[
{
"path": "README.md",
"content": "# BatchFormer: Learning to Explore Sample Relationships for Robust Representation Learning\n\n## Introduction\n\nThis is the official PyTorch implementation of [BatchFormer](https://arxiv.org/abs/2203.01522) for Long-Tailed Recognition, Domain Generalization, Compositional Zero-Shot Learning, Contrastive Learning.\n\n
\n \n
\n\n
\nSample Relationship Exploration for Robust Representation Learning\n
\n\n**Please also refer to [BatchFormerV2](https://arxiv.org/abs/2204.01254), in which we introduce a BatchFormerV2 module for vision Transformers.**\n\n\n
\n \n
\n\n## Main Results\n\n### Long-Tailed Recognition\n\n##### ImageNet-LT\n
\n\n\n
\n
All(R10)
\n
Many(R10)
\n
Med(R10)
\n
Few(R10)
\n
All(R50)
\n
Many(R50)
\n
Med(R50)
\n
Few(R50)
\n\n
RIDE(3 experts)[1]
\n
44.7
\n
57.0
\n
40.3
\n
25.5
\n
53.6
\n
64.9
\n
50.4
\n
33.2
\n
\n\n
+BatchFormer
\n
45.7
\n
56.3
\n
42.1
\n
28.3
\n
54.1
\n
64.3
\n
51.4
\n
35.1
\n\n
PaCo[2]
\n
-
\n
-
\n
-
\n
-
\n
57.0
\n
64.8
\n
55.9
\n
39.1
\n
\n\n
+BatchFormer
\n
-
\n
-
\n
-
\n
-
\n
57.4
\n
62.7
\n
56.7
\n
42.1
\n
\n\nHere, we demonstrate the result on one-stage RIDE (ResNext-50)\n
\n\nHere, we provide the pretrained MoCo-V3 model corresponding to [this strategy](https://github.com/zhihou7/BatchFormer/blob/cc7a060c6395cb1dc8a7c577df77f52a44deaf9d/moco-v3/moco/builder.py#L139).\n\n#### Domain Generalization\n\n##### ResNet-18\n\n
\n\n\n
\n
PACS
\n
VLCS
\n
OfficeHome
\n
Terra
\n\n\n
SWAD[5]
\n
82.9
\n
76.3
\n
62.1
\n
42.1
\n
\n
+BatchFormer
\n
83.7
\n
76.9
\n
64.3
\n
44.8
\n
\n\n\n#### Compositional Zero-Shot Learning\n
\n\n\n
\n
MIT-States(AUC)
\n
MIT-States(HM)
\n
UT-Zap50K(AUC)
\n
UT-Zap50K(HM)
\n
C-GQA(AUC)
\n
C-GQA(HM)
\n\n
CGE*[6]
\n
6.3
\n
20.0
\n
31.5
\n
46.5
\n
3.7
\n
14.9
\n
\n
+BatchFormer
\n
6.7
\n
20.6
\n
34.6
\n
49.0
\n
3.8
\n
15.5
\n
\n\n#### Few-Shot Learning\nExperiments on CUB.\n\n
\n\n\n
\n
Unseen
\n
Seen
\n
Harmonic mean
\n\n\n
CUB[7]*
\n
67.5
\n
65.1
\n
66.3
\n
\n
+BatchFormer
\n
68.2
\n
65.8
\n
67.0
\n
\n\n\n#### Image Classification (V2)\n\n
\n\n\n
\n
Top-1
\n
Top-5
\n\n
DeiT-T
\n
72.2
\n
91.1
\n
\n
+BatchFormerV2
\n
72.7
\n
91.5
\n\n
DeiT-S
\n
79.8
\n
95.0
\n
\n
+BatchFormerV2
\n
80.4
\n
95.2
\n\n
DeiT-B
\n
81.7
\n
95.5
\n
\n
+BatchFormerV2
\n
82.2
\n
95.8
\n
\n\n#### Reference\n\n1. Long-tailed recognition by routing diverse distribution-aware experts. In ICLR, 2021\n2. Parametric contrastive learning. In ICCV, 2021\n3. Improved baselines with momentum contrastive learning.\n4. An empirical study of training self-supervised vision transformers. In CVPR, 2021\n5. Domain generalization by seeking flat minima. In NeurIPS, 2021.\n6. Learning graph embeddings for compositional zero-shot learning. In CVPR, 2021\n7. Contrastive learning based hybrid networks for long- tailed image classification. In CVPR, 2021\n\n\n\n## PyTorch Code\n\nThe proposed BatchFormer can be implemented with a few lines as follows,\n\n def BatchFormer(x, y, encoder, is_training):\n # x: input features with the shape [N, C]\n # encoder: TransformerEncoderLayer(C,4,C,0.5)\n if not is_training:\n return x, y\n pre_x = x\n x = encoder(x.unsqueeze(1)).squeeze(1)\n x = torch.cat([pre_x, x], dim=0)\n y = torch.cat([y, y], dim=0)\n return x, y\n\n\n\n## Citation\nIf you find this repository helpful, please consider cite:\n\n @inproceedings{hou2022batch,\n title={BatchFormer: Learning to Explore Sample Relationships for Robust Representation Learning},\n author={Hou, Zhi and Yu, Baosheng and Tao, Dacheng},\n booktitle={CVPR},\n year={2022}\n }\n @article{hou2022batchformerv2,\n title={BatchFormerV2: Exploring Sample Relationships for Dense Representation Learning},\n author={Hou, Zhi and Yu, Baosheng and Wang, Chaoyue and Zhan, Yibing and Tao, Dacheng},\n journal={arXiv preprint arXiv:2204.01254},\n year={2022}\n }\n\nFeel free to contact \"zhou9878 at uni dot sydney dot edu dot au\" if you have any questions.\n"
},
{
"path": "batchformer-v2/deformable-detr/LICENSE",
"content": "Copyright (c) 2020 SenseTime. All Rights Reserved.\n\n Apache License\n Version 2.0, January 2004\n http://www.apache.org/licenses/\n\n TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION\n\n 1. Definitions.\n\n \"License\" shall mean the terms and conditions for use, reproduction,\n and distribution as defined by Sections 1 through 9 of this document.\n\n \"Licensor\" shall mean the copyright owner or entity authorized by\n the copyright owner that is granting the License.\n\n \"Legal Entity\" shall mean the union of the acting entity and all\n other entities that control, are controlled by, or are under common\n control with that entity. For the purposes of this definition,\n \"control\" means (i) the power, direct or indirect, to cause the\n direction or management of such entity, whether by contract or\n otherwise, or (ii) ownership of fifty percent (50%) or more of the\n outstanding shares, or (iii) beneficial ownership of such entity.\n\n \"You\" (or \"Your\") shall mean an individual or Legal Entity\n exercising permissions granted by this License.\n\n \"Source\" form shall mean the preferred form for making modifications,\n including but not limited to software source code, documentation\n source, and configuration files.\n\n \"Object\" form shall mean any form resulting from mechanical\n transformation or translation of a Source form, including but\n not limited to compiled object code, generated documentation,\n and conversions to other media types.\n\n \"Work\" shall mean the work of authorship, whether in Source or\n Object form, made available under the License, as indicated by a\n copyright notice that is included in or attached to the work\n (an example is provided in the Appendix below).\n\n \"Derivative Works\" shall mean any work, whether in Source or Object\n form, that is based on (or derived from) the Work and for which the\n editorial revisions, annotations, elaborations, or other modifications\n represent, as a whole, an original work of authorship. For the purposes\n of this License, Derivative Works shall not include works that remain\n separable from, or merely link (or bind by name) to the interfaces of,\n the Work and Derivative Works thereof.\n\n \"Contribution\" shall mean any work of authorship, including\n the original version of the Work and any modifications or additions\n to that Work or Derivative Works thereof, that is intentionally\n submitted to Licensor for inclusion in the Work by the copyright owner\n or by an individual or Legal Entity authorized to submit on behalf of\n the copyright owner. For the purposes of this definition, \"submitted\"\n means any form of electronic, verbal, or written communication sent\n to the Licensor or its representatives, including but not limited to\n communication on electronic mailing lists, source code control systems,\n and issue tracking systems that are managed by, or on behalf of, the\n Licensor for the purpose of discussing and improving the Work, but\n excluding communication that is conspicuously marked or otherwise\n designated in writing by the copyright owner as \"Not a Contribution.\"\n\n \"Contributor\" shall mean Licensor and any individual or Legal Entity\n on behalf of whom a Contribution has been received by Licensor and\n subsequently incorporated within the Work.\n\n 2. Grant of Copyright License. Subject to the terms and conditions of\n this License, each Contributor hereby grants to You a perpetual,\n worldwide, non-exclusive, no-charge, royalty-free, irrevocable\n copyright license to reproduce, prepare Derivative Works of,\n publicly display, publicly perform, sublicense, and distribute the\n Work and such Derivative Works in Source or Object form.\n\n 3. Grant of Patent License. Subject to the terms and conditions of\n this License, each Contributor hereby grants to You a perpetual,\n worldwide, non-exclusive, no-charge, royalty-free, irrevocable\n (except as stated in this section) patent license to make, have made,\n use, offer to sell, sell, import, and otherwise transfer the Work,\n where such license applies only to those patent claims licensable\n by such Contributor that are necessarily infringed by their\n Contribution(s) alone or by combination of their Contribution(s)\n with the Work to which such Contribution(s) was submitted. If You\n institute patent litigation against any entity (including a\n cross-claim or counterclaim in a lawsuit) alleging that the Work\n or a Contribution incorporated within the Work constitutes direct\n or contributory patent infringement, then any patent licenses\n granted to You under this License for that Work shall terminate\n as of the date such litigation is filed.\n\n 4. Redistribution. You may reproduce and distribute copies of the\n Work or Derivative Works thereof in any medium, with or without\n modifications, and in Source or Object form, provided that You\n meet the following conditions:\n\n (a) You must give any other recipients of the Work or\n Derivative Works a copy of this License; and\n\n (b) You must cause any modified files to carry prominent notices\n stating that You changed the files; and\n\n (c) You must retain, in the Source form of any Derivative Works\n that You distribute, all copyright, patent, trademark, and\n attribution notices from the Source form of the Work,\n excluding those notices that do not pertain to any part of\n the Derivative Works; and\n\n (d) If the Work includes a \"NOTICE\" text file as part of its\n distribution, then any Derivative Works that You distribute must\n include a readable copy of the attribution notices contained\n within such NOTICE file, excluding those notices that do not\n pertain to any part of the Derivative Works, in at least one\n of the following places: within a NOTICE text file distributed\n as part of the Derivative Works; within the Source form or\n documentation, if provided along with the Derivative Works; or,\n within a display generated by the Derivative Works, if and\n wherever such third-party notices normally appear. The contents\n of the NOTICE file are for informational purposes only and\n do not modify the License. You may add Your own attribution\n notices within Derivative Works that You distribute, alongside\n or as an addendum to the NOTICE text from the Work, provided\n that such additional attribution notices cannot be construed\n as modifying the License.\n\n You may add Your own copyright statement to Your modifications and\n may provide additional or different license terms and conditions\n for use, reproduction, or distribution of Your modifications, or\n for any such Derivative Works as a whole, provided Your use,\n reproduction, and distribution of the Work otherwise complies with\n the conditions stated in this License.\n\n 5. Submission of Contributions. Unless You explicitly state otherwise,\n any Contribution intentionally submitted for inclusion in the Work\n by You to the Licensor shall be under the terms and conditions of\n this License, without any additional terms or conditions.\n Notwithstanding the above, nothing herein shall supersede or modify\n the terms of any separate license agreement you may have executed\n with Licensor regarding such Contributions.\n\n 6. Trademarks. This License does not grant permission to use the trade\n names, trademarks, service marks, or product names of the Licensor,\n except as required for reasonable and customary use in describing the\n origin of the Work and reproducing the content of the NOTICE file.\n\n 7. Disclaimer of Warranty. Unless required by applicable law or\n agreed to in writing, Licensor provides the Work (and each\n Contributor provides its Contributions) on an \"AS IS\" BASIS,\n WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or\n implied, including, without limitation, any warranties or conditions\n of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A\n PARTICULAR PURPOSE. You are solely responsible for determining the\n appropriateness of using or redistributing the Work and assume any\n risks associated with Your exercise of permissions under this License.\n\n 8. Limitation of Liability. In no event and under no legal theory,\n whether in tort (including negligence), contract, or otherwise,\n unless required by applicable law (such as deliberate and grossly\n negligent acts) or agreed to in writing, shall any Contributor be\n liable to You for damages, including any direct, indirect, special,\n incidental, or consequential damages of any character arising as a\n result of this License or out of the use or inability to use the\n Work (including but not limited to damages for loss of goodwill,\n work stoppage, computer failure or malfunction, or any and all\n other commercial damages or losses), even if such Contributor\n has been advised of the possibility of such damages.\n\n 9. Accepting Warranty or Additional Liability. While redistributing\n the Work or Derivative Works thereof, You may choose to offer,\n and charge a fee for, acceptance of support, warranty, indemnity,\n or other liability obligations and/or rights consistent with this\n License. However, in accepting such obligations, You may act only\n on Your own behalf and on Your sole responsibility, not on behalf\n of any other Contributor, and only if You agree to indemnify,\n defend, and hold each Contributor harmless for any liability\n incurred by, or claims asserted against, such Contributor by reason\n of your accepting any such warranty or additional liability.\n\n END OF TERMS AND CONDITIONS\n\n APPENDIX: How to apply the Apache License to your work.\n\n To apply the Apache License to your work, attach the following\n boilerplate notice, with the fields enclosed by brackets \"[]\"\n replaced with your own identifying information. (Don't include\n the brackets!) The text should be enclosed in the appropriate\n comment syntax for the file format. We also recommend that a\n file or class name and description of purpose be included on the\n same \"printed page\" as the copyright notice for easier\n identification within third-party archives.\n\n Copyright 2020 SenseTime\n\n Licensed under the Apache License, Version 2.0 (the \"License\");\n you may not use this file except in compliance with the License.\n You may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\n Unless required by applicable law or agreed to in writing, software\n distributed under the License is distributed on an \"AS IS\" BASIS,\n WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n See the License for the specific language governing permissions and\n limitations under the License.\n\n\nDETR\n\nCopyright 2020 - present, Facebook, Inc\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n"
},
{
"path": "batchformer-v2/deformable-detr/README.md",
"content": "# BatchFormerV2: Exploring Sample Relationships for Dense Representation Learning\n\nThis repository is based on Deformable DETR. One can follow the instructions in Deformable DETR to install Deformable DETR first.\n\nThis repository has changed a few lines of the core code based on Deformable DETR. Check the modifications by:\n```\ndiff models/deformable_transformer.py <(curl https://raw.githubusercontent.com/fundamentalvision/Deformable-DETR/main/models/deformable_transformer.py)\n```\n\nI mainly changes the two parts of the code: \n1. apply batchformer t get two streams, https://github.com/zhihou7/BatchFormer/blob/e22f5ff895b04d10f3aa4e745f9a226f3d9b7641/batchformer-v2/models/deformable_transformer.py#L278\n2. set the labels for batchformerv2 stream. https://github.com/zhihou7/BatchFormer/blob/e22f5ff895b04d10f3aa4e745f9a226f3d9b7641/batchformer-v2/engine.py#L43\n\n## Main Results\n\n| Method | AP |\n| ----------- | ----------- |\n| Deformable DETR | 43.8 |\n| + BatchFormerV2 | 45.5 |\n\n## Installation\n\n### Compiling CUDA operators\n```bash\ncd ./models/ops\nsh ./make.sh\n# unit test (should see all checking is True)\npython test.py\n```\n\n## Usage\n\n### Dataset preparation\n\nPlease download [COCO 2017 dataset](https://cocodataset.org/) and organize them as following:\n\n```\ncode_root/\n└── data/\n └── coco/\n ├── train2017/\n ├── val2017/\n └── annotations/\n \t├── instances_train2017.json\n \t└── instances_val2017.json\n```\n\n### Training\n\n#### Training on single node\n\nFor example, the command for training Deformable DETR on 8 GPUs is as following:\n\n```bash\nGPUS_PER_NODE=8 ./tools/run_dist_launch.sh 8 ./configs/r50_deformable_detr.sh --batch-size 3 --share_bf 0 --bf 1 --insert_idx 0\n```\n\n\"--share_bf 0 --bf 1 --insert_idx 0\" is parameters for BatchFormerV2\n\n### Evaluation\n\nYou can get the config file and pretrained model of Deformable DETR (the link is in \"Main Results\" session), then run following command to evaluate it on COCO 2017 validation set:\n\n```bash\n --resume --eval\n```\n\nIf you find this repository is helpful, please consider to cite\n\n\n @inproceedings{hou2022batch,\n \ttitle={BatchFormer: Learning to Explore Sample Relationships for Robust Representation Learning},\n\tauthor={Hou, Zhi and Yu, Baosheng and Tao, Dacheng},\n \tbooktitle={CVPR},\n\tyear={2022}\n\t}\n @article{hou2022batchformerv2,\n \t title={BatchFormerV2: Exploring Sample Relationships for Dense Representation Learning},\n\t author={Hou, Zhi and Yu, Baosheng and Wang, Chaoyue and Zhan, Yibing and Tao, Dacheng},\n\t journal={arXiv preprint arXiv:2204.01254},\n\t year={2022}\n\t}\n\t\n"
},
{
"path": "batchformer-v2/deformable-detr/benchmark.py",
"content": "# ------------------------------------------------------------------------\n# Deformable DETR\n# Copyright (c) 2020 SenseTime. All Rights Reserved.\n# Licensed under the Apache License, Version 2.0 [see LICENSE for details]\n# ------------------------------------------------------------------------\n\n\"\"\"\nBenchmark inference speed of Deformable DETR.\n\"\"\"\nimport os\nimport time\nimport argparse\n\nimport torch\n\nfrom main import get_args_parser as get_main_args_parser\nfrom models import build_model\nfrom datasets import build_dataset\nfrom util.misc import nested_tensor_from_tensor_list\n\n\ndef get_benckmark_arg_parser():\n parser = argparse.ArgumentParser('Benchmark inference speed of Deformable DETR.')\n parser.add_argument('--num_iters', type=int, default=300, help='total iters to benchmark speed')\n parser.add_argument('--warm_iters', type=int, default=5, help='ignore first several iters that are very slow')\n parser.add_argument('--batch_size', type=int, default=1, help='batch size in inference')\n parser.add_argument('--resume', type=str, help='load the pre-trained checkpoint')\n return parser\n\n\n@torch.no_grad()\ndef measure_average_inference_time(model, inputs, num_iters=100, warm_iters=5):\n ts = []\n for iter_ in range(num_iters):\n torch.cuda.synchronize()\n t_ = time.perf_counter()\n model(inputs)\n torch.cuda.synchronize()\n t = time.perf_counter() - t_\n if iter_ >= warm_iters:\n ts.append(t)\n print(ts)\n return sum(ts) / len(ts)\n\n\ndef benchmark():\n args, _ = get_benckmark_arg_parser().parse_known_args()\n main_args = get_main_args_parser().parse_args(_)\n assert args.warm_iters < args.num_iters and args.num_iters > 0 and args.warm_iters >= 0\n assert args.batch_size > 0\n assert args.resume is None or os.path.exists(args.resume)\n dataset = build_dataset('val', main_args)\n model, _, _ = build_model(main_args)\n model.cuda()\n model.eval()\n if args.resume is not None:\n ckpt = torch.load(args.resume, map_location=lambda storage, loc: storage)\n model.load_state_dict(ckpt['model'])\n inputs = nested_tensor_from_tensor_list([dataset.__getitem__(0)[0].cuda() for _ in range(args.batch_size)])\n t = measure_average_inference_time(model, inputs, args.num_iters, args.warm_iters)\n return 1.0 / t * args.batch_size\n\n\nif __name__ == '__main__':\n fps = benchmark()\n print(f'Inference Speed: {fps:.1f} FPS')\n\n"
},
{
"path": "batchformer-v2/deformable-detr/configs/r101_deformable_detr.sh",
"content": "#!/usr/bin/env bash\n\nsource /usr/local/anaconda/4.2.0/etc/profile.d/conda.sh\nconda activate pyt\n\nset -x\n\nEXP_DIR=exps/r101_deformable_detr\nPY_ARGS=${@:1}\n\npython -u main.py \\\n --output_dir ${EXP_DIR} \\\n ${PY_ARGS}\n"
},
{
"path": "batchformer-v2/deformable-detr/configs/r50_deformable_detr.sh",
"content": "#!/usr/bin/env bash\n\nset -x\n\nEXP_DIR=exps/r50_deformable_detr\nPY_ARGS=${@:1}\n\npython -u main.py \\\n --output_dir ${EXP_DIR} \\\n ${PY_ARGS}\n"
},
{
"path": "batchformer-v2/deformable-detr/configs/r50_deformable_detr_plus_iterative_bbox_refinement.sh",
"content": "#!/usr/bin/env bash\n\nset -x\n\nEXP_DIR=exps/r50_deformable_detr_plus_iterative_bbox_refinement\nPY_ARGS=${@:1}\n\npython -u main.py \\\n --output_dir ${EXP_DIR} \\\n --with_box_refine \\\n ${PY_ARGS}\n"
},
{
"path": "batchformer-v2/deformable-detr/configs/r50_deformable_detr_plus_iterative_bbox_refinement_plus_plus_two_stage.sh",
"content": "#!/usr/bin/env bash\n\nset -x\n\nEXP_DIR=exps/r50_deformable_detr_plus_iterative_bbox_refinement_plus_plus_two_stage\nPY_ARGS=${@:1}\n\npython -u main.py \\\n --output_dir ${EXP_DIR} \\\n --with_box_refine \\\n --two_stage \\\n ${PY_ARGS}\n"
},
{
"path": "batchformer-v2/deformable-detr/configs/r50_deformable_detr_single_scale.sh",
"content": "#!/usr/bin/env bash\n\nset -x\n\nEXP_DIR=exps/r50_deformable_detr_single_scale\nPY_ARGS=${@:1}\n\npython -u main.py \\\n --num_feature_levels 1 \\\n --output_dir ${EXP_DIR} \\\n ${PY_ARGS}\n"
},
{
"path": "batchformer-v2/deformable-detr/configs/r50_deformable_detr_single_scale_dc5.sh",
"content": "#!/usr/bin/env bash\n\nset -x\n\nEXP_DIR=exps/r50_deformable_detr_single_scale_dc5\nPY_ARGS=${@:1}\n\npython -u main.py \\\n --num_feature_levels 1 \\\n --dilation \\\n --output_dir ${EXP_DIR} \\\n ${PY_ARGS}\n"
},
{
"path": "batchformer-v2/deformable-detr/datasets/__init__.py",
"content": "# ------------------------------------------------------------------------\n# Deformable DETR\n# Copyright (c) 2020 SenseTime. All Rights Reserved.\n# Licensed under the Apache License, Version 2.0 [see LICENSE for details]\n# ------------------------------------------------------------------------\n# Modified from DETR (https://github.com/facebookresearch/detr)\n# Copyright (c) Facebook, Inc. and its affiliates. All Rights Reserved\n# ------------------------------------------------------------------------\n\nimport torch.utils.data\nfrom .torchvision_datasets import CocoDetection\n\nfrom .coco import build as build_coco\n\n\ndef get_coco_api_from_dataset(dataset):\n for _ in range(10):\n # if isinstance(dataset, torchvision.datasets.CocoDetection):\n # break\n if isinstance(dataset, torch.utils.data.Subset):\n dataset = dataset.dataset\n if isinstance(dataset, CocoDetection):\n return dataset.coco\n\n\ndef build_dataset(image_set, args):\n if args.dataset_file == 'coco':\n return build_coco(image_set, args)\n if args.dataset_file == 'coco_panoptic':\n # to avoid making panopticapi required for coco\n from .coco_panoptic import build as build_coco_panoptic\n return build_coco_panoptic(image_set, args)\n raise ValueError(f'dataset {args.dataset_file} not supported')\n"
},
{
"path": "batchformer-v2/deformable-detr/datasets/coco.py",
"content": "# ------------------------------------------------------------------------\n# Deformable DETR\n# Copyright (c) 2020 SenseTime. All Rights Reserved.\n# Licensed under the Apache License, Version 2.0 [see LICENSE for details]\n# ------------------------------------------------------------------------\n# Modified from DETR (https://github.com/facebookresearch/detr)\n# Copyright (c) Facebook, Inc. and its affiliates. All Rights Reserved\n# ------------------------------------------------------------------------\n\n\"\"\"\nCOCO dataset which returns image_id for evaluation.\n\nMostly copy-paste from https://github.com/pytorch/vision/blob/13b35ff/references/detection/coco_utils.py\n\"\"\"\nfrom pathlib import Path\n\nimport torch\nimport torch.utils.data\nfrom pycocotools import mask as coco_mask\n\nfrom .torchvision_datasets import CocoDetection as TvCocoDetection\nfrom util.misc import get_local_rank, get_local_size\nimport datasets.transforms as T\n\n\nclass CocoDetection(TvCocoDetection):\n def __init__(self, img_folder, ann_file, transforms, return_masks, cache_mode=False, local_rank=0, local_size=1):\n super(CocoDetection, self).__init__(img_folder, ann_file,\n cache_mode=cache_mode, local_rank=local_rank, local_size=local_size)\n self._transforms = transforms\n self.prepare = ConvertCocoPolysToMask(return_masks)\n\n def __getitem__(self, idx):\n img, target = super(CocoDetection, self).__getitem__(idx)\n image_id = self.ids[idx]\n target = {'image_id': image_id, 'annotations': target}\n img, target = self.prepare(img, target)\n if self._transforms is not None:\n img, target = self._transforms(img, target)\n return img, target\n\n\ndef convert_coco_poly_to_mask(segmentations, height, width):\n masks = []\n for polygons in segmentations:\n rles = coco_mask.frPyObjects(polygons, height, width)\n mask = coco_mask.decode(rles)\n if len(mask.shape) < 3:\n mask = mask[..., None]\n mask = torch.as_tensor(mask, dtype=torch.uint8)\n mask = mask.any(dim=2)\n masks.append(mask)\n if masks:\n masks = torch.stack(masks, dim=0)\n else:\n masks = torch.zeros((0, height, width), dtype=torch.uint8)\n return masks\n\n\nclass ConvertCocoPolysToMask(object):\n def __init__(self, return_masks=False):\n self.return_masks = return_masks\n\n def __call__(self, image, target):\n w, h = image.size\n\n image_id = target[\"image_id\"]\n image_id = torch.tensor([image_id])\n\n anno = target[\"annotations\"]\n\n anno = [obj for obj in anno if 'iscrowd' not in obj or obj['iscrowd'] == 0]\n\n boxes = [obj[\"bbox\"] for obj in anno]\n # guard against no boxes via resizing\n boxes = torch.as_tensor(boxes, dtype=torch.float32).reshape(-1, 4)\n boxes[:, 2:] += boxes[:, :2]\n boxes[:, 0::2].clamp_(min=0, max=w)\n boxes[:, 1::2].clamp_(min=0, max=h)\n\n classes = [obj[\"category_id\"] for obj in anno]\n classes = torch.tensor(classes, dtype=torch.int64)\n\n if self.return_masks:\n segmentations = [obj[\"segmentation\"] for obj in anno]\n masks = convert_coco_poly_to_mask(segmentations, h, w)\n\n keypoints = None\n if anno and \"keypoints\" in anno[0]:\n keypoints = [obj[\"keypoints\"] for obj in anno]\n keypoints = torch.as_tensor(keypoints, dtype=torch.float32)\n num_keypoints = keypoints.shape[0]\n if num_keypoints:\n keypoints = keypoints.view(num_keypoints, -1, 3)\n\n keep = (boxes[:, 3] > boxes[:, 1]) & (boxes[:, 2] > boxes[:, 0])\n boxes = boxes[keep]\n classes = classes[keep]\n if self.return_masks:\n masks = masks[keep]\n if keypoints is not None:\n keypoints = keypoints[keep]\n\n target = {}\n target[\"boxes\"] = boxes\n target[\"labels\"] = classes\n if self.return_masks:\n target[\"masks\"] = masks\n target[\"image_id\"] = image_id\n if keypoints is not None:\n target[\"keypoints\"] = keypoints\n\n # for conversion to coco api\n area = torch.tensor([obj[\"area\"] for obj in anno])\n iscrowd = torch.tensor([obj[\"iscrowd\"] if \"iscrowd\" in obj else 0 for obj in anno])\n target[\"area\"] = area[keep]\n target[\"iscrowd\"] = iscrowd[keep]\n\n target[\"orig_size\"] = torch.as_tensor([int(h), int(w)])\n target[\"size\"] = torch.as_tensor([int(h), int(w)])\n\n return image, target\n\n\ndef make_coco_transforms(image_set):\n\n normalize = T.Compose([\n T.ToTensor(),\n T.Normalize([0.485, 0.456, 0.406], [0.229, 0.224, 0.225])\n ])\n\n scales = [480, 512, 544, 576, 608, 640, 672, 704, 736, 768, 800]\n\n if image_set == 'train':\n return T.Compose([\n T.RandomHorizontalFlip(),\n T.RandomSelect(\n T.RandomResize(scales, max_size=1333),\n T.Compose([\n T.RandomResize([400, 500, 600]),\n T.RandomSizeCrop(384, 600),\n T.RandomResize(scales, max_size=1333),\n ])\n ),\n normalize,\n ])\n\n if image_set == 'val':\n return T.Compose([\n T.RandomResize([800], max_size=1333),\n normalize,\n ])\n\n raise ValueError(f'unknown {image_set}')\n\n\ndef build(image_set, args):\n root = Path(args.coco_path)\n assert root.exists(), f'provided COCO path {root} does not exist'\n mode = 'instances'\n PATHS = {\n \"train\": (root / \"train2017\", root / \"annotations\" / f'{mode}_train2017.json'),\n \"val\": (root / \"val2017\", root / \"annotations\" / f'{mode}_val2017.json'),\n }\n\n img_folder, ann_file = PATHS[image_set]\n dataset = CocoDetection(img_folder, ann_file, transforms=make_coco_transforms(image_set), return_masks=args.masks,\n cache_mode=args.cache_mode, local_rank=get_local_rank(), local_size=get_local_size())\n return dataset\n"
},
{
"path": "batchformer-v2/deformable-detr/datasets/coco_eval.py",
"content": "# ------------------------------------------------------------------------\n# Deformable DETR\n# Copyright (c) 2020 SenseTime. All Rights Reserved.\n# Licensed under the Apache License, Version 2.0 [see LICENSE for details]\n# ------------------------------------------------------------------------\n# Modified from DETR (https://github.com/facebookresearch/detr)\n# Copyright (c) Facebook, Inc. and its affiliates. All Rights Reserved\n# ------------------------------------------------------------------------\n\n\"\"\"\nCOCO evaluator that works in distributed mode.\n\nMostly copy-paste from https://github.com/pytorch/vision/blob/edfd5a7/references/detection/coco_eval.py\nThe difference is that there is less copy-pasting from pycocotools\nin the end of the file, as python3 can suppress prints with contextlib\n\"\"\"\nimport os\nimport contextlib\nimport copy\nimport numpy as np\nimport torch\n\nfrom pycocotools.cocoeval import COCOeval\nfrom pycocotools.coco import COCO\nimport pycocotools.mask as mask_util\n\nfrom util.misc import all_gather\n\n\nclass CocoEvaluator(object):\n def __init__(self, coco_gt, iou_types):\n assert isinstance(iou_types, (list, tuple))\n coco_gt = copy.deepcopy(coco_gt)\n self.coco_gt = coco_gt\n\n self.iou_types = iou_types\n self.coco_eval = {}\n for iou_type in iou_types:\n self.coco_eval[iou_type] = COCOeval(coco_gt, iouType=iou_type)\n\n self.img_ids = []\n self.eval_imgs = {k: [] for k in iou_types}\n\n def update(self, predictions):\n img_ids = list(np.unique(list(predictions.keys())))\n self.img_ids.extend(img_ids)\n\n for iou_type in self.iou_types:\n results = self.prepare(predictions, iou_type)\n\n # suppress pycocotools prints\n with open(os.devnull, 'w') as devnull:\n with contextlib.redirect_stdout(devnull):\n coco_dt = COCO.loadRes(self.coco_gt, results) if results else COCO()\n coco_eval = self.coco_eval[iou_type]\n\n coco_eval.cocoDt = coco_dt\n coco_eval.params.imgIds = list(img_ids)\n img_ids, eval_imgs = evaluate(coco_eval)\n\n self.eval_imgs[iou_type].append(eval_imgs)\n\n def synchronize_between_processes(self):\n for iou_type in self.iou_types:\n self.eval_imgs[iou_type] = np.concatenate(self.eval_imgs[iou_type], 2)\n create_common_coco_eval(self.coco_eval[iou_type], self.img_ids, self.eval_imgs[iou_type])\n\n def accumulate(self):\n for coco_eval in self.coco_eval.values():\n coco_eval.accumulate()\n\n def summarize(self):\n for iou_type, coco_eval in self.coco_eval.items():\n print(\"IoU metric: {}\".format(iou_type))\n coco_eval.summarize()\n\n def prepare(self, predictions, iou_type):\n if iou_type == \"bbox\":\n return self.prepare_for_coco_detection(predictions)\n elif iou_type == \"segm\":\n return self.prepare_for_coco_segmentation(predictions)\n elif iou_type == \"keypoints\":\n return self.prepare_for_coco_keypoint(predictions)\n else:\n raise ValueError(\"Unknown iou type {}\".format(iou_type))\n\n def prepare_for_coco_detection(self, predictions):\n coco_results = []\n for original_id, prediction in predictions.items():\n if len(prediction) == 0:\n continue\n\n boxes = prediction[\"boxes\"]\n boxes = convert_to_xywh(boxes).tolist()\n scores = prediction[\"scores\"].tolist()\n labels = prediction[\"labels\"].tolist()\n\n coco_results.extend(\n [\n {\n \"image_id\": original_id,\n \"category_id\": labels[k],\n \"bbox\": box,\n \"score\": scores[k],\n }\n for k, box in enumerate(boxes)\n ]\n )\n return coco_results\n\n def prepare_for_coco_segmentation(self, predictions):\n coco_results = []\n for original_id, prediction in predictions.items():\n if len(prediction) == 0:\n continue\n\n scores = prediction[\"scores\"]\n labels = prediction[\"labels\"]\n masks = prediction[\"masks\"]\n\n masks = masks > 0.5\n\n scores = prediction[\"scores\"].tolist()\n labels = prediction[\"labels\"].tolist()\n\n rles = [\n mask_util.encode(np.array(mask[0, :, :, np.newaxis], dtype=np.uint8, order=\"F\"))[0]\n for mask in masks\n ]\n for rle in rles:\n rle[\"counts\"] = rle[\"counts\"].decode(\"utf-8\")\n\n coco_results.extend(\n [\n {\n \"image_id\": original_id,\n \"category_id\": labels[k],\n \"segmentation\": rle,\n \"score\": scores[k],\n }\n for k, rle in enumerate(rles)\n ]\n )\n return coco_results\n\n def prepare_for_coco_keypoint(self, predictions):\n coco_results = []\n for original_id, prediction in predictions.items():\n if len(prediction) == 0:\n continue\n\n boxes = prediction[\"boxes\"]\n boxes = convert_to_xywh(boxes).tolist()\n scores = prediction[\"scores\"].tolist()\n labels = prediction[\"labels\"].tolist()\n keypoints = prediction[\"keypoints\"]\n keypoints = keypoints.flatten(start_dim=1).tolist()\n\n coco_results.extend(\n [\n {\n \"image_id\": original_id,\n \"category_id\": labels[k],\n 'keypoints': keypoint,\n \"score\": scores[k],\n }\n for k, keypoint in enumerate(keypoints)\n ]\n )\n return coco_results\n\n\ndef convert_to_xywh(boxes):\n xmin, ymin, xmax, ymax = boxes.unbind(1)\n return torch.stack((xmin, ymin, xmax - xmin, ymax - ymin), dim=1)\n\n\ndef merge(img_ids, eval_imgs):\n all_img_ids = all_gather(img_ids)\n all_eval_imgs = all_gather(eval_imgs)\n\n merged_img_ids = []\n for p in all_img_ids:\n merged_img_ids.extend(p)\n\n merged_eval_imgs = []\n for p in all_eval_imgs:\n merged_eval_imgs.append(p)\n\n merged_img_ids = np.array(merged_img_ids)\n merged_eval_imgs = np.concatenate(merged_eval_imgs, 2)\n\n # keep only unique (and in sorted order) images\n merged_img_ids, idx = np.unique(merged_img_ids, return_index=True)\n merged_eval_imgs = merged_eval_imgs[..., idx]\n\n return merged_img_ids, merged_eval_imgs\n\n\ndef create_common_coco_eval(coco_eval, img_ids, eval_imgs):\n img_ids, eval_imgs = merge(img_ids, eval_imgs)\n img_ids = list(img_ids)\n eval_imgs = list(eval_imgs.flatten())\n\n coco_eval.evalImgs = eval_imgs\n coco_eval.params.imgIds = img_ids\n coco_eval._paramsEval = copy.deepcopy(coco_eval.params)\n\n\n#################################################################\n# From pycocotools, just removed the prints and fixed\n# a Python3 bug about unicode not defined\n#################################################################\n\n\ndef evaluate(self):\n '''\n Run per image evaluation on given images and store results (a list of dict) in self.evalImgs\n :return: None\n '''\n # tic = time.time()\n # print('Running per image evaluation...')\n p = self.params\n # add backward compatibility if useSegm is specified in params\n if p.useSegm is not None:\n p.iouType = 'segm' if p.useSegm == 1 else 'bbox'\n print('useSegm (deprecated) is not None. Running {} evaluation'.format(p.iouType))\n # print('Evaluate annotation type *{}*'.format(p.iouType))\n p.imgIds = list(np.unique(p.imgIds))\n if p.useCats:\n p.catIds = list(np.unique(p.catIds))\n p.maxDets = sorted(p.maxDets)\n self.params = p\n\n self._prepare()\n # loop through images, area range, max detection number\n catIds = p.catIds if p.useCats else [-1]\n\n if p.iouType == 'segm' or p.iouType == 'bbox':\n computeIoU = self.computeIoU\n elif p.iouType == 'keypoints':\n computeIoU = self.computeOks\n self.ious = {\n (imgId, catId): computeIoU(imgId, catId)\n for imgId in p.imgIds\n for catId in catIds}\n\n evaluateImg = self.evaluateImg\n maxDet = p.maxDets[-1]\n evalImgs = [\n evaluateImg(imgId, catId, areaRng, maxDet)\n for catId in catIds\n for areaRng in p.areaRng\n for imgId in p.imgIds\n ]\n # this is NOT in the pycocotools code, but could be done outside\n evalImgs = np.asarray(evalImgs).reshape(len(catIds), len(p.areaRng), len(p.imgIds))\n self._paramsEval = copy.deepcopy(self.params)\n # toc = time.time()\n # print('DONE (t={:0.2f}s).'.format(toc-tic))\n return p.imgIds, evalImgs\n\n#################################################################\n# end of straight copy from pycocotools, just removing the prints\n#################################################################\n"
},
{
"path": "batchformer-v2/deformable-detr/datasets/coco_panoptic.py",
"content": "# ------------------------------------------------------------------------\n# Deformable DETR\n# Copyright (c) 2020 SenseTime. All Rights Reserved.\n# Licensed under the Apache License, Version 2.0 [see LICENSE for details]\n# ------------------------------------------------------------------------\n# Modified from DETR (https://github.com/facebookresearch/detr)\n# Copyright (c) Facebook, Inc. and its affiliates. All Rights Reserved\n# ------------------------------------------------------------------------\n\nimport json\nfrom pathlib import Path\n\nimport numpy as np\nimport torch\nfrom PIL import Image\n\nfrom panopticapi.utils import rgb2id\nfrom util.box_ops import masks_to_boxes\n\nfrom .coco import make_coco_transforms\n\n\nclass CocoPanoptic:\n def __init__(self, img_folder, ann_folder, ann_file, transforms=None, return_masks=True):\n with open(ann_file, 'r') as f:\n self.coco = json.load(f)\n\n # sort 'images' field so that they are aligned with 'annotations'\n # i.e., in alphabetical order\n self.coco['images'] = sorted(self.coco['images'], key=lambda x: x['id'])\n # sanity check\n if \"annotations\" in self.coco:\n for img, ann in zip(self.coco['images'], self.coco['annotations']):\n assert img['file_name'][:-4] == ann['file_name'][:-4]\n\n self.img_folder = img_folder\n self.ann_folder = ann_folder\n self.ann_file = ann_file\n self.transforms = transforms\n self.return_masks = return_masks\n\n def __getitem__(self, idx):\n ann_info = self.coco['annotations'][idx] if \"annotations\" in self.coco else self.coco['images'][idx]\n img_path = Path(self.img_folder) / ann_info['file_name'].replace('.png', '.jpg')\n ann_path = Path(self.ann_folder) / ann_info['file_name']\n\n img = Image.open(img_path).convert('RGB')\n w, h = img.size\n if \"segments_info\" in ann_info:\n masks = np.asarray(Image.open(ann_path), dtype=np.uint32)\n masks = rgb2id(masks)\n\n ids = np.array([ann['id'] for ann in ann_info['segments_info']])\n masks = masks == ids[:, None, None]\n\n masks = torch.as_tensor(masks, dtype=torch.uint8)\n labels = torch.tensor([ann['category_id'] for ann in ann_info['segments_info']], dtype=torch.int64)\n\n target = {}\n target['image_id'] = torch.tensor([ann_info['image_id'] if \"image_id\" in ann_info else ann_info[\"id\"]])\n if self.return_masks:\n target['masks'] = masks\n target['labels'] = labels\n\n target[\"boxes\"] = masks_to_boxes(masks)\n\n target['size'] = torch.as_tensor([int(h), int(w)])\n target['orig_size'] = torch.as_tensor([int(h), int(w)])\n if \"segments_info\" in ann_info:\n for name in ['iscrowd', 'area']:\n target[name] = torch.tensor([ann[name] for ann in ann_info['segments_info']])\n\n if self.transforms is not None:\n img, target = self.transforms(img, target)\n\n return img, target\n\n def __len__(self):\n return len(self.coco['images'])\n\n def get_height_and_width(self, idx):\n img_info = self.coco['images'][idx]\n height = img_info['height']\n width = img_info['width']\n return height, width\n\n\ndef build(image_set, args):\n img_folder_root = Path(args.coco_path)\n ann_folder_root = Path(args.coco_panoptic_path)\n assert img_folder_root.exists(), f'provided COCO path {img_folder_root} does not exist'\n assert ann_folder_root.exists(), f'provided COCO path {ann_folder_root} does not exist'\n mode = 'panoptic'\n PATHS = {\n \"train\": (\"train2017\", Path(\"annotations\") / f'{mode}_train2017.json'),\n \"val\": (\"val2017\", Path(\"annotations\") / f'{mode}_val2017.json'),\n }\n\n img_folder, ann_file = PATHS[image_set]\n img_folder_path = img_folder_root / img_folder\n ann_folder = ann_folder_root / f'{mode}_{img_folder}'\n ann_file = ann_folder_root / ann_file\n\n dataset = CocoPanoptic(img_folder_path, ann_folder, ann_file,\n transforms=make_coco_transforms(image_set), return_masks=args.masks)\n\n return dataset\n"
},
{
"path": "batchformer-v2/deformable-detr/datasets/data_prefetcher.py",
"content": "# ------------------------------------------------------------------------\n# Deformable DETR\n# Copyright (c) 2020 SenseTime. All Rights Reserved.\n# Licensed under the Apache License, Version 2.0 [see LICENSE for details]\n# ------------------------------------------------------------------------\n\nimport torch\n\ndef to_cuda(samples, targets, device):\n samples = samples.to(device, non_blocking=True)\n targets = [{k: v.to(device, non_blocking=True) for k, v in t.items()} for t in targets]\n return samples, targets\n\nclass data_prefetcher():\n def __init__(self, loader, device, prefetch=True):\n self.loader = iter(loader)\n self.prefetch = prefetch\n self.device = device\n if prefetch:\n self.stream = torch.cuda.Stream()\n self.preload()\n\n def preload(self):\n try:\n self.next_samples, self.next_targets = next(self.loader)\n except StopIteration:\n self.next_samples = None\n self.next_targets = None\n return\n # if record_stream() doesn't work, another option is to make sure device inputs are created\n # on the main stream.\n # self.next_input_gpu = torch.empty_like(self.next_input, device='cuda')\n # self.next_target_gpu = torch.empty_like(self.next_target, device='cuda')\n # Need to make sure the memory allocated for next_* is not still in use by the main stream\n # at the time we start copying to next_*:\n # self.stream.wait_stream(torch.cuda.current_stream())\n with torch.cuda.stream(self.stream):\n self.next_samples, self.next_targets = to_cuda(self.next_samples, self.next_targets, self.device)\n # more code for the alternative if record_stream() doesn't work:\n # copy_ will record the use of the pinned source tensor in this side stream.\n # self.next_input_gpu.copy_(self.next_input, non_blocking=True)\n # self.next_target_gpu.copy_(self.next_target, non_blocking=True)\n # self.next_input = self.next_input_gpu\n # self.next_target = self.next_target_gpu\n\n # With Amp, it isn't necessary to manually convert data to half.\n # if args.fp16:\n # self.next_input = self.next_input.half()\n # else:\n\n def next(self):\n if self.prefetch:\n torch.cuda.current_stream().wait_stream(self.stream)\n samples = self.next_samples\n targets = self.next_targets\n if samples is not None:\n samples.record_stream(torch.cuda.current_stream())\n if targets is not None:\n for t in targets:\n for k, v in t.items():\n v.record_stream(torch.cuda.current_stream())\n self.preload()\n else:\n try:\n samples, targets = next(self.loader)\n samples, targets = to_cuda(samples, targets, self.device)\n except StopIteration:\n samples = None\n targets = None\n return samples, targets\n"
},
{
"path": "batchformer-v2/deformable-detr/datasets/panoptic_eval.py",
"content": "# ------------------------------------------------------------------------\n# Deformable DETR\n# Copyright (c) 2020 SenseTime. All Rights Reserved.\n# Licensed under the Apache License, Version 2.0 [see LICENSE for details]\n# ------------------------------------------------------------------------\n# Modified from DETR (https://github.com/facebookresearch/detr)\n# Copyright (c) Facebook, Inc. and its affiliates. All Rights Reserved\n# ------------------------------------------------------------------------\n\nimport json\nimport os\n\nimport util.misc as utils\n\ntry:\n from panopticapi.evaluation import pq_compute\nexcept ImportError:\n pass\n\n\nclass PanopticEvaluator(object):\n def __init__(self, ann_file, ann_folder, output_dir=\"panoptic_eval\"):\n self.gt_json = ann_file\n self.gt_folder = ann_folder\n if utils.is_main_process():\n if not os.path.exists(output_dir):\n os.mkdir(output_dir)\n self.output_dir = output_dir\n self.predictions = []\n\n def update(self, predictions):\n for p in predictions:\n with open(os.path.join(self.output_dir, p[\"file_name\"]), \"wb\") as f:\n f.write(p.pop(\"png_string\"))\n\n self.predictions += predictions\n\n def synchronize_between_processes(self):\n all_predictions = utils.all_gather(self.predictions)\n merged_predictions = []\n for p in all_predictions:\n merged_predictions += p\n self.predictions = merged_predictions\n\n def summarize(self):\n if utils.is_main_process():\n json_data = {\"annotations\": self.predictions}\n predictions_json = os.path.join(self.output_dir, \"predictions.json\")\n with open(predictions_json, \"w\") as f:\n f.write(json.dumps(json_data))\n return pq_compute(self.gt_json, predictions_json, gt_folder=self.gt_folder, pred_folder=self.output_dir)\n return None\n"
},
{
"path": "batchformer-v2/deformable-detr/datasets/samplers.py",
"content": "# ------------------------------------------------------------------------\n# Deformable DETR\n# Copyright (c) 2020 SenseTime. All Rights Reserved.\n# Licensed under the Apache License, Version 2.0 [see LICENSE for details]\n# ------------------------------------------------------------------------\n# Modified from codes in torch.utils.data.distributed\n# ------------------------------------------------------------------------\n\nimport os\nimport math\nimport torch\nimport torch.distributed as dist\nfrom torch.utils.data.sampler import Sampler\n\n\nclass DistributedSampler(Sampler):\n \"\"\"Sampler that restricts data loading to a subset of the dataset.\n It is especially useful in conjunction with\n :class:`torch.nn.parallel.DistributedDataParallel`. In such case, each\n process can pass a DistributedSampler instance as a DataLoader sampler,\n and load a subset of the original dataset that is exclusive to it.\n .. note::\n Dataset is assumed to be of constant size.\n Arguments:\n dataset: Dataset used for sampling.\n num_replicas (optional): Number of processes participating in\n distributed training.\n rank (optional): Rank of the current process within num_replicas.\n \"\"\"\n\n def __init__(self, dataset, num_replicas=None, rank=None, local_rank=None, local_size=None, shuffle=True):\n if num_replicas is None:\n if not dist.is_available():\n raise RuntimeError(\"Requires distributed package to be available\")\n num_replicas = dist.get_world_size()\n if rank is None:\n if not dist.is_available():\n raise RuntimeError(\"Requires distributed package to be available\")\n rank = dist.get_rank()\n self.dataset = dataset\n self.num_replicas = num_replicas\n self.rank = rank\n self.epoch = 0\n self.num_samples = int(math.ceil(len(self.dataset) * 1.0 / self.num_replicas))\n self.total_size = self.num_samples * self.num_replicas\n self.shuffle = shuffle\n\n def __iter__(self):\n if self.shuffle:\n # deterministically shuffle based on epoch\n g = torch.Generator()\n g.manual_seed(self.epoch)\n indices = torch.randperm(len(self.dataset), generator=g).tolist()\n else:\n indices = torch.arange(len(self.dataset)).tolist()\n\n # add extra samples to make it evenly divisible\n indices += indices[: (self.total_size - len(indices))]\n assert len(indices) == self.total_size\n\n # subsample\n offset = self.num_samples * self.rank\n indices = indices[offset : offset + self.num_samples]\n assert len(indices) == self.num_samples\n\n return iter(indices)\n\n def __len__(self):\n return self.num_samples\n\n def set_epoch(self, epoch):\n self.epoch = epoch\n\n\nclass NodeDistributedSampler(Sampler):\n \"\"\"Sampler that restricts data loading to a subset of the dataset.\n It is especially useful in conjunction with\n :class:`torch.nn.parallel.DistributedDataParallel`. In such case, each\n process can pass a DistributedSampler instance as a DataLoader sampler,\n and load a subset of the original dataset that is exclusive to it.\n .. note::\n Dataset is assumed to be of constant size.\n Arguments:\n dataset: Dataset used for sampling.\n num_replicas (optional): Number of processes participating in\n distributed training.\n rank (optional): Rank of the current process within num_replicas.\n \"\"\"\n\n def __init__(self, dataset, num_replicas=None, rank=None, local_rank=None, local_size=None, shuffle=True):\n if num_replicas is None:\n if not dist.is_available():\n raise RuntimeError(\"Requires distributed package to be available\")\n num_replicas = dist.get_world_size()\n if rank is None:\n if not dist.is_available():\n raise RuntimeError(\"Requires distributed package to be available\")\n rank = dist.get_rank()\n if local_rank is None:\n local_rank = int(os.environ.get('LOCAL_RANK', 0))\n if local_size is None:\n local_size = int(os.environ.get('LOCAL_SIZE', 1))\n self.dataset = dataset\n self.shuffle = shuffle\n self.num_replicas = num_replicas\n self.num_parts = local_size\n self.rank = rank\n self.local_rank = local_rank\n self.epoch = 0\n self.num_samples = int(math.ceil(len(self.dataset) * 1.0 / self.num_replicas))\n self.total_size = self.num_samples * self.num_replicas\n\n self.total_size_parts = self.num_samples * self.num_replicas // self.num_parts\n\n def __iter__(self):\n if self.shuffle:\n # deterministically shuffle based on epoch\n g = torch.Generator()\n g.manual_seed(self.epoch)\n indices = torch.randperm(len(self.dataset), generator=g).tolist()\n else:\n indices = torch.arange(len(self.dataset)).tolist()\n indices = [i for i in indices if i % self.num_parts == self.local_rank]\n\n # add extra samples to make it evenly divisible\n indices += indices[:(self.total_size_parts - len(indices))]\n assert len(indices) == self.total_size_parts\n\n # subsample\n indices = indices[self.rank // self.num_parts:self.total_size_parts:self.num_replicas // self.num_parts]\n assert len(indices) == self.num_samples\n\n return iter(indices)\n\n def __len__(self):\n return self.num_samples\n\n def set_epoch(self, epoch):\n self.epoch = epoch\n"
},
{
"path": "batchformer-v2/deformable-detr/datasets/torchvision_datasets/__init__.py",
"content": "# ------------------------------------------------------------------------\n# Deformable DETR\n# Copyright (c) 2020 SenseTime. All Rights Reserved.\n# Licensed under the Apache License, Version 2.0 [see LICENSE for details]\n# ------------------------------------------------------------------------\n\nfrom .coco import CocoDetection\n"
},
{
"path": "batchformer-v2/deformable-detr/datasets/torchvision_datasets/coco.py",
"content": "# ------------------------------------------------------------------------\n# Deformable DETR\n# Copyright (c) 2020 SenseTime. All Rights Reserved.\n# Licensed under the Apache License, Version 2.0 [see LICENSE for details]\n# ------------------------------------------------------------------------\n# Modified from torchvision\n# ------------------------------------------------------------------------\n\n\"\"\"\nCopy-Paste from torchvision, but add utility of caching images on memory\n\"\"\"\nfrom torchvision.datasets.vision import VisionDataset\nfrom PIL import Image\nimport os\nimport os.path\nimport tqdm\nfrom io import BytesIO\n\n\nclass CocoDetection(VisionDataset):\n \"\"\"`MS Coco Detection `_ Dataset.\n Args:\n root (string): Root directory where images are downloaded to.\n annFile (string): Path to json annotation file.\n transform (callable, optional): A function/transform that takes in an PIL image\n and returns a transformed version. E.g, ``transforms.ToTensor``\n target_transform (callable, optional): A function/transform that takes in the\n target and transforms it.\n transforms (callable, optional): A function/transform that takes input sample and its target as entry\n and returns a transformed version.\n \"\"\"\n\n def __init__(self, root, annFile, transform=None, target_transform=None, transforms=None,\n cache_mode=False, local_rank=0, local_size=1):\n super(CocoDetection, self).__init__(root, transforms, transform, target_transform)\n from pycocotools.coco import COCO\n self.coco = COCO(annFile)\n self.ids = list(sorted(self.coco.imgs.keys()))\n self.cache_mode = cache_mode\n self.local_rank = local_rank\n self.local_size = local_size\n if cache_mode:\n self.cache = {}\n self.cache_images()\n\n def cache_images(self):\n self.cache = {}\n for index, img_id in zip(tqdm.trange(len(self.ids)), self.ids):\n if index % self.local_size != self.local_rank:\n continue\n path = self.coco.loadImgs(img_id)[0]['file_name']\n with open(os.path.join(self.root, path), 'rb') as f:\n self.cache[path] = f.read()\n\n def get_image(self, path):\n if self.cache_mode:\n if path not in self.cache.keys():\n with open(os.path.join(self.root, path), 'rb') as f:\n self.cache[path] = f.read()\n return Image.open(BytesIO(self.cache[path])).convert('RGB')\n return Image.open(os.path.join(self.root, path)).convert('RGB')\n\n def __getitem__(self, index):\n \"\"\"\n Args:\n index (int): Index\n Returns:\n tuple: Tuple (image, target). target is the object returned by ``coco.loadAnns``.\n \"\"\"\n coco = self.coco\n img_id = self.ids[index]\n ann_ids = coco.getAnnIds(imgIds=img_id)\n target = coco.loadAnns(ann_ids)\n\n path = coco.loadImgs(img_id)[0]['file_name']\n\n img = self.get_image(path)\n if self.transforms is not None:\n img, target = self.transforms(img, target)\n\n return img, target\n\n def __len__(self):\n return len(self.ids)\n"
},
{
"path": "batchformer-v2/deformable-detr/datasets/transforms.py",
"content": "# ------------------------------------------------------------------------\n# Deformable DETR\n# Copyright (c) 2020 SenseTime. All Rights Reserved.\n# Licensed under the Apache License, Version 2.0 [see LICENSE for details]\n# ------------------------------------------------------------------------\n# Modified from DETR (https://github.com/facebookresearch/detr)\n# Copyright (c) Facebook, Inc. and its affiliates. All Rights Reserved\n# ------------------------------------------------------------------------\n\n\"\"\"\nTransforms and data augmentation for both image + bbox.\n\"\"\"\nimport random\n\nimport PIL\nimport torch\nimport torchvision.transforms as T\nimport torchvision.transforms.functional as F\n\nfrom util.box_ops import box_xyxy_to_cxcywh\nfrom util.misc import interpolate\n\n\ndef crop(image, target, region):\n cropped_image = F.crop(image, *region)\n\n target = target.copy()\n i, j, h, w = region\n\n # should we do something wrt the original size?\n target[\"size\"] = torch.tensor([h, w])\n\n fields = [\"labels\", \"area\", \"iscrowd\"]\n\n if \"boxes\" in target:\n boxes = target[\"boxes\"]\n max_size = torch.as_tensor([w, h], dtype=torch.float32)\n cropped_boxes = boxes - torch.as_tensor([j, i, j, i])\n cropped_boxes = torch.min(cropped_boxes.reshape(-1, 2, 2), max_size)\n cropped_boxes = cropped_boxes.clamp(min=0)\n area = (cropped_boxes[:, 1, :] - cropped_boxes[:, 0, :]).prod(dim=1)\n target[\"boxes\"] = cropped_boxes.reshape(-1, 4)\n target[\"area\"] = area\n fields.append(\"boxes\")\n\n if \"masks\" in target:\n # FIXME should we update the area here if there are no boxes?\n target['masks'] = target['masks'][:, i:i + h, j:j + w]\n fields.append(\"masks\")\n\n # remove elements for which the boxes or masks that have zero area\n if \"boxes\" in target or \"masks\" in target:\n # favor boxes selection when defining which elements to keep\n # this is compatible with previous implementation\n if \"boxes\" in target:\n cropped_boxes = target['boxes'].reshape(-1, 2, 2)\n keep = torch.all(cropped_boxes[:, 1, :] > cropped_boxes[:, 0, :], dim=1)\n else:\n keep = target['masks'].flatten(1).any(1)\n\n for field in fields:\n target[field] = target[field][keep]\n\n return cropped_image, target\n\n\ndef hflip(image, target):\n flipped_image = F.hflip(image)\n\n w, h = image.size\n\n target = target.copy()\n if \"boxes\" in target:\n boxes = target[\"boxes\"]\n boxes = boxes[:, [2, 1, 0, 3]] * torch.as_tensor([-1, 1, -1, 1]) + torch.as_tensor([w, 0, w, 0])\n target[\"boxes\"] = boxes\n\n if \"masks\" in target:\n target['masks'] = target['masks'].flip(-1)\n\n return flipped_image, target\n\n\ndef resize(image, target, size, max_size=None):\n # size can be min_size (scalar) or (w, h) tuple\n\n def get_size_with_aspect_ratio(image_size, size, max_size=None):\n w, h = image_size\n if max_size is not None:\n min_original_size = float(min((w, h)))\n max_original_size = float(max((w, h)))\n if max_original_size / min_original_size * size > max_size:\n size = int(round(max_size * min_original_size / max_original_size))\n\n if (w <= h and w == size) or (h <= w and h == size):\n return (h, w)\n\n if w < h:\n ow = size\n oh = int(size * h / w)\n else:\n oh = size\n ow = int(size * w / h)\n\n return (oh, ow)\n\n def get_size(image_size, size, max_size=None):\n if isinstance(size, (list, tuple)):\n return size[::-1]\n else:\n return get_size_with_aspect_ratio(image_size, size, max_size)\n\n size = get_size(image.size, size, max_size)\n rescaled_image = F.resize(image, size)\n\n if target is None:\n return rescaled_image, None\n\n ratios = tuple(float(s) / float(s_orig) for s, s_orig in zip(rescaled_image.size, image.size))\n ratio_width, ratio_height = ratios\n\n target = target.copy()\n if \"boxes\" in target:\n boxes = target[\"boxes\"]\n scaled_boxes = boxes * torch.as_tensor([ratio_width, ratio_height, ratio_width, ratio_height])\n target[\"boxes\"] = scaled_boxes\n\n if \"area\" in target:\n area = target[\"area\"]\n scaled_area = area * (ratio_width * ratio_height)\n target[\"area\"] = scaled_area\n\n h, w = size\n target[\"size\"] = torch.tensor([h, w])\n\n if \"masks\" in target:\n target['masks'] = interpolate(\n target['masks'][:, None].float(), size, mode=\"nearest\")[:, 0] > 0.5\n\n return rescaled_image, target\n\n\ndef pad(image, target, padding):\n # assumes that we only pad on the bottom right corners\n padded_image = F.pad(image, (0, 0, padding[0], padding[1]))\n if target is None:\n return padded_image, None\n target = target.copy()\n # should we do something wrt the original size?\n target[\"size\"] = torch.tensor(padded_image[::-1])\n if \"masks\" in target:\n target['masks'] = torch.nn.functional.pad(target['masks'], (0, padding[0], 0, padding[1]))\n return padded_image, target\n\n\nclass RandomCrop(object):\n def __init__(self, size):\n self.size = size\n\n def __call__(self, img, target):\n region = T.RandomCrop.get_params(img, self.size)\n return crop(img, target, region)\n\n\nclass RandomSizeCrop(object):\n def __init__(self, min_size: int, max_size: int):\n self.min_size = min_size\n self.max_size = max_size\n\n def __call__(self, img: PIL.Image.Image, target: dict):\n w = random.randint(self.min_size, min(img.width, self.max_size))\n h = random.randint(self.min_size, min(img.height, self.max_size))\n region = T.RandomCrop.get_params(img, [h, w])\n return crop(img, target, region)\n\n\nclass CenterCrop(object):\n def __init__(self, size):\n self.size = size\n\n def __call__(self, img, target):\n image_width, image_height = img.size\n crop_height, crop_width = self.size\n crop_top = int(round((image_height - crop_height) / 2.))\n crop_left = int(round((image_width - crop_width) / 2.))\n return crop(img, target, (crop_top, crop_left, crop_height, crop_width))\n\n\nclass RandomHorizontalFlip(object):\n def __init__(self, p=0.5):\n self.p = p\n\n def __call__(self, img, target):\n if random.random() < self.p:\n return hflip(img, target)\n return img, target\n\n\nclass RandomResize(object):\n def __init__(self, sizes, max_size=None):\n assert isinstance(sizes, (list, tuple))\n self.sizes = sizes\n self.max_size = max_size\n\n def __call__(self, img, target=None):\n size = random.choice(self.sizes)\n return resize(img, target, size, self.max_size)\n\n\nclass RandomPad(object):\n def __init__(self, max_pad):\n self.max_pad = max_pad\n\n def __call__(self, img, target):\n pad_x = random.randint(0, self.max_pad)\n pad_y = random.randint(0, self.max_pad)\n return pad(img, target, (pad_x, pad_y))\n\n\nclass RandomSelect(object):\n \"\"\"\n Randomly selects between transforms1 and transforms2,\n with probability p for transforms1 and (1 - p) for transforms2\n \"\"\"\n def __init__(self, transforms1, transforms2, p=0.5):\n self.transforms1 = transforms1\n self.transforms2 = transforms2\n self.p = p\n\n def __call__(self, img, target):\n if random.random() < self.p:\n return self.transforms1(img, target)\n return self.transforms2(img, target)\n\n\nclass ToTensor(object):\n def __call__(self, img, target):\n return F.to_tensor(img), target\n\n\nclass RandomErasing(object):\n\n def __init__(self, *args, **kwargs):\n self.eraser = T.RandomErasing(*args, **kwargs)\n\n def __call__(self, img, target):\n return self.eraser(img), target\n\n\nclass Normalize(object):\n def __init__(self, mean, std):\n self.mean = mean\n self.std = std\n\n def __call__(self, image, target=None):\n image = F.normalize(image, mean=self.mean, std=self.std)\n if target is None:\n return image, None\n target = target.copy()\n h, w = image.shape[-2:]\n if \"boxes\" in target:\n boxes = target[\"boxes\"]\n boxes = box_xyxy_to_cxcywh(boxes)\n boxes = boxes / torch.tensor([w, h, w, h], dtype=torch.float32)\n target[\"boxes\"] = boxes\n return image, target\n\n\nclass Compose(object):\n def __init__(self, transforms):\n self.transforms = transforms\n\n def __call__(self, image, target):\n for t in self.transforms:\n image, target = t(image, target)\n return image, target\n\n def __repr__(self):\n format_string = self.__class__.__name__ + \"(\"\n for t in self.transforms:\n format_string += \"\\n\"\n format_string += \" {0}\".format(t)\n format_string += \"\\n)\"\n return format_string\n"
},
{
"path": "batchformer-v2/deformable-detr/docs/changelog.md",
"content": "## Changelog\n\n**[2020.12.07]** Fix a bug of sampling offset normalization (see [this issue](https://github.com/fundamentalvision/Deformable-DETR/issues/6)) in the MSDeformAttn module. The final accuracy on COCO is slightly improved. Code and pre-trained models have been updated. This bug only occurs in this released version but not in the original implementation used in our paper."
},
{
"path": "batchformer-v2/deformable-detr/engine.py",
"content": "# ------------------------------------------------------------------------\n# Deformable DETR\n# Copyright (c) 2020 SenseTime. All Rights Reserved.\n# Licensed under the Apache License, Version 2.0 [see LICENSE for details]\n# ------------------------------------------------------------------------\n# Modified from DETR (https://github.com/facebookresearch/detr)\n# Copyright (c) Facebook, Inc. and its affiliates. All Rights Reserved\n# ------------------------------------------------------------------------\n\n\"\"\"\nTrain and eval functions used in main.py\n\"\"\"\nimport math\nimport os\nimport sys\nfrom typing import Iterable\n\nimport torch\nimport util.misc as utils\nfrom datasets.coco_eval import CocoEvaluator\nfrom datasets.panoptic_eval import PanopticEvaluator\nfrom datasets.data_prefetcher import data_prefetcher\n\n\ndef train_one_epoch(model: torch.nn.Module, criterion: torch.nn.Module,\n data_loader: Iterable, optimizer: torch.optim.Optimizer,\n device: torch.device, epoch: int, max_norm: float = 0):\n model.train()\n criterion.train()\n metric_logger = utils.MetricLogger(delimiter=\" \")\n metric_logger.add_meter('lr', utils.SmoothedValue(window_size=1, fmt='{value:.6f}'))\n metric_logger.add_meter('class_error', utils.SmoothedValue(window_size=1, fmt='{value:.2f}'))\n metric_logger.add_meter('grad_norm', utils.SmoothedValue(window_size=1, fmt='{value:.2f}'))\n header = 'Epoch: [{}]'.format(epoch)\n print_freq = 10\n\n prefetcher = data_prefetcher(data_loader, device, prefetch=True)\n samples, targets = prefetcher.next()\n\n # for samples, targets in metric_logger.log_every(data_loader, print_freq, header):\n for _ in metric_logger.log_every(range(len(data_loader)), print_freq, header):\n outputs = model(samples)\n if len(outputs['pred_logits']) > len(targets): # Here, we need to set the labels for the batchformerv2 stream/batch\n targets = targets + targets # targets is a list\n loss_dict = criterion(outputs, targets)\n weight_dict = criterion.weight_dict\n losses = sum(loss_dict[k] * weight_dict[k] for k in loss_dict.keys() if k in weight_dict)\n\n # reduce losses over all GPUs for logging purposes\n loss_dict_reduced = utils.reduce_dict(loss_dict)\n loss_dict_reduced_unscaled = {f'{k}_unscaled': v\n for k, v in loss_dict_reduced.items()}\n loss_dict_reduced_scaled = {k: v * weight_dict[k]\n for k, v in loss_dict_reduced.items() if k in weight_dict}\n losses_reduced_scaled = sum(loss_dict_reduced_scaled.values())\n\n loss_value = losses_reduced_scaled.item()\n\n if not math.isfinite(loss_value):\n print(\"Loss is {}, stopping training\".format(loss_value))\n print(loss_dict_reduced)\n sys.exit(1)\n\n optimizer.zero_grad()\n losses.backward()\n if max_norm > 0:\n grad_total_norm = torch.nn.utils.clip_grad_norm_(model.parameters(), max_norm)\n else:\n grad_total_norm = utils.get_total_grad_norm(model.parameters(), max_norm)\n optimizer.step()\n\n metric_logger.update(loss=loss_value, **loss_dict_reduced_scaled, **loss_dict_reduced_unscaled)\n metric_logger.update(class_error=loss_dict_reduced['class_error'])\n metric_logger.update(lr=optimizer.param_groups[0][\"lr\"])\n metric_logger.update(grad_norm=grad_total_norm)\n\n samples, targets = prefetcher.next()\n # gather the stats from all processes\n metric_logger.synchronize_between_processes()\n print(\"Averaged stats:\", metric_logger)\n return {k: meter.global_avg for k, meter in metric_logger.meters.items()}\n\n\n@torch.no_grad()\ndef evaluate(model, criterion, postprocessors, data_loader, base_ds, device, output_dir):\n model.eval()\n criterion.eval()\n\n metric_logger = utils.MetricLogger(delimiter=\" \")\n metric_logger.add_meter('class_error', utils.SmoothedValue(window_size=1, fmt='{value:.2f}'))\n header = 'Test:'\n\n iou_types = tuple(k for k in ('segm', 'bbox') if k in postprocessors.keys())\n coco_evaluator = CocoEvaluator(base_ds, iou_types)\n # coco_evaluator.coco_eval[iou_types[0]].params.iouThrs = [0, 0.1, 0.5, 0.75]\n\n panoptic_evaluator = None\n if 'panoptic' in postprocessors.keys():\n panoptic_evaluator = PanopticEvaluator(\n data_loader.dataset.ann_file,\n data_loader.dataset.ann_folder,\n output_dir=os.path.join(output_dir, \"panoptic_eval\"),\n )\n\n for samples, targets in metric_logger.log_every(data_loader, 10, header):\n samples = samples.to(device)\n targets = [{k: v.to(device) for k, v in t.items()} for t in targets]\n\n outputs = model(samples)\n loss_dict = criterion(outputs, targets)\n weight_dict = criterion.weight_dict\n\n # reduce losses over all GPUs for logging purposes\n loss_dict_reduced = utils.reduce_dict(loss_dict)\n loss_dict_reduced_scaled = {k: v * weight_dict[k]\n for k, v in loss_dict_reduced.items() if k in weight_dict}\n loss_dict_reduced_unscaled = {f'{k}_unscaled': v\n for k, v in loss_dict_reduced.items()}\n metric_logger.update(loss=sum(loss_dict_reduced_scaled.values()),\n **loss_dict_reduced_scaled,\n **loss_dict_reduced_unscaled)\n metric_logger.update(class_error=loss_dict_reduced['class_error'])\n\n orig_target_sizes = torch.stack([t[\"orig_size\"] for t in targets], dim=0)\n results = postprocessors['bbox'](outputs, orig_target_sizes)\n if 'segm' in postprocessors.keys():\n target_sizes = torch.stack([t[\"size\"] for t in targets], dim=0)\n results = postprocessors['segm'](results, outputs, orig_target_sizes, target_sizes)\n res = {target['image_id'].item(): output for target, output in zip(targets, results)}\n if coco_evaluator is not None:\n coco_evaluator.update(res)\n\n if panoptic_evaluator is not None:\n res_pano = postprocessors[\"panoptic\"](outputs, target_sizes, orig_target_sizes)\n for i, target in enumerate(targets):\n image_id = target[\"image_id\"].item()\n file_name = f\"{image_id:012d}.png\"\n res_pano[i][\"image_id\"] = image_id\n res_pano[i][\"file_name\"] = file_name\n\n panoptic_evaluator.update(res_pano)\n\n # gather the stats from all processes\n metric_logger.synchronize_between_processes()\n print(\"Averaged stats:\", metric_logger)\n if coco_evaluator is not None:\n coco_evaluator.synchronize_between_processes()\n if panoptic_evaluator is not None:\n panoptic_evaluator.synchronize_between_processes()\n\n # accumulate predictions from all images\n if coco_evaluator is not None:\n coco_evaluator.accumulate()\n coco_evaluator.summarize()\n panoptic_res = None\n if panoptic_evaluator is not None:\n panoptic_res = panoptic_evaluator.summarize()\n stats = {k: meter.global_avg for k, meter in metric_logger.meters.items()}\n if coco_evaluator is not None:\n if 'bbox' in postprocessors.keys():\n stats['coco_eval_bbox'] = coco_evaluator.coco_eval['bbox'].stats.tolist()\n if 'segm' in postprocessors.keys():\n stats['coco_eval_masks'] = coco_evaluator.coco_eval['segm'].stats.tolist()\n if panoptic_res is not None:\n stats['PQ_all'] = panoptic_res[\"All\"]\n stats['PQ_th'] = panoptic_res[\"Things\"]\n stats['PQ_st'] = panoptic_res[\"Stuff\"]\n return stats, coco_evaluator\n"
},
{
"path": "batchformer-v2/deformable-detr/main.py",
"content": "# ------------------------------------------------------------------------\n# Deformable DETR\n# Copyright (c) 2020 SenseTime. All Rights Reserved.\n# Licensed under the Apache License, Version 2.0 [see LICENSE for details]\n# ------------------------------------------------------------------------\n# Modified from DETR (https://github.com/facebookresearch/detr)\n# Copyright (c) Facebook, Inc. and its affiliates. All Rights Reserved\n# ------------------------------------------------------------------------\n\n\nimport argparse\nimport datetime\nimport json\nimport random\nimport time\nfrom pathlib import Path\n\nimport numpy as np\nimport torch\nfrom torch.utils.data import DataLoader\nimport datasets\nimport util.misc as utils\nimport datasets.samplers as samplers\nfrom datasets import build_dataset, get_coco_api_from_dataset\nfrom engine import evaluate, train_one_epoch\nfrom models import build_model\n\n\ndef get_args_parser():\n parser = argparse.ArgumentParser('Deformable DETR Detector', add_help=False)\n parser.add_argument('--lr', default=2e-4, type=float)\n parser.add_argument('--lr_backbone_names', default=[\"backbone.0\"], type=str, nargs='+')\n parser.add_argument('--lr_backbone', default=2e-5, type=float)\n parser.add_argument('--lr_linear_proj_names', default=['reference_points', 'sampling_offsets'], type=str, nargs='+')\n parser.add_argument('--lr_linear_proj_mult', default=0.1, type=float)\n parser.add_argument('--batch_size', default=2, type=int)\n parser.add_argument('--weight_decay', default=1e-4, type=float)\n parser.add_argument('--epochs', default=50, type=int)\n parser.add_argument('--lr_drop', default=40, type=int)\n parser.add_argument('--lr_drop_epochs', default=None, type=int, nargs='+')\n parser.add_argument('--clip_max_norm', default=0.1, type=float,\n help='gradient clipping max norm')\n\n\n parser.add_argument('--sgd', action='store_true')\n\n # Variants of Deformable DETR\n parser.add_argument('--with_box_refine', default=False, action='store_true')\n parser.add_argument('--two_stage', default=False, action='store_true')\n\n # Model parameters\n parser.add_argument('--frozen_weights', type=str, default=None,\n help=\"Path to the pretrained model. If set, only the mask head will be trained\")\n\n # * Backbone\n parser.add_argument('--backbone', default='resnet50', type=str,\n help=\"Name of the convolutional backbone to use\")\n parser.add_argument('--dilation', action='store_true',\n help=\"If true, we replace stride with dilation in the last convolutional block (DC5)\")\n parser.add_argument('--position_embedding', default='sine', type=str, choices=('sine', 'learned'),\n help=\"Type of positional embedding to use on top of the image features\")\n parser.add_argument('--position_embedding_scale', default=2 * np.pi, type=float,\n help=\"position / size * scale\")\n parser.add_argument('--num_feature_levels', default=4, type=int, help='number of feature levels')\n\n # * Transformer\n parser.add_argument('--enc_layers', default=6, type=int,\n help=\"Number of encoding layers in the transformer\")\n parser.add_argument('--dec_layers', default=6, type=int,\n help=\"Number of decoding layers in the transformer\")\n parser.add_argument('--dim_feedforward', default=1024, type=int,\n help=\"Intermediate size of the feedforward layers in the transformer blocks\")\n parser.add_argument('--hidden_dim', default=256, type=int,\n help=\"Size of the embeddings (dimension of the transformer)\")\n parser.add_argument('--dropout', default=0.1, type=float,\n help=\"Dropout applied in the transformer\")\n parser.add_argument('--nheads', default=8, type=int,\n help=\"Number of attention heads inside the transformer's attentions\")\n parser.add_argument('--num_queries', default=300, type=int,\n help=\"Number of query slots\")\n parser.add_argument('--dec_n_points', default=4, type=int)\n parser.add_argument('--enc_n_points', default=4, type=int)\n\n # * Segmentation\n parser.add_argument('--masks', action='store_true',\n help=\"Train segmentation head if the flag is provided\")\n\n # Loss\n parser.add_argument('--no_aux_loss', dest='aux_loss', action='store_false',\n help=\"Disables auxiliary decoding losses (loss at each layer)\")\n\n # * Matcher\n parser.add_argument('--set_cost_class', default=2, type=float,\n help=\"Class coefficient in the matching cost\")\n parser.add_argument('--set_cost_bbox', default=5, type=float,\n help=\"L1 box coefficient in the matching cost\")\n parser.add_argument('--set_cost_giou', default=2, type=float,\n help=\"giou box coefficient in the matching cost\")\n\n # * Loss coefficients\n parser.add_argument('--mask_loss_coef', default=1, type=float)\n parser.add_argument('--dice_loss_coef', default=1, type=float)\n parser.add_argument('--cls_loss_coef', default=2, type=float)\n parser.add_argument('--bbox_loss_coef', default=5, type=float)\n parser.add_argument('--giou_loss_coef', default=2, type=float)\n parser.add_argument('--focal_alpha', default=0.25, type=float)\n\n # dataset parameters\n parser.add_argument('--dataset_file', default='coco')\n parser.add_argument('--coco_path', default='./data/coco', type=str)\n parser.add_argument('--coco_panoptic_path', type=str)\n parser.add_argument('--remove_difficult', action='store_true')\n\n parser.add_argument('--output_dir', default='',\n help='path where to save, empty for no saving')\n parser.add_argument('--device', default='cuda',\n help='device to use for training / testing')\n parser.add_argument('--seed', default=42, type=int)\n parser.add_argument('--resume', default='', help='resume from checkpoint')\n parser.add_argument('--start_epoch', default=0, type=int, metavar='N',\n help='start epoch')\n parser.add_argument('--eval', action='store_true')\n parser.add_argument('--num_workers', default=2, type=int)\n parser.add_argument('--bf', default=0, type=int)\n parser.add_argument('--share_bf', default=1, type=int)\n parser.add_argument('--bt_num_layers', default=1, type=int)\n parser.add_argument('--insert_idx', action='append', type=int,\n help='insert idx list')\n parser.add_argument('--eval_bf', action='store_true', help='batch inference with batchformerv2')\n parser.add_argument('--drop_path', default=0., type=float)\n parser.add_argument('--cache_mode', default=False, action='store_true', help='whether to cache images on memory')\n\n return parser\n\n\ndef main(args):\n utils.init_distributed_mode(args)\n print(\"git:\\n {}\\n\".format(utils.get_sha()))\n\n if args.frozen_weights is not None:\n assert args.masks, \"Frozen training is meant for segmentation only\"\n print(args)\n\n device = torch.device(args.device)\n\n # fix the seed for reproducibility\n seed = args.seed + utils.get_rank()\n torch.manual_seed(seed)\n np.random.seed(seed)\n random.seed(seed)\n\n model, criterion, postprocessors = build_model(args)\n model.to(device)\n\n model_without_ddp = model\n n_parameters = sum(p.numel() for p in model.parameters() if p.requires_grad)\n print('number of params:', n_parameters)\n\n dataset_train = build_dataset(image_set='train', args=args)\n dataset_val = build_dataset(image_set='val', args=args)\n\n if args.distributed:\n if args.cache_mode:\n sampler_train = samplers.NodeDistributedSampler(dataset_train)\n sampler_val = samplers.NodeDistributedSampler(dataset_val, shuffle=False)\n else:\n sampler_train = samplers.DistributedSampler(dataset_train)\n sampler_val = samplers.DistributedSampler(dataset_val, shuffle=False)\n else:\n sampler_train = torch.utils.data.RandomSampler(dataset_train)\n sampler_val = torch.utils.data.SequentialSampler(dataset_val)\n\n batch_sampler_train = torch.utils.data.BatchSampler(\n sampler_train, args.batch_size, drop_last=True)\n\n data_loader_train = DataLoader(dataset_train, batch_sampler=batch_sampler_train,\n collate_fn=utils.collate_fn, num_workers=args.num_workers,\n pin_memory=True)\n data_loader_val = DataLoader(dataset_val, args.batch_size, sampler=sampler_val,\n drop_last=False, collate_fn=utils.collate_fn, num_workers=args.num_workers,\n pin_memory=True)\n\n # lr_backbone_names = [\"backbone.0\", \"backbone.neck\", \"input_proj\", \"transformer.encoder\"]\n def match_name_keywords(n, name_keywords):\n out = False\n for b in name_keywords:\n if b in n:\n out = True\n break\n return out\n\n\n param_dicts = [\n {\n \"params\":\n [p for n, p in model_without_ddp.named_parameters()\n if not match_name_keywords(n, args.lr_backbone_names) and not match_name_keywords(n, args.lr_linear_proj_names) and p.requires_grad],\n \"lr\": args.lr,\n },\n {\n \"params\": [p for n, p in model_without_ddp.named_parameters() if match_name_keywords(n, args.lr_backbone_names) and p.requires_grad],\n \"lr\": args.lr_backbone,\n },\n {\n \"params\": [p for n, p in model_without_ddp.named_parameters() if match_name_keywords(n, args.lr_linear_proj_names) and p.requires_grad],\n \"lr\": args.lr * args.lr_linear_proj_mult,\n }\n ]\n if args.sgd:\n optimizer = torch.optim.SGD(param_dicts, lr=args.lr, momentum=0.9,\n weight_decay=args.weight_decay)\n else:\n optimizer = torch.optim.AdamW(param_dicts, lr=args.lr,\n weight_decay=args.weight_decay)\n lr_scheduler = torch.optim.lr_scheduler.StepLR(optimizer, args.lr_drop)\n\n if args.distributed:\n model = torch.nn.parallel.DistributedDataParallel(model, device_ids=[args.gpu])\n model_without_ddp = model.module\n\n if args.dataset_file == \"coco_panoptic\":\n # We also evaluate AP during panoptic training, on original coco DS\n coco_val = datasets.coco.build(\"val\", args)\n base_ds = get_coco_api_from_dataset(coco_val)\n else:\n base_ds = get_coco_api_from_dataset(dataset_val)\n\n if args.frozen_weights is not None:\n checkpoint = torch.load(args.frozen_weights, map_location='cpu')\n model_without_ddp.detr.load_state_dict(checkpoint['model'])\n\n output_dir = Path(args.output_dir)\n if args.resume:\n if args.resume.startswith('https'):\n checkpoint = torch.hub.load_state_dict_from_url(\n args.resume, map_location='cpu', check_hash=True)\n else:\n checkpoint = torch.load(args.resume, map_location='cpu')\n missing_keys, unexpected_keys = model_without_ddp.load_state_dict(checkpoint['model'], strict=False)\n unexpected_keys = [k for k in unexpected_keys if not (k.endswith('total_params') or k.endswith('total_ops'))]\n if len(missing_keys) > 0:\n print('Missing Keys: {}'.format(missing_keys))\n if len(unexpected_keys) > 0:\n print('Unexpected Keys: {}'.format(unexpected_keys))\n if not args.eval and 'optimizer' in checkpoint and 'lr_scheduler' in checkpoint and 'epoch' in checkpoint:\n import copy\n p_groups = copy.deepcopy(optimizer.param_groups)\n optimizer.load_state_dict(checkpoint['optimizer'])\n for pg, pg_old in zip(optimizer.param_groups, p_groups):\n pg['lr'] = pg_old['lr']\n pg['initial_lr'] = pg_old['initial_lr']\n print(optimizer.param_groups)\n lr_scheduler.load_state_dict(checkpoint['lr_scheduler'])\n # todo: this is a hack for doing experiment that resume from checkpoint and also modify lr scheduler (e.g., decrease lr in advance).\n args.override_resumed_lr_drop = True\n if args.override_resumed_lr_drop:\n print('Warning: (hack) args.override_resumed_lr_drop is set to True, so args.lr_drop would override lr_drop in resumed lr_scheduler.')\n lr_scheduler.step_size = args.lr_drop\n lr_scheduler.base_lrs = list(map(lambda group: group['initial_lr'], optimizer.param_groups))\n lr_scheduler.step(lr_scheduler.last_epoch)\n args.start_epoch = checkpoint['epoch'] + 1\n # check the resumed model\n if not args.eval:\n test_stats, coco_evaluator = evaluate(\n model, criterion, postprocessors, data_loader_val, base_ds, device, args.output_dir\n )\n \n if args.eval:\n test_stats, coco_evaluator = evaluate(model, criterion, postprocessors,\n data_loader_val, base_ds, device, args.output_dir)\n if args.output_dir:\n utils.save_on_master(coco_evaluator.coco_eval[\"bbox\"].eval, output_dir / \"eval.pth\")\n return\n\n print(\"Start training\")\n start_time = time.time()\n for epoch in range(args.start_epoch, args.epochs):\n if args.distributed:\n sampler_train.set_epoch(epoch)\n train_stats = train_one_epoch(\n model, criterion, data_loader_train, optimizer, device, epoch, args.clip_max_norm)\n lr_scheduler.step()\n if args.output_dir:\n checkpoint_paths = [output_dir / 'checkpoint.pth']\n # extra checkpoint before LR drop and every 5 epochs\n if (epoch + 1) % args.lr_drop == 0 or (epoch + 1) % 5 == 0:\n checkpoint_paths.append(output_dir / f'checkpoint{epoch:04}.pth')\n for checkpoint_path in checkpoint_paths:\n utils.save_on_master({\n 'model': model_without_ddp.state_dict(),\n 'optimizer': optimizer.state_dict(),\n 'lr_scheduler': lr_scheduler.state_dict(),\n 'epoch': epoch,\n 'args': args,\n }, checkpoint_path)\n\n test_stats, coco_evaluator = evaluate(\n model, criterion, postprocessors, data_loader_val, base_ds, device, args.output_dir\n )\n\n log_stats = {**{f'train_{k}': v for k, v in train_stats.items()},\n **{f'test_{k}': v for k, v in test_stats.items()},\n 'epoch': epoch,\n 'n_parameters': n_parameters}\n\n if args.output_dir and utils.is_main_process():\n with (output_dir / \"log.txt\").open(\"a\") as f:\n f.write(json.dumps(log_stats) + \"\\n\")\n\n # for evaluation logs\n if coco_evaluator is not None:\n (output_dir / 'eval').mkdir(exist_ok=True)\n if \"bbox\" in coco_evaluator.coco_eval:\n filenames = ['latest.pth']\n if epoch % 50 == 0:\n filenames.append(f'{epoch:03}.pth')\n for name in filenames:\n torch.save(coco_evaluator.coco_eval[\"bbox\"].eval,\n output_dir / \"eval\" / name)\n\n total_time = time.time() - start_time\n total_time_str = str(datetime.timedelta(seconds=int(total_time)))\n print('Training time {}'.format(total_time_str))\n\n\nif __name__ == '__main__':\n parser = argparse.ArgumentParser('Deformable DETR training and evaluation script', parents=[get_args_parser()])\n args = parser.parse_args()\n if args.output_dir:\n Path(args.output_dir).mkdir(parents=True, exist_ok=True)\n main(args)\n"
},
{
"path": "batchformer-v2/deformable-detr/models/__init__.py",
"content": "# ------------------------------------------------------------------------\n# Deformable DETR\n# Copyright (c) 2020 SenseTime. All Rights Reserved.\n# Licensed under the Apache License, Version 2.0 [see LICENSE for details]\n# ------------------------------------------------------------------------\n# Modified from DETR (https://github.com/facebookresearch/detr)\n# Copyright (c) Facebook, Inc. and its affiliates. All Rights Reserved\n# ------------------------------------------------------------------------\n\nfrom .deformable_detr import build\n\n\ndef build_model(args):\n return build(args)\n\n"
},
{
"path": "batchformer-v2/deformable-detr/models/backbone.py",
"content": "# ------------------------------------------------------------------------\n# Deformable DETR\n# Copyright (c) 2020 SenseTime. All Rights Reserved.\n# Licensed under the Apache License, Version 2.0 [see LICENSE for details]\n# ------------------------------------------------------------------------\n# Modified from DETR (https://github.com/facebookresearch/detr)\n# Copyright (c) Facebook, Inc. and its affiliates. All Rights Reserved\n# ------------------------------------------------------------------------\n\n\"\"\"\nBackbone modules.\n\"\"\"\nfrom collections import OrderedDict\n\nimport torch\nimport torch.nn.functional as F\nimport torchvision\nfrom torch import nn\nfrom torchvision.models._utils import IntermediateLayerGetter\nfrom typing import Dict, List\n\nfrom util.misc import NestedTensor, is_main_process\n\nfrom .position_encoding import build_position_encoding\n\n\nclass FrozenBatchNorm2d(torch.nn.Module):\n \"\"\"\n BatchNorm2d where the batch statistics and the affine parameters are fixed.\n\n Copy-paste from torchvision.misc.ops with added eps before rqsrt,\n without which any other models than torchvision.models.resnet[18,34,50,101]\n produce nans.\n \"\"\"\n\n def __init__(self, n, eps=1e-5):\n super(FrozenBatchNorm2d, self).__init__()\n self.register_buffer(\"weight\", torch.ones(n))\n self.register_buffer(\"bias\", torch.zeros(n))\n self.register_buffer(\"running_mean\", torch.zeros(n))\n self.register_buffer(\"running_var\", torch.ones(n))\n self.eps = eps\n\n def _load_from_state_dict(self, state_dict, prefix, local_metadata, strict,\n missing_keys, unexpected_keys, error_msgs):\n num_batches_tracked_key = prefix + 'num_batches_tracked'\n if num_batches_tracked_key in state_dict:\n del state_dict[num_batches_tracked_key]\n\n super(FrozenBatchNorm2d, self)._load_from_state_dict(\n state_dict, prefix, local_metadata, strict,\n missing_keys, unexpected_keys, error_msgs)\n\n def forward(self, x):\n # move reshapes to the beginning\n # to make it fuser-friendly\n w = self.weight.reshape(1, -1, 1, 1)\n b = self.bias.reshape(1, -1, 1, 1)\n rv = self.running_var.reshape(1, -1, 1, 1)\n rm = self.running_mean.reshape(1, -1, 1, 1)\n eps = self.eps\n scale = w * (rv + eps).rsqrt()\n bias = b - rm * scale\n return x * scale + bias\n\n\nclass BackboneBase(nn.Module):\n\n def __init__(self, backbone: nn.Module, train_backbone: bool, return_interm_layers: bool):\n super().__init__()\n for name, parameter in backbone.named_parameters():\n if not train_backbone or 'layer2' not in name and 'layer3' not in name and 'layer4' not in name:\n parameter.requires_grad_(False)\n if return_interm_layers:\n # return_layers = {\"layer1\": \"0\", \"layer2\": \"1\", \"layer3\": \"2\", \"layer4\": \"3\"}\n return_layers = {\"layer2\": \"0\", \"layer3\": \"1\", \"layer4\": \"2\"}\n self.strides = [8, 16, 32]\n self.num_channels = [512, 1024, 2048]\n else:\n return_layers = {'layer4': \"0\"}\n self.strides = [32]\n self.num_channels = [2048]\n self.body = IntermediateLayerGetter(backbone, return_layers=return_layers)\n\n def forward(self, tensor_list: NestedTensor):\n xs = self.body(tensor_list.tensors)\n out: Dict[str, NestedTensor] = {}\n for name, x in xs.items():\n m = tensor_list.mask\n assert m is not None\n mask = F.interpolate(m[None].float(), size=x.shape[-2:]).to(torch.bool)[0]\n out[name] = NestedTensor(x, mask)\n return out\n\n\nclass Backbone(BackboneBase):\n \"\"\"ResNet backbone with frozen BatchNorm.\"\"\"\n def __init__(self, name: str,\n train_backbone: bool,\n return_interm_layers: bool,\n dilation: bool):\n norm_layer = FrozenBatchNorm2d\n backbone = getattr(torchvision.models, name)(\n replace_stride_with_dilation=[False, False, dilation],\n pretrained=is_main_process(), norm_layer=norm_layer)\n assert name not in ('resnet18', 'resnet34'), \"number of channels are hard coded\"\n super().__init__(backbone, train_backbone, return_interm_layers)\n if dilation:\n self.strides[-1] = self.strides[-1] // 2\n\n\nclass Joiner(nn.Sequential):\n def __init__(self, backbone, position_embedding):\n super().__init__(backbone, position_embedding)\n self.strides = backbone.strides\n self.num_channels = backbone.num_channels\n\n def forward(self, tensor_list: NestedTensor):\n xs = self[0](tensor_list)\n out: List[NestedTensor] = []\n pos = []\n for name, x in sorted(xs.items()):\n out.append(x)\n\n # position encoding\n for x in out:\n pos.append(self[1](x).to(x.tensors.dtype))\n\n return out, pos\n\n\ndef build_backbone(args):\n position_embedding = build_position_encoding(args)\n train_backbone = args.lr_backbone > 0\n return_interm_layers = args.masks or (args.num_feature_levels > 1)\n backbone = Backbone(args.backbone, train_backbone, return_interm_layers, args.dilation)\n model = Joiner(backbone, position_embedding)\n return model\n"
},
{
"path": "batchformer-v2/deformable-detr/models/deformable_detr.py",
"content": "# ------------------------------------------------------------------------\n# Deformable DETR\n# Copyright (c) 2020 SenseTime. All Rights Reserved.\n# Licensed under the Apache License, Version 2.0 [see LICENSE for details]\n# ------------------------------------------------------------------------\n# Modified from DETR (https://github.com/facebookresearch/detr)\n# Copyright (c) Facebook, Inc. and its affiliates. All Rights Reserved\n# ------------------------------------------------------------------------\n\n\"\"\"\nDeformable DETR model and criterion classes.\n\"\"\"\nimport torch\nimport torch.nn.functional as F\nfrom torch import nn\nimport math\n\nfrom util import box_ops\nfrom util.misc import (NestedTensor, nested_tensor_from_tensor_list,\n accuracy, get_world_size, interpolate,\n is_dist_avail_and_initialized, inverse_sigmoid)\n\nfrom .backbone import build_backbone\nfrom .matcher import build_matcher\nfrom .segmentation import (DETRsegm, PostProcessPanoptic, PostProcessSegm,\n dice_loss, sigmoid_focal_loss)\nfrom .deformable_transformer import build_deforamble_transformer\nimport copy\n\n\ndef _get_clones(module, N):\n return nn.ModuleList([copy.deepcopy(module) for i in range(N)])\n\n\nclass DeformableDETR(nn.Module):\n \"\"\" This is the Deformable DETR module that performs object detection \"\"\"\n def __init__(self, backbone, transformer, num_classes, num_queries, num_feature_levels,\n aux_loss=True, with_box_refine=False, two_stage=False):\n \"\"\" Initializes the model.\n Parameters:\n backbone: torch module of the backbone to be used. See backbone.py\n transformer: torch module of the transformer architecture. See transformer.py\n num_classes: number of object classes\n num_queries: number of object queries, ie detection slot. This is the maximal number of objects\n DETR can detect in a single image. For COCO, we recommend 100 queries.\n aux_loss: True if auxiliary decoding losses (loss at each decoder layer) are to be used.\n with_box_refine: iterative bounding box refinement\n two_stage: two-stage Deformable DETR\n \"\"\"\n super().__init__()\n self.num_queries = num_queries\n self.transformer = transformer\n hidden_dim = transformer.d_model\n self.class_embed = nn.Linear(hidden_dim, num_classes)\n self.bbox_embed = MLP(hidden_dim, hidden_dim, 4, 3)\n self.num_feature_levels = num_feature_levels\n if not two_stage:\n self.query_embed = nn.Embedding(num_queries, hidden_dim*2)\n if num_feature_levels > 1:\n num_backbone_outs = len(backbone.strides)\n input_proj_list = []\n for _ in range(num_backbone_outs):\n in_channels = backbone.num_channels[_]\n input_proj_list.append(nn.Sequential(\n nn.Conv2d(in_channels, hidden_dim, kernel_size=1),\n nn.GroupNorm(32, hidden_dim),\n ))\n for _ in range(num_feature_levels - num_backbone_outs):\n input_proj_list.append(nn.Sequential(\n nn.Conv2d(in_channels, hidden_dim, kernel_size=3, stride=2, padding=1),\n nn.GroupNorm(32, hidden_dim),\n ))\n in_channels = hidden_dim\n self.input_proj = nn.ModuleList(input_proj_list)\n else:\n self.input_proj = nn.ModuleList([\n nn.Sequential(\n nn.Conv2d(backbone.num_channels[0], hidden_dim, kernel_size=1),\n nn.GroupNorm(32, hidden_dim),\n )])\n self.backbone = backbone\n self.aux_loss = aux_loss\n self.with_box_refine = with_box_refine\n self.two_stage = two_stage\n\n prior_prob = 0.01\n bias_value = -math.log((1 - prior_prob) / prior_prob)\n self.class_embed.bias.data = torch.ones(num_classes) * bias_value\n nn.init.constant_(self.bbox_embed.layers[-1].weight.data, 0)\n nn.init.constant_(self.bbox_embed.layers[-1].bias.data, 0)\n for proj in self.input_proj:\n nn.init.xavier_uniform_(proj[0].weight, gain=1)\n nn.init.constant_(proj[0].bias, 0)\n\n # if two-stage, the last class_embed and bbox_embed is for region proposal generation\n num_pred = (transformer.decoder.num_layers + 1) if two_stage else transformer.decoder.num_layers\n if with_box_refine:\n self.class_embed = _get_clones(self.class_embed, num_pred)\n self.bbox_embed = _get_clones(self.bbox_embed, num_pred)\n nn.init.constant_(self.bbox_embed[0].layers[-1].bias.data[2:], -2.0)\n # hack implementation for iterative bounding box refinement\n self.transformer.decoder.bbox_embed = self.bbox_embed\n else:\n nn.init.constant_(self.bbox_embed.layers[-1].bias.data[2:], -2.0)\n self.class_embed = nn.ModuleList([self.class_embed for _ in range(num_pred)])\n self.bbox_embed = nn.ModuleList([self.bbox_embed for _ in range(num_pred)])\n self.transformer.decoder.bbox_embed = None\n if two_stage:\n # hack implementation for two-stage\n self.transformer.decoder.class_embed = self.class_embed\n for box_embed in self.bbox_embed:\n nn.init.constant_(box_embed.layers[-1].bias.data[2:], 0.0)\n\n def forward(self, samples: NestedTensor):\n \"\"\" The forward expects a NestedTensor, which consists of:\n - samples.tensor: batched images, of shape [batch_size x 3 x H x W]\n - samples.mask: a binary mask of shape [batch_size x H x W], containing 1 on padded pixels\n\n It returns a dict with the following elements:\n - \"pred_logits\": the classification logits (including no-object) for all queries.\n Shape= [batch_size x num_queries x (num_classes + 1)]\n - \"pred_boxes\": The normalized boxes coordinates for all queries, represented as\n (center_x, center_y, height, width). These values are normalized in [0, 1],\n relative to the size of each individual image (disregarding possible padding).\n See PostProcess for information on how to retrieve the unnormalized bounding box.\n - \"aux_outputs\": Optional, only returned when auxilary losses are activated. It is a list of\n dictionnaries containing the two above keys for each decoder layer.\n \"\"\"\n if not isinstance(samples, NestedTensor):\n samples = nested_tensor_from_tensor_list(samples)\n features, pos = self.backbone(samples)\n\n srcs = []\n masks = []\n for l, feat in enumerate(features):\n src, mask = feat.decompose()\n srcs.append(self.input_proj[l](src))\n masks.append(mask)\n assert mask is not None\n if self.num_feature_levels > len(srcs):\n _len_srcs = len(srcs)\n for l in range(_len_srcs, self.num_feature_levels):\n if l == _len_srcs:\n src = self.input_proj[l](features[-1].tensors)\n else:\n src = self.input_proj[l](srcs[-1])\n m = samples.mask\n mask = F.interpolate(m[None].float(), size=src.shape[-2:]).to(torch.bool)[0]\n pos_l = self.backbone[1](NestedTensor(src, mask)).to(src.dtype)\n srcs.append(src)\n masks.append(mask)\n pos.append(pos_l)\n\n query_embeds = None\n if not self.two_stage:\n query_embeds = self.query_embed.weight\n hs, init_reference, inter_references, enc_outputs_class, enc_outputs_coord_unact = self.transformer(srcs, masks, pos, query_embeds)\n\n outputs_classes = []\n outputs_coords = []\n for lvl in range(hs.shape[0]):\n if lvl == 0:\n reference = init_reference\n else:\n reference = inter_references[lvl - 1]\n reference = inverse_sigmoid(reference)\n outputs_class = self.class_embed[lvl](hs[lvl])\n tmp = self.bbox_embed[lvl](hs[lvl])\n if reference.shape[-1] == 4:\n tmp += reference\n else:\n assert reference.shape[-1] == 2\n tmp[..., :2] += reference\n outputs_coord = tmp.sigmoid()\n outputs_classes.append(outputs_class)\n outputs_coords.append(outputs_coord)\n outputs_class = torch.stack(outputs_classes)\n outputs_coord = torch.stack(outputs_coords)\n\n out = {'pred_logits': outputs_class[-1], 'pred_boxes': outputs_coord[-1]}\n if self.aux_loss:\n out['aux_outputs'] = self._set_aux_loss(outputs_class, outputs_coord)\n\n if self.two_stage:\n enc_outputs_coord = enc_outputs_coord_unact.sigmoid()\n out['enc_outputs'] = {'pred_logits': enc_outputs_class, 'pred_boxes': enc_outputs_coord}\n return out\n\n @torch.jit.unused\n def _set_aux_loss(self, outputs_class, outputs_coord):\n # this is a workaround to make torchscript happy, as torchscript\n # doesn't support dictionary with non-homogeneous values, such\n # as a dict having both a Tensor and a list.\n return [{'pred_logits': a, 'pred_boxes': b}\n for a, b in zip(outputs_class[:-1], outputs_coord[:-1])]\n\n\nclass SetCriterion(nn.Module):\n \"\"\" This class computes the loss for DETR.\n The process happens in two steps:\n 1) we compute hungarian assignment between ground truth boxes and the outputs of the model\n 2) we supervise each pair of matched ground-truth / prediction (supervise class and box)\n \"\"\"\n def __init__(self, num_classes, matcher, weight_dict, losses, focal_alpha=0.25):\n \"\"\" Create the criterion.\n Parameters:\n num_classes: number of object categories, omitting the special no-object category\n matcher: module able to compute a matching between targets and proposals\n weight_dict: dict containing as key the names of the losses and as values their relative weight.\n losses: list of all the losses to be applied. See get_loss for list of available losses.\n focal_alpha: alpha in Focal Loss\n \"\"\"\n super().__init__()\n self.num_classes = num_classes\n self.matcher = matcher\n self.weight_dict = weight_dict\n self.losses = losses\n self.focal_alpha = focal_alpha\n\n def loss_labels(self, outputs, targets, indices, num_boxes, log=True):\n \"\"\"Classification loss (NLL)\n targets dicts must contain the key \"labels\" containing a tensor of dim [nb_target_boxes]\n \"\"\"\n assert 'pred_logits' in outputs\n src_logits = outputs['pred_logits']\n\n idx = self._get_src_permutation_idx(indices)\n target_classes_o = torch.cat([t[\"labels\"][J] for t, (_, J) in zip(targets, indices)])\n target_classes = torch.full(src_logits.shape[:2], self.num_classes,\n dtype=torch.int64, device=src_logits.device)\n target_classes[idx] = target_classes_o\n\n target_classes_onehot = torch.zeros([src_logits.shape[0], src_logits.shape[1], src_logits.shape[2] + 1],\n dtype=src_logits.dtype, layout=src_logits.layout, device=src_logits.device)\n target_classes_onehot.scatter_(2, target_classes.unsqueeze(-1), 1)\n\n target_classes_onehot = target_classes_onehot[:,:,:-1]\n loss_ce = sigmoid_focal_loss(src_logits, target_classes_onehot, num_boxes, alpha=self.focal_alpha, gamma=2) * src_logits.shape[1]\n losses = {'loss_ce': loss_ce}\n\n if log:\n # TODO this should probably be a separate loss, not hacked in this one here\n losses['class_error'] = 100 - accuracy(src_logits[idx], target_classes_o)[0]\n return losses\n\n @torch.no_grad()\n def loss_cardinality(self, outputs, targets, indices, num_boxes):\n \"\"\" Compute the cardinality error, ie the absolute error in the number of predicted non-empty boxes\n This is not really a loss, it is intended for logging purposes only. It doesn't propagate gradients\n \"\"\"\n pred_logits = outputs['pred_logits']\n device = pred_logits.device\n tgt_lengths = torch.as_tensor([len(v[\"labels\"]) for v in targets], device=device)\n # Count the number of predictions that are NOT \"no-object\" (which is the last class)\n card_pred = (pred_logits.argmax(-1) != pred_logits.shape[-1] - 1).sum(1)\n card_err = F.l1_loss(card_pred.float(), tgt_lengths.float())\n losses = {'cardinality_error': card_err}\n return losses\n\n def loss_boxes(self, outputs, targets, indices, num_boxes):\n \"\"\"Compute the losses related to the bounding boxes, the L1 regression loss and the GIoU loss\n targets dicts must contain the key \"boxes\" containing a tensor of dim [nb_target_boxes, 4]\n The target boxes are expected in format (center_x, center_y, h, w), normalized by the image size.\n \"\"\"\n assert 'pred_boxes' in outputs\n idx = self._get_src_permutation_idx(indices)\n src_boxes = outputs['pred_boxes'][idx]\n target_boxes = torch.cat([t['boxes'][i] for t, (_, i) in zip(targets, indices)], dim=0)\n\n loss_bbox = F.l1_loss(src_boxes, target_boxes, reduction='none')\n\n losses = {}\n losses['loss_bbox'] = loss_bbox.sum() / num_boxes\n\n loss_giou = 1 - torch.diag(box_ops.generalized_box_iou(\n box_ops.box_cxcywh_to_xyxy(src_boxes),\n box_ops.box_cxcywh_to_xyxy(target_boxes)))\n losses['loss_giou'] = loss_giou.sum() / num_boxes\n return losses\n\n def loss_masks(self, outputs, targets, indices, num_boxes):\n \"\"\"Compute the losses related to the masks: the focal loss and the dice loss.\n targets dicts must contain the key \"masks\" containing a tensor of dim [nb_target_boxes, h, w]\n \"\"\"\n assert \"pred_masks\" in outputs\n\n src_idx = self._get_src_permutation_idx(indices)\n tgt_idx = self._get_tgt_permutation_idx(indices)\n\n src_masks = outputs[\"pred_masks\"]\n\n # TODO use valid to mask invalid areas due to padding in loss\n target_masks, valid = nested_tensor_from_tensor_list([t[\"masks\"] for t in targets]).decompose()\n target_masks = target_masks.to(src_masks)\n\n src_masks = src_masks[src_idx]\n # upsample predictions to the target size\n src_masks = interpolate(src_masks[:, None], size=target_masks.shape[-2:],\n mode=\"bilinear\", align_corners=False)\n src_masks = src_masks[:, 0].flatten(1)\n\n target_masks = target_masks[tgt_idx].flatten(1)\n\n losses = {\n \"loss_mask\": sigmoid_focal_loss(src_masks, target_masks, num_boxes),\n \"loss_dice\": dice_loss(src_masks, target_masks, num_boxes),\n }\n return losses\n\n def _get_src_permutation_idx(self, indices):\n # permute predictions following indices\n batch_idx = torch.cat([torch.full_like(src, i) for i, (src, _) in enumerate(indices)])\n src_idx = torch.cat([src for (src, _) in indices])\n return batch_idx, src_idx\n\n def _get_tgt_permutation_idx(self, indices):\n # permute targets following indices\n batch_idx = torch.cat([torch.full_like(tgt, i) for i, (_, tgt) in enumerate(indices)])\n tgt_idx = torch.cat([tgt for (_, tgt) in indices])\n return batch_idx, tgt_idx\n\n def get_loss(self, loss, outputs, targets, indices, num_boxes, **kwargs):\n loss_map = {\n 'labels': self.loss_labels,\n 'cardinality': self.loss_cardinality,\n 'boxes': self.loss_boxes,\n 'masks': self.loss_masks\n }\n assert loss in loss_map, f'do you really want to compute {loss} loss?'\n return loss_map[loss](outputs, targets, indices, num_boxes, **kwargs)\n\n def forward(self, outputs, targets):\n \"\"\" This performs the loss computation.\n Parameters:\n outputs: dict of tensors, see the output specification of the model for the format\n targets: list of dicts, such that len(targets) == batch_size.\n The expected keys in each dict depends on the losses applied, see each loss' doc\n \"\"\"\n outputs_without_aux = {k: v for k, v in outputs.items() if k != 'aux_outputs' and k != 'enc_outputs'}\n\n # Retrieve the matching between the outputs of the last layer and the targets\n indices = self.matcher(outputs_without_aux, targets)\n\n # Compute the average number of target boxes accross all nodes, for normalization purposes\n num_boxes = sum(len(t[\"labels\"]) for t in targets)\n num_boxes = torch.as_tensor([num_boxes], dtype=torch.float, device=next(iter(outputs.values())).device)\n if is_dist_avail_and_initialized():\n torch.distributed.all_reduce(num_boxes)\n num_boxes = torch.clamp(num_boxes / get_world_size(), min=1).item()\n\n # Compute all the requested losses\n losses = {}\n for loss in self.losses:\n kwargs = {}\n losses.update(self.get_loss(loss, outputs, targets, indices, num_boxes, **kwargs))\n\n # In case of auxiliary losses, we repeat this process with the output of each intermediate layer.\n if 'aux_outputs' in outputs:\n for i, aux_outputs in enumerate(outputs['aux_outputs']):\n indices = self.matcher(aux_outputs, targets)\n for loss in self.losses:\n if loss == 'masks':\n # Intermediate masks losses are too costly to compute, we ignore them.\n continue\n kwargs = {}\n if loss == 'labels':\n # Logging is enabled only for the last layer\n kwargs['log'] = False\n l_dict = self.get_loss(loss, aux_outputs, targets, indices, num_boxes, **kwargs)\n l_dict = {k + f'_{i}': v for k, v in l_dict.items()}\n losses.update(l_dict)\n\n if 'enc_outputs' in outputs:\n enc_outputs = outputs['enc_outputs']\n bin_targets = copy.deepcopy(targets)\n for bt in bin_targets:\n bt['labels'] = torch.zeros_like(bt['labels'])\n indices = self.matcher(enc_outputs, bin_targets)\n for loss in self.losses:\n if loss == 'masks':\n # Intermediate masks losses are too costly to compute, we ignore them.\n continue\n kwargs = {}\n if loss == 'labels':\n # Logging is enabled only for the last layer\n kwargs['log'] = False\n l_dict = self.get_loss(loss, enc_outputs, bin_targets, indices, num_boxes, **kwargs)\n l_dict = {k + f'_enc': v for k, v in l_dict.items()}\n losses.update(l_dict)\n\n return losses\n\n\nclass PostProcess(nn.Module):\n \"\"\" This module converts the model's output into the format expected by the coco api\"\"\"\n\n @torch.no_grad()\n def forward(self, outputs, target_sizes):\n \"\"\" Perform the computation\n Parameters:\n outputs: raw outputs of the model\n target_sizes: tensor of dimension [batch_size x 2] containing the size of each images of the batch\n For evaluation, this must be the original image size (before any data augmentation)\n For visualization, this should be the image size after data augment, but before padding\n \"\"\"\n out_logits, out_bbox = outputs['pred_logits'], outputs['pred_boxes']\n\n assert len(out_logits) == len(target_sizes)\n assert target_sizes.shape[1] == 2\n\n prob = out_logits.sigmoid()\n topk_values, topk_indexes = torch.topk(prob.view(out_logits.shape[0], -1), 100, dim=1)\n scores = topk_values\n topk_boxes = topk_indexes // out_logits.shape[2]\n labels = topk_indexes % out_logits.shape[2]\n boxes = box_ops.box_cxcywh_to_xyxy(out_bbox)\n boxes = torch.gather(boxes, 1, topk_boxes.unsqueeze(-1).repeat(1,1,4))\n\n # and from relative [0, 1] to absolute [0, height] coordinates\n img_h, img_w = target_sizes.unbind(1)\n scale_fct = torch.stack([img_w, img_h, img_w, img_h], dim=1)\n boxes = boxes * scale_fct[:, None, :]\n\n results = [{'scores': s, 'labels': l, 'boxes': b} for s, l, b in zip(scores, labels, boxes)]\n\n return results\n\n\nclass MLP(nn.Module):\n \"\"\" Very simple multi-layer perceptron (also called FFN)\"\"\"\n\n def __init__(self, input_dim, hidden_dim, output_dim, num_layers):\n super().__init__()\n self.num_layers = num_layers\n h = [hidden_dim] * (num_layers - 1)\n self.layers = nn.ModuleList(nn.Linear(n, k) for n, k in zip([input_dim] + h, h + [output_dim]))\n\n def forward(self, x):\n for i, layer in enumerate(self.layers):\n x = F.relu(layer(x)) if i < self.num_layers - 1 else layer(x)\n return x\n\n\ndef build(args):\n num_classes = 20 if args.dataset_file != 'coco' else 91\n if args.dataset_file == \"coco_panoptic\":\n num_classes = 250\n device = torch.device(args.device)\n\n backbone = build_backbone(args)\n\n transformer = build_deforamble_transformer(args)\n model = DeformableDETR(\n backbone,\n transformer,\n num_classes=num_classes,\n num_queries=args.num_queries,\n num_feature_levels=args.num_feature_levels,\n aux_loss=args.aux_loss,\n with_box_refine=args.with_box_refine,\n two_stage=args.two_stage,\n )\n if args.masks:\n model = DETRsegm(model, freeze_detr=(args.frozen_weights is not None))\n matcher = build_matcher(args)\n weight_dict = {'loss_ce': args.cls_loss_coef, 'loss_bbox': args.bbox_loss_coef}\n weight_dict['loss_giou'] = args.giou_loss_coef\n if args.masks:\n weight_dict[\"loss_mask\"] = args.mask_loss_coef\n weight_dict[\"loss_dice\"] = args.dice_loss_coef\n # TODO this is a hack\n if args.aux_loss:\n aux_weight_dict = {}\n for i in range(args.dec_layers - 1):\n aux_weight_dict.update({k + f'_{i}': v for k, v in weight_dict.items()})\n aux_weight_dict.update({k + f'_enc': v for k, v in weight_dict.items()})\n weight_dict.update(aux_weight_dict)\n\n losses = ['labels', 'boxes', 'cardinality']\n if args.masks:\n losses += [\"masks\"]\n # num_classes, matcher, weight_dict, losses, focal_alpha=0.25\n criterion = SetCriterion(num_classes, matcher, weight_dict, losses, focal_alpha=args.focal_alpha)\n criterion.to(device)\n postprocessors = {'bbox': PostProcess()}\n if args.masks:\n postprocessors['segm'] = PostProcessSegm()\n if args.dataset_file == \"coco_panoptic\":\n is_thing_map = {i: i <= 90 for i in range(201)}\n postprocessors[\"panoptic\"] = PostProcessPanoptic(is_thing_map, threshold=0.85)\n\n return model, criterion, postprocessors\n"
},
{
"path": "batchformer-v2/deformable-detr/models/deformable_transformer.py",
"content": "\nimport copy\nfrom typing import Optional, List\nimport math\n\nimport torch\nimport torch.nn.functional as F\nfrom torch import nn, Tensor\nfrom torch.nn.init import xavier_uniform_, constant_, uniform_, normal_\n\nfrom util.misc import inverse_sigmoid\nfrom models.ops.modules import MSDeformAttn\n\n\nclass DeformableTransformer(nn.Module):\n def __init__(self, d_model=256, nhead=8,\n num_encoder_layers=6, num_decoder_layers=6, dim_feedforward=1024, dropout=0.1,\n activation=\"relu\", return_intermediate_dec=False,\n num_feature_levels=4, dec_n_points=4, enc_n_points=4,\n two_stage=False, two_stage_num_proposals=300, args=None):\n super().__init__()\n assert args is not None\n\n self.d_model = d_model\n self.nhead = nhead\n self.two_stage = two_stage\n self.two_stage_num_proposals = two_stage_num_proposals\n\n encoder_layer = DeformableTransformerEncoderLayer(d_model, dim_feedforward,\n dropout, activation,\n num_feature_levels, nhead, enc_n_points)\n self.bf = None\n insert_idx = []\n if args is not None and args.bf:\n def generate_bf(type, args):\n if type == 1:\n encoder = torch.nn.TransformerEncoderLayer(d_model, 4, d_model, dropout=0.5)\n elif type >= 2:\n encoder = torch.nn.TransformerEncoder(torch.nn.TransformerEncoderLayer(d_model, 4, d_model, dropout=0.5), type)\n else:\n encoder = torch.nn.TransformerEncoderLayer(d_model, 4, d_model, dropout=0.5)\n return encoder\n self.bf = generate_bf(args.bt_num_layers, args)\n insert_idx = args.insert_idx\n if not args.share_bf:\n self.bf = torch.nn.ModuleList([generate_bf(args.bt_num_layers, args) if i in insert_idx else torch.nn.Identity() for i in range(0, num_encoder_layers)])\n self.encoder = DeformableTransformerEncoder(encoder_layer, num_encoder_layers, bf=self.bf, bf_idx = args.bf, insert_idx=insert_idx,\n eval_bf=args.eval_bf)\n\n decoder_layer = DeformableTransformerDecoderLayer(d_model, dim_feedforward,\n dropout, activation,\n num_feature_levels, nhead, dec_n_points)\n self.decoder = DeformableTransformerDecoder(decoder_layer, num_decoder_layers, return_intermediate_dec)\n\n self.level_embed = nn.Parameter(torch.Tensor(num_feature_levels, d_model))\n\n if two_stage:\n self.enc_output = nn.Linear(d_model, d_model)\n self.enc_output_norm = nn.LayerNorm(d_model)\n self.pos_trans = nn.Linear(d_model * 2, d_model * 2)\n self.pos_trans_norm = nn.LayerNorm(d_model * 2)\n else:\n self.reference_points = nn.Linear(d_model, 2)\n\n self._reset_parameters()\n\n def _reset_parameters(self):\n for p in self.parameters():\n if p.dim() > 1:\n nn.init.xavier_uniform_(p)\n for m in self.modules():\n if isinstance(m, MSDeformAttn):\n m._reset_parameters()\n if not self.two_stage:\n xavier_uniform_(self.reference_points.weight.data, gain=1.0)\n constant_(self.reference_points.bias.data, 0.)\n normal_(self.level_embed)\n\n def get_proposal_pos_embed(self, proposals):\n num_pos_feats = 128\n temperature = 10000\n scale = 2 * math.pi\n\n dim_t = torch.arange(num_pos_feats, dtype=torch.float32, device=proposals.device)\n dim_t = temperature ** (2 * (dim_t // 2) / num_pos_feats)\n # N, L, 4\n proposals = proposals.sigmoid() * scale\n # N, L, 4, 128\n pos = proposals[:, :, :, None] / dim_t\n # N, L, 4, 64, 2\n pos = torch.stack((pos[:, :, :, 0::2].sin(), pos[:, :, :, 1::2].cos()), dim=4).flatten(2)\n return pos\n\n def gen_encoder_output_proposals(self, memory, memory_padding_mask, spatial_shapes):\n N_, S_, C_ = memory.shape\n base_scale = 4.0\n proposals = []\n _cur = 0\n for lvl, (H_, W_) in enumerate(spatial_shapes):\n mask_flatten_ = memory_padding_mask[:, _cur:(_cur + H_ * W_)].view(N_, H_, W_, 1)\n valid_H = torch.sum(~mask_flatten_[:, :, 0, 0], 1)\n valid_W = torch.sum(~mask_flatten_[:, 0, :, 0], 1)\n\n grid_y, grid_x = torch.meshgrid(torch.linspace(0, H_ - 1, H_, dtype=torch.float32, device=memory.device),\n torch.linspace(0, W_ - 1, W_, dtype=torch.float32, device=memory.device))\n grid = torch.cat([grid_x.unsqueeze(-1), grid_y.unsqueeze(-1)], -1)\n\n scale = torch.cat([valid_W.unsqueeze(-1), valid_H.unsqueeze(-1)], 1).view(N_, 1, 1, 2)\n grid = (grid.unsqueeze(0).expand(N_, -1, -1, -1) + 0.5) / scale\n wh = torch.ones_like(grid) * 0.05 * (2.0 ** lvl)\n proposal = torch.cat((grid, wh), -1).view(N_, -1, 4)\n proposals.append(proposal)\n _cur += (H_ * W_)\n output_proposals = torch.cat(proposals, 1)\n output_proposals_valid = ((output_proposals > 0.01) & (output_proposals < 0.99)).all(-1, keepdim=True)\n output_proposals = torch.log(output_proposals / (1 - output_proposals))\n output_proposals = output_proposals.masked_fill(memory_padding_mask.unsqueeze(-1), float('inf'))\n output_proposals = output_proposals.masked_fill(~output_proposals_valid, float('inf'))\n\n output_memory = memory\n output_memory = output_memory.masked_fill(memory_padding_mask.unsqueeze(-1), float(0))\n output_memory = output_memory.masked_fill(~output_proposals_valid, float(0))\n output_memory = self.enc_output_norm(self.enc_output(output_memory))\n return output_memory, output_proposals\n\n def get_valid_ratio(self, mask):\n _, H, W = mask.shape\n valid_H = torch.sum(~mask[:, :, 0], 1)\n valid_W = torch.sum(~mask[:, 0, :], 1)\n valid_ratio_h = valid_H.float() / H\n valid_ratio_w = valid_W.float() / W\n valid_ratio = torch.stack([valid_ratio_w, valid_ratio_h], -1)\n return valid_ratio\n\n def forward(self, srcs, masks, pos_embeds, query_embed=None):\n assert self.two_stage or query_embed is not None\n\n # prepare input for encoder\n src_flatten = []\n mask_flatten = []\n lvl_pos_embed_flatten = []\n spatial_shapes = []\n for lvl, (src, mask, pos_embed) in enumerate(zip(srcs, masks, pos_embeds)):\n bs, c, h, w = src.shape\n spatial_shape = (h, w)\n spatial_shapes.append(spatial_shape)\n src = src.flatten(2).transpose(1, 2)\n mask = mask.flatten(1)\n pos_embed = pos_embed.flatten(2).transpose(1, 2)\n lvl_pos_embed = pos_embed + self.level_embed[lvl].view(1, 1, -1)\n lvl_pos_embed_flatten.append(lvl_pos_embed)\n src_flatten.append(src)\n mask_flatten.append(mask)\n src_flatten = torch.cat(src_flatten, 1)\n mask_flatten = torch.cat(mask_flatten, 1)\n lvl_pos_embed_flatten = torch.cat(lvl_pos_embed_flatten, 1)\n spatial_shapes = torch.as_tensor(spatial_shapes, dtype=torch.long, device=src_flatten.device)\n level_start_index = torch.cat((spatial_shapes.new_zeros((1, )), spatial_shapes.prod(1).cumsum(0)[:-1]))\n valid_ratios = torch.stack([self.get_valid_ratio(m) for m in masks], 1)\n\n # encoder\n memory = self.encoder(src_flatten, spatial_shapes, level_start_index, valid_ratios, lvl_pos_embed_flatten, mask_flatten)\n\n if len(memory) > len(mask_flatten):\n mask_flatten = torch.cat([mask_flatten, mask_flatten], dim=0)\n valid_ratios = torch.cat([valid_ratios, valid_ratios], dim=0)\n # prepare input for decoder\n bs, _, c = memory.shape\n if self.two_stage:\n output_memory, output_proposals = self.gen_encoder_output_proposals(memory, mask_flatten, spatial_shapes)\n\n # hack implementation for two-stage Deformable DETR\n enc_outputs_class = self.decoder.class_embed[self.decoder.num_layers](output_memory)\n enc_outputs_coord_unact = self.decoder.bbox_embed[self.decoder.num_layers](output_memory) + output_proposals\n\n topk = self.two_stage_num_proposals\n topk_proposals = torch.topk(enc_outputs_class[..., 0], topk, dim=1)[1]\n topk_coords_unact = torch.gather(enc_outputs_coord_unact, 1, topk_proposals.unsqueeze(-1).repeat(1, 1, 4))\n topk_coords_unact = topk_coords_unact.detach()\n reference_points = topk_coords_unact.sigmoid()\n init_reference_out = reference_points\n pos_trans_out = self.pos_trans_norm(self.pos_trans(self.get_proposal_pos_embed(topk_coords_unact)))\n query_embed, tgt = torch.split(pos_trans_out, c, dim=2)\n else:\n query_embed, tgt = torch.split(query_embed, c, dim=1)\n query_embed = query_embed.unsqueeze(0).expand(bs, -1, -1)\n tgt = tgt.unsqueeze(0).expand(bs, -1, -1)\n reference_points = self.reference_points(query_embed).sigmoid()\n init_reference_out = reference_points\n # decoder\n hs, inter_references = self.decoder(tgt, reference_points, memory,\n spatial_shapes, level_start_index, valid_ratios, query_embed, mask_flatten)\n\n inter_references_out = inter_references\n if self.two_stage:\n return hs, init_reference_out, inter_references_out, enc_outputs_class, enc_outputs_coord_unact\n return hs, init_reference_out, inter_references_out, None, None\n\n\nclass DeformableTransformerEncoderLayer(nn.Module):\n def __init__(self,\n d_model=256, d_ffn=1024,\n dropout=0.1, activation=\"relu\",\n n_levels=4, n_heads=8, n_points=4):\n super().__init__()\n\n # self attention\n self.self_attn = MSDeformAttn(d_model, n_levels, n_heads, n_points)\n self.dropout1 = nn.Dropout(dropout)\n self.norm1 = nn.LayerNorm(d_model)\n\n # ffn\n self.linear1 = nn.Linear(d_model, d_ffn)\n self.activation = _get_activation_fn(activation)\n self.dropout2 = nn.Dropout(dropout)\n self.linear2 = nn.Linear(d_ffn, d_model)\n self.dropout3 = nn.Dropout(dropout)\n self.norm2 = nn.LayerNorm(d_model)\n\n @staticmethod\n def with_pos_embed(tensor, pos):\n return tensor if pos is None else tensor + pos\n\n def forward_ffn(self, src):\n src2 = self.linear2(self.dropout2(self.activation(self.linear1(src))))\n src = src + self.dropout3(src2)\n src = self.norm2(src)\n return src\n\n def forward(self, src, pos, reference_points, spatial_shapes, level_start_index, padding_mask=None):\n # self attention\n src2 = self.self_attn(self.with_pos_embed(src, pos), reference_points, src, spatial_shapes, level_start_index, padding_mask)\n src = src + self.dropout1(src2)\n src = self.norm1(src)\n\n # ffn\n src = self.forward_ffn(src)\n\n return src\n\n\nclass DeformableTransformerEncoder(nn.Module):\n def __init__(self, encoder_layer, num_layers, bf=None, bf_idx = 0, insert_idx=2, eval_bf=0):\n super().__init__()\n self.layers = _get_clones(encoder_layer, num_layers)\n self.num_layers = num_layers\n self.insert_idx = insert_idx\n if type(bf) != torch.nn.ModuleList and bf is not None:\n self.bf = [bf]*num_layers\n else:\n self.bf = bf\n self.bf_idx = bf_idx\n self.eval_bf = eval_bf\n\n @staticmethod\n def get_reference_points(spatial_shapes, valid_ratios, device):\n reference_points_list = []\n for lvl, (H_, W_) in enumerate(spatial_shapes):\n\n ref_y, ref_x = torch.meshgrid(torch.linspace(0.5, H_ - 0.5, H_, dtype=torch.float32, device=device),\n torch.linspace(0.5, W_ - 0.5, W_, dtype=torch.float32, device=device))\n ref_y = ref_y.reshape(-1)[None] / (valid_ratios[:, None, lvl, 1] * H_)\n ref_x = ref_x.reshape(-1)[None] / (valid_ratios[:, None, lvl, 0] * W_)\n ref = torch.stack((ref_x, ref_y), -1)\n reference_points_list.append(ref)\n reference_points = torch.cat(reference_points_list, 1)\n reference_points = reference_points[:, :, None] * valid_ratios[:, None]\n return reference_points\n\n def forward(self, src, spatial_shapes, level_start_index, valid_ratios, pos=None, padding_mask=None):\n output = src\n reference_points = self.get_reference_points(spatial_shapes, valid_ratios, device=src.device)\n for i, layer in enumerate(self.layers):\n output = layer(output, pos, reference_points, spatial_shapes, level_start_index, padding_mask)\n\n if i in self.insert_idx and self.bf is not None and self.bf_idx in [1] and (self.training or self.eval_bf):\n old_output = output\n if i != self.insert_idx[0]: # If this is not the first layer, we split the batch into two streams because we have concatenate the two streams.\n old_output = output[ :len(output)//2, :, :] # stream without batchformerv2\n output = output[len(output)//2:, :, :] # strea with batchformerv2\n output = self.bf[i](output) # if this is th first layer, we apply the batchformerv2 on the batch. then concatenate old_output and output\n if self.training:\n output = torch.cat([old_output, output], dim=0)\n if i == self.insert_idx[0] and self.training: # for other variables, we just share it i the next modules. We thus repeat it.\n pos = torch.cat([pos, pos], dim=0) \n reference_points = torch.cat([reference_points, reference_points], dim=0)\n padding_mask = torch.cat([padding_mask, padding_mask], dim=0)\n elif i in self.insert_idx and self.bf is not None and self.bf_idx == 7 and self.training:\n # single stream\n output = self.bf[i](output)\n\n return output\n\n\nclass DeformableTransformerDecoderLayer(nn.Module):\n def __init__(self, d_model=256, d_ffn=1024,\n dropout=0.1, activation=\"relu\",\n n_levels=4, n_heads=8, n_points=4):\n super().__init__()\n\n # cross attention\n self.cross_attn = MSDeformAttn(d_model, n_levels, n_heads, n_points)\n self.dropout1 = nn.Dropout(dropout)\n self.norm1 = nn.LayerNorm(d_model)\n\n # self attention\n self.self_attn = nn.MultiheadAttention(d_model, n_heads, dropout=dropout)\n self.dropout2 = nn.Dropout(dropout)\n self.norm2 = nn.LayerNorm(d_model)\n\n # ffn\n self.linear1 = nn.Linear(d_model, d_ffn)\n self.activation = _get_activation_fn(activation)\n self.dropout3 = nn.Dropout(dropout)\n self.linear2 = nn.Linear(d_ffn, d_model)\n self.dropout4 = nn.Dropout(dropout)\n self.norm3 = nn.LayerNorm(d_model)\n\n @staticmethod\n def with_pos_embed(tensor, pos):\n return tensor if pos is None else tensor + pos\n\n def forward_ffn(self, tgt):\n tgt2 = self.linear2(self.dropout3(self.activation(self.linear1(tgt))))\n tgt = tgt + self.dropout4(tgt2)\n tgt = self.norm3(tgt)\n return tgt\n\n def forward(self, tgt, query_pos, reference_points, src, src_spatial_shapes, level_start_index, src_padding_mask=None):\n # self attention\n q = k = self.with_pos_embed(tgt, query_pos)\n tgt2 = self.self_attn(q.transpose(0, 1), k.transpose(0, 1), tgt.transpose(0, 1))[0].transpose(0, 1)\n tgt = tgt + self.dropout2(tgt2)\n tgt = self.norm2(tgt)\n\n # cross attention\n tgt2 = self.cross_attn(self.with_pos_embed(tgt, query_pos),\n reference_points,\n src, src_spatial_shapes, level_start_index, src_padding_mask)\n tgt = tgt + self.dropout1(tgt2)\n tgt = self.norm1(tgt)\n\n # ffn\n tgt = self.forward_ffn(tgt)\n\n return tgt\n\n\nclass DeformableTransformerDecoder(nn.Module):\n def __init__(self, decoder_layer, num_layers, return_intermediate=False):\n super().__init__()\n self.layers = _get_clones(decoder_layer, num_layers)\n self.num_layers = num_layers\n self.return_intermediate = return_intermediate\n # hack implementation for iterative bounding box refinement and two-stage Deformable DETR\n self.bbox_embed = None\n self.class_embed = None\n\n def forward(self, tgt, reference_points, src, src_spatial_shapes, src_level_start_index, src_valid_ratios,\n query_pos=None, src_padding_mask=None):\n output = tgt\n\n intermediate = []\n intermediate_reference_points = []\n for lid, layer in enumerate(self.layers):\n if reference_points.shape[-1] == 4:\n reference_points_input = reference_points[:, :, None] \\\n * torch.cat([src_valid_ratios, src_valid_ratios], -1)[:, None]\n else:\n assert reference_points.shape[-1] == 2\n reference_points_input = reference_points[:, :, None] * src_valid_ratios[:, None]\n output = layer(output, query_pos, reference_points_input, src, src_spatial_shapes, src_level_start_index, src_padding_mask)\n # hack implementation for iterative bounding box refinement\n if self.bbox_embed is not None:\n tmp = self.bbox_embed[lid](output)\n if reference_points.shape[-1] == 4:\n new_reference_points = tmp + inverse_sigmoid(reference_points)\n new_reference_points = new_reference_points.sigmoid()\n else:\n assert reference_points.shape[-1] == 2\n new_reference_points = tmp\n new_reference_points[..., :2] = tmp[..., :2] + inverse_sigmoid(reference_points)\n new_reference_points = new_reference_points.sigmoid()\n reference_points = new_reference_points.detach()\n\n if self.return_intermediate:\n intermediate.append(output)\n intermediate_reference_points.append(reference_points)\n\n if self.return_intermediate:\n return torch.stack(intermediate), torch.stack(intermediate_reference_points)\n\n return output, reference_points\n\n\ndef _get_clones(module, N):\n return nn.ModuleList([copy.deepcopy(module) for i in range(N)])\n\n\ndef _get_activation_fn(activation):\n \"\"\"Return an activation function given a string\"\"\"\n if activation == \"relu\":\n return F.relu\n if activation == \"gelu\":\n return F.gelu\n if activation == \"glu\":\n return F.glu\n raise RuntimeError(F\"activation should be relu/gelu, not {activation}.\")\n\n\ndef build_deforamble_transformer(args):\n return DeformableTransformer(\n d_model=args.hidden_dim,\n nhead=args.nheads,\n num_encoder_layers=args.enc_layers,\n num_decoder_layers=args.dec_layers,\n dim_feedforward=args.dim_feedforward,\n dropout=args.dropout,\n activation=\"relu\",\n return_intermediate_dec=True,\n num_feature_levels=args.num_feature_levels,\n dec_n_points=args.dec_n_points,\n enc_n_points=args.enc_n_points,\n two_stage=args.two_stage,\n two_stage_num_proposals=args.num_queries,\n args=args)\n\n\n"
},
{
"path": "batchformer-v2/deformable-detr/models/matcher.py",
"content": "# ------------------------------------------------------------------------\n# Deformable DETR\n# Copyright (c) 2020 SenseTime. All Rights Reserved.\n# Licensed under the Apache License, Version 2.0 [see LICENSE for details]\n# ------------------------------------------------------------------------\n# Modified from DETR (https://github.com/facebookresearch/detr)\n# Copyright (c) Facebook, Inc. and its affiliates. All Rights Reserved\n# ------------------------------------------------------------------------\n\n\"\"\"\nModules to compute the matching cost and solve the corresponding LSAP.\n\"\"\"\nimport torch\nfrom scipy.optimize import linear_sum_assignment\nfrom torch import nn\n\nfrom util.box_ops import box_cxcywh_to_xyxy, generalized_box_iou\n\n\nclass HungarianMatcher(nn.Module):\n \"\"\"This class computes an assignment between the targets and the predictions of the network\n\n For efficiency reasons, the targets don't include the no_object. Because of this, in general,\n there are more predictions than targets. In this case, we do a 1-to-1 matching of the best predictions,\n while the others are un-matched (and thus treated as non-objects).\n \"\"\"\n\n def __init__(self,\n cost_class: float = 1,\n cost_bbox: float = 1,\n cost_giou: float = 1):\n \"\"\"Creates the matcher\n\n Params:\n cost_class: This is the relative weight of the classification error in the matching cost\n cost_bbox: This is the relative weight of the L1 error of the bounding box coordinates in the matching cost\n cost_giou: This is the relative weight of the giou loss of the bounding box in the matching cost\n \"\"\"\n super().__init__()\n self.cost_class = cost_class\n self.cost_bbox = cost_bbox\n self.cost_giou = cost_giou\n assert cost_class != 0 or cost_bbox != 0 or cost_giou != 0, \"all costs cant be 0\"\n\n def forward(self, outputs, targets):\n \"\"\" Performs the matching\n\n Params:\n outputs: This is a dict that contains at least these entries:\n \"pred_logits\": Tensor of dim [batch_size, num_queries, num_classes] with the classification logits\n \"pred_boxes\": Tensor of dim [batch_size, num_queries, 4] with the predicted box coordinates\n\n targets: This is a list of targets (len(targets) = batch_size), where each target is a dict containing:\n \"labels\": Tensor of dim [num_target_boxes] (where num_target_boxes is the number of ground-truth\n objects in the target) containing the class labels\n \"boxes\": Tensor of dim [num_target_boxes, 4] containing the target box coordinates\n\n Returns:\n A list of size batch_size, containing tuples of (index_i, index_j) where:\n - index_i is the indices of the selected predictions (in order)\n - index_j is the indices of the corresponding selected targets (in order)\n For each batch element, it holds:\n len(index_i) = len(index_j) = min(num_queries, num_target_boxes)\n \"\"\"\n with torch.no_grad():\n bs, num_queries = outputs[\"pred_logits\"].shape[:2]\n\n # We flatten to compute the cost matrices in a batch\n out_prob = outputs[\"pred_logits\"].flatten(0, 1).sigmoid()\n out_bbox = outputs[\"pred_boxes\"].flatten(0, 1) # [batch_size * num_queries, 4]\n\n # Also concat the target labels and boxes\n tgt_ids = torch.cat([v[\"labels\"] for v in targets])\n tgt_bbox = torch.cat([v[\"boxes\"] for v in targets])\n\n # Compute the classification cost.\n alpha = 0.25\n gamma = 2.0\n neg_cost_class = (1 - alpha) * (out_prob ** gamma) * (-(1 - out_prob + 1e-8).log())\n pos_cost_class = alpha * ((1 - out_prob) ** gamma) * (-(out_prob + 1e-8).log())\n cost_class = pos_cost_class[:, tgt_ids] - neg_cost_class[:, tgt_ids]\n\n # Compute the L1 cost between boxes\n cost_bbox = torch.cdist(out_bbox, tgt_bbox, p=1)\n\n # Compute the giou cost betwen boxes\n cost_giou = -generalized_box_iou(box_cxcywh_to_xyxy(out_bbox),\n box_cxcywh_to_xyxy(tgt_bbox))\n\n # Final cost matrix\n C = self.cost_bbox * cost_bbox + self.cost_class * cost_class + self.cost_giou * cost_giou\n C = C.view(bs, num_queries, -1).cpu()\n\n sizes = [len(v[\"boxes\"]) for v in targets]\n indices = [linear_sum_assignment(c[i]) for i, c in enumerate(C.split(sizes, -1))]\n return [(torch.as_tensor(i, dtype=torch.int64), torch.as_tensor(j, dtype=torch.int64)) for i, j in indices]\n\n\ndef build_matcher(args):\n return HungarianMatcher(cost_class=args.set_cost_class,\n cost_bbox=args.set_cost_bbox,\n cost_giou=args.set_cost_giou)\n"
},
{
"path": "batchformer-v2/deformable-detr/models/ops/functions/__init__.py",
"content": "# ------------------------------------------------------------------------------------------------\n# Deformable DETR\n# Copyright (c) 2020 SenseTime. All Rights Reserved.\n# Licensed under the Apache License, Version 2.0 [see LICENSE for details]\n# ------------------------------------------------------------------------------------------------\n# Modified from https://github.com/chengdazhi/Deformable-Convolution-V2-PyTorch/tree/pytorch_1.0.0\n# ------------------------------------------------------------------------------------------------\n\nfrom .ms_deform_attn_func import MSDeformAttnFunction\n\n"
},
{
"path": "batchformer-v2/deformable-detr/models/ops/functions/ms_deform_attn_func.py",
"content": "# ------------------------------------------------------------------------------------------------\n# Deformable DETR\n# Copyright (c) 2020 SenseTime. All Rights Reserved.\n# Licensed under the Apache License, Version 2.0 [see LICENSE for details]\n# ------------------------------------------------------------------------------------------------\n# Modified from https://github.com/chengdazhi/Deformable-Convolution-V2-PyTorch/tree/pytorch_1.0.0\n# ------------------------------------------------------------------------------------------------\n\nfrom __future__ import absolute_import\nfrom __future__ import print_function\nfrom __future__ import division\n\nimport torch\nimport torch.nn.functional as F\nfrom torch.autograd import Function\nfrom torch.autograd.function import once_differentiable\n\nimport MultiScaleDeformableAttention as MSDA\n\n\nclass MSDeformAttnFunction(Function):\n @staticmethod\n def forward(ctx, value, value_spatial_shapes, value_level_start_index, sampling_locations, attention_weights, im2col_step):\n ctx.im2col_step = im2col_step\n output = MSDA.ms_deform_attn_forward(\n value, value_spatial_shapes, value_level_start_index, sampling_locations, attention_weights, ctx.im2col_step)\n ctx.save_for_backward(value, value_spatial_shapes, value_level_start_index, sampling_locations, attention_weights)\n return output\n\n @staticmethod\n @once_differentiable\n def backward(ctx, grad_output):\n value, value_spatial_shapes, value_level_start_index, sampling_locations, attention_weights = ctx.saved_tensors\n grad_value, grad_sampling_loc, grad_attn_weight = \\\n MSDA.ms_deform_attn_backward(\n value, value_spatial_shapes, value_level_start_index, sampling_locations, attention_weights, grad_output, ctx.im2col_step)\n\n return grad_value, None, None, grad_sampling_loc, grad_attn_weight, None\n\n\ndef ms_deform_attn_core_pytorch(value, value_spatial_shapes, sampling_locations, attention_weights):\n # for debug and test only,\n # need to use cuda version instead\n N_, S_, M_, D_ = value.shape\n _, Lq_, M_, L_, P_, _ = sampling_locations.shape\n value_list = value.split([H_ * W_ for H_, W_ in value_spatial_shapes], dim=1)\n sampling_grids = 2 * sampling_locations - 1\n sampling_value_list = []\n for lid_, (H_, W_) in enumerate(value_spatial_shapes):\n # N_, H_*W_, M_, D_ -> N_, H_*W_, M_*D_ -> N_, M_*D_, H_*W_ -> N_*M_, D_, H_, W_\n value_l_ = value_list[lid_].flatten(2).transpose(1, 2).reshape(N_*M_, D_, H_, W_)\n # N_, Lq_, M_, P_, 2 -> N_, M_, Lq_, P_, 2 -> N_*M_, Lq_, P_, 2\n sampling_grid_l_ = sampling_grids[:, :, :, lid_].transpose(1, 2).flatten(0, 1)\n # N_*M_, D_, Lq_, P_\n sampling_value_l_ = F.grid_sample(value_l_, sampling_grid_l_,\n mode='bilinear', padding_mode='zeros', align_corners=False)\n sampling_value_list.append(sampling_value_l_)\n # (N_, Lq_, M_, L_, P_) -> (N_, M_, Lq_, L_, P_) -> (N_, M_, 1, Lq_, L_*P_)\n attention_weights = attention_weights.transpose(1, 2).reshape(N_*M_, 1, Lq_, L_*P_)\n output = (torch.stack(sampling_value_list, dim=-2).flatten(-2) * attention_weights).sum(-1).view(N_, M_*D_, Lq_)\n return output.transpose(1, 2).contiguous()\n"
},
{
"path": "batchformer-v2/deformable-detr/models/ops/make.sh",
"content": "#!/usr/bin/env bash\n# ------------------------------------------------------------------------------------------------\n# Deformable DETR\n# Copyright (c) 2020 SenseTime. All Rights Reserved.\n# Licensed under the Apache License, Version 2.0 [see LICENSE for details]\n# ------------------------------------------------------------------------------------------------\n# Modified from https://github.com/chengdazhi/Deformable-Convolution-V2-PyTorch/tree/pytorch_1.0.0\n# ------------------------------------------------------------------------------------------------\n\npython setup.py build install\n"
},
{
"path": "batchformer-v2/deformable-detr/models/ops/modules/__init__.py",
"content": "# ------------------------------------------------------------------------------------------------\n# Deformable DETR\n# Copyright (c) 2020 SenseTime. All Rights Reserved.\n# Licensed under the Apache License, Version 2.0 [see LICENSE for details]\n# ------------------------------------------------------------------------------------------------\n# Modified from https://github.com/chengdazhi/Deformable-Convolution-V2-PyTorch/tree/pytorch_1.0.0\n# ------------------------------------------------------------------------------------------------\n\nfrom .ms_deform_attn import MSDeformAttn\n"
},
{
"path": "batchformer-v2/deformable-detr/models/ops/modules/ms_deform_attn.py",
"content": "# ------------------------------------------------------------------------------------------------\n# Deformable DETR\n# Copyright (c) 2020 SenseTime. All Rights Reserved.\n# Licensed under the Apache License, Version 2.0 [see LICENSE for details]\n# ------------------------------------------------------------------------------------------------\n# Modified from https://github.com/chengdazhi/Deformable-Convolution-V2-PyTorch/tree/pytorch_1.0.0\n# ------------------------------------------------------------------------------------------------\n\nfrom __future__ import absolute_import\nfrom __future__ import print_function\nfrom __future__ import division\n\nimport warnings\nimport math\n\nimport torch\nfrom torch import nn\nimport torch.nn.functional as F\nfrom torch.nn.init import xavier_uniform_, constant_\n\nfrom ..functions import MSDeformAttnFunction\n\n\ndef _is_power_of_2(n):\n if (not isinstance(n, int)) or (n < 0):\n raise ValueError(\"invalid input for _is_power_of_2: {} (type: {})\".format(n, type(n)))\n return (n & (n-1) == 0) and n != 0\n\n\nclass MSDeformAttn(nn.Module):\n def __init__(self, d_model=256, n_levels=4, n_heads=8, n_points=4):\n \"\"\"\n Multi-Scale Deformable Attention Module\n :param d_model hidden dimension\n :param n_levels number of feature levels\n :param n_heads number of attention heads\n :param n_points number of sampling points per attention head per feature level\n \"\"\"\n super().__init__()\n if d_model % n_heads != 0:\n raise ValueError('d_model must be divisible by n_heads, but got {} and {}'.format(d_model, n_heads))\n _d_per_head = d_model // n_heads\n # you'd better set _d_per_head to a power of 2 which is more efficient in our CUDA implementation\n if not _is_power_of_2(_d_per_head):\n warnings.warn(\"You'd better set d_model in MSDeformAttn to make the dimension of each attention head a power of 2 \"\n \"which is more efficient in our CUDA implementation.\")\n\n self.im2col_step = 64\n\n self.d_model = d_model\n self.n_levels = n_levels\n self.n_heads = n_heads\n self.n_points = n_points\n\n self.sampling_offsets = nn.Linear(d_model, n_heads * n_levels * n_points * 2)\n self.attention_weights = nn.Linear(d_model, n_heads * n_levels * n_points)\n self.value_proj = nn.Linear(d_model, d_model)\n self.output_proj = nn.Linear(d_model, d_model)\n\n self._reset_parameters()\n\n def _reset_parameters(self):\n constant_(self.sampling_offsets.weight.data, 0.)\n thetas = torch.arange(self.n_heads, dtype=torch.float32) * (2.0 * math.pi / self.n_heads)\n grid_init = torch.stack([thetas.cos(), thetas.sin()], -1)\n grid_init = (grid_init / grid_init.abs().max(-1, keepdim=True)[0]).view(self.n_heads, 1, 1, 2).repeat(1, self.n_levels, self.n_points, 1)\n for i in range(self.n_points):\n grid_init[:, :, i, :] *= i + 1\n with torch.no_grad():\n self.sampling_offsets.bias = nn.Parameter(grid_init.view(-1))\n constant_(self.attention_weights.weight.data, 0.)\n constant_(self.attention_weights.bias.data, 0.)\n xavier_uniform_(self.value_proj.weight.data)\n constant_(self.value_proj.bias.data, 0.)\n xavier_uniform_(self.output_proj.weight.data)\n constant_(self.output_proj.bias.data, 0.)\n\n def forward(self, query, reference_points, input_flatten, input_spatial_shapes, input_level_start_index, input_padding_mask=None):\n \"\"\"\n :param query (N, Length_{query}, C)\n :param reference_points (N, Length_{query}, n_levels, 2), range in [0, 1], top-left (0,0), bottom-right (1, 1), including padding area\n or (N, Length_{query}, n_levels, 4), add additional (w, h) to form reference boxes\n :param input_flatten (N, \\sum_{l=0}^{L-1} H_l \\cdot W_l, C)\n :param input_spatial_shapes (n_levels, 2), [(H_0, W_0), (H_1, W_1), ..., (H_{L-1}, W_{L-1})]\n :param input_level_start_index (n_levels, ), [0, H_0*W_0, H_0*W_0+H_1*W_1, H_0*W_0+H_1*W_1+H_2*W_2, ..., H_0*W_0+H_1*W_1+...+H_{L-1}*W_{L-1}]\n :param input_padding_mask (N, \\sum_{l=0}^{L-1} H_l \\cdot W_l), True for padding elements, False for non-padding elements\n\n :return output (N, Length_{query}, C)\n \"\"\"\n N, Len_q, _ = query.shape\n N, Len_in, _ = input_flatten.shape\n assert (input_spatial_shapes[:, 0] * input_spatial_shapes[:, 1]).sum() == Len_in\n\n value = self.value_proj(input_flatten)\n if input_padding_mask is not None:\n value = value.masked_fill(input_padding_mask[..., None], float(0))\n value = value.view(N, Len_in, self.n_heads, self.d_model // self.n_heads)\n sampling_offsets = self.sampling_offsets(query).view(N, Len_q, self.n_heads, self.n_levels, self.n_points, 2)\n attention_weights = self.attention_weights(query).view(N, Len_q, self.n_heads, self.n_levels * self.n_points)\n attention_weights = F.softmax(attention_weights, -1).view(N, Len_q, self.n_heads, self.n_levels, self.n_points)\n # N, Len_q, n_heads, n_levels, n_points, 2\n if reference_points.shape[-1] == 2:\n offset_normalizer = torch.stack([input_spatial_shapes[..., 1], input_spatial_shapes[..., 0]], -1)\n sampling_locations = reference_points[:, :, None, :, None, :] \\\n + sampling_offsets / offset_normalizer[None, None, None, :, None, :]\n elif reference_points.shape[-1] == 4:\n sampling_locations = reference_points[:, :, None, :, None, :2] \\\n + sampling_offsets / self.n_points * reference_points[:, :, None, :, None, 2:] * 0.5\n else:\n raise ValueError(\n 'Last dim of reference_points must be 2 or 4, but get {} instead.'.format(reference_points.shape[-1]))\n output = MSDeformAttnFunction.apply(\n value, input_spatial_shapes, input_level_start_index, sampling_locations, attention_weights, self.im2col_step)\n output = self.output_proj(output)\n return output\n"
},
{
"path": "batchformer-v2/deformable-detr/models/ops/setup.py",
"content": "# ------------------------------------------------------------------------------------------------\n# Deformable DETR\n# Copyright (c) 2020 SenseTime. All Rights Reserved.\n# Licensed under the Apache License, Version 2.0 [see LICENSE for details]\n# ------------------------------------------------------------------------------------------------\n# Modified from https://github.com/chengdazhi/Deformable-Convolution-V2-PyTorch/tree/pytorch_1.0.0\n# ------------------------------------------------------------------------------------------------\n\nimport os\nimport glob\n\nimport torch\n\nfrom torch.utils.cpp_extension import CUDA_HOME\nfrom torch.utils.cpp_extension import CppExtension\nfrom torch.utils.cpp_extension import CUDAExtension\n\nfrom setuptools import find_packages\nfrom setuptools import setup\n\nrequirements = [\"torch\", \"torchvision\"]\n\ndef get_extensions():\n this_dir = os.path.dirname(os.path.abspath(__file__))\n extensions_dir = os.path.join(this_dir, \"src\")\n\n main_file = glob.glob(os.path.join(extensions_dir, \"*.cpp\"))\n source_cpu = glob.glob(os.path.join(extensions_dir, \"cpu\", \"*.cpp\"))\n source_cuda = glob.glob(os.path.join(extensions_dir, \"cuda\", \"*.cu\"))\n\n sources = main_file + source_cpu\n extension = CppExtension\n extra_compile_args = {\"cxx\": []}\n define_macros = []\n\n if torch.cuda.is_available() and CUDA_HOME is not None:\n extension = CUDAExtension\n sources += source_cuda\n define_macros += [(\"WITH_CUDA\", None)]\n extra_compile_args[\"nvcc\"] = [\n \"-DCUDA_HAS_FP16=1\",\n \"-D__CUDA_NO_HALF_OPERATORS__\",\n \"-D__CUDA_NO_HALF_CONVERSIONS__\",\n \"-D__CUDA_NO_HALF2_OPERATORS__\",\n ]\n else:\n raise NotImplementedError('Cuda is not availabel')\n\n sources = [os.path.join(extensions_dir, s) for s in sources]\n include_dirs = [extensions_dir]\n ext_modules = [\n extension(\n \"MultiScaleDeformableAttention\",\n sources,\n include_dirs=include_dirs,\n define_macros=define_macros,\n extra_compile_args=extra_compile_args,\n )\n ]\n return ext_modules\n\nsetup(\n name=\"MultiScaleDeformableAttention\",\n version=\"1.0\",\n author=\"Weijie Su\",\n url=\"https://github.com/fundamentalvision/Deformable-DETR\",\n description=\"PyTorch Wrapper for CUDA Functions of Multi-Scale Deformable Attention\",\n packages=find_packages(exclude=(\"configs\", \"tests\",)),\n ext_modules=get_extensions(),\n cmdclass={\"build_ext\": torch.utils.cpp_extension.BuildExtension},\n)\n"
},
{
"path": "batchformer-v2/deformable-detr/models/ops/src/cpu/ms_deform_attn_cpu.cpp",
"content": "/*!\n**************************************************************************************************\n* Deformable DETR\n* Copyright (c) 2020 SenseTime. All Rights Reserved.\n* Licensed under the Apache License, Version 2.0 [see LICENSE for details]\n**************************************************************************************************\n* Modified from https://github.com/chengdazhi/Deformable-Convolution-V2-PyTorch/tree/pytorch_1.0.0\n**************************************************************************************************\n*/\n\n#include \n\n#include \n#include \n\n\nat::Tensor\nms_deform_attn_cpu_forward(\n const at::Tensor &value, \n const at::Tensor &spatial_shapes,\n const at::Tensor &level_start_index,\n const at::Tensor &sampling_loc,\n const at::Tensor &attn_weight,\n const int im2col_step)\n{\n AT_ERROR(\"Not implement on cpu\");\n}\n\nstd::vector\nms_deform_attn_cpu_backward(\n const at::Tensor &value, \n const at::Tensor &spatial_shapes,\n const at::Tensor &level_start_index,\n const at::Tensor &sampling_loc,\n const at::Tensor &attn_weight,\n const at::Tensor &grad_output,\n const int im2col_step)\n{\n AT_ERROR(\"Not implement on cpu\");\n}\n\n"
},
{
"path": "batchformer-v2/deformable-detr/models/ops/src/cpu/ms_deform_attn_cpu.h",
"content": "/*!\n**************************************************************************************************\n* Deformable DETR\n* Copyright (c) 2020 SenseTime. All Rights Reserved.\n* Licensed under the Apache License, Version 2.0 [see LICENSE for details]\n**************************************************************************************************\n* Modified from https://github.com/chengdazhi/Deformable-Convolution-V2-PyTorch/tree/pytorch_1.0.0\n**************************************************************************************************\n*/\n\n#pragma once\n#include \n\nat::Tensor\nms_deform_attn_cpu_forward(\n const at::Tensor &value, \n const at::Tensor &spatial_shapes,\n const at::Tensor &level_start_index,\n const at::Tensor &sampling_loc,\n const at::Tensor &attn_weight,\n const int im2col_step);\n\nstd::vector\nms_deform_attn_cpu_backward(\n const at::Tensor &value, \n const at::Tensor &spatial_shapes,\n const at::Tensor &level_start_index,\n const at::Tensor &sampling_loc,\n const at::Tensor &attn_weight,\n const at::Tensor &grad_output,\n const int im2col_step);\n\n\n"
},
{
"path": "batchformer-v2/deformable-detr/models/ops/src/cuda/ms_deform_attn_cuda.cu",
"content": "/*!\n**************************************************************************************************\n* Deformable DETR\n* Copyright (c) 2020 SenseTime. All Rights Reserved.\n* Licensed under the Apache License, Version 2.0 [see LICENSE for details]\n**************************************************************************************************\n* Modified from https://github.com/chengdazhi/Deformable-Convolution-V2-PyTorch/tree/pytorch_1.0.0\n**************************************************************************************************\n*/\n\n#include \n#include \"cuda/ms_deform_im2col_cuda.cuh\"\n\n#include \n#include \n#include \n#include \n\n\nat::Tensor ms_deform_attn_cuda_forward(\n const at::Tensor &value, \n const at::Tensor &spatial_shapes,\n const at::Tensor &level_start_index,\n const at::Tensor &sampling_loc,\n const at::Tensor &attn_weight,\n const int im2col_step)\n{\n AT_ASSERTM(value.is_contiguous(), \"value tensor has to be contiguous\");\n AT_ASSERTM(spatial_shapes.is_contiguous(), \"spatial_shapes tensor has to be contiguous\");\n AT_ASSERTM(level_start_index.is_contiguous(), \"level_start_index tensor has to be contiguous\");\n AT_ASSERTM(sampling_loc.is_contiguous(), \"sampling_loc tensor has to be contiguous\");\n AT_ASSERTM(attn_weight.is_contiguous(), \"attn_weight tensor has to be contiguous\");\n\n AT_ASSERTM(value.type().is_cuda(), \"value must be a CUDA tensor\");\n AT_ASSERTM(spatial_shapes.type().is_cuda(), \"spatial_shapes must be a CUDA tensor\");\n AT_ASSERTM(level_start_index.type().is_cuda(), \"level_start_index must be a CUDA tensor\");\n AT_ASSERTM(sampling_loc.type().is_cuda(), \"sampling_loc must be a CUDA tensor\");\n AT_ASSERTM(attn_weight.type().is_cuda(), \"attn_weight must be a CUDA tensor\");\n\n const int batch = value.size(0);\n const int spatial_size = value.size(1);\n const int num_heads = value.size(2);\n const int channels = value.size(3);\n\n const int num_levels = spatial_shapes.size(0);\n\n const int num_query = sampling_loc.size(1);\n const int num_point = sampling_loc.size(4);\n\n const int im2col_step_ = std::min(batch, im2col_step);\n\n AT_ASSERTM(batch % im2col_step_ == 0, \"batch(%d) must divide im2col_step(%d)\", batch, im2col_step_);\n \n auto output = at::zeros({batch, num_query, num_heads, channels}, value.options());\n\n const int batch_n = im2col_step_;\n auto output_n = output.view({batch/im2col_step_, batch_n, num_query, num_heads, channels});\n auto per_value_size = spatial_size * num_heads * channels;\n auto per_sample_loc_size = num_query * num_heads * num_levels * num_point * 2;\n auto per_attn_weight_size = num_query * num_heads * num_levels * num_point;\n for (int n = 0; n < batch/im2col_step_; ++n)\n {\n auto columns = output_n.select(0, n);\n AT_DISPATCH_FLOATING_TYPES(value.type(), \"ms_deform_attn_forward_cuda\", ([&] {\n ms_deformable_im2col_cuda(at::cuda::getCurrentCUDAStream(),\n value.data() + n * im2col_step_ * per_value_size,\n spatial_shapes.data(),\n level_start_index.data(),\n sampling_loc.data() + n * im2col_step_ * per_sample_loc_size,\n attn_weight.data() + n * im2col_step_ * per_attn_weight_size,\n batch_n, spatial_size, num_heads, channels, num_levels, num_query, num_point,\n columns.data());\n\n }));\n }\n\n output = output.view({batch, num_query, num_heads*channels});\n\n return output;\n}\n\n\nstd::vector ms_deform_attn_cuda_backward(\n const at::Tensor &value, \n const at::Tensor &spatial_shapes,\n const at::Tensor &level_start_index,\n const at::Tensor &sampling_loc,\n const at::Tensor &attn_weight,\n const at::Tensor &grad_output,\n const int im2col_step)\n{\n\n AT_ASSERTM(value.is_contiguous(), \"value tensor has to be contiguous\");\n AT_ASSERTM(spatial_shapes.is_contiguous(), \"spatial_shapes tensor has to be contiguous\");\n AT_ASSERTM(level_start_index.is_contiguous(), \"level_start_index tensor has to be contiguous\");\n AT_ASSERTM(sampling_loc.is_contiguous(), \"sampling_loc tensor has to be contiguous\");\n AT_ASSERTM(attn_weight.is_contiguous(), \"attn_weight tensor has to be contiguous\");\n AT_ASSERTM(grad_output.is_contiguous(), \"grad_output tensor has to be contiguous\");\n\n AT_ASSERTM(value.type().is_cuda(), \"value must be a CUDA tensor\");\n AT_ASSERTM(spatial_shapes.type().is_cuda(), \"spatial_shapes must be a CUDA tensor\");\n AT_ASSERTM(level_start_index.type().is_cuda(), \"level_start_index must be a CUDA tensor\");\n AT_ASSERTM(sampling_loc.type().is_cuda(), \"sampling_loc must be a CUDA tensor\");\n AT_ASSERTM(attn_weight.type().is_cuda(), \"attn_weight must be a CUDA tensor\");\n AT_ASSERTM(grad_output.type().is_cuda(), \"grad_output must be a CUDA tensor\");\n\n const int batch = value.size(0);\n const int spatial_size = value.size(1);\n const int num_heads = value.size(2);\n const int channels = value.size(3);\n\n const int num_levels = spatial_shapes.size(0);\n\n const int num_query = sampling_loc.size(1);\n const int num_point = sampling_loc.size(4);\n\n const int im2col_step_ = std::min(batch, im2col_step);\n\n AT_ASSERTM(batch % im2col_step_ == 0, \"batch(%d) must divide im2col_step(%d)\", batch, im2col_step_);\n\n auto grad_value = at::zeros_like(value);\n auto grad_sampling_loc = at::zeros_like(sampling_loc);\n auto grad_attn_weight = at::zeros_like(attn_weight);\n\n const int batch_n = im2col_step_;\n auto per_value_size = spatial_size * num_heads * channels;\n auto per_sample_loc_size = num_query * num_heads * num_levels * num_point * 2;\n auto per_attn_weight_size = num_query * num_heads * num_levels * num_point;\n auto grad_output_n = grad_output.view({batch/im2col_step_, batch_n, num_query, num_heads, channels});\n \n for (int n = 0; n < batch/im2col_step_; ++n)\n {\n auto grad_output_g = grad_output_n.select(0, n);\n AT_DISPATCH_FLOATING_TYPES(value.type(), \"ms_deform_attn_backward_cuda\", ([&] {\n ms_deformable_col2im_cuda(at::cuda::getCurrentCUDAStream(),\n grad_output_g.data(),\n value.data() + n * im2col_step_ * per_value_size,\n spatial_shapes.data(),\n level_start_index.data(),\n sampling_loc.data() + n * im2col_step_ * per_sample_loc_size,\n attn_weight.data() + n * im2col_step_ * per_attn_weight_size,\n batch_n, spatial_size, num_heads, channels, num_levels, num_query, num_point,\n grad_value.data() + n * im2col_step_ * per_value_size,\n grad_sampling_loc.data() + n * im2col_step_ * per_sample_loc_size,\n grad_attn_weight.data() + n * im2col_step_ * per_attn_weight_size);\n\n }));\n }\n\n return {\n grad_value, grad_sampling_loc, grad_attn_weight\n };\n}"
},
{
"path": "batchformer-v2/deformable-detr/models/ops/src/cuda/ms_deform_attn_cuda.h",
"content": "/*!\n**************************************************************************************************\n* Deformable DETR\n* Copyright (c) 2020 SenseTime. All Rights Reserved.\n* Licensed under the Apache License, Version 2.0 [see LICENSE for details]\n**************************************************************************************************\n* Modified from https://github.com/chengdazhi/Deformable-Convolution-V2-PyTorch/tree/pytorch_1.0.0\n**************************************************************************************************\n*/\n\n#pragma once\n#include \n\nat::Tensor ms_deform_attn_cuda_forward(\n const at::Tensor &value, \n const at::Tensor &spatial_shapes,\n const at::Tensor &level_start_index,\n const at::Tensor &sampling_loc,\n const at::Tensor &attn_weight,\n const int im2col_step);\n\nstd::vector ms_deform_attn_cuda_backward(\n const at::Tensor &value, \n const at::Tensor &spatial_shapes,\n const at::Tensor &level_start_index,\n const at::Tensor &sampling_loc,\n const at::Tensor &attn_weight,\n const at::Tensor &grad_output,\n const int im2col_step);\n\n"
},
{
"path": "batchformer-v2/deformable-detr/models/ops/src/cuda/ms_deform_im2col_cuda.cuh",
"content": "/*!\n**************************************************************************\n* Deformable DETR\n* Copyright (c) 2020 SenseTime. All Rights Reserved.\n* Licensed under the Apache License, Version 2.0 [see LICENSE for details]\n**************************************************************************\n* Modified from DCN (https://github.com/msracver/Deformable-ConvNets)\n* Copyright (c) 2018 Microsoft\n**************************************************************************\n*/\n\n#include \n#include \n#include \n\n#include \n#include \n\n#include \n\n#define CUDA_KERNEL_LOOP(i, n) \\\n for (int i = blockIdx.x * blockDim.x + threadIdx.x; \\\n i < (n); \\\n i += blockDim.x * gridDim.x)\n\nconst int CUDA_NUM_THREADS = 1024;\ninline int GET_BLOCKS(const int N, const int num_threads)\n{\n return (N + num_threads - 1) / num_threads;\n}\n\n\ntemplate \n__device__ scalar_t ms_deform_attn_im2col_bilinear(const scalar_t* &bottom_data, \n const int &height, const int &width, const int &nheads, const int &channels,\n const scalar_t &h, const scalar_t &w, const int &m, const int &c)\n{\n const int h_low = floor(h);\n const int w_low = floor(w);\n const int h_high = h_low + 1;\n const int w_high = w_low + 1;\n\n const scalar_t lh = h - h_low;\n const scalar_t lw = w - w_low;\n const scalar_t hh = 1 - lh, hw = 1 - lw;\n\n const int w_stride = nheads * channels;\n const int h_stride = width * w_stride;\n const int h_low_ptr_offset = h_low * h_stride;\n const int h_high_ptr_offset = h_low_ptr_offset + h_stride;\n const int w_low_ptr_offset = w_low * w_stride;\n const int w_high_ptr_offset = w_low_ptr_offset + w_stride;\n const int base_ptr = m * channels + c;\n\n scalar_t v1 = 0;\n if (h_low >= 0 && w_low >= 0)\n {\n const int ptr1 = h_low_ptr_offset + w_low_ptr_offset + base_ptr;\n v1 = bottom_data[ptr1];\n }\n scalar_t v2 = 0;\n if (h_low >= 0 && w_high <= width - 1)\n {\n const int ptr2 = h_low_ptr_offset + w_high_ptr_offset + base_ptr;\n v2 = bottom_data[ptr2];\n }\n scalar_t v3 = 0;\n if (h_high <= height - 1 && w_low >= 0)\n {\n const int ptr3 = h_high_ptr_offset + w_low_ptr_offset + base_ptr;\n v3 = bottom_data[ptr3];\n }\n scalar_t v4 = 0;\n if (h_high <= height - 1 && w_high <= width - 1)\n {\n const int ptr4 = h_high_ptr_offset + w_high_ptr_offset + base_ptr;\n v4 = bottom_data[ptr4];\n }\n\n const scalar_t w1 = hh * hw, w2 = hh * lw, w3 = lh * hw, w4 = lh * lw;\n\n const scalar_t val = (w1 * v1 + w2 * v2 + w3 * v3 + w4 * v4);\n return val;\n}\n\n\ntemplate \n__device__ void ms_deform_attn_col2im_bilinear(const scalar_t* &bottom_data, \n const int &height, const int &width, const int &nheads, const int &channels,\n const scalar_t &h, const scalar_t &w, const int &m, const int &c,\n const scalar_t &top_grad,\n const scalar_t &attn_weight,\n scalar_t* &grad_value, \n scalar_t* grad_sampling_loc,\n scalar_t* grad_attn_weight)\n{\n const int h_low = floor(h);\n const int w_low = floor(w);\n const int h_high = h_low + 1;\n const int w_high = w_low + 1;\n\n const scalar_t lh = h - h_low;\n const scalar_t lw = w - w_low;\n const scalar_t hh = 1 - lh, hw = 1 - lw;\n\n const int w_stride = nheads * channels;\n const int h_stride = width * w_stride;\n const int h_low_ptr_offset = h_low * h_stride;\n const int h_high_ptr_offset = h_low_ptr_offset + h_stride;\n const int w_low_ptr_offset = w_low * w_stride;\n const int w_high_ptr_offset = w_low_ptr_offset + w_stride;\n const int base_ptr = m * channels + c;\n\n const scalar_t w1 = hh * hw, w2 = hh * lw, w3 = lh * hw, w4 = lh * lw;\n const scalar_t top_grad_value = top_grad * attn_weight;\n scalar_t grad_h_weight = 0, grad_w_weight = 0;\n\n scalar_t v1 = 0;\n if (h_low >= 0 && w_low >= 0)\n {\n const int ptr1 = h_low_ptr_offset + w_low_ptr_offset + base_ptr;\n v1 = bottom_data[ptr1];\n grad_h_weight -= hw * v1;\n grad_w_weight -= hh * v1;\n atomicAdd(grad_value+ptr1, w1*top_grad_value);\n }\n scalar_t v2 = 0;\n if (h_low >= 0 && w_high <= width - 1)\n {\n const int ptr2 = h_low_ptr_offset + w_high_ptr_offset + base_ptr;\n v2 = bottom_data[ptr2];\n grad_h_weight -= lw * v2;\n grad_w_weight += hh * v2;\n atomicAdd(grad_value+ptr2, w2*top_grad_value);\n }\n scalar_t v3 = 0;\n if (h_high <= height - 1 && w_low >= 0)\n {\n const int ptr3 = h_high_ptr_offset + w_low_ptr_offset + base_ptr;\n v3 = bottom_data[ptr3];\n grad_h_weight += hw * v3;\n grad_w_weight -= lh * v3;\n atomicAdd(grad_value+ptr3, w3*top_grad_value); \n }\n scalar_t v4 = 0;\n if (h_high <= height - 1 && w_high <= width - 1)\n {\n const int ptr4 = h_high_ptr_offset + w_high_ptr_offset + base_ptr;\n v4 = bottom_data[ptr4];\n grad_h_weight += lw * v4;\n grad_w_weight += lh * v4;\n atomicAdd(grad_value+ptr4, w4*top_grad_value);\n }\n\n const scalar_t val = (w1 * v1 + w2 * v2 + w3 * v3 + w4 * v4);\n *grad_attn_weight = top_grad * val;\n *grad_sampling_loc = width * grad_w_weight * top_grad_value;\n *(grad_sampling_loc + 1) = height * grad_h_weight * top_grad_value;\n}\n\n\ntemplate \n__device__ void ms_deform_attn_col2im_bilinear_gm(const scalar_t* &bottom_data, \n const int &height, const int &width, const int &nheads, const int &channels,\n const scalar_t &h, const scalar_t &w, const int &m, const int &c,\n const scalar_t &top_grad,\n const scalar_t &attn_weight,\n scalar_t* &grad_value, \n scalar_t* grad_sampling_loc,\n scalar_t* grad_attn_weight)\n{\n const int h_low = floor(h);\n const int w_low = floor(w);\n const int h_high = h_low + 1;\n const int w_high = w_low + 1;\n\n const scalar_t lh = h - h_low;\n const scalar_t lw = w - w_low;\n const scalar_t hh = 1 - lh, hw = 1 - lw;\n\n const int w_stride = nheads * channels;\n const int h_stride = width * w_stride;\n const int h_low_ptr_offset = h_low * h_stride;\n const int h_high_ptr_offset = h_low_ptr_offset + h_stride;\n const int w_low_ptr_offset = w_low * w_stride;\n const int w_high_ptr_offset = w_low_ptr_offset + w_stride;\n const int base_ptr = m * channels + c;\n\n const scalar_t w1 = hh * hw, w2 = hh * lw, w3 = lh * hw, w4 = lh * lw;\n const scalar_t top_grad_value = top_grad * attn_weight;\n scalar_t grad_h_weight = 0, grad_w_weight = 0;\n\n scalar_t v1 = 0;\n if (h_low >= 0 && w_low >= 0)\n {\n const int ptr1 = h_low_ptr_offset + w_low_ptr_offset + base_ptr;\n v1 = bottom_data[ptr1];\n grad_h_weight -= hw * v1;\n grad_w_weight -= hh * v1;\n atomicAdd(grad_value+ptr1, w1*top_grad_value);\n }\n scalar_t v2 = 0;\n if (h_low >= 0 && w_high <= width - 1)\n {\n const int ptr2 = h_low_ptr_offset + w_high_ptr_offset + base_ptr;\n v2 = bottom_data[ptr2];\n grad_h_weight -= lw * v2;\n grad_w_weight += hh * v2;\n atomicAdd(grad_value+ptr2, w2*top_grad_value);\n }\n scalar_t v3 = 0;\n if (h_high <= height - 1 && w_low >= 0)\n {\n const int ptr3 = h_high_ptr_offset + w_low_ptr_offset + base_ptr;\n v3 = bottom_data[ptr3];\n grad_h_weight += hw * v3;\n grad_w_weight -= lh * v3;\n atomicAdd(grad_value+ptr3, w3*top_grad_value); \n }\n scalar_t v4 = 0;\n if (h_high <= height - 1 && w_high <= width - 1)\n {\n const int ptr4 = h_high_ptr_offset + w_high_ptr_offset + base_ptr;\n v4 = bottom_data[ptr4];\n grad_h_weight += lw * v4;\n grad_w_weight += lh * v4;\n atomicAdd(grad_value+ptr4, w4*top_grad_value);\n }\n\n const scalar_t val = (w1 * v1 + w2 * v2 + w3 * v3 + w4 * v4);\n atomicAdd(grad_attn_weight, top_grad * val); \n atomicAdd(grad_sampling_loc, width * grad_w_weight * top_grad_value);\n atomicAdd(grad_sampling_loc + 1, height * grad_h_weight * top_grad_value);\n}\n\n\ntemplate \n__global__ void ms_deformable_im2col_gpu_kernel(const int n,\n const scalar_t *data_value, \n const int64_t *data_spatial_shapes,\n const int64_t *data_level_start_index, \n const scalar_t *data_sampling_loc,\n const scalar_t *data_attn_weight,\n const int batch_size, \n const int spatial_size, \n const int num_heads,\n const int channels, \n const int num_levels,\n const int num_query,\n const int num_point,\n scalar_t *data_col)\n{\n CUDA_KERNEL_LOOP(index, n)\n {\n int _temp = index;\n const int c_col = _temp % channels;\n _temp /= channels;\n const int sampling_index = _temp; \n const int m_col = _temp % num_heads;\n _temp /= num_heads;\n const int q_col = _temp % num_query;\n _temp /= num_query;\n const int b_col = _temp;\n\n scalar_t *data_col_ptr = data_col + index;\n int data_weight_ptr = sampling_index * num_levels * num_point;\n int data_loc_w_ptr = data_weight_ptr << 1;\n const int qid_stride = num_heads * channels;\n const int data_value_ptr_init_offset = b_col * spatial_size * qid_stride;\n scalar_t col = 0;\n \n for (int l_col=0; l_col < num_levels; ++l_col)\n {\n const int level_start_id = data_level_start_index[l_col];\n const int spatial_h_ptr = l_col << 1;\n const int spatial_h = data_spatial_shapes[spatial_h_ptr];\n const int spatial_w = data_spatial_shapes[spatial_h_ptr + 1];\n const scalar_t *data_value_ptr = data_value + (data_value_ptr_init_offset + level_start_id * qid_stride);\n for (int p_col=0; p_col < num_point; ++p_col)\n {\n const scalar_t loc_w = data_sampling_loc[data_loc_w_ptr];\n const scalar_t loc_h = data_sampling_loc[data_loc_w_ptr + 1];\n const scalar_t weight = data_attn_weight[data_weight_ptr];\n\n const scalar_t h_im = loc_h * spatial_h - 0.5;\n const scalar_t w_im = loc_w * spatial_w - 0.5;\n\n if (h_im > -1 && w_im > -1 && h_im < spatial_h && w_im < spatial_w)\n {\n col += ms_deform_attn_im2col_bilinear(data_value_ptr, spatial_h, spatial_w, num_heads, channels, h_im, w_im, m_col, c_col) * weight;\n }\n\n data_weight_ptr += 1;\n data_loc_w_ptr += 2;\n }\n }\n *data_col_ptr = col;\n }\n}\n\ntemplate \n__global__ void ms_deformable_col2im_gpu_kernel_shm_blocksize_aware_reduce_v1(const int n,\n const scalar_t *grad_col,\n const scalar_t *data_value,\n const int64_t *data_spatial_shapes,\n const int64_t *data_level_start_index, \n const scalar_t *data_sampling_loc,\n const scalar_t *data_attn_weight,\n const int batch_size, \n const int spatial_size, \n const int num_heads,\n const int channels, \n const int num_levels,\n const int num_query,\n const int num_point,\n scalar_t *grad_value,\n scalar_t *grad_sampling_loc,\n scalar_t *grad_attn_weight)\n{\n CUDA_KERNEL_LOOP(index, n)\n {\n __shared__ scalar_t cache_grad_sampling_loc[blockSize * 2];\n __shared__ scalar_t cache_grad_attn_weight[blockSize];\n unsigned int tid = threadIdx.x;\n int _temp = index;\n const int c_col = _temp % channels;\n _temp /= channels;\n const int sampling_index = _temp; \n const int m_col = _temp % num_heads;\n _temp /= num_heads;\n const int q_col = _temp % num_query;\n _temp /= num_query;\n const int b_col = _temp;\n\n const scalar_t top_grad = grad_col[index];\n\n int data_weight_ptr = sampling_index * num_levels * num_point;\n int data_loc_w_ptr = data_weight_ptr << 1;\n const int grad_sampling_ptr = data_weight_ptr;\n grad_sampling_loc += grad_sampling_ptr << 1;\n grad_attn_weight += grad_sampling_ptr;\n const int grad_weight_stride = 1;\n const int grad_loc_stride = 2;\n const int qid_stride = num_heads * channels;\n const int data_value_ptr_init_offset = b_col * spatial_size * qid_stride;\n\n for (int l_col=0; l_col < num_levels; ++l_col)\n {\n const int level_start_id = data_level_start_index[l_col];\n const int spatial_h_ptr = l_col << 1;\n const int spatial_h = data_spatial_shapes[spatial_h_ptr];\n const int spatial_w = data_spatial_shapes[spatial_h_ptr + 1];\n const int value_ptr_offset = data_value_ptr_init_offset + level_start_id * qid_stride;\n const scalar_t *data_value_ptr = data_value + value_ptr_offset;\n scalar_t *grad_value_ptr = grad_value + value_ptr_offset;\n\n for (int p_col=0; p_col < num_point; ++p_col)\n {\n const scalar_t loc_w = data_sampling_loc[data_loc_w_ptr];\n const scalar_t loc_h = data_sampling_loc[data_loc_w_ptr + 1];\n const scalar_t weight = data_attn_weight[data_weight_ptr];\n\n const scalar_t h_im = loc_h * spatial_h - 0.5;\n const scalar_t w_im = loc_w * spatial_w - 0.5;\n *(cache_grad_sampling_loc+(threadIdx.x << 1)) = 0;\n *(cache_grad_sampling_loc+((threadIdx.x << 1) + 1)) = 0;\n *(cache_grad_attn_weight+threadIdx.x)=0;\n if (h_im > -1 && w_im > -1 && h_im < spatial_h && w_im < spatial_w)\n {\n ms_deform_attn_col2im_bilinear(\n data_value_ptr, spatial_h, spatial_w, num_heads, channels, h_im, w_im, m_col, c_col,\n top_grad, weight, grad_value_ptr, \n cache_grad_sampling_loc+(threadIdx.x << 1), cache_grad_attn_weight+threadIdx.x);\n }\n \n __syncthreads();\n if (tid == 0)\n {\n scalar_t _grad_w=cache_grad_sampling_loc[0], _grad_h=cache_grad_sampling_loc[1], _grad_a=cache_grad_attn_weight[0];\n int sid=2;\n for (unsigned int tid = 1; tid < blockSize; ++tid)\n {\n _grad_w += cache_grad_sampling_loc[sid];\n _grad_h += cache_grad_sampling_loc[sid + 1];\n _grad_a += cache_grad_attn_weight[tid];\n sid += 2;\n }\n \n \n *grad_sampling_loc = _grad_w;\n *(grad_sampling_loc + 1) = _grad_h;\n *grad_attn_weight = _grad_a;\n }\n __syncthreads();\n\n data_weight_ptr += 1;\n data_loc_w_ptr += 2;\n grad_attn_weight += grad_weight_stride;\n grad_sampling_loc += grad_loc_stride;\n }\n }\n }\n}\n\n\ntemplate \n__global__ void ms_deformable_col2im_gpu_kernel_shm_blocksize_aware_reduce_v2(const int n,\n const scalar_t *grad_col,\n const scalar_t *data_value,\n const int64_t *data_spatial_shapes,\n const int64_t *data_level_start_index, \n const scalar_t *data_sampling_loc,\n const scalar_t *data_attn_weight,\n const int batch_size, \n const int spatial_size, \n const int num_heads,\n const int channels, \n const int num_levels,\n const int num_query,\n const int num_point,\n scalar_t *grad_value,\n scalar_t *grad_sampling_loc,\n scalar_t *grad_attn_weight)\n{\n CUDA_KERNEL_LOOP(index, n)\n {\n __shared__ scalar_t cache_grad_sampling_loc[blockSize * 2];\n __shared__ scalar_t cache_grad_attn_weight[blockSize];\n unsigned int tid = threadIdx.x;\n int _temp = index;\n const int c_col = _temp % channels;\n _temp /= channels;\n const int sampling_index = _temp; \n const int m_col = _temp % num_heads;\n _temp /= num_heads;\n const int q_col = _temp % num_query;\n _temp /= num_query;\n const int b_col = _temp;\n\n const scalar_t top_grad = grad_col[index];\n\n int data_weight_ptr = sampling_index * num_levels * num_point;\n int data_loc_w_ptr = data_weight_ptr << 1;\n const int grad_sampling_ptr = data_weight_ptr;\n grad_sampling_loc += grad_sampling_ptr << 1;\n grad_attn_weight += grad_sampling_ptr;\n const int grad_weight_stride = 1;\n const int grad_loc_stride = 2;\n const int qid_stride = num_heads * channels;\n const int data_value_ptr_init_offset = b_col * spatial_size * qid_stride;\n\n for (int l_col=0; l_col < num_levels; ++l_col)\n {\n const int level_start_id = data_level_start_index[l_col];\n const int spatial_h_ptr = l_col << 1;\n const int spatial_h = data_spatial_shapes[spatial_h_ptr];\n const int spatial_w = data_spatial_shapes[spatial_h_ptr + 1];\n const int value_ptr_offset = data_value_ptr_init_offset + level_start_id * qid_stride;\n const scalar_t *data_value_ptr = data_value + value_ptr_offset;\n scalar_t *grad_value_ptr = grad_value + value_ptr_offset;\n\n for (int p_col=0; p_col < num_point; ++p_col)\n {\n const scalar_t loc_w = data_sampling_loc[data_loc_w_ptr];\n const scalar_t loc_h = data_sampling_loc[data_loc_w_ptr + 1];\n const scalar_t weight = data_attn_weight[data_weight_ptr];\n\n const scalar_t h_im = loc_h * spatial_h - 0.5;\n const scalar_t w_im = loc_w * spatial_w - 0.5;\n *(cache_grad_sampling_loc+(threadIdx.x << 1)) = 0;\n *(cache_grad_sampling_loc+((threadIdx.x << 1) + 1)) = 0;\n *(cache_grad_attn_weight+threadIdx.x)=0;\n if (h_im > -1 && w_im > -1 && h_im < spatial_h && w_im < spatial_w)\n {\n ms_deform_attn_col2im_bilinear(\n data_value_ptr, spatial_h, spatial_w, num_heads, channels, h_im, w_im, m_col, c_col,\n top_grad, weight, grad_value_ptr, \n cache_grad_sampling_loc+(threadIdx.x << 1), cache_grad_attn_weight+threadIdx.x);\n }\n \n __syncthreads();\n\n for (unsigned int s=blockSize/2; s>0; s>>=1)\n {\n if (tid < s) {\n const unsigned int xid1 = tid << 1;\n const unsigned int xid2 = (tid + s) << 1;\n cache_grad_attn_weight[tid] += cache_grad_attn_weight[tid + s];\n cache_grad_sampling_loc[xid1] += cache_grad_sampling_loc[xid2];\n cache_grad_sampling_loc[xid1 + 1] += cache_grad_sampling_loc[xid2 + 1];\n }\n __syncthreads();\n }\n\n if (tid == 0)\n { \n *grad_sampling_loc = cache_grad_sampling_loc[0];\n *(grad_sampling_loc + 1) = cache_grad_sampling_loc[1];\n *grad_attn_weight = cache_grad_attn_weight[0];\n }\n __syncthreads();\n\n data_weight_ptr += 1;\n data_loc_w_ptr += 2;\n grad_attn_weight += grad_weight_stride;\n grad_sampling_loc += grad_loc_stride;\n }\n }\n }\n}\n\n\ntemplate \n__global__ void ms_deformable_col2im_gpu_kernel_shm_reduce_v1(const int n,\n const scalar_t *grad_col,\n const scalar_t *data_value,\n const int64_t *data_spatial_shapes,\n const int64_t *data_level_start_index, \n const scalar_t *data_sampling_loc,\n const scalar_t *data_attn_weight,\n const int batch_size, \n const int spatial_size, \n const int num_heads,\n const int channels, \n const int num_levels,\n const int num_query,\n const int num_point,\n scalar_t *grad_value,\n scalar_t *grad_sampling_loc,\n scalar_t *grad_attn_weight)\n{\n CUDA_KERNEL_LOOP(index, n)\n {\n extern __shared__ int _s[];\n scalar_t* cache_grad_sampling_loc = (scalar_t*)_s;\n scalar_t* cache_grad_attn_weight = cache_grad_sampling_loc + 2 * blockDim.x;\n unsigned int tid = threadIdx.x;\n int _temp = index;\n const int c_col = _temp % channels;\n _temp /= channels;\n const int sampling_index = _temp; \n const int m_col = _temp % num_heads;\n _temp /= num_heads;\n const int q_col = _temp % num_query;\n _temp /= num_query;\n const int b_col = _temp;\n\n const scalar_t top_grad = grad_col[index];\n\n int data_weight_ptr = sampling_index * num_levels * num_point;\n int data_loc_w_ptr = data_weight_ptr << 1;\n const int grad_sampling_ptr = data_weight_ptr;\n grad_sampling_loc += grad_sampling_ptr << 1;\n grad_attn_weight += grad_sampling_ptr;\n const int grad_weight_stride = 1;\n const int grad_loc_stride = 2;\n const int qid_stride = num_heads * channels;\n const int data_value_ptr_init_offset = b_col * spatial_size * qid_stride;\n\n for (int l_col=0; l_col < num_levels; ++l_col)\n {\n const int level_start_id = data_level_start_index[l_col];\n const int spatial_h_ptr = l_col << 1;\n const int spatial_h = data_spatial_shapes[spatial_h_ptr];\n const int spatial_w = data_spatial_shapes[spatial_h_ptr + 1];\n const int value_ptr_offset = data_value_ptr_init_offset + level_start_id * qid_stride;\n const scalar_t *data_value_ptr = data_value + value_ptr_offset;\n scalar_t *grad_value_ptr = grad_value + value_ptr_offset;\n\n for (int p_col=0; p_col < num_point; ++p_col)\n {\n const scalar_t loc_w = data_sampling_loc[data_loc_w_ptr];\n const scalar_t loc_h = data_sampling_loc[data_loc_w_ptr + 1];\n const scalar_t weight = data_attn_weight[data_weight_ptr];\n\n const scalar_t h_im = loc_h * spatial_h - 0.5;\n const scalar_t w_im = loc_w * spatial_w - 0.5;\n *(cache_grad_sampling_loc+(threadIdx.x << 1)) = 0;\n *(cache_grad_sampling_loc+((threadIdx.x << 1) + 1)) = 0;\n *(cache_grad_attn_weight+threadIdx.x)=0;\n if (h_im > -1 && w_im > -1 && h_im < spatial_h && w_im < spatial_w)\n {\n ms_deform_attn_col2im_bilinear(\n data_value_ptr, spatial_h, spatial_w, num_heads, channels, h_im, w_im, m_col, c_col,\n top_grad, weight, grad_value_ptr, \n cache_grad_sampling_loc+(threadIdx.x << 1), cache_grad_attn_weight+threadIdx.x);\n }\n \n __syncthreads();\n if (tid == 0)\n {\n scalar_t _grad_w=cache_grad_sampling_loc[0], _grad_h=cache_grad_sampling_loc[1], _grad_a=cache_grad_attn_weight[0];\n int sid=2;\n for (unsigned int tid = 1; tid < blockDim.x; ++tid)\n {\n _grad_w += cache_grad_sampling_loc[sid];\n _grad_h += cache_grad_sampling_loc[sid + 1];\n _grad_a += cache_grad_attn_weight[tid];\n sid += 2;\n }\n \n \n *grad_sampling_loc = _grad_w;\n *(grad_sampling_loc + 1) = _grad_h;\n *grad_attn_weight = _grad_a;\n }\n __syncthreads();\n\n data_weight_ptr += 1;\n data_loc_w_ptr += 2;\n grad_attn_weight += grad_weight_stride;\n grad_sampling_loc += grad_loc_stride;\n }\n }\n }\n}\n\ntemplate \n__global__ void ms_deformable_col2im_gpu_kernel_shm_reduce_v2(const int n,\n const scalar_t *grad_col,\n const scalar_t *data_value,\n const int64_t *data_spatial_shapes,\n const int64_t *data_level_start_index, \n const scalar_t *data_sampling_loc,\n const scalar_t *data_attn_weight,\n const int batch_size, \n const int spatial_size, \n const int num_heads,\n const int channels, \n const int num_levels,\n const int num_query,\n const int num_point,\n scalar_t *grad_value,\n scalar_t *grad_sampling_loc,\n scalar_t *grad_attn_weight)\n{\n CUDA_KERNEL_LOOP(index, n)\n {\n extern __shared__ int _s[];\n scalar_t* cache_grad_sampling_loc = (scalar_t*)_s;\n scalar_t* cache_grad_attn_weight = cache_grad_sampling_loc + 2 * blockDim.x;\n unsigned int tid = threadIdx.x;\n int _temp = index;\n const int c_col = _temp % channels;\n _temp /= channels;\n const int sampling_index = _temp; \n const int m_col = _temp % num_heads;\n _temp /= num_heads;\n const int q_col = _temp % num_query;\n _temp /= num_query;\n const int b_col = _temp;\n\n const scalar_t top_grad = grad_col[index];\n\n int data_weight_ptr = sampling_index * num_levels * num_point;\n int data_loc_w_ptr = data_weight_ptr << 1;\n const int grad_sampling_ptr = data_weight_ptr;\n grad_sampling_loc += grad_sampling_ptr << 1;\n grad_attn_weight += grad_sampling_ptr;\n const int grad_weight_stride = 1;\n const int grad_loc_stride = 2;\n const int qid_stride = num_heads * channels;\n const int data_value_ptr_init_offset = b_col * spatial_size * qid_stride;\n\n for (int l_col=0; l_col < num_levels; ++l_col)\n {\n const int level_start_id = data_level_start_index[l_col];\n const int spatial_h_ptr = l_col << 1;\n const int spatial_h = data_spatial_shapes[spatial_h_ptr];\n const int spatial_w = data_spatial_shapes[spatial_h_ptr + 1];\n const int value_ptr_offset = data_value_ptr_init_offset + level_start_id * qid_stride;\n const scalar_t *data_value_ptr = data_value + value_ptr_offset;\n scalar_t *grad_value_ptr = grad_value + value_ptr_offset;\n\n for (int p_col=0; p_col < num_point; ++p_col)\n {\n const scalar_t loc_w = data_sampling_loc[data_loc_w_ptr];\n const scalar_t loc_h = data_sampling_loc[data_loc_w_ptr + 1];\n const scalar_t weight = data_attn_weight[data_weight_ptr];\n\n const scalar_t h_im = loc_h * spatial_h - 0.5;\n const scalar_t w_im = loc_w * spatial_w - 0.5;\n *(cache_grad_sampling_loc+(threadIdx.x << 1)) = 0;\n *(cache_grad_sampling_loc+((threadIdx.x << 1) + 1)) = 0;\n *(cache_grad_attn_weight+threadIdx.x)=0;\n if (h_im > -1 && w_im > -1 && h_im < spatial_h && w_im < spatial_w)\n {\n ms_deform_attn_col2im_bilinear(\n data_value_ptr, spatial_h, spatial_w, num_heads, channels, h_im, w_im, m_col, c_col,\n top_grad, weight, grad_value_ptr, \n cache_grad_sampling_loc+(threadIdx.x << 1), cache_grad_attn_weight+threadIdx.x);\n }\n \n __syncthreads();\n\n for (unsigned int s=blockDim.x/2, spre=blockDim.x; s>0; s>>=1, spre>>=1)\n {\n if (tid < s) {\n const unsigned int xid1 = tid << 1;\n const unsigned int xid2 = (tid + s) << 1;\n cache_grad_attn_weight[tid] += cache_grad_attn_weight[tid + s];\n cache_grad_sampling_loc[xid1] += cache_grad_sampling_loc[xid2];\n cache_grad_sampling_loc[xid1 + 1] += cache_grad_sampling_loc[xid2 + 1];\n if (tid + (s << 1) < spre)\n {\n cache_grad_attn_weight[tid] += cache_grad_attn_weight[tid + (s << 1)];\n cache_grad_sampling_loc[xid1] += cache_grad_sampling_loc[xid2 + (s << 1)];\n cache_grad_sampling_loc[xid1 + 1] += cache_grad_sampling_loc[xid2 + 1 + (s << 1)];\n } \n }\n __syncthreads();\n }\n\n if (tid == 0)\n {\n *grad_sampling_loc = cache_grad_sampling_loc[0];\n *(grad_sampling_loc + 1) = cache_grad_sampling_loc[1];\n *grad_attn_weight = cache_grad_attn_weight[0];\n }\n __syncthreads();\n\n data_weight_ptr += 1;\n data_loc_w_ptr += 2;\n grad_attn_weight += grad_weight_stride;\n grad_sampling_loc += grad_loc_stride;\n }\n }\n }\n}\n\ntemplate \n__global__ void ms_deformable_col2im_gpu_kernel_shm_reduce_v2_multi_blocks(const int n,\n const scalar_t *grad_col,\n const scalar_t *data_value,\n const int64_t *data_spatial_shapes,\n const int64_t *data_level_start_index, \n const scalar_t *data_sampling_loc,\n const scalar_t *data_attn_weight,\n const int batch_size, \n const int spatial_size, \n const int num_heads,\n const int channels, \n const int num_levels,\n const int num_query,\n const int num_point,\n scalar_t *grad_value,\n scalar_t *grad_sampling_loc,\n scalar_t *grad_attn_weight)\n{\n CUDA_KERNEL_LOOP(index, n)\n {\n extern __shared__ int _s[];\n scalar_t* cache_grad_sampling_loc = (scalar_t*)_s;\n scalar_t* cache_grad_attn_weight = cache_grad_sampling_loc + 2 * blockDim.x;\n unsigned int tid = threadIdx.x;\n int _temp = index;\n const int c_col = _temp % channels;\n _temp /= channels;\n const int sampling_index = _temp; \n const int m_col = _temp % num_heads;\n _temp /= num_heads;\n const int q_col = _temp % num_query;\n _temp /= num_query;\n const int b_col = _temp;\n\n const scalar_t top_grad = grad_col[index];\n\n int data_weight_ptr = sampling_index * num_levels * num_point;\n int data_loc_w_ptr = data_weight_ptr << 1;\n const int grad_sampling_ptr = data_weight_ptr;\n grad_sampling_loc += grad_sampling_ptr << 1;\n grad_attn_weight += grad_sampling_ptr;\n const int grad_weight_stride = 1;\n const int grad_loc_stride = 2;\n const int qid_stride = num_heads * channels;\n const int data_value_ptr_init_offset = b_col * spatial_size * qid_stride;\n\n for (int l_col=0; l_col < num_levels; ++l_col)\n {\n const int level_start_id = data_level_start_index[l_col];\n const int spatial_h_ptr = l_col << 1;\n const int spatial_h = data_spatial_shapes[spatial_h_ptr];\n const int spatial_w = data_spatial_shapes[spatial_h_ptr + 1];\n const int value_ptr_offset = data_value_ptr_init_offset + level_start_id * qid_stride;\n const scalar_t *data_value_ptr = data_value + value_ptr_offset;\n scalar_t *grad_value_ptr = grad_value + value_ptr_offset;\n\n for (int p_col=0; p_col < num_point; ++p_col)\n {\n const scalar_t loc_w = data_sampling_loc[data_loc_w_ptr];\n const scalar_t loc_h = data_sampling_loc[data_loc_w_ptr + 1];\n const scalar_t weight = data_attn_weight[data_weight_ptr];\n\n const scalar_t h_im = loc_h * spatial_h - 0.5;\n const scalar_t w_im = loc_w * spatial_w - 0.5;\n *(cache_grad_sampling_loc+(threadIdx.x << 1)) = 0;\n *(cache_grad_sampling_loc+((threadIdx.x << 1) + 1)) = 0;\n *(cache_grad_attn_weight+threadIdx.x)=0;\n if (h_im > -1 && w_im > -1 && h_im < spatial_h && w_im < spatial_w)\n {\n ms_deform_attn_col2im_bilinear(\n data_value_ptr, spatial_h, spatial_w, num_heads, channels, h_im, w_im, m_col, c_col,\n top_grad, weight, grad_value_ptr, \n cache_grad_sampling_loc+(threadIdx.x << 1), cache_grad_attn_weight+threadIdx.x);\n }\n \n __syncthreads();\n\n for (unsigned int s=blockDim.x/2, spre=blockDim.x; s>0; s>>=1, spre>>=1)\n {\n if (tid < s) {\n const unsigned int xid1 = tid << 1;\n const unsigned int xid2 = (tid + s) << 1;\n cache_grad_attn_weight[tid] += cache_grad_attn_weight[tid + s];\n cache_grad_sampling_loc[xid1] += cache_grad_sampling_loc[xid2];\n cache_grad_sampling_loc[xid1 + 1] += cache_grad_sampling_loc[xid2 + 1];\n if (tid + (s << 1) < spre)\n {\n cache_grad_attn_weight[tid] += cache_grad_attn_weight[tid + (s << 1)];\n cache_grad_sampling_loc[xid1] += cache_grad_sampling_loc[xid2 + (s << 1)];\n cache_grad_sampling_loc[xid1 + 1] += cache_grad_sampling_loc[xid2 + 1 + (s << 1)];\n }\n }\n __syncthreads();\n }\n\n if (tid == 0)\n {\n atomicAdd(grad_sampling_loc, cache_grad_sampling_loc[0]);\n atomicAdd(grad_sampling_loc + 1, cache_grad_sampling_loc[1]);\n atomicAdd(grad_attn_weight, cache_grad_attn_weight[0]);\n }\n __syncthreads();\n\n data_weight_ptr += 1;\n data_loc_w_ptr += 2;\n grad_attn_weight += grad_weight_stride;\n grad_sampling_loc += grad_loc_stride;\n }\n }\n }\n}\n\n\ntemplate \n__global__ void ms_deformable_col2im_gpu_kernel_gm(const int n,\n const scalar_t *grad_col,\n const scalar_t *data_value,\n const int64_t *data_spatial_shapes,\n const int64_t *data_level_start_index, \n const scalar_t *data_sampling_loc,\n const scalar_t *data_attn_weight,\n const int batch_size, \n const int spatial_size, \n const int num_heads,\n const int channels, \n const int num_levels,\n const int num_query,\n const int num_point,\n scalar_t *grad_value,\n scalar_t *grad_sampling_loc,\n scalar_t *grad_attn_weight)\n{\n CUDA_KERNEL_LOOP(index, n)\n {\n int _temp = index;\n const int c_col = _temp % channels;\n _temp /= channels;\n const int sampling_index = _temp; \n const int m_col = _temp % num_heads;\n _temp /= num_heads;\n const int q_col = _temp % num_query;\n _temp /= num_query;\n const int b_col = _temp;\n\n const scalar_t top_grad = grad_col[index];\n\n int data_weight_ptr = sampling_index * num_levels * num_point;\n int data_loc_w_ptr = data_weight_ptr << 1;\n const int grad_sampling_ptr = data_weight_ptr;\n grad_sampling_loc += grad_sampling_ptr << 1;\n grad_attn_weight += grad_sampling_ptr;\n const int grad_weight_stride = 1;\n const int grad_loc_stride = 2;\n const int qid_stride = num_heads * channels;\n const int data_value_ptr_init_offset = b_col * spatial_size * qid_stride;\n\n for (int l_col=0; l_col < num_levels; ++l_col)\n {\n const int level_start_id = data_level_start_index[l_col];\n const int spatial_h_ptr = l_col << 1;\n const int spatial_h = data_spatial_shapes[spatial_h_ptr];\n const int spatial_w = data_spatial_shapes[spatial_h_ptr + 1];\n const int value_ptr_offset = data_value_ptr_init_offset + level_start_id * qid_stride;\n const scalar_t *data_value_ptr = data_value + value_ptr_offset;\n scalar_t *grad_value_ptr = grad_value + value_ptr_offset;\n\n for (int p_col=0; p_col < num_point; ++p_col)\n {\n const scalar_t loc_w = data_sampling_loc[data_loc_w_ptr];\n const scalar_t loc_h = data_sampling_loc[data_loc_w_ptr + 1];\n const scalar_t weight = data_attn_weight[data_weight_ptr];\n\n const scalar_t h_im = loc_h * spatial_h - 0.5;\n const scalar_t w_im = loc_w * spatial_w - 0.5;\n if (h_im > -1 && w_im > -1 && h_im < spatial_h && w_im < spatial_w)\n {\n ms_deform_attn_col2im_bilinear_gm(\n data_value_ptr, spatial_h, spatial_w, num_heads, channels, h_im, w_im, m_col, c_col,\n top_grad, weight, grad_value_ptr, \n grad_sampling_loc, grad_attn_weight);\n }\n data_weight_ptr += 1;\n data_loc_w_ptr += 2;\n grad_attn_weight += grad_weight_stride;\n grad_sampling_loc += grad_loc_stride;\n }\n }\n }\n}\n\n\ntemplate \nvoid ms_deformable_im2col_cuda(cudaStream_t stream,\n const scalar_t* data_value,\n const int64_t* data_spatial_shapes, \n const int64_t* data_level_start_index, \n const scalar_t* data_sampling_loc,\n const scalar_t* data_attn_weight,\n const int batch_size,\n const int spatial_size, \n const int num_heads, \n const int channels, \n const int num_levels, \n const int num_query,\n const int num_point,\n scalar_t* data_col)\n{\n const int num_kernels = batch_size * num_query * num_heads * channels;\n const int num_actual_kernels = batch_size * num_query * num_heads * channels;\n const int num_threads = CUDA_NUM_THREADS;\n ms_deformable_im2col_gpu_kernel\n <<>>(\n num_kernels, data_value, data_spatial_shapes, data_level_start_index, data_sampling_loc, data_attn_weight, \n batch_size, spatial_size, num_heads, channels, num_levels, num_query, num_point, data_col);\n \n cudaError_t err = cudaGetLastError();\n if (err != cudaSuccess)\n {\n printf(\"error in ms_deformable_im2col_cuda: %s\\n\", cudaGetErrorString(err));\n }\n\n}\n\ntemplate \nvoid ms_deformable_col2im_cuda(cudaStream_t stream,\n const scalar_t* grad_col,\n const scalar_t* data_value,\n const int64_t * data_spatial_shapes,\n const int64_t * data_level_start_index,\n const scalar_t * data_sampling_loc,\n const scalar_t * data_attn_weight,\n const int batch_size, \n const int spatial_size, \n const int num_heads,\n const int channels, \n const int num_levels,\n const int num_query,\n const int num_point, \n scalar_t* grad_value,\n scalar_t* grad_sampling_loc,\n scalar_t* grad_attn_weight)\n{\n const int num_threads = (channels > CUDA_NUM_THREADS)?CUDA_NUM_THREADS:channels;\n const int num_kernels = batch_size * num_query * num_heads * channels;\n const int num_actual_kernels = batch_size * num_query * num_heads * channels;\n if (channels > 1024)\n {\n if ((channels & 1023) == 0)\n {\n ms_deformable_col2im_gpu_kernel_shm_reduce_v2_multi_blocks\n <<>>(\n num_kernels, \n grad_col,\n data_value,\n data_spatial_shapes,\n data_level_start_index, \n data_sampling_loc,\n data_attn_weight,\n batch_size, \n spatial_size, \n num_heads,\n channels, \n num_levels,\n num_query,\n num_point,\n grad_value,\n grad_sampling_loc,\n grad_attn_weight);\n }\n else\n {\n ms_deformable_col2im_gpu_kernel_gm\n <<>>(\n num_kernels, \n grad_col,\n data_value,\n data_spatial_shapes,\n data_level_start_index, \n data_sampling_loc,\n data_attn_weight,\n batch_size, \n spatial_size, \n num_heads,\n channels, \n num_levels,\n num_query,\n num_point,\n grad_value,\n grad_sampling_loc,\n grad_attn_weight);\n }\n }\n else{\n switch(channels)\n {\n case 1:\n ms_deformable_col2im_gpu_kernel_shm_blocksize_aware_reduce_v1\n <<>>(\n num_kernels, \n grad_col,\n data_value,\n data_spatial_shapes,\n data_level_start_index, \n data_sampling_loc,\n data_attn_weight,\n batch_size, \n spatial_size, \n num_heads,\n channels, \n num_levels,\n num_query,\n num_point,\n grad_value,\n grad_sampling_loc,\n grad_attn_weight);\n break;\n case 2:\n ms_deformable_col2im_gpu_kernel_shm_blocksize_aware_reduce_v1\n <<>>(\n num_kernels, \n grad_col,\n data_value,\n data_spatial_shapes,\n data_level_start_index, \n data_sampling_loc,\n data_attn_weight,\n batch_size, \n spatial_size, \n num_heads,\n channels, \n num_levels,\n num_query,\n num_point,\n grad_value,\n grad_sampling_loc,\n grad_attn_weight);\n break;\n case 4:\n ms_deformable_col2im_gpu_kernel_shm_blocksize_aware_reduce_v1\n <<>>(\n num_kernels, \n grad_col,\n data_value,\n data_spatial_shapes,\n data_level_start_index, \n data_sampling_loc,\n data_attn_weight,\n batch_size, \n spatial_size, \n num_heads,\n channels, \n num_levels,\n num_query,\n num_point,\n grad_value,\n grad_sampling_loc,\n grad_attn_weight);\n break;\n case 8:\n ms_deformable_col2im_gpu_kernel_shm_blocksize_aware_reduce_v1\n <<>>(\n num_kernels, \n grad_col,\n data_value,\n data_spatial_shapes,\n data_level_start_index, \n data_sampling_loc,\n data_attn_weight,\n batch_size, \n spatial_size, \n num_heads,\n channels, \n num_levels,\n num_query,\n num_point,\n grad_value,\n grad_sampling_loc,\n grad_attn_weight);\n break;\n case 16:\n ms_deformable_col2im_gpu_kernel_shm_blocksize_aware_reduce_v1\n <<>>(\n num_kernels, \n grad_col,\n data_value,\n data_spatial_shapes,\n data_level_start_index, \n data_sampling_loc,\n data_attn_weight,\n batch_size, \n spatial_size, \n num_heads,\n channels, \n num_levels,\n num_query,\n num_point,\n grad_value,\n grad_sampling_loc,\n grad_attn_weight);\n break;\n case 32:\n ms_deformable_col2im_gpu_kernel_shm_blocksize_aware_reduce_v1\n <<>>(\n num_kernels, \n grad_col,\n data_value,\n data_spatial_shapes,\n data_level_start_index, \n data_sampling_loc,\n data_attn_weight,\n batch_size, \n spatial_size, \n num_heads,\n channels, \n num_levels,\n num_query,\n num_point,\n grad_value,\n grad_sampling_loc,\n grad_attn_weight);\n break;\n case 64:\n ms_deformable_col2im_gpu_kernel_shm_blocksize_aware_reduce_v2\n <<>>(\n num_kernels, \n grad_col,\n data_value,\n data_spatial_shapes,\n data_level_start_index, \n data_sampling_loc,\n data_attn_weight,\n batch_size, \n spatial_size, \n num_heads,\n channels, \n num_levels,\n num_query,\n num_point,\n grad_value,\n grad_sampling_loc,\n grad_attn_weight);\n break;\n case 128:\n ms_deformable_col2im_gpu_kernel_shm_blocksize_aware_reduce_v2\n <<>>(\n num_kernels, \n grad_col,\n data_value,\n data_spatial_shapes,\n data_level_start_index, \n data_sampling_loc,\n data_attn_weight,\n batch_size, \n spatial_size, \n num_heads,\n channels, \n num_levels,\n num_query,\n num_point,\n grad_value,\n grad_sampling_loc,\n grad_attn_weight);\n break;\n case 256:\n ms_deformable_col2im_gpu_kernel_shm_blocksize_aware_reduce_v2\n <<>>(\n num_kernels, \n grad_col,\n data_value,\n data_spatial_shapes,\n data_level_start_index, \n data_sampling_loc,\n data_attn_weight,\n batch_size, \n spatial_size, \n num_heads,\n channels, \n num_levels,\n num_query,\n num_point,\n grad_value,\n grad_sampling_loc,\n grad_attn_weight);\n break;\n case 512:\n ms_deformable_col2im_gpu_kernel_shm_blocksize_aware_reduce_v2\n <<>>(\n num_kernels, \n grad_col,\n data_value,\n data_spatial_shapes,\n data_level_start_index, \n data_sampling_loc,\n data_attn_weight,\n batch_size, \n spatial_size, \n num_heads,\n channels, \n num_levels,\n num_query,\n num_point,\n grad_value,\n grad_sampling_loc,\n grad_attn_weight);\n break;\n case 1024:\n ms_deformable_col2im_gpu_kernel_shm_blocksize_aware_reduce_v2\n <<>>(\n num_kernels, \n grad_col,\n data_value,\n data_spatial_shapes,\n data_level_start_index, \n data_sampling_loc,\n data_attn_weight,\n batch_size, \n spatial_size, \n num_heads,\n channels, \n num_levels,\n num_query,\n num_point,\n grad_value,\n grad_sampling_loc,\n grad_attn_weight);\n break;\n default:\n if (channels < 64)\n {\n ms_deformable_col2im_gpu_kernel_shm_reduce_v1\n <<>>(\n num_kernels, \n grad_col,\n data_value,\n data_spatial_shapes,\n data_level_start_index, \n data_sampling_loc,\n data_attn_weight,\n batch_size, \n spatial_size, \n num_heads,\n channels, \n num_levels,\n num_query,\n num_point,\n grad_value,\n grad_sampling_loc,\n grad_attn_weight);\n }\n else\n {\n ms_deformable_col2im_gpu_kernel_shm_reduce_v2\n <<>>(\n num_kernels, \n grad_col,\n data_value,\n data_spatial_shapes,\n data_level_start_index, \n data_sampling_loc,\n data_attn_weight,\n batch_size, \n spatial_size, \n num_heads,\n channels, \n num_levels,\n num_query,\n num_point,\n grad_value,\n grad_sampling_loc,\n grad_attn_weight);\n }\n }\n }\n cudaError_t err = cudaGetLastError();\n if (err != cudaSuccess)\n {\n printf(\"error in ms_deformable_col2im_cuda: %s\\n\", cudaGetErrorString(err));\n }\n\n}"
},
{
"path": "batchformer-v2/deformable-detr/models/ops/src/ms_deform_attn.h",
"content": "/*!\n**************************************************************************************************\n* Deformable DETR\n* Copyright (c) 2020 SenseTime. All Rights Reserved.\n* Licensed under the Apache License, Version 2.0 [see LICENSE for details]\n**************************************************************************************************\n* Modified from https://github.com/chengdazhi/Deformable-Convolution-V2-PyTorch/tree/pytorch_1.0.0\n**************************************************************************************************\n*/\n\n#pragma once\n\n#include \"cpu/ms_deform_attn_cpu.h\"\n\n#ifdef WITH_CUDA\n#include \"cuda/ms_deform_attn_cuda.h\"\n#endif\n\n\nat::Tensor\nms_deform_attn_forward(\n const at::Tensor &value, \n const at::Tensor &spatial_shapes,\n const at::Tensor &level_start_index,\n const at::Tensor &sampling_loc,\n const at::Tensor &attn_weight,\n const int im2col_step)\n{\n if (value.type().is_cuda())\n {\n#ifdef WITH_CUDA\n return ms_deform_attn_cuda_forward(\n value, spatial_shapes, level_start_index, sampling_loc, attn_weight, im2col_step);\n#else\n AT_ERROR(\"Not compiled with GPU support\");\n#endif\n }\n AT_ERROR(\"Not implemented on the CPU\");\n}\n\nstd::vector\nms_deform_attn_backward(\n const at::Tensor &value, \n const at::Tensor &spatial_shapes,\n const at::Tensor &level_start_index,\n const at::Tensor &sampling_loc,\n const at::Tensor &attn_weight,\n const at::Tensor &grad_output,\n const int im2col_step)\n{\n if (value.type().is_cuda())\n {\n#ifdef WITH_CUDA\n return ms_deform_attn_cuda_backward(\n value, spatial_shapes, level_start_index, sampling_loc, attn_weight, grad_output, im2col_step);\n#else\n AT_ERROR(\"Not compiled with GPU support\");\n#endif\n }\n AT_ERROR(\"Not implemented on the CPU\");\n}\n\n"
},
{
"path": "batchformer-v2/deformable-detr/models/ops/src/vision.cpp",
"content": "/*!\n**************************************************************************************************\n* Deformable DETR\n* Copyright (c) 2020 SenseTime. All Rights Reserved.\n* Licensed under the Apache License, Version 2.0 [see LICENSE for details]\n**************************************************************************************************\n* Modified from https://github.com/chengdazhi/Deformable-Convolution-V2-PyTorch/tree/pytorch_1.0.0\n**************************************************************************************************\n*/\n\n#include \"ms_deform_attn.h\"\n\nPYBIND11_MODULE(TORCH_EXTENSION_NAME, m) {\n m.def(\"ms_deform_attn_forward\", &ms_deform_attn_forward, \"ms_deform_attn_forward\");\n m.def(\"ms_deform_attn_backward\", &ms_deform_attn_backward, \"ms_deform_attn_backward\");\n}\n"
},
{
"path": "batchformer-v2/deformable-detr/models/ops/test.py",
"content": "# ------------------------------------------------------------------------------------------------\n# Deformable DETR\n# Copyright (c) 2020 SenseTime. All Rights Reserved.\n# Licensed under the Apache License, Version 2.0 [see LICENSE for details]\n# ------------------------------------------------------------------------------------------------\n# Modified from https://github.com/chengdazhi/Deformable-Convolution-V2-PyTorch/tree/pytorch_1.0.0\n# ------------------------------------------------------------------------------------------------\n\nfrom __future__ import absolute_import\nfrom __future__ import print_function\nfrom __future__ import division\n\nimport time\nimport torch\nimport torch.nn as nn\nfrom torch.autograd import gradcheck\n\nfrom functions.ms_deform_attn_func import MSDeformAttnFunction, ms_deform_attn_core_pytorch\n\n\nN, M, D = 1, 2, 2\nLq, L, P = 2, 2, 2\nshapes = torch.as_tensor([(6, 4), (3, 2)], dtype=torch.long).cuda()\nlevel_start_index = torch.cat((shapes.new_zeros((1, )), shapes.prod(1).cumsum(0)[:-1]))\nS = sum([(H*W).item() for H, W in shapes])\n\n\ntorch.manual_seed(3)\n\n\n@torch.no_grad()\ndef check_forward_equal_with_pytorch_double():\n value = torch.rand(N, S, M, D).cuda() * 0.01\n sampling_locations = torch.rand(N, Lq, M, L, P, 2).cuda()\n attention_weights = torch.rand(N, Lq, M, L, P).cuda() + 1e-5\n attention_weights /= attention_weights.sum(-1, keepdim=True).sum(-2, keepdim=True)\n im2col_step = 2\n output_pytorch = ms_deform_attn_core_pytorch(value.double(), shapes, sampling_locations.double(), attention_weights.double()).detach().cpu()\n output_cuda = MSDeformAttnFunction.apply(value.double(), shapes, level_start_index, sampling_locations.double(), attention_weights.double(), im2col_step).detach().cpu()\n fwdok = torch.allclose(output_cuda, output_pytorch)\n max_abs_err = (output_cuda - output_pytorch).abs().max()\n max_rel_err = ((output_cuda - output_pytorch).abs() / output_pytorch.abs()).max()\n\n print(f'* {fwdok} check_forward_equal_with_pytorch_double: max_abs_err {max_abs_err:.2e} max_rel_err {max_rel_err:.2e}')\n\n\n@torch.no_grad()\ndef check_forward_equal_with_pytorch_float():\n value = torch.rand(N, S, M, D).cuda() * 0.01\n sampling_locations = torch.rand(N, Lq, M, L, P, 2).cuda()\n attention_weights = torch.rand(N, Lq, M, L, P).cuda() + 1e-5\n attention_weights /= attention_weights.sum(-1, keepdim=True).sum(-2, keepdim=True)\n im2col_step = 2\n output_pytorch = ms_deform_attn_core_pytorch(value, shapes, sampling_locations, attention_weights).detach().cpu()\n output_cuda = MSDeformAttnFunction.apply(value, shapes, level_start_index, sampling_locations, attention_weights, im2col_step).detach().cpu()\n fwdok = torch.allclose(output_cuda, output_pytorch, rtol=1e-2, atol=1e-3)\n max_abs_err = (output_cuda - output_pytorch).abs().max()\n max_rel_err = ((output_cuda - output_pytorch).abs() / output_pytorch.abs()).max()\n\n print(f'* {fwdok} check_forward_equal_with_pytorch_float: max_abs_err {max_abs_err:.2e} max_rel_err {max_rel_err:.2e}')\n\n\ndef check_gradient_numerical(channels=4, grad_value=True, grad_sampling_loc=True, grad_attn_weight=True):\n\n value = torch.rand(N, S, M, channels).cuda() * 0.01\n sampling_locations = torch.rand(N, Lq, M, L, P, 2).cuda()\n attention_weights = torch.rand(N, Lq, M, L, P).cuda() + 1e-5\n attention_weights /= attention_weights.sum(-1, keepdim=True).sum(-2, keepdim=True)\n im2col_step = 2\n func = MSDeformAttnFunction.apply\n\n value.requires_grad = grad_value\n sampling_locations.requires_grad = grad_sampling_loc\n attention_weights.requires_grad = grad_attn_weight\n\n gradok = gradcheck(func, (value.double(), shapes, level_start_index, sampling_locations.double(), attention_weights.double(), im2col_step))\n\n print(f'* {gradok} check_gradient_numerical(D={channels})')\n\n\nif __name__ == '__main__':\n check_forward_equal_with_pytorch_double()\n check_forward_equal_with_pytorch_float()\n\n for channels in [30, 32, 64, 71, 1025, 2048, 3096]:\n check_gradient_numerical(channels, True, True, True)\n\n\n\n"
},
{
"path": "batchformer-v2/deformable-detr/models/position_encoding.py",
"content": "# ------------------------------------------------------------------------\n# Deformable DETR\n# Copyright (c) 2020 SenseTime. All Rights Reserved.\n# Licensed under the Apache License, Version 2.0 [see LICENSE for details]\n# ------------------------------------------------------------------------\n# Modified from DETR (https://github.com/facebookresearch/detr)\n# Copyright (c) Facebook, Inc. and its affiliates. All Rights Reserved\n# ------------------------------------------------------------------------\n\n\"\"\"\nVarious positional encodings for the transformer.\n\"\"\"\nimport math\nimport torch\nfrom torch import nn\n\nfrom util.misc import NestedTensor\n\n\nclass PositionEmbeddingSine(nn.Module):\n \"\"\"\n This is a more standard version of the position embedding, very similar to the one\n used by the Attention is all you need paper, generalized to work on images.\n \"\"\"\n def __init__(self, num_pos_feats=64, temperature=10000, normalize=False, scale=None):\n super().__init__()\n self.num_pos_feats = num_pos_feats\n self.temperature = temperature\n self.normalize = normalize\n if scale is not None and normalize is False:\n raise ValueError(\"normalize should be True if scale is passed\")\n if scale is None:\n scale = 2 * math.pi\n self.scale = scale\n\n def forward(self, tensor_list: NestedTensor):\n x = tensor_list.tensors\n mask = tensor_list.mask\n assert mask is not None\n not_mask = ~mask\n y_embed = not_mask.cumsum(1, dtype=torch.float32)\n x_embed = not_mask.cumsum(2, dtype=torch.float32)\n if self.normalize:\n eps = 1e-6\n y_embed = (y_embed - 0.5) / (y_embed[:, -1:, :] + eps) * self.scale\n x_embed = (x_embed - 0.5) / (x_embed[:, :, -1:] + eps) * self.scale\n\n dim_t = torch.arange(self.num_pos_feats, dtype=torch.float32, device=x.device)\n dim_t = self.temperature ** (2 * (dim_t // 2) / self.num_pos_feats)\n\n pos_x = x_embed[:, :, :, None] / dim_t\n pos_y = y_embed[:, :, :, None] / dim_t\n pos_x = torch.stack((pos_x[:, :, :, 0::2].sin(), pos_x[:, :, :, 1::2].cos()), dim=4).flatten(3)\n pos_y = torch.stack((pos_y[:, :, :, 0::2].sin(), pos_y[:, :, :, 1::2].cos()), dim=4).flatten(3)\n pos = torch.cat((pos_y, pos_x), dim=3).permute(0, 3, 1, 2)\n return pos\n\n\nclass PositionEmbeddingLearned(nn.Module):\n \"\"\"\n Absolute pos embedding, learned.\n \"\"\"\n def __init__(self, num_pos_feats=256):\n super().__init__()\n self.row_embed = nn.Embedding(50, num_pos_feats)\n self.col_embed = nn.Embedding(50, num_pos_feats)\n self.reset_parameters()\n\n def reset_parameters(self):\n nn.init.uniform_(self.row_embed.weight)\n nn.init.uniform_(self.col_embed.weight)\n\n def forward(self, tensor_list: NestedTensor):\n x = tensor_list.tensors\n h, w = x.shape[-2:]\n i = torch.arange(w, device=x.device)\n j = torch.arange(h, device=x.device)\n x_emb = self.col_embed(i)\n y_emb = self.row_embed(j)\n pos = torch.cat([\n x_emb.unsqueeze(0).repeat(h, 1, 1),\n y_emb.unsqueeze(1).repeat(1, w, 1),\n ], dim=-1).permute(2, 0, 1).unsqueeze(0).repeat(x.shape[0], 1, 1, 1)\n return pos\n\n\ndef build_position_encoding(args):\n N_steps = args.hidden_dim // 2\n if args.position_embedding in ('v2', 'sine'):\n # TODO find a better way of exposing other arguments\n position_embedding = PositionEmbeddingSine(N_steps, normalize=True)\n elif args.position_embedding in ('v3', 'learned'):\n position_embedding = PositionEmbeddingLearned(N_steps)\n else:\n raise ValueError(f\"not supported {args.position_embedding}\")\n\n return position_embedding\n"
},
{
"path": "batchformer-v2/deformable-detr/models/segmentation.py",
"content": "# ------------------------------------------------------------------------\n# Deformable DETR\n# Copyright (c) 2020 SenseTime. All Rights Reserved.\n# Licensed under the Apache License, Version 2.0 [see LICENSE for details]\n# ------------------------------------------------------------------------\n# Modified from DETR (https://github.com/facebookresearch/detr)\n# Copyright (c) Facebook, Inc. and its affiliates. All Rights Reserved\n# ------------------------------------------------------------------------\n\n\"\"\"\nThis file provides the definition of the convolutional heads used to predict masks, as well as the losses\n\"\"\"\nimport io\nfrom collections import defaultdict\n\nimport torch\nimport torch.nn as nn\nimport torch.nn.functional as F\nfrom PIL import Image\n\nimport util.box_ops as box_ops\nfrom util.misc import NestedTensor, interpolate, nested_tensor_from_tensor_list\n\ntry:\n from panopticapi.utils import id2rgb, rgb2id\nexcept ImportError:\n pass\n\n\nclass DETRsegm(nn.Module):\n def __init__(self, detr, freeze_detr=False):\n super().__init__()\n self.detr = detr\n\n if freeze_detr:\n for p in self.parameters():\n p.requires_grad_(False)\n\n hidden_dim, nheads = detr.transformer.d_model, detr.transformer.nhead\n self.bbox_attention = MHAttentionMap(hidden_dim, hidden_dim, nheads, dropout=0)\n self.mask_head = MaskHeadSmallConv(hidden_dim + nheads, [1024, 512, 256], hidden_dim)\n\n def forward(self, samples: NestedTensor):\n if not isinstance(samples, NestedTensor):\n samples = nested_tensor_from_tensor_list(samples)\n features, pos = self.detr.backbone(samples)\n\n bs = features[-1].tensors.shape[0]\n\n src, mask = features[-1].decompose()\n src_proj = self.detr.input_proj(src)\n hs, memory = self.detr.transformer(src_proj, mask, self.detr.query_embed.weight, pos[-1])\n\n outputs_class = self.detr.class_embed(hs)\n outputs_coord = self.detr.bbox_embed(hs).sigmoid()\n out = {\"pred_logits\": outputs_class[-1], \"pred_boxes\": outputs_coord[-1]}\n if self.detr.aux_loss:\n out[\"aux_outputs\"] = [\n {\"pred_logits\": a, \"pred_boxes\": b} for a, b in zip(outputs_class[:-1], outputs_coord[:-1])\n ]\n\n # FIXME h_boxes takes the last one computed, keep this in mind\n bbox_mask = self.bbox_attention(hs[-1], memory, mask=mask)\n\n seg_masks = self.mask_head(src_proj, bbox_mask, [features[2].tensors, features[1].tensors, features[0].tensors])\n outputs_seg_masks = seg_masks.view(bs, self.detr.num_queries, seg_masks.shape[-2], seg_masks.shape[-1])\n\n out[\"pred_masks\"] = outputs_seg_masks\n return out\n\n\nclass MaskHeadSmallConv(nn.Module):\n \"\"\"\n Simple convolutional head, using group norm.\n Upsampling is done using a FPN approach\n \"\"\"\n\n def __init__(self, dim, fpn_dims, context_dim):\n super().__init__()\n\n inter_dims = [dim, context_dim // 2, context_dim // 4, context_dim // 8, context_dim // 16, context_dim // 64]\n self.lay1 = torch.nn.Conv2d(dim, dim, 3, padding=1)\n self.gn1 = torch.nn.GroupNorm(8, dim)\n self.lay2 = torch.nn.Conv2d(dim, inter_dims[1], 3, padding=1)\n self.gn2 = torch.nn.GroupNorm(8, inter_dims[1])\n self.lay3 = torch.nn.Conv2d(inter_dims[1], inter_dims[2], 3, padding=1)\n self.gn3 = torch.nn.GroupNorm(8, inter_dims[2])\n self.lay4 = torch.nn.Conv2d(inter_dims[2], inter_dims[3], 3, padding=1)\n self.gn4 = torch.nn.GroupNorm(8, inter_dims[3])\n self.lay5 = torch.nn.Conv2d(inter_dims[3], inter_dims[4], 3, padding=1)\n self.gn5 = torch.nn.GroupNorm(8, inter_dims[4])\n self.out_lay = torch.nn.Conv2d(inter_dims[4], 1, 3, padding=1)\n\n self.dim = dim\n\n self.adapter1 = torch.nn.Conv2d(fpn_dims[0], inter_dims[1], 1)\n self.adapter2 = torch.nn.Conv2d(fpn_dims[1], inter_dims[2], 1)\n self.adapter3 = torch.nn.Conv2d(fpn_dims[2], inter_dims[3], 1)\n\n for m in self.modules():\n if isinstance(m, nn.Conv2d):\n nn.init.kaiming_uniform_(m.weight, a=1)\n nn.init.constant_(m.bias, 0)\n\n def forward(self, x, bbox_mask, fpns):\n def expand(tensor, length):\n return tensor.unsqueeze(1).repeat(1, int(length), 1, 1, 1).flatten(0, 1)\n\n x = torch.cat([expand(x, bbox_mask.shape[1]), bbox_mask.flatten(0, 1)], 1)\n\n x = self.lay1(x)\n x = self.gn1(x)\n x = F.relu(x)\n x = self.lay2(x)\n x = self.gn2(x)\n x = F.relu(x)\n\n cur_fpn = self.adapter1(fpns[0])\n if cur_fpn.size(0) != x.size(0):\n cur_fpn = expand(cur_fpn, x.size(0) / cur_fpn.size(0))\n x = cur_fpn + F.interpolate(x, size=cur_fpn.shape[-2:], mode=\"nearest\")\n x = self.lay3(x)\n x = self.gn3(x)\n x = F.relu(x)\n\n cur_fpn = self.adapter2(fpns[1])\n if cur_fpn.size(0) != x.size(0):\n cur_fpn = expand(cur_fpn, x.size(0) / cur_fpn.size(0))\n x = cur_fpn + F.interpolate(x, size=cur_fpn.shape[-2:], mode=\"nearest\")\n x = self.lay4(x)\n x = self.gn4(x)\n x = F.relu(x)\n\n cur_fpn = self.adapter3(fpns[2])\n if cur_fpn.size(0) != x.size(0):\n cur_fpn = expand(cur_fpn, x.size(0) / cur_fpn.size(0))\n x = cur_fpn + F.interpolate(x, size=cur_fpn.shape[-2:], mode=\"nearest\")\n x = self.lay5(x)\n x = self.gn5(x)\n x = F.relu(x)\n\n x = self.out_lay(x)\n return x\n\n\nclass MHAttentionMap(nn.Module):\n \"\"\"This is a 2D attention module, which only returns the attention softmax (no multiplication by value)\"\"\"\n\n def __init__(self, query_dim, hidden_dim, num_heads, dropout=0, bias=True):\n super().__init__()\n self.num_heads = num_heads\n self.hidden_dim = hidden_dim\n self.dropout = nn.Dropout(dropout)\n\n self.q_linear = nn.Linear(query_dim, hidden_dim, bias=bias)\n self.k_linear = nn.Linear(query_dim, hidden_dim, bias=bias)\n\n nn.init.zeros_(self.k_linear.bias)\n nn.init.zeros_(self.q_linear.bias)\n nn.init.xavier_uniform_(self.k_linear.weight)\n nn.init.xavier_uniform_(self.q_linear.weight)\n self.normalize_fact = float(hidden_dim / self.num_heads) ** -0.5\n\n def forward(self, q, k, mask=None):\n q = self.q_linear(q)\n k = F.conv2d(k, self.k_linear.weight.unsqueeze(-1).unsqueeze(-1), self.k_linear.bias)\n qh = q.view(q.shape[0], q.shape[1], self.num_heads, self.hidden_dim // self.num_heads)\n kh = k.view(k.shape[0], self.num_heads, self.hidden_dim // self.num_heads, k.shape[-2], k.shape[-1])\n weights = torch.einsum(\"bqnc,bnchw->bqnhw\", qh * self.normalize_fact, kh)\n\n if mask is not None:\n weights.masked_fill_(mask.unsqueeze(1).unsqueeze(1), float(\"-inf\"))\n weights = F.softmax(weights.flatten(2), dim=-1).view_as(weights)\n weights = self.dropout(weights)\n return weights\n\n\ndef dice_loss(inputs, targets, num_boxes):\n \"\"\"\n Compute the DICE loss, similar to generalized IOU for masks\n Args:\n inputs: A float tensor of arbitrary shape.\n The predictions for each example.\n targets: A float tensor with the same shape as inputs. Stores the binary\n classification label for each element in inputs\n (0 for the negative class and 1 for the positive class).\n \"\"\"\n inputs = inputs.sigmoid()\n inputs = inputs.flatten(1)\n numerator = 2 * (inputs * targets).sum(1)\n denominator = inputs.sum(-1) + targets.sum(-1)\n loss = 1 - (numerator + 1) / (denominator + 1)\n return loss.sum() / num_boxes\n\n\ndef sigmoid_focal_loss(inputs, targets, num_boxes, alpha: float = 0.25, gamma: float = 2):\n \"\"\"\n Loss used in RetinaNet for dense detection: https://arxiv.org/abs/1708.02002.\n Args:\n inputs: A float tensor of arbitrary shape.\n The predictions for each example.\n targets: A float tensor with the same shape as inputs. Stores the binary\n classification label for each element in inputs\n (0 for the negative class and 1 for the positive class).\n alpha: (optional) Weighting factor in range (0,1) to balance\n positive vs negative examples. Default = -1 (no weighting).\n gamma: Exponent of the modulating factor (1 - p_t) to\n balance easy vs hard examples.\n Returns:\n Loss tensor\n \"\"\"\n prob = inputs.sigmoid()\n ce_loss = F.binary_cross_entropy_with_logits(inputs, targets, reduction=\"none\")\n p_t = prob * targets + (1 - prob) * (1 - targets)\n loss = ce_loss * ((1 - p_t) ** gamma)\n\n if alpha >= 0:\n alpha_t = alpha * targets + (1 - alpha) * (1 - targets)\n loss = alpha_t * loss\n\n return loss.mean(1).sum() / num_boxes\n\n\nclass PostProcessSegm(nn.Module):\n def __init__(self, threshold=0.5):\n super().__init__()\n self.threshold = threshold\n\n @torch.no_grad()\n def forward(self, results, outputs, orig_target_sizes, max_target_sizes):\n assert len(orig_target_sizes) == len(max_target_sizes)\n max_h, max_w = max_target_sizes.max(0)[0].tolist()\n outputs_masks = outputs[\"pred_masks\"].squeeze(2)\n outputs_masks = F.interpolate(outputs_masks, size=(max_h, max_w), mode=\"bilinear\", align_corners=False)\n outputs_masks = (outputs_masks.sigmoid() > self.threshold).cpu()\n\n for i, (cur_mask, t, tt) in enumerate(zip(outputs_masks, max_target_sizes, orig_target_sizes)):\n img_h, img_w = t[0], t[1]\n results[i][\"masks\"] = cur_mask[:, :img_h, :img_w].unsqueeze(1)\n results[i][\"masks\"] = F.interpolate(\n results[i][\"masks\"].float(), size=tuple(tt.tolist()), mode=\"nearest\"\n ).byte()\n\n return results\n\n\nclass PostProcessPanoptic(nn.Module):\n \"\"\"This class converts the output of the model to the final panoptic result, in the format expected by the\n coco panoptic API \"\"\"\n\n def __init__(self, is_thing_map, threshold=0.85):\n \"\"\"\n Parameters:\n is_thing_map: This is a whose keys are the class ids, and the values a boolean indicating whether\n the class is a thing (True) or a stuff (False) class\n threshold: confidence threshold: segments with confidence lower than this will be deleted\n \"\"\"\n super().__init__()\n self.threshold = threshold\n self.is_thing_map = is_thing_map\n\n def forward(self, outputs, processed_sizes, target_sizes=None):\n \"\"\" This function computes the panoptic prediction from the model's predictions.\n Parameters:\n outputs: This is a dict coming directly from the model. See the model doc for the content.\n processed_sizes: This is a list of tuples (or torch tensors) of sizes of the images that were passed to the\n model, ie the size after data augmentation but before batching.\n target_sizes: This is a list of tuples (or torch tensors) corresponding to the requested final size\n of each prediction. If left to None, it will default to the processed_sizes\n \"\"\"\n if target_sizes is None:\n target_sizes = processed_sizes\n assert len(processed_sizes) == len(target_sizes)\n out_logits, raw_masks, raw_boxes = outputs[\"pred_logits\"], outputs[\"pred_masks\"], outputs[\"pred_boxes\"]\n assert len(out_logits) == len(raw_masks) == len(target_sizes)\n preds = []\n\n def to_tuple(tup):\n if isinstance(tup, tuple):\n return tup\n return tuple(tup.cpu().tolist())\n\n for cur_logits, cur_masks, cur_boxes, size, target_size in zip(\n out_logits, raw_masks, raw_boxes, processed_sizes, target_sizes\n ):\n # we filter empty queries and detection below threshold\n scores, labels = cur_logits.softmax(-1).max(-1)\n keep = labels.ne(outputs[\"pred_logits\"].shape[-1] - 1) & (scores > self.threshold)\n cur_scores, cur_classes = cur_logits.softmax(-1).max(-1)\n cur_scores = cur_scores[keep]\n cur_classes = cur_classes[keep]\n cur_masks = cur_masks[keep]\n cur_masks = interpolate(cur_masks[None], to_tuple(size), mode=\"bilinear\").squeeze(0)\n cur_boxes = box_ops.box_cxcywh_to_xyxy(cur_boxes[keep])\n\n h, w = cur_masks.shape[-2:]\n assert len(cur_boxes) == len(cur_classes)\n\n # It may be that we have several predicted masks for the same stuff class.\n # In the following, we track the list of masks ids for each stuff class (they are merged later on)\n cur_masks = cur_masks.flatten(1)\n stuff_equiv_classes = defaultdict(lambda: [])\n for k, label in enumerate(cur_classes):\n if not self.is_thing_map[label.item()]:\n stuff_equiv_classes[label.item()].append(k)\n\n def get_ids_area(masks, scores, dedup=False):\n # This helper function creates the final panoptic segmentation image\n # It also returns the area of the masks that appears on the image\n\n m_id = masks.transpose(0, 1).softmax(-1)\n\n if m_id.shape[-1] == 0:\n # We didn't detect any mask :(\n m_id = torch.zeros((h, w), dtype=torch.long, device=m_id.device)\n else:\n m_id = m_id.argmax(-1).view(h, w)\n\n if dedup:\n # Merge the masks corresponding to the same stuff class\n for equiv in stuff_equiv_classes.values():\n if len(equiv) > 1:\n for eq_id in equiv:\n m_id.masked_fill_(m_id.eq(eq_id), equiv[0])\n\n final_h, final_w = to_tuple(target_size)\n\n seg_img = Image.fromarray(id2rgb(m_id.view(h, w).cpu().numpy()))\n seg_img = seg_img.resize(size=(final_w, final_h), resample=Image.NEAREST)\n\n np_seg_img = (\n torch.ByteTensor(torch.ByteStorage.from_buffer(seg_img.tobytes())).view(final_h, final_w, 3).numpy()\n )\n m_id = torch.from_numpy(rgb2id(np_seg_img))\n\n area = []\n for i in range(len(scores)):\n area.append(m_id.eq(i).sum().item())\n return area, seg_img\n\n area, seg_img = get_ids_area(cur_masks, cur_scores, dedup=True)\n if cur_classes.numel() > 0:\n # We know filter empty masks as long as we find some\n while True:\n filtered_small = torch.as_tensor(\n [area[i] <= 4 for i, c in enumerate(cur_classes)], dtype=torch.bool, device=keep.device\n )\n if filtered_small.any().item():\n cur_scores = cur_scores[~filtered_small]\n cur_classes = cur_classes[~filtered_small]\n cur_masks = cur_masks[~filtered_small]\n area, seg_img = get_ids_area(cur_masks, cur_scores)\n else:\n break\n\n else:\n cur_classes = torch.ones(1, dtype=torch.long, device=cur_classes.device)\n\n segments_info = []\n for i, a in enumerate(area):\n cat = cur_classes[i].item()\n segments_info.append({\"id\": i, \"isthing\": self.is_thing_map[cat], \"category_id\": cat, \"area\": a})\n del cur_classes\n\n with io.BytesIO() as out:\n seg_img.save(out, format=\"PNG\")\n predictions = {\"png_string\": out.getvalue(), \"segments_info\": segments_info}\n preds.append(predictions)\n return preds\n"
},
{
"path": "batchformer-v2/deformable-detr/requirements.txt",
"content": "pycocotools\ntqdm\ncython\nscipy\n"
},
{
"path": "batchformer-v2/deformable-detr/tools/launch.py",
"content": "# --------------------------------------------------------------------------------------------------------------------------\n# Deformable DETR\n# Copyright (c) 2020 SenseTime. All Rights Reserved.\n# Licensed under the Apache License, Version 2.0 [see LICENSE for details]\n# --------------------------------------------------------------------------------------------------------------------------\n# Modified from https://github.com/pytorch/pytorch/blob/173f224570017b4b1a3a1a13d0bff280a54d9cd9/torch/distributed/launch.py\n# --------------------------------------------------------------------------------------------------------------------------\n\nr\"\"\"\n`torch.distributed.launch` is a module that spawns up multiple distributed\ntraining processes on each of the training nodes.\nThe utility can be used for single-node distributed training, in which one or\nmore processes per node will be spawned. The utility can be used for either\nCPU training or GPU training. If the utility is used for GPU training,\neach distributed process will be operating on a single GPU. This can achieve\nwell-improved single-node training performance. It can also be used in\nmulti-node distributed training, by spawning up multiple processes on each node\nfor well-improved multi-node distributed training performance as well.\nThis will especially be benefitial for systems with multiple Infiniband\ninterfaces that have direct-GPU support, since all of them can be utilized for\naggregated communication bandwidth.\nIn both cases of single-node distributed training or multi-node distributed\ntraining, this utility will launch the given number of processes per node\n(``--nproc_per_node``). If used for GPU training, this number needs to be less\nor euqal to the number of GPUs on the current system (``nproc_per_node``),\nand each process will be operating on a single GPU from *GPU 0 to\nGPU (nproc_per_node - 1)*.\n**How to use this module:**\n1. Single-Node multi-process distributed training\n::\n >>> python -m torch.distributed.launch --nproc_per_node=NUM_GPUS_YOU_HAVE\n YOUR_TRAINING_SCRIPT.py (--arg1 --arg2 --arg3 and all other\n arguments of your training script)\n2. Multi-Node multi-process distributed training: (e.g. two nodes)\nNode 1: *(IP: 192.168.1.1, and has a free port: 1234)*\n::\n >>> python -m torch.distributed.launch --nproc_per_node=NUM_GPUS_YOU_HAVE\n --nnodes=2 --node_rank=0 --master_addr=\"192.168.1.1\"\n --master_port=1234 YOUR_TRAINING_SCRIPT.py (--arg1 --arg2 --arg3\n and all other arguments of your training script)\nNode 2:\n::\n >>> python -m torch.distributed.launch --nproc_per_node=NUM_GPUS_YOU_HAVE\n --nnodes=2 --node_rank=1 --master_addr=\"192.168.1.1\"\n --master_port=1234 YOUR_TRAINING_SCRIPT.py (--arg1 --arg2 --arg3\n and all other arguments of your training script)\n3. To look up what optional arguments this module offers:\n::\n >>> python -m torch.distributed.launch --help\n**Important Notices:**\n1. This utilty and multi-process distributed (single-node or\nmulti-node) GPU training currently only achieves the best performance using\nthe NCCL distributed backend. Thus NCCL backend is the recommended backend to\nuse for GPU training.\n2. In your training program, you must parse the command-line argument:\n``--local_rank=LOCAL_PROCESS_RANK``, which will be provided by this module.\nIf your training program uses GPUs, you should ensure that your code only\nruns on the GPU device of LOCAL_PROCESS_RANK. This can be done by:\nParsing the local_rank argument\n::\n >>> import argparse\n >>> parser = argparse.ArgumentParser()\n >>> parser.add_argument(\"--local_rank\", type=int)\n >>> args = parser.parse_args()\nSet your device to local rank using either\n::\n >>> torch.cuda.set_device(arg.local_rank) # before your code runs\nor\n::\n >>> with torch.cuda.device(arg.local_rank):\n >>> # your code to run\n3. In your training program, you are supposed to call the following function\nat the beginning to start the distributed backend. You need to make sure that\nthe init_method uses ``env://``, which is the only supported ``init_method``\nby this module.\n::\n torch.distributed.init_process_group(backend='YOUR BACKEND',\n init_method='env://')\n4. In your training program, you can either use regular distributed functions\nor use :func:`torch.nn.parallel.DistributedDataParallel` module. If your\ntraining program uses GPUs for training and you would like to use\n:func:`torch.nn.parallel.DistributedDataParallel` module,\nhere is how to configure it.\n::\n model = torch.nn.parallel.DistributedDataParallel(model,\n device_ids=[arg.local_rank],\n output_device=arg.local_rank)\nPlease ensure that ``device_ids`` argument is set to be the only GPU device id\nthat your code will be operating on. This is generally the local rank of the\nprocess. In other words, the ``device_ids`` needs to be ``[args.local_rank]``,\nand ``output_device`` needs to be ``args.local_rank`` in order to use this\nutility\n5. Another way to pass ``local_rank`` to the subprocesses via environment variable\n``LOCAL_RANK``. This behavior is enabled when you launch the script with\n``--use_env=True``. You must adjust the subprocess example above to replace\n``args.local_rank`` with ``os.environ['LOCAL_RANK']``; the launcher\nwill not pass ``--local_rank`` when you specify this flag.\n.. warning::\n ``local_rank`` is NOT globally unique: it is only unique per process\n on a machine. Thus, don't use it to decide if you should, e.g.,\n write to a networked filesystem. See\n https://github.com/pytorch/pytorch/issues/12042 for an example of\n how things can go wrong if you don't do this correctly.\n\"\"\"\n\n\nimport sys\nimport subprocess\nimport os\nimport socket\nfrom argparse import ArgumentParser, REMAINDER\n\nimport torch\n\n\ndef parse_args():\n \"\"\"\n Helper function parsing the command line options\n @retval ArgumentParser\n \"\"\"\n parser = ArgumentParser(description=\"PyTorch distributed training launch \"\n \"helper utilty that will spawn up \"\n \"multiple distributed processes\")\n\n # Optional arguments for the launch helper\n parser.add_argument(\"--nnodes\", type=int, default=1,\n help=\"The number of nodes to use for distributed \"\n \"training\")\n parser.add_argument(\"--node_rank\", type=int, default=0,\n help=\"The rank of the node for multi-node distributed \"\n \"training\")\n parser.add_argument(\"--nproc_per_node\", type=int, default=1,\n help=\"The number of processes to launch on each node, \"\n \"for GPU training, this is recommended to be set \"\n \"to the number of GPUs in your system so that \"\n \"each process can be bound to a single GPU.\")\n parser.add_argument(\"--master_addr\", default=\"127.0.0.1\", type=str,\n help=\"Master node (rank 0)'s address, should be either \"\n \"the IP address or the hostname of node 0, for \"\n \"single node multi-proc training, the \"\n \"--master_addr can simply be 127.0.0.1\")\n parser.add_argument(\"--master_port\", default=29500, type=int,\n help=\"Master node (rank 0)'s free port that needs to \"\n \"be used for communciation during distributed \"\n \"training\")\n\n # positional\n parser.add_argument(\"training_script\", type=str,\n help=\"The full path to the single GPU training \"\n \"program/script to be launched in parallel, \"\n \"followed by all the arguments for the \"\n \"training script\")\n\n # rest from the training program\n parser.add_argument('training_script_args', nargs=REMAINDER)\n return parser.parse_args()\n\n\ndef main():\n args = parse_args()\n\n # world size in terms of number of processes\n dist_world_size = args.nproc_per_node * args.nnodes\n\n # set PyTorch distributed related environmental variables\n current_env = os.environ.copy()\n current_env[\"MASTER_ADDR\"] = args.master_addr\n current_env[\"MASTER_PORT\"] = str(args.master_port)\n current_env[\"WORLD_SIZE\"] = str(dist_world_size)\n\n processes = []\n\n for local_rank in range(0, args.nproc_per_node):\n # each process's rank\n dist_rank = args.nproc_per_node * args.node_rank + local_rank\n current_env[\"RANK\"] = str(dist_rank)\n current_env[\"LOCAL_RANK\"] = str(local_rank)\n\n cmd = [args.training_script] + args.training_script_args\n\n process = subprocess.Popen(cmd, env=current_env)\n processes.append(process)\n\n for process in processes:\n process.wait()\n if process.returncode != 0:\n raise subprocess.CalledProcessError(returncode=process.returncode,\n cmd=process.args)\n\n\nif __name__ == \"__main__\":\n main()"
},
{
"path": "batchformer-v2/deformable-detr/tools/run_dist_launch.sh",
"content": "#!/usr/bin/env bash\n# ------------------------------------------------------------------------\n# Deformable DETR\n# Copyright (c) 2020 SenseTime. All Rights Reserved.\n# Licensed under the Apache License, Version 2.0 [see LICENSE for details]\n# ------------------------------------------------------------------------\n\nset -x\n\nGPUS=$1\nRUN_COMMAND=${@:2}\nif [ $GPUS -lt 8 ]; then\n GPUS_PER_NODE=${GPUS_PER_NODE:-$GPUS}\nelse\n GPUS_PER_NODE=${GPUS_PER_NODE:-8}\nfi\nMASTER_ADDR=${MASTER_ADDR:-\"127.0.0.1\"}\nMASTER_PORT=${MASTER_PORT:-\"29500\"}\nNODE_RANK=${NODE_RANK:-0}\n\nlet \"NNODES=GPUS/GPUS_PER_NODE\"\n\npython ./tools/launch.py \\\n --nnodes ${NNODES} \\\n --node_rank ${NODE_RANK} \\\n --master_addr ${MASTER_ADDR} \\\n --master_port ${MASTER_PORT} \\\n --nproc_per_node ${GPUS_PER_NODE} \\\n ${RUN_COMMAND}"
},
{
"path": "batchformer-v2/deformable-detr/tools/run_dist_slurm.sh",
"content": "#!/usr/bin/env bash\n# --------------------------------------------------------------------------------------------------------------------------\n# Deformable DETR\n# Copyright (c) 2020 SenseTime. All Rights Reserved.\n# Licensed under the Apache License, Version 2.0 [see LICENSE for details]\n# --------------------------------------------------------------------------------------------------------------------------\n# Modified from https://github.com/open-mmlab/mmdetection/blob/3b53fe15d87860c6941f3dda63c0f27422da6266/tools/slurm_train.sh\n# --------------------------------------------------------------------------------------------------------------------------\n\nset -x\n\nPARTITION=$1\nJOB_NAME=$2\nGPUS=$3\nRUN_COMMAND=${@:4}\nif [ $GPUS -lt 8 ]; then\n GPUS_PER_NODE=${GPUS_PER_NODE:-$GPUS}\nelse\n GPUS_PER_NODE=${GPUS_PER_NODE:-8}\nfi\nCPUS_PER_TASK=${CPUS_PER_TASK:-4}\nSRUN_ARGS=${SRUN_ARGS:-\"\"}\n\nsrun -p ${PARTITION} \\\n --job-name=${JOB_NAME} \\\n --gres=gpu:${GPUS_PER_NODE} \\\n --ntasks=${GPUS} \\\n --ntasks-per-node=${GPUS_PER_NODE} \\\n --cpus-per-task=${CPUS_PER_TASK} \\\n --kill-on-bad-exit=1 \\\n ${SRUN_ARGS} \\\n ${RUN_COMMAND}\n\n"
},
{
"path": "batchformer-v2/deformable-detr/util/__init__.py",
"content": "# ------------------------------------------------------------------------\n# Deformable DETR\n# Copyright (c) 2020 SenseTime. All Rights Reserved.\n# Licensed under the Apache License, Version 2.0 [see LICENSE for details]\n# ------------------------------------------------------------------------\n# Modified from DETR (https://github.com/facebookresearch/detr)\n# Copyright (c) Facebook, Inc. and its affiliates. All Rights Reserved\n# ------------------------------------------------------------------------\n"
},
{
"path": "batchformer-v2/deformable-detr/util/box_ops.py",
"content": "# ------------------------------------------------------------------------\n# Deformable DETR\n# Copyright (c) 2020 SenseTime. All Rights Reserved.\n# Licensed under the Apache License, Version 2.0 [see LICENSE for details]\n# ------------------------------------------------------------------------\n# Modified from DETR (https://github.com/facebookresearch/detr)\n# Copyright (c) Facebook, Inc. and its affiliates. All Rights Reserved\n# ------------------------------------------------------------------------\n\n\"\"\"\nUtilities for bounding box manipulation and GIoU.\n\"\"\"\nimport torch\nfrom torchvision.ops.boxes import box_area\n\n\ndef box_cxcywh_to_xyxy(x):\n x_c, y_c, w, h = x.unbind(-1)\n b = [(x_c - 0.5 * w), (y_c - 0.5 * h),\n (x_c + 0.5 * w), (y_c + 0.5 * h)]\n return torch.stack(b, dim=-1)\n\n\ndef box_xyxy_to_cxcywh(x):\n x0, y0, x1, y1 = x.unbind(-1)\n b = [(x0 + x1) / 2, (y0 + y1) / 2,\n (x1 - x0), (y1 - y0)]\n return torch.stack(b, dim=-1)\n\n\n# modified from torchvision to also return the union\ndef box_iou(boxes1, boxes2):\n area1 = box_area(boxes1)\n area2 = box_area(boxes2)\n\n lt = torch.max(boxes1[:, None, :2], boxes2[:, :2]) # [N,M,2]\n rb = torch.min(boxes1[:, None, 2:], boxes2[:, 2:]) # [N,M,2]\n\n wh = (rb - lt).clamp(min=0) # [N,M,2]\n inter = wh[:, :, 0] * wh[:, :, 1] # [N,M]\n\n union = area1[:, None] + area2 - inter\n\n iou = inter / union\n return iou, union\n\n\ndef generalized_box_iou(boxes1, boxes2):\n \"\"\"\n Generalized IoU from https://giou.stanford.edu/\n\n The boxes should be in [x0, y0, x1, y1] format\n\n Returns a [N, M] pairwise matrix, where N = len(boxes1)\n and M = len(boxes2)\n \"\"\"\n # degenerate boxes gives inf / nan results\n # so do an early check\n assert (boxes1[:, 2:] >= boxes1[:, :2]).all()\n assert (boxes2[:, 2:] >= boxes2[:, :2]).all()\n iou, union = box_iou(boxes1, boxes2)\n\n lt = torch.min(boxes1[:, None, :2], boxes2[:, :2])\n rb = torch.max(boxes1[:, None, 2:], boxes2[:, 2:])\n\n wh = (rb - lt).clamp(min=0) # [N,M,2]\n area = wh[:, :, 0] * wh[:, :, 1]\n\n return iou - (area - union) / area\n\n\ndef masks_to_boxes(masks):\n \"\"\"Compute the bounding boxes around the provided masks\n\n The masks should be in format [N, H, W] where N is the number of masks, (H, W) are the spatial dimensions.\n\n Returns a [N, 4] tensors, with the boxes in xyxy format\n \"\"\"\n if masks.numel() == 0:\n return torch.zeros((0, 4), device=masks.device)\n\n h, w = masks.shape[-2:]\n\n y = torch.arange(0, h, dtype=torch.float)\n x = torch.arange(0, w, dtype=torch.float)\n y, x = torch.meshgrid(y, x)\n\n x_mask = (masks * x.unsqueeze(0))\n x_max = x_mask.flatten(1).max(-1)[0]\n x_min = x_mask.masked_fill(~(masks.bool()), 1e8).flatten(1).min(-1)[0]\n\n y_mask = (masks * y.unsqueeze(0))\n y_max = y_mask.flatten(1).max(-1)[0]\n y_min = y_mask.masked_fill(~(masks.bool()), 1e8).flatten(1).min(-1)[0]\n\n return torch.stack([x_min, y_min, x_max, y_max], 1)\n"
},
{
"path": "batchformer-v2/deformable-detr/util/misc.py",
"content": "# ------------------------------------------------------------------------\n# Deformable DETR\n# Copyright (c) 2020 SenseTime. All Rights Reserved.\n# Licensed under the Apache License, Version 2.0 [see LICENSE for details]\n# ------------------------------------------------------------------------\n# Modified from DETR (https://github.com/facebookresearch/detr)\n# Copyright (c) Facebook, Inc. and its affiliates. All Rights Reserved\n# ------------------------------------------------------------------------\n\n\"\"\"\nMisc functions, including distributed helpers.\n\nMostly copy-paste from torchvision references.\n\"\"\"\nimport os\nimport subprocess\nimport time\nfrom collections import defaultdict, deque\nimport datetime\nimport pickle\nfrom typing import Optional, List\n\nimport torch\nimport torch.nn as nn\nimport torch.distributed as dist\nfrom torch import Tensor\n\n# needed due to empty tensor bug in pytorch and torchvision 0.5\nimport torchvision\nif float(torchvision.__version__[:3]) < 0.5 and float(torchvision.__version__[:3]) > 0.1:\n import math\n from torchvision.ops.misc import _NewEmptyTensorOp\n def _check_size_scale_factor(dim, size, scale_factor):\n # type: (int, Optional[List[int]], Optional[float]) -> None\n if size is None and scale_factor is None:\n raise ValueError(\"either size or scale_factor should be defined\")\n if size is not None and scale_factor is not None:\n raise ValueError(\"only one of size or scale_factor should be defined\")\n if not (scale_factor is not None and len(scale_factor) != dim):\n raise ValueError(\n \"scale_factor shape must match input shape. \"\n \"Input is {}D, scale_factor size is {}\".format(dim, len(scale_factor))\n )\n def _output_size(dim, input, size, scale_factor):\n # type: (int, Tensor, Optional[List[int]], Optional[float]) -> List[int]\n assert dim == 2\n _check_size_scale_factor(dim, size, scale_factor)\n if size is not None:\n return size\n # if dim is not 2 or scale_factor is iterable use _ntuple instead of concat\n assert scale_factor is not None and isinstance(scale_factor, (int, float))\n scale_factors = [scale_factor, scale_factor]\n # math.floor might return float in py2.7\n return [\n int(math.floor(input.size(i + 2) * scale_factors[i])) for i in range(dim)\n ]\nelif float(torchvision.__version__[:3]) < 0.7 and float(torchvision.__version__[:3]) > 0.1:\n from torchvision.ops import _new_empty_tensor\n from torchvision.ops.misc import _output_size\n\n\nclass SmoothedValue(object):\n \"\"\"Track a series of values and provide access to smoothed values over a\n window or the global series average.\n \"\"\"\n\n def __init__(self, window_size=20, fmt=None):\n if fmt is None:\n fmt = \"{median:.4f} ({global_avg:.4f})\"\n self.deque = deque(maxlen=window_size)\n self.total = 0.0\n self.count = 0\n self.fmt = fmt\n\n def update(self, value, n=1):\n self.deque.append(value)\n self.count += n\n self.total += value * n\n\n def synchronize_between_processes(self):\n \"\"\"\n Warning: does not synchronize the deque!\n \"\"\"\n if not is_dist_avail_and_initialized():\n return\n t = torch.tensor([self.count, self.total], dtype=torch.float64, device='cuda')\n dist.barrier()\n dist.all_reduce(t)\n t = t.tolist()\n self.count = int(t[0])\n self.total = t[1]\n\n @property\n def median(self):\n d = torch.tensor(list(self.deque))\n return d.median().item()\n\n @property\n def avg(self):\n d = torch.tensor(list(self.deque), dtype=torch.float32)\n return d.mean().item()\n\n @property\n def global_avg(self):\n return self.total / self.count\n\n @property\n def max(self):\n return max(self.deque)\n\n @property\n def value(self):\n return self.deque[-1]\n\n def __str__(self):\n return self.fmt.format(\n median=self.median,\n avg=self.avg,\n global_avg=self.global_avg,\n max=self.max,\n value=self.value)\n\n\ndef all_gather(data):\n \"\"\"\n Run all_gather on arbitrary picklable data (not necessarily tensors)\n Args:\n data: any picklable object\n Returns:\n list[data]: list of data gathered from each rank\n \"\"\"\n world_size = get_world_size()\n if world_size == 1:\n return [data]\n\n # serialized to a Tensor\n buffer = pickle.dumps(data)\n storage = torch.ByteStorage.from_buffer(buffer)\n tensor = torch.ByteTensor(storage).to(\"cuda\")\n\n # obtain Tensor size of each rank\n local_size = torch.tensor([tensor.numel()], device=\"cuda\")\n size_list = [torch.tensor([0], device=\"cuda\") for _ in range(world_size)]\n dist.all_gather(size_list, local_size)\n size_list = [int(size.item()) for size in size_list]\n max_size = max(size_list)\n\n # receiving Tensor from all ranks\n # we pad the tensor because torch all_gather does not support\n # gathering tensors of different shapes\n tensor_list = []\n for _ in size_list:\n tensor_list.append(torch.empty((max_size,), dtype=torch.uint8, device=\"cuda\"))\n if local_size != max_size:\n padding = torch.empty(size=(max_size - local_size,), dtype=torch.uint8, device=\"cuda\")\n tensor = torch.cat((tensor, padding), dim=0)\n dist.all_gather(tensor_list, tensor)\n\n data_list = []\n for size, tensor in zip(size_list, tensor_list):\n buffer = tensor.cpu().numpy().tobytes()[:size]\n data_list.append(pickle.loads(buffer))\n\n return data_list\n\n\ndef reduce_dict(input_dict, average=True):\n \"\"\"\n Args:\n input_dict (dict): all the values will be reduced\n average (bool): whether to do average or sum\n Reduce the values in the dictionary from all processes so that all processes\n have the averaged results. Returns a dict with the same fields as\n input_dict, after reduction.\n \"\"\"\n world_size = get_world_size()\n if world_size < 2:\n return input_dict\n with torch.no_grad():\n names = []\n values = []\n # sort the keys so that they are consistent across processes\n for k in sorted(input_dict.keys()):\n names.append(k)\n values.append(input_dict[k])\n values = torch.stack(values, dim=0)\n dist.all_reduce(values)\n if average:\n values /= world_size\n reduced_dict = {k: v for k, v in zip(names, values)}\n return reduced_dict\n\n\nclass MetricLogger(object):\n def __init__(self, delimiter=\"\\t\"):\n self.meters = defaultdict(SmoothedValue)\n self.delimiter = delimiter\n\n def update(self, **kwargs):\n for k, v in kwargs.items():\n if isinstance(v, torch.Tensor):\n v = v.item()\n assert isinstance(v, (float, int))\n self.meters[k].update(v)\n\n def __getattr__(self, attr):\n if attr in self.meters:\n return self.meters[attr]\n if attr in self.__dict__:\n return self.__dict__[attr]\n raise AttributeError(\"'{}' object has no attribute '{}'\".format(\n type(self).__name__, attr))\n\n def __str__(self):\n loss_str = []\n for name, meter in self.meters.items():\n loss_str.append(\n \"{}: {}\".format(name, str(meter))\n )\n return self.delimiter.join(loss_str)\n\n def synchronize_between_processes(self):\n for meter in self.meters.values():\n meter.synchronize_between_processes()\n\n def add_meter(self, name, meter):\n self.meters[name] = meter\n\n def log_every(self, iterable, print_freq, header=None):\n i = 0\n if not header:\n header = ''\n start_time = time.time()\n end = time.time()\n iter_time = SmoothedValue(fmt='{avg:.4f}')\n data_time = SmoothedValue(fmt='{avg:.4f}')\n space_fmt = ':' + str(len(str(len(iterable)))) + 'd'\n if torch.cuda.is_available():\n log_msg = self.delimiter.join([\n header,\n '[{0' + space_fmt + '}/{1}]',\n 'eta: {eta}',\n '{meters}',\n 'time: {time}',\n 'data: {data}',\n 'max mem: {memory:.0f}'\n ])\n else:\n log_msg = self.delimiter.join([\n header,\n '[{0' + space_fmt + '}/{1}]',\n 'eta: {eta}',\n '{meters}',\n 'time: {time}',\n 'data: {data}'\n ])\n MB = 1024.0 * 1024.0\n for obj in iterable:\n data_time.update(time.time() - end)\n yield obj\n iter_time.update(time.time() - end)\n if i % print_freq == 0 or i == len(iterable) - 1:\n eta_seconds = iter_time.global_avg * (len(iterable) - i)\n eta_string = str(datetime.timedelta(seconds=int(eta_seconds)))\n if torch.cuda.is_available():\n print(log_msg.format(\n i, len(iterable), eta=eta_string,\n meters=str(self),\n time=str(iter_time), data=str(data_time),\n memory=torch.cuda.max_memory_allocated() / MB))\n else:\n print(log_msg.format(\n i, len(iterable), eta=eta_string,\n meters=str(self),\n time=str(iter_time), data=str(data_time)))\n i += 1\n end = time.time()\n total_time = time.time() - start_time\n total_time_str = str(datetime.timedelta(seconds=int(total_time)))\n print('{} Total time: {} ({:.4f} s / it)'.format(\n header, total_time_str, total_time / len(iterable)))\n\n\ndef get_sha():\n cwd = os.path.dirname(os.path.abspath(__file__))\n\n def _run(command):\n return subprocess.check_output(command, cwd=cwd).decode('ascii').strip()\n sha = 'N/A'\n diff = \"clean\"\n branch = 'N/A'\n try:\n sha = _run(['git', 'rev-parse', 'HEAD'])\n subprocess.check_output(['git', 'diff'], cwd=cwd)\n diff = _run(['git', 'diff-index', 'HEAD'])\n diff = \"has uncommited changes\" if diff else \"clean\"\n branch = _run(['git', 'rev-parse', '--abbrev-ref', 'HEAD'])\n except Exception:\n pass\n message = f\"sha: {sha}, status: {diff}, branch: {branch}\"\n return message\n\n\ndef collate_fn(batch):\n batch = list(zip(*batch))\n batch[0] = nested_tensor_from_tensor_list(batch[0])\n return tuple(batch)\n\n\ndef _max_by_axis(the_list):\n # type: (List[List[int]]) -> List[int]\n maxes = the_list[0]\n for sublist in the_list[1:]:\n for index, item in enumerate(sublist):\n maxes[index] = max(maxes[index], item)\n return maxes\n\n\ndef nested_tensor_from_tensor_list(tensor_list: List[Tensor]):\n # TODO make this more general\n if tensor_list[0].ndim == 3:\n # TODO make it support different-sized images\n max_size = _max_by_axis([list(img.shape) for img in tensor_list])\n # min_size = tuple(min(s) for s in zip(*[img.shape for img in tensor_list]))\n batch_shape = [len(tensor_list)] + max_size\n b, c, h, w = batch_shape\n dtype = tensor_list[0].dtype\n device = tensor_list[0].device\n tensor = torch.zeros(batch_shape, dtype=dtype, device=device)\n mask = torch.ones((b, h, w), dtype=torch.bool, device=device)\n for img, pad_img, m in zip(tensor_list, tensor, mask):\n pad_img[: img.shape[0], : img.shape[1], : img.shape[2]].copy_(img)\n m[: img.shape[1], :img.shape[2]] = False\n else:\n raise ValueError('not supported')\n return NestedTensor(tensor, mask)\n\n\nclass NestedTensor(object):\n def __init__(self, tensors, mask: Optional[Tensor]):\n self.tensors = tensors\n self.mask = mask\n\n def to(self, device, non_blocking=False):\n # type: (Device) -> NestedTensor # noqa\n cast_tensor = self.tensors.to(device, non_blocking=non_blocking)\n mask = self.mask\n if mask is not None:\n assert mask is not None\n cast_mask = mask.to(device, non_blocking=non_blocking)\n else:\n cast_mask = None\n return NestedTensor(cast_tensor, cast_mask)\n\n def record_stream(self, *args, **kwargs):\n self.tensors.record_stream(*args, **kwargs)\n if self.mask is not None:\n self.mask.record_stream(*args, **kwargs)\n\n def decompose(self):\n return self.tensors, self.mask\n\n def __repr__(self):\n return str(self.tensors)\n\n\ndef setup_for_distributed(is_master):\n \"\"\"\n This function disables printing when not in master process\n \"\"\"\n import builtins as __builtin__\n builtin_print = __builtin__.print\n\n def print(*args, **kwargs):\n force = kwargs.pop('force', False)\n if is_master or force:\n builtin_print(*args, **kwargs)\n\n __builtin__.print = print\n\n\ndef is_dist_avail_and_initialized():\n if not dist.is_available():\n return False\n if not dist.is_initialized():\n return False\n return True\n\n\ndef get_world_size():\n if not is_dist_avail_and_initialized():\n return 1\n return dist.get_world_size()\n\n\ndef get_rank():\n if not is_dist_avail_and_initialized():\n return 0\n return dist.get_rank()\n\n\ndef get_local_size():\n if not is_dist_avail_and_initialized():\n return 1\n return int(os.environ['LOCAL_SIZE'])\n\n\ndef get_local_rank():\n if not is_dist_avail_and_initialized():\n return 0\n return int(os.environ['LOCAL_RANK'])\n\n\ndef is_main_process():\n return get_rank() == 0\n\n\ndef save_on_master(*args, **kwargs):\n if is_main_process():\n torch.save(*args, **kwargs)\n\n\ndef init_distributed_mode(args):\n if 'RANK' in os.environ and 'WORLD_SIZE' in os.environ:\n args.rank = int(os.environ[\"RANK\"])\n args.world_size = int(os.environ['WORLD_SIZE'])\n args.gpu = int(os.environ['LOCAL_RANK'])\n args.dist_url = 'env://'\n os.environ['LOCAL_SIZE'] = str(torch.cuda.device_count())\n elif 'SLURM_PROCID' in os.environ:\n proc_id = int(os.environ['SLURM_PROCID'])\n ntasks = int(os.environ['SLURM_NTASKS'])\n node_list = os.environ['SLURM_NODELIST']\n num_gpus = torch.cuda.device_count()\n addr = subprocess.getoutput(\n 'scontrol show hostname {} | head -n1'.format(node_list))\n os.environ['MASTER_PORT'] = os.environ.get('MASTER_PORT', '29500')\n os.environ['MASTER_ADDR'] = addr\n os.environ['WORLD_SIZE'] = str(ntasks)\n os.environ['RANK'] = str(proc_id)\n os.environ['LOCAL_RANK'] = str(proc_id % num_gpus)\n os.environ['LOCAL_SIZE'] = str(num_gpus)\n args.dist_url = 'env://'\n args.world_size = ntasks\n args.rank = proc_id\n args.gpu = proc_id % num_gpus\n else:\n print('Not using distributed mode')\n args.distributed = False\n return\n\n args.distributed = True\n\n torch.cuda.set_device(args.gpu)\n args.dist_backend = 'nccl'\n print('| distributed init (rank {}): {}'.format(\n args.rank, args.dist_url), flush=True)\n torch.distributed.init_process_group(backend=args.dist_backend, init_method=args.dist_url,\n world_size=args.world_size, rank=args.rank)\n torch.distributed.barrier()\n setup_for_distributed(args.rank == 0)\n\n\n@torch.no_grad()\ndef accuracy(output, target, topk=(1,)):\n \"\"\"Computes the precision@k for the specified values of k\"\"\"\n if target.numel() == 0:\n return [torch.zeros([], device=output.device)]\n maxk = max(topk)\n batch_size = target.size(0)\n\n _, pred = output.topk(maxk, 1, True, True)\n pred = pred.t()\n correct = pred.eq(target.view(1, -1).expand_as(pred))\n\n res = []\n for k in topk:\n correct_k = correct[:k].view(-1).float().sum(0)\n res.append(correct_k.mul_(100.0 / batch_size))\n return res\n\n\ndef interpolate(input, size=None, scale_factor=None, mode=\"nearest\", align_corners=None):\n # type: (Tensor, Optional[List[int]], Optional[float], str, Optional[bool]) -> Tensor\n \"\"\"\n Equivalent to nn.functional.interpolate, but with support for empty batch sizes.\n This will eventually be supported natively by PyTorch, and this\n class can go away.\n \"\"\"\n if float(torchvision.__version__[:3]) < 0.7:\n if input.numel() > 0:\n return torch.nn.functional.interpolate(\n input, size, scale_factor, mode, align_corners\n )\n\n output_shape = _output_size(2, input, size, scale_factor)\n output_shape = list(input.shape[:-2]) + list(output_shape)\n if float(torchvision.__version__[:3]) < 0.5:\n return _NewEmptyTensorOp.apply(input, output_shape)\n return _new_empty_tensor(input, output_shape)\n else:\n return torchvision.ops.misc.interpolate(input, size, scale_factor, mode, align_corners)\n\n\ndef get_total_grad_norm(parameters, norm_type=2):\n parameters = list(filter(lambda p: p.grad is not None, parameters))\n norm_type = float(norm_type)\n device = parameters[0].grad.device\n total_norm = torch.norm(torch.stack([torch.norm(p.grad.detach(), norm_type).to(device) for p in parameters]),\n norm_type)\n return total_norm\n\ndef inverse_sigmoid(x, eps=1e-5):\n x = x.clamp(min=0, max=1)\n x1 = x.clamp(min=eps)\n x2 = (1 - x).clamp(min=eps)\n return torch.log(x1/x2)\n\n"
},
{
"path": "batchformer-v2/deformable-detr/util/plot_utils.py",
"content": "# ------------------------------------------------------------------------\n# Deformable DETR\n# Copyright (c) 2020 SenseTime. All Rights Reserved.\n# Licensed under the Apache License, Version 2.0 [see LICENSE for details]\n# ------------------------------------------------------------------------\n# Modified from DETR (https://github.com/facebookresearch/detr)\n# Copyright (c) Facebook, Inc. and its affiliates. All Rights Reserved\n# ------------------------------------------------------------------------\n\n\"\"\"\nPlotting utilities to visualize training logs.\n\"\"\"\nimport torch\nimport pandas as pd\nimport seaborn as sns\nimport matplotlib.pyplot as plt\n\nfrom pathlib import Path, PurePath\n\n\ndef plot_logs(logs, fields=('class_error', 'loss_bbox_unscaled', 'mAP'), ewm_col=0, log_name='log.txt'):\n '''\n Function to plot specific fields from training log(s). Plots both training and test results.\n\n :: Inputs - logs = list containing Path objects, each pointing to individual dir with a log file\n - fields = which results to plot from each log file - plots both training and test for each field.\n - ewm_col = optional, which column to use as the exponential weighted smoothing of the plots\n - log_name = optional, name of log file if different than default 'log.txt'.\n\n :: Outputs - matplotlib plots of results in fields, color coded for each log file.\n - solid lines are training results, dashed lines are test results.\n\n '''\n func_name = \"plot_utils.py::plot_logs\"\n\n # verify logs is a list of Paths (list[Paths]) or single Pathlib object Path,\n # convert single Path to list to avoid 'not iterable' error\n\n if not isinstance(logs, list):\n if isinstance(logs, PurePath):\n logs = [logs]\n print(f\"{func_name} info: logs param expects a list argument, converted to list[Path].\")\n else:\n raise ValueError(f\"{func_name} - invalid argument for logs parameter.\\n \\\n Expect list[Path] or single Path obj, received {type(logs)}\")\n\n # verify valid dir(s) and that every item in list is Path object\n for i, dir in enumerate(logs):\n if not isinstance(dir, PurePath):\n raise ValueError(f\"{func_name} - non-Path object in logs argument of {type(dir)}: \\n{dir}\")\n if dir.exists():\n continue\n raise ValueError(f\"{func_name} - invalid directory in logs argument:\\n{dir}\")\n\n # load log file(s) and plot\n dfs = [pd.read_json(Path(p) / log_name, lines=True) for p in logs]\n\n fig, axs = plt.subplots(ncols=len(fields), figsize=(16, 5))\n\n for df, color in zip(dfs, sns.color_palette(n_colors=len(logs))):\n for j, field in enumerate(fields):\n if field == 'mAP':\n coco_eval = pd.DataFrame(pd.np.stack(df.test_coco_eval.dropna().values)[:, 1]).ewm(com=ewm_col).mean()\n axs[j].plot(coco_eval, c=color)\n else:\n df.interpolate().ewm(com=ewm_col).mean().plot(\n y=[f'train_{field}', f'test_{field}'],\n ax=axs[j],\n color=[color] * 2,\n style=['-', '--']\n )\n for ax, field in zip(axs, fields):\n ax.legend([Path(p).name for p in logs])\n ax.set_title(field)\n\n\ndef plot_precision_recall(files, naming_scheme='iter'):\n if naming_scheme == 'exp_id':\n # name becomes exp_id\n names = [f.parts[-3] for f in files]\n elif naming_scheme == 'iter':\n names = [f.stem for f in files]\n else:\n raise ValueError(f'not supported {naming_scheme}')\n fig, axs = plt.subplots(ncols=2, figsize=(16, 5))\n for f, color, name in zip(files, sns.color_palette(\"Blues\", n_colors=len(files)), names):\n data = torch.load(f)\n # precision is n_iou, n_points, n_cat, n_area, max_det\n precision = data['precision']\n recall = data['params'].recThrs\n scores = data['scores']\n # take precision for all classes, all areas and 100 detections\n precision = precision[0, :, :, 0, -1].mean(1)\n scores = scores[0, :, :, 0, -1].mean(1)\n prec = precision.mean()\n rec = data['recall'][0, :, 0, -1].mean()\n print(f'{naming_scheme} {name}: mAP@50={prec * 100: 05.1f}, ' +\n f'score={scores.mean():0.3f}, ' +\n f'f1={2 * prec * rec / (prec + rec + 1e-8):0.3f}'\n )\n axs[0].plot(recall, precision, c=color)\n axs[1].plot(recall, scores, c=color)\n\n axs[0].set_title('Precision / Recall')\n axs[0].legend(names)\n axs[1].set_title('Scores / Recall')\n axs[1].legend(names)\n return fig, axs\n\n\n\n"
},
{
"path": "batchformer-v2/deformable-detr/util/visualizer.py",
"content": "\nimport argparse\n\nimport cv2\nimport numpy as np\nimport re\n\nfrom detectron2 import model_zoo\nfrom detectron2.config import get_cfg, CfgNode\nfrom detectron2.data import MetadataCatalog\nfrom detectron2.engine import DefaultPredictor\nfrom detectron2.structures import Instances\nfrom detectron2.utils.visualizer import Visualizer, VisImage\n\n\nif __name__ == \"__main__\":\n\n image_file: str\n for image_file in args.images:\n img: np.ndarray = cv2.imread(image_file)\n\n output: Instances = predictor(img)[\"instances\"]\n v = Visualizer(img[:, :, ::-1],\n MetadataCatalog.get(cfg.DATASETS.TRAIN[0]),\n scale=1.0)\n result: VisImage = v.draw_instance_predictions(output.to(\"cpu\"))\n result_image: np.ndarray = result.get_image()[:, :, ::-1]\n\n # get file name without extension, -1 to remove \".\" at the end\n out_file_name: str = re.search(r\"(.*)\\.\", image_file).group(0)[:-1]\n out_file_name += \"_processed.png\"\n\n cv2.imwrite(out_file_name, result_image)"
},
{
"path": "batchformer-v2/deit_share/.circleci/config.yml",
"content": "version: 2.1\n\njobs:\n python_lint:\n docker:\n - image: circleci/python:3.7\n steps:\n - checkout\n - run:\n command: |\n pip install --user --progress-bar off flake8 typing\n flake8 .\n test:\n docker:\n - image: circleci/python:3.7\n steps:\n - checkout\n - run:\n command: |\n pip install --user --progress-bar off pytest\n pip install --user --progress-bar off torch torchvision\n pip install --user --progress-bar off timm==0.3.2\n pytest .\n\nworkflows:\n build:\n jobs:\n - python_lint\n"
},
{
"path": "batchformer-v2/deit_share/.github/CODE_OF_CONDUCT.md",
"content": "# Code of Conduct\n\nFacebook has adopted a Code of Conduct that we expect project participants to adhere to.\nPlease read the [full text](https://code.fb.com/codeofconduct/)\nso that you can understand what actions will and will not be tolerated.\n"
},
{
"path": "batchformer-v2/deit_share/.github/CONTRIBUTING.md",
"content": "# Contributing to DeiT\nWe want to make contributing to this project as easy and transparent as\npossible.\n\n## Our Development Process\nMinor changes and improvements will be released on an ongoing basis. Larger changes (e.g., changesets implementing a new paper) will be released on a more periodic basis.\n\n## Pull Requests\nWe actively welcome your pull requests.\n\n1. Fork the repo and create your branch from `main`.\n2. If you've added code that should be tested, add tests.\n3. If you've changed APIs, update the documentation.\n4. Ensure the test suite passes.\n5. Make sure your code lints.\n6. If you haven't already, complete the Contributor License Agreement (\"CLA\").\n\n## Contributor License Agreement (\"CLA\")\nIn order to accept your pull request, we need you to submit a CLA. You only need\nto do this once to work on any of Facebook's open source projects.\n\nComplete your CLA here: \n\n## Issues\nWe use GitHub issues to track public bugs. Please ensure your description is\nclear and has sufficient instructions to be able to reproduce the issue.\n\nFacebook has a [bounty program](https://www.facebook.com/whitehat/) for the safe\ndisclosure of security bugs. In those cases, please go through the process\noutlined on that page and do not file a public issue.\n\n## Coding Style \n* 4 spaces for indentation rather than tabs\n* 80 character line length\n* PEP8 formatting following [Black](https://black.readthedocs.io/en/stable/)\n\n## License\nBy contributing to DeiT, you agree that your contributions will be licensed\nunder the LICENSE file in the root directory of this source tree.\n"
},
{
"path": "batchformer-v2/deit_share/.gitignore",
"content": "*.swp\n**/__pycache__/**\nimnet_resnet50_scratch/timm_temp/\n.dumbo.json\ncheckpoints/\n"
},
{
"path": "batchformer-v2/deit_share/.idea/.gitignore",
"content": "# Default ignored files\n/shelf/\n/workspace.xml\n# Datasource local storage ignored files\n/dataSources/\n/dataSources.local.xml\n# Editor-based HTTP Client requests\n/httpRequests/\n"
},
{
"path": "batchformer-v2/deit_share/.idea/deit_share.iml",
"content": "\n\n \n \n \n \n \n \n \n \n \n"
},
{
"path": "batchformer-v2/deit_share/.idea/inspectionProfiles/profiles_settings.xml",
"content": "\n \n \n \n \n"
},
{
"path": "batchformer-v2/deit_share/.idea/misc.xml",
"content": "\n\n \n"
},
{
"path": "batchformer-v2/deit_share/.idea/modules.xml",
"content": "\n\n \n \n \n \n \n"
},
{
"path": "batchformer-v2/deit_share/.idea/vcs.xml",
"content": "\n\n \n \n \n \n"
},
{
"path": "batchformer-v2/deit_share/LICENSE",
"content": " Apache License\n Version 2.0, January 2004\n http://www.apache.org/licenses/\n\n TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION\n\n 1. Definitions.\n\n \"License\" shall mean the terms and conditions for use, reproduction,\n and distribution as defined by Sections 1 through 9 of this document.\n\n \"Licensor\" shall mean the copyright owner or entity authorized by\n the copyright owner that is granting the License.\n\n \"Legal Entity\" shall mean the union of the acting entity and all\n other entities that control, are controlled by, or are under common\n control with that entity. For the purposes of this definition,\n \"control\" means (i) the power, direct or indirect, to cause the\n direction or management of such entity, whether by contract or\n otherwise, or (ii) ownership of fifty percent (50%) or more of the\n outstanding shares, or (iii) beneficial ownership of such entity.\n\n \"You\" (or \"Your\") shall mean an individual or Legal Entity\n exercising permissions granted by this License.\n\n \"Source\" form shall mean the preferred form for making modifications,\n including but not limited to software source code, documentation\n source, and configuration files.\n\n \"Object\" form shall mean any form resulting from mechanical\n transformation or translation of a Source form, including but\n not limited to compiled object code, generated documentation,\n and conversions to other media types.\n\n \"Work\" shall mean the work of authorship, whether in Source or\n Object form, made available under the License, as indicated by a\n copyright notice that is included in or attached to the work\n (an example is provided in the Appendix below).\n\n \"Derivative Works\" shall mean any work, whether in Source or Object\n form, that is based on (or derived from) the Work and for which the\n editorial revisions, annotations, elaborations, or other modifications\n represent, as a whole, an original work of authorship. For the purposes\n of this License, Derivative Works shall not include works that remain\n separable from, or merely link (or bind by name) to the interfaces of,\n the Work and Derivative Works thereof.\n\n \"Contribution\" shall mean any work of authorship, including\n the original version of the Work and any modifications or additions\n to that Work or Derivative Works thereof, that is intentionally\n submitted to Licensor for inclusion in the Work by the copyright owner\n or by an individual or Legal Entity authorized to submit on behalf of\n the copyright owner. For the purposes of this definition, \"submitted\"\n means any form of electronic, verbal, or written communication sent\n to the Licensor or its representatives, including but not limited to\n communication on electronic mailing lists, source code control systems,\n and issue tracking systems that are managed by, or on behalf of, the\n Licensor for the purpose of discussing and improving the Work, but\n excluding communication that is conspicuously marked or otherwise\n designated in writing by the copyright owner as \"Not a Contribution.\"\n\n \"Contributor\" shall mean Licensor and any individual or Legal Entity\n on behalf of whom a Contribution has been received by Licensor and\n subsequently incorporated within the Work.\n\n 2. Grant of Copyright License. Subject to the terms and conditions of\n this License, each Contributor hereby grants to You a perpetual,\n worldwide, non-exclusive, no-charge, royalty-free, irrevocable\n copyright license to reproduce, prepare Derivative Works of,\n publicly display, publicly perform, sublicense, and distribute the\n Work and such Derivative Works in Source or Object form.\n\n 3. Grant of Patent License. Subject to the terms and conditions of\n this License, each Contributor hereby grants to You a perpetual,\n worldwide, non-exclusive, no-charge, royalty-free, irrevocable\n (except as stated in this section) patent license to make, have made,\n use, offer to sell, sell, import, and otherwise transfer the Work,\n where such license applies only to those patent claims licensable\n by such Contributor that are necessarily infringed by their\n Contribution(s) alone or by combination of their Contribution(s)\n with the Work to which such Contribution(s) was submitted. If You\n institute patent litigation against any entity (including a\n cross-claim or counterclaim in a lawsuit) alleging that the Work\n or a Contribution incorporated within the Work constitutes direct\n or contributory patent infringement, then any patent licenses\n granted to You under this License for that Work shall terminate\n as of the date such litigation is filed.\n\n 4. Redistribution. You may reproduce and distribute copies of the\n Work or Derivative Works thereof in any medium, with or without\n modifications, and in Source or Object form, provided that You\n meet the following conditions:\n\n (a) You must give any other recipients of the Work or\n Derivative Works a copy of this License; and\n\n (b) You must cause any modified files to carry prominent notices\n stating that You changed the files; and\n\n (c) You must retain, in the Source form of any Derivative Works\n that You distribute, all copyright, patent, trademark, and\n attribution notices from the Source form of the Work,\n excluding those notices that do not pertain to any part of\n the Derivative Works; and\n\n (d) If the Work includes a \"NOTICE\" text file as part of its\n distribution, then any Derivative Works that You distribute must\n include a readable copy of the attribution notices contained\n within such NOTICE file, excluding those notices that do not\n pertain to any part of the Derivative Works, in at least one\n of the following places: within a NOTICE text file distributed\n as part of the Derivative Works; within the Source form or\n documentation, if provided along with the Derivative Works; or,\n within a display generated by the Derivative Works, if and\n wherever such third-party notices normally appear. The contents\n of the NOTICE file are for informational purposes only and\n do not modify the License. You may add Your own attribution\n notices within Derivative Works that You distribute, alongside\n or as an addendum to the NOTICE text from the Work, provided\n that such additional attribution notices cannot be construed\n as modifying the License.\n\n You may add Your own copyright statement to Your modifications and\n may provide additional or different license terms and conditions\n for use, reproduction, or distribution of Your modifications, or\n for any such Derivative Works as a whole, provided Your use,\n reproduction, and distribution of the Work otherwise complies with\n the conditions stated in this License.\n\n 5. Submission of Contributions. Unless You explicitly state otherwise,\n any Contribution intentionally submitted for inclusion in the Work\n by You to the Licensor shall be under the terms and conditions of\n this License, without any additional terms or conditions.\n Notwithstanding the above, nothing herein shall supersede or modify\n the terms of any separate license agreement you may have executed\n with Licensor regarding such Contributions.\n\n 6. Trademarks. This License does not grant permission to use the trade\n names, trademarks, service marks, or product names of the Licensor,\n except as required for reasonable and customary use in describing the\n origin of the Work and reproducing the content of the NOTICE file.\n\n 7. Disclaimer of Warranty. Unless required by applicable law or\n agreed to in writing, Licensor provides the Work (and each\n Contributor provides its Contributions) on an \"AS IS\" BASIS,\n WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or\n implied, including, without limitation, any warranties or conditions\n of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A\n PARTICULAR PURPOSE. You are solely responsible for determining the\n appropriateness of using or redistributing the Work and assume any\n risks associated with Your exercise of permissions under this License.\n\n 8. Limitation of Liability. In no event and under no legal theory,\n whether in tort (including negligence), contract, or otherwise,\n unless required by applicable law (such as deliberate and grossly\n negligent acts) or agreed to in writing, shall any Contributor be\n liable to You for damages, including any direct, indirect, special,\n incidental, or consequential damages of any character arising as a\n result of this License or out of the use or inability to use the\n Work (including but not limited to damages for loss of goodwill,\n work stoppage, computer failure or malfunction, or any and all\n other commercial damages or losses), even if such Contributor\n has been advised of the possibility of such damages.\n\n 9. Accepting Warranty or Additional Liability. While redistributing\n the Work or Derivative Works thereof, You may choose to offer,\n and charge a fee for, acceptance of support, warranty, indemnity,\n or other liability obligations and/or rights consistent with this\n License. However, in accepting such obligations, You may act only\n on Your own behalf and on Your sole responsibility, not on behalf\n of any other Contributor, and only if You agree to indemnify,\n defend, and hold each Contributor harmless for any liability\n incurred by, or claims asserted against, such Contributor by reason\n of your accepting any such warranty or additional liability.\n\n END OF TERMS AND CONDITIONS\n\n APPENDIX: How to apply the Apache License to your work.\n\n To apply the Apache License to your work, attach the following\n boilerplate notice, with the fields enclosed by brackets \"[]\"\n replaced with your own identifying information. (Don't include\n the brackets!) The text should be enclosed in the appropriate\n comment syntax for the file format. We also recommend that a\n file or class name and description of purpose be included on the\n same \"printed page\" as the copyright notice for easier\n identification within third-party archives.\n\n Copyright 2020 - present, Facebook, Inc\n\n Licensed under the Apache License, Version 2.0 (the \"License\");\n you may not use this file except in compliance with the License.\n You may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\n Unless required by applicable law or agreed to in writing, software\n distributed under the License is distributed on an \"AS IS\" BASIS,\n WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n See the License for the specific language governing permissions and\n limitations under the License.\n"
},
{
"path": "batchformer-v2/deit_share/README.md",
"content": "#BatchFormerV2\n\nThe hyperparameters\n\n--add_global 2 --insert_idx 8 --bt_lr 0.5 # for larger model, maybe, it requires to avoid overfitting. e.g. a smaller lr for bt\n\n--add_global 2 --start_idx 0 # CIFAR\n"
},
{
"path": "batchformer-v2/deit_share/README_cait.md",
"content": "\n# CaiT: Going deeper with Image Transformers\n\nThis repository contains PyTorch evaluation code, training code and pretrained models for DeiT (Data-Efficient Image Transformers) and CaiT (Going deeper with Image Transformers). All models are trained during 400 epochs. \n\nCaiT obtain competitive tradeoffs in terms of flops / precision:\n\n
\n \n
\n\nFor details see [Going deeper with Image Transformers](https://arxiv.org/abs/2103.17239) by Hugo Touvron, Matthieu Cord, Alexandre Sablayrolles, Gabriel Synnaeve and Hervé Jégou\n\nIf you use this code for a paper please cite:\n\n```\n@article{touvron2021cait,\n title={Going deeper with Image Transformers},\n author={Hugo Touvron and Matthieu Cord and Alexandre Sablayrolles and Gabriel Synnaeve and Herv\\'e J\\'egou},\n journal={arXiv preprint arXiv:2103.17239},\n year={2021}\n}\n```\n\n# Model Zoo\n\nWe provide baseline CaiT models pretrained on ImageNet1k 2012 only, using the distilled version of our method.\n\n| name | acc@1 | res | FLOPs| #params | url |\n| --- | --- | --- | --- | --- | --- | \n| S24 | 83.5 | 224 |9.4B| 47M| [model](https://dl.fbaipublicfiles.com/deit/S24_224.pth) |\n| XS24| 84.1 | 384 | 19.3B |27M | [model](https://dl.fbaipublicfiles.com/deit/XS24_384.pth) |\n| S24 | 85.1 | 384 | 32.2B |47M | [model](https://dl.fbaipublicfiles.com/deit/S24_384.pth) |\n| S36 | 85.4 | 384 | 48.0B| 68M| [model](https://dl.fbaipublicfiles.com/deit/S36_384.pth) |\n| M36 | 86.1 | 384 | 173.3B| 271M | [model](https://dl.fbaipublicfiles.com/deit/M36_384.pth) |\n| M48 | 86.5 | 448 | 329.6B| 356M | [model](https://dl.fbaipublicfiles.com/deit/M48_448.pth) |\n\n\nThe models are also available via torch hub.\nBefore using it, make sure you have the pytorch-image-models package [`timm==0.3.2`](https://github.com/rwightman/pytorch-image-models) by [Ross Wightman](https://github.com/rwightman) installed. \n\n# Evaluation transforms\n\nCaiT employs a slightly different pre-processing, in particular a crop-ratio of 1.0 at test time. To reproduce the results of our paper please use the following pre-processing:\n\n```\ndef get_test_transforms(input_size):\n mean, std = [0.485, 0.456, 0.406],[0.229, 0.224, 0.225] \n transformations = {}\n transformations= transforms.Compose(\n [transforms.Resize(input_size, interpolation=3),\n transforms.CenterCrop(input_size),\n transforms.ToTensor(),\n transforms.Normalize(mean, std)])\n return transformations\n ``` \n\nRemark: for CaiT M48 it is best to evaluate with FP32 precision\n\n# License\nThis repository is released under the Apache 2.0 license as found in the [LICENSE](LICENSE) file.\n\n# Contributing\nWe actively welcome your pull requests! Please see [CONTRIBUTING.md](.github/CONTRIBUTING.md) and [CODE_OF_CONDUCT.md](.github/CODE_OF_CONDUCT.md) for more info.\n"
},
{
"path": "batchformer-v2/deit_share/README_resmlp.md",
"content": "\n# ResMLP: Feedforward networks for image classification with data-efficient training\n\nThis repository contains PyTorch evaluation code, training code and pretrained models for the following projects:\n* [DeiT](README.md) (Data-Efficient Image Transformers) \n* [CaiT](README_cait.md) (Going deeper with Image Transformers)\n* [ResMLP](README_resmlp.md) (ResMLP: Feedforward networks for image classification with data-efficient training)\n\nResMLP obtain good performance given its simplicity:\n\n
\n \n
\n\nFor details see [ResMLP: Feedforward networks for image classification with data-efficient training](https://arxiv.org/abs/2105.03404) by Hugo Touvron, Piotr Bojanowski, Mathilde Caron, Matthieu Cord, Alaaeldin El-Nouby, Edouard Grave, Armand Joulin, Gabriel Synnaeve, Jakob Verbeek and Hervé Jégou.\n\nIf you use this code for a paper please cite:\n\n```\n@article{touvron2021resmlp,\n title={ResMLP: Feedforward networks for image classification with data-efficient training},\n author={Hugo Touvron and Piotr Bojanowski and Mathilde Caron and M. Cord and Alaaeldin El-Nouby and Edouard Grave and Armand Joulin and Gabriel Synnaeve and Jakob Verbeek and Herv'e J'egou},\n journal={arXiv preprint arXiv:2105.03404},\n year={2021},\n}\n```\n\n# Model Zoo\n\nWe provide baseline ResMLP models pretrained on ImageNet1k 2012, using the distilled version of our method:\n\n| name | acc@1 | res | FLOPs| #params | url |\n| --- | --- | --- | --- | --- | --- | \n| ResMLP-S12 | 77.8 | 224 |3B| 15M| [model](https://dl.fbaipublicfiles.com/deit/resmlp_12_dist.pth) |\n| ResMLP-S24| 80.8 | 224 | 6B |30M | [model](https://dl.fbaipublicfiles.com/deit/resmlp_24_dist.pth) |\n| ResMLP-S36 | 81.1 | 224 | 23B |116M | [model](https://dl.fbaipublicfiles.com/deit/resmlp_36_dist.pth) |\n| ResMLP-B24 |83.6 | 224 | 100B |129M | [model](https://dl.fbaipublicfiles.com/deit/resmlpB_24_dist.pth) |\n\nModel pretrained on ImageNet-22k with finetuning on ImageNet1k 2012:\n\n| name | acc@1 | res | FLOPs| #params | url |\n| --- | --- | --- | --- | --- | --- | \n| ResMLP-B24 |84.4 | 224 | 100B |129M | [model](https://dl.fbaipublicfiles.com/deit/resmlpB_24_22k.pth) |\n\nModels pretrained with DINO without finetuning:\n\n| name | acc@1 (knn)| res | FLOPs| #params | url |\n| --- | --- | --- | --- | --- | --- | \n| ResMLP-S12 | 62.6 | 224 |3B| 15M| [model](https://dl.fbaipublicfiles.com/deit/resmlp_12_dino.pth) |\n| ResMLP-S24| 69.4 | 224 | 6B |30M | [model](https://dl.fbaipublicfiles.com/deit/resmlp_24_dino.pth) |\n\nThe models are also available via torch hub.\nBefore using it, make sure you have the pytorch-image-models package [`timm==0.3.2`](https://github.com/rwightman/pytorch-image-models) by [Ross Wightman](https://github.com/rwightman) installed. \n\n# Evaluation transforms\n\nResMLP employs a slightly different pre-processing, in particular a crop-ratio of 0.9 at test time. To reproduce the results of our paper please use the following pre-processing:\n\n```\ndef get_test_transforms(input_size):\n mean, std = [0.485, 0.456, 0.406],[0.229, 0.224, 0.225] \n transformations = {}\n Rs_size=int(input_size/0.9)\n transformations= transforms.Compose(\n [transforms.Resize(Rs_size, interpolation=3),\n transforms.CenterCrop(input_size),\n transforms.ToTensor(),\n transforms.Normalize(mean, std)])\n return transformations\n ``` \n\n# License\nThis repository is released under the Apache 2.0 license as found in the [LICENSE](LICENSE) file.\n\n# Contributing\nWe actively welcome your pull requests! Please see [CONTRIBUTING.md](.github/CONTRIBUTING.md) and [CODE_OF_CONDUCT.md](.github/CODE_OF_CONDUCT.md) for more info.\n"
},
{
"path": "batchformer-v2/deit_share/bt.py",
"content": "import torch\nfrom timm.models.vision_transformer import Attention, Block\n\n\nclass BlockBF(torch.nn.Module):\n\n def __init__(self, block: Block, fp16_type=0):\n super().__init__()\n self.block = block\n self.fp16_type = fp16_type\n\n def forward(self, x):\n from torch.cuda.amp import autocast\n if self.fp16_type == 2:\n\n with autocast(enabled=False):\n old_x = self.block.norm1(x)\n x = x + torch.transpose(self.block.drop_path(torch.transpose(self.block.attn(old_x.float()), 1, 0)), 1, 0)\n old_x = self.block.norm2(x)\n x = x + torch.transpose(self.block.drop_path(torch.transpose(self.block.mlp(old_x), 1, 0)), 1, 0)\n elif self.fp16_type == 0:\n x = x + torch.transpose(self.block.drop_path(torch.transpose(self.block.attn(self.block.norm1(x)), 1, 0)), 1, 0)\n x = x + torch.transpose(self.block.drop_path(torch.transpose(self.block.mlp(self.block.norm2(x)), 1, 0)), 1, 0)\n else:\n old_x = self.block.norm1(x)\n with autocast(enabled=False):\n x = x + torch.transpose(self.block.drop_path(torch.transpose(self.block.attn(old_x.float()), 1, 0)), 1, 0)\n x = x + torch.transpose(self.block.drop_path(torch.transpose(self.block.mlp(self.block.norm2(x)), 1, 0)), 1, 0)\n\n\n return x\n\n\nclass BlockWrap32(torch.nn.Module):\n\n def __init__(self, block: Block, fp16_type: int):\n super().__init__()\n self.block = block\n self.fp16_type = fp16_type\n\n def forward(self, x):\n from torch.cuda.amp import autocast\n if self.fp16_type == 2:\n\n with autocast(enabled=False):\n old_x = self.block.norm1(x)\n x = x + self.block.drop_path(self.block.attn(old_x.float()))\n old_x = self.block.norm2(x)\n x = x + self.block.drop_path(self.block.mlp(old_x))\n else:\n old_x = self.block.norm1(x)\n with autocast(enabled=False):\n x = x + self.block.drop_path(self.block.attn(old_x.float()))\n x = x + self.block.drop_path(self.block.mlp(self.block.norm2(x)))\n return x\n\nclass BlockWrapDebug(torch.nn.Module):\n\n def __init__(self, block: Block):\n super().__init__()\n self.block = block\n\n def forward(self, x):\n tmp_x = self.block.drop_path(self.block.attn(self.block.norm1(x)))\n for n,p in self.block.named_parameters():\n if n.__contains__('attn.qkv.weight'):\n print(n, p.detach().cpu().numpy(), p.grad)\n x = x + tmp_x\n tmp_x = self.block.drop_path(self.block.mlp(self.block.norm2(x)))\n\n x = x + tmp_x\n return x\n\nclass AttentionOnly(torch.nn.Module):\n\n def __init__(self, block: Block, drop_path = 0. , add_bt=70):\n super().__init__()\n self.block = block\n from timm.models.layers import DropPath\n self.add_bt=add_bt\n self.drop_path = DropPath(drop_path) if drop_path > 0. else torch.nn.Identity()\n self.block.mlp = None\n self.block.norm2 = None\n\n def forward(self, x):\n if self.add_bt==70:\n x = x + torch.transpose(self.drop_path(torch.transpose(self.block.attn(self.block.norm1(x)), 0, 1)), 0, 1)\n else:\n x = x + self.drop_path(self.block.attn(self.block.norm1(x)))\n # x = x + self.block.drop_path(self.block.mlp(self.block.norm2(x)))\n return x\n\nclass MLPDecorder(torch.nn.Module):\n\n def __init__(self, dim, mlp_dim, skip_mlp=False):\n super().__init__()\n self.skip_mlp = skip_mlp\n self.mlp = torch.nn.Sequential(torch.nn.Linear(dim, mlp_dim), torch.nn.BatchNorm1d(mlp_dim), torch.nn.ReLU(),\n torch.nn.Linear(mlp_dim, dim), torch.nn.BatchNorm1d(dim), )\n\n def forward(self, x):\n if self.training and not self.skip_mlp :\n bt_x = x[len(x)//2:]\n old_x = x[:len(x)//2]\n bt_x = self.mlp(bt_x)\n x = torch.cat([old_x, bt_x], dim=0)\n return x\n\nclass MLPEncoder(torch.nn.Module):\n\n def __init__(self, d_model, batch_size, dim_feedforward=2048, dropout=0.1, activation=\"relu\"):\n super(MLPEncoder, self).__init__()\n # self.self_attn = MultiheadAttention(d_model, nhead, dropout=dropout)\n # Implementation of Feedforward model\n self.linear1 = torch.nn.Linear(batch_size, batch_size)\n\n self.norm1 = torch.nn.LayerNorm(d_model)\n self.dropout1 = torch.nn.Dropout(dropout)\n import torch.nn.functional as F\n self.activation = F.relu\n\n def __setstate__(self, state):\n if 'activation' not in state:\n import torch.nn.functional as F\n state['activation'] = F.relu\n super(MLPEncoder, self).__setstate__(state)\n\n def forward(self, src):\n r\"\"\"Pass the input through the encoder layer.\n\n Args:\n src: the sequence to the encoder layer (required).\n src_mask: the mask for the src sequence (optional).\n src_key_padding_mask: the mask for the src keys per batch (optional).\n\n Shape:\n see the docs in Transformer class.\n \"\"\"\n src2 = torch.transpose(src, 0, 2)\n src2 = self.linear1(src2)\n src2 = torch.transpose(src2, 0, 2)\n src = src + self.dropout1(src2)\n src = self.norm1(src)\n\n return src\n\nclass TransformerDecorator1(torch.nn.Module):\n def __init__(self, add_bt=3, dim=2048, eval_global=0, args=None, first_layer=False, drop_path=0.1):\n super(TransformerDecorator1, self).__init__()\n self.not_cls_token = False\n self.no_fp16_bt = 0\n self.shuffle_patch=False\n self.cls_token_only=False\n heads = 4\n self.empty_bt = 0\n self.mlp_enc = 0\n self.batch_size = 128\n self.skip_bt = False\n if args is not None:\n self.not_cls_token = args.not_cls_token\n self.cls_token_only = args.cls_token_only\n self.no_fp16_bt = args.no_fp16_bt\n self.mlp_enc = args.mlp_enc\n heads = args.num_heads\n self.shuffle_patch = args.shuffle_patch\n self.empty_bt = args.empty_bt\n self.batch_size = args.batch_size\n self.all_patches = args.all_patches\n self.skip_bt = args.skip_bt\n self.atten_weights = None\n self.encoder_layers = torch.nn.TransformerEncoderLayer(dim, heads, dim, args.bt_atten_drop)\n self.eval_global = eval_global\n self.add_bt = add_bt\n self.first_layer = first_layer\n if self.mlp_enc == 1:\n self.encoder_layers = MLPEncoder(dim, args.batch_size)\n\n if self.empty_bt:\n self.encoder_layers = torch.nn.Identity()\n self.idx = 0\n self.noise = None\n self.layer_num = 1\n if self.first_layer:\n self.add_bt = 1\n\n def forward(self, feature):\n if self.training and self.add_bt > 0 and not self.skip_bt:\n old_feature = feature\n if self.add_bt in [1, 2, 3] and not self.first_layer:\n # split\n old_feature = feature[:len(feature)//2]\n feature = feature[-len(feature)//2:]\n if self.shuffle_patch: # We do not use this\n B, L, C = feature.shape\n idx_orig_list = []\n idx_shuffle_list = []\n for i in range(len(feature)):\n idx = torch.arange(0, L).type(torch.LongTensor)\n idx_orig = torch.zeros(L).type(torch.LongTensor)\n idx_shuffle = torch.cat([torch.randperm(1).type(torch.LongTensor),torch.randperm(L -1 ).type(torch.LongTensor)], 0)\n\n idx_orig[idx_shuffle] = idx + i*L\n idx_shuffle_list.append(idx_shuffle+ i * L)\n # new_features.append(feature[i][idx_shuffle])\n idx_orig_list.append(idx_orig)\n idx_shuffle = torch.stack(idx_shuffle_list, 0).reshape(-1).to(feature.device)\n idx_orig_list = torch.stack(idx_orig_list, 0).reshape(-1).to(feature.device)\n feature = feature.view(B*L, C)[idx_shuffle].view(B, L, C)\n pass\n\n size = feature.shape\n if len(size) == 3: #vit\n if self.all_patches:\n stride = 4\n # N L C\n size = feature.shape\n feature = torch.transpose(feature, 1, 0) # L N C\n feature = torch.reshape(feature, (stride * size[1], size[0] // stride, size[2]))\n if isinstance(self.encoder_layers, Block) or \\\n isinstance(self.encoder_layers, Attention):\n feature = feature.transpose(0, 1)\n elif self.not_cls_token: # do not use in the paper\n feature1 = self.encoder_layers(feature[:, 1:, :])\n feature = torch.cat([feature[:, :1, :], feature1], dim=1)\n elif self.cls_token_only: # do not use in the paper\n feature1 = self.encoder_layers(feature[:, :1, :])\n feature = torch.cat([feature1, feature[:, 1:, :]], dim=1)\n elif self.no_fp16_bt:\n from torch.cuda.amp import autocast\n with autocast(enabled=False):\n feature = self.encoder_layers(feature.float())\n else:\n feature = self.encoder_layers(feature)\n if isinstance(self.encoder_layers, Block) or \\\n isinstance(self.encoder_layers, Attention): # The dimension is different from torch.nn.Transformer\n feature = feature.transpose(0, 1)\n if self.all_patches:\n # recover\n feature = torch.reshape(feature, (size[1], size[0], size[2]))\n feature = torch.transpose(feature, 1, 0)\n else:\n feature = feature.view(feature.size(0), feature.size(1), -1)\n feature = torch.transpose(feature, 2, 1)\n feature = self.encoder_layers(feature)\n feature = torch.transpose(feature, 2, 1)\n feature = feature.view(size)\n if self.shuffle_patch:\n feature = feature.view(B*L, C)[idx_orig_list].view(B, L, C)\n if self.add_bt not in [1]:\n feature = torch.cat([old_feature, feature], dim=0)\n # print(self.noise, self.idx)\n # print(feature.norm())\n return feature\n elif self.add_bt and self.eval_global:\n if isinstance(self.encoder_layers, Block) or \\\n isinstance(self.encoder_layers, Attention):\n feature = feature.transpose(0, 1)\n if self.add_bt and self.eval_global == 2:\n size = feature.shape\n old_feature = feature\n feature = torch.reshape(feature, (size[0] * size[1], 1, size[2]))\n feature = self.encoder_layers(feature)\n feature = torch.reshape(feature, (size[0], size[1], size[2])) # Acc@1 80.152 Acc@5 95.160 loss 0.849\n # feature = torch.cat([old_feature, feature], dim=0)\n elif self.add_bt and self.eval_global == 1:\n feature = self.encoder_layers(feature)\n elif self.add_bt and self.eval_global == 3:\n old_feature = feature\n feature = self.encoder_layers(feature)\n # feature = old_feature + feature # 79.628\n feature = torch.cat([old_feature, feature], dim=0)\n if isinstance(self.encoder_layers, Block) or \\\n isinstance(self.encoder_layers, Attention):\n feature = feature.transpose(0, 1)\n return feature\n"
},
{
"path": "batchformer-v2/deit_share/cait_models.py",
"content": "# Copyright (c) 2015-present, Facebook, Inc.\n# All rights reserved.\n\nimport torch\nimport torch.nn as nn\nfrom functools import partial\n\nfrom timm.models.vision_transformer import Mlp, PatchEmbed , _cfg\nfrom timm.models.registry import register_model\nfrom timm.models.layers import trunc_normal_, DropPath\n\n\n__all__ = [\n 'cait_M48', 'cait_M36', 'cait_M4',\n 'cait_S36', 'cait_S24','cait_S24_224',\n 'cait_XS24','cait_XXS24','cait_XXS24_224',\n 'cait_XXS36','cait_XXS36_224'\n]\n\n\nclass Class_Attention(nn.Module):\n # taken from https://github.com/rwightman/pytorch-image-models/blob/master/timm/models/vision_transformer.py\n # with slight modifications to do CA \n def __init__(self, dim, num_heads=8, qkv_bias=False, qk_scale=None, attn_drop=0., proj_drop=0.):\n super().__init__()\n self.num_heads = num_heads\n head_dim = dim // num_heads\n self.scale = qk_scale or head_dim ** -0.5\n\n self.q = nn.Linear(dim, dim, bias=qkv_bias)\n self.k = nn.Linear(dim, dim, bias=qkv_bias)\n self.v = nn.Linear(dim, dim, bias=qkv_bias)\n self.attn_drop = nn.Dropout(attn_drop)\n self.proj = nn.Linear(dim, dim)\n self.proj_drop = nn.Dropout(proj_drop)\n\n \n def forward(self, x ):\n \n B, N, C = x.shape\n q = self.q(x[:,0]).unsqueeze(1).reshape(B, 1, self.num_heads, C // self.num_heads).permute(0, 2, 1, 3)\n k = self.k(x).reshape(B, N, self.num_heads, C // self.num_heads).permute(0, 2, 1, 3)\n\n q = q * self.scale\n v = self.v(x).reshape(B, N, self.num_heads, C // self.num_heads).permute(0, 2, 1, 3)\n\n attn = (q @ k.transpose(-2, -1)) \n attn = attn.softmax(dim=-1)\n attn = self.attn_drop(attn)\n\n x_cls = (attn @ v).transpose(1, 2).reshape(B, 1, C)\n x_cls = self.proj(x_cls)\n x_cls = self.proj_drop(x_cls)\n \n return x_cls \n \nclass LayerScale_Block_CA(nn.Module):\n # taken from https://github.com/rwightman/pytorch-image-models/blob/master/timm/models/vision_transformer.py\n # with slight modifications to add CA and LayerScale\n def __init__(self, dim, num_heads, mlp_ratio=4., qkv_bias=False, qk_scale=None, drop=0., attn_drop=0.,\n drop_path=0., act_layer=nn.GELU, norm_layer=nn.LayerNorm, Attention_block = Class_Attention,\n Mlp_block=Mlp,init_values=1e-4):\n super().__init__()\n self.norm1 = norm_layer(dim)\n self.attn = Attention_block(\n dim, num_heads=num_heads, qkv_bias=qkv_bias, qk_scale=qk_scale, attn_drop=attn_drop, proj_drop=drop)\n self.drop_path = DropPath(drop_path) if drop_path > 0. else nn.Identity()\n self.norm2 = norm_layer(dim)\n mlp_hidden_dim = int(dim * mlp_ratio)\n self.mlp = Mlp_block(in_features=dim, hidden_features=mlp_hidden_dim, act_layer=act_layer, drop=drop)\n self.gamma_1 = nn.Parameter(init_values * torch.ones((dim)),requires_grad=True)\n self.gamma_2 = nn.Parameter(init_values * torch.ones((dim)),requires_grad=True)\n\n \n def forward(self, x, x_cls):\n \n u = torch.cat((x_cls,x),dim=1)\n \n \n x_cls = x_cls + self.drop_path(self.gamma_1 * self.attn(self.norm1(u)))\n \n x_cls = x_cls + self.drop_path(self.gamma_2 * self.mlp(self.norm2(x_cls)))\n \n return x_cls \n \n \nclass Attention_talking_head(nn.Module):\n # taken from https://github.com/rwightman/pytorch-image-models/blob/master/timm/models/vision_transformer.py\n # with slight modifications to add Talking Heads Attention (https://arxiv.org/pdf/2003.02436v1.pdf)\n def __init__(self, dim, num_heads=8, qkv_bias=False, qk_scale=None, attn_drop=0., proj_drop=0.):\n super().__init__()\n \n self.num_heads = num_heads\n \n head_dim = dim // num_heads\n \n self.scale = qk_scale or head_dim ** -0.5\n\n self.qkv = nn.Linear(dim, dim * 3, bias=qkv_bias)\n self.attn_drop = nn.Dropout(attn_drop)\n \n self.proj = nn.Linear(dim, dim)\n \n self.proj_l = nn.Linear(num_heads, num_heads)\n self.proj_w = nn.Linear(num_heads, num_heads)\n \n self.proj_drop = nn.Dropout(proj_drop)\n\n\n \n def forward(self, x):\n B, N, C = x.shape\n qkv = self.qkv(x).reshape(B, N, 3, self.num_heads, C // self.num_heads).permute(2, 0, 3, 1, 4)\n q, k, v = qkv[0] * self.scale , qkv[1], qkv[2] \n \n attn = (q @ k.transpose(-2, -1)) \n \n attn = self.proj_l(attn.permute(0,2,3,1)).permute(0,3,1,2)\n \n attn = attn.softmax(dim=-1)\n \n attn = self.proj_w(attn.permute(0,2,3,1)).permute(0,3,1,2)\n attn = self.attn_drop(attn)\n\n x = (attn @ v).transpose(1, 2).reshape(B, N, C)\n x = self.proj(x)\n x = self.proj_drop(x)\n return x\n \nclass LayerScale_Block(nn.Module):\n # taken from https://github.com/rwightman/pytorch-image-models/blob/master/timm/models/vision_transformer.py\n # with slight modifications to add layerScale\n def __init__(self, dim, num_heads, mlp_ratio=4., qkv_bias=False, qk_scale=None, drop=0., attn_drop=0.,\n drop_path=0., act_layer=nn.GELU, norm_layer=nn.LayerNorm,Attention_block = Attention_talking_head,\n Mlp_block=Mlp,init_values=1e-4):\n super().__init__()\n self.norm1 = norm_layer(dim)\n self.attn = Attention_block(\n dim, num_heads=num_heads, qkv_bias=qkv_bias, qk_scale=qk_scale, attn_drop=attn_drop, proj_drop=drop)\n self.drop_path = DropPath(drop_path) if drop_path > 0. else nn.Identity()\n self.norm2 = norm_layer(dim)\n mlp_hidden_dim = int(dim * mlp_ratio)\n self.mlp = Mlp_block(in_features=dim, hidden_features=mlp_hidden_dim, act_layer=act_layer, drop=drop)\n self.gamma_1 = nn.Parameter(init_values * torch.ones((dim)),requires_grad=True)\n self.gamma_2 = nn.Parameter(init_values * torch.ones((dim)),requires_grad=True)\n\n def forward(self, x): \n x = x + self.drop_path(self.gamma_1 * self.attn(self.norm1(x)))\n x = x + self.drop_path(self.gamma_2 * self.mlp(self.norm2(x)))\n return x \n \n \n \n \nclass cait_models(nn.Module):\n # taken from https://github.com/rwightman/pytorch-image-models/blob/master/timm/models/vision_transformer.py\n # with slight modifications to adapt to our cait models\n def __init__(self, img_size=224, patch_size=16, in_chans=3, num_classes=1000, embed_dim=768, depth=12,\n num_heads=12, mlp_ratio=4., qkv_bias=False, qk_scale=None, drop_rate=0., attn_drop_rate=0.,\n drop_path_rate=0., norm_layer=nn.LayerNorm, global_pool=None,\n block_layers = LayerScale_Block,\n block_layers_token = LayerScale_Block_CA,\n Patch_layer=PatchEmbed,act_layer=nn.GELU,\n Attention_block = Attention_talking_head,Mlp_block=Mlp,\n init_scale=1e-4,\n Attention_block_token_only=Class_Attention,\n Mlp_block_token_only= Mlp, \n depth_token_only=2,\n mlp_ratio_clstk = 4.0):\n super().__init__()\n \n\n \n self.num_classes = num_classes\n self.num_features = self.embed_dim = embed_dim \n\n self.patch_embed = Patch_layer(\n img_size=img_size, patch_size=patch_size, in_chans=in_chans, embed_dim=embed_dim)\n \n num_patches = self.patch_embed.num_patches\n\n self.cls_token = nn.Parameter(torch.zeros(1, 1, embed_dim))\n self.pos_embed = nn.Parameter(torch.zeros(1, num_patches, embed_dim))\n self.pos_drop = nn.Dropout(p=drop_rate)\n\n dpr = [drop_path_rate for i in range(depth)] \n self.blocks = nn.ModuleList([\n block_layers(\n dim=embed_dim, num_heads=num_heads, mlp_ratio=mlp_ratio, qkv_bias=qkv_bias, qk_scale=qk_scale,\n drop=drop_rate, attn_drop=attn_drop_rate, drop_path=dpr[i], norm_layer=norm_layer,\n act_layer=act_layer,Attention_block=Attention_block,Mlp_block=Mlp_block,init_values=init_scale)\n for i in range(depth)])\n \n\n self.blocks_token_only = nn.ModuleList([\n block_layers_token(\n dim=embed_dim, num_heads=num_heads, mlp_ratio=mlp_ratio_clstk, qkv_bias=qkv_bias, qk_scale=qk_scale,\n drop=0.0, attn_drop=0.0, drop_path=0.0, norm_layer=norm_layer,\n act_layer=act_layer,Attention_block=Attention_block_token_only,\n Mlp_block=Mlp_block_token_only,init_values=init_scale)\n for i in range(depth_token_only)])\n \n self.norm = norm_layer(embed_dim)\n\n\n self.feature_info = [dict(num_chs=embed_dim, reduction=0, module='head')]\n self.head = nn.Linear(embed_dim, num_classes) if num_classes > 0 else nn.Identity()\n\n trunc_normal_(self.pos_embed, std=.02)\n trunc_normal_(self.cls_token, std=.02)\n self.apply(self._init_weights)\n\n def _init_weights(self, m):\n if isinstance(m, nn.Linear):\n trunc_normal_(m.weight, std=.02)\n if isinstance(m, nn.Linear) and m.bias is not None:\n nn.init.constant_(m.bias, 0)\n elif isinstance(m, nn.LayerNorm):\n nn.init.constant_(m.bias, 0)\n nn.init.constant_(m.weight, 1.0)\n\n @torch.jit.ignore\n def no_weight_decay(self):\n return {'pos_embed', 'cls_token'}\n\n\n def forward_features(self, x):\n B = x.shape[0]\n x = self.patch_embed(x)\n\n cls_tokens = self.cls_token.expand(B, -1, -1) \n \n x = x + self.pos_embed\n x = self.pos_drop(x)\n\n for i , blk in enumerate(self.blocks):\n x = blk(x)\n \n for i , blk in enumerate(self.blocks_token_only):\n cls_tokens = blk(x,cls_tokens)\n\n x = torch.cat((cls_tokens, x), dim=1)\n \n \n x = self.norm(x)\n return x[:, 0]\n\n def forward(self, x):\n x = self.forward_features(x)\n \n x = self.head(x)\n\n return x \n \n@register_model\ndef cait_XXS24_224(pretrained=False, **kwargs):\n model = cait_models(\n img_size= 224,patch_size=16, embed_dim=192, depth=24, num_heads=4, mlp_ratio=4, qkv_bias=True,\n norm_layer=partial(nn.LayerNorm, eps=1e-6),\n init_scale=1e-5,\n depth_token_only=2,**kwargs)\n \n model.default_cfg = _cfg()\n if pretrained:\n checkpoint = torch.hub.load_state_dict_from_url(\n url=\"https://dl.fbaipublicfiles.com/deit/XXS24_224.pth\",\n map_location=\"cpu\", check_hash=True\n )\n checkpoint_no_module = {}\n for k in model.state_dict().keys():\n checkpoint_no_module[k] = checkpoint[\"model\"]['module.'+k]\n \n model.load_state_dict(checkpoint_no_module)\n \n return model \n\n@register_model\ndef cait_XXS24(pretrained=False, **kwargs):\n model = cait_models(\n img_size= 384,patch_size=16, embed_dim=192, depth=24, num_heads=4, mlp_ratio=4, qkv_bias=True,\n norm_layer=partial(nn.LayerNorm, eps=1e-6),\n init_scale=1e-5,\n depth_token_only=2,**kwargs)\n \n model.default_cfg = _cfg()\n if pretrained:\n checkpoint = torch.hub.load_state_dict_from_url(\n url=\"https://dl.fbaipublicfiles.com/deit/XXS24_384.pth\",\n map_location=\"cpu\", check_hash=True\n )\n checkpoint_no_module = {}\n for k in model.state_dict().keys():\n checkpoint_no_module[k] = checkpoint[\"model\"]['module.'+k]\n \n model.load_state_dict(checkpoint_no_module)\n \n return model \n@register_model\ndef cait_XXS36_224(pretrained=False, **kwargs):\n model = cait_models(\n img_size= 224,patch_size=16, embed_dim=192, depth=36, num_heads=4, mlp_ratio=4, qkv_bias=True,\n norm_layer=partial(nn.LayerNorm, eps=1e-6),\n init_scale=1e-5,\n depth_token_only=2,**kwargs)\n \n model.default_cfg = _cfg()\n if pretrained:\n checkpoint = torch.hub.load_state_dict_from_url(\n url=\"https://dl.fbaipublicfiles.com/deit/XXS36_224.pth\",\n map_location=\"cpu\", check_hash=True\n )\n checkpoint_no_module = {}\n for k in model.state_dict().keys():\n checkpoint_no_module[k] = checkpoint[\"model\"]['module.'+k]\n \n model.load_state_dict(checkpoint_no_module)\n \n return model \n\n@register_model\ndef cait_XXS36(pretrained=False, **kwargs):\n model = cait_models(\n img_size= 384,patch_size=16, embed_dim=192, depth=36, num_heads=4, mlp_ratio=4, qkv_bias=True,\n norm_layer=partial(nn.LayerNorm, eps=1e-6),\n init_scale=1e-5,\n depth_token_only=2,**kwargs)\n \n model.default_cfg = _cfg()\n if pretrained:\n checkpoint = torch.hub.load_state_dict_from_url(\n url=\"https://dl.fbaipublicfiles.com/deit/XXS36_384.pth\",\n map_location=\"cpu\", check_hash=True\n )\n checkpoint_no_module = {}\n for k in model.state_dict().keys():\n checkpoint_no_module[k] = checkpoint[\"model\"]['module.'+k]\n \n model.load_state_dict(checkpoint_no_module)\n \n return model \n\n@register_model\ndef cait_XS24(pretrained=False, **kwargs):\n model = cait_models(\n img_size= 384,patch_size=16, embed_dim=288, depth=24, num_heads=6, mlp_ratio=4, qkv_bias=True,\n norm_layer=partial(nn.LayerNorm, eps=1e-6),\n init_scale=1e-5,\n depth_token_only=2,**kwargs)\n \n model.default_cfg = _cfg()\n if pretrained:\n checkpoint = torch.hub.load_state_dict_from_url(\n url=\"https://dl.fbaipublicfiles.com/deit/XS24_384.pth\",\n map_location=\"cpu\", check_hash=True\n )\n checkpoint_no_module = {}\n for k in model.state_dict().keys():\n checkpoint_no_module[k] = checkpoint[\"model\"]['module.'+k]\n \n model.load_state_dict(checkpoint_no_module)\n \n return model \n\n\n\n\n@register_model\ndef cait_S24_224(pretrained=False, **kwargs):\n model = cait_models(\n img_size= 224,patch_size=16, embed_dim=384, depth=24, num_heads=8, mlp_ratio=4, qkv_bias=True,\n norm_layer=partial(nn.LayerNorm, eps=1e-6),\n init_scale=1e-5,\n depth_token_only=2,**kwargs)\n \n model.default_cfg = _cfg()\n if pretrained:\n checkpoint = torch.hub.load_state_dict_from_url(\n url=\"https://dl.fbaipublicfiles.com/deit/S24_224.pth\",\n map_location=\"cpu\", check_hash=True\n )\n checkpoint_no_module = {}\n for k in model.state_dict().keys():\n checkpoint_no_module[k] = checkpoint[\"model\"]['module.'+k]\n \n model.load_state_dict(checkpoint_no_module)\n \n return model \n\n@register_model\ndef cait_S24(pretrained=False, **kwargs):\n model = cait_models(\n img_size= 384,patch_size=16, embed_dim=384, depth=24, num_heads=8, mlp_ratio=4, qkv_bias=True,\n norm_layer=partial(nn.LayerNorm, eps=1e-6),\n init_scale=1e-5,\n depth_token_only=2,**kwargs)\n \n model.default_cfg = _cfg()\n if pretrained:\n checkpoint = torch.hub.load_state_dict_from_url(\n url=\"https://dl.fbaipublicfiles.com/deit/S24_384.pth\",\n map_location=\"cpu\", check_hash=True\n )\n checkpoint_no_module = {}\n for k in model.state_dict().keys():\n checkpoint_no_module[k] = checkpoint[\"model\"]['module.'+k]\n \n model.load_state_dict(checkpoint_no_module)\n \n return model \n\n@register_model\ndef cait_S36(pretrained=False, **kwargs):\n model = cait_models(\n img_size= 384,patch_size=16, embed_dim=384, depth=36, num_heads=8, mlp_ratio=4, qkv_bias=True,\n norm_layer=partial(nn.LayerNorm, eps=1e-6),\n init_scale=1e-6,\n depth_token_only=2,**kwargs)\n \n model.default_cfg = _cfg()\n if pretrained:\n checkpoint = torch.hub.load_state_dict_from_url(\n url=\"https://dl.fbaipublicfiles.com/deit/S36_384.pth\",\n map_location=\"cpu\", check_hash=True\n )\n checkpoint_no_module = {}\n for k in model.state_dict().keys():\n checkpoint_no_module[k] = checkpoint[\"model\"]['module.'+k]\n \n model.load_state_dict(checkpoint_no_module)\n\n return model \n\n\n\n\n\n@register_model\ndef cait_M36(pretrained=False, **kwargs):\n model = cait_models(\n img_size= 384, patch_size=16, embed_dim=768, depth=36, num_heads=16, mlp_ratio=4, qkv_bias=True,\n norm_layer=partial(nn.LayerNorm, eps=1e-6),\n init_scale=1e-6,\n depth_token_only=2,**kwargs)\n \n model.default_cfg = _cfg()\n if pretrained:\n checkpoint = torch.hub.load_state_dict_from_url(\n url=\"https://dl.fbaipublicfiles.com/deit/M36_384.pth\",\n map_location=\"cpu\", check_hash=True\n )\n checkpoint_no_module = {}\n for k in model.state_dict().keys():\n checkpoint_no_module[k] = checkpoint[\"model\"]['module.'+k]\n \n model.load_state_dict(checkpoint_no_module)\n\n return model \n\n\n@register_model\ndef cait_M48(pretrained=False, **kwargs):\n model = cait_models(\n img_size= 448 , patch_size=16, embed_dim=768, depth=48, num_heads=16, mlp_ratio=4, qkv_bias=True,\n norm_layer=partial(nn.LayerNorm, eps=1e-6),\n init_scale=1e-6,\n depth_token_only=2,**kwargs)\n \n model.default_cfg = _cfg()\n if pretrained:\n checkpoint = torch.hub.load_state_dict_from_url(\n url=\"https://dl.fbaipublicfiles.com/deit/M48_448.pth\",\n map_location=\"cpu\", check_hash=True\n )\n checkpoint_no_module = {}\n for k in model.state_dict().keys():\n checkpoint_no_module[k] = checkpoint[\"model\"]['module.'+k]\n \n model.load_state_dict(checkpoint_no_module)\n \n return model \n"
},
{
"path": "batchformer-v2/deit_share/datasets.py",
"content": "# Copyright (c) 2015-present, Facebook, Inc.\n# All rights reserved.\nimport os\nimport json\n\nfrom torchvision import datasets, transforms\nfrom torchvision.datasets.folder import ImageFolder, default_loader\n\nfrom timm.data.constants import IMAGENET_DEFAULT_MEAN, IMAGENET_DEFAULT_STD\nfrom timm.data import create_transform\n\n\nclass INatDataset(ImageFolder):\n def __init__(self, root, train=True, year=2018, transform=None, target_transform=None,\n category='name', loader=default_loader):\n self.transform = transform\n self.loader = loader\n self.target_transform = target_transform\n self.year = year\n # assert category in ['kingdom','phylum','class','order','supercategory','family','genus','name']\n path_json = os.path.join(root, f'{\"train\" if train else \"val\"}{year}.json')\n with open(path_json) as json_file:\n data = json.load(json_file)\n\n with open(os.path.join(root, 'categories.json')) as json_file:\n data_catg = json.load(json_file)\n\n path_json_for_targeter = os.path.join(root, f\"train{year}.json\")\n\n with open(path_json_for_targeter) as json_file:\n data_for_targeter = json.load(json_file)\n\n targeter = {}\n indexer = 0\n for elem in data_for_targeter['annotations']:\n king = []\n king.append(data_catg[int(elem['category_id'])][category])\n if king[0] not in targeter.keys():\n targeter[king[0]] = indexer\n indexer += 1\n self.nb_classes = len(targeter)\n\n self.samples = []\n for elem in data['images']:\n cut = elem['file_name'].split('/')\n target_current = int(cut[2])\n path_current = os.path.join(root, cut[0], cut[2], cut[3])\n\n categors = data_catg[target_current]\n target_current_true = targeter[categors[category]]\n self.samples.append((path_current, target_current_true))\n\n # __getitem__ and __len__ inherited from ImageFolder\n\n\ndef build_dataset(is_train, args):\n transform = build_transform(is_train, args)\n\n if args.data_set == 'CIFAR':\n dataset = datasets.CIFAR100(args.data_path, train=is_train, transform=transform, download=True)\n nb_classes = 100\n elif args.data_set == 'IMNET':\n root = os.path.join(args.data_path, 'train' if is_train else 'val')\n dataset = datasets.ImageFolder(root, transform=transform)\n nb_classes = len(dataset.classes)\n elif args.data_set == 'INAT':\n dataset = INatDataset(args.data_path, train=is_train, year=2018,\n category=args.inat_category, transform=transform)\n nb_classes = dataset.nb_classes\n elif args.data_set == 'INAT19':\n dataset = INatDataset(args.data_path, train=is_train, year=2019,\n category=args.inat_category, transform=transform)\n nb_classes = dataset.nb_classes\n\n return dataset, nb_classes\n\n\ndef build_transform(is_train, args):\n resize_im = args.input_size > 32\n if is_train:\n # this should always dispatch to transforms_imagenet_train\n transform = create_transform(\n input_size=args.input_size,\n is_training=True,\n color_jitter=args.color_jitter,\n auto_augment=args.aa,\n interpolation=args.train_interpolation,\n re_prob=args.reprob,\n re_mode=args.remode,\n re_count=args.recount,\n )\n if not resize_im:\n # replace RandomResizedCropAndInterpolation with\n # RandomCrop\n transform.transforms[0] = transforms.RandomCrop(\n args.input_size, padding=4)\n return transform\n\n t = []\n if resize_im:\n size = int((256 / 224) * args.input_size)\n t.append(\n transforms.Resize(size, interpolation=3), # to maintain same ratio w.r.t. 224 images\n )\n t.append(transforms.CenterCrop(args.input_size))\n\n t.append(transforms.ToTensor())\n t.append(transforms.Normalize(IMAGENET_DEFAULT_MEAN, IMAGENET_DEFAULT_STD))\n return transforms.Compose(t)\n"
},
{
"path": "batchformer-v2/deit_share/engine.py",
"content": "# Copyright (c) 2015-present, Facebook, Inc.\n# All rights reserved.\n\"\"\"\nTrain and eval functions used in main.py\n\"\"\"\nimport math\nimport os\nimport sys\nfrom typing import Iterable, Optional\n\nimport torch\n\nfrom timm.data import Mixup\nfrom timm.utils import accuracy, ModelEma\n\nfrom losses import DistillationLoss\nimport utils\n\n\ndef train_one_epoch(model: torch.nn.Module, criterion: DistillationLoss,\n data_loader: Iterable, optimizer: torch.optim.Optimizer,\n device: torch.device, epoch: int, loss_scaler, max_norm: float = 0,\n model_ema: Optional[ModelEma] = None, mixup_fn: Optional[Mixup] = None,\n set_training_mode=True, no_fp16_bt=0,\n model_without_ddp=None, num_classes=1000):\n \n metric_logger = utils.MetricLogger(delimiter=\" \")\n metric_logger.add_meter('lr', utils.SmoothedValue(window_size=1, fmt='{value:.6f}'))\n header = 'Epoch: [{}]'.format(epoch)\n print_freq = 10\n for samples, targets in metric_logger.log_every(data_loader, print_freq, header):\n samples = samples.to(device, non_blocking=True)\n targets = targets.to(device, non_blocking=True)\n\n if mixup_fn is not None:\n samples, targets = mixup_fn(samples, targets)\n\n if no_fp16_bt in [4]:\n # remove fp16\n outputs = model(samples)\n real_loss = criterion(samples[:len(targets)], outputs[:len(targets)], targets[:len(targets)])\n while len(outputs) > len(targets):\n targets = torch.cat([targets, targets], dim=0)\n loss = criterion(samples, outputs, targets)\n else:\n with torch.cuda.amp.autocast():\n outputs = model(samples)\n real_loss = criterion(samples[:len(targets)], outputs[:len(targets)], targets[:len(targets)])\n while len(outputs) > len(targets):\n targets = torch.cat([targets, targets], dim=0)\n loss = criterion(samples, outputs, targets)\n\n loss_value = loss.item()\n real_loss_value = real_loss.item()\n\n if not math.isfinite(loss_value):\n print(loss_value)\n # open(\"/checkpoints/debug.log\", 'a').write('loss nan').close()\n print(\"Loss is {}, stopping training\".format(loss_value))\n # os.system(\"ps -ux | grep slurm | awk '{print $2}' | xargs kill\")\n # sys.exit(1)\n optimizer.zero_grad()\n continue\n\n # continue\n\n optimizer.zero_grad()\n\n # this attribute is added by timm on one optimizer (adahessian)\n is_second_order = hasattr(optimizer, 'is_second_order') and optimizer.is_second_order\n if no_fp16_bt in [3, 4]:\n loss.backward()\n optimizer.step()\n else:\n loss_scaler(loss, optimizer, clip_grad=max_norm,\n parameters=model.parameters(), create_graph=is_second_order)\n\n torch.cuda.synchronize()\n if model_ema is not None:\n model_ema.update(model)\n\n metric_logger.update(loss=loss_value)\n metric_logger.update(rloss=real_loss_value)\n metric_logger.update(lr=optimizer.param_groups[0][\"lr\"])\n # gather the stats from all processes\n metric_logger.synchronize_between_processes()\n print(\"Averaged stats:\", metric_logger)\n return {k: meter.global_avg for k, meter in metric_logger.meters.items()}\n\n\n@torch.no_grad()\ndef evaluate(data_loader, model, device):\n criterion = torch.nn.CrossEntropyLoss()\n\n metric_logger = utils.MetricLogger(delimiter=\" \")\n header = 'Test:'\n\n # switch to evaluation mode\n model.eval()\n\n for images, target in metric_logger.log_every(data_loader, 10, header):\n images = images.to(device, non_blocking=True)\n target = target.to(device, non_blocking=True)\n\n # compute output\n with torch.cuda.amp.autocast():\n output = model(images)\n if len(output) > len(target):\n output = (output[:len(target)] + output[len(target):])\n # output = output[:len(target)]\n # output = output[len(target):]\n loss = criterion(output, target)\n\n acc1, acc5 = accuracy(output, target, topk=(1, 5))\n\n batch_size = images.shape[0]\n metric_logger.update(loss=loss.item())\n metric_logger.meters['acc1'].update(acc1.item(), n=batch_size)\n metric_logger.meters['acc5'].update(acc5.item(), n=batch_size)\n # gather the stats from all processes\n metric_logger.synchronize_between_processes()\n print('* Acc@1 {top1.global_avg:.3f} Acc@5 {top5.global_avg:.3f} loss {losses.global_avg:.3f}'\n .format(top1=metric_logger.acc1, top5=metric_logger.acc5, losses=metric_logger.loss))\n\n return {k: meter.global_avg for k, meter in metric_logger.meters.items()}\n\ndef reduce_tensor(acc1, args):\n import torch.distributed as dist\n rt = acc1.clone()\n dist.all_reduce(rt, op=dist.ReduceOp.SUM)\n rt = rt / args.world_size\n return rt\n\ndef validate(val_loader, training_labels, model, device, num_classes, rank):\n\n # switch to evaluate mode\n model.eval()\n total_logits = torch.empty((0, num_classes)).cuda()\n total_labels = torch.empty(0, dtype=torch.long).cuda()\n\n with torch.no_grad():\n for i, (images, target) in enumerate(val_loader):\n images = images.to(device, non_blocking=True)\n target = target.to(device, non_blocking=True)\n\n # compute output\n output = model(images)\n\n total_logits = torch.cat((total_logits, output))\n total_labels = torch.cat((total_labels, target))\n\n # measure accuracy and record loss\n acc1, acc5 = accuracy(output, target, topk=(1, 5))\n # if args.wor\n\n import torch.nn.functional as F\n probs, preds = F.softmax(total_logits.detach(), dim=1).max(dim=1)\n\n if rank <= 0:\n many_acc_top1, median_acc_top1, low_acc_top1, cls_accs = shot_acc(preds, total_labels, training_labels, acc_per_cls=True)\n print('Many_acc: %.5f, Medium_acc: %.5f Low_acc: %.5f'%(many_acc_top1, median_acc_top1, low_acc_top1))\n\n\ndef shot_acc(preds, labels, train_data, many_shot_thr=100, low_shot_thr=20, acc_per_cls=False):\n import numpy as np\n if isinstance(train_data, np.ndarray):\n training_labels = np.array(train_data).astype(int)\n else:\n training_labels = np.array(train_data.dataset.labels).astype(int)\n\n if isinstance(preds, torch.Tensor):\n preds = preds.detach().cpu().numpy()\n labels = labels.detach().cpu().numpy()\n elif isinstance(preds, np.ndarray):\n pass\n else:\n raise TypeError('Type ({}) of preds not supported'.format(type(preds)))\n train_class_count = []\n test_class_count = []\n class_correct = []\n for l in np.unique(labels):\n train_class_count.append(len(training_labels[training_labels == l]))\n test_class_count.append(len(labels[labels == l]))\n class_correct.append((preds[labels == l] == labels[labels == l]).sum())\n\n many_shot = []\n median_shot = []\n low_shot = []\n for i in range(len(train_class_count)):\n if train_class_count[i] > many_shot_thr:\n many_shot.append((class_correct[i] / test_class_count[i]))\n elif train_class_count[i] < low_shot_thr:\n low_shot.append((class_correct[i] / test_class_count[i]))\n else:\n median_shot.append((class_correct[i] / test_class_count[i]))\n\n if len(many_shot) == 0:\n many_shot.append(0)\n if len(median_shot) == 0:\n median_shot.append(0)\n if len(low_shot) == 0:\n low_shot.append(0)\n\n if acc_per_cls:\n class_accs = [c / cnt for c, cnt in zip(class_correct, test_class_count)]\n return np.mean(many_shot), np.mean(median_shot), np.mean(low_shot), class_accs\n else:\n return np.mean(many_shot), np.mean(median_shot), np.mean(low_shot)\n"
},
{
"path": "batchformer-v2/deit_share/gpu.pbs",
"content": "#!/bin/bash\n#PBS -P ZHIHOU\n#PBS -l select=4:ngpus=1:ncpus=6:mem=24GB\n#PBS -l walltime=72:00:00\n#PBS -q alloc-dt\nsource activate pyt\n\n#env\n\ncd \"$PBS_O_WORKDIR\"\n\n"
},
{
"path": "batchformer-v2/deit_share/hubconf.py",
"content": "# Copyright (c) 2015-present, Facebook, Inc.\n# All rights reserved.\nfrom models import *\n\ndependencies = [\"torch\", \"torchvision\", \"timm\"]\n"
},
{
"path": "batchformer-v2/deit_share/losses.py",
"content": "# Copyright (c) 2015-present, Facebook, Inc.\n# All rights reserved.\n\"\"\"\nImplements the knowledge distillation loss\n\"\"\"\nimport torch\nfrom torch.nn import functional as F\nimport torch\nfrom torch.nn.modules.loss import _Loss\nimport torch.nn.functional as F\nimport json\n\nclass DistillationLoss(torch.nn.Module):\n \"\"\"\n This module wraps a standard criterion and adds an extra knowledge distillation loss by\n taking a teacher model prediction and using it as additional supervision.\n \"\"\"\n def __init__(self, base_criterion: torch.nn.Module, teacher_model: torch.nn.Module,\n distillation_type: str, alpha: float, tau: float):\n super().__init__()\n self.base_criterion = base_criterion\n self.teacher_model = teacher_model\n assert distillation_type in ['none', 'soft', 'hard']\n self.distillation_type = distillation_type\n self.alpha = alpha\n self.tau = tau\n\n def forward(self, inputs, outputs, labels):\n \"\"\"\n Args:\n inputs: The original inputs that are feed to the teacher model\n outputs: the outputs of the model to be trained. It is expected to be\n either a Tensor, or a Tuple[Tensor, Tensor], with the original output\n in the first position and the distillation predictions as the second output\n labels: the labels for the base criterion\n \"\"\"\n outputs_kd = None\n if not isinstance(outputs, torch.Tensor):\n # assume that the model outputs a tuple of [outputs, outputs_kd]\n outputs, outputs_kd = outputs\n base_loss = self.base_criterion(outputs, labels)\n if self.distillation_type == 'none':\n return base_loss\n\n if outputs_kd is None:\n raise ValueError(\"When knowledge distillation is enabled, the model is \"\n \"expected to return a Tuple[Tensor, Tensor] with the output of the \"\n \"class_token and the dist_token\")\n # don't backprop throught the teacher\n with torch.no_grad():\n teacher_outputs = self.teacher_model(inputs)\n\n if self.distillation_type == 'soft':\n T = self.tau\n # taken from https://github.com/peterliht/knowledge-distillation-pytorch/blob/master/model/net.py#L100\n # with slight modifications\n distillation_loss = F.kl_div(\n F.log_softmax(outputs_kd / T, dim=1),\n #We provide the teacher's targets in log probability because we use log_target=True \n #(as recommended in pytorch https://github.com/pytorch/pytorch/blob/9324181d0ac7b4f7949a574dbc3e8be30abe7041/torch/nn/functional.py#L2719)\n #but it is possible to give just the probabilities and set log_target=False. In our experiments we tried both.\n F.log_softmax(teacher_outputs / T, dim=1),\n reduction='sum',\n log_target=True\n ) * (T * T) / outputs_kd.numel()\n #We divide by outputs_kd.numel() to have the legacy PyTorch behavior. \n #But we also experiments output_kd.size(0) \n #see issue 61(https://github.com/facebookresearch/deit/issues/61) for more details\n elif self.distillation_type == 'hard':\n distillation_loss = F.cross_entropy(outputs_kd, teacher_outputs.argmax(dim=1))\n\n loss = base_loss * (1 - self.alpha) + distillation_loss * self.alpha\n return loss\n\n\nclass BalancedSoftmax(_Loss):\n \"\"\"\n Balanced Softmax Loss\n \"\"\"\n def __init__(self, freq_path):\n super(BalancedSoftmax, self).__init__()\n with open(freq_path, 'r') as fd:\n freq = json.load(fd)\n freq = torch.tensor(freq)\n self.sample_per_class = freq\n\n def forward(self, inputs, outputs, labels, reduction='mean'):\n return balanced_softmax_loss(labels, outputs, self.sample_per_class, reduction)\n\n\ndef balanced_softmax_loss(labels, logits, sample_per_class, reduction):\n \"\"\"Compute the Balanced Softmax Loss between `logits` and the ground truth `labels`.\n Args:\n labels: A int tensor of size [batch].\n logits: A float tensor of size [batch, no_of_classes].\n sample_per_class: A int tensor of size [no of classes].\n reduction: string. One of \"none\", \"mean\", \"sum\"\n Returns:\n loss: A float tensor. Balanced Softmax Loss.\n \"\"\"\n spc = sample_per_class.type_as(logits)\n spc = spc.unsqueeze(0).expand(logits.shape[0], -1)\n logits = logits + spc.log()\n loss = F.binary_cross_entropy_with_logits(input=logits, target=labels, reduction=reduction)\n return loss\n\n\ndef create_loss(freq_path):\n print('Loading Balanced Softmax Loss.')\n return BalancedSoftmax(freq_path)"
},
{
"path": "batchformer-v2/deit_share/main.py",
"content": "# Copyright (c) 2015-present, Facebook, Inc.\n# All rights reserved.\nimport argparse\nimport datetime\nimport numpy as np\nimport time\nimport torch\nimport torch.backends.cudnn as cudnn\nimport json\n\nfrom pathlib import Path\n\nfrom timm.data import Mixup\nfrom timm.models import create_model\nfrom timm.loss import LabelSmoothingCrossEntropy, SoftTargetCrossEntropy\nfrom timm.scheduler import create_scheduler\nfrom timm.optim import create_optimizer\nfrom timm.utils import NativeScaler, get_state_dict, ModelEma\n\n\nfrom datasets import build_dataset\nfrom engine import train_one_epoch, evaluate\nfrom losses import DistillationLoss\nfrom samplers import RASampler\nimport models\nimport utils\n\n\ndef get_args_parser():\n parser = argparse.ArgumentParser('DeiT training and evaluation script', add_help=False)\n parser.add_argument('--batch-size', default=64, type=int)\n parser.add_argument('--epochs', default=300, type=int)\n\n # Model parameters\n parser.add_argument('--model', default='deit_small_patch16_224', type=str, metavar='MODEL',\n help='Name of model to train')\n parser.add_argument('--input-size', default=224, type=int, help='images input size')\n\n parser.add_argument('--drop', type=float, default=0.0, metavar='PCT',\n help='Dropout rate (default: 0.)')\n parser.add_argument('--drop-path', type=float, default=0.1, metavar='PCT',\n help='Drop path rate (default: 0.1)')\n\n parser.add_argument('--model-ema', action='store_true')\n parser.add_argument('--no-model-ema', action='store_false', dest='model_ema')\n parser.set_defaults(model_ema=True)\n parser.add_argument('--model-ema-decay', type=float, default=0.99996, help='')\n parser.add_argument('--model-ema-force-cpu', action='store_true', default=False, help='')\n\n # Optimizer parameters\n parser.add_argument('--opt', default='adamw', type=str, metavar='OPTIMIZER',\n help='Optimizer (default: \"adamw\"')\n parser.add_argument('--opt-eps', default=1e-8, type=float, metavar='EPSILON',\n help='Optimizer Epsilon (default: 1e-8)')\n parser.add_argument('--opt-betas', default=None, type=float, nargs='+', metavar='BETA',\n help='Optimizer Betas (default: None, use opt default)')\n parser.add_argument('--clip-grad', type=float, default=None, metavar='NORM',\n help='Clip gradient norm (default: None, no clipping)')\n parser.add_argument('--momentum', type=float, default=0.9, metavar='M',\n help='SGD momentum (default: 0.9)')\n parser.add_argument('--weight-decay', type=float, default=0.05,\n help='weight decay (default: 0.05)')\n # Learning rate schedule parameters\n parser.add_argument('--sched', default='cosine', type=str, metavar='SCHEDULER',\n help='LR scheduler (default: \"cosine\"')\n parser.add_argument('--lr', type=float, default=5e-4, metavar='LR',\n help='learning rate (default: 5e-4)')\n parser.add_argument('--lr-noise', type=float, nargs='+', default=None, metavar='pct, pct',\n help='learning rate noise on/off epoch percentages')\n parser.add_argument('--lr-noise-pct', type=float, default=0.67, metavar='PERCENT',\n help='learning rate noise limit percent (default: 0.67)')\n parser.add_argument('--lr-noise-std', type=float, default=1.0, metavar='STDDEV',\n help='learning rate noise std-dev (default: 1.0)')\n parser.add_argument('--warmup-lr', type=float, default=1e-6, metavar='LR',\n help='warmup learning rate (default: 1e-6)')\n parser.add_argument('--min-lr', type=float, default=1e-5, metavar='LR',\n help='lower lr bound for cyclic schedulers that hit 0 (1e-5)')\n\n parser.add_argument('--decay-epochs', type=float, default=30, metavar='N',\n help='epoch interval to decay LR')\n parser.add_argument('--warmup-epochs', type=int, default=5, metavar='N',\n help='epochs to warmup LR, if scheduler supports')\n parser.add_argument('--cooldown-epochs', type=int, default=10, metavar='N',\n help='epochs to cooldown LR at min_lr, after cyclic schedule ends')\n parser.add_argument('--patience-epochs', type=int, default=10, metavar='N',\n help='patience epochs for Plateau LR scheduler (default: 10')\n parser.add_argument('--decay-rate', '--dr', type=float, default=0.1, metavar='RATE',\n help='LR decay rate (default: 0.1)')\n\n # Augmentation parameters\n parser.add_argument('--color-jitter', type=float, default=0.4, metavar='PCT',\n help='Color jitter factor (default: 0.4)')\n parser.add_argument('--aa', type=str, default='rand-m9-mstd0.5-inc1', metavar='NAME',\n help='Use AutoAugment policy. \"v0\" or \"original\". \" + \\\n \"(default: rand-m9-mstd0.5-inc1)'),\n parser.add_argument('--smoothing', type=float, default=0.1, help='Label smoothing (default: 0.1)')\n parser.add_argument('--train-interpolation', type=str, default='bicubic',\n help='Training interpolation (random, bilinear, bicubic default: \"bicubic\")')\n\n parser.add_argument('--repeated-aug', action='store_true')\n parser.add_argument('--no-repeated-aug', action='store_false', dest='repeated_aug')\n parser.set_defaults(repeated_aug=True)\n\n # * Random Erase params\n parser.add_argument('--reprob', type=float, default=0.25, metavar='PCT',\n help='Random erase prob (default: 0.25)')\n parser.add_argument('--remode', type=str, default='pixel',\n help='Random erase mode (default: \"pixel\")')\n parser.add_argument('--recount', type=int, default=1,\n help='Random erase count (default: 1)')\n parser.add_argument('--resplit', action='store_true', default=False,\n help='Do not random erase first (clean) augmentation split')\n\n # * Mixup params\n parser.add_argument('--mixup', type=float, default=0.8,\n help='mixup alpha, mixup enabled if > 0. (default: 0.8)')\n parser.add_argument('--cutmix', type=float, default=1.0,\n help='cutmix alpha, cutmix enabled if > 0. (default: 1.0)')\n parser.add_argument('--cutmix-minmax', type=float, nargs='+', default=None,\n help='cutmix min/max ratio, overrides alpha and enables cutmix if set (default: None)')\n parser.add_argument('--mixup-prob', type=float, default=1.0,\n help='Probability of performing mixup or cutmix when either/both is enabled')\n parser.add_argument('--mixup-switch-prob', type=float, default=0.5,\n help='Probability of switching to cutmix when both mixup and cutmix enabled')\n parser.add_argument('--mixup-mode', type=str, default='batch',\n help='How to apply mixup/cutmix params. Per \"batch\", \"pair\", or \"elem\"')\n\n # Distillation parameters\n parser.add_argument('--teacher-model', default='regnety_160', type=str, metavar='MODEL',\n help='Name of teacher model to train (default: \"regnety_160\"')\n parser.add_argument('--teacher-path', type=str, default='')\n parser.add_argument('--distillation-type', default='none', choices=['none', 'soft', 'hard'], type=str, help=\"\")\n parser.add_argument('--distillation-alpha', default=0.5, type=float, help=\"\")\n parser.add_argument('--distillation-tau', default=1.0, type=float, help=\"\")\n\n # * Finetuning params\n parser.add_argument('--finetune', default='', help='finetune from checkpoint')\n\n # Dataset parameters\n parser.add_argument('--data-path', default='//project/ZHIHOUDATA/Code/RIDE-LongTailRecognition/data/ImageNet_LT/', type=str,\n help='dataset path')\n parser.add_argument('--data-set', default='IMNET', choices=['CIFAR', 'IMNET', 'INAT', 'INAT19'],\n type=str, help='Image Net dataset path')\n parser.add_argument('--inat-category', default='name',\n choices=['kingdom', 'phylum', 'class', 'order', 'supercategory', 'family', 'genus', 'name'],\n type=str, help='semantic granularity')\n\n parser.add_argument('--output_dir', default='',\n help='path where to save, empty for no saving')\n parser.add_argument('--device', default='cuda',\n help='device to use for training / testing')\n parser.add_argument('--seed', default=0, type=int)\n parser.add_argument('--resume', default='', help='resume from checkpoint')\n parser.add_argument('--start_epoch', default=0, type=int, metavar='N',\n help='start epoch')\n parser.add_argument('--eval', action='store_true', help='Perform evaluation only')\n parser.add_argument('--dist-eval', action='store_true', default=False, help='Enabling distributed evaluation')\n parser.add_argument('--num_workers', default=10, type=int)\n parser.add_argument('--pin-mem', action='store_true',\n help='Pin CPU memory in DataLoader for more efficient (sometimes) transfer to GPU.')\n parser.add_argument('--no-pin-mem', action='store_false', dest='pin_mem',\n help='')\n parser.set_defaults(pin_mem=True)\n\n # distributed training parameters\n parser.add_argument('--world_size', default=1, type=int,\n help='number of distributed processes')\n parser.add_argument('--add_bt', default=0, type=int,\n help='add_bt')\n parser.add_argument('--eval_global', default=0, type=int,\n help='eval_global') # This is for evaluation with a mini-batch\n parser.add_argument('--dropout', default=0., type=float,\n help='dropout')\n parser.add_argument('--exp_id', default=\"\", type=str,\n help='exp_id')\n\n # For BT\n parser.add_argument('--decay_drop_bt', default=0., type=float,\n help='drop_patch')\n parser.add_argument('--drop_patch', default=0., type=float,\n help='drop_patch')\n parser.add_argument('--shuffle_patch', action='store_true', )\n parser.add_argument('--no_fp16_bt', default=0, type=int,\n help='no_fp16_bt')\n parser.add_argument('--num_heads', default=4, type=int,\n help='num_heads')\n parser.add_argument('--start_idx', default=8, type=int,\n help='start_idx: start bt layer index')\n parser.add_argument('--start_bt_epoch', default=0, type=int,\n help='start_bt_epoch: start bt layer index')\n parser.add_argument('--insert_idx', action='append', type=int,\n help='insert idx list')\n parser.add_argument('--all_patches', action='store_true',)\n parser.add_argument('--drop_path_bt', default=0., type=float)\n parser.add_argument('--not_cls_token', action='store_true'),\n parser.add_argument('--cls_token_only', action='store_true'),\n parser.add_argument('--shared_bt', default=1, type=int,),\n parser.add_argument('--empty_bt', default=0, type=int,),\n parser.add_argument('--add_norm_bt', default=0, type=int,), # This is to add BT in the last layer (normalization)\n parser.add_argument('--add_mlp_bt', default=0, type=int,),\n parser.add_argument('--mlp_enc', default=0, type=int,),\n parser.add_argument('--no_grad_bt', default=0, type=int,),\n parser.add_argument('--bt_decay', default=0., type=float,\n help='drop_patch')\n parser.add_argument('--mlp_decay', default=0., type=float,\n help='drop_patch')\n parser.add_argument('--bt_lr', default=0.5, type=float,)\n parser.add_argument('--balanced_softmax', action='store_true',) # This is for the experiments on Long-Tailed Recognition\n parser.add_argument('--bt_atten_drop', default=0.5, type=float)\n\n parser.add_argument('--skip_bt', action='store_true',)\n parser.add_argument('--pretrained', action='store_true',\n help='pretrained')\n parser.add_argument('--dist_url', default='env://', help='url used to set up distributed training')\n return parser\n\ndef main(args):\n # torch.autograd.set_detect_anomaly(True)\n utils.init_distributed_mode(args)\n\n print(args)\n\n if args.distillation_type != 'none' and args.finetune and not args.eval:\n raise NotImplementedError(\"Finetuning with distillation not yet supported\")\n\n device = torch.device(args.device)\n\n # fix the seed for reproducibility\n seed = args.seed + utils.get_rank()\n torch.manual_seed(seed)\n np.random.seed(seed)\n # random.seed(seed)\n\n cudnn.benchmark = True\n\n dataset_train, args.nb_classes = build_dataset(is_train=True, args=args)\n dataset_val, _ = build_dataset(is_train=False, args=args)\n\n if True: # args.distributed:\n num_tasks = utils.get_world_size()\n global_rank = utils.get_rank()\n if args.repeated_aug:\n sampler_train = RASampler(\n dataset_train, num_replicas=num_tasks, rank=global_rank, shuffle=True\n )\n else:\n sampler_train = torch.utils.data.DistributedSampler(\n dataset_train, num_replicas=num_tasks, rank=global_rank, shuffle=True\n )\n if args.dist_eval:\n if len(dataset_val) % num_tasks != 0:\n print('Warning: Enabling distributed evaluation with an eval dataset not divisible by process number. '\n 'This will slightly alter validation results as extra duplicate entries are added to achieve '\n 'equal num of samples per-process.')\n sampler_val = torch.utils.data.DistributedSampler(\n dataset_val, num_replicas=num_tasks, rank=global_rank, shuffle=False)\n else:\n sampler_val = torch.utils.data.SequentialSampler(dataset_val)\n else:\n sampler_train = torch.utils.data.RandomSampler(dataset_train)\n sampler_val = torch.utils.data.SequentialSampler(dataset_val)\n\n data_loader_train = torch.utils.data.DataLoader(\n dataset_train, sampler=sampler_train,\n batch_size=args.batch_size,\n num_workers=args.num_workers,\n pin_memory=args.pin_mem,\n drop_last=True,\n )\n eval_batch_size = int(1.5 * args.batch_size)\n import os\n if os.environ['HOME'].endswith('root'):\n eval_batch_size = int(4 * args.batch_size)\n data_loader_val = torch.utils.data.DataLoader(\n dataset_val, sampler=sampler_val,\n batch_size=eval_batch_size,\n num_workers=args.num_workers,\n pin_memory=args.pin_mem,\n drop_last=False\n )\n\n mixup_fn = None\n mixup_active = args.mixup > 0 or args.cutmix > 0. or args.cutmix_minmax is not None\n if mixup_active:\n mixup_fn = Mixup(\n mixup_alpha=args.mixup, cutmix_alpha=args.cutmix, cutmix_minmax=args.cutmix_minmax,\n prob=args.mixup_prob, switch_prob=args.mixup_switch_prob, mode=args.mixup_mode,\n label_smoothing=args.smoothing, num_classes=args.nb_classes)\n\n print(f\"Creating model: {args.model}\")\n model = create_model(\n args.model,\n pretrained=args.pretrained,\n num_classes=args.nb_classes,\n drop_rate=args.drop,\n drop_path_rate=args.drop_path,\n drop_block_rate=None,\n )\n linear_scaled_lr = args.lr * args.batch_size * utils.get_world_size() / 512.0\n args.lr = linear_scaled_lr\n args.bt_lr=linear_scaled_lr*args.bt_lr\n from optimizer import create_bt_optimizer\n model_o = model\n optimizer1 = create_bt_optimizer(args, model, bt_decay=args.bt_decay)\n lr_scheduler1, _ = create_scheduler(args, optimizer1)\n\n if args.add_bt:\n from bt import TransformerDecorator1, BlockBF, BlockWrap32\n encoder_global = TransformerDecorator1(args.add_bt, model.num_features, args.eval_global,\n args=args, drop_path=args.drop_path_bt)\n\n if args.add_bt in [1]:\n # This is for the last layer\n res_blocks = []\n for i, block in enumerate(model.blocks):\n if args.no_fp16_bt and args.no_fp16_bt not in [4]:\n # This is for ease nan\n res_blocks.append(BlockWrap32(block, args.no_fp16_bt))\n else:\n res_blocks.append(block)\n model.blocks = torch.nn.Sequential(*res_blocks)\n model.norm = torch.nn.Sequential(model.norm, encoder_global)\n elif args.add_bt in [3]:\n # for insert into a single layer\n res_blocks = []\n drop_rate_list = [args.drop_path_bt]*(len(model.blocks)+1)\n if args.insert_idx is not None and len(args.insert_idx) > 0:\n insert_list = args.insert_idx\n else:\n insert_list = list(range(args.start_idx, len(model.blocks)))\n for i, block in enumerate(model.blocks):\n if args.no_fp16_bt and args.no_fp16_bt not in [4]:\n res_blocks.append(BlockWrap32(block, args.no_fp16_bt))\n else:\n res_blocks.append(block)\n if i in insert_list:\n if not args.shared_bt:\n encoder_global = TransformerDecorator1(args.add_bt, model.num_features, args.eval_global,\n args=args, drop_path=drop_rate_list[i])\n res_blocks.append(encoder_global)\n model.blocks = torch.nn.Sequential(*res_blocks)\n if args.add_norm_bt:\n if not args.shared_bt:\n encoder_global = TransformerDecorator1(args.add_bt, model.num_features, args.eval_global,\n args=args, drop_path=args.drop_path_bt)\n model.norm = torch.nn.Sequential(model.norm, encoder_global)\n elif args.add_bt in [2]:\n # This is for multiple layers\n # 61, 62 is for half batch\n # model.norm = torch.nn.Sequential(model.norm, encoder_global)\n res_blocks = []\n first_enc = TransformerDecorator1(args.add_bt, model.num_features, args.eval_global,\n args=args, first_layer=True, drop_path=args.drop_path_bt)\n old_enc = first_enc.encoder_layers\n del old_enc\n first_enc.encoder_layers = encoder_global.encoder_layers\n nums = 0\n if args.insert_idx is not None and len(args.insert_idx) > 0:\n insert_list = args.insert_idx\n else:\n insert_list = list(range(args.start_idx, len(model.blocks)))\n for i, block in enumerate(model.blocks):\n if args.no_fp16_bt and args.no_fp16_bt not in [4]:\n res_blocks.append(BlockWrap32(block, args.no_fp16_bt))\n else:\n res_blocks.append(block)\n if i in insert_list:\n if insert_list[0] == i: # first layer\n res_blocks.append(first_enc)\n first_enc = None\n else:\n if not args.shared_bt:\n encoder_global = TransformerDecorator1(args.add_bt, model.num_features, args.eval_global,\n args=args, drop_path=args.drop_path_bt)\n res_blocks.append(encoder_global)\n # model.norm = torch.nn.Identity()\n if args.add_norm_bt:\n if not args.shared_bt:\n encoder_global = TransformerDecorator1(args.add_bt, model.num_features, args.eval_global,\n args=args, drop_path=args.drop_path_bt)\n\n model.norm = torch.nn.Sequential(model.norm, encoder_global)\n model.blocks = torch.nn.Sequential(*res_blocks)\n\n if args.drop_patch != 0:\n # what's the relationship between drop_patch and shared prediction modules\n # This is not for the paper.\n res_blocks = []\n for i, block in enumerate(model.blocks):\n if i > args.start_idx:\n if args.drop_patch > 0:\n from timm.models.layers import DropPath\n block.drop_path = DropPath(args.drop_patch)\n res_blocks.append(BlockBF(block, args.no_fp16_bt))\n else:\n res_blocks.append(BlockBF(block, args.no_fp16_bt))\n else:\n res_blocks.append(block)\n model.blocks = torch.nn.Sequential(*res_blocks)\n print(model)\n if args.finetune:\n if args.finetune.startswith('https'):\n checkpoint = torch.hub.load_state_dict_from_url(\n args.finetune, map_location='cpu', check_hash=True)\n else:\n checkpoint = torch.load(args.finetune, map_location='cpu')\n\n checkpoint_model = checkpoint['model']\n state_dict = model.state_dict()\n for k in ['head.weight', 'head.bias', 'head_dist.weight', 'head_dist.bias']:\n if k in checkpoint_model and checkpoint_model[k].shape != state_dict[k].shape:\n print(f\"Removing key {k} from pretrained checkpoint\")\n del checkpoint_model[k]\n\n # interpolate position embedding\n pos_embed_checkpoint = checkpoint_model['pos_embed']\n embedding_size = pos_embed_checkpoint.shape[-1]\n num_patches = model.patch_embed.num_patches\n num_extra_tokens = model.pos_embed.shape[-2] - num_patches\n # height (== width) for the checkpoint position embedding\n orig_size = int((pos_embed_checkpoint.shape[-2] - num_extra_tokens) ** 0.5)\n # height (== width) for the new position embedding\n new_size = int(num_patches ** 0.5)\n # class_token and dist_token are kept unchanged\n extra_tokens = pos_embed_checkpoint[:, :num_extra_tokens]\n # only the position tokens are interpolated\n pos_tokens = pos_embed_checkpoint[:, num_extra_tokens:]\n pos_tokens = pos_tokens.reshape(-1, orig_size, orig_size, embedding_size).permute(0, 3, 1, 2)\n pos_tokens = torch.nn.functional.interpolate(\n pos_tokens, size=(new_size, new_size), mode='bicubic', align_corners=False)\n pos_tokens = pos_tokens.permute(0, 2, 3, 1).flatten(1, 2)\n new_pos_embed = torch.cat((extra_tokens, pos_tokens), dim=1)\n checkpoint_model['pos_embed'] = new_pos_embed\n\n model.load_state_dict(checkpoint_model, strict=False)\n if args.add_mlp_bt:\n from bt import MLPDecorder\n model.pre_logits = MLPDecorder(model.num_features, 4096, skip_mlp=args.skip_bt)\n\n print(model)\n model.to(device)\n\n model_ema = None\n if args.model_ema:\n # Important to create EMA model after cuda(), DP wrapper, and AMP but before SyncBN and DDP wrapper\n model_ema = ModelEma(\n model,\n decay=args.model_ema_decay,\n device='cpu' if args.model_ema_force_cpu else '',\n resume='')\n\n model_without_ddp = model\n if args.distributed:\n model = torch.nn.parallel.DistributedDataParallel(model, device_ids=[args.gpu], find_unused_parameters=True if args.skip_bt else False)\n model_without_ddp = model.module\n n_parameters = sum(p.numel() for p in model.parameters() if p.requires_grad)\n print('number of params:', n_parameters)\n\n # linear_scaled_lr = args.lr * args.batch_size * utils.get_world_size() / 512.0\n args.lr = linear_scaled_lr\n from optimizer import create_bt_optimizer\n optimizer = create_bt_optimizer(args, model_without_ddp, bt_decay=args.bt_decay)\n loss_scaler = NativeScaler()\n\n lr_scheduler, _ = create_scheduler(args, optimizer)\n\n criterion = LabelSmoothingCrossEntropy()\n\n if mixup_active:\n # smoothing is handled with mixup label transform\n criterion = SoftTargetCrossEntropy()\n elif args.smoothing:\n criterion = LabelSmoothingCrossEntropy(smoothing=args.smoothing)\n else:\n criterion = torch.nn.CrossEntropyLoss()\n\n teacher_model = None\n if args.distillation_type != 'none':\n assert args.teacher_path, 'need to specify teacher-path when using distillation'\n print(f\"Creating teacher model: {args.teacher_model}\")\n teacher_model = create_model(\n args.teacher_model,\n pretrained=args.pretrained,\n num_classes=args.nb_classes,\n global_pool='avg',\n )\n if args.teacher_path.startswith('https'):\n checkpoint = torch.hub.load_state_dict_from_url(\n args.teacher_path, map_location='cpu', check_hash=True)\n else:\n checkpoint = torch.load(args.teacher_path, map_location='cpu')\n teacher_model.load_state_dict(checkpoint['model'])\n teacher_model.to(device)\n teacher_model.eval()\n\n # wrap the criterion in our custom DistillationLoss, which\n # just dispatches to the original criterion if args.distillation_type is 'none'\n criterion = DistillationLoss(\n criterion, teacher_model, args.distillation_type, args.distillation_alpha, args.distillation_tau\n )\n if args.balanced_softmax:\n from losses import BalancedSoftmax\n criterion = BalancedSoftmax('./cls_freq/ImageNet_LT.json')\n output_dir = Path(args.output_dir)\n if args.resume:\n if args.resume.startswith('https'):\n checkpoint = torch.hub.load_state_dict_from_url(\n args.resume, map_location='cpu', check_hash=True)\n else:\n checkpoint = torch.load(args.resume, map_location='cpu')\n if args.skip_bt:\n model_without_ddp.load_state_dict(checkpoint['model'], strict=False)\n else:\n model_without_ddp.load_state_dict(checkpoint['model'], strict=True)\n if not args.eval and 'optimizer' in checkpoint and 'lr_scheduler' in checkpoint and 'epoch' in checkpoint:\n optimizer.load_state_dict(checkpoint['optimizer'])\n lr_scheduler.load_state_dict(checkpoint['lr_scheduler'])\n args.start_epoch = checkpoint['epoch'] + 1\n if args.model_ema:\n utils._load_checkpoint_for_ema(model_ema, checkpoint['model_ema'])\n if 'scaler' in checkpoint:\n loss_scaler.load_state_dict(checkpoint['scaler'])\n\n if args.skip_bt:\n optimizer.param_groups.pop(1)\n if args.add_mlp_bt:\n logits_list = [item for item in model_without_ddp.pre_logits.parameters()]\n print(len(optimizer.param_groups), len(logits_list))\n new_list = []\n if len(optimizer.param_groups) == 1:\n for item in optimizer.param_groups[0]['params']:\n is_logits=False\n for item1 in logits_list:\n if item.size == item1.size and torch.all(torch.eq(item, item1)):\n is_logits = True\n break\n if not is_logits:\n new_list.append(item)\n optimizer.param_groups[0]['params'] = new_list\n else:\n for item in optimizer.param_groups[1]['params']:\n is_logits=False\n for item1 in logits_list:\n if item.size == item1.size and torch.all(torch.eq(item, item1)):\n is_logits = True\n break\n if not is_logits:\n new_list.append(item)\n optimizer.param_groups[1]['params'] = new_list\n\n if args.eval:\n # test_stats = evaluate(data_loader_val, model, device)\n training_labels = dataset_train.targets\n training_labels = np.array(training_labels).astype(int)\n from engine import validate\n validate(data_loader_val, training_labels, model, device, args.nb_classes, utils.get_rank())\n\n print(f\"Accuracy of the network on the {len(dataset_val)} test images: {test_stats['acc1']:.1f}%\")\n return\n\n print(f\"Start training for {args.epochs} epochs\")\n start_time = time.time()\n max_accuracy = 0.0\n opt2=optimizer\n model2 = model\n lr_sched = lr_scheduler\n for epoch in range(args.start_epoch, args.epochs):\n if args.distributed:\n data_loader_train.sampler.set_epoch(epoch)\n\n if epoch < args.start_bt_epoch:\n for item in model.modules():\n if isinstance(item, TransformerDecorator1):\n item.eval()\n for n,p in item.named_parameters():\n p.requires_grad=False\n optimizer=optimizer1\n model = model_o\n lr_scheduler = lr_scheduler1\n elif epoch == args.start_bt_epoch:\n\n for n,p in model.named_parameters():\n p.requires_grad=True\n optimizer=opt2\n model = model2\n lr_scheduler = lr_sched\n model.train(args.finetune == '')\n\n train_stats = train_one_epoch(\n model, criterion, data_loader_train,\n optimizer, device, epoch, loss_scaler,\n args.clip_grad, model_ema, mixup_fn,\n set_training_mode=args.finetune == '', # keep in eval mode during finetuning\n no_fp16_bt=args.no_fp16_bt, model_without_ddp = model_without_ddp,\n num_classes=args.nb_classes\n )\n if epoch < args.start_bt_epoch:\n lr_sched.step(epoch)\n lr_scheduler.step(epoch)\n if args.output_dir:\n checkpoint_paths = [output_dir / str(args.exp_id+'checkpoint.pth')]\n for checkpoint_path in checkpoint_paths:\n utils.save_on_master({\n 'model': model_without_ddp.state_dict(),\n 'optimizer': optimizer.state_dict() if not args.skip_bt else 0,\n 'lr_scheduler': lr_scheduler.state_dict(),\n 'epoch': epoch,\n 'model_ema': get_state_dict(model_ema),\n 'scaler': loss_scaler.state_dict(),\n 'args': args,\n }, checkpoint_path)\n\n test_stats = evaluate(data_loader_val, model, device)\n print(f\"Accuracy of the network on the {len(dataset_val)} test images: {test_stats['acc1']:.1f}%\")\n max_accuracy = max(max_accuracy, test_stats[\"acc1\"])\n print(f'Max accuracy: {max_accuracy:.2f}%')\n\n log_stats = {**{f'train_{k}': v for k, v in train_stats.items()},\n **{f'test_{k}': v for k, v in test_stats.items()},\n 'epoch': epoch,\n 'n_parameters': n_parameters}\n\n if args.output_dir and utils.is_main_process():\n with (output_dir / str(args.exp_id+\"log.txt\")).open(\"a\") as f:\n f.write(json.dumps(log_stats) + \"\\n\")\n\n total_time = time.time() - start_time\n total_time_str = str(datetime.timedelta(seconds=int(total_time)))\n print('Training time {}'.format(total_time_str))\n\n\nif __name__ == '__main__':\n parser = argparse.ArgumentParser('DeiT training and evaluation script', parents=[get_args_parser()])\n args = parser.parse_args()\n if args.output_dir:\n Path(args.output_dir).mkdir(parents=True, exist_ok=True)\n main(args)\n"
},
{
"path": "batchformer-v2/deit_share/models.py",
"content": "# Copyright (c) 2015-present, Facebook, Inc.\n# All rights reserved.\nimport os\n\nimport torch\nimport torch.nn as nn\nfrom functools import partial\n\nfrom timm.models.vision_transformer import VisionTransformer, _cfg\nfrom timm.models.registry import register_model\nfrom timm.models.layers import trunc_normal_\n\n\n__all__ = [\n 'deit_tiny_patch16_224', 'deit_small_patch16_224', 'deit_small_patch16_224_12', 'deit_base_patch16_224',\n 'deit_tiny_distilled_patch16_224', 'deit_small_distilled_patch16_224',\n 'deit_base_distilled_patch16_224', 'deit_base_patch16_384',\n 'deit_base_distilled_patch16_384',\n]\n\n\nclass DistilledVisionTransformer(VisionTransformer):\n def __init__(self, *args, **kwargs):\n super().__init__(*args, **kwargs)\n self.dist_token = nn.Parameter(torch.zeros(1, 1, self.embed_dim))\n num_patches = self.patch_embed.num_patches\n self.pos_embed = nn.Parameter(torch.zeros(1, num_patches + 2, self.embed_dim))\n self.head_dist = nn.Linear(self.embed_dim, self.num_classes) if self.num_classes > 0 else nn.Identity()\n\n trunc_normal_(self.dist_token, std=.02)\n trunc_normal_(self.pos_embed, std=.02)\n self.head_dist.apply(self._init_weights)\n\n def forward_features(self, x):\n # taken from https://github.com/rwightman/pytorch-image-models/blob/master/timm/models/vision_transformer.py\n # with slight modifications to add the dist_token\n B = x.shape[0]\n x = self.patch_embed(x)\n\n cls_tokens = self.cls_token.expand(B, -1, -1) # stole cls_tokens impl from Phil Wang, thanks\n dist_token = self.dist_token.expand(B, -1, -1)\n x = torch.cat((cls_tokens, dist_token, x), dim=1)\n\n x = x + self.pos_embed\n x = self.pos_drop(x)\n\n for blk in self.blocks:\n x = blk(x)\n\n x = self.norm(x)\n return x[:, 0], x[:, 1]\n\n def forward(self, x):\n x, x_dist = self.forward_features(x)\n x = self.head(x)\n x_dist = self.head_dist(x_dist)\n if self.training:\n return x, x_dist\n else:\n # during inference, return the average of both classifier predictions\n return (x + x_dist) / 2\n\n\n@register_model\ndef deit_tiny_patch16_224(pretrained=False, **kwargs):\n model = VisionTransformer(\n patch_size=16, embed_dim=192, depth=12, num_heads=3, mlp_ratio=4, qkv_bias=True,\n norm_layer=partial(nn.LayerNorm, eps=1e-6), **kwargs)\n model.default_cfg = _cfg()\n if pretrained:\n checkpoint = torch.hub.load_state_dict_from_url(\n url=\"https://dl.fbaipublicfiles.com/deit/deit_tiny_patch16_224-a1311bcf.pth\",\n map_location=\"cpu\", check_hash=True\n )\n model.load_state_dict(checkpoint[\"model\"])\n return model\n\n\n@register_model\ndef deit_small_patch16_224(pretrained=False, **kwargs):\n model = VisionTransformer(\n patch_size=16, embed_dim=384, depth=12, num_heads=6, mlp_ratio=4, qkv_bias=True,\n norm_layer=partial(nn.LayerNorm, eps=1e-6), **kwargs)\n model.default_cfg = _cfg()\n if pretrained:\n checkpoint = torch.hub.load_state_dict_from_url(\n url=\"https://dl.fbaipublicfiles.com/deit/deit_small_patch16_224-cd65a155.pth\",\n map_location=\"cpu\", check_hash=True\n )\n state_dict = model.state_dict()\n for k in ['head.weight', 'head.bias']:\n if k in checkpoint['model'] and checkpoint['model'][k].shape != state_dict[k].shape:\n print(f\"Removing key {k} from pretrained checkpoint\")\n del checkpoint['model'][k]\n res = model.load_state_dict(checkpoint[\"model\"], strict=False)\n print(res)\n return model\n\n\n@register_model\ndef deit_small_patch16_224_12(pretrained=False, **kwargs):\n model = VisionTransformer(\n patch_size=16, embed_dim=384, depth=12, num_heads=12, mlp_ratio=4, qkv_bias=True,\n norm_layer=partial(nn.LayerNorm, eps=1e-6), **kwargs)\n model.default_cfg = _cfg()\n if pretrained:\n if os.environ['HOME'].endswith('zhou9878'):\n checkpoint = torch.load(\"deit_small_patch16_224-cd65a155.pth\")\n else:\n checkpoint = torch.load(\"/checkpoints/deit_small_patch16_224-cd65a155.pth\")\n # checkpoint = torch.hub.load_state_dict_from_url(\n # url=\"https://dl.fbaipublicfiles.com/deit/deit_small_patch16_224-cd65a155.pth\",\n # map_location=\"cpu\", check_hash=True\n # )\n model.load_state_dict(checkpoint[\"model\"])\n return model\n\n@register_model\ndef deit_base_patch16_224(pretrained=False, **kwargs):\n model = VisionTransformer(\n patch_size=16, embed_dim=768, depth=12, num_heads=12, mlp_ratio=4, qkv_bias=True,\n norm_layer=partial(nn.LayerNorm, eps=1e-6), **kwargs)\n model.default_cfg = _cfg()\n if pretrained:\n # checkpoint = torch.hub.load_state_dict_from_url(\n # url=\"https://dl.fbaipublicfiles.com/deit/deit_base_patch16_224-b5f2ef4d.pth\",\n # map_location=\"cpu\", check_hash=True\n # )\n if os.environ['HOME'].endswith('zhou9878'):\n checkpoint = torch.load(\"mae_pretrain_vit_base.pth\")\n else:\n checkpoint = torch.load(\"/checkpoints/deit_base_patch16_224-b5f2ef4d.pth\")\n print(\"load mae\")\n state_dict = model.state_dict()\n for k in ['head.weight', 'head.bias']:\n if k in checkpoint['model'] and checkpoint['model'][k].shape != state_dict[k].shape:\n print(f\"Removing key {k} from pretrained checkpoint\")\n del checkpoint['model'][k]\n res = model.load_state_dict(checkpoint[\"model\"], strict=False)\n print(res)\n return model\n\n\n@register_model\ndef deit_tiny_distilled_patch16_224(pretrained=False, **kwargs):\n model = DistilledVisionTransformer(\n patch_size=16, embed_dim=192, depth=12, num_heads=3, mlp_ratio=4, qkv_bias=True,\n norm_layer=partial(nn.LayerNorm, eps=1e-6), **kwargs)\n model.default_cfg = _cfg()\n if pretrained:\n checkpoint = torch.hub.load_state_dict_from_url(\n url=\"https://dl.fbaipublicfiles.com/deit/deit_tiny_distilled_patch16_224-b40b3cf7.pth\",\n map_location=\"cpu\", check_hash=True\n )\n model.load_state_dict(checkpoint[\"model\"])\n return model\n\n\n@register_model\ndef deit_small_distilled_patch16_224(pretrained=False, **kwargs):\n model = DistilledVisionTransformer(\n patch_size=16, embed_dim=384, depth=12, num_heads=6, mlp_ratio=4, qkv_bias=True,\n norm_layer=partial(nn.LayerNorm, eps=1e-6), **kwargs)\n model.default_cfg = _cfg()\n if pretrained:\n checkpoint = torch.hub.load_state_dict_from_url(\n url=\"https://dl.fbaipublicfiles.com/deit/deit_small_distilled_patch16_224-649709d9.pth\",\n map_location=\"cpu\", check_hash=True\n )\n model.load_state_dict(checkpoint[\"model\"])\n return model\n\n\n@register_model\ndef deit_base_distilled_patch16_224(pretrained=False, **kwargs):\n model = DistilledVisionTransformer(\n patch_size=16, embed_dim=768, depth=12, num_heads=12, mlp_ratio=4, qkv_bias=True,\n norm_layer=partial(nn.LayerNorm, eps=1e-6), **kwargs)\n model.default_cfg = _cfg()\n if pretrained:\n checkpoint = torch.hub.load_state_dict_from_url(\n url=\"https://dl.fbaipublicfiles.com/deit/deit_base_distilled_patch16_224-df68dfff.pth\",\n map_location=\"cpu\", check_hash=True\n )\n model.load_state_dict(checkpoint[\"model\"])\n return model\n\n\n@register_model\ndef deit_base_patch16_384(pretrained=False, **kwargs):\n model = VisionTransformer(\n img_size=384, patch_size=16, embed_dim=768, depth=12, num_heads=12, mlp_ratio=4, qkv_bias=True,\n norm_layer=partial(nn.LayerNorm, eps=1e-6), **kwargs)\n model.default_cfg = _cfg()\n if pretrained:\n checkpoint = torch.hub.load_state_dict_from_url(\n url=\"https://dl.fbaipublicfiles.com/deit/deit_base_patch16_384-8de9b5d1.pth\",\n map_location=\"cpu\", check_hash=True\n )\n model.load_state_dict(checkpoint[\"model\"])\n return model\n\n\n@register_model\ndef deit_base_distilled_patch16_384(pretrained=False, **kwargs):\n model = DistilledVisionTransformer(\n img_size=384, patch_size=16, embed_dim=768, depth=12, num_heads=12, mlp_ratio=4, qkv_bias=True,\n norm_layer=partial(nn.LayerNorm, eps=1e-6), **kwargs)\n model.default_cfg = _cfg()\n if pretrained:\n checkpoint = torch.hub.load_state_dict_from_url(\n url=\"https://dl.fbaipublicfiles.com/deit/deit_base_distilled_patch16_384-d0272ac0.pth\",\n map_location=\"cpu\", check_hash=True\n )\n model.load_state_dict(checkpoint[\"model\"])\n return model\n"
},
{
"path": "batchformer-v2/deit_share/optimizer.py",
"content": "import torch\nfrom timm.optim import optimizer_kwargs\nfrom timm.optim.optim_factory import add_weight_decay\nfrom torch import nn\nfrom typing import Optional\nimport torch.optim as optim\n\nfrom timm.optim.adafactor import Adafactor\nfrom timm.optim.adahessian import Adahessian\nfrom timm.optim.adamp import AdamP\nfrom timm.optim.lookahead import Lookahead\nfrom timm.optim.nadam import Nadam\nfrom timm.optim.novograd import NovoGrad\nfrom timm.optim.nvnovograd import NvNovoGrad\nfrom timm.optim.radam import RAdam\nfrom timm.optim.rmsprop_tf import RMSpropTF\nfrom timm.optim.sgdp import SGDP\nfrom timm.optim.adabelief import AdaBelief\n\ntry:\n from apex.optimizers import FusedNovoGrad, FusedAdam, FusedLAMB, FusedSGD\n has_apex = True\nexcept ImportError:\n has_apex = False\n\ndef add_bt_weight_decay(model, weight_decay=1e-5, skip_list=(), second_group_tag=None, second_decay=1e-5, lr_bt=1e-4, mlp_decay=0):\n decay = []\n decay2 = []\n decay3 = []\n no_decay = []\n for name, param in model.named_parameters():\n if not param.requires_grad:\n continue # frozen weights\n if len(param.shape) == 1 or name.endswith(\".bias\") or name in skip_list:\n no_decay.append(param)\n elif second_group_tag and name.__contains__(second_group_tag):\n print(name, 'bt group')\n decay2.append(param)\n elif mlp_decay:\n decay3.append(param)\n else:\n decay.append(param)\n return [\n {'params': no_decay, 'weight_decay': 0., },\n {'params': decay2, 'weight_decay': second_decay, 'lr': lr_bt},\n {'params': decay3, 'weight_decay': mlp_decay},\n {'params': decay, 'weight_decay': weight_decay}]\n\ndef create_bt_optimizer(args, model, bt_decay, filter_bias_and_bn=True):\n \"\"\" Legacy optimizer factory for backwards compatibility.\n NOTE: Use create_optimizer_v2 for new code.\n \"\"\"\n return create_optimizer_v2(\n model,\n **optimizer_kwargs(cfg=args),\n filter_bias_and_bn=filter_bias_and_bn,\n bt_decay=bt_decay,\n bt_lr=args.bt_lr,\n mlp_decay=args.mlp_decay,\n )\n\ndef create_optimizer_v2(\n model: nn.Module,\n optimizer_name: str = 'sgd',\n learning_rate: Optional[float] = None,\n weight_decay: float = 0.,\n momentum: float = 0.9,\n filter_bias_and_bn: bool = True,\n bt_decay = 0,\n bt_lr = 1e-4,\n mlp_decay=0,\n **kwargs):\n \"\"\" Create an optimizer.\n\n For more general use an interface that allows selection of parameters to optimize and lr groups, one of:\n * a filter fn interface that further breaks params into groups in a weight_decay compatible fashion\n * expose the parameters interface and leave it up to caller\n\n Args:\n model (nn.Module): model containing parameters to optimize\n optimizer_name: name of optimizer to create\n learning_rate: initial learning rate\n weight_decay: weight decay to apply in optimizer\n momentum: momentum for momentum based optimizers (others may use betas via kwargs)\n filter_bias_and_bn: filter out bias, bn and other 1d params from weight decay\n **kwargs: extra optimizer specific kwargs to pass through\n\n Returns:\n Optimizer\n \"\"\"\n opt_lower = optimizer_name.lower()\n if weight_decay and filter_bias_and_bn:\n skip = {}\n if hasattr(model, 'no_weight_decay'):\n skip = model.no_weight_decay()\n if bt_decay > 0:\n parameters = add_bt_weight_decay(model, weight_decay, skip, 'encoder' if bt_decay > 0 else None, bt_decay, bt_lr, mlp_decay)\n # if skip_bt:\n # parameters = [parameters[0], parameters[2]]\n for item in parameters:\n print(item['weight_decay'])\n else:\n parameters = add_weight_decay(model, weight_decay, skip)\n weight_decay = 0.\n else:\n parameters = model.parameters()\n if 'fused' in opt_lower:\n assert has_apex and torch.cuda.is_available(), 'APEX and CUDA required for fused optimizers'\n\n opt_args = dict(lr=learning_rate, weight_decay=weight_decay, **kwargs)\n opt_split = opt_lower.split('_')\n opt_lower = opt_split[-1]\n if opt_lower == 'sgd' or opt_lower == 'nesterov':\n opt_args.pop('eps', None)\n optimizer = optim.SGD(parameters, momentum=momentum, nesterov=True, **opt_args)\n elif opt_lower == 'momentum':\n opt_args.pop('eps', None)\n optimizer = optim.SGD(parameters, momentum=momentum, nesterov=False, **opt_args)\n elif opt_lower == 'adam':\n optimizer = optim.Adam(parameters, **opt_args)\n elif opt_lower == 'adabelief':\n optimizer = AdaBelief(parameters, rectify=False, **opt_args)\n elif opt_lower == 'adamw':\n optimizer = optim.AdamW(parameters, **opt_args)\n elif opt_lower == 'nadam':\n optimizer = Nadam(parameters, **opt_args)\n elif opt_lower == 'radam':\n optimizer = RAdam(parameters, **opt_args)\n elif opt_lower == 'adamp':\n optimizer = AdamP(parameters, wd_ratio=0.01, nesterov=True, **opt_args)\n elif opt_lower == 'sgdp':\n optimizer = SGDP(parameters, momentum=momentum, nesterov=True, **opt_args)\n elif opt_lower == 'adadelta':\n optimizer = optim.Adadelta(parameters, **opt_args)\n elif opt_lower == 'adafactor':\n if not learning_rate:\n opt_args['lr'] = None\n optimizer = Adafactor(parameters, **opt_args)\n elif opt_lower == 'adahessian':\n optimizer = Adahessian(parameters, **opt_args)\n elif opt_lower == 'rmsprop':\n optimizer = optim.RMSprop(parameters, alpha=0.9, momentum=momentum, **opt_args)\n elif opt_lower == 'rmsproptf':\n optimizer = RMSpropTF(parameters, alpha=0.9, momentum=momentum, **opt_args)\n elif opt_lower == 'novograd':\n optimizer = NovoGrad(parameters, **opt_args)\n elif opt_lower == 'nvnovograd':\n optimizer = NvNovoGrad(parameters, **opt_args)\n elif opt_lower == 'fusedsgd':\n opt_args.pop('eps', None)\n optimizer = FusedSGD(parameters, momentum=momentum, nesterov=True, **opt_args)\n elif opt_lower == 'fusedmomentum':\n opt_args.pop('eps', None)\n optimizer = FusedSGD(parameters, momentum=momentum, nesterov=False, **opt_args)\n elif opt_lower == 'fusedadam':\n optimizer = FusedAdam(parameters, adam_w_mode=False, **opt_args)\n elif opt_lower == 'fusedadamw':\n optimizer = FusedAdam(parameters, adam_w_mode=True, **opt_args)\n elif opt_lower == 'fusedlamb':\n optimizer = FusedLAMB(parameters, **opt_args)\n elif opt_lower == 'fusednovograd':\n opt_args.setdefault('betas', (0.95, 0.98))\n optimizer = FusedNovoGrad(parameters, **opt_args)\n else:\n assert False and \"Invalid optimizer\"\n raise ValueError\n\n if len(opt_split) > 1:\n if opt_split[0] == 'lookahead':\n optimizer = Lookahead(optimizer)\n\n return optimizer"
},
{
"path": "batchformer-v2/deit_share/requirements.txt",
"content": "torch==1.7.0\ntorchvision==0.8.1\ntimm==0.3.2\n"
},
{
"path": "batchformer-v2/deit_share/resmlp_models.py",
"content": "# Copyright (c) 2015-present, Facebook, Inc.\n# All rights reserved.\nimport torch\nimport torch.nn as nn\nfrom functools import partial\n\nfrom timm.models.vision_transformer import Mlp, PatchEmbed , _cfg\nfrom timm.models.registry import register_model\nfrom timm.models.layers import trunc_normal_, DropPath\n\n\n__all__ = [\n 'resMLP_12', 'resMLP_24', 'resMLP_36', 'resmlpB_24'\n]\n\nclass Affine(nn.Module):\n def __init__(self, dim):\n super().__init__()\n self.alpha = nn.Parameter(torch.ones(dim))\n self.beta = nn.Parameter(torch.zeros(dim))\n\n def forward(self, x):\n return self.alpha * x + self.beta \n \nclass layers_scale_mlp_blocks(nn.Module):\n\n def __init__(self, dim, drop=0., drop_path=0., act_layer=nn.GELU,init_values=1e-4,num_patches = 196):\n super().__init__()\n self.norm1 = Affine(dim)\n self.attn = nn.Linear(num_patches, num_patches)\n self.drop_path = DropPath(drop_path) if drop_path > 0. else nn.Identity()\n self.norm2 = Affine(dim)\n self.mlp = Mlp(in_features=dim, hidden_features=int(4.0 * dim), act_layer=act_layer, drop=drop)\n self.gamma_1 = nn.Parameter(init_values * torch.ones((dim)),requires_grad=True)\n self.gamma_2 = nn.Parameter(init_values * torch.ones((dim)),requires_grad=True)\n\n def forward(self, x):\n x = x + self.drop_path(self.gamma_1 * self.attn(self.norm1(x).transpose(1,2)).transpose(1,2))\n x = x + self.drop_path(self.gamma_2 * self.mlp(self.norm2(x)))\n return x \n\n\nclass resmlp_models(nn.Module):\n\n def __init__(self, img_size=224, patch_size=16, in_chans=3, num_classes=1000, embed_dim=768, depth=12,drop_rate=0.,\n Patch_layer=PatchEmbed,act_layer=nn.GELU,\n drop_path_rate=0.0,init_scale=1e-4):\n super().__init__()\n\n\n\n self.num_classes = num_classes\n self.num_features = self.embed_dim = embed_dim \n\n self.patch_embed = Patch_layer(\n img_size=img_size, patch_size=patch_size, in_chans=int(in_chans), embed_dim=embed_dim)\n num_patches = self.patch_embed.num_patches\n dpr = [drop_path_rate for i in range(depth)]\n\n self.blocks = nn.ModuleList([\n layers_scale_mlp_blocks(\n dim=embed_dim,drop=drop_rate,drop_path=dpr[i],\n act_layer=act_layer,init_values=init_scale,\n num_patches=num_patches)\n for i in range(depth)])\n\n\n self.norm = Affine(embed_dim)\n\n\n\n self.feature_info = [dict(num_chs=embed_dim, reduction=0, module='head')]\n self.head = nn.Linear(embed_dim, num_classes) if num_classes > 0 else nn.Identity()\n self.apply(self._init_weights)\n\n def _init_weights(self, m):\n if isinstance(m, nn.Linear):\n trunc_normal_(m.weight, std=0.02)\n if isinstance(m, nn.Linear) and m.bias is not None:\n nn.init.constant_(m.bias, 0)\n elif isinstance(m, nn.LayerNorm):\n nn.init.constant_(m.bias, 0)\n nn.init.constant_(m.weight, 1.0)\n\n\n\n def get_classifier(self):\n return self.head\n\n def reset_classifier(self, num_classes, global_pool=''):\n self.num_classes = num_classes\n self.head = nn.Linear(self.embed_dim, num_classes) if num_classes > 0 else nn.Identity()\n\n def forward_features(self, x):\n B = x.shape[0]\n\n x = self.patch_embed(x)\n\n for i , blk in enumerate(self.blocks):\n x = blk(x)\n\n x = self.norm(x)\n x = x.mean(dim=1).reshape(B,1,-1)\n\n return x[:, 0]\n\n def forward(self, x):\n x = self.forward_features(x)\n x = self.head(x)\n return x \n\n@register_model\ndef resmlp_12(pretrained=False,dist=False, **kwargs):\n model = resmlp_models(\n patch_size=16, embed_dim=384, depth=12,\n Patch_layer=PatchEmbed,\n init_scale=0.1,**kwargs)\n \n model.default_cfg = _cfg()\n if pretrained:\n if dist:\n url_path = \"https://dl.fbaipublicfiles.com/deit/resmlp_12_dist.pth\"\n else:\n url_path = \"https://dl.fbaipublicfiles.com/deit/resmlp_12_no_dist.pth\"\n checkpoint = torch.hub.load_state_dict_from_url(\n url=url_path,\n map_location=\"cpu\", check_hash=True\n )\n \n model.load_state_dict(checkpoint)\n return model\n \n@register_model\ndef resmlp_24(pretrained=False,dist=False,dino=False, **kwargs):\n model = resmlp_models(\n patch_size=16, embed_dim=384, depth=24,\n Patch_layer=PatchEmbed,\n init_scale=1e-5,**kwargs)\n model.default_cfg = _cfg()\n if pretrained:\n if dist:\n url_path = \"https://dl.fbaipublicfiles.com/deit/resmlp_24_dist.pth\"\n elif dino:\n url_path = \"https://dl.fbaipublicfiles.com/deit/resmlp_24_dino.pth\"\n else:\n url_path = \"https://dl.fbaipublicfiles.com/deit/resmlp_24_no_dist.pth\"\n checkpoint = torch.hub.load_state_dict_from_url(\n url=url_path,\n map_location=\"cpu\", check_hash=True\n )\n \n model.load_state_dict(checkpoint)\n return model\n \n@register_model\ndef resmlp_36(pretrained=False,dist=False, **kwargs):\n model = resmlp_models(\n patch_size=16, embed_dim=384, depth=36,\n Patch_layer=PatchEmbed,\n init_scale=1e-6,**kwargs)\n model.default_cfg = _cfg()\n if pretrained:\n if dist:\n url_path = \"https://dl.fbaipublicfiles.com/deit/resmlp_36_dist.pth\"\n else:\n url_path = \"https://dl.fbaipublicfiles.com/deit/resmlp_36_no_dist.pth\"\n checkpoint = torch.hub.load_state_dict_from_url(\n url=url_path,\n map_location=\"cpu\", check_hash=True\n )\n \n model.load_state_dict(checkpoint)\n return model\n\n@register_model\ndef resmlpB_24(pretrained=False,dist=False, in_22k = False, **kwargs):\n model = resmlp_models(\n patch_size=8, embed_dim=768, depth=24,\n Patch_layer=PatchEmbed,\n init_scale=1e-6,**kwargs)\n model.default_cfg = _cfg()\n if pretrained:\n if dist:\n url_path = \"https://dl.fbaipublicfiles.com/deit/resmlpB_24_dist.pth\"\n elif in_22k:\n url_path = \"https://dl.fbaipublicfiles.com/deit/resmlpB_24_22k.pth\"\n else:\n url_path = \"https://dl.fbaipublicfiles.com/deit/resmlpB_24_no_dist.pth\"\n \n checkpoint = torch.hub.load_state_dict_from_url(\n url=url_path,\n map_location=\"cpu\", check_hash=True\n )\n \n model.load_state_dict(checkpoint)\n \n return model\n"
},
{
"path": "batchformer-v2/deit_share/run_with_submitit.py",
"content": "# Copyright (c) 2015-present, Facebook, Inc.\n# All rights reserved.\n\"\"\"\nA script to run multinode training with submitit.\n\"\"\"\nimport argparse\nimport os\nimport uuid\nfrom pathlib import Path\n\nimport main as classification\nimport submitit\n\n\ndef parse_args():\n classification_parser = classification.get_args_parser()\n parser = argparse.ArgumentParser(\"Submitit for DeiT\", parents=[classification_parser])\n parser.add_argument(\"--ngpus\", default=8, type=int, help=\"Number of gpus to request on each node\")\n parser.add_argument(\"--nodes\", default=2, type=int, help=\"Number of nodes to request\")\n parser.add_argument(\"--timeout\", default=2800, type=int, help=\"Duration of the job\")\n parser.add_argument(\"--job_dir\", default=\"\", type=str, help=\"Job dir. Leave empty for automatic.\")\n\n parser.add_argument(\"--partition\", default=\"learnfair\", type=str, help=\"Partition where to submit\")\n parser.add_argument(\"--use_volta32\", action='store_true', help=\"Big models? Use this\")\n parser.add_argument('--comment', default=\"\", type=str,\n help='Comment to pass to scheduler, e.g. priority message')\n return parser.parse_args()\n\n\ndef get_shared_folder() -> Path:\n user = os.getenv(\"USER\")\n if Path(\"/checkpoint/\").is_dir():\n p = Path(f\"/checkpoint/{user}/experiments\")\n p.mkdir(exist_ok=True)\n return p\n raise RuntimeError(\"No shared folder available\")\n\n\ndef get_init_file():\n # Init file must not exist, but it's parent dir must exist.\n os.makedirs(str(get_shared_folder()), exist_ok=True)\n init_file = get_shared_folder() / f\"{uuid.uuid4().hex}_init\"\n if init_file.exists():\n os.remove(str(init_file))\n return init_file\n\n\nclass Trainer(object):\n def __init__(self, args):\n self.args = args\n\n def __call__(self):\n import main as classification\n\n self._setup_gpu_args()\n classification.main(self.args)\n\n def checkpoint(self):\n import os\n import submitit\n\n self.args.dist_url = get_init_file().as_uri()\n checkpoint_file = os.path.join(self.args.output_dir, \"checkpoint.pth\")\n if os.path.exists(checkpoint_file):\n self.args.resume = checkpoint_file\n print(\"Requeuing \", self.args)\n empty_trainer = type(self)(self.args)\n return submitit.helpers.DelayedSubmission(empty_trainer)\n\n def _setup_gpu_args(self):\n import submitit\n from pathlib import Path\n\n job_env = submitit.JobEnvironment()\n self.args.output_dir = Path(str(self.args.output_dir).replace(\"%j\", str(job_env.job_id)))\n self.args.gpu = job_env.local_rank\n self.args.rank = job_env.global_rank\n self.args.world_size = job_env.num_tasks\n print(f\"Process group: {job_env.num_tasks} tasks, rank: {job_env.global_rank}\")\n\n\ndef main():\n args = parse_args()\n if args.job_dir == \"\":\n args.job_dir = get_shared_folder() / \"%j\"\n\n # Note that the folder will depend on the job_id, to easily track experiments\n executor = submitit.AutoExecutor(folder=args.job_dir, slurm_max_num_timeout=30)\n\n num_gpus_per_node = args.ngpus\n nodes = args.nodes\n timeout_min = args.timeout\n\n partition = args.partition\n kwargs = {}\n if args.use_volta32:\n kwargs['slurm_constraint'] = 'volta32gb'\n if args.comment:\n kwargs['slurm_comment'] = args.comment\n\n executor.update_parameters(\n mem_gb=40 * num_gpus_per_node,\n gpus_per_node=num_gpus_per_node,\n tasks_per_node=num_gpus_per_node, # one task per GPU\n cpus_per_task=10,\n nodes=nodes,\n timeout_min=timeout_min, # max is 60 * 72\n # Below are cluster dependent parameters\n slurm_partition=partition,\n slurm_signal_delay_s=120,\n **kwargs\n )\n\n executor.update_parameters(name=\"deit\")\n\n args.dist_url = get_init_file().as_uri()\n args.output_dir = args.job_dir\n\n trainer = Trainer(args)\n job = executor.submit(trainer)\n\n print(\"Submitted job_id:\", job.job_id)\n\n\nif __name__ == \"__main__\":\n main()\n"
},
{
"path": "batchformer-v2/deit_share/samplers.py",
"content": "# Copyright (c) 2015-present, Facebook, Inc.\n# All rights reserved.\nimport torch\nimport torch.distributed as dist\nimport math\n\n\nclass RASampler(torch.utils.data.Sampler):\n \"\"\"Sampler that restricts data loading to a subset of the dataset for distributed,\n with repeated augmentation.\n It ensures that different each augmented version of a sample will be visible to a\n different process (GPU)\n Heavily based on torch.utils.data.DistributedSampler\n \"\"\"\n\n def __init__(self, dataset, num_replicas=None, rank=None, shuffle=True):\n if num_replicas is None:\n if not dist.is_available():\n raise RuntimeError(\"Requires distributed package to be available\")\n num_replicas = dist.get_world_size()\n if rank is None:\n if not dist.is_available():\n raise RuntimeError(\"Requires distributed package to be available\")\n rank = dist.get_rank()\n self.dataset = dataset\n self.num_replicas = num_replicas\n self.rank = rank\n self.epoch = 0\n self.num_samples = int(math.ceil(len(self.dataset) * 3.0 / self.num_replicas))\n self.total_size = self.num_samples * self.num_replicas\n # self.num_selected_samples = int(math.ceil(len(self.dataset) / self.num_replicas))\n self.num_selected_samples = int(math.floor(len(self.dataset) // 256 * 256 / self.num_replicas))\n self.shuffle = shuffle\n\n def __iter__(self):\n # deterministically shuffle based on epoch\n g = torch.Generator()\n g.manual_seed(self.epoch)\n if self.shuffle:\n indices = torch.randperm(len(self.dataset), generator=g).tolist()\n else:\n indices = list(range(len(self.dataset)))\n\n # add extra samples to make it evenly divisible\n indices = [ele for ele in indices for i in range(3)]\n indices += indices[:(self.total_size - len(indices))]\n assert len(indices) == self.total_size\n\n # subsample\n indices = indices[self.rank:self.total_size:self.num_replicas]\n assert len(indices) == self.num_samples\n\n return iter(indices[:self.num_selected_samples])\n\n def __len__(self):\n return self.num_selected_samples\n\n def set_epoch(self, epoch):\n self.epoch = epoch\n"
},
{
"path": "batchformer-v2/deit_share/temp.txt",
"content": "n01443537\nn01629819\nn01641577\nn01644900\nn01698640\nn01742172\nn01768244\nn01770393\nn01774384\nn01774750\nn01784675\nn01855672\nn01882714\nn01910747\nn01917289\nn01944390\nn01945685\nn01950731\nn01983481\nn01984695\nn02002724\nn02056570\nn02058221\nn02074367\nn02085620\nn02094433\nn02099601\nn02099712\nn02106662\nn02113799\nn02123045\nn02123394\nn02124075\nn02125311\nn02129165\nn02132136\nn02165456\nn02190166\nn02206856\nn02226429\nn02231487\nn02233338\nn02236044\nn02268443\nn02279972\nn02281406\nn02321529\nn02364673\nn02395406\nn02403003\nn02410509\nn02415577\nn02423022\nn02437312\nn02480495\nn02481823\nn02486410\nn02504458\nn02509815\nn02666196\nn02669723\nn02699494\nn02730930\nn02769748\nn02788148\nn02791270\nn02793495\nn02795169\nn02802426\nn02808440\nn02814533\nn02814860\nn02815834\nn02823428\nn02837789\nn02841315\nn02843684\nn02883205\nn02892201\nn02906734\nn02909870\nn02917067\nn02927161\nn02948072\nn02950826\nn02963159\nn02977058\nn02988304\nn02999410\nn03014705\nn03026506\nn03042490\nn03085013\nn03089624\nn03100240\nn03126707\nn03160309\nn03179701\nn03201208\nn03250847\nn03255030\nn03355925\nn03388043\nn03393912\nn03400231\nn03404251\nn03424325\nn03444034\nn03447447\nn03544143\nn03584254\nn03599486\nn03617480\nn03637318\nn03649909\nn03662601\nn03670208\nn03706229\nn03733131\nn03763968\nn03770439\nn03796401\nn03804744\nn03814639\nn03837869\nn03838899\nn03854065\nn03891332\nn03902125\nn03930313\nn03937543\nn03970156\nn03976657\nn03977966\nn03980874\nn03983396\nn03992509\nn04008634\nn04023962\nn04067472\nn04070727\nn04074963\nn04099969\nn04118538\nn04133789\nn04146614\nn04149813\nn04179913\nn04251144\nn04254777\nn04259630\nn04265275\nn04275548\nn04285008\nn04311004\nn04328186\nn04356056\nn04366367\nn04371430\nn04376876\nn04398044\nn04399382\nn04417672\nn04456115\nn04465501\nn04486054\nn04487081\nn04501370\nn04507155\nn04532106\nn04532670\nn04540053\nn04560804\nn04562935\nn04596742\nn04597913\nn06596364\nn07579787\nn07583066\nn07614500\nn07615774\nn07695742\nn07711569\nn07715103\nn07720875\nn07734744\nn07747607\nn07749582\nn07753592\nn07768694\nn07871810\nn07873807\nn07875152\nn07920052\nn09193705\nn09246464\nn09256479\nn09332890\nn09428293\nn12267677\n"
},
{
"path": "batchformer-v2/deit_share/tiny_img.py",
"content": "import os\nimport sys\nos.makedirs('/ImageNetTiny')\nos.makedirs('/ImageNetTiny/train')\nos.makedirs('/ImageNetTiny/val')\n\nwith open('temp.txt') as f:\n for item in f.readlines():\n if len(item) < 3:\n continue\n os.system('ln -s /data/train/{} /ImageNetTiny/train/'.format(item.strip()))\n os.system('ln -s /data/val/{} /ImageNetTiny/val/'.format(item.strip()))"
},
{
"path": "batchformer-v2/deit_share/tox.ini",
"content": "[flake8]\nmax-line-length = 120\nignore = F401,E402,F403,W503,W504\n"
},
{
"path": "batchformer-v2/deit_share/utils.py",
"content": "# Copyright (c) 2015-present, Facebook, Inc.\n# All rights reserved.\n\"\"\"\nMisc functions, including distributed helpers.\n\nMostly copy-paste from torchvision references.\n\"\"\"\nimport io\nimport os\nimport time\nfrom collections import defaultdict, deque\nimport datetime\n\nimport torch\nimport torch.distributed as dist\n\n\nclass SmoothedValue(object):\n \"\"\"Track a series of values and provide access to smoothed values over a\n window or the global series average.\n \"\"\"\n\n def __init__(self, window_size=20, fmt=None):\n if fmt is None:\n fmt = \"{median:.4f} ({global_avg:.4f})\"\n self.deque = deque(maxlen=window_size)\n self.total = 0.0\n self.count = 0\n self.fmt = fmt\n\n def update(self, value, n=1):\n self.deque.append(value)\n self.count += n\n self.total += value * n\n\n def synchronize_between_processes(self):\n \"\"\"\n Warning: does not synchronize the deque!\n \"\"\"\n if not is_dist_avail_and_initialized():\n return\n t = torch.tensor([self.count, self.total], dtype=torch.float64, device='cuda')\n dist.barrier()\n dist.all_reduce(t)\n t = t.tolist()\n self.count = int(t[0])\n self.total = t[1]\n\n @property\n def median(self):\n d = torch.tensor(list(self.deque))\n return d.median().item()\n\n @property\n def avg(self):\n d = torch.tensor(list(self.deque), dtype=torch.float32)\n return d.mean().item()\n\n @property\n def global_avg(self):\n return self.total / self.count\n\n @property\n def max(self):\n return max(self.deque)\n\n @property\n def value(self):\n return self.deque[-1]\n\n def __str__(self):\n return self.fmt.format(\n median=self.median,\n avg=self.avg,\n global_avg=self.global_avg,\n max=self.max,\n value=self.value)\n\n\nclass MetricLogger(object):\n def __init__(self, delimiter=\"\\t\"):\n self.meters = defaultdict(SmoothedValue)\n self.delimiter = delimiter\n\n def update(self, **kwargs):\n for k, v in kwargs.items():\n if isinstance(v, torch.Tensor):\n v = v.item()\n assert isinstance(v, (float, int))\n self.meters[k].update(v)\n\n def __getattr__(self, attr):\n if attr in self.meters:\n return self.meters[attr]\n if attr in self.__dict__:\n return self.__dict__[attr]\n raise AttributeError(\"'{}' object has no attribute '{}'\".format(\n type(self).__name__, attr))\n\n def __str__(self):\n loss_str = []\n for name, meter in self.meters.items():\n loss_str.append(\n \"{}: {}\".format(name, str(meter))\n )\n return self.delimiter.join(loss_str)\n\n def synchronize_between_processes(self):\n for meter in self.meters.values():\n meter.synchronize_between_processes()\n\n def add_meter(self, name, meter):\n self.meters[name] = meter\n\n def log_every(self, iterable, print_freq, header=None):\n i = 0\n if not header:\n header = ''\n start_time = time.time()\n end = time.time()\n iter_time = SmoothedValue(fmt='{avg:.4f}')\n data_time = SmoothedValue(fmt='{avg:.4f}')\n space_fmt = ':' + str(len(str(len(iterable)))) + 'd'\n log_msg = [\n header,\n '[{0' + space_fmt + '}/{1}]',\n 'eta: {eta}',\n '{meters}',\n 'time: {time}',\n 'data: {data}'\n ]\n if torch.cuda.is_available():\n log_msg.append('max mem: {memory:.0f}')\n log_msg = self.delimiter.join(log_msg)\n MB = 1024.0 * 1024.0\n for obj in iterable:\n data_time.update(time.time() - end)\n yield obj\n iter_time.update(time.time() - end)\n if i % print_freq == 0 or i == len(iterable) - 1:\n eta_seconds = iter_time.global_avg * (len(iterable) - i)\n eta_string = str(datetime.timedelta(seconds=int(eta_seconds)))\n if torch.cuda.is_available():\n print(log_msg.format(\n i, len(iterable), eta=eta_string,\n meters=str(self),\n time=str(iter_time), data=str(data_time),\n memory=torch.cuda.max_memory_allocated() / MB))\n else:\n print(log_msg.format(\n i, len(iterable), eta=eta_string,\n meters=str(self),\n time=str(iter_time), data=str(data_time)))\n i += 1\n end = time.time()\n total_time = time.time() - start_time\n total_time_str = str(datetime.timedelta(seconds=int(total_time)))\n print('{} Total time: {} ({:.4f} s / it)'.format(\n header, total_time_str, total_time / len(iterable)))\n\n\ndef _load_checkpoint_for_ema(model_ema, checkpoint):\n \"\"\"\n Workaround for ModelEma._load_checkpoint to accept an already-loaded object\n \"\"\"\n mem_file = io.BytesIO()\n torch.save(checkpoint, mem_file)\n mem_file.seek(0)\n model_ema._load_checkpoint(mem_file)\n\n\ndef setup_for_distributed(is_master):\n \"\"\"\n This function disables printing when not in master process\n \"\"\"\n import builtins as __builtin__\n builtin_print = __builtin__.print\n\n def print(*args, **kwargs):\n force = kwargs.pop('force', False)\n if is_master or force:\n builtin_print(*args, **kwargs)\n\n __builtin__.print = print\n\n\ndef is_dist_avail_and_initialized():\n if not dist.is_available():\n return False\n if not dist.is_initialized():\n return False\n return True\n\n\ndef get_world_size():\n if not is_dist_avail_and_initialized():\n return 1\n return dist.get_world_size()\n\n\ndef get_rank():\n if not is_dist_avail_and_initialized():\n return 0\n return dist.get_rank()\n\n\ndef is_main_process():\n return get_rank() == 0\n\n\ndef save_on_master(*args, **kwargs):\n if is_main_process():\n torch.save(*args, **kwargs)\n\n\ndef init_distributed_mode(args):\n if 'RANK' in os.environ and 'WORLD_SIZE' in os.environ:\n args.rank = int(os.environ[\"RANK\"])\n args.world_size = int(os.environ['WORLD_SIZE'])\n args.gpu = int(os.environ['LOCAL_RANK'])\n elif 'SLURM_PROCID' in os.environ:\n args.rank = int(os.environ['SLURM_PROCID'])\n args.gpu = args.rank % torch.cuda.device_count()\n else:\n print('Not using distributed mode')\n args.distributed = False\n return\n\n args.distributed = True\n\n torch.cuda.set_device(args.gpu)\n args.dist_backend = 'nccl'\n print('| distributed init (rank {}): {}'.format(\n args.rank, args.dist_url), flush=True)\n torch.distributed.init_process_group(backend=args.dist_backend, init_method=args.dist_url,\n world_size=args.world_size, rank=args.rank)\n torch.distributed.barrier()\n setup_for_distributed(args.rank == 0)\n"
},
{
"path": "batchformer-v2/detr/.circleci/config.yml",
"content": "version: 2.1\n\njobs:\n python_lint:\n docker:\n - image: circleci/python:3.7\n steps:\n - checkout\n - run:\n command: |\n pip install --user --progress-bar off flake8 typing\n flake8 .\n\n test:\n docker:\n - image: circleci/python:3.7\n steps:\n - checkout\n - run:\n command: |\n pip install --user --progress-bar off scipy pytest\n pip install --user --progress-bar off --pre torch torchvision -f https://download.pytorch.org/whl/nightly/cpu/torch_nightly.html\n pip install --user --progress-bar off onnx onnxruntime\n pytest .\n\nworkflows:\n build:\n jobs:\n - python_lint\n - test\n"
},
{
"path": "batchformer-v2/detr/.github/CODE_OF_CONDUCT.md",
"content": "# Code of Conduct\n\nFacebook has adopted a Code of Conduct that we expect project participants to adhere to.\nPlease read the [full text](https://code.fb.com/codeofconduct/)\nso that you can understand what actions will and will not be tolerated.\n"
},
{
"path": "batchformer-v2/detr/.github/CONTRIBUTING.md",
"content": "# Contributing to DETR\nWe want to make contributing to this project as easy and transparent as\npossible.\n\n## Our Development Process\nMinor changes and improvements will be released on an ongoing basis. Larger changes (e.g., changesets implementing a new paper) will be released on a more periodic basis.\n\n## Pull Requests\nWe actively welcome your pull requests.\n\n1. Fork the repo and create your branch from `master`.\n2. If you've added code that should be tested, add tests.\n3. If you've changed APIs, update the documentation.\n4. Ensure the test suite passes.\n5. Make sure your code lints.\n6. If you haven't already, complete the Contributor License Agreement (\"CLA\").\n\n## Contributor License Agreement (\"CLA\")\nIn order to accept your pull request, we need you to submit a CLA. You only need\nto do this once to work on any of Facebook's open source projects.\n\nComplete your CLA here: \n\n## Issues\nWe use GitHub issues to track public bugs. Please ensure your description is\nclear and has sufficient instructions to be able to reproduce the issue.\n\nFacebook has a [bounty program](https://www.facebook.com/whitehat/) for the safe\ndisclosure of security bugs. In those cases, please go through the process\noutlined on that page and do not file a public issue.\n\n## Coding Style \n* 4 spaces for indentation rather than tabs\n* 80 character line length\n* PEP8 formatting following [Black](https://black.readthedocs.io/en/stable/)\n\n## License\nBy contributing to DETR, you agree that your contributions will be licensed\nunder the LICENSE file in the root directory of this source tree.\n"
},
{
"path": "batchformer-v2/detr/.github/ISSUE_TEMPLATE/bugs.md",
"content": "---\nname: \"🐛 Bugs\"\nabout: Report bugs in DETR\ntitle: Please read & provide the following\n\n---\n\n## Instructions To Reproduce the 🐛 Bug:\n\n1. what changes you made (`git diff`) or what code you wrote\n```\n\n```\n2. what exact command you run:\n3. what you observed (including __full logs__):\n```\n\n```\n4. please simplify the steps as much as possible so they do not require additional resources to\n\t run, such as a private dataset.\n\n## Expected behavior:\n\nIf there are no obvious error in \"what you observed\" provided above,\nplease tell us the expected behavior.\n\n## Environment:\n\nProvide your environment information using the following command:\n```\npython -m torch.utils.collect_env\n```\n"
},
{
"path": "batchformer-v2/detr/.github/ISSUE_TEMPLATE/questions-help-support.md",
"content": "---\nname: \"How to do something❓\"\nabout: How to do something using DETR?\n\n---\n\n## ❓ How to do something using DETR\n\nDescribe what you want to do, including:\n1. what inputs you will provide, if any:\n2. what outputs you are expecting:\n\n\nNOTE:\n\n1. Only general answers are provided.\n If you want to ask about \"why X did not work\", please use the\n [Unexpected behaviors](https://github.com/facebookresearch/detr/issues/new/choose) issue template.\n\n2. About how to implement new models / new dataloader / new training logic, etc., check documentation first.\n\n3. We do not answer general machine learning / computer vision questions that are not specific to DETR, such as how a model works, how to improve your training/make it converge, or what algorithm/methods can be used to achieve X.\n"
},
{
"path": "batchformer-v2/detr/.github/ISSUE_TEMPLATE/unexpected-problems-bugs.md",
"content": "---\nname: \"Unexpected behaviors\"\nabout: Run into unexpected behaviors when using DETR\ntitle: Please read & provide the following\n\n---\n\nIf you do not know the root cause of the problem, and wish someone to help you, please\npost according to this template:\n\n## Instructions To Reproduce the Issue:\n\n1. what changes you made (`git diff`) or what code you wrote\n```\n\n```\n2. what exact command you run:\n3. what you observed (including __full logs__):\n```\n\n```\n4. please simplify the steps as much as possible so they do not require additional resources to\n\t run, such as a private dataset.\n\n## Expected behavior:\n\nIf there are no obvious error in \"what you observed\" provided above,\nplease tell us the expected behavior.\n\nIf you expect the model to converge / work better, note that we do not give suggestions\non how to train a new model.\nOnly in one of the two conditions we will help with it:\n(1) You're unable to reproduce the results in DETR model zoo.\n(2) It indicates a DETR bug.\n\n## Environment:\n\nProvide your environment information using the following command:\n```\npython -m torch.utils.collect_env\n```\n"
},
{
"path": "batchformer-v2/detr/.gitignore",
"content": ".nfs*\n*.ipynb\n*.pyc\n.dumbo.json\n.DS_Store\n.*.swp\n*.pth\n**/__pycache__/**\n.ipynb_checkpoints/\ndatasets/data/\nexperiment-*\n*.tmp\n*.pkl\n**/.mypy_cache/*\n.mypy_cache/*\nnot_tracked_dir/\n.vscode\n"
},
{
"path": "batchformer-v2/detr/Dockerfile",
"content": "FROM pytorch/pytorch:1.5-cuda10.1-cudnn7-runtime\n\nENV DEBIAN_FRONTEND=noninteractive\n\nRUN apt-get update -qq && \\\n apt-get install -y git vim libgtk2.0-dev && \\\n rm -rf /var/cache/apk/*\n\nRUN pip --no-cache-dir install Cython\n\nCOPY requirements.txt /workspace\n\nRUN pip --no-cache-dir install -r /workspace/requirements.txt\n"
},
{
"path": "batchformer-v2/detr/LICENSE",
"content": " Apache License\n Version 2.0, January 2004\n http://www.apache.org/licenses/\n\n TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION\n\n 1. Definitions.\n\n \"License\" shall mean the terms and conditions for use, reproduction,\n and distribution as defined by Sections 1 through 9 of this document.\n\n \"Licensor\" shall mean the copyright owner or entity authorized by\n the copyright owner that is granting the License.\n\n \"Legal Entity\" shall mean the union of the acting entity and all\n other entities that control, are controlled by, or are under common\n control with that entity. For the purposes of this definition,\n \"control\" means (i) the power, direct or indirect, to cause the\n direction or management of such entity, whether by contract or\n otherwise, or (ii) ownership of fifty percent (50%) or more of the\n outstanding shares, or (iii) beneficial ownership of such entity.\n\n \"You\" (or \"Your\") shall mean an individual or Legal Entity\n exercising permissions granted by this License.\n\n \"Source\" form shall mean the preferred form for making modifications,\n including but not limited to software source code, documentation\n source, and configuration files.\n\n \"Object\" form shall mean any form resulting from mechanical\n transformation or translation of a Source form, including but\n not limited to compiled object code, generated documentation,\n and conversions to other media types.\n\n \"Work\" shall mean the work of authorship, whether in Source or\n Object form, made available under the License, as indicated by a\n copyright notice that is included in or attached to the work\n (an example is provided in the Appendix below).\n\n \"Derivative Works\" shall mean any work, whether in Source or Object\n form, that is based on (or derived from) the Work and for which the\n editorial revisions, annotations, elaborations, or other modifications\n represent, as a whole, an original work of authorship. For the purposes\n of this License, Derivative Works shall not include works that remain\n separable from, or merely link (or bind by name) to the interfaces of,\n the Work and Derivative Works thereof.\n\n \"Contribution\" shall mean any work of authorship, including\n the original version of the Work and any modifications or additions\n to that Work or Derivative Works thereof, that is intentionally\n submitted to Licensor for inclusion in the Work by the copyright owner\n or by an individual or Legal Entity authorized to submit on behalf of\n the copyright owner. For the purposes of this definition, \"submitted\"\n means any form of electronic, verbal, or written communication sent\n to the Licensor or its representatives, including but not limited to\n communication on electronic mailing lists, source code control systems,\n and issue tracking systems that are managed by, or on behalf of, the\n Licensor for the purpose of discussing and improving the Work, but\n excluding communication that is conspicuously marked or otherwise\n designated in writing by the copyright owner as \"Not a Contribution.\"\n\n \"Contributor\" shall mean Licensor and any individual or Legal Entity\n on behalf of whom a Contribution has been received by Licensor and\n subsequently incorporated within the Work.\n\n 2. Grant of Copyright License. Subject to the terms and conditions of\n this License, each Contributor hereby grants to You a perpetual,\n worldwide, non-exclusive, no-charge, royalty-free, irrevocable\n copyright license to reproduce, prepare Derivative Works of,\n publicly display, publicly perform, sublicense, and distribute the\n Work and such Derivative Works in Source or Object form.\n\n 3. Grant of Patent License. Subject to the terms and conditions of\n this License, each Contributor hereby grants to You a perpetual,\n worldwide, non-exclusive, no-charge, royalty-free, irrevocable\n (except as stated in this section) patent license to make, have made,\n use, offer to sell, sell, import, and otherwise transfer the Work,\n where such license applies only to those patent claims licensable\n by such Contributor that are necessarily infringed by their\n Contribution(s) alone or by combination of their Contribution(s)\n with the Work to which such Contribution(s) was submitted. If You\n institute patent litigation against any entity (including a\n cross-claim or counterclaim in a lawsuit) alleging that the Work\n or a Contribution incorporated within the Work constitutes direct\n or contributory patent infringement, then any patent licenses\n granted to You under this License for that Work shall terminate\n as of the date such litigation is filed.\n\n 4. Redistribution. You may reproduce and distribute copies of the\n Work or Derivative Works thereof in any medium, with or without\n modifications, and in Source or Object form, provided that You\n meet the following conditions:\n\n (a) You must give any other recipients of the Work or\n Derivative Works a copy of this License; and\n\n (b) You must cause any modified files to carry prominent notices\n stating that You changed the files; and\n\n (c) You must retain, in the Source form of any Derivative Works\n that You distribute, all copyright, patent, trademark, and\n attribution notices from the Source form of the Work,\n excluding those notices that do not pertain to any part of\n the Derivative Works; and\n\n (d) If the Work includes a \"NOTICE\" text file as part of its\n distribution, then any Derivative Works that You distribute must\n include a readable copy of the attribution notices contained\n within such NOTICE file, excluding those notices that do not\n pertain to any part of the Derivative Works, in at least one\n of the following places: within a NOTICE text file distributed\n as part of the Derivative Works; within the Source form or\n documentation, if provided along with the Derivative Works; or,\n within a display generated by the Derivative Works, if and\n wherever such third-party notices normally appear. The contents\n of the NOTICE file are for informational purposes only and\n do not modify the License. You may add Your own attribution\n notices within Derivative Works that You distribute, alongside\n or as an addendum to the NOTICE text from the Work, provided\n that such additional attribution notices cannot be construed\n as modifying the License.\n\n You may add Your own copyright statement to Your modifications and\n may provide additional or different license terms and conditions\n for use, reproduction, or distribution of Your modifications, or\n for any such Derivative Works as a whole, provided Your use,\n reproduction, and distribution of the Work otherwise complies with\n the conditions stated in this License.\n\n 5. Submission of Contributions. Unless You explicitly state otherwise,\n any Contribution intentionally submitted for inclusion in the Work\n by You to the Licensor shall be under the terms and conditions of\n this License, without any additional terms or conditions.\n Notwithstanding the above, nothing herein shall supersede or modify\n the terms of any separate license agreement you may have executed\n with Licensor regarding such Contributions.\n\n 6. Trademarks. This License does not grant permission to use the trade\n names, trademarks, service marks, or product names of the Licensor,\n except as required for reasonable and customary use in describing the\n origin of the Work and reproducing the content of the NOTICE file.\n\n 7. Disclaimer of Warranty. Unless required by applicable law or\n agreed to in writing, Licensor provides the Work (and each\n Contributor provides its Contributions) on an \"AS IS\" BASIS,\n WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or\n implied, including, without limitation, any warranties or conditions\n of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A\n PARTICULAR PURPOSE. You are solely responsible for determining the\n appropriateness of using or redistributing the Work and assume any\n risks associated with Your exercise of permissions under this License.\n\n 8. Limitation of Liability. In no event and under no legal theory,\n whether in tort (including negligence), contract, or otherwise,\n unless required by applicable law (such as deliberate and grossly\n negligent acts) or agreed to in writing, shall any Contributor be\n liable to You for damages, including any direct, indirect, special,\n incidental, or consequential damages of any character arising as a\n result of this License or out of the use or inability to use the\n Work (including but not limited to damages for loss of goodwill,\n work stoppage, computer failure or malfunction, or any and all\n other commercial damages or losses), even if such Contributor\n has been advised of the possibility of such damages.\n\n 9. Accepting Warranty or Additional Liability. While redistributing\n the Work or Derivative Works thereof, You may choose to offer,\n and charge a fee for, acceptance of support, warranty, indemnity,\n or other liability obligations and/or rights consistent with this\n License. However, in accepting such obligations, You may act only\n on Your own behalf and on Your sole responsibility, not on behalf\n of any other Contributor, and only if You agree to indemnify,\n defend, and hold each Contributor harmless for any liability\n incurred by, or claims asserted against, such Contributor by reason\n of your accepting any such warranty or additional liability.\n\n END OF TERMS AND CONDITIONS\n\n APPENDIX: How to apply the Apache License to your work.\n\n To apply the Apache License to your work, attach the following\n boilerplate notice, with the fields enclosed by brackets \"[]\"\n replaced with your own identifying information. (Don't include\n the brackets!) The text should be enclosed in the appropriate\n comment syntax for the file format. We also recommend that a\n file or class name and description of purpose be included on the\n same \"printed page\" as the copyright notice for easier\n identification within third-party archives.\n\n Copyright 2020 - present, Facebook, Inc\n\n Licensed under the Apache License, Version 2.0 (the \"License\");\n you may not use this file except in compliance with the License.\n You may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\n Unless required by applicable law or agreed to in writing, software\n distributed under the License is distributed on an \"AS IS\" BASIS,\n WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n See the License for the specific language governing permissions and\n limitations under the License.\n"
},
{
"path": "batchformer-v2/detr/README.md",
"content": "This is the repository based on DETR. This repository includes the implementation of batchformer based on DETR. The corresponding code starts from here [https://github.com/zhihou7/detr/blob/449d425a8ae8465512234bc3f8750ff07f5c473b/models/transformer.py#L33](https://github.com/zhihou7/detr/blob/449d425a8ae8465512234bc3f8750ff07f5c473b/models/transformer.py#L33). \n\nPlease ignore the \"checkpoint\" that I actually do not use it.\nCore part of the code is \n\n[https://github.com/zhihou7/detr/blob/449d425a8ae8465512234bc3f8750ff07f5c473b/models/transformer.py#L121](https://github.com/zhihou7/detr/blob/449d425a8ae8465512234bc3f8750ff07f5c473b/models/transformer.py#L121)\n\nHowever, this repository might include some codes for my other projects. For the redundant code, you can just ignore it.\n\n\nFor this batchformerv2 experiment, I insert the batchformerv2 module in the first layer (i.e. insert_index=0 and bf_idx=3). Meanwhile, I run the experiments on 16G V100 and set the batch size as 8 with 4 gpus.\n"
},
{
"path": "batchformer-v2/detr/configs/base.sh",
"content": "#!/usr/bin/env bash\n\nsource /usr/local/anaconda/4.2.0/etc/profile.d/conda.sh\nconda activate pyt\n\nset -x\n\npython main.py --coco_path /project/ZHIHOU/Dataset/coco/ --bf 1 --batch_size 2 --output_dir bf1 --resume bf1/checkpoint.pth\n\n"
},
{
"path": "batchformer-v2/detr/configs/panoptic.sh",
"content": "#!/usr/bin/env bash\n\nsource /usr/local/anaconda/4.2.0/etc/profile.d/conda.sh\nconda activate pyt\n\nset -x\n\n#python main.py --coco_path /project/ZHIHOU/Dataset/coco/ --bf 1 --batch_size 2 --output_dir bf1\n\n#python -m torch.distributed.launch --nproc_per_node=8 --use_env\npython main.py --coco_path /project/ZHIHOU/Dataset/coco/ \\\n--coco_panoptic_path /project/ZHIHOU/Dataset/coco/ \\\n--dataset_file coco_panoptic \\\n--output_dir panoptic_bf3 --bf 3 --resume panoptic_bf3/checkpoint.pth"
},
{
"path": "batchformer-v2/detr/configs/panoptic_1.sh",
"content": "#!/usr/bin/env bash\n\nsource /usr/local/anaconda/4.2.0/etc/profile.d/conda.sh\nconda activate pyt\n\nset -x\n\n\npython main.py --masks --epochs 25 --lr_drop 15 --coco_path /project/ZHIHOU/Dataset/coco/ \\\n--coco_panoptic_path /project/ZHIHOU/Dataset/coco/ \\\n--dataset_file coco_panoptic \\\n--frozen_weights panoptic_bf3/checkpoint0249.pth \\\n--output_dir panoptic_bf0_segm --batch_size 1"
},
{
"path": "batchformer-v2/detr/configs/panoptic_2.sh",
"content": "#!/usr/bin/env bash\n\nsource /usr/local/anaconda/4.2.0/etc/profile.d/conda.sh\nconda activate pyt\n\nset -x\n\n\npython main.py --masks --epochs 25 --lr_drop 15 --coco_path /project/ZHIHOU/Dataset/coco/ \\\n--coco_panoptic_path /project/ZHIHOU/Dataset/coco/ \\\n--dataset_file coco_panoptic \\\n--frozen_weights panoptic_bf3/checkpoint0249.pth \\\n--use_checkpoint \\\n--output_dir panoptic_bf3_segm --bf 3 --batch_size 2"
},
{
"path": "batchformer-v2/detr/configs/swin.sh",
"content": "#!/usr/bin/env bash\n\nsource /usr/local/anaconda/4.2.0/etc/profile.d/conda.sh\nconda activate pyt\n\nset -x\n\npython main.py --coco_path /project/ZHIHOU/Dataset/coco/ --bf 3 --batch_size 2 --output_dir output_bf3_7 --resume output_bf3_7/checkpoint.pth\n\n"
},
{
"path": "batchformer-v2/detr/d2/README.md",
"content": "Detectron2 wrapper for DETR\n=======\n\nWe provide a Detectron2 wrapper for DETR, thus providing a way to better integrate it in the existing detection ecosystem. It can be used for example to easily leverage datasets or backbones provided in Detectron2.\n\nThis wrapper currently supports only box detection, and is intended to be as close as possible to the original implementation, and we checked that it indeed match the results. Some notable facts and caveats:\n- The data augmentation matches DETR's original data augmentation. This required patching the RandomCrop augmentation from Detectron2, so you'll need a version from the master branch from June 24th 2020 or more recent.\n- To match DETR's original backbone initialization, we use the weights of a ResNet50 trained on imagenet using torchvision. This network uses a different pixel mean and std than most of the backbones available in Detectron2 by default, so extra care must be taken when switching to another one. Note that no other torchvision models are available in Detectron2 as of now, though it may change in the future.\n- The gradient clipping mode is \"full_model\", which is not the default in Detectron2.\n\n# Usage\n\nTo install Detectron2, please follow the [official installation instructions](https://github.com/facebookresearch/detectron2/blob/master/INSTALL.md).\n\n## Evaluating a model\n\nFor convenience, we provide a conversion script to convert models trained by the main DETR training loop into the format of this wrapper. To download and convert the main Resnet50 model, simply do:\n\n```\npython converter.py --source_model https://dl.fbaipublicfiles.com/detr/detr-r50-e632da11.pth --output_model converted_model.pth\n```\n\nYou can then evaluate it using:\n```\npython train_net.py --eval-only --config configs/detr_256_6_6_torchvision.yaml MODEL.WEIGHTS \"converted_model.pth\"\n```\n\n\n## Training\n\nTo train DETR on a single node with 8 gpus, simply use:\n```\npython train_net.py --config configs/detr_256_6_6_torchvision.yaml --num-gpus 8\n```\n\nTo fine-tune DETR for instance segmentation on a single node with 8 gpus, simply use:\n```\npython train_net.py --config configs/detr_segm_256_6_6_torchvision.yaml --num-gpus 8 MODEL.DETR.FROZEN_WEIGHTS \n```\n"
},
{
"path": "batchformer-v2/detr/d2/configs/detr_256_6_6_torchvision.yaml",
"content": "MODEL:\n META_ARCHITECTURE: \"Detr\"\n WEIGHTS: \"detectron2://ImageNetPretrained/torchvision/R-50.pkl\"\n PIXEL_MEAN: [123.675, 116.280, 103.530]\n PIXEL_STD: [58.395, 57.120, 57.375]\n MASK_ON: False\n RESNETS:\n DEPTH: 50\n STRIDE_IN_1X1: False\n OUT_FEATURES: [\"res2\", \"res3\", \"res4\", \"res5\"]\n DETR:\n GIOU_WEIGHT: 2.0\n L1_WEIGHT: 5.0\n NUM_OBJECT_QUERIES: 100\nDATASETS:\n TRAIN: (\"coco_2017_train\",)\n TEST: (\"coco_2017_val\",)\nSOLVER:\n IMS_PER_BATCH: 64\n BASE_LR: 0.0001\n STEPS: (369600,)\n MAX_ITER: 554400\n WARMUP_FACTOR: 1.0\n WARMUP_ITERS: 10\n WEIGHT_DECAY: 0.0001\n OPTIMIZER: \"ADAMW\"\n BACKBONE_MULTIPLIER: 0.1\n CLIP_GRADIENTS:\n ENABLED: True\n CLIP_TYPE: \"full_model\"\n CLIP_VALUE: 0.01\n NORM_TYPE: 2.0\nINPUT:\n MIN_SIZE_TRAIN: (480, 512, 544, 576, 608, 640, 672, 704, 736, 768, 800)\n CROP:\n ENABLED: True\n TYPE: \"absolute_range\"\n SIZE: (384, 600)\n FORMAT: \"RGB\"\nTEST:\n EVAL_PERIOD: 4000\nDATALOADER:\n FILTER_EMPTY_ANNOTATIONS: False\n NUM_WORKERS: 4\nVERSION: 2\n"
},
{
"path": "batchformer-v2/detr/d2/configs/detr_segm_256_6_6_torchvision.yaml",
"content": "MODEL:\n META_ARCHITECTURE: \"Detr\"\n# WEIGHTS: \"detectron2://ImageNetPretrained/torchvision/R-50.pkl\"\n PIXEL_MEAN: [123.675, 116.280, 103.530]\n PIXEL_STD: [58.395, 57.120, 57.375]\n MASK_ON: True\n RESNETS:\n DEPTH: 50\n STRIDE_IN_1X1: False\n OUT_FEATURES: [\"res2\", \"res3\", \"res4\", \"res5\"]\n DETR:\n GIOU_WEIGHT: 2.0\n L1_WEIGHT: 5.0\n NUM_OBJECT_QUERIES: 100\n FROZEN_WEIGHTS: ''\nDATASETS:\n TRAIN: (\"coco_2017_train\",)\n TEST: (\"coco_2017_val\",)\nSOLVER:\n IMS_PER_BATCH: 64\n BASE_LR: 0.0001\n STEPS: (55440,)\n MAX_ITER: 92400\n WARMUP_FACTOR: 1.0\n WARMUP_ITERS: 10\n WEIGHT_DECAY: 0.0001\n OPTIMIZER: \"ADAMW\"\n BACKBONE_MULTIPLIER: 0.1\n CLIP_GRADIENTS:\n ENABLED: True\n CLIP_TYPE: \"full_model\"\n CLIP_VALUE: 0.01\n NORM_TYPE: 2.0\nINPUT:\n MIN_SIZE_TRAIN: (480, 512, 544, 576, 608, 640, 672, 704, 736, 768, 800)\n CROP:\n ENABLED: True\n TYPE: \"absolute_range\"\n SIZE: (384, 600)\n FORMAT: \"RGB\"\nTEST:\n EVAL_PERIOD: 4000\nDATALOADER:\n FILTER_EMPTY_ANNOTATIONS: False\n NUM_WORKERS: 4\nVERSION: 2\n"
},
{
"path": "batchformer-v2/detr/d2/converter.py",
"content": "# Copyright (c) Facebook, Inc. and its affiliates. All Rights Reserved\n\"\"\"\nHelper script to convert models trained with the main version of DETR to be used with the Detectron2 version.\n\"\"\"\nimport json\nimport argparse\n\nimport numpy as np\nimport torch\n\n\ndef parse_args():\n parser = argparse.ArgumentParser(\"D2 model converter\")\n\n parser.add_argument(\"--source_model\", default=\"\", type=str, help=\"Path or url to the DETR model to convert\")\n parser.add_argument(\"--output_model\", default=\"\", type=str, help=\"Path where to save the converted model\")\n return parser.parse_args()\n\n\ndef main():\n args = parse_args()\n\n # D2 expects contiguous classes, so we need to remap the 92 classes from DETR\n # fmt: off\n coco_idx = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25,\n 27, 28, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 46, 47, 48, 49, 50, 51,\n 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 67, 70, 72, 73, 74, 75, 76, 77,\n 78, 79, 80, 81, 82, 84, 85, 86, 87, 88, 89, 90, 91]\n # fmt: on\n\n coco_idx = np.array(coco_idx)\n\n if args.source_model.startswith(\"https\"):\n checkpoint = torch.hub.load_state_dict_from_url(args.source_model, map_location=\"cpu\", check_hash=True)\n else:\n checkpoint = torch.load(args.source_model, map_location=\"cpu\")\n model_to_convert = checkpoint[\"model\"]\n\n model_converted = {}\n for k in model_to_convert.keys():\n old_k = k\n if \"backbone\" in k:\n k = k.replace(\"backbone.0.body.\", \"\")\n if \"layer\" not in k:\n k = \"stem.\" + k\n for t in [1, 2, 3, 4]:\n k = k.replace(f\"layer{t}\", f\"res{t + 1}\")\n for t in [1, 2, 3]:\n k = k.replace(f\"bn{t}\", f\"conv{t}.norm\")\n k = k.replace(\"downsample.0\", \"shortcut\")\n k = k.replace(\"downsample.1\", \"shortcut.norm\")\n k = \"backbone.0.backbone.\" + k\n k = \"detr.\" + k\n print(old_k, \"->\", k)\n if \"class_embed\" in old_k:\n v = model_to_convert[old_k].detach()\n if v.shape[0] == 92:\n shape_old = v.shape\n model_converted[k] = v[coco_idx]\n print(\"Head conversion: changing shape from {} to {}\".format(shape_old, model_converted[k].shape))\n continue\n model_converted[k] = model_to_convert[old_k].detach()\n\n model_to_save = {\"model\": model_converted}\n torch.save(model_to_save, args.output_model)\n\n\nif __name__ == \"__main__\":\n main()\n"
},
{
"path": "batchformer-v2/detr/d2/detr/__init__.py",
"content": "# Copyright (c) Facebook, Inc. and its affiliates. All Rights Reserved\nfrom .config import add_detr_config\nfrom .detr import Detr\nfrom .dataset_mapper import DetrDatasetMapper\n"
},
{
"path": "batchformer-v2/detr/d2/detr/config.py",
"content": "# -*- coding: utf-8 -*-\n# Copyright (c) Facebook, Inc. and its affiliates. All Rights Reserved\nfrom detectron2.config import CfgNode as CN\n\n\ndef add_detr_config(cfg):\n \"\"\"\n Add config for DETR.\n \"\"\"\n cfg.MODEL.DETR = CN()\n cfg.MODEL.DETR.NUM_CLASSES = 80\n\n # For Segmentation\n cfg.MODEL.DETR.FROZEN_WEIGHTS = ''\n\n # LOSS\n cfg.MODEL.DETR.GIOU_WEIGHT = 2.0\n cfg.MODEL.DETR.L1_WEIGHT = 5.0\n cfg.MODEL.DETR.DEEP_SUPERVISION = True\n cfg.MODEL.DETR.NO_OBJECT_WEIGHT = 0.1\n\n # TRANSFORMER\n cfg.MODEL.DETR.NHEADS = 8\n cfg.MODEL.DETR.DROPOUT = 0.1\n cfg.MODEL.DETR.DIM_FEEDFORWARD = 2048\n cfg.MODEL.DETR.ENC_LAYERS = 6\n cfg.MODEL.DETR.DEC_LAYERS = 6\n cfg.MODEL.DETR.PRE_NORM = False\n\n cfg.MODEL.DETR.HIDDEN_DIM = 256\n cfg.MODEL.DETR.NUM_OBJECT_QUERIES = 100\n\n cfg.SOLVER.OPTIMIZER = \"ADAMW\"\n cfg.SOLVER.BACKBONE_MULTIPLIER = 0.1\n"
},
{
"path": "batchformer-v2/detr/d2/detr/dataset_mapper.py",
"content": "# Copyright (c) Facebook, Inc. and its affiliates. All Rights Reserved\nimport copy\nimport logging\n\nimport numpy as np\nimport torch\n\nfrom detectron2.data import detection_utils as utils\nfrom detectron2.data import transforms as T\nfrom detectron2.data.transforms import TransformGen\n\n__all__ = [\"DetrDatasetMapper\"]\n\n\ndef build_transform_gen(cfg, is_train):\n \"\"\"\n Create a list of :class:`TransformGen` from config.\n Returns:\n list[TransformGen]\n \"\"\"\n if is_train:\n min_size = cfg.INPUT.MIN_SIZE_TRAIN\n max_size = cfg.INPUT.MAX_SIZE_TRAIN\n sample_style = cfg.INPUT.MIN_SIZE_TRAIN_SAMPLING\n else:\n min_size = cfg.INPUT.MIN_SIZE_TEST\n max_size = cfg.INPUT.MAX_SIZE_TEST\n sample_style = \"choice\"\n if sample_style == \"range\":\n assert len(min_size) == 2, \"more than 2 ({}) min_size(s) are provided for ranges\".format(len(min_size))\n\n logger = logging.getLogger(__name__)\n tfm_gens = []\n if is_train:\n tfm_gens.append(T.RandomFlip())\n tfm_gens.append(T.ResizeShortestEdge(min_size, max_size, sample_style))\n if is_train:\n logger.info(\"TransformGens used in training: \" + str(tfm_gens))\n return tfm_gens\n\n\nclass DetrDatasetMapper:\n \"\"\"\n A callable which takes a dataset dict in Detectron2 Dataset format,\n and map it into a format used by DETR.\n\n The callable currently does the following:\n\n 1. Read the image from \"file_name\"\n 2. Applies geometric transforms to the image and annotation\n 3. Find and applies suitable cropping to the image and annotation\n 4. Prepare image and annotation to Tensors\n \"\"\"\n\n def __init__(self, cfg, is_train=True):\n if cfg.INPUT.CROP.ENABLED and is_train:\n self.crop_gen = [\n T.ResizeShortestEdge([400, 500, 600], sample_style=\"choice\"),\n T.RandomCrop(cfg.INPUT.CROP.TYPE, cfg.INPUT.CROP.SIZE),\n ]\n else:\n self.crop_gen = None\n\n self.mask_on = cfg.MODEL.MASK_ON\n self.tfm_gens = build_transform_gen(cfg, is_train)\n logging.getLogger(__name__).info(\n \"Full TransformGens used in training: {}, crop: {}\".format(str(self.tfm_gens), str(self.crop_gen))\n )\n\n self.img_format = cfg.INPUT.FORMAT\n self.is_train = is_train\n\n def __call__(self, dataset_dict):\n \"\"\"\n Args:\n dataset_dict (dict): Metadata of one image, in Detectron2 Dataset format.\n\n Returns:\n dict: a format that builtin models in detectron2 accept\n \"\"\"\n dataset_dict = copy.deepcopy(dataset_dict) # it will be modified by code below\n image = utils.read_image(dataset_dict[\"file_name\"], format=self.img_format)\n utils.check_image_size(dataset_dict, image)\n\n if self.crop_gen is None:\n image, transforms = T.apply_transform_gens(self.tfm_gens, image)\n else:\n if np.random.rand() > 0.5:\n image, transforms = T.apply_transform_gens(self.tfm_gens, image)\n else:\n image, transforms = T.apply_transform_gens(\n self.tfm_gens[:-1] + self.crop_gen + self.tfm_gens[-1:], image\n )\n\n image_shape = image.shape[:2] # h, w\n\n # Pytorch's dataloader is efficient on torch.Tensor due to shared-memory,\n # but not efficient on large generic data structures due to the use of pickle & mp.Queue.\n # Therefore it's important to use torch.Tensor.\n dataset_dict[\"image\"] = torch.as_tensor(np.ascontiguousarray(image.transpose(2, 0, 1)))\n\n if not self.is_train:\n # USER: Modify this if you want to keep them for some reason.\n dataset_dict.pop(\"annotations\", None)\n return dataset_dict\n\n if \"annotations\" in dataset_dict:\n # USER: Modify this if you want to keep them for some reason.\n for anno in dataset_dict[\"annotations\"]:\n if not self.mask_on:\n anno.pop(\"segmentation\", None)\n anno.pop(\"keypoints\", None)\n\n # USER: Implement additional transformations if you have other types of data\n annos = [\n utils.transform_instance_annotations(obj, transforms, image_shape)\n for obj in dataset_dict.pop(\"annotations\")\n if obj.get(\"iscrowd\", 0) == 0\n ]\n instances = utils.annotations_to_instances(annos, image_shape)\n dataset_dict[\"instances\"] = utils.filter_empty_instances(instances)\n return dataset_dict\n"
},
{
"path": "batchformer-v2/detr/d2/detr/detr.py",
"content": "# Copyright (c) Facebook, Inc. and its affiliates. All Rights Reserved\nimport logging\nimport math\nfrom typing import List\n\nimport numpy as np\nimport torch\nimport torch.distributed as dist\nimport torch.nn.functional as F\nfrom scipy.optimize import linear_sum_assignment\nfrom torch import nn\n\nfrom detectron2.layers import ShapeSpec\nfrom detectron2.modeling import META_ARCH_REGISTRY, build_backbone, detector_postprocess\nfrom detectron2.structures import Boxes, ImageList, Instances, BitMasks, PolygonMasks\nfrom detectron2.utils.logger import log_first_n\nfrom fvcore.nn import giou_loss, smooth_l1_loss\nfrom models.backbone import Joiner\nfrom models.detr import DETR, SetCriterion\nfrom models.matcher import HungarianMatcher\nfrom models.position_encoding import PositionEmbeddingSine\nfrom models.transformer import Transformer\nfrom models.segmentation import DETRsegm, PostProcessPanoptic, PostProcessSegm\nfrom util.box_ops import box_cxcywh_to_xyxy, box_xyxy_to_cxcywh\nfrom util.misc import NestedTensor\nfrom datasets.coco import convert_coco_poly_to_mask\n\n__all__ = [\"Detr\"]\n\n\nclass MaskedBackbone(nn.Module):\n \"\"\" This is a thin wrapper around D2's backbone to provide padding masking\"\"\"\n\n def __init__(self, cfg):\n super().__init__()\n self.backbone = build_backbone(cfg)\n backbone_shape = self.backbone.output_shape()\n self.feature_strides = [backbone_shape[f].stride for f in backbone_shape.keys()]\n self.num_channels = backbone_shape[list(backbone_shape.keys())[-1]].channels\n\n def forward(self, images):\n features = self.backbone(images.tensor)\n masks = self.mask_out_padding(\n [features_per_level.shape for features_per_level in features.values()],\n images.image_sizes,\n images.tensor.device,\n )\n assert len(features) == len(masks)\n for i, k in enumerate(features.keys()):\n features[k] = NestedTensor(features[k], masks[i])\n return features\n\n def mask_out_padding(self, feature_shapes, image_sizes, device):\n masks = []\n assert len(feature_shapes) == len(self.feature_strides)\n for idx, shape in enumerate(feature_shapes):\n N, _, H, W = shape\n masks_per_feature_level = torch.ones((N, H, W), dtype=torch.bool, device=device)\n for img_idx, (h, w) in enumerate(image_sizes):\n masks_per_feature_level[\n img_idx,\n : int(np.ceil(float(h) / self.feature_strides[idx])),\n : int(np.ceil(float(w) / self.feature_strides[idx])),\n ] = 0\n masks.append(masks_per_feature_level)\n return masks\n\n\n@META_ARCH_REGISTRY.register()\nclass Detr(nn.Module):\n \"\"\"\n Implement Detr\n \"\"\"\n\n def __init__(self, cfg):\n super().__init__()\n\n self.device = torch.device(cfg.MODEL.DEVICE)\n\n self.num_classes = cfg.MODEL.DETR.NUM_CLASSES\n self.mask_on = cfg.MODEL.MASK_ON\n hidden_dim = cfg.MODEL.DETR.HIDDEN_DIM\n num_queries = cfg.MODEL.DETR.NUM_OBJECT_QUERIES\n # Transformer parameters:\n nheads = cfg.MODEL.DETR.NHEADS\n dropout = cfg.MODEL.DETR.DROPOUT\n dim_feedforward = cfg.MODEL.DETR.DIM_FEEDFORWARD\n enc_layers = cfg.MODEL.DETR.ENC_LAYERS\n dec_layers = cfg.MODEL.DETR.DEC_LAYERS\n pre_norm = cfg.MODEL.DETR.PRE_NORM\n\n # Loss parameters:\n giou_weight = cfg.MODEL.DETR.GIOU_WEIGHT\n l1_weight = cfg.MODEL.DETR.L1_WEIGHT\n deep_supervision = cfg.MODEL.DETR.DEEP_SUPERVISION\n no_object_weight = cfg.MODEL.DETR.NO_OBJECT_WEIGHT\n\n N_steps = hidden_dim // 2\n d2_backbone = MaskedBackbone(cfg)\n backbone = Joiner(d2_backbone, PositionEmbeddingSine(N_steps, normalize=True))\n backbone.num_channels = d2_backbone.num_channels\n\n transformer = Transformer(\n d_model=hidden_dim,\n dropout=dropout,\n nhead=nheads,\n dim_feedforward=dim_feedforward,\n num_encoder_layers=enc_layers,\n num_decoder_layers=dec_layers,\n normalize_before=pre_norm,\n return_intermediate_dec=deep_supervision,\n )\n\n self.detr = DETR(\n backbone, transformer, num_classes=self.num_classes, num_queries=num_queries, aux_loss=deep_supervision\n )\n if self.mask_on:\n frozen_weights = cfg.MODEL.DETR.FROZEN_WEIGHTS\n if frozen_weights != '':\n print(\"LOAD pre-trained weights\")\n weight = torch.load(frozen_weights, map_location=lambda storage, loc: storage)['model']\n new_weight = {}\n for k, v in weight.items():\n if 'detr.' in k:\n new_weight[k.replace('detr.', '')] = v\n else:\n print(f\"Skipping loading weight {k} from frozen model\")\n del weight\n self.detr.load_state_dict(new_weight)\n del new_weight\n self.detr = DETRsegm(self.detr, freeze_detr=(frozen_weights != ''))\n self.seg_postprocess = PostProcessSegm\n\n self.detr.to(self.device)\n\n # building criterion\n matcher = HungarianMatcher(cost_class=1, cost_bbox=l1_weight, cost_giou=giou_weight)\n weight_dict = {\"loss_ce\": 1, \"loss_bbox\": l1_weight}\n weight_dict[\"loss_giou\"] = giou_weight\n if deep_supervision:\n aux_weight_dict = {}\n for i in range(dec_layers - 1):\n aux_weight_dict.update({k + f\"_{i}\": v for k, v in weight_dict.items()})\n weight_dict.update(aux_weight_dict)\n losses = [\"labels\", \"boxes\", \"cardinality\"]\n if self.mask_on:\n losses += [\"masks\"]\n self.criterion = SetCriterion(\n self.num_classes, matcher=matcher, weight_dict=weight_dict, eos_coef=no_object_weight, losses=losses,\n )\n self.criterion.to(self.device)\n\n pixel_mean = torch.Tensor(cfg.MODEL.PIXEL_MEAN).to(self.device).view(3, 1, 1)\n pixel_std = torch.Tensor(cfg.MODEL.PIXEL_STD).to(self.device).view(3, 1, 1)\n self.normalizer = lambda x: (x - pixel_mean) / pixel_std\n self.to(self.device)\n\n def forward(self, batched_inputs):\n \"\"\"\n Args:\n batched_inputs: a list, batched outputs of :class:`DatasetMapper` .\n Each item in the list contains the inputs for one image.\n For now, each item in the list is a dict that contains:\n\n * image: Tensor, image in (C, H, W) format.\n * instances: Instances\n\n Other information that's included in the original dicts, such as:\n\n * \"height\", \"width\" (int): the output resolution of the model, used in inference.\n See :meth:`postprocess` for details.\n Returns:\n dict[str: Tensor]:\n mapping from a named loss to a tensor storing the loss. Used during training only.\n \"\"\"\n images = self.preprocess_image(batched_inputs)\n output = self.detr(images)\n\n if self.training:\n gt_instances = [x[\"instances\"].to(self.device) for x in batched_inputs]\n\n targets = self.prepare_targets(gt_instances)\n loss_dict = self.criterion(output, targets)\n weight_dict = self.criterion.weight_dict\n for k in loss_dict.keys():\n if k in weight_dict:\n loss_dict[k] *= weight_dict[k]\n return loss_dict\n else:\n box_cls = output[\"pred_logits\"]\n box_pred = output[\"pred_boxes\"]\n mask_pred = output[\"pred_masks\"] if self.mask_on else None\n results = self.inference(box_cls, box_pred, mask_pred, images.image_sizes)\n processed_results = []\n for results_per_image, input_per_image, image_size in zip(results, batched_inputs, images.image_sizes):\n height = input_per_image.get(\"height\", image_size[0])\n width = input_per_image.get(\"width\", image_size[1])\n r = detector_postprocess(results_per_image, height, width)\n processed_results.append({\"instances\": r})\n return processed_results\n\n def prepare_targets(self, targets):\n new_targets = []\n for targets_per_image in targets:\n h, w = targets_per_image.image_size\n image_size_xyxy = torch.as_tensor([w, h, w, h], dtype=torch.float, device=self.device)\n gt_classes = targets_per_image.gt_classes\n gt_boxes = targets_per_image.gt_boxes.tensor / image_size_xyxy\n gt_boxes = box_xyxy_to_cxcywh(gt_boxes)\n new_targets.append({\"labels\": gt_classes, \"boxes\": gt_boxes})\n if self.mask_on and hasattr(targets_per_image, 'gt_masks'):\n gt_masks = targets_per_image.gt_masks\n gt_masks = convert_coco_poly_to_mask(gt_masks.polygons, h, w)\n new_targets[-1].update({'masks': gt_masks})\n return new_targets\n\n def inference(self, box_cls, box_pred, mask_pred, image_sizes):\n \"\"\"\n Arguments:\n box_cls (Tensor): tensor of shape (batch_size, num_queries, K).\n The tensor predicts the classification probability for each query.\n box_pred (Tensor): tensors of shape (batch_size, num_queries, 4).\n The tensor predicts 4-vector (x,y,w,h) box\n regression values for every queryx\n image_sizes (List[torch.Size]): the input image sizes\n\n Returns:\n results (List[Instances]): a list of #images elements.\n \"\"\"\n assert len(box_cls) == len(image_sizes)\n results = []\n\n # For each box we assign the best class or the second best if the best on is `no_object`.\n scores, labels = F.softmax(box_cls, dim=-1)[:, :, :-1].max(-1)\n\n for i, (scores_per_image, labels_per_image, box_pred_per_image, image_size) in enumerate(zip(\n scores, labels, box_pred, image_sizes\n )):\n result = Instances(image_size)\n result.pred_boxes = Boxes(box_cxcywh_to_xyxy(box_pred_per_image))\n\n result.pred_boxes.scale(scale_x=image_size[1], scale_y=image_size[0])\n if self.mask_on:\n mask = F.interpolate(mask_pred[i].unsqueeze(0), size=image_size, mode='bilinear', align_corners=False)\n mask = mask[0].sigmoid() > 0.5\n B, N, H, W = mask_pred.shape\n mask = BitMasks(mask.cpu()).crop_and_resize(result.pred_boxes.tensor.cpu(), 32)\n result.pred_masks = mask.unsqueeze(1).to(mask_pred[0].device)\n\n result.scores = scores_per_image\n result.pred_classes = labels_per_image\n results.append(result)\n return results\n\n def preprocess_image(self, batched_inputs):\n \"\"\"\n Normalize, pad and batch the input images.\n \"\"\"\n images = [self.normalizer(x[\"image\"].to(self.device)) for x in batched_inputs]\n images = ImageList.from_tensors(images)\n return images\n"
},
{
"path": "batchformer-v2/detr/d2/train_net.py",
"content": "# Copyright (c) Facebook, Inc. and its affiliates. All Rights Reserved\n\"\"\"\nDETR Training Script.\n\nThis script is a simplified version of the training script in detectron2/tools.\n\"\"\"\nimport os\nimport sys\nimport itertools\n\n# fmt: off\nsys.path.insert(1, os.path.join(sys.path[0], '..'))\n# fmt: on\n\nimport time\nfrom typing import Any, Dict, List, Set\n\nimport torch\n\nimport detectron2.utils.comm as comm\nfrom d2.detr import DetrDatasetMapper, add_detr_config\nfrom detectron2.checkpoint import DetectionCheckpointer\nfrom detectron2.config import get_cfg\nfrom detectron2.data import MetadataCatalog, build_detection_train_loader\nfrom detectron2.engine import DefaultTrainer, default_argument_parser, default_setup, launch\nfrom detectron2.evaluation import COCOEvaluator, verify_results\n\nfrom detectron2.solver.build import maybe_add_gradient_clipping\n\n\nclass Trainer(DefaultTrainer):\n \"\"\"\n Extension of the Trainer class adapted to DETR.\n \"\"\"\n\n @classmethod\n def build_evaluator(cls, cfg, dataset_name, output_folder=None):\n \"\"\"\n Create evaluator(s) for a given dataset.\n This uses the special metadata \"evaluator_type\" associated with each builtin dataset.\n For your own dataset, you can simply create an evaluator manually in your\n script and do not have to worry about the hacky if-else logic here.\n \"\"\"\n if output_folder is None:\n output_folder = os.path.join(cfg.OUTPUT_DIR, \"inference\")\n return COCOEvaluator(dataset_name, cfg, True, output_folder)\n\n @classmethod\n def build_train_loader(cls, cfg):\n if \"Detr\" == cfg.MODEL.META_ARCHITECTURE:\n mapper = DetrDatasetMapper(cfg, True)\n else:\n mapper = None\n return build_detection_train_loader(cfg, mapper=mapper)\n\n @classmethod\n def build_optimizer(cls, cfg, model):\n params: List[Dict[str, Any]] = []\n memo: Set[torch.nn.parameter.Parameter] = set()\n for key, value in model.named_parameters(recurse=True):\n if not value.requires_grad:\n continue\n # Avoid duplicating parameters\n if value in memo:\n continue\n memo.add(value)\n lr = cfg.SOLVER.BASE_LR\n weight_decay = cfg.SOLVER.WEIGHT_DECAY\n if \"backbone\" in key:\n lr = lr * cfg.SOLVER.BACKBONE_MULTIPLIER\n params += [{\"params\": [value], \"lr\": lr, \"weight_decay\": weight_decay}]\n\n def maybe_add_full_model_gradient_clipping(optim): # optim: the optimizer class\n # detectron2 doesn't have full model gradient clipping now\n clip_norm_val = cfg.SOLVER.CLIP_GRADIENTS.CLIP_VALUE\n enable = (\n cfg.SOLVER.CLIP_GRADIENTS.ENABLED\n and cfg.SOLVER.CLIP_GRADIENTS.CLIP_TYPE == \"full_model\"\n and clip_norm_val > 0.0\n )\n\n class FullModelGradientClippingOptimizer(optim):\n def step(self, closure=None):\n all_params = itertools.chain(*[x[\"params\"] for x in self.param_groups])\n torch.nn.utils.clip_grad_norm_(all_params, clip_norm_val)\n super().step(closure=closure)\n\n return FullModelGradientClippingOptimizer if enable else optim\n\n optimizer_type = cfg.SOLVER.OPTIMIZER\n if optimizer_type == \"SGD\":\n optimizer = maybe_add_full_model_gradient_clipping(torch.optim.SGD)(\n params, cfg.SOLVER.BASE_LR, momentum=cfg.SOLVER.MOMENTUM\n )\n elif optimizer_type == \"ADAMW\":\n optimizer = maybe_add_full_model_gradient_clipping(torch.optim.AdamW)(\n params, cfg.SOLVER.BASE_LR\n )\n else:\n raise NotImplementedError(f\"no optimizer type {optimizer_type}\")\n if not cfg.SOLVER.CLIP_GRADIENTS.CLIP_TYPE == \"full_model\":\n optimizer = maybe_add_gradient_clipping(cfg, optimizer)\n return optimizer\n\n\ndef setup(args):\n \"\"\"\n Create configs and perform basic setups.\n \"\"\"\n cfg = get_cfg()\n add_detr_config(cfg)\n cfg.merge_from_file(args.config_file)\n cfg.merge_from_list(args.opts)\n cfg.freeze()\n default_setup(cfg, args)\n return cfg\n\n\ndef main(args):\n cfg = setup(args)\n\n if args.eval_only:\n model = Trainer.build_model(cfg)\n DetectionCheckpointer(model, save_dir=cfg.OUTPUT_DIR).resume_or_load(cfg.MODEL.WEIGHTS, resume=args.resume)\n res = Trainer.test(cfg, model)\n if comm.is_main_process():\n verify_results(cfg, res)\n return res\n\n trainer = Trainer(cfg)\n trainer.resume_or_load(resume=args.resume)\n return trainer.train()\n\n\nif __name__ == \"__main__\":\n args = default_argument_parser().parse_args()\n print(\"Command Line Args:\", args)\n launch(\n main,\n args.num_gpus,\n num_machines=args.num_machines,\n machine_rank=args.machine_rank,\n dist_url=args.dist_url,\n args=(args,),\n )\n"
},
{
"path": "batchformer-v2/detr/datasets/__init__.py",
"content": "# Copyright (c) Facebook, Inc. and its affiliates. All Rights Reserved\nimport torch.utils.data\nimport torchvision\n\nfrom .coco import build as build_coco\nfrom .coco import build_hico, build_coco2014\n\ndef get_coco_api_from_dataset(dataset):\n for _ in range(10):\n # if isinstance(dataset, torchvision.datasets.CocoDetection):\n # break\n if isinstance(dataset, torch.utils.data.Subset):\n dataset = dataset.dataset\n if isinstance(dataset, torchvision.datasets.CocoDetection):\n return dataset.coco\n\n\ndef build_dataset(image_set, args):\n if args.dataset_file == 'coco':\n return build_coco(image_set, args)\n elif args.dataset_file == 'hico':\n return build_hico(image_set, args)\n elif args.dataset_file == 'coco2014':\n return build_coco2014(image_set, args)\n if args.dataset_file == 'coco_panoptic':\n # to avoid making panopticapi required for coco\n from .coco_panoptic import build as build_coco_panoptic\n return build_coco_panoptic(image_set, args)\n raise ValueError(f'dataset {args.dataset_file} not supported')\n"
},
{
"path": "batchformer-v2/detr/datasets/coco.py",
"content": "# Copyright (c) Facebook, Inc. and its affiliates. All Rights Reserved\n\"\"\"\nCOCO dataset which returns image_id for evaluation.\n\nMostly copy-paste from https://github.com/pytorch/vision/blob/13b35ff/references/detection/coco_utils.py\n\"\"\"\nfrom pathlib import Path\n\nimport torch\nimport torch.utils.data\nimport torchvision\nfrom pycocotools import mask as coco_mask\n\nimport datasets.transforms as T\n\n\nclass CocoDetection(torchvision.datasets.CocoDetection):\n def __init__(self, img_folder, ann_file, transforms, return_masks):\n super(CocoDetection, self).__init__(img_folder, ann_file)\n self._transforms = transforms\n self.prepare = ConvertCocoPolysToMask(return_masks)\n\n def __getitem__(self, idx):\n img, target = super(CocoDetection, self).__getitem__(idx)\n image_id = self.ids[idx]\n target = {'image_id': image_id, 'annotations': target}\n img, target = self.prepare(img, target)\n if self._transforms is not None:\n img, target = self._transforms(img, target)\n return img, target\n\n\ndef convert_coco_poly_to_mask(segmentations, height, width):\n masks = []\n for polygons in segmentations:\n rles = coco_mask.frPyObjects(polygons, height, width)\n mask = coco_mask.decode(rles)\n if len(mask.shape) < 3:\n mask = mask[..., None]\n mask = torch.as_tensor(mask, dtype=torch.uint8)\n mask = mask.any(dim=2)\n masks.append(mask)\n if masks:\n masks = torch.stack(masks, dim=0)\n else:\n masks = torch.zeros((0, height, width), dtype=torch.uint8)\n return masks\n\n\nclass ConvertCocoPolysToMask(object):\n def __init__(self, return_masks=False):\n self.return_masks = return_masks\n\n def __call__(self, image, target):\n w, h = image.size\n\n image_id = target[\"image_id\"]\n image_id = torch.tensor([image_id])\n\n anno = target[\"annotations\"]\n\n anno = [obj for obj in anno if 'iscrowd' not in obj or obj['iscrowd'] == 0]\n\n boxes = [obj[\"bbox\"] for obj in anno]\n # guard against no boxes via resizing\n boxes = torch.as_tensor(boxes, dtype=torch.float32).reshape(-1, 4)\n boxes[:, 2:] += boxes[:, :2]\n boxes[:, 0::2].clamp_(min=0, max=w)\n boxes[:, 1::2].clamp_(min=0, max=h)\n\n classes = [obj[\"category_id\"] for obj in anno]\n classes = torch.tensor(classes, dtype=torch.int64)\n\n if self.return_masks:\n segmentations = [obj[\"segmentation\"] for obj in anno]\n masks = convert_coco_poly_to_mask(segmentations, h, w)\n\n keypoints = None\n if anno and \"keypoints\" in anno[0]:\n keypoints = [obj[\"keypoints\"] for obj in anno]\n keypoints = torch.as_tensor(keypoints, dtype=torch.float32)\n num_keypoints = keypoints.shape[0]\n if num_keypoints:\n keypoints = keypoints.view(num_keypoints, -1, 3)\n\n keep = (boxes[:, 3] > boxes[:, 1]) & (boxes[:, 2] > boxes[:, 0])\n boxes = boxes[keep]\n classes = classes[keep]\n if self.return_masks:\n masks = masks[keep]\n if keypoints is not None:\n keypoints = keypoints[keep]\n\n target = {}\n target[\"boxes\"] = boxes\n target[\"labels\"] = classes\n if self.return_masks:\n target[\"masks\"] = masks\n target[\"image_id\"] = image_id\n if keypoints is not None:\n target[\"keypoints\"] = keypoints\n\n # for conversion to coco api\n area = torch.tensor([obj[\"area\"] for obj in anno])\n iscrowd = torch.tensor([obj[\"iscrowd\"] if \"iscrowd\" in obj else 0 for obj in anno])\n target[\"area\"] = area[keep]\n target[\"iscrowd\"] = iscrowd[keep]\n\n target[\"orig_size\"] = torch.as_tensor([int(h), int(w)])\n target[\"size\"] = torch.as_tensor([int(h), int(w)])\n\n return image, target\n\n\ndef make_coco_transforms(image_set):\n\n normalize = T.Compose([\n T.ToTensor(),\n T.Normalize([0.485, 0.456, 0.406], [0.229, 0.224, 0.225])\n ])\n\n scales = [480, 512, 544, 576, 608, 640, 672, 704, 736, 768, 800]\n\n if image_set == 'train':\n return T.Compose([\n T.RandomHorizontalFlip(),\n T.RandomSelect(\n T.RandomResize(scales, max_size=1333),\n T.Compose([\n T.RandomResize([400, 500, 600]),\n T.RandomSizeCrop(384, 600),\n T.RandomResize(scales, max_size=1333),\n ])\n ),\n normalize,\n ])\n\n if image_set == 'val':\n return T.Compose([\n T.RandomResize([800], max_size=1333),\n normalize,\n ])\n\n raise ValueError(f'unknown {image_set}')\n\n\ndef build(image_set, args):\n root = Path(args.coco_path)\n assert root.exists(), f'provided COCO path {root} does not exist'\n mode = 'instances'\n PATHS = {\n \"train\": (root / \"train2017\", root / \"annotations\" / f'{mode}_train2017.json'),\n \"val\": (root / \"val2017\", root / \"annotations\" / f'{mode}_val2017.json'),\n }\n\n img_folder, ann_file = PATHS[image_set]\n dataset = CocoDetection(img_folder, ann_file, transforms=make_coco_transforms(image_set), return_masks=args.masks)\n return dataset\n\ndef build_coco2014(image_set, args):\n root = Path(args.coco_path)\n assert root.exists(), f'provided COCO path {root} does not exist'\n mode = 'instances'\n PATHS = {\n \"train\": (root / \"train2014\", root / \"annotations\" / f'{mode}_train2014.json'),\n \"val\": (root / \"val2014\", root / \"annotations\" / f'{mode}_val2014.json'),\n }\n\n img_folder, ann_file = PATHS[image_set]\n dataset = CocoDetection(img_folder, ann_file, transforms=make_coco_transforms(image_set), return_masks=args.masks)\n return dataset\n\ndef build_hico(image_set, args):\n root = Path(args.coco_path)\n assert root.exists(), f'provided COCO path {root} does not exist'\n mode = 'instances'\n PATHS = {\n \"train\": (root / \"train2015\", root / \"annotations\" / 'hico_train.json'),\n \"val\": (root / \"test2015\", root / \"annotations\" / 'hico_test.json'),\n }\n\n img_folder, ann_file = PATHS[image_set]\n dataset = CocoDetection(img_folder, ann_file, transforms=make_coco_transforms(image_set), return_masks=args.masks)\n return dataset"
},
{
"path": "batchformer-v2/detr/datasets/coco_eval.py",
"content": "# Copyright (c) Facebook, Inc. and its affiliates. All Rights Reserved\n\"\"\"\nCOCO evaluator that works in distributed mode.\n\nMostly copy-paste from https://github.com/pytorch/vision/blob/edfd5a7/references/detection/coco_eval.py\nThe difference is that there is less copy-pasting from pycocotools\nin the end of the file, as python3 can suppress prints with contextlib\n\"\"\"\nimport os\nimport contextlib\nimport copy\nimport numpy as np\nimport torch\n\nfrom pycocotools.cocoeval import COCOeval\nfrom pycocotools.coco import COCO\nimport pycocotools.mask as mask_util\n\nfrom util.misc import all_gather\n\n\nclass CocoEvaluator(object):\n def __init__(self, coco_gt, iou_types):\n assert isinstance(iou_types, (list, tuple))\n coco_gt = copy.deepcopy(coco_gt)\n self.coco_gt = coco_gt\n\n self.iou_types = iou_types\n self.coco_eval = {}\n for iou_type in iou_types:\n self.coco_eval[iou_type] = COCOeval(coco_gt, iouType=iou_type)\n\n self.img_ids = []\n self.eval_imgs = {k: [] for k in iou_types}\n\n def update(self, predictions):\n img_ids = list(np.unique(list(predictions.keys())))\n self.img_ids.extend(img_ids)\n\n for iou_type in self.iou_types:\n results = self.prepare(predictions, iou_type)\n\n # suppress pycocotools prints\n with open(os.devnull, 'w') as devnull:\n with contextlib.redirect_stdout(devnull):\n coco_dt = COCO.loadRes(self.coco_gt, results) if results else COCO()\n coco_eval = self.coco_eval[iou_type]\n\n coco_eval.cocoDt = coco_dt\n coco_eval.params.imgIds = list(img_ids)\n img_ids, eval_imgs = evaluate(coco_eval)\n\n self.eval_imgs[iou_type].append(eval_imgs)\n\n def synchronize_between_processes(self):\n for iou_type in self.iou_types:\n self.eval_imgs[iou_type] = np.concatenate(self.eval_imgs[iou_type], 2)\n create_common_coco_eval(self.coco_eval[iou_type], self.img_ids, self.eval_imgs[iou_type])\n\n def accumulate(self):\n for coco_eval in self.coco_eval.values():\n coco_eval.accumulate()\n\n def summarize(self):\n for iou_type, coco_eval in self.coco_eval.items():\n print(\"IoU metric: {}\".format(iou_type))\n coco_eval.summarize()\n\n def prepare(self, predictions, iou_type):\n if iou_type == \"bbox\":\n return self.prepare_for_coco_detection(predictions)\n elif iou_type == \"segm\":\n return self.prepare_for_coco_segmentation(predictions)\n elif iou_type == \"keypoints\":\n return self.prepare_for_coco_keypoint(predictions)\n else:\n raise ValueError(\"Unknown iou type {}\".format(iou_type))\n\n def prepare_for_coco_detection(self, predictions):\n coco_results = []\n for original_id, prediction in predictions.items():\n if len(prediction) == 0:\n continue\n\n boxes = prediction[\"boxes\"]\n boxes = convert_to_xywh(boxes).tolist()\n scores = prediction[\"scores\"].tolist()\n labels = prediction[\"labels\"].tolist()\n\n coco_results.extend(\n [\n {\n \"image_id\": original_id,\n \"category_id\": labels[k],\n \"bbox\": box,\n \"score\": scores[k],\n }\n for k, box in enumerate(boxes)\n ]\n )\n return coco_results\n\n def prepare_for_coco_segmentation(self, predictions):\n coco_results = []\n for original_id, prediction in predictions.items():\n if len(prediction) == 0:\n continue\n\n scores = prediction[\"scores\"]\n labels = prediction[\"labels\"]\n masks = prediction[\"masks\"]\n\n masks = masks > 0.5\n\n scores = prediction[\"scores\"].tolist()\n labels = prediction[\"labels\"].tolist()\n\n rles = [\n mask_util.encode(np.array(mask[0, :, :, np.newaxis], dtype=np.uint8, order=\"F\"))[0]\n for mask in masks\n ]\n for rle in rles:\n rle[\"counts\"] = rle[\"counts\"].decode(\"utf-8\")\n\n coco_results.extend(\n [\n {\n \"image_id\": original_id,\n \"category_id\": labels[k],\n \"segmentation\": rle,\n \"score\": scores[k],\n }\n for k, rle in enumerate(rles)\n ]\n )\n return coco_results\n\n def prepare_for_coco_keypoint(self, predictions):\n coco_results = []\n for original_id, prediction in predictions.items():\n if len(prediction) == 0:\n continue\n\n boxes = prediction[\"boxes\"]\n boxes = convert_to_xywh(boxes).tolist()\n scores = prediction[\"scores\"].tolist()\n labels = prediction[\"labels\"].tolist()\n keypoints = prediction[\"keypoints\"]\n keypoints = keypoints.flatten(start_dim=1).tolist()\n\n coco_results.extend(\n [\n {\n \"image_id\": original_id,\n \"category_id\": labels[k],\n 'keypoints': keypoint,\n \"score\": scores[k],\n }\n for k, keypoint in enumerate(keypoints)\n ]\n )\n return coco_results\n\n\ndef convert_to_xywh(boxes):\n xmin, ymin, xmax, ymax = boxes.unbind(1)\n return torch.stack((xmin, ymin, xmax - xmin, ymax - ymin), dim=1)\n\n\ndef merge(img_ids, eval_imgs):\n all_img_ids = all_gather(img_ids)\n all_eval_imgs = all_gather(eval_imgs)\n\n merged_img_ids = []\n for p in all_img_ids:\n merged_img_ids.extend(p)\n\n merged_eval_imgs = []\n for p in all_eval_imgs:\n merged_eval_imgs.append(p)\n\n merged_img_ids = np.array(merged_img_ids)\n merged_eval_imgs = np.concatenate(merged_eval_imgs, 2)\n\n # keep only unique (and in sorted order) images\n merged_img_ids, idx = np.unique(merged_img_ids, return_index=True)\n merged_eval_imgs = merged_eval_imgs[..., idx]\n\n return merged_img_ids, merged_eval_imgs\n\n\ndef create_common_coco_eval(coco_eval, img_ids, eval_imgs):\n img_ids, eval_imgs = merge(img_ids, eval_imgs)\n img_ids = list(img_ids)\n eval_imgs = list(eval_imgs.flatten())\n\n coco_eval.evalImgs = eval_imgs\n coco_eval.params.imgIds = img_ids\n coco_eval._paramsEval = copy.deepcopy(coco_eval.params)\n\n\n#################################################################\n# From pycocotools, just removed the prints and fixed\n# a Python3 bug about unicode not defined\n#################################################################\n\n\ndef evaluate(self):\n '''\n Run per image evaluation on given images and store results (a list of dict) in self.evalImgs\n :return: None\n '''\n # tic = time.time()\n # print('Running per image evaluation...')\n p = self.params\n # add backward compatibility if useSegm is specified in params\n if p.useSegm is not None:\n p.iouType = 'segm' if p.useSegm == 1 else 'bbox'\n print('useSegm (deprecated) is not None. Running {} evaluation'.format(p.iouType))\n # print('Evaluate annotation type *{}*'.format(p.iouType))\n p.imgIds = list(np.unique(p.imgIds))\n if p.useCats:\n p.catIds = list(np.unique(p.catIds))\n p.maxDets = sorted(p.maxDets)\n self.params = p\n\n self._prepare()\n # loop through images, area range, max detection number\n catIds = p.catIds if p.useCats else [-1]\n\n if p.iouType == 'segm' or p.iouType == 'bbox':\n computeIoU = self.computeIoU\n elif p.iouType == 'keypoints':\n computeIoU = self.computeOks\n self.ious = {\n (imgId, catId): computeIoU(imgId, catId)\n for imgId in p.imgIds\n for catId in catIds}\n\n evaluateImg = self.evaluateImg\n maxDet = p.maxDets[-1]\n evalImgs = [\n evaluateImg(imgId, catId, areaRng, maxDet)\n for catId in catIds\n for areaRng in p.areaRng\n for imgId in p.imgIds\n ]\n # this is NOT in the pycocotools code, but could be done outside\n evalImgs = np.asarray(evalImgs).reshape(len(catIds), len(p.areaRng), len(p.imgIds))\n self._paramsEval = copy.deepcopy(self.params)\n # toc = time.time()\n # print('DONE (t={:0.2f}s).'.format(toc-tic))\n return p.imgIds, evalImgs\n\n#################################################################\n# end of straight copy from pycocotools, just removing the prints\n#################################################################\n"
},
{
"path": "batchformer-v2/detr/datasets/coco_panoptic.py",
"content": "# Copyright (c) Facebook, Inc. and its affiliates. All Rights Reserved\nimport json\nfrom pathlib import Path\n\nimport numpy as np\nimport torch\nfrom PIL import Image\n\nfrom panopticapi.utils import rgb2id\nfrom util.box_ops import masks_to_boxes\n\nfrom .coco import make_coco_transforms\n\n\nclass CocoPanoptic:\n def __init__(self, img_folder, ann_folder, ann_file, transforms=None, return_masks=True):\n with open(ann_file, 'r') as f:\n self.coco = json.load(f)\n\n # sort 'images' field so that they are aligned with 'annotations'\n # i.e., in alphabetical order\n self.coco['images'] = sorted(self.coco['images'], key=lambda x: x['id'])\n # sanity check\n if \"annotations\" in self.coco:\n for img, ann in zip(self.coco['images'], self.coco['annotations']):\n assert img['file_name'][:-4] == ann['file_name'][:-4]\n\n self.img_folder = img_folder\n self.ann_folder = ann_folder\n self.ann_file = ann_file\n self.transforms = transforms\n self.return_masks = return_masks\n\n def __getitem__(self, idx):\n ann_info = self.coco['annotations'][idx] if \"annotations\" in self.coco else self.coco['images'][idx]\n img_path = Path(self.img_folder) / ann_info['file_name'].replace('.png', '.jpg')\n ann_path = Path(self.ann_folder) / ann_info['file_name']\n\n img = Image.open(img_path).convert('RGB')\n w, h = img.size\n if \"segments_info\" in ann_info:\n masks = np.asarray(Image.open(ann_path), dtype=np.uint32)\n masks = rgb2id(masks)\n\n ids = np.array([ann['id'] for ann in ann_info['segments_info']])\n masks = masks == ids[:, None, None]\n\n masks = torch.as_tensor(masks, dtype=torch.uint8)\n labels = torch.tensor([ann['category_id'] for ann in ann_info['segments_info']], dtype=torch.int64)\n\n target = {}\n target['image_id'] = torch.tensor([ann_info['image_id'] if \"image_id\" in ann_info else ann_info[\"id\"]])\n if self.return_masks:\n target['masks'] = masks\n target['labels'] = labels\n\n target[\"boxes\"] = masks_to_boxes(masks)\n\n target['size'] = torch.as_tensor([int(h), int(w)])\n target['orig_size'] = torch.as_tensor([int(h), int(w)])\n if \"segments_info\" in ann_info:\n for name in ['iscrowd', 'area']:\n target[name] = torch.tensor([ann[name] for ann in ann_info['segments_info']])\n\n if self.transforms is not None:\n img, target = self.transforms(img, target)\n\n return img, target\n\n def __len__(self):\n return len(self.coco['images'])\n\n def get_height_and_width(self, idx):\n img_info = self.coco['images'][idx]\n height = img_info['height']\n width = img_info['width']\n return height, width\n\n\ndef build(image_set, args):\n img_folder_root = Path(args.coco_path)\n ann_folder_root = Path(args.coco_panoptic_path)\n assert img_folder_root.exists(), f'provided COCO path {img_folder_root} does not exist'\n assert ann_folder_root.exists(), f'provided COCO path {ann_folder_root} does not exist'\n mode = 'panoptic'\n PATHS = {\n \"train\": (\"train2017\", Path(\"annotations\") / f'{mode}_train2017.json'),\n \"val\": (\"val2017\", Path(\"annotations\") / f'{mode}_val2017.json'),\n }\n\n img_folder, ann_file = PATHS[image_set]\n img_folder_path = img_folder_root / img_folder\n ann_folder = ann_folder_root / f'{mode}_{img_folder}'\n ann_file = ann_folder_root / ann_file\n\n dataset = CocoPanoptic(img_folder_path, ann_folder, ann_file,\n transforms=make_coco_transforms(image_set), return_masks=args.masks)\n\n return dataset\n"
},
{
"path": "batchformer-v2/detr/datasets/panoptic_eval.py",
"content": "# Copyright (c) Facebook, Inc. and its affiliates. All Rights Reserved\nimport json\nimport os\n\nimport util.misc as utils\n\ntry:\n from panopticapi.evaluation import pq_compute\nexcept ImportError:\n pass\n\n\nclass PanopticEvaluator(object):\n def __init__(self, ann_file, ann_folder, output_dir=\"panoptic_eval\"):\n self.gt_json = ann_file\n self.gt_folder = ann_folder\n if utils.is_main_process():\n if not os.path.exists(output_dir):\n os.mkdir(output_dir)\n self.output_dir = output_dir\n self.predictions = []\n\n def update(self, predictions):\n for p in predictions:\n with open(os.path.join(self.output_dir, p[\"file_name\"]), \"wb\") as f:\n f.write(p.pop(\"png_string\"))\n\n self.predictions += predictions\n\n def synchronize_between_processes(self):\n all_predictions = utils.all_gather(self.predictions)\n merged_predictions = []\n for p in all_predictions:\n merged_predictions += p\n self.predictions = merged_predictions\n\n def summarize(self):\n if utils.is_main_process():\n json_data = {\"annotations\": self.predictions}\n predictions_json = os.path.join(self.output_dir, \"predictions.json\")\n with open(predictions_json, \"w\") as f:\n f.write(json.dumps(json_data))\n return pq_compute(self.gt_json, predictions_json, gt_folder=self.gt_folder, pred_folder=self.output_dir)\n return None\n"
},
{
"path": "batchformer-v2/detr/datasets/transforms.py",
"content": "# Copyright (c) Facebook, Inc. and its affiliates. All Rights Reserved\n\"\"\"\nTransforms and data augmentation for both image + bbox.\n\"\"\"\nimport random\n\nimport PIL\nimport torch\nimport torchvision.transforms as T\nimport torchvision.transforms.functional as F\n\nfrom util.box_ops import box_xyxy_to_cxcywh\nfrom util.misc import interpolate\n\n\ndef crop(image, target, region):\n cropped_image = F.crop(image, *region)\n\n target = target.copy()\n i, j, h, w = region\n\n # should we do something wrt the original size?\n target[\"size\"] = torch.tensor([h, w])\n\n fields = [\"labels\", \"area\", \"iscrowd\"]\n\n if \"boxes\" in target:\n boxes = target[\"boxes\"]\n max_size = torch.as_tensor([w, h], dtype=torch.float32)\n cropped_boxes = boxes - torch.as_tensor([j, i, j, i])\n cropped_boxes = torch.min(cropped_boxes.reshape(-1, 2, 2), max_size)\n cropped_boxes = cropped_boxes.clamp(min=0)\n area = (cropped_boxes[:, 1, :] - cropped_boxes[:, 0, :]).prod(dim=1)\n target[\"boxes\"] = cropped_boxes.reshape(-1, 4)\n target[\"area\"] = area\n fields.append(\"boxes\")\n\n if \"masks\" in target:\n # FIXME should we update the area here if there are no boxes?\n target['masks'] = target['masks'][:, i:i + h, j:j + w]\n fields.append(\"masks\")\n\n # remove elements for which the boxes or masks that have zero area\n if \"boxes\" in target or \"masks\" in target:\n # favor boxes selection when defining which elements to keep\n # this is compatible with previous implementation\n if \"boxes\" in target:\n cropped_boxes = target['boxes'].reshape(-1, 2, 2)\n keep = torch.all(cropped_boxes[:, 1, :] > cropped_boxes[:, 0, :], dim=1)\n else:\n keep = target['masks'].flatten(1).any(1)\n\n for field in fields:\n target[field] = target[field][keep]\n\n return cropped_image, target\n\n\ndef hflip(image, target):\n flipped_image = F.hflip(image)\n\n w, h = image.size\n\n target = target.copy()\n if \"boxes\" in target:\n boxes = target[\"boxes\"]\n boxes = boxes[:, [2, 1, 0, 3]] * torch.as_tensor([-1, 1, -1, 1]) + torch.as_tensor([w, 0, w, 0])\n target[\"boxes\"] = boxes\n\n if \"masks\" in target:\n target['masks'] = target['masks'].flip(-1)\n\n return flipped_image, target\n\n\ndef resize(image, target, size, max_size=None):\n # size can be min_size (scalar) or (w, h) tuple\n\n def get_size_with_aspect_ratio(image_size, size, max_size=None):\n w, h = image_size\n if max_size is not None:\n min_original_size = float(min((w, h)))\n max_original_size = float(max((w, h)))\n if max_original_size / min_original_size * size > max_size:\n size = int(round(max_size * min_original_size / max_original_size))\n\n if (w <= h and w == size) or (h <= w and h == size):\n return (h, w)\n\n if w < h:\n ow = size\n oh = int(size * h / w)\n else:\n oh = size\n ow = int(size * w / h)\n\n return (oh, ow)\n\n def get_size(image_size, size, max_size=None):\n if isinstance(size, (list, tuple)):\n return size[::-1]\n else:\n return get_size_with_aspect_ratio(image_size, size, max_size)\n\n size = get_size(image.size, size, max_size)\n rescaled_image = F.resize(image, size)\n\n if target is None:\n return rescaled_image, None\n\n ratios = tuple(float(s) / float(s_orig) for s, s_orig in zip(rescaled_image.size, image.size))\n ratio_width, ratio_height = ratios\n\n target = target.copy()\n if \"boxes\" in target:\n boxes = target[\"boxes\"]\n scaled_boxes = boxes * torch.as_tensor([ratio_width, ratio_height, ratio_width, ratio_height])\n target[\"boxes\"] = scaled_boxes\n\n if \"area\" in target:\n area = target[\"area\"]\n scaled_area = area * (ratio_width * ratio_height)\n target[\"area\"] = scaled_area\n\n h, w = size\n target[\"size\"] = torch.tensor([h, w])\n\n if \"masks\" in target:\n target['masks'] = interpolate(\n target['masks'][:, None].float(), size, mode=\"nearest\")[:, 0] > 0.5\n\n return rescaled_image, target\n\n\ndef pad(image, target, padding):\n # assumes that we only pad on the bottom right corners\n padded_image = F.pad(image, (0, 0, padding[0], padding[1]))\n if target is None:\n return padded_image, None\n target = target.copy()\n # should we do something wrt the original size?\n target[\"size\"] = torch.tensor(padded_image.size[::-1])\n if \"masks\" in target:\n target['masks'] = torch.nn.functional.pad(target['masks'], (0, padding[0], 0, padding[1]))\n return padded_image, target\n\n\nclass RandomCrop(object):\n def __init__(self, size):\n self.size = size\n\n def __call__(self, img, target):\n region = T.RandomCrop.get_params(img, self.size)\n return crop(img, target, region)\n\n\nclass RandomSizeCrop(object):\n def __init__(self, min_size: int, max_size: int):\n self.min_size = min_size\n self.max_size = max_size\n\n def __call__(self, img: PIL.Image.Image, target: dict):\n w = random.randint(self.min_size, min(img.width, self.max_size))\n h = random.randint(self.min_size, min(img.height, self.max_size))\n region = T.RandomCrop.get_params(img, [h, w])\n return crop(img, target, region)\n\n\nclass CenterCrop(object):\n def __init__(self, size):\n self.size = size\n\n def __call__(self, img, target):\n image_width, image_height = img.size\n crop_height, crop_width = self.size\n crop_top = int(round((image_height - crop_height) / 2.))\n crop_left = int(round((image_width - crop_width) / 2.))\n return crop(img, target, (crop_top, crop_left, crop_height, crop_width))\n\n\nclass RandomHorizontalFlip(object):\n def __init__(self, p=0.5):\n self.p = p\n\n def __call__(self, img, target):\n if random.random() < self.p:\n return hflip(img, target)\n return img, target\n\n\nclass RandomResize(object):\n def __init__(self, sizes, max_size=None):\n assert isinstance(sizes, (list, tuple))\n self.sizes = sizes\n self.max_size = max_size\n\n def __call__(self, img, target=None):\n size = random.choice(self.sizes)\n return resize(img, target, size, self.max_size)\n\n\nclass RandomPad(object):\n def __init__(self, max_pad):\n self.max_pad = max_pad\n\n def __call__(self, img, target):\n pad_x = random.randint(0, self.max_pad)\n pad_y = random.randint(0, self.max_pad)\n return pad(img, target, (pad_x, pad_y))\n\n\nclass RandomSelect(object):\n \"\"\"\n Randomly selects between transforms1 and transforms2,\n with probability p for transforms1 and (1 - p) for transforms2\n \"\"\"\n def __init__(self, transforms1, transforms2, p=0.5):\n self.transforms1 = transforms1\n self.transforms2 = transforms2\n self.p = p\n\n def __call__(self, img, target):\n if random.random() < self.p:\n return self.transforms1(img, target)\n return self.transforms2(img, target)\n\n\nclass ToTensor(object):\n def __call__(self, img, target):\n return F.to_tensor(img), target\n\n\nclass RandomErasing(object):\n\n def __init__(self, *args, **kwargs):\n self.eraser = T.RandomErasing(*args, **kwargs)\n\n def __call__(self, img, target):\n return self.eraser(img), target\n\n\nclass Normalize(object):\n def __init__(self, mean, std):\n self.mean = mean\n self.std = std\n\n def __call__(self, image, target=None):\n image = F.normalize(image, mean=self.mean, std=self.std)\n if target is None:\n return image, None\n target = target.copy()\n h, w = image.shape[-2:]\n if \"boxes\" in target:\n boxes = target[\"boxes\"]\n boxes = box_xyxy_to_cxcywh(boxes)\n boxes = boxes / torch.tensor([w, h, w, h], dtype=torch.float32)\n target[\"boxes\"] = boxes\n return image, target\n\n\nclass Compose(object):\n def __init__(self, transforms):\n self.transforms = transforms\n\n def __call__(self, image, target):\n for t in self.transforms:\n image, target = t(image, target)\n return image, target\n\n def __repr__(self):\n format_string = self.__class__.__name__ + \"(\"\n for t in self.transforms:\n format_string += \"\\n\"\n format_string += \" {0}\".format(t)\n format_string += \"\\n)\"\n return format_string\n"
},
{
"path": "batchformer-v2/detr/detr.pbs",
"content": "#!/bin/bash\n#PBS -P ZHIHOU\n#PBS -l select=8:ngpus=1:ncpus=6:mem=24GB\n#PBS -l walltime=48:00:00\n##PBS -q alloc-dt\nsource activate pyt\n\n#env\n\ncd \"$PBS_O_WORKDIR\"\n\n\n#python run.py --nproc_per_node 1 --master_port 29543 --nnodes 8 ./configs/base.sh &>> basebf1.out\n\npython run.py --nproc_per_node 1 --master_port 29547 --nnodes 8 ./configs/panoptic_2.sh &>> panoptic_bf3_segm.out\n\n\n#python run.py --nproc_per_node 1 --master_port 29545 --nnodes 8 ./configs/panoptic.sh &>> panoptic_bf3.out\n\n#python run.py --nproc_per_node 1 --master_port 29546 --nnodes 7 ./configs/swin.sh &>> output_bf3_7.out\n\n"
},
{
"path": "batchformer-v2/detr/engine.py",
"content": "# Copyright (c) Facebook, Inc. and its affiliates. All Rights Reserved\n\"\"\"\nTrain and eval functions used in main.py\n\"\"\"\nimport math\nimport os\nimport sys\nfrom typing import Iterable\n\nimport torch\n\nimport util.misc as utils\nfrom datasets.coco_eval import CocoEvaluator\nfrom datasets.panoptic_eval import PanopticEvaluator\n\n\ndef train_one_epoch(model: torch.nn.Module, criterion: torch.nn.Module,\n data_loader: Iterable, optimizer: torch.optim.Optimizer,\n device: torch.device, epoch: int, max_norm: float = 0):\n model.train()\n criterion.train()\n metric_logger = utils.MetricLogger(delimiter=\" \")\n metric_logger.add_meter('lr', utils.SmoothedValue(window_size=1, fmt='{value:.6f}'))\n metric_logger.add_meter('class_error', utils.SmoothedValue(window_size=1, fmt='{value:.2f}'))\n header = 'Epoch: [{}]'.format(epoch)\n print_freq = 10\n\n for samples, targets in metric_logger.log_every(data_loader, print_freq, header):\n samples = samples.to(device)\n targets = [{k: v.to(device) for k, v in t.items()} for t in targets]\n\n outputs = model(samples)\n if len(targets) < len(outputs['pred_logits']):\n targets = targets + targets\n loss_dict = criterion(outputs, targets)\n weight_dict = criterion.weight_dict\n losses = sum(loss_dict[k] * weight_dict[k] for k in loss_dict.keys() if k in weight_dict)\n\n # reduce losses over all GPUs for logging purposes\n loss_dict_reduced = utils.reduce_dict(loss_dict)\n loss_dict_reduced_unscaled = {f'{k}_unscaled': v\n for k, v in loss_dict_reduced.items()}\n loss_dict_reduced_scaled = {k: v * weight_dict[k]\n for k, v in loss_dict_reduced.items() if k in weight_dict}\n losses_reduced_scaled = sum(loss_dict_reduced_scaled.values())\n\n loss_value = losses_reduced_scaled.item()\n\n if not math.isfinite(loss_value):\n print(\"Loss is {}, stopping training\".format(loss_value))\n print(loss_dict_reduced)\n sys.exit(1)\n\n optimizer.zero_grad()\n losses.backward()\n if max_norm > 0:\n torch.nn.utils.clip_grad_norm_(model.parameters(), max_norm)\n optimizer.step()\n\n metric_logger.update(loss=loss_value, **loss_dict_reduced_scaled, **loss_dict_reduced_unscaled)\n metric_logger.update(class_error=loss_dict_reduced['class_error'])\n metric_logger.update(lr=optimizer.param_groups[0][\"lr\"])\n # gather the stats from all processes\n metric_logger.synchronize_between_processes()\n print(\"Averaged stats:\", metric_logger)\n return {k: meter.global_avg for k, meter in metric_logger.meters.items()}\n\n\n@torch.no_grad()\ndef evaluate(model, criterion, postprocessors, data_loader, base_ds, device, output_dir):\n model.eval()\n criterion.eval()\n\n metric_logger = utils.MetricLogger(delimiter=\" \")\n metric_logger.add_meter('class_error', utils.SmoothedValue(window_size=1, fmt='{value:.2f}'))\n header = 'Test:'\n\n iou_types = tuple(k for k in ('segm', 'bbox') if k in postprocessors.keys())\n coco_evaluator = CocoEvaluator(base_ds, iou_types)\n # coco_evaluator.coco_eval[iou_types[0]].params.iouThrs = [0, 0.1, 0.5, 0.75]\n\n panoptic_evaluator = None\n if 'panoptic' in postprocessors.keys():\n panoptic_evaluator = PanopticEvaluator(\n data_loader.dataset.ann_file,\n data_loader.dataset.ann_folder,\n output_dir=os.path.join(output_dir, \"panoptic_eval\"),\n )\n\n for samples, targets in metric_logger.log_every(data_loader, 10, header):\n samples = samples.to(device)\n targets = [{k: v.to(device) for k, v in t.items()} for t in targets]\n\n outputs = model(samples)\n loss_dict = criterion(outputs, targets)\n weight_dict = criterion.weight_dict\n\n # reduce losses over all GPUs for logging purposes\n loss_dict_reduced = utils.reduce_dict(loss_dict)\n loss_dict_reduced_scaled = {k: v * weight_dict[k]\n for k, v in loss_dict_reduced.items() if k in weight_dict}\n loss_dict_reduced_unscaled = {f'{k}_unscaled': v\n for k, v in loss_dict_reduced.items()}\n metric_logger.update(loss=sum(loss_dict_reduced_scaled.values()),\n **loss_dict_reduced_scaled,\n **loss_dict_reduced_unscaled)\n metric_logger.update(class_error=loss_dict_reduced['class_error'])\n\n orig_target_sizes = torch.stack([t[\"orig_size\"] for t in targets], dim=0)\n results = postprocessors['bbox'](outputs, orig_target_sizes)\n if 'segm' in postprocessors.keys():\n target_sizes = torch.stack([t[\"size\"] for t in targets], dim=0)\n results = postprocessors['segm'](results, outputs, orig_target_sizes, target_sizes)\n res = {target['image_id'].item(): output for target, output in zip(targets, results)}\n if coco_evaluator is not None:\n coco_evaluator.update(res)\n\n if panoptic_evaluator is not None:\n res_pano = postprocessors[\"panoptic\"](outputs, target_sizes, orig_target_sizes)\n for i, target in enumerate(targets):\n image_id = target[\"image_id\"].item()\n file_name = f\"{image_id:012d}.png\"\n res_pano[i][\"image_id\"] = image_id\n res_pano[i][\"file_name\"] = file_name\n\n panoptic_evaluator.update(res_pano)\n\n # gather the stats from all processes\n metric_logger.synchronize_between_processes()\n print(\"Averaged stats:\", metric_logger)\n if coco_evaluator is not None:\n coco_evaluator.synchronize_between_processes()\n if panoptic_evaluator is not None:\n panoptic_evaluator.synchronize_between_processes()\n\n # accumulate predictions from all images\n if coco_evaluator is not None:\n coco_evaluator.accumulate()\n coco_evaluator.summarize()\n panoptic_res = None\n if panoptic_evaluator is not None:\n panoptic_res = panoptic_evaluator.summarize()\n stats = {k: meter.global_avg for k, meter in metric_logger.meters.items()}\n if coco_evaluator is not None:\n if 'bbox' in postprocessors.keys():\n stats['coco_eval_bbox'] = coco_evaluator.coco_eval['bbox'].stats.tolist()\n if 'segm' in postprocessors.keys():\n stats['coco_eval_masks'] = coco_evaluator.coco_eval['segm'].stats.tolist()\n if panoptic_res is not None:\n stats['PQ_all'] = panoptic_res[\"All\"]\n stats['PQ_th'] = panoptic_res[\"Things\"]\n stats['PQ_st'] = panoptic_res[\"Stuff\"]\n return stats, coco_evaluator\n"
},
{
"path": "batchformer-v2/detr/hubconf.py",
"content": "# Copyright (c) Facebook, Inc. and its affiliates. All Rights Reserved\nimport torch\n\nfrom models.backbone import Backbone, Joiner\nfrom models.detr import DETR, PostProcess\nfrom models.position_encoding import PositionEmbeddingSine\nfrom models.segmentation import DETRsegm, PostProcessPanoptic\nfrom models.transformer import Transformer\n\ndependencies = [\"torch\", \"torchvision\"]\n\n\ndef _make_detr(backbone_name: str, dilation=False, num_classes=91, mask=False):\n hidden_dim = 256\n backbone = Backbone(backbone_name, train_backbone=True, return_interm_layers=mask, dilation=dilation)\n pos_enc = PositionEmbeddingSine(hidden_dim // 2, normalize=True)\n backbone_with_pos_enc = Joiner(backbone, pos_enc)\n backbone_with_pos_enc.num_channels = backbone.num_channels\n transformer = Transformer(d_model=hidden_dim, return_intermediate_dec=True)\n detr = DETR(backbone_with_pos_enc, transformer, num_classes=num_classes, num_queries=100)\n if mask:\n return DETRsegm(detr)\n return detr\n\n\ndef detr_resnet50(pretrained=False, num_classes=91, return_postprocessor=False):\n \"\"\"\n DETR R50 with 6 encoder and 6 decoder layers.\n\n Achieves 42/62.4 AP/AP50 on COCO val5k.\n \"\"\"\n model = _make_detr(\"resnet50\", dilation=False, num_classes=num_classes)\n if pretrained:\n checkpoint = torch.hub.load_state_dict_from_url(\n url=\"https://dl.fbaipublicfiles.com/detr/detr-r50-e632da11.pth\", map_location=\"cpu\", check_hash=True\n )\n model.load_state_dict(checkpoint[\"model\"])\n if return_postprocessor:\n return model, PostProcess()\n return model\n\n\ndef detr_resnet50_dc5(pretrained=False, num_classes=91, return_postprocessor=False):\n \"\"\"\n DETR-DC5 R50 with 6 encoder and 6 decoder layers.\n\n The last block of ResNet-50 has dilation to increase\n output resolution.\n Achieves 43.3/63.1 AP/AP50 on COCO val5k.\n \"\"\"\n model = _make_detr(\"resnet50\", dilation=True, num_classes=num_classes)\n if pretrained:\n checkpoint = torch.hub.load_state_dict_from_url(\n url=\"https://dl.fbaipublicfiles.com/detr/detr-r50-dc5-f0fb7ef5.pth\", map_location=\"cpu\", check_hash=True\n )\n model.load_state_dict(checkpoint[\"model\"])\n if return_postprocessor:\n return model, PostProcess()\n return model\n\n\ndef detr_resnet101(pretrained=False, num_classes=91, return_postprocessor=False):\n \"\"\"\n DETR-DC5 R101 with 6 encoder and 6 decoder layers.\n\n Achieves 43.5/63.8 AP/AP50 on COCO val5k.\n \"\"\"\n model = _make_detr(\"resnet101\", dilation=False, num_classes=num_classes)\n if pretrained:\n checkpoint = torch.hub.load_state_dict_from_url(\n url=\"https://dl.fbaipublicfiles.com/detr/detr-r101-2c7b67e5.pth\", map_location=\"cpu\", check_hash=True\n )\n model.load_state_dict(checkpoint[\"model\"])\n if return_postprocessor:\n return model, PostProcess()\n return model\n\n\ndef detr_resnet101_dc5(pretrained=False, num_classes=91, return_postprocessor=False):\n \"\"\"\n DETR-DC5 R101 with 6 encoder and 6 decoder layers.\n\n The last block of ResNet-101 has dilation to increase\n output resolution.\n Achieves 44.9/64.7 AP/AP50 on COCO val5k.\n \"\"\"\n model = _make_detr(\"resnet101\", dilation=True, num_classes=num_classes)\n if pretrained:\n checkpoint = torch.hub.load_state_dict_from_url(\n url=\"https://dl.fbaipublicfiles.com/detr/detr-r101-dc5-a2e86def.pth\", map_location=\"cpu\", check_hash=True\n )\n model.load_state_dict(checkpoint[\"model\"])\n if return_postprocessor:\n return model, PostProcess()\n return model\n\n\ndef detr_resnet50_panoptic(\n pretrained=False, num_classes=250, threshold=0.85, return_postprocessor=False\n):\n \"\"\"\n DETR R50 with 6 encoder and 6 decoder layers.\n Achieves 43.4 PQ on COCO val5k.\n\n threshold is the minimum confidence required for keeping segments in the prediction\n \"\"\"\n model = _make_detr(\"resnet50\", dilation=False, num_classes=num_classes, mask=True)\n is_thing_map = {i: i <= 90 for i in range(250)}\n if pretrained:\n checkpoint = torch.hub.load_state_dict_from_url(\n url=\"https://dl.fbaipublicfiles.com/detr/detr-r50-panoptic-00ce5173.pth\",\n map_location=\"cpu\",\n check_hash=True,\n )\n model.load_state_dict(checkpoint[\"model\"])\n if return_postprocessor:\n return model, PostProcessPanoptic(is_thing_map, threshold=threshold)\n return model\n\n\ndef detr_resnet50_dc5_panoptic(\n pretrained=False, num_classes=250, threshold=0.85, return_postprocessor=False\n):\n \"\"\"\n DETR-DC5 R50 with 6 encoder and 6 decoder layers.\n\n The last block of ResNet-50 has dilation to increase\n output resolution.\n Achieves 44.6 on COCO val5k.\n\n threshold is the minimum confidence required for keeping segments in the prediction\n \"\"\"\n model = _make_detr(\"resnet50\", dilation=True, num_classes=num_classes, mask=True)\n is_thing_map = {i: i <= 90 for i in range(250)}\n if pretrained:\n checkpoint = torch.hub.load_state_dict_from_url(\n url=\"https://dl.fbaipublicfiles.com/detr/detr-r50-dc5-panoptic-da08f1b1.pth\",\n map_location=\"cpu\",\n check_hash=True,\n )\n model.load_state_dict(checkpoint[\"model\"])\n if return_postprocessor:\n return model, PostProcessPanoptic(is_thing_map, threshold=threshold)\n return model\n\n\ndef detr_resnet101_panoptic(\n pretrained=False, num_classes=250, threshold=0.85, return_postprocessor=False\n):\n \"\"\"\n DETR-DC5 R101 with 6 encoder and 6 decoder layers.\n\n Achieves 45.1 PQ on COCO val5k.\n\n threshold is the minimum confidence required for keeping segments in the prediction\n \"\"\"\n model = _make_detr(\"resnet101\", dilation=False, num_classes=num_classes, mask=True)\n is_thing_map = {i: i <= 90 for i in range(250)}\n if pretrained:\n checkpoint = torch.hub.load_state_dict_from_url(\n url=\"https://dl.fbaipublicfiles.com/detr/detr-r101-panoptic-40021d53.pth\",\n map_location=\"cpu\",\n check_hash=True,\n )\n model.load_state_dict(checkpoint[\"model\"])\n if return_postprocessor:\n return model, PostProcessPanoptic(is_thing_map, threshold=threshold)\n return model\n"
},
{
"path": "batchformer-v2/detr/main.py",
"content": "# Copyright (c) Facebook, Inc. and its affiliates. All Rights Reserved\nimport argparse\nimport datetime\nimport json\nimport random\nimport time\nfrom pathlib import Path\n\nimport numpy as np\nimport torch\nfrom torch.utils.data import DataLoader, DistributedSampler\n\nimport datasets\nimport util.misc as utils\nfrom datasets import build_dataset, get_coco_api_from_dataset\nfrom engine import evaluate, train_one_epoch\nfrom models import build_model\n\n\ndef get_args_parser():\n parser = argparse.ArgumentParser('Set transformer detector', add_help=False)\n parser.add_argument('--lr', default=1e-4, type=float)\n parser.add_argument('--lr_backbone', default=1e-5, type=float)\n parser.add_argument('--batch_size', default=2, type=int)\n parser.add_argument('--weight_decay', default=1e-4, type=float)\n parser.add_argument('--epochs', default=300, type=int)\n parser.add_argument('--lr_drop', default=200, type=int)\n parser.add_argument('--clip_max_norm', default=0.1, type=float,\n help='gradient clipping max norm')\n\n # Model parameters\n parser.add_argument('--frozen_weights', type=str, default=None,\n help=\"Path to the pretrained model. If set, only the mask head will be trained\")\n parser.add_argument('--pretrained', type=str, default=None,\n help=\"Path to the pretrained model. If set, only the mask head will be trained\")\n # * Backbone\n parser.add_argument('--backbone', default='resnet50', type=str,\n help=\"Name of the convolutional backbone to use\")\n parser.add_argument('--dilation', action='store_true',\n help=\"If true, we replace stride with dilation in the last convolutional block (DC5)\")\n parser.add_argument('--position_embedding', default='sine', type=str, choices=('sine', 'learned'),\n help=\"Type of positional embedding to use on top of the image features\")\n\n # * Transformer\n parser.add_argument('--enc_layers', default=6, type=int,\n help=\"Number of encoding layers in the transformer\")\n parser.add_argument('--dec_layers', default=6, type=int,\n help=\"Number of decoding layers in the transformer\")\n parser.add_argument('--dim_feedforward', default=2048, type=int,\n help=\"Intermediate size of the feedforward layers in the transformer blocks\")\n parser.add_argument('--hidden_dim', default=256, type=int,\n help=\"Size of the embeddings (dimension of the transformer)\")\n parser.add_argument('--dropout', default=0.1, type=float,\n help=\"Dropout applied in the transformer\")\n parser.add_argument('--nheads', default=8, type=int,\n help=\"Number of attention heads inside the transformer's attentions\")\n parser.add_argument('--num_queries', default=100, type=int,\n help=\"Number of query slots\")\n parser.add_argument('--pre_norm', action='store_true')\n\n # * Segmentation\n parser.add_argument('--masks', action='store_true',\n help=\"Train segmentation head if the flag is provided\")\n\n # Loss\n parser.add_argument('--no_aux_loss', dest='aux_loss', action='store_false',\n help=\"Disables auxiliary decoding losses (loss at each layer)\")\n # * Matcher\n parser.add_argument('--set_cost_class', default=1, type=float,\n help=\"Class coefficient in the matching cost\")\n parser.add_argument('--set_cost_bbox', default=5, type=float,\n help=\"L1 box coefficient in the matching cost\")\n parser.add_argument('--set_cost_giou', default=2, type=float,\n help=\"giou box coefficient in the matching cost\")\n # * Loss coefficients\n parser.add_argument('--mask_loss_coef', default=1, type=float)\n parser.add_argument('--dice_loss_coef', default=1, type=float)\n parser.add_argument('--bbox_loss_coef', default=5, type=float)\n parser.add_argument('--giou_loss_coef', default=2, type=float)\n parser.add_argument('--eos_coef', default=0.1, type=float,\n help=\"Relative classification weight of the no-object class\")\n parser.add_argument('--focal_alpha', default=0., type=float)\n\n\n # dataset parameters\n parser.add_argument('--dataset_file', default='coco')\n parser.add_argument('--coco_path', type=str)\n parser.add_argument('--coco_panoptic_path', type=str)\n parser.add_argument('--remove_difficult', action='store_true')\n\n parser.add_argument('--output_dir', default='',\n help='path where to save, empty for no saving')\n parser.add_argument('--device', default='cuda',\n help='device to use for training / testing')\n parser.add_argument('--seed', default=42, type=int)\n parser.add_argument('--resume', default='', help='resume from checkpoint')\n parser.add_argument('--start_epoch', default=0, type=int, metavar='N',\n help='start epoch')\n parser.add_argument('--eval', action='store_true')\n parser.add_argument('--num_workers', default=2, type=int)\n parser.add_argument('--bf', default=0, type=int)\n parser.add_argument('--base_bf', default=0, type=int)\n parser.add_argument('--bf_lr', default=1., type=float)\n parser.add_argument('--bt_decay', default=1e-4, type=float)\n parser.add_argument('--share_bf', default=1, type=int)\n parser.add_argument('--start_idx', default=0, type=int)\n parser.add_argument('--insert_idx', action='append', type=int,\n help='insert idx list')\n parser.add_argument('--use_checkpoint', action='store_true')\n\n # distributed training parameters\n parser.add_argument('--world_size', default=1, type=int,\n help='number of distributed processes')\n parser.add_argument('--dist_url', default='env://', help='url used to set up distributed training')\n return parser\n\n\ndef main(args):\n utils.init_distributed_mode(args)\n if args.insert_idx is None or len(args.insert_idx) == 0:\n args.insert_idx = [i for i in range(args.start_idx, args.enc_layers)]\n print(\"git:\\n {}\\n\".format(utils.get_sha()))\n\n if args.frozen_weights is not None:\n assert args.masks, \"Frozen training is meant for segmentation only\"\n print(args)\n\n device = torch.device(args.device)\n\n # fix the seed for reproducibility\n seed = args.seed + utils.get_rank()\n torch.manual_seed(seed)\n np.random.seed(seed)\n random.seed(seed)\n\n model, criterion, postprocessors = build_model(args)\n model.to(device)\n\n model_without_ddp = model\n if args.distributed:\n model = torch.nn.parallel.DistributedDataParallel(model, device_ids=[args.gpu], find_unused_parameters=False)\n model_without_ddp = model.module\n n_parameters = sum(p.numel() for p in model.parameters() if p.requires_grad)\n print('number of params:', n_parameters)\n\n param_dicts = [\n {\"params\": [p for n, p in model_without_ddp.named_parameters() if \"backbone\" not in n and p.requires_grad and \"bf\" not in n]},\n {\"params\": [p for n, p in model_without_ddp.named_parameters() if \"backbone\" not in n and \"bf\" in n and p.requires_grad],\n \"lr\": args.lr*args.bf_lr, 'weight_decay': args.bt_decay},\n {\n \"params\": [p for n, p in model_without_ddp.named_parameters() if \"backbone\" in n and p.requires_grad],\n \"lr\": args.lr_backbone,\n },\n ]\n print([n for n, p in model_without_ddp.named_parameters() if \"backbone\" not in n and \"bf\" in n and p.requires_grad])\n optimizer = torch.optim.AdamW(param_dicts, lr=args.lr,\n weight_decay=args.weight_decay)\n lr_scheduler = torch.optim.lr_scheduler.StepLR(optimizer, args.lr_drop)\n\n dataset_train = build_dataset(image_set='train', args=args)\n dataset_val = build_dataset(image_set='val', args=args)\n\n if args.distributed:\n sampler_train = DistributedSampler(dataset_train)\n sampler_val = DistributedSampler(dataset_val, shuffle=False)\n else:\n sampler_train = torch.utils.data.RandomSampler(dataset_train)\n sampler_val = torch.utils.data.SequentialSampler(dataset_val)\n\n batch_sampler_train = torch.utils.data.BatchSampler(\n sampler_train, args.batch_size, drop_last=True)\n\n data_loader_train = DataLoader(dataset_train, batch_sampler=batch_sampler_train,\n collate_fn=utils.collate_fn, num_workers=args.num_workers)\n data_loader_val = DataLoader(dataset_val, args.batch_size, sampler=sampler_val,\n drop_last=False, collate_fn=utils.collate_fn, num_workers=args.num_workers)\n\n if args.dataset_file == \"coco_panoptic\":\n # We also evaluate AP during panoptic training, on original coco DS\n coco_val = datasets.coco.build(\"val\", args)\n base_ds = get_coco_api_from_dataset(coco_val)\n else:\n base_ds = get_coco_api_from_dataset(dataset_val)\n\n if args.frozen_weights is not None:\n checkpoint = torch.load(args.frozen_weights, map_location='cpu')\n model_without_ddp.detr.load_state_dict(checkpoint['model'], False)\n\n output_dir = Path(args.output_dir)\n if args.resume:\n if args.resume.startswith('https'):\n checkpoint = torch.hub.load_state_dict_from_url(\n args.resume, map_location='cpu', check_hash=True)\n else:\n checkpoint = torch.load(args.resume, map_location='cpu')\n model_without_ddp.load_state_dict(checkpoint['model'])\n if not args.eval and 'optimizer' in checkpoint and 'lr_scheduler' in checkpoint and 'epoch' in checkpoint:\n optimizer.load_state_dict(checkpoint['optimizer'])\n lr_scheduler.load_state_dict(checkpoint['lr_scheduler'])\n lr_scheduler.step_size = args.lr_drop\n lr_scheduler.base_lrs = list(map(lambda group: group['initial_lr'], optimizer.param_groups))\n lr_scheduler.step(lr_scheduler.last_epoch)\n args.start_epoch = checkpoint['epoch'] + 1\n elif args.pretrained:\n checkpoint = torch.load(args.pretrained, map_location='cpu')\n if 'model' in checkpoint:\n checkpoint = checkpoint['model']\n if args.eval:\n model_without_ddp.load_state_dict(checkpoint, strict=True)\n else:\n print(type(checkpoint))\n new_checkpoint = {}\n for k in checkpoint:\n k1 = k.replace('backbone.bottom_up.', 'backbone.0.body.')\n if not k1.__contains__('backbone.0.body.'):\n k1 = 'backbone.0.body.'+k1\n new_checkpoint[k1] = checkpoint[k]\n res = model_without_ddp.load_state_dict(new_checkpoint, strict=False)\n # print(res)\n for k in res.missing_keys:\n if not k.__contains__('transformer.encoder') and not k.__contains__('transformer.decoder'):\n print(k)\n print('unexcep======')\n for k in res.unexpected_keys:\n if not k.__contains__('transformer.encoder') and not k.__contains__('transformer.decoder'):\n print(k)\n\n if args.eval:\n test_stats, coco_evaluator = evaluate(model, criterion, postprocessors,\n data_loader_val, base_ds, device, args.output_dir)\n if args.output_dir:\n utils.save_on_master(coco_evaluator.coco_eval[\"bbox\"].eval, output_dir / \"eval.pth\")\n return\n\n print(\"Start training\")\n start_time = time.time()\n for epoch in range(args.start_epoch, args.epochs):\n for n,p in model_without_ddp.named_parameters():\n print(n, p.requires_grad)\n if args.distributed:\n sampler_train.set_epoch(epoch)\n train_stats = train_one_epoch(\n model, criterion, data_loader_train, optimizer, device, epoch,\n args.clip_max_norm)\n lr_scheduler.step()\n if args.output_dir:\n checkpoint_paths = [output_dir / 'checkpoint.pth']\n # extra checkpoint before LR drop and every 100 epochs\n if (epoch + 1) % args.lr_drop == 0 or (epoch + 1) % 10 == 0:\n checkpoint_paths.append(output_dir / f'checkpoint{epoch:04}.pth')\n for checkpoint_path in checkpoint_paths:\n utils.save_on_master({\n 'model': model_without_ddp.state_dict(),\n 'optimizer': optimizer.state_dict(),\n 'lr_scheduler': lr_scheduler.state_dict(),\n 'epoch': epoch,\n 'args': args,\n }, checkpoint_path)\n\n test_stats, coco_evaluator = evaluate(\n model, criterion, postprocessors, data_loader_val, base_ds, device, args.output_dir\n )\n\n log_stats = {**{f'train_{k}': v for k, v in train_stats.items()},\n **{f'test_{k}': v for k, v in test_stats.items()},\n 'epoch': epoch,\n 'n_parameters': n_parameters}\n\n if args.output_dir and utils.is_main_process():\n with (output_dir / \"log.txt\").open(\"a\") as f:\n f.write(json.dumps(log_stats) + \"\\n\")\n\n # for evaluation logs\n if coco_evaluator is not None:\n (output_dir / 'eval').mkdir(exist_ok=True)\n if \"bbox\" in coco_evaluator.coco_eval:\n filenames = ['latest.pth']\n if epoch % 50 == 0:\n filenames.append(f'{epoch:03}.pth')\n for name in filenames:\n torch.save(coco_evaluator.coco_eval[\"bbox\"].eval,\n output_dir / \"eval\" / name)\n\n total_time = time.time() - start_time\n total_time_str = str(datetime.timedelta(seconds=int(total_time)))\n print('Training time {}'.format(total_time_str))\n\n\nif __name__ == '__main__':\n parser = argparse.ArgumentParser('DETR training and evaluation script', parents=[get_args_parser()])\n args = parser.parse_args()\n if args.output_dir:\n Path(args.output_dir).mkdir(parents=True, exist_ok=True)\n main(args)\n"
},
{
"path": "batchformer-v2/detr/models/__init__.py",
"content": "# Copyright (c) Facebook, Inc. and its affiliates. All Rights Reserved\nfrom .detr import build\n\n\ndef build_model(args):\n return build(args)\n"
},
{
"path": "batchformer-v2/detr/models/backbone.py",
"content": "# Copyright (c) Facebook, Inc. and its affiliates. All Rights Reserved\n\"\"\"\nBackbone modules.\n\"\"\"\nfrom collections import OrderedDict\n\nimport torch\nimport torch.nn.functional as F\nimport torchvision\nfrom torch import nn\nfrom torchvision.models._utils import IntermediateLayerGetter\nfrom typing import Dict, List\n\nfrom util.misc import NestedTensor, is_main_process\n\nfrom .position_encoding import build_position_encoding\n\n\nclass FrozenBatchNorm2d(torch.nn.Module):\n \"\"\"\n BatchNorm2d where the batch statistics and the affine parameters are fixed.\n\n Copy-paste from torchvision.misc.ops with added eps before rqsrt,\n without which any other models than torchvision.models.resnet[18,34,50,101]\n produce nans.\n \"\"\"\n\n def __init__(self, n):\n super(FrozenBatchNorm2d, self).__init__()\n self.register_buffer(\"weight\", torch.ones(n))\n self.register_buffer(\"bias\", torch.zeros(n))\n self.register_buffer(\"running_mean\", torch.zeros(n))\n self.register_buffer(\"running_var\", torch.ones(n))\n\n def _load_from_state_dict(self, state_dict, prefix, local_metadata, strict,\n missing_keys, unexpected_keys, error_msgs):\n num_batches_tracked_key = prefix + 'num_batches_tracked'\n if num_batches_tracked_key in state_dict:\n del state_dict[num_batches_tracked_key]\n\n super(FrozenBatchNorm2d, self)._load_from_state_dict(\n state_dict, prefix, local_metadata, strict,\n missing_keys, unexpected_keys, error_msgs)\n\n def forward(self, x):\n # move reshapes to the beginning\n # to make it fuser-friendly\n w = self.weight.reshape(1, -1, 1, 1)\n b = self.bias.reshape(1, -1, 1, 1)\n rv = self.running_var.reshape(1, -1, 1, 1)\n rm = self.running_mean.reshape(1, -1, 1, 1)\n eps = 1e-5\n scale = w * (rv + eps).rsqrt()\n bias = b - rm * scale\n return x * scale + bias\n\n\nclass BackboneBase(nn.Module):\n\n def __init__(self, backbone: nn.Module, train_backbone: bool, num_channels: int, return_interm_layers: bool,\n args=None):\n super().__init__()\n for name, parameter in backbone.named_parameters():\n if not train_backbone or 'layer2' not in name and 'layer3' not in name and 'layer4' not in name:\n parameter.requires_grad_(False)\n if return_interm_layers:\n return_layers = {\"layer1\": \"0\", \"layer2\": \"1\", \"layer3\": \"2\", \"layer4\": \"3\"}\n else:\n return_layers = {'layer4': \"0\"}\n self.body = IntermediateLayerGetter(backbone, return_layers=return_layers)\n self.num_channels = num_channels\n self.use_checkpoint = args.use_checkpoint\n\n def forward(self, tensor_list: NestedTensor):\n xs = self.body(tensor_list.tensors)\n out: Dict[str, NestedTensor] = {}\n for name, x in xs.items():\n m = tensor_list.mask\n assert m is not None\n mask = F.interpolate(m[None].float(), size=x.shape[-2:]).to(torch.bool)[0]\n out[name] = NestedTensor(x, mask)\n return out\n\n\nclass Backbone(BackboneBase):\n \"\"\"ResNet backbone with frozen BatchNorm.\"\"\"\n def __init__(self, name: str,\n train_backbone: bool,\n return_interm_layers: bool,\n dilation: bool,\n args=None):\n backbone = getattr(torchvision.models, name)(\n replace_stride_with_dilation=[False, False, dilation],\n pretrained=is_main_process(), norm_layer=FrozenBatchNorm2d)\n num_channels = 512 if name in ('resnet18', 'resnet34') else 2048\n super().__init__(backbone, train_backbone, num_channels, return_interm_layers, args)\n\n\nclass Joiner(nn.Sequential):\n def __init__(self, backbone, position_embedding):\n super().__init__(backbone, position_embedding)\n\n def forward(self, tensor_list: NestedTensor):\n xs = self[0](tensor_list)\n out: List[NestedTensor] = []\n pos = []\n for name, x in xs.items():\n out.append(x)\n # position encoding\n pos.append(self[1](x).to(x.tensors.dtype))\n\n return out, pos\n\n\ndef build_backbone(args):\n position_embedding = build_position_encoding(args)\n train_backbone = args.lr_backbone > 0\n return_interm_layers = args.masks\n backbone = Backbone(args.backbone, train_backbone, return_interm_layers, args.dilation, args)\n model = Joiner(backbone, position_embedding)\n model.num_channels = backbone.num_channels\n return model\n\nclass SwinBackbone(nn.Module):\n def __init__(self, name: str,\n train_backbone: bool,\n return_interm_layers: bool,\n dilation: bool):\n super().__init__()\n from models.swin_transformer1 import SwinTransformer\n self.body = SwinTransformer(\n patch_size=4,\n in_chans=3,\n embed_dim=96,\n depths=[2, 2, 6, 2],\n num_heads=[3, 6, 12, 24],\n window_size=7,\n mlp_ratio=4,\n qkv_bias=True,\n qk_scale=None,\n drop_rate=0.,\n attn_drop_rate=0.,\n drop_path_rate=0.2,\n norm_layer=nn.LayerNorm,\n ape=False,\n patch_norm=True,\n frozen_stages=-1,\n out_features=[\"stage2\", \"stage3\", \"stage4\", \"stage5\"]\n )\n if return_interm_layers:\n return_layers = {\"stage2\": \"0\", \"stage3\": \"1\", \"stage4\": \"2\", \"stage5\": \"3\"}\n else:\n return_layers = {'stage5': \"0\"}\n num_channels = 512 if name in ('resnet18', 'resnet34') else 2048\n self.num_channels = 768\n\n def forward(self, tensor_list: NestedTensor):\n xs = self.body(tensor_list.tensors)\n out: Dict[str, NestedTensor] = {}\n for name, x in xs.items():\n m = tensor_list.mask\n assert m is not None\n mask = F.interpolate(m[None].float(), size=x.shape[-2:]).to(torch.bool)[0]\n out[name] = NestedTensor(x, mask)\n return out\n\n\ndef build_swin_backbone(args):\n from models.position_encoding import build_position_encoding\n position_embedding = build_position_encoding(args)\n train_backbone = args.lr_backbone > 0\n return_interm_layers = args.masks\n backbone = SwinBackbone(args.backbone, train_backbone, return_interm_layers, args.dilation)\n model = Joiner(backbone, position_embedding)\n model.num_channels = backbone.num_channels\n\n return model\n"
},
{
"path": "batchformer-v2/detr/models/detr.py",
"content": "# Copyright (c) Facebook, Inc. and its affiliates. All Rights Reserved\n\"\"\"\nDETR model and criterion classes.\n\"\"\"\nimport torch\nimport torch.nn.functional as F\nfrom torch import nn\n\nfrom util import box_ops\nfrom util.misc import (NestedTensor, nested_tensor_from_tensor_list,\n accuracy, get_world_size, interpolate,\n is_dist_avail_and_initialized)\n\nfrom .backbone import build_backbone\nfrom .matcher import build_matcher\nfrom .segmentation import (DETRsegm, PostProcessPanoptic, PostProcessSegm,\n dice_loss, sigmoid_focal_loss)\nfrom .transformer import build_transformer\n\n\nclass DETR(nn.Module):\n \"\"\" This is the DETR module that performs object detection \"\"\"\n def __init__(self, backbone, transformer, num_classes, num_queries, aux_loss=False, args=None):\n \"\"\" Initializes the model.\n Parameters:\n backbone: torch module of the backbone to be used. See backbone.py\n transformer: torch module of the transformer architecture. See transformer.py\n num_classes: number of object classes\n num_queries: number of object queries, ie detection slot. This is the maximal number of objects\n DETR can detect in a single image. For COCO, we recommend 100 queries.\n aux_loss: True if auxiliary decoding losses (loss at each decoder layer) are to be used.\n \"\"\"\n super().__init__()\n self.num_queries = num_queries\n self.transformer = transformer\n hidden_dim = transformer.d_model\n self.class_embed = nn.Linear(hidden_dim, num_classes + 1)\n self.bbox_embed = MLP(hidden_dim, hidden_dim, 4, 3)\n self.query_embed = nn.Embedding(num_queries, hidden_dim)\n self.input_proj = nn.Conv2d(backbone.num_channels, hidden_dim, kernel_size=1)\n self.backbone = backbone\n self.aux_loss = aux_loss\n self.use_checkpoint=args.use_checkpoint\n\n def forward(self, samples: NestedTensor):\n \"\"\" The forward expects a NestedTensor, which consists of:\n - samples.tensor: batched images, of shape [batch_size x 3 x H x W]\n - samples.mask: a binary mask of shape [batch_size x H x W], containing 1 on padded pixels\n\n It returns a dict with the following elements:\n - \"pred_logits\": the classification logits (including no-object) for all queries.\n Shape= [batch_size x num_queries x (num_classes + 1)]\n - \"pred_boxes\": The normalized boxes coordinates for all queries, represented as\n (center_x, center_y, height, width). These values are normalized in [0, 1],\n relative to the size of each individual image (disregarding possible padding).\n See PostProcess for information on how to retrieve the unnormalized bounding box.\n - \"aux_outputs\": Optional, only returned when auxilary losses are activated. It is a list of\n dictionnaries containing the two above keys for each decoder layer.\n \"\"\"\n if isinstance(samples, (list, torch.Tensor)):\n samples = nested_tensor_from_tensor_list(samples)\n features, pos = self.backbone(samples)\n\n src, mask = features[-1].decompose()\n assert mask is not None\n hs = self.transformer(self.input_proj(src), mask, self.query_embed.weight, pos[-1])[0]\n\n outputs_class = self.class_embed(hs)\n outputs_coord = self.bbox_embed(hs).sigmoid()\n out = {'pred_logits': outputs_class[-1], 'pred_boxes': outputs_coord[-1]}\n if self.aux_loss:\n out['aux_outputs'] = self._set_aux_loss(outputs_class, outputs_coord)\n return out\n\n @torch.jit.unused\n def _set_aux_loss(self, outputs_class, outputs_coord):\n # this is a workaround to make torchscript happy, as torchscript\n # doesn't support dictionary with non-homogeneous values, such\n # as a dict having both a Tensor and a list.\n return [{'pred_logits': a, 'pred_boxes': b}\n for a, b in zip(outputs_class[:-1], outputs_coord[:-1])]\n\n\nclass SetCriterion(nn.Module):\n \"\"\" This class computes the loss for DETR.\n The process happens in two steps:\n 1) we compute hungarian assignment between ground truth boxes and the outputs of the model\n 2) we supervise each pair of matched ground-truth / prediction (supervise class and box)\n \"\"\"\n def __init__(self, num_classes, matcher, weight_dict, eos_coef, losses, focal_alpha=0.25):\n \"\"\" Create the criterion.\n Parameters:\n num_classes: number of object categories, omitting the special no-object category\n matcher: module able to compute a matching between targets and proposals\n weight_dict: dict containing as key the names of the losses and as values their relative weight.\n eos_coef: relative classification weight applied to the no-object category\n losses: list of all the losses to be applied. See get_loss for list of available losses.\n \"\"\"\n super().__init__()\n self.num_classes = num_classes\n self.matcher = matcher\n self.weight_dict = weight_dict\n self.eos_coef = eos_coef\n self.losses = losses\n self.focal_alpha = focal_alpha\n\n empty_weight = torch.ones(self.num_classes + 1)\n empty_weight[-1] = self.eos_coef\n self.register_buffer('empty_weight', empty_weight)\n\n def loss_labels(self, outputs, targets, indices, num_boxes, log=True):\n \"\"\"Classification loss (NLL)\n targets dicts must contain the key \"labels\" containing a tensor of dim [nb_target_boxes]\n \"\"\"\n assert 'pred_logits' in outputs\n src_logits = outputs['pred_logits']\n\n idx = self._get_src_permutation_idx(indices)\n target_classes_o = torch.cat([t[\"labels\"][J] for t, (_, J) in zip(targets, indices)])\n target_classes = torch.full(src_logits.shape[:2], self.num_classes,\n dtype=torch.int64, device=src_logits.device)\n target_classes[idx] = target_classes_o\n\n if self.focal_alpha > 0:\n target_classes_onehot = torch.zeros([src_logits.shape[0], src_logits.shape[1], src_logits.shape[2] + 1],\n dtype=src_logits.dtype, layout=src_logits.layout, device=src_logits.device)\n target_classes_onehot.scatter_(2, target_classes.unsqueeze(-1), 1)\n\n target_classes_onehot = target_classes_onehot[:,:,:-1]\n loss_ce = sigmoid_focal_loss(src_logits, target_classes_onehot, num_boxes, alpha=self.focal_alpha, gamma=2) * src_logits.shape[1]\n else:\n loss_ce = F.cross_entropy(src_logits.transpose(1, 2), target_classes, self.empty_weight)\n losses = {'loss_ce': loss_ce}\n\n if log:\n # TODO this should probably be a separate loss, not hacked in this one here\n losses['class_error'] = 100 - accuracy(src_logits[idx], target_classes_o)[0]\n return losses\n\n @torch.no_grad()\n def loss_cardinality(self, outputs, targets, indices, num_boxes):\n \"\"\" Compute the cardinality error, ie the absolute error in the number of predicted non-empty boxes\n This is not really a loss, it is intended for logging purposes only. It doesn't propagate gradients\n \"\"\"\n pred_logits = outputs['pred_logits']\n device = pred_logits.device\n tgt_lengths = torch.as_tensor([len(v[\"labels\"]) for v in targets], device=device)\n # Count the number of predictions that are NOT \"no-object\" (which is the last class)\n card_pred = (pred_logits.argmax(-1) != pred_logits.shape[-1] - 1).sum(1)\n card_err = F.l1_loss(card_pred.float(), tgt_lengths.float())\n losses = {'cardinality_error': card_err}\n return losses\n\n def loss_boxes(self, outputs, targets, indices, num_boxes):\n \"\"\"Compute the losses related to the bounding boxes, the L1 regression loss and the GIoU loss\n targets dicts must contain the key \"boxes\" containing a tensor of dim [nb_target_boxes, 4]\n The target boxes are expected in format (center_x, center_y, w, h), normalized by the image size.\n \"\"\"\n assert 'pred_boxes' in outputs\n idx = self._get_src_permutation_idx(indices)\n src_boxes = outputs['pred_boxes'][idx]\n target_boxes = torch.cat([t['boxes'][i] for t, (_, i) in zip(targets, indices)], dim=0)\n\n loss_bbox = F.l1_loss(src_boxes, target_boxes, reduction='none')\n\n losses = {}\n losses['loss_bbox'] = loss_bbox.sum() / num_boxes\n\n loss_giou = 1 - torch.diag(box_ops.generalized_box_iou(\n box_ops.box_cxcywh_to_xyxy(src_boxes),\n box_ops.box_cxcywh_to_xyxy(target_boxes)))\n losses['loss_giou'] = loss_giou.sum() / num_boxes\n return losses\n\n def loss_masks(self, outputs, targets, indices, num_boxes):\n \"\"\"Compute the losses related to the masks: the focal loss and the dice loss.\n targets dicts must contain the key \"masks\" containing a tensor of dim [nb_target_boxes, h, w]\n \"\"\"\n assert \"pred_masks\" in outputs\n\n src_idx = self._get_src_permutation_idx(indices)\n tgt_idx = self._get_tgt_permutation_idx(indices)\n src_masks = outputs[\"pred_masks\"]\n src_masks = src_masks[src_idx]\n masks = [t[\"masks\"] for t in targets]\n # TODO use valid to mask invalid areas due to padding in loss\n target_masks, valid = nested_tensor_from_tensor_list(masks).decompose()\n target_masks = target_masks.to(src_masks)\n target_masks = target_masks[tgt_idx]\n\n # upsample predictions to the target size\n src_masks = interpolate(src_masks[:, None], size=target_masks.shape[-2:],\n mode=\"bilinear\", align_corners=False)\n src_masks = src_masks[:, 0].flatten(1)\n\n target_masks = target_masks.flatten(1)\n target_masks = target_masks.view(src_masks.shape)\n losses = {\n \"loss_mask\": sigmoid_focal_loss(src_masks, target_masks, num_boxes),\n \"loss_dice\": dice_loss(src_masks, target_masks, num_boxes),\n }\n return losses\n\n def _get_src_permutation_idx(self, indices):\n # permute predictions following indices\n batch_idx = torch.cat([torch.full_like(src, i) for i, (src, _) in enumerate(indices)])\n src_idx = torch.cat([src for (src, _) in indices])\n return batch_idx, src_idx\n\n def _get_tgt_permutation_idx(self, indices):\n # permute targets following indices\n batch_idx = torch.cat([torch.full_like(tgt, i) for i, (_, tgt) in enumerate(indices)])\n tgt_idx = torch.cat([tgt for (_, tgt) in indices])\n return batch_idx, tgt_idx\n\n def get_loss(self, loss, outputs, targets, indices, num_boxes, **kwargs):\n loss_map = {\n 'labels': self.loss_labels,\n 'cardinality': self.loss_cardinality,\n 'boxes': self.loss_boxes,\n 'masks': self.loss_masks\n }\n assert loss in loss_map, f'do you really want to compute {loss} loss?'\n return loss_map[loss](outputs, targets, indices, num_boxes, **kwargs)\n\n def forward(self, outputs, targets):\n \"\"\" This performs the loss computation.\n Parameters:\n outputs: dict of tensors, see the output specification of the model for the format\n targets: list of dicts, such that len(targets) == batch_size.\n The expected keys in each dict depends on the losses applied, see each loss' doc\n \"\"\"\n outputs_without_aux = {k: v for k, v in outputs.items() if k != 'aux_outputs'}\n\n # Retrieve the matching between the outputs of the last layer and the targets\n indices = self.matcher(outputs_without_aux, targets)\n\n # Compute the average number of target boxes accross all nodes, for normalization purposes\n num_boxes = sum(len(t[\"labels\"]) for t in targets)\n num_boxes = torch.as_tensor([num_boxes], dtype=torch.float, device=next(iter(outputs.values())).device)\n if is_dist_avail_and_initialized():\n torch.distributed.all_reduce(num_boxes)\n num_boxes = torch.clamp(num_boxes / get_world_size(), min=1).item()\n\n # Compute all the requested losses\n losses = {}\n for loss in self.losses:\n losses.update(self.get_loss(loss, outputs, targets, indices, num_boxes))\n\n # In case of auxiliary losses, we repeat this process with the output of each intermediate layer.\n if 'aux_outputs' in outputs:\n for i, aux_outputs in enumerate(outputs['aux_outputs']):\n indices = self.matcher(aux_outputs, targets)\n for loss in self.losses:\n if loss == 'masks':\n # Intermediate masks losses are too costly to compute, we ignore them.\n continue\n kwargs = {}\n if loss == 'labels':\n # Logging is enabled only for the last layer\n kwargs = {'log': False}\n l_dict = self.get_loss(loss, aux_outputs, targets, indices, num_boxes, **kwargs)\n l_dict = {k + f'_{i}': v for k, v in l_dict.items()}\n losses.update(l_dict)\n\n return losses\n\n\nclass PostProcess(nn.Module):\n \"\"\" This module converts the model's output into the format expected by the coco api\"\"\"\n @torch.no_grad()\n def forward(self, outputs, target_sizes):\n \"\"\" Perform the computation\n Parameters:\n outputs: raw outputs of the model\n target_sizes: tensor of dimension [batch_size x 2] containing the size of each images of the batch\n For evaluation, this must be the original image size (before any data augmentation)\n For visualization, this should be the image size after data augment, but before padding\n \"\"\"\n out_logits, out_bbox = outputs['pred_logits'], outputs['pred_boxes']\n\n assert len(out_logits) == len(target_sizes)\n assert target_sizes.shape[1] == 2\n\n prob = F.softmax(out_logits, -1)\n scores, labels = prob[..., :-1].max(-1)\n\n # convert to [x0, y0, x1, y1] format\n boxes = box_ops.box_cxcywh_to_xyxy(out_bbox)\n # and from relative [0, 1] to absolute [0, height] coordinates\n img_h, img_w = target_sizes.unbind(1)\n scale_fct = torch.stack([img_w, img_h, img_w, img_h], dim=1)\n boxes = boxes * scale_fct[:, None, :]\n\n results = [{'scores': s, 'labels': l, 'boxes': b} for s, l, b in zip(scores, labels, boxes)]\n\n return results\n\n\nclass MLP(nn.Module):\n \"\"\" Very simple multi-layer perceptron (also called FFN)\"\"\"\n\n def __init__(self, input_dim, hidden_dim, output_dim, num_layers):\n super().__init__()\n self.num_layers = num_layers\n h = [hidden_dim] * (num_layers - 1)\n self.layers = nn.ModuleList(nn.Linear(n, k) for n, k in zip([input_dim] + h, h + [output_dim]))\n\n def forward(self, x):\n for i, layer in enumerate(self.layers):\n x = F.relu(layer(x)) if i < self.num_layers - 1 else layer(x)\n return x\n\n\ndef build(args):\n # the `num_classes` naming here is somewhat misleading.\n # it indeed corresponds to `max_obj_id + 1`, where max_obj_id\n # is the maximum id for a class in your dataset. For example,\n # COCO has a max_obj_id of 90, so we pass `num_classes` to be 91.\n # As another example, for a dataset that has a single class with id 1,\n # you should pass `num_classes` to be 2 (max_obj_id + 1).\n # For more details on this, check the following discussion\n # https://github.com/facebookresearch/detr/issues/108#issuecomment-650269223\n num_classes = 20 if args.dataset_file != 'coco' else 91\n if args.dataset_file == 'hico':\n num_classes = 91\n if args.dataset_file == \"coco_panoptic\":\n # for panoptic, we just add a num_classes that is large enough to hold\n # max_obj_id + 1, but the exact value doesn't really matter\n num_classes = 250\n device = torch.device(args.device)\n\n if args.backbone.__contains__('swin'):\n from .backbone import build_swin_backbone\n backbone = build_swin_backbone(args)\n else:\n backbone = build_backbone(args)\n\n transformer = build_transformer(args)\n\n model = DETR(\n backbone,\n transformer,\n num_classes=num_classes,\n num_queries=args.num_queries,\n aux_loss=args.aux_loss,\n args=args,\n )\n if args.masks:\n model = DETRsegm(model, freeze_detr=(args.frozen_weights is not None))\n matcher = build_matcher(args)\n weight_dict = {'loss_ce': 1, 'loss_bbox': args.bbox_loss_coef}\n weight_dict['loss_giou'] = args.giou_loss_coef\n if args.masks:\n weight_dict[\"loss_mask\"] = args.mask_loss_coef\n weight_dict[\"loss_dice\"] = args.dice_loss_coef\n # TODO this is a hack\n if args.aux_loss:\n aux_weight_dict = {}\n for i in range(args.dec_layers - 1):\n aux_weight_dict.update({k + f'_{i}': v for k, v in weight_dict.items()})\n weight_dict.update(aux_weight_dict)\n\n losses = ['labels', 'boxes', 'cardinality']\n if args.masks:\n losses += [\"masks\"]\n criterion = SetCriterion(num_classes, matcher=matcher, weight_dict=weight_dict,\n eos_coef=args.eos_coef, losses=losses, focal_alpha=args.focal_alpha)\n criterion.to(device)\n postprocessors = {'bbox': PostProcess()}\n if args.masks:\n postprocessors['segm'] = PostProcessSegm()\n if args.dataset_file == \"coco_panoptic\":\n is_thing_map = {i: i <= 90 for i in range(201)}\n postprocessors[\"panoptic\"] = PostProcessPanoptic(is_thing_map, threshold=0.85)\n\n return model, criterion, postprocessors\n"
},
{
"path": "batchformer-v2/detr/models/matcher.py",
"content": "# Copyright (c) Facebook, Inc. and its affiliates. All Rights Reserved\n\"\"\"\nModules to compute the matching cost and solve the corresponding LSAP.\n\"\"\"\nimport torch\nfrom scipy.optimize import linear_sum_assignment\nfrom torch import nn\n\nfrom util.box_ops import box_cxcywh_to_xyxy, generalized_box_iou\n\n\nclass HungarianMatcher(nn.Module):\n \"\"\"This class computes an assignment between the targets and the predictions of the network\n\n For efficiency reasons, the targets don't include the no_object. Because of this, in general,\n there are more predictions than targets. In this case, we do a 1-to-1 matching of the best predictions,\n while the others are un-matched (and thus treated as non-objects).\n \"\"\"\n\n def __init__(self, cost_class: float = 1, cost_bbox: float = 1, cost_giou: float = 1):\n \"\"\"Creates the matcher\n\n Params:\n cost_class: This is the relative weight of the classification error in the matching cost\n cost_bbox: This is the relative weight of the L1 error of the bounding box coordinates in the matching cost\n cost_giou: This is the relative weight of the giou loss of the bounding box in the matching cost\n \"\"\"\n super().__init__()\n self.cost_class = cost_class\n self.cost_bbox = cost_bbox\n self.cost_giou = cost_giou\n assert cost_class != 0 or cost_bbox != 0 or cost_giou != 0, \"all costs cant be 0\"\n\n @torch.no_grad()\n def forward(self, outputs, targets):\n \"\"\" Performs the matching\n\n Params:\n outputs: This is a dict that contains at least these entries:\n \"pred_logits\": Tensor of dim [batch_size, num_queries, num_classes] with the classification logits\n \"pred_boxes\": Tensor of dim [batch_size, num_queries, 4] with the predicted box coordinates\n\n targets: This is a list of targets (len(targets) = batch_size), where each target is a dict containing:\n \"labels\": Tensor of dim [num_target_boxes] (where num_target_boxes is the number of ground-truth\n objects in the target) containing the class labels\n \"boxes\": Tensor of dim [num_target_boxes, 4] containing the target box coordinates\n\n Returns:\n A list of size batch_size, containing tuples of (index_i, index_j) where:\n - index_i is the indices of the selected predictions (in order)\n - index_j is the indices of the corresponding selected targets (in order)\n For each batch element, it holds:\n len(index_i) = len(index_j) = min(num_queries, num_target_boxes)\n \"\"\"\n bs, num_queries = outputs[\"pred_logits\"].shape[:2]\n\n # We flatten to compute the cost matrices in a batch\n out_prob = outputs[\"pred_logits\"].flatten(0, 1).softmax(-1) # [batch_size * num_queries, num_classes]\n out_bbox = outputs[\"pred_boxes\"].flatten(0, 1) # [batch_size * num_queries, 4]\n\n # Also concat the target labels and boxes\n tgt_ids = torch.cat([v[\"labels\"] for v in targets])\n tgt_bbox = torch.cat([v[\"boxes\"] for v in targets])\n\n # Compute the classification cost. Contrary to the loss, we don't use the NLL,\n # but approximate it in 1 - proba[target class].\n # The 1 is a constant that doesn't change the matching, it can be ommitted.\n cost_class = -out_prob[:, tgt_ids]\n\n # Compute the L1 cost between boxes\n cost_bbox = torch.cdist(out_bbox, tgt_bbox, p=1)\n\n # Compute the giou cost betwen boxes\n cost_giou = -generalized_box_iou(box_cxcywh_to_xyxy(out_bbox), box_cxcywh_to_xyxy(tgt_bbox))\n\n # Final cost matrix\n C = self.cost_bbox * cost_bbox + self.cost_class * cost_class + self.cost_giou * cost_giou\n C = C.view(bs, num_queries, -1).cpu()\n\n sizes = [len(v[\"boxes\"]) for v in targets]\n indices = [linear_sum_assignment(c[i]) for i, c in enumerate(C.split(sizes, -1))]\n return [(torch.as_tensor(i, dtype=torch.int64), torch.as_tensor(j, dtype=torch.int64)) for i, j in indices]\n\n\ndef build_matcher(args):\n return HungarianMatcher(cost_class=args.set_cost_class, cost_bbox=args.set_cost_bbox, cost_giou=args.set_cost_giou)\n"
},
{
"path": "batchformer-v2/detr/models/position_encoding.py",
"content": "# Copyright (c) Facebook, Inc. and its affiliates. All Rights Reserved\n\"\"\"\nVarious positional encodings for the transformer.\n\"\"\"\nimport math\nimport torch\nfrom torch import nn\n\nfrom util.misc import NestedTensor\n\n\nclass PositionEmbeddingSine(nn.Module):\n \"\"\"\n This is a more standard version of the position embedding, very similar to the one\n used by the Attention is all you need paper, generalized to work on images.\n \"\"\"\n def __init__(self, num_pos_feats=64, temperature=10000, normalize=False, scale=None):\n super().__init__()\n self.num_pos_feats = num_pos_feats\n self.temperature = temperature\n self.normalize = normalize\n if scale is not None and normalize is False:\n raise ValueError(\"normalize should be True if scale is passed\")\n if scale is None:\n scale = 2 * math.pi\n self.scale = scale\n\n def forward(self, tensor_list: NestedTensor):\n x = tensor_list.tensors\n mask = tensor_list.mask\n assert mask is not None\n not_mask = ~mask\n y_embed = not_mask.cumsum(1, dtype=torch.float32)\n x_embed = not_mask.cumsum(2, dtype=torch.float32)\n if self.normalize:\n eps = 1e-6\n y_embed = y_embed / (y_embed[:, -1:, :] + eps) * self.scale\n x_embed = x_embed / (x_embed[:, :, -1:] + eps) * self.scale\n\n dim_t = torch.arange(self.num_pos_feats, dtype=torch.float32, device=x.device)\n dim_t = self.temperature ** (2 * (dim_t // 2) / self.num_pos_feats)\n\n pos_x = x_embed[:, :, :, None] / dim_t\n pos_y = y_embed[:, :, :, None] / dim_t\n pos_x = torch.stack((pos_x[:, :, :, 0::2].sin(), pos_x[:, :, :, 1::2].cos()), dim=4).flatten(3)\n pos_y = torch.stack((pos_y[:, :, :, 0::2].sin(), pos_y[:, :, :, 1::2].cos()), dim=4).flatten(3)\n pos = torch.cat((pos_y, pos_x), dim=3).permute(0, 3, 1, 2)\n return pos\n\n\nclass PositionEmbeddingLearned(nn.Module):\n \"\"\"\n Absolute pos embedding, learned.\n \"\"\"\n def __init__(self, num_pos_feats=256):\n super().__init__()\n self.row_embed = nn.Embedding(50, num_pos_feats)\n self.col_embed = nn.Embedding(50, num_pos_feats)\n self.reset_parameters()\n\n def reset_parameters(self):\n nn.init.uniform_(self.row_embed.weight)\n nn.init.uniform_(self.col_embed.weight)\n\n def forward(self, tensor_list: NestedTensor):\n x = tensor_list.tensors\n h, w = x.shape[-2:]\n i = torch.arange(w, device=x.device)\n j = torch.arange(h, device=x.device)\n x_emb = self.col_embed(i)\n y_emb = self.row_embed(j)\n pos = torch.cat([\n x_emb.unsqueeze(0).repeat(h, 1, 1),\n y_emb.unsqueeze(1).repeat(1, w, 1),\n ], dim=-1).permute(2, 0, 1).unsqueeze(0).repeat(x.shape[0], 1, 1, 1)\n return pos\n\n\ndef build_position_encoding(args):\n N_steps = args.hidden_dim // 2\n if args.position_embedding in ('v2', 'sine'):\n # TODO find a better way of exposing other arguments\n position_embedding = PositionEmbeddingSine(N_steps, normalize=True)\n elif args.position_embedding in ('v3', 'learned'):\n position_embedding = PositionEmbeddingLearned(N_steps)\n else:\n raise ValueError(f\"not supported {args.position_embedding}\")\n\n return position_embedding\n"
},
{
"path": "batchformer-v2/detr/models/segmentation.py",
"content": "# Copyright (c) Facebook, Inc. and its affiliates. All Rights Reserved\n\"\"\"\nThis file provides the definition of the convolutional heads used to predict masks, as well as the losses\n\"\"\"\nimport io\nfrom collections import defaultdict\nfrom typing import List, Optional\n\nimport torch\nimport torch.nn as nn\nimport torch.nn.functional as F\nfrom torch import Tensor\nfrom PIL import Image\n\nimport util.box_ops as box_ops\nfrom util.misc import NestedTensor, interpolate, nested_tensor_from_tensor_list\n\ntry:\n from panopticapi.utils import id2rgb, rgb2id\nexcept ImportError:\n pass\n\n\nclass DETRsegm(nn.Module):\n def __init__(self, detr, freeze_detr=False):\n super().__init__()\n self.detr = detr\n\n if freeze_detr:\n for p in self.parameters():\n p.requires_grad_(False)\n\n hidden_dim, nheads = detr.transformer.d_model, detr.transformer.nhead\n self.bbox_attention = MHAttentionMap(hidden_dim, hidden_dim, nheads, dropout=0.0)\n self.mask_head = MaskHeadSmallConv(hidden_dim + nheads, [1024, 512, 256], hidden_dim)\n\n def forward(self, samples: NestedTensor):\n if isinstance(samples, (list, torch.Tensor)):\n samples = nested_tensor_from_tensor_list(samples)\n features, pos = self.detr.backbone(samples)\n\n bs = features[-1].tensors.shape[0]\n\n src, mask = features[-1].decompose()\n assert mask is not None\n src_proj = self.detr.input_proj(src)\n hs, memory = self.detr.transformer(src_proj, mask, self.detr.query_embed.weight, pos[-1])\n feat2 = features[2].tensors\n feat1 = features[1].tensors\n feat0 = features[0].tensors\n if len(memory) > len(src_proj):\n mask = torch.cat([mask, mask], dim=0)\n bs = bs + bs\n src_proj = torch.cat([src_proj, src_proj], dim=0)\n feat2 = torch.cat([feat2, feat2], dim=0)\n feat1 = torch.cat([feat1, feat1], dim=0)\n feat0 = torch.cat([feat0, feat0], dim=0)\n # import ipdb;ipdb.set_trace()\n outputs_class = self.detr.class_embed(hs)\n outputs_coord = self.detr.bbox_embed(hs).sigmoid()\n out = {\"pred_logits\": outputs_class[-1], \"pred_boxes\": outputs_coord[-1]}\n if self.detr.aux_loss:\n out['aux_outputs'] = self.detr._set_aux_loss(outputs_class, outputs_coord)\n\n # FIXME h_boxes takes the last one computed, keep this in mind\n bbox_mask = self.bbox_attention(hs[-1], memory, mask=mask)\n\n seg_masks = self.mask_head(src_proj, bbox_mask, [feat2, feat1, feat0])\n outputs_seg_masks = seg_masks.view(bs, self.detr.num_queries, seg_masks.shape[-2], seg_masks.shape[-1])\n\n out[\"pred_masks\"] = outputs_seg_masks\n return out\n\n\ndef _expand(tensor, length: int):\n return tensor.unsqueeze(1).repeat(1, int(length), 1, 1, 1).flatten(0, 1)\n\n\nclass MaskHeadSmallConv(nn.Module):\n \"\"\"\n Simple convolutional head, using group norm.\n Upsampling is done using a FPN approach\n \"\"\"\n\n def __init__(self, dim, fpn_dims, context_dim):\n super().__init__()\n\n inter_dims = [dim, context_dim // 2, context_dim // 4, context_dim // 8, context_dim // 16, context_dim // 64]\n self.lay1 = torch.nn.Conv2d(dim, dim, 3, padding=1)\n self.gn1 = torch.nn.GroupNorm(8, dim)\n self.lay2 = torch.nn.Conv2d(dim, inter_dims[1], 3, padding=1)\n self.gn2 = torch.nn.GroupNorm(8, inter_dims[1])\n self.lay3 = torch.nn.Conv2d(inter_dims[1], inter_dims[2], 3, padding=1)\n self.gn3 = torch.nn.GroupNorm(8, inter_dims[2])\n self.lay4 = torch.nn.Conv2d(inter_dims[2], inter_dims[3], 3, padding=1)\n self.gn4 = torch.nn.GroupNorm(8, inter_dims[3])\n self.lay5 = torch.nn.Conv2d(inter_dims[3], inter_dims[4], 3, padding=1)\n self.gn5 = torch.nn.GroupNorm(8, inter_dims[4])\n self.out_lay = torch.nn.Conv2d(inter_dims[4], 1, 3, padding=1)\n\n self.dim = dim\n\n self.adapter1 = torch.nn.Conv2d(fpn_dims[0], inter_dims[1], 1)\n self.adapter2 = torch.nn.Conv2d(fpn_dims[1], inter_dims[2], 1)\n self.adapter3 = torch.nn.Conv2d(fpn_dims[2], inter_dims[3], 1)\n\n for m in self.modules():\n if isinstance(m, nn.Conv2d):\n nn.init.kaiming_uniform_(m.weight, a=1)\n nn.init.constant_(m.bias, 0)\n\n def forward(self, x: Tensor, bbox_mask: Tensor, fpns: List[Tensor]):\n x = torch.cat([_expand(x, bbox_mask.shape[1]), bbox_mask.flatten(0, 1)], 1)\n\n x = self.lay1(x)\n x = self.gn1(x)\n x = F.relu(x)\n x = self.lay2(x)\n x = self.gn2(x)\n x = F.relu(x)\n\n cur_fpn = self.adapter1(fpns[0])\n if cur_fpn.size(0) != x.size(0):\n cur_fpn = _expand(cur_fpn, x.size(0) // cur_fpn.size(0))\n x = cur_fpn + F.interpolate(x, size=cur_fpn.shape[-2:], mode=\"nearest\")\n x = self.lay3(x)\n x = self.gn3(x)\n x = F.relu(x)\n\n cur_fpn = self.adapter2(fpns[1])\n if cur_fpn.size(0) != x.size(0):\n cur_fpn = _expand(cur_fpn, x.size(0) // cur_fpn.size(0))\n x = cur_fpn + F.interpolate(x, size=cur_fpn.shape[-2:], mode=\"nearest\")\n x = self.lay4(x)\n x = self.gn4(x)\n x = F.relu(x)\n\n cur_fpn = self.adapter3(fpns[2])\n if cur_fpn.size(0) != x.size(0):\n cur_fpn = _expand(cur_fpn, x.size(0) // cur_fpn.size(0))\n x = cur_fpn + F.interpolate(x, size=cur_fpn.shape[-2:], mode=\"nearest\")\n x = self.lay5(x)\n x = self.gn5(x)\n x = F.relu(x)\n\n x = self.out_lay(x)\n return x\n\n\nclass MHAttentionMap(nn.Module):\n \"\"\"This is a 2D attention module, which only returns the attention softmax (no multiplication by value)\"\"\"\n\n def __init__(self, query_dim, hidden_dim, num_heads, dropout=0.0, bias=True):\n super().__init__()\n self.num_heads = num_heads\n self.hidden_dim = hidden_dim\n self.dropout = nn.Dropout(dropout)\n\n self.q_linear = nn.Linear(query_dim, hidden_dim, bias=bias)\n self.k_linear = nn.Linear(query_dim, hidden_dim, bias=bias)\n\n nn.init.zeros_(self.k_linear.bias)\n nn.init.zeros_(self.q_linear.bias)\n nn.init.xavier_uniform_(self.k_linear.weight)\n nn.init.xavier_uniform_(self.q_linear.weight)\n self.normalize_fact = float(hidden_dim / self.num_heads) ** -0.5\n\n def forward(self, q, k, mask: Optional[Tensor] = None):\n q = self.q_linear(q)\n k = F.conv2d(k, self.k_linear.weight.unsqueeze(-1).unsqueeze(-1), self.k_linear.bias)\n qh = q.view(q.shape[0], q.shape[1], self.num_heads, self.hidden_dim // self.num_heads)\n kh = k.view(k.shape[0], self.num_heads, self.hidden_dim // self.num_heads, k.shape[-2], k.shape[-1])\n weights = torch.einsum(\"bqnc,bnchw->bqnhw\", qh * self.normalize_fact, kh)\n\n if mask is not None:\n weights.masked_fill_(mask.unsqueeze(1).unsqueeze(1), float(\"-inf\"))\n weights = F.softmax(weights.flatten(2), dim=-1).view(weights.size())\n weights = self.dropout(weights)\n return weights\n\n\ndef dice_loss(inputs, targets, num_boxes):\n \"\"\"\n Compute the DICE loss, similar to generalized IOU for masks\n Args:\n inputs: A float tensor of arbitrary shape.\n The predictions for each example.\n targets: A float tensor with the same shape as inputs. Stores the binary\n classification label for each element in inputs\n (0 for the negative class and 1 for the positive class).\n \"\"\"\n inputs = inputs.sigmoid()\n inputs = inputs.flatten(1)\n numerator = 2 * (inputs * targets).sum(1)\n denominator = inputs.sum(-1) + targets.sum(-1)\n loss = 1 - (numerator + 1) / (denominator + 1)\n return loss.sum() / num_boxes\n\n\ndef sigmoid_focal_loss(inputs, targets, num_boxes, alpha: float = 0.25, gamma: float = 2):\n \"\"\"\n Loss used in RetinaNet for dense detection: https://arxiv.org/abs/1708.02002.\n Args:\n inputs: A float tensor of arbitrary shape.\n The predictions for each example.\n targets: A float tensor with the same shape as inputs. Stores the binary\n classification label for each element in inputs\n (0 for the negative class and 1 for the positive class).\n alpha: (optional) Weighting factor in range (0,1) to balance\n positive vs negative examples. Default = -1 (no weighting).\n gamma: Exponent of the modulating factor (1 - p_t) to\n balance easy vs hard examples.\n Returns:\n Loss tensor\n \"\"\"\n prob = inputs.sigmoid()\n ce_loss = F.binary_cross_entropy_with_logits(inputs, targets, reduction=\"none\")\n p_t = prob * targets + (1 - prob) * (1 - targets)\n loss = ce_loss * ((1 - p_t) ** gamma)\n\n if alpha >= 0:\n alpha_t = alpha * targets + (1 - alpha) * (1 - targets)\n loss = alpha_t * loss\n\n return loss.mean(1).sum() / num_boxes\n\n\nclass PostProcessSegm(nn.Module):\n def __init__(self, threshold=0.5):\n super().__init__()\n self.threshold = threshold\n\n @torch.no_grad()\n def forward(self, results, outputs, orig_target_sizes, max_target_sizes):\n assert len(orig_target_sizes) == len(max_target_sizes)\n max_h, max_w = max_target_sizes.max(0)[0].tolist()\n outputs_masks = outputs[\"pred_masks\"].squeeze(2)\n outputs_masks = F.interpolate(outputs_masks, size=(max_h, max_w), mode=\"bilinear\", align_corners=False)\n outputs_masks = (outputs_masks.sigmoid() > self.threshold).cpu()\n\n for i, (cur_mask, t, tt) in enumerate(zip(outputs_masks, max_target_sizes, orig_target_sizes)):\n img_h, img_w = t[0], t[1]\n results[i][\"masks\"] = cur_mask[:, :img_h, :img_w].unsqueeze(1)\n results[i][\"masks\"] = F.interpolate(\n results[i][\"masks\"].float(), size=tuple(tt.tolist()), mode=\"nearest\"\n ).byte()\n\n return results\n\n\nclass PostProcessPanoptic(nn.Module):\n \"\"\"This class converts the output of the model to the final panoptic result, in the format expected by the\n coco panoptic API \"\"\"\n\n def __init__(self, is_thing_map, threshold=0.85):\n \"\"\"\n Parameters:\n is_thing_map: This is a whose keys are the class ids, and the values a boolean indicating whether\n the class is a thing (True) or a stuff (False) class\n threshold: confidence threshold: segments with confidence lower than this will be deleted\n \"\"\"\n super().__init__()\n self.threshold = threshold\n self.is_thing_map = is_thing_map\n\n def forward(self, outputs, processed_sizes, target_sizes=None):\n \"\"\" This function computes the panoptic prediction from the model's predictions.\n Parameters:\n outputs: This is a dict coming directly from the model. See the model doc for the content.\n processed_sizes: This is a list of tuples (or torch tensors) of sizes of the images that were passed to the\n model, ie the size after data augmentation but before batching.\n target_sizes: This is a list of tuples (or torch tensors) corresponding to the requested final size\n of each prediction. If left to None, it will default to the processed_sizes\n \"\"\"\n if target_sizes is None:\n target_sizes = processed_sizes\n assert len(processed_sizes) == len(target_sizes)\n out_logits, raw_masks, raw_boxes = outputs[\"pred_logits\"], outputs[\"pred_masks\"], outputs[\"pred_boxes\"]\n assert len(out_logits) == len(raw_masks) == len(target_sizes)\n preds = []\n\n def to_tuple(tup):\n if isinstance(tup, tuple):\n return tup\n return tuple(tup.cpu().tolist())\n\n for cur_logits, cur_masks, cur_boxes, size, target_size in zip(\n out_logits, raw_masks, raw_boxes, processed_sizes, target_sizes\n ):\n # we filter empty queries and detection below threshold\n scores, labels = cur_logits.softmax(-1).max(-1)\n keep = labels.ne(outputs[\"pred_logits\"].shape[-1] - 1) & (scores > self.threshold)\n cur_scores, cur_classes = cur_logits.softmax(-1).max(-1)\n cur_scores = cur_scores[keep]\n cur_classes = cur_classes[keep]\n cur_masks = cur_masks[keep]\n cur_masks = interpolate(cur_masks[:, None], to_tuple(size), mode=\"bilinear\").squeeze(1)\n cur_boxes = box_ops.box_cxcywh_to_xyxy(cur_boxes[keep])\n\n h, w = cur_masks.shape[-2:]\n assert len(cur_boxes) == len(cur_classes)\n\n # It may be that we have several predicted masks for the same stuff class.\n # In the following, we track the list of masks ids for each stuff class (they are merged later on)\n cur_masks = cur_masks.flatten(1)\n stuff_equiv_classes = defaultdict(lambda: [])\n for k, label in enumerate(cur_classes):\n if not self.is_thing_map[label.item()]:\n stuff_equiv_classes[label.item()].append(k)\n\n def get_ids_area(masks, scores, dedup=False):\n # This helper function creates the final panoptic segmentation image\n # It also returns the area of the masks that appears on the image\n\n m_id = masks.transpose(0, 1).softmax(-1)\n\n if m_id.shape[-1] == 0:\n # We didn't detect any mask :(\n m_id = torch.zeros((h, w), dtype=torch.long, device=m_id.device)\n else:\n m_id = m_id.argmax(-1).view(h, w)\n\n if dedup:\n # Merge the masks corresponding to the same stuff class\n for equiv in stuff_equiv_classes.values():\n if len(equiv) > 1:\n for eq_id in equiv:\n m_id.masked_fill_(m_id.eq(eq_id), equiv[0])\n\n final_h, final_w = to_tuple(target_size)\n\n seg_img = Image.fromarray(id2rgb(m_id.view(h, w).cpu().numpy()))\n seg_img = seg_img.resize(size=(final_w, final_h), resample=Image.NEAREST)\n\n np_seg_img = (\n torch.ByteTensor(torch.ByteStorage.from_buffer(seg_img.tobytes())).view(final_h, final_w, 3).numpy()\n )\n m_id = torch.from_numpy(rgb2id(np_seg_img))\n\n area = []\n for i in range(len(scores)):\n area.append(m_id.eq(i).sum().item())\n return area, seg_img\n\n area, seg_img = get_ids_area(cur_masks, cur_scores, dedup=True)\n if cur_classes.numel() > 0:\n # We know filter empty masks as long as we find some\n while True:\n filtered_small = torch.as_tensor(\n [area[i] <= 4 for i, c in enumerate(cur_classes)], dtype=torch.bool, device=keep.device\n )\n if filtered_small.any().item():\n cur_scores = cur_scores[~filtered_small]\n cur_classes = cur_classes[~filtered_small]\n cur_masks = cur_masks[~filtered_small]\n area, seg_img = get_ids_area(cur_masks, cur_scores)\n else:\n break\n\n else:\n cur_classes = torch.ones(1, dtype=torch.long, device=cur_classes.device)\n\n segments_info = []\n for i, a in enumerate(area):\n cat = cur_classes[i].item()\n segments_info.append({\"id\": i, \"isthing\": self.is_thing_map[cat], \"category_id\": cat, \"area\": a})\n del cur_classes\n\n with io.BytesIO() as out:\n seg_img.save(out, format=\"PNG\")\n predictions = {\"png_string\": out.getvalue(), \"segments_info\": segments_info}\n preds.append(predictions)\n return preds\n"
},
{
"path": "batchformer-v2/detr/models/transformer.py",
"content": "# Copyright (c) Facebook, Inc. and its affiliates. All Rights Reserved\n\"\"\"\nDETR Transformer class.\n\nCopy-paste from torch.nn.Transformer with modifications:\n * positional encodings are passed in MHattention\n * extra LN at the end of encoder is removed\n * decoder returns a stack of activations from all decoding layers\n\"\"\"\nimport copy\nfrom typing import Optional, List\n\nimport torch\nimport torch.nn.functional as F\nfrom torch import nn, Tensor\n\n\nclass Transformer(nn.Module):\n\n def __init__(self, d_model=512, nhead=8, num_encoder_layers=6,\n num_decoder_layers=6, dim_feedforward=2048, dropout=0.1,\n activation=\"relu\", normalize_before=False,\n return_intermediate_dec=False, args=None):\n super().__init__()\n\n encoder_layer = TransformerEncoderLayer(d_model, nhead, dim_feedforward,\n dropout, activation, normalize_before)\n encoder_norm = nn.LayerNorm(d_model) if normalize_before else None\n self.bf = None\n self.use_checkpoint=False\n self.base_bf = 0\n insert_idx = []\n if args is not None and args.bf:\n self.use_checkpoint = args.use_checkpoint\n self.base_bf = args.base_bf\n insert_idx = args.insert_idx\n self.bf = torch.nn.TransformerEncoderLayer(d_model, 4, d_model, dropout=0.5)\n if not args.share_bf:\n self.bf = torch.nn.ModuleList([torch.nn.TransformerEncoderLayer(d_model, 4, d_model, dropout=0.5) if i in insert_idx else torch.nn.Identity() for i in range(0, num_encoder_layers)])\n elif self.base_bf:\n self.bf = torch.nn.Identity()\n args.bf = 3\n self.encoder = TransformerEncoder(encoder_layer, num_encoder_layers, encoder_norm, bf=self.bf, bf_idx=args.bf, insert_idx=insert_idx,\n use_checkpoint=args.use_checkpoint)\n\n decoder_layer = TransformerDecoderLayer(d_model, nhead, dim_feedforward,\n dropout, activation, normalize_before)\n decoder_norm = nn.LayerNorm(d_model)\n self.decoder = TransformerDecoder(decoder_layer, num_decoder_layers, decoder_norm,\n return_intermediate=return_intermediate_dec, use_checkpoint=args.use_checkpoint)\n\n self._reset_parameters()\n\n self.d_model = d_model\n self.nhead = nhead\n\n def _reset_parameters(self):\n for p in self.parameters():\n if p.dim() > 1:\n nn.init.xavier_uniform_(p)\n\n def forward(self, src, mask, query_embed, pos_embed):\n # flatten NxCxHxW to HWxNxC\n bs, c, h, w = src.shape\n src = src.flatten(2).permute(2, 0, 1)\n pos_embed = pos_embed.flatten(2).permute(2, 0, 1)\n query_embed = query_embed.unsqueeze(1).repeat(1, bs, 1)\n mask = mask.flatten(1)\n\n tgt = torch.zeros_like(query_embed)\n memory = self.encoder(src, src_key_padding_mask=mask, pos=pos_embed)\n if memory.shape[1] > bs:\n tgt = torch.cat([tgt,tgt], dim=1)\n mask = torch.cat([mask, mask], dim=0)\n pos_embed = torch.cat([pos_embed, pos_embed], dim=1)\n query_embed = torch.cat([query_embed, query_embed], dim=1)\n bs = bs * 2\n if self.use_checkpoint:\n print('use checkpoint')\n from torch.utils import checkpoint\n # output = checkpoint.checkpoint(layer, output, mask, src_key_padding_mask, pos)\n hs = checkpoint.checkpoint(self.decoder, tgt, memory, None, None, None, mask,\n pos_embed, query_embed)\n # tgt, memory,\n # tgt_mask: Optional[Tensor] = None,\n # memory_mask: Optional[Tensor] = None,\n # tgt_key_padding_mask: Optional[Tensor] = None,\n # memory_key_padding_mask: Optional[Tensor] = None,\n # pos: Optional[Tensor] = None,\n # query_pos: Optional[Tensor] = None):\n else:\n hs = self.decoder(tgt, memory, memory_key_padding_mask=mask,\n pos=pos_embed, query_pos=query_embed)\n return hs.transpose(1, 2), memory.permute(1, 2, 0).view(bs, c, h, w)\n\n\nclass TransformerEncoder(nn.Module):\n\n def __init__(self, encoder_layer, num_layers, norm=None, bf=None, bf_idx =0, insert_idx=[], use_checkpoint=False):\n super().__init__()\n self.layers = _get_clones(encoder_layer, num_layers)\n self.num_layers = num_layers\n self.norm = norm\n if type(bf) != torch.nn.ModuleList and bf is not None:\n self.bf = [bf]*num_layers\n else:\n self.bf = bf\n self.insert_idx = insert_idx\n self.bf_idx = bf_idx\n self.use_checkpoint = use_checkpoint\n\n def forward(self, src,\n mask: Optional[Tensor] = None,\n src_key_padding_mask: Optional[Tensor] = None,\n pos: Optional[Tensor] = None):\n output = src\n\n for i, layer in enumerate(self.layers):\n output = layer(output, src_mask=mask,\n src_key_padding_mask=src_key_padding_mask, pos=pos)\n if i in self.insert_idx and self.bf is not None and self.bf_idx == 3 and self.training:\n\n L, B, C = output.shape\n old_output = output\n if i != self.insert_idx[0]:\n old_output = output[:, :B//2, :]\n output = output[:, B//2:, :]\n # old_output = output[:, :len(output)//2, :]\n # output = output[:, len(output)//2:, :]\n # the original batches\n output = self.bf[i](torch.transpose(output, 1, 0))\n output = torch.transpose(output, 1, 0)\n output = torch.cat([old_output, output], dim=1)\n if i == self.insert_idx[0]:\n pos = torch.cat([pos, pos], dim=1)\n src_key_padding_mask = torch.cat([src_key_padding_mask, src_key_padding_mask], dim=0)\n elif i == 4 and self.bf is not None and self.bf_idx == 2 and self.training:\n old_output = output\n output = self.bf[i](torch.transpose(output, 1, 0))\n output = torch.transpose(output, 1, 0)\n output = torch.cat([old_output, output], dim=1)\n pos = torch.cat([pos, pos], dim=1)\n src_key_padding_mask = torch.cat([src_key_padding_mask, src_key_padding_mask], dim=0)\n\n\n if self.norm is not None:\n output = self.norm(output)\n if self.bf is not None and self.bf_idx == 1 and self.training:\n # print(output.shape, self.norm)\n # import ipdb;ipdb.set_trace()\n old_output = output\n output = self.bf[i](torch.transpose(output, 1, 0))\n output = torch.transpose(output, 1, 0)\n output = torch.cat([old_output, output], dim=1)\n pos = torch.cat([pos, pos], dim=1)\n src_key_padding_mask = torch.cat([src_key_padding_mask, src_key_padding_mask], dim=0)\n return output\n\n\nclass TransformerDecoder(nn.Module):\n\n def __init__(self, decoder_layer, num_layers, norm=None, return_intermediate=False, use_checkpoint=False):\n super().__init__()\n self.layers = _get_clones(decoder_layer, num_layers)\n self.num_layers = num_layers\n self.norm = norm\n self.return_intermediate = return_intermediate\n self.use_checkpoint = use_checkpoint\n\n def forward(self, tgt, memory,\n tgt_mask: Optional[Tensor] = None,\n memory_mask: Optional[Tensor] = None,\n tgt_key_padding_mask: Optional[Tensor] = None,\n memory_key_padding_mask: Optional[Tensor] = None,\n pos: Optional[Tensor] = None,\n query_pos: Optional[Tensor] = None):\n output = tgt\n\n intermediate = []\n\n for layer in self.layers:\n if self.use_checkpoint:\n from torch.utils import checkpoint\n output = checkpoint.checkpoint(layer, output, memory, tgt_mask,\n memory_mask,\n tgt_key_padding_mask,\n memory_key_padding_mask,\n pos, query_pos)\n else:\n output = layer(output, memory, tgt_mask=tgt_mask,\n memory_mask=memory_mask,\n tgt_key_padding_mask=tgt_key_padding_mask,\n memory_key_padding_mask=memory_key_padding_mask,\n pos=pos, query_pos=query_pos)\n if self.return_intermediate:\n intermediate.append(self.norm(output))\n\n if self.norm is not None:\n output = self.norm(output)\n if self.return_intermediate:\n intermediate.pop()\n intermediate.append(output)\n\n if self.return_intermediate:\n return torch.stack(intermediate)\n\n return output.unsqueeze(0)\n\n\nclass TransformerEncoderLayer(nn.Module):\n\n def __init__(self, d_model, nhead, dim_feedforward=2048, dropout=0.1,\n activation=\"relu\", normalize_before=False):\n super().__init__()\n self.self_attn = nn.MultiheadAttention(d_model, nhead, dropout=dropout)\n # Implementation of Feedforward model\n self.linear1 = nn.Linear(d_model, dim_feedforward)\n self.dropout = nn.Dropout(dropout)\n self.linear2 = nn.Linear(dim_feedforward, d_model)\n\n self.norm1 = nn.LayerNorm(d_model)\n self.norm2 = nn.LayerNorm(d_model)\n self.dropout1 = nn.Dropout(dropout)\n self.dropout2 = nn.Dropout(dropout)\n\n self.activation = _get_activation_fn(activation)\n self.normalize_before = normalize_before\n\n def with_pos_embed(self, tensor, pos: Optional[Tensor]):\n return tensor if pos is None else tensor + pos\n\n def forward_post(self,\n src,\n src_mask: Optional[Tensor] = None,\n src_key_padding_mask: Optional[Tensor] = None,\n pos: Optional[Tensor] = None):\n q = k = self.with_pos_embed(src, pos)\n src2 = self.self_attn(q, k, value=src, attn_mask=src_mask,\n key_padding_mask=src_key_padding_mask)[0]\n src = src + self.dropout1(src2)\n src = self.norm1(src)\n src2 = self.linear2(self.dropout(self.activation(self.linear1(src))))\n src = src + self.dropout2(src2)\n src = self.norm2(src)\n return src\n\n def forward_pre(self, src,\n src_mask: Optional[Tensor] = None,\n src_key_padding_mask: Optional[Tensor] = None,\n pos: Optional[Tensor] = None):\n src2 = self.norm1(src)\n q = k = self.with_pos_embed(src2, pos)\n src2 = self.self_attn(q, k, value=src2, attn_mask=src_mask,\n key_padding_mask=src_key_padding_mask)[0]\n src = src + self.dropout1(src2)\n src2 = self.norm2(src)\n src2 = self.linear2(self.dropout(self.activation(self.linear1(src2))))\n src = src + self.dropout2(src2)\n return src\n\n def forward(self, src,\n src_mask: Optional[Tensor] = None,\n src_key_padding_mask: Optional[Tensor] = None,\n pos: Optional[Tensor] = None):\n if self.normalize_before:\n return self.forward_pre(src, src_mask, src_key_padding_mask, pos)\n return self.forward_post(src, src_mask, src_key_padding_mask, pos)\n\n\nclass TransformerDecoderLayer(nn.Module):\n\n def __init__(self, d_model, nhead, dim_feedforward=2048, dropout=0.1,\n activation=\"relu\", normalize_before=False):\n super().__init__()\n self.self_attn = nn.MultiheadAttention(d_model, nhead, dropout=dropout)\n self.multihead_attn = nn.MultiheadAttention(d_model, nhead, dropout=dropout)\n # Implementation of Feedforward model\n self.linear1 = nn.Linear(d_model, dim_feedforward)\n self.dropout = nn.Dropout(dropout)\n self.linear2 = nn.Linear(dim_feedforward, d_model)\n\n self.norm1 = nn.LayerNorm(d_model)\n self.norm2 = nn.LayerNorm(d_model)\n self.norm3 = nn.LayerNorm(d_model)\n self.dropout1 = nn.Dropout(dropout)\n self.dropout2 = nn.Dropout(dropout)\n self.dropout3 = nn.Dropout(dropout)\n\n self.activation = _get_activation_fn(activation)\n self.normalize_before = normalize_before\n\n def with_pos_embed(self, tensor, pos: Optional[Tensor]):\n return tensor if pos is None else tensor + pos\n\n def forward_post(self, tgt, memory,\n tgt_mask: Optional[Tensor] = None,\n memory_mask: Optional[Tensor] = None,\n tgt_key_padding_mask: Optional[Tensor] = None,\n memory_key_padding_mask: Optional[Tensor] = None,\n pos: Optional[Tensor] = None,\n query_pos: Optional[Tensor] = None):\n q = k = self.with_pos_embed(tgt, query_pos)\n tgt2 = self.self_attn(q, k, value=tgt, attn_mask=tgt_mask,\n key_padding_mask=tgt_key_padding_mask)[0]\n tgt = tgt + self.dropout1(tgt2)\n tgt = self.norm1(tgt)\n tgt2 = self.multihead_attn(query=self.with_pos_embed(tgt, query_pos),\n key=self.with_pos_embed(memory, pos),\n value=memory, attn_mask=memory_mask,\n key_padding_mask=memory_key_padding_mask)[0]\n tgt = tgt + self.dropout2(tgt2)\n tgt = self.norm2(tgt)\n tgt2 = self.linear2(self.dropout(self.activation(self.linear1(tgt))))\n tgt = tgt + self.dropout3(tgt2)\n tgt = self.norm3(tgt)\n return tgt\n\n def forward_pre(self, tgt, memory,\n tgt_mask: Optional[Tensor] = None,\n memory_mask: Optional[Tensor] = None,\n tgt_key_padding_mask: Optional[Tensor] = None,\n memory_key_padding_mask: Optional[Tensor] = None,\n pos: Optional[Tensor] = None,\n query_pos: Optional[Tensor] = None):\n tgt2 = self.norm1(tgt)\n q = k = self.with_pos_embed(tgt2, query_pos)\n tgt2 = self.self_attn(q, k, value=tgt2, attn_mask=tgt_mask,\n key_padding_mask=tgt_key_padding_mask)[0]\n tgt = tgt + self.dropout1(tgt2)\n tgt2 = self.norm2(tgt)\n tgt2 = self.multihead_attn(query=self.with_pos_embed(tgt2, query_pos),\n key=self.with_pos_embed(memory, pos),\n value=memory, attn_mask=memory_mask,\n key_padding_mask=memory_key_padding_mask)[0]\n tgt = tgt + self.dropout2(tgt2)\n tgt2 = self.norm3(tgt)\n tgt2 = self.linear2(self.dropout(self.activation(self.linear1(tgt2))))\n tgt = tgt + self.dropout3(tgt2)\n return tgt\n\n def forward(self, tgt, memory,\n tgt_mask: Optional[Tensor] = None,\n memory_mask: Optional[Tensor] = None,\n tgt_key_padding_mask: Optional[Tensor] = None,\n memory_key_padding_mask: Optional[Tensor] = None,\n pos: Optional[Tensor] = None,\n query_pos: Optional[Tensor] = None):\n if self.normalize_before:\n return self.forward_pre(tgt, memory, tgt_mask, memory_mask,\n tgt_key_padding_mask, memory_key_padding_mask, pos, query_pos)\n return self.forward_post(tgt, memory, tgt_mask, memory_mask,\n tgt_key_padding_mask, memory_key_padding_mask, pos, query_pos)\n\n\ndef _get_clones(module, N):\n return nn.ModuleList([copy.deepcopy(module) for i in range(N)])\n\n\ndef build_transformer(args):\n return Transformer(\n d_model=args.hidden_dim,\n dropout=args.dropout,\n nhead=args.nheads,\n dim_feedforward=args.dim_feedforward,\n num_encoder_layers=args.enc_layers,\n num_decoder_layers=args.dec_layers,\n normalize_before=args.pre_norm,\n return_intermediate_dec=True,\n args=args\n )\n\n\ndef _get_activation_fn(activation):\n \"\"\"Return an activation function given a string\"\"\"\n if activation == \"relu\":\n return F.relu\n if activation == \"gelu\":\n return F.gelu\n if activation == \"glu\":\n return F.glu\n raise RuntimeError(F\"activation should be relu/gelu, not {activation}.\")\n"
},
{
"path": "batchformer-v2/detr/requirements.txt",
"content": "cython\ngit+https://github.com/cocodataset/cocoapi.git#subdirectory=PythonAPI&egg=pycocotools\nsubmitit\ntorch>=1.5.0\ntorchvision>=0.6.0\ngit+https://github.com/cocodataset/panopticapi.git#egg=panopticapi\nscipy\nonnx\nonnxruntime\n"
},
{
"path": "batchformer-v2/detr/run.py",
"content": "#!/usr/local/anaconda/4.2.0/bin/python\n\n##################################\n'''\nCode is based on hpc_check_py36_withquotawarning_pbs.py\nYou can put this file into ~/bin/ and rename filename to ssub (without suffix .py).\nMeanwhile chmod a+x ~/bin/ssub\nThen, you can submit job by ssub xxx.pbs\nThis script will search suitable node to submit job with host=xxxx\n'''\n#################################\n\n\nimport subprocess\nfrom argparse import ArgumentParser\n\nimport numpy as np\nimport re\n\nimport time\nimport json\nimport os\n\nMEM_PER_NODE = 187\nCPU_PER_NODE = 36\nGPU_PER_NODE = 4\n\n# from user_quota_config import *\nHPC_NODE_INFO_ALLOC_DT = [ 'hpc223', 'hpc224', 'hpc225', 'hpc226', 'hpc227', 'hpc228', 'hpc229', 'hpc230', 'hpc231',\n 'hpc232',\n 'hpc233', 'hpc234', 'hpc235', 'hpc236', 'hpc237', 'hpc238', 'hpc239', 'hpc246', 'hpc247',\n 'hpc248']\nHPC_NODE_INFO_ALLOC_DT = ['hpc'+str(i) for i in range(216, 223, 1)] + HPC_NODE_INFO_ALLOC_DT\n\nprint(HPC_NODE_INFO_ALLOC_DT )\nHOST_IP = {}\nfor host in HPC_NODE_INFO_ALLOC_DT:\n ip = host.replace('hpc', '')\n ip = '192.168.66.'+ip\n HOST_IP[host+'.hpc.sydney.edu.au'] = ip\nprint(HOST_IP)\n\ndef parse_args():\n \"\"\"\n Helper function parsing the command line options\n @retval ArgumentParser\n \"\"\"\n parser = ArgumentParser(description=\"PyTorch distributed training launch \"\n \"helper utilty that will spawn up \"\n \"multiple distributed processes\")\n\n # Optional arguments for the launch helper\n parser.add_argument(\"--nnodes\", type=int, default=1,\n help=\"The number of nodes to use for distributed \"\n \"training\")\n parser.add_argument(\"--nproc_per_node\", type=int, default=1,\n help=\"The number of processes to launch on each node, \"\n \"for GPU training, this is recommended to be set \"\n \"to the number of GPUs in your system so that \"\n \"each process can be bound to a single GPU.\")\n parser.add_argument(\"--master_addr\", default=\"127.0.0.1\", type=str,\n help=\"Master node (rank 0)'s address, should be either \"\n \"the IP address or the hostname of node 0, for \"\n \"single node multi-proc training, the \"\n \"--master_addr can simply be 127.0.0.1\")\n parser.add_argument(\"--master_port\", default=29500, type=int,\n help=\"Master node (rank 0)'s free port that needs to \"\n \"be used for communciation during distributed \"\n \"training\")\n\n # positional\n parser.add_argument(\"training_script\", type=str, default='./configs/r50_deformable_detr.sh',\n help=\"The full path to the single GPU training \"\n \"program/script to be launched in parallel, \"\n \"followed by all the arguments for the \"\n \"training script\")\n\n # rest from the training program\n # parser.add_argument('training_script_args', nargs=REMAINDER)\n return parser.parse_args()\n\n\n\nif __name__ == \"__main__\":\n import sys\n import os\n\n args = parse_args()\n os.system('chmod a+x {}'.format(args.training_script))\n\n fpbs = open(os.environ['PBS_NODEFILE'])\n hosts = [item.strip() for item in fpbs.readlines()]\n uniq_hosts = set(hosts)\n host_gpus = {k:0 for k in uniq_hosts}\n host_rank = {k:0 for k in uniq_hosts}\n current_gpus = {}\n for host in hosts:\n host_gpus[host] += args.nproc_per_node\n # host_rank[host] += sum()\n current_gpus[host] = 0\n print(uniq_hosts, hosts)\n uniq_hosts = hosts\n pbs_jobid = os.environ['PBS_JOBID']\n pbs_jobid = pbs_jobid.split('.')[0]\n\n current_env = os.environ.copy()\n current_env[\"MASTER_ADDR\"] = HOST_IP[hosts[0]]\n current_env[\"MASTER_PORT\"] = str(args.master_port)\n current_env[\"WORLD_SIZE\"] = str(args.nnodes*args.nproc_per_node)\n current_env[\"GPUS_PER_NODE\"] = str(args.nproc_per_node)\n current_env[\"NNODES\"] = str(len(uniq_hosts))\n processes = []\n\n alloc_procs = 0\n tmp_i = 0\n for ni in range(len(uniq_hosts)-1, -1, -1):\n # each process's rank\n current_env[\"GPUS_PER_NODE\"] = str(args.nproc_per_node)\n command_remote = 'NNODES={} MASTER_ADDR={} MASTER_PORT={} NODE_RANK={} GPUS_PER_NODE={} ./tools/run_dist_launch.sh {} {}'.format(\n args.nnodes, HOST_IP[hosts[0]], str(args.master_port), ni, args.nproc_per_node, str(args.nnodes*args.nproc_per_node), args.training_script)\n current_env[\"NODE_RANK\"] = str(ni)\n command = './tools/run_dist_launch.sh {} {}'.format(str(args.nnodes*args.nproc_per_node), args.training_script)\n\n if ni == 0:\n print('cur', ni, command)\n # print(host_gpus)\n # print(current_env[\"GPUS_PER_NODE\"], args.nproc_per_node, 'test')\n process = subprocess.Popen(command.split(' '), env=current_env)\n # processes.append(process)\n else:\n # print('ssh', ni)\n cmd = 'ssh {} cd {} & {}'.format(hosts[ni], os.path.abspath('.'), command)\n cmd = \"ssh {} 'cd {} ; {}'\".format(hosts[ni], os.path.abspath('.'), command_remote)\n # print(cmd)\n # os.system(cmd)\n # continue\n\n if host_gpus[hosts[ni]] == 4 and args.nproc_per_node == 2:\n tmp_i += 1\n current_gpus[hosts[ni]] += args.nproc_per_node\n if current_gpus[hosts[ni]] == 4:\n arr = ['ssh', hosts[ni], 'cd {} ;CUDA_VISIBLE_DEVICES=2,3 {} '.format(os.path.abspath('.'), command_remote)]\n else:\n arr = ['ssh', hosts[ni], 'cd {} ;CUDA_VISIBLE_DEVICES=0,1 {} '.format(os.path.abspath('.'), command_remote)]\n elif host_gpus[hosts[ni]] > args.nproc_per_node:\n arr = ['ssh', hosts[ni],\n 'cd {} ;CUDA_VISIBLE_DEVICES={} {} '.format(os.path.abspath('.'), current_gpus[hosts[ni]], command_remote)]\n\n current_gpus[hosts[ni]] += args.nproc_per_node\n else:\n arr = ['ssh', hosts[ni], 'cd {} ; {} '.format(os.path.abspath('.'), command_remote)]\n print(' '.join(arr))\n process = subprocess.Popen(arr,\n env=current_env)\n processes.append(process)\n time.sleep(3)\n\n for process in processes:\n process.wait()\n if process.returncode != 0:\n raise subprocess.CalledProcessError(returncode=process.returncode,\n cmd=process.args)\n"
},
{
"path": "batchformer-v2/detr/run_with_submitit.py",
"content": "# Copyright (c) Facebook, Inc. and its affiliates. All Rights Reserved\n\"\"\"\nA script to run multinode training with submitit.\n\"\"\"\nimport argparse\nimport os\nimport uuid\nfrom pathlib import Path\n\nimport main as detection\nimport submitit\n\n\ndef parse_args():\n detection_parser = detection.get_args_parser()\n parser = argparse.ArgumentParser(\"Submitit for detection\", parents=[detection_parser])\n parser.add_argument(\"--ngpus\", default=8, type=int, help=\"Number of gpus to request on each node\")\n parser.add_argument(\"--nodes\", default=4, type=int, help=\"Number of nodes to request\")\n parser.add_argument(\"--timeout\", default=60, type=int, help=\"Duration of the job\")\n parser.add_argument(\"--job_dir\", default=\"\", type=str, help=\"Job dir. Leave empty for automatic.\")\n return parser.parse_args()\n\n\ndef get_shared_folder() -> Path:\n user = os.getenv(\"USER\")\n if Path(\"/checkpoint/\").is_dir():\n p = Path(f\"/checkpoint/{user}/experiments\")\n p.mkdir(exist_ok=True)\n return p\n raise RuntimeError(\"No shared folder available\")\n\n\ndef get_init_file():\n # Init file must not exist, but it's parent dir must exist.\n os.makedirs(str(get_shared_folder()), exist_ok=True)\n init_file = get_shared_folder() / f\"{uuid.uuid4().hex}_init\"\n if init_file.exists():\n os.remove(str(init_file))\n return init_file\n\n\nclass Trainer(object):\n def __init__(self, args):\n self.args = args\n\n def __call__(self):\n import main as detection\n\n self._setup_gpu_args()\n detection.main(self.args)\n\n def checkpoint(self):\n import os\n import submitit\n from pathlib import Path\n\n self.args.dist_url = get_init_file().as_uri()\n checkpoint_file = os.path.join(self.args.output_dir, \"checkpoint.pth\")\n if os.path.exists(checkpoint_file):\n self.args.resume = checkpoint_file\n print(\"Requeuing \", self.args)\n empty_trainer = type(self)(self.args)\n return submitit.helpers.DelayedSubmission(empty_trainer)\n\n def _setup_gpu_args(self):\n import submitit\n from pathlib import Path\n\n job_env = submitit.JobEnvironment()\n self.args.output_dir = Path(str(self.args.output_dir).replace(\"%j\", str(job_env.job_id)))\n self.args.gpu = job_env.local_rank\n self.args.rank = job_env.global_rank\n self.args.world_size = job_env.num_tasks\n print(f\"Process group: {job_env.num_tasks} tasks, rank: {job_env.global_rank}\")\n\n\ndef main():\n args = parse_args()\n if args.job_dir == \"\":\n args.job_dir = get_shared_folder() / \"%j\"\n\n # Note that the folder will depend on the job_id, to easily track experiments\n executor = submitit.AutoExecutor(folder=args.job_dir, slurm_max_num_timeout=30)\n\n # cluster setup is defined by environment variables\n num_gpus_per_node = args.ngpus\n nodes = args.nodes\n timeout_min = args.timeout\n\n executor.update_parameters(\n mem_gb=40 * num_gpus_per_node,\n gpus_per_node=num_gpus_per_node,\n tasks_per_node=num_gpus_per_node, # one task per GPU\n cpus_per_task=10,\n nodes=nodes,\n timeout_min=timeout_min, # max is 60 * 72\n )\n\n executor.update_parameters(name=\"detr\")\n\n args.dist_url = get_init_file().as_uri()\n args.output_dir = args.job_dir\n\n trainer = Trainer(args)\n job = executor.submit(trainer)\n\n print(\"Submitted job_id:\", job.job_id)\n\n\nif __name__ == \"__main__\":\n main()\n"
},
{
"path": "batchformer-v2/detr/test_all.py",
"content": "# Copyright (c) Facebook, Inc. and its affiliates. All Rights Reserved\nimport io\nimport unittest\n\nimport torch\nfrom torch import nn, Tensor\nfrom typing import List\n\nfrom models.matcher import HungarianMatcher\nfrom models.position_encoding import PositionEmbeddingSine, PositionEmbeddingLearned\nfrom models.backbone import Backbone, Joiner, BackboneBase\nfrom util import box_ops\nfrom util.misc import nested_tensor_from_tensor_list\nfrom hubconf import detr_resnet50, detr_resnet50_panoptic\n\n# onnxruntime requires python 3.5 or above\ntry:\n import onnxruntime\nexcept ImportError:\n onnxruntime = None\n\n\nclass Tester(unittest.TestCase):\n\n def test_box_cxcywh_to_xyxy(self):\n t = torch.rand(10, 4)\n r = box_ops.box_xyxy_to_cxcywh(box_ops.box_cxcywh_to_xyxy(t))\n self.assertLess((t - r).abs().max(), 1e-5)\n\n @staticmethod\n def indices_torch2python(indices):\n return [(i.tolist(), j.tolist()) for i, j in indices]\n\n def test_hungarian(self):\n n_queries, n_targets, n_classes = 100, 15, 91\n logits = torch.rand(1, n_queries, n_classes + 1)\n boxes = torch.rand(1, n_queries, 4)\n tgt_labels = torch.randint(high=n_classes, size=(n_targets,))\n tgt_boxes = torch.rand(n_targets, 4)\n matcher = HungarianMatcher()\n targets = [{'labels': tgt_labels, 'boxes': tgt_boxes}]\n indices_single = matcher({'pred_logits': logits, 'pred_boxes': boxes}, targets)\n indices_batched = matcher({'pred_logits': logits.repeat(2, 1, 1),\n 'pred_boxes': boxes.repeat(2, 1, 1)}, targets * 2)\n self.assertEqual(len(indices_single[0][0]), n_targets)\n self.assertEqual(len(indices_single[0][1]), n_targets)\n self.assertEqual(self.indices_torch2python(indices_single),\n self.indices_torch2python([indices_batched[0]]))\n self.assertEqual(self.indices_torch2python(indices_single),\n self.indices_torch2python([indices_batched[1]]))\n\n # test with empty targets\n tgt_labels_empty = torch.randint(high=n_classes, size=(0,))\n tgt_boxes_empty = torch.rand(0, 4)\n targets_empty = [{'labels': tgt_labels_empty, 'boxes': tgt_boxes_empty}]\n indices = matcher({'pred_logits': logits.repeat(2, 1, 1),\n 'pred_boxes': boxes.repeat(2, 1, 1)}, targets + targets_empty)\n self.assertEqual(len(indices[1][0]), 0)\n indices = matcher({'pred_logits': logits.repeat(2, 1, 1),\n 'pred_boxes': boxes.repeat(2, 1, 1)}, targets_empty * 2)\n self.assertEqual(len(indices[0][0]), 0)\n\n def test_position_encoding_script(self):\n m1, m2 = PositionEmbeddingSine(), PositionEmbeddingLearned()\n mm1, mm2 = torch.jit.script(m1), torch.jit.script(m2) # noqa\n\n def test_backbone_script(self):\n backbone = Backbone('resnet50', True, False, False)\n torch.jit.script(backbone) # noqa\n\n def test_model_script_detection(self):\n model = detr_resnet50(pretrained=False).eval()\n scripted_model = torch.jit.script(model)\n x = nested_tensor_from_tensor_list([torch.rand(3, 200, 200), torch.rand(3, 200, 250)])\n out = model(x)\n out_script = scripted_model(x)\n self.assertTrue(out[\"pred_logits\"].equal(out_script[\"pred_logits\"]))\n self.assertTrue(out[\"pred_boxes\"].equal(out_script[\"pred_boxes\"]))\n\n def test_model_script_panoptic(self):\n model = detr_resnet50_panoptic(pretrained=False).eval()\n scripted_model = torch.jit.script(model)\n x = nested_tensor_from_tensor_list([torch.rand(3, 200, 200), torch.rand(3, 200, 250)])\n out = model(x)\n out_script = scripted_model(x)\n self.assertTrue(out[\"pred_logits\"].equal(out_script[\"pred_logits\"]))\n self.assertTrue(out[\"pred_boxes\"].equal(out_script[\"pred_boxes\"]))\n self.assertTrue(out[\"pred_masks\"].equal(out_script[\"pred_masks\"]))\n\n def test_model_detection_different_inputs(self):\n model = detr_resnet50(pretrained=False).eval()\n # support NestedTensor\n x = nested_tensor_from_tensor_list([torch.rand(3, 200, 200), torch.rand(3, 200, 250)])\n out = model(x)\n self.assertIn('pred_logits', out)\n # and 4d Tensor\n x = torch.rand(1, 3, 200, 200)\n out = model(x)\n self.assertIn('pred_logits', out)\n # and List[Tensor[C, H, W]]\n x = torch.rand(3, 200, 200)\n out = model([x])\n self.assertIn('pred_logits', out)\n\n def test_warpped_model_script_detection(self):\n class WrappedDETR(nn.Module):\n def __init__(self, model):\n super().__init__()\n self.model = model\n\n def forward(self, inputs: List[Tensor]):\n sample = nested_tensor_from_tensor_list(inputs)\n return self.model(sample)\n\n model = detr_resnet50(pretrained=False)\n wrapped_model = WrappedDETR(model)\n wrapped_model.eval()\n scripted_model = torch.jit.script(wrapped_model)\n x = [torch.rand(3, 200, 200), torch.rand(3, 200, 250)]\n out = wrapped_model(x)\n out_script = scripted_model(x)\n self.assertTrue(out[\"pred_logits\"].equal(out_script[\"pred_logits\"]))\n self.assertTrue(out[\"pred_boxes\"].equal(out_script[\"pred_boxes\"]))\n\n\n@unittest.skipIf(onnxruntime is None, 'ONNX Runtime unavailable')\nclass ONNXExporterTester(unittest.TestCase):\n @classmethod\n def setUpClass(cls):\n torch.manual_seed(123)\n\n def run_model(self, model, inputs_list, tolerate_small_mismatch=False, do_constant_folding=True, dynamic_axes=None,\n output_names=None, input_names=None):\n model.eval()\n\n onnx_io = io.BytesIO()\n # export to onnx with the first input\n torch.onnx.export(model, inputs_list[0], onnx_io,\n do_constant_folding=do_constant_folding, opset_version=12,\n dynamic_axes=dynamic_axes, input_names=input_names, output_names=output_names)\n # validate the exported model with onnx runtime\n for test_inputs in inputs_list:\n with torch.no_grad():\n if isinstance(test_inputs, torch.Tensor) or isinstance(test_inputs, list):\n test_inputs = (nested_tensor_from_tensor_list(test_inputs),)\n test_ouputs = model(*test_inputs)\n if isinstance(test_ouputs, torch.Tensor):\n test_ouputs = (test_ouputs,)\n self.ort_validate(onnx_io, test_inputs, test_ouputs, tolerate_small_mismatch)\n\n def ort_validate(self, onnx_io, inputs, outputs, tolerate_small_mismatch=False):\n\n inputs, _ = torch.jit._flatten(inputs)\n outputs, _ = torch.jit._flatten(outputs)\n\n def to_numpy(tensor):\n if tensor.requires_grad:\n return tensor.detach().cpu().numpy()\n else:\n return tensor.cpu().numpy()\n\n inputs = list(map(to_numpy, inputs))\n outputs = list(map(to_numpy, outputs))\n\n ort_session = onnxruntime.InferenceSession(onnx_io.getvalue())\n # compute onnxruntime output prediction\n ort_inputs = dict((ort_session.get_inputs()[i].name, inpt) for i, inpt in enumerate(inputs))\n ort_outs = ort_session.run(None, ort_inputs)\n for i in range(0, len(outputs)):\n try:\n torch.testing.assert_allclose(outputs[i], ort_outs[i], rtol=1e-03, atol=1e-05)\n except AssertionError as error:\n if tolerate_small_mismatch:\n self.assertIn(\"(0.00%)\", str(error), str(error))\n else:\n raise\n\n def test_model_onnx_detection(self):\n model = detr_resnet50(pretrained=False).eval()\n dummy_image = torch.ones(1, 3, 800, 800) * 0.3\n model(dummy_image)\n\n # Test exported model on images of different size, or dummy input\n self.run_model(\n model,\n [(torch.rand(1, 3, 750, 800),)],\n input_names=[\"inputs\"],\n output_names=[\"pred_logits\", \"pred_boxes\"],\n tolerate_small_mismatch=True,\n )\n\n @unittest.skip(\"CI doesn't have enough memory\")\n def test_model_onnx_detection_panoptic(self):\n model = detr_resnet50_panoptic(pretrained=False).eval()\n dummy_image = torch.ones(1, 3, 800, 800) * 0.3\n model(dummy_image)\n\n # Test exported model on images of different size, or dummy input\n self.run_model(\n model,\n [(torch.rand(1, 3, 750, 800),)],\n input_names=[\"inputs\"],\n output_names=[\"pred_logits\", \"pred_boxes\", \"pred_masks\"],\n tolerate_small_mismatch=True,\n )\n\n\nif __name__ == '__main__':\n unittest.main()\n"
},
{
"path": "batchformer-v2/detr/tools/launch.py",
"content": "# --------------------------------------------------------------------------------------------------------------------------\n# Deformable DETR\n# Copyright (c) 2020 SenseTime. All Rights Reserved.\n# Licensed under the Apache License, Version 2.0 [see LICENSE for details]\n# --------------------------------------------------------------------------------------------------------------------------\n# Modified from https://github.com/pytorch/pytorch/blob/173f224570017b4b1a3a1a13d0bff280a54d9cd9/torch/distributed/launch.py\n# --------------------------------------------------------------------------------------------------------------------------\n\nr\"\"\"\n`torch.distributed.launch` is a module that spawns up multiple distributed\ntraining processes on each of the training nodes.\nThe utility can be used for single-node distributed training, in which one or\nmore processes per node will be spawned. The utility can be used for either\nCPU training or GPU training. If the utility is used for GPU training,\neach distributed process will be operating on a single GPU. This can achieve\nwell-improved single-node training performance. It can also be used in\nmulti-node distributed training, by spawning up multiple processes on each node\nfor well-improved multi-node distributed training performance as well.\nThis will especially be benefitial for systems with multiple Infiniband\ninterfaces that have direct-GPU support, since all of them can be utilized for\naggregated communication bandwidth.\nIn both cases of single-node distributed training or multi-node distributed\ntraining, this utility will launch the given number of processes per node\n(``--nproc_per_node``). If used for GPU training, this number needs to be less\nor euqal to the number of GPUs on the current system (``nproc_per_node``),\nand each process will be operating on a single GPU from *GPU 0 to\nGPU (nproc_per_node - 1)*.\n**How to use this module:**\n1. Single-Node multi-process distributed training\n::\n >>> python -m torch.distributed.launch --nproc_per_node=NUM_GPUS_YOU_HAVE\n YOUR_TRAINING_SCRIPT.py (--arg1 --arg2 --arg3 and all other\n arguments of your training script)\n2. Multi-Node multi-process distributed training: (e.g. two nodes)\nNode 1: *(IP: 192.168.1.1, and has a free port: 1234)*\n::\n >>> python -m torch.distributed.launch --nproc_per_node=NUM_GPUS_YOU_HAVE\n --nnodes=2 --node_rank=0 --master_addr=\"192.168.1.1\"\n --master_port=1234 YOUR_TRAINING_SCRIPT.py (--arg1 --arg2 --arg3\n and all other arguments of your training script)\nNode 2:\n::\n >>> python -m torch.distributed.launch --nproc_per_node=NUM_GPUS_YOU_HAVE\n --nnodes=2 --node_rank=1 --master_addr=\"192.168.1.1\"\n --master_port=1234 YOUR_TRAINING_SCRIPT.py (--arg1 --arg2 --arg3\n and all other arguments of your training script)\n3. To look up what optional arguments this module offers:\n::\n >>> python -m torch.distributed.launch --help\n**Important Notices:**\n1. This utilty and multi-process distributed (single-node or\nmulti-node) GPU training currently only achieves the best performance using\nthe NCCL distributed backend. Thus NCCL backend is the recommended backend to\nuse for GPU training.\n2. In your training program, you must parse the command-line argument:\n``--local_rank=LOCAL_PROCESS_RANK``, which will be provided by this module.\nIf your training program uses GPUs, you should ensure that your code only\nruns on the GPU device of LOCAL_PROCESS_RANK. This can be done by:\nParsing the local_rank argument\n::\n >>> import argparse\n >>> parser = argparse.ArgumentParser()\n >>> parser.add_argument(\"--local_rank\", type=int)\n >>> args = parser.parse_args()\nSet your device to local rank using either\n::\n >>> torch.cuda.set_device(arg.local_rank) # before your code runs\nor\n::\n >>> with torch.cuda.device(arg.local_rank):\n >>> # your code to run\n3. In your training program, you are supposed to call the following function\nat the beginning to start the distributed backend. You need to make sure that\nthe init_method uses ``env://``, which is the only supported ``init_method``\nby this module.\n::\n torch.distributed.init_process_group(backend='YOUR BACKEND',\n init_method='env://')\n4. In your training program, you can either use regular distributed functions\nor use :func:`torch.nn.parallel.DistributedDataParallel` module. If your\ntraining program uses GPUs for training and you would like to use\n:func:`torch.nn.parallel.DistributedDataParallel` module,\nhere is how to configure it.\n::\n model = torch.nn.parallel.DistributedDataParallel(model,\n device_ids=[arg.local_rank],\n output_device=arg.local_rank)\nPlease ensure that ``device_ids`` argument is set to be the only GPU device id\nthat your code will be operating on. This is generally the local rank of the\nprocess. In other words, the ``device_ids`` needs to be ``[args.local_rank]``,\nand ``output_device`` needs to be ``args.local_rank`` in order to use this\nutility\n5. Another way to pass ``local_rank`` to the subprocesses via environment variable\n``LOCAL_RANK``. This behavior is enabled when you launch the script with\n``--use_env=True``. You must adjust the subprocess example above to replace\n``args.local_rank`` with ``os.environ['LOCAL_RANK']``; the launcher\nwill not pass ``--local_rank`` when you specify this flag.\n.. warning::\n ``local_rank`` is NOT globally unique: it is only unique per process\n on a machine. Thus, don't use it to decide if you should, e.g.,\n write to a networked filesystem. See\n https://github.com/pytorch/pytorch/issues/12042 for an example of\n how things can go wrong if you don't do this correctly.\n\"\"\"\n\n\nimport sys\nimport subprocess\nimport os\nimport socket\nfrom argparse import ArgumentParser, REMAINDER\n\nimport torch\n\n\ndef parse_args():\n \"\"\"\n Helper function parsing the command line options\n @retval ArgumentParser\n \"\"\"\n parser = ArgumentParser(description=\"PyTorch distributed training launch \"\n \"helper utilty that will spawn up \"\n \"multiple distributed processes\")\n\n # Optional arguments for the launch helper\n parser.add_argument(\"--nnodes\", type=int, default=1,\n help=\"The number of nodes to use for distributed \"\n \"training\")\n parser.add_argument(\"--node_rank\", type=int, default=0,\n help=\"The rank of the node for multi-node distributed \"\n \"training\")\n parser.add_argument(\"--base_rank\", type=int, default=0,\n help=\"The rank of the node for multi-node distributed \"\n \"training\")\n parser.add_argument(\"--nproc_per_node\", type=int, default=1,\n help=\"The number of processes to launch on each node, \"\n \"for GPU training, this is recommended to be set \"\n \"to the number of GPUs in your system so that \"\n \"each process can be bound to a single GPU.\")\n parser.add_argument(\"--master_addr\", default=\"127.0.0.1\", type=str,\n help=\"Master node (rank 0)'s address, should be either \"\n \"the IP address or the hostname of node 0, for \"\n \"single node multi-proc training, the \"\n \"--master_addr can simply be 127.0.0.1\")\n parser.add_argument(\"--master_port\", default=29500, type=int,\n help=\"Master node (rank 0)'s free port that needs to \"\n \"be used for communciation during distributed \"\n \"training\")\n\n # positional\n parser.add_argument(\"training_script\", type=str,\n help=\"The full path to the single GPU training \"\n \"program/script to be launched in parallel, \"\n \"followed by all the arguments for the \"\n \"training script\")\n\n # rest from the training program\n parser.add_argument('training_script_args', nargs=REMAINDER)\n return parser.parse_args()\n\n\ndef main():\n args = parse_args()\n\n # world size in terms of number of processes\n dist_world_size = args.nproc_per_node * args.nnodes\n\n # set PyTorch distributed related environmental variables\n current_env = os.environ.copy()\n current_env[\"MASTER_ADDR\"] = args.master_addr\n current_env[\"MASTER_PORT\"] = str(args.master_port)\n current_env[\"WORLD_SIZE\"] = str(dist_world_size)\n\n processes = []\n\n for local_rank in range(0, args.nproc_per_node):\n # each process's rank\n dist_rank = args.nproc_per_node * args.node_rank + local_rank\n current_env[\"RANK\"] = str(dist_rank)\n current_env[\"LOCAL_RANK\"] = str(local_rank)\n\n cmd = [args.training_script] + args.training_script_args\n print(cmd)\n process = subprocess.Popen(cmd, env=current_env)\n processes.append(process)\n\n for process in processes:\n process.wait()\n if process.returncode != 0:\n raise subprocess.CalledProcessError(returncode=process.returncode,\n cmd=process.args)\n\n\nif __name__ == \"__main__\":\n main()"
},
{
"path": "batchformer-v2/detr/tools/run_dist_launch.sh",
"content": "#!/usr/bin/env bash\n# ------------------------------------------------------------------------\n# Deformable DETR\n# Copyright (c) 2020 SenseTime. All Rights Reserved.\n# Licensed under the Apache License, Version 2.0 [see LICENSE for details]\n# ------------------------------------------------------------------------\n#source ~/.bashrc\n#conda activate torch150\n\nsource /usr/local/anaconda/4.2.0/etc/profile.d/conda.sh\nconda activate pyt\n#module load gcc\nset -x\n\nGPUS=$1\nRUN_COMMAND=${@:2}\n#if [ $GPUS -lt 8 ]; then\n# GPUS_PER_NODE=${GPUS_PER_NODE:-$GPUS}\n#else\n# GPUS_PER_NODE=${GPUS_PER_NODE:-8}\n#fi\nMASTER_ADDR=${MASTER_ADDR:-\"127.0.0.1\"}\nMASTER_PORT=${MASTER_PORT:-\"29500\"}\nNODE_RANK=${NODE_RANK:-0}\nBASE_RANK=${BASE_RANK:-0}\n#env\necho $GPUS\necho $GPUS_PER_NODE\n\n#let \"NNODES=GPUS/GPUS_PER_NODE\"\n\npython ./tools/launch.py \\\n --nnodes ${NNODES} \\\n --node_rank ${NODE_RANK} \\\n --base_rank ${BASE_RANK} \\\n --master_addr ${MASTER_ADDR} \\\n --master_port ${MASTER_PORT} \\\n --nproc_per_node ${GPUS_PER_NODE} \\\n ${RUN_COMMAND}"
},
{
"path": "batchformer-v2/detr/tools/run_dist_slurm.sh",
"content": "#!/usr/bin/env bash\n# --------------------------------------------------------------------------------------------------------------------------\n# Deformable DETR\n# Copyright (c) 2020 SenseTime. All Rights Reserved.\n# Licensed under the Apache License, Version 2.0 [see LICENSE for details]\n# --------------------------------------------------------------------------------------------------------------------------\n# Modified from https://github.com/open-mmlab/mmdetection/blob/3b53fe15d87860c6941f3dda63c0f27422da6266/tools/slurm_train.sh\n# --------------------------------------------------------------------------------------------------------------------------\n\nset -x\n\nPARTITION=$1\nJOB_NAME=$2\nGPUS=$3\nRUN_COMMAND=${@:4}\nif [ $GPUS -lt 8 ]; then\n GPUS_PER_NODE=${GPUS_PER_NODE:-$GPUS}\nelse\n GPUS_PER_NODE=${GPUS_PER_NODE:-8}\nfi\nCPUS_PER_TASK=${CPUS_PER_TASK:-4}\nSRUN_ARGS=${SRUN_ARGS:-\"\"}\n\nsrun -p ${PARTITION} \\\n --job-name=${JOB_NAME} \\\n --gres=gpu:${GPUS_PER_NODE} \\\n --ntasks=${GPUS} \\\n --ntasks-per-node=${GPUS_PER_NODE} \\\n --cpus-per-task=${CPUS_PER_TASK} \\\n --kill-on-bad-exit=1 \\\n ${SRUN_ARGS} \\\n ${RUN_COMMAND}\n\n"
},
{
"path": "batchformer-v2/detr/tox.ini",
"content": "[flake8]\nmax-line-length = 120\nignore = F401,E402,F403,W503,W504\n"
},
{
"path": "batchformer-v2/detr/util/__init__.py",
"content": "# Copyright (c) Facebook, Inc. and its affiliates. All Rights Reserved\n"
},
{
"path": "batchformer-v2/detr/util/box_ops.py",
"content": "# Copyright (c) Facebook, Inc. and its affiliates. All Rights Reserved\n\"\"\"\nUtilities for bounding box manipulation and GIoU.\n\"\"\"\nimport torch\nfrom torchvision.ops.boxes import box_area\n\n\ndef box_cxcywh_to_xyxy(x):\n x_c, y_c, w, h = x.unbind(-1)\n b = [(x_c - 0.5 * w), (y_c - 0.5 * h),\n (x_c + 0.5 * w), (y_c + 0.5 * h)]\n return torch.stack(b, dim=-1)\n\n\ndef box_xyxy_to_cxcywh(x):\n x0, y0, x1, y1 = x.unbind(-1)\n b = [(x0 + x1) / 2, (y0 + y1) / 2,\n (x1 - x0), (y1 - y0)]\n return torch.stack(b, dim=-1)\n\n\n# modified from torchvision to also return the union\ndef box_iou(boxes1, boxes2):\n area1 = box_area(boxes1)\n area2 = box_area(boxes2)\n\n lt = torch.max(boxes1[:, None, :2], boxes2[:, :2]) # [N,M,2]\n rb = torch.min(boxes1[:, None, 2:], boxes2[:, 2:]) # [N,M,2]\n\n wh = (rb - lt).clamp(min=0) # [N,M,2]\n inter = wh[:, :, 0] * wh[:, :, 1] # [N,M]\n\n union = area1[:, None] + area2 - inter\n\n iou = inter / union\n return iou, union\n\n\ndef generalized_box_iou(boxes1, boxes2):\n \"\"\"\n Generalized IoU from https://giou.stanford.edu/\n\n The boxes should be in [x0, y0, x1, y1] format\n\n Returns a [N, M] pairwise matrix, where N = len(boxes1)\n and M = len(boxes2)\n \"\"\"\n # degenerate boxes gives inf / nan results\n # so do an early check\n assert (boxes1[:, 2:] >= boxes1[:, :2]).all()\n assert (boxes2[:, 2:] >= boxes2[:, :2]).all()\n iou, union = box_iou(boxes1, boxes2)\n\n lt = torch.min(boxes1[:, None, :2], boxes2[:, :2])\n rb = torch.max(boxes1[:, None, 2:], boxes2[:, 2:])\n\n wh = (rb - lt).clamp(min=0) # [N,M,2]\n area = wh[:, :, 0] * wh[:, :, 1]\n\n return iou - (area - union) / area\n\n\ndef masks_to_boxes(masks):\n \"\"\"Compute the bounding boxes around the provided masks\n\n The masks should be in format [N, H, W] where N is the number of masks, (H, W) are the spatial dimensions.\n\n Returns a [N, 4] tensors, with the boxes in xyxy format\n \"\"\"\n if masks.numel() == 0:\n return torch.zeros((0, 4), device=masks.device)\n\n h, w = masks.shape[-2:]\n\n y = torch.arange(0, h, dtype=torch.float)\n x = torch.arange(0, w, dtype=torch.float)\n y, x = torch.meshgrid(y, x)\n\n x_mask = (masks * x.unsqueeze(0))\n x_max = x_mask.flatten(1).max(-1)[0]\n x_min = x_mask.masked_fill(~(masks.bool()), 1e8).flatten(1).min(-1)[0]\n\n y_mask = (masks * y.unsqueeze(0))\n y_max = y_mask.flatten(1).max(-1)[0]\n y_min = y_mask.masked_fill(~(masks.bool()), 1e8).flatten(1).min(-1)[0]\n\n return torch.stack([x_min, y_min, x_max, y_max], 1)\n"
},
{
"path": "batchformer-v2/detr/util/misc.py",
"content": "# Copyright (c) Facebook, Inc. and its affiliates. All Rights Reserved\n\"\"\"\nMisc functions, including distributed helpers.\n\nMostly copy-paste from torchvision references.\n\"\"\"\nimport os\nimport subprocess\nimport time\nfrom collections import defaultdict, deque\nimport datetime\nimport pickle\nfrom typing import Optional, List\n\nimport torch\nimport torch.distributed as dist\nfrom torch import Tensor\n\n# needed due to empty tensor bug in pytorch and torchvision 0.5\nimport torchvision\nprint(torchvision.__version__)\nimport math\nif float(torchvision.__version__[:3]) < 0.5 and float(torchvision.__version__[:3]) > 0.1:\n import math\n from torchvision.ops.misc import _NewEmptyTensorOp\n def _check_size_scale_factor(dim, size, scale_factor):\n # type: (int, Optional[List[int]], Optional[float]) -> None\n if size is None and scale_factor is None:\n raise ValueError(\"either size or scale_factor should be defined\")\n if size is not None and scale_factor is not None:\n raise ValueError(\"only one of size or scale_factor should be defined\")\n if not (scale_factor is not None and len(scale_factor) != dim):\n raise ValueError(\n \"scale_factor shape must match input shape. \"\n \"Input is {}D, scale_factor size is {}\".format(dim, len(scale_factor))\n )\n def _output_size(dim, input, size, scale_factor):\n # type: (int, Tensor, Optional[List[int]], Optional[float]) -> List[int]\n assert dim == 2\n _check_size_scale_factor(dim, size, scale_factor)\n if size is not None:\n return size\n # if dim is not 2 or scale_factor is iterable use _ntuple instead of concat\n assert scale_factor is not None and isinstance(scale_factor, (int, float))\n scale_factors = [scale_factor, scale_factor]\n # math.floor might return float in py2.7\n return [\n int(math.floor(input.size(i + 2) * scale_factors[i])) for i in range(dim)\n ]\nelif float(torchvision.__version__[:3]) < 0.7 and float(torchvision.__version__[:3]) > 0.1:\n from torchvision.ops import _new_empty_tensor\n from torchvision.ops.misc import _output_size\n\n\nclass SmoothedValue(object):\n \"\"\"Track a series of values and provide access to smoothed values over a\n window or the global series average.\n \"\"\"\n\n def __init__(self, window_size=20, fmt=None):\n if fmt is None:\n fmt = \"{median:.4f} ({global_avg:.4f})\"\n self.deque = deque(maxlen=window_size)\n self.total = 0.0\n self.count = 0\n self.fmt = fmt\n\n def update(self, value, n=1):\n self.deque.append(value)\n self.count += n\n self.total += value * n\n\n def synchronize_between_processes(self):\n \"\"\"\n Warning: does not synchronize the deque!\n \"\"\"\n if not is_dist_avail_and_initialized():\n return\n t = torch.tensor([self.count, self.total], dtype=torch.float64, device='cuda')\n dist.barrier()\n dist.all_reduce(t)\n t = t.tolist()\n self.count = int(t[0])\n self.total = t[1]\n\n @property\n def median(self):\n d = torch.tensor(list(self.deque))\n return d.median().item()\n\n @property\n def avg(self):\n d = torch.tensor(list(self.deque), dtype=torch.float32)\n return d.mean().item()\n\n @property\n def global_avg(self):\n return self.total / self.count\n\n @property\n def max(self):\n return max(self.deque)\n\n @property\n def value(self):\n return self.deque[-1]\n\n def __str__(self):\n return self.fmt.format(\n median=self.median,\n avg=self.avg,\n global_avg=self.global_avg,\n max=self.max,\n value=self.value)\n\n\ndef all_gather(data):\n \"\"\"\n Run all_gather on arbitrary picklable data (not necessarily tensors)\n Args:\n data: any picklable object\n Returns:\n list[data]: list of data gathered from each rank\n \"\"\"\n world_size = get_world_size()\n if world_size == 1:\n return [data]\n\n # serialized to a Tensor\n buffer = pickle.dumps(data)\n storage = torch.ByteStorage.from_buffer(buffer)\n tensor = torch.ByteTensor(storage).to(\"cuda\")\n\n # obtain Tensor size of each rank\n local_size = torch.tensor([tensor.numel()], device=\"cuda\")\n size_list = [torch.tensor([0], device=\"cuda\") for _ in range(world_size)]\n dist.all_gather(size_list, local_size)\n size_list = [int(size.item()) for size in size_list]\n max_size = max(size_list)\n\n # receiving Tensor from all ranks\n # we pad the tensor because torch all_gather does not support\n # gathering tensors of different shapes\n tensor_list = []\n for _ in size_list:\n tensor_list.append(torch.empty((max_size,), dtype=torch.uint8, device=\"cuda\"))\n if local_size != max_size:\n padding = torch.empty(size=(max_size - local_size,), dtype=torch.uint8, device=\"cuda\")\n tensor = torch.cat((tensor, padding), dim=0)\n dist.all_gather(tensor_list, tensor)\n\n data_list = []\n for size, tensor in zip(size_list, tensor_list):\n buffer = tensor.cpu().numpy().tobytes()[:size]\n data_list.append(pickle.loads(buffer))\n\n return data_list\n\n\ndef reduce_dict(input_dict, average=True):\n \"\"\"\n Args:\n input_dict (dict): all the values will be reduced\n average (bool): whether to do average or sum\n Reduce the values in the dictionary from all processes so that all processes\n have the averaged results. Returns a dict with the same fields as\n input_dict, after reduction.\n \"\"\"\n world_size = get_world_size()\n if world_size < 2:\n return input_dict\n with torch.no_grad():\n names = []\n values = []\n # sort the keys so that they are consistent across processes\n for k in sorted(input_dict.keys()):\n names.append(k)\n values.append(input_dict[k])\n values = torch.stack(values, dim=0)\n dist.all_reduce(values)\n if average:\n values /= world_size\n reduced_dict = {k: v for k, v in zip(names, values)}\n return reduced_dict\n\n\nclass MetricLogger(object):\n def __init__(self, delimiter=\"\\t\"):\n self.meters = defaultdict(SmoothedValue)\n self.delimiter = delimiter\n\n def update(self, **kwargs):\n for k, v in kwargs.items():\n if isinstance(v, torch.Tensor):\n v = v.item()\n assert isinstance(v, (float, int))\n self.meters[k].update(v)\n\n def __getattr__(self, attr):\n if attr in self.meters:\n return self.meters[attr]\n if attr in self.__dict__:\n return self.__dict__[attr]\n raise AttributeError(\"'{}' object has no attribute '{}'\".format(\n type(self).__name__, attr))\n\n def __str__(self):\n loss_str = []\n for name, meter in self.meters.items():\n loss_str.append(\n \"{}: {}\".format(name, str(meter))\n )\n return self.delimiter.join(loss_str)\n\n def synchronize_between_processes(self):\n for meter in self.meters.values():\n meter.synchronize_between_processes()\n\n def add_meter(self, name, meter):\n self.meters[name] = meter\n\n def log_every(self, iterable, print_freq, header=None):\n i = 0\n if not header:\n header = ''\n start_time = time.time()\n end = time.time()\n iter_time = SmoothedValue(fmt='{avg:.4f}')\n data_time = SmoothedValue(fmt='{avg:.4f}')\n space_fmt = ':' + str(len(str(len(iterable)))) + 'd'\n if torch.cuda.is_available():\n log_msg = self.delimiter.join([\n header,\n '[{0' + space_fmt + '}/{1}]',\n 'eta: {eta}',\n '{meters}',\n 'time: {time}',\n 'data: {data}',\n 'max mem: {memory:.0f}'\n ])\n else:\n log_msg = self.delimiter.join([\n header,\n '[{0' + space_fmt + '}/{1}]',\n 'eta: {eta}',\n '{meters}',\n 'time: {time}',\n 'data: {data}'\n ])\n MB = 1024.0 * 1024.0\n for obj in iterable:\n data_time.update(time.time() - end)\n yield obj\n iter_time.update(time.time() - end)\n if i % print_freq == 0 or i == len(iterable) - 1:\n eta_seconds = iter_time.global_avg * (len(iterable) - i)\n eta_string = str(datetime.timedelta(seconds=int(eta_seconds)))\n if torch.cuda.is_available():\n print(log_msg.format(\n i, len(iterable), eta=eta_string,\n meters=str(self),\n time=str(iter_time), data=str(data_time),\n memory=torch.cuda.max_memory_allocated() / MB))\n else:\n print(log_msg.format(\n i, len(iterable), eta=eta_string,\n meters=str(self),\n time=str(iter_time), data=str(data_time)))\n i += 1\n end = time.time()\n total_time = time.time() - start_time\n total_time_str = str(datetime.timedelta(seconds=int(total_time)))\n print('{} Total time: {} ({:.4f} s / it)'.format(\n header, total_time_str, total_time / len(iterable)))\n\n\ndef get_sha():\n cwd = os.path.dirname(os.path.abspath(__file__))\n\n def _run(command):\n return subprocess.check_output(command, cwd=cwd).decode('ascii').strip()\n sha = 'N/A'\n diff = \"clean\"\n branch = 'N/A'\n try:\n sha = _run(['git', 'rev-parse', 'HEAD'])\n subprocess.check_output(['git', 'diff'], cwd=cwd)\n diff = _run(['git', 'diff-index', 'HEAD'])\n diff = \"has uncommited changes\" if diff else \"clean\"\n branch = _run(['git', 'rev-parse', '--abbrev-ref', 'HEAD'])\n except Exception:\n pass\n message = f\"sha: {sha}, status: {diff}, branch: {branch}\"\n return message\n\n\ndef collate_fn(batch):\n batch = list(zip(*batch))\n batch[0] = nested_tensor_from_tensor_list(batch[0])\n return tuple(batch)\n\n\ndef _max_by_axis(the_list):\n # type: (List[List[int]]) -> List[int]\n maxes = the_list[0]\n for sublist in the_list[1:]:\n for index, item in enumerate(sublist):\n maxes[index] = max(maxes[index], item)\n return maxes\n\n\nclass NestedTensor(object):\n def __init__(self, tensors, mask: Optional[Tensor]):\n self.tensors = tensors\n self.mask = mask\n\n def to(self, device):\n # type: (Device) -> NestedTensor # noqa\n cast_tensor = self.tensors.to(device)\n mask = self.mask\n if mask is not None:\n assert mask is not None\n cast_mask = mask.to(device)\n else:\n cast_mask = None\n return NestedTensor(cast_tensor, cast_mask)\n\n def decompose(self):\n return self.tensors, self.mask\n\n def __repr__(self):\n return str(self.tensors)\n\n\ndef nested_tensor_from_tensor_list(tensor_list: List[Tensor]):\n # TODO make this more general\n if tensor_list[0].ndim == 3:\n if torchvision._is_tracing():\n # nested_tensor_from_tensor_list() does not export well to ONNX\n # call _onnx_nested_tensor_from_tensor_list() instead\n return _onnx_nested_tensor_from_tensor_list(tensor_list)\n\n # TODO make it support different-sized images\n max_size = _max_by_axis([list(img.shape) for img in tensor_list])\n # min_size = tuple(min(s) for s in zip(*[img.shape for img in tensor_list]))\n batch_shape = [len(tensor_list)] + max_size\n b, c, h, w = batch_shape\n dtype = tensor_list[0].dtype\n device = tensor_list[0].device\n tensor = torch.zeros(batch_shape, dtype=dtype, device=device)\n mask = torch.ones((b, h, w), dtype=torch.bool, device=device)\n for img, pad_img, m in zip(tensor_list, tensor, mask):\n pad_img[: img.shape[0], : img.shape[1], : img.shape[2]].copy_(img)\n m[: img.shape[1], :img.shape[2]] = False\n else:\n raise ValueError('not supported')\n return NestedTensor(tensor, mask)\n\n\n# _onnx_nested_tensor_from_tensor_list() is an implementation of\n# nested_tensor_from_tensor_list() that is supported by ONNX tracing.\n@torch.jit.unused\ndef _onnx_nested_tensor_from_tensor_list(tensor_list: List[Tensor]) -> NestedTensor:\n max_size = []\n for i in range(tensor_list[0].dim()):\n max_size_i = torch.max(torch.stack([img.shape[i] for img in tensor_list]).to(torch.float32)).to(torch.int64)\n max_size.append(max_size_i)\n max_size = tuple(max_size)\n\n # work around for\n # pad_img[: img.shape[0], : img.shape[1], : img.shape[2]].copy_(img)\n # m[: img.shape[1], :img.shape[2]] = False\n # which is not yet supported in onnx\n padded_imgs = []\n padded_masks = []\n for img in tensor_list:\n padding = [(s1 - s2) for s1, s2 in zip(max_size, tuple(img.shape))]\n padded_img = torch.nn.functional.pad(img, (0, padding[2], 0, padding[1], 0, padding[0]))\n padded_imgs.append(padded_img)\n\n m = torch.zeros_like(img[0], dtype=torch.int, device=img.device)\n padded_mask = torch.nn.functional.pad(m, (0, padding[2], 0, padding[1]), \"constant\", 1)\n padded_masks.append(padded_mask.to(torch.bool))\n\n tensor = torch.stack(padded_imgs)\n mask = torch.stack(padded_masks)\n\n return NestedTensor(tensor, mask=mask)\n\n\ndef setup_for_distributed(is_master):\n \"\"\"\n This function disables printing when not in master process\n \"\"\"\n import builtins as __builtin__\n builtin_print = __builtin__.print\n\n def print(*args, **kwargs):\n force = kwargs.pop('force', False)\n if is_master or force:\n builtin_print(*args, **kwargs)\n\n __builtin__.print = print\n\n\ndef is_dist_avail_and_initialized():\n if not dist.is_available():\n return False\n if not dist.is_initialized():\n return False\n return True\n\n\ndef get_world_size():\n if not is_dist_avail_and_initialized():\n return 1\n return dist.get_world_size()\n\n\ndef get_rank():\n if not is_dist_avail_and_initialized():\n return 0\n return dist.get_rank()\n\n\ndef is_main_process():\n return get_rank() == 0\n\n\ndef save_on_master(*args, **kwargs):\n if is_main_process():\n torch.save(*args, **kwargs)\n\n\ndef init_distributed_mode(args):\n if 'RANK' in os.environ and 'WORLD_SIZE' in os.environ:\n args.rank = int(os.environ[\"RANK\"])\n args.world_size = int(os.environ['WORLD_SIZE'])\n args.gpu = int(os.environ['LOCAL_RANK'])\n elif 'SLURM_PROCID' in os.environ:\n args.rank = int(os.environ['SLURM_PROCID'])\n args.gpu = args.rank % torch.cuda.device_count()\n else:\n print('Not using distributed mode')\n args.distributed = False\n return\n\n args.distributed = True\n\n torch.cuda.set_device(args.gpu)\n args.dist_backend = 'nccl'\n print('| distributed init (rank {}): {}'.format(\n args.rank, args.dist_url), flush=True)\n torch.distributed.init_process_group(backend=args.dist_backend, init_method=args.dist_url,\n world_size=args.world_size, rank=args.rank)\n torch.distributed.barrier()\n setup_for_distributed(args.rank == 0)\n\n\n@torch.no_grad()\ndef accuracy(output, target, topk=(1,)):\n \"\"\"Computes the precision@k for the specified values of k\"\"\"\n if target.numel() == 0:\n return [torch.zeros([], device=output.device)]\n maxk = max(topk)\n batch_size = target.size(0)\n\n _, pred = output.topk(maxk, 1, True, True)\n pred = pred.t()\n correct = pred.eq(target.view(1, -1).expand_as(pred))\n\n res = []\n for k in topk:\n correct_k = correct[:k].view(-1).float().sum(0)\n res.append(correct_k.mul_(100.0 / batch_size))\n return res\n\n\ndef interpolate(input, size=None, scale_factor=None, mode=\"nearest\", align_corners=None):\n # type: (Tensor, Optional[List[int]], Optional[float], str, Optional[bool]) -> Tensor\n \"\"\"\n Equivalent to nn.functional.interpolate, but with support for empty batch sizes.\n This will eventually be supported natively by PyTorch, and this\n class can go away.\n \"\"\"\n if float(torchvision.__version__[:3]) < 0.7 and not torchvision.__version__.startswith('0.10'):\n if input.numel() > 0:\n return torch.nn.functional.interpolate(\n input, size, scale_factor, mode, align_corners\n )\n\n output_shape = _output_size(2, input, size, scale_factor)\n output_shape = list(input.shape[:-2]) + list(output_shape)\n return _new_empty_tensor(input, output_shape)\n else:\n return torchvision.ops.misc.interpolate(input, size, scale_factor, mode, align_corners)\n"
},
{
"path": "batchformer-v2/detr/util/plot_utils.py",
"content": "\"\"\"\nPlotting utilities to visualize training logs.\n\"\"\"\nimport torch\nimport pandas as pd\nimport numpy as np\nimport seaborn as sns\nimport matplotlib.pyplot as plt\n\nfrom pathlib import Path, PurePath\n\n\ndef plot_logs(logs, fields=('class_error', 'loss_bbox_unscaled', 'mAP'), ewm_col=0, log_name='log.txt'):\n '''\n Function to plot specific fields from training log(s). Plots both training and test results.\n\n :: Inputs - logs = list containing Path objects, each pointing to individual dir with a log file\n - fields = which results to plot from each log file - plots both training and test for each field.\n - ewm_col = optional, which column to use as the exponential weighted smoothing of the plots\n - log_name = optional, name of log file if different than default 'log.txt'.\n\n :: Outputs - matplotlib plots of results in fields, color coded for each log file.\n - solid lines are training results, dashed lines are test results.\n\n '''\n func_name = \"plot_utils.py::plot_logs\"\n\n # verify logs is a list of Paths (list[Paths]) or single Pathlib object Path,\n # convert single Path to list to avoid 'not iterable' error\n\n if not isinstance(logs, list):\n if isinstance(logs, PurePath):\n logs = [logs]\n print(f\"{func_name} info: logs param expects a list argument, converted to list[Path].\")\n else:\n raise ValueError(f\"{func_name} - invalid argument for logs parameter.\\n \\\n Expect list[Path] or single Path obj, received {type(logs)}\")\n\n # Quality checks - verify valid dir(s), that every item in list is Path object, and that log_name exists in each dir\n for i, dir in enumerate(logs):\n if not isinstance(dir, PurePath):\n raise ValueError(f\"{func_name} - non-Path object in logs argument of {type(dir)}: \\n{dir}\")\n if not dir.exists():\n raise ValueError(f\"{func_name} - invalid directory in logs argument:\\n{dir}\")\n # verify log_name exists\n fn = Path(dir / log_name)\n if not fn.exists():\n print(f\"-> missing {log_name}. Have you gotten to Epoch 1 in training?\")\n print(f\"--> full path of missing log file: {fn}\")\n return\n\n # load log file(s) and plot\n dfs = [pd.read_json(Path(p) / log_name, lines=True) for p in logs]\n\n fig, axs = plt.subplots(ncols=len(fields), figsize=(16, 5))\n\n for df, color in zip(dfs, sns.color_palette(n_colors=len(logs))):\n for j, field in enumerate(fields):\n if field == 'mAP':\n coco_eval = pd.DataFrame(\n np.stack(df.test_coco_eval_bbox.dropna().values)[:, 1]\n ).ewm(com=ewm_col).mean()\n axs[j].plot(coco_eval, c=color)\n else:\n df.interpolate().ewm(com=ewm_col).mean().plot(\n y=[f'train_{field}', f'test_{field}'],\n ax=axs[j],\n color=[color] * 2,\n style=['-', '--']\n )\n for ax, field in zip(axs, fields):\n ax.legend([Path(p).name for p in logs])\n ax.set_title(field)\n\n\ndef plot_precision_recall(files, naming_scheme='iter'):\n if naming_scheme == 'exp_id':\n # name becomes exp_id\n names = [f.parts[-3] for f in files]\n elif naming_scheme == 'iter':\n names = [f.stem for f in files]\n else:\n raise ValueError(f'not supported {naming_scheme}')\n fig, axs = plt.subplots(ncols=2, figsize=(16, 5))\n for f, color, name in zip(files, sns.color_palette(\"Blues\", n_colors=len(files)), names):\n data = torch.load(f)\n # precision is n_iou, n_points, n_cat, n_area, max_det\n precision = data['precision']\n recall = data['params'].recThrs\n scores = data['scores']\n # take precision for all classes, all areas and 100 detections\n precision = precision[0, :, :, 0, -1].mean(1)\n scores = scores[0, :, :, 0, -1].mean(1)\n prec = precision.mean()\n rec = data['recall'][0, :, 0, -1].mean()\n print(f'{naming_scheme} {name}: mAP@50={prec * 100: 05.1f}, ' +\n f'score={scores.mean():0.3f}, ' +\n f'f1={2 * prec * rec / (prec + rec + 1e-8):0.3f}'\n )\n axs[0].plot(recall, precision, c=color)\n axs[1].plot(recall, scores, c=color)\n\n axs[0].set_title('Precision / Recall')\n axs[0].legend(names)\n axs[1].set_title('Scores / Recall')\n axs[1].legend(names)\n return fig, axs\n"
},
{
"path": "czsl/LICENSE",
"content": " GNU GENERAL PUBLIC LICENSE\n Version 3, 29 June 2007\n\n Copyright (C) 2007 Free Software Foundation, Inc. \n Everyone is permitted to copy and distribute verbatim copies\n of this license document, but changing it is not allowed.\n\n Preamble\n\n The GNU General Public License is a free, copyleft license for\nsoftware and other kinds of works.\n\n The licenses for most software and other practical works are designed\nto take away your freedom to share and change the works. By contrast,\nthe GNU General Public License is intended to guarantee your freedom to\nshare and change all versions of a program--to make sure it remains free\nsoftware for all its users. We, the Free Software Foundation, use the\nGNU General Public License for most of our software; it applies also to\nany other work released this way by its authors. You can apply it to\nyour programs, too.\n\n When we speak of free software, we are referring to freedom, not\nprice. Our General Public Licenses are designed to make sure that you\nhave the freedom to distribute copies of free software (and charge for\nthem if you wish), that you receive source code or can get it if you\nwant it, that you can change the software or use pieces of it in new\nfree programs, and that you know you can do these things.\n\n To protect your rights, we need to prevent others from denying you\nthese rights or asking you to surrender the rights. Therefore, you have\ncertain responsibilities if you distribute copies of the software, or if\nyou modify it: responsibilities to respect the freedom of others.\n\n For example, if you distribute copies of such a program, whether\ngratis or for a fee, you must pass on to the recipients the same\nfreedoms that you received. You must make sure that they, too, receive\nor can get the source code. And you must show them these terms so they\nknow their rights.\n\n Developers that use the GNU GPL protect your rights with two steps:\n(1) assert copyright on the software, and (2) offer you this License\ngiving you legal permission to copy, distribute and/or modify it.\n\n For the developers' and authors' protection, the GPL clearly explains\nthat there is no warranty for this free software. For both users' and\nauthors' sake, the GPL requires that modified versions be marked as\nchanged, so that their problems will not be attributed erroneously to\nauthors of previous versions.\n\n Some devices are designed to deny users access to install or run\nmodified versions of the software inside them, although the manufacturer\ncan do so. This is fundamentally incompatible with the aim of\nprotecting users' freedom to change the software. The systematic\npattern of such abuse occurs in the area of products for individuals to\nuse, which is precisely where it is most unacceptable. Therefore, we\nhave designed this version of the GPL to prohibit the practice for those\nproducts. If such problems arise substantially in other domains, we\nstand ready to extend this provision to those domains in future versions\nof the GPL, as needed to protect the freedom of users.\n\n Finally, every program is threatened constantly by software patents.\nStates should not allow patents to restrict development and use of\nsoftware on general-purpose computers, but in those that do, we wish to\navoid the special danger that patents applied to a free program could\nmake it effectively proprietary. To prevent this, the GPL assures that\npatents cannot be used to render the program non-free.\n\n The precise terms and conditions for copying, distribution and\nmodification follow.\n\n TERMS AND CONDITIONS\n\n 0. Definitions.\n\n \"This License\" refers to version 3 of the GNU General Public License.\n\n \"Copyright\" also means copyright-like laws that apply to other kinds of\nworks, such as semiconductor masks.\n\n \"The Program\" refers to any copyrightable work licensed under this\nLicense. Each licensee is addressed as \"you\". \"Licensees\" and\n\"recipients\" may be individuals or organizations.\n\n To \"modify\" a work means to copy from or adapt all or part of the work\nin a fashion requiring copyright permission, other than the making of an\nexact copy. The resulting work is called a \"modified version\" of the\nearlier work or a work \"based on\" the earlier work.\n\n A \"covered work\" means either the unmodified Program or a work based\non the Program.\n\n To \"propagate\" a work means to do anything with it that, without\npermission, would make you directly or secondarily liable for\ninfringement under applicable copyright law, except executing it on a\ncomputer or modifying a private copy. Propagation includes copying,\ndistribution (with or without modification), making available to the\npublic, and in some countries other activities as well.\n\n To \"convey\" a work means any kind of propagation that enables other\nparties to make or receive copies. Mere interaction with a user through\na computer network, with no transfer of a copy, is not conveying.\n\n An interactive user interface displays \"Appropriate Legal Notices\"\nto the extent that it includes a convenient and prominently visible\nfeature that (1) displays an appropriate copyright notice, and (2)\ntells the user that there is no warranty for the work (except to the\nextent that warranties are provided), that licensees may convey the\nwork under this License, and how to view a copy of this License. If\nthe interface presents a list of user commands or options, such as a\nmenu, a prominent item in the list meets this criterion.\n\n 1. Source Code.\n\n The \"source code\" for a work means the preferred form of the work\nfor making modifications to it. \"Object code\" means any non-source\nform of a work.\n\n A \"Standard Interface\" means an interface that either is an official\nstandard defined by a recognized standards body, or, in the case of\ninterfaces specified for a particular programming language, one that\nis widely used among developers working in that language.\n\n The \"System Libraries\" of an executable work include anything, other\nthan the work as a whole, that (a) is included in the normal form of\npackaging a Major Component, but which is not part of that Major\nComponent, and (b) serves only to enable use of the work with that\nMajor Component, or to implement a Standard Interface for which an\nimplementation is available to the public in source code form. A\n\"Major Component\", in this context, means a major essential component\n(kernel, window system, and so on) of the specific operating system\n(if any) on which the executable work runs, or a compiler used to\nproduce the work, or an object code interpreter used to run it.\n\n The \"Corresponding Source\" for a work in object code form means all\nthe source code needed to generate, install, and (for an executable\nwork) run the object code and to modify the work, including scripts to\ncontrol those activities. However, it does not include the work's\nSystem Libraries, or general-purpose tools or generally available free\nprograms which are used unmodified in performing those activities but\nwhich are not part of the work. For example, Corresponding Source\nincludes interface definition files associated with source files for\nthe work, and the source code for shared libraries and dynamically\nlinked subprograms that the work is specifically designed to require,\nsuch as by intimate data communication or control flow between those\nsubprograms and other parts of the work.\n\n The Corresponding Source need not include anything that users\ncan regenerate automatically from other parts of the Corresponding\nSource.\n\n The Corresponding Source for a work in source code form is that\nsame work.\n\n 2. Basic Permissions.\n\n All rights granted under this License are granted for the term of\ncopyright on the Program, and are irrevocable provided the stated\nconditions are met. This License explicitly affirms your unlimited\npermission to run the unmodified Program. The output from running a\ncovered work is covered by this License only if the output, given its\ncontent, constitutes a covered work. This License acknowledges your\nrights of fair use or other equivalent, as provided by copyright law.\n\n You may make, run and propagate covered works that you do not\nconvey, without conditions so long as your license otherwise remains\nin force. You may convey covered works to others for the sole purpose\nof having them make modifications exclusively for you, or provide you\nwith facilities for running those works, provided that you comply with\nthe terms of this License in conveying all material for which you do\nnot control copyright. Those thus making or running the covered works\nfor you must do so exclusively on your behalf, under your direction\nand control, on terms that prohibit them from making any copies of\nyour copyrighted material outside their relationship with you.\n\n Conveying under any other circumstances is permitted solely under\nthe conditions stated below. Sublicensing is not allowed; section 10\nmakes it unnecessary.\n\n 3. Protecting Users' Legal Rights From Anti-Circumvention Law.\n\n No covered work shall be deemed part of an effective technological\nmeasure under any applicable law fulfilling obligations under article\n11 of the WIPO copyright treaty adopted on 20 December 1996, or\nsimilar laws prohibiting or restricting circumvention of such\nmeasures.\n\n When you convey a covered work, you waive any legal power to forbid\ncircumvention of technological measures to the extent such circumvention\nis effected by exercising rights under this License with respect to\nthe covered work, and you disclaim any intention to limit operation or\nmodification of the work as a means of enforcing, against the work's\nusers, your or third parties' legal rights to forbid circumvention of\ntechnological measures.\n\n 4. Conveying Verbatim Copies.\n\n You may convey verbatim copies of the Program's source code as you\nreceive it, in any medium, provided that you conspicuously and\nappropriately publish on each copy an appropriate copyright notice;\nkeep intact all notices stating that this License and any\nnon-permissive terms added in accord with section 7 apply to the code;\nkeep intact all notices of the absence of any warranty; and give all\nrecipients a copy of this License along with the Program.\n\n You may charge any price or no price for each copy that you convey,\nand you may offer support or warranty protection for a fee.\n\n 5. Conveying Modified Source Versions.\n\n You may convey a work based on the Program, or the modifications to\nproduce it from the Program, in the form of source code under the\nterms of section 4, provided that you also meet all of these conditions:\n\n a) The work must carry prominent notices stating that you modified\n it, and giving a relevant date.\n\n b) The work must carry prominent notices stating that it is\n released under this License and any conditions added under section\n 7. This requirement modifies the requirement in section 4 to\n \"keep intact all notices\".\n\n c) You must license the entire work, as a whole, under this\n License to anyone who comes into possession of a copy. This\n License will therefore apply, along with any applicable section 7\n additional terms, to the whole of the work, and all its parts,\n regardless of how they are packaged. This License gives no\n permission to license the work in any other way, but it does not\n invalidate such permission if you have separately received it.\n\n d) If the work has interactive user interfaces, each must display\n Appropriate Legal Notices; however, if the Program has interactive\n interfaces that do not display Appropriate Legal Notices, your\n work need not make them do so.\n\n A compilation of a covered work with other separate and independent\nworks, which are not by their nature extensions of the covered work,\nand which are not combined with it such as to form a larger program,\nin or on a volume of a storage or distribution medium, is called an\n\"aggregate\" if the compilation and its resulting copyright are not\nused to limit the access or legal rights of the compilation's users\nbeyond what the individual works permit. Inclusion of a covered work\nin an aggregate does not cause this License to apply to the other\nparts of the aggregate.\n\n 6. Conveying Non-Source Forms.\n\n You may convey a covered work in object code form under the terms\nof sections 4 and 5, provided that you also convey the\nmachine-readable Corresponding Source under the terms of this License,\nin one of these ways:\n\n a) Convey the object code in, or embodied in, a physical product\n (including a physical distribution medium), accompanied by the\n Corresponding Source fixed on a durable physical medium\n customarily used for software interchange.\n\n b) Convey the object code in, or embodied in, a physical product\n (including a physical distribution medium), accompanied by a\n written offer, valid for at least three years and valid for as\n long as you offer spare parts or customer support for that product\n model, to give anyone who possesses the object code either (1) a\n copy of the Corresponding Source for all the software in the\n product that is covered by this License, on a durable physical\n medium customarily used for software interchange, for a price no\n more than your reasonable cost of physically performing this\n conveying of source, or (2) access to copy the\n Corresponding Source from a network server at no charge.\n\n c) Convey individual copies of the object code with a copy of the\n written offer to provide the Corresponding Source. This\n alternative is allowed only occasionally and noncommercially, and\n only if you received the object code with such an offer, in accord\n with subsection 6b.\n\n d) Convey the object code by offering access from a designated\n place (gratis or for a charge), and offer equivalent access to the\n Corresponding Source in the same way through the same place at no\n further charge. You need not require recipients to copy the\n Corresponding Source along with the object code. If the place to\n copy the object code is a network server, the Corresponding Source\n may be on a different server (operated by you or a third party)\n that supports equivalent copying facilities, provided you maintain\n clear directions next to the object code saying where to find the\n Corresponding Source. Regardless of what server hosts the\n Corresponding Source, you remain obligated to ensure that it is\n available for as long as needed to satisfy these requirements.\n\n e) Convey the object code using peer-to-peer transmission, provided\n you inform other peers where the object code and Corresponding\n Source of the work are being offered to the general public at no\n charge under subsection 6d.\n\n A separable portion of the object code, whose source code is excluded\nfrom the Corresponding Source as a System Library, need not be\nincluded in conveying the object code work.\n\n A \"User Product\" is either (1) a \"consumer product\", which means any\ntangible personal property which is normally used for personal, family,\nor household purposes, or (2) anything designed or sold for incorporation\ninto a dwelling. In determining whether a product is a consumer product,\ndoubtful cases shall be resolved in favor of coverage. For a particular\nproduct received by a particular user, \"normally used\" refers to a\ntypical or common use of that class of product, regardless of the status\nof the particular user or of the way in which the particular user\nactually uses, or expects or is expected to use, the product. A product\nis a consumer product regardless of whether the product has substantial\ncommercial, industrial or non-consumer uses, unless such uses represent\nthe only significant mode of use of the product.\n\n \"Installation Information\" for a User Product means any methods,\nprocedures, authorization keys, or other information required to install\nand execute modified versions of a covered work in that User Product from\na modified version of its Corresponding Source. The information must\nsuffice to ensure that the continued functioning of the modified object\ncode is in no case prevented or interfered with solely because\nmodification has been made.\n\n If you convey an object code work under this section in, or with, or\nspecifically for use in, a User Product, and the conveying occurs as\npart of a transaction in which the right of possession and use of the\nUser Product is transferred to the recipient in perpetuity or for a\nfixed term (regardless of how the transaction is characterized), the\nCorresponding Source conveyed under this section must be accompanied\nby the Installation Information. But this requirement does not apply\nif neither you nor any third party retains the ability to install\nmodified object code on the User Product (for example, the work has\nbeen installed in ROM).\n\n The requirement to provide Installation Information does not include a\nrequirement to continue to provide support service, warranty, or updates\nfor a work that has been modified or installed by the recipient, or for\nthe User Product in which it has been modified or installed. Access to a\nnetwork may be denied when the modification itself materially and\nadversely affects the operation of the network or violates the rules and\nprotocols for communication across the network.\n\n Corresponding Source conveyed, and Installation Information provided,\nin accord with this section must be in a format that is publicly\ndocumented (and with an implementation available to the public in\nsource code form), and must require no special password or key for\nunpacking, reading or copying.\n\n 7. Additional Terms.\n\n \"Additional permissions\" are terms that supplement the terms of this\nLicense by making exceptions from one or more of its conditions.\nAdditional permissions that are applicable to the entire Program shall\nbe treated as though they were included in this License, to the extent\nthat they are valid under applicable law. If additional permissions\napply only to part of the Program, that part may be used separately\nunder those permissions, but the entire Program remains governed by\nthis License without regard to the additional permissions.\n\n When you convey a copy of a covered work, you may at your option\nremove any additional permissions from that copy, or from any part of\nit. (Additional permissions may be written to require their own\nremoval in certain cases when you modify the work.) You may place\nadditional permissions on material, added by you to a covered work,\nfor which you have or can give appropriate copyright permission.\n\n Notwithstanding any other provision of this License, for material you\nadd to a covered work, you may (if authorized by the copyright holders of\nthat material) supplement the terms of this License with terms:\n\n a) Disclaiming warranty or limiting liability differently from the\n terms of sections 15 and 16 of this License; or\n\n b) Requiring preservation of specified reasonable legal notices or\n author attributions in that material or in the Appropriate Legal\n Notices displayed by works containing it; or\n\n c) Prohibiting misrepresentation of the origin of that material, or\n requiring that modified versions of such material be marked in\n reasonable ways as different from the original version; or\n\n d) Limiting the use for publicity purposes of names of licensors or\n authors of the material; or\n\n e) Declining to grant rights under trademark law for use of some\n trade names, trademarks, or service marks; or\n\n f) Requiring indemnification of licensors and authors of that\n material by anyone who conveys the material (or modified versions of\n it) with contractual assumptions of liability to the recipient, for\n any liability that these contractual assumptions directly impose on\n those licensors and authors.\n\n All other non-permissive additional terms are considered \"further\nrestrictions\" within the meaning of section 10. If the Program as you\nreceived it, or any part of it, contains a notice stating that it is\ngoverned by this License along with a term that is a further\nrestriction, you may remove that term. If a license document contains\na further restriction but permits relicensing or conveying under this\nLicense, you may add to a covered work material governed by the terms\nof that license document, provided that the further restriction does\nnot survive such relicensing or conveying.\n\n If you add terms to a covered work in accord with this section, you\nmust place, in the relevant source files, a statement of the\nadditional terms that apply to those files, or a notice indicating\nwhere to find the applicable terms.\n\n Additional terms, permissive or non-permissive, may be stated in the\nform of a separately written license, or stated as exceptions;\nthe above requirements apply either way.\n\n 8. Termination.\n\n You may not propagate or modify a covered work except as expressly\nprovided under this License. Any attempt otherwise to propagate or\nmodify it is void, and will automatically terminate your rights under\nthis License (including any patent licenses granted under the third\nparagraph of section 11).\n\n However, if you cease all violation of this License, then your\nlicense from a particular copyright holder is reinstated (a)\nprovisionally, unless and until the copyright holder explicitly and\nfinally terminates your license, and (b) permanently, if the copyright\nholder fails to notify you of the violation by some reasonable means\nprior to 60 days after the cessation.\n\n Moreover, your license from a particular copyright holder is\nreinstated permanently if the copyright holder notifies you of the\nviolation by some reasonable means, this is the first time you have\nreceived notice of violation of this License (for any work) from that\ncopyright holder, and you cure the violation prior to 30 days after\nyour receipt of the notice.\n\n Termination of your rights under this section does not terminate the\nlicenses of parties who have received copies or rights from you under\nthis License. If your rights have been terminated and not permanently\nreinstated, you do not qualify to receive new licenses for the same\nmaterial under section 10.\n\n 9. Acceptance Not Required for Having Copies.\n\n You are not required to accept this License in order to receive or\nrun a copy of the Program. Ancillary propagation of a covered work\noccurring solely as a consequence of using peer-to-peer transmission\nto receive a copy likewise does not require acceptance. However,\nnothing other than this License grants you permission to propagate or\nmodify any covered work. These actions infringe copyright if you do\nnot accept this License. Therefore, by modifying or propagating a\ncovered work, you indicate your acceptance of this License to do so.\n\n 10. Automatic Licensing of Downstream Recipients.\n\n Each time you convey a covered work, the recipient automatically\nreceives a license from the original licensors, to run, modify and\npropagate that work, subject to this License. You are not responsible\nfor enforcing compliance by third parties with this License.\n\n An \"entity transaction\" is a transaction transferring control of an\norganization, or substantially all assets of one, or subdividing an\norganization, or merging organizations. If propagation of a covered\nwork results from an entity transaction, each party to that\ntransaction who receives a copy of the work also receives whatever\nlicenses to the work the party's predecessor in interest had or could\ngive under the previous paragraph, plus a right to possession of the\nCorresponding Source of the work from the predecessor in interest, if\nthe predecessor has it or can get it with reasonable efforts.\n\n You may not impose any further restrictions on the exercise of the\nrights granted or affirmed under this License. For example, you may\nnot impose a license fee, royalty, or other charge for exercise of\nrights granted under this License, and you may not initiate litigation\n(including a cross-claim or counterclaim in a lawsuit) alleging that\nany patent claim is infringed by making, using, selling, offering for\nsale, or importing the Program or any portion of it.\n\n 11. Patents.\n\n A \"contributor\" is a copyright holder who authorizes use under this\nLicense of the Program or a work on which the Program is based. The\nwork thus licensed is called the contributor's \"contributor version\".\n\n A contributor's \"essential patent claims\" are all patent claims\nowned or controlled by the contributor, whether already acquired or\nhereafter acquired, that would be infringed by some manner, permitted\nby this License, of making, using, or selling its contributor version,\nbut do not include claims that would be infringed only as a\nconsequence of further modification of the contributor version. For\npurposes of this definition, \"control\" includes the right to grant\npatent sublicenses in a manner consistent with the requirements of\nthis License.\n\n Each contributor grants you a non-exclusive, worldwide, royalty-free\npatent license under the contributor's essential patent claims, to\nmake, use, sell, offer for sale, import and otherwise run, modify and\npropagate the contents of its contributor version.\n\n In the following three paragraphs, a \"patent license\" is any express\nagreement or commitment, however denominated, not to enforce a patent\n(such as an express permission to practice a patent or covenant not to\nsue for patent infringement). To \"grant\" such a patent license to a\nparty means to make such an agreement or commitment not to enforce a\npatent against the party.\n\n If you convey a covered work, knowingly relying on a patent license,\nand the Corresponding Source of the work is not available for anyone\nto copy, free of charge and under the terms of this License, through a\npublicly available network server or other readily accessible means,\nthen you must either (1) cause the Corresponding Source to be so\navailable, or (2) arrange to deprive yourself of the benefit of the\npatent license for this particular work, or (3) arrange, in a manner\nconsistent with the requirements of this License, to extend the patent\nlicense to downstream recipients. \"Knowingly relying\" means you have\nactual knowledge that, but for the patent license, your conveying the\ncovered work in a country, or your recipient's use of the covered work\nin a country, would infringe one or more identifiable patents in that\ncountry that you have reason to believe are valid.\n\n If, pursuant to or in connection with a single transaction or\narrangement, you convey, or propagate by procuring conveyance of, a\ncovered work, and grant a patent license to some of the parties\nreceiving the covered work authorizing them to use, propagate, modify\nor convey a specific copy of the covered work, then the patent license\nyou grant is automatically extended to all recipients of the covered\nwork and works based on it.\n\n A patent license is \"discriminatory\" if it does not include within\nthe scope of its coverage, prohibits the exercise of, or is\nconditioned on the non-exercise of one or more of the rights that are\nspecifically granted under this License. You may not convey a covered\nwork if you are a party to an arrangement with a third party that is\nin the business of distributing software, under which you make payment\nto the third party based on the extent of your activity of conveying\nthe work, and under which the third party grants, to any of the\nparties who would receive the covered work from you, a discriminatory\npatent license (a) in connection with copies of the covered work\nconveyed by you (or copies made from those copies), or (b) primarily\nfor and in connection with specific products or compilations that\ncontain the covered work, unless you entered into that arrangement,\nor that patent license was granted, prior to 28 March 2007.\n\n Nothing in this License shall be construed as excluding or limiting\nany implied license or other defenses to infringement that may\notherwise be available to you under applicable patent law.\n\n 12. No Surrender of Others' Freedom.\n\n If conditions are imposed on you (whether by court order, agreement or\notherwise) that contradict the conditions of this License, they do not\nexcuse you from the conditions of this License. If you cannot convey a\ncovered work so as to satisfy simultaneously your obligations under this\nLicense and any other pertinent obligations, then as a consequence you may\nnot convey it at all. For example, if you agree to terms that obligate you\nto collect a royalty for further conveying from those to whom you convey\nthe Program, the only way you could satisfy both those terms and this\nLicense would be to refrain entirely from conveying the Program.\n\n 13. Use with the GNU Affero General Public License.\n\n Notwithstanding any other provision of this License, you have\npermission to link or combine any covered work with a work licensed\nunder version 3 of the GNU Affero General Public License into a single\ncombined work, and to convey the resulting work. The terms of this\nLicense will continue to apply to the part which is the covered work,\nbut the special requirements of the GNU Affero General Public License,\nsection 13, concerning interaction through a network will apply to the\ncombination as such.\n\n 14. Revised Versions of this License.\n\n The Free Software Foundation may publish revised and/or new versions of\nthe GNU General Public License from time to time. Such new versions will\nbe similar in spirit to the present version, but may differ in detail to\naddress new problems or concerns.\n\n Each version is given a distinguishing version number. If the\nProgram specifies that a certain numbered version of the GNU General\nPublic License \"or any later version\" applies to it, you have the\noption of following the terms and conditions either of that numbered\nversion or of any later version published by the Free Software\nFoundation. If the Program does not specify a version number of the\nGNU General Public License, you may choose any version ever published\nby the Free Software Foundation.\n\n If the Program specifies that a proxy can decide which future\nversions of the GNU General Public License can be used, that proxy's\npublic statement of acceptance of a version permanently authorizes you\nto choose that version for the Program.\n\n Later license versions may give you additional or different\npermissions. However, no additional obligations are imposed on any\nauthor or copyright holder as a result of your choosing to follow a\nlater version.\n\n 15. Disclaimer of Warranty.\n\n THERE IS NO WARRANTY FOR THE PROGRAM, TO THE EXTENT PERMITTED BY\nAPPLICABLE LAW. EXCEPT WHEN OTHERWISE STATED IN WRITING THE COPYRIGHT\nHOLDERS AND/OR OTHER PARTIES PROVIDE THE PROGRAM \"AS IS\" WITHOUT WARRANTY\nOF ANY KIND, EITHER EXPRESSED OR IMPLIED, INCLUDING, BUT NOT LIMITED TO,\nTHE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR\nPURPOSE. THE ENTIRE RISK AS TO THE QUALITY AND PERFORMANCE OF THE PROGRAM\nIS WITH YOU. SHOULD THE PROGRAM PROVE DEFECTIVE, YOU ASSUME THE COST OF\nALL NECESSARY SERVICING, REPAIR OR CORRECTION.\n\n 16. Limitation of Liability.\n\n IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING\nWILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MODIFIES AND/OR CONVEYS\nTHE PROGRAM AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES, INCLUDING ANY\nGENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING OUT OF THE\nUSE OR INABILITY TO USE THE PROGRAM (INCLUDING BUT NOT LIMITED TO LOSS OF\nDATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY YOU OR THIRD\nPARTIES OR A FAILURE OF THE PROGRAM TO OPERATE WITH ANY OTHER PROGRAMS),\nEVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE POSSIBILITY OF\nSUCH DAMAGES.\n\n 17. Interpretation of Sections 15 and 16.\n\n If the disclaimer of warranty and limitation of liability provided\nabove cannot be given local legal effect according to their terms,\nreviewing courts shall apply local law that most closely approximates\nan absolute waiver of all civil liability in connection with the\nProgram, unless a warranty or assumption of liability accompanies a\ncopy of the Program in return for a fee.\n\n END OF TERMS AND CONDITIONS\n\n How to Apply These Terms to Your New Programs\n\n If you develop a new program, and you want it to be of the greatest\npossible use to the public, the best way to achieve this is to make it\nfree software which everyone can redistribute and change under these terms.\n\n To do so, attach the following notices to the program. It is safest\nto attach them to the start of each source file to most effectively\nstate the exclusion of warranty; and each file should have at least\nthe \"copyright\" line and a pointer to where the full notice is found.\n\n \n Copyright (C) \n\n This program is free software: you can redistribute it and/or modify\n it under the terms of the GNU General Public License as published by\n the Free Software Foundation, either version 3 of the License, or\n (at your option) any later version.\n\n This program is distributed in the hope that it will be useful,\n but WITHOUT ANY WARRANTY; without even the implied warranty of\n MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the\n GNU General Public License for more details.\n\n You should have received a copy of the GNU General Public License\n along with this program. If not, see .\n\nAlso add information on how to contact you by electronic and paper mail.\n\n If the program does terminal interaction, make it output a short\nnotice like this when it starts in an interactive mode:\n\n Copyright (C) \n This program comes with ABSOLUTELY NO WARRANTY; for details type `show w'.\n This is free software, and you are welcome to redistribute it\n under certain conditions; type `show c' for details.\n\nThe hypothetical commands `show w' and `show c' should show the appropriate\nparts of the General Public License. Of course, your program's commands\nmight be different; for a GUI interface, you would use an \"about box\".\n\n You should also get your employer (if you work as a programmer) or school,\nif any, to sign a \"copyright disclaimer\" for the program, if necessary.\nFor more information on this, and how to apply and follow the GNU GPL, see\n.\n\n The GNU General Public License does not permit incorporating your program\ninto proprietary programs. If your program is a subroutine library, you\nmay consider it more useful to permit linking proprietary applications with\nthe library. If this is what you want to do, use the GNU Lesser General\nPublic License instead of this License. But first, please read\n.\n"
},
{
"path": "czsl/README.md",
"content": "\n# BatchFormer for Compositional Zero-Shot Learning\n\n\n## Setup \n\n1. Clone the repo and \n\n```\ncd czsl\n```\n\n2. We recommend using Anaconda for environment setup. To create the environment and activate it, please run:\n```\n conda env create --file environment.yml\n conda activate czsl\n```\n\n4. Go to the cloned repo and open a terminal. Download the datasets and embeddings, specifying the desired path (e.g. `DATA_ROOT` in the example):\n```\n bash ./utils/download_data.sh DATA_ROOT\n mkdir logs\n```\n\n## Training\n**Closed World.** To train a model, the command is simply:\n```\n python train.py --config CONFIG_FILE\n```\nwhere `CONFIG_FILE` is the path to the configuration file of the model. \nThe folder `configs` contains configuration files for all methods, i.e. CGE in `configs/cge`, CompCos in `configs/compcos`, and the other methods in `configs/baselines`. \n\nTo run CGE on MitStates, the command is just:\n```\n python train.py --config configs/cge/mit.yml --add_bt 1\n```\nOn UT-Zappos, the command is:\n```\n python train.py --config configs/cge/utzappos.yml --add_bt 1\n```\n\n## Test\n \n\n**Closed World.** To test a model, the code is simple:\n```\n python test.py --logpath LOG_DIR\n```\nwhere `LOG_DIR` is the directory containing the logs of a model.\n\n\n## References\nIf you use this code, please cite\n```\n@inproceedings{hou2022batch,\n title={BatchFormer: Learning to Explore Sample Relationships for Robust Representation Learning},\n author={Hou, Zhi and Yu, Baosheng and Tao, Dacheng},\n booktitle={CVPR},\n year={2022}\n}\n```\n\n**Note**: Code are based on czsl:\n```\n@inproceedings{naeem2021learning,\n title={Learning Graph Embeddings for Compositional Zero-shot Learning},\n author={Naeem, MF and Xian, Y and Tombari, F and Akata, Zeynep},\n booktitle={34th IEEE Conference on Computer Vision and Pattern Recognition},\n year={2021},\n organization={IEEE}\n}\n```\n"
},
{
"path": "czsl/configs/baselines/cgqa/aopp.yml",
"content": "---\nexperiment:\n name: aopp/cgqa/\ndataset:\n data_dir: cgqa\n dataset: cgqa\n splitname: compositional-split-natural\nmodel_params:\n model: attributeop\n emb_dim: 300\n emb_init: glove\n eval_type: dist_fast\n image_extractor: resnet18\n train_only: true\n static_inp: false\n composition: add\nloss:\n lambda_aux: 1.0\n lambda_comm: 1.0\ntraining:\n batch_size: 128\n eval_val_every: 2\n load: \n lr: 5.0e-05\n lrg: 0.001\n margin: 0.5\n max_epochs: 1000\n norm_family: imagenet\n save_every: 10000\n test_batch_size: 32\n test_set: val\n topk: 1\n wd: 5.0e-05\n workers: 8\n update_features: false\n freeze_features: false\nextra:\n lambda_attr: 0\n lambda_obj: 0\n lambda_sub: 0\n graph: false\n hardk: null"
},
{
"path": "czsl/configs/baselines/cgqa/le+.yml",
"content": "---\nexperiment:\n name: labelembed/cgqa/\ndataset:\n data_dir: cgqa\n dataset: cgqa\n splitname: compositional-split-natural\nmodel_params:\n model: labelembed+\n dropout: false\n norm: false\n nlayers: 2\n fc_emb: 512,512\n emb_dim: 300\n emb_init: glove\n eval_type: dist_fast\n image_extractor: resnet18\n train_only: true\n static_inp: false\n composition: add\nloss:\n lambda_ce: 1.0\n lambda_trip_smart: 0\ntraining:\n batch_size: 512\n eval_val_every: 2\n load: \n lr: 5.0e-05\n margin: 0.5\n max_epochs: 2000\n norm_family: imagenet\n save_every: 10000\n test_batch_size: 32\n test_set: val\n topk: 1\n wd: 5.0e-05\n workers: 8\n update_features: false\n freeze_features: false"
},
{
"path": "czsl/configs/baselines/cgqa/symnet.yml",
"content": "---\nexperiment:\n name: symnet/cgqa/\ndataset:\n data_dir: cgqa\n dataset: cgqa\n splitname: compositional-split-natural\nmodel_params:\n model: symnet\n dropout: true\n norm: true\n nlayers: 3\n fc_emb: 768,1024,1200\n emb_dim: 300\n emb_init: glove\n image_extractor: resnet18\nloss:\n lambda_cls_attr: 1.0\n lambda_cls_obj: 1.0\n lambda_trip: 0.5\n lambda_sym: 0.01\n lambda_axiom: 0.03\ntraining:\n batch_size: 512\n eval_val_every: 2\n load: \n lr: 5.0e-05\n lrg: 0.001\n margin: 0.5\n max_epochs: 2000\n norm_family: imagenet\n save_every: 10000\n test_batch_size: 32\n test_set: val\n topk: 1\n wd: 5.0e-05\n workers: 8\n update_features: false\n freeze_features: false"
},
{
"path": "czsl/configs/baselines/cgqa/tmn.yml",
"content": "---\nexperiment:\n name: tmn/cgqa/\ndataset:\n data_dir: cgqa\n dataset: cgqa\n splitname: compositional-split-natural\nmodel_params:\n model: tmn\n emb_init: glove\n image_extractor: resnet18\n num_negs: 600\n embed_rank: 64\n emb_dim: 16\n nmods: 24\ntraining:\n batch_size: 256\n eval_val_every: 2\n load: \n lr: 0.0001\n lrg: 0.01\n margin: 0.5\n max_epochs: 100\n norm_family: imagenet\n save_every: 10000\n test_batch_size: 32\n test_set: val\n topk: 1\n wd: 5.0e-05\n workers: 8\n update_features: false\n freeze_features: false"
},
{
"path": "czsl/configs/baselines/mit/aopp.yml",
"content": "---\nexperiment:\n name: aopp/mitstates/\ndataset:\n data_dir: mit-states\n dataset: mitstates\n splitname: compositional-split-natural\nmodel_params:\n model: attributeop\n emb_dim: 300\n emb_init: glove\n eval_type: dist_fast\n image_extractor: resnet18\n train_only: true\n static_inp: false\n composition: add\nloss:\n lambda_aux: 1000.0\n lambda_comm: 1.0\ntraining:\n batch_size: 512\n eval_val_every: 2\n load: \n lr: 5.0e-05\n lrg: 0.001\n margin: 0.5\n max_epochs: 1000\n norm_family: imagenet\n save_every: 10000\n test_batch_size: 32\n test_set: val\n topk: 1\n wd: 5.0e-05\n workers: 8\n update_features: false\n freeze_features: false\nextra:\n lambda_attr: 0\n lambda_obj: 0\n lambda_sub: 0\n graph: false\n hardk: null"
},
{
"path": "czsl/configs/baselines/mit/le+.yml",
"content": "---\nexperiment:\n name: labelembed/mitstates/\ndataset:\n data_dir: mit-states\n dataset: mitstates\n splitname: compositional-split-natural\nmodel_params:\n model: labelembed+\n dropout: false\n norm: false\n nlayers: 2\n fc_emb: 512,512\n emb_dim: 300\n emb_init: glove\n eval_type: dist_fast\n image_extractor: resnet18\n train_only: true\n static_inp: false\n composition: add\ntraining:\n batch_size: 512\n eval_val_every: 2\n load: \n lr: 5.0e-05\n margin: 0.5\n max_epochs: 2000\n norm_family: imagenet\n save_every: 10000\n test_batch_size: 32\n test_set: val\n topk: 1\n wd: 5.0e-05\n workers: 8\n update_features: false\n freeze_features: false"
},
{
"path": "czsl/configs/baselines/mit/symnet.yml",
"content": "---\nexperiment:\n name: symnet/mit-states/\ndataset:\n data_dir: mit-states\n dataset: mitstates\n splitname: compositional-split-natural\nmodel_params:\n model: symnet\n dropout: true\n norm: true\n nlayers: 3\n fc_emb: 768,1024,1200\n emb_dim: 300\n emb_init: glove\n image_extractor: resnet18\n obj_fixed: True\nloss:\n lambda_cls_attr: 1.0\n lambda_cls_obj: 0.1\n lambda_trip: 0.5\n lambda_sym: 0.01\n lambda_axiom: 0.03\ntraining:\n batch_size: 450\n eval_val_every: 2\n load: \n lr: 5.0e-05\n lrg: 5.0e-06\n margin: 0.5\n max_epochs: 2000\n norm_family: imagenet\n save_every: 10000\n test_batch_size: 32\n test_set: val\n topk: 1\n wd: 5.0e-05\n workers: 8\n update_features: false\n freeze_features: false"
},
{
"path": "czsl/configs/baselines/mit/tmn.yml",
"content": "---\nexperiment:\n name: tmn/mitstates/\ndataset:\n data_dir: mit-states\n dataset: mitstates\n splitname: compositional-split-natural\nmodel_params:\n model: tmn\n emb_init: glove\n image_extractor: resnet18\n num_negs: 600\n embed_rank: 64\n emb_dim: 16\n nmods: 24\ntraining:\n batch_size: 256\n eval_val_every: 2\n load: \n lr: 0.0001\n lrg: 1.0e-06\n margin: 0.5\n max_epochs: 100\n norm_family: imagenet\n save_every: 10000\n test_batch_size: 32\n test_set: val\n topk: 1\n wd: 5.0e-05\n workers: 8\n update_features: true\n freeze_features: false"
},
{
"path": "czsl/configs/baselines/utzppos/aopp.yml",
"content": "---\nexperiment:\n name: aopp/utzappos/\ndataset:\n data_dir: ut-zap50k\n dataset: utzappos\n splitname: compositional-split-natural\nmodel_params:\n model: attributeop\n emb_dim: 300\n emb_init: glove\n eval_type: dist_fast\n image_extractor: resnet18\n train_only: true\n static_inp: false\n composition: add\nloss:\n lambda_aux: 1.0\n lambda_comm: 1.0\ntraining:\n batch_size: 512\n eval_val_every: 2\n load: \n lr: 5.0e-05\n lrg: 0.001\n margin: 0.5\n max_epochs: 1000\n norm_family: imagenet\n save_every: 10000\n test_batch_size: 32\n test_set: val\n topk: 1\n wd: 5.0e-05\n workers: 8\n update_features: false\n freeze_features: false\nextra:\n lambda_attr: 0\n lambda_obj: 0\n lambda_sub: 0\n graph: false\n hardk: null"
},
{
"path": "czsl/configs/baselines/utzppos/le+.yml",
"content": "---\nexperiment:\n name: labelembed/utzappos/\ndataset:\n data_dir: ut-zap50k\n dataset: utzappos\n splitname: compositional-split-natural\nmodel_params:\n model: labelembed+\n dropout: false\n norm: false\n nlayers: 2\n fc_emb: 512,512\n emb_dim: 300\n emb_init: glove\n eval_type: dist_fast\n image_extractor: resnet18\n train_only: true\n static_inp: false\n composition: add\ntraining:\n batch_size: 512\n eval_val_every: 2\n load: \n lr: 5.0e-05\n margin: 0.5\n max_epochs: 2000\n norm_family: imagenet\n save_every: 10000\n test_batch_size: 32\n test_set: val\n topk: 1\n wd: 5.0e-05\n workers: 8\n update_features: false\n freeze_features: false"
},
{
"path": "czsl/configs/baselines/utzppos/symnet.yml",
"content": "---\nexperiment:\n name: symnet/utzappos/\ndataset:\n data_dir: ut-zap50k\n dataset: utzappos\n splitname: compositional-split-natural\nmodel_params:\n model: symnet\n dropout: true\n norm: true\n nlayers: 3\n fc_emb: 768,1024,1200\n emb_dim: 300\n emb_init: fasttext\n image_extractor: resnet18\nloss:\n lambda_cls_attr: 1.0\n lambda_cls_obj: 1.0\n lambda_trip: 0.5\n lambda_sym: 0.01\n lambda_axiom: 0.03\ntraining:\n batch_size: 512\n eval_val_every: 2\n load: \n lr: 5.0e-05\n lrg: 0.001\n margin: 0.5\n max_epochs: 2000\n norm_family: imagenet\n save_every: 10000\n test_batch_size: 32\n test_set: val\n topk: 1\n wd: 5.0e-05\n workers: 8\n update_features: false\n freeze_features: false"
},
{
"path": "czsl/configs/baselines/utzppos/tmn.yml",
"content": "---\nexperiment:\n name: tmn/utzappos/\ndataset:\n data_dir: ut-zap50k\n dataset: utzappos\n splitname: compositional-split-natural\nmodel_params:\n model: tmn\n emb_init: word2vec\n emb_dim: 300\n image_extractor: resnet18\n train_only: true\n static_inp: true\n num_negs: 600\n embed_rank: 64\n nmods: 24\ntraining:\n batch_size: 256\n eval_val_every: 10\n load:\n lr: 0.0001\n lrg: 1.0e-06\n margin: 0.5\n max_epochs: 100\n norm_family: imagenet\n save_every: 10000\n test_batch_size: 32\n test_set: val\n topk: 1\n wd: 5.0e-05\n workers: 8\n update_features: false\n freeze_features: false\n(czsl) mmancini68@EML-Jupi"
},
{
"path": "czsl/configs/cge/cgqa.yml",
"content": "experiment:\n name: graphembed/cgqa\ndataset:\n data_dir: cgqa\n dataset: cgqa\n splitname: compositional-split-natural\nmodel_params:\n model: graphfull\n dropout: true\n norm: true\n nlayers:\n fc_emb: 768,1024,1200\n gr_emb: d4096,d\n emb_dim: 512\n emb_init: null\n graph_init: utils/cgqa-graph.t7\n eval_type: otherdist\n image_extractor: resnet18\n train_only: true\n static_inp: false\ngcnii:\n hidden_dim: 1024\n gcn_type: gcn\n gcn_nlayers: 10\ntraining:\n batch_size: 256\n eval_val_every: 2\n load:\n lr: 5.0e-05\n wd: 5.0e-05\n lrg: 5.0e-06\n margin: 2\n max_epochs: 200\n norm_family: imagenet\n save_every: 10000\n test_batch_size: 32\n test_set: val\n topk: 1\n workers: 8\n update_features: true\n freeze_featues: false\n"
},
{
"path": "czsl/configs/cge/mit.yml",
"content": "---\nexperiment:\n name: graphembed/mitstates/base\ndataset:\n data_dir: mit-states\n dataset: mitstates\n splitname: compositional-split-natural\nmodel_params:\n model: graphfull\n dropout: true\n norm: true\n nlayers: \n gr_emb: d4096,d\n emb_dim: 512\n graph_init: utils/mitstates-graph.t7\n image_extractor: resnet18\n train_only: true\n static_inp: false\ntraining:\n batch_size: 128\n eval_val_every: 2\n load: \n lr: 5.0e-05\n wd: 5.0e-05\n lrg: 5.0e-6\n margin: 0.5\n max_epochs: 100\n norm_family: imagenet\n save_every: 10000\n test_batch_size: 32\n test_set: val\n topk: 1\n workers: 8\n update_features: true\n freeze_features: false\n\n"
},
{
"path": "czsl/configs/cge/utzappos.yml",
"content": "---\nexperiment:\n name: graphembed/utzappos/base\ndataset:\n data_dir: ut-zap50k\n dataset: utzappos\n splitname: compositional-split-natural\nmodel_params:\n model: graphfull\n dropout: true\n norm: true\n nlayers: 1\n fc_emb: 768,1024,1200\n gr_emb: d4096,d\n emb_dim: 300\n emb_init: null\n graph_init: utils/utzappos-graph.t7\n eval_type: otherdist\n image_extractor: resnet18\n train_only: true\n static_inp: false\n composition: mlp_add\ngcnii:\n hidden_dim: 1024\n gcn_type: gcn\n gcn_nlayers: 10\ntraining:\n batch_size: 128\n eval_val_every: 2\n load: \n lr: 5.0e-05\n wd: 5.0e-05\n lrg: 5.0e-6\n margin: 0.5\n max_epochs: 100\n norm_family: imagenet\n save_every: 10000\n test_batch_size: 32\n test_set: val\n topk: 1\n workers: 8\n update_features: true\n freeze_features: false\nextra:\n lambda_attr: 0\n lambda_obj: 0\n lambda_sub: 0\n graph: false\n hardk: null\n cutmix: false\n cutmix_prob: 1.0\n beta: 1.0"
},
{
"path": "czsl/configs/compcos/cgqa/compcos.yml",
"content": "---\nexperiment:\n name: compcos/cgqa\ndataset:\n data_dir: cgqa\n dataset: cgqa\n splitname: compositional-split-natural\n open_world: true\nmodel_params:\n model: compcos\n dropout: true\n norm: true\n nlayers: 2\n relu: false\n fc_emb: 768,1024,1200\n emb_dim: 300\n emb_init: word2vec\n image_extractor: resnet18\n train_only: false\n static_inp: false\ntraining:\n batch_size: 128\n load:\n lr: 5.0e-05\n lrg: 0.001\n margin: 0.4\n cosine_scale: 100\n max_epochs: 300\n norm_family: imagenet\n save_every: 10000\n test_batch_size: 64\n test_set: val\n topk: 1\n wd: 5.0e-05\n workers: 8\n update_features: false\n freeze_features: false\n epoch_max_margin: 15\n"
},
{
"path": "czsl/configs/compcos/cgqa/compcos_cw.yml",
"content": "---\nexperiment:\n name: compcos_cw/cgqa/\ndataset:\n data_dir: cgqa\n dataset: cgqa\n splitname: compositional-split-natural\nmodel_params:\n model: compcos\n dropout: true\n norm: true\n nlayers: 2\n relu: false\n fc_emb: 768,1024,1200\n emb_dim: 300\n emb_init: word2vec\n image_extractor: resnet18\n train_only: true\n static_inp: false\ntraining:\n batch_size: 128\n load:\n lr: 5.0e-05\n lrg: 0.001\n margin: 0.4\n cosine_scale: 100\n max_epochs: 300\n norm_family: imagenet\n save_every: 10000\n test_batch_size: 64\n test_set: val\n topk: 1\n wd: 5.0e-05\n workers: 8\n update_features: false\n freeze_features: false\n epoch_max_margin: 15\n"
},
{
"path": "czsl/configs/compcos/mit/compcos.yml",
"content": "---\nexperiment:\n name: compcos/mitstates\ndataset:\n data_dir: mit-states\n dataset: mitstates\n splitname: compositional-split-natural\n open_world: true\nmodel_params:\n model: compcos\n dropout: true\n norm: true\n nlayers: 2\n relu: false\n fc_emb: 768,1024,1200\n emb_dim: 600\n emb_init: ft+w2v\n image_extractor: resnet18\n train_only: false\n static_inp: false\ntraining:\n batch_size: 128\n load:\n lr: 5.0e-05\n lrg: 0.001\n margin: 0.4\n cosine_scale: 20\n max_epochs: 300\n norm_family: imagenet\n save_every: 10000\n test_batch_size: 64\n test_set: val\n topk: 1\n wd: 5.0e-05\n workers: 8\n update_features: false\n freeze_features: false\n epoch_max_margin: 15\n"
},
{
"path": "czsl/configs/compcos/mit/compcos_cw.yml",
"content": "---\nexperiment:\n name: compcos_cw/mitstates\ndataset:\n data_dir: mit-states\n dataset: mitstates\n splitname: compositional-split-natural\nmodel_params:\n model: compcos\n dropout: true\n norm: true\n nlayers: 2\n relu: false\n fc_emb: 768,1024,1200\n emb_dim: 600\n emb_init: ft+w2v\n image_extractor: resnet18\n train_only: true\n static_inp: false\ntraining:\n batch_size: 128\n load:\n lr: 5.0e-05\n lrg: 0.001\n margin: 0.4\n cosine_scale: 20\n max_epochs: 300\n norm_family: imagenet\n save_every: 10000\n test_batch_size: 64\n test_set: val\n topk: 1\n wd: 5.0e-05\n workers: 8\n update_features: false\n freeze_features: false\n epoch_max_margin: 15\n"
},
{
"path": "czsl/configs/compcos/utzppos/compcos.yml",
"content": "---\nexperiment:\n name: compcos/utzappos\ndataset:\n data_dir: ut-zap50k\n dataset: utzappos\n splitname: compositional-split-natural\n open_world: true\nmodel_params:\n model: compcos\n dropout: true\n norm: true\n nlayers: 2\n relu: false\n fc_emb: 768,1024\n emb_dim: 300\n emb_init: word2vec\n image_extractor: resnet18\n train_only: false\n static_inp: false\ntraining:\n batch_size: 128\n load:\n lr: 5.0e-05\n lrg: 0.001\n margin: 1.0\n cosine_scale: 50\n max_epochs: 300\n norm_family: imagenet\n save_every: 10000\n test_batch_size: 64\n test_set: val\n topk: 1\n wd: 5.0e-05\n workers: 8\n update_features: false\n freeze_features: false\n epoch_max_margin: 100"
},
{
"path": "czsl/configs/compcos/utzppos/compcos_cw.yml",
"content": "---\nexperiment:\n name: compcos/utzappos\ndataset:\n data_dir: ut-zap50k\n dataset: utzappos\n splitname: compositional-split-natural\nmodel_params:\n model: compcos\n dropout: true\n norm: true\n nlayers: 2\n relu: false\n fc_emb: 768,1024\n emb_dim: 300\n emb_init: word2vec\n image_extractor: resnet18\n train_only: true\n static_inp: false\ntraining:\n batch_size: 128\n load:\n lr: 5.0e-05\n lrg: 0.001\n margin: 1.0\n cosine_scale: 50\n max_epochs: 300\n norm_family: imagenet\n save_every: 10000\n test_batch_size: 64\n test_set: val\n topk: 1\n wd: 5.0e-05\n workers: 8\n update_features: false\n freeze_features: false\n epoch_max_margin: 100"
},
{
"path": "czsl/data/__init__.py",
"content": ""
},
{
"path": "czsl/data/dataset.py",
"content": "#external libs\nimport numpy as np\nfrom tqdm import tqdm\nfrom PIL import Image\nimport os\nimport random\nfrom os.path import join as ospj\nfrom glob import glob \n#torch libs\nfrom torch.utils.data import Dataset\nimport torch\nimport torchvision.transforms as transforms\n#local libs\nfrom utils.utils import get_norm_values, chunks\nfrom models.image_extractor import get_image_extractor\nfrom itertools import product\n\ndevice = 'cuda' if torch.cuda.is_available() else 'cpu'\n\nclass ImageLoader:\n def __init__(self, root):\n self.root_dir = root\n\n def __call__(self, img):\n img = Image.open(ospj(self.root_dir,img)).convert('RGB') #We don't want alpha\n return img\n\n\ndef dataset_transform(phase, norm_family = 'imagenet'):\n '''\n Inputs\n phase: String controlling which set of transforms to use\n norm_family: String controlling which normaliztion values to use\n \n Returns\n transform: A list of pytorch transforms\n '''\n mean, std = get_norm_values(norm_family=norm_family)\n\n if phase == 'train':\n transform = transforms.Compose([\n transforms.RandomResizedCrop(224),\n transforms.RandomHorizontalFlip(),\n transforms.ToTensor(),\n transforms.Normalize(mean, std)\n ])\n\n elif phase == 'val' or phase == 'test':\n transform = transforms.Compose([\n transforms.Resize(256),\n transforms.CenterCrop(224),\n transforms.ToTensor(),\n transforms.Normalize(mean, std)\n ])\n elif phase == 'all':\n transform = transforms.Compose([\n transforms.Resize(256),\n transforms.CenterCrop(224),\n transforms.ToTensor(),\n transforms.Normalize(mean, std)\n ])\n else:\n raise ValueError('Invalid transform')\n\n return transform\n\ndef filter_data(all_data, pairs_gt, topk = 5):\n '''\n Helper function to clean data\n '''\n valid_files = []\n with open('/home/ubuntu/workspace/top'+str(topk)+'.txt') as f:\n for line in f:\n valid_files.append(line.strip())\n\n data, pairs, attr, obj = [], [], [], []\n for current in all_data:\n if current[0] in valid_files:\n data.append(current)\n pairs.append((current[1],current[2]))\n attr.append(current[1])\n obj.append(current[2])\n \n counter = 0\n for current in pairs_gt:\n if current in pairs:\n counter+=1\n print('Matches ', counter, ' out of ', len(pairs_gt))\n print('Samples ', len(data), ' out of ', len(all_data))\n return data, sorted(list(set(pairs))), sorted(list(set(attr))), sorted(list(set(obj)))\n\n# Dataset class now\n\nclass CompositionDataset(Dataset):\n '''\n Inputs\n root: String of base dir of dataset\n phase: String train, val, test\n split: String dataset split\n subset: Boolean if true uses a subset of train at each epoch\n num_negs: Int, numbers of negative pairs per batch\n pair_dropout: Percentage of pairs to leave in current epoch\n '''\n def __init__(\n self,\n root,\n phase,\n split = 'compositional-split',\n model = 'resnet18',\n norm_family = 'imagenet',\n subset = False,\n num_negs = 1,\n pair_dropout = 0.0,\n update_features = False,\n return_images = False,\n train_only = False,\n open_world=False\n ):\n self.root = root\n self.phase = phase\n self.split = split\n self.num_negs = num_negs\n self.pair_dropout = pair_dropout\n self.norm_family = norm_family\n self.return_images = return_images\n self.update_features = update_features\n self.feat_dim = 512 if 'resnet18' in model else 2048 # todo, unify this with models\n self.open_world = open_world\n\n self.attrs, self.objs, self.pairs, self.train_pairs, \\\n self.val_pairs, self.test_pairs = self.parse_split()\n self.train_data, self.val_data, self.test_data = self.get_split_info()\n self.full_pairs = list(product(self.attrs,self.objs))\n \n # Clean only was here\n self.obj2idx = {obj: idx for idx, obj in enumerate(self.objs)}\n self.attr2idx = {attr : idx for idx, attr in enumerate(self.attrs)}\n if self.open_world:\n self.pairs = self.full_pairs\n\n self.all_pair2idx = {pair: idx for idx, pair in enumerate(self.pairs)}\n\n if train_only and self.phase == 'train':\n print('Using only train pairs')\n self.pair2idx = {pair : idx for idx, pair in enumerate(self.train_pairs)}\n else:\n print('Using all pairs')\n self.pair2idx = {pair : idx for idx, pair in enumerate(self.pairs)}\n \n if self.phase == 'train':\n self.data = self.train_data\n elif self.phase == 'val':\n self.data = self.val_data\n elif self.phase == 'test':\n self.data = self.test_data\n elif self.phase == 'all':\n print('Using all data')\n self.data = self.train_data + self.val_data + self.test_data\n else:\n raise ValueError('Invalid training phase')\n \n self.all_data = self.train_data + self.val_data + self.test_data\n print('Dataset loaded')\n print('Train pairs: {}, Validation pairs: {}, Test Pairs: {}'.format(\n len(self.train_pairs), len(self.val_pairs), len(self.test_pairs)))\n print('Train images: {}, Validation images: {}, Test images: {}'.format(\n len(self.train_data), len(self.val_data), len(self.test_data)))\n\n if subset:\n ind = np.arange(len(self.data))\n ind = ind[::len(ind) // 1000]\n self.data = [self.data[i] for i in ind]\n\n\n # Keeping a list of all pairs that occur with each object\n self.obj_affordance = {}\n self.train_obj_affordance = {}\n for _obj in self.objs:\n candidates = [attr for (_, attr, obj) in self.train_data+self.test_data if obj==_obj]\n self.obj_affordance[_obj] = list(set(candidates))\n\n candidates = [attr for (_, attr, obj) in self.train_data if obj==_obj]\n self.train_obj_affordance[_obj] = list(set(candidates))\n\n self.sample_indices = list(range(len(self.data)))\n self.sample_pairs = self.train_pairs\n\n # Load based on what to output\n self.transform = dataset_transform(self.phase, self.norm_family)\n self.loader = ImageLoader(ospj(self.root, 'images'))\n if not self.update_features:\n feat_file = ospj(root, model+'_featurers.t7')\n print(f'Using {model} and feature file {feat_file}')\n if not os.path.exists(feat_file):\n with torch.no_grad():\n self.generate_features(feat_file, model)\n self.phase = phase\n activation_data = torch.load(feat_file)\n self.activations = dict(\n zip(activation_data['files'], activation_data['features']))\n self.feat_dim = activation_data['features'].size(1)\n print('{} activations loaded'.format(len(self.activations)))\n\n\n def parse_split(self):\n '''\n Helper function to read splits of object atrribute pair\n Returns\n all_attrs: List of all attributes\n all_objs: List of all objects\n all_pairs: List of all combination of attrs and objs\n tr_pairs: List of train pairs of attrs and objs\n vl_pairs: List of validation pairs of attrs and objs\n ts_pairs: List of test pairs of attrs and objs\n '''\n def parse_pairs(pair_list):\n '''\n Helper function to parse each phase to object attrribute vectors\n Inputs\n pair_list: path to textfile\n '''\n with open(pair_list, 'r') as f:\n pairs = f.read().strip().split('\\n')\n pairs = [line.split() for line in pairs]\n pairs = list(map(tuple, pairs))\n\n attrs, objs = zip(*pairs)\n return attrs, objs, pairs\n\n tr_attrs, tr_objs, tr_pairs = parse_pairs(\n ospj(self.root, self.split, 'train_pairs.txt')\n )\n vl_attrs, vl_objs, vl_pairs = parse_pairs(\n ospj(self.root, self.split, 'val_pairs.txt')\n )\n ts_attrs, ts_objs, ts_pairs = parse_pairs(\n ospj(self.root, self.split, 'test_pairs.txt')\n )\n \n #now we compose all objs, attrs and pairs\n all_attrs, all_objs = sorted(\n list(set(tr_attrs + vl_attrs + ts_attrs))), sorted(\n list(set(tr_objs + vl_objs + ts_objs)))\n all_pairs = sorted(list(set(tr_pairs + vl_pairs + ts_pairs)))\n\n return all_attrs, all_objs, all_pairs, tr_pairs, vl_pairs, ts_pairs\n\n def get_split_info(self):\n '''\n Helper method to read image, attrs, objs samples\n\n Returns\n train_data, val_data, test_data: List of tuple of image, attrs, obj\n '''\n data = torch.load(ospj(self.root, 'metadata_{}.t7'.format(self.split)))\n\n train_data, val_data, test_data = [], [], []\n\n for instance in data:\n image, attr, obj, settype = instance['image'], instance['attr'], \\\n instance['obj'], instance['set']\n curr_data = [image, attr, obj]\n\n if attr == 'NA' or (attr, obj) not in self.pairs or settype == 'NA':\n # Skip incomplete pairs, unknown pairs and unknown set\n continue\n\n if settype == 'train':\n train_data.append(curr_data)\n elif settype == 'val':\n val_data.append(curr_data)\n else:\n test_data.append(curr_data)\n\n return train_data, val_data, test_data\n\n def get_dict_data(self, data, pairs):\n data_dict = {}\n for current in pairs:\n data_dict[current] = []\n\n for current in data:\n image, attr, obj = current\n data_dict[(attr, obj)].append(image)\n \n return data_dict\n\n\n def reset_dropout(self):\n ''' \n Helper function to sample new subset of data containing a subset of pairs of objs and attrs\n '''\n self.sample_indices = list(range(len(self.data)))\n self.sample_pairs = self.train_pairs\n\n # Using sampling from random instead of 2 step numpy\n n_pairs = int((1 - self.pair_dropout) * len(self.train_pairs))\n\n self.sample_pairs = random.sample(self.train_pairs, n_pairs)\n print('Sampled new subset')\n print('Using {} pairs out of {} pairs right now'.format(\n n_pairs, len(self.train_pairs)))\n\n self.sample_indices = [ i for i in range(len(self.data))\n if (self.data[i][1], self.data[i][2]) in self.sample_pairs\n ]\n print('Using {} images out of {} images right now'.format(\n len(self.sample_indices), len(self.data)))\n\n def sample_negative(self, attr, obj):\n '''\n Inputs\n attr: String of valid attribute\n obj: String of valid object\n Returns\n Tuple of a different attribute, object indexes\n '''\n new_attr, new_obj = self.sample_pairs[np.random.choice(\n len(self.sample_pairs))]\n\n while new_attr == attr and new_obj == obj:\n new_attr, new_obj = self.sample_pairs[np.random.choice(\n len(self.sample_pairs))]\n \n return (self.attr2idx[new_attr], self.obj2idx[new_obj])\n\n def sample_affordance(self, attr, obj):\n '''\n Inputs\n attr: String of valid attribute\n obj: String of valid object\n Return\n Idx of a different attribute for the same object\n '''\n new_attr = np.random.choice(self.obj_affordance[obj])\n \n while new_attr == attr:\n new_attr = np.random.choice(self.obj_affordance[obj])\n \n return self.attr2idx[new_attr]\n\n def sample_train_affordance(self, attr, obj):\n '''\n Inputs\n attr: String of valid attribute\n obj: String of valid object\n Return\n Idx of a different attribute for the same object from the training pairs\n '''\n new_attr = np.random.choice(self.train_obj_affordance[obj])\n \n while new_attr == attr:\n new_attr = np.random.choice(self.train_obj_affordance[obj])\n \n return self.attr2idx[new_attr]\n\n def generate_features(self, out_file, model):\n '''\n Inputs\n out_file: Path to save features\n model: String of extraction model\n '''\n # data = self.all_data\n data = ospj(self.root,'images')\n files_before = glob(ospj(data, '**', '*.jpg'), recursive=True)\n files_all = []\n for current in files_before:\n parts = current.split('/')\n if \"cgqa\" in self.root:\n files_all.append(parts[-1])\n else:\n files_all.append(os.path.join(parts[-2],parts[-1]))\n transform = dataset_transform('test', self.norm_family)\n feat_extractor = get_image_extractor(arch = model).eval()\n feat_extractor = feat_extractor.to(device)\n\n image_feats = []\n image_files = []\n for chunk in tqdm(\n chunks(files_all, 512), total=len(files_all) // 512, desc=f'Extracting features {model}'):\n\n files = chunk\n imgs = list(map(self.loader, files))\n imgs = list(map(transform, imgs))\n feats = feat_extractor(torch.stack(imgs, 0).to(device))\n image_feats.append(feats.data.cpu())\n image_files += files\n image_feats = torch.cat(image_feats, 0)\n print('features for %d images generated' % (len(image_files)))\n\n torch.save({'features': image_feats, 'files': image_files}, out_file)\n\n def __getitem__(self, index):\n '''\n Call for getting samples\n '''\n index = self.sample_indices[index]\n\n image, attr, obj = self.data[index]\n\n # Decide what to output\n if not self.update_features:\n img = self.activations[image]\n else:\n img = self.loader(image)\n img = self.transform(img)\n\n data = [img, self.attr2idx[attr], self.obj2idx[obj], self.pair2idx[(attr, obj)]]\n \n if self.phase == 'train':\n all_neg_attrs = []\n all_neg_objs = []\n\n for curr in range(self.num_negs):\n neg_attr, neg_obj = self.sample_negative(attr, obj) # negative for triplet lose\n all_neg_attrs.append(neg_attr)\n all_neg_objs.append(neg_obj)\n\n neg_attr, neg_obj = torch.LongTensor(all_neg_attrs), torch.LongTensor(all_neg_objs)\n \n #note here\n if len(self.train_obj_affordance[obj])>1:\n inv_attr = self.sample_train_affordance(attr, obj) # attribute for inverse regularizer\n else:\n inv_attr = (all_neg_attrs[0]) \n\n comm_attr = self.sample_affordance(inv_attr, obj) # attribute for commutative regularizer\n \n\n data += [neg_attr, neg_obj, inv_attr, comm_attr]\n\n # Return image paths if requested as the last element of the list\n if self.return_images and self.phase != 'train':\n data.append(image)\n\n return data\n \n def __len__(self):\n '''\n Call for length\n '''\n return len(self.sample_indices)\n"
},
{
"path": "czsl/environment.yml",
"content": "name: czsl\nchannels:\n - pytorch\n - defaults\ndependencies:\n - _libgcc_mutex=0.1=main\n - blas=1.0=mkl\n - ca-certificates=2021.1.19=h06a4308_0\n - certifi=2020.12.5=py38h06a4308_0\n - cudatoolkit=11.0.221=h6bb024c_0\n - freetype=2.10.4=h5ab3b9f_0\n - intel-openmp=2020.2=254\n - jpeg=9b=h024ee3a_2\n - lcms2=2.11=h396b838_0\n - ld_impl_linux-64=2.33.1=h53a641e_7\n - libedit=3.1.20191231=h14c3975_1\n - libffi=3.3=he6710b0_2\n - libgcc-ng=9.1.0=hdf63c60_0\n - libgfortran-ng=7.3.0=hdf63c60_0\n - libpng=1.6.37=hbc83047_0\n - libstdcxx-ng=9.1.0=hdf63c60_0\n - libtiff=4.1.0=h2733197_1\n - libuv=1.40.0=h7b6447c_0\n - lz4-c=1.9.3=h2531618_0\n - mkl=2020.2=256\n - mkl-service=2.3.0=py38he904b0f_0\n - mkl_fft=1.2.0=py38h23d657b_0\n - mkl_random=1.1.1=py38h0573a6f_0\n - ncurses=6.2=he6710b0_1\n - ninja=1.10.2=py38hff7bd54_0\n - numpy=1.19.2=py38h54aff64_0\n - numpy-base=1.19.2=py38hfa32c7d_0\n - olefile=0.46=py_0\n - openssl=1.1.1i=h27cfd23_0\n - pillow=8.1.0=py38he98fc37_0\n - pip=20.3.3=py38h06a4308_0\n - python=3.8.5=h7579374_1\n - pytorch=1.7.1=py3.8_cuda11.0.221_cudnn8.0.5_0\n - readline=8.1=h27cfd23_0\n - scipy=1.5.2=py38h0b6359f_0\n - setuptools=52.0.0=py38h06a4308_0\n - six=1.15.0=py38h06a4308_0\n - sqlite=3.33.0=h62c20be_0\n - tk=8.6.10=hbc83047_0\n - torchaudio=0.7.2=py38\n - torchvision=0.8.2=py38_cu110\n - typing_extensions=3.7.4.3=pyh06a4308_0\n - wheel=0.36.2=pyhd3eb1b0_0\n - xz=5.2.5=h7b6447c_0\n - yaml=0.2.5=h7b6447c_0\n - zlib=1.2.11=h7b6447c_3\n - zstd=1.4.5=h9ceee32_0\n - pip:\n - absl-py==0.11.0\n - cachetools==4.2.1\n - chardet==4.0.0\n - fasttext==0.9.2\n - gensim==3.8.3\n - google-auth==1.24.0\n - google-auth-oauthlib==0.4.2\n - grpcio==1.35.0\n - idna==2.10\n - markdown==3.3.3\n - oauthlib==3.1.0\n - protobuf==3.14.0\n - pyasn1==0.4.8\n - pyasn1-modules==0.2.8\n - pybind11==2.6.2\n - pyyaml==5.4.1\n - requests==2.25.1\n - requests-oauthlib==1.3.0\n - rsa==4.7\n - smart-open==4.1.2\n - tensorboard==2.4.1\n - tensorboard-plugin-wit==1.8.0\n - tqdm==4.56.0\n - urllib3==1.26.3\n - werkzeug==1.0.1\n\n"
},
{
"path": "czsl/flags.py",
"content": "import argparse\n\nDATA_FOLDER = \"/project/ZHIHOUDATA/CZSL\"\nparser = argparse.ArgumentParser(formatter_class=argparse.ArgumentDefaultsHelpFormatter)\n\nparser.add_argument('--config', default='configs/args.yml', help='path of the config file (training only)')\nparser.add_argument('--dataset', default='mitstates', help='mitstates|zappos')\nparser.add_argument('--data_dir', default='mit-states', help='local path to data root dir from ' + DATA_FOLDER)\nparser.add_argument('--logpath', default=None, help='Path to dir where to logs are stored (test only)')\nparser.add_argument('--splitname', default='compositional-split-natural', help=\"dataset split\")\nparser.add_argument('--cv_dir', default='logs/', help='dir to save checkpoints and logs to')\nparser.add_argument('--name', default='temp', help='Name of exp used to name models')\nparser.add_argument('--load', default=None, help='path to checkpoint to load from')\nparser.add_argument('--image_extractor', default = 'resnet18', help = 'Feature extractor model')\nparser.add_argument('--norm_family', default = 'imagenet', help = 'Normalization values from dataset')\nparser.add_argument('--num_negs', type=int, default=1, help='Number of negatives to sample per positive (triplet loss)')\nparser.add_argument('--pair_dropout', type=float, default=0.0, help='Each epoch drop this fraction of train pairs')\nparser.add_argument('--test_set', default='val', help='val|test mode')\nparser.add_argument('--clean_only', action='store_true', default=False, help='use only clean subset of data (mitstates)')\nparser.add_argument('--subset', action='store_true', default=False, help='test on a 1000 image subset (debug purpose)')\nparser.add_argument('--open_world', action='store_true', default=False, help='perform open world experiment')\nparser.add_argument('--test_batch_size', type=int, default=32, help=\"Batch size at test/eval time\")\n\n# Model parameters\nparser.add_argument('--model', default='graphfull', help='visprodNN|redwine|labelembed+|attributeop|tmn|compcos')\nparser.add_argument('--emb_dim', type=int, default=300, help='dimension of share embedding space')\nparser.add_argument('--nlayers', type=int, default=3, help='Layers in the image embedder')\nparser.add_argument('--nmods', type=int, default=24, help='number of mods per layer for TMN')\nparser.add_argument('--embed_rank', type=int, default=64, help='intermediate dimension in the gating model for TMN')\nparser.add_argument('--bias', type=float, default=1e3, help='Bias value for unseen concepts')\nparser.add_argument('--update_features', action = 'store_true', default=False, help='If specified, train feature extractor')\nparser.add_argument('--freeze_features', action = 'store_true', default=False, help='If specified, put extractor in eval mode')\nparser.add_argument('--emb_init', default=None, help='w2v|ft|gl|glove|word2vec|fasttext, name of embeddings to use for initializing the primitives')\nparser.add_argument('--clf_init', action='store_true', default=False, help='initialize inputs with SVM weights')\nparser.add_argument('--static_inp', action='store_true', default=False, help='do not optimize primitives representations')\nparser.add_argument('--composition', default='mlp_add', help='add|mul|mlp|mlp_add, how to compose primitives')\nparser.add_argument('--relu', action='store_true', default=False, help='Use relu in image embedder')\nparser.add_argument('--dropout', action='store_true', default=False, help='Use dropout in image embedder')\nparser.add_argument('--norm', action='store_true', default=False, help='Use normalization in image embedder')\nparser.add_argument('--train_only', action='store_true', default=False, help='Optimize only for train pairs')\nparser.add_argument('--add_bt', type=int, default=0.0, help='global images')\nparser.add_argument('--eval_bt', type=int, default=0, help='global images')\n#CGE\nparser.add_argument('--graph', action='store_true', default=False, help='graph l2 distance triplet') # Do we need this\nparser.add_argument('--graph_init', default='graph', help='filename, file from which initializing the nodes and adjacency matrix of the graph')\nparser.add_argument('--gcn_type', default='gcn', help='GCN Version')\n\n# Forward\nparser.add_argument('--eval_type', default='dist', help='dist|prod|direct, function for computing the predictions')\n\n# Primitive-based loss\nparser.add_argument('--lambda_aux', type=float, default=0.0, help='weight of auxiliar losses on the primitives')\n\n# AoP\nparser.add_argument('--lambda_inv', type=float, default=0.0, help='weight of inverse consistency loss')\nparser.add_argument('--lambda_comm', type=float, default=0.0, help='weight of commutative loss')\nparser.add_argument('--lambda_ant', type=float, default=0.0, help='weight of antonyms loss')\n\n\n# SymNet\nparser.add_argument(\"--lambda_trip\", type=float, default=0, help='weight of triplet loss')\nparser.add_argument(\"--lambda_sym\", type=float, default=0, help='weight of symmetry loss')\nparser.add_argument(\"--lambda_axiom\", type=float, default=0, help='weight of axiom loss')\nparser.add_argument(\"--lambda_cls_attr\", type=float, default=0, help='weight of classification loss on attributes')\nparser.add_argument(\"--lambda_cls_obj\", type=float, default=0, help='weight of classification loss on objects')\n\n# CompCos (for the margin, see below)\nparser.add_argument('--cosine_scale', type=float, default=20,help=\"Scale for cosine similarity\")\nparser.add_argument('--epoch_max_margin', type=float, default=15,help=\"Epoch of max margin\")\nparser.add_argument('--update_feasibility_every', type=float, default=1,help=\"Frequency of feasibility scores update in epochs\")\nparser.add_argument('--hard_masking', action='store_true', default=False, help='hard masking during test time')\nparser.add_argument('--threshold', default=None,help=\"Apply a specific threshold at test time for the hard masking\")\nparser.add_argument('--threshold_trials', type=int, default=50,help=\"how many threshold values to try\")\n\n# Hyperparameters\nparser.add_argument('--topk', type=int, default=1,help=\"Compute topk accuracy\")\nparser.add_argument('--margin', type=float, default=2,help=\"Margin for triplet loss or feasibility scores in CompCos\")\nparser.add_argument('--workers', type=int, default=8,help=\"Number of workers\")\nparser.add_argument('--batch_size', type=int, default=512,help=\"Training batch size\")\nparser.add_argument('--lr', type=float, default=5e-5,help=\"Learning rate\")\nparser.add_argument('--lrg', type=float, default=1e-3,help=\"Learning rate feature extractor\")\nparser.add_argument('--lrb', type=float, default=5e-5,help=\"Learning rate encoder\")\nparser.add_argument('--wd', type=float, default=5e-5,help=\"Weight decay\")\nparser.add_argument('--save_every', type=int, default=10000,help=\"Frequency of snapshots in epochs\")\nparser.add_argument('--eval_val_every', type=int, default=1,help=\"Frequency of eval in epochs\")\nparser.add_argument('--max_epochs', type=int, default=800,help=\"Max number of epochs\")\nparser.add_argument(\"--fc_emb\", default='768,1024,1200',help=\"Image embedder layer config\")\nparser.add_argument(\"--gr_emb\", default='d4096,d',help=\"graph layers config\")\n"
},
{
"path": "czsl/models/__init__.py",
"content": ""
},
{
"path": "czsl/models/common.py",
"content": "import torch\nimport torch.nn as nn\nimport torch.nn.functional as F\nimport numpy as np\nimport copy\nfrom scipy.stats import hmean\n\ndevice = 'cuda' if torch.cuda.is_available() else 'cpu'\n\nclass MLP(nn.Module):\n '''\n Baseclass to create a simple MLP\n Inputs\n inp_dim: Int, Input dimension\n out-dim: Int, Output dimension\n num_layer: Number of hidden layers\n relu: Bool, Use non linear function at output\n bias: Bool, Use bias\n '''\n def __init__(self, inp_dim, out_dim, num_layers = 1, relu = True, bias = True, dropout = False, norm = False, layers = []):\n super(MLP, self).__init__()\n mod = []\n incoming = inp_dim\n for layer in range(num_layers - 1):\n if len(layers) == 0:\n outgoing = incoming\n else:\n outgoing = layers.pop(0)\n mod.append(nn.Linear(incoming, outgoing, bias = bias))\n \n incoming = outgoing\n if norm:\n mod.append(nn.LayerNorm(outgoing))\n # mod.append(nn.BatchNorm1d(outgoing))\n mod.append(nn.ReLU(inplace = True))\n # mod.append(nn.LeakyReLU(inplace=True, negative_slope=0.2))\n if dropout:\n mod.append(nn.Dropout(p = 0.5))\n\n mod.append(nn.Linear(incoming, out_dim, bias = bias))\n\n if relu:\n mod.append(nn.ReLU(inplace = True))\n # mod.append(nn.LeakyReLU(inplace=True, negative_slope=0.2))\n self.mod = nn.Sequential(*mod)\n \n def forward(self, x):\n return self.mod(x)\n\nclass Reshape(nn.Module):\n def __init__(self, *args):\n super(Reshape, self).__init__()\n self.shape = args\n\n def forward(self, x):\n return x.view(self.shape)\n\ndef calculate_margines(domain_embedding, gt, margin_range = 5):\n '''\n domain_embedding: pairs * feats\n gt: batch * feats\n '''\n batch_size, pairs, features = gt.shape[0], domain_embedding.shape[0], domain_embedding.shape[1]\n gt_expanded = gt[:,None,:].expand(-1, pairs, -1)\n domain_embedding_expanded = domain_embedding[None, :, :].expand(batch_size, -1, -1)\n margin = (gt_expanded - domain_embedding_expanded)**2\n margin = margin.sum(2)\n max_margin, _ = torch.max(margin, dim = 0)\n margin /= max_margin\n margin *= margin_range\n return margin\n\ndef l2_all_batched(image_embedding, domain_embedding):\n '''\n Image Embedding: Tensor of Batch_size * pairs * Feature_dim\n domain_embedding: Tensor of pairs * Feature_dim\n '''\n pairs = image_embedding.shape[1]\n domain_embedding_extended = image_embedding[:,None,:].expand(-1,pairs,-1)\n l2_loss = (image_embedding - domain_embedding_extended) ** 2\n l2_loss = l2_loss.sum(2)\n l2_loss = l2_loss.sum() / l2_loss.numel()\n return l2_loss\n\ndef same_domain_triplet_loss(image_embedding, trip_images, gt, hard_k = None, margin = 2):\n '''\n Image Embedding: Tensor of Batch_size * Feature_dim\n Triplet Images: Tensor of Batch_size * num_pairs * Feature_dim\n GT: Tensor of Batch_size\n '''\n batch_size, pairs, features = trip_images.shape\n batch_iterator = torch.arange(batch_size).to(device)\n image_embedding_expanded = image_embedding[:,None,:].expand(-1, pairs, -1)\n\n diff = (image_embedding_expanded - trip_images)**2\n diff = diff.sum(2)\n\n positive_anchor = diff[batch_iterator, gt][:,None]\n positive_anchor = positive_anchor.expand(-1, pairs)\n\n # Calculating triplet loss\n triplet_loss = positive_anchor - diff + margin\n \n # Setting positive anchor loss to 0\n triplet_loss[batch_iterator, gt] = 0\n \n # Removing easy triplets\n triplet_loss[triplet_loss < 0] = 0\n\n # If only mining hard triplets\n if hard_k:\n triplet_loss, _ = triplet_loss.topk(hard_k)\n\n # Counting number of valid pairs\n num_positive_triplets = triplet_loss[triplet_loss > 1e-16].size(0)\n \n # Calculating the final loss\n triplet_loss = triplet_loss.sum() / (num_positive_triplets + 1e-16)\n \n return triplet_loss\n\n\ndef cross_domain_triplet_loss(image_embedding, domain_embedding, gt, hard_k = None, margin = 2):\n '''\n Image Embedding: Tensor of Batch_size * Feature_dim\n Domain Embedding: Tensor of Num_pairs * Feature_dim\n gt: Tensor of Batch_size with ground truth labels\n margin: Float of margin\n Returns:\n Triplet loss of all valid triplets\n '''\n batch_size, pairs, features = image_embedding.shape[0], domain_embedding.shape[0], domain_embedding.shape[1]\n batch_iterator = torch.arange(batch_size).to(device)\n # Now dimensions will be Batch_size * Num_pairs * Feature_dim\n image_embedding = image_embedding[:,None,:].expand(-1, pairs, -1)\n domain_embedding = domain_embedding[None,:,:].expand(batch_size, -1, -1)\n \n # Calculating difference\n diff = (image_embedding - domain_embedding)**2\n diff = diff.sum(2)\n \n # Getting the positive pair\n positive_anchor = diff[batch_iterator, gt][:,None]\n positive_anchor = positive_anchor.expand(-1, pairs)\n \n # Calculating triplet loss\n triplet_loss = positive_anchor - diff + margin\n \n # Setting positive anchor loss to 0\n triplet_loss[batch_iterator, gt] = 0\n \n # Removing easy triplets\n triplet_loss[triplet_loss < 0] = 0\n \n # If only mining hard triplets\n if hard_k:\n triplet_loss, _ = triplet_loss.topk(hard_k)\n\n # Counting number of valid pairs\n num_positive_triplets = triplet_loss[triplet_loss > 1e-16].size(0)\n \n # Calculating the final loss\n triplet_loss = triplet_loss.sum() / (num_positive_triplets + 1e-16)\n \n return triplet_loss\n\ndef same_domain_triplet_loss_old(image_embedding, positive_anchor, negative_anchor, margin = 2):\n '''\n Image Embedding: Tensor of Batch_size * Feature_dim\n Positive anchor: Tensor of Batch_size * Feature_dim\n negative anchor: Tensor of Batch_size * negs *Feature_dim\n '''\n batch_size, negs, features = negative_anchor.shape\n dist_pos = (image_embedding - positive_anchor)**2\n dist_pos = dist_pos.sum(1)\n dist_pos = dist_pos[:, None].expand(-1, negs)\n\n image_embedding_expanded = image_embedding[:,None,:].expand(-1, negs, -1)\n dist_neg = (image_embedding_expanded - negative_anchor)**2\n dist_neg = dist_neg.sum(2)\n\n triplet_loss = dist_pos - dist_neg + margin\n triplet_loss[triplet_loss < 0] = 0\n num_positive_triplets = triplet_loss[triplet_loss > 1e-16].size(0)\n\n triplet_loss = triplet_loss.sum() / (num_positive_triplets + 1e-16)\n return triplet_loss\n\n\ndef pairwise_distances(x, y=None):\n '''\n Input: x is a Nxd matrix\n y is an optional Mxd matirx\n Output: dist is a NxM matrix where dist[i,j] is the square norm between x[i,:] and y[j,:]\n if y is not given then use 'y=x'.\n i.e. dist[i,j] = ||x[i,:]-y[j,:]||^2\n '''\n x_norm = (x**2).sum(1).view(-1, 1)\n if y is not None:\n y_t = torch.transpose(y, 0, 1)\n y_norm = (y**2).sum(1).view(1, -1)\n else:\n y_t = torch.transpose(x, 0, 1)\n y_norm = x_norm.view(1, -1)\n \n dist = x_norm + y_norm - 2.0 * torch.mm(x, y_t)\n # Ensure diagonal is zero if x=y\n # if y is None:\n # dist = dist - torch.diag(dist.diag)\n return torch.clamp(dist, 0.0, np.inf)\n\nclass Evaluator:\n\n def __init__(self, dset, model):\n\n self.dset = dset\n\n # Convert text pairs to idx tensors: [('sliced', 'apple'), ('ripe', 'apple'), ...] --> torch.LongTensor([[0,1],[1,1], ...])\n pairs = [(dset.attr2idx[attr], dset.obj2idx[obj]) for attr, obj in dset.pairs]\n self.train_pairs = [(dset.attr2idx[attr], dset.obj2idx[obj]) for attr, obj in dset.train_pairs]\n self.pairs = torch.LongTensor(pairs)\n\n # Mask over pairs that occur in closed world\n # Select set based on phase\n if dset.phase == 'train':\n print('Evaluating with train pairs')\n test_pair_set = set(dset.train_pairs)\n test_pair_gt = set(dset.train_pairs)\n elif dset.phase == 'val':\n print('Evaluating with validation pairs')\n test_pair_set = set(dset.val_pairs + dset.train_pairs)\n test_pair_gt = set(dset.val_pairs)\n else:\n print('Evaluating with test pairs')\n test_pair_set = set(dset.test_pairs + dset.train_pairs)\n test_pair_gt = set(dset.test_pairs)\n\n self.test_pair_dict = [(dset.attr2idx[attr], dset.obj2idx[obj]) for attr, obj in test_pair_gt]\n self.test_pair_dict = dict.fromkeys(self.test_pair_dict, 0)\n\n # dict values are pair val, score, total\n for attr, obj in test_pair_gt:\n pair_val = dset.pair2idx[(attr,obj)]\n key = (dset.attr2idx[attr], dset.obj2idx[obj])\n self.test_pair_dict[key] = [pair_val, 0, 0]\n\n if dset.open_world:\n masks = [1 for _ in dset.pairs]\n else:\n masks = [1 if pair in test_pair_set else 0 for pair in dset.pairs]\n\n self.closed_mask = torch.BoolTensor(masks)\n # Mask of seen concepts\n seen_pair_set = set(dset.train_pairs)\n mask = [1 if pair in seen_pair_set else 0 for pair in dset.pairs]\n self.seen_mask = torch.BoolTensor(mask)\n\n # Object specific mask over which pairs occur in the object oracle setting\n oracle_obj_mask = []\n for _obj in dset.objs:\n mask = [1 if _obj == obj else 0 for attr, obj in dset.pairs]\n oracle_obj_mask.append(torch.BoolTensor(mask))\n self.oracle_obj_mask = torch.stack(oracle_obj_mask, 0)\n\n # Decide if the model under evaluation is a manifold model or not\n self.score_model = self.score_manifold_model\n\n # Generate mask for each settings, mask scores, and get prediction labels\n def generate_predictions(self, scores, obj_truth, bias = 0.0, topk = 5): # (Batch, #pairs)\n '''\n Inputs\n scores: Output scores\n obj_truth: Ground truth object\n Returns\n results: dict of results in 3 settings\n '''\n def get_pred_from_scores(_scores, topk):\n '''\n Given list of scores, returns top 10 attr and obj predictions\n Check later\n '''\n _, pair_pred = _scores.topk(topk, dim = 1) #sort returns indices of k largest values\n pair_pred = pair_pred.contiguous().view(-1)\n attr_pred, obj_pred = self.pairs[pair_pred][:, 0].view(-1, topk), \\\n self.pairs[pair_pred][:, 1].view(-1, topk)\n return (attr_pred, obj_pred)\n\n results = {}\n orig_scores = scores.clone()\n mask = self.seen_mask.repeat(scores.shape[0],1) # Repeat mask along pairs dimension\n scores[~mask] += bias # Add bias to test pairs\n\n # Unbiased setting\n \n # Open world setting --no mask, all pairs of the dataset\n results.update({'open': get_pred_from_scores(scores, topk)})\n results.update({'unbiased_open': get_pred_from_scores(orig_scores, topk)})\n # Closed world setting - set the score for all Non test pairs to -1e10, \n # this excludes the pairs from set not in evaluation\n mask = self.closed_mask.repeat(scores.shape[0], 1)\n closed_scores = scores.clone()\n closed_scores[~mask] = -1e10 \n closed_orig_scores = orig_scores.clone()\n closed_orig_scores[~mask] = -1e10\n results.update({'closed': get_pred_from_scores(closed_scores, topk)})\n results.update({'unbiased_closed': get_pred_from_scores(closed_orig_scores, topk)})\n\n # Object_oracle setting - set the score to -1e10 for all pairs where the true object does Not participate, can also use the closed score\n mask = self.oracle_obj_mask[obj_truth]\n oracle_obj_scores = scores.clone()\n oracle_obj_scores[~mask] = -1e10\n oracle_obj_scores_unbiased = orig_scores.clone()\n oracle_obj_scores_unbiased[~mask] = -1e10\n results.update({'object_oracle': get_pred_from_scores(oracle_obj_scores, 1)})\n results.update({'object_oracle_unbiased': get_pred_from_scores(oracle_obj_scores_unbiased, 1)})\n\n return results\n\n def score_clf_model(self, scores, obj_truth, topk = 5):\n '''\n Wrapper function to call generate_predictions for CLF models\n '''\n attr_pred, obj_pred = scores\n\n # Go to CPU\n attr_pred, obj_pred, obj_truth = attr_pred.to('cpu'), obj_pred.to('cpu'), obj_truth.to('cpu')\n\n # Gather scores (P(a), P(o)) for all relevant (a,o) pairs\n # Multiply P(a) * P(o) to get P(pair)\n attr_subset = attr_pred.index_select(1, self.pairs[:,0]) # Return only attributes that are in our pairs\n obj_subset = obj_pred.index_select(1, self.pairs[:, 1])\n scores = (attr_subset * obj_subset) # (Batch, #pairs)\n\n results = self.generate_predictions(scores, obj_truth)\n results['biased_scores'] = scores\n\n return results\n\n def score_manifold_model(self, scores, obj_truth, bias = 0.0, topk = 5):\n '''\n Wrapper function to call generate_predictions for manifold models\n '''\n # Go to CPU\n scores = {k: v.to('cpu') for k, v in scores.items()}\n obj_truth = obj_truth.to(device)\n\n # Gather scores for all relevant (a,o) pairs\n scores = torch.stack(\n [scores[(attr,obj)] for attr, obj in self.dset.pairs], 1\n ) # (Batch, #pairs)\n orig_scores = scores.clone()\n results = self.generate_predictions(scores, obj_truth, bias, topk)\n results['scores'] = orig_scores\n return results\n\n def score_fast_model(self, scores, obj_truth, bias = 0.0, topk = 5):\n '''\n Wrapper function to call generate_predictions for manifold models\n '''\n \n results = {}\n mask = self.seen_mask.repeat(scores.shape[0],1) # Repeat mask along pairs dimension\n scores[~mask] += bias # Add bias to test pairs\n\n mask = self.closed_mask.repeat(scores.shape[0], 1)\n closed_scores = scores.clone()\n closed_scores[~mask] = -1e10 \n\n _, pair_pred = closed_scores.topk(topk, dim = 1) #sort returns indices of k largest values\n pair_pred = pair_pred.contiguous().view(-1)\n attr_pred, obj_pred = self.pairs[pair_pred][:, 0].view(-1, topk), \\\n self.pairs[pair_pred][:, 1].view(-1, topk)\n\n results.update({'closed': (attr_pred, obj_pred)})\n return results\n\n def evaluate_predictions(self, predictions, attr_truth, obj_truth, pair_truth, allpred, topk = 1):\n # Go to CPU\n attr_truth, obj_truth, pair_truth = attr_truth.to('cpu'), obj_truth.to('cpu'), pair_truth.to('cpu')\n\n pairs = list(\n zip(list(attr_truth.numpy()), list(obj_truth.numpy())))\n \n\n seen_ind, unseen_ind = [], []\n for i in range(len(attr_truth)):\n if pairs[i] in self.train_pairs:\n seen_ind.append(i)\n else:\n unseen_ind.append(i)\n\n \n seen_ind, unseen_ind = torch.LongTensor(seen_ind), torch.LongTensor(unseen_ind)\n def _process(_scores):\n # Top k pair accuracy\n # Attribute, object and pair\n attr_match = (attr_truth.unsqueeze(1).repeat(1, topk) == _scores[0][:, :topk])\n obj_match = (obj_truth.unsqueeze(1).repeat(1, topk) == _scores[1][:, :topk])\n\n # Match of object pair\n match = (attr_match * obj_match).any(1).float()\n attr_match = attr_match.any(1).float()\n obj_match = obj_match.any(1).float()\n # Match of seen and unseen pairs\n seen_match = match[seen_ind]\n unseen_match = match[unseen_ind]\n ### Calculating class average accuracy\n \n # local_score_dict = copy.deepcopy(self.test_pair_dict)\n # for pair_gt, pair_pred in zip(pairs, match):\n # # print(pair_gt)\n # local_score_dict[pair_gt][2] += 1.0 #increase counter\n # if int(pair_pred) == 1:\n # local_score_dict[pair_gt][1] += 1.0\n\n # # Now we have hits and totals for classes in evaluation set\n # seen_score, unseen_score = [], []\n # for key, (idx, hits, total) in local_score_dict.items():\n # score = hits/total\n # if bool(self.seen_mask[idx]) == True:\n # seen_score.append(score)\n # else:\n # unseen_score.append(score)\n \n seen_score, unseen_score = torch.ones(512,5), torch.ones(512,5)\n\n return attr_match, obj_match, match, seen_match, unseen_match, \\\n torch.Tensor(seen_score+unseen_score), torch.Tensor(seen_score), torch.Tensor(unseen_score)\n\n def _add_to_dict(_scores, type_name, stats):\n base = ['_attr_match', '_obj_match', '_match', '_seen_match', '_unseen_match', '_ca', '_seen_ca', '_unseen_ca']\n for val, name in zip(_scores, base):\n stats[type_name + name] = val\n\n ##################### Match in places where corrent object\n obj_oracle_match = (attr_truth == predictions['object_oracle'][0][:, 0]).float() #object is already conditioned\n obj_oracle_match_unbiased = (attr_truth == predictions['object_oracle_unbiased'][0][:, 0]).float()\n\n stats = dict(obj_oracle_match = obj_oracle_match, obj_oracle_match_unbiased = obj_oracle_match_unbiased)\n\n #################### Closed world\n closed_scores = _process(predictions['closed'])\n unbiased_closed = _process(predictions['unbiased_closed'])\n _add_to_dict(closed_scores, 'closed', stats)\n _add_to_dict(unbiased_closed, 'closed_ub', stats)\n\n #################### Calculating AUC\n scores = predictions['scores']\n # getting score for each ground truth class\n correct_scores = scores[torch.arange(scores.shape[0]), pair_truth][unseen_ind]\n\n # Getting top predicted score for these unseen classes\n max_seen_scores = predictions['scores'][unseen_ind][:, self.seen_mask].topk(topk, dim=1)[0][:, topk - 1]\n\n # Getting difference between these scores\n unseen_score_diff = max_seen_scores - correct_scores\n\n # Getting matched classes at max bias for diff\n unseen_matches = stats['closed_unseen_match'].bool()\n correct_unseen_score_diff = unseen_score_diff[unseen_matches] - 1e-4\n\n # sorting these diffs\n correct_unseen_score_diff = torch.sort(correct_unseen_score_diff)[0]\n magic_binsize = 20\n # getting step size for these bias values\n bias_skip = max(len(correct_unseen_score_diff) // magic_binsize, 1)\n # Getting list\n biaslist = correct_unseen_score_diff[::bias_skip]\n\n seen_match_max = float(stats['closed_seen_match'].mean())\n unseen_match_max = float(stats['closed_unseen_match'].mean())\n seen_accuracy, unseen_accuracy = [], []\n\n # Go to CPU\n base_scores = {k: v.to('cpu') for k, v in allpred.items()}\n obj_truth = obj_truth.to('cpu')\n\n # Gather scores for all relevant (a,o) pairs\n base_scores = torch.stack(\n [allpred[(attr,obj)] for attr, obj in self.dset.pairs], 1\n ) # (Batch, #pairs)\n\n for bias in biaslist:\n scores = base_scores.clone()\n results = self.score_fast_model(scores, obj_truth, bias = bias, topk = topk)\n results = results['closed'] # we only need biased\n results = _process(results)\n seen_match = float(results[3].mean())\n unseen_match = float(results[4].mean())\n seen_accuracy.append(seen_match)\n unseen_accuracy.append(unseen_match)\n\n seen_accuracy.append(seen_match_max)\n unseen_accuracy.append(unseen_match_max)\n seen_accuracy, unseen_accuracy = np.array(seen_accuracy), np.array(unseen_accuracy)\n area = np.trapz(seen_accuracy, unseen_accuracy)\n\n for key in stats:\n stats[key] = float(stats[key].mean())\n\n harmonic_mean = hmean([seen_accuracy, unseen_accuracy], axis = 0)\n max_hm = np.max(harmonic_mean)\n idx = np.argmax(harmonic_mean)\n if idx == len(biaslist):\n bias_term = 1e3\n else:\n bias_term = biaslist[idx]\n stats['biasterm'] = float(bias_term)\n stats['best_unseen'] = np.max(unseen_accuracy)\n stats['best_seen'] = np.max(seen_accuracy)\n stats['AUC'] = area\n stats['hm_unseen'] = unseen_accuracy[idx]\n stats['hm_seen'] = seen_accuracy[idx]\n stats['best_hm'] = max_hm\n return stats"
},
{
"path": "czsl/models/compcos.py",
"content": "import torch\nimport torch.nn as nn\nimport torch.nn.functional as F\nfrom .word_embedding import load_word_embeddings\nfrom .common import MLP\n\nfrom itertools import product\n\ndevice = 'cuda' if torch.cuda.is_available() else 'cpu'\n\ndef compute_cosine_similarity(names, weights, return_dict=True):\n pairing_names = list(product(names, names))\n normed_weights = F.normalize(weights,dim=1)\n similarity = torch.mm(normed_weights, normed_weights.t())\n if return_dict:\n dict_sim = {}\n for i,n in enumerate(names):\n for j,m in enumerate(names):\n dict_sim[(n,m)]=similarity[i,j].item()\n return dict_sim\n return pairing_names, similarity.to('cpu')\n\nclass CompCos(nn.Module):\n\n def __init__(self, dset, args):\n super(CompCos, self).__init__()\n self.args = args\n self.dset = dset\n\n def get_all_ids(relevant_pairs):\n # Precompute validation pairs\n attrs, objs = zip(*relevant_pairs)\n attrs = [dset.attr2idx[attr] for attr in attrs]\n objs = [dset.obj2idx[obj] for obj in objs]\n pairs = [a for a in range(len(relevant_pairs))]\n attrs = torch.LongTensor(attrs).to(device)\n objs = torch.LongTensor(objs).to(device)\n pairs = torch.LongTensor(pairs).to(device)\n return attrs, objs, pairs\n\n # Validation\n self.val_attrs, self.val_objs, self.val_pairs = get_all_ids(self.dset.pairs)\n\n # for indivual projections\n self.uniq_attrs, self.uniq_objs = torch.arange(len(self.dset.attrs)).long().to(device), \\\n torch.arange(len(self.dset.objs)).long().to(device)\n self.factor = 2\n\n self.scale = self.args.cosine_scale\n self.add_bt = args.add_bt\n\n if dset.open_world:\n self.train_forward = self.train_forward_open\n self.known_pairs = dset.train_pairs\n seen_pair_set = set(self.known_pairs)\n mask = [1 if pair in seen_pair_set else 0 for pair in dset.pairs]\n self.seen_mask = torch.BoolTensor(mask).to(device) * 1.\n\n self.activated = False\n\n # Init feasibility-related variables\n self.attrs = dset.attrs\n self.objs = dset.objs\n self.possible_pairs = dset.pairs\n\n self.validation_pairs = dset.val_pairs\n\n self.feasibility_margin = (1-self.seen_mask).float()\n self.epoch_max_margin = self.args.epoch_max_margin\n self.cosine_margin_factor = -args.margin\n\n # Intantiate attribut-object relations, needed just to evaluate mined pairs\n self.obj_by_attrs_train = {k: [] for k in self.attrs}\n for (a, o) in self.known_pairs:\n self.obj_by_attrs_train[a].append(o)\n\n # Intantiate attribut-object relations, needed just to evaluate mined pairs\n self.attrs_by_obj_train = {k: [] for k in self.objs}\n for (a, o) in self.known_pairs:\n self.attrs_by_obj_train[o].append(a)\n\n else:\n self.train_forward = self.train_forward_closed\n\n # Precompute training compositions\n if args.train_only:\n self.train_attrs, self.train_objs, self.train_pairs = get_all_ids(self.dset.train_pairs)\n else:\n self.train_attrs, self.train_objs, self.train_pairs = self.val_attrs, self.val_objs, self.val_pairs\n\n try:\n self.args.fc_emb = self.args.fc_emb.split(',')\n except:\n self.args.fc_emb = [self.args.fc_emb]\n layers = []\n for a in self.args.fc_emb:\n a = int(a)\n layers.append(a)\n\n\n self.image_embedder = MLP(dset.feat_dim, int(args.emb_dim), relu=args.relu, num_layers=args.nlayers,\n dropout=self.args.dropout,\n norm=self.args.norm, layers=layers)\n\n # Fixed\n self.composition = args.composition\n\n input_dim = args.emb_dim\n self.attr_embedder = nn.Embedding(len(dset.attrs), input_dim)\n self.obj_embedder = nn.Embedding(len(dset.objs), input_dim)\n\n # init with word embeddings\n if args.emb_init:\n pretrained_weight = load_word_embeddings(args.emb_init, dset.attrs)\n self.attr_embedder.weight.data.copy_(pretrained_weight)\n pretrained_weight = load_word_embeddings(args.emb_init, dset.objs)\n self.obj_embedder.weight.data.copy_(pretrained_weight)\n\n # static inputs\n if args.static_inp:\n for param in self.attr_embedder.parameters():\n param.requires_grad = False\n for param in self.obj_embedder.parameters():\n param.requires_grad = False\n\n # Composition MLP\n self.projection = nn.Linear(input_dim * 2, args.emb_dim)\n\n\n def freeze_representations(self):\n print('Freezing representations')\n for param in self.image_embedder.parameters():\n param.requires_grad = False\n for param in self.attr_embedder.parameters():\n param.requires_grad = False\n for param in self.obj_embedder.parameters():\n param.requires_grad = False\n\n\n def compose(self, attrs, objs):\n attrs, objs = self.attr_embedder(attrs), self.obj_embedder(objs)\n inputs = torch.cat([attrs, objs], 1)\n output = self.projection(inputs)\n output = F.normalize(output, dim=1)\n return output\n\n\n def compute_feasibility(self):\n obj_embeddings = self.obj_embedder(torch.arange(len(self.objs)).long().to('cuda'))\n obj_embedding_sim = compute_cosine_similarity(self.objs, obj_embeddings,\n return_dict=True)\n attr_embeddings = self.attr_embedder(torch.arange(len(self.attrs)).long().to('cuda'))\n attr_embedding_sim = compute_cosine_similarity(self.attrs, attr_embeddings,\n return_dict=True)\n\n feasibility_scores = self.seen_mask.clone().float()\n for a in self.attrs:\n for o in self.objs:\n if (a, o) not in self.known_pairs:\n idx = self.dset.all_pair2idx[(a, o)]\n score_obj = self.get_pair_scores_objs(a, o, obj_embedding_sim)\n score_attr = self.get_pair_scores_attrs(a, o, attr_embedding_sim)\n score = (score_obj + score_attr) / 2\n feasibility_scores[idx] = score\n\n self.feasibility_scores = feasibility_scores\n\n return feasibility_scores * (1 - self.seen_mask.float())\n\n\n def get_pair_scores_objs(self, attr, obj, obj_embedding_sim):\n score = -1.\n for o in self.objs:\n if o!=obj and attr in self.attrs_by_obj_train[o]:\n temp_score = obj_embedding_sim[(obj,o)]\n if temp_score>score:\n score=temp_score\n return score\n\n def get_pair_scores_attrs(self, attr, obj, attr_embedding_sim):\n score = -1.\n for a in self.attrs:\n if a != attr and obj in self.obj_by_attrs_train[a]:\n temp_score = attr_embedding_sim[(attr, a)]\n if temp_score > score:\n score = temp_score\n return score\n\n def update_feasibility(self,epoch):\n self.activated = True\n feasibility_scores = self.compute_feasibility()\n self.feasibility_margin = min(1.,epoch/self.epoch_max_margin) * \\\n (self.cosine_margin_factor*feasibility_scores.float().to(device))\n\n\n def val_forward(self, x):\n img = x[0]\n img_feats = self.image_embedder(img)\n img_feats_normed = F.normalize(img_feats, dim=1)\n pair_embeds = self.compose(self.val_attrs, self.val_objs).permute(1, 0) # Evaluate all pairs\n score = torch.matmul(img_feats_normed, pair_embeds)\n\n scores = {}\n for itr, pair in enumerate(self.dset.pairs):\n scores[pair] = score[:, self.dset.all_pair2idx[pair]]\n\n return None, scores\n\n\n def val_forward_with_threshold(self, x, th=0.):\n img = x[0]\n img_feats = self.image_embedder(img)\n img_feats_normed = F.normalize(img_feats, dim=1)\n pair_embeds = self.compose(self.val_attrs, self.val_objs).permute(1, 0) # Evaluate all pairs\n score = torch.matmul(img_feats_normed, pair_embeds)\n\n # Note: Pairs are already aligned here\n mask = (self.feasibility_scores>=th).float()\n score = score*mask + (1.-mask)*(-1.)\n\n scores = {}\n for itr, pair in enumerate(self.dset.pairs):\n scores[pair] = score[:, self.dset.all_pair2idx[pair]]\n\n return None, scores\n\n\n def train_forward_open(self, x):\n img, attrs, objs, pairs = x[0], x[1], x[2], x[3]\n img_feats = self.image_embedder(img)\n\n pair_embed = self.compose(self.train_attrs, self.train_objs).permute(1, 0)\n img_feats_normed = F.normalize(img_feats, dim=1)\n\n pair_pred = torch.matmul(img_feats_normed, pair_embed)\n\n if self.add_bt > 1:\n pairs = torch.cat([pairs, pairs], dim=0)\n if self.activated:\n pair_pred += (1 - self.seen_mask) * self.feasibility_margin\n loss_cos = F.cross_entropy(self.scale * pair_pred, pairs)\n else:\n pair_pred = pair_pred * self.seen_mask + (1 - self.seen_mask) * (-10)\n loss_cos = F.cross_entropy(self.scale * pair_pred, pairs)\n\n return loss_cos.mean(), None\n\n\n def train_forward_closed(self, x):\n img, attrs, objs, pairs = x[0], x[1], x[2], x[3]\n img_feats = self.image_embedder(img)\n\n pair_embed = self.compose(self.train_attrs, self.train_objs).permute(1, 0)\n img_feats_normed = F.normalize(img_feats, dim=1)\n\n pair_pred = torch.matmul(img_feats_normed, pair_embed)\n\n loss_cos = F.cross_entropy(self.scale * pair_pred, pairs)\n\n return loss_cos.mean(), None\n\n\n def forward(self, x):\n if self.training:\n loss, pred = self.train_forward(x)\n else:\n with torch.no_grad():\n loss, pred = self.val_forward(x)\n return loss, pred\n\n\n\n"
},
{
"path": "czsl/models/gcn.py",
"content": "import numpy as np\nimport scipy.sparse as sp\n\nimport math\nimport torch\nimport torch.nn as nn\nimport torch.nn.functional as F\nfrom torch.nn.init import xavier_uniform_\n\ndevice = 'cuda' if torch.cuda.is_available() else 'cpu'\n\ndef normt_spm(mx, method='in'):\n if method == 'in':\n mx = mx.transpose()\n rowsum = np.array(mx.sum(1))\n r_inv = np.power(rowsum, -1).flatten()\n r_inv[np.isinf(r_inv)] = 0.\n r_mat_inv = sp.diags(r_inv)\n mx = r_mat_inv.dot(mx)\n return mx\n\n if method == 'sym':\n rowsum = np.array(mx.sum(1))\n r_inv = np.power(rowsum, -0.5).flatten()\n r_inv[np.isinf(r_inv)] = 0.\n r_mat_inv = sp.diags(r_inv)\n mx = mx.dot(r_mat_inv).transpose().dot(r_mat_inv)\n return mx\n\ndef spm_to_tensor(sparse_mx):\n sparse_mx = sparse_mx.tocoo().astype(np.float32)\n indices = torch.from_numpy(np.vstack(\n (sparse_mx.row, sparse_mx.col))).long()\n values = torch.from_numpy(sparse_mx.data)\n shape = torch.Size(sparse_mx.shape)\n return torch.sparse.FloatTensor(indices, values, shape)\n\nclass GraphConv(nn.Module):\n\n def __init__(self, in_channels, out_channels, dropout=False, relu=True):\n super().__init__()\n\n if dropout:\n self.dropout = nn.Dropout(p=0.5)\n else:\n self.dropout = None\n\n self.layer = nn.Linear(in_channels, out_channels)\n\n if relu:\n # self.relu = nn.LeakyReLU(negative_slope=0.2)\n self.relu = nn.ReLU()\n else:\n self.relu = None\n\n def forward(self, inputs, adj):\n if self.dropout is not None:\n inputs = self.dropout(inputs)\n\n outputs = torch.mm(adj, torch.mm(inputs, self.layer.weight.T)) + self.layer.bias\n\n if self.relu is not None:\n outputs = self.relu(outputs)\n return outputs\n\n\nclass GCN(nn.Module):\n\n def __init__(self, adj, in_channels, out_channels, hidden_layers):\n super().__init__()\n\n adj = normt_spm(adj, method='in')\n adj = spm_to_tensor(adj)\n self.adj = adj.to(device)\n\n self.train_adj = self.adj\n\n hl = hidden_layers.split(',')\n if hl[-1] == 'd':\n dropout_last = True\n hl = hl[:-1]\n else:\n dropout_last = False\n\n i = 0\n layers = []\n last_c = in_channels\n for c in hl:\n if c[0] == 'd':\n dropout = True\n c = c[1:]\n else:\n dropout = False\n c = int(c)\n\n i += 1\n conv = GraphConv(last_c, c, dropout=dropout)\n self.add_module('conv{}'.format(i), conv)\n layers.append(conv)\n\n last_c = c\n\n conv = GraphConv(last_c, out_channels, relu=False, dropout=dropout_last)\n self.add_module('conv-last', conv)\n layers.append(conv)\n\n self.layers = layers\n\n def forward(self, x):\n if self.training:\n for conv in self.layers:\n x = conv(x, self.train_adj)\n else:\n for conv in self.layers:\n x = conv(x, self.adj)\n return F.normalize(x)\n\n### GCNII\nclass GraphConvolution(nn.Module):\n\n def __init__(self, in_features, out_features, dropout=False, relu=True, residual=False, variant=False):\n super(GraphConvolution, self).__init__() \n self.variant = variant\n if self.variant:\n self.in_features = 2*in_features \n else:\n self.in_features = in_features\n\n if dropout:\n self.dropout = nn.Dropout(p=0.5)\n else:\n self.dropout = None\n\n if relu:\n self.relu = nn.ReLU()\n else:\n self.relu = None\n\n self.out_features = out_features\n self.residual = residual\n self.layer = nn.Linear(self.in_features, self.out_features, bias = False)\n\n def reset_parameters(self):\n stdv = 1. / math.sqrt(self.out_features)\n self.weight.data.uniform_(-stdv, stdv)\n\n def forward(self, input, adj , h0 , lamda, alpha, l):\n if self.dropout is not None:\n input = self.dropout(input)\n\n theta = math.log(lamda/l+1)\n hi = torch.spmm(adj, input)\n if self.variant:\n support = torch.cat([hi,h0],1)\n r = (1-alpha)*hi+alpha*h0\n else:\n support = (1-alpha)*hi+alpha*h0\n r = support\n\n mm_term = torch.mm(support, self.layer.weight.T)\n\n output = theta*mm_term+(1-theta)*r\n if self.residual:\n output = output+input\n\n if self.relu is not None:\n output = self.relu(output)\n\n return output\n\nclass GCNII(nn.Module):\n def __init__(self, adj, in_channels , out_channels, hidden_dim, hidden_layers, lamda, alpha, variant, dropout = True):\n super(GCNII, self).__init__()\n\n self.alpha = alpha\n self.lamda = lamda\n\n adj = normt_spm(adj, method='in')\n adj = spm_to_tensor(adj)\n self.adj = adj.to(device)\n\n i = 0\n layers = nn.ModuleList()\n self.fc_dim = nn.Linear(in_channels, hidden_dim)\n self.relu = nn.ReLU()\n self.dropout = nn.Dropout()\n\n for i, c in enumerate(range(hidden_layers)):\n conv = GraphConvolution(hidden_dim, hidden_dim, variant=variant, dropout=dropout)\n layers.append(conv)\n\n self.layers = layers\n self.fc_out = nn.Linear(hidden_dim, out_channels)\n\n def forward(self, x):\n _layers = []\n layer_inner = self.relu(self.fc_dim(self.dropout(x)))\n # layer_inner = x\n _layers.append(layer_inner)\n\n for i,con in enumerate(self.layers):\n layer_inner = con(layer_inner,self.adj,_layers[0],self.lamda,self.alpha,i+1)\n \n layer_inner = self.fc_out(self.dropout(layer_inner))\n return layer_inner"
},
{
"path": "czsl/models/graph_method.py",
"content": "import torch\nimport torch.nn as nn\nimport torch.nn.functional as F\nimport numpy as np\nfrom .common import MLP, cross_domain_triplet_loss\nfrom .gcn import GCN, GCNII\n\ndevice = 'cuda' if torch.cuda.is_available() else 'cpu'\n\nclass GraphFull(nn.Module):\n def __init__(self, dset, args):\n super(GraphFull, self).__init__()\n self.args = args\n self.dset = dset\n\n self.val_forward = self.val_forward_dotpr\n self.train_forward = self.train_forward_normal\n self.add_bt = args.add_bt\n # Image Embedder\n self.num_attrs, self.num_objs, self.num_pairs = len(dset.attrs), len(dset.objs), len(dset.pairs)\n\n if self.args.train_only:\n train_idx = []\n for current in dset.train_pairs:\n train_idx.append(dset.all_pair2idx[current]+self.num_attrs+self.num_objs)\n self.train_idx = torch.LongTensor(train_idx).to(device)\n\n if self.args.fc_emb.__contains__(','):\n self.args.fc_emb = self.args.fc_emb.split(',')\n layers = []\n for a in self.args.fc_emb:\n a = int(a)\n layers.append(a)\n\n if args.nlayers:\n self.image_embedder = MLP(dset.feat_dim, args.emb_dim, num_layers= args.nlayers, dropout = self.args.dropout,\n norm = self.args.norm, layers = layers, relu = True)\n\n path = args.graph_init\n graph = torch.load(path)\n self.embeddings = graph['embeddings'].to(device)\n adj = graph['adj']\n\n hidden_layers = self.args.gr_emb\n\n if args.gcn_type == 'gcn':\n self.gcn = GCN(adj, self.embeddings.shape[1], args.emb_dim, hidden_layers)\n else:\n self.gcn = GCNII(adj, self.embeddings.shape[1], args.emb_dim, args.hidden_dim, args.gcn_nlayers, lamda = 0.5, alpha = 0.1, variant = False)\n\n if args.static_inp:\n for param in self.parameters():\n param.requires_grad = False\n\n\n def train_forward_normal(self, x):\n img, _, _, pairs = x[0], x[1], x[2], x[3]\n\n if self.args.nlayers:\n img_feats = self.image_embedder(img)\n else:\n img_feats = (img)\n\n current_embeddings = self.gcn(self.embeddings)\n \n if self.args.train_only:\n pair_embed = current_embeddings[self.train_idx]\n else:\n pair_embed = current_embeddings[self.num_attrs+self.num_objs:self.num_attrs+self.num_objs+self.num_pairs,:]\n\n pair_embed = pair_embed.permute(1,0)\n pair_pred = torch.matmul(img_feats, pair_embed)\n if self.add_bt >= 1:\n pairs = torch.cat([pairs, pairs], dim=0)\n loss = F.cross_entropy(pair_pred, pairs)\n\n return loss, None\n\n def val_forward_dotpr(self, x):\n img = x[0]\n\n if self.args.nlayers:\n img_feats = self.image_embedder(img)\n else:\n img_feats = (img)\n\n current_embedddings = self.gcn(self.embeddings)\n\n pair_embeds = current_embedddings[self.num_attrs+self.num_objs:self.num_attrs+self.num_objs+self.num_pairs,:].permute(1,0)\n\n score = torch.matmul(img_feats, pair_embeds)\n\n scores = {}\n for itr, pair in enumerate(self.dset.pairs):\n scores[pair] = score[:,self.dset.all_pair2idx[pair]]\n\n return None, scores\n\n def val_forward_distance_fast(self, x):\n img = x[0]\n\n img_feats = (self.image_embedder(img))\n current_embeddings = self.gcn(self.embeddings)\n pair_embeds = current_embeddings[self.num_attrs+self.num_objs:,:]\n\n batch_size, pairs, features = img_feats.shape[0], pair_embeds.shape[0], pair_embeds.shape[1]\n img_feats = img_feats[:,None,:].expand(-1, pairs, -1)\n pair_embeds = pair_embeds[None,:,:].expand(batch_size, -1, -1)\n diff = (img_feats - pair_embeds)**2\n score = diff.sum(2) * -1\n\n scores = {}\n for itr, pair in enumerate(self.dset.pairs):\n scores[pair] = score[:,self.dset.all_pair2idx[pair]]\n\n return None, scores\n\n def forward(self, x):\n if self.training:\n loss, pred = self.train_forward(x)\n else:\n with torch.no_grad():\n loss, pred = self.val_forward(x)\n return loss, pred"
},
{
"path": "czsl/models/image_extractor.py",
"content": "import torch\nimport torch.nn as nn\nimport torch.nn.functional as F\nfrom torchvision import models\nfrom torchvision.models.resnet import ResNet, BasicBlock\n\nclass ResNet18_conv(ResNet):\n def __init__(self):\n super(ResNet18_conv, self).__init__(BasicBlock, [2, 2, 2, 2])\n \n def forward(self, x):\n # change forward here\n x = self.conv1(x)\n x = self.bn1(x)\n x = self.relu(x)\n x = self.maxpool(x)\n\n x = self.layer1(x)\n x = self.layer2(x)\n x = self.layer3(x)\n x = self.layer4(x)\n\n return x\n\n\ndef get_image_extractor(arch = 'resnet18', pretrained = True, feature_dim = None, checkpoint = ''):\n '''\n Inputs\n arch: Base architecture\n pretrained: Bool, Imagenet weights\n feature_dim: Int, output feature dimension\n checkpoint: String, not implemented\n Returns\n Pytorch model\n '''\n\n if arch == 'resnet18':\n model = models.resnet18(pretrained = pretrained)\n if feature_dim is None:\n model.fc = nn.Sequential()\n else:\n model.fc = nn.Linear(512, feature_dim) \n\n if arch == 'resnet18_conv':\n model = ResNet18_conv()\n model.load_state_dict(models.resnet18(pretrained=True).state_dict())\n\n elif arch == 'resnet50':\n model = models.resnet50(pretrained = pretrained)\n if feature_dim is None:\n model.fc = nn.Sequential()\n else:\n model.fc = nn.Linear(2048, feature_dim) \n\n elif arch == 'resnet50_cutmix':\n model = models.resnet50(pretrained = pretrained)\n checkpoint = torch.load('/home/ubuntu/workspace/pretrained/resnet50_cutmix.tar')\n model.load_state_dict(checkpoint['state_dict'], strict=False)\n if feature_dim is None:\n model.fc = nn.Sequential()\n else:\n model.fc = nn.Linear(2048, feature_dim) \n\n elif arch == 'resnet152':\n model = models.resnet152(pretrained = pretrained)\n if feature_dim is None:\n model.fc = nn.Sequential()\n else:\n model.fc = nn.Linear(2048, feature_dim) \n\n elif arch == 'vgg16':\n model = models.vgg16(pretrained = pretrained)\n modules = list(model.classifier.children())[:-3]\n model.classifier=torch.nn.Sequential(*modules)\n if feature_dim is not None:\n model.classifier[3]=torch.nn.Linear(4096,feature_dim)\n\n return model\n\n"
},
{
"path": "czsl/models/manifold_methods.py",
"content": "import torch\nimport torch.nn as nn\nimport torch.nn.functional as F\nimport numpy as np\nfrom .word_embedding import load_word_embeddings\nfrom .common import MLP, Reshape\nfrom flags import DATA_FOLDER\n\ndevice = 'cuda' if torch.cuda.is_available() else 'cpu'\n\nclass ManifoldModel(nn.Module):\n\n def __init__(self, dset, args):\n super(ManifoldModel, self).__init__()\n self.args = args\n self.dset = dset\n\n def get_all_ids(relevant_pairs):\n # Precompute validation pairs\n attrs, objs = zip(*relevant_pairs)\n attrs = [dset.attr2idx[attr] for attr in attrs]\n objs = [dset.obj2idx[obj] for obj in objs]\n pairs = [a for a in range(len(relevant_pairs))]\n attrs = torch.LongTensor(attrs).to(device)\n objs = torch.LongTensor(objs).to(device)\n pairs = torch.LongTensor(pairs).to(device)\n return attrs, objs, pairs\n\n #Validation\n self.val_attrs, self.val_objs, self.val_pairs = get_all_ids(self.dset.pairs)\n # for indivual projections\n self.uniq_attrs, self.uniq_objs = torch.arange(len(self.dset.attrs)).long().to(device), \\\n torch.arange(len(self.dset.objs)).long().to(device)\n self.factor = 2\n # Precompute training compositions\n if args.train_only:\n self.train_attrs, self.train_objs, self.train_pairs = get_all_ids(self.dset.train_pairs)\n else:\n self.train_attrs, self.train_objs = self.val_subs, self.val_attrs, self.val_objs\n\n if args.lambda_aux > 0 or args.lambda_cls_attr > 0 or args.lambda_cls_obj > 0:\n print('Initializing classifiers')\n self.obj_clf = nn.Linear(args.emb_dim, len(dset.objs))\n self.attr_clf = nn.Linear(args.emb_dim, len(dset.attrs))\n\n def train_forward_bce(self, x):\n img, attrs, objs = x[0], x[1], x[2]\n neg_attrs, neg_objs = x[4][:,0],x[5][:,0] #todo: do a less hacky version\n \n img_feat = self.image_embedder(img)\n # Sample 25% positive and 75% negative pairs\n labels = np.random.binomial(1, 0.25, attrs.shape[0])\n labels = torch.from_numpy(labels).bool().to(device)\n sampled_attrs, sampled_objs = neg_attrs.clone(), neg_objs.clone()\n sampled_attrs[labels] = attrs[labels]\n sampled_objs[labels] = objs[labels]\n labels = labels.float()\n\n composed_clf = self.compose(attrs, objs)\n p = torch.sigmoid((img_feat*composed_clf).sum(1))\n loss = F.binary_cross_entropy(p, labels)\n\n return loss, None\n\n def train_forward_triplet(self, x):\n img, attrs, objs = x[0], x[1], x[2]\n neg_attrs, neg_objs = x[4][:,0], x[5][:,0] #todo:do a less hacky version\n\n img_feats = self.image_embedder(img)\n positive = self.compose(attrs, objs)\n negative = self.compose(neg_attrs, neg_objs)\n loss = F.triplet_margin_loss(img_feats, positive, negative, margin = self.args.margin)\n\n # Auxiliary object/ attribute prediction loss both need to be correct\n if self.args.lambda_aux > 0:\n obj_pred = self.obj_clf(positive)\n attr_pred = self.attr_clf(positive)\n loss_aux = F.cross_entropy(attr_pred, attrs) + F.cross_entropy(obj_pred, objs)\n loss += self.args.lambda_aux * loss_aux\n\n return loss, None\n\n\n def val_forward_distance(self, x):\n img = x[0]\n batch_size = img.shape[0]\n\n img_feats = self.image_embedder(img)\n scores = {}\n pair_embeds = self.compose(self.val_attrs, self.val_objs)\n \n for itr, pair in enumerate(self.dset.pairs):\n pair_embed = pair_embeds[itr, None].expand(batch_size, pair_embeds.size(1))\n score = self.compare_metric(img_feats, pair_embed)\n scores[pair] = score\n\n return None, scores\n\n def val_forward_distance_fast(self, x):\n img = x[0]\n batch_size = img.shape[0]\n\n img_feats = self.image_embedder(img)\n pair_embeds = self.compose(self.val_attrs, self.val_objs) # Evaluate all pairs\n\n batch_size, pairs, features = img_feats.shape[0], pair_embeds.shape[0], pair_embeds.shape[1]\n img_feats = img_feats[:,None,:].expand(-1, pairs, -1)\n pair_embeds = pair_embeds[None,:,:].expand(batch_size, -1, -1)\n diff = (img_feats - pair_embeds)**2\n score = diff.sum(2) * -1\n\n scores = {}\n for itr, pair in enumerate(self.dset.pairs):\n scores[pair] = score[:,self.dset.all_pair2idx[pair]]\n\n return None, scores\n \n def val_forward_direct(self, x):\n img = x[0]\n batch_size = img.shape[0]\n\n img_feats = self.image_embedder(img)\n pair_embeds = self.compose(self.val_attrs, self.val_objs).permute(1,0) # Evaluate all pairs\n score = torch.matmul(img_feats, pair_embeds)\n\n scores = {}\n for itr, pair in enumerate(self.dset.pairs):\n scores[pair] = score[:,self.dset.all_pair2idx[pair]]\n\n return None, scores\n\n def forward(self, x):\n if self.training:\n loss, pred = self.train_forward(x)\n else:\n with torch.no_grad():\n loss, pred = self.val_forward(x)\n return loss, pred\n\n#########################################\nclass RedWine(ManifoldModel):\n\n def __init__(self, dset, args):\n super(RedWine, self).__init__(dset, args)\n self.image_embedder = lambda img: img\n self.compare_metric = lambda img_feats, pair_embed: torch.sigmoid((img_feats*pair_embed).sum(1))\n self.train_forward = self.train_forward_bce\n self.val_forward = self.val_forward_distance\n\n in_dim = dset.feat_dim if not args.glove_init else 300\n self.T = nn.Sequential(\n nn.Linear(2*in_dim, 3*in_dim),\n nn.LeakyReLU(0.1, True),\n nn.Linear(3*in_dim, 3*in_dim//2),\n nn.LeakyReLU(0.1, True),\n nn.Linear(3*in_dim//2, dset.feat_dim))\n\n self.attr_embedder = nn.Embedding(len(dset.attrs), in_dim)\n self.obj_embedder = nn.Embedding(len(dset.objs), in_dim)\n\n # initialize the weights of the embedders with the svm weights\n if args.glove_init:\n pretrained_weight = load_word_embeddings(DATA_FOLDER+'/glove/glove.6B.300d.txt', dset.attrs)\n self.attr_embedder.weight.data.copy_(pretrained_weight)\n pretrained_weight = load_word_embeddings(DATA_FOLDER+'/glove/glove.6B.300d.txt', dset.objs)\n self.obj_embedder.weight.data.copy_(pretrained_weight)\n\n elif args.clf_init:\n for idx, attr in enumerate(dset.attrs):\n at_id = self.dset.attr2idx[attr]\n weight = torch.load('%s/svm/attr_%d'%(args.data_dir, at_id)).coef_.squeeze()\n self.attr_embedder.weight[idx].data.copy_(torch.from_numpy(weight))\n for idx, obj in enumerate(dset.objs):\n obj_id = self.dset.obj2idx[obj]\n weight = torch.load('%s/svm/obj_%d'%(args.data_dir, obj_id)).coef_.squeeze()\n self.obj_embedder.weight[idx].data.copy_(torch.from_numpy(weight))\n else:\n print ('init must be either glove or clf')\n return\n\n if args.static_inp:\n for param in self.attr_embedder.parameters():\n param.requires_grad = False\n for param in self.obj_embedder.parameters():\n param.requires_grad = False\n\n def compose(self, attrs, objs):\n attr_wt = self.attr_embedder(attrs)\n obj_wt = self.obj_embedder(objs)\n inp_wts = torch.cat([attr_wt, obj_wt], 1) # 2D\n composed_clf = self.T(inp_wts)\n return composed_clf\n\nclass LabelEmbedPlus(ManifoldModel):\n def __init__(self, dset, args):\n super(LabelEmbedPlus, self).__init__(dset, args)\n if 'conv' in args.image_extractor:\n self.image_embedder = torch.nn.Sequential(torch.nn.Conv2d(dset.feat_dim,args.emb_dim,7),\n torch.nn.ReLU(True),\n Reshape(-1,args.emb_dim)\n )\n else:\n self.image_embedder = MLP(dset.feat_dim, args.emb_dim)\n\n self.compare_metric = lambda img_feats, pair_embed: -F.pairwise_distance(img_feats, pair_embed)\n self.train_forward = self.train_forward_triplet\n self.val_forward = self.val_forward_distance_fast\n\n input_dim = dset.feat_dim if args.clf_init else args.emb_dim\n self.attr_embedder = nn.Embedding(len(dset.attrs), input_dim)\n self.obj_embedder = nn.Embedding(len(dset.objs), input_dim)\n self.T = MLP(2*input_dim, args.emb_dim, num_layers= args.nlayers)\n\n # init with word embeddings\n if args.emb_init:\n pretrained_weight = load_word_embeddings(args.emb_init, dset.attrs)\n self.attr_embedder.weight.data.copy_(pretrained_weight)\n pretrained_weight = load_word_embeddings(args.emb_init, dset.objs)\n self.obj_embedder.weight.data.copy_(pretrained_weight)\n\n # init with classifier weights\n elif args.clf_init:\n for idx, attr in enumerate(dset.attrs):\n at_id = dset.attrs.index(attr)\n weight = torch.load('%s/svm/attr_%d'%(args.data_dir, at_id)).coef_.squeeze()\n self.attr_embedder.weight[idx].data.copy_(torch.from_numpy(weight))\n for idx, obj in enumerate(dset.objs):\n obj_id = dset.objs.index(obj)\n weight = torch.load('%s/svm/obj_%d'%(args.data_dir, obj_id)).coef_.squeeze()\n self.obj_emb.weight[idx].data.copy_(torch.from_numpy(weight))\n\n # static inputs\n if args.static_inp:\n for param in self.attr_embedder.parameters():\n param.requires_grad = False\n for param in self.obj_embedder.parameters():\n param.requires_grad = False\n\n def compose(self, attrs, objs):\n inputs = [self.attr_embedder(attrs), self.obj_embedder(objs)]\n inputs = torch.cat(inputs, 1)\n output = self.T(inputs)\n return output\n\nclass AttributeOperator(ManifoldModel):\n def __init__(self, dset, args):\n super(AttributeOperator, self).__init__(dset, args)\n self.image_embedder = MLP(dset.feat_dim, args.emb_dim)\n self.compare_metric = lambda img_feats, pair_embed: -F.pairwise_distance(img_feats, pair_embed)\n self.val_forward = self.val_forward_distance_fast\n\n self.attr_ops = nn.ParameterList([nn.Parameter(torch.eye(args.emb_dim)) for _ in range(len(self.dset.attrs))])\n self.obj_embedder = nn.Embedding(len(dset.objs), args.emb_dim)\n\n if args.emb_init:\n pretrained_weight = load_word_embeddings('glove', dset.objs)\n self.obj_embedder.weight.data.copy_(pretrained_weight)\n \n self.inverse_cache = {}\n\n if args.lambda_ant>0 and args.dataset=='mitstates':\n antonym_list = open(DATA_FOLDER+'/data/antonyms.txt').read().strip().split('\\n')\n antonym_list = [l.split() for l in antonym_list]\n antonym_list = [[self.dset.attrs.index(a1), self.dset.attrs.index(a2)] for a1, a2 in antonym_list]\n antonyms = {}\n antonyms.update({a1:a2 for a1, a2 in antonym_list})\n antonyms.update({a2:a1 for a1, a2 in antonym_list})\n self.antonyms, self.antonym_list = antonyms, antonym_list\n\n if args.static_inp:\n for param in self.obj_embedder.parameters():\n param.requires_grad = False\n\n def apply_ops(self, ops, rep):\n out = torch.bmm(ops, rep.unsqueeze(2)).squeeze(2)\n out = F.relu(out)\n return out\n\n def compose(self, attrs, objs):\n obj_rep = self.obj_embedder(objs)\n attr_ops = torch.stack([self.attr_ops[attr.item()] for attr in attrs])\n embedded_reps = self.apply_ops(attr_ops, obj_rep)\n return embedded_reps\n\n def apply_inverse(self, img_rep, attrs):\n inverse_ops = []\n for i in range(img_rep.size(0)):\n attr = attrs[i]\n if attr not in self.inverse_cache:\n self.inverse_cache[attr] = self.attr_ops[attr].inverse()\n inverse_ops.append(self.inverse_cache[attr])\n inverse_ops = torch.stack(inverse_ops) # (B,512,512)\n obj_rep = self.apply_ops(inverse_ops, img_rep)\n return obj_rep\n\n def train_forward(self, x):\n img, attrs, objs = x[0], x[1], x[2]\n neg_attrs, neg_objs, inv_attrs, comm_attrs = x[4][:,0], x[5][:,0], x[6], x[7]\n batch_size = img.size(0)\n\n loss = []\n\n anchor = self.image_embedder(img)\n\n obj_emb = self.obj_embedder(objs)\n pos_ops = torch.stack([self.attr_ops[attr.item()] for attr in attrs])\n positive = self.apply_ops(pos_ops, obj_emb)\n\n neg_obj_emb = self.obj_embedder(neg_objs)\n neg_ops = torch.stack([self.attr_ops[attr.item()] for attr in neg_attrs])\n negative = self.apply_ops(neg_ops, neg_obj_emb)\n\n loss_triplet = F.triplet_margin_loss(anchor, positive, negative, margin=self.args.margin)\n loss.append(loss_triplet)\n\n\n # Auxiliary object/attribute loss\n if self.args.lambda_aux>0:\n obj_pred = self.obj_clf(positive)\n attr_pred = self.attr_clf(positive)\n loss_aux = F.cross_entropy(attr_pred, attrs) + F.cross_entropy(obj_pred, objs)\n loss.append(self.args.lambda_aux*loss_aux)\n\n # Inverse Consistency\n if self.args.lambda_inv>0:\n obj_rep = self.apply_inverse(anchor, attrs)\n new_ops = torch.stack([self.attr_ops[attr.item()] for attr in inv_attrs])\n new_rep = self.apply_ops(new_ops, obj_rep)\n new_positive = self.apply_ops(new_ops, obj_emb)\n loss_inv = F.triplet_margin_loss(new_rep, new_positive, positive, margin=self.args.margin)\n loss.append(self.args.lambda_inv*loss_inv)\n\n # Commutative Operators\n if self.args.lambda_comm>0:\n B = torch.stack([self.attr_ops[attr.item()] for attr in comm_attrs])\n BA = self.apply_ops(B, positive)\n AB = self.apply_ops(pos_ops, self.apply_ops(B, obj_emb))\n loss_comm = ((AB-BA)**2).sum(1).mean()\n loss.append(self.args.lambda_comm*loss_comm)\n\n # Antonym Consistency\n if self.args.lambda_ant>0:\n\n select_idx = [i for i in range(batch_size) if attrs[i].item() in self.antonyms]\n if len(select_idx)>0:\n select_idx = torch.LongTensor(select_idx).cuda()\n attr_subset = attrs[select_idx]\n antonym_ops = torch.stack([self.attr_ops[self.antonyms[attr.item()]] for attr in attr_subset])\n\n Ao = anchor[select_idx]\n if self.args.lambda_inv>0:\n o = obj_rep[select_idx]\n else:\n o = self.apply_inverse(Ao, attr_subset)\n BAo = self.apply_ops(antonym_ops, Ao)\n\n loss_cycle = ((BAo-o)**2).sum(1).mean()\n loss.append(self.args.lambda_ant*loss_cycle)\n\n\n loss = sum(loss)\n return loss, None\n\n def forward(self, x):\n loss, pred = super(AttributeOperator, self).forward(x)\n self.inverse_cache = {}\n return loss, pred"
},
{
"path": "czsl/models/modular_methods.py",
"content": "# Copyright (c) Facebook, Inc. and its affiliates.\n# All rights reserved.\n#\n# This source code is licensed under the license found in the\n# LICENSE file in the root directory of this source tree.\n#\nimport torch\nimport torch.nn as nn\nimport torch.nn.functional as F\nimport torchvision.models as tmodels\nimport numpy as np\nfrom .word_embedding import load_word_embeddings\nimport itertools\nimport math\nimport collections\nfrom torch.distributions.bernoulli import Bernoulli\nimport pdb\nimport sys\n\n\nclass Flatten(nn.Module):\n def forward(self, input):\n return input.view(input.size(0), -1)\n\n\nclass GatingSampler(nn.Module):\n \"\"\"Docstring for GatingSampler. \"\"\"\n\n def __init__(self, gater, stoch_sample=True, temperature=1.0):\n \"\"\"TODO: to be defined1.\n\n \"\"\"\n nn.Module.__init__(self)\n self.gater = gater\n self._stoch_sample = stoch_sample\n self._temperature = temperature\n\n def disable_stochastic_sampling(self):\n self._stoch_sample = False\n\n def enable_stochastic_sampling(self):\n self._stoch_sample = True\n\n def forward(self, tdesc=None, return_additional=False, gating_wt=None):\n if self.gater is None and return_additional:\n return None, None\n elif self.gater is None:\n return None\n\n if gating_wt is not None:\n return_wts = gating_wt\n gating_g = self.gater(tdesc, gating_wt=gating_wt)\n else:\n gating_g = self.gater(tdesc)\n return_wts = None\n if isinstance(gating_g, tuple):\n return_wts = gating_g[1]\n gating_g = gating_g[0]\n\n if not self._stoch_sample:\n sampled_g = gating_g\n else:\n raise (NotImplementedError)\n\n if return_additional:\n return sampled_g, return_wts\n return sampled_g\n\n\nclass GatedModularNet(nn.Module):\n \"\"\"\n An interface for creating modular nets\n \"\"\"\n\n def __init__(self,\n module_list,\n start_modules=None,\n end_modules=None,\n single_head=False,\n chain=False):\n \"\"\"TODO: to be defined1.\n\n :module_list: TODO\n :g: TODO\n\n \"\"\"\n nn.Module.__init__(self)\n self._module_list = nn.ModuleList(\n [nn.ModuleList(m) for m in module_list])\n\n self.num_layers = len(self._module_list)\n if start_modules is not None:\n self._start_modules = nn.ModuleList(start_modules)\n else:\n self._start_modules = None\n\n if end_modules is not None:\n self._end_modules = nn.ModuleList(end_modules)\n self.num_layers += 1\n else:\n self._end_modules = None\n\n self.sampled_g = None\n self.single_head = single_head\n self._chain = chain\n\n def forward(self, x, sampled_g=None, t=None, return_feat=False):\n \"\"\"TODO: Docstring for forward.\n\n :x: Input data\n :g: Gating tensor (#Task x )#num_layer x #num_mods x #num_mods\n :t: task ID\n :returns: TODO\n\n \"\"\"\n\n if t is None:\n t_not_set = True\n t = torch.tensor([0] * x.shape[0], dtype=x.dtype).long()\n else:\n t_not_set = False\n t = t.squeeze()\n\n if self._start_modules is not None:\n prev_out = [mod(x) for mod in self._start_modules]\n else:\n prev_out = [x]\n\n if sampled_g is None:\n # NON-Gated Module network\n prev_out = sum(prev_out) / float(len(prev_out))\n #prev_out = torch.mean(prev_out, 0)\n for li in range(len(self._module_list)):\n prev_out = sum([\n mod(prev_out) for mod in self._module_list[li]\n ]) / float(len(self._module_list[li]))\n features = prev_out\n if self._end_modules is not None:\n if t_not_set or self.single_head:\n prev_out = self._end_modules[0](prev_out)\n else:\n prev_out = torch.cat([\n self._end_modules[tid](prev_out[bi:bi + 1])\n for bi, tid in enumerate(t)\n ], 0)\n if return_feat:\n return prev_out, features\n return prev_out\n else:\n # Forward prop with sampled Gs\n for li in range(len(self._module_list)):\n curr_out = []\n for j in range(len(self._module_list[li])):\n gind = j if not self._chain else 0\n # Dim: #Batch x C\n module_in_wt = sampled_g[li + 1][gind]\n # Module input weights rearranged to match inputs\n module_in_wt = module_in_wt.transpose(0, 1)\n add_dims = prev_out[0].dim() + 1 - module_in_wt.dim()\n module_in_wt = module_in_wt.view(*module_in_wt.shape,\n *([1] * add_dims))\n module_in_wt = module_in_wt.expand(\n len(prev_out), *prev_out[0].shape)\n module_in = sum([\n module_in_wt[i] * prev_out[i]\n for i in range(len(prev_out))\n ])\n mod = self._module_list[li][j]\n curr_out.append(mod(module_in))\n prev_out = curr_out\n\n # Output modules (with sampled Gs)\n if self._end_modules is not None:\n li = self.num_layers - 1\n if t_not_set or self.single_head:\n # Dim: #Batch x C\n module_in_wt = sampled_g[li + 1][0]\n # Module input weights rearranged to match inputs\n module_in_wt = module_in_wt.transpose(0, 1)\n add_dims = prev_out[0].dim() + 1 - module_in_wt.dim()\n module_in_wt = module_in_wt.view(*module_in_wt.shape,\n *([1] * add_dims))\n module_in_wt = module_in_wt.expand(\n len(prev_out), *prev_out[0].shape)\n module_in = sum([\n module_in_wt[i] * prev_out[i]\n for i in range(len(prev_out))\n ])\n features = module_in\n prev_out = self._end_modules[0](module_in)\n else:\n curr_out = []\n for bi, tid in enumerate(t):\n # Dim: #Batch x C\n gind = tid if not self._chain else 0\n module_in_wt = sampled_g[li + 1][gind]\n # Module input weights rearranged to match inputs\n module_in_wt = module_in_wt.transpose(0, 1)\n add_dims = prev_out[0].dim() + 1 - module_in_wt.dim()\n module_in_wt = module_in_wt.view(\n *module_in_wt.shape, *([1] * add_dims))\n module_in_wt = module_in_wt.expand(\n len(prev_out), *prev_out[0].shape)\n module_in = sum([\n module_in_wt[i] * prev_out[i]\n for i in range(len(prev_out))\n ])\n features = module_in\n mod = self._end_modules[tid]\n curr_out.append(mod(module_in[bi:bi + 1]))\n prev_out = curr_out\n prev_out = torch.cat(prev_out, 0)\n if return_feat:\n return prev_out, features\n return prev_out\n\n\nclass CompositionalModel(nn.Module):\n def __init__(self, dset, args):\n super(CompositionalModel, self).__init__()\n self.args = args\n self.dset = dset\n\n # precompute validation pairs\n attrs, objs = zip(*self.dset.pairs)\n attrs = [dset.attr2idx[attr] for attr in attrs]\n objs = [dset.obj2idx[obj] for obj in objs]\n self.val_attrs = torch.LongTensor(attrs).cuda()\n self.val_objs = torch.LongTensor(objs).cuda()\n\n def train_forward_softmax(self, x):\n img, attrs, objs = x[0], x[1], x[2]\n neg_attrs, neg_objs = x[4], x[5]\n inv_attrs, comm_attrs = x[6], x[7]\n\n sampled_attrs = torch.cat((attrs.unsqueeze(1), neg_attrs), 1)\n sampled_objs = torch.cat((objs.unsqueeze(1), neg_objs), 1)\n img_ind = torch.arange(sampled_objs.shape[0]).unsqueeze(1).repeat(\n 1, sampled_attrs.shape[1])\n\n flat_sampled_attrs = sampled_attrs.view(-1)\n flat_sampled_objs = sampled_objs.view(-1)\n flat_img_ind = img_ind.view(-1)\n labels = torch.zeros_like(sampled_attrs[:, 0]).long()\n\n self.composed_g = self.compose(flat_sampled_attrs, flat_sampled_objs)\n\n cls_scores, feat = self.comp_network(\n img[flat_img_ind], self.composed_g, return_feat=True)\n pair_scores = cls_scores[:, :1]\n pair_scores = pair_scores.view(*sampled_attrs.shape)\n\n loss = 0\n loss_cls = F.cross_entropy(pair_scores, labels)\n loss += loss_cls\n\n loss_obj = torch.FloatTensor([0])\n loss_attr = torch.FloatTensor([0])\n loss_sparse = torch.FloatTensor([0])\n loss_unif = torch.FloatTensor([0])\n loss_aux = torch.FloatTensor([0])\n\n acc = (pair_scores.argmax(1) == labels).sum().float() / float(\n len(labels))\n all_losses = {}\n all_losses['total_loss'] = loss\n all_losses['main_loss'] = loss_cls\n all_losses['aux_loss'] = loss_aux\n all_losses['obj_loss'] = loss_obj\n all_losses['attr_loss'] = loss_attr\n all_losses['sparse_loss'] = loss_sparse\n all_losses['unif_loss'] = loss_unif\n\n return loss, all_losses, acc, (pair_scores, feat)\n\n def val_forward(self, x):\n img = x[0]\n batch_size = img.shape[0]\n pair_scores = torch.zeros(batch_size, len(self.val_attrs))\n pair_feats = torch.zeros(batch_size, len(self.val_attrs),\n self.args.emb_dim)\n pair_bs = len(self.val_attrs)\n\n for pi in range(math.ceil(len(self.val_attrs) / pair_bs)):\n self.compose_g = self.compose(\n self.val_attrs[pi * pair_bs:(pi + 1) * pair_bs],\n self.val_objs[pi * pair_bs:(pi + 1) * pair_bs])\n compose_g = self.compose_g\n expanded_im = img.unsqueeze(1).repeat(\n 1, compose_g[0][0].shape[0],\n *tuple([1] * (img.dim() - 1))).view(-1, *img.shape[1:])\n expanded_compose_g = [[\n g.unsqueeze(0).repeat(batch_size, *tuple([1] * g.dim())).view(\n -1, *g.shape[1:]) for g in layer_g\n ] for layer_g in compose_g]\n this_pair_scores, this_feat = self.comp_network(\n expanded_im, expanded_compose_g, return_feat=True)\n featnorm = torch.norm(this_feat, p=2, dim=-1)\n this_feat = this_feat.div(\n featnorm.unsqueeze(-1).expand_as(this_feat))\n\n this_pair_scores = this_pair_scores[:, :1].view(batch_size, -1)\n this_feat = this_feat.view(batch_size, -1, self.args.emb_dim)\n\n pair_scores[:, pi * pair_bs:pi * pair_bs +\n this_pair_scores.shape[1]] = this_pair_scores[:, :]\n pair_feats[:, pi * pair_bs:pi * pair_bs +\n this_pair_scores.shape[1], :] = this_feat[:]\n\n scores = {}\n feats = {}\n for i, (attr, obj) in enumerate(self.dset.pairs):\n scores[(attr, obj)] = pair_scores[:, i]\n feats[(attr, obj)] = pair_feats[:, i]\n\n # return None, (scores, feats)\n return None, scores\n\n def forward(self, x, with_grad=False):\n if self.training:\n loss, loss_aux, acc, pred = self.train_forward(x)\n else:\n loss_aux = torch.Tensor([0])\n loss = torch.Tensor([0])\n if not with_grad:\n with torch.no_grad():\n acc, pred = self.val_forward(x)\n else:\n acc, pred = self.val_forward(x)\n # return loss, loss_aux, acc, pred\n return loss, pred\n\n\nclass GatedGeneralNN(CompositionalModel):\n \"\"\"Docstring for GatedCompositionalModel. \"\"\"\n\n def __init__(self,\n dset,\n args,\n num_layers=2,\n num_modules_per_layer=3,\n stoch_sample=False,\n use_full_model=False,\n num_classes=[2],\n gater_type='general'):\n \"\"\"TODO: to be defined1.\n\n :dset: TODO\n :args: TODO\n\n \"\"\"\n CompositionalModel.__init__(self, dset, args)\n\n self.train_forward = self.train_forward_softmax\n self.compose_type = 'nn' #todo: could be different\n\n gating_in_dim = 128\n if args.emb_init:\n gating_in_dim = 300\n elif args.clf_init:\n gating_in_dim = 512\n\n if self.compose_type == 'nn':\n tdim = gating_in_dim * 2\n inter_tdim = self.args.embed_rank\n # Change this to allow only obj, only attr gatings\n self.attr_embedder = nn.Embedding(\n len(dset.attrs) + 1,\n gating_in_dim,\n padding_idx=len(dset.attrs),\n )\n self.obj_embedder = nn.Embedding(\n len(dset.objs) + 1,\n gating_in_dim,\n padding_idx=len(dset.objs),\n )\n\n # initialize the weights of the embedders with the svm weights\n if args.emb_init:\n pretrained_weight = load_word_embeddings(\n args.emb_init, dset.attrs)\n self.attr_embedder.weight[:-1, :].data.copy_(pretrained_weight)\n pretrained_weight = load_word_embeddings(\n args.emb_init, dset.objs)\n self.obj_embedder.weight.data[:-1, :].copy_(pretrained_weight)\n elif args.clf_init:\n for idx, attr in enumerate(dset.attrs):\n at_id = self.dset.attr2idx[attr]\n weight = torch.load(\n '%s/svm/attr_%d' % (args.data_dir,\n at_id)).coef_.squeeze()\n self.attr_embedder.weight[idx].data.copy_(\n torch.from_numpy(weight))\n for idx, obj in enumerate(dset.objs):\n obj_id = self.dset.obj2idx[obj]\n weight = torch.load(\n '%s/svm/obj_%d' % (args.data_dir,\n obj_id)).coef_.squeeze()\n self.obj_embedder.weight[idx].data.copy_(\n torch.from_numpy(weight))\n else:\n n_attr = len(dset.attrs)\n gating_in_dim = 300\n tdim = gating_in_dim * 2 + n_attr\n self.attr_embedder = nn.Embedding(\n n_attr,\n n_attr,\n )\n self.attr_embedder.weight.data.copy_(\n torch.from_numpy(np.eye(n_attr)))\n self.obj_embedder = nn.Embedding(\n len(dset.objs) + 1,\n gating_in_dim,\n padding_idx=len(dset.objs),\n )\n pretrained_weight = load_word_embeddings(\n '/home/ubuntu/workspace/czsl/data/glove/glove.6B.300d.txt', dset.objs)\n self.obj_embedder.weight.data[:-1, :].copy_(pretrained_weight)\n else:\n raise (NotImplementedError)\n\n self.comp_network, self.gating_network, self.nummods, _ = modular_general(\n num_layers=num_layers,\n num_modules_per_layer=num_modules_per_layer,\n feat_dim=dset.feat_dim,\n inter_dim=args.emb_dim,\n stoch_sample=stoch_sample,\n use_full_model=use_full_model,\n tdim=tdim,\n inter_tdim=inter_tdim,\n gater_type=gater_type,\n )\n\n if args.static_inp:\n for param in self.attr_embedder.parameters():\n param.requires_grad = False\n for param in self.obj_embedder.parameters():\n param.requires_grad = False\n\n def compose(self, attrs, objs):\n obj_wt = self.obj_embedder(objs)\n if self.compose_type == 'nn':\n attr_wt = self.attr_embedder(attrs)\n inp_wts = torch.cat([attr_wt, obj_wt], 1) # 2D\n else:\n raise (NotImplementedError)\n composed_g, composed_g_wt = self.gating_network(\n inp_wts, return_additional=True)\n\n return composed_g\n\n\nclass GeneralNormalizedNN(nn.Module):\n \"\"\"Docstring for GatedCompositionalModel. \"\"\"\n\n def __init__(self, num_layers, num_modules_per_layer, in_dim, inter_dim):\n \"\"\"TODO: to be defined1. \"\"\"\n nn.Module.__init__(self)\n self.start_modules = [nn.Sequential()]\n self.layer1 = [[\n nn.Sequential(\n nn.Linear(in_dim, inter_dim), nn.BatchNorm1d(inter_dim),\n nn.ReLU()) for _ in range(num_modules_per_layer)\n ]]\n if num_layers > 1:\n self.layer2 = [[\n nn.Sequential(\n nn.BatchNorm1d(inter_dim), nn.Linear(inter_dim, inter_dim),\n nn.BatchNorm1d(inter_dim), nn.ReLU())\n for _m in range(num_modules_per_layer)\n ] for _l in range(num_layers - 1)]\n self.avgpool = nn.Sequential()\n self.fc = [\n nn.Sequential(nn.BatchNorm1d(inter_dim), nn.Linear(inter_dim, 1))\n ]\n\n\nclass GeneralGatingNN(nn.Module):\n def __init__(\n self,\n num_mods,\n tdim,\n inter_tdim,\n randinit=False,\n ):\n \"\"\"TODO: to be defined1.\n\n :num_mods: TODO\n :tdim: TODO\n\n \"\"\"\n nn.Module.__init__(self)\n\n self._num_mods = num_mods\n self._tdim = tdim\n self._inter_tdim = inter_tdim\n task_outdim = self._inter_tdim\n\n self.task_linear1 = nn.Linear(self._tdim, task_outdim, bias=False)\n self.task_bn1 = nn.BatchNorm1d(task_outdim)\n self.task_linear2 = nn.Linear(task_outdim, task_outdim, bias=False)\n self.task_bn2 = nn.BatchNorm1d(task_outdim)\n self.joint_linear1 = nn.Linear(task_outdim, task_outdim, bias=False)\n self.joint_bn1 = nn.BatchNorm1d(task_outdim)\n\n num_out = [[1]] + [[\n self._num_mods[i - 1] for _ in range(self._num_mods[i])\n ] for i in range(1, len(self._num_mods))]\n count = 0\n out_ind = []\n for i in range(len(num_out)):\n this_out_ind = []\n for j in range(len(num_out[i])):\n this_out_ind.append([count, count + num_out[i][j]])\n count += num_out[i][j]\n out_ind.append(this_out_ind)\n self.out_ind = out_ind\n self.out_count = count\n\n self.joint_linear2 = nn.Linear(task_outdim, count, bias=False)\n\n def apply_init(m):\n if isinstance(m, nn.Linear):\n nn.init.normal_(m.weight, 0, 0.1)\n if m.bias is not None:\n nn.init.constant_(m.bias, 0)\n elif isinstance(m, nn.BatchNorm1d):\n nn.init.constant_(m.weight, 1)\n nn.init.constant_(m.bias, 0)\n\n for m in self.modules():\n if isinstance(m, nn.ModuleList):\n for subm in m:\n if isinstance(subm, nn.ModuleList):\n for subsubm in subm:\n apply_init(subsubm)\n else:\n apply_init(subm)\n else:\n apply_init(m)\n\n if not randinit:\n self.joint_linear2.weight.data.zero_()\n\n def forward(self, tdesc=None):\n \"\"\"TODO: Docstring for function.\n\n :arg1: TODO\n :returns: TODO\n\n \"\"\"\n if tdesc is None:\n return None\n\n x = tdesc\n task_embeds1 = F.relu(self.task_bn1(self.task_linear1(x)))\n joint_embed = task_embeds1\n joint_embed = self.joint_linear2(joint_embed)\n joint_embed = [[\n joint_embed[:, self.out_ind[i][j][0]:self.out_ind[i][j][1]]\n for j in range(len(self.out_ind[i]))\n ] for i in range(len(self.out_ind))]\n\n gating_wt = joint_embed\n prob_g = [[F.softmax(wt, -1) for wt in gating_wt[i]]\n for i in range(len(gating_wt))]\n return prob_g, gating_wt\n\n\ndef modularize_network(\n model,\n stoch_sample=False,\n use_full_model=False,\n tdim=200,\n inter_tdim=200,\n gater_type='general',\n single_head=True,\n num_classes=[2],\n num_lookup_gating=10,\n):\n # Copy start modules and end modules\n start_modules = model.start_modules\n end_modules = [\n nn.Sequential(model.avgpool, Flatten(), fci) for fci in model.fc\n ]\n\n # Create module_list as list of lists [[layer1 modules], [layer2 modules], ...]\n module_list = []\n li = 1\n while True:\n if hasattr(model, 'layer{}'.format(li)):\n module_list.extend(getattr(model, 'layer{}'.format(li)))\n li += 1\n else:\n break\n\n num_module_list = [len(start_modules)] + [len(layer) for layer in module_list] \\\n + [len(end_modules)]\n gated_model_func = GatedModularNet\n gated_net = gated_model_func(\n module_list,\n start_modules=start_modules,\n end_modules=end_modules,\n single_head=single_head)\n\n gater_func = GeneralGatingNN\n gater = gater_func(\n num_mods=num_module_list, tdim=tdim, inter_tdim=inter_tdim)\n\n fan_in = num_module_list\n if use_full_model:\n gater = None\n\n # Create Gating Sampler\n gating_sampler = GatingSampler(gater=gater, stoch_sample=stoch_sample)\n return gated_net, gating_sampler, num_module_list, fan_in\n\n\ndef modular_general(\n num_layers,\n num_modules_per_layer,\n feat_dim,\n inter_dim,\n stoch_sample=False,\n use_full_model=False,\n num_classes=[2],\n single_head=True,\n gater_type='general',\n tdim=300,\n inter_tdim=300,\n num_lookup_gating=10,\n):\n # First create a ResNext model\n model = GeneralNormalizedNN(\n num_layers,\n num_modules_per_layer,\n feat_dim,\n inter_dim,\n )\n\n # Modularize the model and create gating funcs\n gated_net, gating_sampler, num_module_list, fan_in = modularize_network(\n model,\n stoch_sample,\n use_full_model=use_full_model,\n gater_type=gater_type,\n tdim=tdim,\n inter_tdim=inter_tdim,\n single_head=single_head,\n num_classes=num_classes,\n num_lookup_gating=num_lookup_gating,\n )\n return gated_net, gating_sampler, num_module_list, fan_in"
},
{
"path": "czsl/models/svm.py",
"content": "import numpy as np\nimport tqdm\nfrom data import dataset as dset\nimport os\nfrom utils import utils\nimport torch\nfrom torch.autograd import Variable\nimport h5py\nfrom sklearn.svm import LinearSVC\nfrom sklearn.model_selection import GridSearchCV\nimport torch.nn.functional as F\nfrom joblib import Parallel, delayed\nimport glob\nimport scipy.io\nfrom sklearn.calibration import CalibratedClassifierCV\n\n\nimport argparse\nparser = argparse.ArgumentParser()\nparser.add_argument('--batch_size', type=int, default=512)\nparser.add_argument('--dataset', default='mitstates', help='mitstates|zappos')\nparser.add_argument('--data_dir', default='data/mit-states/')\nparser.add_argument('--generate', action ='store_true', default=False)\nparser.add_argument('--evalsvm', action ='store_true', default=False)\nparser.add_argument('--evaltf', action ='store_true', default=False)\nparser.add_argument('--completed', default='tensor-completion/completed/complete.mat')\nargs = parser.parse_args()\n\n#----------------------------------------------------------------------------------------#\n\nsearch_params = {'C': np.logspace(-5,5,11)}\ndef train_svm(Y_train, sample_weight=None):\n \n try:\n clf = GridSearchCV(LinearSVC(class_weight='balanced', fit_intercept=False), search_params, scoring='f1', cv=4)\n clf.fit(X_train, Y_train, sample_weight=sample_weight)\n except:\n clf = LinearSVC(C=0.1, class_weight='balanced', fit_intercept=False)\n if Y_train.sum()==len(Y_train) or Y_train.sum()==0:\n return None\n clf.fit(X_train, Y_train, sample_weight=sample_weight)\n\n return clf\n\ndef generate_svms():\n # train an SVM for every attribute, object and pair primitive\n\n Y = [(train_attrs==attr).astype(np.int) for attr in range(len(dataset.attrs))]\n attr_clfs = Parallel(n_jobs=32, verbose=16)(delayed(train_svm)(Y[attr]) for attr in range(len(dataset.attrs)))\n for attr, clf in enumerate(attr_clfs):\n print (attr, dataset.attrs[attr])\n print ('params:', clf.best_params_)\n print ('-'*30)\n torch.save(clf.best_estimator_, '%s/svm/attr_%d'%(args.data_dir, attr))\n\n Y = [(train_objs==obj).astype(np.int) for obj in range(len(dataset.objs))]\n obj_clfs = Parallel(n_jobs=32, verbose=16)(delayed(train_svm)(Y[obj]) for obj in range(len(dataset.objs)))\n for obj, clf in enumerate(obj_clfs):\n print (obj, dataset.objs[obj])\n print ('params:', clf.best_params_)\n print ('-'*30)\n torch.save(clf.best_estimator_, '%s/svm/obj_%d'%(args.data_dir, obj))\n\n\n Y, Y_attr, sample_weight = [], [], []\n for idx, (attr, obj) in enumerate(dataset.train_pairs):\n Y_train = ((train_attrs==attr)*(train_objs==obj)).astype(np.int)\n\n # reweight instances to get a little more training data\n Y_train_attr = (train_attrs==attr).astype(np.int)\n instance_weights = 0.1*Y_train_attr\n instance_weights[Y_train.nonzero()[0]] = 1.0\n\n Y.append(Y_train)\n Y_attr.append(Y_train_attr)\n sample_weight.append(instance_weights)\n\n pair_clfs = Parallel(n_jobs=32, verbose=16)(delayed(train_svm)(Y[pair]) for pair in range(len(dataset.train_pairs)))\n for idx, (attr, obj) in enumerate(dataset.train_pairs):\n clf = pair_clfs[idx]\n print (dataset.attrs[attr], dataset.objs[obj])\n\n try:\n print ('params:', clf.best_params_)\n torch.save(clf.best_estimator_, '%s/svm/pair_%d_%d'%(args.data_dir, attr, obj))\n except:\n print ('FAILED! #positive:', Y[idx].sum(), len(Y[idx]))\n\n return\n\ndef make_svm_tensor():\n subs, vals, size = [], [], (len(dataset.attrs), len(dataset.objs), X.shape[1])\n fullsubs, fullvals = [], []\n composite_clfs = glob.glob('%s/svm/pair*'%args.data_dir)\n print ('%d composite classifiers found'%(len(composite_clfs)))\n for clf in tqdm.tqdm(composite_clfs):\n _, attr, obj = os.path.basename(clf).split('_')\n attr, obj = int(attr), int(obj)\n\n clf = torch.load(clf)\n weight = clf.coef_.squeeze()\n for i in range(len(weight)):\n subs.append((attr, obj, i))\n vals.append(weight[i])\n\n for attr, obj in dataset.pairs:\n for i in range(X.shape[1]):\n fullsubs.append((attr, obj, i))\n\n subs, vals = np.array(subs), np.array(vals).reshape(-1,1)\n fullsubs, fullvals = np.array(fullsubs), np.ones(len(fullsubs)).reshape(-1,1)\n savedat = {'subs':subs, 'vals':vals, 'size':size, 'fullsubs':fullsubs, 'fullvals':fullvals}\n scipy.io.savemat('tensor-completion/incomplete/%s.mat'%args.dataset, savedat)\n print (subs.shape, vals.shape, size)\n\ndef evaluate_svms():\n\n attr_clfs = [torch.load('%s/svm/attr_%d'%(args.data_dir, attr)) for attr in range(len(dataset.attrs))]\n obj_clfs = [torch.load('%s/svm/obj_%d'%(args.data_dir, obj)) for obj in range(len(dataset.objs))]\n\n # Calibrate all classifiers first\n Y = [(train_attrs==attr).astype(np.int) for attr in range(len(dataset.attrs))]\n for attr in tqdm.tqdm(range(len(dataset.attrs))):\n clf = attr_clfs[attr]\n calibrated = CalibratedClassifierCV(clf, method='sigmoid', cv='prefit')\n calibrated.fit(X_train, Y[attr])\n attr_clfs[attr] = calibrated\n\n Y = [(train_objs==obj).astype(np.int) for obj in range(len(dataset.objs))]\n for obj in tqdm.tqdm(range(len(dataset.objs))):\n clf = obj_clfs[obj]\n calibrated = CalibratedClassifierCV(clf, method='sigmoid', cv='prefit')\n calibrated.fit(X_train, Y[obj])\n obj_clfs[obj] = calibrated\n\n # Generate all the scores\n attr_scores, obj_scores = [], []\n for attr in tqdm.tqdm(range(len(dataset.attrs))):\n clf = attr_clfs[attr]\n score = clf.predict_proba(X_test)[:,1]\n attr_scores.append(score)\n attr_scores = np.vstack(attr_scores)\n\n for obj in tqdm.tqdm(range(len(dataset.objs))):\n clf = obj_clfs[obj]\n score = clf.predict_proba(X_test)[:,1]\n obj_scores.append(score)\n obj_scores = np.vstack(obj_scores)\n\n attr_pred = torch.from_numpy(attr_scores).transpose(0,1)\n obj_pred = torch.from_numpy(obj_scores).transpose(0,1)\n\n x = [None, Variable(torch.from_numpy(test_attrs)).long(), Variable(torch.from_numpy(test_objs)).long(), Variable(torch.from_numpy(test_pairs)).long()]\n attr_pred, obj_pred, _ = utils.generate_prediction_tensors([attr_pred, obj_pred], dataset, x[2].data, source='classification')\n attr_match, obj_match, zsl_match, gzsl_match, fixobj_match = utils.performance_stats(attr_pred, obj_pred, x)\n print (attr_match.mean(), obj_match.mean(), zsl_match.mean(), gzsl_match.mean(), fixobj_match.mean())\n\ndef evaluate_tensorcompletion():\n\n def parse_tensor(fl):\n tensor = scipy.io.loadmat(fl)\n nz_idx = zip(*(tensor['subs']))\n composite_clfs = np.zeros((len(dataset.attrs), len(dataset.objs), X.shape[1]))\n composite_clfs[nz_idx[0], nz_idx[1], nz_idx[2]] = tensor['vals'].squeeze()\n return composite_clfs, nz_idx, tensor['vals'].squeeze()\n\n # see recon error\n tr_file = 'tensor-completion/incomplete/%s.mat'%args.dataset\n ts_file = args.completed\n\n tr_clfs, tr_nz_idx, tr_vals = parse_tensor(tr_file)\n ts_clfs, ts_nz_idx, ts_vals = parse_tensor(ts_file)\n\n print (tr_vals.min(), tr_vals.max(), tr_vals.mean())\n print (ts_vals.min(), ts_vals.max(), ts_vals.mean())\n\n print ('Completed Tensor: %s'%args.completed)\n\n # see train recon error\n err = 1.0*((tr_clfs[tr_nz_idx[0], tr_nz_idx[1], tr_nz_idx[2]]-ts_clfs[tr_nz_idx[0], tr_nz_idx[1], tr_nz_idx[2]])**2).sum()/(len(tr_vals))\n print ('recon error:', err)\n\n # Create and scale classifiers for each pair\n clfs = {}\n test_pair_set = set(map(tuple, dataset.test_pairs.numpy().tolist()))\n for idx, (attr, obj) in tqdm.tqdm(enumerate(dataset.pairs), total=len(dataset.pairs)):\n clf = LinearSVC(fit_intercept=False)\n clf.fit(np.eye(2), [0,1])\n\n if (attr, obj) in test_pair_set:\n X_ = X_test\n Y_ = (test_attrs==attr).astype(np.int)*(test_objs==obj).astype(np.int)\n clf.coef_ = ts_clfs[attr, obj][None,:]\n else:\n X_ = X_train\n Y_ = (train_attrs==attr).astype(np.int)*(train_objs==obj).astype(np.int)\n clf.coef_ = tr_clfs[attr, obj][None,:]\n \n calibrated = CalibratedClassifierCV(clf, method='sigmoid', cv='prefit')\n calibrated.fit(X_, Y_)\n clfs[(attr, obj)] = calibrated\n\n scores = {}\n for attr, obj in tqdm.tqdm(dataset.pairs):\n score = clfs[(attr, obj)].predict_proba(X_test)[:,1]\n scores[(attr, obj)] = torch.from_numpy(score).float().unsqueeze(1)\n\n x = [None, Variable(torch.from_numpy(test_attrs)).long(), Variable(torch.from_numpy(test_objs)).long(), Variable(torch.from_numpy(test_pairs)).long()]\n attr_pred, obj_pred, _ = utils.generate_prediction_tensors(scores, dataset, x[2].data, source='manifold')\n attr_match, obj_match, zsl_match, gzsl_match, fixobj_match = utils.performance_stats(attr_pred, obj_pred, x)\n print (attr_match.mean(), obj_match.mean(), zsl_match.mean(), gzsl_match.mean(), fixobj_match.mean())\n\n\n\n#----------------------------------------------------------------------------------------#\nif args.dataset == 'mitstates':\n DSet = dset.MITStatesActivations\nelif args.dataset == 'zappos':\n DSet = dset.UTZapposActivations\n\ndataset = DSet(root=args.data_dir, phase='train')\ntrain_idx, train_attrs, train_objs, train_pairs = map(np.array, zip(*dataset.train_data))\ntest_idx, test_attrs, test_objs, test_pairs = map(np.array, zip(*dataset.test_data))\n\nX = dataset.activations.numpy()\nX_train, X_test = X[train_idx,:], X[test_idx,:]\n\nprint (len(dataset.attrs), len(dataset.objs), len(dataset.pairs))\nprint (X_train.shape, X_test.shape)\n\nif args.generate:\n generate_svms()\n make_svm_tensor()\n\nif args.evalsvm:\n evaluate_svms()\n\nif args.evaltf:\n evaluate_tensorcompletion()\n"
},
{
"path": "czsl/models/symnet.py",
"content": "import torch\nimport torch.nn as nn\nimport torch.nn.functional as F\nimport numpy as np\nfrom .word_embedding import load_word_embeddings\nfrom .common import MLP\n\ndevice = 'cuda' if torch.cuda.is_available() else 'cpu'\n\n\n\nclass Symnet(nn.Module):\n def __init__(self, dset, args):\n super(Symnet, self).__init__()\n self.dset = dset\n self.args = args\n self.num_attrs = len(dset.attrs)\n self.num_objs = len(dset.objs)\n\n self.image_embedder = MLP(dset.feat_dim, args.emb_dim, relu = False)\n self.ce_loss = nn.CrossEntropyLoss()\n self.mse_loss = nn.MSELoss()\n self.nll_loss = nn.NLLLoss()\n self.softmax = nn.Softmax(dim=1)\n # Attribute embedder and object classifier\n self.attr_embedder = nn.Embedding(len(dset.attrs), args.emb_dim)\n\n self.obj_classifier = MLP(args.emb_dim, len(dset.objs), num_layers = 2, relu = False, dropout = True, layers = [512]) # original paper uses a second fc classifier for final product\n self.attr_classifier = MLP(args.emb_dim, len(dset.attrs), num_layers = 2, relu = False, dropout = True, layers = [512]) \n # CoN and DecoN\n self.CoN_fc_attention = MLP(args.emb_dim, args.emb_dim, num_layers = 2, relu = False, dropout = True, layers = [512])\n self.CoN_emb = MLP(args.emb_dim + args.emb_dim, args.emb_dim, num_layers = 2, relu = False, dropout = True, layers = [768])\n self.DecoN_fc_attention = MLP(args.emb_dim, args.emb_dim, num_layers = 2, relu = False, dropout = True, layers = [512])\n self.DecoN_emb = MLP(args.emb_dim + args.emb_dim, args.emb_dim, num_layers = 2, relu = False, dropout = True, layers = [768])\n\n # if args.glove_init:\n pretrained_weight = load_word_embeddings(args.emb_init, dset.attrs)\n self.attr_embedder.weight.data.copy_(pretrained_weight)\n\n for param in self.attr_embedder.parameters():\n param.requires_grad = False\n\n def CoN(self, img_embedding, attr_embedding):\n attention = torch.sigmoid(self.CoN_fc_attention(attr_embedding))\n img_embedding = attention*img_embedding + img_embedding\n\n hidden = torch.cat([img_embedding, attr_embedding], dim = 1)\n output = self.CoN_emb(hidden)\n return output\n\n def DeCoN(self, img_embedding, attr_embedding):\n attention = torch.sigmoid(self.DecoN_fc_attention(attr_embedding))\n img_embedding = attention*img_embedding + img_embedding\n\n hidden = torch.cat([img_embedding, attr_embedding], dim = 1)\n output = self.DecoN_emb(hidden)\n return output\n\n\n def distance_metric(self, a, b):\n return torch.norm(a-b, dim = -1)\n\n def RMD_prob(self, feat_plus, feat_minus, repeat_img_feat):\n \"\"\"return attribute classification probability with our RMD\"\"\"\n # feat_plus, feat_minus: shape=(bz, #attr, dim_emb)\n # d_plus: distance between feature before&after CoN\n # d_minus: distance between feature before&after DecoN\n d_plus = self.distance_metric(feat_plus, repeat_img_feat).reshape(-1, self.num_attrs)\n d_minus = self.distance_metric(feat_minus, repeat_img_feat).reshape(-1, self.num_attrs)\n # not adding softmax because it is part of cross entropy loss\n return d_minus - d_plus\n\n def train_forward(self, x):\n pos_image_feat, pos_attr_id, pos_obj_id = x[0], x[1], x[2]\n neg_attr_id = x[4][:,0]\n\n batch_size = pos_image_feat.size(0)\n\n loss = []\n pos_attr_emb = self.attr_embedder(pos_attr_id)\n neg_attr_emb = self.attr_embedder(neg_attr_id)\n\n pos_img = self.image_embedder(pos_image_feat)\n\n # rA = remove positive attribute A\n # aA = add positive attribute A\n # rB = remove negative attribute B\n # aB = add negative attribute B\n pos_rA = self.DeCoN(pos_img, pos_attr_emb)\n pos_aA = self.CoN(pos_img, pos_attr_emb)\n pos_rB = self.DeCoN(pos_img, neg_attr_emb)\n pos_aB = self.CoN(pos_img, neg_attr_emb)\n\n # get all attr embedding #attr, embedding\n attr_emb = torch.LongTensor(np.arange(self.num_attrs)).to(device)\n attr_emb = self.attr_embedder(attr_emb)\n tile_attr_emb = attr_emb.repeat(batch_size, 1) # (batch*attr, dim_emb)\n\n # Now we calculate all the losses\n if self.args.lambda_cls_attr > 0:\n # Original image\n score_pos_A = self.attr_classifier(pos_img)\n loss_cls_pos_a = self.ce_loss(score_pos_A, pos_attr_id)\n\n # After removing pos_attr\n score_pos_rA_A = self.attr_classifier(pos_rA)\n total = sum(score_pos_rA_A)\n # prob_pos_rA_A = 1 - self.softmax(score_pos_rA_A)\n # loss_cls_pos_rA_a = self.nll_loss(torch.log(prob_pos_rA_A), pos_attr_id)\n loss_cls_pos_rA_a = self.ce_loss(total - score_pos_rA_A, pos_attr_id) #should be maximum for the gt label\n\n # rmd time\n repeat_img_feat = torch.repeat_interleave(pos_img, self.num_attrs, 0) #(batch*attr, dim_rep)\n feat_plus = self.CoN(repeat_img_feat, tile_attr_emb)\n feat_minus = self.DeCoN(repeat_img_feat, tile_attr_emb)\n score_cls_rmd = self.RMD_prob(feat_plus, feat_minus, repeat_img_feat)\n loss_cls_rmd = self.ce_loss(score_cls_rmd, pos_attr_id)\n\n loss_cls_attr = self.args.lambda_cls_attr*sum([loss_cls_pos_a, loss_cls_pos_rA_a, loss_cls_rmd])\n loss.append(loss_cls_attr)\n\n if self.args.lambda_cls_obj > 0:\n # Original image\n score_pos_O = self.obj_classifier(pos_img)\n loss_cls_pos_o = self.ce_loss(score_pos_O, pos_obj_id)\n\n # After removing pos attr\n score_pos_rA_O = self.obj_classifier(pos_rA)\n loss_cls_pos_rA_o = self.ce_loss(score_pos_rA_O, pos_obj_id)\n\n # After adding neg attr\n score_pos_aB_O = self.obj_classifier(pos_aB)\n loss_cls_pos_aB_o = self.ce_loss(score_pos_aB_O, pos_obj_id)\n\n loss_cls_obj = self.args.lambda_cls_obj * sum([loss_cls_pos_o, loss_cls_pos_rA_o, loss_cls_pos_aB_o])\n\n loss.append(loss_cls_obj)\n\n if self.args.lambda_sym > 0:\n\n loss_sys_pos = self.mse_loss(pos_aA, pos_img)\n loss_sys_neg = self.mse_loss(pos_rB, pos_img)\n loss_sym = self.args.lambda_sym * (loss_sys_pos + loss_sys_neg)\n\n loss.append(loss_sym)\n\n ##### Axiom losses\n if self.args.lambda_axiom > 0:\n loss_clo = loss_inv = loss_com = 0\n # closure\n pos_aA_rA = self.DeCoN(pos_aA, pos_attr_emb)\n pos_rB_aB = self.CoN(pos_rB, neg_attr_emb)\n loss_clo = self.mse_loss(pos_aA_rA, pos_rA) + self.mse_loss(pos_rB_aB, pos_aB)\n\n # invertibility\n pos_rA_aA = self.CoN(pos_rA, pos_attr_emb)\n pos_aB_rB = self.DeCoN(pos_aB, neg_attr_emb)\n loss_inv = self.mse_loss(pos_rA_aA, pos_img) + self.mse_loss(pos_aB_rB, pos_img)\n\n # commutative\n pos_aA_rB = self.DeCoN(pos_aA, neg_attr_emb)\n pos_rB_aA = self.DeCoN(pos_rB, pos_attr_emb)\n loss_com = self.mse_loss(pos_aA_rB, pos_rB_aA)\n\n loss_axiom = self.args.lambda_axiom * (loss_clo + loss_inv + loss_com)\n loss.append(loss_axiom)\n \n # triplet loss\n if self.args.lambda_trip > 0:\n pos_triplet = F.triplet_margin_loss(pos_img, pos_aA, pos_rA)\n neg_triplet = F.triplet_margin_loss(pos_img, pos_rB, pos_aB)\n\n loss_triplet = self.args.lambda_trip * (pos_triplet + neg_triplet)\n loss.append(loss_triplet)\n\n\n loss = sum(loss)\n return loss, None\n\n def val_forward(self, x):\n pos_image_feat, pos_attr_id, pos_obj_id = x[0], x[1], x[2]\n batch_size = pos_image_feat.shape[0]\n pos_img = self.image_embedder(pos_image_feat)\n repeat_img_feat = torch.repeat_interleave(pos_img, self.num_attrs, 0) #(batch*attr, dim_rep)\n \n # get all attr embedding #attr, embedding\n attr_emb = torch.LongTensor(np.arange(self.num_attrs)).to(device)\n attr_emb = self.attr_embedder(attr_emb)\n tile_attr_emb = attr_emb.repeat(batch_size, 1) # (batch*attr, dim_emb)\n\n feat_plus = self.CoN(repeat_img_feat, tile_attr_emb)\n feat_minus = self.DeCoN(repeat_img_feat, tile_attr_emb)\n score_cls_rmd = self.RMD_prob(feat_plus, feat_minus, repeat_img_feat)\n prob_A_rmd = F.softmax(score_cls_rmd, dim = 1)\n\n score_obj = self.obj_classifier(pos_img)\n prob_O = F.softmax(score_obj, dim = 1)\n\n scores = {}\n for itr, (attr, obj) in enumerate(self.dset.pairs):\n attr_id, obj_id = self.dset.attr2idx[attr], self.dset.obj2idx[obj]\n score = prob_A_rmd[:,attr_id] * prob_O[:, obj_id]\n \n scores[(attr, obj)] = score\n \n return None, scores\n\n def forward(self, x):\n if self.training:\n loss, pred = self.train_forward(x)\n else:\n with torch.no_grad():\n loss, pred = self.val_forward(x)\n return loss, pred"
},
{
"path": "czsl/models/visual_product.py",
"content": "import torch\nimport torch.nn as nn\nimport torch.nn.functional as F\nimport torchvision.models as models\nimport numpy as np\nfrom .common import MLP\n\nclass VisualProductNN(nn.Module):\n def __init__(self, dset, args):\n super(VisualProductNN, self).__init__()\n self.attr_clf = MLP(dset.feat_dim, len(dset.attrs), 2, relu = False)\n self.obj_clf = MLP(dset.feat_dim, len(dset.objs), 2, relu = False)\n self.dset = dset\n\n def train_forward(self, x):\n img, attrs, objs = x[0],x[1], x[2]\n\n attr_pred = self.attr_clf(img)\n obj_pred = self.obj_clf(img)\n\n attr_loss = F.cross_entropy(attr_pred, attrs)\n obj_loss = F.cross_entropy(obj_pred, objs)\n\n loss = attr_loss + obj_loss\n\n return loss, None\n\n def val_forward(self, x):\n img = x[0]\n attr_pred = F.softmax(self.attr_clf(img), dim =1)\n obj_pred = F.softmax(self.obj_clf(img), dim = 1)\n\n scores = {}\n for itr, (attr, obj) in enumerate(self.dset.pairs):\n attr_id, obj_id = self.dset.attr2idx[attr], self.dset.obj2idx[obj]\n score = attr_pred[:,attr_id] * obj_pred[:, obj_id]\n \n scores[(attr, obj)] = score\n return None, scores\n\n def forward(self, x):\n if self.training:\n loss, pred = self.train_forward(x)\n else:\n with torch.no_grad():\n loss, pred = self.val_forward(x)\n\n return loss, pred"
},
{
"path": "czsl/models/word_embedding.py",
"content": "import torch\nimport numpy as np\nfrom flags import DATA_FOLDER\ndef load_word_embeddings(emb_type, vocab):\n if emb_type == 'glove':\n embeds = load_glove_embeddings(vocab)\n elif emb_type == 'fasttext':\n embeds = load_fasttext_embeddings(vocab)\n elif emb_type == 'word2vec':\n embeds = load_word2vec_embeddings(vocab)\n elif emb_type == 'ft+w2v':\n embeds1 = load_fasttext_embeddings(vocab)\n embeds2 = load_word2vec_embeddings(vocab)\n embeds = torch.cat([embeds1, embeds2], dim = 1)\n print('Combined embeddings are ',embeds.shape)\n elif emb_type == 'ft+gl':\n embeds1 = load_fasttext_embeddings(vocab)\n embeds2 = load_glove_embeddings(vocab)\n embeds = torch.cat([embeds1, embeds2], dim = 1)\n print('Combined embeddings are ',embeds.shape)\n elif emb_type == 'ft+ft':\n embeds1 = load_fasttext_embeddings(vocab)\n embeds = torch.cat([embeds1, embeds1], dim = 1)\n print('Combined embeddings are ',embeds.shape)\n elif emb_type == 'gl+w2v':\n embeds1 = load_glove_embeddings(vocab)\n embeds2 = load_word2vec_embeddings(vocab)\n embeds = torch.cat([embeds1, embeds2], dim = 1)\n print('Combined embeddings are ',embeds.shape)\n elif emb_type == 'ft+w2v+gl':\n embeds1 = load_fasttext_embeddings(vocab)\n embeds2 = load_word2vec_embeddings(vocab)\n embeds3 = load_glove_embeddings(vocab)\n embeds = torch.cat([embeds1, embeds2, embeds3], dim = 1)\n print('Combined embeddings are ',embeds.shape)\n else:\n raise ValueError('Invalid embedding')\n return embeds\n\ndef load_fasttext_embeddings(vocab):\n custom_map = {\n 'Faux.Fur': 'fake fur',\n 'Faux.Leather': 'fake leather',\n 'Full.grain.leather': 'thick leather',\n 'Hair.Calf': 'hairy leather',\n 'Patent.Leather': 'shiny leather',\n 'Boots.Ankle': 'ankle boots',\n 'Boots.Knee.High': 'kneehigh boots',\n 'Boots.Mid-Calf': 'midcalf boots',\n 'Shoes.Boat.Shoes': 'boatshoes',\n 'Shoes.Clogs.and.Mules': 'clogs shoes',\n 'Shoes.Flats': 'flats shoes',\n 'Shoes.Heels': 'heels',\n 'Shoes.Loafers': 'loafers',\n 'Shoes.Oxfords': 'oxford shoes',\n 'Shoes.Sneakers.and.Athletic.Shoes': 'sneakers',\n 'traffic_light': 'traficlight',\n 'trash_can': 'trashcan',\n 'dry-erase_board' : 'dry_erase_board',\n 'black_and_white' : 'black_white',\n 'eiffel_tower' : 'tower'\n }\n vocab_lower = [v.lower() for v in vocab]\n vocab = []\n for current in vocab_lower:\n if current in custom_map:\n vocab.append(custom_map[current])\n else:\n vocab.append(current)\n\n import fasttext.util\n ft = fasttext.load_model(DATA_FOLDER+'/fast/cc.en.300.bin')\n embeds = []\n for k in vocab:\n if '_' in k:\n ks = k.split('_')\n emb = np.stack([ft.get_word_vector(it) for it in ks]).mean(axis=0)\n else:\n emb = ft.get_word_vector(k)\n embeds.append(emb)\n\n embeds = torch.Tensor(np.stack(embeds))\n print('Fasttext Embeddings loaded, total embeddings: {}'.format(embeds.size()))\n return embeds\n\ndef load_word2vec_embeddings(vocab):\n # vocab = [v.lower() for v in vocab]\n\n from gensim import models\n model = models.KeyedVectors.load_word2vec_format(\n DATA_FOLDER+'/w2v/GoogleNews-vectors-negative300.bin', binary=True)\n\n custom_map = {\n 'Faux.Fur': 'fake_fur',\n 'Faux.Leather': 'fake_leather',\n 'Full.grain.leather': 'thick_leather',\n 'Hair.Calf': 'hair_leather',\n 'Patent.Leather': 'shiny_leather',\n 'Boots.Ankle': 'ankle_boots',\n 'Boots.Knee.High': 'knee_high_boots',\n 'Boots.Mid-Calf': 'midcalf_boots',\n 'Shoes.Boat.Shoes': 'boat_shoes',\n 'Shoes.Clogs.and.Mules': 'clogs_shoes',\n 'Shoes.Flats': 'flats_shoes',\n 'Shoes.Heels': 'heels',\n 'Shoes.Loafers': 'loafers',\n 'Shoes.Oxfords': 'oxford_shoes',\n 'Shoes.Sneakers.and.Athletic.Shoes': 'sneakers',\n 'traffic_light': 'traffic_light',\n 'trash_can': 'trashcan',\n 'dry-erase_board' : 'dry_erase_board',\n 'black_and_white' : 'black_white',\n 'eiffel_tower' : 'tower'\n }\n\n embeds = []\n for k in vocab:\n if k in custom_map:\n k = custom_map[k]\n if '_' in k and k not in model:\n ks = k.split('_')\n emb = np.stack([model[it] for it in ks]).mean(axis=0)\n else:\n emb = model[k]\n embeds.append(emb)\n embeds = torch.Tensor(np.stack(embeds))\n print('Word2Vec Embeddings loaded, total embeddings: {}'.format(embeds.size()))\n return embeds\n\ndef load_glove_embeddings(vocab):\n '''\n Inputs\n emb_file: Text file with word embedding pairs e.g. Glove, Processed in lower case.\n vocab: List of words\n Returns\n Embedding Matrix\n '''\n vocab = [v.lower() for v in vocab]\n emb_file = DATA_FOLDER+'/glove/glove.6B.300d.txt'\n model = {} # populating a dictionary of word and embeddings\n for line in open(emb_file, 'r'):\n line = line.strip().split(' ') # Word-embedding\n wvec = torch.FloatTensor(list(map(float, line[1:])))\n model[line[0]] = wvec\n\n # Adding some vectors for UT Zappos\n custom_map = {\n 'faux.fur': 'fake_fur',\n 'faux.leather': 'fake_leather',\n 'full.grain.leather': 'thick_leather',\n 'hair.calf': 'hair_leather',\n 'patent.leather': 'shiny_leather',\n 'boots.ankle': 'ankle_boots',\n 'boots.knee.high': 'knee_high_boots',\n 'boots.mid-calf': 'midcalf_boots',\n 'shoes.boat.shoes': 'boat_shoes',\n 'shoes.clogs.and.mules': 'clogs_shoes',\n 'shoes.flats': 'flats_shoes',\n 'shoes.heels': 'heels',\n 'shoes.loafers': 'loafers',\n 'shoes.oxfords': 'oxford_shoes',\n 'shoes.sneakers.and.athletic.shoes': 'sneakers',\n 'traffic_light': 'traffic_light',\n 'trash_can': 'trashcan',\n 'dry-erase_board' : 'dry_erase_board',\n 'black_and_white' : 'black_white',\n 'eiffel_tower' : 'tower',\n 'nubuck' : 'grainy_leather',\n }\n\n embeds = []\n for k in vocab:\n if k in custom_map:\n k = custom_map[k]\n if '_' in k:\n ks = k.split('_')\n emb = torch.stack([model[it] for it in ks]).mean(dim=0)\n else:\n emb = model[k]\n embeds.append(emb)\n embeds = torch.stack(embeds)\n print('Glove Embeddings loaded, total embeddings: {}'.format(embeds.size()))\n return embeds"
},
{
"path": "czsl/notebooks/analysis.ipynb",
"content": "{\n \"cells\": [\n {\n \"cell_type\": \"code\",\n \"execution_count\": null,\n \"metadata\": {},\n \"outputs\": [],\n \"source\": [\n \"import sys\\n\",\n \"sys.path.append('../')\\n\",\n \"# Torch imports\\n\",\n \"import torch\\n\",\n \"import torch.nn as nn\\n\",\n \"import torch.optim as optim\\n\",\n \"import torch.nn.functional as F\\n\",\n \"from torch.utils.tensorboard import SummaryWriter\\n\",\n \"import torch.backends.cudnn as cudnn\\n\",\n \"cudnn.benchmark = True\\n\",\n \"\\n\",\n \"# Python imports\\n\",\n \"import numpy as np\\n\",\n \"import tqdm\\n\",\n \"import torchvision.models as tmodels\\n\",\n \"from tqdm import tqdm\\n\",\n \"import os\\n\",\n \"from os.path import join as ospj\\n\",\n \"import itertools\\n\",\n \"import glob\\n\",\n \"import random\\n\",\n \"\\n\",\n \"#Local imports\\n\",\n \"from data import dataset as dset\\n\",\n \"from models.common import Evaluator\\n\",\n \"from models.image_extractor import get_image_extractor\\n\",\n \"from models.manifold_methods import RedWine, LabelEmbedPlus, AttributeOperator\\n\",\n \"from models.modular_methods import GatedGeneralNN\\n\",\n \"from models.symnet import Symnet\\n\",\n \"from utils.utils import save_args, UnNormalizer, load_args\\n\",\n \"from utils.config_model import configure_model\\n\",\n \"from flags import parser\\n\",\n \"from PIL import Image\\n\",\n \"import matplotlib.pyplot as plt\\n\",\n \"import importlib\\n\",\n \"\\n\",\n \"device = 'cuda' if torch.cuda.is_available() else 'cpu'\\n\",\n \"args, unknown = parser.parse_known_args()\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"### Run one of the cells to load the dataset you want to run test for and move to the next section\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": null,\n \"metadata\": {},\n \"outputs\": [],\n \"source\": [\n \"best_mit = '../logs/graphembed/mitstates/base/mit.yml'\\n\",\n \"load_args(best_mit,args)\\n\",\n \"args.graph_init = '../'+args.graph_init\\n\",\n \"args.load = best_mit[:-7] + 'ckpt_best_auc.t7'\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": null,\n \"metadata\": {},\n \"outputs\": [],\n \"source\": [\n \"best_ut = '../logs/graphembed/utzappos/base/utzappos.yml'\\n\",\n \"load_args(best_ut,args)\\n\",\n \"args.graph_init = '../'+args.graph_init\\n\",\n \"args.load = best_ut[:-12] + 'ckpt_best_auc.t7'\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"### Loading arguments and dataset\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": null,\n \"metadata\": {},\n \"outputs\": [],\n \"source\": [\n \"args.data_dir = '../'+args.data_dir\\n\",\n \"args.test_set = 'test'\\n\",\n \"testset = dset.CompositionDataset(\\n\",\n \" root= args.data_dir,\\n\",\n \" phase=args.test_set,\\n\",\n \" split=args.splitname,\\n\",\n \" model =args.image_extractor,\\n\",\n \" subset=args.subset,\\n\",\n \" return_images = True,\\n\",\n \" update_features = args.update_features,\\n\",\n \" clean_only = args.clean_only\\n\",\n \" )\\n\",\n \"testloader = torch.utils.data.DataLoader(\\n\",\n \" testset,\\n\",\n \" batch_size=args.test_batch_size,\\n\",\n \" shuffle=True,\\n\",\n \" num_workers=args.workers)\\n\",\n \"\\n\",\n \"print('Objs ', len(testset.objs), ' Attrs ', len(testset.attrs))\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": null,\n \"metadata\": {},\n \"outputs\": [],\n \"source\": [\n \"image_extractor, model, optimizer = configure_model(args, testset)\\n\",\n \"evaluator = Evaluator(testset, model)\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": null,\n \"metadata\": {},\n \"outputs\": [],\n \"source\": [\n \"if args.load is not None:\\n\",\n \" checkpoint = torch.load(args.load)\\n\",\n \" if image_extractor:\\n\",\n \" try:\\n\",\n \" image_extractor.load_state_dict(checkpoint['image_extractor'])\\n\",\n \" image_extractor.eval()\\n\",\n \" except:\\n\",\n \" print('No Image extractor in checkpoint')\\n\",\n \" model.load_state_dict(checkpoint['net'])\\n\",\n \" model.eval()\\n\",\n \" print('Loaded model from ', args.load)\\n\",\n \" print('Best AUC: ', checkpoint['AUC'])\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": null,\n \"metadata\": {},\n \"outputs\": [],\n \"source\": [\n \"def print_results(scores, exp):\\n\",\n \" print(exp)\\n\",\n \" result = scores[exp]\\n\",\n \" attr = [evaluator.dset.attrs[result[0][idx,a]] for a in range(topk)]\\n\",\n \" obj = [evaluator.dset.objs[result[1][idx,a]] for a in range(topk)]\\n\",\n \" attr_gt, obj_gt = evaluator.dset.attrs[data[1][idx]], evaluator.dset.objs[data[2][idx]]\\n\",\n \" print(f'Ground truth: {attr_gt} {obj_gt}')\\n\",\n \" prediction = ''\\n\",\n \" for a,o in zip(attr, obj):\\n\",\n \" prediction += a + ' ' + o + '| '\\n\",\n \" print('Predictions: ', prediction)\\n\",\n \" print('__'*50)\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"### An example of predictions\\n\",\n \"closed -> Biased for unseen classes\\n\",\n \"\\n\",\n \"unbiiased -> Biased against unseen classes\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": null,\n \"metadata\": {},\n \"outputs\": [],\n \"source\": [\n \"data = next(iter(testloader))\\n\",\n \"images = data[-1]\\n\",\n \"data = [d.to(device) for d in data[:-1]]\\n\",\n \"if image_extractor:\\n\",\n \" data[0] = image_extractor(data[0])\\n\",\n \"_, predictions = model(data)\\n\",\n \"data = [d.to('cpu') for d in data]\\n\",\n \"topk = 5\\n\",\n \"results = evaluator.score_model(predictions, data[2], bias = 1000, topk=topk)\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": null,\n \"metadata\": {\n \"scrolled\": true\n },\n \"outputs\": [],\n \"source\": [\n \"for idx in range(len(images)):\\n\",\n \" seen = bool(evaluator.seen_mask[data[3][idx]])\\n\",\n \" if seen:\\n\",\n \" continue\\n\",\n \" image = Image.open(ospj( args.data_dir,'images', images[idx]))\\n\",\n \" \\n\",\n \" plt.figure(dpi=300)\\n\",\n \" plt.imshow(image)\\n\",\n \" plt.axis('off')\\n\",\n \" plt.show()\\n\",\n \" \\n\",\n \" print(f'GT pair seen: {seen}')\\n\",\n \" print_results(results, 'closed')\\n\",\n \" print_results(results, 'unbiased_closed')\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"### Run Evaluation\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": null,\n \"metadata\": {},\n \"outputs\": [],\n \"source\": [\n \"model.eval()\\n\",\n \"args.bias = 1e3\\n\",\n \"accuracies, all_attr_gt, all_obj_gt, all_pair_gt, all_pred = [], [], [], [], []\\n\",\n \"\\n\",\n \"for idx, data in tqdm(enumerate(testloader), total=len(testloader), desc = 'Testing'):\\n\",\n \" data.pop()\\n\",\n \" data = [d.to(device) for d in data]\\n\",\n \" if image_extractor:\\n\",\n \" data[0] = image_extractor(data[0])\\n\",\n \"\\n\",\n \" _, predictions = model(data) # todo: Unify outputs across models\\n\",\n \"\\n\",\n \" attr_truth, obj_truth, pair_truth = data[1], data[2], data[3]\\n\",\n \" all_pred.append(predictions)\\n\",\n \" all_attr_gt.append(attr_truth)\\n\",\n \" all_obj_gt.append(obj_truth)\\n\",\n \" all_pair_gt.append(pair_truth)\\n\",\n \"\\n\",\n \"all_attr_gt, all_obj_gt, all_pair_gt = torch.cat(all_attr_gt), torch.cat(all_obj_gt), torch.cat(all_pair_gt)\\n\",\n \"\\n\",\n \"all_pred_dict = {}\\n\",\n \"# Gather values as dict of (attr, obj) as key and list of predictions as values\\n\",\n \"for k in all_pred[0].keys():\\n\",\n \" all_pred_dict[k] = torch.cat(\\n\",\n \" [all_pred[i][k] for i in range(len(all_pred))])\\n\",\n \"\\n\",\n \"# Calculate best unseen accuracy\\n\",\n \"attr_truth, obj_truth = all_attr_gt.to('cpu'), all_obj_gt.to('cpu')\\n\",\n \"pairs = list(\\n\",\n \" zip(list(attr_truth.numpy()), list(obj_truth.numpy())))\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": null,\n \"metadata\": {},\n \"outputs\": [],\n \"source\": [\n \"topk = 1 ### For topk results\\n\",\n \"our_results = evaluator.score_model(all_pred_dict, all_obj_gt, bias = 1e3, topk = topk)\\n\",\n \"stats = evaluator.evaluate_predictions(our_results, all_attr_gt, all_obj_gt, all_pair_gt, all_pred_dict, topk = topk)\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": null,\n \"metadata\": {},\n \"outputs\": [],\n \"source\": [\n \"for k, v in stats.items():\\n\",\n \" print(k, v)\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": null,\n \"metadata\": {},\n \"outputs\": [],\n \"source\": []\n }\n ],\n \"metadata\": {\n \"kernelspec\": {\n \"display_name\": \"Python 3\",\n \"language\": \"python\",\n \"name\": \"python3\"\n },\n \"language_info\": {\n \"codemirror_mode\": {\n \"name\": \"ipython\",\n \"version\": 3\n },\n \"file_extension\": \".py\",\n \"mimetype\": \"text/x-python\",\n \"name\": \"python\",\n \"nbconvert_exporter\": \"python\",\n \"pygments_lexer\": \"ipython3\",\n \"version\": \"3.8.5\"\n }\n },\n \"nbformat\": 4,\n \"nbformat_minor\": 4\n}\n"
},
{
"path": "czsl/test.py",
"content": "# Torch imports\nimport torch\nfrom torch.utils.tensorboard import SummaryWriter\nimport torch.backends.cudnn as cudnn\nimport numpy as np\nfrom flags import DATA_FOLDER\n\ncudnn.benchmark = True\n\n# Python imports\nimport tqdm\nfrom tqdm import tqdm\nimport os\nfrom os.path import join as ospj\n\n# Local imports\nfrom data import dataset as dset\nfrom models.common import Evaluator\nfrom utils.utils import load_args\nfrom utils.config_model import configure_model\nfrom flags import parser\n\n\n\ndevice = 'cuda' if torch.cuda.is_available() else 'cpu'\n\ndef main():\n # Get arguments and start logging\n args = parser.parse_args()\n logpath = args.logpath\n config = [os.path.join(logpath, _) for _ in os.listdir(logpath) if _.endswith('yml')][0]\n load_args(config, args)\n\n # Get dataset\n trainset = dset.CompositionDataset(\n root=os.path.join(DATA_FOLDER,args.data_dir),\n phase='train',\n split=args.splitname,\n model=args.image_extractor,\n update_features=args.update_features,\n train_only=args.train_only,\n subset=args.subset,\n open_world=args.open_world\n )\n\n valset = dset.CompositionDataset(\n root=os.path.join(DATA_FOLDER,args.data_dir),\n phase='val',\n split=args.splitname,\n model=args.image_extractor,\n subset=args.subset,\n update_features=args.update_features,\n open_world=args.open_world\n )\n\n valoader = torch.utils.data.DataLoader(\n valset,\n batch_size=args.test_batch_size,\n shuffle=False,\n num_workers=8)\n\n testset = dset.CompositionDataset(\n root=os.path.join(DATA_FOLDER,args.data_dir),\n phase='test',\n split=args.splitname,\n model =args.image_extractor,\n subset=args.subset,\n update_features = args.update_features,\n open_world=args.open_world\n )\n testloader = torch.utils.data.DataLoader(\n testset,\n batch_size=args.test_batch_size,\n shuffle=False,\n num_workers=args.workers)\n\n\n # Get model and optimizer\n image_extractor, model, optimizer = configure_model(args, trainset)\n args.extractor = image_extractor\n\n if args.add_bt:\n args.load = ospj(logpath,'ckpt_global{}1e-06_best_auc.t7'.format(args.add_bt))\n else:\n args.load = ospj(logpath,'ckpt_best_auc.t7')\n\n print(args.load)\n checkpoint = torch.load(args.load)\n print(checkpoint['epoch'])\n if image_extractor:\n try:\n image_extractor.load_state_dict(checkpoint['image_extractor'])\n image_extractor.eval()\n except:\n print('No Image extractor in checkpoint')\n net_params = {}\n for k in checkpoint['net'].keys():\n nk = k\n # if k.startswith('gcn.'):\n # nk = 'model.' + k\n net_params[nk] = checkpoint['net'][k]\n print(net_params.keys())\n model.load_state_dict(net_params, False)\n model.eval()\n\n threshold = None\n if args.open_world and args.hard_masking:\n assert args.model == 'compcos', args.model + ' does not have hard masking.'\n if args.threshold is not None:\n threshold = args.threshold\n else:\n evaluator_val = Evaluator(valset, model)\n unseen_scores = model.compute_feasibility().to('cpu')\n seen_mask = model.seen_mask.to('cpu')\n min_feasibility = (unseen_scores+seen_mask*10.).min()\n max_feasibility = (unseen_scores-seen_mask*10.).max()\n thresholds = np.linspace(min_feasibility,max_feasibility, num=args.threshold_trials)\n best_auc = 0.\n best_th = -10\n with torch.no_grad():\n for th in thresholds:\n results = test(image_extractor,model,valoader,evaluator_val,args,threshold=th,print_results=False)\n auc = results['AUC']\n if auc > best_auc:\n best_auc = auc\n best_th = th\n print('New best AUC',best_auc)\n print('Threshold',best_th)\n\n threshold = best_th\n\n evaluator = Evaluator(testset, model)\n\n with torch.no_grad():\n test(image_extractor, model, testloader, evaluator, args, threshold)\n\n\ndef test(image_extractor, model, testloader, evaluator, args, threshold=None, print_results=True):\n if image_extractor:\n image_extractor.eval()\n\n model.eval()\n\n accuracies, all_sub_gt, all_attr_gt, all_obj_gt, all_pair_gt, all_pred = [], [], [], [], [], []\n\n for idx, data in tqdm(enumerate(testloader), total=len(testloader), desc='Testing'):\n data = [d.to(device) for d in data]\n\n if image_extractor:\n data[0] = image_extractor(data[0])\n if threshold is None:\n _, predictions = model(data)\n else:\n _, predictions = model.val_forward_with_threshold(data,threshold)\n\n attr_truth, obj_truth, pair_truth = data[1], data[2], data[3]\n\n all_pred.append(predictions)\n all_attr_gt.append(attr_truth)\n all_obj_gt.append(obj_truth)\n all_pair_gt.append(pair_truth)\n\n all_attr_gt, all_obj_gt, all_pair_gt = torch.cat(all_attr_gt), torch.cat(all_obj_gt), torch.cat(all_pair_gt)\n\n all_pred_dict = {}\n # Gather values as dict of (attr, obj) as key and list of predictions as values\n for k in all_pred[0].keys():\n all_pred_dict[k] = torch.cat(\n [all_pred[i][k] for i in range(len(all_pred))])\n\n # Calculate best unseen accuracy\n results = evaluator.score_model(all_pred_dict, all_obj_gt, bias=args.bias, topk=args.topk)\n stats = evaluator.evaluate_predictions(results, all_attr_gt, all_obj_gt, all_pair_gt, all_pred_dict,\n topk=args.topk)\n\n\n result = ''\n for key in stats:\n result = result + key + ' ' + str(round(stats[key], 4)) + '| '\n\n result = result + args.name\n if print_results:\n print(f'Results')\n print(result)\n return results\n\n\nif __name__ == '__main__':\n main()\n"
},
{
"path": "czsl/train.py",
"content": "# Torch imports\nimport torch\nimport torch.nn as nn\nimport torch.nn.functional as F\nfrom torch.utils.tensorboard import SummaryWriter\nimport torch.backends.cudnn as cudnn\ncudnn.benchmark = True\n\n# Python imports\nimport tqdm\nfrom tqdm import tqdm\nimport os\nfrom os.path import join as ospj\nimport csv\n\n#Local imports\nfrom data import dataset as dset\nfrom models.common import Evaluator\nfrom utils.utils import save_args, load_args\nfrom utils.config_model import configure_model\nfrom flags import parser, DATA_FOLDER\n\nbest_auc = 0\nbest_hm = 0\ncompose_switch = True\ndevice = 'cuda' if torch.cuda.is_available() else 'cpu'\n\ndef main():\n # Get arguments and start logging\n args = parser.parse_args()\n load_args(args.config, args)\n logpath = os.path.join(args.cv_dir, args.name)\n os.makedirs(logpath, exist_ok=True)\n save_args(args, logpath, args.config)\n writer = SummaryWriter(log_dir = logpath, flush_secs = 30)\n\n # Get dataset\n trainset = dset.CompositionDataset(\n root=os.path.join(DATA_FOLDER,args.data_dir),\n phase='train',\n split=args.splitname,\n model =args.image_extractor,\n num_negs=args.num_negs,\n pair_dropout=args.pair_dropout,\n update_features = args.update_features,\n train_only= args.train_only,\n open_world=args.open_world\n )\n trainloader = torch.utils.data.DataLoader(\n trainset,\n batch_size=args.batch_size,\n shuffle=True,\n num_workers=args.workers)\n testset = dset.CompositionDataset(\n root=os.path.join(DATA_FOLDER,args.data_dir),\n phase=args.test_set,\n split=args.splitname,\n model =args.image_extractor,\n subset=args.subset,\n update_features = args.update_features,\n open_world=args.open_world\n )\n testloader = torch.utils.data.DataLoader(\n testset,\n batch_size=args.test_batch_size,\n shuffle=False,\n num_workers=args.workers)\n\n\n # Get model and optimizer\n image_extractor, model, optimizer = configure_model(args, trainset)\n args.extractor = image_extractor\n\n train = train_normal\n\n evaluator_val = Evaluator(testset, model)\n\n print(model)\n\n start_epoch = 0\n # Load checkpoint\n if args.load is not None:\n checkpoint = torch.load(args.load)\n if image_extractor:\n try:\n image_extractor.load_state_dict(checkpoint['image_extractor'])\n if args.freeze_features:\n print('Freezing image extractor')\n image_extractor.eval()\n for param in image_extractor.parameters():\n param.requires_grad = False\n except:\n print('No Image extractor in checkpoint')\n model.load_state_dict(checkpoint['net'])\n start_epoch = checkpoint['epoch']\n print('Loaded model from ', args.load)\n \n for epoch in tqdm(range(start_epoch, args.max_epochs + 1), desc = 'Current epoch'):\n train(epoch, image_extractor, model, trainloader, optimizer, writer)\n if model.is_open and args.model=='compcos' and ((epoch+1)%args.update_feasibility_every)==0 :\n print('Updating feasibility scores')\n model.update_feasibility(epoch+1.)\n\n if epoch % args.eval_val_every == 0:\n with torch.no_grad(): # todo: might not be needed\n test(epoch, image_extractor, model, testloader, evaluator_val, writer, args, logpath)\n print('Best AUC achieved is ', best_auc)\n print('Best HM achieved is ', best_hm)\n\n\ndef train_normal(epoch, image_extractor, model, trainloader, optimizer, writer):\n '''\n Runs training for an epoch\n '''\n\n if image_extractor:\n image_extractor.train()\n model.train() # Let's switch to training\n\n train_loss = 0.0 \n for idx, data in tqdm(enumerate(trainloader), total=len(trainloader), desc = 'Training'):\n data = [d.to(device) for d in data]\n\n if image_extractor:\n data[0] = image_extractor(data[0])\n \n loss, _ = model(data)\n\n optimizer.zero_grad()\n loss.backward()\n optimizer.step()\n \n train_loss += loss.item()\n\n train_loss = train_loss/len(trainloader)\n writer.add_scalar('Loss/train_total', train_loss, epoch)\n print('Epoch: {}| Loss: {}'.format(epoch, round(train_loss, 2)))\n\n\ndef test(epoch, image_extractor, model, testloader, evaluator, writer, args, logpath):\n '''\n Runs testing for an epoch\n '''\n global best_auc, best_hm\n\n def save_checkpoint(filename):\n state = {\n 'net': model.state_dict(),\n 'epoch': epoch,\n 'AUC': stats['AUC']\n }\n if image_extractor:\n state['image_extractor'] = image_extractor.state_dict()\n filename = 'global{}_'.format(str(args.add_bt)+str(args.lrb)) + filename\n\n torch.save(state, os.path.join(logpath, 'ckpt_{}.t7'.format(filename)))\n\n if image_extractor:\n image_extractor.eval()\n\n model.eval()\n\n accuracies, all_sub_gt, all_attr_gt, all_obj_gt, all_pair_gt, all_pred = [], [], [], [], [], []\n\n for idx, data in tqdm(enumerate(testloader), total=len(testloader), desc='Testing'):\n data = [d.to(device) for d in data]\n\n if image_extractor:\n data[0] = image_extractor(data[0])\n\n _, predictions = model(data)\n\n attr_truth, obj_truth, pair_truth = data[1], data[2], data[3]\n\n all_pred.append({k: v.to('cpu') for k, v in predictions.items()})\n all_attr_gt.append(attr_truth)\n all_obj_gt.append(obj_truth)\n all_pair_gt.append(pair_truth)\n\n all_attr_gt, all_obj_gt, all_pair_gt = torch.cat(all_attr_gt), torch.cat(all_obj_gt), torch.cat(all_pair_gt)\n\n all_pred_dict = {}\n # Gather values as dict of (attr, obj) as key and list of predictions as values\n for k in all_pred[0].keys():\n all_pred_dict[k] = torch.cat(\n [all_pred[i][k] for i in range(len(all_pred))])\n\n # Calculate best unseen accuracy\n results = evaluator.score_model(all_pred_dict, all_obj_gt, bias=args.bias, topk=args.topk)\n stats = evaluator.evaluate_predictions(results, all_attr_gt, all_obj_gt, all_pair_gt, all_pred_dict, topk=args.topk)\n\n stats['a_epoch'] = epoch\n\n result = ''\n # write to Tensorboard\n for key in stats:\n writer.add_scalar(key, stats[key], epoch)\n result = result + key + ' ' + str(round(stats[key], 4)) + '| '\n\n result = result + args.name\n print(f'Test Epoch: {epoch}')\n print(result)\n if epoch > 0 and epoch % args.save_every == 0:\n save_checkpoint(epoch)\n if stats['AUC'] > best_auc:\n best_auc = stats['AUC']\n print('New best AUC ', best_auc)\n save_checkpoint('best_auc')\n\n if stats['best_hm'] > best_hm:\n best_hm = stats['best_hm']\n print('New best HM ', best_hm)\n save_checkpoint('best_hm')\n\n # Logs\n with open(ospj(logpath, 'logs.csv'), 'a') as f:\n w = csv.DictWriter(f, stats.keys())\n if epoch == 0:\n w.writeheader()\n w.writerow(stats)\n\n\nif __name__ == '__main__':\n try:\n main()\n except KeyboardInterrupt:\n print('Best AUC achieved is ', best_auc)\n print('Best HM achieved is ', best_hm)"
},
{
"path": "czsl/utils/__init__.py",
"content": ""
},
{
"path": "czsl/utils/config_model.py",
"content": "import torch\nimport torch.optim as optim\n\nfrom models.image_extractor import get_image_extractor\nfrom models.visual_product import VisualProductNN\nfrom models.manifold_methods import RedWine, LabelEmbedPlus, AttributeOperator\nfrom models.modular_methods import GatedGeneralNN\nfrom models.graph_method import GraphFull\nfrom models.symnet import Symnet\nfrom models.compcos import CompCos\n\ndevice = 'cuda' if torch.cuda.is_available() else 'cpu'\n\nclass TransformerDecorator(torch.nn.Module):\n def __init__(self, model, model_name, add_bt=0, eval_bt=0):\n super(TransformerDecorator, self).__init__()\n self.model = model\n self.model_name = model_name\n self.add_bt = add_bt\n self.eval_bt = eval_bt\n\n if self.add_bt:\n self.encoder_layers = torch.nn.TransformerEncoderLayer(512, 4, 512, 0.5)\n\n @property\n def is_open(self):\n return self.model.is_open\n\n @is_open.setter\n def is_open(self, value):\n self.model.is_open = value\n\n def update_feasibility(self, value):\n self.model.update_feasibility(value)\n\n def forward_feats(self, feature):\n feature[0] = feature[0].view(feature[0].size(0), -1)\n\n old_feat = feature[0]\n if self.model.training and self.add_bt or self.eval_bt:\n feature[0] = feature[0].unsqueeze(1)\n feature[0] = self.encoder_layers(feature[0])\n feature[0] = feature[0].squeeze(1)\n if self.add_bt > 1:\n feature[0] = torch.cat([old_feat, feature[0]], dim=0)\n return feature\n\n def forward(self, feature):\n feature = self.forward_feats(feature)\n feature[0] = self.model(feature)\n\n return feature[0]\n\n\ndef configure_model(args, dataset):\n image_extractor = None\n is_open = False\n\n if args.model == 'visprodNN':\n model = VisualProductNN(dataset, args)\n elif args.model == 'redwine':\n model = RedWine(dataset, args)\n elif args.model == 'labelembed+':\n model = LabelEmbedPlus(dataset, args)\n elif args.model == 'attributeop':\n model = AttributeOperator(dataset, args)\n elif args.model == 'tmn':\n model = GatedGeneralNN(dataset, args, num_layers=args.nlayers, num_modules_per_layer=args.nmods)\n elif args.model == 'symnet':\n model = Symnet(dataset, args)\n elif args.model == 'graphfull':\n model = GraphFull(dataset, args)\n elif args.model == 'compcos':\n model = CompCos(dataset, args)\n if dataset.open_world and not args.train_only:\n is_open = True\n else:\n raise NotImplementedError\n\n # add Transformer\n model = TransformerDecorator(model, args.model, args.add_bt)\n model = model.to(device)\n\n if args.update_features:\n print('Learnable image_embeddings')\n image_extractor = get_image_extractor(arch = args.image_extractor, pretrained = True)\n image_extractor = image_extractor.to(device)\n\n # configuring optimizer\n if args.model=='redwine':\n optim_params = filter(lambda p: p.requires_grad, model.parameters())\n elif args.model=='attributeop':\n attr_params = [param for name, param in model.named_parameters() if 'attr_op' in name and param.requires_grad]\n other_params = [param for name, param in model.named_parameters() if 'attr_op' not in name and param.requires_grad]\n optim_params = [{'params':attr_params, 'lr':0.1*args.lr}, {'params':other_params}]\n elif args.model=='tmn':\n gating_params = [\n param for name, param in model.named_parameters()\n if 'gating_network' in name and param.requires_grad\n ]\n network_params = [\n param for name, param in model.named_parameters()\n if 'gating_network' not in name and param.requires_grad\n ]\n optim_params = [\n {\n 'params': network_params,\n },\n {\n 'params': gating_params,\n 'lr': args.lrg\n },\n ]\n else:\n model_params = [param for name, param in model.named_parameters() if param.requires_grad and not name.__contains__('encoder_layers')]\n optim_params = [{'params':model_params}]\n\n if args.add_bt:\n model_params = [param for name, param in model.named_parameters() if param.requires_grad and name.__contains__('encoder_layers')]\n optim_params.append({'params':model_params, 'lr': args.lrb})\n\n if args.update_features:\n ie_parameters = [param for name, param in image_extractor.named_parameters()]\n optim_params.append({'params': ie_parameters,\n 'lr': args.lrg})\n optimizer = optim.Adam(optim_params, lr=args.lr, weight_decay=args.wd)\n\n model.is_open = is_open\n\n return image_extractor, model, optimizer"
},
{
"path": "czsl/utils/download_data.sh",
"content": "# Copyright (c) Facebook, Inc. and its affiliates.\n# All rights reserved.\n#\n# This source code is licensed under the license found in the\n# LICENSE file in the root directory of this source tree.\n#\n\nCURRENT_DIR=$(pwd)\n\nFOLDER=$1\n\nmkdir $FOLDER\nmkdir $FOLDER/fast\nmkdir $FOLDER/glove\nmkdir $FOLDER/w2v\n\nwget --show-progress -O utils/cgqa-graph.t7 https://raw.githubusercontent.com/ExplainableML/czsl/main/utils/cgqa-graph.t7\nwget --show-progress -O utils/mitstates-graph.t7 https://raw.githubusercontent.com/ExplainableML/czsl/main/utils/mitstates-graph.t7\nwget --show-progress -O utils/utzappos-graph.t7 https://raw.githubusercontent.com/ExplainableML/czsl/main/utils/utzappos-graph.t7\n\ncp utils/download_embeddings.py $FOLDER/fast\n\n\n# Download everything\nwget --show-progress -O $FOLDER/attr-ops-data.tar.gz https://www.cs.utexas.edu/~tushar/attribute-ops/attr-ops-data.tar.gz\nwget --show-progress -O $FOLDER/mitstates.zip http://wednesday.csail.mit.edu/joseph_result/state_and_transformation/release_dataset.zip\nwget --show-progress -O $FOLDER/utzap.zip http://vision.cs.utexas.edu/projects/finegrained/utzap50k/ut-zap50k-images.zip\nwget --show-progress -O $FOLDER/splits.tar.gz https://www.senthilpurushwalkam.com/publication/compositional/compositional_split_natural.tar.gz\nwget --show-progress -O $FOLDER/cgqa.zip https://s3.mlcloud.uni-tuebingen.de/czsl/cgqa.zip\n\necho \"Data downloaded. Extracting files...\"\nsed -i \"s|ROOT_FOLDER|$FOLDER|g\" utils/reorganize_utzap.py\nsed -i \"s|ROOT_FOLDER|$FOLDER|g\" flags.py\n\n# Dataset metadata, pretrained SVMs and features, tensor completion data\n\ncd $FOLDER\n\ntar -zxvf attr-ops-data.tar.gz --strip 1\n\n# MIT-States\nunzip mitstates.zip 'release_dataset/images/*' -d mit-states/\nmv mit-states/release_dataset/images mit-states/images/\nrm -r mit-states/release_dataset\nrename \"s/ /_/g\" mit-states/images/*\n\n# UT-Zappos50k\nunzip utzap.zip -d ut-zap50k/\nmv ut-zap50k/ut-zap50k-images ut-zap50k/_images/\n\n# C-GQA\nunzip cgqa.zip -d cgqa/\n\n# Download new splits for Purushwalkam et. al\ntar -zxvf splits.tar.gz\n\n# remove all zip files and temporary files\nrm -r attr-ops-data.tar.gz mitstates.zip utzap.zip splits.tar.gz cgqa.zip\n\n# Download embeddings\n\n# Glove (from attribute as operators)\nmv data/glove/* glove/\n\n# FastText\ncd fast\npython download_embeddings.py\nrm cc.en.300.bin.gz\n\n# Word2Vec\ncd ../w2v\nwget -c \"https://s3.amazonaws.com/dl4j-distribution/GoogleNews-vectors-negative300.bin.gz\"\ngzip -d GoogleNews-vectors-negative300.bin.gz\nrm GoogleNews-vectors-negative300.bin.gz\n\ncd ..\nrm -r data\n\ncd $CURRENT_DIR\npython utils/reorganize_utzap.py\n"
},
{
"path": "czsl/utils/download_embeddings.py",
"content": "import fasttext.util\nfasttext.util.download_model('en', if_exists='ignore') # English\n\n"
},
{
"path": "czsl/utils/reorganize_utzap.py",
"content": "# Copyright (c) Facebook, Inc. and its affiliates.\n# All rights reserved.\n#\n# This source code is licensed under the license found in the\n# LICENSE file in the root directory of this source tree.\n#\n\"\"\"\nReorganize the UT-Zappos dataset to resemble the MIT-States dataset\nroot/attr_obj/img1.jpg\nroot/attr_obj/img2.jpg\nroot/attr_obj/img3.jpg\n...\n\"\"\"\n\nimport os\nimport torch\nimport shutil\nimport tqdm\n\nDATA_FOLDER= \"/scratch/ZHIHOU/CZSL\"\n\nroot = DATA_FOLDER+'/ut-zap50k/'\nos.makedirs(root+'/images',exist_ok=True)\n\ndata = torch.load(root+'/metadata_compositional-split-natural.t7')\nfor instance in tqdm.tqdm(data):\n\timage, attr, obj = instance['_image'], instance['attr'], instance['obj']\n\told_file = '%s/_images/%s'%(root, image)\n\tnew_dir = '%s/images/%s_%s/'%(root, attr, obj)\n\tos.makedirs(new_dir, exist_ok=True)\n\tshutil.copy(old_file, new_dir)\n"
},
{
"path": "czsl/utils/utils.py",
"content": "import os\nfrom os.path import join as ospj\nimport torch\nimport random\nimport copy\nimport shutil\nimport sys\nimport yaml\n\n\ndef chunks(l, n):\n \"\"\"Yield successive n-sized chunks from l.\"\"\"\n for i in range(0, len(l), n):\n yield l[i:i + n]\n\ndef get_norm_values(norm_family = 'imagenet'):\n '''\n Inputs\n norm_family: String of norm_family\n Returns\n mean, std : tuple of 3 channel values\n '''\n if norm_family == 'imagenet':\n mean, std = [0.485, 0.456, 0.406], [0.229, 0.224, 0.225]\n else:\n raise ValueError('Incorrect normalization family')\n return mean, std\n\ndef save_args(args, log_path, argfile):\n shutil.copy('train.py', log_path)\n modelfiles = ospj(log_path, 'models')\n try:\n shutil.copy(argfile, log_path)\n except:\n print('Config exists')\n try:\n shutil.copytree('models/', modelfiles)\n except:\n print('Already exists')\n with open(ospj(log_path,'args_all.yaml'),'w') as f:\n yaml.dump(args, f, default_flow_style=False, allow_unicode=True)\n with open(ospj(log_path, 'args.txt'), 'w') as f:\n f.write('\\n'.join(sys.argv[1:]))\n\nclass UnNormalizer:\n '''\n Unnormalize a given tensor using mean and std of a dataset family\n Inputs\n norm_family: String, dataset\n tensor: Torch tensor\n Outputs\n tensor: Unnormalized tensor\n '''\n def __init__(self, norm_family = 'imagenet'):\n self.mean, self.std = get_norm_values(norm_family=norm_family)\n self.mean, self.std = torch.Tensor(self.mean).view(1, 3, 1, 1), torch.Tensor(self.std).view(1, 3, 1, 1)\n\n def __call__(self, tensor):\n return (tensor * self.std) + self.mean\n\ndef load_args(filename, args):\n with open(filename, 'r') as stream:\n data_loaded = yaml.safe_load(stream)\n for key, group in data_loaded.items():\n for key, val in group.items():\n setattr(args, key, val)"
},
{
"path": "domain_generalization/README.md",
"content": "# BatchFormer for Domain Generalization\n\n## Installation\nCode is based on Transfer-Learning-Library (https://github.com/thuml/Transfer-Learning-Library.git). One shoud first install this library, or copy the files the files in this directory into Transfer-Learning-Library/examples/domain_generalization/image_classification/\n\n\n\nThis repository has changed a few lines of code based on Transfer-Learning-Library. Check the modifications by:\n```\ndiff baseline.py <(curl https://raw.githubusercontent.com/thuml/Transfer-Learning-Library/master/examples/domain_generalization/image_classification/baseline.py)\n```\n\nI have also included Transfer-Learning-Library in this directory. You can find the examples in Transfer-Learning-Library/master/examples/domain_generalization/image_classification/ (e.g. baseline.py, irm.py)\n\nIn Transfer-Learning-Library/master/examples/domain_adaption/image_classification/, you can find corresponding code for Domain Adaption.\n\n\nBatchFormer based on SWAD is provided in swad.\n\n\n## Experiment and Results\n\nFor example, if you want to reproduce IRM on Office-Home, use the following script\n\n```shell script\nCUDA_VISIBLE_DEVICES=0 python irm.py data/office-home -d OfficeHome -s Ar Cl Rw -t Pr -a resnet50 --seed 0 --log logs/irm/OfficeHome_Pr --add_bt 1\n```\n\nYou can find the corresponding *.pbs files in Transfer-Learning-Library/master/examples/domain_generalization/image_classification/ to get the running scripts for other methods. Please refer to Transfer-Learning-Library for More suggestions.\n\nif you find this repository helpful, please consider cite:\n\n @inproceedings{hou2022batch,\n title={BatchFormer: Learning to Explore Sample Relationships for Robust Representation Learning},\n author={Hou, Zhi and Yu, Baosheng and Tao, Dacheng},\n booktitle={CVPR},\n year={2022}\n }\n\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/.github/ISSUE_TEMPLATE/bug_report.md",
"content": "---\nname: Bug report\nabout: Create a report to help us improve\ntitle: ''\nlabels: ''\nassignees: ''\n\n---\n\n**Describe the bug**\nA clear and concise description of what the bug is.\n\n**To Reproduce**\nSteps to reproduce the behavior:\n1. Go to '...'\n2. Click on '....'\n3. Scroll down to '....'\n4. See error\n\n**Expected behavior**\nA clear and concise description of what you expected to happen.\n\n**Screenshots**\nIf applicable, add screenshots to help explain your problem.\n\n**Desktop (please complete the following information):**\n - OS: [e.g. iOS]\n - Browser [e.g. chrome, safari]\n - Version [e.g. 22]\n\n**Smartphone (please complete the following information):**\n - Device: [e.g. iPhone6]\n - OS: [e.g. iOS8.1]\n - Browser [e.g. stock browser, safari]\n - Version [e.g. 22]\n\n**Additional context**\nAdd any other context about the problem here.\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/.github/ISSUE_TEMPLATE/custom.md",
"content": "---\nname: Custom issue template\nabout: Describe this issue template's purpose here.\ntitle: ''\nlabels: ''\nassignees: ''\n\n---\n\n\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/.github/ISSUE_TEMPLATE/feature_request.md",
"content": "---\nname: Feature request\nabout: Suggest an idea for this project\ntitle: ''\nlabels: ''\nassignees: ''\n\n---\n\n**Is your feature request related to a problem? Please describe.**\nA clear and concise description of what the problem is. Ex. I'm always frustrated when [...]\n\n**Describe the solution you'd like**\nA clear and concise description of what you want to happen.\n\n**Describe alternatives you've considered**\nA clear and concise description of any alternative solutions or features you've considered.\n\n**Additional context**\nAdd any other context or screenshots about the feature request here.\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/.gitignore",
"content": "# Byte-compiled / optimized / DLL files\n__pycache__/\n*.py[cod]\n*$py.class\n\n# C extensions\n*.so\n\n# Distribution / packaging\n.Python\nbuild/\ndevelop-eggs/\ndist/\ndownloads/\neggs/\n.eggs/\nlib/\nlib64/\nparts/\nsdist/\nvar/\nwheels/\npip-wheel-metadata/\nshare/python-wheels/\n*.egg-info/\n.installed.cfg\n*.egg\nMANIFEST\n\n# PyInstaller\n# Usually these files are written by a python script from a template\n# before PyInstaller builds the exe, so as to inject date/other infos into it.\n*.manifest\n*.spec\n\n# Installer logs\npip-log.txt\npip-delete-this-directory.txt\n\n# Unit test / coverage reports\nhtmlcov/\n.tox/\n.nox/\n.coverage\n.coverage.*\n.cache\nnosetests.xml\ncoverage.xml\n*.cover\n.hypothesis/\n.pytest_cache/\n\n# Translations\n*.mo\n*.pot\n\n# Django stuff:\n*.log\nlocal_settings.py\ndb.sqlite3\ndb.sqlite3-journal\n\n# Flask stuff:\ninstance/\n.webassets-cache\n\n# Scrapy stuff:\n.scrapy\n\n# Sphinx documentation\ndocs/_build/\ndocs/build/*\ndocs/pytorch_sphinx_theme/*\n\n# PyBuilder\ntarget/\n\n# Jupyter Notebook\n.ipynb_checkpoints\n\n# IPython\nprofile_default/\nipython_config.py\n\n# pyenv\n.python-version\n\n# pipenv\n# According to pypa/pipenv#598, it is recommended to include Pipfile.lock in version control.\n# However, in case of collaboration, if having platform-specific dependencies or dependencies\n# having no cross-platform support, pipenv may install dependencies that don't work, or not\n# install all needed dependencies.\n#Pipfile.lock\n\n# celery beat schedule file\ncelerybeat-schedule\n\n# SageMath parsed files\n*.sage.py\n\n# Environments\n.env\n.venv\nenv/\nvenv/\nENV/\nenv.bak/\nvenv.bak/\n\n# Spyder project settings\n.spyderproject\n.spyproject\n\n# Rope project settings\n.ropeproject\n\n# mkdocs documentation\n/site\n\n# mypy\n.mypy_cache/\n.dmypy.json\ndmypy.json\n\n# Pyre type checker\n.pyre/\n\n.idea/\n\nexp/*\ntrash/*\nexamples/domain_adaptation/digits/logs/*\nexamples/domain_adaptation/digits/data/*\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/CONTRIBUTING.md",
"content": "## Contributing to Transfer-Learning-Library\n\nAll kinds of contributions are welcome, including but not limited to the following.\n\n- Fix typo or bugs\n- Add documentation\n- Add new features and components\n\n### Workflow\n\n1. fork and pull the latest Transfer-Learning-Library repository\n2. checkout a new branch (do not use master branch for PRs)\n3. commit your changes\n4. create a PR\n\n```{note}\nIf you plan to add some new features that involve large changes, it is encouraged to open an issue for discussion first.\n```\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/LICENSE",
"content": "Copyright (c) 2018 The Python Packaging Authority\n\nPermission is hereby granted, free of charge, to any person obtaining a copy\nof this software and associated documentation files (the \"Software\"), to deal\nin the Software without restriction, including without limitation the rights\nto use, copy, modify, merge, publish, distribute, sublicense, and/or sell\ncopies of the Software, and to permit persons to whom the Software is\nfurnished to do so, subject to the following conditions:\n\nThe above copyright notice and this permission notice shall be included in all\ncopies or substantial portions of the Software.\n\nTHE SOFTWARE IS PROVIDED \"AS IS\", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR\nIMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,\nFITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE\nAUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER\nLIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,\nOUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE\nSOFTWARE."
},
{
"path": "domain_generalization/Transfer-Learning-Library/README.md",
"content": "\n\n## Introduction\n*Transfer-Learn* is an open-source and well-documented library for Transfer Learning. It is based on pure PyTorch with high performance and friendly API. Our code is pythonic, and the design is consistent with torchvision. You can easily develop new algorithms, or readily apply existing algorithms.\n\nThe currently supported algorithms include:\n\n##### [Domain Adaptation for Classification](/examples/domain_adaptation/image_classification)\n- [Domain Adversarial Neural Network (DANN)](https://arxiv.org/abs/1505.07818)\n- [Deep Adaptation Network (DAN)](https://arxiv.org/pdf/1502.02791)\n- [Joint Adaptation Network (JAN)](https://arxiv.org/abs/1605.06636)\n- [Adversarial Discriminative Domain Adaptation (ADDA)](https://arxiv.org/pdf/1702.05464.pdf)\n- [Conditional Domain Adversarial Network (CDAN)](https://arxiv.org/abs/1705.10667)\n- [Maximum Classifier Discrepancy (MCD)](https://arxiv.org/abs/1712.02560)\n- [Adaptive Feature Norm (AFN)](https://arxiv.org/pdf/1811.07456v2.pdf)\n- [Margin Disparity Discrepancy (MDD)](https://arxiv.org/abs/1904.05801)\n- [Batch Spectral Penalization (BSP)](http://ise.thss.tsinghua.edu.cn/~mlong/doc/batch-spectral-penalization-icml19.pdf)\n- [Minimum Class Confusion (MCC)](https://arxiv.org/abs/1912.03699)\n\n##### [Partial Domain Adaptation](/examples/domain_adaptation/partial_domain_adaptation/)\n- [Domain Adversarial Neural Network (DANN)](https://arxiv.org/abs/1505.07818)\n- [Partial Adversarial Domain Adaptation (PADA)](https://arxiv.org/abs/1808.04205)\n- [Importance Weighted Adversarial Nets (IWAN)](https://arxiv.org/abs/1803.09210)\n- [Adaptive Feature Norm (AFN)](https://arxiv.org/pdf/1811.07456v2.pdf)\n\n##### [Open-set Domain Adaptation](/examples/domain_adaptation/openset_domain_adaptation/)\n- [Open Set Domain Adaptation (OSBP)](https://arxiv.org/abs/1804.10427)\n\n##### [Domain Adaptation for Semantic Segmentation](/examples/domain_adaptation/semantic_segmentation/)\n- [Cycle-Consistent Adversarial Networks (CycleGAN)](https://arxiv.org/pdf/1703.10593.pdf)\n- [CyCADA: Cycle-Consistent Adversarial Domain Adaptation](https://arxiv.org/abs/1711.03213)\n- [Adversarial Entropy Minimization (ADVENT)](https://arxiv.org/abs/1811.12833)\n- [Fourier Domain Adaptation (FDA)](https://arxiv.org/abs/2004.05498)\n\n##### [Domain Adaptation for Keypoint Detection](/examples/domain_adaptation/keypoint_detection)\n- [Regressive Domain Adaptation for Unsupervised Keypoint Detection (RegDA, CVPR 2021)](http://ise.thss.tsinghua.edu.cn/~mlong/doc/regressive-domain-adaptation-cvpr21.pdf)\n\n##### [Domain Adaptation for Person Re-identification](/examples/domain_adaptation/re_identification/)\n- [Two at Once: Enhancing Learning and Generalization Capacities via IBN-Net (IBN-Net, ECCV 2018)](https://openaccess.thecvf.com/content_ECCV_2018/papers/Xingang_Pan_Two_at_Once_ECCV_2018_paper.pdf)\n- [Mutual Mean-Teaching: Pseudo Label Refinery for Unsupervised Domain Adaptation on Person Re-identification (MMT, ICLR 2020)](https://arxiv.org/abs/2001.01526)\n- [Similarity Preserving Generative Adversarial Network (SPGAN, CVPR 2018)](https://arxiv.org/pdf/1811.10551.pdf)\n\n##### [Task Adaptation for Image Classification](/examples/task_adaptation/image_classification/)\n- [Explicit inductive bias for transfer learning with convolutional networks\n (L2-SP, ICML 2018)](https://arxiv.org/abs/1802.01483)\n- [Catastrophic Forgetting Meets Negative Transfer: Batch Spectral Shrinkage for Safe Transfer Learning (BSS, NIPS 2019)](https://proceedings.neurips.cc/paper/2019/file/c6bff625bdb0393992c9d4db0c6bbe45-Paper.pdf)\n- [DEep Learning Transfer using Fea- ture Map with Attention for convolutional networks (DELTA, ICLR 2019)](https://openreview.net/pdf?id=rkgbwsAcYm)\n- [Co-Tuning for Transfer Learning (Co-Tuning, NIPS 2020)](http://ise.thss.tsinghua.edu.cn/~mlong/doc/co-tuning-for-transfer-learning-nips20.pdf)\n- [Stochastic Normalization (StochNorm, NIPS 2020)](https://papers.nips.cc/paper/2020/file/bc573864331a9e42e4511de6f678aa83-Paper.pdf)\n- [Learning Without Forgetting (LWF, ECCV 2016)](https://arxiv.org/abs/1606.09282)\n- [Bi-tuning of Pre-trained Representations (Bi-Tuning)](https://arxiv.org/abs/2011.06182?utm_source=feedburner&utm_medium=feed&utm_campaign=Feed%3A+arxiv%2FQSXk+%28ExcitingAds%21+cs+updates+on+arXiv.org%29)\n\n##### [Domain Generalization for Classification](/examples/domain_generalization/image_classification/)\n- [Two at Once: Enhancing Learning and Generalization Capacities via IBN-Net (IBN-Net, ECCV 2018)](https://openaccess.thecvf.com/content_ECCV_2018/papers/Xingang_Pan_Two_at_Once_ECCV_2018_paper.pdf)\n- [Domain Generalization with MixStyle (MixStyle, ICLR 2021)](https://arxiv.org/abs/2104.02008)\n- [Learning to Generalize: Meta-Learning for Domain Generalization (MLDG, AAAI 2018)](https://arxiv.org/pdf/1710.03463.pdf)\n- [Invariant Risk Minimization (IRM)](https://arxiv.org/abs/1907.02893)\n- [Out-of-Distribution Generalization via Risk Extrapolation (VREx, ICML 2021)](https://arxiv.org/abs/2003.00688)\n- [Distributionally Robust Neural Networks for Group Shifts: On the Importance of Regularization for Worst-Case Generalization(GroupDRO)](https://arxiv.org/abs/1911.08731)\n- [Deep CORAL: Correlation Alignment for Deep Domain Adaptation (Deep Coral, ECCV 2016)](https://arxiv.org/abs/1607.01719)\n\n##### [Domain Generalization for Person Re-identification](/examples/domain_generalization/re_identification/)\n- [Two at Once: Enhancing Learning and Generalization Capacities via IBN-Net (IBN-Net, ECCV 2018)](https://openaccess.thecvf.com/content_ECCV_2018/papers/Xingang_Pan_Two_at_Once_ECCV_2018_paper.pdf)\n- [Domain Generalization with MixStyle (MixStyle, ICLR 2021)](https://arxiv.org/abs/2104.02008)\n\nWe are planning to add\n- DA for Object Detection\n- TA for text classification\n\n## Installation\n\nTo use ``dalib``, ``talib``, ``dglib``, and ``common`` in other places, you need to install Transfer-Learn,\n```shell\npython setup.py install\n```\nNote that we do not support *pip install* currently.\n\nFor flexible use and modification of Transfer-Learn, please git clone the library.\n\n\n## Documentation\nYou can find the tutorial and API documentation on the website: [Documentation (please open in Firefox or Safari)](http://170.106.108.162/index.html). Note that this link is only for temporary use. You can also build the doc by yourself following the instructions in http://170.106.108.162/get_started/faq.html.\n\nAlso, we have examples in the directory `examples`. A typical usage is \n```shell script\n# Train a DANN on Office-31 Amazon -> Webcam task using ResNet 50.\n# Assume you have put the datasets under the path `data/office-31`, \n# or you are glad to download the datasets automatically from the Internet to this path\npython dann.py data/office31 -d Office31 -s A -t W -a resnet50 --epochs 20\n```\n\nIn the directory `examples`, you can find all the necessary running scripts to reproduce the benchmarks with specified hyper-parameters.\n\n## Contributing\nWe appreciate all contributions. If you are planning to contribute back bug-fixes, please do so without any further discussion. If you plan to contribute new features, utility functions or extensions, please first open an issue and discuss the feature with us. \n\n## Disclaimer on Datasets\n\nThis is a utility library that downloads and prepares public datasets. We do not host or distribute these datasets, vouch for their quality or fairness, or claim that you have licenses to use the dataset. It is your responsibility to determine whether you have permission to use the dataset under the dataset's license.\n\nIf you're a dataset owner and wish to update any part of it (description, citation, etc.), or do not want your dataset to be included in this library, please get in touch through a GitHub issue. Thanks for your contribution to the ML community!\n\n\n## Contact\nIf you have any problem with our code or have some suggestions, including the future feature, feel free to contact \n- Junguang Jiang (JiangJunguang1123@outlook.com)\n- Baixu Chen (cbx_99_hasta@outlook.com)\n- Mingsheng Long (longmingsheng@gmail.com)\n\nor describe it in Issues.\n\nFor Q&A in Chinese, you can choose to ask questions here before sending an email. [迁移学习算法库答疑专区](https://zhuanlan.zhihu.com/p/248104070)\n\n## Citation\n\nIf you use this toolbox or benchmark in your research, please cite this project. \n\n```latex\n@misc{dalib,\n author = {Junguang Jiang, Baixu Chen, Bo Fu, Mingsheng Long},\n title = {Transfer-Learning-library},\n year = {2020},\n publisher = {GitHub},\n journal = {GitHub repository},\n howpublished = {\\url{https://github.com/thuml/Transfer-Learning-Library}},\n}\n```\n\n## Acknowledgment\n\nWe would like to thank School of Software, Tsinghua University and The National Engineering Laboratory for Big Data Software for providing such an excellent ML research platform.\n\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/common/__init__.py",
"content": "__all__ = ['modules', 'utils', 'vision']\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/common/loss/__init__.py",
"content": "import torch.nn as nn\nimport torch.nn.functional as F\n\n\nclass KnowledgeDistillationLoss(nn.Module):\n \"\"\"Knowledge Distillation Loss.\n\n Args:\n T (double): Temperature. Default: 1.\n reduction (str, optional): Specifies the reduction to apply to the output:\n ``'none'`` | ``'mean'`` | ``'sum'``. ``'none'``: no reduction will be applied,\n ``'mean'``: the sum of the output will be divided by the number of\n elements in the output, ``'sum'``: the output will be summed. Default: ``'batchmean'``\n\n Inputs:\n - y_student (tensor): logits output of the student\n - y_teacher (tensor): logits output of the teacher\n\n Shape:\n - y_student: (minibatch, `num_classes`)\n - y_teacher: (minibatch, `num_classes`)\n\n \"\"\"\n def __init__(self, T=1., reduction='batchmean'):\n super(KnowledgeDistillationLoss, self).__init__()\n self.T = T\n self.kl = nn.KLDivLoss(reduction=reduction)\n\n def forward(self, y_student, y_teacher):\n \"\"\"\"\"\"\n return self.kl(F.log_softmax(y_student / self.T, dim=-1), F.softmax(y_teacher / self.T, dim=-1))\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/common/modules/__init__.py",
"content": "from .classifier import *\nfrom .regressor import *\n\n__all__ = ['classifier', 'regressor']"
},
{
"path": "domain_generalization/Transfer-Learning-Library/common/modules/classifier.py",
"content": "\"\"\"\n@author: Junguang Jiang\n@contact: JiangJunguang1123@outlook.com\n\"\"\"\nfrom typing import Tuple, Optional, List, Dict\nimport torch.nn as nn\nimport torch\n\n__all__ = ['Classifier']\n\n\nclass Classifier(nn.Module):\n \"\"\"A generic Classifier class for domain adaptation.\n\n Args:\n backbone (torch.nn.Module): Any backbone to extract 2-d features from data\n num_classes (int): Number of classes\n bottleneck (torch.nn.Module, optional): Any bottleneck layer. Use no bottleneck by default\n bottleneck_dim (int, optional): Feature dimension of the bottleneck layer. Default: -1\n head (torch.nn.Module, optional): Any classifier head. Use :class:`torch.nn.Linear` by default\n finetune (bool): Whether finetune the classifier or train from scratch. Default: True\n\n .. note::\n Different classifiers are used in different domain adaptation algorithms to achieve better accuracy\n respectively, and we provide a suggested `Classifier` for different algorithms.\n Remember they are not the core of algorithms. You can implement your own `Classifier` and combine it with\n the domain adaptation algorithm in this algorithm library.\n\n .. note::\n The learning rate of this classifier is set 10 times to that of the feature extractor for better accuracy\n by default. If you have other optimization strategies, please over-ride :meth:`~Classifier.get_parameters`.\n\n Inputs:\n - x (tensor): input data fed to `backbone`\n\n Outputs:\n - predictions: classifier's predictions\n - features: features after `bottleneck` layer and before `head` layer\n\n Shape:\n - Inputs: (minibatch, *) where * means, any number of additional dimensions\n - predictions: (minibatch, `num_classes`)\n - features: (minibatch, `features_dim`)\n\n \"\"\"\n\n def __init__(self, backbone: nn.Module, num_classes: int, bottleneck: Optional[nn.Module] = None,\n bottleneck_dim: Optional[int] = -1, head: Optional[nn.Module] = None, finetune=True, pool_layer=None):\n super(Classifier, self).__init__()\n self.backbone = backbone\n self.num_classes = num_classes\n if pool_layer is None:\n self.pool_layer = nn.Sequential(\n nn.AdaptiveAvgPool2d(output_size=(1, 1)),\n nn.Flatten()\n )\n else:\n self.pool_layer = pool_layer\n if bottleneck is None:\n self.bottleneck = nn.Identity()\n self._features_dim = backbone.out_features\n else:\n self.bottleneck = bottleneck\n assert bottleneck_dim > 0\n self._features_dim = bottleneck_dim\n\n if head is None:\n self.head = nn.Linear(self._features_dim, num_classes)\n else:\n self.head = head\n self.finetune = finetune\n\n @property\n def features_dim(self) -> int:\n \"\"\"The dimension of features before the final `head` layer\"\"\"\n return self._features_dim\n\n def forward(self, x: torch.Tensor) -> Tuple[torch.Tensor, torch.Tensor]:\n \"\"\"\"\"\"\n f = self.pool_layer(self.backbone(x))\n f = self.bottleneck(f)\n predictions = self.head(f)\n if self.training:\n return predictions, f\n else:\n return predictions\n\n def get_parameters(self, base_lr=1.0) -> List[Dict]:\n \"\"\"A parameter list which decides optimization hyper-parameters,\n such as the relative learning rate of each layer\n \"\"\"\n params = [\n {\"params\": self.backbone.parameters(), \"lr\": 0.1 * base_lr if self.finetune else 1.0 * base_lr},\n {\"params\": self.bottleneck.parameters(), \"lr\": 1.0 * base_lr},\n {\"params\": self.head.parameters(), \"lr\": 1.0 * base_lr},\n ]\n\n return params\n\n\nclass ImageClassifier(Classifier):\n pass\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/common/modules/regressor.py",
"content": "\"\"\"\n@author: Junguang Jiang\n@contact: JiangJunguang1123@outlook.com\n\"\"\"\nfrom typing import Tuple, Optional, List, Dict\nimport torch.nn as nn\nimport torch\n\n__all__ = ['Regressor']\n\n\nclass Regressor(nn.Module):\n \"\"\"A generic Regressor class for domain adaptation.\n\n Args:\n backbone (torch.nn.Module): Any backbone to extract 2-d features from data\n num_factors (int): Number of factors\n bottleneck (torch.nn.Module, optional): Any bottleneck layer. Use no bottleneck by default\n bottleneck_dim (int, optional): Feature dimension of the bottleneck layer. Default: -1\n head (torch.nn.Module, optional): Any classifier head. Use `nn.Linear` by default\n finetune (bool): Whether finetune the classifier or train from scratch. Default: True\n\n .. note::\n The learning rate of this regressor is set 10 times to that of the feature extractor for better accuracy\n by default. If you have other optimization strategies, please over-ride :meth:`~Regressor.get_parameters`.\n\n Inputs:\n - x (tensor): input data fed to `backbone`\n\n Outputs:\n - predictions: regressor's predictions\n - features: features after `bottleneck` layer and before `head` layer\n\n Shape:\n - Inputs: (minibatch, *) where * means, any number of additional dimensions\n - predictions: (minibatch, `num_factors`)\n - features: (minibatch, `features_dim`)\n\n \"\"\"\n\n def __init__(self, backbone: nn.Module, num_factors: int, bottleneck: Optional[nn.Module] = None,\n bottleneck_dim=-1, head: Optional[nn.Module] = None, finetune=True):\n super(Regressor, self).__init__()\n self.backbone = backbone\n self.num_factors = num_factors\n if bottleneck is None:\n feature_dim = backbone.out_features\n self.bottleneck = nn.Sequential(\n nn.Conv2d(feature_dim, feature_dim, kernel_size=3, stride=1, padding=1),\n nn.BatchNorm2d(feature_dim, feature_dim),\n nn.ReLU(),\n nn.AdaptiveAvgPool2d(output_size=(1, 1)),\n nn.Flatten()\n )\n self._features_dim = feature_dim\n else:\n self.bottleneck = bottleneck\n assert bottleneck_dim > 0\n self._features_dim = bottleneck_dim\n\n if head is None:\n self.head = nn.Sequential(\n nn.Linear(self._features_dim, num_factors),\n nn.Sigmoid()\n )\n else:\n self.head = head\n self.finetune = finetune\n\n @property\n def features_dim(self) -> int:\n \"\"\"The dimension of features before the final `head` layer\"\"\"\n return self._features_dim\n\n def forward(self, x: torch.Tensor) -> Tuple[torch.Tensor, torch.Tensor]:\n \"\"\"\"\"\"\n f = self.backbone(x)\n f = self.bottleneck(f)\n predictions = self.head(f)\n if self.training:\n return predictions, f\n else:\n return predictions\n\n def get_parameters(self, base_lr=1.0) -> List[Dict]:\n \"\"\"A parameter list which decides optimization hyper-parameters,\n such as the relative learning rate of each layer\n \"\"\"\n params = [\n {\"params\": self.backbone.parameters(), \"lr\": 0.1 * base_lr if self.finetune else 1.0 * base_lr},\n {\"params\": self.bottleneck.parameters(), \"lr\": 1.0 * base_lr},\n {\"params\": self.head.parameters(), \"lr\": 1.0 * base_lr},\n ]\n\n return params\n\n\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/common/utils/__init__.py",
"content": "from .logger import CompleteLogger\nfrom .meter import *\nfrom .data import ForeverDataIterator\n\n__all__ = ['metric', 'analysis', 'meter', 'data', 'logger']"
},
{
"path": "domain_generalization/Transfer-Learning-Library/common/utils/analysis/__init__.py",
"content": "import torch\nfrom torch.utils.data import DataLoader\nimport torch.nn as nn\nimport tqdm\n\n\ndef collect_feature(data_loader: DataLoader, feature_extractor: nn.Module,\n device: torch.device, max_num_features=None) -> torch.Tensor:\n \"\"\"\n Fetch data from `data_loader`, and then use `feature_extractor` to collect features\n\n Args:\n data_loader (torch.utils.data.DataLoader): Data loader.\n feature_extractor (torch.nn.Module): A feature extractor.\n device (torch.device)\n max_num_features (int): The max number of features to return\n\n Returns:\n Features in shape (min(len(data_loader), max_num_features * mini-batch size), :math:`|\\mathcal{F}|`).\n \"\"\"\n feature_extractor.eval()\n all_features = []\n with torch.no_grad():\n for i, (images, target) in enumerate(tqdm.tqdm(data_loader)):\n if max_num_features is not None and i >= max_num_features:\n break\n images = images.to(device)\n feature = feature_extractor(images).cpu()\n all_features.append(feature)\n return torch.cat(all_features, dim=0)\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/common/utils/analysis/a_distance.py",
"content": "\"\"\"\n@author: Junguang Jiang\n@contact: JiangJunguang1123@outlook.com\n\"\"\"\nfrom torch.utils.data import TensorDataset\nimport torch\nimport torch.nn as nn\nimport torch.nn.functional as F\nfrom torch.utils.data import DataLoader\nfrom torch.optim import SGD\nfrom ..meter import AverageMeter\nfrom ..metric import binary_accuracy\n\n\nclass ANet(nn.Module):\n def __init__(self, in_feature):\n super(ANet, self).__init__()\n self.layer = nn.Linear(in_feature, 1)\n self.sigmoid = nn.Sigmoid()\n\n def forward(self, x):\n x = self.layer(x)\n x = self.sigmoid(x)\n return x\n\n\ndef calculate(source_feature: torch.Tensor, target_feature: torch.Tensor,\n device, progress=True, training_epochs=10):\n \"\"\"\n Calculate the :math:`\\mathcal{A}`-distance, which is a measure for distribution discrepancy.\n\n The definition is :math:`dist_\\mathcal{A} = 2 (1-2\\epsilon)`, where :math:`\\epsilon` is the\n test error of a classifier trained to discriminate the source from the target.\n\n Args:\n source_feature (tensor): features from source domain in shape :math:`(minibatch, F)`\n target_feature (tensor): features from target domain in shape :math:`(minibatch, F)`\n device (torch.device)\n progress (bool): if True, displays a the progress of training A-Net\n training_epochs (int): the number of epochs when training the classifier\n\n Returns:\n :math:`\\mathcal{A}`-distance\n \"\"\"\n source_label = torch.ones((source_feature.shape[0], 1))\n target_label = torch.zeros((target_feature.shape[0], 1))\n feature = torch.cat([source_feature, target_feature], dim=0)\n label = torch.cat([source_label, target_label], dim=0)\n\n dataset = TensorDataset(feature, label)\n length = len(dataset)\n train_size = int(0.8 * length)\n val_size = length - train_size\n train_set, val_set = torch.utils.data.random_split(dataset, [train_size, val_size])\n train_loader = DataLoader(train_set, batch_size=2, shuffle=True)\n val_loader = DataLoader(val_set, batch_size=8, shuffle=False)\n\n anet = ANet(feature.shape[1]).to(device)\n optimizer = SGD(anet.parameters(), lr=0.01)\n a_distance = 2.0\n for epoch in range(training_epochs):\n anet.train()\n for (x, label) in train_loader:\n x = x.to(device)\n label = label.to(device)\n anet.zero_grad()\n y = anet(x)\n loss = F.binary_cross_entropy(y, label)\n loss.backward()\n optimizer.step()\n\n anet.eval()\n meter = AverageMeter(\"accuracy\", \":4.2f\")\n with torch.no_grad():\n for (x, label) in val_loader:\n x = x.to(device)\n label = label.to(device)\n y = anet(x)\n acc = binary_accuracy(y, label)\n meter.update(acc, x.shape[0])\n error = 1 - meter.avg / 100\n a_distance = 2 * (1 - 2 * error)\n if progress:\n print(\"epoch {} accuracy: {} A-dist: {}\".format(epoch, meter.avg, a_distance))\n\n return a_distance\n\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/common/utils/analysis/tsne.py",
"content": "\"\"\"\n@author: Junguang Jiang\n@contact: JiangJunguang1123@outlook.com\n\"\"\"\nimport torch\nimport matplotlib\n\nmatplotlib.use('Agg')\nfrom sklearn.manifold import TSNE\nimport numpy as np\nimport matplotlib.pyplot as plt\nimport matplotlib.colors as col\n\n\ndef visualize(source_feature: torch.Tensor, target_feature: torch.Tensor,\n filename: str, source_color='r', target_color='b'):\n \"\"\"\n Visualize features from different domains using t-SNE.\n\n Args:\n source_feature (tensor): features from source domain in shape :math:`(minibatch, F)`\n target_feature (tensor): features from target domain in shape :math:`(minibatch, F)`\n filename (str): the file name to save t-SNE\n source_color (str): the color of the source features. Default: 'r'\n target_color (str): the color of the target features. Default: 'b'\n\n \"\"\"\n source_feature = source_feature.numpy()\n target_feature = target_feature.numpy()\n features = np.concatenate([source_feature, target_feature], axis=0)\n\n # map features to 2-d using TSNE\n X_tsne = TSNE(n_components=2, random_state=33).fit_transform(features)\n\n # domain labels, 1 represents source while 0 represents target\n domains = np.concatenate((np.ones(len(source_feature)), np.zeros(len(target_feature))))\n\n # visualize using matplotlib\n fig, ax = plt.subplots(figsize=(10, 10))\n ax.spines['top'].set_visible(False)\n ax.spines['right'].set_visible(False)\n ax.spines['bottom'].set_visible(False)\n ax.spines['left'].set_visible(False)\n plt.scatter(X_tsne[:, 0], X_tsne[:, 1], c=domains, cmap=col.ListedColormap([target_color, source_color]), s=20)\n plt.xticks([])\n plt.yticks([])\n plt.savefig(filename)\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/common/utils/data.py",
"content": "\"\"\"\n@author: Junguang Jiang\n@contact: JiangJunguang1123@outlook.com\n\"\"\"\nimport itertools\nimport random\nimport numpy as np\n\nimport torch\nfrom torch.utils.data import Sampler\nfrom torch.utils.data import DataLoader, Dataset\nfrom typing import TypeVar, Iterable, Dict, List\n\nT_co = TypeVar('T_co', covariant=True)\nT = TypeVar('T')\n\n\ndef send_to_device(tensor, device):\n \"\"\"\n Recursively sends the elements in a nested list/tuple/dictionary of tensors to a given device.\n\n Args:\n tensor (nested list/tuple/dictionary of :obj:`torch.Tensor`):\n The data to send to a given device.\n device (:obj:`torch.device`):\n The device to send the data to\n\n Returns:\n The same data structure as :obj:`tensor` with all tensors sent to the proper device.\n \"\"\"\n if isinstance(tensor, (list, tuple)):\n return type(tensor)(send_to_device(t, device) for t in tensor)\n elif isinstance(tensor, dict):\n return type(tensor)({k: send_to_device(v, device) for k, v in tensor.items()})\n elif not hasattr(tensor, \"to\"):\n return tensor\n return tensor.to(device)\n\n\nclass ForeverDataIterator:\n r\"\"\"A data iterator that will never stop producing data\"\"\"\n\n def __init__(self, data_loader: DataLoader, device=None):\n self.data_loader = data_loader\n self.iter = iter(self.data_loader)\n self.device = device\n\n def __next__(self):\n try:\n data = next(self.iter)\n if self.device is not None:\n data = send_to_device(data, self.device)\n except StopIteration:\n self.iter = iter(self.data_loader)\n data = next(self.iter)\n if self.device is not None:\n data = send_to_device(data, self.device)\n return data\n\n def __len__(self):\n return len(self.data_loader)\n\n\nclass RandomMultipleGallerySampler(Sampler):\n r\"\"\"Sampler from `In defense of the Triplet Loss for Person Re-Identification\n (ICCV 2017) `_. Assume there are :math:`N` identities in the dataset, this\n implementation simply samples :math:`K` images for every identity to form an iter of size :math:`N\\times K`. During\n training, we will call ``__iter__`` method of pytorch dataloader once we reach a ``StopIteration``, this guarantees\n every image in the dataset will eventually be selected and we are not wasting any training data.\n\n Args:\n dataset(list): each element of this list is a tuple (image_path, person_id, camera_id)\n num_instances(int, optional): number of images to sample for every identity (:math:`K` here)\n \"\"\"\n\n def __init__(self, dataset, num_instances=4):\n super(RandomMultipleGallerySampler, self).__init__(dataset)\n self.dataset = dataset\n self.num_instances = num_instances\n\n self.idx_to_pid = {}\n self.cid_list_per_pid = {}\n self.idx_list_per_pid = {}\n\n for idx, (_, pid, cid) in enumerate(dataset):\n if pid not in self.cid_list_per_pid:\n self.cid_list_per_pid[pid] = []\n self.idx_list_per_pid[pid] = []\n\n self.idx_to_pid[idx] = pid\n self.cid_list_per_pid[pid].append(cid)\n self.idx_list_per_pid[pid].append(idx)\n\n self.pid_list = list(self.idx_list_per_pid.keys())\n self.num_samples = len(self.pid_list)\n\n def __len__(self):\n return self.num_samples * self.num_instances\n\n def __iter__(self):\n def select_idxes(element_list, target_element):\n assert isinstance(element_list, list)\n return [i for i, element in enumerate(element_list) if element != target_element]\n\n pid_idxes = torch.randperm(len(self.pid_list)).tolist()\n final_idxes = []\n\n for perm_id in pid_idxes:\n i = random.choice(self.idx_list_per_pid[self.pid_list[perm_id]])\n _, _, cid = self.dataset[i]\n\n final_idxes.append(i)\n\n pid_i = self.idx_to_pid[i]\n cid_list = self.cid_list_per_pid[pid_i]\n idx_list = self.idx_list_per_pid[pid_i]\n selected_cid_list = select_idxes(cid_list, cid)\n\n if selected_cid_list:\n if len(selected_cid_list) >= self.num_instances:\n cid_idxes = np.random.choice(selected_cid_list, size=self.num_instances - 1, replace=False)\n else:\n cid_idxes = np.random.choice(selected_cid_list, size=self.num_instances - 1, replace=True)\n for cid_idx in cid_idxes:\n final_idxes.append(idx_list[cid_idx])\n else:\n selected_idxes = select_idxes(idx_list, i)\n if not selected_idxes:\n continue\n if len(selected_idxes) >= self.num_instances:\n pid_idxes = np.random.choice(selected_idxes, size=self.num_instances - 1, replace=False)\n else:\n pid_idxes = np.random.choice(selected_idxes, size=self.num_instances - 1, replace=True)\n\n for pid_idx in pid_idxes:\n final_idxes.append(idx_list[pid_idx])\n\n return iter(final_idxes)\n\n\nclass CombineDataset(Dataset[T_co]):\n r\"\"\"Dataset as a combination of multiple datasets.\n The element of each dataset must be a list, and the i-th element of the combined dataset\n is a list splicing of the i-th element of each sub dataset.\n The length of the combined dataset is the minimum of the lengths of all sub datasets.\n\n Arguments:\n datasets (sequence): List of datasets to be concatenated\n \"\"\"\n\n def __init__(self, datasets: Iterable[Dataset]) -> None:\n super(CombineDataset, self).__init__()\n # Cannot verify that datasets is Sized\n assert len(datasets) > 0, 'datasets should not be an empty iterable' # type: ignore\n self.datasets = list(datasets)\n\n def __len__(self):\n return min([len(d) for d in self.datasets])\n\n def __getitem__(self, idx):\n return list(itertools.chain(*[d[idx] for d in self.datasets]))\n\n\ndef concatenate(tensors):\n \"\"\"concatenate multiple batches into one batch.\n ``tensors`` can be :class:`torch.Tensor`, List or Dict, but they must be the same data format.\n \"\"\"\n if isinstance(tensors[0], torch.Tensor):\n return torch.cat(tensors, dim=0)\n elif isinstance(tensors[0], List):\n ret = []\n for i in range(len(tensors[0])):\n ret.append(concatenate([t[i] for t in tensors]))\n return ret\n elif isinstance(tensors[0], Dict):\n ret = dict()\n for k in tensors[0].keys():\n ret[k] = concatenate([t[k] for t in tensors])\n return ret\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/common/utils/logger.py",
"content": "\"\"\"\n@author: Junguang Jiang\n@contact: JiangJunguang1123@outlook.com\n\"\"\"\nimport os\nimport sys\nimport time\n\nclass TextLogger(object):\n \"\"\"Writes stream output to external text file.\n\n Args:\n filename (str): the file to write stream output\n stream: the stream to read from. Default: sys.stdout\n \"\"\"\n def __init__(self, filename, stream=sys.stdout):\n self.terminal = stream\n self.log = open(filename, 'a')\n\n def write(self, message):\n self.terminal.write(message)\n self.log.write(message)\n self.flush()\n\n def flush(self):\n self.terminal.flush()\n self.log.flush()\n\n def close(self):\n self.terminal.close()\n self.log.close()\n\n\nclass CompleteLogger:\n \"\"\"\n A useful logger that\n\n - writes outputs to files and displays them on the console at the same time.\n - manages the directory of checkpoints and debugging images.\n\n Args:\n root (str): the root directory of logger\n phase (str): the phase of training.\n\n \"\"\"\n\n def __init__(self, root, phase='train'):\n self.root = root\n self.phase = phase\n self.visualize_directory = os.path.join(self.root, \"visualize\")\n self.checkpoint_directory = os.path.join(self.root, \"checkpoints\")\n self.epoch = 0\n\n os.makedirs(self.root, exist_ok=True)\n os.makedirs(self.visualize_directory, exist_ok=True)\n os.makedirs(self.checkpoint_directory, exist_ok=True)\n\n # redirect std out\n now = time.strftime(\"%Y-%m-%d-%H_%M_%S\", time.localtime(time.time()))\n log_filename = os.path.join(self.root, \"{}-{}.txt\".format(phase, now))\n if os.path.exists(log_filename):\n os.remove(log_filename)\n self.logger = TextLogger(log_filename)\n sys.stdout = self.logger\n sys.stderr = self.logger\n if phase != 'train':\n self.set_epoch(phase)\n\n def set_epoch(self, epoch):\n \"\"\"Set the epoch number. Please use it during training.\"\"\"\n os.makedirs(os.path.join(self.visualize_directory, str(epoch)), exist_ok=True)\n self.epoch = epoch\n\n def _get_phase_or_epoch(self):\n if self.phase == 'train':\n return str(self.epoch)\n else:\n return self.phase\n\n def get_image_path(self, filename: str):\n \"\"\"\n Get the full image path for a specific filename\n \"\"\"\n return os.path.join(self.visualize_directory, self._get_phase_or_epoch(), filename)\n\n def get_checkpoint_path(self, name=None):\n \"\"\"\n Get the full checkpoint path.\n\n Args:\n name (optional): the filename (without file extension) to save checkpoint.\n If None, when the phase is ``train``, checkpoint will be saved to ``{epoch}.pth``.\n Otherwise, will be saved to ``{phase}.pth``.\n\n \"\"\"\n if name is None:\n name = self._get_phase_or_epoch()\n name = str(name)\n return os.path.join(self.checkpoint_directory, name + \".pth\")\n\n def close(self):\n self.logger.close()\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/common/utils/meter.py",
"content": "\"\"\"\n@author: Junguang Jiang\n@contact: JiangJunguang1123@outlook.com\n\"\"\"\nfrom typing import Optional, List\n\n\nclass AverageMeter(object):\n r\"\"\"Computes and stores the average and current value.\n\n Examples::\n\n >>> # Initialize a meter to record loss\n >>> losses = AverageMeter()\n >>> # Update meter after every minibatch update\n >>> losses.update(loss_value, batch_size)\n \"\"\"\n def __init__(self, name: str, fmt: Optional[str] = ':f'):\n self.name = name\n self.fmt = fmt\n self.reset()\n\n def reset(self):\n self.val = 0\n self.avg = 0\n self.sum = 0\n self.count = 0\n\n def update(self, val, n=1):\n self.val = val\n self.sum += val * n\n self.count += n\n if self.count > 0:\n self.avg = self.sum / self.count\n\n def __str__(self):\n fmtstr = '{name} {val' + self.fmt + '} ({avg' + self.fmt + '})'\n return fmtstr.format(**self.__dict__)\n\n\nclass AverageMeterDict(object):\n def __init__(self, names: List, fmt: Optional[str] = ':f'):\n self.dict = {\n name: AverageMeter(name, fmt) for name in names\n }\n\n def reset(self):\n for meter in self.dict.values():\n meter.reset()\n\n def update(self, accuracies, n=1):\n for name, acc in accuracies.items():\n self.dict[name].update(acc, n)\n\n def average(self):\n return {\n name: meter.avg for name, meter in self.dict.items()\n }\n\n def __getitem__(self, item):\n return self.dict[item]\n\n\nclass Meter(object):\n \"\"\"Computes and stores the current value.\"\"\"\n def __init__(self, name: str, fmt: Optional[str] = ':f'):\n self.name = name\n self.fmt = fmt\n self.reset()\n\n def reset(self):\n self.val = 0\n\n def update(self, val):\n self.val = val\n\n def __str__(self):\n fmtstr = '{name} {val' + self.fmt + '}'\n return fmtstr.format(**self.__dict__)\n\n\nclass ProgressMeter(object):\n def __init__(self, num_batches, meters, prefix=\"\"):\n self.batch_fmtstr = self._get_batch_fmtstr(num_batches)\n self.meters = meters\n self.prefix = prefix\n\n def display(self, batch):\n entries = [self.prefix + self.batch_fmtstr.format(batch)]\n entries += [str(meter) for meter in self.meters]\n print('\\t'.join(entries))\n\n def _get_batch_fmtstr(self, num_batches):\n num_digits = len(str(num_batches // 1))\n fmt = '{:' + str(num_digits) + 'd}'\n return '[' + fmt + '/' + fmt.format(num_batches) + ']'\n\n\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/common/utils/metric/__init__.py",
"content": "import torch\nimport prettytable\n\n__all__ = ['keypoint_detection']\n\ndef binary_accuracy(output: torch.Tensor, target: torch.Tensor) -> float:\n \"\"\"Computes the accuracy for binary classification\"\"\"\n with torch.no_grad():\n batch_size = target.size(0)\n pred = (output >= 0.5).float().t().view(-1)\n correct = pred.eq(target.view(-1)).float().sum()\n correct.mul_(100. / batch_size)\n return correct\n\n\ndef accuracy(output, target, topk=(1,)):\n r\"\"\"\n Computes the accuracy over the k top predictions for the specified values of k\n\n Args:\n output (tensor): Classification outputs, :math:`(N, C)` where `C = number of classes`\n target (tensor): :math:`(N)` where each value is :math:`0 \\leq \\text{targets}[i] \\leq C-1`\n topk (sequence[int]): A list of top-N number.\n\n Returns:\n Top-N accuracies (N :math:`\\in` topK).\n \"\"\"\n with torch.no_grad():\n maxk = max(topk)\n batch_size = target.size(0)\n\n _, pred = output.topk(maxk, 1, True, True)\n pred = pred.t()\n correct = pred.eq(target[None])\n\n res = []\n for k in topk:\n correct_k = correct[:k].flatten().sum(dtype=torch.float32)\n res.append(correct_k * (100.0 / batch_size))\n return res\n\n\nclass ConfusionMatrix(object):\n def __init__(self, num_classes):\n self.num_classes = num_classes\n self.mat = None\n\n def update(self, target, output):\n \"\"\"\n Update confusion matrix.\n\n Args:\n target: ground truth\n output: predictions of models\n\n Shape:\n - target: :math:`(minibatch, C)` where C means the number of classes.\n - output: :math:`(minibatch, C)` where C means the number of classes.\n \"\"\"\n n = self.num_classes\n if self.mat is None:\n self.mat = torch.zeros((n, n), dtype=torch.int64, device=target.device)\n with torch.no_grad():\n k = (target >= 0) & (target < n)\n inds = n * target[k].to(torch.int64) + output[k]\n self.mat += torch.bincount(inds, minlength=n**2).reshape(n, n)\n\n def reset(self):\n self.mat.zero_()\n\n def compute(self):\n \"\"\"compute global accuracy, per-class accuracy and per-class IoU\"\"\"\n h = self.mat.float()\n acc_global = torch.diag(h).sum() / h.sum()\n acc = torch.diag(h) / h.sum(1)\n iu = torch.diag(h) / (h.sum(1) + h.sum(0) - torch.diag(h))\n return acc_global, acc, iu\n\n # def reduce_from_all_processes(self):\n # if not torch.distributed.is_available():\n # return\n # if not torch.distributed.is_initialized():\n # return\n # torch.distributed.barrier()\n # torch.distributed.all_reduce(self.mat)\n\n def __str__(self):\n acc_global, acc, iu = self.compute()\n return (\n 'global correct: {:.1f}\\n'\n 'average row correct: {}\\n'\n 'IoU: {}\\n'\n 'mean IoU: {:.1f}').format(\n acc_global.item() * 100,\n ['{:.1f}'.format(i) for i in (acc * 100).tolist()],\n ['{:.1f}'.format(i) for i in (iu * 100).tolist()],\n iu.mean().item() * 100)\n\n def format(self, classes: list):\n \"\"\"Get the accuracy and IoU for each class in the table format\"\"\"\n acc_global, acc, iu = self.compute()\n\n table = prettytable.PrettyTable([\"class\", \"acc\", \"iou\"])\n for i, class_name, per_acc, per_iu in zip(range(len(classes)), classes, (acc * 100).tolist(), (iu * 100).tolist()):\n table.add_row([class_name, per_acc, per_iu])\n\n return 'global correct: {:.1f}\\nmean correct:{:.1f}\\nmean IoU: {:.1f}\\n{}'.format(\n acc_global.item() * 100, acc.mean().item() * 100, iu.mean().item() * 100, table.get_string())\n\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/common/utils/metric/keypoint_detection.py",
"content": "\"\"\"\n@author: Junguang Jiang\n@contact: JiangJunguang1123@outlook.com\n\"\"\"\n# TODO: add documentation\nimport numpy as np\n\n\ndef get_max_preds(batch_heatmaps):\n '''\n get predictions from score maps\n heatmaps: numpy.ndarray([batch_size, num_joints, height, width])\n '''\n assert isinstance(batch_heatmaps, np.ndarray), \\\n 'batch_heatmaps should be numpy.ndarray'\n assert batch_heatmaps.ndim == 4, 'batch_images should be 4-ndim'\n\n batch_size = batch_heatmaps.shape[0]\n num_joints = batch_heatmaps.shape[1]\n width = batch_heatmaps.shape[3]\n heatmaps_reshaped = batch_heatmaps.reshape((batch_size, num_joints, -1))\n idx = np.argmax(heatmaps_reshaped, 2)\n maxvals = np.amax(heatmaps_reshaped, 2)\n\n maxvals = maxvals.reshape((batch_size, num_joints, 1))\n idx = idx.reshape((batch_size, num_joints, 1))\n\n preds = np.tile(idx, (1, 1, 2)).astype(np.float32)\n\n preds[:, :, 0] = (preds[:, :, 0]) % width\n preds[:, :, 1] = np.floor((preds[:, :, 1]) / width)\n\n pred_mask = np.tile(np.greater(maxvals, 0.0), (1, 1, 2))\n pred_mask = pred_mask.astype(np.float32)\n\n preds *= pred_mask\n return preds, maxvals\n\n\ndef calc_dists(preds, target, normalize):\n preds = preds.astype(np.float32)\n target = target.astype(np.float32)\n dists = np.zeros((preds.shape[1], preds.shape[0]))\n for n in range(preds.shape[0]):\n for c in range(preds.shape[1]):\n if target[n, c, 0] > 1 and target[n, c, 1] > 1:\n normed_preds = preds[n, c, :] / normalize[n]\n normed_targets = target[n, c, :] / normalize[n]\n dists[c, n] = np.linalg.norm(normed_preds - normed_targets)\n else:\n dists[c, n] = -1\n return dists\n\n\ndef dist_acc(dists, thr=0.5):\n ''' Return percentage below threshold while ignoring values with a -1 '''\n dist_cal = np.not_equal(dists, -1)\n num_dist_cal = dist_cal.sum()\n if num_dist_cal > 0:\n return np.less(dists[dist_cal], thr).sum() * 1.0 / num_dist_cal\n else:\n return -1\n\n\ndef accuracy(output, target, hm_type='gaussian', thr=0.5):\n '''\n Calculate accuracy according to PCK,\n but uses ground truth heatmap rather than x,y locations\n First value to be returned is average accuracy across 'idxs',\n followed by individual accuracies\n '''\n idx = list(range(output.shape[1]))\n norm = 1.0\n if hm_type == 'gaussian':\n pred, _ = get_max_preds(output)\n target, _ = get_max_preds(target)\n h = output.shape[2]\n w = output.shape[3]\n norm = np.ones((pred.shape[0], 2)) * np.array([h, w]) / 10\n dists = calc_dists(pred, target, norm)\n\n acc = np.zeros(len(idx))\n avg_acc = 0\n cnt = 0\n\n for i in range(len(idx)):\n acc[i] = dist_acc(dists[idx[i]], thr)\n if acc[i] >= 0:\n avg_acc = avg_acc + acc[i]\n cnt += 1\n\n avg_acc = avg_acc / cnt if cnt != 0 else 0\n\n return acc, avg_acc, cnt, pred\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/common/utils/metric/reid.py",
"content": "# TODO: add documentation\n\"\"\"\nModified from https://github.com/yxgeee/MMT\n@author: Baixu Chen\n@contact: cbx_99_hasta@outlook.com\n\"\"\"\nimport os\nimport os.path as osp\nfrom collections import defaultdict\nimport time\nimport numpy as np\nimport torch\nimport torch.nn.functional as F\nfrom sklearn.metrics import average_precision_score\nfrom common.utils.meter import AverageMeter, ProgressMeter\n\n\ndef unique_sample(ids_dict, num):\n \"\"\"Randomly choose one instance for each person id, these instances will not be selected again\"\"\"\n mask = np.zeros(num, dtype=np.bool)\n for _, indices in ids_dict.items():\n i = np.random.choice(indices)\n mask[i] = True\n return mask\n\n\ndef cmc(dist_mat, query_ids, gallery_ids, query_cams, gallery_cams, topk=100, separate_camera_set=False,\n single_gallery_shot=False, first_match_break=False):\n \"\"\"Compute Cumulative Matching Characteristics (CMC)\"\"\"\n dist_mat = dist_mat.cpu().numpy()\n m, n = dist_mat.shape\n query_ids = np.asarray(query_ids)\n gallery_ids = np.asarray(gallery_ids)\n query_cams = np.asarray(query_cams)\n gallery_cams = np.asarray(gallery_cams)\n # Sort and find correct matches\n indices = np.argsort(dist_mat, axis=1)\n matches = (gallery_ids[indices] == query_ids[:, np.newaxis])\n # Compute CMC for each query\n ret = np.zeros(topk)\n num_valid_queries = 0\n for i in range(m):\n # Filter out the same id and same camera\n valid = ((gallery_ids[indices[i]] != query_ids[i]) |\n (gallery_cams[indices[i]] != query_cams[i]))\n if separate_camera_set:\n # Filter out samples from same camera\n valid &= (gallery_cams[indices[i]] != query_cams[i])\n if not np.any(matches[i, valid]): continue\n if single_gallery_shot:\n repeat = 10\n gids = gallery_ids[indices[i][valid]]\n inds = np.where(valid)[0]\n ids_dict = defaultdict(list)\n for j, x in zip(inds, gids):\n ids_dict[x].append(j)\n else:\n repeat = 1\n for _ in range(repeat):\n if single_gallery_shot:\n # Randomly choose one instance for each id\n sampled = (valid & unique_sample(ids_dict, len(valid)))\n index = np.nonzero(matches[i, sampled])[0]\n else:\n index = np.nonzero(matches[i, valid])[0]\n delta = 1. / (len(index) * repeat)\n for j, k in enumerate(index):\n if k - j >= topk: break\n if first_match_break:\n ret[k - j] += 1\n break\n ret[k - j] += delta\n num_valid_queries += 1\n if num_valid_queries == 0:\n raise RuntimeError(\"No valid query\")\n return ret.cumsum() / num_valid_queries\n\n\ndef mean_ap(dist_mat, query_ids, gallery_ids, query_cams, gallery_cams):\n \"\"\"Compute mean average precision (mAP)\"\"\"\n dist_mat = dist_mat.cpu().numpy()\n m, n = dist_mat.shape\n query_ids = np.asarray(query_ids)\n gallery_ids = np.asarray(gallery_ids)\n query_cams = np.asarray(query_cams)\n gallery_cams = np.asarray(gallery_cams)\n # Sort and find correct matches\n indices = np.argsort(dist_mat, axis=1)\n matches = (gallery_ids[indices] == query_ids[:, np.newaxis])\n # Compute AP for each query\n aps = []\n for i in range(m):\n # Filter out the same id and same camera\n valid = ((gallery_ids[indices[i]] != query_ids[i]) |\n (gallery_cams[indices[i]] != query_cams[i]))\n y_true = matches[i, valid]\n y_score = -dist_mat[i][indices[i]][valid]\n if not np.any(y_true): continue\n aps.append(average_precision_score(y_true, y_score))\n if len(aps) == 0:\n raise RuntimeError(\"No valid query\")\n return np.mean(aps)\n\n\ndef re_ranking(q_g_dist, q_q_dist, g_g_dist, k1=20, k2=6, lambda_value=0.3):\n \"\"\"Perform re-ranking with distance matrix between query and gallery images `q_g_dist`, distance matrix between\n query and query images `q_q_dist` and distance matrix between gallery and gallery images `g_g_dist`.\n \"\"\"\n q_g_dist = q_g_dist.cpu().numpy()\n q_q_dist = q_q_dist.cpu().numpy()\n g_g_dist = g_g_dist.cpu().numpy()\n\n original_dist = np.concatenate(\n [np.concatenate([q_q_dist, q_g_dist], axis=1),\n np.concatenate([q_g_dist.T, g_g_dist], axis=1)],\n axis=0)\n original_dist = np.power(original_dist, 2).astype(np.float32)\n original_dist = np.transpose(1. * original_dist / np.max(original_dist, axis=0))\n V = np.zeros_like(original_dist).astype(np.float32)\n initial_rank = np.argsort(original_dist).astype(np.int32)\n\n query_num = q_g_dist.shape[0]\n gallery_num = q_g_dist.shape[0] + q_g_dist.shape[1]\n all_num = gallery_num\n\n for i in range(all_num):\n # k-reciprocal neighbors\n forward_k_neigh_index = initial_rank[i, :k1 + 1]\n backward_k_neigh_index = initial_rank[forward_k_neigh_index, :k1 + 1]\n fi = np.where(backward_k_neigh_index == i)[0]\n k_reciprocal_index = forward_k_neigh_index[fi]\n k_reciprocal_expansion_index = k_reciprocal_index\n for j in range(len(k_reciprocal_index)):\n candidate = k_reciprocal_index[j]\n candidate_forward_k_neigh_index = initial_rank[candidate, :int(np.around(k1 / 2.)) + 1]\n candidate_backward_k_neigh_index = initial_rank[candidate_forward_k_neigh_index,\n :int(np.around(k1 / 2.)) + 1]\n fi_candidate = np.where(candidate_backward_k_neigh_index == candidate)[0]\n candidate_k_reciprocal_index = candidate_forward_k_neigh_index[fi_candidate]\n if len(np.intersect1d(candidate_k_reciprocal_index, k_reciprocal_index)) > 2. / 3 * len(\n candidate_k_reciprocal_index):\n k_reciprocal_expansion_index = np.append(k_reciprocal_expansion_index, candidate_k_reciprocal_index)\n\n k_reciprocal_expansion_index = np.unique(k_reciprocal_expansion_index)\n weight = np.exp(-original_dist[i, k_reciprocal_expansion_index])\n V[i, k_reciprocal_expansion_index] = 1. * weight / np.sum(weight)\n original_dist = original_dist[:query_num, ]\n if k2 != 1:\n V_qe = np.zeros_like(V, dtype=np.float32)\n for i in range(all_num):\n V_qe[i, :] = np.mean(V[initial_rank[i, :k2], :], axis=0)\n V = V_qe\n del V_qe\n del initial_rank\n invIndex = []\n for i in range(gallery_num):\n invIndex.append(np.where(V[:, i] != 0)[0])\n\n jaccard_dist = np.zeros_like(original_dist, dtype=np.float32)\n\n for i in range(query_num):\n temp_min = np.zeros(shape=[1, gallery_num], dtype=np.float32)\n indNonZero = np.where(V[i, :] != 0)[0]\n indImages = [invIndex[ind] for ind in indNonZero]\n for j in range(len(indNonZero)):\n temp_min[0, indImages[j]] = temp_min[0, indImages[j]] + np.minimum(V[i, indNonZero[j]],\n V[indImages[j], indNonZero[j]])\n jaccard_dist[i] = 1 - temp_min / (2. - temp_min)\n\n final_dist = jaccard_dist * (1 - lambda_value) + original_dist * lambda_value\n del original_dist\n del V\n del jaccard_dist\n final_dist = final_dist[:query_num, query_num:]\n return final_dist\n\n\ndef extract_reid_feature(data_loader, model, device, normalize, print_freq=200):\n \"\"\"Extract feature for person ReID. If `normalize` is True, `cosine` distance will be employed as distance\n metric, otherwise `euclidean` distance.\n \"\"\"\n batch_time = AverageMeter('Time', ':6.3f')\n progress = ProgressMeter(\n len(data_loader),\n [batch_time],\n prefix='Collect feature: ')\n\n # switch to eval mode\n model.eval()\n feature_dict = dict()\n\n with torch.no_grad():\n end = time.time()\n for i, (images_batch, filenames_batch, _, _) in enumerate(data_loader):\n\n images_batch = images_batch.to(device)\n features_batch = model(images_batch)\n if normalize:\n features_batch = F.normalize(features_batch)\n\n for filename, feature in zip(filenames_batch, features_batch):\n feature_dict[filename] = feature\n\n # measure elapsed time\n batch_time.update(time.time() - end)\n end = time.time()\n\n if i % print_freq == 0:\n progress.display(i)\n\n return feature_dict\n\n\ndef pairwise_distance(feature_dict, query, gallery):\n \"\"\"Compute pairwise distance between two sets of features\"\"\"\n\n # concat features and convert to pytorch tensor\n # we compute pairwise distance metric on cpu because it may require a large amount of GPU memory, if you are using\n # gpu with a larger capacity, it's faster to calculate on gpu\n x = torch.cat([feature_dict[f].unsqueeze(0) for f, _, _ in query], dim=0).cpu()\n y = torch.cat([feature_dict[f].unsqueeze(0) for f, _, _ in gallery], dim=0).cpu()\n m, n = x.size(0), y.size(0)\n # flatten\n x = x.view(m, -1)\n y = y.view(n, -1)\n # compute dist_mat\n dist_mat = torch.pow(x, 2).sum(dim=1, keepdim=True).expand(m, n) + \\\n torch.pow(y, 2).sum(dim=1, keepdim=True).expand(n, m).t() - \\\n 2 * torch.matmul(x, y.t())\n return dist_mat\n\n\ndef evaluate_all(dist_mat, query, gallery, cmc_topk=(1, 5, 10), cmc_flag=False):\n \"\"\"Compute CMC score, mAP and return\"\"\"\n query_ids = [pid for _, pid, _ in query]\n gallery_ids = [pid for _, pid, _ in gallery]\n query_cams = [cid for _, _, cid in query]\n gallery_cams = [cid for _, _, cid in gallery]\n\n # Compute mean AP\n mAP = mean_ap(dist_mat, query_ids, gallery_ids, query_cams, gallery_cams)\n print('Mean AP: {:4.1%}'.format(mAP))\n\n if not cmc_flag:\n return mAP\n\n cmc_configs = {\n 'config': dict(separate_camera_set=False, single_gallery_shot=False, first_match_break=True)\n }\n cmc_scores = {name: cmc(dist_mat, query_ids, gallery_ids, query_cams, gallery_cams, **params) for name, params in\n cmc_configs.items()}\n\n print('CMC Scores:')\n for k in cmc_topk:\n print(' top-{:<4}{:12.1%}'.format(k, cmc_scores['config'][k - 1]))\n return cmc_scores['config'][0], mAP\n\n\ndef validate(val_loader, model, query, gallery, device, criterion='cosine', cmc_flag=False, rerank=False):\n assert criterion in ['cosine', 'euclidean']\n # when criterion == 'cosine', normalize feature of single image into unit norm\n normalize = (criterion == 'cosine')\n\n feature_dict = extract_reid_feature(val_loader, model, device, normalize)\n dist_mat = pairwise_distance(feature_dict, query, gallery)\n results = evaluate_all(dist_mat, query=query, gallery=gallery, cmc_flag=cmc_flag)\n if not rerank:\n return results\n # apply person re-ranking\n print('Applying person re-ranking')\n dist_mat_query = pairwise_distance(feature_dict, query, query)\n dist_mat_gallery = pairwise_distance(feature_dict, gallery, gallery)\n dist_mat = re_ranking(dist_mat, dist_mat_query, dist_mat_gallery)\n return evaluate_all(dist_mat, query=query, gallery=gallery, cmc_flag=cmc_flag)\n\n\n# location parameters for visualization\nGRID_SPACING = 10\nQUERY_EXTRA_SPACING = 90\n# border width\nBW = 5\nGREEN = (0, 255, 0)\nRED = (0, 0, 255)\n\n\ndef visualize_ranked_results(data_loader, model, query, gallery, device, visualize_dir, criterion='cosine',\n rerank=False, width=128, height=256, topk=10):\n \"\"\"Visualize ranker results. We first compute pair-wise distance between query images and gallery images. Then for\n every query image, `topk` gallery images with least distance between given query image are selected. We plot the\n query image and selected gallery images together. A green border denotes a match, and a red one denotes a mis-match.\n \"\"\"\n assert criterion in ['cosine', 'euclidean']\n normalize = (criterion == 'cosine')\n\n # compute pairwise distance matrix\n feature_dict = extract_reid_feature(data_loader, model, device, normalize)\n dist_mat = pairwise_distance(feature_dict, query, gallery)\n\n if rerank:\n dist_mat_query = pairwise_distance(feature_dict, query, query)\n dist_mat_gallery = pairwise_distance(feature_dict, gallery, gallery)\n dist_mat = re_ranking(dist_mat, dist_mat_query, dist_mat_gallery)\n\n # make dir if not exists\n os.makedirs(visualize_dir, exist_ok=True)\n\n dist_mat = dist_mat.numpy()\n num_q, num_g = dist_mat.shape\n print('query images: {}'.format(num_q))\n print('gallery images: {}'.format(num_g))\n\n assert num_q == len(query)\n assert num_g == len(gallery)\n\n # start visualizing\n import cv2\n sorted_idxes = np.argsort(dist_mat, axis=1)\n for q_idx in range(num_q):\n q_img_path, q_pid, q_cid = query[q_idx]\n\n q_img = cv2.imread(q_img_path)\n q_img = cv2.resize(q_img, (width, height))\n # use black border to denote query image\n q_img = cv2.copyMakeBorder(\n q_img, BW, BW, BW, BW, cv2.BORDER_CONSTANT, value=(0, 0, 0)\n )\n q_img = cv2.resize(q_img, (width, height))\n num_cols = topk + 1\n grid_img = 255 * np.ones(\n (height, num_cols * width + topk * GRID_SPACING + QUERY_EXTRA_SPACING, 3), dtype=np.uint8\n )\n grid_img[:, :width, :] = q_img\n\n # collect top-k gallery images with smallest distance\n rank_idx = 1\n for g_idx in sorted_idxes[q_idx, :]:\n g_img_path, g_pid, g_cid = gallery[g_idx]\n invalid = (q_pid == g_pid) & (q_cid == g_cid)\n if not invalid:\n matched = (g_pid == q_pid)\n border_color = GREEN if matched else RED\n g_img = cv2.imread(g_img_path)\n g_img = cv2.resize(g_img, (width, height))\n g_img = cv2.copyMakeBorder(\n g_img, BW, BW, BW, BW, cv2.BORDER_CONSTANT, value=border_color\n )\n g_img = cv2.resize(g_img, (width, height))\n start = rank_idx * width + rank_idx * GRID_SPACING + QUERY_EXTRA_SPACING\n end = (rank_idx + 1) * width + rank_idx * GRID_SPACING + QUERY_EXTRA_SPACING\n grid_img[:, start:end, :] = g_img\n\n rank_idx += 1\n if rank_idx > topk:\n break\n\n save_path = osp.basename(osp.splitext(q_img_path)[0])\n cv2.imwrite(osp.join(visualize_dir, save_path + '.jpg'), grid_img)\n\n if (q_idx + 1) % 100 == 0:\n print('Visualize {}/{}'.format(q_idx + 1, num_q))\n\n print('Visualization process is done, ranked results are saved to {}'.format(visualize_dir))\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/common/utils/scheduler.py",
"content": "\"\"\"\nModified from https://github.com/yxgeee/MMT\n@author: Baixu Chen\n@contact: cbx_99_hasta@outlook.com\n\"\"\"\nimport torch\nfrom bisect import bisect_right\n\n\nclass WarmupMultiStepLR(torch.optim.lr_scheduler._LRScheduler):\n r\"\"\"Starts with a warm-up phase, then decays the learning rate of each parameter group by gamma once the\n number of epoch reaches one of the milestones. When last_epoch=-1, sets initial lr as lr.\n\n Args:\n optimizer (Optimizer): Wrapped optimizer.\n milestones (list): List of epoch indices. Must be increasing.\n gamma (float): Multiplicative factor of learning rate decay.\n Default: 0.1.\n warmup_factor (float): a float number :math:`k` between 0 and 1, the start learning rate of warmup phase\n will be set to :math:`k*initial\\_lr`\n warmup_steps (int): number of warm-up steps.\n warmup_method (str): \"constant\" denotes a constant learning rate during warm-up phase and \"linear\" denotes a\n linear-increasing learning rate during warm-up phase.\n last_epoch (int): The index of last epoch. Default: -1.\n \"\"\"\n\n def __init__(\n self,\n optimizer,\n milestones,\n gamma=0.1,\n warmup_factor=1.0 / 3,\n warmup_steps=500,\n warmup_method=\"linear\",\n last_epoch=-1,\n ):\n if not list(milestones) == sorted(milestones):\n raise ValueError(\n \"Milestones should be a list of\" \" increasing integers. Got {}\",\n milestones,\n )\n\n if warmup_method not in (\"constant\", \"linear\"):\n raise ValueError(\n \"Only 'constant' or 'linear' warmup_method accepted\"\n \"got {}\".format(warmup_method)\n )\n self.milestones = milestones\n self.gamma = gamma\n self.warmup_factor = warmup_factor\n self.warmup_steps = warmup_steps\n self.warmup_method = warmup_method\n super(WarmupMultiStepLR, self).__init__(optimizer, last_epoch)\n\n def get_lr(self):\n warmup_factor = 1\n if self.last_epoch < self.warmup_steps:\n if self.warmup_method == \"constant\":\n warmup_factor = self.warmup_factor\n elif self.warmup_method == \"linear\":\n alpha = float(self.last_epoch) / float(self.warmup_steps)\n warmup_factor = self.warmup_factor * (1 - alpha) + alpha\n return [\n base_lr\n * warmup_factor\n * self.gamma ** bisect_right(self.milestones, self.last_epoch)\n for base_lr in self.base_lrs\n ]\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/common/vision/__init__.py",
"content": "__all__ = ['datasets', 'models', 'transforms']\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/common/vision/datasets/__init__.py",
"content": "from .imagelist import ImageList\nfrom .office31 import Office31\nfrom .officehome import OfficeHome\nfrom .visda2017 import VisDA2017\nfrom .officecaltech import OfficeCaltech\nfrom .domainnet import DomainNet\nfrom .imagenet_r import ImageNetR\nfrom .imagenet_sketch import ImageNetSketch\nfrom .pacs import PACS\nfrom .digits import *\nfrom .aircrafts import Aircraft\nfrom .cub200 import CUB200\nfrom .stanford_cars import StanfordCars\nfrom .stanford_dogs import StanfordDogs\nfrom .coco70 import COCO70\nfrom .oxfordpet import OxfordIIITPet\nfrom .dtd import DTD\nfrom .oxfordflowers import OxfordFlowers102\nfrom .patchcamelyon import PatchCamelyon\nfrom .retinopathy import Retinopathy\nfrom .eurosat import EuroSAT\nfrom .resisc45 import Resisc45\n\n__all__ = ['ImageList', 'Office31', 'OfficeHome', \"VisDA2017\", \"OfficeCaltech\", \"DomainNet\", \"ImageNetR\", \"ImageNetSketch\",\n \"Aircraft\", \"cub200\", \"StanfordCars\", \"StanfordDogs\", \"COCO70\", \"OxfordIIITPet\", \"PACS\", \"DTD\",\n \"OxfordFlowers102\", \"PatchCamelyon\", \"Retinopathy\", \"EuroSAT\", \"Resisc45\"]\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/common/vision/datasets/_util.py",
"content": "\"\"\"\n@author: Junguang Jiang\n@contact: JiangJunguang1123@outlook.com\n\"\"\"\nimport os\nfrom typing import List\nfrom torchvision.datasets.utils import download_and_extract_archive\n\n\ndef download(root: str, file_name: str, archive_name: str, url_link: str):\n \"\"\"\n Download file from internet url link.\n\n Args:\n root (str) The directory to put downloaded files.\n file_name: (str) The name of the unzipped file.\n archive_name: (str) The name of archive(zipped file) downloaded.\n url_link: (str) The url link to download data.\n\n .. note::\n If `file_name` already exists under path `root`, then it is not downloaded again.\n Else `archive_name` will be downloaded from `url_link` and extracted to `file_name`.\n \"\"\"\n if not os.path.exists(os.path.join(root, file_name)):\n print(\"Downloading {}\".format(file_name))\n # if os.path.exists(os.path.join(root, archive_name)):\n # os.remove(os.path.join(root, archive_name))\n try:\n download_and_extract_archive(url_link, download_root=root, filename=archive_name, remove_finished=False)\n except Exception:\n print(\"Fail to download {} from url link {}\".format(archive_name, url_link))\n print('Please check you internet connection.'\n \"Simply trying again may be fine.\")\n exit(0)\n\n\ndef check_exits(root: str, file_name: str):\n \"\"\"Check whether `file_name` exists under directory `root`. \"\"\"\n if not os.path.exists(os.path.join(root, file_name)):\n print(\"Dataset directory {} not found under {}\".format(file_name, root))\n exit(-1)\n\n\ndef read_list_from_file(file_name: str) -> List[str]:\n \"\"\"Read data from file and convert each line into an element in the list\"\"\"\n result = []\n with open(file_name, \"r\") as f:\n for line in f.readlines():\n result.append(line.strip())\n return result\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/common/vision/datasets/aircrafts.py",
"content": "\"\"\"\n@author: Yifei Ji\n@contact: jiyf990330@163.com\n\"\"\"\nimport os\nfrom typing import Optional\nfrom .imagelist import ImageList\nfrom ._util import download as download_data, check_exits\n\n\nclass Aircraft(ImageList):\n \"\"\"`FVGC-Aircraft `_ \\\n is a benchmark for the fine-grained visual categorization of aircraft. \\\n The dataset contains 10,200 images of aircraft, with 100 images for each \\\n of the 102 different aircraft variants.\n\n Args:\n root (str): Root directory of dataset\n split (str, optional): The dataset split, supports ``train``, or ``test``.\n sample_rate (int): The sampling rates to sample random ``training`` images for each category.\n Choices include 100, 50, 30, 15. Default: 100.\n download (bool, optional): If true, downloads the dataset from the internet and puts it \\\n in root directory. If dataset is already downloaded, it is not downloaded again.\n transform (callable, optional): A function/transform that takes in an PIL image and returns a \\\n transformed version. E.g, :class:`torchvision.transforms.RandomCrop`.\n target_transform (callable, optional): A function/transform that takes in the target and transforms it.\n\n .. note:: In `root`, there will exist following files after downloading.\n ::\n train/\n test/\n image_list/\n train_100.txt\n train_50.txt\n train_30.txt\n train_15.txt\n test.txt\n \"\"\"\n download_list = [\n (\"image_list\", \"image_list.zip\", \"https://cloud.tsinghua.edu.cn/f/04356d49d0054092b07e/?dl=1\"),\n (\"train\", \"train.tgz\", \"https://cloud.tsinghua.edu.cn/f/9fed22eba03046d69012/?dl=1\"),\n (\"test\", \"test.tgz\", \"https://cloud.tsinghua.edu.cn/f/8d5e4c1b031a4a608c68/?dl=1\"),\n ]\n image_list = {\n \"train\": \"image_list/train_100.txt\",\n \"train100\": \"image_list/train_100.txt\",\n \"train50\": \"image_list/train_50.txt\",\n \"train30\": \"image_list/train_30.txt\",\n \"train15\": \"image_list/train_15.txt\",\n \"test\": \"image_list/test.txt\",\n \"test100\": \"image_list/test.txt\",\n }\n CLASSES = ['707-320', '727-200', '737-200', '737-300', '737-400', '737-500', '737-600', '737-700', '737-800', '737-900', '747-100',\n '747-200', '747-300', '747-400', '757-200', '757-300', '767-200', '767-300', '767-400', '777-200', '777-300', 'A300B4',\n 'A310', 'A318', 'A319', 'A320', 'A321', 'A330-200', 'A330-300', 'A340-200', 'A340-300', 'A340-500', 'A340-600', 'A380',\n 'ATR-42', 'ATR-72', 'An-12', 'BAE 146-200', 'BAE 146-300', 'BAE-125', 'Beechcraft 1900', 'Boeing 717', 'C-130', 'C-47',\n 'CRJ-200', 'CRJ-700', 'CRJ-900', 'Cessna 172', 'Cessna 208', 'Cessna 525', 'Cessna 560', 'Challenger 600', 'DC-10',\n 'DC-3', 'DC-6', 'DC-8', 'DC-9-30', 'DH-82', 'DHC-1', 'DHC-6', 'DHC-8-100', 'DHC-8-300', 'DR-400', 'Dornier 328', 'E-170',\n 'E-190', 'E-195', 'EMB-120', 'ERJ 135', 'ERJ 145', 'Embraer Legacy 600', 'Eurofighter Typhoon', 'F-16A-B', 'F-A-18',\n 'Falcon 2000', 'Falcon 900', 'Fokker 100', 'Fokker 50', 'Fokker 70', 'Global Express', 'Gulfstream IV', 'Gulfstream V',\n 'Hawk T1', 'Il-76', 'L-1011', 'MD-11', 'MD-80', 'MD-87', 'MD-90', 'Metroliner', 'Model B200', 'PA-28', 'SR-20',\n 'Saab 2000', 'Saab 340', 'Spitfire', 'Tornado', 'Tu-134', 'Tu-154', 'Yak-42']\n\n def __init__(self, root: str, split: str, sample_rate: Optional[int]=100, download: Optional[bool] = False, **kwargs):\n\n if split == 'train':\n list_name = 'train' + str(sample_rate)\n assert list_name in self.image_list\n data_list_file = os.path.join(root, self.image_list[list_name])\n else:\n data_list_file = os.path.join(root, self.image_list['test'])\n\n if download:\n list(map(lambda args: download_data(root, *args), self.download_list))\n else:\n list(map(lambda file_name, _: check_exits(root, file_name), self.download_list))\n\n super(Aircraft, self).__init__(root, Aircraft.CLASSES, data_list_file=data_list_file, **kwargs)"
},
{
"path": "domain_generalization/Transfer-Learning-Library/common/vision/datasets/coco70.py",
"content": "\"\"\"\n@author: Yifei Ji\n@contact: jiyf990330@163.com\n\"\"\"\nimport os\nfrom typing import Optional\nfrom .imagelist import ImageList\nfrom ._util import download as download_data, check_exits\n\n\nclass COCO70(ImageList):\n \"\"\"COCO-70 dataset is a large-scale classification dataset (1000 images per class) created from\n `COCO `_ Dataset.\n It is used to explore the effect of fine-tuning with a large amount of data.\n\n Args:\n root (str): Root directory of dataset\n split (str, optional): The dataset split, supports ``train``, or ``test``.\n sample_rate (int): The sampling rates to sample random ``training`` images for each category.\n Choices include 100, 50, 30, 15. Default: 100.\n download (bool, optional): If true, downloads the dataset from the internet and puts it \\\n in root directory. If dataset is already downloaded, it is not downloaded again.\n transform (callable, optional): A function/transform that takes in an PIL image and returns a \\\n transformed version. E.g, :class:`torchvision.transforms.RandomCrop`.\n target_transform (callable, optional): A function/transform that takes in the target and transforms it.\n\n .. note:: In `root`, there will exist following files after downloading.\n ::\n train/\n test/\n image_list/\n train_100.txt\n train_50.txt\n train_30.txt\n train_15.txt\n test.txt\n \"\"\"\n download_list = [\n (\"image_list\", \"image_list.zip\", \"https://cloud.tsinghua.edu.cn/f/d2ffb62fe3d140f1a73c/?dl=1\"),\n (\"train\", \"train.tgz\", \"https://cloud.tsinghua.edu.cn/f/e0dc4368342948c5bb2a/?dl=1\"),\n (\"test\", \"test.tgz\", \"https://cloud.tsinghua.edu.cn/f/59393a55c818429fb8d1/?dl=1\"),\n ]\n image_list = {\n \"train\": \"image_list/train_100.txt\",\n \"train100\": \"image_list/train_100.txt\",\n \"train50\": \"image_list/train_50.txt\",\n \"train30\": \"image_list/train_30.txt\",\n \"train15\": \"image_list/train_15.txt\",\n \"test\": \"image_list/test.txt\",\n \"test100\": \"image_list/test.txt\",\n }\n CLASSES =['person', 'bicycle', 'car', 'motorcycle', 'airplane', 'bus', 'train', 'truck',\n 'boat', 'traffic_light', 'fire_hydrant', 'stop_sign', 'bench', 'bird', 'cat', 'dog',\n 'horse', 'sheep', 'cow', 'elephant', 'bear', 'zebra', 'giraffe', 'backpack', 'umbrella',\n 'handbag', 'tie', 'suitcase', 'skis', 'kite', 'baseball_bat', 'skateboard', 'surfboard',\n 'tennis_racket', 'bottle', 'wine_glass', 'cup', 'fork', 'knife', 'spoon', 'bowl', 'banana',\n 'apple', 'sandwich', 'orange', 'broccoli', 'carrot', 'hot_dog', 'pizza', 'donut', 'cake',\n 'chair', 'couch', 'potted_plant', 'bed', 'dining_table', 'toilet', 'tv', 'laptop',\n 'remote', 'keyboard', 'cell_phone', 'microwave', 'oven', 'sink', 'refrigerator', 'book', 'clock', 'vase', 'teddy_bear']\n\n def __init__(self, root: str, split: str, sample_rate: Optional[int] =100, download: Optional[bool] = False, **kwargs):\n\n if split == 'train':\n list_name = 'train' + str(sample_rate)\n assert list_name in self.image_list\n data_list_file = os.path.join(root, self.image_list[list_name])\n else:\n data_list_file = os.path.join(root, self.image_list['test'])\n\n if download:\n list(map(lambda args: download_data(root, *args), self.download_list))\n else:\n list(map(lambda file_name, _: check_exits(root, file_name), self.download_list))\n\n super(COCO70, self).__init__(root, COCO70.CLASSES, data_list_file=data_list_file, **kwargs)\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/common/vision/datasets/cub200.py",
"content": "\"\"\"\n@author: Yifei Ji\n@contact: jiyf990330@163.com\n\"\"\"\nimport os\nfrom typing import Optional\nfrom .imagelist import ImageList\nfrom ._util import download as download_data, check_exits\n\n\nclass CUB200(ImageList):\n \"\"\"`Caltech-UCSD Birds-200-2011 `_ \\\n is a dataset for fine-grained visual recognition with 11,788 images in 200 bird species. \\\n It is an extended version of the CUB-200 dataset, roughly doubling the number of images.\n\n Args:\n root (str): Root directory of dataset\n split (str, optional): The dataset split, supports ``train``, or ``test``.\n sample_rate (int): The sampling rates to sample random ``training`` images for each category.\n Choices include 100, 50, 30, 15. Default: 100.\n download (bool, optional): If true, downloads the dataset from the internet and puts it \\\n in root directory. If dataset is already downloaded, it is not downloaded again.\n transform (callable, optional): A function/transform that takes in an PIL image and returns a \\\n transformed version. E.g, :class:`torchvision.transforms.RandomCrop`.\n target_transform (callable, optional): A function/transform that takes in the target and transforms it.\n\n .. note:: In `root`, there will exist following files after downloading.\n ::\n train/\n test/\n image_list/\n train_100.txt\n train_50.txt\n train_30.txt\n train_15.txt\n test.txt\n \"\"\"\n download_list = [\n (\"image_list\", \"image_list.zip\", \"https://cloud.tsinghua.edu.cn/f/fa398a321b094a24a347/?dl=1\"),\n (\"train\", \"train.tgz\", \"https://cloud.tsinghua.edu.cn/f/521ba92bafc04ee69c20/?dl=1\"),\n (\"test\", \"test.tgz\", \"https://cloud.tsinghua.edu.cn/f/cc7ef72081e64bc7a218/?dl=1\"),\n ]\n image_list = {\n \"train\": \"image_list/train_100.txt\",\n \"train100\": \"image_list/train_100.txt\",\n \"train50\": \"image_list/train_50.txt\",\n \"train30\": \"image_list/train_30.txt\",\n \"train15\": \"image_list/train_15.txt\",\n \"test\": \"image_list/test.txt\",\n \"test100\": \"image_list/test.txt\",\n }\n CLASSES = ['001.Black_footed_Albatross', '002.Laysan_Albatross', '003.Sooty_Albatross', '004.Groove_billed_Ani', '005.Crested_Auklet', '006.Least_Auklet', '007.Parakeet_Auklet', '008.Rhinoceros_Auklet', '009.Brewer_Blackbird', '010.Red_winged_Blackbird', '011.Rusty_Blackbird', '012.Yellow_headed_Blackbird', '013.Bobolink', '014.Indigo_Bunting', '015.Lazuli_Bunting', '016.Painted_Bunting', '017.Cardinal', '018.Spotted_Catbird', '019.Gray_Catbird', '020.Yellow_breasted_Chat', '021.Eastern_Towhee', '022.Chuck_will_Widow', '023.Brandt_Cormorant', '024.Red_faced_Cormorant', '025.Pelagic_Cormorant', '026.Bronzed_Cowbird', '027.Shiny_Cowbird', '028.Brown_Creeper', '029.American_Crow', '030.Fish_Crow', '031.Black_billed_Cuckoo', '032.Mangrove_Cuckoo', '033.Yellow_billed_Cuckoo', '034.Gray_crowned_Rosy_Finch', '035.Purple_Finch', '036.Northern_Flicker', '037.Acadian_Flycatcher', '038.Great_Crested_Flycatcher', '039.Least_Flycatcher', '040.Olive_sided_Flycatcher', '041.Scissor_tailed_Flycatcher', '042.Vermilion_Flycatcher', '043.Yellow_bellied_Flycatcher', '044.Frigatebird', '045.Northern_Fulmar', '046.Gadwall', '047.American_Goldfinch', '048.European_Goldfinch', '049.Boat_tailed_Grackle', '050.Eared_Grebe',\n '051.Horned_Grebe', '052.Pied_billed_Grebe', '053.Western_Grebe', '054.Blue_Grosbeak', '055.Evening_Grosbeak', '056.Pine_Grosbeak', '057.Rose_breasted_Grosbeak', '058.Pigeon_Guillemot', '059.California_Gull', '060.Glaucous_winged_Gull', '061.Heermann_Gull', '062.Herring_Gull', '063.Ivory_Gull', '064.Ring_billed_Gull', '065.Slaty_backed_Gull', '066.Western_Gull', '067.Anna_Hummingbird', '068.Ruby_throated_Hummingbird', '069.Rufous_Hummingbird', '070.Green_Violetear', '071.Long_tailed_Jaeger', '072.Pomarine_Jaeger', '073.Blue_Jay', '074.Florida_Jay', '075.Green_Jay', '076.Dark_eyed_Junco', '077.Tropical_Kingbird', '078.Gray_Kingbird', '079.Belted_Kingfisher', '080.Green_Kingfisher', '081.Pied_Kingfisher', '082.Ringed_Kingfisher', '083.White_breasted_Kingfisher', '084.Red_legged_Kittiwake', '085.Horned_Lark', '086.Pacific_Loon', '087.Mallard', '088.Western_Meadowlark', '089.Hooded_Merganser', '090.Red_breasted_Merganser', '091.Mockingbird', '092.Nighthawk', '093.Clark_Nutcracker', '094.White_breasted_Nuthatch', '095.Baltimore_Oriole', '096.Hooded_Oriole', '097.Orchard_Oriole', '098.Scott_Oriole', '099.Ovenbird', '100.Brown_Pelican',\n '101.White_Pelican', '102.Western_Wood_Pewee', '103.Sayornis', '104.American_Pipit', '105.Whip_poor_Will', '106.Horned_Puffin', '107.Common_Raven', '108.White_necked_Raven', '109.American_Redstart', '110.Geococcyx', '111.Loggerhead_Shrike', '112.Great_Grey_Shrike', '113.Baird_Sparrow', '114.Black_throated_Sparrow', '115.Brewer_Sparrow', '116.Chipping_Sparrow', '117.Clay_colored_Sparrow', '118.House_Sparrow', '119.Field_Sparrow', '120.Fox_Sparrow', '121.Grasshopper_Sparrow', '122.Harris_Sparrow', '123.Henslow_Sparrow', '124.Le_Conte_Sparrow', '125.Lincoln_Sparrow', '126.Nelson_Sharp_tailed_Sparrow', '127.Savannah_Sparrow', '128.Seaside_Sparrow', '129.Song_Sparrow', '130.Tree_Sparrow', '131.Vesper_Sparrow', '132.White_crowned_Sparrow', '133.White_throated_Sparrow', '134.Cape_Glossy_Starling', '135.Bank_Swallow', '136.Barn_Swallow', '137.Cliff_Swallow', '138.Tree_Swallow', '139.Scarlet_Tanager', '140.Summer_Tanager', '141.Artic_Tern', '142.Black_Tern', '143.Caspian_Tern', '144.Common_Tern', '145.Elegant_Tern', '146.Forsters_Tern', '147.Least_Tern', '148.Green_tailed_Towhee', '149.Brown_Thrasher', '150.Sage_Thrasher',\n '151.Black_capped_Vireo', '152.Blue_headed_Vireo', '153.Philadelphia_Vireo', '154.Red_eyed_Vireo', '155.Warbling_Vireo', '156.White_eyed_Vireo', '157.Yellow_throated_Vireo', '158.Bay_breasted_Warbler', '159.Black_and_white_Warbler', '160.Black_throated_Blue_Warbler', '161.Blue_winged_Warbler', '162.Canada_Warbler', '163.Cape_May_Warbler', '164.Cerulean_Warbler', '165.Chestnut_sided_Warbler', '166.Golden_winged_Warbler', '167.Hooded_Warbler', '168.Kentucky_Warbler', '169.Magnolia_Warbler', '170.Mourning_Warbler', '171.Myrtle_Warbler', '172.Nashville_Warbler', '173.Orange_crowned_Warbler', '174.Palm_Warbler', '175.Pine_Warbler', '176.Prairie_Warbler', '177.Prothonotary_Warbler', '178.Swainson_Warbler', '179.Tennessee_Warbler', '180.Wilson_Warbler', '181.Worm_eating_Warbler', '182.Yellow_Warbler', '183.Northern_Waterthrush', '184.Louisiana_Waterthrush', '185.Bohemian_Waxwing', '186.Cedar_Waxwing', '187.American_Three_toed_Woodpecker', '188.Pileated_Woodpecker', '189.Red_bellied_Woodpecker', '190.Red_cockaded_Woodpecker', '191.Red_headed_Woodpecker', '192.Downy_Woodpecker', '193.Bewick_Wren', '194.Cactus_Wren', '195.Carolina_Wren', '196.House_Wren', '197.Marsh_Wren', '198.Rock_Wren', '199.Winter_Wren', '200.Common_Yellowthroat']\n\n def __init__(self, root: str, split: str, sample_rate: Optional[int] =100, download: Optional[bool] = False, **kwargs):\n\n if split == 'train':\n list_name = 'train' + str(sample_rate)\n assert list_name in self.image_list\n data_list_file = os.path.join(root, self.image_list[list_name])\n else:\n data_list_file = os.path.join(root, self.image_list['test'])\n\n if download:\n list(map(lambda args: download_data(root, *args), self.download_list))\n else:\n list(map(lambda file_name, _: check_exits(root, file_name), self.download_list))\n\n super(CUB200, self).__init__(root, CUB200.CLASSES, data_list_file=data_list_file, **kwargs)\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/common/vision/datasets/digits.py",
"content": "\"\"\"\n@author: Junguang Jiang, Baixu Chen\n@contact: JiangJunguang1123@outlook.com, cbx_99_hasta@outlook.com\n\"\"\"\nimport os\nfrom typing import Optional, Tuple, Any\nfrom .imagelist import ImageList\nfrom ._util import download as download_data, check_exits\n\n\nclass MNIST(ImageList):\n \"\"\"`MNIST `_ Dataset.\n\n Args:\n root (str): Root directory of dataset where ``MNIST/processed/training.pt``\n and ``MNIST/processed/test.pt`` exist.\n mode (str): The channel mode for image. Choices includes ``\"L\"```, ``\"RGB\"``.\n Default: ``\"L\"```\n split (str, optional): The dataset split, supports ``train``, or ``test``.\n download (bool, optional): If true, downloads the dataset from the internet and\n puts it in root directory. If dataset is already downloaded, it is not\n downloaded again.\n transform (callable, optional): A function/transform that takes in an PIL image\n and returns a transformed version. E.g, ``transforms.RandomCrop``\n \"\"\"\n download_list = [\n (\"image_list\", \"image_list.zip\", \"https://cloud.tsinghua.edu.cn/f/f18f1e115de94644b900/?dl=1\"),\n (\"mnist_train_image\", \"mnist_image.tar.gz\", \"https://cloud.tsinghua.edu.cn/f/fdf45c75d2e746acba93/?dl=1\"),\n # (\"mnist_test_image\", \"mnist_image.tar.gz\", \"https://cloud.tsinghua.edu.cn/f/fdf45c75d2e746acba93/?dl=1\")\n ]\n image_list = {\n \"train\": \"image_list/mnist_train.txt\",\n \"test\": \"image_list/mnist_test.txt\"\n }\n CLASSES = ['0 - zero', '1 - one', '2 - two', '3 - three', '4 - four',\n '5 - five', '6 - six', '7 - seven', '8 - eight', '9 - nine']\n\n def __init__(self, root, mode=\"L\", split='train', download: Optional[bool] = True, **kwargs):\n assert split in ['train', 'test']\n data_list_file = os.path.join(root, self.image_list[split])\n\n if download:\n list(map(lambda args: download_data(root, *args), self.download_list))\n else:\n list(map(lambda file_name, _: check_exits(root, file_name), self.download_list))\n\n assert mode in ['L', 'RGB']\n self.mode = mode\n super(MNIST, self).__init__(root, MNIST.CLASSES, data_list_file=data_list_file, **kwargs)\n\n def __getitem__(self, index: int) -> Tuple[Any, int]:\n \"\"\"\n Args:\n index (int): Index\n\n return (tuple): (image, target) where target is index of the target class.\n \"\"\"\n path, target = self.samples[index]\n img = self.loader(path).convert(self.mode)\n if self.transform is not None:\n img = self.transform(img)\n if self.target_transform is not None and target is not None:\n target = self.target_transform(target)\n return img, target\n\n @classmethod\n def get_classes(self):\n return MNIST.CLASSES\n\n\nclass USPS(ImageList):\n \"\"\"`USPS `_ Dataset.\n The data-format is : [label [index:value ]*256 \\\\n] * num_lines, where ``label`` lies in ``[1, 10]``.\n The value for each pixel lies in ``[-1, 1]``. Here we transform the ``label`` into ``[0, 9]``\n and make pixel values in ``[0, 255]``.\n\n Args:\n root (str): Root directory of dataset to store``USPS`` data files.\n mode (str): The channel mode for image. Choices includes ``\"L\"```, ``\"RGB\"``.\n Default: ``\"L\"```\n split (str, optional): The dataset split, supports ``train``, or ``test``.\n transform (callable, optional): A function/transform that takes in an PIL image\n and returns a transformed version. E.g, ``transforms.RandomCrop``\n download (bool, optional): If true, downloads the dataset from the internet and\n puts it in root directory. If dataset is already downloaded, it is not\n downloaded again.\n\n \"\"\"\n download_list = [\n (\"image_list\", \"image_list.zip\", \"https://cloud.tsinghua.edu.cn/f/10ddb319c24e40a08e58/?dl=1\"),\n (\"usps_train_image\", \"usps_image.tar.gz\", \"https://cloud.tsinghua.edu.cn/f/1d3d7e2540bd4392b715/?dl=1\"),\n # (\"usps_test_image\", \"usps_image.tar.gz\", \"https://cloud.tsinghua.edu.cn/f/1d3d7e2540bd4392b715/?dl=1\")\n ]\n image_list = {\n \"train\": \"image_list/usps_train.txt\",\n \"test\": \"image_list/usps_test.txt\"\n }\n CLASSES = ['0 - zero', '1 - one', '2 - two', '3 - three', '4 - four',\n '5 - five', '6 - six', '7 - seven', '8 - eight', '9 - nine']\n\n def __init__(self, root, mode=\"L\", split='train', download: Optional[bool] = True, **kwargs):\n assert split in ['train', 'test']\n data_list_file = os.path.join(root, self.image_list[split])\n\n if download:\n list(map(lambda args: download_data(root, *args), self.download_list))\n else:\n list(map(lambda file_name, _: check_exits(root, file_name), self.download_list))\n\n assert mode in ['L', 'RGB']\n self.mode = mode\n super(USPS, self).__init__(root, USPS.CLASSES, data_list_file=data_list_file, **kwargs)\n\n def __getitem__(self, index: int) -> Tuple[Any, int]:\n \"\"\"\n Args:\n index (int): Index\n\n return (tuple): (image, target) where target is index of the target class.\n \"\"\"\n path, target = self.samples[index]\n img = self.loader(path).convert(self.mode)\n if self.transform is not None:\n img = self.transform(img)\n if self.target_transform is not None and target is not None:\n target = self.target_transform(target)\n return img, target\n\n\nclass SVHN(ImageList):\n \"\"\"`SVHN `_ Dataset.\n Note: The SVHN dataset assigns the label `10` to the digit `0`. However, in this Dataset,\n we assign the label `0` to the digit `0` to be compatible with PyTorch loss functions which\n expect the class labels to be in the range `[0, C-1]`\n\n .. warning::\n\n This class needs `scipy `_ to load data from `.mat` format.\n\n Args:\n root (str): Root directory of dataset where directory\n ``SVHN`` exists.\n mode (str): The channel mode for image. Choices includes ``\"L\"```, ``\"RGB\"``.\n Default: ``\"RGB\"```\n split (str, optional): The dataset split, supports ``train``, or ``test``.\n transform (callable, optional): A function/transform that takes in an PIL image\n and returns a transformed version. E.g, ``transforms.RandomCrop``\n download (bool, optional): If true, downloads the dataset from the internet and\n puts it in root directory. If dataset is already downloaded, it is not\n downloaded again.\n\n \"\"\"\n download_list = [\n (\"image_list\", \"image_list.zip\", \"https://cloud.tsinghua.edu.cn/f/80a8a06c4a324c59a5e4/?dl=1\"),\n (\"svhn_image\", \"svhn_image.tar.gz\", \"https://cloud.tsinghua.edu.cn/f/0e48a871e00345eb91a9/?dl=1\")\n ]\n image_list = \"image_list/svhn_balanced.txt\"\n # image_list = \"image_list/svhn.txt\"\n CLASSES = ['0 - zero', '1 - one', '2 - two', '3 - three', '4 - four',\n '5 - five', '6 - six', '7 - seven', '8 - eight', '9 - nine']\n\n def __init__(self, root, mode=\"L\", download: Optional[bool] = True, **kwargs):\n data_list_file = os.path.join(root, self.image_list)\n\n if download:\n list(map(lambda args: download_data(root, *args), self.download_list))\n else:\n list(map(lambda file_name, _: check_exits(root, file_name), self.download_list))\n\n assert mode in ['L', 'RGB']\n self.mode = mode\n super(SVHN, self).__init__(root, SVHN.CLASSES, data_list_file=data_list_file, **kwargs)\n\n def __getitem__(self, index: int) -> Tuple[Any, int]:\n \"\"\"\n Args:\n index (int): Index\n\n return (tuple): (image, target) where target is index of the target class.\n \"\"\"\n path, target = self.samples[index]\n img = self.loader(path).convert(self.mode)\n if self.transform is not None:\n img = self.transform(img)\n if self.target_transform is not None and target is not None:\n target = self.target_transform(target)\n return img, target\n\n\nclass MNISTRGB(MNIST):\n def __init__(self, root, **kwargs):\n super(MNISTRGB, self).__init__(root, mode='RGB', **kwargs)\n\n\nclass USPSRGB(USPS):\n def __init__(self, root, **kwargs):\n super(USPSRGB, self).__init__(root, mode='RGB', **kwargs)\n\n\nclass SVHNRGB(SVHN):\n def __init__(self, root, **kwargs):\n super(SVHNRGB, self).__init__(root, mode='RGB', **kwargs)\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/common/vision/datasets/domainnet.py",
"content": "\"\"\"\n@author: Junguang Jiang\n@contact: JiangJunguang1123@outlook.com\n\"\"\"\nimport os\nfrom typing import Optional\nfrom .imagelist import ImageList\nfrom ._util import download as download_data, check_exits\n\n\nclass DomainNet(ImageList):\n \"\"\"`DomainNet `_ (cleaned version, recommended)\n\n See `Moment Matching for Multi-Source Domain Adaptation `_ for details.\n\n Args:\n root (str): Root directory of dataset\n task (str): The task (domain) to create dataset. Choices include ``'c'``:clipart, \\\n ``'i'``: infograph, ``'p'``: painting, ``'q'``: quickdraw, ``'r'``: real, ``'s'``: sketch\n split (str, optional): The dataset split, supports ``train``, or ``test``.\n download (bool, optional): If true, downloads the dataset from the internet and puts it \\\n in root directory. If dataset is already downloaded, it is not downloaded again.\n transform (callable, optional): A function/transform that takes in an PIL image and returns a \\\n transformed version. E.g, :class:`torchvision.transforms.RandomCrop`.\n target_transform (callable, optional): A function/transform that takes in the target and transforms it.\n\n .. note:: In `root`, there will exist following files after downloading.\n ::\n clipart/\n infograph/\n painting/\n quickdraw/\n real/\n sketch/\n image_list/\n clipart.txt\n ...\n \"\"\"\n download_list = [\n (\"image_list\", \"image_list.zip\", \"https://cloud.tsinghua.edu.cn/f/90ecb35bbd374e5e8c41/?dl=1\"),\n (\"clipart\", \"clipart.zip\", \"http://csr.bu.edu/ftp/visda/2019/multi-source/groundtruth/clipart.zip\"),\n (\"infograph\", \"infograph.zip\", \"http://csr.bu.edu/ftp/visda/2019/multi-source/infograph.zip\"),\n (\"painting\", \"painting.zip\", \"http://csr.bu.edu/ftp/visda/2019/multi-source/groundtruth/painting.zip\"),\n (\"quickdraw\", \"quickdraw.zip\", \"http://csr.bu.edu/ftp/visda/2019/multi-source/quickdraw.zip\"),\n (\"real\", \"real.zip\", \"http://csr.bu.edu/ftp/visda/2019/multi-source/real.zip\"),\n (\"sketch\", \"sketch.zip\", \"http://csr.bu.edu/ftp/visda/2019/multi-source/sketch.zip\"),\n ]\n image_list = {\n \"c\": \"clipart\",\n \"i\": \"infograph\",\n \"p\": \"painting\",\n \"q\": \"quickdraw\",\n \"r\": \"real\",\n \"s\": \"sketch\",\n }\n CLASSES = ['aircraft_carrier', 'airplane', 'alarm_clock', 'ambulance', 'angel', 'animal_migration', 'ant', 'anvil',\n 'apple', 'arm', 'asparagus', 'axe', 'backpack', 'banana', 'bandage', 'barn', 'baseball', 'baseball_bat',\n 'basket', 'basketball', 'bat', 'bathtub', 'beach', 'bear', 'beard', 'bed', 'bee', 'belt', 'bench',\n 'bicycle', 'binoculars', 'bird', 'birthday_cake', 'blackberry', 'blueberry', 'book', 'boomerang',\n 'bottlecap', 'bowtie', 'bracelet', 'brain', 'bread', 'bridge', 'broccoli', 'broom', 'bucket',\n 'bulldozer', 'bus', 'bush', 'butterfly', 'cactus', 'cake', 'calculator', 'calendar', 'camel', 'camera',\n 'camouflage', 'campfire', 'candle', 'cannon', 'canoe', 'car', 'carrot', 'castle', 'cat', 'ceiling_fan',\n 'cello', 'cell_phone', 'chair', 'chandelier', 'church', 'circle', 'clarinet', 'clock', 'cloud',\n 'coffee_cup', 'compass', 'computer', 'cookie', 'cooler', 'couch', 'cow', 'crab', 'crayon', 'crocodile',\n 'crown', 'cruise_ship', 'cup', 'diamond', 'dishwasher', 'diving_board', 'dog', 'dolphin', 'donut',\n 'door', 'dragon', 'dresser', 'drill', 'drums', 'duck', 'dumbbell', 'ear', 'elbow', 'elephant',\n 'envelope', 'eraser', 'eye', 'eyeglasses', 'face', 'fan', 'feather', 'fence', 'finger', 'fire_hydrant',\n 'fireplace', 'firetruck', 'fish', 'flamingo', 'flashlight', 'flip_flops', 'floor_lamp', 'flower',\n 'flying_saucer', 'foot', 'fork', 'frog', 'frying_pan', 'garden', 'garden_hose', 'giraffe', 'goatee',\n 'golf_club', 'grapes', 'grass', 'guitar', 'hamburger', 'hammer', 'hand', 'harp', 'hat', 'headphones',\n 'hedgehog', 'helicopter', 'helmet', 'hexagon', 'hockey_puck', 'hockey_stick', 'horse', 'hospital',\n 'hot_air_balloon', 'hot_dog', 'hot_tub', 'hourglass', 'house', 'house_plant', 'hurricane', 'ice_cream',\n 'jacket', 'jail', 'kangaroo', 'key', 'keyboard', 'knee', 'knife', 'ladder', 'lantern', 'laptop', 'leaf',\n 'leg', 'light_bulb', 'lighter', 'lighthouse', 'lightning', 'line', 'lion', 'lipstick', 'lobster',\n 'lollipop', 'mailbox', 'map', 'marker', 'matches', 'megaphone', 'mermaid', 'microphone', 'microwave',\n 'monkey', 'moon', 'mosquito', 'motorbike', 'mountain', 'mouse', 'moustache', 'mouth', 'mug', 'mushroom',\n 'nail', 'necklace', 'nose', 'ocean', 'octagon', 'octopus', 'onion', 'oven', 'owl', 'paintbrush',\n 'paint_can', 'palm_tree', 'panda', 'pants', 'paper_clip', 'parachute', 'parrot', 'passport', 'peanut',\n 'pear', 'peas', 'pencil', 'penguin', 'piano', 'pickup_truck', 'picture_frame', 'pig', 'pillow',\n 'pineapple', 'pizza', 'pliers', 'police_car', 'pond', 'pool', 'popsicle', 'postcard', 'potato',\n 'power_outlet', 'purse', 'rabbit', 'raccoon', 'radio', 'rain', 'rainbow', 'rake', 'remote_control',\n 'rhinoceros', 'rifle', 'river', 'roller_coaster', 'rollerskates', 'sailboat', 'sandwich', 'saw',\n 'saxophone', 'school_bus', 'scissors', 'scorpion', 'screwdriver', 'sea_turtle', 'see_saw', 'shark',\n 'sheep', 'shoe', 'shorts', 'shovel', 'sink', 'skateboard', 'skull', 'skyscraper', 'sleeping_bag',\n 'smiley_face', 'snail', 'snake', 'snorkel', 'snowflake', 'snowman', 'soccer_ball', 'sock', 'speedboat',\n 'spider', 'spoon', 'spreadsheet', 'square', 'squiggle', 'squirrel', 'stairs', 'star', 'steak', 'stereo',\n 'stethoscope', 'stitches', 'stop_sign', 'stove', 'strawberry', 'streetlight', 'string_bean', 'submarine',\n 'suitcase', 'sun', 'swan', 'sweater', 'swing_set', 'sword', 'syringe', 'table', 'teapot', 'teddy-bear',\n 'telephone', 'television', 'tennis_racquet', 'tent', 'The_Eiffel_Tower', 'The_Great_Wall_of_China',\n 'The_Mona_Lisa', 'tiger', 'toaster', 'toe', 'toilet', 'tooth', 'toothbrush', 'toothpaste', 'tornado',\n 'tractor', 'traffic_light', 'train', 'tree', 'triangle', 'trombone', 'truck', 'trumpet', 't-shirt',\n 'umbrella', 'underwear', 'van', 'vase', 'violin', 'washing_machine', 'watermelon', 'waterslide',\n 'whale', 'wheel', 'windmill', 'wine_bottle', 'wine_glass', 'wristwatch', 'yoga', 'zebra', 'zigzag']\n\n def __init__(self, root: str, task: str, split: Optional[str] = 'train', download: Optional[float] = False, **kwargs):\n assert task in self.image_list\n assert split in ['train', 'test']\n data_list_file = os.path.join(root, \"image_list\", \"{}_{}.txt\".format(self.image_list[task], split))\n print(\"loading {}\".format(data_list_file))\n\n if download:\n list(map(lambda args: download_data(root, *args), self.download_list))\n else:\n list(map(lambda args: check_exits(root, args[0]), self.download_list))\n\n super(DomainNet, self).__init__(root, DomainNet.CLASSES, data_list_file=data_list_file, **kwargs)\n\n @classmethod\n def domains(cls):\n return list(cls.image_list.keys())\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/common/vision/datasets/dtd.py",
"content": "\"\"\"\n@author: Junguang Jiang\n@contact: JiangJunguang1123@outlook.com\n\"\"\"\nimport os\nfrom common.vision.datasets import ImageList\nfrom ._util import download as download_data, check_exits\n\n\nclass DTD(ImageList):\n \"\"\"\n `The Describable Textures Dataset (DTD) `_ is an \\\n evolving collection of textural images in the wild, annotated with a series of human-centric attributes, \\\n inspired by the perceptual properties of textures. \\\n The task consists in classifying images of textural patterns (47 classes, with 120 training images each). \\\n Some of the textures are banded, bubbly, meshed, lined, or porous. \\\n The image size ranges between 300x300 and 640x640 pixels.\n\n Args:\n root (str): Root directory of dataset\n split (str, optional): The dataset split, supports ``train``, or ``test``.\n download (bool, optional): If true, downloads the dataset from the internet and puts it \\\n in root directory. If dataset is already downloaded, it is not downloaded again.\n transform (callable, optional): A function/transform that takes in an PIL image and returns a \\\n transformed version. E.g, :class:`torchvision.transforms.RandomCrop`.\n target_transform (callable, optional): A function/transform that takes in the target and transforms it.\n \"\"\"\n CLASSES =['banded', 'blotchy', 'braided', 'bubbly', 'bumpy', 'chequered', 'cobwebbed', 'cracked',\n 'crosshatched', 'crystalline', 'dotted', 'fibrous', 'flecked', 'freckled', 'frilly', 'gauzy',\n 'grid', 'grooved', 'honeycombed', 'interlaced', 'knitted', 'lacelike', 'lined', 'marbled',\n 'matted', 'meshed', 'paisley', 'perforated', 'pitted', 'pleated', 'polka-dotted', 'porous',\n 'potholed', 'scaly', 'smeared', 'spiralled', 'sprinkled', 'stained', 'stratified', 'striped',\n 'studded', 'swirly', 'veined', 'waffled', 'woven', 'wrinkled', 'zigzagged']\n\n def __init__(self, root, split, download=False, **kwargs):\n if download:\n download_data(root, \"dtd\", \"dtd.tgz\", \"https://cloud.tsinghua.edu.cn/f/97e7d188f0d74b5c9d36/?dl=1\")\n else:\n check_exits(root, \"dtd\")\n\n root = os.path.join(root, \"dtd\")\n super(DTD, self).__init__(root, DTD.CLASSES, os.path.join(root, \"imagelist\", \"{}.txt\".format(split)), **kwargs)\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/common/vision/datasets/eurosat.py",
"content": "\"\"\"\n@author: Junguang Jiang\n@contact: JiangJunguang1123@outlook.com\n\"\"\"\nimport os\nfrom common.vision.datasets import ImageList\nfrom ._util import download as download_data, check_exits\n\n\nclass EuroSAT(ImageList):\n \"\"\"\n `EuroSAT `_ dataset consists in classifying \\\n Sentinel-2 satellite images into 10 different types of land use (Residential, \\\n Industrial, River, Highway, etc). \\\n The spatial resolution corresponds to 10 meters per pixel, and the image size \\\n is 64x64 pixels.\n\n Args:\n root (str): Root directory of dataset\n split (str, optional): The dataset split, supports ``train``, or ``test``.\n download (bool, optional): If true, downloads the dataset from the internet and puts it \\\n in root directory. If dataset is already downloaded, it is not downloaded again.\n transform (callable, optional): A function/transform that takes in an PIL image and returns a \\\n transformed version. E.g, :class:`torchvision.transforms.RandomCrop`.\n target_transform (callable, optional): A function/transform that takes in the target and transforms it.\n \"\"\"\n CLASSES =['AnnualCrop', 'Forest', 'HerbaceousVegetation', 'Highway', 'Industrial', 'Pasture',\n 'PermanentCrop', 'Residential', 'River', 'SeaLake']\n\n def __init__(self, root, split='train', download=False, **kwargs):\n if download:\n download_data(root, \"eurosat\", \"eurosat.tgz\", \"https://cloud.tsinghua.edu.cn/f/9983d7ab86184d74bb17/?dl=1\")\n else:\n check_exits(root, \"eurosat\")\n split = 'train[:21600]' if split == 'train' else 'train[21600:]'\n\n root = os.path.join(root, \"eurosat\")\n super(EuroSAT, self).__init__(root, EuroSAT.CLASSES, os.path.join(root, \"imagelist\", \"{}.txt\".format(split)), **kwargs)\n\n\n\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/common/vision/datasets/imagelist.py",
"content": "\"\"\"\n@author: Junguang Jiang\n@contact: JiangJunguang1123@outlook.com\n\"\"\"\nimport os\nfrom typing import Optional, Callable, Tuple, Any, List\nimport torchvision.datasets as datasets\nfrom torchvision.datasets.folder import default_loader\n\n\nclass ImageList(datasets.VisionDataset):\n \"\"\"A generic Dataset class for image classification\n\n Args:\n root (str): Root directory of dataset\n classes (list[str]): The names of all the classes\n data_list_file (str): File to read the image list from.\n transform (callable, optional): A function/transform that takes in an PIL image \\\n and returns a transformed version. E.g, :class:`torchvision.transforms.RandomCrop`.\n target_transform (callable, optional): A function/transform that takes in the target and transforms it.\n\n .. note:: In `data_list_file`, each line has 2 values in the following format.\n ::\n source_dir/dog_xxx.png 0\n source_dir/cat_123.png 1\n target_dir/dog_xxy.png 0\n target_dir/cat_nsdf3.png 1\n\n The first value is the relative path of an image, and the second value is the label of the corresponding image.\n If your data_list_file has different formats, please over-ride :meth:`~ImageList.parse_data_file`.\n \"\"\"\n\n def __init__(self, root: str, classes: List[str], data_list_file: str,\n transform: Optional[Callable] = None, target_transform: Optional[Callable] = None):\n super().__init__(root, transform=transform, target_transform=target_transform)\n self.samples = self.parse_data_file(data_list_file)\n self.classes = classes\n self.class_to_idx = {cls: idx\n for idx, cls in enumerate(self.classes)}\n self.loader = default_loader\n self.data_list_file = data_list_file\n\n def __getitem__(self, index: int) -> Tuple[Any, int]:\n \"\"\"\n Args:\n index (int): Index\n return (tuple): (image, target) where target is index of the target class.\n \"\"\"\n path, target = self.samples[index]\n img = self.loader(path)\n if self.transform is not None:\n img = self.transform(img)\n if self.target_transform is not None and target is not None:\n target = self.target_transform(target)\n return img, target\n\n def __len__(self) -> int:\n return len(self.samples)\n\n def parse_data_file(self, file_name: str) -> List[Tuple[str, int]]:\n \"\"\"Parse file to data list\n\n Args:\n file_name (str): The path of data file\n return (list): List of (image path, class_index) tuples\n \"\"\"\n with open(file_name, \"r\") as f:\n data_list = []\n for line in f.readlines():\n split_line = line.split()\n target = split_line[-1]\n path = ' '.join(split_line[:-1])\n if not os.path.isabs(path):\n path = os.path.join(self.root, path)\n target = int(target)\n data_list.append((path, target))\n return data_list\n\n @property\n def num_classes(self) -> int:\n \"\"\"Number of classes\"\"\"\n return len(self.classes)\n\n @classmethod\n def domains(cls):\n \"\"\"All possible domain in this dataset\"\"\"\n raise NotImplemented"
},
{
"path": "domain_generalization/Transfer-Learning-Library/common/vision/datasets/imagenet_r.py",
"content": "\"\"\"\n@author: Junguang Jiang\n@contact: JiangJunguang1123@outlook.com\n\"\"\"\nfrom typing import Optional\nimport os\nfrom .imagelist import ImageList\nfrom ._util import download as download_data, check_exits\n\n\nclass ImageNetR(ImageList):\n \"\"\"ImageNet-R Dataset.\n\n Args:\n root (str): Root directory of dataset\n task (str): The task (domain) to create dataset. Choices include ``'A'``: amazon, \\\n ``'D'``: dslr and ``'W'``: webcam.\n download (bool, optional): If true, downloads the dataset from the internet and puts it \\\n in root directory. If dataset is already downloaded, it is not downloaded again.\n transform (callable, optional): A function/transform that takes in an PIL image and returns a \\\n transformed version. E.g, :class:`torchvision.transforms.RandomCrop`.\n target_transform (callable, optional): A function/transform that takes in the target and transforms it.\n\n .. note:: You need to put ``train`` directory of ImageNet-1K and ``imagenet_r`` directory of ImageNet-R\n manually in `root` directory.\n\n DALIB will only download ImageList automatically.\n In `root`, there will exist following files after preparing.\n ::\n train/\n n02128385/\n ...\n val/\n imagenet-r/\n n02128385/\n image_list/\n imagenet-train.txt\n imagenet-r.txt\n art.txt\n ...\n \"\"\"\n download_list = [\n (\"image_list\", \"image_list.zip\", \"https://cloud.tsinghua.edu.cn/f/8066e6c5a8974be6a702/?dl=1\"),\n ]\n image_list = {\n \"IN\": \"image_list/imagenet-train.txt\",\n \"IN-val\": \"image_list/imagenet-val.txt\",\n \"INR\": \"image_list/imagenet-r.txt\",\n \"art\": \"art.txt\",\n \"embroidery\": \"embroidery.txt\",\n \"misc\": \"misc.txt\",\n \"sculpture\": \"sculpture.txt\",\n \"tattoo\": \"tattoo.txt\",\n \"cartoon\": \"cartoon.txt\",\n \"graffiti\": \"graffiti.txt\",\n \"origami\": \"origami.txt\",\n \"sketch\": \"sketch.txt\",\n \"toy\": \"toy.txt\",\n \"deviantart\": \"deviantart.txt\",\n \"graphic\": \"graphic.txt\",\n \"painting\": \"painting.txt\",\n \"sticker\": \"sticker.txt\",\n \"videogame\": \"videogame.txt\"\n }\n CLASSES = ['n01443537', 'n01484850', 'n01494475', 'n01498041', 'n01514859', 'n01518878', 'n01531178', 'n01534433', 'n01614925', 'n01616318', 'n01630670', 'n01632777', 'n01644373', 'n01677366', 'n01694178', 'n01748264', 'n01770393', 'n01774750', 'n01784675', 'n01806143', 'n01820546', 'n01833805', 'n01843383', 'n01847000', 'n01855672', 'n01860187', 'n01882714', 'n01910747', 'n01944390', 'n01983481', 'n01986214'\n, 'n02007558', 'n02009912', 'n02051845', 'n02056570', 'n02066245', 'n02071294', 'n02077923', 'n02085620', 'n02086240', 'n02088094', 'n02088238', 'n02088364', 'n02088466', 'n02091032', 'n02091134', 'n02092339', 'n02094433', 'n02096585', 'n02097298', 'n02098286', 'n02099601', 'n02099712', 'n02102318', 'n02106030', 'n02106166', 'n02106550', 'n02106662', 'n02108089', 'n02108915', 'n02109525', 'n02110185', 'n02110341', 'n02110958', 'n02112018', 'n02112137', 'n02113023', 'n02113624', 'n02113799', 'n02114367', 'n02117135', 'n02119022', 'n02123045', 'n02128385', 'n02128757', 'n02129165', 'n02129604', 'n02130308', 'n02134084', 'n02138441', 'n02165456', 'n02190166', 'n02206856', 'n02219486', 'n02226429', 'n02233338', 'n02236044', 'n02268443', 'n02279972', 'n02317335', 'n02325366', 'n02346627', 'n02356798', 'n02363005', 'n02364673', 'n02391049', 'n02395406', 'n02398521', 'n02410509', 'n02423022', 'n02437616', 'n02445715', 'n02447366', 'n02480495', 'n02480855', 'n02481823', 'n02483362', 'n02486410', 'n02510455', 'n02526121', 'n02607072', 'n02655020', 'n02672831', 'n02701002', 'n02749479', 'n02769748', 'n02793495', 'n02797295', 'n02802426', 'n02808440', 'n02814860', 'n02823750', 'n02841315', 'n02843684', 'n02883205', 'n02906734', 'n02909870', 'n02939185', 'n02948072', 'n02950826', 'n02951358', 'n02966193', 'n02980441', 'n02992529', 'n03124170', 'n03272010', 'n03345487', 'n03372029', 'n03424325', 'n03452741', 'n03467068', 'n03481172', 'n03494278', 'n03495258', 'n03498962', 'n03594945', 'n03602883', 'n03630383', 'n03649909', 'n03676483', 'n03710193', 'n03773504', 'n03775071', 'n03888257', 'n03930630', 'n03947888', 'n04086273', 'n04118538', 'n04133789', 'n04141076', 'n04146614', 'n04147183', 'n04192698', 'n04254680', 'n04266014', 'n04275548', 'n04310018', 'n04325704', 'n04347754', 'n04389033', 'n04409515', 'n04465501', 'n04487394', 'n04522168', 'n04536866', 'n04552348', 'n04591713', 'n07614500', 'n07693725', 'n07695742', 'n07697313', 'n07697537', 'n07714571', 'n07714990', 'n07718472', 'n07720875', 'n07734744', 'n07742313', 'n07745940', 'n07749582', 'n07753275', 'n07753592', 'n07768694', 'n07873807', 'n07880968', 'n07920052', 'n09472597', 'n09835506', 'n10565667', 'n12267677']\n\n def __init__(self, root: str, task: str, split: Optional[str] = 'all', download: Optional[bool] = True, **kwargs):\n assert task in self.image_list\n assert split in [\"train\", \"val\", \"all\"]\n if task == \"IN\" and split == \"val\":\n task = \"IN-val\"\n\n data_list_file = os.path.join(root, self.image_list[task])\n\n if download:\n list(map(lambda args: download_data(root, *args), self.download_list))\n else:\n list(map(lambda file_name, _: check_exits(root, file_name), self.download_list))\n\n super(ImageNetR, self).__init__(root, ImageNetR.CLASSES, data_list_file=data_list_file, **kwargs)\n\n @classmethod\n def domains(cls):\n return list(cls.image_list.keys())"
},
{
"path": "domain_generalization/Transfer-Learning-Library/common/vision/datasets/imagenet_sketch.py",
"content": "\"\"\"\n@author: Junguang Jiang\n@contact: JiangJunguang1123@outlook.com\n\"\"\"\nfrom typing import Optional\nimport os\nfrom torchvision.datasets.imagenet import ImageNet\nfrom .imagelist import ImageList\nfrom ._util import download as download_data, check_exits\n\n\nclass ImageNetSketch(ImageList):\n \"\"\"ImageNet-Sketch Dataset.\n\n Args:\n root (str): Root directory of dataset\n task (str): The task (domain) to create dataset. Choices include ``'A'``: amazon, \\\n ``'D'``: dslr and ``'W'``: webcam.\n download (bool, optional): If true, downloads the dataset from the internet and puts it \\\n in root directory. If dataset is already downloaded, it is not downloaded again.\n transform (callable, optional): A function/transform that takes in an PIL image and returns a \\\n transformed version. E.g, :class:`torchvision.transforms.RandomCrop`.\n target_transform (callable, optional): A function/transform that takes in the target and transforms it.\n\n .. note:: You need to put ``train`` directory, ``metabin`` of ImageNet-1K and ``sketch`` directory of ImageNet-Sketch\n manually in `root` directory.\n\n DALIB will only download ImageList automatically.\n In `root`, there will exist following files after preparing.\n ::\n metabin (from ImageNet)\n train/\n n02128385/\n ...\n val/\n sketch/\n n02128385/\n image_list/\n imagenet-train.txt\n sketch.txt\n ...\n \"\"\"\n # TODO\n download_list = [\n (\"image_list\", \"image_list.zip\", \"https://cloud.tsinghua.edu.cn/f/8e12fff7b6224e5fa62b/?dl=1\"),\n ]\n image_list = {\n \"IN\": \"image_list/imagenet-train.txt\",\n \"IN-val\": \"image_list/imagenet-val.txt\",\n \"sketch\": \"image_list/sketch.txt\",\n }\n\n def __init__(self, root: str, task: str, split: Optional[str] = 'all', download: Optional[bool] = True, **kwargs):\n assert task in self.image_list\n assert split in [\"train\", \"val\", \"all\"]\n if task == \"IN\" and split == \"val\":\n task = \"IN-val\"\n\n data_list_file = os.path.join(root, self.image_list[task])\n\n if download:\n list(map(lambda args: download_data(root, *args), self.download_list))\n else:\n list(map(lambda file_name, _: check_exits(root, file_name), self.download_list))\n\n super(ImageNetSketch, self).__init__(root, ImageNet(root).classes, data_list_file=data_list_file, **kwargs)\n\n @classmethod\n def domains(cls):\n return list(cls.image_list.keys())"
},
{
"path": "domain_generalization/Transfer-Learning-Library/common/vision/datasets/keypoint_detection/__init__.py",
"content": "from .rendered_hand_pose import RenderedHandPose\nfrom .hand_3d_studio import Hand3DStudio, Hand3DStudioAll\nfrom .freihand import FreiHand\n\nfrom .surreal import SURREAL\nfrom .lsp import LSP\nfrom .human36m import Human36M\n\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/common/vision/datasets/keypoint_detection/freihand.py",
"content": "\"\"\"\n@author: Junguang Jiang\n@contact: JiangJunguang1123@outlook.com\n\"\"\"\nimport json\nimport time\nimport torch\nimport os\nimport os.path as osp\nfrom torchvision.datasets.utils import download_and_extract_archive\n\nfrom ...transforms.keypoint_detection import *\nfrom .keypoint_dataset import Hand21KeypointDataset\nfrom .util import *\n\n\n\"\"\" General util functions. \"\"\"\ndef _assert_exist(p):\n msg = 'File does not exists: %s' % p\n assert os.path.exists(p), msg\n\n\ndef json_load(p):\n _assert_exist(p)\n with open(p, 'r') as fi:\n d = json.load(fi)\n return d\n\n\ndef load_db_annotation(base_path, set_name=None):\n if set_name is None:\n # only training set annotations are released so this is a valid default choice\n set_name = 'training'\n\n print('Loading FreiHAND dataset index ...')\n t = time.time()\n\n # assumed paths to data containers\n k_path = os.path.join(base_path, '%s_K.json' % set_name)\n mano_path = os.path.join(base_path, '%s_mano.json' % set_name)\n xyz_path = os.path.join(base_path, '%s_xyz.json' % set_name)\n\n # load if exist\n K_list = json_load(k_path)\n mano_list = json_load(mano_path)\n xyz_list = json_load(xyz_path)\n\n # should have all the same length\n assert len(K_list) == len(mano_list), 'Size mismatch.'\n assert len(K_list) == len(xyz_list), 'Size mismatch.'\n\n print('Loading of %d samples done in %.2f seconds' % (len(K_list), time.time()-t))\n return list(zip(K_list, mano_list, xyz_list))\n\n\ndef projectPoints(xyz, K):\n \"\"\" Project 3D coordinates into image space. \"\"\"\n xyz = np.array(xyz)\n K = np.array(K)\n uv = np.matmul(K, xyz.T).T\n return uv[:, :2] / uv[:, -1:]\n\n\n\"\"\" Dataset related functions. \"\"\"\ndef db_size(set_name):\n \"\"\" Hardcoded size of the datasets. \"\"\"\n if set_name == 'training':\n return 32560 # number of unique samples (they exists in multiple 'versions')\n elif set_name == 'evaluation':\n return 3960\n else:\n assert 0, 'Invalid choice.'\n\n\nclass sample_version:\n gs = 'gs' # green screen\n hom = 'hom' # homogenized\n sample = 'sample' # auto colorization with sample points\n auto = 'auto' # auto colorization without sample points: automatic color hallucination\n\n db_size = db_size('training')\n\n @classmethod\n def valid_options(cls):\n return [cls.gs, cls.hom, cls.sample, cls.auto]\n\n\n @classmethod\n def check_valid(cls, version):\n msg = 'Invalid choice: \"%s\" (must be in %s)' % (version, cls.valid_options())\n assert version in cls.valid_options(), msg\n\n @classmethod\n def map_id(cls, id, version):\n cls.check_valid(version)\n return id + cls.db_size*cls.valid_options().index(version)\n\n\nclass FreiHand(Hand21KeypointDataset):\n \"\"\"`FreiHand Dataset `_\n\n Args:\n root (str): Root directory of dataset\n split (str, optional): The dataset split, supports ``train``, ``test``, or ``all``.\n task (str, optional): The post-processing option to create dataset. Choices include ``'gs'``: green screen \\\n recording, ``'auto'``: auto colorization without sample points: automatic color hallucination, \\\n ``'sample'``: auto colorization with sample points, ``'hom'``: homogenized, \\\n and ``'all'``: all hands. Default: 'all'.\n download (bool, optional): If true, downloads the dataset from the internet and puts it \\\n in root directory. If dataset is already downloaded, it is not downloaded again.\n transforms (callable, optional): A function/transform that takes in a dict (which contains PIL image and\n its labels) and returns a transformed version. E.g, :class:`~common.vision.transforms.keypoint_detection.Resize`.\n image_size (tuple): (width, height) of the image. Default: (256, 256)\n heatmap_size (tuple): (width, height) of the heatmap. Default: (64, 64)\n sigma (int): sigma parameter when generate the heatmap. Default: 2\n\n .. note:: In `root`, there will exist following files after downloading.\n ::\n *.json\n training/\n evaluation/\n \"\"\"\n def __init__(self, root, split='train', task='all', download=True, **kwargs):\n if download:\n if not osp.exists(osp.join(root, \"training\")) or not osp.exists(osp.join(root, \"evaluation\")):\n download_and_extract_archive(\"https://lmb.informatik.uni-freiburg.de/data/freihand/FreiHAND_pub_v2.zip\",\n download_root=root, filename=\"FreiHAND_pub_v2.zip\", remove_finished=False,\n extract_root=root)\n\n assert split in ['train', 'test', 'all']\n self.split = split\n\n assert task in ['all', 'gs', 'auto', 'sample', 'hom']\n self.task = task\n if task == 'all':\n samples = self.get_samples(root, 'gs') + self.get_samples(root, 'auto') + self.get_samples(root, 'sample') + self.get_samples(root, 'hom')\n else:\n samples = self.get_samples(root, task)\n random.seed(42)\n random.shuffle(samples)\n samples_len = len(samples)\n samples_split = min(int(samples_len * 0.2), 3200)\n if self.split == 'train':\n samples = samples[samples_split:]\n elif self.split == 'test':\n samples = samples[:samples_split]\n\n super(FreiHand, self).__init__(root, samples, **kwargs)\n\n def __getitem__(self, index):\n sample = self.samples[index]\n image_name = sample['name']\n image_path = os.path.join(self.root, image_name)\n image = Image.open(image_path)\n keypoint3d_camera = np.array(sample['keypoint3d']) # NUM_KEYPOINTS x 3\n keypoint2d = np.array(sample['keypoint2d']) # NUM_KEYPOINTS x 2\n intrinsic_matrix = np.array(sample['intrinsic_matrix'])\n Zc = keypoint3d_camera[:, 2]\n\n # Crop the images such that the hand is at the center of the image\n # The images will be 1.5 times larger than the hand\n # The crop process will change Xc and Yc, leaving Zc with no changes\n bounding_box = get_bounding_box(keypoint2d)\n w, h = image.size\n left, upper, right, lower = scale_box(bounding_box, w, h, 1.5)\n image, keypoint2d = crop(image, upper, left, lower - upper, right - left, keypoint2d)\n\n # Change all hands to right hands\n if sample['left'] is False:\n image, keypoint2d = hflip(image, keypoint2d)\n\n image, data = self.transforms(image, keypoint2d=keypoint2d, intrinsic_matrix=intrinsic_matrix)\n keypoint2d = data['keypoint2d']\n intrinsic_matrix = data['intrinsic_matrix']\n keypoint3d_camera = keypoint2d_to_3d(keypoint2d, intrinsic_matrix, Zc)\n\n # noramlize 2D pose:\n visible = np.ones((self.num_keypoints, ), dtype=np.float32)\n visible = visible[:, np.newaxis]\n # 2D heatmap\n target, target_weight = generate_target(keypoint2d, visible, self.heatmap_size, self.sigma, self.image_size)\n target = torch.from_numpy(target)\n target_weight = torch.from_numpy(target_weight)\n\n # normalize 3D pose:\n # put middle finger metacarpophalangeal (MCP) joint in the center of the coordinate system\n # and make distance between wrist and middle finger MCP joint to be of length 1\n keypoint3d_n = keypoint3d_camera - keypoint3d_camera[9:10, :]\n keypoint3d_n = keypoint3d_n / np.sqrt(np.sum(keypoint3d_n[0, :] ** 2))\n z = keypoint3d_n[:, 2]\n\n meta = {\n 'image': image_name,\n 'keypoint2d': keypoint2d, # （NUM_KEYPOINTS x 2）\n 'keypoint3d': keypoint3d_n, # （NUM_KEYPOINTS x 3）\n 'z': z,\n }\n\n return image, target, target_weight, meta\n\n def get_samples(self, root, version='gs'):\n set = 'training'\n # load annotations of this set\n db_data_anno = load_db_annotation(root, set)\n\n version_map = {\n 'gs': sample_version.gs,\n 'hom': sample_version.hom,\n 'sample': sample_version.sample,\n 'auto': sample_version.auto\n }\n samples = []\n for idx in range(db_size(set)):\n image_name = os.path.join(set, 'rgb',\n '%08d.jpg' % sample_version.map_id(idx, version_map[version]))\n mask_name = os.path.join(set, 'mask', '%08d.jpg' % idx)\n intrinsic_matrix, mano, keypoint3d = db_data_anno[idx]\n keypoint2d = projectPoints(keypoint3d, intrinsic_matrix)\n\n sample = {\n 'name': image_name,\n 'mask_name': mask_name,\n 'keypoint2d': keypoint2d,\n 'keypoint3d': keypoint3d,\n 'intrinsic_matrix': intrinsic_matrix,\n 'left': False\n }\n samples.append(sample)\n\n return samples\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/common/vision/datasets/keypoint_detection/hand_3d_studio.py",
"content": "\"\"\"\n@author: Junguang Jiang\n@contact: JiangJunguang1123@outlook.com\n\"\"\"\nimport os\nimport json\nimport random\nfrom PIL import ImageFile, Image\nimport torch\nimport os.path as osp\n\nfrom .._util import download as download_data, check_exits\nfrom .keypoint_dataset import Hand21KeypointDataset\nfrom .util import *\n\nImageFile.LOAD_TRUNCATED_IMAGES = True\n\n\nclass Hand3DStudio(Hand21KeypointDataset):\n \"\"\"`Hand-3d-Studio Dataset `_\n\n Args:\n root (str): Root directory of dataset\n split (str, optional): The dataset split, supports ``train``, ``test``, or ``all``.\n task (str, optional): The task to create dataset. Choices include ``'noobject'``: only hands without objects, \\\n ``'object'``: only hands interacting with hands, and ``'all'``: all hands. Default: 'noobject'.\n download (bool, optional): If true, downloads the dataset from the internet and puts it \\\n in root directory. If dataset is already downloaded, it is not downloaded again.\n transforms (callable, optional): A function/transform that takes in a dict (which contains PIL image and\n its labels) and returns a transformed version. E.g, :class:`~common.vision.transforms.keypoint_detection.Resize`.\n image_size (tuple): (width, height) of the image. Default: (256, 256)\n heatmap_size (tuple): (width, height) of the heatmap. Default: (64, 64)\n sigma (int): sigma parameter when generate the heatmap. Default: 2\n\n .. note::\n We found that the original H3D image is in high resolution while most part in an image is background,\n thus we crop the image and keep only the surrounding area of hands (1.5x bigger than hands) to speed up training.\n\n .. note:: In `root`, there will exist following files after downloading.\n ::\n H3D_crop/\n annotation.json\n part1/\n part2/\n part3/\n part4/\n part5/\n \"\"\"\n def __init__(self, root, split='train', task='noobject', download=True, **kwargs):\n assert split in ['train', 'test', 'all']\n self.split = split\n assert task in ['noobject', 'object', 'all']\n self.task = task\n\n if download:\n download_data(root, \"H3D_crop\", \"H3D_crop.tar\", \"https://cloud.tsinghua.edu.cn/f/d4e612e44dc04d8eb01f/?dl=1\")\n else:\n check_exits(root, \"H3D_crop\")\n\n root = osp.join(root, \"H3D_crop\")\n # load labels\n annotation_file = os.path.join(root, 'annotation.json')\n print(\"loading from {}\".format(annotation_file))\n with open(annotation_file) as f:\n samples = list(json.load(f))\n if task == 'noobject':\n samples = [sample for sample in samples if int(sample['without_object']) == 1]\n elif task == 'object':\n samples = [sample for sample in samples if int(sample['without_object']) == 0]\n\n random.seed(42)\n random.shuffle(samples)\n samples_len = len(samples)\n samples_split = min(int(samples_len * 0.2), 3200)\n if split == 'train':\n samples = samples[samples_split:]\n elif split == 'test':\n samples = samples[:samples_split]\n\n super(Hand3DStudio, self).__init__(root, samples, **kwargs)\n\n def __getitem__(self, index):\n sample = self.samples[index]\n image_name = sample['name']\n image_path = os.path.join(self.root, image_name)\n image = Image.open(image_path)\n keypoint3d_camera = np.array(sample['keypoint3d']) # NUM_KEYPOINTS x 3\n keypoint2d = np.array(sample['keypoint2d']) # NUM_KEYPOINTS x 2\n intrinsic_matrix = np.array(sample['intrinsic_matrix'])\n Zc = keypoint3d_camera[:, 2]\n\n image, data = self.transforms(image, keypoint2d=keypoint2d, intrinsic_matrix=intrinsic_matrix)\n keypoint2d = data['keypoint2d']\n intrinsic_matrix = data['intrinsic_matrix']\n keypoint3d_camera = keypoint2d_to_3d(keypoint2d, intrinsic_matrix, Zc)\n\n # noramlize 2D pose:\n visible = np.ones((self.num_keypoints, ), dtype=np.float32)\n visible = visible[:, np.newaxis]\n # 2D heatmap\n target, target_weight = generate_target(keypoint2d, visible, self.heatmap_size, self.sigma, self.image_size)\n target = torch.from_numpy(target)\n target_weight = torch.from_numpy(target_weight)\n\n # normalize 3D pose:\n # put middle finger metacarpophalangeal (MCP) joint in the center of the coordinate system\n # and make distance between wrist and middle finger MCP joint to be of length 1\n keypoint3d_n = keypoint3d_camera - keypoint3d_camera[9:10, :]\n keypoint3d_n = keypoint3d_n / np.sqrt(np.sum(keypoint3d_n[0, :] ** 2))\n\n meta = {\n 'image': image_name,\n 'keypoint2d': keypoint2d, # （NUM_KEYPOINTS x 2）\n 'keypoint3d': keypoint3d_n, # （NUM_KEYPOINTS x 3）\n }\n return image, target, target_weight, meta\n\n\nclass Hand3DStudioAll(Hand3DStudio):\n \"\"\"\n `Hand-3d-Studio Dataset `_\n\n \"\"\"\n def __init__(self, root, task='all', **kwargs):\n super(Hand3DStudioAll, self).__init__(root, task=task, **kwargs)"
},
{
"path": "domain_generalization/Transfer-Learning-Library/common/vision/datasets/keypoint_detection/human36m.py",
"content": "\"\"\"\n@author: Junguang Jiang\n@contact: JiangJunguang1123@outlook.com\n\"\"\"\nimport os\nimport json\nimport tqdm\nfrom PIL import ImageFile\nimport torch\nfrom .keypoint_dataset import Body16KeypointDataset\nfrom ...transforms.keypoint_detection import *\nfrom .util import *\n\nImageFile.LOAD_TRUNCATED_IMAGES = True\n\n\nclass Human36M(Body16KeypointDataset):\n \"\"\"`Human3.6M Dataset `_\n\n Args:\n root (str): Root directory of dataset\n split (str, optional): The dataset split, supports ``train``, ``test``, or ``all``.\n Default: ``train``.\n task (str, optional): Placeholder.\n download (bool, optional): Placeholder.\n transforms (callable, optional): A function/transform that takes in a dict (which contains PIL image and\n its labels) and returns a transformed version. E.g, :class:`~common.vision.transforms.keypoint_detection.Resize`.\n image_size (tuple): (width, height) of the image. Default: (256, 256)\n heatmap_size (tuple): (width, height) of the heatmap. Default: (64, 64)\n sigma (int): sigma parameter when generate the heatmap. Default: 2\n\n .. note:: You need to download Human36M manually.\n Ensure that there exist following files in the `root` directory before you using this class.\n ::\n annotations/\n Human36M_subject11_joint_3d.json\n ...\n images/\n\n .. note::\n We found that the original Human3.6M image is in high resolution while most part in an image is background,\n thus we crop the image and keep only the surrounding area of hands (1.5x bigger than hands) to speed up training.\n In `root`, there will exist following files after crop.\n ::\n Human36M_crop/\n annotations/\n keypoints2d_11.json\n ...\n \"\"\"\n def __init__(self, root, split='train', task='all', download=True, **kwargs):\n assert split in ['train', 'test', 'all']\n self.split = split\n\n samples = []\n if self.split == 'train':\n parts = [1, 5, 6, 7, 8]\n elif self.split == 'test':\n parts = [9, 11]\n else:\n parts = [1, 5, 6, 7, 8, 9, 11]\n\n for part in parts:\n annotation_file = os.path.join(root, 'annotations/keypoints2d_{}.json'.format(part))\n if not os.path.exists(annotation_file):\n self.preprocess(part, root)\n print(\"loading\", annotation_file)\n with open(annotation_file) as f:\n samples.extend(json.load(f))\n # decrease the number of test samples to decrease the time spent on test\n random.seed(42)\n if self.split == 'test':\n samples = random.choices(samples, k=3200)\n super(Human36M, self).__init__(root, samples, **kwargs)\n\n def __getitem__(self, index):\n sample = self.samples[index]\n image_name = sample['name']\n image_path = os.path.join(self.root, \"crop_images\", image_name)\n image = Image.open(image_path)\n keypoint3d_camera = np.array(sample['keypoint3d']) # NUM_KEYPOINTS x 3\n keypoint2d = np.array(sample['keypoint2d']) # NUM_KEYPOINTS x 2\n intrinsic_matrix = np.array(sample['intrinsic_matrix'])\n Zc = keypoint3d_camera[:, 2]\n\n image, data = self.transforms(image, keypoint2d=keypoint2d, intrinsic_matrix=intrinsic_matrix)\n keypoint2d = data['keypoint2d']\n intrinsic_matrix = data['intrinsic_matrix']\n keypoint3d_camera = keypoint2d_to_3d(keypoint2d, intrinsic_matrix, Zc)\n\n # noramlize 2D pose:\n visible = np.ones((self.num_keypoints, ), dtype=np.float32)\n visible = visible[:, np.newaxis]\n # 2D heatmap\n target, target_weight = generate_target(keypoint2d, visible, self.heatmap_size, self.sigma, self.image_size)\n target = torch.from_numpy(target)\n target_weight = torch.from_numpy(target_weight)\n\n # normalize 3D pose:\n # put middle finger metacarpophalangeal (MCP) joint in the center of the coordinate system\n # and make distance between wrist and middle finger MCP joint to be of length 1\n keypoint3d_n = keypoint3d_camera - keypoint3d_camera[9:10, :]\n keypoint3d_n = keypoint3d_n / np.sqrt(np.sum(keypoint3d_n[0, :] ** 2))\n\n meta = {\n 'image': image_name,\n 'keypoint2d': keypoint2d, # （NUM_KEYPOINTS x 2）\n 'keypoint3d': keypoint3d_n, # （NUM_KEYPOINTS x 3）\n }\n return image, target, target_weight, meta\n\n def preprocess(self, part, root):\n body_index = [3, 2, 1, 4, 5, 6, 0, 11, 8, 10, 16, 15, 14, 11, 12, 13]\n image_size = 512\n print(\"preprocessing part\", part)\n camera_json = os.path.join(root, \"annotations\", \"Human36M_subject{}_camera.json\".format(part))\n data_json = os.path.join(root, \"annotations\", \"Human36M_subject{}_data.json\".format(part))\n joint_3d_json = os.path.join(root, \"annotations\", \"Human36M_subject{}_joint_3d.json\".format(part))\n with open(camera_json, \"r\") as f:\n cameras = json.load(f)\n with open(data_json, \"r\") as f:\n data = json.load(f)\n images = data['images']\n\n with open(joint_3d_json, \"r\") as f:\n joints_3d = json.load(f)\n\n data = []\n\n for i, image_data in enumerate(tqdm.tqdm(images)):\n # downsample\n if i % 5 == 0:\n keypoint3d = np.array(joints_3d[str(image_data[\"action_idx\"])][str(image_data[\"subaction_idx\"])][\n str(image_data[\"frame_idx\"])])\n keypoint3d = keypoint3d[body_index, :]\n keypoint3d[7, :] = 0.5 * (keypoint3d[12, :] + keypoint3d[13, :])\n camera = cameras[str(image_data[\"cam_idx\"])]\n R, T = np.array(camera[\"R\"]), np.array(camera['t'])[:, np.newaxis]\n extrinsic_matrix = np.concatenate([R, T], axis=1)\n keypoint3d_camera = np.matmul(extrinsic_matrix, np.hstack(\n (keypoint3d, np.ones((keypoint3d.shape[0], 1)))).T) # (3 x NUM_KEYPOINTS)\n Z_c = keypoint3d_camera[2:3, :] # 1 x NUM_KEYPOINTS\n\n f, c = np.array(camera[\"f\"]), np.array(camera['c'])\n intrinsic_matrix = np.zeros((3, 3))\n intrinsic_matrix[0, 0] = f[0]\n intrinsic_matrix[1, 1] = f[1]\n intrinsic_matrix[0, 2] = c[0]\n intrinsic_matrix[1, 2] = c[1]\n intrinsic_matrix[2, 2] = 1\n keypoint2d = np.matmul(intrinsic_matrix, keypoint3d_camera) # (3 x NUM_KEYPOINTS)\n keypoint2d = keypoint2d[0: 2, :] / Z_c\n keypoint2d = keypoint2d.T\n src_image_path = os.path.join(root, \"images\", image_data['file_name'])\n tgt_image_path = os.path.join(root, \"crop_images\", image_data['file_name'])\n os.makedirs(os.path.dirname(tgt_image_path), exist_ok=True)\n image = Image.open(src_image_path)\n\n bounding_box = get_bounding_box(keypoint2d)\n w, h = image.size\n left, upper, right, lower = scale_box(bounding_box, w, h, 1.5)\n image, keypoint2d = crop(image, upper, left, lower-upper+1, right-left+1, keypoint2d)\n Z_c = Z_c.T\n\n # Calculate XYZ from uvz\n uv1 = np.concatenate([np.copy(keypoint2d), np.ones((16, 1))],\n axis=1) # NUM_KEYPOINTS x 3\n uv1 = uv1 * Z_c # NUM_KEYPOINTS x 3\n keypoint3d_camera = np.matmul(np.linalg.inv(intrinsic_matrix), uv1.T).T\n\n # resize image will change camera intrinsic matrix\n w, h = image.size\n image = image.resize((image_size, image_size))\n image.save(tgt_image_path)\n\n zoom_factor = float(w) / float(image_size)\n keypoint2d /= zoom_factor\n intrinsic_matrix[0, 0] /= zoom_factor\n intrinsic_matrix[1, 1] /= zoom_factor\n intrinsic_matrix[0, 2] /= zoom_factor\n intrinsic_matrix[1, 2] /= zoom_factor\n\n data.append({\n \"name\": image_data['file_name'],\n 'keypoint2d': keypoint2d.tolist(),\n 'keypoint3d': keypoint3d_camera.tolist(),\n 'intrinsic_matrix': intrinsic_matrix.tolist(),\n })\n\n with open(os.path.join(root, \"annotations\", \"keypoints2d_{}.json\".format(part)), \"w\") as f:\n json.dump(data, f)"
},
{
"path": "domain_generalization/Transfer-Learning-Library/common/vision/datasets/keypoint_detection/keypoint_dataset.py",
"content": "\"\"\"\n@author: Junguang Jiang\n@contact: JiangJunguang1123@outlook.com\n\"\"\"\nfrom abc import ABC\nimport numpy as np\nfrom torch.utils.data.dataset import Dataset\nfrom webcolors import name_to_rgb\nimport cv2\n\n\nclass KeypointDataset(Dataset, ABC):\n \"\"\"A generic dataset class for image keypoint detection\n\n Args:\n root (str): Root directory of dataset\n num_keypoints (int): Number of keypoints\n samples (list): list of data\n transforms (callable, optional): A function/transform that takes in a dict (which contains PIL image and\n its labels) and returns a transformed version. E.g, :class:`~common.vision.transforms.keypoint_detection.Resize`.\n image_size (tuple): (width, height) of the image. Default: (256, 256)\n heatmap_size (tuple): (width, height) of the heatmap. Default: (64, 64)\n sigma (int): sigma parameter when generate the heatmap. Default: 2\n keypoints_group (dict): a dict that stores the index of different types of keypoints\n colored_skeleton (dict): a dict that stores the index and color of different skeleton\n \"\"\"\n def __init__(self, root, num_keypoints, samples, transforms=None, image_size=(256, 256), heatmap_size=(64, 64),\n sigma=2, keypoints_group=None, colored_skeleton=None):\n self.root = root\n self.num_keypoints = num_keypoints\n self.samples = samples\n self.transforms = transforms\n self.image_size = image_size\n self.heatmap_size = heatmap_size\n self.sigma = sigma\n self.keypoints_group = keypoints_group\n self.colored_skeleton = colored_skeleton\n\n def __len__(self):\n return len(self.samples)\n\n def visualize(self, image, keypoints, filename):\n \"\"\"Visualize an image with its keypoints, and store the result into a file\n\n Args:\n image (PIL.Image):\n keypoints (torch.Tensor): keypoints in shape K x 2\n filename (str): the name of file to store\n \"\"\"\n assert self.colored_skeleton is not None\n\n image = cv2.cvtColor(np.asarray(image), cv2.COLOR_RGB2BGR).copy()\n for (_, (line, color)) in self.colored_skeleton.items():\n for i in range(len(line) - 1):\n start, end = keypoints[line[i]], keypoints[line[i + 1]]\n cv2.line(image, (int(start[0]), int(start[1])), (int(end[0]), int(end[1])), color=name_to_rgb(color),\n thickness=3)\n for keypoint in keypoints:\n cv2.circle(image, (int(keypoint[0]), int(keypoint[1])), 3, name_to_rgb('black'), 1)\n cv2.imwrite(filename, image)\n\n def group_accuracy(self, accuracies):\n \"\"\" Group the accuracy of K keypoints into different kinds.\n\n Args:\n accuracies (list): accuracy of the K keypoints\n\n Returns:\n accuracy of ``N=len(keypoints_group)`` kinds of keypoints\n\n \"\"\"\n grouped_accuracies = dict()\n for name, keypoints in self.keypoints_group.items():\n grouped_accuracies[name] = sum([accuracies[idx] for idx in keypoints]) / len(keypoints)\n return grouped_accuracies\n\n\nclass Body16KeypointDataset(KeypointDataset, ABC):\n \"\"\"\n Dataset with 16 body keypoints.\n \"\"\"\n # TODO: add image\n head = (9,)\n shoulder = (12, 13)\n elbow = (11, 14)\n wrist = (10, 15)\n hip = (2, 3)\n knee = (1, 4)\n ankle = (0, 5)\n all = (12, 13, 11, 14, 10, 15, 2, 3, 1, 4, 0, 5)\n right_leg = (0, 1, 2, 8)\n left_leg = (5, 4, 3, 8)\n backbone = (8, 9)\n right_arm = (10, 11, 12, 8)\n left_arm = (15, 14, 13, 8)\n\n def __init__(self, root, samples, **kwargs):\n colored_skeleton = {\n \"right_leg\": (self.right_leg, 'yellow'),\n \"left_leg\": (self.left_leg, 'green'),\n \"backbone\": (self.backbone, 'blue'),\n \"right_arm\": (self.right_arm, 'purple'),\n \"left_arm\": (self.left_arm, 'red'),\n }\n keypoints_group = {\n \"head\": self.head,\n \"shoulder\": self.shoulder,\n \"elbow\": self.elbow,\n \"wrist\": self.wrist,\n \"hip\": self.hip,\n \"knee\": self.knee,\n \"ankle\": self.ankle,\n \"all\": self.all\n }\n super(Body16KeypointDataset, self).__init__(root, 16, samples, keypoints_group=keypoints_group,\n colored_skeleton=colored_skeleton, **kwargs)\n\n\nclass Hand21KeypointDataset(KeypointDataset, ABC):\n \"\"\"\n Dataset with 21 hand keypoints.\n \"\"\"\n # TODO: add image\n MCP = (1, 5, 9, 13, 17)\n PIP = (2, 6, 10, 14, 18)\n DIP = (3, 7, 11, 15, 19)\n fingertip = (4, 8, 12, 16, 20)\n all = tuple(range(21))\n thumb = (0, 1, 2, 3, 4)\n index_finger = (0, 5, 6, 7, 8)\n middle_finger = (0, 9, 10, 11, 12)\n ring_finger = (0, 13, 14, 15, 16)\n little_finger = (0, 17, 18, 19, 20)\n\n def __init__(self, root, samples, **kwargs):\n colored_skeleton = {\n \"thumb\": (self.thumb, 'yellow'),\n \"index_finger\": (self.index_finger, 'green'),\n \"middle_finger\": (self.middle_finger, 'blue'),\n \"ring_finger\": (self.ring_finger, 'purple'),\n \"little_finger\": (self.little_finger, 'red'),\n }\n keypoints_group = {\n \"MCP\": self.MCP,\n \"PIP\": self.PIP,\n \"DIP\": self.DIP,\n \"fingertip\": self.fingertip,\n \"all\": self.all\n }\n super(Hand21KeypointDataset, self).__init__(root, 21, samples, keypoints_group=keypoints_group,\n colored_skeleton=colored_skeleton, **kwargs)\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/common/vision/datasets/keypoint_detection/lsp.py",
"content": "\"\"\"\n@author: Junguang Jiang\n@contact: JiangJunguang1123@outlook.com\n\"\"\"\nimport scipy.io as scio\nimport os\n\nfrom PIL import ImageFile\nimport torch\nfrom .keypoint_dataset import Body16KeypointDataset\nfrom ...transforms.keypoint_detection import *\nfrom .util import *\nfrom .._util import download as download_data, check_exits\n\n\nImageFile.LOAD_TRUNCATED_IMAGES = True\n\n\nclass LSP(Body16KeypointDataset):\n \"\"\"`Leeds Sports Pose Dataset `_\n\n Args:\n root (str): Root directory of dataset\n split (str, optional): PlaceHolder.\n task (str, optional): Placeholder.\n download (bool, optional): If true, downloads the dataset from the internet and puts it \\\n in root directory. If dataset is already downloaded, it is not downloaded again.\n transforms (callable, optional): PlaceHolder.\n heatmap_size (tuple): (width, height) of the heatmap. Default: (64, 64)\n sigma (int): sigma parameter when generate the heatmap. Default: 2\n\n .. note:: In `root`, there will exist following files after downloading.\n ::\n lsp/\n images/\n joints.mat\n\n .. note::\n LSP is only used for target domain. Due to the small dataset size, the whole dataset is used\n no matter what ``split`` is. Also, the transform is fixed.\n \"\"\"\n def __init__(self, root, split='train', task='all', download=True, image_size=(256, 256), transforms=None, **kwargs):\n if download:\n download_data(root, \"images\", \"lsp_dataset.zip\",\n \"https://cloud.tsinghua.edu.cn/f/46ea73c89abc46bfb125/?dl=1\")\n else:\n check_exits(root, \"lsp\")\n\n assert split in ['train', 'test', 'all']\n self.split = split\n\n samples = []\n annotations = scio.loadmat(os.path.join(root, \"joints.mat\"))['joints'].transpose((2, 1, 0))\n for i in range(0, 2000):\n image = \"im{0:04d}.jpg\".format(i+1)\n annotation = annotations[i]\n samples.append((image, annotation))\n\n self.joints_index = (0, 1, 2, 3, 4, 5, 13, 13, 12, 13, 6, 7, 8, 9, 10, 11)\n self.visible = np.array([1.] * 6 + [0, 0] + [1.] * 8, dtype=np.float32)\n normalize = Normalize([0.485, 0.456, 0.406], [0.229, 0.224, 0.225])\n transforms = Compose([\n ResizePad(image_size[0]),\n ToTensor(),\n normalize\n ])\n super(LSP, self).__init__(root, samples, transforms=transforms, image_size=image_size, **kwargs)\n\n def __getitem__(self, index):\n sample = self.samples[index]\n image_name = sample[0]\n image = Image.open(os.path.join(self.root, \"images\", image_name))\n keypoint2d = sample[1][self.joints_index, :2]\n image, data = self.transforms(image, keypoint2d=keypoint2d)\n keypoint2d = data['keypoint2d']\n visible = self.visible * (1-sample[1][self.joints_index, 2])\n visible = visible[:, np.newaxis]\n\n # 2D heatmap\n target, target_weight = generate_target(keypoint2d, visible, self.heatmap_size, self.sigma, self.image_size)\n target = torch.from_numpy(target)\n target_weight = torch.from_numpy(target_weight)\n\n meta = {\n 'image': image_name,\n 'keypoint2d': keypoint2d, # （NUM_KEYPOINTS x 2）\n 'keypoint3d': np.zeros((self.num_keypoints, 3)).astype(keypoint2d.dtype), # （NUM_KEYPOINTS x 3）\n }\n return image, target, target_weight, meta\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/common/vision/datasets/keypoint_detection/rendered_hand_pose.py",
"content": "\"\"\"\n@author: Junguang Jiang\n@contact: JiangJunguang1123@outlook.com\n\"\"\"\nimport torch\nimport os\nimport pickle\n\nfrom .._util import download as download_data, check_exits\nfrom ...transforms.keypoint_detection import *\nfrom .keypoint_dataset import Hand21KeypointDataset\nfrom .util import *\n\n\nclass RenderedHandPose(Hand21KeypointDataset):\n \"\"\"`Rendered Handpose Dataset `_\n\n Args:\n root (str): Root directory of dataset\n split (str, optional): The dataset split, supports ``train``, ``test``, or ``all``.\n task (str, optional): Placeholder.\n download (bool, optional): If true, downloads the dataset from the internet and puts it \\\n in root directory. If dataset is already downloaded, it is not downloaded again.\n transforms (callable, optional): A function/transform that takes in a dict (which contains PIL image and\n its labels) and returns a transformed version. E.g, :class:`~common.vision.transforms.keypoint_detection.Resize`.\n image_size (tuple): (width, height) of the image. Default: (256, 256)\n heatmap_size (tuple): (width, height) of the heatmap. Default: (64, 64)\n sigma (int): sigma parameter when generate the heatmap. Default: 2\n\n .. note:: In `root`, there will exist following files after downloading.\n ::\n RHD_published_v2/\n training/\n evaluation/\n \"\"\"\n def __init__(self, root, split='train', task='all', download=True, **kwargs):\n if download:\n download_data(root, \"RHD_published_v2\", \"RHD_v1-1.zip\", \"https://lmb.informatik.uni-freiburg.de/data/RenderedHandpose/RHD_v1-1.zip\")\n else:\n check_exits(root, \"RHD_published_v2\")\n\n root = os.path.join(root, \"RHD_published_v2\")\n\n assert split in ['train', 'test', 'all']\n self.split = split\n if split == 'all':\n samples = self.get_samples(root, 'train') + self.get_samples(root, 'test')\n else:\n samples = self.get_samples(root, split)\n\n super(RenderedHandPose, self).__init__(\n root, samples, **kwargs)\n\n def __getitem__(self, index):\n sample = self.samples[index]\n image_name = sample['name']\n image_path = os.path.join(self.root, image_name)\n image = Image.open(image_path)\n\n keypoint3d_camera = np.array(sample['keypoint3d']) # NUM_KEYPOINTS x 3\n keypoint2d = np.array(sample['keypoint2d']) # NUM_KEYPOINTS x 2\n intrinsic_matrix = np.array(sample['intrinsic_matrix'])\n Zc = keypoint3d_camera[:, 2]\n\n # Crop the images such that the hand is at the center of the image\n # The images will be 1.5 times larger than the hand\n # The crop process will change Xc and Yc, leaving Zc with no changes\n bounding_box = get_bounding_box(keypoint2d)\n w, h = image.size\n left, upper, right, lower = scale_box(bounding_box, w, h, 1.5)\n image, keypoint2d = crop(image, upper, left, lower - upper, right - left, keypoint2d)\n\n # Change all hands to right hands\n if sample['left'] is False:\n image, keypoint2d = hflip(image, keypoint2d)\n\n image, data = self.transforms(image, keypoint2d=keypoint2d, intrinsic_matrix=intrinsic_matrix)\n keypoint2d = data['keypoint2d']\n intrinsic_matrix = data['intrinsic_matrix']\n keypoint3d_camera = keypoint2d_to_3d(keypoint2d, intrinsic_matrix, Zc)\n\n # noramlize 2D pose:\n visible = np.array(sample['visible'], dtype=np.float32)\n visible = visible[:, np.newaxis]\n # 2D heatmap\n target, target_weight = generate_target(keypoint2d, visible, self.heatmap_size, self.sigma, self.image_size)\n target = torch.from_numpy(target)\n target_weight = torch.from_numpy(target_weight)\n\n # normalize 3D pose:\n # put middle finger metacarpophalangeal (MCP) joint in the center of the coordinate system\n # and make distance between wrist and middle finger MCP joint to be of length 1\n keypoint3d_n = keypoint3d_camera - keypoint3d_camera[9:10, :]\n keypoint3d_n = keypoint3d_n / np.sqrt(np.sum(keypoint3d_n[0, :] ** 2))\n z = keypoint3d_n[:, 2]\n\n meta = {\n 'image': image_name,\n 'keypoint2d': keypoint2d, # （NUM_KEYPOINTS x 2）\n 'keypoint3d': keypoint3d_n, # （NUM_KEYPOINTS x 3）\n 'z': z,\n }\n\n return image, target, target_weight, meta\n\n def get_samples(self, root, task, min_size=64):\n if task == 'train':\n set = 'training'\n else:\n set = 'evaluation'\n # load annotations of this set\n with open(os.path.join(root, set, 'anno_%s.pickle' % set), 'rb') as fi:\n anno_all = pickle.load(fi)\n\n samples = []\n left_hand_index = [0, 4, 3, 2, 1, 8, 7, 6, 5, 12, 11, 10, 9, 16, 15, 14, 13, 20, 19, 18, 17]\n right_hand_index = [i+21 for i in left_hand_index]\n for sample_id, anno in anno_all.items():\n image_name = os.path.join(set, 'color', '%.5d.png' % sample_id)\n mask_name = os.path.join(set, 'mask', '%.5d.png' % sample_id)\n keypoint2d = anno['uv_vis'][:, :2]\n keypoint3d = anno['xyz']\n intrinsic_matrix = anno['K']\n visible = anno['uv_vis'][:, 2]\n\n left_hand_keypoint2d = keypoint2d[left_hand_index] # NUM_KEYPOINTS x 2\n left_box = get_bounding_box(left_hand_keypoint2d)\n right_hand_keypoint2d = keypoint2d[right_hand_index] # NUM_KEYPOINTS x 2\n right_box = get_bounding_box(right_hand_keypoint2d)\n\n w, h = 320, 320\n scaled_left_box = scale_box(left_box, w, h, 1.5)\n left, upper, right, lower = scaled_left_box\n size = max(right - left, lower - upper)\n if size > min_size and np.sum(visible[left_hand_index]) > 16 and area(*intersection(scaled_left_box, right_box)) / area(*scaled_left_box) < 0.3:\n sample = {\n 'name': image_name,\n 'mask_name': mask_name,\n 'keypoint2d': left_hand_keypoint2d,\n 'visible': visible[left_hand_index],\n 'keypoint3d': keypoint3d[left_hand_index],\n 'intrinsic_matrix': intrinsic_matrix,\n 'left': True\n }\n samples.append(sample)\n\n scaled_right_box = scale_box(right_box, w, h, 1.5)\n left, upper, right, lower = scaled_right_box\n size = max(right - left, lower - upper)\n if size > min_size and np.sum(visible[right_hand_index]) > 16 and area(*intersection(scaled_right_box, left_box)) / area(*scaled_right_box) < 0.3:\n sample = {\n 'name': image_name,\n 'mask_name': mask_name,\n 'keypoint2d': right_hand_keypoint2d,\n 'visible': visible[right_hand_index],\n 'keypoint3d': keypoint3d[right_hand_index],\n 'intrinsic_matrix': intrinsic_matrix,\n 'left': False\n }\n samples.append(sample)\n\n return samples"
},
{
"path": "domain_generalization/Transfer-Learning-Library/common/vision/datasets/keypoint_detection/surreal.py",
"content": "\"\"\"\n@author: Junguang Jiang\n@contact: JiangJunguang1123@outlook.com\n\"\"\"\nimport os\nimport json\nfrom PIL import ImageFile\nimport torch\nfrom ...transforms.keypoint_detection import *\nfrom .util import *\nfrom .._util import download as download_data, check_exits\nfrom .keypoint_dataset import Body16KeypointDataset\n\nImageFile.LOAD_TRUNCATED_IMAGES = True\n\n\nclass SURREAL(Body16KeypointDataset):\n \"\"\"`Surreal Dataset `_\n\n Args:\n root (str): Root directory of dataset\n split (str, optional): The dataset split, supports ``train``, ``test``, or ``all``.\n Default: ``train``.\n task (str, optional): Placeholder.\n download (bool, optional): If true, downloads the dataset from the internet and puts it \\\n in root directory. If dataset is already downloaded, it is not downloaded again.\n transforms (callable, optional): A function/transform that takes in a dict (which contains PIL image and\n its labels) and returns a transformed version. E.g, :class:`~common.vision.transforms.keypoint_detection.Resize`.\n image_size (tuple): (width, height) of the image. Default: (256, 256)\n heatmap_size (tuple): (width, height) of the heatmap. Default: (64, 64)\n sigma (int): sigma parameter when generate the heatmap. Default: 2\n\n .. note::\n We found that the original Surreal image is in high resolution while most part in an image is background,\n thus we crop the image and keep only the surrounding area of hands (1.5x bigger than hands) to speed up training.\n\n .. note:: In `root`, there will exist following files after downloading.\n ::\n train/\n test/\n val/\n \"\"\"\n def __init__(self, root, split='train', task='all', download=True, **kwargs):\n assert split in ['train', 'test', 'val']\n self.split = split\n\n if download:\n download_data(root, \"train/run0\", \"train0.tgz\", \"https://cloud.tsinghua.edu.cn/f/b13604f06ff1445c830a/?dl=1\")\n download_data(root, \"train/run1\", \"train1.tgz\", \"https://cloud.tsinghua.edu.cn/f/919aefe2de3541c3b940/?dl=1\")\n download_data(root, \"train/run1\", \"train2.tgz\", \"https://cloud.tsinghua.edu.cn/f/34864760ad4945b9bcd6/?dl=1\")\n download_data(root, \"val\", \"val.tgz\", \"https://cloud.tsinghua.edu.cn/f/16b20f2e76684f848dc1/?dl=1\")\n download_data(root, \"test\", \"test.tgz\", \"https://cloud.tsinghua.edu.cn/f/36c72d86e43540e0a913/?dl=1\")\n else:\n check_exits(root, \"train/run0\")\n check_exits(root, \"train/run1\")\n check_exits(root, \"train/run2\")\n check_exits(root, \"val\")\n check_exits(root, \"test\")\n\n all_samples = []\n for part in [0, 1, 2]:\n annotation_file = os.path.join(root, split, 'run{}.json'.format(part))\n print(\"loading\", annotation_file)\n with open(annotation_file) as f:\n samples = json.load(f)\n for sample in samples:\n sample[\"image_path\"] = os.path.join(root, self.split, 'run{}'.format(part), sample['name'])\n all_samples.extend(samples)\n\n random.seed(42)\n random.shuffle(all_samples)\n samples_len = len(all_samples)\n samples_split = min(int(samples_len * 0.2), 3200)\n if self.split == 'train':\n all_samples = all_samples[samples_split:]\n elif self.split == 'test':\n all_samples = all_samples[:samples_split]\n self.joints_index = (7, 4, 1, 2, 5, 8, 0, 9, 12, 15, 20, 18, 13, 14, 19, 21)\n\n super(SURREAL, self).__init__(root, all_samples, **kwargs)\n\n def __getitem__(self, index):\n sample = self.samples[index]\n image_name = sample['name']\n\n image_path = sample['image_path']\n image = Image.open(image_path)\n keypoint3d_camera = np.array(sample['keypoint3d'])[self.joints_index, :] # NUM_KEYPOINTS x 3\n keypoint2d = np.array(sample['keypoint2d'])[self.joints_index, :] # NUM_KEYPOINTS x 2\n intrinsic_matrix = np.array(sample['intrinsic_matrix'])\n Zc = keypoint3d_camera[:, 2]\n\n image, data = self.transforms(image, keypoint2d=keypoint2d, intrinsic_matrix=intrinsic_matrix)\n keypoint2d = data['keypoint2d']\n intrinsic_matrix = data['intrinsic_matrix']\n keypoint3d_camera = keypoint2d_to_3d(keypoint2d, intrinsic_matrix, Zc)\n\n # noramlize 2D pose:\n visible = np.array([1.] * 16, dtype=np.float32)\n visible = visible[:, np.newaxis]\n\n # 2D heatmap\n target, target_weight = generate_target(keypoint2d, visible, self.heatmap_size, self.sigma, self.image_size)\n target = torch.from_numpy(target)\n target_weight = torch.from_numpy(target_weight)\n\n # normalize 3D pose:\n # put middle finger metacarpophalangeal (MCP) joint in the center of the coordinate system\n # and make distance between wrist and middle finger MCP joint to be of length 1\n keypoint3d_n = keypoint3d_camera - keypoint3d_camera[9:10, :]\n keypoint3d_n = keypoint3d_n / np.sqrt(np.sum(keypoint3d_n[0, :] ** 2))\n\n meta = {\n 'image': image_name,\n 'keypoint2d': keypoint2d, # （NUM_KEYPOINTS x 2）\n 'keypoint3d': keypoint3d_n, # （NUM_KEYPOINTS x 3）\n }\n return image, target, target_weight, meta\n\n def __len__(self):\n return len(self.samples)\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/common/vision/datasets/keypoint_detection/util.py",
"content": "\"\"\"\n@author: Junguang Jiang\n@contact: JiangJunguang1123@outlook.com\n\"\"\"\nimport numpy as np\nimport cv2\n\n\ndef generate_target(joints, joints_vis, heatmap_size, sigma, image_size):\n \"\"\"Generate heatamap for joints.\n\n Args:\n joints: (K, 2)\n joints_vis: (K, 1)\n heatmap_size: W, H\n sigma:\n image_size:\n\n Returns:\n\n \"\"\"\n num_joints = joints.shape[0]\n target_weight = np.ones((num_joints, 1), dtype=np.float32)\n target_weight[:, 0] = joints_vis[:, 0]\n\n target = np.zeros((num_joints,\n heatmap_size[1],\n heatmap_size[0]),\n dtype=np.float32)\n\n tmp_size = sigma * 3\n image_size = np.array(image_size)\n heatmap_size = np.array(heatmap_size)\n\n for joint_id in range(num_joints):\n feat_stride = image_size / heatmap_size\n mu_x = int(joints[joint_id][0] / feat_stride[0] + 0.5)\n mu_y = int(joints[joint_id][1] / feat_stride[1] + 0.5)\n # Check that any part of the gaussian is in-bounds\n ul = [int(mu_x - tmp_size), int(mu_y - tmp_size)]\n br = [int(mu_x + tmp_size + 1), int(mu_y + tmp_size + 1)]\n if mu_x >= heatmap_size[0] or mu_y >= heatmap_size[1] \\\n or mu_x < 0 or mu_y < 0:\n # If not, just return the image as is\n target_weight[joint_id] = 0\n continue\n\n # Generate gaussian\n size = 2 * tmp_size + 1\n x = np.arange(0, size, 1, np.float32)\n y = x[:, np.newaxis]\n x0 = y0 = size // 2\n # The gaussian is not normalized, we want the center value to equal 1\n g = np.exp(- ((x - x0) ** 2 + (y - y0) ** 2) / (2 * sigma ** 2))\n\n # Usable gaussian range\n g_x = max(0, -ul[0]), min(br[0], heatmap_size[0]) - ul[0]\n g_y = max(0, -ul[1]), min(br[1], heatmap_size[1]) - ul[1]\n # Image range\n img_x = max(0, ul[0]), min(br[0], heatmap_size[0])\n img_y = max(0, ul[1]), min(br[1], heatmap_size[1])\n\n v = target_weight[joint_id]\n if v > 0.5:\n target[joint_id][img_y[0]:img_y[1], img_x[0]:img_x[1]] = \\\n g[g_y[0]:g_y[1], g_x[0]:g_x[1]]\n\n return target, target_weight\n\n\ndef keypoint2d_to_3d(keypoint2d: np.ndarray, intrinsic_matrix: np.ndarray, Zc: np.ndarray):\n \"\"\"Convert 2D keypoints to 3D keypoints\"\"\"\n uv1 = np.concatenate([np.copy(keypoint2d), np.ones((keypoint2d.shape[0], 1))], axis=1).T * Zc # 3 x NUM_KEYPOINTS\n xyz = np.matmul(np.linalg.inv(intrinsic_matrix), uv1).T # NUM_KEYPOINTS x 3\n return xyz\n\n\ndef keypoint3d_to_2d(keypoint3d: np.ndarray, intrinsic_matrix: np.ndarray):\n \"\"\"Convert 3D keypoints to 2D keypoints\"\"\"\n keypoint2d = np.matmul(intrinsic_matrix, keypoint3d.T).T # NUM_KEYPOINTS x 3\n keypoint2d = keypoint2d[:, :2] / keypoint2d[:, 2:3] # NUM_KEYPOINTS x 2\n return keypoint2d\n\n\ndef scale_box(box, image_width, image_height, scale):\n \"\"\"\n Change `box` to a square box.\n The side with of the square box will be `scale` * max(w, h)\n where w and h is the width and height of the origin box\n \"\"\"\n left, upper, right, lower = box\n center_x, center_y = (left + right) / 2, (upper + lower) / 2\n w, h = right - left, lower - upper\n side_with = min(round(scale * max(w, h)), min(image_width, image_height))\n left = round(center_x - side_with / 2)\n right = left + side_with - 1\n upper = round(center_y - side_with / 2)\n lower = upper + side_with - 1\n if left < 0:\n left = 0\n right = side_with - 1\n if right >= image_width:\n right = image_width - 1\n left = image_width - side_with\n if upper < 0:\n upper = 0\n lower = side_with -1\n if lower >= image_height:\n lower = image_height - 1\n upper = image_height - side_with\n return left, upper, right, lower\n\n\ndef get_bounding_box(keypoint2d: np.array):\n \"\"\"Get the bounding box for keypoints\"\"\"\n left = np.min(keypoint2d[:, 0])\n right = np.max(keypoint2d[:, 0])\n upper = np.min(keypoint2d[:, 1])\n lower = np.max(keypoint2d[:, 1])\n return left, upper, right, lower\n\n\ndef visualize_heatmap(image, heatmaps, filename):\n image = cv2.cvtColor(np.asarray(image), cv2.COLOR_RGB2BGR).copy()\n H, W = heatmaps.shape[1], heatmaps.shape[2]\n resized_image = cv2.resize(image, (int(W), int(H)))\n heatmaps = heatmaps.mul(255).clamp(0, 255).byte().cpu().numpy()\n for k in range(heatmaps.shape[0]):\n heatmap = heatmaps[k]\n colored_heatmap = cv2.applyColorMap(heatmap, cv2.COLORMAP_JET)\n masked_image = colored_heatmap * 0.7 + resized_image * 0.3\n cv2.imwrite(filename.format(k), masked_image)\n \n\ndef area(left, upper, right, lower):\n return max(right - left + 1, 0) * max(lower - upper + 1, 0)\n\n\ndef intersection(box_a, box_b):\n left_a, upper_a, right_a, lower_a = box_a\n left_b, upper_b, right_b, lower_b = box_b\n return max(left_a, left_b), max(upper_a, upper_b), min(right_a, right_b), min(lower_a, lower_b)\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/common/vision/datasets/office31.py",
"content": "\"\"\"\n@author: Junguang Jiang\n@contact: JiangJunguang1123@outlook.com\n\"\"\"\nfrom typing import Optional\nimport os\nfrom .imagelist import ImageList\nfrom ._util import download as download_data, check_exits\n\n\nclass Office31(ImageList):\n \"\"\"Office31 Dataset.\n\n Args:\n root (str): Root directory of dataset\n task (str): The task (domain) to create dataset. Choices include ``'A'``: amazon, \\\n ``'D'``: dslr and ``'W'``: webcam.\n download (bool, optional): If true, downloads the dataset from the internet and puts it \\\n in root directory. If dataset is already downloaded, it is not downloaded again.\n transform (callable, optional): A function/transform that takes in an PIL image and returns a \\\n transformed version. E.g, :class:`torchvision.transforms.RandomCrop`.\n target_transform (callable, optional): A function/transform that takes in the target and transforms it.\n\n .. note:: In `root`, there will exist following files after downloading.\n ::\n amazon/\n images/\n backpack/\n *.jpg\n ...\n dslr/\n webcam/\n image_list/\n amazon.txt\n dslr.txt\n webcam.txt\n \"\"\"\n download_list = [\n (\"image_list\", \"image_list.zip\", \"https://cloud.tsinghua.edu.cn/f/d9bca681c71249f19da2/?dl=1\"),\n (\"amazon\", \"amazon.tgz\", \"https://cloud.tsinghua.edu.cn/f/edc8d1bba1c740dc821c/?dl=1\"),\n (\"dslr\", \"dslr.tgz\", \"https://cloud.tsinghua.edu.cn/f/ca6df562b7e64850ad7f/?dl=1\"),\n (\"webcam\", \"webcam.tgz\", \"https://cloud.tsinghua.edu.cn/f/82b24ed2e08f4a3c8888/?dl=1\"),\n ]\n image_list = {\n \"A\": \"image_list/amazon.txt\",\n \"D\": \"image_list/dslr.txt\",\n \"W\": \"image_list/webcam.txt\"\n }\n CLASSES = ['back_pack', 'bike', 'bike_helmet', 'bookcase', 'bottle', 'calculator', 'desk_chair', 'desk_lamp',\n 'desktop_computer', 'file_cabinet', 'headphones', 'keyboard', 'laptop_computer', 'letter_tray',\n 'mobile_phone', 'monitor', 'mouse', 'mug', 'paper_notebook', 'pen', 'phone', 'printer', 'projector',\n 'punchers', 'ring_binder', 'ruler', 'scissors', 'speaker', 'stapler', 'tape_dispenser', 'trash_can']\n\n def __init__(self, root: str, task: str, download: Optional[bool] = True, **kwargs):\n assert task in self.image_list\n data_list_file = os.path.join(root, self.image_list[task])\n\n if download:\n list(map(lambda args: download_data(root, *args), self.download_list))\n else:\n list(map(lambda file_name, _: check_exits(root, file_name), self.download_list))\n\n super(Office31, self).__init__(root, Office31.CLASSES, data_list_file=data_list_file, **kwargs)\n\n @classmethod\n def domains(cls):\n return list(cls.image_list.keys())"
},
{
"path": "domain_generalization/Transfer-Learning-Library/common/vision/datasets/officecaltech.py",
"content": "\"\"\"\n@author: Junguang Jiang\n@contact: JiangJunguang1123@outlook.com\n\"\"\"\nimport os\nfrom typing import Optional\nfrom torchvision.datasets.folder import DatasetFolder, IMG_EXTENSIONS, default_loader\nfrom torchvision.datasets.utils import download_and_extract_archive\nfrom ._util import check_exits\n\n\nclass OfficeCaltech(DatasetFolder):\n \"\"\"Office+Caltech Dataset.\n\n Args:\n root (str): Root directory of dataset\n task (str): The task (domain) to create dataset. Choices include ``'A'``: amazon, \\\n ``'D'``: dslr, ``'W'``:webcam and ``'C'``: caltech.\n download (bool, optional): If true, downloads the dataset from the internet and puts it \\\n in root directory. If dataset is already downloaded, it is not downloaded again.\n transform (callable, optional): A function/transform that takes in an PIL image and returns a \\\n transformed version. E.g, :class:`torchvision.transforms.RandomCrop`.\n target_transform (callable, optional): A function/transform that takes in the target and transforms it.\n\n .. note:: In `root`, there will exist following files after downloading.\n ::\n amazon/\n images/\n backpack/\n *.jpg\n ...\n dslr/\n webcam/\n caltech/\n image_list/\n amazon.txt\n dslr.txt\n webcam.txt\n caltech.txt\n \"\"\"\n directories = {\n \"A\": \"amazon\",\n \"D\": \"dslr\",\n \"W\": \"webcam\",\n \"C\": \"caltech\"\n }\n CLASSES = ['back_pack', 'bike', 'calculator', 'headphones', 'keyboard',\n 'laptop_computer', 'monitor', 'mouse', 'mug', 'projector']\n\n def __init__(self, root: str, task: str, download: Optional[bool] = False, **kwargs):\n if download:\n for dir in self.directories.values():\n if not os.path.exists(os.path.join(root, dir)):\n download_and_extract_archive(url=\"https://cloud.tsinghua.edu.cn/f/e93f2e07d93243d6b57e/?dl=1\",\n download_root=os.path.join(root, 'download'),\n filename=\"officecaltech.tgz\", remove_finished=False, extract_root=root)\n break\n else:\n list(map(lambda dir, _: check_exits(root, dir), self.directories.values()))\n\n super(OfficeCaltech, self).__init__(\n os.path.join(root, self.directories[task]), default_loader, extensions=IMG_EXTENSIONS, **kwargs)\n self.classes = OfficeCaltech.CLASSES\n self.class_to_idx = {cls: idx\n for idx, clss in enumerate(self.classes)\n for cls in clss}\n\n @property\n def num_classes(self):\n \"\"\"Number of classes\"\"\"\n return len(self.classes)\n\n @classmethod\n def domains(cls):\n return list(cls.directories.keys())"
},
{
"path": "domain_generalization/Transfer-Learning-Library/common/vision/datasets/officehome.py",
"content": "\"\"\"\n@author: Junguang Jiang\n@contact: JiangJunguang1123@outlook.com\n\"\"\"\nimport os\nfrom typing import Optional\nfrom .imagelist import ImageList\nfrom ._util import download as download_data, check_exits\n\n\nclass OfficeHome(ImageList):\n \"\"\"`OfficeHome `_ Dataset.\n\n Args:\n root (str): Root directory of dataset\n task (str): The task (domain) to create dataset. Choices include ``'Ar'``: Art, \\\n ``'Cl'``: Clipart, ``'Pr'``: Product and ``'Rw'``: Real_World.\n download (bool, optional): If true, downloads the dataset from the internet and puts it \\\n in root directory. If dataset is already downloaded, it is not downloaded again.\n transform (callable, optional): A function/transform that takes in an PIL image and returns a \\\n transformed version. E.g, :class:`torchvision.transforms.RandomCrop`.\n target_transform (callable, optional): A function/transform that takes in the target and transforms it.\n\n .. note:: In `root`, there will exist following files after downloading.\n ::\n Art/\n Alarm_Clock/*.jpg\n ...\n Clipart/\n Product/\n Real_World/\n image_list/\n Art.txt\n Clipart.txt\n Product.txt\n Real_World.txt\n \"\"\"\n download_list = [\n (\"image_list\", \"image_list.zip\", \"https://cloud.tsinghua.edu.cn/f/ca3a3b6a8d554905b4cd/?dl=1\"),\n (\"Art\", \"Art.tgz\", \"https://cloud.tsinghua.edu.cn/f/4691878067d04755beab/?dl=1\"),\n (\"Clipart\", \"Clipart.tgz\", \"https://cloud.tsinghua.edu.cn/f/0d41e7da4558408ea5aa/?dl=1\"),\n (\"Product\", \"Product.tgz\", \"https://cloud.tsinghua.edu.cn/f/76186deacd7c4fa0a679/?dl=1\"),\n (\"Real_World\", \"Real_World.tgz\", \"https://cloud.tsinghua.edu.cn/f/dee961894cc64b1da1d7/?dl=1\")\n ]\n image_list = {\n \"Ar\": \"image_list/Art.txt\",\n \"Cl\": \"image_list/Clipart.txt\",\n \"Pr\": \"image_list/Product.txt\",\n \"Rw\": \"image_list/Real_World.txt\",\n }\n CLASSES = ['Drill', 'Exit_Sign', 'Bottle', 'Glasses', 'Computer', 'File_Cabinet', 'Shelf', 'Toys', 'Sink',\n 'Laptop', 'Kettle', 'Folder', 'Keyboard', 'Flipflops', 'Pencil', 'Bed', 'Hammer', 'ToothBrush', 'Couch',\n 'Bike', 'Postit_Notes', 'Mug', 'Webcam', 'Desk_Lamp', 'Telephone', 'Helmet', 'Mouse', 'Pen', 'Monitor',\n 'Mop', 'Sneakers', 'Notebook', 'Backpack', 'Alarm_Clock', 'Push_Pin', 'Paper_Clip', 'Batteries', 'Radio',\n 'Fan', 'Ruler', 'Pan', 'Screwdriver', 'Trash_Can', 'Printer', 'Speaker', 'Eraser', 'Bucket', 'Chair',\n 'Calendar', 'Calculator', 'Flowers', 'Lamp_Shade', 'Spoon', 'Candles', 'Clipboards', 'Scissors', 'TV',\n 'Curtains', 'Fork', 'Soda', 'Table', 'Knives', 'Oven', 'Refrigerator', 'Marker']\n\n def __init__(self, root: str, task: str, download: Optional[bool] = False, **kwargs):\n assert task in self.image_list\n data_list_file = os.path.join(root, self.image_list[task])\n\n if download:\n list(map(lambda args: download_data(root, *args), self.download_list))\n else:\n list(map(lambda file_name, _: check_exits(root, file_name), self.download_list))\n\n super(OfficeHome, self).__init__(root, OfficeHome.CLASSES, data_list_file=data_list_file, **kwargs)\n\n @classmethod\n def domains(cls):\n return list(cls.image_list.keys())"
},
{
"path": "domain_generalization/Transfer-Learning-Library/common/vision/datasets/openset/__init__.py",
"content": "\"\"\"\n@author: Junguang Jiang\n@contact: JiangJunguang1123@outlook.com\n\"\"\"\nfrom ..imagelist import ImageList\nfrom ..office31 import Office31\nfrom ..officehome import OfficeHome\nfrom ..visda2017 import VisDA2017\n\nfrom typing import Optional, ClassVar, Sequence\nfrom copy import deepcopy\n\n\n__all__ = ['Office31', 'OfficeHome', \"VisDA2017\"]\n\n\ndef open_set(dataset_class: ClassVar, public_classes: Sequence[str],\n private_classes: Optional[Sequence[str]] = ()) -> ClassVar:\n \"\"\"\n Convert a dataset into its open-set version.\n\n In other words, those samples which doesn't belong to `private_classes` will be marked as \"unknown\".\n\n Be aware that `open_set` will change the label number of each category.\n\n Args:\n dataset_class (class): Dataset class. Only subclass of ``ImageList`` can be open-set.\n public_classes (sequence[str]): A sequence of which categories need to be kept in the open-set dataset.\\\n Each element of `public_classes` must belong to the `classes` list of `dataset_class`.\n private_classes (sequence[str], optional): A sequence of which categories need to be marked as \"unknown\" \\\n in the open-set dataset. Each element of `private_classes` must belong to the `classes` list of \\\n `dataset_class`. Default: ().\n\n Examples::\n\n >>> public_classes = ['back_pack', 'bike', 'calculator', 'headphones', 'keyboard']\n >>> private_classes = ['laptop_computer', 'monitor', 'mouse', 'mug', 'projector']\n >>> # create a open-set dataset class which has classes\n >>> # 'back_pack', 'bike', 'calculator', 'headphones', 'keyboard' and 'unknown'.\n >>> OpenSetOffice31 = open_set(Office31, public_classes, private_classes)\n >>> # create an instance of the open-set dataset\n >>> dataset = OpenSetDataset(root=\"data/office31\", task=\"A\")\n\n \"\"\"\n if not (issubclass(dataset_class, ImageList)):\n raise Exception(\"Only subclass of ImageList can be openset\")\n\n class OpenSetDataset(dataset_class):\n def __init__(self, **kwargs):\n super(OpenSetDataset, self).__init__(**kwargs)\n samples = []\n all_classes = list(deepcopy(public_classes)) + [\"unknown\"]\n for (path, label) in self.samples:\n class_name = self.classes[label]\n if class_name in public_classes:\n samples.append((path, all_classes.index(class_name)))\n elif class_name in private_classes:\n samples.append((path, all_classes.index(\"unknown\")))\n self.samples = samples\n self.classes = all_classes\n self.class_to_idx = {cls: idx\n for idx, cls in enumerate(self.classes)}\n\n return OpenSetDataset\n\n\ndef default_open_set(dataset_class: ClassVar, source: bool) -> ClassVar:\n \"\"\"\n Default open-set used in some paper.\n\n Args:\n dataset_class (class): Dataset class. Currently, dataset_class must be one of\n :class:`~common.vision.datasets.office31.Office31`, :class:`~common.vision.datasets.officehome.OfficeHome`,\n :class:`~common.vision.datasets.visda2017.VisDA2017`,\n source (bool): Whether the dataset is used for source domain or not.\n \"\"\"\n if dataset_class == Office31:\n public_classes = Office31.CLASSES[:20]\n if source:\n private_classes = ()\n else:\n private_classes = Office31.CLASSES[20:]\n elif dataset_class == OfficeHome:\n public_classes = sorted(OfficeHome.CLASSES)[:25]\n if source:\n private_classes = ()\n else:\n private_classes = sorted(OfficeHome.CLASSES)[25:]\n elif dataset_class == VisDA2017:\n public_classes = ('bicycle', 'bus', 'car', 'motorcycle', 'train', 'truck')\n if source:\n private_classes = ()\n else:\n private_classes = ('aeroplane', 'horse', 'knife', 'person', 'plant', 'skateboard')\n else:\n raise NotImplementedError(\"Unknown openset domain adaptation dataset: {}\".format(dataset_class.__name__))\n return open_set(dataset_class, public_classes, private_classes)\n\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/common/vision/datasets/oxfordflowers.py",
"content": "\"\"\"\n@author: Junguang Jiang\n@contact: JiangJunguang1123@outlook.com\n\"\"\"\nimport os\nfrom common.vision.datasets import ImageList\nfrom ._util import download as download_data, check_exits\n\n\nclass OxfordFlowers102(ImageList):\n \"\"\"\n `The Oxford Flowers 102 `_ is a \\\n consistent of 102 flower categories commonly occurring in the United Kingdom. \\\n Each class consists of between 40 and 258 images. The images have large scale, \\\n pose and light variations. In addition, there are categories that have large \\\n variations within the category and several very similar categories. \\\n The dataset is divided into a training set, a validation set and a test set. \\\n The training set and validation set each consist of 10 images per class \\\n (totalling 1020 images each). \\\n The test set consists of the remaining 6149 images (minimum 20 per class).\n\n Args:\n root (str): Root directory of dataset\n split (str, optional): The dataset split, supports ``train``, or ``test``.\n download (bool, optional): If true, downloads the dataset from the internet and puts it \\\n in root directory. If dataset is already downloaded, it is not downloaded again.\n transform (callable, optional): A function/transform that takes in an PIL image and returns a \\\n transformed version. E.g, :class:`torchvision.transforms.RandomCrop`.\n target_transform (callable, optional): A function/transform that takes in the target and transforms it.\n \"\"\"\n CLASSES = ['pink primrose', 'hard-leaved pocket orchid', 'canterbury bells', 'sweet pea', 'english marigold',\n 'tiger lily', 'moon orchid', 'bird of paradise', 'monkshood', 'globe thistle', 'snapdragon',\n \"colt's foot\", 'king protea', 'spear thistle', 'yellow iris', 'globe-flower', 'purple coneflower',\n 'peruvian lily', 'balloon flower', 'giant white arum lily', 'fire lily', 'pincushion flower',\n 'fritillary', 'red ginger', 'grape hyacinth', 'corn poppy', 'prince of wales feathers',\n 'stemless gentian', 'artichoke', 'sweet william', 'carnation', 'garden phlox', 'love in the mist',\n 'mexican aster', 'alpine sea holly', 'ruby-lipped cattleya', 'cape flower', 'great masterwort',\n 'siam tulip', 'lenten rose', 'barbeton daisy', 'daffodil', 'sword lily', 'poinsettia',\n 'bolero deep blue', 'wallflower', 'marigold', 'buttercup', 'oxeye daisy', 'common dandelion',\n 'petunia', 'wild pansy', 'primula', 'sunflower', 'pelargonium', 'bishop of llandaff', 'gaura',\n 'geranium', 'orange dahlia', 'pink-yellow dahlia?', 'cautleya spicata', 'japanese anemone',\n 'black-eyed susan', 'silverbush', 'californian poppy', 'osteospermum', 'spring crocus',\n 'bearded iris', 'windflower', 'tree poppy', 'gazania', 'azalea', 'water lily', 'rose', 'thorn apple',\n 'morning glory', 'passion flower', 'lotus', 'toad lily', 'anthurium', 'frangipani', 'clematis',\n 'hibiscus', 'columbine', 'desert-rose', 'tree mallow', 'magnolia', 'cyclamen', 'watercress',\n 'canna lily', 'hippeastrum', 'bee balm', 'ball moss', 'foxglove', 'bougainvillea', 'camellia',\n 'mallow', 'mexican petunia', 'bromelia', 'blanket flower', 'trumpet creeper', 'blackberry lily']\n def __init__(self, root, split='train', download=False, **kwargs):\n if download:\n download_data(root, \"oxford_flowers102\", \"oxford_flowers102.tgz\", \"https://cloud.tsinghua.edu.cn/f/9910751054764d568281/?dl=1\")\n else:\n check_exits(root, \"oxford_flowers102\")\n root = os.path.join(root, \"oxford_flowers102\")\n super(OxfordFlowers102, self).__init__(root, OxfordFlowers102.CLASSES, os.path.join(root, 'imagelist', '{}.txt'.format(split)), **kwargs)\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/common/vision/datasets/oxfordpet.py",
"content": "\"\"\"\n@author: Yifei Ji\n@contact: jiyf990330@163.com\n\"\"\"\nimport os\nfrom typing import Optional\nfrom .imagelist import ImageList\nfrom ._util import download as download_data, check_exits\n\n\nclass OxfordIIITPet(ImageList):\n \"\"\"`The Oxford-IIIT Pet `_ \\\n is a 37-category pet dataset with roughly 200 images for each class.\n\n Args:\n root (str): Root directory of dataset\n split (str, optional): The dataset split, supports ``train``, or ``test``.\n sample_rate (int): The sampling rates to sample random ``training`` images for each category.\n Choices include 100, 50, 30, 15. Default: 100.\n download (bool, optional): If true, downloads the dataset from the internet and puts it \\\n in root directory. If dataset is already downloaded, it is not downloaded again.\n transform (callable, optional): A function/transform that takes in an PIL image and returns a \\\n transformed version. E.g, :class:`torchvision.transforms.RandomCrop`.\n target_transform (callable, optional): A function/transform that takes in the target and transforms it.\n\n .. note:: In `root`, there will exist following files after downloading.\n ::\n train/\n test/\n image_list/\n train_100.txt\n train_50.txt\n train_30.txt\n train_15.txt\n test.txt\n \"\"\"\n download_list = [\n (\"image_list\", \"image_list.zip\", \"https://cloud.tsinghua.edu.cn/f/738d75de56844bd0951b/?dl=1\"),\n (\"train\", \"train.tgz\", \"https://cloud.tsinghua.edu.cn/f/f40cf9f8c9ac4b04ba9b/?dl=1\"),\n (\"test\", \"test.tgz\", \"https://cloud.tsinghua.edu.cn/f/72b4bff8b5c84f4ba240/?dl=1\"),\n ]\n image_list = {\n \"train\": \"image_list/train_100.txt\",\n \"train100\": \"image_list/train_100.txt\",\n \"train50\": \"image_list/train_50.txt\",\n \"train30\": \"image_list/train_30.txt\",\n \"train15\": \"image_list/train_15.txt\",\n \"test\": \"image_list/test.txt\",\n \"test100\": \"image_list/test.txt\",\n }\n CLASSES = ['Abyssinian', 'american_bulldog', 'american_pit_bull_terrier', 'basset_hound', 'beagle', 'Bengal',\n 'Birman', 'Bombay', 'boxer', 'British_Shorthair', 'chihuahua', 'Egyptian_Mau', 'english_cocker_spaniel',\n 'english_setter', 'german_shorthaired', 'great_pyrenees', 'havanese', 'japanese_chin', 'keeshond', 'leonberger',\n 'Maine_Coon', 'miniature_pinscher', 'newfoundland', 'Persian', 'pomeranian', 'pug', 'Ragdoll',\n 'Russian_Blue', 'saint_bernard', 'samoyed', 'scottish_terrier', 'shiba_inu', 'Siamese', 'Sphynx',\n 'staffordshire_bull_terrier', 'wheaten_terrier', 'yorkshire_terrier']\n\n def __init__(self, root: str, split: str, sample_rate: Optional[int] =100, download: Optional[bool] = False, **kwargs):\n\n if split == 'train':\n list_name = 'train' + str(sample_rate)\n assert list_name in self.image_list\n data_list_file = os.path.join(root, self.image_list[list_name])\n else:\n data_list_file = os.path.join(root, self.image_list['test'])\n\n if download:\n list(map(lambda args: download_data(root, *args), self.download_list))\n else:\n list(map(lambda file_name, _: check_exits(root, file_name), self.download_list))\n\n super(OxfordIIITPet, self).__init__(root, OxfordIIITPet.CLASSES, data_list_file=data_list_file, **kwargs)\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/common/vision/datasets/pacs.py",
"content": "from typing import Optional\nimport os\nfrom .imagelist import ImageList\nfrom ._util import download as download_data, check_exits\n\n\nclass PACS(ImageList):\n \"\"\"`PACS Dataset `_.\n\n Args:\n root (str): Root directory of dataset\n task (str): The task (domain) to create dataset. Choices include ``'A'``: amazon, \\\n ``'D'``: dslr and ``'W'``: webcam.\n download (bool, optional): If true, downloads the dataset from the internet and puts it \\\n in root directory. If dataset is already downloaded, it is not downloaded again.\n transform (callable, optional): A function/transform that takes in an PIL image and returns a \\\n transformed version. E.g, :class:`torchvision.transforms.RandomCrop`.\n target_transform (callable, optional): A function/transform that takes in the target and transforms it.\n\n .. note:: In `root`, there will exist following files after downloading.\n ::\n art_painting/\n dog/\n *.jpg\n ...\n cartoon/\n photo/\n sketch\n image_list/\n art_painting.txt\n cartoon.txt\n photo.txt\n sketch.txt\n \"\"\"\n download_list = [\n (\"image_list\", \"image_list.zip\", \"https://cloud.tsinghua.edu.cn/f/add42cc3859847bc988c/?dl=1\"),\n (\"art_painting\", \"art_painting.tgz\", \"https://cloud.tsinghua.edu.cn/f/4eb7db4f3eec41719856/?dl=1\"),\n (\"cartoon\", \"cartoon.tgz\", \"https://cloud.tsinghua.edu.cn/f/d847ac22497b4826889f/?dl=1\"),\n (\"photo\", \"photo.tgz\", \"https://cloud.tsinghua.edu.cn/f/458ad21483da4a45935b/?dl=1\"),\n (\"sketch\", \"sketch.tgz\", \"https://cloud.tsinghua.edu.cn/f/c892ac2d94a44b1196b8/?dl=1\"),\n ]\n image_list = {\n \"A\": \"image_list/art_painting_{}.txt\",\n \"C\": \"image_list/cartoon_{}.txt\",\n \"P\": \"image_list/photo_{}.txt\",\n \"S\": \"image_list/sketch_{}.txt\"\n }\n CLASSES = ['dog', 'elephant', 'giraffe', 'guitar', 'horse', 'house', 'person']\n\n def __init__(self, root: str, task: str, split='all', download: Optional[bool] = True, **kwargs):\n assert task in self.image_list\n assert split in [\"train\", \"val\", \"all\", \"test\"]\n if split == \"test\":\n split = \"all\"\n data_list_file = os.path.join(root, self.image_list[task].format(split))\n\n if download:\n list(map(lambda args: download_data(root, *args), self.download_list))\n else:\n list(map(lambda file_name, _: check_exits(root, file_name), self.download_list))\n\n super(PACS, self).__init__(root, PACS.CLASSES, data_list_file=data_list_file, target_transform=lambda x: x - 1,\n **kwargs)\n\n @classmethod\n def domains(cls):\n return list(cls.image_list.keys())\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/common/vision/datasets/partial/__init__.py",
"content": "\"\"\"\n@author: Junguang Jiang\n@contact: JiangJunguang1123@outlook.com\n\"\"\"\nfrom ..imagelist import ImageList\nfrom ..office31 import Office31\nfrom ..officehome import OfficeHome\nfrom ..visda2017 import VisDA2017\nfrom ..officecaltech import OfficeCaltech\nfrom .imagenet_caltech import ImageNetCaltech\nfrom .caltech_imagenet import CaltechImageNet\nfrom common.vision.datasets.partial.imagenet_caltech import ImageNetCaltech\nfrom typing import Sequence, ClassVar\n\n\n__all__ = ['Office31', 'OfficeHome', \"VisDA2017\", \"CaltechImageNet\", \"ImageNetCaltech\"]\n\n\ndef partial(dataset_class: ClassVar, partial_classes: Sequence[str]) -> ClassVar:\n \"\"\"\n Convert a dataset into its partial version.\n\n In other words, those samples which doesn't belong to `partial_classes` will be discarded.\n Yet `partial` will not change the label space of `dataset_class`.\n\n Args:\n dataset_class (class): Dataset class. Only subclass of ``ImageList`` can be partial.\n partial_classes (sequence[str]): A sequence of which categories need to be kept in the partial dataset.\\\n Each element of `partial_classes` must belong to the `classes` list of `dataset_class`.\n\n Examples::\n\n >>> partial_classes = ['back_pack', 'bike', 'calculator', 'headphones', 'keyboard']\n >>> # create a partial dataset class\n >>> PartialOffice31 = partial(Office31, partial_classes)\n >>> # create an instance of the partial dataset\n >>> dataset = PartialDataset(root=\"data/office31\", task=\"A\")\n\n \"\"\"\n if not (issubclass(dataset_class, ImageList)):\n raise Exception(\"Only subclass of ImageList can be partial\")\n\n class PartialDataset(dataset_class):\n def __init__(self, **kwargs):\n super(PartialDataset, self).__init__(**kwargs)\n assert all([c in self.classes for c in partial_classes])\n samples = []\n for (path, label) in self.samples:\n class_name = self.classes[label]\n if class_name in partial_classes:\n samples.append((path, label))\n self.samples = samples\n self.partial_classes = partial_classes\n self.partial_classes_idx = [self.class_to_idx[c] for c in partial_classes]\n\n return PartialDataset\n\n\ndef default_partial(dataset_class: ClassVar) -> ClassVar:\n \"\"\"\n Default partial used in some paper.\n\n Args:\n dataset_class (class): Dataset class. Currently, dataset_class must be one of\n :class:`~common.vision.datasets.office31.Office31`, :class:`~common.vision.datasets.officehome.OfficeHome`,\n :class:`~common.vision.datasets.visda2017.VisDA2017`,\n :class:`~common.vision.datasets.partial.imagenet_caltech.ImageNetCaltech`\n and :class:`~common.vision.datasets.partial.caltech_imagenet.CaltechImageNet`.\n \"\"\"\n if dataset_class == Office31:\n kept_classes = OfficeCaltech.CLASSES\n elif dataset_class == OfficeHome:\n kept_classes = sorted(OfficeHome.CLASSES)[:25]\n elif dataset_class == VisDA2017:\n kept_classes = sorted(VisDA2017.CLASSES)[:6]\n elif dataset_class in [ImageNetCaltech, CaltechImageNet]:\n kept_classes = dataset_class.CLASSES\n else:\n raise NotImplementedError(\"Unknown partial domain adaptation dataset: {}\".format(dataset_class.__name__))\n return partial(dataset_class, kept_classes)"
},
{
"path": "domain_generalization/Transfer-Learning-Library/common/vision/datasets/partial/caltech_imagenet.py",
"content": "\"\"\"\n@author: Junguang Jiang\n@contact: JiangJunguang1123@outlook.com\n\"\"\"\nfrom typing import Optional\nimport os\nfrom ..imagelist import ImageList\nfrom .._util import download as download_data, check_exits\n\n_CLASSES = ['ak47', 'american flag', 'backpack', 'baseball bat', 'baseball glove', 'basketball hoop', 'bat',\n 'bathtub', 'bear', 'beer mug', 'billiards', 'binoculars', 'birdbath', 'blimp', 'bonsai 101',\n 'boom box', 'bowling ball', 'bowling pin', 'boxing glove', 'brain 101', 'breadmaker', 'buddha 101',\n 'bulldozer', 'butterfly', 'cactus', 'cake', 'calculator', 'camel', 'cannon', 'canoe', 'car tire',\n 'cartman', 'cd', 'centipede', 'cereal box', 'chandelier 101', 'chess board', 'chimp', 'chopsticks',\n 'cockroach', 'coffee mug', 'coffin', 'coin', 'comet', 'computer keyboard', 'computer monitor',\n 'computer mouse', 'conch', 'cormorant', 'covered wagon', 'cowboy hat', 'crab 101', 'desk globe',\n 'diamond ring', 'dice', 'dog', 'dolphin 101', 'doorknob', 'drinking straw', 'duck', 'dumb bell',\n 'eiffel tower', 'electric guitar 101', 'elephant 101', 'elk', 'ewer 101', 'eyeglasses', 'fern',\n 'fighter jet', 'fire extinguisher', 'fire hydrant', 'fire truck', 'fireworks', 'flashlight',\n 'floppy disk', 'football helmet', 'french horn', 'fried egg', 'frisbee', 'frog', 'frying pan',\n 'galaxy', 'gas pump', 'giraffe', 'goat', 'golden gate bridge', 'goldfish', 'golf ball', 'goose',\n 'gorilla', 'grand piano 101', 'grapes', 'grasshopper', 'guitar pick', 'hamburger', 'hammock',\n 'harmonica', 'harp', 'harpsichord', 'hawksbill 101', 'head phones', 'helicopter 101', 'hibiscus',\n 'homer simpson', 'horse', 'horseshoe crab', 'hot air balloon', 'hot dog', 'hot tub', 'hourglass',\n 'house fly', 'human skeleton', 'hummingbird', 'ibis 101', 'ice cream cone', 'iguana', 'ipod',\n 'iris', 'jesus christ', 'joy stick', 'kangaroo 101', 'kayak', 'ketch 101', 'killer whale', 'knife',\n 'ladder', 'laptop 101', 'lathe', 'leopards 101', 'license plate', 'lightbulb', 'light house',\n 'lightning', 'llama 101', 'mailbox', 'mandolin', 'mars', 'mattress', 'megaphone', 'menorah 101',\n 'microscope', 'microwave', 'minaret', 'minotaur', 'motorbikes 101', 'mountain bike', 'mushroom',\n 'mussels', 'necktie', 'octopus', 'ostrich', 'owl', 'palm pilot', 'palm tree', 'paperclip',\n 'paper shredder', 'pci card', 'penguin', 'people', 'pez dispenser', 'photocopier', 'picnic table',\n 'playing card', 'porcupine', 'pram', 'praying mantis', 'pyramid', 'raccoon', 'radio telescope',\n 'rainbow', 'refrigerator', 'revolver 101', 'rifle', 'rotary phone', 'roulette wheel', 'saddle',\n 'saturn', 'school bus', 'scorpion 101', 'screwdriver', 'segway', 'self propelled lawn mower',\n 'sextant', 'sheet music', 'skateboard', 'skunk', 'skyscraper', 'smokestack', 'snail', 'snake',\n 'sneaker', 'snowmobile', 'soccer ball', 'socks', 'soda can', 'spaghetti', 'speed boat', 'spider',\n 'spoon', 'stained glass', 'starfish 101', 'steering wheel', 'stirrups', 'sunflower 101', 'superman',\n 'sushi', 'swan', 'swiss army knife', 'sword', 'syringe', 'tambourine', 'teapot', 'teddy bear',\n 'teepee', 'telephone box', 'tennis ball', 'tennis court', 'tennis racket', 'theodolite', 'toaster',\n 'tomato', 'tombstone', 'top hat', 'touring bike', 'tower pisa', 'traffic light', 'treadmill',\n 'triceratops', 'tricycle', 'trilobite 101', 'tripod', 't shirt', 'tuning fork', 'tweezer',\n 'umbrella 101', 'unicorn', 'vcr', 'video projector', 'washing machine', 'watch 101', 'waterfall',\n 'watermelon', 'welding mask', 'wheelbarrow', 'windmill', 'wine bottle', 'xylophone', 'yarmulke',\n 'yo yo', 'zebra', 'airplanes 101', 'car side 101', 'faces easy 101', 'greyhound', 'tennis shoes',\n 'toad']\n\n\nclass CaltechImageNet(ImageList):\n \"\"\"Caltech-ImageNet is constructed from `Caltech-256 `_ and\n `ImageNet-1K `_ .\n\n They share 84 common classes. Caltech-ImageNet keeps all classes of Caltech-256.\n The label is based on the Caltech256 (class 0-255) . The private classes of ImageNet-1K is discarded.\n\n\n Args:\n root (str): Root directory of dataset\n task (str): The task (domain) to create dataset. Choices include ``'C'``:Caltech-256, \\\n ``'I'``: ImageNet-1K validation set.\n download (bool, optional): If true, downloads the dataset from the internet and puts it \\\n in root directory. If dataset is already downloaded, it is not downloaded again.\n transform (callable, optional): A function/transform that takes in an PIL image and returns a \\\n transformed version. E.g, :class:`torchvision.transforms.RandomCrop`.\n target_transform (callable, optional): A function/transform that takes in the target and transforms it.\n\n .. note:: You need to put ``train`` and ``val`` directory of ImageNet-1K manually in `root` directory\n since ImageNet-1K is no longer publicly accessible. DALIB will only download Caltech-256 and ImageList automatically.\n In `root`, there will exist following files after downloading.\n ::\n train/\n n01440764/\n ...\n val/\n 256_ObjectCategories/\n 001.ak47/\n ...\n image_list/\n caltech_256_list.txt\n ...\n \"\"\"\n image_list = {\n \"C\": \"image_list/caltech_256_list.txt\",\n \"I\": \"image_list/imagenet_val_84_list.txt\",\n }\n CLASSES = _CLASSES\n\n def __init__(self, root: str, task: str, download: Optional[bool] = True, **kwargs):\n assert task in self.image_list\n data_list_file = os.path.join(root, self.image_list[task])\n\n if download:\n list(map(lambda args: download_data(root, *args), download_list))\n else:\n list(map(lambda file_name, _: check_exits(root, file_name), download_list))\n\n if not os.path.exists(os.path.join(root, 'val')):\n print(\"Please put train and val directory of ImageNet-1K manually under {} \"\n \"since ImageNet-1K is no longer publicly accessible.\".format(root))\n exit(-1)\n\n super(CaltechImageNet, self).__init__(root, CaltechImageNet.CLASSES, data_list_file=data_list_file, **kwargs)\n\n\nclass CaltechImageNetUniversal(ImageList):\n \"\"\"Caltech-ImageNet-Universal is constructed from `Caltech-256 `_\n and `ImageNet-1K `_ .\n\n They share 84 common classes. Caltech-ImageNet keeps all classes of Caltech-256.\n The label is based on the Caltech256 (class 0-255) . The private classes of ImageNet-1K is grouped into class 256 (\"unknown\").\n Thus, CaltechImageNetUniversal has 257 classes in total.\n\n Args:\n root (str): Root directory of dataset\n task (str): The task (domain) to create dataset. Choices include ``'C'``:Caltech-256, \\\n ``'I'``: ImageNet-1K validation set.\n download (bool, optional): If true, downloads the dataset from the internet and puts it \\\n in root directory. If dataset is already downloaded, it is not downloaded again.\n transform (callable, optional): A function/transform that takes in an PIL image and returns a \\\n transformed version. E.g, :class:`torchvision.transforms.RandomCrop`.\n target_transform (callable, optional): A function/transform that takes in the target and transforms it.\n\n .. note:: You need to put ``train`` and ``val`` directory of ImageNet-1K manually in `root` directory\n since ImageNet-1K is no longer publicly accessible. DALIB will only download Caltech-256 and ImageList automatically.\n In `root`, there will exist following files after downloading.\n ::\n train/\n n01440764/\n ...\n val/\n 256_ObjectCategories/\n 001.ak47/\n ...\n image_list/\n caltech_256_list.txt\n ...\n \"\"\"\n image_list = {\n \"C\": \"image_list/caltech_256_list.txt\",\n \"I\": \"image_list/imagenet_val_85_list.txt\",\n }\n CLASSES = _CLASSES + ['unknown']\n\n def __init__(self, root: str, task: str, download: Optional[bool] = True, **kwargs):\n assert task in self.image_list\n data_list_file = os.path.join(root, self.image_list[task])\n\n if download:\n list(map(lambda args: download_data(root, *args), download_list))\n else:\n list(map(lambda file_name, _: check_exits(root, file_name), download_list))\n\n if not os.path.exists(os.path.join(root, 'val')):\n print(\"Please put train and val directory of ImageNet-1K manually under {} \"\n \"since ImageNet-1K is no longer publicly accessible.\".format(root))\n exit(-1)\n\n super(CaltechImageNetUniversal, self).__init__(root, CaltechImageNetUniversal.CLASSES,\n data_list_file=data_list_file, **kwargs)\n\n\n\ndownload_list = [\n (\"image_list\", \"image_list.zip\", \"https://cloud.tsinghua.edu.cn/f/a0d7ea37026946f98965/?dl=1\"),\n (\"256_ObjectCategories\", \"256_ObjectCategories.tar\",\n \"http://www.vision.caltech.edu/Image_Datasets/Caltech256/256_ObjectCategories.tar\"),\n]\n\n\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/common/vision/datasets/partial/imagenet_caltech.py",
"content": "\"\"\"\n@author: Junguang Jiang\n@contact: JiangJunguang1123@outlook.com\n\"\"\"\nimport os\nfrom typing import Optional\nfrom ..imagelist import ImageList\nfrom .._util import download as download_data, check_exits\n\n\n_CLASSES = [c[0] for c in [('tench', 'Tinca tinca'), ('goldfish', 'Carassius auratus'),\n ('great white shark', 'white shark', 'man-eater', 'man-eating shark', 'Carcharodon carcharias'),\n ('tiger shark', 'Galeocerdo cuvieri'), ('hammerhead', 'hammerhead shark'),\n ('electric ray', 'crampfish', 'numbfish', 'torpedo'), ('stingray',), ('cock',), ('hen',),\n ('ostrich', 'Struthio camelus'), ('brambling', 'Fringilla montifringilla'),\n ('goldfinch', 'Carduelis carduelis'), ('house finch', 'linnet', 'Carpodacus mexicanus'),\n ('junco', 'snowbird'), ('indigo bunting', 'indigo finch', 'indigo bird', 'Passerina cyanea'),\n ('robin', 'American robin', 'Turdus migratorius'), ('bulbul',), ('jay',), ('magpie',),\n ('chickadee',), ('water ouzel', 'dipper'), ('kite',),\n ('bald eagle', 'American eagle', 'Haliaeetus leucocephalus'), ('vulture',),\n ('great grey owl', 'great gray owl', 'Strix nebulosa'),\n ('European fire salamander', 'Salamandra salamandra'), ('common newt', 'Triturus vulgaris'),\n ('eft',), ('spotted salamander', 'Ambystoma maculatum'),\n ('axolotl', 'mud puppy', 'Ambystoma mexicanum'), ('bullfrog', 'Rana catesbeiana'),\n ('tree frog', 'tree-frog'),\n ('tailed frog', 'bell toad', 'ribbed toad', 'tailed toad', 'Ascaphus trui'),\n ('loggerhead', 'loggerhead turtle', 'Caretta caretta'),\n ('leatherback turtle', 'leatherback', 'leathery turtle', 'Dermochelys coriacea'),\n ('mud turtle',), ('terrapin',), ('box turtle', 'box tortoise'), ('banded gecko',),\n ('common iguana', 'iguana', 'Iguana iguana'),\n ('American chameleon', 'anole', 'Anolis carolinensis'), ('whiptail', 'whiptail lizard'),\n ('agama',), ('frilled lizard', 'Chlamydosaurus kingi'), ('alligator lizard',),\n ('Gila monster', 'Heloderma suspectum'), ('green lizard', 'Lacerta viridis'),\n ('African chameleon', 'Chamaeleo chamaeleon'),\n ('Komodo dragon', 'Komodo lizard', 'dragon lizard', 'giant lizard', 'Varanus komodoensis'),\n ('African crocodile', 'Nile crocodile', 'Crocodylus niloticus'),\n ('American alligator', 'Alligator mississipiensis'), ('triceratops',),\n ('thunder snake', 'worm snake', 'Carphophis amoenus'),\n ('ringneck snake', 'ring-necked snake', 'ring snake'),\n ('hognose snake', 'puff adder', 'sand viper'), ('green snake', 'grass snake'),\n ('king snake', 'kingsnake'), ('garter snake', 'grass snake'), ('water snake',),\n ('vine snake',), ('night snake', 'Hypsiglena torquata'),\n ('boa constrictor', 'Constrictor constrictor'), ('rock python', 'rock snake', 'Python sebae'),\n ('Indian cobra', 'Naja naja'), ('green mamba',), ('sea snake',),\n ('horned viper', 'cerastes', 'sand viper', 'horned asp', 'Cerastes cornutus'),\n ('diamondback', 'diamondback rattlesnake', 'Crotalus adamanteus'),\n ('sidewinder', 'horned rattlesnake', 'Crotalus cerastes'), ('trilobite',),\n ('harvestman', 'daddy longlegs', 'Phalangium opilio'), ('scorpion',),\n ('black and gold garden spider', 'Argiope aurantia'), ('barn spider', 'Araneus cavaticus'),\n ('garden spider', 'Aranea diademata'), ('black widow', 'Latrodectus mactans'), ('tarantula',),\n ('wolf spider', 'hunting spider'), ('tick',), ('centipede',), ('black grouse',),\n ('ptarmigan',), ('ruffed grouse', 'partridge', 'Bonasa umbellus'),\n ('prairie chicken', 'prairie grouse', 'prairie fowl'), ('peacock',), ('quail',),\n ('partridge',), ('African grey', 'African gray', 'Psittacus erithacus'), ('macaw',),\n ('sulphur-crested cockatoo', 'Kakatoe galerita', 'Cacatua galerita'), ('lorikeet',),\n ('coucal',), ('bee eater',), ('hornbill',), ('hummingbird',), ('jacamar',), ('toucan',),\n ('drake',), ('red-breasted merganser', 'Mergus serrator'), ('goose',),\n ('black swan', 'Cygnus atratus'), ('tusker',), ('echidna', 'spiny anteater', 'anteater'), (\n 'platypus', 'duckbill', 'duckbilled platypus', 'duck-billed platypus',\n 'Ornithorhynchus anatinus'), ('wallaby', 'brush kangaroo'),\n ('koala', 'koalabear', 'kangaroo bear', 'native bear', 'Phascolarctos cinereus'), ('wombat',),\n ('jellyfish',), ('sea anemone', 'anemone'), ('brain coral',), ('flatworm', 'platyhelminth'),\n ('nematode', 'nematode worm', 'roundworm'), ('conch',), ('snail',), ('slug',),\n ('sea slug', 'nudibranch'), ('chiton', 'coat-of-mail shell', 'sea cradle', 'polyplacophore'),\n ('chambered nautilus', 'pearly nautilus', 'nautilus'), ('Dungeness crab', 'Cancer magister'),\n ('rock crab', 'Cancer irroratus'), ('fiddler crab',), (\n 'king crab', 'Alaska crab', 'Alaskan king crab', 'Alaska king crab',\n 'Paralithodes camtschatica'),\n ('American lobster', 'Northernlobster', 'Maine lobster', 'Homarus americanus'),\n ('spiny lobster', 'langouste', 'rock lobster', 'crawfish', 'crayfish', 'sea crawfish'),\n ('crayfish', 'crawfish', 'crawdad', 'crawdaddy'), ('hermit crab',), ('isopod',),\n ('white stork', 'Ciconia ciconia'), ('black stork', 'Ciconia nigra'), ('spoonbill',),\n ('flamingo',), ('little blue heron', 'Egretta caerulea'),\n ('American egret', 'great white heron', 'Egretta albus'), ('bittern',), ('crane',),\n ('limpkin', 'Aramus pictus'), ('European gallinule', 'Porphyrio porphyrio'),\n ('American coot', 'marsh hen', 'mud hen', 'water hen', 'Fulica americana'), ('bustard',),\n ('ruddy turnstone', 'Arenaria interpres'),\n ('red-backed sandpiper', 'dunlin', 'Erolia alpina'), ('redshank', 'Tringa totanus'),\n ('dowitcher',), ('oystercatcher', 'oyster catcher'), ('pelican',),\n ('king penguin', 'Aptenodytes patagonica'), ('albatross', 'mollymawk'),\n ('grey whale', 'gray whale', 'devilfish', 'Eschrichtius gibbosus', 'Eschrichtius robustus'),\n ('killer whale', 'killer', 'orca', 'grampus', 'sea wolf', 'Orcinusorca'),\n ('dugong', 'Dugong dugon'), ('sea lion',), ('Chihuahua',), ('Japanese spaniel',),\n ('Maltese dog', 'Maltese terrier', 'Maltese'), ('Pekinese', 'Pekingese', 'Peke'),\n ('Shih-Tzu',), ('Blenheim spaniel',), ('papillon',), ('toy terrier',),\n ('Rhodesian ridgeback',), ('Afghan hound', 'Afghan'), ('basset', 'basset hound'), ('beagle',),\n ('bloodhound', 'sleuthhound'), ('bluetick',), ('black-and-tan coonhound',),\n ('Walker hound', 'Walker foxhound'), ('English foxhound',), ('redbone',),\n ('borzoi', 'Russian wolfhound'), ('Irish wolfhound',), ('Italian greyhound',), ('whippet',),\n ('Ibizan hound', 'Ibizan Podenco'), ('Norwegian elkhound', 'elkhound'),\n ('otterhound', 'otter hound'), ('Saluki', 'gazelle hound'),\n ('Scottish deerhound', 'deerhound'), ('Weimaraner',),\n ('Staffordshire bullterrier', 'Staffordshire bull terrier'), (\n 'American Staffordshire terrier', 'Staffordshire terrier', 'American pit bull terrier',\n 'pit bull terrier'), ('Bedlington terrier',), ('Border terrier',),\n ('Kerry blue terrier',),\n ('Irish terrier',), ('Norfolkterrier',), ('Norwich terrier',), ('Yorkshire terrier',),\n ('wire-haired fox terrier',), ('Lakeland terrier',), ('Sealyham terrier', 'Sealyham'),\n ('Airedale', 'Airedale terrier'), ('cairn', 'cairn terrier'), ('Australian terrier',),\n ('Dandie Dinmont', 'Dandie Dinmont terrier'), ('Boston bull', 'Boston terrier'),\n ('miniature schnauzer',), ('giant schnauzer',), ('standard schnauzer',),\n ('Scotch terrier', 'Scottish terrier', 'Scottie'), ('Tibetan terrier', 'chrysanthemum dog'),\n ('silky terrier', 'Sydney silky'), ('soft-coated wheaten terrier',),\n ('West Highland white terrier',), ('Lhasa', 'Lhasa apso'), ('flat-coated retriever',),\n ('curly-coated retriever',), ('golden retriever',), ('Labrador retriever',),\n ('Chesapeake Bay retriever',), ('German short-haired pointer',),\n ('vizsla', 'Hungarian pointer'), ('English setter',), ('Irish setter', 'red setter'),\n ('Gordon setter',), ('Brittany spaniel',), ('clumber', 'clumber spaniel'),\n ('English springer', 'English springer spaniel'), ('Welsh springer spaniel',),\n ('cocker spaniel', 'English cocker spaniel', 'cocker'), ('Sussex spaniel',),\n ('Irish water spaniel',), ('kuvasz',), ('schipperke',), ('groenendael',), ('malinois',),\n ('briard',), ('kelpie',), ('komondor',), ('Old English sheepdog', 'bobtail'),\n ('Shetland sheepdog', 'Shetland sheep dog', 'Shetland'), ('collie',), ('Border collie',),\n ('Bouvier des Flandres', 'Bouviers des Flandres'), ('Rottweiler',),\n ('German shepherd', 'German shepherd dog', 'German police dog', 'alsatian'),\n ('Doberman', 'Doberman pinscher'), ('miniature pinscher',), ('Greater Swiss Mountain dog',),\n ('Bernese mountain dog',), ('Appenzeller',), ('EntleBucher',), ('boxer',), ('bull mastiff',),\n ('Tibetan mastiff',), ('French bulldog',), ('Great Dane',), ('Saint Bernard', 'St Bernard'),\n ('Eskimo dog', 'husky'), ('malamute', 'malemute', 'Alaskan malamute'), ('Siberian husky',),\n ('dalmatian', 'coach dog', 'carriage dog'),\n ('affenpinscher', 'monkey pinscher', 'monkey dog'), ('basenji',), ('pug', 'pug-dog'),\n ('Leonberg',), ('Newfoundland', 'Newfoundland dog'), ('Great Pyrenees',),\n ('Samoyed', 'Samoyede'), ('Pomeranian',), ('chow', 'chow chow'), ('keeshond',),\n ('Brabancon griffon',), ('Pembroke', 'Pembroke Welsh corgi'),\n ('Cardigan', 'Cardigan Welsh corgi'), ('toy poodle',), ('miniature poodle',),\n ('standard poodle',), ('Mexican hairless',),\n ('timber wolf', 'grey wolf', 'gray wolf', 'Canis lupus'),\n ('white wolf', 'Arctic wolf', 'Canis lupus tundrarum'),\n ('red wolf', 'maned wolf', 'Canis rufus', 'Canis niger'),\n ('coyote', 'prairie wolf', 'brush wolf', 'Canis latrans'),\n ('dingo', 'warrigal', 'warragal', 'Canis dingo'), ('dhole', 'Cuon alpinus'),\n ('African hunting dog', 'hyena dog', 'Cape hunting dog', 'Lycaon pictus'),\n ('hyena', 'hyaena'), ('red fox', 'Vulpes vulpes'), ('kit fox', 'Vulpes macrotis'),\n ('Arctic fox', 'white fox', 'Alopex lagopus'),\n ('grey fox', 'gray fox', 'Urocyon cinereoargenteus'), ('tabby', 'tabby cat'), ('tiger cat',),\n ('Persian cat',), ('Siamese cat', 'Siamese'), ('Egyptian cat',),\n ('cougar', 'puma', 'catamount', 'mountain lion', 'painter', 'panther', 'Felis concolor'),\n ('lynx', 'catamount'), ('leopard', 'Panthera pardus'),\n ('snow leopard', 'ounce', 'Panthera uncia'),\n ('jaguar', 'panther', 'Panthera onca', 'Felis onca'),\n ('lion', 'king of beasts', 'Panthera leo'), ('tiger', 'Panthera tigris'),\n ('cheetah', 'chetah', 'Acinonyx jubatus'), ('brown bear', 'bruin', 'Ursus arctos'),\n ('American black bear', 'black bear', 'Ursus americanus', 'Euarctos americanus'),\n ('ice bear', 'polar bear', 'Ursus Maritimus', 'Thalarctos maritimus'),\n ('sloth bear', 'Melursus ursinus', 'Ursus ursinus'), ('mongoose',), ('meerkat', 'mierkat'),\n ('tiger beetle',), ('ladybug', 'ladybeetle', 'lady beetle', 'ladybird', 'ladybird beetle'),\n ('ground beetle', 'carabid beetle'), ('long-horned beetle', 'longicorn', 'longicorn beetle'),\n ('leaf beetle', 'chrysomelid'), ('dung beetle',), ('rhinoceros beetle',), ('weevil',),\n ('fly',), ('bee',), ('ant', 'emmet', 'pismire'), ('grasshopper', 'hopper'), ('cricket',),\n ('walking stick', 'walkingstick', 'stick insect'), ('cockroach', 'roach'),\n ('mantis', 'mantid'), ('cicada', 'cicala'), ('leafhopper',), ('lacewing', 'lacewing fly'), (\n 'dragonfly', 'darning needle', \"devil's darning needle\", 'sewing needle', 'snake feeder',\n 'snake doctor', 'mosquito hawk', 'skeeter hawk'), ('damselfly',), ('admiral',),\n ('ringlet', 'ringlet butterfly'),\n ('monarch', 'monarch butterfly', 'milkweed butterfly', 'Danaus plexippus'),\n ('cabbage butterfly',), ('sulphur butterfly', 'sulfur butterfly'),\n ('lycaenid', 'lycaenid butterfly'), ('starfish', 'sea star'), ('sea urchin',),\n ('sea cucumber', 'holothurian'), ('wood rabbit', 'cottontail', 'cottontail rabbit'),\n ('hare',), ('Angora', 'Angora rabbit'), ('hamster',), ('porcupine', 'hedgehog'),\n ('fox squirrel', 'eastern fox squirrel', 'Sciurus niger'), ('marmot',), ('beaver',),\n ('guinea pig', 'Cavia cobaya'), ('sorrel',), ('zebra',),\n ('hog', 'pig', 'grunter', 'squealer', 'Sus scrofa'), ('wild boar', 'boar', 'Sus scrofa'),\n ('warthog',), ('hippopotamus', 'hippo', 'river horse', 'Hippopotamus amphibius'), ('ox',),\n ('water buffalo', 'water ox', 'Asiatic buffalo', 'Bubalus bubalis'), ('bison',),\n ('ram', 'tup'), (\n 'bighorn', 'bighorn sheep', 'cimarron', 'Rocky Mountain bighorn', 'Rocky Mountain sheep',\n 'Ovis canadensis'), ('ibex', 'Capra ibex'), ('hartebeest',),\n ('impala', 'Aepyceros melampus'),\n ('gazelle',), ('Arabian camel', 'dromedary', 'Camelus dromedarius'), ('llama',), ('weasel',),\n ('mink',), ('polecat', 'fitch', 'foulmart', 'foumart', 'Mustela putorius'),\n ('black-footed ferret', 'ferret', 'Mustela nigripes'), ('otter',),\n ('skunk', 'polecat', 'wood pussy'), ('badger',), ('armadillo',),\n ('three-toed sloth', 'ai', 'Bradypus tridactylus'),\n ('orangutan', 'orang', 'orangutang', 'Pongo pygmaeus'), ('gorilla', 'Gorilla gorilla'),\n ('chimpanzee', 'chimp', 'Pan troglodytes'), ('gibbon', 'Hylobates lar'),\n ('siamang', 'Hylobates syndactylus', 'Symphalangus syndactylus'), ('guenon', 'guenon monkey'),\n ('patas', 'hussar monkey', 'Erythrocebus patas'), ('baboon',), ('macaque',), ('langur',),\n ('colobus', 'colobus monkey'), ('proboscis monkey', 'Nasalis larvatus'), ('marmoset',),\n ('capuchin', 'ringtail', 'Cebus capucinus'), ('howler monkey', 'howler'),\n ('titi', 'titi monkey'), ('spider monkey', 'Ateles geoffroyi'),\n ('squirrel monkey', 'Saimiri sciureus'),\n ('Madagascar cat', 'ring-tailed lemur', 'Lemur catta'),\n ('indri', 'indris', 'Indri indri', 'Indri brevicaudatus'),\n ('Indian elephant', 'Elephas maximus'), ('African elephant', 'Loxodonta africana'),\n ('lesser panda', 'red panda', 'panda', 'bear cat', 'cat bear', 'Ailurus fulgens'),\n ('giant panda', 'panda', 'panda bear', 'coon bear', 'Ailuropoda melanoleuca'),\n ('barracouta', 'snoek'), ('eel',),\n ('coho', 'cohoe', 'coho salmon', 'blue jack', 'silver salmon', 'Oncorhynchus kisutch'),\n ('rock beauty', 'Holocanthus tricolor'), ('anemone fish',), ('sturgeon',),\n ('gar', 'garfish', 'garpike', 'billfish', 'Lepisosteus osseus'), ('lionfish',),\n ('puffer', 'pufferfish', 'blowfish', 'globefish'), ('abacus',), ('abaya',),\n ('academic gown', 'academic robe', \"judge's robe\"),\n ('accordion', 'piano accordion', 'squeeze box'), ('acoustic guitar',),\n ('aircraft carrier', 'carrier', 'flattop', 'attack aircraft carrier'), ('airliner',),\n ('airship', 'dirigible'), ('altar',), ('ambulance',), ('amphibian', 'amphibious vehicle'),\n ('analog clock',), ('apiary', 'bee house'), ('apron',), (\n 'ashcan', 'trash can', 'garbage can', 'wastebin', 'ash bin', 'ash-bin', 'ashbin',\n 'dustbin',\n 'trash barrel', 'trash bin'), ('assault rifle', 'assault gun'),\n ('backpack', 'back pack', 'knapsack', 'packsack', 'rucksack', 'haversack'),\n ('bakery', 'bakeshop', 'bakehouse'), ('balance beam', 'beam'), ('balloon',),\n ('ballpoint', 'ballpoint pen', 'ballpen', 'Biro'), ('Band Aid',), ('banjo',),\n ('bannister', 'banister', 'balustrade', 'balusters', 'handrail'), ('barbell',),\n ('barber chair',), ('barbershop',), ('barn',), ('barometer',), ('barrel', 'cask'),\n ('barrow', 'garden cart', 'lawn cart', 'wheelbarrow'), ('baseball',), ('basketball',),\n ('bassinet',), ('bassoon',), ('bathing cap', 'swimming cap'), ('bath towel',),\n ('bathtub', 'bathing tub', 'bath', 'tub'), (\n 'beach wagon', 'station wagon', 'wagon', 'estate car', 'beach waggon', 'station waggon',\n 'waggon'), ('beacon', 'lighthouse', 'beacon light', 'pharos'), ('beaker',),\n ('bearskin', 'busby', 'shako'), ('beer bottle',), ('beer glass',), ('bell cote', 'bell cot'),\n ('bib',), ('bicycle-built-for-two', 'tandem bicycle', 'tandem'), ('bikini', 'two-piece'),\n ('binder', 'ring-binder'), ('binoculars', 'field glasses', 'opera glasses'), ('birdhouse',),\n ('boathouse',), ('bobsled', 'bobsleigh', 'bob'), ('bolo tie', 'bolo', 'bola tie', 'bola'),\n ('bonnet', 'poke bonnet'), ('bookcase',), ('bookshop', 'bookstore', 'bookstall'),\n ('bottlecap',), ('bow',), ('bow tie', 'bow-tie', 'bowtie'),\n ('brass', 'memorial tablet', 'plaque'), ('brassiere', 'bra', 'bandeau'),\n ('breakwater', 'groin', 'groyne', 'mole', 'bulwark', 'seawall', 'jetty'),\n ('breastplate', 'aegis', 'egis'), ('broom',), ('bucket', 'pail'), ('buckle',),\n ('bulletproof vest',), ('bullet train', 'bullet'), ('butcher shop', 'meat market'),\n ('cab', 'hack', 'taxi', 'taxicab'), ('caldron', 'cauldron'), ('candle', 'taper', 'wax light'),\n ('cannon',), ('canoe',), ('can opener', 'tin opener'), ('cardigan',), ('car mirror',),\n ('carousel', 'carrousel', 'merry-go-round', 'roundabout', 'whirligig'),\n (\"carpenter's kit\", 'tool kit'), ('carton',), ('car wheel',), (\n 'cash machine', 'cash dispenser', 'automated teller machine', 'automatic teller machine',\n 'automated teller', 'automatic teller', 'ATM'), ('cassette',), ('cassette player',),\n ('castle',), ('catamaran',), ('CD player',), ('cello', 'violoncello'),\n ('cellular telephone', 'cellular phone', 'cellphone', 'cell', 'mobile phone'), ('chain',),\n ('chainlink fence',), (\n 'chain mail', 'ring mail', 'mail', 'chain armor', 'chain armour', 'ring armor',\n 'ring armour'), ('chain saw', 'chainsaw'), ('chest',), ('chiffonier', 'commode'),\n ('chime', 'bell', 'gong'), ('china cabinet', 'china closet'), ('Christmas stocking',),\n ('church', 'church building'),\n ('cinema', 'movie theater', 'movie theatre', 'movie house', 'picture palace'),\n ('cleaver', 'meat cleaver', 'chopper'), ('cliff dwelling',), ('cloak',),\n ('clog', 'geta', 'patten', 'sabot'), ('cocktail shaker',), ('coffee mug',), ('coffeepot',),\n ('coil', 'spiral', 'volute', 'whorl', 'helix'), ('combination lock',),\n ('computer keyboard', 'keypad'), ('confectionery', 'confectionary', 'candy store'),\n ('container ship', 'containership', 'container vessel'), ('convertible',),\n ('corkscrew', 'bottle screw'), ('cornet', 'horn', 'trumpet', 'trump'), ('cowboy boot',),\n ('cowboy hat', 'ten-gallon hat'), ('cradle',), ('crane',), ('crash helmet',), ('crate',),\n ('crib', 'cot'), ('Crock Pot',), ('croquet ball',), ('crutch',), ('cuirass',),\n ('dam', 'dike', 'dyke'), ('desk',), ('desktop computer',), ('dial telephone', 'dial phone'),\n ('diaper', 'nappy', 'napkin'), ('digital clock',), ('digital watch',),\n ('dining table', 'board'), ('dishrag', 'dishcloth'),\n ('dishwasher', 'dish washer', 'dishwashing machine'), ('disk brake', 'disc brake'),\n ('dock', 'dockage', 'docking facility'), ('dogsled', 'dog sled', 'dog sleigh'), ('dome',),\n ('doormat', 'welcome mat'), ('drilling platform', 'offshore rig'),\n ('drum', 'membranophone', 'tympan'), ('drumstick',), ('dumbbell',), ('Dutch oven',),\n ('electric fan', 'blower'), ('electric guitar',), ('electric locomotive',),\n ('entertainment center',), ('envelope',), ('espresso maker',), ('face powder',),\n ('feather boa', 'boa'), ('file', 'file cabinet', 'filing cabinet'), ('fireboat',),\n ('fire engine', 'fire truck'), ('fire screen', 'fireguard'), ('flagpole', 'flagstaff'),\n ('flute', 'transverse flute'), ('folding chair',), ('football helmet',), ('forklift',),\n ('fountain',), ('fountain pen',), ('four-poster',), ('freight car',), ('French horn', 'horn'),\n ('frying pan', 'frypan', 'skillet'), ('fur coat',), ('garbage truck', 'dustcart'),\n ('gasmask', 'respirator', 'gas helmet'),\n ('gas pump', 'gasoline pump', 'petrol pump', 'island dispenser'), ('goblet',), ('go-kart',),\n ('golf ball',), ('golfcart', 'golf cart'), ('gondola',), ('gong', 'tam-tam'), ('gown',),\n ('grand piano', 'grand'), ('greenhouse', 'nursery', 'glasshouse'),\n ('grille', 'radiator grille'), ('grocery store', 'grocery', 'food market', 'market'),\n ('guillotine',), ('hair slide',), ('hair spray',), ('half track',), ('hammer',), ('hamper',),\n ('hand blower', 'blow dryer', 'blow drier', 'hair dryer', 'hair drier'),\n ('hand-held computer', 'hand-held microcomputer'),\n ('handkerchief', 'hankie', 'hanky', 'hankey'), ('hard disc', 'hard disk', 'fixed disk'),\n ('harmonica', 'mouth organ', 'harp', 'mouth harp'), ('harp',), ('harvester', 'reaper'),\n ('hatchet',), ('holster',), ('home theater', 'home theatre'), ('honeycomb',),\n ('hook', 'claw'), ('hoopskirt', 'crinoline'), ('horizontal bar', 'high bar'),\n ('horse cart', 'horse-cart'), ('hourglass',), ('iPod',), ('iron', 'smoothing iron'),\n (\"jack-o'-lantern\",), ('jean', 'blue jean', 'denim'), ('jeep', 'landrover'),\n ('jersey', 'T-shirt', 'tee shirt'), ('jigsaw puzzle',), ('jinrikisha', 'ricksha', 'rickshaw'),\n ('joystick',), ('kimono',), ('knee pad',), ('knot',), ('lab coat', 'laboratory coat'),\n ('ladle',), ('lampshade', 'lamp shade'), ('laptop', 'laptop computer'),\n ('lawn mower', 'mower'), ('lens cap', 'lens cover'),\n ('letter opener', 'paper knife', 'paperknife'), ('library',), ('lifeboat',),\n ('lighter', 'light', 'igniter', 'ignitor'), ('limousine', 'limo'), ('liner', 'ocean liner'),\n ('lipstick', 'lip rouge'), ('Loafer',), ('lotion',),\n ('loudspeaker', 'speaker', 'speaker unit', 'loudspeaker system', 'speaker system'),\n ('loupe', \"jeweler's loupe\"), ('lumbermill', 'sawmill'), ('magnetic compass',),\n ('mailbag', 'postbag'), ('mailbox', 'letter box'), ('maillot',), ('maillot', 'tank suit'),\n ('manhole cover',), ('maraca',), ('marimba', 'xylophone'), ('mask',), ('matchstick',),\n ('maypole',), ('maze', 'labyrinth'), ('measuring cup',),\n ('medicine chest', 'medicine cabinet'), ('megalith', 'megalithic structure'),\n ('microphone', 'mike'), ('microwave', 'microwave oven'), ('military uniform',), ('milk can',),\n ('minibus',), ('miniskirt', 'mini'), ('minivan',), ('missile',), ('mitten',),\n ('mixing bowl',), ('mobile home', 'manufactured home'), ('Model T',), ('modem',),\n ('monastery',), ('monitor',), ('moped',), ('mortar',), ('mortarboard',), ('mosque',),\n ('mosquito net',), ('motor scooter', 'scooter'),\n ('mountain bike', 'all-terrain bike', 'off-roader'), ('mountain tent',),\n ('mouse', 'computer mouse'), ('mousetrap',), ('moving van',), ('muzzle',), ('nail',),\n ('neck brace',), ('necklace',), ('nipple',), ('notebook', 'notebook computer'), ('obelisk',),\n ('oboe', 'hautboy', 'hautbois'), ('ocarina', 'sweet potato'),\n ('odometer', 'hodometer', 'mileometer', 'milometer'), ('oil filter',),\n ('organ', 'pipe organ'), ('oscilloscope', 'scope', 'cathode-ray oscilloscope', 'CRO'),\n ('overskirt',), ('oxcart',), ('oxygen mask',), ('packet',), ('paddle', 'boat paddle'),\n ('paddlewheel', 'paddle wheel'), ('padlock',), ('paintbrush',),\n ('pajama', 'pyjama', \"pj's\", 'jammies'), ('palace',), ('panpipe', 'pandean pipe', 'syrinx'),\n ('paper towel',), ('parachute', 'chute'), ('parallel bars', 'bars'), ('park bench',),\n ('parking meter',), ('passenger car', 'coach', 'carriage'), ('patio', 'terrace'),\n ('pay-phone', 'pay-station'), ('pedestal', 'plinth', 'footstall'),\n ('pencil box', 'pencil case'), ('pencil sharpener',), ('perfume', 'essence'), ('Petri dish',),\n ('photocopier',), ('pick', 'plectrum', 'plectron'), ('pickelhaube',),\n ('picket fence', 'paling'), ('pickup', 'pickup truck'), ('pier',),\n ('piggy bank', 'penny bank'), ('pill bottle',), ('pillow',), ('ping-pong ball',),\n ('pinwheel',), ('pirate', 'pirate ship'), ('pitcher', 'ewer'),\n ('plane', \"carpenter's plane\", 'woodworking plane'), ('planetarium',), ('plastic bag',),\n ('plate rack',), ('plow', 'plough'), ('plunger', \"plumber's helper\"),\n ('Polaroid camera', 'Polaroid Land camera'), ('pole',),\n ('police van', 'police wagon', 'paddy wagon', 'patrol wagon', 'wagon', 'black Maria'),\n ('poncho',), ('pool table', 'billiard table', 'snooker table'), ('pop bottle', 'soda bottle'),\n ('pot', 'flowerpot'), (\"potter's wheel\",), ('power drill',), ('prayer rug', 'prayer mat'),\n ('printer',), ('prison', 'prison house'), ('projectile', 'missile'), ('projector',),\n ('puck', 'hockey puck'), ('punching bag', 'punch bag', 'punching ball', 'punchball'),\n ('purse',), ('quill', 'quill pen'), ('quilt', 'comforter', 'comfort', 'puff'),\n ('racer', 'race car', 'racing car'), ('racket', 'racquet'), ('radiator',),\n ('radio', 'wireless'), ('radio telescope', 'radio reflector'), ('rain barrel',),\n ('recreational vehicle', 'RV', 'R.V.'), ('reel',), ('reflex camera',),\n ('refrigerator', 'icebox'), ('remote control', 'remote'),\n ('restaurant', 'eating house', 'eating place', 'eatery'),\n ('revolver', 'six-gun', 'six-shooter'), ('rifle',), ('rocking chair', 'rocker'),\n ('rotisserie',), ('rubber eraser', 'rubber', 'pencil eraser'), ('rugby ball',),\n ('rule', 'ruler'), ('running shoe',), ('safe',), ('safety pin',),\n ('saltshaker', 'salt shaker'), ('sandal',), ('sarong',), ('sax', 'saxophone'), ('scabbard',),\n ('scale', 'weighing machine'), ('school bus',), ('schooner',), ('scoreboard',),\n ('screen', 'CRT screen'), ('screw',), ('screwdriver',), ('seat belt', 'seatbelt'),\n ('sewing machine',), ('shield', 'buckler'), ('shoe shop', 'shoe-shop', 'shoe store'),\n ('shoji',), ('shopping basket',), ('shopping cart',), ('shovel',), ('shower cap',),\n ('shower curtain',), ('ski',), ('ski mask',), ('sleeping bag',), ('slide rule', 'slipstick'),\n ('sliding door',), ('slot', 'one-armed bandit'), ('snorkel',), ('snowmobile',),\n ('snowplow', 'snowplough'), ('soap dispenser',), ('soccer ball',), ('sock',),\n ('solar dish', 'solar collector', 'solar furnace'), ('sombrero',), ('soup bowl',),\n ('space bar',), ('space heater',), ('space shuttle',), ('spatula',), ('speedboat',),\n ('spider web', \"spider's web\"), ('spindle',), ('sports car', 'sport car'),\n ('spotlight', 'spot'), ('stage',), ('steam locomotive',), ('steel arch bridge',),\n ('steel drum',), ('stethoscope',), ('stole',), ('stone wall',), ('stopwatch', 'stop watch'),\n ('stove',), ('strainer',), ('streetcar', 'tram', 'tramcar', 'trolley', 'trolley car'),\n ('stretcher',), ('studio couch', 'day bed'), ('stupa', 'tope'),\n ('submarine', 'pigboat', 'sub', 'U-boat'), ('suit', 'suit of clothes'), ('sundial',),\n ('sunglass',), ('sunglasses', 'dark glasses', 'shades'),\n ('sunscreen', 'sunblock', 'sun blocker'), ('suspension bridge',), ('swab', 'swob', 'mop'),\n ('sweatshirt',), ('swimming trunks', 'bathing trunks'), ('swing',),\n ('switch', 'electric switch', 'electrical switch'), ('syringe',), ('table lamp',),\n ('tank', 'army tank', 'armored combat vehicle', 'armoured combat vehicle'), ('tape player',),\n ('teapot',), ('teddy', 'teddy bear'), ('television', 'television system'), ('tennis ball',),\n ('thatch', 'thatched roof'), ('theater curtain', 'theatre curtain'), ('thimble',),\n ('thresher', 'thrasher', 'threshing machine'), ('throne',), ('tile roof',), ('toaster',),\n ('tobacco shop', 'tobacconist shop', 'tobacconist'), ('toilet seat',), ('torch',),\n ('totem pole',), ('tow truck', 'tow car', 'wrecker'), ('toyshop',), ('tractor',),\n ('trailer truck', 'tractor trailer', 'trucking rig', 'rig', 'articulated lorry', 'semi'),\n ('tray',), ('trench coat',), ('tricycle', 'trike', 'velocipede'), ('trimaran',), ('tripod',),\n ('triumphal arch',), ('trolleybus', 'trolley coach', 'trackless trolley'), ('trombone',),\n ('tub', 'vat'), ('turnstile',), ('typewriter keyboard',), ('umbrella',),\n ('unicycle', 'monocycle'), ('upright', 'upright piano'), ('vacuum', 'vacuum cleaner'),\n ('vase',), ('vault',), ('velvet',), ('vending machine',), ('vestment',), ('viaduct',),\n ('violin', 'fiddle'), ('volleyball',), ('waffle iron',), ('wall clock',),\n ('wallet', 'billfold', 'notecase', 'pocketbook'), ('wardrobe', 'closet', 'press'),\n ('warplane', 'military plane'),\n ('washbasin', 'handbasin', 'washbowl', 'lavabo', 'wash-hand basin'),\n ('washer', 'automatic washer', 'washing machine'), ('water bottle',), ('water jug',),\n ('water tower',), ('whiskey jug',), ('whistle',), ('wig',), ('window screen',),\n ('window shade',), ('Windsor tie',), ('wine bottle',), ('wing',), ('wok',), ('wooden spoon',),\n ('wool', 'woolen', 'woollen'),\n ('worm fence', 'snake fence', 'snake-rail fence', 'Virginia fence'), ('wreck',), ('yawl',),\n ('yurt',), ('web site', 'website', 'internet site', 'site'), ('comic book',),\n ('crossword puzzle', 'crossword'), ('street sign',),\n ('traffic light', 'traffic signal', 'stoplight'),\n ('book jacket', 'dust cover', 'dust jacket', 'dust wrapper'), ('menu',), ('plate',),\n ('guacamole',), ('consomme',), ('hot pot', 'hotpot'), ('trifle',), ('ice cream', 'icecream'),\n ('ice lolly', 'lolly', 'lollipop', 'popsicle'), ('French loaf',), ('bagel', 'beigel'),\n ('pretzel',), ('cheeseburger',), ('hotdog', 'hot dog', 'red hot'), ('mashed potato',),\n ('head cabbage',), ('broccoli',), ('cauliflower',), ('zucchini', 'courgette'),\n ('spaghetti squash',), ('acorn squash',), ('butternut squash',), ('cucumber', 'cuke'),\n ('artichoke', 'globe artichoke'), ('bell pepper',), ('cardoon',), ('mushroom',),\n ('Granny Smith',), ('strawberry',), ('orange',), ('lemon',), ('fig',),\n ('pineapple', 'ananas'), ('banana',), ('jackfruit', 'jak', 'jack'), ('custard apple',),\n ('pomegranate',), ('hay',), ('carbonara',), ('chocolate sauce', 'chocolate syrup'),\n ('dough',), ('meat loaf', 'meatloaf'), ('pizza', 'pizza pie'), ('potpie',), ('burrito',),\n ('red wine',), ('espresso',), ('cup',), ('eggnog',), ('alp',), ('bubble',),\n ('cliff', 'drop', 'drop-off'), ('coral reef',), ('geyser',), ('lakeside', 'lakeshore'),\n ('promontory', 'headland', 'head', 'foreland'), ('sandbar', 'sand bar'),\n ('seashore', 'coast', 'seacoast', 'sea-coast'), ('valley', 'vale'), ('volcano',),\n ('ballplayer', 'baseball player'), ('groom', 'bridegroom'), ('scuba diver',), ('rapeseed',),\n ('daisy',), (\"yellow lady's slipper\", 'yellow lady-slipper', 'Cypripedium calceolus',\n 'Cypripedium parviflorum'), ('corn',), ('acorn',),\n ('hip', 'rose hip', 'rosehip'), ('buckeye', 'horse chestnut', 'conker'), ('coral fungus',),\n ('agaric',), ('gyromitra',), ('stinkhorn', 'carrion fungus'), ('earthstar',),\n ('hen-of-the-woods', 'hen of the woods', 'Polyporus frondosus', 'Grifola frondosa'),\n ('bolete',), ('ear', 'spike', 'capitulum'),\n ('toilet tissue', 'toilet paper', 'bathroom tissue')]]\n\n\nclass ImageNetCaltech(ImageList):\n \"\"\"ImageNet-Caltech is constructed from `Caltech-256 `_ and\n `ImageNet-1K `_ .\n\n They share 84 common classes. ImageNet-Caltech keeps all classes of ImageNet-1K.\n The label is based on the ImageNet-1K (class 0-999) . The private classes of Caltech-256 is discarded.\n\n\n Args:\n root (str): Root directory of dataset\n task (str): The task (domain) to create dataset. Choices include ``'C'``:Caltech-256, \\\n ``'I'``: ImageNet-1K training set.\n download (bool, optional): If true, downloads the dataset from the internet and puts it \\\n in root directory. If dataset is already downloaded, it is not downloaded again.\n transform (callable, optional): A function/transform that takes in an PIL image and returns a \\\n transformed version. E.g, :class:`torchvision.transforms.RandomCrop`.\n target_transform (callable, optional): A function/transform that takes in the target and transforms it.\n\n .. note:: You need to put ``train`` and ``val`` directory of ImageNet-1K manually in `root` directory\n since ImageNet-1K is no longer publicly accessible. DALIB will only download Caltech-256 and ImageList automatically.\n In `root`, there will exist following files after downloading.\n ::\n train/\n n01440764/\n ...\n val/\n 256_ObjectCategories/\n 001.ak47/\n ...\n image_list/\n caltech_256_list.txt\n ...\n \"\"\"\n image_list = {\n \"I\": \"image_list/imagenet_train_1000_list.txt\",\n \"C\": \"image_list/caltech_84_list.txt\",\n }\n CLASSES = _CLASSES\n\n def __init__(self, root: str, task: str, download: Optional[bool] = True, **kwargs):\n assert task in self.image_list\n data_list_file = os.path.join(root, self.image_list[task])\n\n if download:\n list(map(lambda args: download_data(root, *args), download_list))\n else:\n list(map(lambda file_name, _: check_exits(root, file_name), download_list))\n\n if not os.path.exists(os.path.join(root, 'train')):\n print(\"Please put train and val directory of ImageNet-1K manually under {} \"\n \"since ImageNet-1K is no longer publicly accessible.\".format(root))\n exit(-1)\n\n super(ImageNetCaltech, self).__init__(root, ImageNetCaltech.CLASSES, data_list_file=data_list_file, **kwargs)\n\n\nclass ImageNetCaltechUniversal(ImageList):\n \"\"\"ImageNet-Caltech-Universal is constructed from `Caltech-256 `_\n and `ImageNet-1K `_ .\n\n They share 84 common classes. ImageNet-Caltech keeps all classes of ImageNet-1K.\n The label is based on the ImageNet-1K (class 0-999) . The private classes of Caltech-256 is grouped into class 1000 (\"unknown\").\n Thus, ImageNetCaltechUniversal has 1001 classes in total.\n\n\n Args:\n root (str): Root directory of dataset\n task (str): The task (domain) to create dataset. Choices include ``'C'``:Caltech-256, \\\n ``'I'``: ImageNet-1K training set.\n download (bool, optional): If true, downloads the dataset from the internet and puts it \\\n in root directory. If dataset is already downloaded, it is not downloaded again.\n transform (callable, optional): A function/transform that takes in an PIL image and returns a \\\n transformed version. E.g, :class:`torchvision.transforms.RandomCrop`.\n target_transform (callable, optional): A function/transform that takes in the target and transforms it.\n\n .. note:: You need to put ``train`` and ``val`` directory of ImageNet-1K manually in `root` directory\n since ImageNet-1K is no longer publicly accessible. DALIB will only download Caltech-256 and ImageList automatically.\n In `root`, there will exist following files after downloading.\n ::\n train/\n n01440764/\n ...\n val/\n 256_ObjectCategories/\n 001.ak47/\n ...\n image_list/\n caltech_256_list.txt\n ...\n \"\"\"\n image_list = {\n \"I\": \"image_list/imagenet_train_1000_list.txt\",\n \"C\": \"image_list/caltech_85_list.txt\",\n }\n CLASSES = _CLASSES + [\"unknown\"]\n\n def __init__(self, root: str, task: str, download: Optional[bool] = True, **kwargs):\n assert task in self.image_list\n data_list_file = os.path.join(root, self.image_list[task])\n\n if download:\n list(map(lambda args: download_data(root, *args), download_list))\n else:\n list(map(lambda file_name, _: check_exits(root, file_name), download_list))\n\n if not os.path.exists(os.path.join(root, 'train')):\n print(\"Please put train and val directory of ImageNet-1K manually under {} \"\n \"since ImageNet-1K is no longer publicly accessible.\".format(root))\n exit(-1)\n\n super(ImageNetCaltechUniversal, self).__init__(root, ImageNetCaltechUniversal.CLASSES, data_list_file=data_list_file, **kwargs)\n\n\ndownload_list = [\n (\"image_list\", \"image_list.zip\", \"https://cloud.tsinghua.edu.cn/f/a0d7ea37026946f98965/?dl=1\"),\n (\"256_ObjectCategories\", \"256_ObjectCategories.tar\",\n \"http://www.vision.caltech.edu/Image_Datasets/Caltech256/256_ObjectCategories.tar\"),\n]\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/common/vision/datasets/patchcamelyon.py",
"content": "\"\"\"\n@author: Junguang Jiang\n@contact: JiangJunguang1123@outlook.com\n\"\"\"\nimport os\nfrom common.vision.datasets import ImageList\nfrom ._util import download as download_data, check_exits\n\n\nclass PatchCamelyon(ImageList):\n \"\"\"\n The `PatchCamelyon `_ dataset contains \\\n 327680 images of histopathologic scans of lymph node sections. \\\n The classification task consists in predicting the presence of metastatic tissue \\\n in given image (i.e., two classes). All images are 96x96 pixels\n\n Args:\n root (str): Root directory of dataset\n split (str, optional): The dataset split, supports ``train``, or ``test``.\n download (bool, optional): If true, downloads the dataset from the internet and puts it \\\n in root directory. If dataset is already downloaded, it is not downloaded again.\n transform (callable, optional): A function/transform that takes in an PIL image and returns a \\\n transformed version. E.g, :class:`torchvision.transforms.RandomCrop`.\n target_transform (callable, optional): A function/transform that takes in the target and transforms it.\n \"\"\"\n CLASSES = ['0', '1']\n\n def __init__(self, root, split, download=False, **kwargs):\n if download:\n download_data(root, \"patch_camelyon\", \"patch_camelyon.tgz\", \"https://cloud.tsinghua.edu.cn/f/21360b3441a54274b843/?dl=1\")\n else:\n check_exits(root, \"patch_camelyon\")\n\n root = os.path.join(root, \"patch_camelyon\")\n super(PatchCamelyon, self).__init__(root, PatchCamelyon.CLASSES, os.path.join(root, \"imagelist\", \"{}.txt\".format(split)), **kwargs)\n\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/common/vision/datasets/regression/__init__.py",
"content": "from .image_regression import ImageRegression\nfrom .dsprites import DSprites\nfrom .mpi3d import MPI3D"
},
{
"path": "domain_generalization/Transfer-Learning-Library/common/vision/datasets/regression/dsprites.py",
"content": "\"\"\"\n@author: Junguang Jiang\n@contact: JiangJunguang1123@outlook.com\n\"\"\"\nfrom typing import Optional, Sequence\nimport os\nfrom .._util import download as download_data, check_exits\nfrom .image_regression import ImageRegression\n\n\nclass DSprites(ImageRegression):\n \"\"\"`DSprites `_ Dataset.\n\n Args:\n root (str): Root directory of dataset\n task (str): The task (domain) to create dataset. Choices include ``'C'``: Color, \\\n ``'N'``: Noisy and ``'S'``: Scream.\n split (str, optional): The dataset split, supports ``train``, or ``test``.\n factors (sequence[str]): Factors selected. Default: ('scale', 'position x', 'position y').\n download (bool, optional): If true, downloads the dataset from the internet and puts it \\\n in root directory. If dataset is already downloaded, it is not downloaded again.\n transform (callable, optional): A function/transform that takes in an PIL image and returns a \\\n transformed version. E.g, :class:`torchvision.transforms.RandomCrop`.\n target_transform (callable, optional): A function/transform that takes in the target and transforms it.\n\n .. note:: In `root`, there will exist following files after downloading.\n ::\n color/\n ...\n noisy/\n scream/\n image_list/\n color_train.txt\n noisy_train.txt\n scream_train.txt\n color_test.txt\n noisy_test.txt\n scream_test.txt\n \"\"\"\n download_list = [\n (\"image_list\", \"image_list.zip\", \"https://cloud.tsinghua.edu.cn/f/fbbb6b1a43034712b34d/?dl=1\"),\n (\"color\", \"color.tgz\", \"https://cloud.tsinghua.edu.cn/f/9ce9f2abc61f49ed995a/?dl=1\"),\n (\"noisy\", \"noisy.tgz\", \"https://cloud.tsinghua.edu.cn/f/674435c8cb914ca0ad10/?dl=1\"),\n (\"scream\", \"scream.tgz\", \"https://cloud.tsinghua.edu.cn/f/0613675916ac4c3bb6bd/?dl=1\"),\n ]\n image_list = {\n \"C\": \"color\",\n \"N\": \"noisy\",\n \"S\": \"scream\"\n }\n FACTORS = ('none', 'shape', 'scale', 'orientation', 'position x', 'position y')\n\n def __init__(self, root: str, task: str, split: Optional[str] = 'train',\n factors: Sequence[str] = ('scale', 'position x', 'position y'),\n download: Optional[bool] = True, target_transform=None, **kwargs):\n assert task in self.image_list\n assert split in ['train', 'test']\n for factor in factors:\n assert factor in self.FACTORS\n\n factor_index = [self.FACTORS.index(factor) for factor in factors]\n\n if target_transform is None:\n target_transform = lambda x: x[list(factor_index)]\n else:\n target_transform = lambda x: target_transform(x[list(factor_index)])\n\n data_list_file = os.path.join(root, \"image_list\", \"{}_{}.txt\".format(self.image_list[task], split))\n\n if download:\n list(map(lambda args: download_data(root, *args), self.download_list))\n else:\n list(map(lambda file_name, _: check_exits(root, file_name), self.download_list))\n\n super(DSprites, self).__init__(root, factors, data_list_file=data_list_file, target_transform=target_transform, **kwargs)\n\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/common/vision/datasets/regression/image_regression.py",
"content": "\"\"\"\n@author: Junguang Jiang\n@contact: JiangJunguang1123@outlook.com\n\"\"\"\nimport os\nfrom typing import Optional, Callable, Tuple, Any, List, Sequence\nimport torchvision.datasets as datasets\nfrom torchvision.datasets.folder import default_loader\nimport numpy as np\n\n\nclass ImageRegression(datasets.VisionDataset):\n \"\"\"A generic Dataset class for domain adaptation in image regression\n\n Args:\n root (str): Root directory of dataset\n factors (sequence[str]): Factors selected. Default: ('scale', 'position x', 'position y').\n data_list_file (str): File to read the image list from.\n transform (callable, optional): A function/transform that takes in an PIL image and returns a \\\n transformed version. E.g, :class:`torchvision.transforms.RandomCrop`.\n target_transform (callable, optional): A function/transform that takes in the target and transforms it.\n\n .. note::\n In `data_list_file`, each line has `1+len(factors)` values in the following format.\n ::\n source_dir/dog_xxx.png x11, x12, ...\n source_dir/cat_123.png x21, x22, ...\n target_dir/dog_xxy.png x31, x32, ...\n target_dir/cat_nsdf3.png x41, x42, ...\n\n The first value is the relative path of an image, and the rest values are the ground truth of the corresponding factors.\n If your data_list_file has different formats, please over-ride :meth:`ImageRegression.parse_data_file`.\n \"\"\"\n def __init__(self, root: str, factors: Sequence[str], data_list_file: str,\n transform: Optional[Callable] = None, target_transform: Optional[Callable] = None):\n super().__init__(root, transform=transform, target_transform=target_transform)\n self.samples = self.parse_data_file(data_list_file)\n self.factors = factors\n self.loader = default_loader\n self.data_list_file = data_list_file\n\n def __getitem__(self, index: int) -> Tuple[Any, Tuple[float]]:\n \"\"\"\n Args:\n index (int): Index\n\n Returns:\n (image, target) where target is a numpy float array.\n \"\"\"\n path, target = self.samples[index]\n img = self.loader(path)\n if self.transform is not None:\n img = self.transform(img)\n if self.target_transform is not None and target is not None:\n target = self.target_transform(target)\n return img, target\n\n def __len__(self) -> int:\n return len(self.samples)\n\n def parse_data_file(self, file_name: str) -> List[Tuple[str, Any]]:\n \"\"\"Parse file to data list\n\n Args:\n file_name (str): The path of data file\n\n Returns:\n List of (image path, (factors)) tuples\n \"\"\"\n with open(file_name, \"r\") as f:\n data_list = []\n for line in f.readlines():\n data = line.split()\n path = str(data[0])\n target = np.array([float(d) for d in data[1:]], dtype=np.float)\n if not os.path.isabs(path):\n path = os.path.join(self.root, path)\n data_list.append((path, target))\n return data_list\n\n @property\n def num_factors(self) -> int:\n return len(self.factors)"
},
{
"path": "domain_generalization/Transfer-Learning-Library/common/vision/datasets/regression/mpi3d.py",
"content": "\"\"\"\n@author: Junguang Jiang\n@contact: JiangJunguang1123@outlook.com\n\"\"\"\nfrom typing import Optional, Sequence\nimport os\nfrom .._util import download as download_data, check_exits\nfrom .image_regression import ImageRegression\n\n\nclass MPI3D(ImageRegression):\n \"\"\"`MPI3D `_ Dataset.\n\n Args:\n root (str): Root directory of dataset\n task (str): The task (domain) to create dataset. Choices include ``'C'``: Color, \\\n ``'N'``: Noisy and ``'S'``: Scream.\n split (str, optional): The dataset split, supports ``train``, or ``test``.\n factors (sequence[str]): Factors selected. Default: ('horizontal axis', 'vertical axis').\n download (bool, optional): If true, downloads the dataset from the internet and puts it \\\n in root directory. If dataset is already downloaded, it is not downloaded again.\n transform (callable, optional): A function/transform that takes in an PIL image and returns a \\\n transformed version. E.g, :class:`torchvision.transforms.RandomCrop`.\n target_transform (callable, optional): A function/transform that takes in the target and transforms it.\n\n .. note:: In `root`, there will exist following files after downloading.\n ::\n real/\n ...\n realistic/\n toy/\n image_list/\n real_train.txt\n realistic_train.txt\n toy_train.txt\n real_test.txt\n realistic_test.txt\n toy_test.txt\n \"\"\"\n download_list = [\n (\"image_list\", \"image_list.zip\", \"https://cloud.tsinghua.edu.cn/f/f0ff24df967b42479d9e/?dl=1\"),\n (\"real\", \"real.tgz\", \"https://cloud.tsinghua.edu.cn/f/04c1318555fc4283862b/?dl=1\"),\n (\"realistic\", \"realistic.tgz\", \"https://cloud.tsinghua.edu.cn/f/2c0f7dacc73148cea593/?dl=1\"),\n (\"toy\", \"toy.tgz\", \"https://cloud.tsinghua.edu.cn/f/6327912a50374e20af95/?dl=1\"),\n ]\n image_list = {\n \"RL\": \"real\",\n \"RC\": \"realistic\",\n \"T\": \"toy\"\n }\n FACTORS = ('horizontal axis', 'vertical axis')\n\n def __init__(self, root: str, task: str, split: Optional[str] = 'train',\n factors: Sequence[str] = ('horizontal axis', 'vertical axis'),\n download: Optional[bool] = True, target_transform=None, **kwargs):\n assert task in self.image_list\n assert split in ['train', 'test']\n for factor in factors:\n assert factor in self.FACTORS\n\n factor_index = [self.FACTORS.index(factor) for factor in factors]\n\n if target_transform is None:\n target_transform = lambda x: x[list(factor_index)] / 39.\n else:\n target_transform = lambda x: target_transform(x[list(factor_index)]) / 39.\n\n data_list_file = os.path.join(root, \"image_list\", \"{}_{}.txt\".format(self.image_list[task], split))\n\n if download:\n list(map(lambda args: download_data(root, *args), self.download_list))\n else:\n list(map(lambda file_name, _: check_exits(root, file_name), self.download_list))\n\n super(MPI3D, self).__init__(root, factors, data_list_file=data_list_file, target_transform=target_transform, **kwargs)\n\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/common/vision/datasets/reid/__init__.py",
"content": "from .market1501 import Market1501\nfrom .dukemtmc import DukeMTMC\nfrom .msmt17 import MSMT17\nfrom .personx import PersonX\nfrom .unreal import UnrealPerson\n\n__all__ = ['Market1501', 'DukeMTMC', 'MSMT17', 'PersonX', 'UnrealPerson']\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/common/vision/datasets/reid/basedataset.py",
"content": "\"\"\"\nModified from https://github.com/yxgeee/MMT\n@author: Baixu Chen\n@contact: cbx_99_hasta@outlook.com\n\"\"\"\nimport os.path as osp\nimport numpy as np\n\n\nclass BaseDataset(object):\n \"\"\"\n Base class of reid dataset\n \"\"\"\n\n def get_imagedata_info(self, data):\n pids, cams = [], []\n for _, pid, camid in data:\n pids += [pid]\n cams += [camid]\n pids = set(pids)\n cams = set(cams)\n num_pids = len(pids)\n num_cams = len(cams)\n num_imgs = len(data)\n return num_pids, num_imgs, num_cams\n\n def get_videodata_info(self, data, return_tracklet_stats=False):\n pids, cams, tracklet_stats = [], [], []\n for img_paths, pid, camid in data:\n pids += [pid]\n cams += [camid]\n tracklet_stats += [len(img_paths)]\n pids = set(pids)\n cams = set(cams)\n num_pids = len(pids)\n num_cams = len(cams)\n num_tracklets = len(data)\n if return_tracklet_stats:\n return num_pids, num_tracklets, num_cams, tracklet_stats\n return num_pids, num_tracklets, num_cams\n\n def print_dataset_statistics(self, train, query, galler):\n raise NotImplementedError\n\n def check_before_run(self, required_files):\n \"\"\"Checks if required files exist before going deeper.\n Args:\n required_files (str or list): string file name(s).\n \"\"\"\n if isinstance(required_files, str):\n required_files = [required_files]\n\n for fpath in required_files:\n if not osp.exists(fpath):\n raise RuntimeError('\"{}\" is not found'.format(fpath))\n\n @property\n def images_dir(self):\n return None\n\n\nclass BaseImageDataset(BaseDataset):\n \"\"\"\n Base class of image reid dataset\n \"\"\"\n\n def print_dataset_statistics(self, train, query, gallery):\n num_train_pids, num_train_imgs, num_train_cams = self.get_imagedata_info(train)\n num_query_pids, num_query_imgs, num_query_cams = self.get_imagedata_info(query)\n num_gallery_pids, num_gallery_imgs, num_gallery_cams = self.get_imagedata_info(gallery)\n\n print(\"Dataset statistics:\")\n print(\" ----------------------------------------\")\n print(\" subset | # ids | # images | # cameras\")\n print(\" ----------------------------------------\")\n print(\" train | {:5d} | {:8d} | {:9d}\".format(num_train_pids, num_train_imgs, num_train_cams))\n print(\" query | {:5d} | {:8d} | {:9d}\".format(num_query_pids, num_query_imgs, num_query_cams))\n print(\" gallery | {:5d} | {:8d} | {:9d}\".format(num_gallery_pids, num_gallery_imgs, num_gallery_cams))\n print(\" ----------------------------------------\")\n\n\nclass BaseVideoDataset(BaseDataset):\n \"\"\"\n Base class of video reid dataset\n \"\"\"\n\n def print_dataset_statistics(self, train, query, gallery):\n num_train_pids, num_train_tracklets, num_train_cams, train_tracklet_stats = \\\n self.get_videodata_info(train, return_tracklet_stats=True)\n\n num_query_pids, num_query_tracklets, num_query_cams, query_tracklet_stats = \\\n self.get_videodata_info(query, return_tracklet_stats=True)\n\n num_gallery_pids, num_gallery_tracklets, num_gallery_cams, gallery_tracklet_stats = \\\n self.get_videodata_info(gallery, return_tracklet_stats=True)\n\n tracklet_stats = train_tracklet_stats + query_tracklet_stats + gallery_tracklet_stats\n min_num = np.min(tracklet_stats)\n max_num = np.max(tracklet_stats)\n avg_num = np.mean(tracklet_stats)\n\n print(\"Dataset statistics:\")\n print(\" -------------------------------------------\")\n print(\" subset | # ids | # tracklets | # cameras\")\n print(\" -------------------------------------------\")\n print(\" train | {:5d} | {:11d} | {:9d}\".format(num_train_pids, num_train_tracklets, num_train_cams))\n print(\" query | {:5d} | {:11d} | {:9d}\".format(num_query_pids, num_query_tracklets, num_query_cams))\n print(\" gallery | {:5d} | {:11d} | {:9d}\".format(num_gallery_pids, num_gallery_tracklets, num_gallery_cams))\n print(\" -------------------------------------------\")\n print(\" number of images per tracklet: {} ~ {}, average {:.2f}\".format(min_num, max_num, avg_num))\n print(\" -------------------------------------------\")\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/common/vision/datasets/reid/convert.py",
"content": "\"\"\"\n@author: Baixu Chen\n@contact: cbx_99_hasta@outlook.com\n\"\"\"\nimport os.path as osp\nfrom torch.utils.data import Dataset\nfrom PIL import Image\n\n\ndef convert_to_pytorch_dataset(dataset, root=None, transform=None, return_idxes=False):\n class ReidDataset(Dataset):\n def __init__(self, dataset, root, transform):\n super(ReidDataset, self).__init__()\n self.dataset = dataset\n self.root = root\n self.transform = transform\n self.return_idxes = return_idxes\n\n def __len__(self):\n return len(self.dataset)\n\n def __getitem__(self, index):\n fname, pid, cid = self.dataset[index]\n fpath = fname\n if self.root is not None:\n fpath = osp.join(self.root, fname)\n\n img = Image.open(fpath).convert('RGB')\n\n if self.transform is not None:\n img = self.transform(img)\n\n if not self.return_idxes:\n return img, fname, pid, cid\n else:\n return img, fname, pid, cid, index\n\n return ReidDataset(dataset, root, transform)\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/common/vision/datasets/reid/dukemtmc.py",
"content": "\"\"\"\n@author: Baixu Chen\n@contact: cbx_99_hasta@outlook.com\n\"\"\"\nfrom .basedataset import BaseImageDataset\nfrom typing import Callable\nfrom PIL import Image\nimport os\nimport os.path as osp\nimport glob\nimport re\nfrom common.vision.datasets._util import download\n\n\nclass DukeMTMC(BaseImageDataset):\n \"\"\"DukeMTMC-reID dataset from `Performance Measures and a Data Set for Multi-Target, Multi-Camera Tracking\n (ECCV 2016) `_.\n\n Dataset statistics:\n - identities: 1404 (train + query)\n - images:16522 (train) + 2228 (query) + 17661 (gallery)\n - cameras: 8\n\n Args:\n root (str): Root directory of dataset\n verbose (bool, optional): If true, print dataset statistics after loading the dataset. Default: True\n \"\"\"\n dataset_dir = '.'\n archive_name = 'DukeMTMC-reID.tgz'\n dataset_url = 'https://cloud.tsinghua.edu.cn/f/89f1edaf0f83434f8070/?dl=1'\n\n def __init__(self, root, verbose=True):\n super(DukeMTMC, self).__init__()\n download(root, self.dataset_dir, self.archive_name, self.dataset_url)\n self.relative_dataset_dir = self.dataset_dir\n self.dataset_dir = osp.join(root, self.dataset_dir)\n self.train_dir = osp.join(self.dataset_dir, 'DukeMTMC-reID/bounding_box_train')\n self.query_dir = osp.join(self.dataset_dir, 'DukeMTMC-reID/query')\n self.gallery_dir = osp.join(self.dataset_dir, 'DukeMTMC-reID/bounding_box_test')\n\n required_files = [self.dataset_dir, self.train_dir, self.query_dir, self.gallery_dir]\n self.check_before_run(required_files)\n\n train = self.process_dir(self.train_dir, relabel=True)\n query = self.process_dir(self.query_dir, relabel=False)\n gallery = self.process_dir(self.gallery_dir, relabel=False)\n\n if verbose:\n print(\"=> DukeMTMC-reID loaded\")\n self.print_dataset_statistics(train, query, gallery)\n\n self.train = train\n self.query = query\n self.gallery = gallery\n\n self.num_train_pids, self.num_train_imgs, self.num_train_cams = self.get_imagedata_info(self.train)\n self.num_query_pids, self.num_query_imgs, self.num_query_cams = self.get_imagedata_info(self.query)\n self.num_gallery_pids, self.num_gallery_imgs, self.num_gallery_cams = self.get_imagedata_info(self.gallery)\n\n def process_dir(self, dir_path, relabel=False):\n img_paths = glob.glob(osp.join(dir_path, '*.jpg'))\n pattern = re.compile(r'([-\\d]+)_c(\\d)')\n\n pid_container = set()\n for img_path in img_paths:\n pid, _ = map(int, pattern.search(img_path).groups())\n pid_container.add(pid)\n pid2label = {pid: label for label, pid in enumerate(pid_container)}\n\n dataset = []\n for img_path in img_paths:\n pid, cid = map(int, pattern.search(img_path).groups())\n assert 1 <= cid <= 8\n cid -= 1 # index starts from 0\n if relabel:\n pid = pid2label[pid]\n dataset.append((img_path, pid, cid))\n\n return dataset\n\n def translate(self, transform: Callable, target_root: str):\n \"\"\" Translate an image and save it into a specified directory\n\n Args:\n transform (callable): a transform function that maps images from one domain to another domain\n target_root (str): the root directory to save images\n\n \"\"\"\n os.makedirs(target_root, exist_ok=True)\n translated_dataset_dir = osp.join(target_root, self.relative_dataset_dir)\n\n translated_train_dir = osp.join(translated_dataset_dir, 'DukeMTMC-reID/bounding_box_train')\n translated_query_dir = osp.join(translated_dataset_dir, 'DukeMTMC-reID/query')\n translated_gallery_dir = osp.join(translated_dataset_dir, 'DukeMTMC-reID/bounding_box_test')\n\n print(\"Translating dataset with image to image transform...\")\n self.translate_dir(transform, self.train_dir, translated_train_dir)\n self.translate_dir(None, self.query_dir, translated_query_dir)\n self.translate_dir(None, self.gallery_dir, translated_gallery_dir)\n print(\"Translation process is done, save dataset to {}\".format(translated_dataset_dir))\n\n def translate_dir(self, transform, origin_dir: str, target_dir: str):\n image_list = os.listdir(origin_dir)\n for image_name in image_list:\n if not image_name.endswith(\".jpg\"):\n continue\n image_path = osp.join(origin_dir, image_name)\n image = Image.open(image_path)\n translated_image_path = osp.join(target_dir, image_name)\n translated_image = image\n if transform:\n translated_image = transform(image)\n\n os.makedirs(os.path.dirname(translated_image_path), exist_ok=True)\n translated_image.save(translated_image_path)\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/common/vision/datasets/reid/market1501.py",
"content": "\"\"\"\n@author: Baixu Chen\n@contact: cbx_99_hasta@outlook.com\n\"\"\"\nfrom .basedataset import BaseImageDataset\nfrom typing import Callable\nfrom PIL import Image\nimport os\nimport os.path as osp\nimport glob\nimport re\nfrom common.vision.datasets._util import download\n\n\nclass Market1501(BaseImageDataset):\n \"\"\"Market1501 dataset from `Scalable Person Re-identification: A Benchmark (ICCV 2015)\n `_.\n\n Dataset statistics:\n - identities: 1501 (+1 for background)\n - images: 12936 (train) + 3368 (query) + 15913 (gallery)\n - cameras: 6\n\n Args:\n root (str): Root directory of dataset\n verbose (bool, optional): If true, print dataset statistics after loading the dataset. Default: True\n \"\"\"\n dataset_dir = 'Market-1501-v15.09.15'\n archive_name = 'Market-1501-v15.09.15.tgz'\n dataset_url = 'https://cloud.tsinghua.edu.cn/f/aad1f51c221f40c7b2a7/?dl=1'\n\n def __init__(self, root, verbose=True):\n super(Market1501, self).__init__()\n download(root, self.dataset_dir, self.archive_name, self.dataset_url)\n self.relative_dataset_dir = self.dataset_dir\n self.dataset_dir = osp.join(root, self.dataset_dir)\n self.train_dir = osp.join(self.dataset_dir, 'bounding_box_train')\n self.query_dir = osp.join(self.dataset_dir, 'query')\n self.gallery_dir = osp.join(self.dataset_dir, 'bounding_box_test')\n\n required_files = [self.dataset_dir, self.train_dir, self.query_dir, self.gallery_dir]\n self.check_before_run(required_files)\n\n train = self.process_dir(self.train_dir, relabel=True)\n query = self.process_dir(self.query_dir, relabel=False)\n gallery = self.process_dir(self.gallery_dir, relabel=False)\n\n if verbose:\n print(\"=> Market1501 loaded\")\n self.print_dataset_statistics(train, query, gallery)\n\n self.train = train\n self.query = query\n self.gallery = gallery\n\n self.num_train_pids, self.num_train_imgs, self.num_train_cams = self.get_imagedata_info(self.train)\n self.num_query_pids, self.num_query_imgs, self.num_query_cams = self.get_imagedata_info(self.query)\n self.num_gallery_pids, self.num_gallery_imgs, self.num_gallery_cams = self.get_imagedata_info(self.gallery)\n\n def process_dir(self, dir_path, relabel=False):\n img_paths = glob.glob(osp.join(dir_path, '*.jpg'))\n pattern = re.compile(r'([-\\d]+)_c(\\d)')\n\n pid_container = set()\n for img_path in img_paths:\n pid, _ = map(int, pattern.search(img_path).groups())\n if pid == -1:\n continue # junk images are just ignored\n pid_container.add(pid)\n pid2label = {pid: label for label, pid in enumerate(pid_container)}\n\n dataset = []\n for img_path in img_paths:\n pid, cid = map(int, pattern.search(img_path).groups())\n if pid == -1:\n continue # junk images are just ignored\n assert 0 <= pid <= 1501 # pid == 0 means background\n assert 1 <= cid <= 6\n cid -= 1 # index starts from 0\n if relabel:\n pid = pid2label[pid]\n dataset.append((img_path, pid, cid))\n\n return dataset\n\n def translate(self, transform: Callable, target_root: str):\n \"\"\" Translate an image and save it into a specified directory\n\n Args:\n transform (callable): a transform function that maps images from one domain to another domain\n target_root (str): the root directory to save images\n\n \"\"\"\n os.makedirs(target_root, exist_ok=True)\n translated_dataset_dir = osp.join(target_root, self.relative_dataset_dir)\n\n translated_train_dir = osp.join(translated_dataset_dir, 'bounding_box_train')\n translated_query_dir = osp.join(translated_dataset_dir, 'query')\n translated_gallery_dir = osp.join(translated_dataset_dir, 'bounding_box_test')\n\n print(\"Translating dataset with image to image transform...\")\n self.translate_dir(transform, self.train_dir, translated_train_dir)\n self.translate_dir(None, self.query_dir, translated_query_dir)\n self.translate_dir(None, self.gallery_dir, translated_gallery_dir)\n print(\"Translation process is done, save dataset to {}\".format(translated_dataset_dir))\n\n def translate_dir(self, transform, origin_dir: str, target_dir: str):\n image_list = os.listdir(origin_dir)\n for image_name in image_list:\n if not image_name.endswith(\".jpg\"):\n continue\n image_path = osp.join(origin_dir, image_name)\n image = Image.open(image_path)\n translated_image_path = osp.join(target_dir, image_name)\n translated_image = image\n if transform:\n translated_image = transform(image)\n\n os.makedirs(os.path.dirname(translated_image_path), exist_ok=True)\n translated_image.save(translated_image_path)\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/common/vision/datasets/reid/msmt17.py",
"content": "\"\"\"\n@author: Baixu Chen\n@contact: cbx_99_hasta@outlook.com\n\"\"\"\nfrom .basedataset import BaseImageDataset\nfrom typing import Callable\nfrom PIL import Image\nimport os\nimport os.path as osp\nfrom common.vision.datasets._util import download\n\n\nclass MSMT17(BaseImageDataset):\n \"\"\"MSMT17 dataset from `Person Transfer GAN to Bridge Domain Gap for Person Re-Identification (CVPR 2018)\n `_.\n\n Dataset statistics:\n - identities: 4101\n - images: 32621 (train) + 11659 (query) + 82161 (gallery)\n - cameras: 15\n\n Args:\n root (str): Root directory of dataset\n verbose (bool, optional): If true, print dataset statistics after loading the dataset. Default: True\n \"\"\"\n dataset_dir = '.'\n archive_name = 'MSMT17_V1.zip'\n dataset_url = 'https://cloud.tsinghua.edu.cn/f/55d7e5aa3c224f49b908/?dl=1'\n\n def __init__(self, root, verbose=True):\n super(MSMT17, self).__init__()\n download(root, self.dataset_dir, self.archive_name, self.dataset_url)\n self.relative_dataset_dir = self.dataset_dir\n self.dataset_dir = osp.join(root, self.dataset_dir)\n self.train_dir = osp.join(self.dataset_dir, 'bounding_box_train')\n self.query_dir = osp.join(self.dataset_dir, 'query')\n self.gallery_dir = osp.join(self.dataset_dir, 'bounding_box_test')\n\n required_files = [self.dataset_dir, self.train_dir, self.query_dir, self.gallery_dir]\n self.check_before_run(required_files)\n\n self.train = self.process_dir(self.train_dir)\n self.query = self.process_dir(self.query_dir)\n self.gallery = self.process_dir(self.gallery_dir)\n if verbose:\n print(\"=> MSMT17 loaded\")\n self.print_dataset_statistics(self.train, self.query, self.gallery)\n\n self.num_train_pids, self.num_train_imgs, self.num_train_cams = self.get_imagedata_info(self.train)\n self.num_query_pids, self.num_query_imgs, self.num_query_cams = self.get_imagedata_info(self.query)\n self.num_gallery_pids, self.num_gallery_imgs, self.num_gallery_cams = self.get_imagedata_info(self.gallery)\n\n def process_dir(self, dir_path):\n image_list = os.listdir(dir_path)\n dataset = []\n pid_container = set()\n\n for image_path in image_list:\n pid, cid, _ = image_path.split('_')\n pid = int(pid)\n cid = int(cid[1:]) - 1 # index starts from 0\n full_image_path = osp.join(dir_path, image_path)\n dataset.append((full_image_path, pid, cid))\n pid_container.add(pid)\n\n # check if pid starts from 0 and increments with 1\n for idx, pid in enumerate(pid_container):\n assert idx == pid, \"See code comment for explanation\"\n return dataset\n\n def translate(self, transform: Callable, target_root: str):\n \"\"\" Translate an image and save it into a specified directory\n\n Args:\n transform (callable): a transform function that maps images from one domain to another domain\n target_root (str): the root directory to save images\n\n \"\"\"\n os.makedirs(target_root, exist_ok=True)\n translated_dataset_dir = osp.join(target_root, self.relative_dataset_dir)\n\n translated_train_dir = osp.join(translated_dataset_dir, 'bounding_box_train')\n translated_query_dir = osp.join(translated_dataset_dir, 'query')\n translated_gallery_dir = osp.join(translated_dataset_dir, 'bounding_box_test')\n\n print(\"Translating dataset with image to image transform...\")\n self.translate_dir(transform, self.train_dir, translated_train_dir)\n self.translate_dir(None, self.query_dir, translated_query_dir)\n self.translate_dir(None, self.gallery_dir, translated_gallery_dir)\n print(\"Translation process is done, save dataset to {}\".format(translated_dataset_dir))\n\n def translate_dir(self, transform, origin_dir: str, target_dir: str):\n image_list = os.listdir(origin_dir)\n for image_name in image_list:\n if not image_name.endswith(\".jpg\"):\n continue\n image_path = osp.join(origin_dir, image_name)\n image = Image.open(image_path)\n translated_image_path = osp.join(target_dir, image_name)\n translated_image = image\n if transform:\n translated_image = transform(image)\n\n os.makedirs(os.path.dirname(translated_image_path), exist_ok=True)\n translated_image.save(translated_image_path)\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/common/vision/datasets/reid/personx.py",
"content": "\"\"\"\nModified from https://github.com/yxgeee/SpCL\n@author: Baixu Chen\n@contact: cbx_99_hasta@outlook.com\n\"\"\"\nfrom .basedataset import BaseImageDataset\nfrom typing import Callable\nfrom PIL import Image\nimport os\nimport os.path as osp\nimport glob\nimport re\nfrom common.vision.datasets._util import download\n\n\nclass PersonX(BaseImageDataset):\n \"\"\"PersonX dataset from `Dissecting Person Re-identification from the Viewpoint of Viewpoint (CVPR 2019)\n `_.\n\n Dataset statistics:\n - identities: 1266\n - images: 9840 (train) + 5136 (query) + 30816 (gallery)\n - cameras: 6\n\n Args:\n root (str): Root directory of dataset\n verbose (bool, optional): If true, print dataset statistics after loading the dataset. Default: True\n \"\"\"\n dataset_dir = '.'\n archive_name = 'PersonX.zip'\n dataset_url = 'https://cloud.tsinghua.edu.cn/f/f506cd11d6b646729bd1/?dl=1'\n\n def __init__(self, root, verbose=True):\n super(PersonX, self).__init__()\n download(root, self.dataset_dir, self.archive_name, self.dataset_url)\n self.relative_dataset_dir = self.dataset_dir\n self.dataset_dir = osp.join(root, self.dataset_dir)\n self.train_dir = osp.join(self.dataset_dir, 'bounding_box_train')\n self.query_dir = osp.join(self.dataset_dir, 'query')\n self.gallery_dir = osp.join(self.dataset_dir, 'bounding_box_test')\n\n required_files = [self.dataset_dir, self.train_dir, self.query_dir, self.gallery_dir]\n self.check_before_run(required_files)\n\n train = self.process_dir(self.train_dir, relabel=True)\n query = self.process_dir(self.query_dir, relabel=False)\n gallery = self.process_dir(self.gallery_dir, relabel=False)\n\n if verbose:\n print(\"=> PersonX loaded\")\n self.print_dataset_statistics(train, query, gallery)\n\n self.train = train\n self.query = query\n self.gallery = gallery\n\n self.num_train_pids, self.num_train_imgs, self.num_train_cams = self.get_imagedata_info(self.train)\n self.num_query_pids, self.num_query_imgs, self.num_query_cams = self.get_imagedata_info(self.query)\n self.num_gallery_pids, self.num_gallery_imgs, self.num_gallery_cams = self.get_imagedata_info(self.gallery)\n\n def process_dir(self, dir_path, relabel=False):\n img_paths = glob.glob(osp.join(dir_path, '*.jpg'))\n pattern = re.compile(r'([-\\d]+)_c([-\\d]+)')\n cam2label = {3: 1, 4: 2, 8: 3, 10: 4, 11: 5, 12: 6}\n\n pid_container = set()\n for img_path in img_paths:\n pid, _ = map(int, pattern.search(img_path).groups())\n pid_container.add(pid)\n pid2label = {pid: label for label, pid in enumerate(pid_container)}\n\n dataset = []\n for img_path in img_paths:\n pid, cid = map(int, pattern.search(img_path).groups())\n assert (cid in cam2label.keys())\n cid = cam2label[cid]\n cid -= 1 # index starts from 0\n if relabel:\n pid = pid2label[pid]\n dataset.append((img_path, pid, cid))\n\n return dataset\n\n def translate(self, transform: Callable, target_root: str):\n \"\"\" Translate an image and save it into a specified directory\n\n Args:\n transform (callable): a transform function that maps images from one domain to another domain\n target_root (str): the root directory to save images\n\n \"\"\"\n os.makedirs(target_root, exist_ok=True)\n translated_dataset_dir = osp.join(target_root, self.relative_dataset_dir)\n\n translated_train_dir = osp.join(translated_dataset_dir, 'bounding_box_train')\n translated_query_dir = osp.join(translated_dataset_dir, 'query')\n translated_gallery_dir = osp.join(translated_dataset_dir, 'bounding_box_test')\n\n print(\"Translating dataset with image to image transform...\")\n self.translate_dir(transform, self.train_dir, translated_train_dir)\n self.translate_dir(None, self.query_dir, translated_query_dir)\n self.translate_dir(None, self.gallery_dir, translated_gallery_dir)\n print(\"Translation process is done, save dataset to {}\".format(translated_dataset_dir))\n\n def translate_dir(self, transform, origin_dir: str, target_dir: str):\n image_list = os.listdir(origin_dir)\n for image_name in image_list:\n if not image_name.endswith(\".jpg\"):\n continue\n image_path = osp.join(origin_dir, image_name)\n image = Image.open(image_path)\n translated_image_path = osp.join(target_dir, image_name)\n translated_image = image\n if transform:\n translated_image = transform(image)\n\n os.makedirs(os.path.dirname(translated_image_path), exist_ok=True)\n translated_image.save(translated_image_path)\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/common/vision/datasets/reid/unreal.py",
"content": "\"\"\"\nModified from https://github.com/SikaStar/IDM\n@author: Baixu Chen\n@contact: cbx_99_hasta@outlook.com\n\"\"\"\nfrom .basedataset import BaseImageDataset\nfrom typing import Callable\nimport os.path as osp\nfrom common.vision.datasets._util import download\n\n\nclass UnrealPerson(BaseImageDataset):\n \"\"\"UnrealPerson dataset from `UnrealPerson: An Adaptive Pipeline towards Costless Person Re-identification\n (CVPR 2021) `_.\n\n Dataset statistics:\n - identities: 3000\n - images: 120,000\n - cameras: 34\n\n Args:\n root (str): Root directory of dataset\n verbose (bool, optional): If true, print dataset statistics after loading the dataset. Default: True\n \"\"\"\n dataset_dir = '.'\n download_list = [\n (\"list_unreal_train.txt\", \"image_list.zip\", \"https://cloud.tsinghua.edu.cn/f/a51b22fd760743e7bca6/?dl=1\"),\n (\"unreal_v1.1\", \"unreal_v1.1.tar\", \"https://cloud.tsinghua.edu.cn/f/a8806bb3bf1744dda5b1/?dl=1\"),\n (\"unreal_v1.2\", \"unreal_v1.2.tar\", \"https://cloud.tsinghua.edu.cn/f/449224485a654c5baa8f/?dl=1\"),\n (\"unreal_v1.3\", \"unreal_v1.3.tar\", \"https://cloud.tsinghua.edu.cn/f/069f3162f74849c09c10/?dl=1\"),\n (\"unreal_v2.1\", \"unreal_v2.1.tar\", \"https://cloud.tsinghua.edu.cn/f/a791aaa42674466eb183/?dl=1\"),\n (\"unreal_v2.2\", \"unreal_v2.2.tar\", \"https://cloud.tsinghua.edu.cn/f/b601d9f54f964248bd0e/?dl=1\"),\n (\"unreal_v2.3\", \"unreal_v2.3.tar\", \"https://cloud.tsinghua.edu.cn/f/311ec60e810b42d48d12/?dl=1\"),\n (\"unreal_v3.1\", \"unreal_v3.1.tar\", \"https://cloud.tsinghua.edu.cn/f/d51b7c1d125e4632bcf9/?dl=1\"),\n (\"unreal_v3.2\", \"unreal_v3.2.tar\", \"https://cloud.tsinghua.edu.cn/f/4efbd969ea2f4e8197e8/?dl=1\"),\n (\"unreal_v3.3\", \"unreal_v3.3.tar\", \"https://cloud.tsinghua.edu.cn/f/a3cc3d9c460247848fb7/?dl=1\"),\n (\"unreal_v4.1\", \"unreal_v4.1.tar\", \"https://cloud.tsinghua.edu.cn/f/ca05183ac9cd4be5a53b/?dl=1\"),\n (\"unreal_v4.2\", \"unreal_v4.2.tar\", \"https://cloud.tsinghua.edu.cn/f/b90722cbd754496f9f40/?dl=1\"),\n (\"unreal_v4.3\", \"unreal_v4.3.tar\", \"https://cloud.tsinghua.edu.cn/f/547ae646c3d346038297/?dl=1\"),\n ]\n\n def __init__(self, root, verbose=True):\n super(UnrealPerson, self).__init__()\n list(map(lambda args: download(root, *args), self.download_list))\n self.dataset_dir = osp.join(root, self.dataset_dir)\n self.train_list = osp.join(self.dataset_dir, 'list_unreal_train.txt')\n\n required_files = [self.dataset_dir, self.train_list]\n self.check_before_run(required_files)\n\n train = self.process_dir(self.train_list)\n self.train = train\n self.query = []\n self.gallery = []\n self.num_train_pids, self.num_train_imgs, self.num_train_cams = self.get_imagedata_info(self.train)\n\n if verbose:\n print(\"=> UnrealPerson loaded\")\n print(\" ----------------------------------------\")\n print(\" subset | # ids | # cams | # images\")\n print(\" ----------------------------------------\")\n print(\" train | {:5d} | {:5d} | {:8d}\"\n .format(self.num_train_pids, self.num_train_cams, self.num_train_imgs))\n print(\" ----------------------------------------\")\n\n def process_dir(self, list_file):\n with open(list_file, 'r') as f:\n lines = f.readlines()\n dataset = []\n pid_container = set()\n for line in lines:\n line = line.strip()\n pid = line.split(' ')[1]\n pid_container.add(pid)\n\n pid2label = {pid: label for label, pid in enumerate(sorted(pid_container))}\n\n for line in lines:\n line = line.strip()\n fname, pid, cid = line.split(' ')[0], line.split(' ')[1], int(line.split(' ')[2])\n img_path = osp.join(self.dataset_dir, fname)\n dataset.append((img_path, pid2label[pid], cid))\n\n return dataset\n\n def translate(self, transform: Callable, target_root: str):\n raise NotImplementedError\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/common/vision/datasets/resisc45.py",
"content": "\"\"\"\n@author: Junguang Jiang\n@contact: JiangJunguang1123@outlook.com\n\"\"\"\n\nfrom torchvision.datasets.folder import ImageFolder\nimport random\n\n\nclass Resisc45(ImageFolder):\n \"\"\"`Resisc45 `_ dataset \\\n is a scene classification task from remote sensing images. There are 45 classes, \\\n containing 700 images each, including tennis court, ship, island, lake, \\\n parking lot, sparse residential, or stadium. \\\n The image size is RGB 256x256 pixels.\n\n .. note:: You need to download the source data manually into `root` directory.\n\n Args:\n root (str): Root directory of dataset\n split (str, optional): The dataset split, supports ``train``, or ``test``.\n transform (callable, optional): A function/transform that takes in an PIL image and returns a \\\n transformed version. E.g, :class:`torchvision.transforms.RandomCrop`.\n target_transform (callable, optional): A function/transform that takes in the target and transforms it.\n\n \"\"\"\n def __init__(self, root, split='train', download=False, **kwargs):\n super(Resisc45, self).__init__(root, **kwargs)\n random.seed(0)\n random.shuffle(self.samples)\n if split == 'train':\n self.samples = self.samples[:25200]\n else:\n self.samples = self.samples[25200:]\n\n @property\n def num_classes(self) -> int:\n \"\"\"Number of classes\"\"\"\n return len(self.classes)\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/common/vision/datasets/retinopathy.py",
"content": "\"\"\"\n@author: Junguang Jiang\n@contact: JiangJunguang1123@outlook.com\n\"\"\"\nimport os\nfrom common.vision.datasets import ImageList\n\n\nclass Retinopathy(ImageList):\n \"\"\"`Retinopathy `_ dataset \\\n consists of image-label pairs with high-resolution retina images, and labels that indicate \\\n the presence of Diabetic Retinopahy (DR) in a 0-4 scale (No DR, Mild, Moderate, Severe, \\\n or Proliferative DR).\n\n .. note:: You need to download the source data manually into `root` directory.\n\n Args:\n root (str): Root directory of dataset\n split (str, optional): The dataset split, supports ``train``, or ``test``.\n transform (callable, optional): A function/transform that takes in an PIL image and returns a \\\n transformed version. E.g, :class:`torchvision.transforms.RandomCrop`.\n target_transform (callable, optional): A function/transform that takes in the target and transforms it.\n\n \"\"\"\n CLASSES = ['No DR', 'Mild', 'Moderate', 'Severe', 'Proliferative DR']\n\n def __init__(self, root, split, download=False, **kwargs):\n\n super(Retinopathy, self).__init__(os.path.join(root, split), Retinopathy.CLASSES, os.path.join(root, \"image_list\", \"{}.txt\".format(split)), **kwargs)\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/common/vision/datasets/segmentation/__init__.py",
"content": "from .segmentation_list import SegmentationList\nfrom .cityscapes import Cityscapes, FoggyCityscapes\nfrom .gta5 import GTA5\nfrom .synthia import Synthia\n\n__all__ = [\"SegmentationList\", \"Cityscapes\", \"GTA5\", \"Synthia\", \"FoggyCityscapes\"]\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/common/vision/datasets/segmentation/cityscapes.py",
"content": "\"\"\"\n@author: Junguang Jiang\n@contact: JiangJunguang1123@outlook.com\n\"\"\"\nimport os\nfrom .segmentation_list import SegmentationList\nfrom .._util import download as download_data\n\n\nclass Cityscapes(SegmentationList):\n \"\"\"`Cityscapes `_ is a real-world semantic segmentation dataset collected\n in driving scenarios.\n\n Args:\n root (str): Root directory of dataset\n split (str, optional): The dataset split, supports ``train``, or ``val``.\n data_folder (str, optional): Sub-directory of the image. Default: 'leftImg8bit'.\n label_folder (str, optional): Sub-directory of the label. Default: 'gtFine'.\n mean (seq[float]): mean BGR value. Normalize the image if not None. Default: None.\n transforms (callable, optional): A function/transform that takes in (PIL image, label) pair \\\n and returns a transformed version. E.g, :class:`~common.vision.transforms.segmentation.Resize`.\n\n .. note:: You need to download Cityscapes manually.\n Ensure that there exist following files in the `root` directory before you using this class.\n ::\n leftImg8bit/\n train/\n val/\n test/\n gtFine/\n train/\n val/\n test/\n \"\"\"\n\n CLASSES = ['road', 'sidewalk', 'building', 'wall', 'fence', 'pole', 'traffic light', 'traffic sign',\n 'vegetation', 'terrain', 'sky', 'person', 'rider', 'car', 'truck', 'bus', 'train', 'motorcycle',\n 'bicycle']\n\n ID_TO_TRAIN_ID = {\n 7: 0, 8: 1, 11: 2, 12: 3, 13: 4, 17: 5,\n 19: 6, 20: 7, 21: 8, 22: 9, 23: 10, 24: 11, 25: 12,\n 26: 13, 27: 14, 28: 15, 31: 16, 32: 17, 33: 18\n }\n TRAIN_ID_TO_COLOR = [(128, 64, 128), (244, 35, 232), (70, 70, 70), (102, 102, 156),\n (190, 153, 153), (153, 153, 153), (250, 170, 30), (220, 220, 0),\n (107, 142, 35), (152, 251, 152), (70, 130, 180), (220, 20, 60),\n (255, 0, 0), (0, 0, 142), (0, 0, 70), (0, 60, 100), (0, 80, 100),\n (0, 0, 230), (119, 11, 32), [0, 0, 0]]\n download_list = [\n (\"image_list\", \"image_list.zip\", \"https://cloud.tsinghua.edu.cn/f/08745e798b16483db4bf/?dl=1\"),\n ]\n EVALUATE_CLASSES = CLASSES\n\n def __init__(self, root, split='train', data_folder='leftImg8bit', label_folder='gtFine', **kwargs):\n assert split in ['train', 'val']\n\n # download meta information from Internet\n list(map(lambda args: download_data(root, *args), self.download_list))\n data_list_file = os.path.join(root, \"image_list\", \"{}.txt\".format(split))\n self.split = split\n super(Cityscapes, self).__init__(root, Cityscapes.CLASSES, data_list_file, data_list_file,\n os.path.join(data_folder, split), os.path.join(label_folder, split),\n id_to_train_id=Cityscapes.ID_TO_TRAIN_ID,\n train_id_to_color=Cityscapes.TRAIN_ID_TO_COLOR, **kwargs)\n\n def parse_label_file(self, label_list_file):\n with open(label_list_file, \"r\") as f:\n label_list = [line.strip().replace(\"leftImg8bit\", \"gtFine_labelIds\") for line in f.readlines()]\n return label_list\n\n\nclass FoggyCityscapes(Cityscapes):\n \"\"\"`Foggy Cityscapes `_ is a real-world semantic segmentation dataset collected\n in foggy driving scenarios.\n\n Args:\n root (str): Root directory of dataset\n split (str, optional): The dataset split, supports ``train``, or ``val``.\n data_folder (str, optional): Sub-directory of the image. Default: 'leftImg8bit'.\n label_folder (str, optional): Sub-directory of the label. Default: 'gtFine'.\n beta (float, optional): The parameter for foggy. Choices includes: 0.005, 0.01, 0.02. Default: 0.02\n mean (seq[float]): mean BGR value. Normalize the image if not None. Default: None.\n transforms (callable, optional): A function/transform that takes in (PIL image, label) pair \\\n and returns a transformed version. E.g, :class:`~common.vision.transforms.segmentation.Resize`.\n\n .. note:: You need to download Cityscapes manually.\n Ensure that there exist following files in the `root` directory before you using this class.\n ::\n leftImg8bit_foggy/\n train/\n val/\n test/\n gtFine/\n train/\n val/\n test/\n \"\"\"\n def __init__(self, root, split='train', data_folder='leftImg8bit_foggy', label_folder='gtFine', beta=0.02, **kwargs):\n assert beta in [0.02, 0.01, 0.005]\n self.beta = beta\n super(FoggyCityscapes, self).__init__(root, split, data_folder, label_folder, **kwargs)\n\n def parse_data_file(self, file_name):\n \"\"\"Parse file to image list\n\n Args:\n file_name (str): The path of data file\n\n Returns:\n List of image path\n \"\"\"\n with open(file_name, \"r\") as f:\n data_list = [line.strip().replace(\"leftImg8bit\", \"leftImg8bit_foggy_beta_{}\".format(self.beta)) for line in f.readlines()]\n return data_list\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/common/vision/datasets/segmentation/gta5.py",
"content": "\"\"\"\n@author: Junguang Jiang\n@contact: JiangJunguang1123@outlook.com\n\"\"\"\nimport os\nfrom .segmentation_list import SegmentationList\nfrom .cityscapes import Cityscapes\nfrom .._util import download as download_data\n\n\nclass GTA5(SegmentationList):\n \"\"\"`GTA5 `_\n\n Args:\n root (str): Root directory of dataset\n split (str, optional): The dataset split, supports ``train``.\n data_folder (str, optional): Sub-directory of the image. Default: 'images'.\n label_folder (str, optional): Sub-directory of the label. Default: 'labels'.\n mean (seq[float]): mean BGR value. Normalize the image if not None. Default: None.\n transforms (callable, optional): A function/transform that takes in (PIL image, label) pair \\\n and returns a transformed version. E.g, :class:`~common.vision.transforms.segmentation.Resize`.\n\n .. note:: You need to download GTA5 manually.\n Ensure that there exist following directories in the `root` directory before you using this class.\n ::\n images/\n labels/\n \"\"\"\n download_list = [\n (\"image_list\", \"image_list.zip\", \"https://cloud.tsinghua.edu.cn/f/c77ff6fc4eea435791f4/?dl=1\"),\n ]\n\n def __init__(self, root, split='train', data_folder='images', label_folder='labels', **kwargs):\n assert split in ['train']\n # download meta information from Internet\n list(map(lambda args: download_data(root, *args), self.download_list))\n data_list_file = os.path.join(root, \"image_list\", \"{}.txt\".format(split))\n self.split = split\n super(GTA5, self).__init__(root, Cityscapes.CLASSES, data_list_file, data_list_file, data_folder, label_folder,\n id_to_train_id=Cityscapes.ID_TO_TRAIN_ID, train_id_to_color=Cityscapes.TRAIN_ID_TO_COLOR, **kwargs)"
},
{
"path": "domain_generalization/Transfer-Learning-Library/common/vision/datasets/segmentation/segmentation_list.py",
"content": "\"\"\"\n@author: Junguang Jiang\n@contact: JiangJunguang1123@outlook.com\n\"\"\"\nimport os\nfrom typing import Sequence, Optional, Dict, Callable\nfrom PIL import Image\nimport tqdm\nimport numpy as np\nfrom torch.utils import data\nimport torch\n\n\nclass SegmentationList(data.Dataset):\n \"\"\"A generic Dataset class for domain adaptation in image segmentation\n\n Args:\n root (str): Root directory of dataset\n classes (seq[str]): The names of all the classes\n data_list_file (str): File to read the image list from.\n label_list_file (str): File to read the label list from.\n data_folder (str): Sub-directory of the image.\n label_folder (str): Sub-directory of the label.\n mean (seq[float]): mean BGR value. Normalize and convert to the image if not None. Default: None.\n id_to_train_id (dict, optional): the map between the id on the label and the actual train id.\n train_id_to_color (seq, optional): the map between the train id and the color.\n transforms (callable, optional): A function/transform that takes in (PIL Image, label) pair \\\n and returns a transformed version. E.g, :class:`~common.vision.transforms.segmentation.Resize`.\n\n .. note:: In ``data_list_file``, each line is the relative path of an image.\n If your data_list_file has different formats, please over-ride :meth:`~SegmentationList.parse_data_file`.\n ::\n source_dir/dog_xxx.png\n target_dir/dog_xxy.png\n\n In ``label_list_file``, each line is the relative path of an label.\n If your label_list_file has different formats, please over-ride :meth:`~SegmentationList.parse_label_file`.\n\n .. warning:: When mean is not None, please do not provide Normalize and ToTensor in transforms.\n\n \"\"\"\n def __init__(self, root: str, classes: Sequence[str], data_list_file: str, label_list_file: str,\n data_folder: str, label_folder: str,\n id_to_train_id: Optional[Dict] = None, train_id_to_color: Optional[Sequence] = None,\n transforms: Optional[Callable] = None):\n self.root = root\n self.classes = classes\n self.data_list_file = data_list_file\n self.label_list_file = label_list_file\n self.data_folder = data_folder\n self.label_folder = label_folder\n self.ignore_label = 255\n self.id_to_train_id = id_to_train_id\n self.train_id_to_color = np.array(train_id_to_color)\n self.data_list = self.parse_data_file(self.data_list_file)\n self.label_list = self.parse_label_file(self.label_list_file)\n self.transforms = transforms\n\n def parse_data_file(self, file_name):\n \"\"\"Parse file to image list\n\n Args:\n file_name (str): The path of data file\n\n Returns:\n List of image path\n \"\"\"\n with open(file_name, \"r\") as f:\n data_list = [line.strip() for line in f.readlines()]\n return data_list\n\n def parse_label_file(self, file_name):\n \"\"\"Parse file to label list\n\n Args:\n file_name (str): The path of data file\n\n Returns:\n List of label path\n \"\"\"\n with open(file_name, \"r\") as f:\n label_list = [line.strip() for line in f.readlines()]\n return label_list\n\n def __len__(self):\n return len(self.data_list)\n\n def __getitem__(self, index):\n image_name = self.data_list[index]\n label_name = self.label_list[index]\n image = Image.open(os.path.join(self.root, self.data_folder, image_name)).convert('RGB')\n label = Image.open(os.path.join(self.root, self.label_folder, label_name))\n image, label = self.transforms(image, label)\n\n # remap label\n if isinstance(label, torch.Tensor):\n label = label.numpy()\n label = np.asarray(label, np.int64)\n label_copy = self.ignore_label * np.ones(label.shape, dtype=np.int64)\n if self.id_to_train_id:\n for k, v in self.id_to_train_id.items():\n label_copy[label == k] = v\n\n return image, label_copy.copy()\n\n @property\n def num_classes(self) -> int:\n \"\"\"Number of classes\"\"\"\n return len(self.classes)\n\n def decode_target(self, target):\n \"\"\" Decode label (each value is integer) into the corresponding RGB value.\n\n Args:\n target (numpy.array): label in shape H x W\n\n Returns:\n RGB label (PIL Image) in shape H x W x 3\n \"\"\"\n target = target.copy()\n target[target == 255] = self.num_classes # unknown label is black on the RGB label\n target = self.train_id_to_color[target]\n return Image.fromarray(target.astype(np.uint8))\n\n def collect_image_paths(self):\n \"\"\"Return a list of the absolute path of all the images\"\"\"\n return [os.path.join(self.root, self.data_folder, image_name) for image_name in self.data_list]\n\n @staticmethod\n def _save_pil_image(image, path):\n os.makedirs(os.path.dirname(path), exist_ok=True)\n image.save(path)\n\n def translate(self, transform: Callable, target_root: str, color=False):\n \"\"\" Translate an image and save it into a specified directory\n\n Args:\n transform (callable): a transform function that maps (image, label) pair from one domain to another domain\n target_root (str): the root directory to save images and labels\n\n \"\"\"\n os.makedirs(target_root, exist_ok=True)\n for image_name, label_name in zip(tqdm.tqdm(self.data_list), self.label_list):\n image_path = os.path.join(target_root, self.data_folder, image_name)\n label_path = os.path.join(target_root, self.label_folder, label_name)\n if os.path.exists(image_path) and os.path.exists(label_path):\n continue\n image = Image.open(os.path.join(self.root, self.data_folder, image_name)).convert('RGB')\n label = Image.open(os.path.join(self.root, self.label_folder, label_name))\n\n translated_image, translated_label = transform(image, label)\n self._save_pil_image(translated_image, image_path)\n self._save_pil_image(translated_label, label_path)\n if color:\n colored_label = self.decode_target(np.array(translated_label))\n file_name, file_ext = os.path.splitext(label_name)\n self._save_pil_image(colored_label, os.path.join(target_root, self.label_folder,\n \"{}_color{}\".format(file_name, file_ext)))\n\n @property\n def evaluate_classes(self):\n \"\"\"The name of classes to be evaluated\"\"\"\n return self.classes\n\n @property\n def ignore_classes(self):\n \"\"\"The name of classes to be ignored\"\"\"\n return list(set(self.classes) - set(self.evaluate_classes))"
},
{
"path": "domain_generalization/Transfer-Learning-Library/common/vision/datasets/segmentation/synthia.py",
"content": "\"\"\"\n@author: Junguang Jiang\n@contact: JiangJunguang1123@outlook.com\n\"\"\"\nimport os\nfrom .segmentation_list import SegmentationList\nfrom .cityscapes import Cityscapes\nfrom .._util import download as download_data\n\n\nclass Synthia(SegmentationList):\n \"\"\"`SYNTHIA `_\n\n Args:\n root (str): Root directory of dataset\n split (str, optional): The dataset split, supports ``train``.\n data_folder (str, optional): Sub-directory of the image. Default: 'RGB'.\n label_folder (str, optional): Sub-directory of the label. Default: 'synthia_mapped_to_cityscapes'.\n mean (seq[float]): mean BGR value. Normalize the image if not None. Default: None.\n transforms (callable, optional): A function/transform that takes in (PIL image, label) pair \\\n and returns a transformed version. E.g, :class:`~common.vision.transforms.segmentation.Resize`.\n\n .. note:: You need to download GTA5 manually.\n Ensure that there exist following directories in the `root` directory before you using this class.\n ::\n RGB/\n synthia_mapped_to_cityscapes/\n \"\"\"\n ID_TO_TRAIN_ID = {\n 3: 0, 4: 1, 2: 2, 21: 3, 5: 4, 7: 5,\n 15: 6, 9: 7, 6: 8, 16: 9, 1: 10, 10: 11, 17: 12,\n 8: 13, 18: 14, 19: 15, 20: 16, 12: 17, 11: 18\n }\n download_list = [\n (\"image_list\", \"image_list.zip\", \"https://cloud.tsinghua.edu.cn/f/15c4d0f8e62e45d9a6b7/?dl=1\"),\n ]\n\n def __init__(self, root, split='train', data_folder='RGB', label_folder='synthia_mapped_to_cityscapes', **kwargs):\n assert split in ['train']\n # download meta information from Internet\n list(map(lambda args: download_data(root, *args), self.download_list))\n data_list_file = os.path.join(root, \"image_list\", \"{}.txt\".format(split))\n super(Synthia, self).__init__(root, Cityscapes.CLASSES, data_list_file, data_list_file, data_folder,\n label_folder, id_to_train_id=Synthia.ID_TO_TRAIN_ID,\n train_id_to_color=Cityscapes.TRAIN_ID_TO_COLOR, **kwargs)\n\n @property\n def evaluate_classes(self):\n return [\n 'road', 'sidewalk', 'building', 'traffic light', 'traffic sign',\n 'vegetation', 'sky', 'person', 'rider', 'car', 'bus', 'motorcycle', 'bicycle'\n ]\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/common/vision/datasets/stanford_cars.py",
"content": "\"\"\"\n@author: Yifei Ji\n@contact: jiyf990330@163.com\n\"\"\"\nimport os\nfrom typing import Optional\nfrom .imagelist import ImageList\nfrom ._util import download as download_data, check_exits\n\n\nclass StanfordCars(ImageList):\n \"\"\"`The Stanford Cars `_ \\\n contains 16,185 images of 196 classes of cars. \\\n Each category has been split roughly in a 50-50 split. \\\n There are 8,144 images for training and 8,041 images for testing.\n\n Args:\n root (str): Root directory of dataset\n split (str, optional): The dataset split, supports ``train``, or ``test``.\n sample_rate (int): The sampling rates to sample random ``training`` images for each category.\n Choices include 100, 50, 30, 15. Default: 100.\n download (bool, optional): If true, downloads the dataset from the internet and puts it \\\n in root directory. If dataset is already downloaded, it is not downloaded again.\n transform (callable, optional): A function/transform that takes in an PIL image and returns a \\\n transformed version. E.g, :class:`torchvision.transforms.RandomCrop`.\n target_transform (callable, optional): A function/transform that takes in the target and transforms it.\n\n .. note:: In `root`, there will exist following files after downloading.\n ::\n train/\n test/\n image_list/\n train_100.txt\n train_50.txt\n train_30.txt\n train_15.txt\n test.txt\n \"\"\"\n download_list = [\n (\"image_list\", \"image_list.zip\", \"https://cloud.tsinghua.edu.cn/f/d95c188cc49c404aba70/?dl=1\"),\n (\"train\", \"train.tgz\", \"https://cloud.tsinghua.edu.cn/f/d5ab63c391a949509db0/?dl=1\"),\n (\"test\", \"test.tgz\", \"https://cloud.tsinghua.edu.cn/f/04e6fd5222a84d0a8ff5/?dl=1\"),\n ]\n image_list = {\n \"train\": \"image_list/train_100.txt\",\n \"train100\": \"image_list/train_100.txt\",\n \"train50\": \"image_list/train_50.txt\",\n \"train30\": \"image_list/train_30.txt\",\n \"train15\": \"image_list/train_15.txt\",\n \"test\": \"image_list/test.txt\",\n \"test100\": \"image_list/test.txt\",\n }\n CLASSES = ['1', '2', '3', '4', '5', '6', '7', '8', '9', '10', '11', '12', '13', '14', '15', '16', '17', '18', '19', '20',\n '21', '22', '23', '24', '25', '26', '27', '28', '29', '30', '31', '32', '33', '34', '35', '36', '37', '38', '39', '40',\n '41', '42', '43', '44', '45', '46', '47', '48', '49', '50', '51', '52', '53', '54', '55', '56', '57', '58', '59', '60',\n '61', '62', '63', '64', '65', '66', '67', '68', '69', '70', '71', '72', '73', '74', '75', '76', '77', '78', '79', '80',\n '81', '82', '83', '84', '85', '86', '87', '88', '89', '90', '91', '92', '93', '94', '95', '96', '97', '98', '99', '100',\n '101', '102', '103', '104', '105', '106', '107', '108', '109', '110', '111', '112', '113', '114', '115', '116', '117', '118', '119', '120',\n '121', '122', '123', '124', '125', '126', '127', '128', '129', '130', '131', '132', '133', '134', '135', '136', '137', '138', '139', '140',\n '141', '142', '143', '144', '145', '146', '147', '148', '149', '150', '151', '152', '153', '154', '155', '156', '157', '158', '159', '160',\n '161', '162', '163', '164', '165', '166', '167', '168', '169', '170', '171', '172', '173', '174', '175', '176', '177', '178', '179', '180',\n '181', '182', '183', '184', '185', '186', '187', '188', '189', '190', '191', '192', '193', '194', '195', '196']\n\n def __init__(self, root: str, split: str, sample_rate: Optional[int] =100, download: Optional[bool] = False, **kwargs):\n\n if split == 'train':\n list_name = 'train' + str(sample_rate)\n assert list_name in self.image_list\n data_list_file = os.path.join(root, self.image_list[list_name])\n else:\n data_list_file = os.path.join(root, self.image_list['test'])\n\n if download:\n list(map(lambda args: download_data(root, *args), self.download_list))\n else:\n list(map(lambda file_name, _: check_exits(root, file_name), self.download_list))\n\n super(StanfordCars, self).__init__(root, StanfordCars.CLASSES, data_list_file=data_list_file, **kwargs)\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/common/vision/datasets/stanford_dogs.py",
"content": "\"\"\"\n@author: Yifei Ji\n@contact: jiyf990330@163.com\n\"\"\"\nimport os\nfrom typing import Optional\nfrom .imagelist import ImageList\nfrom ._util import download as download_data, check_exits\n\n\nclass StanfordDogs(ImageList):\n \"\"\"`The Stanford Dogs `_ \\\n contains 20,580 images of 120 breeds of dogs from around the world. \\\n Each category is composed of exactly 100 training examples and around 72 testing examples.\n\n Args:\n root (str): Root directory of dataset\n split (str, optional): The dataset split, supports ``train``, or ``test``.\n sample_rate (int): The sampling rates to sample random ``training`` images for each category.\n Choices include 100, 50, 30, 15. Default: 100.\n download (bool, optional): If true, downloads the dataset from the internet and puts it \\\n in root directory. If dataset is already downloaded, it is not downloaded again.\n transform (callable, optional): A function/transform that takes in an PIL image and returns a \\\n transformed version. E.g, :class:`torchvision.transforms.RandomCrop`.\n target_transform (callable, optional): A function/transform that takes in the target and transforms it.\n\n .. note:: In `root`, there will exist following files after downloading.\n ::\n train/\n test/\n image_list/\n train_100.txt\n train_50.txt\n train_30.txt\n train_15.txt\n test.txt\n \"\"\"\n download_list = [\n (\"image_list\", \"image_list.zip\", \"https://cloud.tsinghua.edu.cn/f/b7b3dd297ec54e038238/?dl=1\"),\n (\"train\", \"train.tgz\", \"https://cloud.tsinghua.edu.cn/f/cd09d9ca88f044bfa5d3/?dl=1\"),\n (\"test\", \"test.tgz\", \"https://cloud.tsinghua.edu.cn/f/64a67e97069948c7b2c9/?dl=1\"),\n ]\n image_list = {\n \"train\": \"image_list/train_100.txt\",\n \"train100\": \"image_list/train_100.txt\",\n \"train50\": \"image_list/train_50.txt\",\n \"train30\": \"image_list/train_30.txt\",\n \"train15\": \"image_list/train_15.txt\",\n \"test\": \"image_list/test.txt\",\n \"test100\": \"image_list/test.txt\",\n }\n CLASSES = ['n02085620-Chihuahua', 'n02085782-Japanese_spaniel', 'n02085936-Maltese_dog', 'n02086079-Pekinese', 'n02086240-Shih-Tzu',\n 'n02086646-Blenheim_spaniel', 'n02086910-papillon', 'n02087046-toy_terrier', 'n02087394-Rhodesian_ridgeback',\n 'n02088094-Afghan_hound', 'n02088238-basset', 'n02088364-beagle', 'n02088466-bloodhound', 'n02088632-bluetick', 'n02089078-black-and-tan_coonhound',\n 'n02089867-Walker_hound', 'n02089973-English_foxhound', 'n02090379-redbone', 'n02090622-borzoi', 'n02090721-Irish_wolfhound', 'n02091032-Italian_greyhound',\n 'n02091134-whippet', 'n02091244-Ibizan_hound', 'n02091467-Norwegian_elkhound', 'n02091635-otterhound', 'n02091831-Saluki', 'n02092002-Scottish_deerhound',\n 'n02092339-Weimaraner', 'n02093256-Staffordshire_bullterrier', 'n02093428-American_Staffordshire_terrier', 'n02093647-Bedlington_terrier', 'n02093754-Border_terrier',\n 'n02093859-Kerry_blue_terrier', 'n02093991-Irish_terrier', 'n02094114-Norfolk_terrier', 'n02094258-Norwich_terrier', 'n02094433-Yorkshire_terrier',\n 'n02095314-wire-haired_fox_terrier', 'n02095570-Lakeland_terrier', 'n02095889-Sealyham_terrier', 'n02096051-Airedale', 'n02096177-cairn', 'n02096294-Australian_terrier',\n 'n02096437-Dandie_Dinmont', 'n02096585-Boston_bull', 'n02097047-miniature_schnauzer', 'n02097130-giant_schnauzer', 'n02097209-standard_schnauzer',\n 'n02097298-Scotch_terrier', 'n02097474-Tibetan_terrier', 'n02097658-silky_terrier', 'n02098105-soft-coated_wheaten_terrier', 'n02098286-West_Highland_white_terrier',\n 'n02098413-Lhasa', 'n02099267-flat-coated_retriever', 'n02099429-curly-coated_retriever', 'n02099601-golden_retriever', 'n02099712-Labrador_retriever',\n 'n02099849-Chesapeake_Bay_retriever', 'n02100236-German_short-haired_pointer', 'n02100583-vizsla', 'n02100735-English_setter', 'n02100877-Irish_setter',\n 'n02101006-Gordon_setter', 'n02101388-Brittany_spaniel', 'n02101556-clumber', 'n02102040-English_springer', 'n02102177-Welsh_springer_spaniel', 'n02102318-cocker_spaniel',\n 'n02102480-Sussex_spaniel', 'n02102973-Irish_water_spaniel', 'n02104029-kuvasz', 'n02104365-schipperke', 'n02105056-groenendael', 'n02105162-malinois', 'n02105251-briard', 'n02105412-kelpie',\n 'n02105505-komondor', 'n02105641-Old_English_sheepdog', 'n02105855-Shetland_sheepdog', 'n02106030-collie', 'n02106166-Border_collie', 'n02106382-Bouvier_des_Flandres', 'n02106550-Rottweiler',\n 'n02106662-German_shepherd', 'n02107142-Doberman', 'n02107312-miniature_pinscher', 'n02107574-Greater_Swiss_Mountain_dog',\n 'n02107683-Bernese_mountain_dog', 'n02107908-Appenzeller', 'n02108000-EntleBucher', 'n02108089-boxer', 'n02108422-bull_mastiff', 'n02108551-Tibetan_mastiff',\n 'n02108915-French_bulldog', 'n02109047-Great_Dane', 'n02109525-Saint_Bernard', 'n02109961-Eskimo_dog', 'n02110063-malamute', 'n02110185-Siberian_husky',\n 'n02110627-affenpinscher', 'n02110806-basenji', 'n02110958-pug', 'n02111129-Leonberg', 'n02111277-Newfoundland', 'n02111500-Great_Pyrenees', 'n02111889-Samoyed', 'n02112018-Pomeranian',\n 'n02112137-chow', 'n02112350-keeshond', 'n02112706-Brabancon_griffon', 'n02113023-Pembroke', 'n02113186-Cardigan',\n 'n02113624-toy_poodle', 'n02113712-miniature_poodle', 'n02113799-standard_poodle', 'n02113978-Mexican_hairless', 'n02115641-dingo', 'n02115913-dhole', 'n02116738-African_hunting_dog']\n\n\n def __init__(self, root: str, split: str, sample_rate: Optional[int] =100, download: Optional[bool] = False, **kwargs):\n\n if split == 'train':\n list_name = 'train' + str(sample_rate)\n assert list_name in self.image_list\n data_list_file = os.path.join(root, self.image_list[list_name])\n else:\n data_list_file = os.path.join(root, self.image_list['test'])\n\n if download:\n list(map(lambda args: download_data(root, *args), self.download_list))\n else:\n list(map(lambda file_name, _: check_exits(root, file_name), self.download_list))\n\n super(StanfordDogs, self).__init__(root, StanfordDogs.CLASSES, data_list_file=data_list_file, **kwargs)"
},
{
"path": "domain_generalization/Transfer-Learning-Library/common/vision/datasets/visda2017.py",
"content": "\"\"\"\n@author: Junguang Jiang\n@contact: JiangJunguang1123@outlook.com\n\"\"\"\nimport os\nfrom typing import Optional\nfrom .imagelist import ImageList\nfrom ._util import download as download_data, check_exits\n\n\nclass VisDA2017(ImageList):\n \"\"\"`VisDA-2017 `_ Dataset\n\n Args:\n root (str): Root directory of dataset\n task (str): The task (domain) to create dataset. Choices include ``'Synthetic'``: synthetic images and \\\n ``'Real'``: real-world images.\n download (bool, optional): If true, downloads the dataset from the internet and puts it \\\n in root directory. If dataset is already downloaded, it is not downloaded again.\n transform (callable, optional): A function/transform that takes in an PIL image and returns a \\\n transformed version. E.g, ``transforms.RandomCrop``.\n target_transform (callable, optional): A function/transform that takes in the target and transforms it.\n\n .. note:: In `root`, there will exist following files after downloading.\n ::\n train/\n aeroplance/\n *.png\n ...\n validation/\n image_list/\n train.txt\n validation.txt\n \"\"\"\n download_list = [\n (\"image_list\", \"image_list.zip\", \"https://cloud.tsinghua.edu.cn/f/b25b2b990e8f42e691f0/?dl=1\"),\n (\"train\", \"train.tar\", \"http://csr.bu.edu/ftp/visda17/clf/train.tar\"),\n (\"validation\", \"validation.tar\", \"http://csr.bu.edu/ftp/visda17/clf/validation.tar\")\n ]\n image_list = {\n \"Synthetic\": \"image_list/train.txt\",\n \"Real\": \"image_list/validation.txt\"\n }\n CLASSES = ['aeroplane', 'bicycle', 'bus', 'car', 'horse', 'knife',\n 'motorcycle', 'person', 'plant', 'skateboard', 'train', 'truck']\n\n def __init__(self, root: str, task: str, download: Optional[bool] = False, **kwargs):\n assert task in self.image_list\n data_list_file = os.path.join(root, self.image_list[task])\n\n if download:\n list(map(lambda args: download_data(root, *args), self.download_list))\n else:\n list(map(lambda file_name, _: check_exits(root, file_name), self.download_list))\n\n super(VisDA2017, self).__init__(root, VisDA2017.CLASSES, data_list_file=data_list_file, **kwargs)\n\n @classmethod\n def domains(cls):\n return list(cls.image_list.keys())"
},
{
"path": "domain_generalization/Transfer-Learning-Library/common/vision/models/__init__.py",
"content": "from .resnet import *\nfrom .ibn import *\nfrom .digits import *\n\n__all__ = ['resnet', 'digits', 'ibn']\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/common/vision/models/digits.py",
"content": "\"\"\"\n@author: Junguang Jiang\n@contact: JiangJunguang1123@outlook.com\n\"\"\"\nimport torch.nn as nn\n\n\nclass LeNet(nn.Sequential):\n def __init__(self, num_classes=10):\n super(LeNet, self).__init__(\n nn.Conv2d(1, 20, kernel_size=5),\n nn.MaxPool2d(2),\n nn.ReLU(),\n nn.Conv2d(20, 50, kernel_size=5),\n nn.Dropout2d(p=0.5),\n nn.MaxPool2d(2),\n nn.ReLU(),\n nn.Flatten(start_dim=1),\n nn.Linear(50 * 4 * 4, 500),\n nn.ReLU(),\n nn.Dropout(p=0.5),\n )\n self.num_classes = num_classes\n self.out_features = 500\n\n def copy_head(self):\n return nn.Linear(500, self.num_classes)\n\n\nclass DTN(nn.Sequential):\n def __init__(self, num_classes=10):\n super(DTN, self).__init__(\n nn.Conv2d(3, 64, kernel_size=5, stride=2, padding=2),\n nn.BatchNorm2d(64),\n nn.Dropout2d(0.1),\n nn.ReLU(),\n nn.Conv2d(64, 128, kernel_size=5, stride=2, padding=2),\n nn.BatchNorm2d(128),\n nn.Dropout2d(0.3),\n nn.ReLU(),\n nn.Conv2d(128, 256, kernel_size=5, stride=2, padding=2),\n nn.BatchNorm2d(256),\n nn.Dropout2d(0.5),\n nn.ReLU(),\n nn.Flatten(start_dim=1),\n nn.Linear(256 * 4 * 4, 512),\n nn.BatchNorm1d(512),\n nn.ReLU(),\n nn.Dropout(),\n )\n self.num_classes = num_classes\n self.out_features = 512\n\n def copy_head(self):\n return nn.Linear(512, self.num_classes)\n\n\n\ndef lenet(pretrained=False, **kwargs):\n \"\"\"LeNet model from\n `\"Gradient-based learning applied to document recognition\" `_\n\n Args:\n num_classes (int): number of classes. Default: 10\n\n .. note::\n The input image size must be 28 x 28.\n\n \"\"\"\n return LeNet(**kwargs)\n\n\ndef dtn(pretrained=False, **kwargs):\n \"\"\" DTN model\n\n Args:\n num_classes (int): number of classes. Default: 10\n\n .. note::\n The input image size must be 32 x 32.\n\n \"\"\"\n return DTN(**kwargs)"
},
{
"path": "domain_generalization/Transfer-Learning-Library/common/vision/models/ibn.py",
"content": "\"\"\"\nModified from https://github.com/XingangPan/IBN-Net\n@author: Baixu Chen\n@contact: cbx_99_hasta@outlook.com\n\"\"\"\nimport math\nimport torch\nimport torch.nn as nn\n\n__all__ = ['resnet18_ibn_a', 'resnet18_ibn_b', 'resnet34_ibn_a', 'resnet34_ibn_b', 'resnet50_ibn_a', 'resnet50_ibn_b',\n 'resnet101_ibn_a', 'resnet101_ibn_b']\n\nmodel_urls = {\n 'resnet18_ibn_a': 'https://github.com/XingangPan/IBN-Net/releases/download/v1.0/resnet18_ibn_a-2f571257.pth',\n 'resnet34_ibn_a': 'https://github.com/XingangPan/IBN-Net/releases/download/v1.0/resnet34_ibn_a-94bc1577.pth',\n 'resnet50_ibn_a': 'https://github.com/XingangPan/IBN-Net/releases/download/v1.0/resnet50_ibn_a-d9d0bb7b.pth',\n 'resnet101_ibn_a': 'https://github.com/XingangPan/IBN-Net/releases/download/v1.0/resnet101_ibn_a-59ea0ac6.pth',\n 'resnet18_ibn_b': 'https://github.com/XingangPan/IBN-Net/releases/download/v1.0/resnet18_ibn_b-bc2f3c11.pth',\n 'resnet34_ibn_b': 'https://github.com/XingangPan/IBN-Net/releases/download/v1.0/resnet34_ibn_b-04134c37.pth',\n 'resnet50_ibn_b': 'https://github.com/XingangPan/IBN-Net/releases/download/v1.0/resnet50_ibn_b-9ca61e85.pth',\n 'resnet101_ibn_b': 'https://github.com/XingangPan/IBN-Net/releases/download/v1.0/resnet101_ibn_b-c55f6dba.pth',\n}\n\n\nclass IBN(nn.Module):\n r\"\"\"Instance-Batch Normalization layer from\n `Two at Once: Enhancing Learning and Generalization Capacities via IBN-Net (ECCV 2018)\n `_.\n\n Given input feature map :math:`f\\_input` of dimension :math:`(C,H,W)`, we first split :math:`f\\_input` into\n two parts along `channel` dimension. They are denoted as :math:`f_1` of dimension :math:`(C_1,H,W)` and\n :math:`f_2` of dimension :math:`(C_2,H,W)`, where :math:`C_1+C_2=C`. Then we pass :math:`f_1` and :math:`f_2`\n through IN and BN layer, respectively, to get :math:`IN(f_1)` and :math:`BN(f_2)`. Last, we concat them along\n `channel` dimension to create :math:`f\\_output=concat(IN(f_1), BN(f_2))`.\n\n Args:\n planes (int): Number of channels for the input tensor\n ratio (float): Ratio of instance normalization in the IBN layer\n \"\"\"\n\n def __init__(self, planes, ratio=0.5):\n super(IBN, self).__init__()\n self.half = int(planes * ratio)\n self.IN = nn.InstanceNorm2d(self.half, affine=True)\n self.BN = nn.BatchNorm2d(planes - self.half)\n\n def forward(self, x):\n split = torch.split(x, self.half, 1)\n out1 = self.IN(split[0].contiguous())\n out2 = self.BN(split[1].contiguous())\n out = torch.cat((out1, out2), 1)\n return out\n\n\nclass BasicBlock_IBN(nn.Module):\n expansion = 1\n\n def __init__(self, inplanes, planes, ibn=None, stride=1, downsample=None):\n super(BasicBlock_IBN, self).__init__()\n self.conv1 = nn.Conv2d(inplanes, planes, kernel_size=3, stride=stride,\n padding=1, bias=False)\n if ibn == 'a':\n self.bn1 = IBN(planes)\n else:\n self.bn1 = nn.BatchNorm2d(planes)\n self.relu = nn.ReLU(inplace=True)\n self.conv2 = nn.Conv2d(planes, planes, kernel_size=3, padding=1, bias=False)\n self.bn2 = nn.BatchNorm2d(planes)\n self.IN = nn.InstanceNorm2d(planes, affine=True) if ibn == 'b' else None\n self.downsample = downsample\n self.stride = stride\n\n def forward(self, x):\n residual = x\n\n out = self.conv1(x)\n out = self.bn1(out)\n out = self.relu(out)\n\n out = self.conv2(out)\n out = self.bn2(out)\n\n if self.downsample is not None:\n residual = self.downsample(x)\n\n out += residual\n if self.IN is not None:\n out = self.IN(out)\n out = self.relu(out)\n\n return out\n\n\nclass Bottleneck_IBN(nn.Module):\n expansion = 4\n\n def __init__(self, inplanes, planes, ibn=None, stride=1, downsample=None):\n super(Bottleneck_IBN, self).__init__()\n self.conv1 = nn.Conv2d(inplanes, planes, kernel_size=1, bias=False)\n if ibn == 'a':\n self.bn1 = IBN(planes)\n else:\n self.bn1 = nn.BatchNorm2d(planes)\n self.conv2 = nn.Conv2d(planes, planes, kernel_size=3, stride=stride,\n padding=1, bias=False)\n self.bn2 = nn.BatchNorm2d(planes)\n self.conv3 = nn.Conv2d(planes, planes * self.expansion, kernel_size=1, bias=False)\n self.bn3 = nn.BatchNorm2d(planes * self.expansion)\n self.IN = nn.InstanceNorm2d(planes * 4, affine=True) if ibn == 'b' else None\n self.relu = nn.ReLU(inplace=True)\n self.downsample = downsample\n self.stride = stride\n\n def forward(self, x):\n residual = x\n\n out = self.conv1(x)\n out = self.bn1(out)\n out = self.relu(out)\n\n out = self.conv2(out)\n out = self.bn2(out)\n out = self.relu(out)\n\n out = self.conv3(out)\n out = self.bn3(out)\n\n if self.downsample is not None:\n residual = self.downsample(x)\n\n out += residual\n if self.IN is not None:\n out = self.IN(out)\n out = self.relu(out)\n\n return out\n\n\nclass ResNet_IBN(nn.Module):\n r\"\"\"\n ResNets-IBN without fully connected layer\n \"\"\"\n\n def __init__(self, block, layers, ibn_cfg=('a', 'a', 'a', None)):\n self.inplanes = 64\n super(ResNet_IBN, self).__init__()\n self.conv1 = nn.Conv2d(3, 64, kernel_size=7, stride=2, padding=3,\n bias=False)\n if ibn_cfg[0] == 'b':\n self.bn1 = nn.InstanceNorm2d(64, affine=True)\n else:\n self.bn1 = nn.BatchNorm2d(64)\n self.relu = nn.ReLU(inplace=True)\n self.maxpool = nn.MaxPool2d(kernel_size=3, stride=2, padding=1)\n self.layer1 = self._make_layer(block, 64, layers[0], ibn=ibn_cfg[0])\n self.layer2 = self._make_layer(block, 128, layers[1], stride=2, ibn=ibn_cfg[1])\n self.layer3 = self._make_layer(block, 256, layers[2], stride=2, ibn=ibn_cfg[2])\n self.layer4 = self._make_layer(block, 512, layers[3], stride=2, ibn=ibn_cfg[3])\n self._out_features = 512 * block.expansion\n\n for m in self.modules():\n if isinstance(m, nn.Conv2d):\n n = m.kernel_size[0] * m.kernel_size[1] * m.out_channels\n m.weight.data.normal_(0, math.sqrt(2. / n))\n elif isinstance(m, nn.BatchNorm2d) or isinstance(m, nn.InstanceNorm2d):\n m.weight.data.fill_(1)\n m.bias.data.zero_()\n\n def _make_layer(self, block, planes, blocks, stride=1, ibn=None):\n downsample = None\n if stride != 1 or self.inplanes != planes * block.expansion:\n downsample = nn.Sequential(\n nn.Conv2d(self.inplanes, planes * block.expansion,\n kernel_size=1, stride=stride, bias=False),\n nn.BatchNorm2d(planes * block.expansion),\n )\n\n layers = []\n layers.append(block(self.inplanes, planes,\n None if ibn == 'b' else ibn,\n stride, downsample))\n self.inplanes = planes * block.expansion\n for i in range(1, blocks):\n layers.append(block(self.inplanes, planes,\n None if (ibn == 'b' and i < blocks - 1) else ibn))\n\n return nn.Sequential(*layers)\n\n def forward(self, x):\n \"\"\"\"\"\"\n x = self.conv1(x)\n x = self.bn1(x)\n x = self.relu(x)\n x = self.maxpool(x)\n\n x = self.layer1(x)\n x = self.layer2(x)\n x = self.layer3(x)\n x = self.layer4(x)\n\n return x\n\n @property\n def out_features(self) -> int:\n \"\"\"The dimension of output features\"\"\"\n return self._out_features\n\n\ndef resnet18_ibn_a(pretrained=False):\n \"\"\"Constructs a ResNet-18-IBN-a model.\n\n Args:\n pretrained (bool): If True, returns a model pre-trained on ImageNet\n \"\"\"\n model = ResNet_IBN(block=BasicBlock_IBN,\n layers=[2, 2, 2, 2],\n ibn_cfg=('a', 'a', 'a', None))\n if pretrained:\n model.load_state_dict(torch.hub.load_state_dict_from_url(model_urls['resnet18_ibn_a']), strict=False)\n return model\n\n\ndef resnet34_ibn_a(pretrained=False):\n \"\"\"Constructs a ResNet-34-IBN-a model.\n\n Args:\n pretrained (bool): If True, returns a model pre-trained on ImageNet\n \"\"\"\n model = ResNet_IBN(block=BasicBlock_IBN,\n layers=[3, 4, 6, 3],\n ibn_cfg=('a', 'a', 'a', None))\n if pretrained:\n model.load_state_dict(torch.hub.load_state_dict_from_url(model_urls['resnet34_ibn_a']), strict=False)\n return model\n\n\ndef resnet50_ibn_a(pretrained=False):\n \"\"\"Constructs a ResNet-50-IBN-a model.\n\n Args:\n pretrained (bool): If True, returns a model pre-trained on ImageNet\n \"\"\"\n model = ResNet_IBN(block=Bottleneck_IBN,\n layers=[3, 4, 6, 3],\n ibn_cfg=('a', 'a', 'a', None))\n if pretrained:\n model.load_state_dict(torch.hub.load_state_dict_from_url(model_urls['resnet50_ibn_a']), strict=False)\n return model\n\n\ndef resnet101_ibn_a(pretrained=False):\n \"\"\"Constructs a ResNet-101-IBN-a model.\n\n Args:\n pretrained (bool): If True, returns a model pre-trained on ImageNet\n \"\"\"\n model = ResNet_IBN(block=Bottleneck_IBN,\n layers=[3, 4, 23, 3],\n ibn_cfg=('a', 'a', 'a', None))\n if pretrained:\n model.load_state_dict(torch.hub.load_state_dict_from_url(model_urls['resnet101_ibn_a']), strict=False)\n return model\n\n\ndef resnet18_ibn_b(pretrained=False):\n \"\"\"Constructs a ResNet-18-IBN-b model.\n\n Args:\n pretrained (bool): If True, returns a model pre-trained on ImageNet\n \"\"\"\n model = ResNet_IBN(block=BasicBlock_IBN,\n layers=[2, 2, 2, 2],\n ibn_cfg=('b', 'b', None, None))\n if pretrained:\n model.load_state_dict(torch.hub.load_state_dict_from_url(model_urls['resnet18_ibn_b']), strict=False)\n return model\n\n\ndef resnet34_ibn_b(pretrained=False):\n \"\"\"Constructs a ResNet-34-IBN-b model.\n\n Args:\n pretrained (bool): If True, returns a model pre-trained on ImageNet\n \"\"\"\n model = ResNet_IBN(block=BasicBlock_IBN,\n layers=[3, 4, 6, 3],\n ibn_cfg=('b', 'b', None, None))\n if pretrained:\n model.load_state_dict(torch.hub.load_state_dict_from_url(model_urls['resnet34_ibn_b']), strict=False)\n return model\n\n\ndef resnet50_ibn_b(pretrained=False):\n \"\"\"Constructs a ResNet-50-IBN-b model.\n\n Args:\n pretrained (bool): If True, returns a model pre-trained on ImageNet\n \"\"\"\n model = ResNet_IBN(block=Bottleneck_IBN,\n layers=[3, 4, 6, 3],\n ibn_cfg=('b', 'b', None, None))\n if pretrained:\n model.load_state_dict(torch.hub.load_state_dict_from_url(model_urls['resnet50_ibn_b']), strict=False)\n return model\n\n\ndef resnet101_ibn_b(pretrained=False):\n \"\"\"Constructs a ResNet-101-IBN-b model.\n\n Args:\n pretrained (bool): If True, returns a model pre-trained on ImageNet\n \"\"\"\n model = ResNet_IBN(block=Bottleneck_IBN,\n layers=[3, 4, 23, 3],\n ibn_cfg=('b', 'b', None, None))\n if pretrained:\n model.load_state_dict(torch.hub.load_state_dict_from_url(model_urls['resnet101_ibn_b']), strict=False)\n return model\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/common/vision/models/keypoint_detection/__init__.py",
"content": "from .pose_resnet import *\nfrom . import loss\n\n__all__ = ['pose_resnet']"
},
{
"path": "domain_generalization/Transfer-Learning-Library/common/vision/models/keypoint_detection/loss.py",
"content": "\"\"\"\nModified from https://github.com/microsoft/human-pose-estimation.pytorch\n@author: Junguang Jiang\n@contact: JiangJunguang1123@outlook.com\n\"\"\"\nimport torch.nn as nn\nimport torch.nn.functional as F\n\n\nclass JointsMSELoss(nn.Module):\n \"\"\"\n Typical MSE loss for keypoint detection.\n\n Args:\n reduction (str, optional): Specifies the reduction to apply to the output:\n ``'none'`` | ``'mean'``. ``'none'``: no reduction will be applied,\n ``'mean'``: the sum of the output will be divided by the number of\n elements in the output. Default: ``'mean'``\n\n Inputs:\n - output (tensor): heatmap predictions\n - target (tensor): heatmap labels\n - target_weight (tensor): whether the keypoint is visible. All keypoint is visible if None. Default: None.\n\n Shape:\n - output: :math:`(minibatch, K, H, W)` where K means the number of keypoints,\n H and W is the height and width of the heatmap respectively.\n - target: :math:`(minibatch, K, H, W)`.\n - target_weight: :math:`(minibatch, K)`.\n - Output: scalar by default. If :attr:`reduction` is ``'none'``, then :math:`(minibatch, K)`.\n\n \"\"\"\n def __init__(self, reduction='mean'):\n super(JointsMSELoss, self).__init__()\n self.criterion = nn.MSELoss(reduction='none')\n self.reduction = reduction\n\n def forward(self, output, target, target_weight=None):\n B, K, _, _ = output.shape\n heatmaps_pred = output.reshape((B, K, -1))\n heatmaps_gt = target.reshape((B, K, -1))\n loss = self.criterion(heatmaps_pred, heatmaps_gt) * 0.5\n if target_weight is not None:\n loss = loss * target_weight.view((B, K, 1))\n if self.reduction == 'mean':\n return loss.mean()\n elif self.reduction == 'none':\n return loss.mean(dim=-1)\n\n\nclass JointsKLLoss(nn.Module):\n \"\"\"\n KL Divergence for keypoint detection proposed by\n `Regressive Domain Adaptation for Unsupervised Keypoint Detection `_.\n\n Args:\n reduction (str, optional): Specifies the reduction to apply to the output:\n ``'none'`` | ``'mean'``. ``'none'``: no reduction will be applied,\n ``'mean'``: the sum of the output will be divided by the number of\n elements in the output. Default: ``'mean'``\n\n Inputs:\n - output (tensor): heatmap predictions\n - target (tensor): heatmap labels\n - target_weight (tensor): whether the keypoint is visible. All keypoint is visible if None. Default: None.\n\n Shape:\n - output: :math:`(minibatch, K, H, W)` where K means the number of keypoints,\n H and W is the height and width of the heatmap respectively.\n - target: :math:`(minibatch, K, H, W)`.\n - target_weight: :math:`(minibatch, K)`.\n - Output: scalar by default. If :attr:`reduction` is ``'none'``, then :math:`(minibatch, K)`.\n\n \"\"\"\n def __init__(self, reduction='mean', epsilon=0.):\n super(JointsKLLoss, self).__init__()\n self.criterion = nn.KLDivLoss(reduction='none')\n self.reduction = reduction\n self.epsilon = epsilon\n\n def forward(self, output, target, target_weight=None):\n B, K, _, _ = output.shape\n heatmaps_pred = output.reshape((B, K, -1))\n heatmaps_pred = F.log_softmax(heatmaps_pred, dim=-1)\n heatmaps_gt = target.reshape((B, K, -1))\n heatmaps_gt = heatmaps_gt + self.epsilon\n heatmaps_gt = heatmaps_gt / heatmaps_gt.sum(dim=-1, keepdims=True)\n loss = self.criterion(heatmaps_pred, heatmaps_gt).sum(dim=-1)\n if target_weight is not None:\n loss = loss * target_weight.view((B, K))\n if self.reduction == 'mean':\n return loss.mean()\n elif self.reduction == 'none':\n return loss.mean(dim=-1)\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/common/vision/models/keypoint_detection/pose_resnet.py",
"content": "\"\"\"\nModified from https://github.com/microsoft/human-pose-estimation.pytorch\n@author: Junguang Jiang\n@contact: JiangJunguang1123@outlook.com\n\"\"\"\nimport torch.nn as nn\nfrom ..resnet import _resnet, Bottleneck\n\n\nclass Upsampling(nn.Sequential):\n \"\"\"\n 3-layers deconvolution used in `Simple Baseline `_.\n \"\"\"\n def __init__(self, in_channel=2048, hidden_dims=(256, 256, 256), kernel_sizes=(4, 4, 4), bias=False):\n assert len(hidden_dims) == len(kernel_sizes), \\\n 'ERROR: len(hidden_dims) is different len(kernel_sizes)'\n\n layers = []\n for hidden_dim, kernel_size in zip(hidden_dims, kernel_sizes):\n if kernel_size == 4:\n padding = 1\n output_padding = 0\n elif kernel_size == 3:\n padding = 1\n output_padding = 1\n elif kernel_size == 2:\n padding = 0\n output_padding = 0\n else:\n raise NotImplementedError(\"kernel_size is {}\".format(kernel_size))\n\n layers.append(\n nn.ConvTranspose2d(\n in_channels=in_channel,\n out_channels=hidden_dim,\n kernel_size=kernel_size,\n stride=2,\n padding=padding,\n output_padding=output_padding,\n bias=bias))\n layers.append(nn.BatchNorm2d(hidden_dim))\n layers.append(nn.ReLU(inplace=True))\n in_channel = hidden_dim\n\n super(Upsampling, self).__init__(*layers)\n\n # init following Simple Baseline\n for name, m in self.named_modules():\n if isinstance(m, nn.ConvTranspose2d):\n nn.init.normal_(m.weight, std=0.001)\n if bias:\n nn.init.constant_(m.bias, 0)\n elif isinstance(m, nn.BatchNorm2d):\n nn.init.constant_(m.weight, 1)\n nn.init.constant_(m.bias, 0)\n\n\nclass PoseResNet(nn.Module):\n \"\"\"\n `Simple Baseline `_ for keypoint detection.\n\n Args:\n backbone (torch.nn.Module): Backbone to extract 2-d features from data\n upsampling (torch.nn.Module): Layer to upsample image feature to heatmap size\n feature_dim (int): The dimension of the features from upsampling layer.\n num_keypoints (int): Number of keypoints\n finetune (bool, optional): Whether use 10x smaller learning rate in the backbone. Default: False\n \"\"\"\n def __init__(self, backbone, upsampling, feature_dim, num_keypoints, finetune=False):\n super(PoseResNet, self).__init__()\n self.backbone = backbone\n self.upsampling = upsampling\n self.head = nn.Conv2d(in_channels=feature_dim, out_channels=num_keypoints, kernel_size=1, stride=1, padding=0)\n self.finetune = finetune\n for m in self.head.modules():\n nn.init.normal_(m.weight, std=0.001)\n nn.init.constant_(m.bias, 0)\n\n def forward(self, x):\n x = self.backbone(x)\n x = self.upsampling(x)\n x = self.head(x)\n return x\n\n def get_parameters(self, lr=1.):\n return [\n {'params': self.backbone.parameters(), 'lr': 0.1 * lr if self.finetune else lr},\n {'params': self.upsampling.parameters(), 'lr': lr},\n {'params': self.head.parameters(), 'lr': lr},\n ]\n\n\ndef _pose_resnet(arch, num_keypoints, block, layers, pretrained_backbone, deconv_with_bias, finetune=False, progress=True, **kwargs):\n backbone = _resnet(arch, block, layers, pretrained_backbone, progress, **kwargs)\n upsampling = Upsampling(backbone.out_features, bias=deconv_with_bias)\n model = PoseResNet(backbone, upsampling, 256, num_keypoints, finetune)\n return model\n\n\ndef pose_resnet101(num_keypoints, pretrained_backbone=True, deconv_with_bias=False, finetune=False, progress=True, **kwargs):\n \"\"\"Constructs a Simple Baseline model with a ResNet-101 backbone.\n\n Args:\n num_keypoints (int): number of keypoints\n pretrained_backbone (bool, optional): If True, returns a model pre-trained on ImageNet. Default: True.\n deconv_with_bias (bool, optional): Whether use bias in the deconvolution layer. Default: False\n finetune (bool, optional): Whether use 10x smaller learning rate in the backbone. Default: False\n progress (bool, optional): If True, displays a progress bar of the download to stderr. Default: True\n \"\"\"\n return _pose_resnet('resnet101', num_keypoints, Bottleneck, [3, 4, 23, 3], pretrained_backbone, deconv_with_bias, finetune, progress, **kwargs)"
},
{
"path": "domain_generalization/Transfer-Learning-Library/common/vision/models/reid/__init__.py",
"content": "from .resnet import *\nfrom ..ibn import *\n\n__all__ = ['resnet']\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/common/vision/models/reid/identifier.py",
"content": "\"\"\"\n@author: Baixu Chen\n@contact: cbx_99_hasta@outlook.com\n\"\"\"\nfrom typing import List, Dict, Optional\nimport torch\nimport torch.nn as nn\nfrom torch.nn import init\n\n\nclass ReIdentifier(nn.Module):\n r\"\"\"Person reIdentifier from `Bag of Tricks and A Strong Baseline for Deep Person Re-identification (CVPR 2019)\n `_.\n Given 2-d features :math:`f` from backbone network, the authors pass :math:`f` through another `BatchNorm1d` layer\n and get :math:`bn\\_f`, which will then pass through a `Linear` layer to output predictions. During training, we\n use :math:`f` to compute triplet loss. While during testing, :math:`bn\\_f` is used as feature. This may be a little\n confusing. The figures in the origin paper will help you understand better.\n \"\"\"\n\n def __init__(self, backbone: nn.Module, num_classes: int, bottleneck: Optional[nn.Module] = None,\n bottleneck_dim: Optional[int] = -1, finetune=True, pool_layer=None):\n super(ReIdentifier, self).__init__()\n if pool_layer is None:\n self.pool_layer = nn.Sequential(\n nn.AdaptiveAvgPool2d(output_size=(1, 1)),\n nn.Flatten()\n )\n else:\n self.pool_layer = pool_layer\n self.backbone = backbone\n self.num_classes = num_classes\n if bottleneck is None:\n feature_bn = nn.BatchNorm1d(backbone.out_features)\n self.bottleneck = feature_bn\n self._features_dim = backbone.out_features\n else:\n feature_bn = nn.BatchNorm1d(bottleneck_dim)\n self.bottleneck = nn.Sequential(\n bottleneck,\n feature_bn\n )\n self._features_dim = bottleneck_dim\n\n self.head = nn.Linear(self.features_dim, num_classes, bias=False)\n self.finetune = finetune\n\n # initialize feature_bn and head\n feature_bn.bias.requires_grad_(False)\n init.constant_(feature_bn.weight, 1)\n init.constant_(feature_bn.bias, 0)\n init.normal_(self.head.weight, std=0.001)\n\n @property\n def features_dim(self) -> int:\n \"\"\"The dimension of features before the final `head` layer\"\"\"\n return self._features_dim\n\n def forward(self, x: torch.Tensor):\n \"\"\"\"\"\"\n f = self.pool_layer(self.backbone(x))\n bn_f = self.bottleneck(f)\n if not self.training:\n return bn_f\n predictions = self.head(bn_f)\n return predictions, f\n\n def get_parameters(self, base_lr=1.0, rate=0.1) -> List[Dict]:\n \"\"\"A parameter list which decides optimization hyper-parameters,\n such as the relative learning rate of each layer\n \"\"\"\n params = [\n {\"params\": self.backbone.parameters(), \"lr\": rate * base_lr if self.finetune else 1.0 * base_lr},\n {\"params\": self.bottleneck.parameters(), \"lr\": 1.0 * base_lr},\n {\"params\": self.head.parameters(), \"lr\": 1.0 * base_lr},\n ]\n\n return params\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/common/vision/models/reid/loss.py",
"content": "\"\"\"\nModified from https://github.com/yxgeee/MMT\n@author: Baixu Chen\n@contact: cbx_99_hasta@outlook.com\n\"\"\"\nimport torch\nimport torch.nn as nn\nimport torch.nn.functional as F\n\n\ndef pairwise_euclidean_distance(x, y):\n \"\"\"Compute pairwise euclidean distance between two sets of features\"\"\"\n m, n = x.size(0), y.size(0)\n dist_mat = torch.pow(x, 2).sum(1, keepdim=True).expand(m, n) + \\\n torch.pow(y, 2).sum(1, keepdim=True).expand(n, m).t() \\\n - 2 * torch.matmul(x, y.t())\n # for numerical stability\n dist_mat = dist_mat.clamp(min=1e-12).sqrt()\n return dist_mat\n\n\ndef hard_examples_mining(dist_mat, identity_mat, return_idxes=False):\n r\"\"\"Select hard positives and hard negatives according to `In defense of the Triplet Loss for Person\n Re-Identification (ICCV 2017) `_\n\n Args:\n dist_mat (tensor): pairwise distance matrix between two sets of features\n identity_mat (tensor): a matrix of shape :math:`(N, M)`. If two images :math:`P[i]` of set :math:`P` and\n :math:`Q[j]` of set :math:`Q` come from the same person, then :math:`identity\\_mat[i, j] = 1`,\n otherwise :math:`identity\\_mat[i, j] = 0`\n return_idxes (bool, optional): if True, also return indexes of hard examples. Default: False\n \"\"\"\n # the implementation here is a little tricky, dist_mat contains pairwise distance between probe image and other\n # images in current mini-batch. As we want to select positive examples of the same person, we add a constant\n # negative offset on other images before sorting. As a result, images of the **same** person will rank first.\n sorted_dist_mat, sorted_idxes = torch.sort(dist_mat + (-1e7) * (1 - identity_mat), dim=1,\n descending=True)\n dist_ap = sorted_dist_mat[:, 0]\n hard_positive_idxes = sorted_idxes[:, 0]\n\n # the implementation here is similar to above code, we add a constant positive offset on images of same person\n # before sorting. Besides, we sort in ascending order. As a result, images of **different** persons will rank first.\n sorted_dist_mat, sorted_idxes = torch.sort(dist_mat + 1e7 * identity_mat, dim=1,\n descending=False)\n dist_an = sorted_dist_mat[:, 0]\n hard_negative_idxes = sorted_idxes[:, 0]\n if return_idxes:\n return dist_ap, dist_an, hard_positive_idxes, hard_negative_idxes\n return dist_ap, dist_an\n\n\nclass CrossEntropyLossWithLabelSmooth(nn.Module):\n r\"\"\"Cross entropy loss with label smooth from `Rethinking the Inception Architecture for Computer Vision\n (CVPR 2016) `_.\n\n Given one-hot labels :math:`labels \\in R^C`, where :math:`C` is the number of classes,\n smoothed labels are calculated as\n\n .. math::\n smoothed\\_labels = (1 - \\epsilon) \\times labels + \\epsilon \\times \\frac{1}{C}\n\n We use smoothed labels when calculating cross entropy loss and this can be helpful for preventing over-fitting.\n\n Args:\n num_classes (int): number of classes.\n epsilon (float): a float number that controls the smoothness.\n\n Inputs:\n - y (tensor): unnormalized classifier predictions, :math:`y`\n - labels (tensor): ground truth labels, :math:`labels`\n\n Shape:\n - y: :math:`(minibatch, C)`, where :math:`C` is the number of classes\n - labels: :math:`(minibatch, )`\n \"\"\"\n\n def __init__(self, num_classes, epsilon=0.1):\n super(CrossEntropyLossWithLabelSmooth, self).__init__()\n self.num_classes = num_classes\n self.epsilon = epsilon\n self.log_softmax = nn.LogSoftmax(dim=1).cuda()\n\n def forward(self, y, labels):\n log_prob = self.log_softmax(y)\n labels = torch.zeros_like(log_prob).scatter_(1, labels.unsqueeze(1), 1)\n labels = (1 - self.epsilon) * labels + self.epsilon / self.num_classes\n loss = (- labels * log_prob).mean(0).sum()\n return loss\n\n\nclass TripletLoss(nn.Module):\n \"\"\"Triplet loss augmented with batch hard from `In defense of the Triplet Loss for Person Re-Identification\n (ICCV 2017) `_.\n\n Args:\n margin (float): margin of triplet loss\n normalize_feature (bool, optional): if True, normalize features into unit norm first before computing loss.\n Default: False.\n \"\"\"\n\n def __init__(self, margin, normalize_feature=False):\n super(TripletLoss, self).__init__()\n self.margin = margin\n self.normalize_feature = normalize_feature\n self.margin_loss = nn.MarginRankingLoss(margin=margin).cuda()\n\n def forward(self, f, labels):\n if self.normalize_feature:\n # equivalent to cosine similarity\n f = F.normalize(f)\n dist_mat = pairwise_euclidean_distance(f, f)\n\n n = dist_mat.size(0)\n identity_mat = labels.expand(n, n).eq(labels.expand(n, n).t()).float()\n\n dist_ap, dist_an = hard_examples_mining(dist_mat, identity_mat)\n y = torch.ones_like(dist_ap)\n loss = self.margin_loss(dist_an, dist_ap, y)\n return loss\n\n\nclass TripletLossXBM(nn.Module):\n r\"\"\"Triplet loss augmented with batch hard from `In defense of the Triplet Loss for Person Re-Identification\n (ICCV 2017) `_. The only difference from triplet loss lies in that\n both features from current mini batch and external storage (XBM) are involved.\n\n Args:\n margin (float, optional): margin of triplet loss. Default: 0.3\n normalize_feature (bool, optional): if True, normalize features into unit norm first before computing loss.\n Default: False\n\n Inputs:\n - f (tensor): features of current mini batch, :math:`f`\n - labels (tensor): identity labels for current mini batch, :math:`labels`\n - xbm_f (tensor): features collected from XBM, :math:`xbm\\_f`\n - xbm_labels (tensor): corresponding identity labels of xbm_f, :math:`xbm\\_labels`\n\n Shape:\n - f: :math:`(minibatch, F)`, where :math:`F` is the feature dimension\n - labels: :math:`(minibatch, )`\n - xbm_f: :math:`(minibatch, F)`\n - xbm_labels: :math:`(minibatch, )`\n \"\"\"\n\n def __init__(self, margin=0.3, normalize_feature=False):\n super(TripletLossXBM, self).__init__()\n self.margin = margin\n self.normalize_feature = normalize_feature\n self.ranking_loss = nn.MarginRankingLoss(margin=margin)\n\n def forward(self, f, labels, xbm_f, xbm_labels):\n if self.normalize_feature:\n # equivalent to cosine similarity\n f = F.normalize(f)\n xbm_f = F.normalize(xbm_f)\n\n dist_mat = pairwise_euclidean_distance(f, xbm_f)\n\n # hard examples mining\n n, m = f.size(0), xbm_f.size(0)\n identity_mat = labels.expand(m, n).t().eq(xbm_labels.expand(n, m)).float()\n dist_ap, dist_an = hard_examples_mining(dist_mat, identity_mat)\n\n # Compute ranking hinge loss\n y = torch.ones_like(dist_an)\n loss = self.ranking_loss(dist_an, dist_ap, y)\n\n return loss\n\n\nclass SoftTripletLoss(nn.Module):\n r\"\"\"Soft triplet loss from `Mutual Mean-Teaching: Pseudo Label Refinery for Unsupervised\n Domain Adaptation on Person Re-identification (ICLR 2020) `_.\n Consider a triplet :math:`x,x_p,x_n` (anchor, positive, negative), corresponding features are :math:`f,f_p,f_n`.\n We optimize for a smaller distance between :math:`f` and :math:`f_p` and a larger distance\n between :math:`f` and :math:`f_n`. Inner product is adopted as their similarity measure, soft triplet loss is thus\n defined as\n\n .. math::\n loss = \\mathcal{L}_{\\text{bce}}(\\frac{\\text{exp}(f^Tf_p)}{\\text{exp}(f^Tf_p)+\\text{exp}(f^Tf_n)}, 1)\n\n where :math:`\\mathcal{L}_{\\text{bce}}` means binary cross entropy loss. We denote the first term in above loss function\n as :math:`T`. When features from another teacher network can be obtained, we can calculate :math:`T_{teacher}` as\n labels, resulting in the following soft version\n\n .. math::\n loss = \\mathcal{L}_{\\text{bce}}(T, T_{teacher})\n\n Args:\n margin (float, optional): margin of triplet loss. If None, soft labels from another network will be adopted when\n computing loss. Default: None.\n normalize_feature (bool, optional): if True, normalize features into unit norm first before computing loss.\n Default: False.\n \"\"\"\n\n def __init__(self, margin=None, normalize_feature=False):\n super(SoftTripletLoss, self).__init__()\n self.margin = margin\n self.normalize_feature = normalize_feature\n\n def forward(self, features_1, features_2, labels):\n if self.normalize_feature:\n # equal to cosine similarity\n features_1 = F.normalize(features_1)\n features_2 = F.normalize(features_2)\n\n dist_mat = pairwise_euclidean_distance(features_1, features_1)\n assert dist_mat.size(0) == dist_mat.size(1)\n\n n = dist_mat.size(0)\n identity_mat = labels.expand(n, n).eq(labels.expand(n, n).t()).float()\n\n dist_ap, dist_an, ap_idxes, an_idxes = hard_examples_mining(dist_mat, identity_mat, return_idxes=True)\n assert dist_an.size(0) == dist_ap.size(0)\n triple_dist = torch.stack((dist_ap, dist_an), dim=1)\n triple_dist = F.log_softmax(triple_dist, dim=1)\n if self.margin is not None:\n loss = (- self.margin * triple_dist[:, 0] - (1 - self.margin) * triple_dist[:, 1]).mean()\n return loss\n\n dist_mat_ref = pairwise_euclidean_distance(features_2, features_2)\n dist_ap_ref = torch.gather(dist_mat_ref, 1, ap_idxes.view(n, 1).expand(n, n))[:, 0]\n dist_an_ref = torch.gather(dist_mat_ref, 1, an_idxes.view(n, 1).expand(n, n))[:, 0]\n triple_dist_ref = torch.stack((dist_ap_ref, dist_an_ref), dim=1)\n triple_dist_ref = F.softmax(triple_dist_ref, dim=1).detach()\n\n loss = (- triple_dist_ref * triple_dist).sum(dim=1).mean()\n return loss\n\n\nclass CrossEntropyLoss(nn.Module):\n r\"\"\"We use :math:`C` to denote the number of classes, :math:`N` to denote mini-batch\n size, this criterion expects unnormalized predictions :math:`y\\_{logits}` of shape :math:`(N, C)` and\n :math:`target\\_{logits}` of the same shape :math:`(N, C)`. Then we first normalize them into\n probability distributions among classes\n\n .. math::\n y = \\text{softmax}(y\\_{logits})\n .. math::\n target = \\text{softmax}(target\\_{logits})\n\n Final objective is calculated as\n\n .. math::\n \\text{loss} = \\frac{1}{N} \\sum_{i=1}^{N} \\sum_{j=1}^C -target_i^j \\times \\text{log} (y_i^j)\n \"\"\"\n\n def __init__(self):\n super(CrossEntropyLoss, self).__init__()\n self.log_softmax = nn.LogSoftmax(dim=1).cuda()\n\n def forward(self, y, labels):\n log_prob = self.log_softmax(y)\n loss = (- F.softmax(labels, dim=1).detach() * log_prob).sum(dim=1).mean()\n return loss\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/common/vision/models/reid/resnet.py",
"content": "\"\"\"\n@author: Baixu Chen\n@contact: cbx_99_hasta@outlook.com\n\"\"\"\nfrom common.vision.models.resnet import ResNet, load_state_dict_from_url, model_urls, BasicBlock, Bottleneck\n\n__all__ = ['reid_resnet18', 'reid_resnet34', 'reid_resnet50', 'reid_resnet101']\n\n\nclass ReidResNet(ResNet):\n r\"\"\"Modified `ResNet` architecture for ReID from `Mutual Mean-Teaching: Pseudo Label Refinery for Unsupervised\n Domain Adaptation on Person Re-identification (ICLR 2020) `_. We change stride\n of :math:`layer4\\_group1\\_conv2, layer4\\_group1\\_downsample1` to 1. During forward pass, we will not activate\n `self.relu`. Please refer to source code for details.\n \"\"\"\n\n def __init__(self, *args, **kwargs):\n super(ReidResNet, self).__init__(*args, **kwargs)\n self.layer4[0].conv2.stride = (1, 1)\n self.layer4[0].downsample[0].stride = (1, 1)\n\n def forward(self, x):\n x = self.conv1(x)\n x = self.bn1(x)\n # x = self.relu(x)\n x = self.maxpool(x)\n\n x = self.layer1(x)\n x = self.layer2(x)\n x = self.layer3(x)\n x = self.layer4(x)\n\n return x\n\n\ndef _reid_resnet(arch, block, layers, pretrained, progress, **kwargs):\n model = ReidResNet(block, layers, **kwargs)\n if pretrained:\n model_dict = model.state_dict()\n pretrained_dict = load_state_dict_from_url(model_urls[arch],\n progress=progress)\n # remove keys from pretrained dict that doesn't appear in model dict\n pretrained_dict = {k: v for k, v in pretrained_dict.items() if k in model_dict}\n model.load_state_dict(pretrained_dict, strict=False)\n return model\n\n\ndef reid_resnet18(pretrained=False, progress=True, **kwargs):\n r\"\"\"Constructs a Reid-ResNet-18 model.\n\n Args:\n pretrained (bool): If True, returns a model pre-trained on ImageNet\n progress (bool): If True, displays a progress bar of the download to stderr\n \"\"\"\n return _reid_resnet('resnet18', BasicBlock, [2, 2, 2, 2], pretrained, progress,\n **kwargs)\n\n\ndef reid_resnet34(pretrained=False, progress=True, **kwargs):\n r\"\"\"Constructs a Reid-ResNet-34 model.\n\n Args:\n pretrained (bool): If True, returns a model pre-trained on ImageNet\n progress (bool): If True, displays a progress bar of the download to stderr\n \"\"\"\n return _reid_resnet('resnet34', BasicBlock, [3, 4, 6, 3], pretrained, progress,\n **kwargs)\n\n\ndef reid_resnet50(pretrained=False, progress=True, **kwargs):\n r\"\"\"Constructs a Reid-ResNet-50 model.\n\n Args:\n pretrained (bool): If True, returns a model pre-trained on ImageNet\n progress (bool): If True, displays a progress bar of the download to stderr\n \"\"\"\n return _reid_resnet('resnet50', Bottleneck, [3, 4, 6, 3], pretrained, progress,\n **kwargs)\n\n\ndef reid_resnet101(pretrained=False, progress=True, **kwargs):\n r\"\"\"Constructs a Reid-ResNet-101 model.\n\n Args:\n pretrained (bool): If True, returns a model pre-trained on ImageNet\n progress (bool): If True, displays a progress bar of the download to stderr\n \"\"\"\n return _reid_resnet('resnet101', Bottleneck, [3, 4, 23, 3], pretrained, progress,\n **kwargs)\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/common/vision/models/resnet.py",
"content": "\"\"\"\nModified based on torchvision.models.resnet.\n@author: Junguang Jiang\n@contact: JiangJunguang1123@outlook.com\n\"\"\"\n\nimport torch.nn as nn\nfrom torchvision import models\nfrom torchvision.models.utils import load_state_dict_from_url\nfrom torchvision.models.resnet import BasicBlock, Bottleneck, model_urls\nimport copy\n\n__all__ = ['ResNet', 'resnet18', 'resnet34', 'resnet50', 'resnet101',\n 'resnet152', 'resnext50_32x4d', 'resnext101_32x8d',\n 'wide_resnet50_2', 'wide_resnet101_2']\n\n\nclass ResNet(models.ResNet):\n \"\"\"ResNets without fully connected layer\"\"\"\n\n def __init__(self, *args, **kwargs):\n super(ResNet, self).__init__(*args, **kwargs)\n self._out_features = self.fc.in_features\n\n def forward(self, x):\n \"\"\"\"\"\"\n x = self.conv1(x)\n x = self.bn1(x)\n x = self.relu(x)\n x = self.maxpool(x)\n\n x = self.layer1(x)\n x = self.layer2(x)\n x = self.layer3(x)\n x = self.layer4(x)\n\n # x = self.avgpool(x)\n # x = torch.flatten(x, 1)\n # x = x.view(-1, self._out_features)\n return x\n\n @property\n def out_features(self) -> int:\n \"\"\"The dimension of output features\"\"\"\n return self._out_features\n\n def copy_head(self) -> nn.Module:\n \"\"\"Copy the origin fully connected layer\"\"\"\n return copy.deepcopy(self.fc)\n\n\ndef _resnet(arch, block, layers, pretrained, progress, **kwargs):\n model = ResNet(block, layers, **kwargs)\n if pretrained:\n model_dict = model.state_dict()\n pretrained_dict = load_state_dict_from_url(model_urls[arch],\n progress=progress)\n # remove keys from pretrained dict that doesn't appear in model dict\n pretrained_dict = {k: v for k, v in pretrained_dict.items() if k in model_dict}\n model.load_state_dict(pretrained_dict, strict=False)\n return model\n\n\ndef resnet18(pretrained=False, progress=True, **kwargs):\n r\"\"\"ResNet-18 model from\n `\"Deep Residual Learning for Image Recognition\" `_\n\n Args:\n pretrained (bool): If True, returns a model pre-trained on ImageNet\n progress (bool): If True, displays a progress bar of the download to stderr\n \"\"\"\n return _resnet('resnet18', BasicBlock, [2, 2, 2, 2], pretrained, progress,\n **kwargs)\n\n\ndef resnet34(pretrained=False, progress=True, **kwargs):\n r\"\"\"ResNet-34 model from\n `\"Deep Residual Learning for Image Recognition\" `_\n\n Args:\n pretrained (bool): If True, returns a model pre-trained on ImageNet\n progress (bool): If True, displays a progress bar of the download to stderr\n \"\"\"\n return _resnet('resnet34', BasicBlock, [3, 4, 6, 3], pretrained, progress,\n **kwargs)\n\n\ndef resnet50(pretrained=False, progress=True, **kwargs):\n r\"\"\"ResNet-50 model from\n `\"Deep Residual Learning for Image Recognition\" `_\n\n Args:\n pretrained (bool): If True, returns a model pre-trained on ImageNet\n progress (bool): If True, displays a progress bar of the download to stderr\n \"\"\"\n return _resnet('resnet50', Bottleneck, [3, 4, 6, 3], pretrained, progress,\n **kwargs)\n\n\ndef resnet101(pretrained=False, progress=True, **kwargs):\n r\"\"\"ResNet-101 model from\n `\"Deep Residual Learning for Image Recognition\" `_\n\n Args:\n pretrained (bool): If True, returns a model pre-trained on ImageNet\n progress (bool): If True, displays a progress bar of the download to stderr\n \"\"\"\n return _resnet('resnet101', Bottleneck, [3, 4, 23, 3], pretrained, progress,\n **kwargs)\n\n\ndef resnet152(pretrained=False, progress=True, **kwargs):\n r\"\"\"ResNet-152 model from\n `\"Deep Residual Learning for Image Recognition\" `_\n\n Args:\n pretrained (bool): If True, returns a model pre-trained on ImageNet\n progress (bool): If True, displays a progress bar of the download to stderr\n \"\"\"\n return _resnet('resnet152', Bottleneck, [3, 8, 36, 3], pretrained, progress,\n **kwargs)\n\n\ndef resnext50_32x4d(pretrained=False, progress=True, **kwargs):\n r\"\"\"ResNeXt-50 32x4d model from\n `\"Aggregated Residual Transformation for Deep Neural Networks\" `_\n\n Args:\n pretrained (bool): If True, returns a model pre-trained on ImageNet\n progress (bool): If True, displays a progress bar of the download to stderr\n \"\"\"\n kwargs['groups'] = 32\n kwargs['width_per_group'] = 4\n return _resnet('resnext50_32x4d', Bottleneck, [3, 4, 6, 3],\n pretrained, progress, **kwargs)\n\n\ndef resnext101_32x8d(pretrained=False, progress=True, **kwargs):\n r\"\"\"ResNeXt-101 32x8d model from\n `\"Aggregated Residual Transformation for Deep Neural Networks\" `_\n\n Args:\n pretrained (bool): If True, returns a model pre-trained on ImageNet\n progress (bool): If True, displays a progress bar of the download to stderr\n \"\"\"\n kwargs['groups'] = 32\n kwargs['width_per_group'] = 8\n return _resnet('resnext101_32x8d', Bottleneck, [3, 4, 23, 3],\n pretrained, progress, **kwargs)\n\n\ndef wide_resnet50_2(pretrained=False, progress=True, **kwargs):\n r\"\"\"Wide ResNet-50-2 model from\n `\"Wide Residual Networks\" `_\n\n The model is the same as ResNet except for the bottleneck number of channels\n which is twice larger in every block. The number of channels in outer 1x1\n convolutions is the same, e.g. last block in ResNet-50 has 2048-512-2048\n channels, and in Wide ResNet-50-2 has 2048-1024-2048.\n\n Args:\n pretrained (bool): If True, returns a model pre-trained on ImageNet\n progress (bool): If True, displays a progress bar of the download to stderr\n \"\"\"\n kwargs['width_per_group'] = 64 * 2\n return _resnet('wide_resnet50_2', Bottleneck, [3, 4, 6, 3],\n pretrained, progress, **kwargs)\n\n\ndef wide_resnet101_2(pretrained=False, progress=True, **kwargs):\n r\"\"\"Wide ResNet-101-2 model from\n `\"Wide Residual Networks\" `_\n\n The model is the same as ResNet except for the bottleneck number of channels\n which is twice larger in every block. The number of channels in outer 1x1\n convolutions is the same, e.g. last block in ResNet-50 has 2048-512-2048\n channels, and in Wide ResNet-50-2 has 2048-1024-2048.\n\n Args:\n pretrained (bool): If True, returns a model pre-trained on ImageNet\n progress (bool): If True, displays a progress bar of the download to stderr\n \"\"\"\n kwargs['width_per_group'] = 64 * 2\n return _resnet('wide_resnet101_2', Bottleneck, [3, 4, 23, 3],\n pretrained, progress, **kwargs)\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/common/vision/models/segmentation/__init__.py",
"content": "from .deeplabv2 import *\n\n__all__ = ['deeplabv2']"
},
{
"path": "domain_generalization/Transfer-Learning-Library/common/vision/models/segmentation/deeplabv2.py",
"content": "\"\"\"\n@author: Junguang Jiang\n@contact: JiangJunguang1123@outlook.com\n\"\"\"\nimport torch.nn as nn\nfrom torchvision.models.utils import load_state_dict_from_url\n\n\nmodel_urls = {\n 'deeplabv2_resnet101': 'https://cloud.tsinghua.edu.cn/f/2d9a7fc43ce34f76803a/?dl=1'\n}\n\naffine_par = True\n\n\nclass Bottleneck(nn.Module):\n expansion = 4\n\n def __init__(self, inplanes, planes, stride=1, dilation=1, downsample=None):\n super(Bottleneck, self).__init__()\n self.conv1 = nn.Conv2d(inplanes, planes, kernel_size=1, stride=stride, bias=False) # change\n self.bn1 = nn.BatchNorm2d(planes, affine=affine_par)\n for i in self.bn1.parameters():\n i.requires_grad = False\n\n padding = dilation\n\n self.conv2 = nn.Conv2d(planes, planes, kernel_size=3, stride=1, # change\n padding=padding, bias=False, dilation=dilation)\n self.bn2 = nn.BatchNorm2d(planes, affine=affine_par)\n for i in self.bn2.parameters():\n i.requires_grad = False\n\n self.conv3 = nn.Conv2d(planes, planes * 4, kernel_size=1, bias=False)\n self.bn3 = nn.BatchNorm2d(planes * 4, affine=affine_par)\n for i in self.bn3.parameters():\n i.requires_grad = False\n\n self.relu = nn.ReLU(inplace=True)\n self.downsample = downsample\n self.stride = stride\n\n def forward(self, x):\n residual = x\n\n out = self.conv1(x)\n out = self.bn1(out)\n out = self.relu(out)\n\n out = self.conv2(out)\n out = self.bn2(out)\n out = self.relu(out)\n\n out = self.conv3(out)\n out = self.bn3(out)\n\n if self.downsample is not None:\n residual = self.downsample(x)\n\n out += residual\n out = self.relu(out)\n\n return out\n\n\nclass ASPP_V2(nn.Module):\n def __init__(self, inplanes, dilation_series, padding_series, num_classes):\n super(ASPP_V2, self).__init__()\n self.conv2d_list = nn.ModuleList()\n for dilation, padding in zip(dilation_series, padding_series):\n self.conv2d_list.append(\n nn.Conv2d(inplanes, num_classes, kernel_size=3, stride=1, padding=padding, dilation=dilation,\n bias=True))\n for m in self.conv2d_list:\n m.weight.data.normal_(0, 0.01)\n\n def forward(self, x):\n out = self.conv2d_list[0](x)\n for i in range(len(self.conv2d_list) - 1):\n out += self.conv2d_list[i + 1](x)\n\n return out\n\n\nclass ResNet(nn.Module):\n def __init__(self, block, layers):\n self.inplanes = 64\n super(ResNet, self).__init__()\n self.conv1 = nn.Conv2d(3, 64, kernel_size=7, stride=2, padding=3,\n bias=False)\n self.bn1 = nn.BatchNorm2d(64, affine=affine_par)\n for i in self.bn1.parameters():\n i.requires_grad = False\n self.relu = nn.ReLU(inplace=True)\n self.maxpool = nn.MaxPool2d(kernel_size=3, stride=2, padding=1, ceil_mode=True) # change\n\n self.layer1 = self._make_layer(block, 64, layers[0])\n self.layer2 = self._make_layer(block, 128, layers[1], stride=2)\n self.layer3 = self._make_layer(block, 256, layers[2], stride=1, dilation=2)\n self.layer4 = self._make_layer(block, 512, layers[3], stride=1, dilation=4)\n\n for m in self.modules():\n if isinstance(m, nn.Conv2d):\n m.weight.data.normal_(0, 0.01)\n elif isinstance(m, nn.BatchNorm2d):\n m.weight.data.fill_(1)\n m.bias.data.zero_()\n\n def _make_layer(self, block, planes, blocks, stride=1, dilation=1):\n downsample = None\n if stride != 1 or self.inplanes != planes * block.expansion or dilation == 2 or dilation == 4:\n downsample = nn.Sequential(\n nn.Conv2d(self.inplanes, planes * block.expansion,\n kernel_size=1, stride=stride, bias=False),\n nn.BatchNorm2d(planes * block.expansion, affine=affine_par))\n for i in downsample._modules['1'].parameters():\n i.requires_grad = False\n layers = []\n layers.append(block(self.inplanes, planes, stride, dilation=dilation, downsample=downsample))\n self.inplanes = planes * block.expansion\n for i in range(1, blocks):\n layers.append(block(self.inplanes, planes, dilation=dilation))\n return nn.Sequential(*layers)\n\n def forward(self, x):\n x = self.conv1(x)\n x = self.bn1(x)\n x = self.relu(x)\n x = self.maxpool(x)\n x = self.layer1(x)\n x = self.layer2(x)\n x = self.layer3(x)\n x = self.layer4(x)\n return x\n\n\nclass Deeplab(nn.Module):\n def __init__(self, backbone, classifier, num_classes):\n super(Deeplab, self).__init__()\n self.backbone = backbone\n self.classifier = classifier\n self.num_classes = num_classes\n\n def forward(self, x):\n x = self.backbone(x)\n y = self.classifier(x)\n return y\n\n def get_1x_lr_params_NOscale(self):\n \"\"\"\n This generator returns all the parameters of the net except for\n the last classification layer. Note that for each batchnorm layer,\n requires_grad is set to False in deeplab_resnet.py, therefore this function does not return\n any batchnorm parameter\n \"\"\"\n layers = [self.backbone.conv1, self.backbone.bn1,\n self.backbone.layer1, self.backbone.layer2, self.backbone.layer3, self.backbone.layer4]\n for layer in layers:\n for module in layer.modules():\n for param in module.parameters():\n if param.requires_grad:\n yield param\n\n def get_10x_lr_params(self):\n \"\"\"\n This generator returns all the parameters for the last layer of the net,\n which does the classification of pixel into classes\n \"\"\"\n for param in self.classifier.parameters():\n yield param\n\n def get_parameters(self, lr=1.):\n return [\n {'params': self.get_1x_lr_params_NOscale(), 'lr': 0.1 * lr},\n {'params': self.get_10x_lr_params(), 'lr': lr}\n ]\n\n\ndef deeplabv2_resnet101(num_classes=19, pretrained_backbone=True):\n \"\"\"Constructs a DeepLabV2 model with a ResNet-101 backbone.\n\n Args:\n num_classes (int, optional): number of classes. Default: 19\n pretrained_backbone (bool, optional): If True, returns a model pre-trained on ImageNet. Default: True.\n \"\"\"\n backbone = ResNet(Bottleneck, [3, 4, 23, 3])\n if pretrained_backbone:\n # download from Internet\n saved_state_dict = load_state_dict_from_url(model_urls['deeplabv2_resnet101'], map_location=lambda storage, loc: storage, file_name=\"deeplabv2_resnet101.pth\")\n new_params = backbone.state_dict().copy()\n for i in saved_state_dict:\n i_parts = i.split('.')\n if not i_parts[1] == 'layer5':\n new_params['.'.join(i_parts[1:])] = saved_state_dict[i]\n backbone.load_state_dict(new_params)\n classifier = ASPP_V2(2048, [6, 12, 18, 24], [6, 12, 18, 24], num_classes)\n return Deeplab(backbone, classifier, num_classes)\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/common/vision/transforms/__init__.py",
"content": "import math\nimport random\nfrom PIL import Image\nimport numpy as np\nimport torch\nfrom torchvision.transforms import Normalize\n\n\nclass ResizeImage(object):\n \"\"\"Resize the input PIL Image to the given size.\n\n Args:\n size (sequence or int): Desired output size. If size is a sequence like\n (h, w), output size will be matched to this. If size is an int,\n output size will be (size, size)\n \"\"\"\n\n def __init__(self, size):\n if isinstance(size, int):\n self.size = (int(size), int(size))\n else:\n self.size = size\n\n def __call__(self, img):\n th, tw = self.size\n return img.resize((th, tw))\n\n def __repr__(self):\n return self.__class__.__name__ + '(size={0})'.format(self.size)\n\n\nclass MultipleApply:\n \"\"\"Apply a list of transformations to an image and get multiple transformed images.\n\n Args:\n transforms (list or tuple): list of transformations\n\n Example:\n \n >>> transform1 = T.Compose([\n ... ResizeImage(256),\n ... T.RandomCrop(224)\n ... ])\n >>> transform2 = T.Compose([\n ... ResizeImage(256),\n ... T.RandomCrop(224),\n ... ])\n >>> multiply_transform = MultipleApply([transform1, transform2])\n \"\"\"\n\n def __init__(self, transforms):\n self.transforms = transforms\n\n def __call__(self, image):\n return [t(image) for t in self.transforms]\n\n def __repr__(self):\n format_string = self.__class__.__name__ + '('\n for t in self.transforms:\n format_string += '\\n'\n format_string += ' {0}'.format(t)\n format_string += '\\n)'\n return format_string\n\n\nclass Denormalize(Normalize):\n \"\"\"DeNormalize a tensor image with mean and standard deviation.\n Given mean: ``(mean[1],...,mean[n])`` and std: ``(std[1],..,std[n])`` for ``n``\n channels, this transform will denormalize each channel of the input\n ``torch.*Tensor`` i.e.,\n ``output[channel] = input[channel] * std[channel] + mean[channel]``\n\n .. note::\n This transform acts out of place, i.e., it does not mutate the input tensor.\n\n Args:\n mean (sequence): Sequence of means for each channel.\n std (sequence): Sequence of standard deviations for each channel.\n\n \"\"\"\n\n def __init__(self, mean, std):\n mean = np.array(mean)\n std = np.array(std)\n super().__init__((-mean / std).tolist(), (1 / std).tolist())\n\n\nclass NormalizeAndTranspose:\n \"\"\"\n First, normalize a tensor image with mean and standard deviation.\n Then, convert the shape (H x W x C) to shape (C x H x W).\n \"\"\"\n\n def __init__(self, mean=(104.00698793, 116.66876762, 122.67891434)):\n self.mean = np.array(mean, dtype=np.float32)\n\n def __call__(self, image):\n if isinstance(image, Image.Image):\n image = np.asarray(image, np.float32)\n # change to BGR\n image = image[:, :, ::-1]\n # normalize\n image -= self.mean\n image = image.transpose((2, 0, 1)).copy()\n elif isinstance(image, torch.Tensor):\n # change to BGR\n image = image[:, :, [2, 1, 0]]\n # normalize\n image -= torch.from_numpy(self.mean).to(image.device)\n image = image.permute((2, 0, 1))\n else:\n raise NotImplementedError(type(image))\n return image\n\n\nclass DeNormalizeAndTranspose:\n \"\"\"\n First, convert a tensor image from the shape (C x H x W ) to shape (H x W x C).\n Then, denormalize it with mean and standard deviation.\n \"\"\"\n\n def __init__(self, mean=(104.00698793, 116.66876762, 122.67891434)):\n self.mean = np.array(mean, dtype=np.float32)\n\n def __call__(self, image):\n image = image.transpose((1, 2, 0))\n # denormalize\n image += self.mean\n # change to RGB\n image = image[:, :, ::-1]\n return image\n\n\nclass RandomErasing(object):\n \"\"\"Random erasing augmentation from `Random Erasing Data Augmentation (CVPR 2017)\n `_. This augmentation randomly selects a rectangle region in an image\n and erases its pixels.\n\n Args:\n probability (float): The probability that the Random Erasing operation will be performed.\n sl (float): Minimum proportion of erased area against input image.\n sh (float): Maximum proportion of erased area against input image.\n r1 (float): Minimum aspect ratio of erased area.\n mean (sequence): Value to fill the erased area.\n \"\"\"\n\n def __init__(self, probability=0.5, sl=0.02, sh=0.4, r1=0.3, mean=(0.4914, 0.4822, 0.4465)):\n self.probability = probability\n self.mean = mean\n self.sl = sl\n self.sh = sh\n self.r1 = r1\n\n def __call__(self, img):\n\n if random.uniform(0, 1) >= self.probability:\n return img\n\n for attempt in range(100):\n area = img.size()[1] * img.size()[2]\n\n target_area = random.uniform(self.sl, self.sh) * area\n aspect_ratio = random.uniform(self.r1, 1 / self.r1)\n\n h = int(round(math.sqrt(target_area * aspect_ratio)))\n w = int(round(math.sqrt(target_area / aspect_ratio)))\n\n if w < img.size()[2] and h < img.size()[1]:\n x1 = random.randint(0, img.size()[1] - h)\n y1 = random.randint(0, img.size()[2] - w)\n if img.size()[0] == 3:\n img[0, x1:x1 + h, y1:y1 + w] = self.mean[0]\n img[1, x1:x1 + h, y1:y1 + w] = self.mean[1]\n img[2, x1:x1 + h, y1:y1 + w] = self.mean[2]\n else:\n img[0, x1:x1 + h, y1:y1 + w] = self.mean[0]\n return img\n\n return img\n\n def __repr__(self):\n return self.__class__.__name__ + '(p={})'.format(self.probability)\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/common/vision/transforms/keypoint_detection.py",
"content": "\"\"\"\n@author: Junguang Jiang\n@contact: JiangJunguang1123@outlook.com\n\"\"\"\n# TODO needs better documentation\nimport numpy as np\nfrom PIL import ImageFilter, Image\nimport torchvision.transforms.functional as F\nimport torchvision.transforms.transforms as T\nimport numbers\nimport random\nimport math\nimport warnings\nfrom typing import ClassVar\n\n\ndef wrapper(transform: ClassVar):\n \"\"\" Wrap a transform for classification to a transform for keypoint detection.\n Note that the keypoint detection label will keep the same before and after wrapper.\n\n Args:\n transform (class, callable): transform for classification\n\n Returns:\n transform for keypoint detection\n \"\"\"\n class WrapperTransform(transform):\n def __call__(self, image, **kwargs):\n image = super().__call__(image)\n return image, kwargs\n return WrapperTransform\n\n\nToTensor = wrapper(T.ToTensor)\nNormalize = wrapper(T.Normalize)\nColorJitter = wrapper(T.ColorJitter)\n\n\ndef resize(image: Image.Image, size: int, interpolation=Image.BILINEAR,\n keypoint2d: np.ndarray=None, intrinsic_matrix: np.ndarray=None):\n width, height = image.size\n assert width == height\n factor = float(size) / float(width)\n image = F.resize(image, size, interpolation)\n keypoint2d = np.copy(keypoint2d)\n keypoint2d *= factor\n intrinsic_matrix = np.copy(intrinsic_matrix)\n intrinsic_matrix[0][0] *= factor\n intrinsic_matrix[0][2] *= factor\n intrinsic_matrix[1][1] *= factor\n intrinsic_matrix[1][2] *= factor\n return image, keypoint2d, intrinsic_matrix\n\n\ndef crop(image: Image.Image, top, left, height, width, keypoint2d: np.ndarray):\n image = F.crop(image, top, left, height, width)\n keypoint2d = np.copy(keypoint2d)\n keypoint2d[:, 0] -= left\n keypoint2d[:, 1] -= top\n return image, keypoint2d\n\n\ndef resized_crop(img, top, left, height, width, size, interpolation=Image.BILINEAR,\n keypoint2d: np.ndarray=None, intrinsic_matrix: np.ndarray=None):\n \"\"\"Crop the given PIL Image and resize it to desired size.\n\n Notably used in :class:`~torchvision.transforms.RandomResizedCrop`.\n\n Args:\n img (PIL Image): Image to be cropped. (0,0) denotes the top left corner of the image.\n top (int): Vertical component of the top left corner of the crop box.\n left (int): Horizontal component of the top left corner of the crop box.\n height (int): Height of the crop box.\n width (int): Width of the crop box.\n size (sequence or int): Desired output size. Same semantics as ``resize``.\n interpolation (int, optional): Desired interpolation. Default is\n ``PIL.Image.BILINEAR``.\n Returns:\n PIL Image: Cropped image.\n \"\"\"\n assert isinstance(img, Image.Image), 'img should be PIL Image'\n img, keypoint2d = crop(img, top, left, height, width, keypoint2d)\n img, keypoint2d, intrinsic_matrix = resize(img, size, interpolation, keypoint2d, intrinsic_matrix)\n return img, keypoint2d, intrinsic_matrix\n\n\ndef center_crop(image, output_size, keypoint2d: np.ndarray):\n \"\"\"Crop the given PIL Image and resize it to desired size.\n\n Args:\n img (PIL Image): Image to be cropped. (0,0) denotes the top left corner of the image.\n output_size (sequence or int): (height, width) of the crop box. If int,\n it is used for both directions\n\n Returns:\n PIL Image: Cropped image.\n \"\"\"\n width, height = image.size\n crop_height, crop_width = output_size\n crop_top = int(round((height - crop_height) / 2.))\n crop_left = int(round((width - crop_width) / 2.))\n return crop(image, crop_top, crop_left, crop_height, crop_width, keypoint2d)\n\n\ndef hflip(image: Image.Image, keypoint2d: np.ndarray):\n width, height = image.size\n image = F.hflip(image)\n keypoint2d = np.copy(keypoint2d)\n keypoint2d[:, 0] = width - 1. - keypoint2d[:, 0]\n return image, keypoint2d\n\n\ndef rotate(image: Image.Image, angle, keypoint2d: np.ndarray):\n image = F.rotate(image, angle)\n\n angle = -np.deg2rad(angle)\n keypoint2d = np.copy(keypoint2d)\n rotation_matrix = np.array([\n [np.cos(angle), -np.sin(angle)],\n [np.sin(angle), np.cos(angle)]\n ])\n width, height = image.size\n keypoint2d[:, 0] = keypoint2d[:, 0] - width / 2\n keypoint2d[:, 1] = keypoint2d[:, 1] - height / 2\n keypoint2d = np.matmul(rotation_matrix, keypoint2d.T).T\n keypoint2d[:, 0] = keypoint2d[:, 0] + width / 2\n keypoint2d[:, 1] = keypoint2d[:, 1] + height / 2\n return image, keypoint2d\n\n\ndef resize_pad(img, keypoint2d, size, interpolation=Image.BILINEAR):\n w, h = img.size\n if w < h:\n oh = size\n ow = int(size * w / h)\n img = img.resize((ow, oh), interpolation)\n pad_top = pad_bottom = 0\n pad_left = math.floor((size - ow) / 2)\n pad_right = math.ceil((size - ow) / 2)\n keypoint2d = keypoint2d * oh / h\n keypoint2d[:, 0] += (size - ow) / 2\n else:\n ow = size\n oh = int(size * h / w)\n img = img.resize((ow, oh), interpolation)\n pad_top = math.floor((size - oh) / 2)\n pad_bottom = math.ceil((size - oh) / 2)\n pad_left = pad_right = 0\n keypoint2d = keypoint2d * ow / w\n keypoint2d[:, 1] += (size - oh) / 2\n keypoint2d[:, 0] += (size - ow) / 2\n img = np.asarray(img)\n\n img = np.pad(img, ((pad_top, pad_bottom), (pad_left, pad_right), (0, 0)), 'constant', constant_values=0)\n return Image.fromarray(img), keypoint2d\n\n\nclass Compose(object):\n \"\"\"Composes several transforms together.\n\n Args:\n transforms (list of ``Transform`` objects): list of transforms to compose.\n \"\"\"\n\n def __init__(self, transforms):\n self.transforms = transforms\n\n def __call__(self, image, **kwargs):\n for t in self.transforms:\n image, kwargs = t(image, **kwargs)\n return image, kwargs\n\n\nclass GaussianBlur(object):\n def __init__(self, low=0, high=0.8):\n self.low = low\n self.high = high\n\n def __call__(self, image: Image, **kwargs):\n radius = np.random.uniform(low=self.low, high=self.high)\n image = image.filter(ImageFilter.GaussianBlur(radius))\n return image, kwargs\n\n\nclass Resize(object):\n \"\"\"Resize the input PIL Image to the given size.\n \"\"\"\n\n def __init__(self, size, interpolation=Image.BILINEAR):\n assert isinstance(size, int)\n self.size = size\n self.interpolation = interpolation\n\n def __call__(self, image, keypoint2d: np.ndarray, intrinsic_matrix: np.ndarray, **kwargs):\n image, keypoint2d, intrinsic_matrix = resize(image, self.size, self.interpolation, keypoint2d, intrinsic_matrix)\n kwargs.update(keypoint2d=keypoint2d, intrinsic_matrix=intrinsic_matrix)\n if 'depth' in kwargs:\n kwargs['depth'] = F.resize(kwargs['depth'], self.size)\n return image, kwargs\n\n\nclass ResizePad(object):\n \"\"\"Pad the given image on all sides with the given \"pad\" value to resize the image to the given size.\n \"\"\"\n def __init__(self, size, interpolation=Image.BILINEAR):\n self.size = size\n self.interpolation = interpolation\n\n def __call__(self, img, keypoint2d, **kwargs):\n image, keypoint2d = resize_pad(img, keypoint2d, self.size, self.interpolation)\n kwargs.update(keypoint2d=keypoint2d)\n return image, kwargs\n\n\nclass CenterCrop(object):\n \"\"\"Crops the given PIL Image at the center.\n \"\"\"\n\n def __init__(self, size):\n if isinstance(size, numbers.Number):\n self.size = (int(size), int(size))\n else:\n self.size = size\n\n def __call__(self, image, keypoint2d, **kwargs):\n \"\"\"\n Args:\n img (PIL Image): Image to be cropped.\n\n Returns:\n PIL Image: Cropped image.\n \"\"\"\n image, keypoint2d = center_crop(image, self.size, keypoint2d)\n kwargs.update(keypoint2d=keypoint2d)\n if 'depth' in kwargs:\n kwargs['depth'] = F.center_crop(kwargs['depth'], self.size)\n return image, kwargs\n\n\nclass RandomRotation(object):\n \"\"\"Rotate the image by angle.\n\n Args:\n degrees (sequence or float or int): Range of degrees to select from.\n If degrees is a number instead of sequence like (min, max), the range of degrees\n will be (-degrees, +degrees).\n \"\"\"\n\n def __init__(self, degrees):\n if isinstance(degrees, numbers.Number):\n if degrees < 0:\n raise ValueError(\"If degrees is a single number, it must be positive.\")\n self.degrees = (-degrees, degrees)\n else:\n if len(degrees) != 2:\n raise ValueError(\"If degrees is a sequence, it must be of len 2.\")\n self.degrees = degrees\n\n\n @staticmethod\n def get_params(degrees):\n \"\"\"Get parameters for ``rotate`` for a random rotation.\n\n Returns:\n sequence: params to be passed to ``rotate`` for random rotation.\n \"\"\"\n angle = random.uniform(degrees[0], degrees[1])\n\n return angle\n\n def __call__(self, image, keypoint2d, **kwargs):\n \"\"\"\n Args:\n img (PIL Image): Image to be rotated.\n\n Returns:\n PIL Image: Rotated image.\n \"\"\"\n\n angle = self.get_params(self.degrees)\n\n image, keypoint2d = rotate(image, angle, keypoint2d)\n kwargs.update(keypoint2d=keypoint2d)\n if 'depth' in kwargs:\n kwargs['depth'] = F.rotate(kwargs['depth'], angle)\n return image, kwargs\n\n\nclass RandomResizedCrop(object):\n \"\"\"Crop the given PIL Image to random size and aspect ratio.\n\n A crop of random size (default: of 0.08 to 1.0) of the original size and a random\n aspect ratio (default: of 3/4 to 4/3) of the original aspect ratio is made. This crop\n is finally resized to given size.\n This is popularly used to train the Inception networks.\n\n Args:\n size: expected output size of each edge\n scale: range of size of the origin size cropped\n ratio: range of aspect ratio of the origin aspect ratio cropped\n interpolation: Default: PIL.Image.BILINEAR\n \"\"\"\n\n def __init__(self, size, scale=(0.6, 1.3), interpolation=Image.BILINEAR):\n self.size = size\n if scale[0] > scale[1]:\n warnings.warn(\"range should be of kind (min, max)\")\n\n self.interpolation = interpolation\n self.scale = scale\n\n @staticmethod\n def get_params(img, scale):\n \"\"\"Get parameters for ``crop`` for a random sized crop.\n\n Args:\n img (PIL Image): Image to be cropped.\n scale (tuple): range of size of the origin size cropped\n\n Returns:\n tuple: params (i, j, h, w) to be passed to ``crop`` for a random\n sized crop.\n \"\"\"\n width, height = img.size\n area = height * width\n\n for attempt in range(10):\n target_area = random.uniform(*scale) * area\n aspect_ratio = 1\n\n w = int(round(math.sqrt(target_area * aspect_ratio)))\n h = int(round(math.sqrt(target_area / aspect_ratio)))\n\n if 0 < w <= width and 0 < h <= height:\n i = random.randint(0, height - h)\n j = random.randint(0, width - w)\n return i, j, h, w\n\n # Fallback to whole image\n return 0, 0, height, width\n\n def __call__(self, image, keypoint2d: np.ndarray, intrinsic_matrix: np.ndarray, **kwargs):\n \"\"\"\n Args:\n img (PIL Image): Image to be cropped and resized.\n\n Returns:\n PIL Image: Randomly cropped and resized image.\n \"\"\"\n i, j, h, w = self.get_params(image, self.scale)\n image, keypoint2d, intrinsic_matrix = resized_crop(image, i, j, h, w, self.size, self.interpolation, keypoint2d, intrinsic_matrix)\n kwargs.update(keypoint2d=keypoint2d, intrinsic_matrix=intrinsic_matrix)\n if 'depth' in kwargs:\n kwargs['depth'] = F.resized_crop(kwargs['depth'], i, j, h, w, self.size, self.interpolation,)\n return image, kwargs\n\n\nclass RandomApply(T.RandomTransforms):\n \"\"\"Apply randomly a list of transformations with a given probability.\n\n Args:\n transforms (list or tuple or torch.nn.Module): list of transformations\n p (float): probability\n \"\"\"\n\n def __init__(self, transforms, p=0.5):\n super(RandomApply, self).__init__(transforms)\n self.p = p\n\n def __call__(self, image, **kwargs):\n if self.p < random.random():\n return image, kwargs\n for t in self.transforms:\n image, kwargs = t(image, **kwargs)\n return image, kwargs\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/common/vision/transforms/segmentation.py",
"content": "\"\"\"\n@author: Junguang Jiang\n@contact: JiangJunguang1123@outlook.com\n\"\"\"\nfrom PIL import Image\nimport random\nimport math\nfrom typing import ClassVar, Sequence, List, Tuple\nfrom torch import Tensor\nimport torch\nimport torchvision.transforms.functional as F\nimport torchvision.transforms.transforms as T\nimport torch.nn as nn\nfrom . import MultipleApply as MultipleApplyBase, NormalizeAndTranspose as NormalizeAndTransposeBase\n\n\ndef wrapper(transform: ClassVar):\n \"\"\" Wrap a transform for classification to a transform for segmentation.\n Note that the segmentation label will keep the same before and after wrapper.\n\n Args:\n transform (class, callable): transform for classification\n\n Returns:\n transform for segmentation\n \"\"\"\n class WrapperTransform(transform):\n def __call__(self, image, label):\n image = super().__call__(image)\n return image, label\n return WrapperTransform\n\n\nColorJitter = wrapper(T.ColorJitter)\nNormalize = wrapper(T.Normalize)\nToTensor = wrapper(T.ToTensor)\nToPILImage = wrapper(T.ToPILImage)\nMultipleApply = wrapper(MultipleApplyBase)\nNormalizeAndTranspose = wrapper(NormalizeAndTransposeBase)\n\n\nclass Compose:\n \"\"\"Composes several transforms together.\n\n Args:\n transforms (list): list of transforms to compose.\n\n Example:\n >>> Compose([\n >>> Resize((512, 512)),\n >>> RandomHorizontalFlip()\n >>> ])\n \"\"\"\n def __init__(self, transforms):\n super(Compose, self).__init__()\n self.transforms = transforms\n\n def __call__(self, image, target):\n for t in self.transforms:\n image, target = t(image, target)\n return image, target\n\n\nclass Resize(nn.Module):\n \"\"\"Resize the input image and the corresponding label to the given size.\n The image should be a PIL Image.\n\n Args:\n image_size (sequence): The requested image size in pixels, as a 2-tuple:\n (width, height).\n label_size (sequence, optional): The requested segmentation label size in pixels, as a 2-tuple:\n (width, height). The same as image_size if None. Default: None.\n \"\"\"\n\n def __init__(self, image_size, label_size=None):\n super(Resize, self).__init__()\n self.image_size = image_size\n if label_size is None:\n self.label_size = image_size\n else:\n self.label_size = label_size\n\n def forward(self, image, label):\n \"\"\"\n Args:\n image: (PIL Image): Image to be scaled.\n label: (PIL Image): Segmentation label to be scaled.\n\n Returns:\n Rescaled image, rescaled segmentation label\n \"\"\"\n # resize\n image = image.resize(self.image_size, Image.BICUBIC)\n label = label.resize(self.label_size, Image.NEAREST)\n return image, label\n\n\nclass RandomCrop(nn.Module):\n \"\"\"Crop the given image at a random location.\n The image can be a PIL Image\n\n Args:\n size (sequence): Desired output size of the crop.\n \"\"\"\n def __init__(self, size):\n super(RandomCrop, self).__init__()\n self.size = size\n\n def forward(self, image, label):\n \"\"\"\n Args:\n image: (PIL Image): Image to be cropped.\n label: (PIL Image): Segmentation label to be cropped.\n\n Returns:\n Cropped image, cropped segmentation label.\n \"\"\"\n # random crop\n left = image.size[0] - self.size[0]\n upper = image.size[1] - self.size[1]\n\n left = random.randint(0, left-1)\n upper = random.randint(0, upper-1)\n right = left + self.size[0]\n lower = upper + self.size[1]\n\n image = image.crop((left, upper, right, lower))\n label = label.crop((left, upper, right, lower))\n return image, label\n\n\nclass RandomHorizontalFlip(nn.Module):\n \"\"\"Horizontally flip the given PIL Image randomly with a given probability.\n\n Args:\n p (float): probability of the image being flipped. Default value is 0.5\n \"\"\"\n\n def __init__(self, p=0.5):\n super(RandomHorizontalFlip, self).__init__()\n self.p = p\n\n def forward(self, image, label):\n \"\"\"\n Args:\n image: (PIL Image): Image to be flipped.\n label: (PIL Image): Segmentation label to be flipped.\n\n Returns:\n Randomly flipped image, randomly flipped segmentation label.\n \"\"\"\n if random.random() < self.p:\n return F.hflip(image), F.hflip(label)\n return image, label\n\n\nclass RandomResizedCrop(T.RandomResizedCrop):\n \"\"\"Crop the given image to random size and aspect ratio.\n The image can be a PIL Image.\n\n A crop of random size (default: of 0.5 to 1.0) of the original size and a random\n aspect ratio (default: of 3/4 to 4/3) of the original aspect ratio is made. This crop\n is finally resized to given size.\n\n Args:\n size (int or sequence): expected output size of each edge. If size is an\n int instead of sequence like (h, w), a square output size ``(size, size)`` is\n made. If provided a tuple or list of length 1, it will be interpreted as (size[0], size[0]).\n scale (tuple of float): range of size of the origin size cropped\n ratio (tuple of float): range of aspect ratio of the origin aspect ratio cropped.\n interpolation: Default: PIL.Image.BILINEAR\n \"\"\"\n\n def __init__(self, size, scale=(0.5, 1.0), ratio=(3. / 4., 4. / 3.), interpolation=Image.BICUBIC):\n super(RandomResizedCrop, self).__init__(size, scale, ratio, interpolation)\n\n @staticmethod\n def get_params(\n img: Tensor, scale: List[float], ratio: List[float]\n ) -> Tuple[int, int, int, int]:\n \"\"\"Get parameters for ``crop`` for a random sized crop.\n\n Args:\n img (PIL Image): Input image.\n scale (list): range of scale of the origin size cropped\n ratio (list): range of aspect ratio of the origin aspect ratio cropped\n\n Returns:\n params (i, j, h, w) to be passed to ``crop`` for a random sized crop.\n \"\"\"\n width, height = F._get_image_size(img)\n area = height * width\n\n for _ in range(10):\n target_area = area * random.uniform(scale[0], scale[1])\n log_ratio = torch.log(torch.tensor(ratio))\n aspect_ratio = math.exp(random.uniform(log_ratio[0], log_ratio[1]))\n\n w = int(round(math.sqrt(target_area * aspect_ratio)))\n h = int(round(math.sqrt(target_area / aspect_ratio)))\n\n if 0 < w <= width and 0 < h <= height:\n i = random.randint(0, height - h)\n j = random.randint(0, width - w)\n return i, j, h, w\n\n # Fallback to central crop\n in_ratio = float(width) / float(height)\n if in_ratio < min(ratio):\n w = width\n h = int(round(w / min(ratio)))\n elif in_ratio > max(ratio):\n h = height\n w = int(round(h * max(ratio)))\n else: # whole image\n w = width\n h = height\n i = (height - h) // 2\n j = (width - w) // 2\n return i, j, h, w\n\n def forward(self, image, label):\n \"\"\"\n Args:\n image: (PIL Image): Image to be cropped and resized.\n label: (PIL Image): Segmentation label to be cropped and resized.\n\n Returns:\n Randomly cropped and resized image, randomly cropped and resized segmentation label.\n \"\"\"\n top, left, height, width = self.get_params(image, self.scale, self.ratio)\n image = image.crop((left, top, left + width, top + height))\n image = image.resize(self.size, self.interpolation)\n label = label.crop((left, top, left + width, top + height))\n label = label.resize(self.size, Image.NEAREST)\n return image, label\n\n\nclass RandomChoice(T.RandomTransforms):\n \"\"\"Apply single transformation randomly picked from a list.\n \"\"\"\n def __call__(self, image, label):\n t = random.choice(self.transforms)\n return t(image, label)\n\n\nclass RandomApply(T.RandomTransforms):\n \"\"\"Apply randomly a list of transformations with a given probability.\n\n Args:\n transforms (list or tuple or torch.nn.Module): list of transformations\n p (float): probability\n \"\"\"\n\n def __init__(self, transforms, p=0.5):\n super(RandomApply, self).__init__(transforms)\n self.p = p\n\n def __call__(self, image, label):\n if self.p < random.random():\n return image\n for t in self.transforms:\n image, label = t(image, label)\n return image\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/dalib/__init__.py",
"content": "__all__ = ['adaptation', 'modules', 'translation']\n\n__version__ = '0.3'"
},
{
"path": "domain_generalization/Transfer-Learning-Library/dalib/adaptation/__init__.py",
"content": "from . import cdan\nfrom . import dann\nfrom . import mdd\nfrom . import dan\nfrom . import jan\nfrom . import mcd\nfrom . import mcc\nfrom . import pada\nfrom . import osbp\nfrom . import iwan\nfrom . import adda\nfrom . import bsp\n\n__all__ = [\"cdan\", \"dann\", \"mdd\", \"dan\", \"jan\", \"mcd\", \"mcc\", \"pada\", \"osbp\", \"iwan\", \"adda\", \"bsp\"]\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/dalib/adaptation/adda.py",
"content": "\"\"\"\n@author: Baixu Chen\n@contact: cbx_99_hasta@outlook.com\n\"\"\"\nfrom typing import Optional, List, Dict\nimport torch\nimport torch.nn as nn\nimport torch.nn.functional as F\nfrom common.modules.classifier import Classifier as ClassifierBase\n\n\nclass DomainAdversarialLoss(nn.Module):\n r\"\"\"Domain adversarial loss from `Adversarial Discriminative Domain Adaptation (CVPR 2017)\n `_.\n Similar to the original `GAN `_ paper, ADDA argues that replacing\n :math:`\\text{log}(1-p)` with :math:`-\\text{log}(p)` in the adversarial loss provides better gradient qualities. Detailed\n optimization process can be found `here\n `_.\n\n Inputs:\n - domain_pred (tensor): predictions of domain discriminator\n - domain_label (str, optional): whether the data comes from source or target.\n Must be 'source' or 'target'. Default: 'source'\n\n Shape:\n - domain_pred: :math:`(minibatch,)`.\n - Outputs: scalar.\n\n \"\"\"\n\n def __init__(self):\n super(DomainAdversarialLoss, self).__init__()\n\n def forward(self, domain_pred, domain_label='source'):\n assert domain_label in ['source', 'target']\n if domain_label == 'source':\n return F.binary_cross_entropy(domain_pred, torch.ones_like(domain_pred).to(domain_pred.device))\n else:\n return F.binary_cross_entropy(domain_pred, torch.zeros_like(domain_pred).to(domain_pred.device))\n\n\nclass ImageClassifier(ClassifierBase):\n def __init__(self, backbone: nn.Module, num_classes: int, bottleneck_dim: Optional[int] = 256, **kwargs):\n bottleneck = nn.Sequential(\n # nn.AdaptiveAvgPool2d(output_size=(1, 1)),\n # nn.Flatten(),\n nn.Linear(backbone.out_features, bottleneck_dim),\n nn.BatchNorm1d(bottleneck_dim),\n nn.ReLU()\n )\n super(ImageClassifier, self).__init__(backbone, num_classes, bottleneck, bottleneck_dim, **kwargs)\n\n def freeze_bn(self):\n for m in self.modules():\n if isinstance(m, nn.BatchNorm2d) or isinstance(m, nn.BatchNorm1d):\n m.eval()\n\n def get_parameters(self, base_lr=1.0, optimize_head=True) -> List[Dict]:\n params = [\n {\"params\": self.backbone.parameters(), \"lr\": 0.1 * base_lr if self.finetune else 1.0 * base_lr},\n {\"params\": self.bottleneck.parameters(), \"lr\": 1.0 * base_lr}\n ]\n if optimize_head:\n params.append({\"params\": self.head.parameters(), \"lr\": 1.0 * base_lr})\n\n return params\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/dalib/adaptation/advent.py",
"content": "\"\"\"\n@author: Junguang Jiang\n@contact: JiangJunguang1123@outlook.com\n\"\"\"\nfrom torch import nn\nimport torch\nimport torch.nn.functional as F\nimport numpy as np\n\n\nclass Discriminator(nn.Sequential):\n \"\"\"\n Domain discriminator model from\n `ADVENT: Adversarial Entropy Minimization for Domain Adaptation in Semantic Segmentation (CVPR 2019) `_\n\n Distinguish pixel-by-pixel whether the input predictions come from the source domain or the target domain.\n The source domain label is 1 and the target domain label is 0.\n\n Args:\n num_classes (int): num of classes in the predictions\n ndf (int): dimension of the hidden features\n\n Shape:\n - Inputs: :math:`(minibatch, C, H, W)` where :math:`C` is the number of classes\n - Outputs: :math:`(minibatch, 1, H, W)`\n \"\"\"\n def __init__(self, num_classes, ndf=64):\n super(Discriminator, self).__init__(\n nn.Conv2d(num_classes, ndf, kernel_size=4, stride=2, padding=1),\n nn.LeakyReLU(negative_slope=0.2, inplace=True),\n nn.Conv2d(ndf, ndf * 2, kernel_size=4, stride=2, padding=1),\n nn.LeakyReLU(negative_slope=0.2, inplace=True),\n nn.Conv2d(ndf * 2, ndf * 4, kernel_size=4, stride=2, padding=1),\n nn.LeakyReLU(negative_slope=0.2, inplace=True),\n nn.Conv2d(ndf * 4, ndf * 8, kernel_size=4, stride=2, padding=1),\n nn.LeakyReLU(negative_slope=0.2, inplace=True),\n nn.Conv2d(ndf * 8, 1, kernel_size=4, stride=2, padding=1),\n )\n\n\ndef prob_2_entropy(prob):\n \"\"\" convert probabilistic prediction maps to weighted self-information maps\n \"\"\"\n n, c, h, w = prob.size()\n return -torch.mul(prob, torch.log2(prob + 1e-30)) / np.log2(c)\n\n\ndef bce_loss(y_pred, y_label):\n y_truth_tensor = torch.FloatTensor(y_pred.size())\n y_truth_tensor.fill_(y_label)\n y_truth_tensor = y_truth_tensor.to(y_pred.get_device())\n return F.binary_cross_entropy_with_logits(y_pred, y_truth_tensor)\n\n\nclass DomainAdversarialEntropyLoss(nn.Module):\n r\"\"\"The `Domain Adversarial Entropy Loss `_\n\n Minimizing entropy with adversarial learning through training a domain discriminator.\n\n Args:\n domain_discriminator (torch.nn.Module): A domain discriminator object, which predicts\n the domains of predictions. Its input shape is :math:`(minibatch, C, H, W)` and output shape is :math:`(minibatch, 1, H, W)`\n\n Inputs:\n - logits (tensor): logits output of segmentation model\n - domain_label (str, optional): whether the data comes from source or target.\n Choices: ['source', 'target']. Default: 'source'\n\n Shape:\n - logits: :math:`(minibatch, C, H, W)` where :math:`C` means the number of classes\n - Outputs: scalar.\n\n Examples::\n\n >>> B, C, H, W = 2, 19, 512, 512\n >>> discriminator = Discriminator(num_classes=C)\n >>> dann = DomainAdversarialEntropyLoss(discriminator)\n >>> # logits output on source domain and target domain\n >>> y_s, y_t = torch.randn(B, C, H, W), torch.randn(B, C, H, W)\n >>> loss = 0.5 * (dann(y_s, \"source\") + dann(y_t, \"target\"))\n \"\"\"\n def __init__(self, discriminator: nn.Module):\n super(DomainAdversarialEntropyLoss, self).__init__()\n self.discriminator = discriminator\n\n def forward(self, logits, domain_label='source'):\n \"\"\"\n \"\"\"\n assert domain_label in ['source', 'target']\n probability = F.softmax(logits, dim=1)\n entropy = prob_2_entropy(probability)\n domain_prediciton = self.discriminator(entropy)\n if domain_label == 'source':\n return bce_loss(domain_prediciton, 1)\n else:\n return bce_loss(domain_prediciton, 0)\n\n def train(self, mode=True):\n r\"\"\"Sets the discriminator in training mode. In the training mode,\n all the parameters in discriminator will be set requires_grad=True.\n\n Args:\n mode (bool): whether to set training mode (``True``) or evaluation mode (``False``). Default: ``True``.\n \"\"\"\n self.discriminator.train(mode)\n for param in self.discriminator.parameters():\n param.requires_grad = mode\n return self\n\n def eval(self):\n r\"\"\"Sets the module in evaluation mode. In the training mode,\n all the parameters in discriminator will be set requires_grad=False.\n\n This is equivalent with :meth:`self.train(False) `.\n \"\"\"\n return self.train(False)\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/dalib/adaptation/afn.py",
"content": "\"\"\"\nModified from https://github.com/jihanyang/AFN\n@author: Baixu Chen\n@contact: cbx_99_hasta@outlook.com\n\"\"\"\nfrom typing import Optional, List, Dict\nimport torch\nimport torch.nn as nn\nimport math\n\nfrom common.modules.classifier import Classifier as ClassfierBase\n\n\nclass AdaptiveFeatureNorm(nn.Module):\n r\"\"\"\n The `Stepwise Adaptive Feature Norm loss (ICCV 2019) `_\n\n Instead of using restrictive scalar R to match the corresponding feature norm, Stepwise Adaptive Feature Norm\n is used in order to learn task-specific features with large norms in a progressive manner.\n We denote parameters of backbone :math:`G` as :math:`\\theta_g`, parameters of bottleneck :math:`F_f` as :math:`\\theta_f`\n , parameters of classifier head :math:`F_y` as :math:`\\theta_y`, and features extracted from sample :math:`x_i` as\n :math:`h(x_i;\\theta)`. Full loss is calculated as follows\n\n .. math::\n L(\\theta_g,\\theta_f,\\theta_y)=\\frac{1}{n_s}\\sum_{(x_i,y_i)\\in D_s}L_y(x_i,y_i)+\\frac{\\lambda}{n_s+n_t}\n \\sum_{x_i\\in D_s\\cup D_t}L_d(h(x_i;\\theta_0)+\\Delta_r,h(x_i;\\theta))\\\\\n\n where :math:`L_y` denotes classification loss, :math:`L_d` denotes norm loss, :math:`\\theta_0` and :math:`\\theta`\n represent the updated and updating model parameters in the last and current iterations respectively.\n\n Args:\n delta (float): positive residual scalar to control the feature norm enlargement.\n\n Inputs:\n - f (tensor): feature representations on source or target domain.\n\n Shape:\n - f: :math:`(N, F)` where F means the dimension of input features.\n - Outputs: scalar.\n\n Examples::\n\n >>> adaptive_feature_norm = AdaptiveFeatureNorm(delta=1)\n >>> f_s = torch.randn(32, 1000)\n >>> f_t = torch.randn(32, 1000)\n >>> norm_loss = adaptive_feature_norm(f_s) + adaptive_feature_norm(f_t)\n \"\"\"\n\n def __init__(self, delta):\n super(AdaptiveFeatureNorm, self).__init__()\n self.delta = delta\n\n def forward(self, f: torch.Tensor) -> torch.Tensor:\n radius = f.norm(p=2, dim=1).detach()\n assert radius.requires_grad == False\n radius = radius + self.delta\n loss = ((f.norm(p=2, dim=1) - radius) ** 2).mean()\n return loss\n\n\nclass Block(nn.Module):\n r\"\"\"\n Basic building block for Image Classifier with structure: FC-BN-ReLU-Dropout.\n We use :math:`L_2` preserved dropout layers.\n Given mask probability :math:`p`, input :math:`x_k`, generated mask :math:`a_k`,\n vanilla dropout layers calculate\n\n .. math::\n \\hat{x}_k = a_k\\frac{1}{1-p}x_k\\\\\n\n While in :math:`L_2` preserved dropout layers\n\n .. math::\n \\hat{x}_k = a_k\\frac{1}{\\sqrt{1-p}}x_k\\\\\n\n Args:\n in_features (int): Dimension of input features\n bottleneck_dim (int, optional): Feature dimension of the bottleneck layer. Default: 1000\n dropout_p (float, optional): dropout probability. Default: 0.5\n \"\"\"\n\n def __init__(self, in_features: int, bottleneck_dim: Optional[int] = 1000, dropout_p: Optional[float] = 0.5):\n super(Block, self).__init__()\n self.fc = nn.Linear(in_features, bottleneck_dim)\n self.bn = nn.BatchNorm1d(bottleneck_dim, affine=True)\n self.relu = nn.ReLU(inplace=True)\n self.dropout = nn.Dropout(dropout_p)\n self.dropout_p = dropout_p\n\n def forward(self, x: torch.Tensor) -> torch.Tensor:\n f = self.fc(x)\n f = self.bn(f)\n f = self.relu(f)\n f = self.dropout(f)\n if self.training:\n f.mul_(math.sqrt(1 - self.dropout_p))\n return f\n\n\nclass ImageClassifier(ClassfierBase):\n r\"\"\"\n ImageClassifier for AFN.\n\n Args:\n backbone (torch.nn.Module): Any backbone to extract 2-d features from data\n num_classes (int): Number of classes\n num_blocks (int, optional): Number of basic blocks. Default: 1\n bottleneck_dim (int, optional): Feature dimension of the bottleneck layer. Default: 1000\n dropout_p (float, optional): dropout probability. Default: 0.5\n \"\"\"\n\n def __init__(self, backbone: nn.Module, num_classes: int, num_blocks: Optional[int] = 1,\n bottleneck_dim: Optional[int] = 1000, dropout_p: Optional[float] = 0.5, **kwargs):\n assert num_blocks >= 1\n layers = [nn.Sequential(\n Block(backbone.out_features, bottleneck_dim, dropout_p)\n )]\n for _ in range(num_blocks - 1):\n layers.append(Block(bottleneck_dim, bottleneck_dim, dropout_p))\n bottleneck = nn.Sequential(*layers)\n super(ImageClassifier, self).__init__(backbone, num_classes, bottleneck, bottleneck_dim, **kwargs)\n # init parameters for bottleneck and head\n for m in self.bottleneck.modules():\n if isinstance(m, nn.BatchNorm1d):\n m.weight.data.normal_(1.0, 0.01)\n m.bias.data.fill_(0)\n if isinstance(m, nn.Linear):\n m.weight.data.normal_(0.0, 0.01)\n m.bias.data.normal_(0.0, 0.01)\n for m in self.head.modules():\n if isinstance(m, nn.Linear):\n m.weight.data.normal_(0.0, 0.01)\n m.bias.data.normal_(0.0, 0.01)\n\n def get_parameters(self, base_lr=1.0) -> List[Dict]:\n params = [\n {\"params\": self.backbone.parameters()},\n {\"params\": self.bottleneck.parameters(), \"momentum\": 0.9},\n {\"params\": self.head.parameters(), \"momentum\": 0.9},\n ]\n return params\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/dalib/adaptation/bsp.py",
"content": "\"\"\"\n@author: Baixu Chen\n@contact: cbx_99_hasta@outlook.com\n\"\"\"\nfrom typing import Optional\nimport torch\nimport torch.nn as nn\nfrom common.modules.classifier import Classifier as ClassifierBase\n\n\nclass BatchSpectralPenalizationLoss(nn.Module):\n r\"\"\"Batch spectral penalization loss from `Transferability vs. Discriminability: Batch\n Spectral Penalization for Adversarial Domain Adaptation (ICML 2019)\n `_.\n\n Given source features :math:`f_s` and target features :math:`f_t` in current mini batch, singular value\n decomposition is first performed\n\n .. math::\n f_s = U_s\\Sigma_sV_s^T\n\n .. math::\n f_t = U_t\\Sigma_tV_t^T\n\n Then batch spectral penalization loss is calculated as\n\n .. math::\n loss=\\sum_{i=1}^k(\\sigma_{s,i}^2+\\sigma_{t,i}^2)\n\n where :math:`\\sigma_{s,i},\\sigma_{t,i}` refer to the :math:`i-th` largest singular value of source features\n and target features respectively. We empirically set :math:`k=1`.\n\n Inputs:\n - f_s (tensor): feature representations on source domain, :math:`f^s`\n - f_t (tensor): feature representations on target domain, :math:`f^t`\n\n Shape:\n - f_s, f_t: :math:`(N, F)` where F means the dimension of input features.\n - Outputs: scalar.\n\n \"\"\"\n\n def __init__(self):\n super(BatchSpectralPenalizationLoss, self).__init__()\n\n def forward(self, f_s, f_t):\n _, s_s, _ = torch.svd(f_s)\n _, s_t, _ = torch.svd(f_t)\n loss = torch.pow(s_s[0], 2) + torch.pow(s_t[0], 2)\n return loss\n\n\nclass ImageClassifier(ClassifierBase):\n def __init__(self, backbone: nn.Module, num_classes: int, bottleneck_dim: Optional[int] = 256, **kwargs):\n bottleneck = nn.Sequential(\n # nn.AdaptiveAvgPool2d(output_size=(1, 1)),\n # nn.Flatten(),\n nn.Linear(backbone.out_features, bottleneck_dim),\n nn.BatchNorm1d(bottleneck_dim),\n nn.ReLU(),\n )\n super(ImageClassifier, self).__init__(backbone, num_classes, bottleneck, bottleneck_dim, **kwargs)\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/dalib/adaptation/cdan.py",
"content": "\"\"\"\n@author: Junguang Jiang\n@contact: JiangJunguang1123@outlook.com\n\"\"\"\nfrom typing import Optional\nimport numpy as np\nimport torch\nimport torch.nn as nn\nimport torch.nn.functional as F\n\nfrom common.modules.classifier import Classifier as ClassifierBase\nfrom common.utils.metric import binary_accuracy\nfrom ..modules.grl import WarmStartGradientReverseLayer\nfrom ..modules.entropy import entropy\n\n\n__all__ = ['ConditionalDomainAdversarialLoss', 'ImageClassifier']\n\n\nclass ConditionalDomainAdversarialLoss(nn.Module):\n r\"\"\"The Conditional Domain Adversarial Loss used in `Conditional Adversarial Domain Adaptation (NIPS 2018) `_\n\n Conditional Domain adversarial loss measures the domain discrepancy through training a domain discriminator in a\n conditional manner. Given domain discriminator :math:`D`, feature representation :math:`f` and\n classifier predictions :math:`g`, the definition of CDAN loss is\n\n .. math::\n loss(\\mathcal{D}_s, \\mathcal{D}_t) &= \\mathbb{E}_{x_i^s \\sim \\mathcal{D}_s} \\text{log}[D(T(f_i^s, g_i^s))] \\\\\n &+ \\mathbb{E}_{x_j^t \\sim \\mathcal{D}_t} \\text{log}[1-D(T(f_j^t, g_j^t))],\\\\\n\n where :math:`T` is a :class:`MultiLinearMap` or :class:`RandomizedMultiLinearMap` which convert two tensors to a single tensor.\n\n Args:\n domain_discriminator (torch.nn.Module): A domain discriminator object, which predicts the domains of\n features. Its input shape is (N, F) and output shape is (N, 1)\n entropy_conditioning (bool, optional): If True, use entropy-aware weight to reweight each training example.\n Default: False\n randomized (bool, optional): If True, use `randomized multi linear map`. Else, use `multi linear map`.\n Default: False\n num_classes (int, optional): Number of classes. Default: -1\n features_dim (int, optional): Dimension of input features. Default: -1\n randomized_dim (int, optional): Dimension of features after randomized. Default: 1024\n reduction (str, optional): Specifies the reduction to apply to the output:\n ``'none'`` | ``'mean'`` | ``'sum'``. ``'none'``: no reduction will be applied,\n ``'mean'``: the sum of the output will be divided by the number of\n elements in the output, ``'sum'``: the output will be summed. Default: ``'mean'``\n\n .. note::\n You need to provide `num_classes`, `features_dim` and `randomized_dim` **only when** `randomized`\n is set True.\n\n Inputs:\n - g_s (tensor): unnormalized classifier predictions on source domain, :math:`g^s`\n - f_s (tensor): feature representations on source domain, :math:`f^s`\n - g_t (tensor): unnormalized classifier predictions on target domain, :math:`g^t`\n - f_t (tensor): feature representations on target domain, :math:`f^t`\n\n Shape:\n - g_s, g_t: :math:`(minibatch, C)` where C means the number of classes.\n - f_s, f_t: :math:`(minibatch, F)` where F means the dimension of input features.\n - Output: scalar by default. If :attr:`reduction` is ``'none'``, then :math:`(minibatch, )`.\n\n Examples::\n\n >>> from dalib.modules.domain_discriminator import DomainDiscriminator\n >>> from dalib.adaptation.cdan import ConditionalDomainAdversarialLoss\n >>> import torch\n >>> num_classes = 2\n >>> feature_dim = 1024\n >>> batch_size = 10\n >>> discriminator = DomainDiscriminator(in_feature=feature_dim * num_classes, hidden_size=1024)\n >>> loss = ConditionalDomainAdversarialLoss(discriminator, reduction='mean')\n >>> # features from source domain and target domain\n >>> f_s, f_t = torch.randn(batch_size, feature_dim), torch.randn(batch_size, feature_dim)\n >>> # logits output from source domain adn target domain\n >>> g_s, g_t = torch.randn(batch_size, num_classes), torch.randn(batch_size, num_classes)\n >>> output = loss(g_s, f_s, g_t, f_t)\n \"\"\"\n\n def __init__(self, domain_discriminator: nn.Module, entropy_conditioning: Optional[bool] = False,\n randomized: Optional[bool] = False, num_classes: Optional[int] = -1,\n features_dim: Optional[int] = -1, randomized_dim: Optional[int] = 1024,\n reduction: Optional[str] = 'mean'):\n super(ConditionalDomainAdversarialLoss, self).__init__()\n self.domain_discriminator = domain_discriminator\n self.grl = WarmStartGradientReverseLayer(alpha=1., lo=0., hi=1., max_iters=1000, auto_step=True)\n self.entropy_conditioning = entropy_conditioning\n\n if randomized:\n assert num_classes > 0 and features_dim > 0 and randomized_dim > 0\n self.map = RandomizedMultiLinearMap(features_dim, num_classes, randomized_dim)\n else:\n self.map = MultiLinearMap()\n\n self.bce = lambda input, target, weight: F.binary_cross_entropy(input, target, weight,\n reduction=reduction) if self.entropy_conditioning \\\n else F.binary_cross_entropy(input, target, reduction=reduction)\n self.domain_discriminator_accuracy = None\n\n def forward(self, g_s: torch.Tensor, f_s: torch.Tensor, g_t: torch.Tensor, f_t: torch.Tensor) -> torch.Tensor:\n f = torch.cat((f_s, f_t), dim=0)\n g = torch.cat((g_s, g_t), dim=0)\n g = F.softmax(g, dim=1).detach()\n h = self.grl(self.map(f, g))\n d = self.domain_discriminator(h)\n d_label = torch.cat((\n torch.ones((g_s.size(0), 1)).to(g_s.device),\n torch.zeros((g_t.size(0), 1)).to(g_t.device),\n ))\n weight = 1.0 + torch.exp(-entropy(g))\n batch_size = f.size(0)\n weight = weight / torch.sum(weight) * batch_size\n self.domain_discriminator_accuracy = binary_accuracy(d, d_label)\n return self.bce(d, d_label, weight.view_as(d))\n\n\nclass RandomizedMultiLinearMap(nn.Module):\n \"\"\"Random multi linear map\n\n Given two inputs :math:`f` and :math:`g`, the definition is\n\n .. math::\n T_{\\odot}(f,g) = \\dfrac{1}{\\sqrt{d}} (R_f f) \\odot (R_g g),\n\n where :math:`\\odot` is element-wise product, :math:`R_f` and :math:`R_g` are random matrices\n sampled only once and fixed in training.\n\n Args:\n features_dim (int): dimension of input :math:`f`\n num_classes (int): dimension of input :math:`g`\n output_dim (int, optional): dimension of output tensor. Default: 1024\n\n Shape:\n - f: (minibatch, features_dim)\n - g: (minibatch, num_classes)\n - Outputs: (minibatch, output_dim)\n \"\"\"\n\n def __init__(self, features_dim: int, num_classes: int, output_dim: Optional[int] = 1024):\n super(RandomizedMultiLinearMap, self).__init__()\n self.Rf = torch.randn(features_dim, output_dim)\n self.Rg = torch.randn(num_classes, output_dim)\n self.output_dim = output_dim\n\n def forward(self, f: torch.Tensor, g: torch.Tensor) -> torch.Tensor:\n f = torch.mm(f, self.Rf.to(f.device))\n g = torch.mm(g, self.Rg.to(g.device))\n output = torch.mul(f, g) / np.sqrt(float(self.output_dim))\n return output\n\n\nclass MultiLinearMap(nn.Module):\n \"\"\"Multi linear map\n\n Shape:\n - f: (minibatch, F)\n - g: (minibatch, C)\n - Outputs: (minibatch, F * C)\n \"\"\"\n\n def __init__(self):\n super(MultiLinearMap, self).__init__()\n\n def forward(self, f: torch.Tensor, g: torch.Tensor) -> torch.Tensor:\n batch_size = f.size(0)\n output = torch.bmm(g.unsqueeze(2), f.unsqueeze(1))\n return output.view(batch_size, -1)\n\n\nclass ImageClassifier(ClassifierBase):\n def __init__(self, backbone: nn.Module, num_classes: int, bottleneck_dim: Optional[int] = 256, **kwargs):\n bottleneck = nn.Sequential(\n # nn.AdaptiveAvgPool2d(output_size=(1, 1)),\n # nn.Flatten(),\n nn.Linear(backbone.out_features, bottleneck_dim),\n nn.BatchNorm1d(bottleneck_dim),\n nn.ReLU()\n )\n super(ImageClassifier, self).__init__(backbone, num_classes, bottleneck, bottleneck_dim, **kwargs)\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/dalib/adaptation/dan.py",
"content": "\"\"\"\n@author: Junguang Jiang\n@contact: JiangJunguang1123@outlook.com\n\"\"\"\nfrom typing import Optional, Sequence\nimport torch\nimport torch.nn as nn\n\nfrom common.modules.classifier import Classifier as ClassifierBase\n\n\n__all__ = ['MultipleKernelMaximumMeanDiscrepancy', 'ImageClassifier']\n\n\nclass MultipleKernelMaximumMeanDiscrepancy(nn.Module):\n r\"\"\"The Multiple Kernel Maximum Mean Discrepancy (MK-MMD) used in\n `Learning Transferable Features with Deep Adaptation Networks (ICML 2015) `_\n\n Given source domain :math:`\\mathcal{D}_s` of :math:`n_s` labeled points and target domain :math:`\\mathcal{D}_t`\n of :math:`n_t` unlabeled points drawn i.i.d. from P and Q respectively, the deep networks will generate\n activations as :math:`\\{z_i^s\\}_{i=1}^{n_s}` and :math:`\\{z_i^t\\}_{i=1}^{n_t}`.\n The MK-MMD :math:`D_k (P, Q)` between probability distributions P and Q is defined as\n\n .. math::\n D_k(P, Q) \\triangleq \\| E_p [\\phi(z^s)] - E_q [\\phi(z^t)] \\|^2_{\\mathcal{H}_k},\n\n :math:`k` is a kernel function in the function space\n\n .. math::\n \\mathcal{K} \\triangleq \\{ k=\\sum_{u=1}^{m}\\beta_{u} k_{u} \\}\n\n where :math:`k_{u}` is a single kernel.\n\n Using kernel trick, MK-MMD can be computed as\n\n .. math::\n \\hat{D}_k(P, Q) &=\n \\dfrac{1}{n_s^2} \\sum_{i=1}^{n_s}\\sum_{j=1}^{n_s} k(z_i^{s}, z_j^{s})\\\\\n &+ \\dfrac{1}{n_t^2} \\sum_{i=1}^{n_t}\\sum_{j=1}^{n_t} k(z_i^{t}, z_j^{t})\\\\\n &- \\dfrac{2}{n_s n_t} \\sum_{i=1}^{n_s}\\sum_{j=1}^{n_t} k(z_i^{s}, z_j^{t}).\\\\\n\n Args:\n kernels (tuple(torch.nn.Module)): kernel functions.\n linear (bool): whether use the linear version of DAN. Default: False\n\n Inputs:\n - z_s (tensor): activations from the source domain, :math:`z^s`\n - z_t (tensor): activations from the target domain, :math:`z^t`\n\n Shape:\n - Inputs: :math:`(minibatch, *)` where * means any dimension\n - Outputs: scalar\n\n .. note::\n Activations :math:`z^{s}` and :math:`z^{t}` must have the same shape.\n\n .. note::\n The kernel values will add up when there are multiple kernels.\n\n Examples::\n\n >>> from dalib.modules.kernels import GaussianKernel\n >>> feature_dim = 1024\n >>> batch_size = 10\n >>> kernels = (GaussianKernel(alpha=0.5), GaussianKernel(alpha=1.), GaussianKernel(alpha=2.))\n >>> loss = MultipleKernelMaximumMeanDiscrepancy(kernels)\n >>> # features from source domain and target domain\n >>> z_s, z_t = torch.randn(batch_size, feature_dim), torch.randn(batch_size, feature_dim)\n >>> output = loss(z_s, z_t)\n \"\"\"\n\n def __init__(self, kernels: Sequence[nn.Module], linear: Optional[bool] = False):\n super(MultipleKernelMaximumMeanDiscrepancy, self).__init__()\n self.kernels = kernels\n self.index_matrix = None\n self.linear = linear\n\n def forward(self, z_s: torch.Tensor, z_t: torch.Tensor) -> torch.Tensor:\n features = torch.cat([z_s, z_t], dim=0)\n batch_size = int(z_s.size(0))\n self.index_matrix = _update_index_matrix(batch_size, self.index_matrix, self.linear).to(z_s.device)\n\n\n kernel_matrix = sum([kernel(features) for kernel in self.kernels]) # Add up the matrix of each kernel\n # Add 2 / (n-1) to make up for the value on the diagonal\n # to ensure loss is positive in the non-linear version\n loss = (kernel_matrix * self.index_matrix).sum() + 2. / float(batch_size - 1)\n\n return loss\n\n\ndef _update_index_matrix(batch_size: int, index_matrix: Optional[torch.Tensor] = None,\n linear: Optional[bool] = True) -> torch.Tensor:\n r\"\"\"\n Update the `index_matrix` which convert `kernel_matrix` to loss.\n If `index_matrix` is a tensor with shape (2 x batch_size, 2 x batch_size), then return `index_matrix`.\n Else return a new tensor with shape (2 x batch_size, 2 x batch_size).\n \"\"\"\n if index_matrix is None or index_matrix.size(0) != batch_size * 2:\n index_matrix = torch.zeros(2 * batch_size, 2 * batch_size)\n if linear:\n for i in range(batch_size):\n s1, s2 = i, (i + 1) % batch_size\n t1, t2 = s1 + batch_size, s2 + batch_size\n index_matrix[s1, s2] = 1. / float(batch_size)\n index_matrix[t1, t2] = 1. / float(batch_size)\n index_matrix[s1, t2] = -1. / float(batch_size)\n index_matrix[s2, t1] = -1. / float(batch_size)\n else:\n for i in range(batch_size):\n for j in range(batch_size):\n if i != j:\n index_matrix[i][j] = 1. / float(batch_size * (batch_size - 1))\n index_matrix[i + batch_size][j + batch_size] = 1. / float(batch_size * (batch_size - 1))\n for i in range(batch_size):\n for j in range(batch_size):\n index_matrix[i][j + batch_size] = -1. / float(batch_size * batch_size)\n index_matrix[i + batch_size][j] = -1. / float(batch_size * batch_size)\n return index_matrix\n\n\nclass ImageClassifier(ClassifierBase):\n def __init__(self, backbone: nn.Module, num_classes: int, bottleneck_dim: Optional[int] = 256, **kwargs):\n bottleneck = nn.Sequential(\n # nn.AdaptiveAvgPool2d(output_size=(1, 1)),\n # nn.Flatten(),\n nn.Linear(backbone.out_features, bottleneck_dim),\n nn.ReLU(),\n nn.Dropout(0.5)\n )\n super(ImageClassifier, self).__init__(backbone, num_classes, bottleneck, bottleneck_dim, **kwargs)"
},
{
"path": "domain_generalization/Transfer-Learning-Library/dalib/adaptation/dann.py",
"content": "\"\"\"\n@author: Junguang Jiang\n@contact: JiangJunguang1123@outlook.com\n\"\"\"\nfrom typing import Optional\nimport torch\nimport torch.nn as nn\nimport torch.nn.functional as F\n\nfrom ..modules.grl import WarmStartGradientReverseLayer\nfrom common.modules.classifier import Classifier as ClassifierBase\nfrom common.utils.metric import binary_accuracy\n\n__all__ = ['DomainAdversarialLoss']\n\n\nclass DomainAdversarialLoss(nn.Module):\n r\"\"\"\n The Domain Adversarial Loss proposed in\n `Domain-Adversarial Training of Neural Networks (ICML 2015) `_\n\n Domain adversarial loss measures the domain discrepancy through training a domain discriminator.\n Given domain discriminator :math:`D`, feature representation :math:`f`, the definition of DANN loss is\n\n .. math::\n loss(\\mathcal{D}_s, \\mathcal{D}_t) = \\mathbb{E}_{x_i^s \\sim \\mathcal{D}_s} \\text{log}[D(f_i^s)]\n + \\mathbb{E}_{x_j^t \\sim \\mathcal{D}_t} \\text{log}[1-D(f_j^t)].\n\n Args:\n domain_discriminator (torch.nn.Module): A domain discriminator object, which predicts the domains of features. Its input shape is (N, F) and output shape is (N, 1)\n reduction (str, optional): Specifies the reduction to apply to the output:\n ``'none'`` | ``'mean'`` | ``'sum'``. ``'none'``: no reduction will be applied,\n ``'mean'``: the sum of the output will be divided by the number of\n elements in the output, ``'sum'``: the output will be summed. Default: ``'mean'``\n grl (WarmStartGradientReverseLayer, optional): Default: None.\n\n Inputs:\n - f_s (tensor): feature representations on source domain, :math:`f^s`\n - f_t (tensor): feature representations on target domain, :math:`f^t`\n - w_s (tensor, optional): a rescaling weight given to each instance from source domain.\n - w_t (tensor, optional): a rescaling weight given to each instance from target domain.\n\n Shape:\n - f_s, f_t: :math:`(N, F)` where F means the dimension of input features.\n - Outputs: scalar by default. If :attr:`reduction` is ``'none'``, then :math:`(N, )`.\n\n Examples::\n\n >>> from dalib.modules.domain_discriminator import DomainDiscriminator\n >>> discriminator = DomainDiscriminator(in_feature=1024, hidden_size=1024)\n >>> loss = DomainAdversarialLoss(discriminator, reduction='mean')\n >>> # features from source domain and target domain\n >>> f_s, f_t = torch.randn(20, 1024), torch.randn(20, 1024)\n >>> # If you want to assign different weights to each instance, you should pass in w_s and w_t\n >>> w_s, w_t = torch.randn(20), torch.randn(20)\n >>> output = loss(f_s, f_t, w_s, w_t)\n \"\"\"\n\n def __init__(self, domain_discriminator: nn.Module, reduction: Optional[str] = 'mean',\n grl: Optional = None):\n super(DomainAdversarialLoss, self).__init__()\n self.grl = WarmStartGradientReverseLayer(alpha=1., lo=0., hi=1., max_iters=1000, auto_step=True) if grl is None else grl\n self.domain_discriminator = domain_discriminator\n self.bce = lambda input, target, weight: \\\n F.binary_cross_entropy(input, target, weight=weight, reduction=reduction)\n self.domain_discriminator_accuracy = None\n\n def forward(self, f_s: torch.Tensor, f_t: torch.Tensor,\n w_s: Optional[torch.Tensor] = None, w_t: Optional[torch.Tensor] = None) -> torch.Tensor:\n f = self.grl(torch.cat((f_s, f_t), dim=0))\n d = self.domain_discriminator(f)\n d_s, d_t = d.chunk(2, dim=0)\n d_label_s = torch.ones((f_s.size(0), 1)).to(f_s.device)\n d_label_t = torch.zeros((f_t.size(0), 1)).to(f_t.device)\n self.domain_discriminator_accuracy = 0.5 * (binary_accuracy(d_s, d_label_s) + binary_accuracy(d_t, d_label_t))\n\n if w_s is None:\n w_s = torch.ones_like(d_label_s)\n if w_t is None:\n w_t = torch.ones_like(d_label_t)\n return 0.5 * (self.bce(d_s, d_label_s, w_s.view_as(d_s)) + self.bce(d_t, d_label_t, w_t.view_as(d_t)))\n\n\nclass ImageClassifier(ClassifierBase):\n def __init__(self, backbone: nn.Module, num_classes: int, bottleneck_dim: Optional[int] = 256, **kwargs):\n bottleneck = nn.Sequential(\n # nn.AdaptiveAvgPool2d(output_size=(1, 1)),\n # nn.Flatten(),\n nn.Linear(backbone.out_features, bottleneck_dim),\n nn.BatchNorm1d(bottleneck_dim),\n nn.ReLU()\n )\n super(ImageClassifier, self).__init__(backbone, num_classes, bottleneck, bottleneck_dim, **kwargs)\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/dalib/adaptation/fda.py",
"content": "\"\"\"\n@author: Junguang Jiang\n@contact: JiangJunguang1123@outlook.com\n\"\"\"\nimport torch.nn.functional as F\nimport math\n\n\ndef robust_entropy(y, ita=1.5, num_classes=19, reduction='mean'):\n \"\"\" Robust entropy proposed in `FDA: Fourier Domain Adaptation for Semantic Segmentation (CVPR 2020) `_\n\n Args:\n y (tensor): logits output of segmentation model in shape of :math:`(N, C, H, W)`\n ita (float, optional): parameters for robust entropy. Default: 1.5\n num_classes (int, optional): number of classes. Default: 19\n reduction (string, optional): Specifies the reduction to apply to the output:\n ``'none'`` | ``'mean'``. ``'none'``: no reduction will be applied,\n ``'mean'``: the sum of the output will be divided by the number of\n elements in the output. Default: ``'mean'``\n\n Returns:\n Scalar by default. If :attr:`reduction` is ``'none'``, then :math:`(N, )`.\n\n \"\"\"\n P = F.softmax(y, dim=1)\n logP = F.log_softmax(y, dim=1)\n PlogP = P * logP\n ent = -1.0 * PlogP.sum(dim=1)\n ent = ent / math.log(num_classes)\n\n # compute robust entropy\n ent = ent ** 2.0 + 1e-8\n ent = ent ** ita\n\n if reduction == 'mean':\n return ent.mean()\n else:\n return ent\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/dalib/adaptation/idm/__init__.py",
"content": ""
},
{
"path": "domain_generalization/Transfer-Learning-Library/dalib/adaptation/idm/loss.py",
"content": "\"\"\"\nModified from https://github.com/SikaStar/IDM\n@author: Baixu Chen\n@contact: cbx_99_hasta@outlook.com\n\"\"\"\nimport torch\nimport torch.nn as nn\n\n\nclass DivLoss(nn.Module):\n \"\"\"Diversity loss, which is defined as negative of standard deviation.\n \"\"\"\n def __init__(self, ):\n super(DivLoss, self).__init__()\n\n def forward(self, lam):\n mu = lam.mean(0)\n std = ((lam - mu) ** 2).mean(0, keepdim=True).clamp(min=1e-12).sqrt()\n loss_std = -std.sum()\n return loss_std\n\n\nclass BridgeFeatLoss(nn.Module):\n \"\"\"Bridge loss on feature space.\n \"\"\"\n def __init__(self):\n super(BridgeFeatLoss, self).__init__()\n\n def forward(self, f_s, f_t, f_mixed, lam):\n dist_mixed2s = ((f_mixed - f_s) ** 2).sum(1, keepdim=True)\n dist_mixed2t = ((f_mixed - f_t) ** 2).sum(1, keepdim=True)\n\n dist_mixed2s = dist_mixed2s.clamp(min=1e-12).sqrt()\n dist_mixed2t = dist_mixed2t.clamp(min=1e-12).sqrt()\n\n dist_mixed = torch.cat((dist_mixed2s, dist_mixed2t), 1)\n lam_dist_mixed = (lam * dist_mixed).sum(1, keepdim=True)\n loss = lam_dist_mixed.mean()\n\n return loss\n\n\nclass BridgeProbLoss(nn.Module):\n \"\"\"Bridge loss on prediction space.\n \"\"\"\n def __init__(self, num_classes, epsilon=0.1):\n super(BridgeProbLoss, self).__init__()\n self.num_classes = num_classes\n self.epsilon = epsilon\n self.log_softmax = nn.LogSoftmax(dim=1).cuda()\n\n def forward(self, y, labels, lam, device_num=4):\n y = y.view(device_num, -1, y.size(-1))\n y_s, y_t, y_mixed = y.split(y.size(1) // 3, dim=1)\n y_ori = torch.cat((y_s, y_t), 1).view(-1, y.size(-1))\n y_mixed = y_mixed.contiguous().view(-1, y.size(-1))\n log_prob_ori = self.log_softmax(y_ori)\n log_prob_mixed = self.log_softmax(y_mixed)\n\n labels = torch.zeros_like(log_prob_ori).scatter_(1, labels.unsqueeze(1), 1)\n labels = labels.view(device_num, -1, labels.size(-1))\n labels_s, labels_t = labels.split(labels.size(1) // 2, dim=1)\n labels_s = labels_s.contiguous().view(-1, labels.size(-1))\n labels_t = labels_t.contiguous().view(-1, labels.size(-1))\n\n labels = labels.view(-1, labels.size(-1))\n soft_labels = (1 - self.epsilon) * labels + self.epsilon / self.num_classes\n\n lam = lam.view(-1, 1)\n soft_labels_mixed = lam * labels_s + (1. - lam) * labels_t\n soft_labels_mixed = (1 - self.epsilon) * soft_labels_mixed + self.epsilon / self.num_classes\n loss_ori = (- soft_labels * log_prob_ori).sum(dim=1).mean()\n loss_bridge_prob = (- soft_labels_mixed * log_prob_mixed).sum(dim=1).mean()\n\n return loss_ori, loss_bridge_prob\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/dalib/adaptation/idm/models/__init__.py",
"content": "from .resnet import *\n\n__all__ = ['resnet']\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/dalib/adaptation/idm/models/dsbn.py",
"content": "\"\"\"\nModified from https://github.com/SikaStar/IDM\n@author: Baixu Chen\n@contact: cbx_99_hasta@outlook.com\n\"\"\"\nimport torch\nimport torch.nn as nn\n\n\nclass DSBN2d(nn.Module):\n \"\"\"Domain-Specific BatchNorm2d layer in `Domain-Specific Batch Normalization for Unsupervised Domain Adaptation (CVPR 2019)\n `_.\n \"\"\"\n def __init__(self, planes):\n super(DSBN2d, self).__init__()\n self.num_features = planes\n self.BN_S = nn.BatchNorm2d(planes)\n self.BN_T = nn.BatchNorm2d(planes)\n\n def forward(self, x):\n if not self.training:\n return self.BN_T(x)\n\n bs = x.size(0)\n assert (bs % 2 == 0)\n split = torch.split(x, int(bs / 2), 0)\n out1 = self.BN_S(split[0].contiguous())\n out2 = self.BN_T(split[1].contiguous())\n out = torch.cat((out1, out2), 0)\n return out\n\n\nclass DSBN1d(nn.Module):\n \"\"\"Domain-Specific BatchNorm1d layer in `Domain-Specific Batch Normalization for Unsupervised Domain Adaptation (CVPR 2019)\n `_.\n \"\"\"\n def __init__(self, planes):\n super(DSBN1d, self).__init__()\n self.num_features = planes\n self.BN_S = nn.BatchNorm1d(planes)\n self.BN_T = nn.BatchNorm1d(planes)\n\n def forward(self, x):\n if not self.training:\n return self.BN_T(x)\n\n bs = x.size(0)\n assert (bs % 2 == 0)\n split = torch.split(x, int(bs / 2), 0)\n out1 = self.BN_S(split[0].contiguous())\n out2 = self.BN_T(split[1].contiguous())\n out = torch.cat((out1, out2), 0)\n return out\n\n\nclass DSBN2d_idm(nn.Module):\n \"\"\"Domain-Specific BatchNorm2d layer with `IDM` module\n in `IDM: An Intermediate Domain Module for Domain Adaptive Person Re-ID (ICCV 2021)\n `_.\n \"\"\"\n def __init__(self, planes):\n super(DSBN2d_idm, self).__init__()\n self.num_features = planes\n self.BN_S = nn.BatchNorm2d(planes)\n self.BN_T = nn.BatchNorm2d(planes)\n self.BN_mix = nn.BatchNorm2d(planes)\n\n def forward(self, x):\n if not self.training:\n return self.BN_T(x)\n\n bs = x.size(0)\n assert (bs % 3 == 0)\n split = torch.split(x, int(bs / 3), 0)\n out1 = self.BN_S(split[0].contiguous())\n out2 = self.BN_T(split[1].contiguous())\n out3 = self.BN_mix(split[2].contiguous())\n out = torch.cat((out1, out2, out3), 0)\n return out\n\n\nclass DSBN1d_idm(nn.Module):\n \"\"\"Domain-Specific BatchNorm1d layer with `IDM` module\n in `IDM: An Intermediate Domain Module for Domain Adaptive Person Re-ID (ICCV 2021)\n `_.\n \"\"\"\n def __init__(self, planes):\n super(DSBN1d_idm, self).__init__()\n self.num_features = planes\n self.BN_S = nn.BatchNorm1d(planes)\n self.BN_T = nn.BatchNorm1d(planes)\n self.BN_mix = nn.BatchNorm1d(planes)\n\n def forward(self, x):\n if not self.training:\n return self.BN_T(x)\n\n bs = x.size(0)\n assert (bs % 3 == 0)\n split = torch.split(x, int(bs / 3), 0)\n out1 = self.BN_S(split[0].contiguous())\n out2 = self.BN_T(split[1].contiguous())\n out3 = self.BN_mix(split[2].contiguous())\n out = torch.cat((out1, out2, out3), 0)\n return out\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/dalib/adaptation/idm/models/identifier.py",
"content": "\"\"\"\nModified from https://github.com/SikaStar/IDM\n@author: Baixu Chen\n@contact: cbx_99_hasta@outlook.com\n\"\"\"\nfrom common.vision.models.reid.identifier import ReIdentifier as ReIdentifierBase\n\n\nclass ReIdentifier(ReIdentifierBase):\n r\"\"\"Person reIdentifier in `IDM: An Intermediate Domain Module for Domain Adaptive Person Re-ID (ICCV 2021)\n `_. During training, model predictions, extracted features as well as\n attention lambda will be returned.\n \"\"\"\n def __init__(self, *args, **kwargs):\n super(ReIdentifier, self).__init__(*args, **kwargs)\n\n def forward(self, x, stage=0):\n if self.training:\n f, attention_lam = self.backbone(x, stage)\n else:\n f = self.backbone(x, stage)\n f = self.pool_layer(f)\n bn_f = self.bottleneck(f)\n if not self.training:\n return bn_f\n predictions = self.head(bn_f)\n return predictions, f, attention_lam\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/dalib/adaptation/idm/models/idm.py",
"content": "\"\"\"\nModified from https://github.com/SikaStar/IDM\n@author: Baixu Chen\n@contact: cbx_99_hasta@outlook.com\n\"\"\"\nimport torch\nimport torch.nn as nn\n\n\nclass IDM(nn.Module):\n \"\"\"IDM module in `IDM: An Intermediate Domain Module for Domain Adaptive Person Re-ID (ICCV 2021)\n `_. This module automatically generates mixed features\n with source and target features.\n \"\"\"\n def __init__(self, channel=64):\n super(IDM, self).__init__()\n self.channel = channel\n self.fc1 = nn.Linear(channel * 2, channel)\n self.fc2 = nn.Linear(channel, int(channel / 2))\n self.fc3 = nn.Linear(int(channel / 2), 2)\n self.softmax = nn.Softmax(dim=1)\n self.avgpool = nn.AdaptiveAvgPool2d(1)\n self.maxpool = nn.AdaptiveMaxPool2d(1)\n\n def forward(self, x):\n if not self.training:\n return x\n\n bs = x.size(0)\n assert bs % 2 == 0\n split = torch.split(x, int(bs / 2), 0)\n x_s = split[0].contiguous()\n x_t = split[1].contiguous()\n\n x_emb_s = torch.cat((self.avgpool(x_s.detach()).squeeze(), self.maxpool(x_s.detach()).squeeze()), 1)\n x_emb_t = torch.cat((self.avgpool(x_t.detach()).squeeze(), self.maxpool(x_t.detach()).squeeze()), 1)\n\n x_emb_s, x_emb_t = self.fc1(x_emb_s), self.fc1(x_emb_t)\n x_emb = x_emb_s + x_emb_t\n x_emb = self.fc2(x_emb)\n lam = self.fc3(x_emb)\n lam = self.softmax(lam)\n x_inter = lam[:, 0].reshape(-1, 1, 1, 1) * x_s + lam[:, 1].reshape(-1, 1, 1, 1) * x_t\n output = torch.cat((x_s, x_t, x_inter), 0)\n\n return output, lam\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/dalib/adaptation/idm/models/resnet.py",
"content": "\"\"\"\nModified from https://github.com/SikaStar/IDM\n@author: Baixu Chen\n@contact: cbx_99_hasta@outlook.com\n\"\"\"\nfrom .idm import IDM\nimport torch.nn as nn\nfrom collections import OrderedDict\nfrom common.vision.models.reid.resnet import ReidResNet as ResNetBase\nfrom common.vision.models.reid.resnet import (\n load_state_dict_from_url, model_urls, BasicBlock, Bottleneck\n)\n\n__all__ = ['reid_resnet18', 'reid_resnet34', 'reid_resnet50', 'reid_resnet101']\n\n\nclass ReidResNet(ResNetBase):\n r\"\"\"Modified `ResNet` architecture with `IDM` module from `IDM: An Intermediate Domain Module for Domain Adaptive\n Person Re-ID (ICCV 2021) `_. Although `IDM` Module can be inserted anywhere,\n original paper places `IDM` after layer0-4. Our implementation follows this idea, but you are free to modify this\n function to try other possibilities.\n \"\"\"\n def __init__(self, *args, **kwargs):\n super(ReidResNet, self).__init__(*args, **kwargs)\n self.conv = nn.Sequential(\n OrderedDict([\n ('conv1', self.conv1),\n ('bn1', self.bn1),\n ('relu', self.relu),\n ('maxpool', self.maxpool)\n ])\n )\n\n self.idm_layer1 = IDM(channel=64)\n self.idm_layer2 = IDM(channel=256)\n self.idm_layer3 = IDM(channel=512)\n self.idm_layer4 = IDM(channel=1024)\n self.idm_layer5 = IDM(channel=2048)\n\n def forward(self, x, stage=0):\n assert stage in range(5)\n\n x = self.conv(x)\n if stage == 0 and self.training:\n x, attention_lam = self.idm_layer1(x)\n x = self.layer1(x)\n if stage == 1 and self.training:\n x, attention_lam = self.idm_layer2(x)\n x = self.layer2(x)\n if stage == 2 and self.training:\n x, attention_lam = self.idm_layer3(x)\n x = self.layer3(x)\n if stage == 3 and self.training:\n x, attention_lam = self.idm_layer4(x)\n x = self.layer4(x)\n if stage == 4 and self.training:\n x, attention_lam = self.idm_layer5(x)\n\n if self.training:\n return x, attention_lam\n else:\n return x\n\n\ndef _reid_resnet(arch, block, layers, pretrained, progress, **kwargs):\n model = ReidResNet(block, layers, **kwargs)\n if pretrained:\n model_dict = model.state_dict()\n pretrained_dict = load_state_dict_from_url(model_urls[arch],\n progress=progress)\n # remove keys from pretrained dict that doesn't appear in model dict\n pretrained_dict = {k: v for k, v in pretrained_dict.items() if k in model_dict}\n model.load_state_dict(pretrained_dict, strict=False)\n return model\n\n\ndef reid_resnet18(pretrained=False, progress=True, **kwargs):\n r\"\"\"Constructs a Reid-ResNet-18 model.\n\n Args:\n pretrained (bool): If True, returns a model pre-trained on ImageNet\n progress (bool): If True, displays a progress bar of the download to stderr\n \"\"\"\n return _reid_resnet('resnet18', BasicBlock, [2, 2, 2, 2], pretrained, progress,\n **kwargs)\n\n\ndef reid_resnet34(pretrained=False, progress=True, **kwargs):\n r\"\"\"Constructs a Reid-ResNet-34 model.\n\n Args:\n pretrained (bool): If True, returns a model pre-trained on ImageNet\n progress (bool): If True, displays a progress bar of the download to stderr\n \"\"\"\n return _reid_resnet('resnet34', BasicBlock, [3, 4, 6, 3], pretrained, progress,\n **kwargs)\n\n\ndef reid_resnet50(pretrained=False, progress=True, **kwargs):\n r\"\"\"Constructs a Reid-ResNet-50 model.\n\n Args:\n pretrained (bool): If True, returns a model pre-trained on ImageNet\n progress (bool): If True, displays a progress bar of the download to stderr\n \"\"\"\n return _reid_resnet('resnet50', Bottleneck, [3, 4, 6, 3], pretrained, progress,\n **kwargs)\n\n\ndef reid_resnet101(pretrained=False, progress=True, **kwargs):\n r\"\"\"Constructs a Reid-ResNet-101 model.\n\n Args:\n pretrained (bool): If True, returns a model pre-trained on ImageNet\n progress (bool): If True, displays a progress bar of the download to stderr\n \"\"\"\n return _reid_resnet('resnet101', Bottleneck, [3, 4, 23, 3], pretrained, progress,\n **kwargs)\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/dalib/adaptation/idm/utils.py",
"content": "\"\"\"\nModified from https://github.com/SikaStar/IDM\n@author: Baixu Chen\n@contact: cbx_99_hasta@outlook.com\n\"\"\"\nimport torch.nn as nn\nfrom .models.dsbn import DSBN1d, DSBN2d, DSBN1d_idm, DSBN2d_idm\n\n\ndef filter_layers(stage):\n layer_names = ['conv', 'layer1', 'layer2', 'layer3', 'layer4', 'bottleneck']\n idm_bn_names = []\n for i in range(len(layer_names)):\n if i >= stage + 1:\n idm_bn_names.append(layer_names[i])\n return idm_bn_names\n\n\ndef convert_dsbn_idm(model, mixup_bn_names, idm=False):\n for _, (child_name, child) in enumerate(model.named_children()):\n # print(child_name)\n idm_flag = idm\n for name in mixup_bn_names:\n if name in child_name:\n idm_flag = True\n if isinstance(child, nn.BatchNorm2d) and not idm_flag:\n m = DSBN2d(child.num_features)\n m.BN_S.load_state_dict(child.state_dict())\n m.BN_T.load_state_dict(child.state_dict())\n setattr(model, child_name, m)\n elif isinstance(child, nn.BatchNorm2d) and idm_flag:\n m = DSBN2d_idm(child.num_features)\n m.BN_S.load_state_dict(child.state_dict())\n m.BN_T.load_state_dict(child.state_dict())\n m.BN_mix.load_state_dict(child.state_dict())\n setattr(model, child_name, m)\n elif isinstance(child, nn.BatchNorm1d) and not idm_flag:\n m = DSBN1d(child.num_features)\n m.BN_S.load_state_dict(child.state_dict())\n m.BN_T.load_state_dict(child.state_dict())\n setattr(model, child_name, m)\n elif isinstance(child, nn.BatchNorm1d) and idm_flag:\n m = DSBN1d_idm(child.num_features)\n m.BN_S.load_state_dict(child.state_dict())\n m.BN_T.load_state_dict(child.state_dict())\n m.BN_mix.load_state_dict(child.state_dict())\n setattr(model, child_name, m)\n else:\n convert_dsbn_idm(child, mixup_bn_names, idm=idm_flag)\n\n\ndef convert_bn_idm(model, use_target=True):\n for _, (child_name, child) in enumerate(model.named_children()):\n assert (not next(model.parameters()).is_cuda)\n if isinstance(child, DSBN2d):\n m = nn.BatchNorm2d(child.num_features)\n if use_target:\n m.load_state_dict(child.BN_T.state_dict())\n else:\n m.load_state_dict(child.BN_S.state_dict())\n setattr(model, child_name, m)\n elif isinstance(child, DSBN2d_idm):\n m = nn.BatchNorm2d(child.num_features)\n if use_target:\n m.load_state_dict(child.BN_T.state_dict())\n else:\n m.load_state_dict(child.BN_S.state_dict())\n setattr(model, child_name, m)\n elif isinstance(child, DSBN1d):\n m = nn.BatchNorm1d(child.num_features)\n if use_target:\n m.load_state_dict(child.BN_T.state_dict())\n else:\n m.load_state_dict(child.BN_S.state_dict())\n setattr(model, child_name, m)\n elif isinstance(child, DSBN1d_idm):\n m = nn.BatchNorm1d(child.num_features)\n if use_target:\n m.load_state_dict(child.BN_T.state_dict())\n else:\n m.load_state_dict(child.BN_S.state_dict())\n setattr(model, child_name, m)\n else:\n convert_bn_idm(child, use_target=use_target)\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/dalib/adaptation/idm/xbm.py",
"content": "\"\"\"\nModified from https://github.com/SikaStar/IDM\n@author: Baixu Chen\n@contact: cbx_99_hasta@outlook.com\n\"\"\"\nimport torch\n\n\nclass XBM(object):\n \"\"\"`XBM` module in `Cross-Batch Memory for Embedding Learning (CVPR 2020) `_.\n \"\"\"\n def __init__(self, memory_size, feature_size):\n self.K = memory_size\n self.D = feature_size\n self.feats = torch.zeros(self.K, self.D).cuda()\n self.targets = torch.zeros(self.K, dtype=torch.long).cuda()\n self.ptr = 0\n\n @property\n def is_full(self):\n return self.targets[-1].item() != 0\n\n def get(self):\n if self.is_full:\n return self.feats, self.targets\n else:\n return self.feats[:self.ptr], self.targets[:self.ptr]\n\n def enqueue_dequeue(self, feats, targets):\n q_size = len(targets)\n if self.ptr + q_size > self.K:\n self.feats[-q_size:] = feats\n self.targets[-q_size:] = targets\n self.ptr = 0\n else:\n self.feats[self.ptr: self.ptr + q_size] = feats\n self.targets[self.ptr: self.ptr + q_size] = targets\n self.ptr += q_size\n\n def clean_target_domain(self, source_classes, target_classes):\n empty_feats = torch.zeros(self.K, self.D).cuda()\n empty_targets = torch.zeros(self.K, dtype=torch.long).cuda()\n j = 0\n for i in range(self.K):\n if source_classes <= self.targets[i] < source_classes + target_classes:\n continue\n else:\n empty_feats[j] = self.feats[i]\n empty_targets[j] = self.targets[i]\n j += 1\n self.feats = empty_feats\n self.targets = empty_targets\n if j > 0:\n self.ptr = j\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/dalib/adaptation/iwan.py",
"content": "\"\"\"\n@author: Baixu Chen\n@contact: cbx_99_hasta@outlook.com\n\"\"\"\nfrom typing import Optional, List, Dict\nimport torch\nimport torch.nn as nn\n\nfrom common.modules.classifier import Classifier as ClassifierBase\n\n\nclass ImportanceWeightModule(object):\n r\"\"\"\n Calculating class weight based on the output of discriminator.\n Introduced by `Importance Weighted Adversarial Nets for Partial Domain Adaptation (CVPR 2018) `_\n\n Args:\n discriminator (torch.nn.Module): A domain discriminator object, which predicts the domains of features.\n Its input shape is :math:`(N, F)` and output shape is :math:`(N, 1)`\n partial_classes_index (list[int], optional): The index of partial classes. Note that this parameter is \\\n just for debugging, since in real-world dataset, we have no access to the index of partial classes. \\\n Default: None.\n\n Examples::\n\n >>> domain_discriminator = DomainDiscriminator(1024, 1024)\n >>> importance_weight_module = ImportanceWeightModule(domain_discriminator)\n >>> num_iterations = 10000\n >>> for _ in range(num_iterations):\n >>> # feature from source domain\n >>> f_s = torch.randn(32, 1024)\n >>> # importance weights for source instance\n >>> w_s = importance_weight_module.get_importance_weight(f_s)\n \"\"\"\n\n def __init__(self, discriminator: nn.Module, partial_classes_index: Optional[List[int]] = None):\n self.discriminator = discriminator\n self.partial_classes_index = partial_classes_index\n\n def get_importance_weight(self, feature):\n \"\"\"\n Get importance weights for each instance.\n\n Args:\n feature (tensor): feature from source domain, in shape :math:`(N, F)`\n\n Returns:\n instance weight in shape :math:`(N, 1)`\n \"\"\"\n weight = 1. - self.discriminator(feature)\n weight = weight / weight.mean()\n weight = weight.detach()\n return weight\n\n def get_partial_classes_weight(self, weights: torch.Tensor, labels: torch.Tensor):\n \"\"\"\n Get class weight averaged on the partial classes and non-partial classes respectively.\n\n Args:\n weights (tensor): instance weight in shape :math:`(N, 1)`\n labels (tensor): ground truth labels in shape :math:`(N, 1)`\n\n .. warning::\n This function is just for debugging, since in real-world dataset, we have no access to the index of \\\n partial classes and this function will throw an error when `partial_classes_index` is None.\n \"\"\"\n assert self.partial_classes_index is not None\n\n weights = weights.squeeze()\n is_partial = torch.Tensor([label in self.partial_classes_index for label in labels]).to(weights.device)\n if is_partial.sum() > 0:\n partial_classes_weight = (weights * is_partial).sum() / is_partial.sum()\n else:\n partial_classes_weight = torch.tensor(0)\n\n not_partial = 1. - is_partial\n if not_partial.sum() > 0:\n not_partial_classes_weight = (weights * not_partial).sum() / not_partial.sum()\n else:\n not_partial_classes_weight = torch.tensor(0)\n return partial_classes_weight, not_partial_classes_weight\n\n\nclass ImageClassifier(ClassifierBase):\n r\"\"\"The Image Classifier for `Importance Weighted Adversarial Nets for Partial Domain Adaptation `_\n \"\"\"\n\n def __init__(self, backbone: nn.Module, num_classes: int, bottleneck_dim: Optional[int] = 256, **kwargs):\n bottleneck = nn.Sequential(\n nn.Linear(backbone.out_features, bottleneck_dim),\n nn.BatchNorm1d(bottleneck_dim),\n nn.ReLU()\n )\n super(ImageClassifier, self).__init__(backbone, num_classes, bottleneck, bottleneck_dim, **kwargs)\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/dalib/adaptation/jan.py",
"content": "\"\"\"\n@author: Junguang Jiang\n@contact: JiangJunguang1123@outlook.com\n\"\"\"\nfrom typing import Optional, Sequence\nimport torch\nimport torch.nn as nn\n\nfrom common.modules.classifier import Classifier as ClassifierBase\nfrom ..modules.grl import GradientReverseLayer\nfrom ..modules.kernels import GaussianKernel\nfrom .dan import _update_index_matrix\n\n\n__all__ = ['JointMultipleKernelMaximumMeanDiscrepancy', 'ImageClassifier']\n\n\n\nclass JointMultipleKernelMaximumMeanDiscrepancy(nn.Module):\n r\"\"\"The Joint Multiple Kernel Maximum Mean Discrepancy (JMMD) used in\n `Deep Transfer Learning with Joint Adaptation Networks (ICML 2017) `_\n\n Given source domain :math:`\\mathcal{D}_s` of :math:`n_s` labeled points and target domain :math:`\\mathcal{D}_t`\n of :math:`n_t` unlabeled points drawn i.i.d. from P and Q respectively, the deep networks will generate\n activations in layers :math:`\\mathcal{L}` as :math:`\\{(z_i^{s1}, ..., z_i^{s|\\mathcal{L}|})\\}_{i=1}^{n_s}` and\n :math:`\\{(z_i^{t1}, ..., z_i^{t|\\mathcal{L}|})\\}_{i=1}^{n_t}`. The empirical estimate of\n :math:`\\hat{D}_{\\mathcal{L}}(P, Q)` is computed as the squared distance between the empirical kernel mean\n embeddings as\n\n .. math::\n \\hat{D}_{\\mathcal{L}}(P, Q) &=\n \\dfrac{1}{n_s^2} \\sum_{i=1}^{n_s}\\sum_{j=1}^{n_s} \\prod_{l\\in\\mathcal{L}} k^l(z_i^{sl}, z_j^{sl}) \\\\\n &+ \\dfrac{1}{n_t^2} \\sum_{i=1}^{n_t}\\sum_{j=1}^{n_t} \\prod_{l\\in\\mathcal{L}} k^l(z_i^{tl}, z_j^{tl}) \\\\\n &- \\dfrac{2}{n_s n_t} \\sum_{i=1}^{n_s}\\sum_{j=1}^{n_t} \\prod_{l\\in\\mathcal{L}} k^l(z_i^{sl}, z_j^{tl}). \\\\\n\n Args:\n kernels (tuple(tuple(torch.nn.Module))): kernel functions, where `kernels[r]` corresponds to kernel :math:`k^{\\mathcal{L}[r]}`.\n linear (bool): whether use the linear version of JAN. Default: False\n thetas (list(Theta): use adversarial version JAN if not None. Default: None\n\n Inputs:\n - z_s (tuple(tensor)): multiple layers' activations from the source domain, :math:`z^s`\n - z_t (tuple(tensor)): multiple layers' activations from the target domain, :math:`z^t`\n\n Shape:\n - :math:`z^{sl}` and :math:`z^{tl}`: :math:`(minibatch, *)` where * means any dimension\n - Outputs: scalar\n\n .. note::\n Activations :math:`z^{sl}` and :math:`z^{tl}` must have the same shape.\n\n .. note::\n The kernel values will add up when there are multiple kernels for a certain layer.\n\n Examples::\n\n >>> feature_dim = 1024\n >>> batch_size = 10\n >>> layer1_kernels = (GaussianKernel(alpha=0.5), GaussianKernel(1.), GaussianKernel(2.))\n >>> layer2_kernels = (GaussianKernel(1.), )\n >>> loss = JointMultipleKernelMaximumMeanDiscrepancy((layer1_kernels, layer2_kernels))\n >>> # layer1 features from source domain and target domain\n >>> z1_s, z1_t = torch.randn(batch_size, feature_dim), torch.randn(batch_size, feature_dim)\n >>> # layer2 features from source domain and target domain\n >>> z2_s, z2_t = torch.randn(batch_size, feature_dim), torch.randn(batch_size, feature_dim)\n >>> output = loss((z1_s, z2_s), (z1_t, z2_t))\n \"\"\"\n\n def __init__(self, kernels: Sequence[Sequence[nn.Module]], linear: Optional[bool] = True, thetas: Sequence[nn.Module] = None):\n super(JointMultipleKernelMaximumMeanDiscrepancy, self).__init__()\n self.kernels = kernels\n self.index_matrix = None\n self.linear = linear\n if thetas:\n self.thetas = thetas\n else:\n self.thetas = [nn.Identity() for _ in kernels]\n\n def forward(self, z_s: torch.Tensor, z_t: torch.Tensor) -> torch.Tensor:\n batch_size = int(z_s[0].size(0))\n self.index_matrix = _update_index_matrix(batch_size, self.index_matrix, self.linear).to(z_s[0].device)\n\n kernel_matrix = torch.ones_like(self.index_matrix)\n for layer_z_s, layer_z_t, layer_kernels, theta in zip(z_s, z_t, self.kernels, self.thetas):\n layer_features = torch.cat([layer_z_s, layer_z_t], dim=0)\n layer_features = theta(layer_features)\n kernel_matrix *= sum(\n [kernel(layer_features) for kernel in layer_kernels]) # Add up the matrix of each kernel\n\n # Add 2 / (n-1) to make up for the value on the diagonal\n # to ensure loss is positive in the non-linear version\n loss = (kernel_matrix * self.index_matrix).sum() + 2. / float(batch_size - 1)\n return loss\n\n\nclass Theta(nn.Module):\n \"\"\"\n maximize loss respect to :math:`\\theta`\n minimize loss respect to features\n \"\"\"\n def __init__(self, dim: int):\n super(Theta, self).__init__()\n self.grl1 = GradientReverseLayer()\n self.grl2 = GradientReverseLayer()\n self.layer1 = nn.Linear(dim, dim)\n nn.init.eye_(self.layer1.weight)\n nn.init.zeros_(self.layer1.bias)\n\n def forward(self, features: torch.Tensor) -> torch.Tensor:\n features = self.grl1(features)\n return self.grl2(self.layer1(features))\n\n\nclass ImageClassifier(ClassifierBase):\n def __init__(self, backbone: nn.Module, num_classes: int, bottleneck_dim: Optional[int] = 256, **kwargs):\n bottleneck = nn.Sequential(\n # nn.AdaptiveAvgPool2d(output_size=(1, 1)),\n # nn.Flatten(),\n nn.Linear(backbone.out_features, bottleneck_dim),\n nn.BatchNorm1d(bottleneck_dim),\n nn.ReLU(),\n nn.Dropout(0.5)\n )\n super(ImageClassifier, self).__init__(backbone, num_classes, bottleneck, bottleneck_dim, **kwargs)"
},
{
"path": "domain_generalization/Transfer-Learning-Library/dalib/adaptation/mcc.py",
"content": "\"\"\"\n@author: Ying Jin\n@contact: sherryying003@gmail.com\n\"\"\"\nfrom typing import Optional\nimport torch\nimport torch.nn as nn\nimport torch.nn.functional as F\n\nfrom common.modules.classifier import Classifier as ClassifierBase\nfrom ..modules.entropy import entropy\n\n\n__all__ = ['MinimumClassConfusionLoss', 'ImageClassifier']\n\n\nclass MinimumClassConfusionLoss(nn.Module):\n r\"\"\"\n Minimum Class Confusion loss minimizes the class confusion in the target predictions.\n\n You can see more details in `Minimum Class Confusion for Versatile Domain Adaptation (ECCV 2020) `_\n\n Args:\n temperature (float) : The temperature for rescaling, the prediction will shrink to vanilla softmax if\n temperature is 1.0.\n\n .. note::\n Make sure that temperature is larger than 0.\n\n Inputs: g_t\n - g_t (tensor): unnormalized classifier predictions on target domain, :math:`g^t`\n\n Shape:\n - g_t: :math:`(minibatch, C)` where C means the number of classes.\n - Output: scalar.\n\n Examples::\n >>> temperature = 2.0\n >>> loss = MinimumClassConfusionLoss(temperature)\n >>> # logits output from target domain\n >>> g_t = torch.randn(batch_size, num_classes)\n >>> output = loss(g_t)\n\n MCC can also serve as a regularizer for existing methods.\n Examples::\n >>> from dalib.modules.domain_discriminator import DomainDiscriminator\n >>> num_classes = 2\n >>> feature_dim = 1024\n >>> batch_size = 10\n >>> temperature = 2.0\n >>> discriminator = DomainDiscriminator(in_feature=feature_dim, hidden_size=1024)\n >>> cdan_loss = ConditionalDomainAdversarialLoss(discriminator, reduction='mean')\n >>> mcc_loss = MinimumClassConfusionLoss(temperature)\n >>> # features from source domain and target domain\n >>> f_s, f_t = torch.randn(batch_size, feature_dim), torch.randn(batch_size, feature_dim)\n >>> # logits output from source domain adn target domain\n >>> g_s, g_t = torch.randn(batch_size, num_classes), torch.randn(batch_size, num_classes)\n >>> total_loss = cdan_loss(g_s, f_s, g_t, f_t) + mcc_loss(g_t)\n \"\"\"\n\n def __init__(self, temperature: float):\n super(MinimumClassConfusionLoss, self).__init__()\n self.temperature = temperature\n\n def forward(self, logits: torch.Tensor) -> torch.Tensor:\n batch_size, num_classes = logits.shape\n predictions = F.softmax(logits / self.temperature, dim=1) # batch_size x num_classes\n entropy_weight = entropy(predictions).detach()\n entropy_weight = 1 + torch.exp(-entropy_weight)\n entropy_weight = (batch_size * entropy_weight / torch.sum(entropy_weight)).unsqueeze(dim=1) # batch_size x 1\n class_confusion_matrix = torch.mm((predictions * entropy_weight).transpose(1, 0), predictions) # num_classes x num_classes\n class_confusion_matrix = class_confusion_matrix / torch.sum(class_confusion_matrix, dim=1)\n mcc_loss = (torch.sum(class_confusion_matrix) - torch.trace(class_confusion_matrix)) / num_classes\n return mcc_loss\n\n\nclass ImageClassifier(ClassifierBase):\n def __init__(self, backbone: nn.Module, num_classes: int, bottleneck_dim: Optional[int] = 256, **kwargs):\n bottleneck = nn.Sequential(\n # nn.AdaptiveAvgPool2d(output_size=(1, 1)),\n # nn.Flatten(),\n nn.Linear(backbone.out_features, bottleneck_dim),\n nn.BatchNorm1d(bottleneck_dim),\n nn.ReLU()\n )\n super(ImageClassifier, self).__init__(backbone, num_classes, bottleneck, bottleneck_dim, **kwargs)\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/dalib/adaptation/mcd.py",
"content": "\"\"\"\n@author: Junguang Jiang\n@contact: JiangJunguang1123@outlook.com\n\"\"\"\nfrom typing import Optional\nimport torch.nn as nn\nimport torch\n\n\ndef classifier_discrepancy(predictions1: torch.Tensor, predictions2: torch.Tensor) -> torch.Tensor:\n r\"\"\"The `Classifier Discrepancy` in\n `Maximum Classifier Discrepancy for Unsupervised Domain Adaptation (CVPR 2018) `_.\n\n The classfier discrepancy between predictions :math:`p_1` and :math:`p_2` can be described as:\n\n .. math::\n d(p_1, p_2) = \\dfrac{1}{K} \\sum_{k=1}^K | p_{1k} - p_{2k} |,\n\n where K is number of classes.\n\n Args:\n predictions1 (torch.Tensor): Classifier predictions :math:`p_1`. Expected to contain raw, normalized scores for each class\n predictions2 (torch.Tensor): Classifier predictions :math:`p_2`\n \"\"\"\n return torch.mean(torch.abs(predictions1 - predictions2))\n\n\ndef entropy(predictions: torch.Tensor) -> torch.Tensor:\n r\"\"\"Entropy of N predictions :math:`(p_1, p_2, ..., p_N)`.\n The definition is:\n\n .. math::\n d(p_1, p_2, ..., p_N) = -\\dfrac{1}{K} \\sum_{k=1}^K \\log \\left( \\dfrac{1}{N} \\sum_{i=1}^N p_{ik} \\right)\n\n where K is number of classes.\n\n .. note::\n This entropy function is specifically used in MCD and different from the usual :meth:`~dalib.modules.entropy.entropy` function.\n\n Args:\n predictions (torch.Tensor): Classifier predictions. Expected to contain raw, normalized scores for each class\n \"\"\"\n return -torch.mean(torch.log(torch.mean(predictions, 0) + 1e-6))\n\n\nclass ImageClassifierHead(nn.Module):\n r\"\"\"Classifier Head for MCD.\n\n Args:\n in_features (int): Dimension of input features\n num_classes (int): Number of classes\n bottleneck_dim (int, optional): Feature dimension of the bottleneck layer. Default: 1024\n\n Shape:\n - Inputs: :math:`(minibatch, F)` where F = `in_features`.\n - Output: :math:`(minibatch, C)` where C = `num_classes`.\n \"\"\"\n\n def __init__(self, in_features: int, num_classes: int, bottleneck_dim: Optional[int] = 1024, pool_layer=None):\n super(ImageClassifierHead, self).__init__()\n self.num_classes = num_classes\n if pool_layer is None:\n self.pool_layer = nn.Sequential(\n nn.AdaptiveAvgPool2d(output_size=(1, 1)),\n nn.Flatten()\n )\n else:\n self.pool_layer = pool_layer\n self.head = nn.Sequential(\n nn.Dropout(0.5),\n nn.Linear(in_features, bottleneck_dim),\n nn.BatchNorm1d(bottleneck_dim),\n nn.ReLU(),\n nn.Dropout(0.5),\n nn.Linear(bottleneck_dim, bottleneck_dim),\n nn.BatchNorm1d(bottleneck_dim),\n nn.ReLU(),\n nn.Linear(bottleneck_dim, num_classes)\n )\n\n def forward(self, inputs: torch.Tensor) -> torch.Tensor:\n return self.head(self.pool_layer(inputs))"
},
{
"path": "domain_generalization/Transfer-Learning-Library/dalib/adaptation/mdd.py",
"content": "\"\"\"\n@author: Junguang Jiang\n@contact: JiangJunguang1123@outlook.com\n\"\"\"\nfrom typing import Optional, List, Dict, Tuple, Callable\nimport torch.nn as nn\nimport torch.nn.functional as F\nimport torch\n\nfrom dalib.modules.grl import WarmStartGradientReverseLayer\n\n\nclass MarginDisparityDiscrepancy(nn.Module):\n r\"\"\"The margin disparity discrepancy (MDD) proposed in `Bridging Theory and Algorithm for Domain Adaptation (ICML 2019) `_.\n\n MDD can measure the distribution discrepancy in domain adaptation.\n\n The :math:`y^s` and :math:`y^t` are logits output by the main head on the source and target domain respectively.\n The :math:`y_{adv}^s` and :math:`y_{adv}^t` are logits output by the adversarial head.\n\n The definition can be described as:\n\n .. math::\n \\mathcal{D}_{\\gamma}(\\hat{\\mathcal{S}}, \\hat{\\mathcal{T}}) =\n -\\gamma \\mathbb{E}_{y^s, y_{adv}^s \\sim\\hat{\\mathcal{S}}} L_s (y^s, y_{adv}^s) +\n \\mathbb{E}_{y^t, y_{adv}^t \\sim\\hat{\\mathcal{T}}} L_t (y^t, y_{adv}^t),\n\n where :math:`\\gamma` is a margin hyper-parameter, :math:`L_s` refers to the disparity function defined on the source domain\n and :math:`L_t` refers to the disparity function defined on the target domain.\n\n Args:\n source_disparity (callable): The disparity function defined on the source domain, :math:`L_s`.\n target_disparity (callable): The disparity function defined on the target domain, :math:`L_t`.\n margin (float): margin :math:`\\gamma`. Default: 4\n reduction (str, optional): Specifies the reduction to apply to the output:\n ``'none'`` | ``'mean'`` | ``'sum'``. ``'none'``: no reduction will be applied,\n ``'mean'``: the sum of the output will be divided by the number of\n elements in the output, ``'sum'``: the output will be summed. Default: ``'mean'``\n\n Inputs:\n - y_s: output :math:`y^s` by the main head on the source domain\n - y_s_adv: output :math:`y^s` by the adversarial head on the source domain\n - y_t: output :math:`y^t` by the main head on the target domain\n - y_t_adv: output :math:`y_{adv}^t` by the adversarial head on the target domain\n - w_s (optional): instance weights for source domain\n - w_t (optional): instance weights for target domain\n\n Examples::\n\n >>> num_outputs = 2\n >>> batch_size = 10\n >>> loss = MarginDisparityDiscrepancy(margin=4., source_disparity=F.l1_loss, target_disparity=F.l1_loss)\n >>> # output from source domain and target domain\n >>> y_s, y_t = torch.randn(batch_size, num_outputs), torch.randn(batch_size, num_outputs)\n >>> # adversarial output from source domain and target domain\n >>> y_s_adv, y_t_adv = torch.randn(batch_size, num_outputs), torch.randn(batch_size, num_outputs)\n >>> output = loss(y_s, y_s_adv, y_t, y_t_adv)\n \"\"\"\n\n def __init__(self, source_disparity: Callable, target_disparity: Callable,\n margin: Optional[float] = 4, reduction: Optional[str] = 'mean'):\n super(MarginDisparityDiscrepancy, self).__init__()\n self.margin = margin\n self.reduction = reduction\n self.source_disparity = source_disparity\n self.target_disparity = target_disparity\n\n def forward(self, y_s: torch.Tensor, y_s_adv: torch.Tensor, y_t: torch.Tensor, y_t_adv: torch.Tensor,\n w_s: Optional[torch.Tensor] = None, w_t: Optional[torch.Tensor] = None) -> torch.Tensor:\n\n source_loss = -self.margin * self.source_disparity(y_s, y_s_adv)\n target_loss = self.target_disparity(y_t, y_t_adv)\n if w_s is None:\n w_s = torch.ones_like(source_loss)\n source_loss = source_loss * w_s\n if w_t is None:\n w_t = torch.ones_like(target_loss)\n target_loss = target_loss * w_t\n\n loss = source_loss + target_loss\n if self.reduction == 'mean':\n loss = loss.mean()\n elif self.reduction == 'sum':\n loss = loss.sum()\n return loss\n\n\nclass ClassificationMarginDisparityDiscrepancy(MarginDisparityDiscrepancy):\n r\"\"\"\n The margin disparity discrepancy (MDD) proposed in `Bridging Theory and Algorithm for Domain Adaptation (ICML 2019) `_.\n\n It measures the distribution discrepancy in domain adaptation\n for classification.\n\n When margin is equal to 1, it's also called disparity discrepancy (DD).\n\n The :math:`y^s` and :math:`y^t` are logits output by the main classifier on the source and target domain respectively.\n The :math:`y_{adv}^s` and :math:`y_{adv}^t` are logits output by the adversarial classifier.\n They are expected to contain raw, unnormalized scores for each class.\n\n The definition can be described as:\n\n .. math::\n \\mathcal{D}_{\\gamma}(\\hat{\\mathcal{S}}, \\hat{\\mathcal{T}}) =\n \\gamma \\mathbb{E}_{y^s, y_{adv}^s \\sim\\hat{\\mathcal{S}}} \\log\\left(\\frac{\\exp(y_{adv}^s[h_{y^s}])}{\\sum_j \\exp(y_{adv}^s[j])}\\right) +\n \\mathbb{E}_{y^t, y_{adv}^t \\sim\\hat{\\mathcal{T}}} \\log\\left(1-\\frac{\\exp(y_{adv}^t[h_{y^t}])}{\\sum_j \\exp(y_{adv}^t[j])}\\right),\n\n where :math:`\\gamma` is a margin hyper-parameter and :math:`h_y` refers to the predicted label when the logits output is :math:`y`.\n You can see more details in `Bridging Theory and Algorithm for Domain Adaptation `_.\n\n Args:\n margin (float): margin :math:`\\gamma`. Default: 4\n reduction (str, optional): Specifies the reduction to apply to the output:\n ``'none'`` | ``'mean'`` | ``'sum'``. ``'none'``: no reduction will be applied,\n ``'mean'``: the sum of the output will be divided by the number of\n elements in the output, ``'sum'``: the output will be summed. Default: ``'mean'``\n\n Inputs:\n - y_s: logits output :math:`y^s` by the main classifier on the source domain\n - y_s_adv: logits output :math:`y^s` by the adversarial classifier on the source domain\n - y_t: logits output :math:`y^t` by the main classifier on the target domain\n - y_t_adv: logits output :math:`y_{adv}^t` by the adversarial classifier on the target domain\n\n Shape:\n - Inputs: :math:`(minibatch, C)` where C = number of classes, or :math:`(minibatch, C, d_1, d_2, ..., d_K)`\n with :math:`K \\geq 1` in the case of `K`-dimensional loss.\n - Output: scalar. If :attr:`reduction` is ``'none'``, then the same size as the target: :math:`(minibatch)`, or\n :math:`(minibatch, d_1, d_2, ..., d_K)` with :math:`K \\geq 1` in the case of K-dimensional loss.\n\n Examples::\n\n >>> num_classes = 2\n >>> batch_size = 10\n >>> loss = ClassificationMarginDisparityDiscrepancy(margin=4.)\n >>> # logits output from source domain and target domain\n >>> y_s, y_t = torch.randn(batch_size, num_classes), torch.randn(batch_size, num_classes)\n >>> # adversarial logits output from source domain and target domain\n >>> y_s_adv, y_t_adv = torch.randn(batch_size, num_classes), torch.randn(batch_size, num_classes)\n >>> output = loss(y_s, y_s_adv, y_t, y_t_adv)\n \"\"\"\n\n def __init__(self, margin: Optional[float] = 4, **kwargs):\n def source_discrepancy(y: torch.Tensor, y_adv: torch.Tensor):\n _, prediction = y.max(dim=1)\n return F.cross_entropy(y_adv, prediction, reduction='none')\n\n def target_discrepancy(y: torch.Tensor, y_adv: torch.Tensor):\n _, prediction = y.max(dim=1)\n return -F.nll_loss(shift_log(1. - F.softmax(y_adv, dim=1)), prediction, reduction='none')\n\n super(ClassificationMarginDisparityDiscrepancy, self).__init__(source_discrepancy, target_discrepancy, margin,\n **kwargs)\n\n\nclass RegressionMarginDisparityDiscrepancy(MarginDisparityDiscrepancy):\n r\"\"\"\n The margin disparity discrepancy (MDD) proposed in `Bridging Theory and Algorithm for Domain Adaptation (ICML 2019) `_.\n\n It measures the distribution discrepancy in domain adaptation\n for regression.\n\n The :math:`y^s` and :math:`y^t` are logits output by the main regressor on the source and target domain respectively.\n The :math:`y_{adv}^s` and :math:`y_{adv}^t` are logits output by the adversarial regressor.\n They are expected to contain ``normalized`` values for each factors.\n\n The definition can be described as:\n\n .. math::\n \\mathcal{D}_{\\gamma}(\\hat{\\mathcal{S}}, \\hat{\\mathcal{T}}) =\n -\\gamma \\mathbb{E}_{y^s, y_{adv}^s \\sim\\hat{\\mathcal{S}}} L (y^s, y_{adv}^s) +\n \\mathbb{E}_{y^t, y_{adv}^t \\sim\\hat{\\mathcal{T}}} L (y^t, y_{adv}^t),\n\n where :math:`\\gamma` is a margin hyper-parameter and :math:`L` refers to the disparity function defined on both domains.\n You can see more details in `Bridging Theory and Algorithm for Domain Adaptation `_.\n\n Args:\n loss_function (callable): The disparity function defined on both domains, :math:`L`.\n margin (float): margin :math:`\\gamma`. Default: 1\n reduction (str, optional): Specifies the reduction to apply to the output:\n ``'none'`` | ``'mean'`` | ``'sum'``. ``'none'``: no reduction will be applied,\n ``'mean'``: the sum of the output will be divided by the number of\n elements in the output, ``'sum'``: the output will be summed. Default: ``'mean'``\n\n Inputs:\n - y_s: logits output :math:`y^s` by the main regressor on the source domain\n - y_s_adv: logits output :math:`y^s` by the adversarial regressor on the source domain\n - y_t: logits output :math:`y^t` by the main regressor on the target domain\n - y_t_adv: logits output :math:`y_{adv}^t` by the adversarial regressor on the target domain\n\n Shape:\n - Inputs: :math:`(minibatch, F)` where F = number of factors, or :math:`(minibatch, F, d_1, d_2, ..., d_K)`\n with :math:`K \\geq 1` in the case of `K`-dimensional loss.\n - Output: scalar. The same size as the target: :math:`(minibatch)`, or\n :math:`(minibatch, d_1, d_2, ..., d_K)` with :math:`K \\geq 1` in the case of K-dimensional loss.\n\n Examples::\n\n >>> num_outputs = 2\n >>> batch_size = 10\n >>> loss = RegressionMarginDisparityDiscrepancy(margin=4., loss_function=F.l1_loss)\n >>> # output from source domain and target domain\n >>> y_s, y_t = torch.randn(batch_size, num_outputs), torch.randn(batch_size, num_outputs)\n >>> # adversarial output from source domain and target domain\n >>> y_s_adv, y_t_adv = torch.randn(batch_size, num_outputs), torch.randn(batch_size, num_outputs)\n >>> output = loss(y_s, y_s_adv, y_t, y_t_adv)\n\n \"\"\"\n\n def __init__(self, margin: Optional[float] = 1, loss_function=F.l1_loss, **kwargs):\n def source_discrepancy(y: torch.Tensor, y_adv: torch.Tensor):\n return loss_function(y_adv, y.detach(), reduction='none')\n\n def target_discrepancy(y: torch.Tensor, y_adv: torch.Tensor):\n return loss_function(y_adv, y.detach(), reduction='none')\n\n super(RegressionMarginDisparityDiscrepancy, self).__init__(source_discrepancy, target_discrepancy, margin,\n **kwargs)\n\n\ndef shift_log(x: torch.Tensor, offset: Optional[float] = 1e-6) -> torch.Tensor:\n r\"\"\"\n First shift, then calculate log, which can be described as:\n\n .. math::\n y = \\max(\\log(x+\\text{offset}), 0)\n\n Used to avoid the gradient explosion problem in log(x) function when x=0.\n\n Args:\n x (torch.Tensor): input tensor\n offset (float, optional): offset size. Default: 1e-6\n\n .. note::\n Input tensor falls in [0., 1.] and the output tensor falls in [-log(offset), 0]\n \"\"\"\n return torch.log(torch.clamp(x + offset, max=1.))\n\n\nclass GeneralModule(nn.Module):\n def __init__(self, backbone: nn.Module, num_classes: int, bottleneck: nn.Module,\n head: nn.Module, adv_head: nn.Module, grl: Optional[WarmStartGradientReverseLayer] = None,\n finetune: Optional[bool] = True):\n super(GeneralModule, self).__init__()\n self.backbone = backbone\n self.num_classes = num_classes\n self.bottleneck = bottleneck\n self.head = head\n self.adv_head = adv_head\n self.finetune = finetune\n self.grl_layer = WarmStartGradientReverseLayer(alpha=1.0, lo=0.0, hi=0.1, max_iters=1000,\n auto_step=False) if grl is None else grl\n\n def forward(self, x: torch.Tensor) -> Tuple[torch.Tensor, torch.Tensor]:\n \"\"\"\"\"\"\n features = self.backbone(x)\n features = self.bottleneck(features)\n outputs = self.head(features)\n features_adv = self.grl_layer(features)\n outputs_adv = self.adv_head(features_adv)\n if self.training:\n return outputs, outputs_adv\n else:\n return outputs\n\n def step(self):\n \"\"\"\n Gradually increase :math:`\\lambda` in GRL layer.\n \"\"\"\n self.grl_layer.step()\n\n def get_parameters(self, base_lr=1.0) -> List[Dict]:\n \"\"\"\n Return a parameters list which decides optimization hyper-parameters,\n such as the relative learning rate of each layer.\n \"\"\"\n params = [\n {\"params\": self.backbone.parameters(), \"lr\": 0.1 * base_lr if self.finetune else base_lr},\n {\"params\": self.bottleneck.parameters(), \"lr\": base_lr},\n {\"params\": self.head.parameters(), \"lr\": base_lr},\n {\"params\": self.adv_head.parameters(), \"lr\": base_lr}\n ]\n return params\n\n\nclass ImageClassifier(GeneralModule):\n r\"\"\"Classifier for MDD.\n\n Classifier for MDD has one backbone, one bottleneck, while two classifier heads.\n The first classifier head is used for final predictions.\n The adversarial classifier head is only used when calculating MarginDisparityDiscrepancy.\n\n\n Args:\n backbone (torch.nn.Module): Any backbone to extract 1-d features from data\n num_classes (int): Number of classes\n bottleneck_dim (int, optional): Feature dimension of the bottleneck layer. Default: 1024\n width (int, optional): Feature dimension of the classifier head. Default: 1024\n grl (nn.Module): Gradient reverse layer. Will use default parameters if None. Default: None.\n finetune (bool, optional): Whether use 10x smaller learning rate in the backbone. Default: True\n\n Inputs:\n - x (tensor): input data\n\n Outputs:\n - outputs: logits outputs by the main classifier\n - outputs_adv: logits outputs by the adversarial classifier\n\n Shape:\n - x: :math:`(minibatch, *)`, same shape as the input of the `backbone`.\n - outputs, outputs_adv: :math:`(minibatch, C)`, where C means the number of classes.\n\n .. note::\n Remember to call function `step()` after function `forward()` **during training phase**! For instance,\n\n >>> # x is inputs, classifier is an ImageClassifier\n >>> outputs, outputs_adv = classifier(x)\n >>> classifier.step()\n\n \"\"\"\n\n def __init__(self, backbone: nn.Module, num_classes: int,\n bottleneck_dim: Optional[int] = 1024, width: Optional[int] = 1024,\n grl: Optional[WarmStartGradientReverseLayer] = None, finetune=True, pool_layer=None):\n grl_layer = WarmStartGradientReverseLayer(alpha=1.0, lo=0.0, hi=0.1, max_iters=1000,\n auto_step=False) if grl is None else grl\n\n if pool_layer is None:\n pool_layer = nn.Sequential(\n nn.AdaptiveAvgPool2d(output_size=(1, 1)),\n nn.Flatten()\n )\n bottleneck = nn.Sequential(\n pool_layer,\n nn.Linear(backbone.out_features, bottleneck_dim),\n nn.BatchNorm1d(bottleneck_dim),\n nn.ReLU(),\n nn.Dropout(0.5)\n )\n bottleneck[1].weight.data.normal_(0, 0.005)\n bottleneck[1].bias.data.fill_(0.1)\n\n # The classifier head used for final predictions.\n head = nn.Sequential(\n nn.Linear(bottleneck_dim, width),\n nn.ReLU(),\n nn.Dropout(0.5),\n nn.Linear(width, num_classes)\n )\n # The adversarial classifier head\n adv_head = nn.Sequential(\n nn.Linear(bottleneck_dim, width),\n nn.ReLU(),\n nn.Dropout(0.5),\n nn.Linear(width, num_classes)\n )\n for dep in range(2):\n head[dep * 3].weight.data.normal_(0, 0.01)\n head[dep * 3].bias.data.fill_(0.0)\n adv_head[dep * 3].weight.data.normal_(0, 0.01)\n adv_head[dep * 3].bias.data.fill_(0.0)\n super(ImageClassifier, self).__init__(backbone, num_classes, bottleneck,\n head, adv_head, grl_layer, finetune)\n\n\nclass ImageRegressor(GeneralModule):\n r\"\"\"Regressor for MDD.\n\n Regressor for MDD has one backbone, one bottleneck, while two regressor heads.\n The first regressor head is used for final predictions.\n The adversarial regressor head is only used when calculating MarginDisparityDiscrepancy.\n\n\n Args:\n backbone (torch.nn.Module): Any backbone to extract 1-d features from data\n num_factors (int): Number of factors\n bottleneck_dim (int, optional): Feature dimension of the bottleneck layer. Default: 1024\n width (int, optional): Feature dimension of the classifier head. Default: 1024\n finetune (bool, optional): Whether use 10x smaller learning rate in the backbone. Default: True\n\n Inputs:\n - x (Tensor): input data\n\n Outputs: (outputs, outputs_adv)\n - outputs: outputs by the main regressor\n - outputs_adv: outputs by the adversarial regressor\n\n Shape:\n - x: :math:`(minibatch, *)`, same shape as the input of the `backbone`.\n - outputs, outputs_adv: :math:`(minibatch, F)`, where F means the number of factors.\n\n .. note::\n Remember to call function `step()` after function `forward()` **during training phase**! For instance,\n\n >>> # x is inputs, regressor is an ImageRegressor\n >>> outputs, outputs_adv = regressor(x)\n >>> regressor.step()\n\n \"\"\"\n\n def __init__(self, backbone: nn.Module, num_factors: int, bottleneck = None, head=None, adv_head=None,\n bottleneck_dim: Optional[int] = 1024, width: Optional[int] = 1024, finetune=True):\n grl_layer = WarmStartGradientReverseLayer(alpha=1.0, lo=0.0, hi=0.1, max_iters=1000, auto_step=False)\n if bottleneck is None:\n bottleneck = nn.Sequential(\n nn.Conv2d(backbone.out_features, bottleneck_dim, kernel_size=3, stride=1, padding=1),\n nn.BatchNorm2d(bottleneck_dim),\n nn.ReLU(),\n )\n\n # The regressor head used for final predictions.\n if head is None:\n head = nn.Sequential(\n nn.Conv2d(bottleneck_dim, width, kernel_size=3, stride=1, padding=1),\n nn.BatchNorm2d(width),\n nn.ReLU(),\n nn.Conv2d(width, width, kernel_size=3, stride=1, padding=1),\n nn.BatchNorm2d(width),\n nn.ReLU(),\n nn.AdaptiveAvgPool2d(output_size=(1, 1)),\n nn.Flatten(),\n nn.Linear(width, num_factors),\n nn.Sigmoid()\n )\n for layer in head:\n if isinstance(layer, nn.Conv2d) or isinstance(layer, nn.Linear):\n nn.init.normal_(layer.weight, 0, 0.01)\n nn.init.constant_(layer.bias, 0)\n # The adversarial regressor head\n if adv_head is None:\n adv_head = nn.Sequential(\n nn.Conv2d(bottleneck_dim, width, kernel_size=3, stride=1, padding=1),\n nn.BatchNorm2d(width),\n nn.ReLU(),\n nn.Conv2d(width, width, kernel_size=3, stride=1, padding=1),\n nn.BatchNorm2d(width),\n nn.ReLU(),\n nn.AdaptiveAvgPool2d(output_size=(1, 1)),\n nn.Flatten(),\n nn.Linear(width, num_factors),\n nn.Sigmoid()\n )\n for layer in adv_head:\n if isinstance(layer, nn.Conv2d) or isinstance(layer, nn.Linear):\n nn.init.normal_(layer.weight, 0, 0.01)\n nn.init.constant_(layer.bias, 0)\n super(ImageRegressor, self).__init__(backbone, num_factors, bottleneck,\n head, adv_head, grl_layer, finetune)\n self.num_factors = num_factors\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/dalib/adaptation/osbp.py",
"content": "\"\"\"\n@author: Junguang Jiang\n@contact: JiangJunguang1123@outlook.com\n\"\"\"\nfrom typing import Optional\nimport torch\nimport torch.nn as nn\nimport torch.nn.functional as F\n\nfrom common.modules.classifier import Classifier as ClassifierBase\nfrom ..modules.grl import GradientReverseLayer\n\n\nclass UnknownClassBinaryCrossEntropy(nn.Module):\n r\"\"\"\n Binary cross entropy loss to make a boundary for unknown samples, proposed by\n `Open Set Domain Adaptation by Backpropagation (ECCV 2018) `_.\n\n Given a sample on target domain :math:`x_t` and its classifcation outputs :math:`y`, the binary cross entropy\n loss is defined as\n\n .. math::\n L_{\\text{adv}}(x_t) = -t \\text{log}(p(y=C+1|x_t)) - (1-t)\\text{log}(1-p(y=C+1|x_t))\n\n where t is a hyper-parameter and C is the number of known classes.\n\n Args:\n t (float): Predefined hyper-parameter. Default: 0.5\n\n Inputs:\n - y (tensor): classification outputs (before softmax).\n\n Shape:\n - y: :math:`(minibatch, C+1)` where C is the number of known classes.\n - Outputs: scalar\n\n \"\"\"\n def __init__(self, t: Optional[float]=0.5):\n super(UnknownClassBinaryCrossEntropy, self).__init__()\n self.t = t\n\n def forward(self, y):\n # y : N x (C+1)\n softmax_output = F.softmax(y, dim=1)\n unknown_class_prob = softmax_output[:, -1].contiguous().view(-1, 1)\n known_class_prob = 1. - unknown_class_prob\n\n unknown_target = torch.ones((y.size(0), 1)).to(y.device) * self.t\n known_target = 1. - unknown_target\n return - torch.mean(unknown_target * torch.log(unknown_class_prob + 1e-6)) \\\n - torch.mean(known_target * torch.log(known_class_prob + 1e-6))\n\n\nclass ImageClassifier(ClassifierBase):\n def __init__(self, backbone: nn.Module, num_classes: int, bottleneck_dim: Optional[int] = 256, **kwargs):\n bottleneck = nn.Sequential(\n # nn.AdaptiveAvgPool2d(output_size=(1, 1)),\n # nn.Flatten(),\n nn.Linear(backbone.out_features, bottleneck_dim),\n nn.BatchNorm1d(bottleneck_dim),\n nn.ReLU(),\n nn.Dropout(),\n nn.Linear(bottleneck_dim, bottleneck_dim),\n nn.BatchNorm1d(bottleneck_dim),\n nn.ReLU(),\n nn.Dropout()\n )\n super(ImageClassifier, self).__init__(backbone, num_classes, bottleneck, bottleneck_dim, **kwargs)\n self.grl = GradientReverseLayer()\n\n def forward(self, x: torch.Tensor, grad_reverse: Optional[bool] = False):\n features = self.pool_layer(self.backbone(x))\n features = self.bottleneck(features)\n if grad_reverse:\n features = self.grl(features)\n outputs = self.head(features)\n if self.training:\n return outputs, features\n else:\n return outputs\n\n\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/dalib/adaptation/pada.py",
"content": "\"\"\"\n@author: Junguang Jiang\n@contact: JiangJunguang1123@outlook.com\n\"\"\"\nfrom typing import Optional, List, Tuple\n\nfrom torch.utils.data.dataloader import DataLoader\nimport torch.nn as nn\nimport torch\nimport torch.nn.functional as F\n\n\nclass AutomaticUpdateClassWeightModule(object):\n r\"\"\"\n Calculating class weight based on the output of classifier. See ``ClassWeightModule`` about the details of the calculation.\n Every N iterations, the class weight is updated automatically.\n\n Args:\n update_steps (int): N, the number of iterations to update class weight.\n data_loader (torch.utils.data.DataLoader): The data loader from which we can collect classification outputs.\n classifier (torch.nn.Module): Classifier.\n num_classes (int): Number of classes.\n device (torch.device): The device to run classifier.\n temperature (float, optional): T, temperature in ClassWeightModule. Default: 0.1\n partial_classes_index (list[int], optional): The index of partial classes. Note that this parameter is \\\n just for debugging, since in real-world dataset, we have no access to the index of partial classes. \\\n Default: None.\n\n Examples::\n\n >>> class_weight_module = AutomaticUpdateClassWeightModule(update_steps=500, ...)\n >>> num_iterations = 10000\n >>> for _ in range(num_iterations):\n >>> class_weight_module.step()\n >>> # weight for F.cross_entropy\n >>> w_c = class_weight_module.get_class_weight_for_cross_entropy_loss()\n >>> # weight for dalib.addaptation.dann.DomainAdversarialLoss\n >>> w_s, w_t = class_weight_module.get_class_weight_for_adversarial_loss()\n \"\"\"\n\n def __init__(self, update_steps: int, data_loader: DataLoader,\n classifier: nn.Module, num_classes: int,\n device: torch.device, temperature: Optional[float] = 0.1,\n partial_classes_index: Optional[List[int]] = None):\n self.update_steps = update_steps\n self.data_loader = data_loader\n self.classifier = classifier\n self.device = device\n self.class_weight_module = ClassWeightModule(temperature)\n self.class_weight = torch.ones(num_classes).to(device)\n self.num_steps = 0\n self.partial_classes_index = partial_classes_index\n if partial_classes_index is not None:\n self.non_partial_classes_index = [c for c in range(num_classes) if c not in partial_classes_index]\n\n def step(self):\n self.num_steps += 1\n if self.num_steps % self.update_steps == 0:\n all_outputs = collect_classification_results(self.data_loader, self.classifier, self.device)\n self.class_weight = self.class_weight_module(all_outputs)\n\n def get_class_weight_for_cross_entropy_loss(self):\n \"\"\"\n Outputs: weight for F.cross_entropy\n\n Shape: :math:`(C, )` where C means the number of classes.\n \"\"\"\n return self.class_weight\n\n def get_class_weight_for_adversarial_loss(self, source_labels: torch.Tensor) -> Tuple[torch.Tensor, torch.Tensor]:\n \"\"\"\n Outputs:\n - w_s: source weight for :py:class:`~dalib.adaptation.dann.DomainAdversarialLoss`\n - w_t: target weight for :py:class:`~dalib.adaptation.dann.DomainAdversarialLoss`\n\n Shape:\n - w_s: :math:`(minibatch, )`\n - w_t: :math:`(minibatch, )`\n \"\"\"\n class_weight_adv_source = self.class_weight[source_labels]\n class_weight_adv_target = torch.ones_like(class_weight_adv_source) * class_weight_adv_source.mean()\n return class_weight_adv_source, class_weight_adv_target\n\n def get_partial_classes_weight(self):\n \"\"\"\n Get class weight averaged on the partial classes and non-partial classes respectively.\n\n .. warning::\n\n This function is just for debugging, since in real-world dataset, we have no access to the index of \\\n partial classes and this function will throw an error when `partial_classes_index` is None.\n \"\"\"\n assert self.partial_classes_index is not None\n return torch.mean(self.class_weight[self.partial_classes_index]), torch.mean(\n self.class_weight[self.non_partial_classes_index])\n\n\nclass ClassWeightModule(nn.Module):\n r\"\"\"\n Calculating class weight based on the output of classifier.\n Introduced by `Partial Adversarial Domain Adaptation (ECCV 2018) `_\n\n Given classification logits outputs :math:`\\{\\hat{y}_i\\}_{i=1}^n`, where :math:`n` is the dataset size,\n the weight indicating the contribution of each class to the training can be calculated as\n follows\n\n .. math::\n \\mathcal{\\gamma} = \\dfrac{1}{n} \\sum_{i=1}^{n}\\text{softmax}( \\hat{y}_i / T),\n\n where :math:`\\mathcal{\\gamma}` is a :math:`|\\mathcal{C}|`-dimensional weight vector quantifying the contribution\n of each class and T is a hyper-parameters called temperature.\n\n In practice, it's possible that some of the weights are very small, thus, we normalize weight :math:`\\mathcal{\\gamma}`\n by dividing its largest element, i.e. :math:`\\mathcal{\\gamma} \\leftarrow \\mathcal{\\gamma} / max(\\mathcal{\\gamma})`\n\n Args:\n temperature (float, optional): hyper-parameters :math:`T`. Default: 0.1\n\n Shape:\n - Inputs: (minibatch, :math:`|\\mathcal{C}|`)\n - Outputs: (:math:`|\\mathcal{C}|`,)\n \"\"\"\n\n def __init__(self, temperature: Optional[float] = 0.1):\n super(ClassWeightModule, self).__init__()\n self.temperature = temperature\n\n def forward(self, outputs: torch.Tensor):\n outputs.detach_()\n softmax_outputs = F.softmax(outputs / self.temperature, dim=1)\n class_weight = torch.mean(softmax_outputs, dim=0)\n class_weight = class_weight / torch.max(class_weight)\n class_weight = class_weight.view(-1)\n return class_weight\n\n\ndef collect_classification_results(data_loader: DataLoader, classifier: nn.Module,\n device: torch.device) -> torch.Tensor:\n \"\"\"\n Fetch data from `data_loader`, and then use `classifier` to collect classification results\n\n Args:\n data_loader (torch.utils.data.DataLoader): Data loader.\n classifier (torch.nn.Module): A classifier.\n device (torch.device)\n\n Returns:\n Classification results in shape (len(data_loader), :math:`|\\mathcal{C}|`).\n \"\"\"\n training = classifier.training\n classifier.eval()\n all_outputs = []\n with torch.no_grad():\n for i, (images, target) in enumerate(data_loader):\n images = images.to(device)\n output = classifier(images)\n all_outputs.append(output)\n classifier.train(training)\n return torch.cat(all_outputs, dim=0)\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/dalib/adaptation/regda.py",
"content": "\"\"\"\n@author: Junguang Jiang\n@contact: JiangJunguang1123@outlook.com\n\"\"\"\nfrom typing import Optional\nimport torch\nimport torch.nn as nn\nimport torch.nn.functional as F\nimport numpy as np\n\nfrom dalib.modules.gl import WarmStartGradientLayer\nfrom common.utils.metric.keypoint_detection import get_max_preds\n\n\nclass FastPseudoLabelGenerator2d(nn.Module):\n def __init__(self, sigma=2):\n super().__init__()\n self.sigma = sigma\n \n def forward(self, heatmap: torch.Tensor):\n heatmap = heatmap.detach()\n height, width = heatmap.shape[-2:]\n idx = heatmap.flatten(-2).argmax(dim=-1) # B, K\n pred_h, pred_w = idx.div(width, rounding_mode='floor'), idx.remainder(width) # B, K\n delta_h = torch.arange(height, device=heatmap.device) - pred_h.unsqueeze(-1) # B, K, H\n delta_w = torch.arange(width, device=heatmap.device) - pred_w.unsqueeze(-1) # B, K, W\n gaussian = (delta_h.square().unsqueeze(-1) + delta_w.square().unsqueeze(-2)).div(-2 * self.sigma * self.sigma).exp() # B, K, H, W\n ground_truth = F.threshold(gaussian, threshold=1e-2, value=0.)\n\n ground_false = (ground_truth.sum(dim=1, keepdim=True) - ground_truth).clamp(0., 1.)\n return ground_truth, ground_false\n\n\nclass PseudoLabelGenerator2d(nn.Module):\n \"\"\"\n Generate ground truth heatmap and ground false heatmap from a prediction.\n\n Args:\n num_keypoints (int): Number of keypoints\n height (int): height of the heatmap. Default: 64\n width (int): width of the heatmap. Default: 64\n sigma (int): sigma parameter when generate the heatmap. Default: 2\n\n Inputs:\n - y: predicted heatmap\n\n Outputs:\n - ground_truth: heatmap conforming to Gaussian distribution\n - ground_false: ground false heatmap\n\n Shape:\n - y: :math:`(minibatch, K, H, W)` where K means the number of keypoints,\n H and W is the height and width of the heatmap respectively.\n - ground_truth: :math:`(minibatch, K, H, W)`\n - ground_false: :math:`(minibatch, K, H, W)`\n \"\"\"\n def __init__(self, num_keypoints, height=64, width=64, sigma=2):\n super(PseudoLabelGenerator2d, self).__init__()\n self.height = height\n self.width = width\n self.sigma = sigma\n\n heatmaps = np.zeros((width, height, height, width), dtype=np.float32)\n\n tmp_size = sigma * 3\n for mu_x in range(width):\n for mu_y in range(height):\n # Check that any part of the gaussian is in-bounds\n ul = [int(mu_x - tmp_size), int(mu_y - tmp_size)]\n br = [int(mu_x + tmp_size + 1), int(mu_y + tmp_size + 1)]\n\n # Generate gaussian\n size = 2 * tmp_size + 1\n x = np.arange(0, size, 1, np.float32)\n y = x[:, np.newaxis]\n x0 = y0 = size // 2\n # The gaussian is not normalized, we want the center value to equal 1\n g = np.exp(- ((x - x0) ** 2 + (y - y0) ** 2) / (2 * sigma ** 2))\n\n # Usable gaussian range\n g_x = max(0, -ul[0]), min(br[0], width) - ul[0]\n g_y = max(0, -ul[1]), min(br[1], height) - ul[1]\n # Image range\n img_x = max(0, ul[0]), min(br[0], width)\n img_y = max(0, ul[1]), min(br[1], height)\n\n heatmaps[mu_x][mu_y][img_y[0]:img_y[1], img_x[0]:img_x[1]] = \\\n g[g_y[0]:g_y[1], g_x[0]:g_x[1]]\n\n self.heatmaps = heatmaps\n self.false_matrix = 1. - np.eye(num_keypoints, dtype=np.float32)\n\n def forward(self, y):\n B, K, H, W = y.shape\n y = y.detach()\n preds, max_vals = get_max_preds(y.cpu().numpy()) # B x K x (x, y)\n preds = preds.reshape(-1, 2).astype(np.int)\n ground_truth = self.heatmaps[preds[:, 0], preds[:, 1], :, :].copy().reshape(B, K, H, W).copy()\n\n ground_false = ground_truth.reshape(B, K, -1).transpose((0, 2, 1))\n ground_false = ground_false.dot(self.false_matrix).clip(max=1., min=0.).transpose((0, 2, 1)).reshape(B, K, H, W).copy()\n return torch.from_numpy(ground_truth).to(y.device), torch.from_numpy(ground_false).to(y.device)\n\n\nclass RegressionDisparity(nn.Module):\n \"\"\"\n Regression Disparity proposed by `Regressive Domain Adaptation for Unsupervised Keypoint Detection (CVPR 2021) `_.\n\n Args:\n pseudo_label_generator (PseudoLabelGenerator2d): generate ground truth heatmap and ground false heatmap\n from a prediction.\n criterion (torch.nn.Module): the loss function to calculate distance between two predictions.\n\n Inputs:\n - y: output by the main head\n - y_adv: output by the adversarial head\n - weight (optional): instance weights\n - mode (str): whether minimize the disparity or maximize the disparity. Choices includes ``min``, ``max``.\n Default: ``min``.\n\n Shape:\n - y: :math:`(minibatch, K, H, W)` where K means the number of keypoints,\n H and W is the height and width of the heatmap respectively.\n - y_adv: :math:`(minibatch, K, H, W)`\n - weight: :math:`(minibatch, K)`.\n - Output: depends on the ``criterion``.\n\n Examples::\n\n >>> num_keypoints = 5\n >>> batch_size = 10\n >>> H = W = 64\n >>> pseudo_label_generator = PseudoLabelGenerator2d(num_keypoints)\n >>> from common.vision.models.keypoint_detection.loss import JointsKLLoss\n >>> loss = RegressionDisparity(pseudo_label_generator, JointsKLLoss())\n >>> # output from source domain and target domain\n >>> y_s, y_t = torch.randn(batch_size, num_keypoints, H, W), torch.randn(batch_size, num_keypoints, H, W)\n >>> # adversarial output from source domain and target domain\n >>> y_s_adv, y_t_adv = torch.randn(batch_size, num_keypoints, H, W), torch.randn(batch_size, num_keypoints, H, W)\n >>> # minimize regression disparity on source domain\n >>> output = loss(y_s, y_s_adv, mode='min')\n >>> # maximize regression disparity on target domain\n >>> output = loss(y_t, y_t_adv, mode='max')\n \"\"\"\n def __init__(self, pseudo_label_generator: PseudoLabelGenerator2d, criterion: nn.Module):\n super(RegressionDisparity, self).__init__()\n self.criterion = criterion\n self.pseudo_label_generator = pseudo_label_generator\n\n def forward(self, y, y_adv, weight=None, mode='min'):\n assert mode in ['min', 'max']\n ground_truth, ground_false = self.pseudo_label_generator(y.detach())\n self.ground_truth = ground_truth\n self.ground_false = ground_false\n if mode == 'min':\n return self.criterion(y_adv, ground_truth, weight)\n else:\n return self.criterion(y_adv, ground_false, weight)\n\n\nclass PoseResNet2d(nn.Module):\n \"\"\"\n Pose ResNet for RegDA has one backbone, one upsampling, while two regression heads.\n\n Args:\n backbone (torch.nn.Module): Backbone to extract 2-d features from data\n upsampling (torch.nn.Module): Layer to upsample image feature to heatmap size\n feature_dim (int): The dimension of the features from upsampling layer.\n num_keypoints (int): Number of keypoints\n gl (WarmStartGradientLayer):\n finetune (bool, optional): Whether use 10x smaller learning rate in the backbone. Default: True\n num_head_layers (int): Number of head layers. Default: 2\n\n Inputs:\n - x (tensor): input data\n\n Outputs:\n - outputs: logits outputs by the main regressor\n - outputs_adv: logits outputs by the adversarial regressor\n\n Shape:\n - x: :math:`(minibatch, *)`, same shape as the input of the `backbone`.\n - outputs, outputs_adv: :math:`(minibatch, K, H, W)`, where K means the number of keypoints.\n\n .. note::\n Remember to call function `step()` after function `forward()` **during training phase**! For instance,\n\n >>> # x is inputs, model is an PoseResNet\n >>> outputs, outputs_adv = model(x)\n >>> model.step()\n \"\"\"\n def __init__(self, backbone, upsampling, feature_dim, num_keypoints,\n gl: Optional[WarmStartGradientLayer] = None, finetune: Optional[bool] = True, num_head_layers=2):\n super(PoseResNet2d, self).__init__()\n self.backbone = backbone\n self.upsampling = upsampling\n self.head = self._make_head(num_head_layers, feature_dim, num_keypoints)\n self.head_adv = self._make_head(num_head_layers, feature_dim, num_keypoints)\n self.finetune = finetune\n self.gl_layer = WarmStartGradientLayer(alpha=1.0, lo=0.0, hi=0.1, max_iters=1000, auto_step=False) if gl is None else gl\n\n @staticmethod\n def _make_head(num_layers, channel_dim, num_keypoints):\n layers = []\n for i in range(num_layers-1):\n layers.extend([\n nn.Conv2d(channel_dim, channel_dim, 3, 1, 1),\n nn.BatchNorm2d(channel_dim),\n nn.ReLU(),\n ])\n layers.append(\n nn.Conv2d(\n in_channels=channel_dim,\n out_channels=num_keypoints,\n kernel_size=1,\n stride=1,\n padding=0\n )\n )\n layers = nn.Sequential(*layers)\n for m in layers.modules():\n if isinstance(m, nn.Conv2d):\n nn.init.normal_(m.weight, std=0.001)\n nn.init.constant_(m.bias, 0)\n return layers\n\n def forward(self, x):\n x = self.backbone(x)\n f = self.upsampling(x)\n f_adv = self.gl_layer(f)\n y = self.head(f)\n y_adv = self.head_adv(f_adv)\n\n if self.training:\n return y, y_adv\n else:\n return y\n\n def get_parameters(self, lr=1.):\n return [\n {'params': self.backbone.parameters(), 'lr': 0.1 * lr if self.finetune else lr},\n {'params': self.upsampling.parameters(), 'lr': lr},\n {'params': self.head.parameters(), 'lr': lr},\n {'params': self.head_adv.parameters(), 'lr': lr},\n ]\n\n def step(self):\n \"\"\"Call step() each iteration during training.\n Will increase :math:`\\lambda` in GL layer.\n \"\"\"\n self.gl_layer.step()"
},
{
"path": "domain_generalization/Transfer-Learning-Library/dalib/adaptation/rsd.py",
"content": "\"\"\"\n@author: Junguang Jiang\n@contact: JiangJunguang1123@outlook.com\n\"\"\"\nimport torch.nn as nn\nimport torch\n\n\nclass RepresentationSubspaceDistance(nn.Module):\n \"\"\"\n `Representation Subspace Distance (ICML 2021) `_\n\n Args:\n trade_off (float): The trade-off value between Representation Subspace Distance\n and Base Mismatch Penalization. Default: 0.1\n\n Inputs:\n - f_s (tensor): feature representations on source domain, :math:`f^s`\n - f_t (tensor): feature representations on target domain, :math:`f^t`\n\n \"\"\"\n def __init__(self, trade_off=0.1):\n super(RepresentationSubspaceDistance, self).__init__()\n self.trade_off = trade_off\n\n def forward(self, f_s, f_t):\n U_s, _, _ = torch.svd(f_s.t())\n U_t, _, _ = torch.svd(f_t.t())\n P_s, cosine, P_t = torch.svd(torch.mm(U_s.t(), U_t))\n sine = torch.sqrt(1 - torch.pow(cosine, 2))\n rsd = torch.norm(sine, 1) # Representation Subspace Distance\n bmp = torch.norm(torch.abs(P_s) - torch.abs(P_t), 2) # Base Mismatch Penalization\n return rsd + self.trade_off * bmp"
},
{
"path": "domain_generalization/Transfer-Learning-Library/dalib/adaptation/self_ensemble.py",
"content": "\"\"\"\n@author: Baixu Chen\n@contact: cbx_99_hasta@outlook.com\n\"\"\"\nfrom typing import Optional, Callable\nimport copy\nimport torch\nimport torch.nn as nn\nimport torch.nn.functional as F\n\nfrom common.modules.classifier import Classifier as ClassifierBase\nfrom dalib.translation.cyclegan.util import set_requires_grad\n\n\nclass ConsistencyLoss(nn.Module):\n r\"\"\"\n Consistency loss between output of student model and output of teacher model.\n Given distance measure :math:`D`, student model's output :math:`y`, teacher\n model's output :math:`y_{teacher}`, binary mask :math:`mask`, consistency loss is\n\n .. math::\n D(y, y_{teacher}) * mask\n\n Args:\n distance_measure (callable): Distance measure function.\n reduction (str, optional): Specifies the reduction to apply to the output:\n ``'none'`` | ``'mean'`` | ``'sum'``. ``'none'``: no reduction will be applied,\n ``'mean'``: the sum of the output will be divided by the number of\n elements in the output, ``'sum'``: the output will be summed. Default: ``'mean'``\n\n Inputs:\n - y: predictions from student model\n - y_teacher: predictions from teacher model\n - mask: binary mask\n\n Shape:\n - y, y_teacher: :math:`(N, C)` where C means the number of classes.\n - mask: :math:`(N, )` where N means mini-batch size.\n \"\"\"\n\n def __init__(self, distance_measure: Callable, reduction: Optional[str] = 'mean'):\n super(ConsistencyLoss, self).__init__()\n self.distance_measure = distance_measure\n self.reduction = reduction\n\n def forward(self, y: torch.Tensor, y_teacher: torch.Tensor, mask: torch.Tensor):\n cons_loss = self.distance_measure(y, y_teacher)\n cons_loss = cons_loss * mask\n if self.reduction == 'mean':\n return cons_loss.mean()\n else:\n return cons_loss\n\n\nclass L2ConsistencyLoss(ConsistencyLoss):\n r\"\"\"\n L2 consistency loss. Given student model's output :math:`y`, teacher model's output :math:`y_{teacher}`\n and binary mask :math:`mask`, L2 consistency loss is\n\n .. math::\n \\text{MSELoss}(y, y_{teacher}) * mask\n\n \"\"\"\n\n def __init__(self, reduction: Optional[str] = 'mean'):\n def l2_distance(y: torch.Tensor, y_teacher: torch.Tensor):\n return ((y - y_teacher) ** 2).sum(dim=1)\n\n super(L2ConsistencyLoss, self).__init__(l2_distance, reduction)\n\n\nclass ClassBalanceLoss(nn.Module):\n r\"\"\"\n Class balance loss that penalises the network for making predictions that exhibit large class imbalance.\n Given predictions :math:`y` with dimension :math:`(N, C)`, we first calculate mean across mini-batch dimension,\n resulting in mini-batch mean per-class probability :math:`y_{mean}` with dimension :math:`(C, )`\n\n .. math::\n y_{mean}^j = \\frac{1}{N} \\sum_{i=1}^N y_i^j\n\n Then we calculate binary cross entropy loss between :math:`y_{mean}` and uniform probability vector :math:`u` with\n the same dimension where :math:`u^j` = :math:`\\frac{1}{C}`\n\n .. math::\n loss = \\text{BCELoss}(y_{mean}, u)\n\n Args:\n num_classes (int): Number of classes\n\n Inputs:\n - y (tensor): predictions from classifier\n\n Shape:\n - y: :math:`(N, C)` where C means the number of classes.\n \"\"\"\n\n def __init__(self, num_classes):\n super(ClassBalanceLoss, self).__init__()\n self.uniform_distribution = torch.ones(num_classes) / num_classes\n\n def forward(self, y: torch.Tensor):\n return F.binary_cross_entropy(y.mean(dim=0), self.uniform_distribution.to(y.device))\n\n\nclass EmaTeacher(object):\n r\"\"\"\n Exponential moving average model used in `Self-ensembling for Visual Domain Adaptation (ICLR 2018) `_\n\n We denote :math:`\\theta_t'` as the parameters of teacher model at training step t, :math:`\\theta_t` as the\n parameters of student model at training step t, :math:`\\alpha` as decay rate. Then we update teacher model in an\n exponential moving average manner as follows\n\n .. math::\n \\theta_t'=\\alpha \\theta_{t-1}' + (1-\\alpha)\\theta_t\n\n Args:\n model (torch.nn.Module): student model\n alpha (float): decay rate for EMA.\n\n Inputs:\n x (tensor): input data fed to teacher model\n\n Examples::\n\n >>> classifier = ImageClassifier(backbone, num_classes=31, bottleneck_dim=256).to(device)\n >>> # initialize teacher model\n >>> teacher = EmaTeacher(classifier, 0.9)\n >>> num_iterations = 1000\n >>> for _ in range(num_iterations):\n >>> # x denotes input of one mini-batch\n >>> # you can get teacher model's output by teacher(x)\n >>> y_teacher = teacher(x)\n >>> # when you want to update teacher, you should call teacher.update()\n >>> teacher.update()\n \"\"\"\n\n def __init__(self, model, alpha):\n self.model = model\n self.alpha = alpha\n self.teacher = copy.deepcopy(model)\n set_requires_grad(self.teacher, False)\n\n def set_alpha(self, alpha: float):\n assert alpha >= 0\n self.alpha = alpha\n\n def update(self):\n for teacher_param, param in zip(self.teacher.parameters(), self.model.parameters()):\n teacher_param.data = self.alpha * teacher_param + (1 - self.alpha) * param\n\n def __call__(self, x: torch.Tensor):\n return self.teacher(x)\n\n def train(self, mode: Optional[bool] = True):\n self.teacher.train(mode)\n\n def eval(self):\n self.train(False)\n\n def state_dict(self):\n return self.teacher.state_dict()\n\n def load_state_dict(self, state_dict):\n self.teacher.load_state_dict(state_dict)\n\n @property\n def module(self):\n return self.teacher.module\n\n\nclass ImageClassifier(ClassifierBase):\n def __init__(self, backbone: nn.Module, num_classes: int, bottleneck_dim: Optional[int] = 256, **kwargs):\n bottleneck = nn.Sequential(\n # nn.AdaptiveAvgPool2d(output_size=(1, 1)),\n # nn.Flatten(),\n nn.Linear(backbone.out_features, bottleneck_dim),\n nn.BatchNorm1d(bottleneck_dim),\n nn.ReLU()\n )\n super(ImageClassifier, self).__init__(backbone, num_classes, bottleneck, bottleneck_dim, **kwargs)\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/dalib/modules/__init__.py",
"content": "from .grl import *\nfrom .domain_discriminator import *\nfrom .kernels import *\nfrom .entropy import *\n\n__all__ = ['grl', 'kernels', 'domain_discriminator', 'entropy']\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/dalib/modules/domain_discriminator.py",
"content": "\"\"\"\n@author: Junguang Jiang\n@contact: JiangJunguang1123@outlook.com\n\"\"\"\nfrom typing import List, Dict\nimport torch.nn as nn\n\n__all__ = ['DomainDiscriminator']\n\n\nclass DomainDiscriminator(nn.Sequential):\n r\"\"\"Domain discriminator model from\n `Domain-Adversarial Training of Neural Networks (ICML 2015) `_\n\n Distinguish whether the input features come from the source domain or the target domain.\n The source domain label is 1 and the target domain label is 0.\n\n Args:\n in_feature (int): dimension of the input feature\n hidden_size (int): dimension of the hidden features\n batch_norm (bool): whether use :class:`~torch.nn.BatchNorm1d`.\n Use :class:`~torch.nn.Dropout` if ``batch_norm`` is False. Default: True.\n\n Shape:\n - Inputs: (minibatch, `in_feature`)\n - Outputs: :math:`(minibatch, 1)`\n \"\"\"\n\n def __init__(self, in_feature: int, hidden_size: int, batch_norm=True):\n if batch_norm:\n super(DomainDiscriminator, self).__init__(\n nn.Linear(in_feature, hidden_size),\n nn.BatchNorm1d(hidden_size),\n nn.ReLU(),\n nn.Linear(hidden_size, hidden_size),\n nn.BatchNorm1d(hidden_size),\n nn.ReLU(),\n nn.Linear(hidden_size, 1),\n nn.Sigmoid()\n )\n else:\n super(DomainDiscriminator, self).__init__(\n nn.Linear(in_feature, hidden_size),\n nn.ReLU(inplace=True),\n nn.Dropout(0.5),\n nn.Linear(hidden_size, hidden_size),\n nn.ReLU(inplace=True),\n nn.Dropout(0.5),\n nn.Linear(hidden_size, 1),\n nn.Sigmoid()\n )\n\n def get_parameters(self) -> List[Dict]:\n return [{\"params\": self.parameters(), \"lr\": 1.}]\n\n\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/dalib/modules/entropy.py",
"content": "\"\"\"\n@author: Junguang Jiang\n@contact: JiangJunguang1123@outlook.com\n\"\"\"\nimport torch\n\n\ndef entropy(predictions: torch.Tensor, reduction='none') -> torch.Tensor:\n r\"\"\"Entropy of prediction.\n The definition is:\n\n .. math::\n entropy(p) = - \\sum_{c=1}^C p_c \\log p_c\n\n where C is number of classes.\n\n Args:\n predictions (tensor): Classifier predictions. Expected to contain raw, normalized scores for each class\n reduction (str, optional): Specifies the reduction to apply to the output:\n ``'none'`` | ``'mean'``. ``'none'``: no reduction will be applied,\n ``'mean'``: the sum of the output will be divided by the number of\n elements in the output. Default: ``'mean'``\n\n Shape:\n - predictions: :math:`(minibatch, C)` where C means the number of classes.\n - Output: :math:`(minibatch, )` by default. If :attr:`reduction` is ``'mean'``, then scalar.\n \"\"\"\n epsilon = 1e-5\n H = -predictions * torch.log(predictions + epsilon)\n H = H.sum(dim=1)\n if reduction == 'mean':\n return H.mean()\n else:\n return H\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/dalib/modules/gl.py",
"content": "\"\"\"\n@author: Junguang Jiang\n@contact: JiangJunguang1123@outlook.com\n\"\"\"\nfrom typing import Optional, Any, Tuple\nimport numpy as np\nimport torch.nn as nn\nfrom torch.autograd import Function\nimport torch\n\n\nclass GradientFunction(Function):\n\n @staticmethod\n def forward(ctx: Any, input: torch.Tensor, coeff: Optional[float] = 1.) -> torch.Tensor:\n ctx.coeff = coeff\n output = input * 1.0\n return output\n\n @staticmethod\n def backward(ctx: Any, grad_output: torch.Tensor) -> Tuple[torch.Tensor, Any]:\n return grad_output * ctx.coeff, None\n\n\nclass WarmStartGradientLayer(nn.Module):\n \"\"\"Warm Start Gradient Layer :math:`\\mathcal{R}(x)` with warm start\n\n The forward and backward behaviours are:\n\n .. math::\n \\mathcal{R}(x) = x,\n\n \\dfrac{ d\\mathcal{R}} {dx} = \\lambda I.\n\n :math:`\\lambda` is initiated at :math:`lo` and is gradually changed to :math:`hi` using the following schedule:\n\n .. math::\n \\lambda = \\dfrac{2(hi-lo)}{1+\\exp(- α \\dfrac{i}{N})} - (hi-lo) + lo\n\n where :math:`i` is the iteration step.\n\n Parameters:\n - **alpha** (float, optional): :math:`α`. Default: 1.0\n - **lo** (float, optional): Initial value of :math:`\\lambda`. Default: 0.0\n - **hi** (float, optional): Final value of :math:`\\lambda`. Default: 1.0\n - **max_iters** (int, optional): :math:`N`. Default: 1000\n - **auto_step** (bool, optional): If True, increase :math:`i` each time `forward` is called.\n Otherwise use function `step` to increase :math:`i`. Default: False\n \"\"\"\n\n def __init__(self, alpha: Optional[float] = 1.0, lo: Optional[float] = 0.0, hi: Optional[float] = 1.,\n max_iters: Optional[int] = 1000., auto_step: Optional[bool] = False):\n super(WarmStartGradientLayer, self).__init__()\n self.alpha = alpha\n self.lo = lo\n self.hi = hi\n self.iter_num = 0\n self.max_iters = max_iters\n self.auto_step = auto_step\n\n def forward(self, input: torch.Tensor) -> torch.Tensor:\n \"\"\"\"\"\"\n coeff = np.float(\n 2.0 * (self.hi - self.lo) / (1.0 + np.exp(-self.alpha * self.iter_num / self.max_iters))\n - (self.hi - self.lo) + self.lo\n )\n if self.auto_step:\n self.step()\n return GradientFunction.apply(input, coeff)\n\n def step(self):\n \"\"\"Increase iteration number :math:`i` by 1\"\"\"\n self.iter_num += 1\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/dalib/modules/grl.py",
"content": "\"\"\"\n@author: Junguang Jiang\n@contact: JiangJunguang1123@outlook.com\n\"\"\"\nfrom typing import Optional, Any, Tuple\nimport numpy as np\nimport torch.nn as nn\nfrom torch.autograd import Function\nimport torch\n\n\nclass GradientReverseFunction(Function):\n\n @staticmethod\n def forward(ctx: Any, input: torch.Tensor, coeff: Optional[float] = 1.) -> torch.Tensor:\n ctx.coeff = coeff\n output = input * 1.0\n return output\n\n @staticmethod\n def backward(ctx: Any, grad_output: torch.Tensor) -> Tuple[torch.Tensor, Any]:\n return grad_output.neg() * ctx.coeff, None\n\n\nclass GradientReverseLayer(nn.Module):\n def __init__(self):\n super(GradientReverseLayer, self).__init__()\n\n def forward(self, *input):\n return GradientReverseFunction.apply(*input)\n\n\nclass WarmStartGradientReverseLayer(nn.Module):\n \"\"\"Gradient Reverse Layer :math:`\\mathcal{R}(x)` with warm start\n\n The forward and backward behaviours are:\n\n .. math::\n \\mathcal{R}(x) = x,\n\n \\dfrac{ d\\mathcal{R}} {dx} = - \\lambda I.\n\n :math:`\\lambda` is initiated at :math:`lo` and is gradually changed to :math:`hi` using the following schedule:\n\n .. math::\n \\lambda = \\dfrac{2(hi-lo)}{1+\\exp(- α \\dfrac{i}{N})} - (hi-lo) + lo\n\n where :math:`i` is the iteration step.\n\n Args:\n alpha (float, optional): :math:`α`. Default: 1.0\n lo (float, optional): Initial value of :math:`\\lambda`. Default: 0.0\n hi (float, optional): Final value of :math:`\\lambda`. Default: 1.0\n max_iters (int, optional): :math:`N`. Default: 1000\n auto_step (bool, optional): If True, increase :math:`i` each time `forward` is called.\n Otherwise use function `step` to increase :math:`i`. Default: False\n \"\"\"\n\n def __init__(self, alpha: Optional[float] = 1.0, lo: Optional[float] = 0.0, hi: Optional[float] = 1.,\n max_iters: Optional[int] = 1000., auto_step: Optional[bool] = False):\n super(WarmStartGradientReverseLayer, self).__init__()\n self.alpha = alpha\n self.lo = lo\n self.hi = hi\n self.iter_num = 0\n self.max_iters = max_iters\n self.auto_step = auto_step\n\n def forward(self, input: torch.Tensor) -> torch.Tensor:\n \"\"\"\"\"\"\n coeff = np.float(\n 2.0 * (self.hi - self.lo) / (1.0 + np.exp(-self.alpha * self.iter_num / self.max_iters))\n - (self.hi - self.lo) + self.lo\n )\n if self.auto_step:\n self.step()\n return GradientReverseFunction.apply(input, coeff)\n\n def step(self):\n \"\"\"Increase iteration number :math:`i` by 1\"\"\"\n self.iter_num += 1\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/dalib/modules/kernels.py",
"content": "\"\"\"\n@author: Junguang Jiang\n@contact: JiangJunguang1123@outlook.com\n\"\"\"\nfrom typing import Optional\nimport torch\nimport torch.nn as nn\n\n\n__all__ = ['GaussianKernel']\n\n\nclass GaussianKernel(nn.Module):\n r\"\"\"Gaussian Kernel Matrix\n\n Gaussian Kernel k is defined by\n\n .. math::\n k(x_1, x_2) = \\exp \\left( - \\dfrac{\\| x_1 - x_2 \\|^2}{2\\sigma^2} \\right)\n\n where :math:`x_1, x_2 \\in R^d` are 1-d tensors.\n\n Gaussian Kernel Matrix K is defined on input group :math:`X=(x_1, x_2, ..., x_m),`\n\n .. math::\n K(X)_{i,j} = k(x_i, x_j)\n\n Also by default, during training this layer keeps running estimates of the\n mean of L2 distances, which are then used to set hyperparameter :math:`\\sigma`.\n Mathematically, the estimation is :math:`\\sigma^2 = \\dfrac{\\alpha}{n^2}\\sum_{i,j} \\| x_i - x_j \\|^2`.\n If :attr:`track_running_stats` is set to ``False``, this layer then does not\n keep running estimates, and use a fixed :math:`\\sigma` instead.\n\n Args:\n sigma (float, optional): bandwidth :math:`\\sigma`. Default: None\n track_running_stats (bool, optional): If ``True``, this module tracks the running mean of :math:`\\sigma^2`.\n Otherwise, it won't track such statistics and always uses fix :math:`\\sigma^2`. Default: ``True``\n alpha (float, optional): :math:`\\alpha` which decides the magnitude of :math:`\\sigma^2` when track_running_stats is set to ``True``\n\n Inputs:\n - X (tensor): input group :math:`X`\n\n Shape:\n - Inputs: :math:`(minibatch, F)` where F means the dimension of input features.\n - Outputs: :math:`(minibatch, minibatch)`\n \"\"\"\n\n def __init__(self, sigma: Optional[float] = None, track_running_stats: Optional[bool] = True,\n alpha: Optional[float] = 1.):\n super(GaussianKernel, self).__init__()\n assert track_running_stats or sigma is not None\n self.sigma_square = torch.tensor(sigma * sigma) if sigma is not None else None\n self.track_running_stats = track_running_stats\n self.alpha = alpha\n\n def forward(self, X: torch.Tensor) -> torch.Tensor:\n l2_distance_square = ((X.unsqueeze(0) - X.unsqueeze(1)) ** 2).sum(2)\n\n if self.track_running_stats:\n self.sigma_square = self.alpha * torch.mean(l2_distance_square.detach())\n\n return torch.exp(-l2_distance_square / (2 * self.sigma_square))"
},
{
"path": "domain_generalization/Transfer-Learning-Library/dalib/translation/__init__.py",
"content": "from . import fourier_transform\n\n__all__ = ['fourier_transform']"
},
{
"path": "domain_generalization/Transfer-Learning-Library/dalib/translation/cycada.py",
"content": "\"\"\"\n@author: Junguang Jiang\n@contact: JiangJunguang1123@outlook.com\n\"\"\"\nimport torch.nn as nn\nfrom torch import Tensor\n\n\nclass SemanticConsistency(nn.Module):\n \"\"\"\n Semantic consistency loss is introduced by\n `CyCADA: Cycle-Consistent Adversarial Domain Adaptation (ICML 2018) `_\n\n This helps to prevent label flipping during image translation.\n\n Args:\n ignore_index (tuple, optional): Specifies target values that are ignored\n and do not contribute to the input gradient. When :attr:`size_average` is\n ``True``, the loss is averaged over non-ignored targets. Default: ().\n reduction (string, optional): Specifies the reduction to apply to the output:\n ``'none'`` | ``'mean'`` | ``'sum'``. ``'none'``: no reduction will\n be applied, ``'mean'``: the weighted mean of the output is taken,\n ``'sum'``: the output will be summed. Note: :attr:`size_average`\n and :attr:`reduce` are in the process of being deprecated, and in\n the meantime, specifying either of those two args will override\n :attr:`reduction`. Default: ``'mean'``\n\n Shape:\n - Input: :math:`(N, C)` where `C = number of classes`, or\n :math:`(N, C, d_1, d_2, ..., d_K)` with :math:`K \\geq 1`\n in the case of `K`-dimensional loss.\n - Target: :math:`(N)` where each value is :math:`0 \\leq \\text{targets}[i] \\leq C-1`, or\n :math:`(N, d_1, d_2, ..., d_K)` with :math:`K \\geq 1` in the case of\n K-dimensional loss.\n - Output: scalar.\n If :attr:`reduction` is ``'none'``, then the same size as the target:\n :math:`(N)`, or\n :math:`(N, d_1, d_2, ..., d_K)` with :math:`K \\geq 1` in the case\n of K-dimensional loss.\n\n Examples::\n\n >>> loss = SemanticConsistency()\n >>> input = torch.randn(3, 5, requires_grad=True)\n >>> target = torch.empty(3, dtype=torch.long).random_(5)\n >>> output = loss(input, target)\n >>> output.backward()\n \"\"\"\n def __init__(self, ignore_index=(), reduction='mean'):\n super(SemanticConsistency, self).__init__()\n self.ignore_index = ignore_index\n self.loss = nn.CrossEntropyLoss(ignore_index=-1, reduction=reduction)\n\n def forward(self, input: Tensor, target: Tensor) -> Tensor:\n for class_idx in self.ignore_index:\n target[target == class_idx] = -1\n return self.loss(input, target)\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/dalib/translation/cyclegan/__init__.py",
"content": "from . import discriminator\nfrom . import generator\nfrom . import loss\nfrom . import transform\n\nfrom .discriminator import *\nfrom .generator import *\nfrom .loss import *\nfrom .transform import *\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/dalib/translation/cyclegan/discriminator.py",
"content": "\"\"\"\nModified from https://github.com/junyanz/pytorch-CycleGAN-and-pix2pix\n@author: Junguang Jiang\n@contact: JiangJunguang1123@outlook.com\n\"\"\"\nimport torch.nn as nn\nfrom torch.nn import init\nimport functools\nfrom .util import get_norm_layer, init_weights\n\n\nclass NLayerDiscriminator(nn.Module):\n \"\"\"Construct a PatchGAN discriminator\n\n Args:\n input_nc (int): the number of channels in input images.\n ndf (int): the number of filters in the last conv layer. Default: 64\n n_layers (int): the number of conv layers in the discriminator. Default: 3\n norm_layer (torch.nn.Module): normalization layer. Default: :class:`nn.BatchNorm2d`\n \"\"\"\n\n def __init__(self, input_nc, ndf=64, n_layers=3, norm_layer=nn.BatchNorm2d):\n super(NLayerDiscriminator, self).__init__()\n if type(norm_layer) == functools.partial: # no need to use bias as BatchNorm2d has affine parameters\n use_bias = norm_layer.func == nn.InstanceNorm2d\n else:\n use_bias = norm_layer == nn.InstanceNorm2d\n\n kw = 4\n padw = 1\n sequence = [nn.Conv2d(input_nc, ndf, kernel_size=kw, stride=2, padding=padw), nn.LeakyReLU(0.2, True)]\n nf_mult = 1\n nf_mult_prev = 1\n for n in range(1, n_layers): # gradually increase the number of filters\n nf_mult_prev = nf_mult\n nf_mult = min(2 ** n, 8)\n sequence += [\n nn.Conv2d(ndf * nf_mult_prev, ndf * nf_mult, kernel_size=kw, stride=2, padding=padw, bias=use_bias),\n norm_layer(ndf * nf_mult),\n nn.LeakyReLU(0.2, True)\n ]\n\n nf_mult_prev = nf_mult\n nf_mult = min(2 ** n_layers, 8)\n sequence += [\n nn.Conv2d(ndf * nf_mult_prev, ndf * nf_mult, kernel_size=kw, stride=1, padding=padw, bias=use_bias),\n norm_layer(ndf * nf_mult),\n nn.LeakyReLU(0.2, True)\n ]\n\n sequence += [nn.Conv2d(ndf * nf_mult, 1, kernel_size=kw, stride=1, padding=padw)] # output 1 channel prediction map\n self.model = nn.Sequential(*sequence)\n\n def forward(self, input):\n return self.model(input)\n\n\nclass PixelDiscriminator(nn.Module):\n \"\"\"Construct a 1x1 PatchGAN discriminator (pixelGAN)\n\n Args:\n input_nc (int): the number of channels in input images.\n ndf (int): the number of filters in the last conv layer. Default: 64\n norm_layer (torch.nn.Module): normalization layer. Default: :class:`nn.BatchNorm2d`\n \"\"\"\n\n def __init__(self, input_nc, ndf=64, norm_layer=nn.BatchNorm2d):\n super(PixelDiscriminator, self).__init__()\n if type(norm_layer) == functools.partial: # no need to use bias as BatchNorm2d has affine parameters\n use_bias = norm_layer.func == nn.InstanceNorm2d\n else:\n use_bias = norm_layer == nn.InstanceNorm2d\n\n self.net = [\n nn.Conv2d(input_nc, ndf, kernel_size=1, stride=1, padding=0),\n nn.LeakyReLU(0.2, True),\n nn.Conv2d(ndf, ndf * 2, kernel_size=1, stride=1, padding=0, bias=use_bias),\n norm_layer(ndf * 2),\n nn.LeakyReLU(0.2, True),\n nn.Conv2d(ndf * 2, 1, kernel_size=1, stride=1, padding=0, bias=use_bias)]\n\n self.net = nn.Sequential(*self.net)\n\n def forward(self, input):\n return self.net(input)\n\n\ndef patch(ndf, input_nc=3, norm='batch', n_layers=3, init_type='normal', init_gain=0.02):\n \"\"\"\n PatchGAN classifier described in the original pix2pix paper.\n It can classify whether 70×70 overlapping patches are real or fake.\n Such a patch-level discriminator architecture has fewer parameters\n than a full-image discriminator and can work on arbitrarily-sized images\n in a fully convolutional fashion.\n\n Args:\n ndf (int): the number of filters in the first conv layer\n input_nc (int): the number of channels in input images. Default: 3\n norm (str): the type of normalization layers used in the network. Default: 'batch'\n n_layers (int): the number of conv layers in the discriminator. Default: 3\n init_type (str): the name of the initialization method. Choices includes: ``normal`` |\n ``xavier`` | ``kaiming`` | ``orthogonal``. Default: 'normal'\n init_gain (float): scaling factor for normal, xavier and orthogonal. Default: 0.02\n \"\"\"\n norm_layer = get_norm_layer(norm_type=norm)\n net = NLayerDiscriminator(input_nc, ndf, n_layers=n_layers, norm_layer=norm_layer)\n init_weights(net, init_type, init_gain=init_gain)\n return net\n\n\ndef pixel(ndf, input_nc=3, norm='batch', init_type='normal', init_gain=0.02):\n \"\"\"\n 1x1 PixelGAN discriminator can classify whether a pixel is real or not.\n It encourages greater color diversity but has no effect on spatial statistics.\n\n Args:\n ndf (int): the number of filters in the first conv layer\n input_nc (int): the number of channels in input images. Default: 3\n norm (str): the type of normalization layers used in the network. Default: 'batch'\n init_type (str): the name of the initialization method. Choices includes: ``normal`` |\n ``xavier`` | ``kaiming`` | ``orthogonal``. Default: 'normal'\n init_gain (float): scaling factor for normal, xavier and orthogonal. Default: 0.02\n \"\"\"\n norm_layer = get_norm_layer(norm_type=norm)\n net = PixelDiscriminator(input_nc, ndf, norm_layer=norm_layer)\n init_weights(net, init_type, init_gain=init_gain)\n return net"
},
{
"path": "domain_generalization/Transfer-Learning-Library/dalib/translation/cyclegan/generator.py",
"content": "\"\"\"\nModified from https://github.com/junyanz/pytorch-CycleGAN-and-pix2pix\n@author: Junguang Jiang\n@contact: JiangJunguang1123@outlook.com\n\"\"\"\nimport torch\nimport torch.nn as nn\nimport functools\nfrom .util import get_norm_layer, init_weights\n\n\nclass ResnetBlock(nn.Module):\n \"\"\"Define a Resnet block\"\"\"\n\n def __init__(self, dim, padding_type, norm_layer, use_dropout, use_bias):\n \"\"\"Initialize the Resnet block\n\n A resnet block is a conv block with skip connections\n We construct a conv block with build_conv_block function,\n and implement skip connections in function.\n Original Resnet paper: https://arxiv.org/pdf/1512.03385.pdf\n \"\"\"\n super(ResnetBlock, self).__init__()\n self.conv_block = self.build_conv_block(dim, padding_type, norm_layer, use_dropout, use_bias)\n\n def build_conv_block(self, dim, padding_type, norm_layer, use_dropout, use_bias):\n \"\"\"Construct a convolutional block.\n\n Args:\n dim (int): the number of channels in the conv layer.\n padding_type (str): the name of padding layer: reflect | replicate | zero\n norm_layer (torch.nn.Module): normalization layer\n use_dropout (bool): if use dropout layers.\n use_bias (bool): if the conv layer uses bias or not\n\n Returns a conv block (with a conv layer, a normalization layer, and a non-linearity layer (ReLU))\n \"\"\"\n conv_block = []\n p = 0\n if padding_type == 'reflect':\n conv_block += [nn.ReflectionPad2d(1)]\n elif padding_type == 'replicate':\n conv_block += [nn.ReplicationPad2d(1)]\n elif padding_type == 'zero':\n p = 1\n else:\n raise NotImplementedError('padding [%s] is not implemented' % padding_type)\n\n conv_block += [nn.Conv2d(dim, dim, kernel_size=3, padding=p, bias=use_bias), norm_layer(dim), nn.ReLU(True)]\n if use_dropout:\n conv_block += [nn.Dropout(0.5)]\n\n p = 0\n if padding_type == 'reflect':\n conv_block += [nn.ReflectionPad2d(1)]\n elif padding_type == 'replicate':\n conv_block += [nn.ReplicationPad2d(1)]\n elif padding_type == 'zero':\n p = 1\n else:\n raise NotImplementedError('padding [%s] is not implemented' % padding_type)\n conv_block += [nn.Conv2d(dim, dim, kernel_size=3, padding=p, bias=use_bias), norm_layer(dim)]\n\n return nn.Sequential(*conv_block)\n\n def forward(self, x):\n \"\"\"Forward function (with skip connections)\"\"\"\n out = x + self.conv_block(x) # add skip connections\n return out\n\n\nclass ResnetGenerator(nn.Module):\n \"\"\"Resnet-based generator that consists of Resnet blocks between a few downsampling/upsampling operations.\n\n We adapt Torch code and idea from Justin Johnson's neural style transfer project(https://github.com/jcjohnson/fast-neural-style)\n\n \"\"\"\n\n def __init__(self, input_nc, output_nc, ngf=64, norm_layer=nn.BatchNorm2d, use_dropout=False, n_blocks=6, padding_type='reflect'):\n \"\"\"Construct a Resnet-based generator\n\n Args:\n input_nc (int): the number of channels in input images\n output_nc (int): the number of channels in output images\n ngf (int): the number of filters in the last conv layer\n norm_layer (torch.nn.Module): normalization layer\n use_dropout (bool): if use dropout layers\n n_blocks (int): the number of ResNet blocks\n padding_type (str): the name of padding layer in conv layers: reflect | replicate | zero\n \"\"\"\n assert(n_blocks >= 0)\n super(ResnetGenerator, self).__init__()\n if type(norm_layer) == functools.partial:\n use_bias = norm_layer.func == nn.InstanceNorm2d\n else:\n use_bias = norm_layer == nn.InstanceNorm2d\n\n model = [nn.ReflectionPad2d(3),\n nn.Conv2d(input_nc, ngf, kernel_size=7, padding=0, bias=use_bias),\n norm_layer(ngf),\n nn.ReLU(True)]\n\n n_downsampling = 2\n for i in range(n_downsampling): # add downsampling layers\n mult = 2 ** i\n model += [nn.Conv2d(ngf * mult, ngf * mult * 2, kernel_size=3, stride=2, padding=1, bias=use_bias),\n norm_layer(ngf * mult * 2),\n nn.ReLU(True)]\n\n mult = 2 ** n_downsampling\n for i in range(n_blocks): # add ResNet blocks\n\n model += [ResnetBlock(ngf * mult, padding_type=padding_type, norm_layer=norm_layer, use_dropout=use_dropout, use_bias=use_bias)]\n\n for i in range(n_downsampling): # add upsampling layers\n mult = 2 ** (n_downsampling - i)\n model += [nn.ConvTranspose2d(ngf * mult, int(ngf * mult / 2),\n kernel_size=3, stride=2,\n padding=1, output_padding=1,\n bias=use_bias),\n norm_layer(int(ngf * mult / 2)),\n nn.ReLU(True)]\n model += [nn.ReflectionPad2d(3)]\n model += [nn.Conv2d(ngf, output_nc, kernel_size=7, padding=0)]\n model += [nn.Tanh()]\n\n self.model = nn.Sequential(*model)\n\n def forward(self, input):\n \"\"\"Standard forward\"\"\"\n return self.model(input)\n\n\nclass UnetGenerator(nn.Module):\n \"\"\"Create a Unet-based generator\"\"\"\n\n def __init__(self, input_nc, output_nc, num_downs, ngf=64, norm_layer=nn.BatchNorm2d, use_dropout=False):\n \"\"\"Construct a Unet generator\n Args:\n input_nc (int): the number of channels in input images\n output_nc (int): the number of channels in output images\n num_downs (int): the number of downsamplings in UNet. For example, # if |num_downs| == 7,\n image of size 128x128 will become of size 1x1 # at the bottleneck\n ngf (int): the number of filters in the last conv layer\n norm_layer(torch.nn.Module): normalization layer\n\n We construct the U-Net from the innermost layer to the outermost layer.\n It is a recursive process.\n \"\"\"\n super(UnetGenerator, self).__init__()\n # construct unet structure\n unet_block = UnetSkipConnectionBlock(ngf * 8, ngf * 8, input_nc=None, submodule=None, norm_layer=norm_layer, innermost=True) # add the innermost layer\n for i in range(num_downs - 5): # add intermediate layers with ngf * 8 filters\n unet_block = UnetSkipConnectionBlock(ngf * 8, ngf * 8, input_nc=None, submodule=unet_block, norm_layer=norm_layer, use_dropout=use_dropout)\n # gradually reduce the number of filters from ngf * 8 to ngf\n unet_block = UnetSkipConnectionBlock(ngf * 4, ngf * 8, input_nc=None, submodule=unet_block, norm_layer=norm_layer)\n unet_block = UnetSkipConnectionBlock(ngf * 2, ngf * 4, input_nc=None, submodule=unet_block, norm_layer=norm_layer)\n unet_block = UnetSkipConnectionBlock(ngf, ngf * 2, input_nc=None, submodule=unet_block, norm_layer=norm_layer)\n self.model = UnetSkipConnectionBlock(output_nc, ngf, input_nc=input_nc, submodule=unet_block, outermost=True, norm_layer=norm_layer) # add the outermost layer\n\n def forward(self, input):\n \"\"\"Standard forward\"\"\"\n return self.model(input)\n\n\nclass UnetSkipConnectionBlock(nn.Module):\n \"\"\"Defines the Unet submodule with skip connection.\n X -------------------identity----------------------\n |-- downsampling -- |submodule| -- upsampling --|\n \"\"\"\n\n def __init__(self, outer_nc, inner_nc, input_nc=None,\n submodule=None, outermost=False, innermost=False, norm_layer=nn.BatchNorm2d, use_dropout=False):\n \"\"\"Construct a Unet submodule with skip connections.\n\n Args:\n outer_nc (int): the number of filters in the outer conv layer\n inner_nc (int): the number of filters in the inner conv layer\n input_nc (int): the number of channels in input images/features\n submodule (UnetSkipConnectionBlock): previously defined submodules\n outermost (bool): if this module is the outermost module\n innermost (bool): if this module is the innermost module\n norm_layer (torch.nn.Module): normalization layer\n use_dropout (bool): if use dropout layers.\n \"\"\"\n super(UnetSkipConnectionBlock, self).__init__()\n self.outermost = outermost\n if type(norm_layer) == functools.partial:\n use_bias = norm_layer.func == nn.InstanceNorm2d\n else:\n use_bias = norm_layer == nn.InstanceNorm2d\n if input_nc is None:\n input_nc = outer_nc\n downconv = nn.Conv2d(input_nc, inner_nc, kernel_size=4,\n stride=2, padding=1, bias=use_bias)\n downrelu = nn.LeakyReLU(0.2, True)\n downnorm = norm_layer(inner_nc)\n uprelu = nn.ReLU(True)\n upnorm = norm_layer(outer_nc)\n\n if outermost:\n upconv = nn.ConvTranspose2d(inner_nc * 2, outer_nc,\n kernel_size=4, stride=2,\n padding=1)\n down = [downconv]\n up = [uprelu, upconv, nn.Tanh()]\n model = down + [submodule] + up\n elif innermost:\n upconv = nn.ConvTranspose2d(inner_nc, outer_nc,\n kernel_size=4, stride=2,\n padding=1, bias=use_bias)\n down = [downrelu, downconv]\n up = [uprelu, upconv, upnorm]\n model = down + up\n else:\n upconv = nn.ConvTranspose2d(inner_nc * 2, outer_nc,\n kernel_size=4, stride=2,\n padding=1, bias=use_bias)\n down = [downrelu, downconv, downnorm]\n up = [uprelu, upconv, upnorm]\n\n if use_dropout:\n model = down + [submodule] + up + [nn.Dropout(0.5)]\n else:\n model = down + [submodule] + up\n\n self.model = nn.Sequential(*model)\n\n def forward(self, x):\n if self.outermost:\n return self.model(x)\n else: # add skip connections\n return torch.cat([x, self.model(x)], 1)\n\n\ndef resnet_9(ngf, input_nc=3, output_nc=3, norm='batch', use_dropout=False,\n init_type='normal', init_gain=0.02):\n \"\"\"\n Resnet-based generator with 9 Resnet blocks.\n\n Args:\n ngf (int): the number of filters in the last conv layer\n input_nc (int): the number of channels in input images. Default: 3\n output_nc (int): the number of channels in output images. Default: 3\n norm (str): the type of normalization layers used in the network. Default: 'batch'\n use_dropout (bool): whether use dropout. Default: False\n init_type (str): the name of the initialization method. Choices includes: ``normal`` |\n ``xavier`` | ``kaiming`` | ``orthogonal``. Default: 'normal'\n init_gain (float): scaling factor for normal, xavier and orthogonal. Default: 0.02\n \"\"\"\n norm_layer = get_norm_layer(norm_type=norm)\n net = ResnetGenerator(input_nc, output_nc, ngf, norm_layer=norm_layer, use_dropout=use_dropout, n_blocks=9)\n init_weights(net, init_type, init_gain)\n return net\n\n\ndef resnet_6(ngf, input_nc=3, output_nc=3, norm='batch', use_dropout=False,\n init_type='normal', init_gain=0.02):\n \"\"\"\n Resnet-based generator with 6 Resnet blocks.\n\n Args:\n ngf (int): the number of filters in the last conv layer\n input_nc (int): the number of channels in input images. Default: 3\n output_nc (int): the number of channels in output images. Default: 3\n norm (str): the type of normalization layers used in the network. Default: 'batch'\n use_dropout (bool): whether use dropout. Default: False\n init_type (str): the name of the initialization method. Choices includes: ``normal`` |\n ``xavier`` | ``kaiming`` | ``orthogonal``. Default: 'normal'\n init_gain (float): scaling factor for normal, xavier and orthogonal. Default: 0.02\n \"\"\"\n norm_layer = get_norm_layer(norm_type=norm)\n net = ResnetGenerator(input_nc, output_nc, ngf, norm_layer=norm_layer, use_dropout=use_dropout, n_blocks=6)\n init_weights(net, init_type, init_gain)\n return net\n\n\ndef unet_256(ngf, input_nc=3, output_nc=3, norm='batch', use_dropout=False,\n init_type='normal', init_gain=0.02):\n \"\"\"\n `U-Net `_ generator for 256x256 input images.\n The size of the input image should be a multiple of 256.\n\n Args:\n ngf (int): the number of filters in the last conv layer\n input_nc (int): the number of channels in input images. Default: 3\n output_nc (int): the number of channels in output images. Default: 3\n norm (str): the type of normalization layers used in the network. Default: 'batch'\n use_dropout (bool): whether use dropout. Default: False\n init_type (str): the name of the initialization method. Choices includes: ``normal`` |\n ``xavier`` | ``kaiming`` | ``orthogonal``. Default: 'normal'\n init_gain (float): scaling factor for normal, xavier and orthogonal. Default: 0.02\n\n \"\"\"\n norm_layer = get_norm_layer(norm_type=norm)\n net = UnetGenerator(input_nc, output_nc, 8, ngf, norm_layer=norm_layer, use_dropout=use_dropout)\n init_weights(net, init_type, init_gain)\n return net\n\n\ndef unet_128(ngf, input_nc=3, output_nc=3, norm='batch', use_dropout=False,\n init_type='normal', init_gain=0.02):\n \"\"\"\n `U-Net `_ generator for 128x128 input images.\n The size of the input image should be a multiple of 128.\n\n Args:\n ngf (int): the number of filters in the last conv layer\n input_nc (int): the number of channels in input images. Default: 3\n output_nc (int): the number of channels in output images. Default: 3\n norm (str): the type of normalization layers used in the network. Default: 'batch'\n use_dropout (bool): whether use dropout. Default: False\n init_type (str): the name of the initialization method. Choices includes: ``normal`` |\n ``xavier`` | ``kaiming`` | ``orthogonal``. Default: 'normal'\n init_gain (float): scaling factor for normal, xavier and orthogonal. Default: 0.02\n\n \"\"\"\n norm_layer = get_norm_layer(norm_type=norm)\n net = UnetGenerator(input_nc, output_nc, 7, ngf, norm_layer=norm_layer, use_dropout=use_dropout)\n init_weights(net, init_type, init_gain)\n return net\n\n\ndef unet_32(ngf, input_nc=3, output_nc=3, norm='batch', use_dropout=False,\n init_type='normal', init_gain=0.02):\n \"\"\"\n `U-Net `_ generator for 32x32 input images\n\n Args:\n ngf (int): the number of filters in the last conv layer\n input_nc (int): the number of channels in input images. Default: 3\n output_nc (int): the number of channels in output images. Default: 3\n norm (str): the type of normalization layers used in the network. Default: 'batch'\n use_dropout (bool): whether use dropout. Default: False\n init_type (str): the name of the initialization method. Choices includes: ``normal`` |\n ``xavier`` | ``kaiming`` | ``orthogonal``. Default: 'normal'\n init_gain (float): scaling factor for normal, xavier and orthogonal. Default: 0.02\n\n \"\"\"\n norm_layer = get_norm_layer(norm_type=norm)\n net = UnetGenerator(input_nc, output_nc, 5, ngf, norm_layer=norm_layer, use_dropout=use_dropout)\n init_weights(net, init_type, init_gain)\n return net"
},
{
"path": "domain_generalization/Transfer-Learning-Library/dalib/translation/cyclegan/loss.py",
"content": "\"\"\"\nModified from https://github.com/junyanz/pytorch-CycleGAN-and-pix2pix\n@author: Junguang Jiang\n@contact: JiangJunguang1123@outlook.com\n\"\"\"\nimport torch.nn as nn\nimport torch\n\n\nclass LeastSquaresGenerativeAdversarialLoss(nn.Module):\n \"\"\"\n Loss for `Least Squares Generative Adversarial Network (LSGAN) `_\n\n Args:\n reduction (str, optional): Specifies the reduction to apply to the output:\n ``'none'`` | ``'mean'`` | ``'sum'``. ``'none'``: no reduction will be applied,\n ``'mean'``: the sum of the output will be divided by the number of\n elements in the output, ``'sum'``: the output will be summed. Default: ``'mean'``\n\n Inputs:\n - prediction (tensor): unnormalized discriminator predictions\n - real (bool): if the ground truth label is for real images or fake images. Default: true\n\n .. warning::\n Do not use sigmoid as the last layer of Discriminator.\n\n \"\"\"\n def __init__(self, reduction='mean'):\n super(LeastSquaresGenerativeAdversarialLoss, self).__init__()\n self.mse_loss = nn.MSELoss(reduction=reduction)\n\n def forward(self, prediction, real=True):\n if real:\n label = torch.ones_like(prediction)\n else:\n label = torch.zeros_like(prediction)\n return self.mse_loss(prediction, label)\n\n\nclass VanillaGenerativeAdversarialLoss(nn.Module):\n \"\"\"\n Loss for `Vanilla Generative Adversarial Network `_\n\n Args:\n reduction (str, optional): Specifies the reduction to apply to the output:\n ``'none'`` | ``'mean'`` | ``'sum'``. ``'none'``: no reduction will be applied,\n ``'mean'``: the sum of the output will be divided by the number of\n elements in the output, ``'sum'``: the output will be summed. Default: ``'mean'``\n\n Inputs:\n - prediction (tensor): unnormalized discriminator predictions\n - real (bool): if the ground truth label is for real images or fake images. Default: true\n\n .. warning::\n Do not use sigmoid as the last layer of Discriminator.\n\n \"\"\"\n def __init__(self, reduction='mean'):\n super(VanillaGenerativeAdversarialLoss, self).__init__()\n self.bce_loss = nn.BCEWithLogitsLoss(reduction=reduction)\n\n def forward(self, prediction, real=True):\n if real:\n label = torch.ones_like(prediction)\n else:\n label = torch.zeros_like(prediction)\n return self.bce_loss(prediction, label)\n\n\nclass WassersteinGenerativeAdversarialLoss(nn.Module):\n \"\"\"\n Loss for `Wasserstein Generative Adversarial Network `_\n\n Args:\n reduction (str, optional): Specifies the reduction to apply to the output:\n ``'none'`` | ``'mean'`` | ``'sum'``. ``'none'``: no reduction will be applied,\n ``'mean'``: the sum of the output will be divided by the number of\n elements in the output, ``'sum'``: the output will be summed. Default: ``'mean'``\n\n Inputs:\n - prediction (tensor): unnormalized discriminator predictions\n - real (bool): if the ground truth label is for real images or fake images. Default: true\n\n .. warning::\n Do not use sigmoid as the last layer of Discriminator.\n\n \"\"\"\n def __init__(self, reduction='mean'):\n super(WassersteinGenerativeAdversarialLoss, self).__init__()\n self.mse_loss = nn.MSELoss(reduction=reduction)\n\n def forward(self, prediction, real=True):\n if real:\n return -prediction.mean()\n else:\n return prediction.mean()"
},
{
"path": "domain_generalization/Transfer-Learning-Library/dalib/translation/cyclegan/transform.py",
"content": "\"\"\"\n@author: Junguang Jiang\n@contact: JiangJunguang1123@outlook.com\n\"\"\"\nimport torch\nimport torch.nn as nn\nimport torchvision.transforms as T\n\nfrom common.vision.transforms import Denormalize\n\n\nclass Translation(nn.Module):\n \"\"\"\n Image Translation Transform Module\n\n Args:\n generator (torch.nn.Module): An image generator, e.g. :meth:`~dalib.translation.cyclegan.resnet_9_generator`\n device (torch.device): device to put the generator. Default: 'cpu'\n mean (tuple): the normalized mean for image\n std (tuple): the normalized std for image\n Input:\n - image (PIL.Image): raw image in shape H x W x C\n\n Output:\n raw image in shape H x W x 3\n\n \"\"\"\n def __init__(self, generator, device=torch.device(\"cpu\"), mean=(0.5, 0.5, 0.5), std=(0.5, 0.5, 0.5)):\n super(Translation, self).__init__()\n self.generator = generator.to(device)\n self.device = device\n self.pre_process = T.Compose([\n T.ToTensor(),\n T.Normalize(mean, std)\n ])\n self.post_process = T.Compose([\n Denormalize(mean, std),\n T.ToPILImage()\n ])\n\n def forward(self, image):\n image = self.pre_process(image.copy()) # C x H x W\n image = image.to(self.device)\n generated_image = self.generator(image.unsqueeze(dim=0)).squeeze(dim=0).cpu()\n return self.post_process(generated_image)\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/dalib/translation/cyclegan/util.py",
"content": "\"\"\"\n@author: Junguang Jiang\n@contact: JiangJunguang1123@outlook.com\n\"\"\"\nimport torch.nn as nn\nimport functools\nimport random\nimport torch\nfrom torch.nn import init\n\n\nclass Identity(nn.Module):\n def forward(self, x):\n return x\n\n\ndef get_norm_layer(norm_type='instance'):\n \"\"\"Return a normalization layer\n\n Parameters:\n norm_type (str) -- the name of the normalization layer: batch | instance | none\n\n For BatchNorm, we use learnable affine parameters and track running statistics (mean/stddev).\n For InstanceNorm, we do not use learnable affine parameters. We do not track running statistics.\n \"\"\"\n if norm_type == 'batch':\n norm_layer = functools.partial(nn.BatchNorm2d, affine=True, track_running_stats=True)\n elif norm_type == 'instance':\n norm_layer = functools.partial(nn.InstanceNorm2d, affine=False, track_running_stats=False)\n elif norm_type == 'none':\n def norm_layer(x): return Identity()\n else:\n raise NotImplementedError('normalization layer [%s] is not found' % norm_type)\n return norm_layer\n\n\ndef init_weights(net, init_type='normal', init_gain=0.02):\n \"\"\"Initialize network weights.\n\n Args:\n net (torch.nn.Module): network to be initialized\n init_type (str): the name of an initialization method. Choices includes: ``normal`` |\n ``xavier`` | ``kaiming`` | ``orthogonal``\n init_gain (float): scaling factor for normal, xavier and orthogonal.\n\n 'normal' is used in the original CycleGAN paper. But xavier and kaiming might\n work better for some applications.\n \"\"\"\n def init_func(m): # define the initialization function\n classname = m.__class__.__name__\n if hasattr(m, 'weight') and (classname.find('Conv') != -1 or classname.find('Linear') != -1):\n if init_type == 'normal':\n init.normal_(m.weight.data, 0.0, init_gain)\n elif init_type == 'xavier':\n init.xavier_normal_(m.weight.data, gain=init_gain)\n elif init_type == 'kaiming':\n init.kaiming_normal_(m.weight.data, a=0, mode='fan_in')\n elif init_type == 'orthogonal':\n init.orthogonal_(m.weight.data, gain=init_gain)\n else:\n raise NotImplementedError('initialization method [%s] is not implemented' % init_type)\n if hasattr(m, 'bias') and m.bias is not None:\n init.constant_(m.bias.data, 0.0)\n elif classname.find('BatchNorm2d') != -1: # BatchNorm Layer's weight is not a matrix; only normal distribution applies.\n init.normal_(m.weight.data, 1.0, init_gain)\n init.constant_(m.bias.data, 0.0)\n\n print('initialize network with %s' % init_type)\n net.apply(init_func) # apply the initialization function \n\n\nclass ImagePool:\n \"\"\"An image buffer that stores previously generated images.\n\n This buffer enables us to update discriminators using a history of generated images\n rather than the ones produced by the latest generators.\n\n Args:\n pool_size (int): the size of image buffer, if pool_size=0, no buffer will be created\n\n \"\"\"\n\n def __init__(self, pool_size):\n self.pool_size = pool_size\n if self.pool_size > 0: # create an empty pool\n self.num_imgs = 0\n self.images = []\n\n def query(self, images):\n \"\"\"Return an image from the pool.\n\n Args:\n images (torch.Tensor): the latest generated images from the generator\n\n Returns:\n By 50/100, the buffer will return input images.\n By 50/100, the buffer will return images previously stored in the buffer,\n and insert the current images to the buffer.\n\n \"\"\"\n if self.pool_size == 0: # if the buffer size is 0, do nothing\n return images\n return_images = []\n for image in images:\n image = torch.unsqueeze(image.data, 0)\n if self.num_imgs < self.pool_size: # if the buffer is not full; keep inserting current images to the buffer\n self.num_imgs = self.num_imgs + 1\n self.images.append(image)\n return_images.append(image)\n else:\n p = random.uniform(0, 1)\n if p > 0.5: # by 50% chance, the buffer will return a previously stored image, and insert the current image into the buffer\n random_id = random.randint(0, self.pool_size - 1) # randint is inclusive\n tmp = self.images[random_id].clone()\n self.images[random_id] = image\n return_images.append(tmp)\n else: # by another 50% chance, the buffer will return the current image\n return_images.append(image)\n return_images = torch.cat(return_images, 0) # collect all the images and return\n return return_images\n\n\ndef set_requires_grad(net, requires_grad=False):\n \"\"\"\n Set requies_grad=Fasle for all the networks to avoid unnecessary computations\n \"\"\"\n for param in net.parameters():\n param.requires_grad = requires_grad\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/dalib/translation/fourier_transform.py",
"content": "\"\"\"\n@author: Junguang Jiang\n@contact: JiangJunguang1123@outlook.com\n\"\"\"\nimport numpy as np\nimport os\nimport tqdm\nimport random\nfrom PIL import Image\nfrom typing import Optional, Sequence\nimport torch.nn as nn\n\n\ndef low_freq_mutate(amp_src: np.ndarray, amp_trg: np.ndarray, beta: Optional[int] = 1):\n \"\"\"\n Args:\n amp_src (numpy.ndarray): amplitude component of the Fourier transform of source image\n amp_trg (numpy.ndarray): amplitude component of the Fourier transform of target image\n beta (int, optional): the size of the center region to be replace. Default: 1\n\n Returns:\n amplitude component of the Fourier transform of source image\n whose low-frequency component is replaced by that of the target image.\n\n \"\"\"\n # Shift the zero-frequency component to the center of the spectrum.\n a_src = np.fft.fftshift(amp_src, axes=(-2, -1))\n a_trg = np.fft.fftshift(amp_trg, axes=(-2, -1))\n\n # The low-frequency component includes\n # the area where the horizontal and vertical distance from the center does not exceed beta\n _, h, w = a_src.shape\n c_h = np.floor(h / 2.0).astype(int)\n c_w = np.floor(w / 2.0).astype(int)\n\n h1 = c_h - beta\n h2 = c_h + beta + 1\n w1 = c_w - beta\n w2 = c_w + beta + 1\n\n # The low-frequency component of source amplitude is replaced by the target amplitude\n a_src[:, h1:h2, w1:w2] = a_trg[:, h1:h2, w1:w2]\n a_src = np.fft.ifftshift(a_src, axes=(-2, -1))\n return a_src\n\n\nclass FourierTransform(nn.Module):\n \"\"\"\n Fourier Transform is introduced by `FDA: Fourier Domain Adaptation for Semantic Segmentation (CVPR 2020) `_\n\n Fourier Transform replace the low frequency component of the amplitude of the source image to that of the target image.\n Denote with :math:`M_{β}` a mask, whose value is zero except for the center region:\n\n .. math::\n M_{β}(h,w) = \\mathbb{1}_{(h, w)\\in [-β,β, -β, β]}\n\n Given images :math:`x^s` from source domain and :math:`x^t` from target domain, the source image in the target style is\n\n .. math::\n x^{s→t} = \\mathcal{F}^{-1}([ M_{β}\\circ\\mathcal{F}^A(x^t) + (1-M_{β})\\circ\\mathcal{F}^A(x^s), \\mathcal{F}^P(x^s) ])\n\n where :math:`\\mathcal{F}^A`, :math:`\\mathcal{F}^P` are the amplitude and phase component of the Fourier\n Transform :math:`\\mathcal{F}` of an RGB image.\n\n Args:\n image_list (sequence[str]): A sequence of image list from the target domain.\n amplitude_dir (str): Specifies the directory to put the amplitude component of the target image.\n beta (int, optional): :math:`β`. Default: 1.\n rebuild (bool, optional): whether rebuild the amplitude component of the target image in the given directory.\n\n Inputs:\n - image (PIL Image): image from the source domain, :math:`x^t`.\n\n Examples:\n\n >>> from dalib.translation.fourier_transform import FourierTransform\n >>> image_list = [\"target_image_path1\", \"target_image_path2\"]\n >>> amplitude_dir = \"path/to/amplitude_dir\"\n >>> fourier_transform = FourierTransform(image_list, amplitude_dir, beta=1, rebuild=False)\n >>> source_image = np.array((256, 256, 3)) # image form source domain\n >>> source_image_in_target_style = fourier_transform(source_image)\n\n .. note::\n The meaning of :math:`β` is different from that of the origin paper. Experimentally, we found that the size of\n the center region in the frequency space should be constant when the image size increases. Thus we make the size\n of the center region independent of the image size. A recommended value for :math:`β` is 1.\n\n .. note::\n The image structure of the source domain and target domain should be as similar as possible,\n thus for segemntation tasks, FourierTransform should be used before RandomResizeCrop and other transformations.\n\n .. note::\n The image size of the source domain and the target domain need to be the same, thus before FourierTransform,\n you should use Resize to convert the source image to the target image size.\n\n Examples:\n\n >>> from dalib.translation.fourier_transform import FourierTransform\n >>> import common.vision.datasets.segmentation.transforms as T\n >>> from PIL import Image\n >>> target_image_list = [\"target_image_path1\", \"target_image_path2\"]\n >>> amplitude_dir = \"path/to/amplitude_dir\"\n >>> # build a fourier transform that translate source images to the target style\n >>> fourier_transform = T.wrapper(FourierTransform)(target_image_list, amplitude_dir)\n >>> transforms=T.Compose([\n ... # convert source image to the size of the target image before fourier transform\n ... T.Resize((2048, 1024)),\n ... fourier_transform,\n ... T.RandomResizedCrop((1024, 512)),\n ... T.RandomHorizontalFlip(),\n ... ])\n >>> source_image = Image.open(\"path/to/source_image\") # image form source domain\n >>> source_image_in_target_style = transforms(source_image)\n \"\"\"\n # TODO add image examples when beta is different\n def __init__(self, image_list: Sequence[str], amplitude_dir: str,\n beta: Optional[int] = 1, rebuild: Optional[bool] = False):\n super(FourierTransform, self).__init__()\n self.amplitude_dir = amplitude_dir\n if not os.path.exists(amplitude_dir) or rebuild:\n os.makedirs(amplitude_dir, exist_ok=True)\n self.build_amplitude(image_list, amplitude_dir)\n self.beta = beta\n self.length = len(image_list)\n\n @staticmethod\n def build_amplitude(image_list, amplitude_dir):\n # extract amplitudes from target domain\n for i, image_name in enumerate(tqdm.tqdm(image_list)):\n image = Image.open(image_name).convert('RGB')\n image = np.asarray(image, np.float32)\n image = image.transpose((2, 0, 1))\n fft = np.fft.fft2(image, axes=(-2, -1))\n amp = np.abs(fft)\n np.save(os.path.join(amplitude_dir, \"{}.npy\".format(i)), amp)\n\n def forward(self, image):\n # randomly sample a target image and load its amplitude component\n amp_trg = np.load(os.path.join(self.amplitude_dir, \"{}.npy\".format(random.randint(0, self.length-1))))\n\n image = np.asarray(image, np.float32)\n image = image.transpose((2, 0, 1))\n\n # get fft, amplitude on source domain\n fft_src = np.fft.fft2(image, axes=(-2, -1))\n amp_src, pha_src = np.abs(fft_src), np.angle(fft_src)\n # mutate the amplitude part of source with target\n amp_src_ = low_freq_mutate(amp_src, amp_trg, beta=self.beta)\n\n # mutated fft of source\n fft_src_ = amp_src_ * np.exp(1j * pha_src)\n\n # get the mutated image\n src_in_trg = np.fft.ifft2(fft_src_, axes=(-2, -1))\n src_in_trg = np.real(src_in_trg)\n\n src_in_trg = src_in_trg.transpose((1, 2, 0))\n src_in_trg = Image.fromarray(src_in_trg.clip(min=0, max=255).astype('uint8')).convert('RGB')\n\n return src_in_trg\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/dalib/translation/spgan/__init__.py",
"content": "from . import siamese\nfrom . import loss\nfrom .siamese import *\nfrom .loss import *\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/dalib/translation/spgan/loss.py",
"content": "\"\"\"\nModified from https://github.com/Simon4Yan/eSPGAN\n@author: Baixu Chen\n@contact: cbx_99_hasta@outlook.com\n\"\"\"\nimport torch\nimport torch.nn.functional as F\n\n\nclass ContrastiveLoss(torch.nn.Module):\n r\"\"\"Contrastive loss from `Dimensionality Reduction by Learning an Invariant Mapping (CVPR 2006)\n `_.\n\n Given output features :math:`f_1, f_2`, we use :math:`D` to denote the pairwise euclidean distance between them,\n :math:`Y` to denote the ground truth labels, :math:`m` to denote a pre-defined margin, then contrastive loss is\n calculated as\n\n .. math::\n (1 - Y)\\frac{1}{2}D^2 + (Y)\\frac{1}{2}\\{\\text{max}(0, m-D)^2\\}\n\n Args:\n margin (float, optional): margin for contrastive loss. Default: 2.0\n\n Inputs:\n - output1 (tensor): feature representations of the first set of samples (:math:`f_1` here).\n - output2 (tensor): feature representations of the second set of samples (:math:`f_2` here).\n - label (tensor): labels (:math:`Y` here).\n\n Shape:\n - output1, output2: :math:`(minibatch, F)` where F means the dimension of input features.\n - label: :math:`(minibatch, )`\n \"\"\"\n def __init__(self, margin=2.0):\n super(ContrastiveLoss, self).__init__()\n self.margin = margin\n\n def forward(self, output1, output2, label):\n euclidean_distance = F.pairwise_distance(output1, output2)\n loss = torch.mean((1 - label) * torch.pow(euclidean_distance, 2) +\n label * torch.pow(torch.clamp(self.margin - euclidean_distance, min=0.0), 2))\n\n return loss\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/dalib/translation/spgan/siamese.py",
"content": "\"\"\"\nModified from https://github.com/Simon4Yan/eSPGAN\n@author: Baixu Chen\n@contact: cbx_99_hasta@outlook.com\n\"\"\"\nimport torch.nn as nn\nimport torch.nn.functional as F\n\n\nclass ConvBlock(nn.Module):\n \"\"\"Basic block with structure Conv-LeakyReLU->Pool\"\"\"\n def __init__(self, in_dim, out_dim):\n super(ConvBlock, self).__init__()\n self.conv_block = nn.Sequential(\n nn.Conv2d(in_dim, out_dim, kernel_size=4, stride=2, padding=1),\n nn.LeakyReLU(0.2),\n nn.MaxPool2d(kernel_size=2, stride=2)\n )\n\n def forward(self, x):\n return self.conv_block(x)\n\n\nclass SiameseNetwork(nn.Module):\n \"\"\"Siamese network whose input is an image of shape :math:`(3,H,W)` and output is an one-dimensional feature vector.\n\n Args:\n nsf (int): dimension of output feature representation.\n \"\"\"\n def __init__(self, nsf=64):\n super(SiameseNetwork, self).__init__()\n self.conv = nn.Sequential(\n nn.Conv2d(3, nsf, kernel_size=4, stride=2, padding=1),\n nn.LeakyReLU(0.2),\n nn.MaxPool2d(kernel_size=2, stride=2),\n ConvBlock(nsf, nsf * 2),\n ConvBlock(nsf * 2, nsf * 4),\n )\n self.flatten = nn.Flatten()\n self.fc1 = nn.Linear(2048, nsf * 2, bias=False)\n self.leaky_relu = nn.LeakyReLU(0.2)\n self.dropout = nn.Dropout(0.5)\n self.fc2 = nn.Linear(nsf * 2, nsf, bias=False)\n\n def forward(self, x):\n x = self.flatten(self.conv(x))\n x = self.fc1(x)\n x = self.leaky_relu(x)\n x = self.dropout(x)\n x = self.fc2(x)\n x = F.normalize(x)\n return x\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/dglib/__init__.py",
"content": "__all__ = ['generalization']\n\n__version__ = '0.3'"
},
{
"path": "domain_generalization/Transfer-Learning-Library/dglib/generalization/__init__.py",
"content": "from . import mixstyle\nfrom . import irm\nfrom . import groupdro\nfrom . import coral\n\n__all__ = ['mixstyle', 'irm', 'groupdro', 'coral']\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/dglib/generalization/coral.py",
"content": "\"\"\"\n@author: Baixu Chen\n@contact: cbx_99_hasta@outlook.com\n\"\"\"\nimport torch\nimport torch.nn as nn\n\n\nclass CorrelationAlignmentLoss(nn.Module):\n r\"\"\"The `Correlation Alignment Loss` in\n `Deep CORAL: Correlation Alignment for Deep Domain Adaptation (ECCV 2016) `_.\n\n Given source features :math:`f_S` and target features :math:`f_T`, the covariance matrices are given by\n\n .. math::\n C_S = \\frac{1}{n_S-1}(f_S^Tf_S-\\frac{1}{n_S}(\\textbf{1}^Tf_S)^T(\\textbf{1}^Tf_S))\n .. math::\n C_T = \\frac{1}{n_T-1}(f_T^Tf_T-\\frac{1}{n_T}(\\textbf{1}^Tf_T)^T(\\textbf{1}^Tf_T))\n\n where :math:`\\textbf{1}` denotes a column vector with all elements equal to 1, :math:`n_S, n_T` denotes number of\n source and target samples, respectively. We use :math:`d` to denote feature dimension, use\n :math:`{\\Vert\\cdot\\Vert}^2_F` to denote the squared matrix `Frobenius norm`. The correlation alignment loss is\n given by\n\n .. math::\n l_{CORAL} = \\frac{1}{4d^2}\\Vert C_S-C_T \\Vert^2_F\n\n Inputs:\n - f_s (tensor): feature representations on source domain, :math:`f^s`\n - f_t (tensor): feature representations on target domain, :math:`f^t`\n\n Shape:\n - f_s, f_t: :math:`(N, d)` where d means the dimension of input features, :math:`N=n_S=n_T` is mini-batch size.\n - Outputs: scalar.\n \"\"\"\n\n def __init__(self):\n super(CorrelationAlignmentLoss, self).__init__()\n\n def forward(self, f_s: torch.Tensor, f_t: torch.Tensor) -> torch.Tensor:\n mean_s = f_s.mean(0, keepdim=True)\n mean_t = f_t.mean(0, keepdim=True)\n cent_s = f_s - mean_s\n cent_t = f_t - mean_t\n cov_s = torch.mm(cent_s.t(), cent_s) / (len(f_s) - 1)\n cov_t = torch.mm(cent_t.t(), cent_t) / (len(f_t) - 1)\n\n mean_diff = (mean_s - mean_t).pow(2).mean()\n cov_diff = (cov_s - cov_t).pow(2).mean()\n\n return mean_diff + cov_diff\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/dglib/generalization/groupdro.py",
"content": "\"\"\"\nModified from https://github.com/facebookresearch/DomainBed\n@author: Baixu Chen\n@contact: cbx_99_hasta@outlook.com\n\"\"\"\nimport torch\n\n\nclass AutomaticUpdateDomainWeightModule(object):\n r\"\"\"\n Maintaining group weight based on loss history of all domains according\n to `Distributionally Robust Neural Networks for Group Shifts: On the Importance of Regularization for Worst-Case\n Generalization (ICLR 2020) `_.\n\n Suppose we have :math:`N` domains. During each iteration, we first calculate unweighted loss among all\n domains, resulting in :math:`loss\\in R^N`. Then we update domain weight by\n\n .. math::\n w_k = w_k * \\text{exp}(loss_k ^{\\eta}), \\forall k \\in [1, N]\n\n where :math:`\\eta` is the hyper parameter which ensures smoother change of weight.\n As :math:`w \\in R^N` denotes a distribution, we `normalize`\n :math:`w` by its sum. At last, weighted loss is calculated as our objective\n\n .. math::\n objective = \\sum_{k=1}^N w_k * loss_k\n\n Args:\n num_domains (int): The number of source domains.\n eta (float): Hyper parameter eta.\n device (torch.device): The device to run on.\n \"\"\"\n\n def __init__(self, num_domains: int, eta: float, device):\n self.domain_weight = torch.ones(num_domains).to(device) / num_domains\n self.eta = eta\n\n def get_domain_weight(self, sampled_domain_idxes):\n \"\"\"Get domain weight to calculate final objective.\n\n Inputs:\n - sampled_domain_idxes (list): sampled domain indexes in current mini-batch\n\n Shape:\n - sampled_domain_idxes: :math:`(D, )` where D means the number of sampled domains in current mini-batch\n - Outputs: :math:`(D, )`\n \"\"\"\n domain_weight = self.domain_weight[sampled_domain_idxes]\n domain_weight = domain_weight / domain_weight.sum()\n return domain_weight\n\n def update(self, sampled_domain_losses: torch.Tensor, sampled_domain_idxes):\n \"\"\"Update domain weight using loss of current mini-batch.\n\n Inputs:\n - sampled_domain_losses (tensor): loss of among sampled domains in current mini-batch\n - sampled_domain_idxes (list): sampled domain indexes in current mini-batch\n\n Shape:\n - sampled_domain_losses: :math:`(D, )` where D means the number of sampled domains in current mini-batch\n - sampled_domain_idxes: :math:`(D, )`\n \"\"\"\n sampled_domain_losses = sampled_domain_losses.detach()\n\n for loss, idx in zip(sampled_domain_losses, sampled_domain_idxes):\n self.domain_weight[idx] *= (self.eta * loss).exp()\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/dglib/generalization/irm.py",
"content": "\"\"\"\nModified from https://github.com/facebookresearch/DomainBed\n@author: Baixu Chen\n@contact: cbx_99_hasta@outlook.com\n\"\"\"\nimport torch\nimport torch.nn as nn\nimport torch.nn.functional as F\nimport torch.autograd as autograd\n\n\nclass InvariancePenaltyLoss(nn.Module):\n r\"\"\"Invariance Penalty Loss from `Invariant Risk Minimization `_.\n We adopt implementation from `DomainBed `_. Given classifier\n output :math:`y` and ground truth :math:`labels`, we split :math:`y` into two parts :math:`y_1, y_2`, corresponding\n labels are :math:`labels_1, labels_2`. Next we calculate cross entropy loss with respect to a dummy classifier\n :math:`w`, resulting in :math:`grad_1, grad_2` . Invariance penalty is then :math:`grad_1*grad_2`.\n\n Inputs:\n - y: predictions from model\n - labels: ground truth\n\n Shape:\n - y: :math:`(N, C)` where C means the number of classes.\n - labels: :math:`(N, )` where N mean mini-batch size\n \"\"\"\n\n def __init__(self):\n super(InvariancePenaltyLoss, self).__init__()\n self.scale = torch.tensor(1.).requires_grad_()\n\n def forward(self, y: torch.Tensor, labels: torch.Tensor) -> torch.Tensor:\n loss_1 = F.cross_entropy(y[::2] * self.scale, labels[::2])\n loss_2 = F.cross_entropy(y[1::2] * self.scale, labels[1::2])\n grad_1 = autograd.grad(loss_1, [self.scale], create_graph=True)[0]\n grad_2 = autograd.grad(loss_2, [self.scale], create_graph=True)[0]\n penalty = torch.sum(grad_1 * grad_2)\n return penalty\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/dglib/generalization/mixstyle/__init__.py",
"content": ""
},
{
"path": "domain_generalization/Transfer-Learning-Library/dglib/generalization/mixstyle/models/__init__.py",
"content": "from .resnet import *\nfrom .mixstyle import *\n\n__all__ = ['resnet']\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/dglib/generalization/mixstyle/models/mixstyle.py",
"content": "\"\"\"\nModified from https://github.com/KaiyangZhou/mixstyle-release\n@author: Baixu Chen\n@contact: cbx_99_hasta@outlook.com\n\"\"\"\nimport random\nimport torch\nimport torch.nn as nn\n\n\nclass MixStyle(nn.Module):\n r\"\"\"MixStyle module from `DOMAIN GENERALIZATION WITH MIXSTYLE (ICLR 2021) `_.\n Given input :math:`x`, we first compute mean :math:`\\mu(x)` and standard deviation :math:`\\sigma(x)` across spatial\n dimension. Then we permute :math:`x` and get :math:`\\tilde{x}`, corresponding mean :math:`\\mu(\\tilde{x})` and\n standard deviation :math:`\\sigma(\\tilde{x})`. `MixUp` is performed using mean and standard deviation\n\n .. math::\n \\gamma_{mix} = \\lambda\\sigma(x) + (1-\\lambda)\\sigma(\\tilde{x})\n\n .. math::\n \\beta_{mix} = \\lambda\\mu(x) + (1-\\lambda)\\mu(\\tilde{x})\n\n where :math:`\\lambda` is instance-wise weight sampled from `Beta distribution`. MixStyle is then\n\n .. math::\n MixStyle(x) = \\gamma_{mix}\\frac{x-\\mu(x)}{\\sigma(x)} + \\beta_{mix}\n\n Args:\n p (float): probability of using MixStyle.\n alpha (float): parameter of the `Beta distribution`.\n eps (float): scaling parameter to avoid numerical issues.\n \"\"\"\n\n def __init__(self, p=0.5, alpha=0.1, eps=1e-6):\n super().__init__()\n self.p = p\n self.beta = torch.distributions.Beta(alpha, alpha)\n self.eps = eps\n self.alpha = alpha\n\n def forward(self, x):\n if not self.training:\n return x\n\n if random.random() > self.p:\n return x\n\n batch_size = x.size(0)\n\n mu = x.mean(dim=[2, 3], keepdim=True)\n var = x.var(dim=[2, 3], keepdim=True)\n sigma = (var + self.eps).sqrt()\n mu, sigma = mu.detach(), sigma.detach()\n x_normed = (x - mu) / sigma\n\n interpolation = self.beta.sample((batch_size, 1, 1, 1))\n interpolation = interpolation.to(x.device)\n\n # split into two halves and swap the order\n perm = torch.arange(batch_size - 1, -1, -1) # inverse index\n perm_b, perm_a = perm.chunk(2)\n perm_b = perm_b[torch.randperm(batch_size // 2)]\n perm_a = perm_a[torch.randperm(batch_size // 2)]\n perm = torch.cat([perm_b, perm_a], 0)\n\n mu_perm, sigma_perm = mu[perm], sigma[perm]\n mu_mix = mu * interpolation + mu_perm * (1 - interpolation)\n sigma_mix = sigma * interpolation + sigma_perm * (1 - interpolation)\n\n return x_normed * sigma_mix + mu_mix\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/dglib/generalization/mixstyle/models/resnet.py",
"content": "\"\"\"\n@author: Baixu Chen\n@contact: cbx_99_hasta@outlook.com\n\"\"\"\nfrom .mixstyle import MixStyle\nfrom common.vision.models.reid.resnet import ReidResNet\nfrom common.vision.models.resnet import ResNet, load_state_dict_from_url, model_urls, BasicBlock, Bottleneck\n\n__all__ = ['resnet18', 'resnet34', 'resnet50', 'resnet101']\n\n\ndef _resnet_with_mix_style(arch, block, layers, pretrained, progress, mix_layers=None, mix_p=0.5, mix_alpha=0.1,\n resnet_class=ResNet, **kwargs):\n \"\"\"Construct `ResNet` with MixStyle modules. Given any resnet architecture **resnet_class** that contains conv1,\n bn1, relu, maxpool, layer1-4, this function define a new class that inherits from **resnet_class** and inserts\n MixStyle module during forward pass. Although MixStyle Module can be inserted anywhere, original paper finds it\n better to place MixStyle after layer1-3. Our implementation follows this idea, but you are free to modify this\n function to try other possibilities.\n\n Args:\n arch (str): resnet architecture (resnet50 for example)\n block (class): class of resnet block\n layers (list): depth list of each block\n pretrained (bool): if True, load imagenet pre-trained model parameters\n progress (bool): whether or not to display a progress bar to stderr\n mix_layers (list): layers to insert MixStyle module after\n mix_p (float): probability to activate MixStyle during forward pass\n mix_alpha (float): parameter alpha for beta distribution\n resnet_class (class): base resnet class to inherit from\n \"\"\"\n\n if mix_layers is None:\n mix_layers = []\n\n available_resnet_class = [ResNet, ReidResNet]\n assert resnet_class in available_resnet_class\n\n class ResNetWithMixStyleModule(resnet_class):\n def __init__(self, mix_layers, mix_p=0.5, mix_alpha=0.1, *args, **kwargs):\n super(ResNetWithMixStyleModule, self).__init__(*args, **kwargs)\n self.mixStyleModule = MixStyle(p=mix_p, alpha=mix_alpha)\n for layer in mix_layers:\n assert layer in ['layer1', 'layer2', 'layer3']\n self.apply_layers = mix_layers\n\n def forward(self, x):\n x = self.conv1(x)\n x = self.bn1(x)\n # turn on relu activation here **except for** reid tasks\n if resnet_class != ReidResNet:\n x = self.relu(x)\n x = self.maxpool(x)\n\n x = self.layer1(x)\n if 'layer1' in self.apply_layers:\n x = self.mixStyleModule(x)\n x = self.layer2(x)\n if 'layer2' in self.apply_layers:\n x = self.mixStyleModule(x)\n x = self.layer3(x)\n if 'layer3' in self.apply_layers:\n x = self.mixStyleModule(x)\n x = self.layer4(x)\n\n return x\n\n model = ResNetWithMixStyleModule(mix_layers=mix_layers, mix_p=mix_p, mix_alpha=mix_alpha, block=block,\n layers=layers, **kwargs)\n if pretrained:\n model_dict = model.state_dict()\n pretrained_dict = load_state_dict_from_url(model_urls[arch],\n progress=progress)\n # remove keys from pretrained dict that doesn't appear in model dict\n pretrained_dict = {k: v for k, v in pretrained_dict.items() if k in model_dict}\n model.load_state_dict(pretrained_dict, strict=False)\n return model\n\n\ndef resnet18(pretrained=False, progress=True, **kwargs):\n \"\"\"Constructs a ResNet-18 model with MixStyle.\n\n Args:\n pretrained (bool): If True, returns a model pre-trained on ImageNet\n progress (bool): If True, displays a progress bar of the download to stderr\n \"\"\"\n return _resnet_with_mix_style('resnet18', BasicBlock, [2, 2, 2, 2], pretrained, progress,\n **kwargs)\n\n\ndef resnet34(pretrained=False, progress=True, **kwargs):\n \"\"\"Constructs a ResNet-34 model with MixStyle.\n\n Args:\n pretrained (bool): If True, returns a model pre-trained on ImageNet\n progress (bool): If True, displays a progress bar of the download to stderr\n \"\"\"\n return _resnet_with_mix_style('resnet34', BasicBlock, [3, 4, 6, 3], pretrained, progress,\n **kwargs)\n\n\ndef resnet50(pretrained=False, progress=True, **kwargs):\n \"\"\"Constructs a ResNet-50 model with MixStyle.\n\n Args:\n pretrained (bool): If True, returns a model pre-trained on ImageNet\n progress (bool): If True, displays a progress bar of the download to stderr\n \"\"\"\n return _resnet_with_mix_style('resnet50', Bottleneck, [3, 4, 6, 3], pretrained, progress,\n **kwargs)\n\n\ndef resnet101(pretrained=False, progress=True, **kwargs):\n \"\"\"Constructs a ResNet-101 model with MixStyle.\n\n Args:\n pretrained (bool): If True, returns a model pre-trained on ImageNet\n progress (bool): If True, displays a progress bar of the download to stderr\n \"\"\"\n return _resnet_with_mix_style('resnet101', Bottleneck, [3, 4, 23, 3], pretrained, progress,\n **kwargs)\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/dglib/generalization/mixstyle/sampler.py",
"content": "\"\"\"\n@author: Baixu Chen\n@contact: cbx_99_hasta@outlook.com\n\"\"\"\nimport random\nimport copy\nfrom torch.utils.data.dataset import ConcatDataset\nfrom torch.utils.data.sampler import Sampler\n\n\nclass RandomDomainMultiInstanceSampler(Sampler):\n r\"\"\"Randomly sample :math:`N` domains, then randomly select :math:`P` instances in each domain, for each instance,\n randomly select :math:`K` images to form a mini-batch of size :math:`N\\times P\\times K`.\n\n Args:\n dataset (ConcatDataset): dataset that contains data from multiple domains\n batch_size (int): mini-batch size (:math:`N\\times P\\times K` here)\n n_domains_per_batch (int): number of domains to select in a single mini-batch (:math:`N` here)\n num_instances (int): number of instances to select in each domain (:math:`K` here)\n \"\"\"\n\n def __init__(self, dataset, batch_size, n_domains_per_batch, num_instances):\n super(Sampler, self).__init__()\n self.dataset = dataset\n self.sample_idxes_per_domain = {}\n for idx, (_, _, domain_id) in enumerate(self.dataset):\n if domain_id not in self.sample_idxes_per_domain:\n self.sample_idxes_per_domain[domain_id] = []\n self.sample_idxes_per_domain[domain_id].append(idx)\n self.n_domains_in_dataset = len(self.sample_idxes_per_domain)\n self.n_domains_per_batch = n_domains_per_batch\n assert self.n_domains_in_dataset >= self.n_domains_per_batch\n\n assert batch_size % n_domains_per_batch == 0\n self.batch_size_per_domain = batch_size // n_domains_per_batch\n\n assert self.batch_size_per_domain % num_instances == 0\n self.num_instances = num_instances\n self.num_classes_per_domain = self.batch_size_per_domain // num_instances\n self.length = len(list(self.__iter__()))\n\n def __iter__(self):\n sample_idxes_per_domain = copy.deepcopy(self.sample_idxes_per_domain)\n domain_idxes = [idx for idx in range(self.n_domains_in_dataset)]\n final_idxes = []\n stop_flag = False\n while not stop_flag:\n selected_domains = random.sample(domain_idxes, self.n_domains_per_batch)\n\n for domain in selected_domains:\n sample_idxes = sample_idxes_per_domain[domain]\n selected_idxes = self.sample_multi_instances(sample_idxes)\n final_idxes.extend(selected_idxes)\n\n for idx in selected_idxes:\n sample_idxes_per_domain[domain].remove(idx)\n\n remaining_size = len(sample_idxes_per_domain[domain])\n if remaining_size < self.batch_size_per_domain:\n stop_flag = True\n\n return iter(final_idxes)\n\n def sample_multi_instances(self, sample_idxes):\n idxes_per_cls = {}\n for idx in sample_idxes:\n _, cls, _ = self.dataset[idx]\n if cls not in idxes_per_cls:\n idxes_per_cls[cls] = []\n idxes_per_cls[cls].append(idx)\n\n cls_list = [cls for cls in idxes_per_cls if len(idxes_per_cls[cls]) >= self.num_instances]\n if len(cls_list) < self.num_classes_per_domain:\n return random.sample(sample_idxes, self.batch_size_per_domain)\n\n selected_idxes = []\n selected_classes = random.sample(cls_list, self.num_classes_per_domain)\n for cls in selected_classes:\n selected_idxes.extend(random.sample(idxes_per_cls[cls], self.num_instances))\n return selected_idxes\n\n def __len__(self):\n return self.length\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/dglib/modules/__init__.py",
"content": ""
},
{
"path": "domain_generalization/Transfer-Learning-Library/dglib/modules/classifier.py",
"content": "\"\"\"\n@author: Baixu Chen\n@contact: cbx_99_hasta@outlook.com\n\"\"\"\nfrom typing import Optional, Tuple\nimport torch\nimport torch.nn as nn\nfrom common.modules.classifier import Classifier as ClassifierBase\n\n\nclass ImageClassifier(ClassifierBase):\n \"\"\"ImageClassifier specific for reproducing results of `DomainBed `_.\n You are free to freeze all `BatchNorm2d` layers and insert one additional `Dropout` layer, this can achieve better\n results for some datasets like PACS but may be worse for others.\n\n Args:\n backbone (torch.nn.Module): Any backbone to extract features from data\n num_classes (int): Number of classes\n freeze_bn (bool, optional): whether to freeze all `BatchNorm2d` layers. Default: False\n dropout_p (float, optional): dropout ratio for additional `Dropout` layer, this layer is only used when `freeze_bn` is True. Default: 0.1\n \"\"\"\n\n def __init__(self, backbone: nn.Module, num_classes: int, freeze_bn: Optional[bool] = False,\n dropout_p: Optional[float] = 0.1, **kwargs):\n super(ImageClassifier, self).__init__(backbone, num_classes, **kwargs)\n self.freeze_bn = freeze_bn\n if freeze_bn:\n self.feature_dropout = nn.Dropout(p=dropout_p)\n\n def forward(self, x: torch.Tensor) -> Tuple[torch.Tensor, torch.Tensor]:\n f = self.pool_layer(self.backbone(x))\n f = self.bottleneck(f)\n if self.freeze_bn:\n f = self.feature_dropout(f)\n predictions = self.head(f)\n if self.training:\n return predictions, f\n else:\n return predictions\n\n def train(self, mode=True):\n super(ImageClassifier, self).train(mode)\n if self.freeze_bn:\n for m in self.modules():\n if isinstance(m, nn.BatchNorm2d):\n m.eval()\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/dglib/modules/sampler.py",
"content": "\"\"\"\n@author: Baixu Chen\n@contact: cbx_99_hasta@outlook.com\n\"\"\"\nimport random\nimport copy\nimport numpy as np\nfrom torch.utils.data.dataset import ConcatDataset\nfrom torch.utils.data.sampler import Sampler\n\n\nclass DefaultSampler(Sampler):\n r\"\"\"Traverse all :math:`N` domains, randomly select :math:`K` samples in each domain to form a mini-batch of size\n :math:`N\\times K`.\n\n Args:\n data_source (ConcatDataset): dataset that contains data from multiple domains\n batch_size (int): mini-batch size (:math:`N\\times K` here)\n \"\"\"\n\n def __init__(self, data_source: ConcatDataset, batch_size: int):\n super(Sampler, self).__init__()\n self.num_all_domains = len(data_source.cumulative_sizes)\n\n self.sample_idxes_per_domain = []\n start = 0\n for end in data_source.cumulative_sizes:\n idxes = [idx for idx in range(start, end)]\n self.sample_idxes_per_domain.append(idxes)\n start = end\n\n assert batch_size % self.num_all_domains == 0\n self.batch_size_per_domain = batch_size // self.num_all_domains\n self.length = len(list(self.__iter__()))\n\n def __iter__(self):\n sample_idxes_per_domain = copy.deepcopy(self.sample_idxes_per_domain)\n final_idxes = []\n stop_flag = False\n while not stop_flag:\n for domain in range(self.num_all_domains):\n sample_idxes = sample_idxes_per_domain[domain]\n selected_idxes = random.sample(sample_idxes, self.batch_size_per_domain)\n final_idxes.extend(selected_idxes)\n\n for idx in selected_idxes:\n sample_idxes_per_domain[domain].remove(idx)\n\n remaining_size = len(sample_idxes_per_domain[domain])\n if remaining_size < self.batch_size_per_domain:\n stop_flag = True\n\n return iter(final_idxes)\n\n def __len__(self):\n return self.length\n\n\nclass RandomDomainSampler(Sampler):\n r\"\"\"Randomly sample :math:`N` domains, then randomly select :math:`K` samples in each domain to form a mini-batch of\n size :math:`N\\times K`.\n\n Args:\n data_source (ConcatDataset): dataset that contains data from multiple domains\n batch_size (int): mini-batch size (:math:`N\\times K` here)\n n_domains_per_batch (int): number of domains to select in a single mini-batch (:math:`N` here)\n \"\"\"\n\n def __init__(self, data_source: ConcatDataset, batch_size: int, n_domains_per_batch: int):\n super(Sampler, self).__init__()\n self.n_domains_in_dataset = len(data_source.cumulative_sizes)\n self.n_domains_per_batch = n_domains_per_batch\n assert self.n_domains_in_dataset >= self.n_domains_per_batch\n\n self.sample_idxes_per_domain = []\n start = 0\n for end in data_source.cumulative_sizes:\n idxes = [idx for idx in range(start, end)]\n self.sample_idxes_per_domain.append(idxes)\n start = end\n\n assert batch_size % n_domains_per_batch == 0\n self.batch_size_per_domain = batch_size // n_domains_per_batch\n self.length = len(list(self.__iter__()))\n\n def __iter__(self):\n sample_idxes_per_domain = copy.deepcopy(self.sample_idxes_per_domain)\n domain_idxes = [idx for idx in range(self.n_domains_in_dataset)]\n final_idxes = []\n stop_flag = False\n while not stop_flag:\n selected_domains = random.sample(domain_idxes, self.n_domains_per_batch)\n\n for domain in selected_domains:\n sample_idxes = sample_idxes_per_domain[domain]\n if len(sample_idxes) < self.batch_size_per_domain:\n selected_idxes = np.random.choice(sample_idxes, self.batch_size_per_domain, replace=True)\n else:\n selected_idxes = random.sample(sample_idxes, self.batch_size_per_domain)\n final_idxes.extend(selected_idxes)\n\n for idx in selected_idxes:\n if idx in sample_idxes_per_domain[domain]:\n sample_idxes_per_domain[domain].remove(idx)\n\n remaining_size = len(sample_idxes_per_domain[domain])\n if remaining_size < self.batch_size_per_domain:\n stop_flag = True\n\n return iter(final_idxes)\n\n def __len__(self):\n return self.length\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/docs/Makefile",
"content": "# Minimal makefile for Sphinx documentation\n#\n\n# You can set these variables from the command line.\nSPHINXOPTS =\nSPHINXBUILD = sphinx-build\nSPHINXPROJ = PyTorchSphinxTheme\nSOURCEDIR = .\nBUILDDIR = build\n\n# Put it first so that \"make\" without argument is like \"make help\".\nhelp:\n\t@$(SPHINXBUILD) -M help \"$(SOURCEDIR)\" \"$(BUILDDIR)\" $(SPHINXOPTS) $(O)\n\n.PHONY: help Makefile\n\n# Catch-all target: route all unknown targets to Sphinx using the new\n# \"make mode\" option. $(O) is meant as a shortcut for $(SPHINXOPTS).\n%: Makefile\n\t@$(SPHINXBUILD) -M $@ \"$(SOURCEDIR)\" \"$(BUILDDIR)\" $(SPHINXOPTS) $(O)\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/docs/common/loss.rst",
"content": "Loss\n=====================\n\n\nKnowledge Distillation\n-------------------------------\n.. autoclass:: common.loss.KnowledgeDistillationLoss\n :members:\n\nCross Entropy with Label Smooth\n----------------------------------------\n.. autoclass:: common.vision.models.reid.loss.CrossEntropyLossWithLabelSmooth\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/docs/common/modules.rst",
"content": "Modules\n=====================\n\n\nClassifier\n-------------------------------\n.. autoclass:: common.modules.classifier.Classifier\n :members:\n\nRegressor\n-------------------------------\n.. autoclass:: common.modules.regressor.Regressor\n :members:\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/docs/common/utils/analysis.rst",
"content": "==============\nAnalysis Tools\n==============\n\n\n.. autofunction:: common.utils.analysis.collect_feature\n\n\n.. autofunction:: common.utils.analysis.a_distance.calculate\n\n\n.. autofunction:: common.utils.analysis.tsne.visualize\n\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/docs/common/utils/base.rst",
"content": "Generic Tools\n==============\n\n\nAverage Meter\n---------------------------------\n\n.. autoclass:: common.utils.meter.AverageMeter\n :members:\n\nProgress Meter\n---------------------------------\n\n.. autoclass:: common.utils.meter.ProgressMeter\n :members:\n\nMeter\n---------------------------------\n\n.. autoclass:: common.utils.meter.Meter\n :members:\n\nData\n---------------------------------\n\n.. autoclass:: common.utils.data.ForeverDataIterator\n :members:\n\n.. autoclass:: common.utils.data.CombineDataset\n :members:\n\n.. autofunction:: common.utils.data.send_to_device\n\n.. autofunction:: common.utils.data.concatenate\n\nLogger\n-----------\n\n.. autoclass:: common.utils.logger.TextLogger\n :members:\n\n\n.. autoclass:: common.utils.logger.CompleteLogger\n :members:\n\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/docs/common/utils/metric.rst",
"content": "===========\nMetrics\n===========\n\nClassification & Segmentation\n==============================\n\n\nAccuracy\n---------------------------------\n\n.. autofunction:: common.utils.metric.accuracy\n\n\nConfusionMatrix\n---------------------------------\n\n.. autoclass:: common.utils.metric.ConfusionMatrix\n :members:\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/docs/common/vision/datasets.rst",
"content": "Vision Datasets\n=============================\n\nCross-Domain Classification\n---------------------------------------------------------\n\n\n--------------------------------------\nImageList\n--------------------------------------\n\n.. autoclass:: common.vision.datasets.imagelist.ImageList\n :members:\n\n-------------------------------------\nOffice-31\n-------------------------------------\n\n.. autoclass:: common.vision.datasets.office31.Office31\n :members:\n :inherited-members:\n\n---------------------------------------\nOffice-Caltech\n---------------------------------------\n\n.. autoclass:: common.vision.datasets.officecaltech.OfficeCaltech\n :members:\n :inherited-members:\n\n---------------------------------------\nOffice-Home\n---------------------------------------\n\n.. autoclass:: common.vision.datasets.officehome.OfficeHome\n :members:\n :inherited-members:\n\n--------------------------------------\nVisDA-2017\n--------------------------------------\n\n.. autoclass:: common.vision.datasets.visda2017.VisDA2017\n :members:\n :inherited-members:\n\n--------------------------------------\nDomainNet\n--------------------------------------\n\n.. autoclass:: common.vision.datasets.domainnet.DomainNet\n :members:\n :inherited-members:\n\n--------------------------------------\nPACS\n--------------------------------------\n\n.. autoclass:: common.vision.datasets.pacs.PACS\n :members:\n\n\n--------------------------------------\nMNIST\n--------------------------------------\n\n.. autoclass:: common.vision.datasets.digits.MNIST\n :members:\n\n\n--------------------------------------\nUSPS\n--------------------------------------\n\n.. autoclass:: common.vision.datasets.digits.USPS\n :members:\n\n\n--------------------------------------\nSVHN\n--------------------------------------\n\n.. autoclass:: common.vision.datasets.digits.SVHN\n :members:\n\n\nPartial Cross-Domain Classification\n----------------------------------------------------\n\n---------------------------------------\nPartial Wrapper\n---------------------------------------\n\n.. autofunction:: common.vision.datasets.partial.partial\n\n.. autofunction:: common.vision.datasets.partial.default_partial\n\n\n---------------------------------------\nCaltech-256->ImageNet-1k\n---------------------------------------\n\n.. autoclass:: common.vision.datasets.partial.caltech_imagenet.CaltechImageNet\n :members:\n\n\n---------------------------------------\nImageNet-1k->Caltech-256\n---------------------------------------\n\n.. autoclass:: common.vision.datasets.partial.imagenet_caltech.ImageNetCaltech\n :members:\n\n\nOpen Set Cross-Domain Classification\n------------------------------------------------------\n\n---------------------------------------\nOpen Set Wrapper\n---------------------------------------\n\n.. autofunction:: common.vision.datasets.openset.open_set\n\n.. autofunction:: common.vision.datasets.openset.default_open_set\n\n\nCross-Domain Regression\n------------------------------------------------------\n\n---------------------------------------\nImageRegression\n---------------------------------------\n\n.. autoclass:: common.vision.datasets.regression.image_regression.ImageRegression\n :members:\n\n---------------------------------------\nDSprites\n---------------------------------------\n.. autoclass:: common.vision.datasets.regression.dsprites.DSprites\n :members:\n\n---------------------------------------\nMPI3D\n---------------------------------------\n.. autoclass:: common.vision.datasets.regression.mpi3d.MPI3D\n :members:\n\n\nCross-Domain Segmentation\n-----------------------------------------------\n\n---------------------------------------\nSegmentationList\n---------------------------------------\n.. autoclass:: common.vision.datasets.segmentation.segmentation_list.SegmentationList\n :members:\n\n---------------------------------------\nCityscapes\n---------------------------------------\n.. autoclass:: common.vision.datasets.segmentation.cityscapes.Cityscapes\n\n---------------------------------------\nGTA5\n---------------------------------------\n.. autoclass:: common.vision.datasets.segmentation.gta5.GTA5\n\n---------------------------------------\nSynthia\n---------------------------------------\n.. autoclass:: common.vision.datasets.segmentation.synthia.Synthia\n\n\n---------------------------------------\nFoggy Cityscapes\n---------------------------------------\n.. autoclass:: common.vision.datasets.segmentation.cityscapes.FoggyCityscapes\n\n\nCross-Domain Keypoint Detection\n-----------------------------------------------\n\n---------------------------------------\nDataset Base for Keypoint Detection\n---------------------------------------\n.. autoclass:: common.vision.datasets.keypoint_detection.keypoint_dataset.KeypointDataset\n :members:\n\n.. autoclass:: common.vision.datasets.keypoint_detection.keypoint_dataset.Body16KeypointDataset\n :members:\n\n.. autoclass:: common.vision.datasets.keypoint_detection.keypoint_dataset.Hand21KeypointDataset\n :members:\n\n---------------------------------------\nRendered Handpose Dataset\n---------------------------------------\n.. autoclass:: common.vision.datasets.keypoint_detection.rendered_hand_pose.RenderedHandPose\n :members:\n\n---------------------------------------\nHand-3d-Studio Dataset\n---------------------------------------\n.. autoclass:: common.vision.datasets.keypoint_detection.hand_3d_studio.Hand3DStudio\n :members:\n\n---------------------------------------\nFreiHAND Dataset\n---------------------------------------\n.. autoclass:: common.vision.datasets.keypoint_detection.freihand.FreiHand\n :members:\n\n---------------------------------------\nSurreal Dataset\n---------------------------------------\n.. autoclass:: common.vision.datasets.keypoint_detection.surreal.SURREAL\n :members:\n\n---------------------------------------\nLSP Dataset\n---------------------------------------\n.. autoclass:: common.vision.datasets.keypoint_detection.lsp.LSP\n :members:\n\n---------------------------------------\nHuman3.6M Dataset\n---------------------------------------\n.. autoclass:: common.vision.datasets.keypoint_detection.human36m.Human36M\n :members:\n\nCross-Domain ReID\n------------------------------------------------------\n\n---------------------------------------\nMarket1501\n---------------------------------------\n\n.. autoclass:: common.vision.datasets.reid.market1501.Market1501\n :members:\n\n---------------------------------------\nDukeMTMC-reID\n---------------------------------------\n\n.. autoclass:: common.vision.datasets.reid.dukemtmc.DukeMTMC\n :members:\n\n---------------------------------------\nMSMT17\n---------------------------------------\n\n.. autoclass:: common.vision.datasets.reid.msmt17.MSMT17\n :members:\n\n\nNatural Object Recognition\n---------------------------------------------------------\n\n\n-------------------------------------\nStanford Dogs\n-------------------------------------\n\n.. autoclass:: common.vision.datasets.stanford_dogs.StanfordDogs\n :members:\n\n-------------------------------------\nStanford Cars\n-------------------------------------\n\n.. autoclass:: common.vision.datasets.stanford_cars.StanfordCars\n :members:\n\n-------------------------------------\nCUB-200-2011\n-------------------------------------\n\n.. autoclass:: common.vision.datasets.cub200.CUB200\n :members:\n\n-------------------------------------\nFVGC Aircraft\n-------------------------------------\n\n.. autoclass:: common.vision.datasets.aircrafts.Aircraft\n :members:\n\n-------------------------------------\nOxford-IIIT Pet\n-------------------------------------\n\n.. autoclass:: common.vision.datasets.oxfordpet.OxfordIIITPet\n :members:\n\n-------------------------------------\nCOCO-70\n-------------------------------------\n\n.. autoclass:: common.vision.datasets.coco70.COCO70\n :members:\n\n-------------------------------------\nDTD\n-------------------------------------\n\n.. autoclass:: common.vision.datasets.dtd.DTD\n :members:\n\n-------------------------------------\nOxfordFlowers102\n-------------------------------------\n\n.. autoclass:: common.vision.datasets.oxfordflowers.OxfordFlowers102\n :members:\n\n\nSpecialized Image Classification\n--------------------------------\n\n-------------------------------------\nPatchCamelyon\n-------------------------------------\n\n.. autoclass:: common.vision.datasets.patchcamelyon.PatchCamelyon\n :members:\n\n-------------------------------------\nRetinopathy\n-------------------------------------\n\n.. autoclass:: common.vision.datasets.retinopathy.Retinopathy\n :members:\n\n-------------------------------------\nEuroSAT\n-------------------------------------\n\n.. autoclass:: common.vision.datasets.eurosat.EuroSAT\n :members:\n\n-------------------------------------\nResisc45\n-------------------------------------\n\n.. autoclass:: common.vision.datasets.resisc45.Resisc45\n :members:"
},
{
"path": "domain_generalization/Transfer-Learning-Library/docs/common/vision/models.rst",
"content": "Vision Models\n===========================\n\n------------------------------\nImage Classification\n------------------------------\n\nResNets\n---------------------------------\n\n.. automodule:: common.vision.models.resnet\n :members:\n\nLeNet\n--------------------------\n\n.. automodule:: common.vision.models.digits.lenet\n :members:\n\nDTN\n--------------------------\n\n.. automodule:: common.vision.models.digits.dtn\n :members:\n\n\n----------------------------------\nSemantic Segmentation\n----------------------------------\n\n.. autofunction:: common.vision.models.segmentation.deeplabv2.deeplabv2_resnet101\n\n\n----------------------------------\nKeypoint Detection\n----------------------------------\n\nPoseResNet\n--------------------------\n\n.. autofunction:: common.vision.models.keypoint_detection.pose_resnet.pose_resnet101\n\n.. autoclass:: common.vision.models.keypoint_detection.pose_resnet.PoseResNet\n\n.. autoclass:: common.vision.models.keypoint_detection.pose_resnet.Upsampling\n\n\nJoint Loss\n----------------------------------\n\n.. autoclass:: common.vision.models.keypoint_detection.loss.JointsMSELoss\n\n.. autoclass:: common.vision.models.keypoint_detection.loss.JointsKLLoss\n\n\n-----------------------------------\nRe-Identification\n-----------------------------------\n\nModels\n---------------\n.. autoclass:: common.vision.models.reid.resnet.ReidResNet\n\n.. automodule:: common.vision.models.reid.resnet\n :members:\n\n.. autoclass:: common.vision.models.reid.identifier.ReIdentifier\n :members:\n\nLoss\n-----------------------------------\n.. autoclass:: common.vision.models.reid.loss.TripletLoss\n\nSampler\n-----------------------------------\n.. autoclass:: common.utils.data.RandomMultipleGallerySampler\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/docs/common/vision/transforms.rst",
"content": "Vision Transforms\n=============================\n\n\nClassification\n---------------------------------\n\n.. automodule:: common.vision.transforms\n :members:\n\n\nSegmentation\n---------------------------------\n\n\n.. automodule:: common.vision.transforms.segmentation\n :members:\n\n\nKeypoint Detection\n---------------------------------\n\n\n.. automodule:: common.vision.transforms.keypoint_detection\n :members:\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/docs/conf.py",
"content": "import sys\nimport os\n\nsys.path.append(os.path.abspath('..'))\nsys.path.append(os.path.abspath('./demo/'))\n\nfrom pytorch_sphinx_theme import __version__\nimport pytorch_sphinx_theme\n\n# If extensions (or modules to document with autodoc) are in another directory,\n# add these directories to sys.path here. If the directory is relative to the\n# documentation root, use os.path.abspath to make it absolute, like shown here.\n#sys.path.insert(0, os.path.abspath('.'))\n\n# -- General configuration -----------------------------------------------------\n\n# If your documentation needs a minimal Sphinx version, state it here.\n#needs_sphinx = '1.0'\n\n# Add any Sphinx extension module names here, as strings. They can be extensions\n# coming with Sphinx (named 'sphinx.ext.*') or your custom ones.\nextensions = [\n 'sphinx.ext.intersphinx',\n 'sphinx.ext.autodoc',\n 'sphinx.ext.viewcode',\n 'sphinxcontrib.httpdomain',\n 'sphinx.ext.autosummary',\n 'sphinx.ext.autosectionlabel',\n 'sphinx.ext.napoleon',\n]\n\n# build the templated autosummary files\nautosummary_generate = True\nnumpydoc_show_class_members = False\n\n# autosectionlabel throws warnings if section names are duplicated.\n# The following tells autosectionlabel to not throw a warning for\n# duplicated section names that are in different documents.\nautosectionlabel_prefix_document = True\n\n\nnapoleon_use_ivar = True\n\n# Do not warn about external images (status badges in README.rst)\nsuppress_warnings = ['image.nonlocal_uri']\n\n# Add any paths that contain templates here, relative to this directory.\ntemplates_path = ['_templates']\n\n# The suffix of source filenames.\nsource_suffix = '.rst'\n\n# The encoding of source files.\n#source_encoding = 'utf-8-sig'\n\n# The master toctree document.\nmaster_doc = 'index'\n\n# General information about the project.\nproject = u'Transfer Learning Library'\ncopyright = u'THUML Group'\n\n# The version info for the project you're documenting, acts as replacement for\n# |version| and |release|, also used in various other places throughout the\n# built documents.\n#\n# The short X.Y version.\nversion = __version__\n# The full version, including alpha/beta/rc tags.\nrelease = __version__\n\n# The language for content autogenerated by Sphinx. Refer to documentation\n# for a list of supported languages.\nlanguage = 'en'\n\n# There are two options for replacing |today|: either, you set today to some\n# non-false value, then it is used:\n#today = ''\n# Else, today_fmt is used as the format for a strftime call.\n#today_fmt = '%B %d, %Y'\n\n# List of patterns, relative to source directory, that match files and\n# directories to ignore when looking for source files.\nexclude_patterns = []\n\n# The reST default role (used for this markup: `text`) to use for all documents.\n#default_role = None\n\n# If true, '()' will be appended to :func: etc. cross-reference text.\n#add_function_parentheses = True\n\n# If true, the current module name will be prepended to all description\n# unit titles (such as .. function::).\n#add_module_names = True\n\n# If true, sectionauthor and moduleauthor directives will be shown in the\n# output. They are ignored by default.\n#show_authors = False\n\n# The name of the Pygments (syntax highlighting) style to use.\npygments_style = 'sphinx'\n\n\n# A list of ignored prefixes for module index sorting.\n#modindex_common_prefix = []\n\nintersphinx_mapping = {\n 'rtd': ('https://docs.readthedocs.io/en/latest/', None),\n 'python': ('https://docs.python.org/3', None),\n 'numpy': ('https://numpy.org/doc/stable', None),\n 'torch': ('https://pytorch.org/docs/stable', None),\n 'torchvision': ('https://pytorch.org/vision/stable', None),\n 'PIL': ('https://pillow.readthedocs.io/en/stable/', None)\n}\n\n\n# -- Options for HTML output ---------------------------------------------------\n\n# The theme to use for HTML and HTML Help pages. See the documentation for\n# a list of builtin themes.\nhtml_theme = 'pytorch_sphinx_theme'\nhtml_theme_path = [pytorch_sphinx_theme.get_html_theme_path()]\n\n# Theme options are theme-specific and customize the look and feel of a theme\n# further. For a list of options available for each theme, see the\n# documentation.\nhtml_theme_options = {\n 'canonical_url': '',\n 'analytics_id': '',\n 'logo_only': False,\n 'display_version': False,\n 'prev_next_buttons_location': 'bottom',\n 'style_external_links': False,\n\n # Toc options\n 'collapse_navigation': True,\n 'sticky_navigation': False,\n 'navigation_depth': 4,\n 'includehidden': True,\n 'titles_only': False\n}\n\n\n# The name for this set of Sphinx documents. If None, it defaults to\n# \" v documentation\".\n#html_title = None\n\n# A shorter title for the navigation bar. Default is the same as html_title.\n#html_short_title = None\n\n# The name of an image file (relative to this directory) to place at the top\n# of the sidebar.\nhtml_logo = \"_static/images/TransLearn.png\"\n\n# The name of an image file (within the static path) to use as favicon of the\n# docs. This file should be a Windows icon file (.ico) being 16x16 or 32x32\n# pixels large.\n#html_favicon = None\n\n# Add any paths that contain custom static files (such as style sheets) here,\n# relative to this directory. They are copied after the builtin static files,\n# so a file named \"default.css\" will overwrite the builtin \"default.css\".\nhtml_static_path = ['_static']\n\n# If not '', a 'Last updated on:' timestamp is inserted at every page bottom,\n# using the given strftime format.\n#html_last_updated_fmt = '%b %d, %Y'\n\n# If true, SmartyPants will be used to convert quotes and dashes to\n# typographically correct entities.\n#html_use_smartypants = True\n\n# Custom sidebar templates, maps document names to template names.\n#html_sidebars = {}\n\n# Additional templates that should be rendered to pages, maps page names to\n# template names.\n#html_additional_pages = {}\n\n# If false, no module index is generated.\n#html_domain_indices = True\n\n# If false, no index is generated.\n#html_use_index = True\n\n# If true, the index is split into individual pages for each letter.\n#html_split_index = False\n\n# If true, links to the reST sources are added to the pages.\nhtml_show_sourcelink = True\n\n# If true, \"Created using Sphinx\" is shown in the HTML footer. Default is True.\n#html_show_sphinx = True\n\n# If true, \"(C) Copyright ...\" is shown in the HTML footer. Default is True.\n#html_show_copyright = True\n\n# If true, an OpenSearch description file will be output, and all pages will\n# contain a tag referring to it. The value of this option must be the\n# base URL from which the finished HTML is served.\n#html_use_opensearch = ''\n\n# This is the file name suffix for HTML files (e.g. \".xhtml\").\n#html_file_suffix = None\n\n\n# Disable displaying type annotations, these can be very verbose\nautodoc_typehints = 'none'\n\n# Output file base name for HTML help builder.\nhtmlhelp_basename = 'TransferLearningLibrary'\n\n\n# -- Options for LaTeX output --------------------------------------------------\n\nlatex_elements = {\n# The paper size ('letterpaper' or 'a4paper').\n#'papersize': 'letterpaper',\n\n# The font size ('10pt', '11pt' or '12pt').\n#'pointsize': '10pt',\n\n# Additional stuff for the LaTeX preamble.\n#'preamble': '',\n}\n\n# Grouping the document tree into LaTeX files. List of tuples\n# (source start file, target name, title, author, documentclass [howto/manual]).\nlatex_documents = [\n ('index', 'TransferLearningLibrary.tex', u'Transfer Learning Library Documentation',\n u'THUML', 'manual'),\n]\n\n# The name of an image file (relative to this directory) to place at the top of\n# the title page.\n#latex_logo = None\n\n# For \"manual\" documents, if this is true, then toplevel headings are parts,\n# not chapters.\n#latex_use_parts = False\n\n# If true, show page references after internal links.\n#latex_show_pagerefs = False\n\n# If true, show URL addresses after external links.\n#latex_show_urls = False\n\n# Documents to append as an appendix to all manuals.\n#latex_appendices = []\n\n# If false, no module index is generated.\n#latex_domain_indices = True\n\n\n# -- Options for manual page output --------------------------------------------\n\n# One entry per manual page. List of tuples\n# (source start file, name, description, authors, manual section).\nman_pages = [\n ('index', 'Transfer Learning Library', u'Transfer Learning Library Documentation',\n [u'THUML'], 1)\n]\n\n# If true, show URL addresses after external links.\n#man_show_urls = False\n\n\n# -- Options for Texinfo output ------------------------------------------------\n\n# Grouping the document tree into Texinfo files. List of tuples\n# (source start file, target name, title, author,\n# dir menu entry, description, category)\ntexinfo_documents = [\n ('index', 'Transfer Learning Library', u'Transfer Learning Library Documentation',\n u'THUML', 'Transfer Learning Library',\n 'One line description of project.', 'Miscellaneous'),\n]\n\n# Documents to append as an appendix to all manuals.\n#texinfo_appendices = []\n\n# If false, no module index is generated.\n#texinfo_domain_indices = True\n\n# How to display URL addresses: 'footnote', 'no', or 'inline'.\n#texinfo_show_urls = 'footnote'\n\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/docs/dalib/adaptation/domain_adversarial.rst",
"content": "=======================================\nDomain Adversarial Learning\n=======================================\n\nDomain Adversarial Training in Close-set DA\n===========================================\n\n.. _DANN:\n\nDANN: Domain Adversarial Neural Network\n----------------------------------------\n\n.. autoclass:: dalib.adaptation.dann.DomainAdversarialLoss\n\n.. _CDAN:\n\nCDAN: Conditional Domain Adversarial Network\n-----------------------------------------------\n\n.. autoclass:: dalib.adaptation.cdan.ConditionalDomainAdversarialLoss\n\n\n.. autoclass:: dalib.adaptation.cdan.RandomizedMultiLinearMap\n\n\n.. autoclass:: dalib.adaptation.cdan.MultiLinearMap\n\n\n.. _ADDA:\n\nADDA: Adversarial Discriminative Domain Adaptation\n-----------------------------------------------------\n\n.. autoclass:: dalib.adaptation.adda.DomainAdversarialLoss\n\n.. note::\n ADDAgrl is also implemented and benchmarked. You can find code\n `here `_.\n\n\n.. _BSP:\n\nBSP: Batch Spectral Penalization\n-----------------------------------\n\n.. autoclass:: dalib.adaptation.bsp.BatchSpectralPenalizationLoss\n\n\nDomain Adversarial Training in Open-set DA\n===========================================\n\n\n.. _OSBP:\n\nOSBP: Open Set Domain Adaptation by Backpropagation\n----------------------------------------------------\n\n.. autoclass:: dalib.adaptation.osbp.UnknownClassBinaryCrossEntropy\n\n\n\n\nDomain Adversarial Training in Partial DA\n===========================================\n\n.. _PADA:\n\nPADA: Partial Adversarial Domain Adaptation\n---------------------------------------------\n\n.. autoclass:: dalib.adaptation.pada.ClassWeightModule\n\n.. autoclass:: dalib.adaptation.pada.AutomaticUpdateClassWeightModule\n :members:\n\n.. autofunction:: dalib.adaptation.pada.collect_classification_results\n\n\n.. _IWAN:\n\nIWAN: Importance Weighted Adversarial Nets\n---------------------------------------------\n\n.. autoclass:: dalib.adaptation.iwan.ImportanceWeightModule\n :members:\n\n\n\nDomain Adversarial Training in Segmentation\n===========================================\n\n.. _ADVENT:\n\nADVENT: Adversarial Entropy Minimization\n------------------------------------------\n\n.. autoclass:: dalib.adaptation.advent.Discriminator\n\n.. autoclass:: dalib.adaptation.advent.DomainAdversarialEntropyLoss\n :members:\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/docs/dalib/adaptation/domain_translation.rst",
"content": "=======================================\nDomain Translation\n=======================================\n\n\n.. _CycleGAN:\n\n------------------------------------------------\nCycleGAN: Cycle-Consistent Adversarial Networks\n------------------------------------------------\n\nDiscriminator\n--------------\n\n.. autofunction:: dalib.translation.cyclegan.pixel\n\n.. autofunction:: dalib.translation.cyclegan.patch\n\nGenerator\n--------------\n\n.. autofunction:: dalib.translation.cyclegan.resnet_9\n\n.. autofunction:: dalib.translation.cyclegan.resnet_6\n\n.. autofunction:: dalib.translation.cyclegan.unet_256\n\n.. autofunction:: dalib.translation.cyclegan.unet_128\n\n\nGAN Loss\n--------------\n\n.. autoclass:: dalib.translation.cyclegan.LeastSquaresGenerativeAdversarialLoss\n\n.. autoclass:: dalib.translation.cyclegan.VanillaGenerativeAdversarialLoss\n\n.. autoclass:: dalib.translation.cyclegan.WassersteinGenerativeAdversarialLoss\n\nTranslation\n--------------\n\n.. autoclass:: dalib.translation.cyclegan.Translation\n\n\nUtil\n----------------\n\n.. autoclass:: dalib.translation.cyclegan.util.ImagePool\n :members:\n\n.. autofunction:: dalib.translation.cyclegan.util.set_requires_grad\n\n\n\n\n.. _Cycada:\n\n--------------------------------------------------------------\nCyCADA: Cycle-Consistent Adversarial Domain Adaptation\n--------------------------------------------------------------\n\n.. autoclass:: dalib.translation.cycada.SemanticConsistency\n\n\n\n.. _SPGAN:\n\n-----------------------------------------------------------\nSPGAN: Similarity Preserving Generative Adversarial Network\n-----------------------------------------------------------\n`Image-Image Domain Adaptation with Preserved Self-Similarity and Domain-Dissimilarity for Person Re-identification\n`_. SPGAN is based on CycleGAN. An additional Siamese network is adopted to force\nthe generator to produce images different from identities in target dataset.\n\nSiamese Network\n-------------------\n\n.. autoclass:: dalib.translation.spgan.siamese.SiameseNetwork\n\nContrastive Loss\n-------------------\n\n.. autoclass:: dalib.translation.spgan.loss.ContrastiveLoss\n\n\n.. _FDA:\n\n------------------------------------------------\nFDA: Fourier Domain Adaptation\n------------------------------------------------\n\n.. autoclass:: dalib.translation.fourier_transform.FourierTransform\n\n.. autofunction:: dalib.translation.fourier_transform.low_freq_mutate\n\n.. autofunction:: dalib.adaptation.fda.robust_entropy\n\n\n\n\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/docs/dalib/adaptation/hypothesis_adversarial.rst",
"content": "=======================================\nHypothesis Adversarial Learning\n=======================================\n\n.. _MCD:\n\nMCD: Maximum Classifier Discrepancy\n=============================================\n\n.. autofunction:: dalib.adaptation.mcd.classifier_discrepancy\n\n.. autofunction:: dalib.adaptation.mcd.entropy\n\n.. autoclass:: dalib.adaptation.mcd.ImageClassifierHead\n\n\n.. _MDD:\n\nMDD: Margin Disparity Discrepancy\n=============================================\n\n\n.. autoclass:: dalib.adaptation.mdd.MarginDisparityDiscrepancy\n\n\nMDD for Classification\n----------------------\n\n.. autoclass:: dalib.adaptation.mdd.ClassificationMarginDisparityDiscrepancy\n\n\n.. autoclass:: dalib.adaptation.mdd.ImageClassifier\n :members:\n\n.. autofunction:: dalib.adaptation.mdd.shift_log\n\n\nMDD for Regression\n------------------\n\n.. autoclass:: dalib.adaptation.mdd.RegressionMarginDisparityDiscrepancy\n\n.. autoclass:: dalib.adaptation.mdd.ImageRegressor\n\n\n.. _RegDA:\n\nRegDA: Regressive Domain Adaptation\n=============================================\n\n.. autoclass:: dalib.adaptation.regda.PseudoLabelGenerator2d\n\n.. autoclass:: dalib.adaptation.regda.RegressionDisparity\n\n.. autoclass:: dalib.adaptation.regda.PoseResNet2d\n\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/docs/dalib/adaptation/others.rst",
"content": "=======================================\nSelf Training Methods\n=======================================\n\n.. _SelfEnsemble:\n\nSelf Ensemble\n-----------------------------\n\n.. autoclass:: dalib.adaptation.self_ensemble.ConsistencyLoss\n\n\n.. autoclass:: dalib.adaptation.self_ensemble.L2ConsistencyLoss\n\n\n.. autoclass:: dalib.adaptation.self_ensemble.ClassBalanceLoss\n\n\n.. autoclass:: dalib.adaptation.self_ensemble.EmaTeacher\n\n\n.. _MCC:\n\nMCC: Minimum Class Confusion\n-----------------------------\n\n.. autoclass:: dalib.adaptation.mcc.MinimumClassConfusionLoss\n\n\n.. _MMT:\n\nMMT: Mutual Mean-Teaching\n--------------------------\n`Mutual Mean-Teaching: Pseudo Label Refinery for Unsupervised\nDomain Adaptation on Person Re-identification (ICLR 2020) `_\n\nState of the art unsupervised domain adaptation methods utilize clustering algorithms to generate pseudo labels on target\ndomain, which are noisy and thus harmful for training. Inspired by the teacher-student approaches, MMT framework\nprovides robust soft pseudo labels in an on-line peer-teaching manner.\n\nWe denote two networks as :math:`f_1,f_2`, their parameters as :math:`\\theta_1,\\theta_2`. The authors also\npropose to use the temporally average model of each network :math:`\\text{ensemble}(f_1),\\text{ensemble}(f_2)` to generate more reliable\nsoft pseudo labels for supervising the other network. Specifically, the parameters of the temporally\naverage models of the two networks at current iteration :math:`T` are denoted as :math:`E^{(T)}[\\theta_1]` and\n:math:`E^{(T)}[\\theta_2]` respectively, which can be calculated as\n\n.. math::\n E^{(T)}[\\theta_1] = \\alpha E^{(T-1)}[\\theta_1] + (1-\\alpha)\\theta_1\n.. math::\n E^{(T)}[\\theta_2] = \\alpha E^{(T-1)}[\\theta_2] + (1-\\alpha)\\theta_2\n\nwhere :math:`E^{(T-1)}[\\theta_1],E^{(T-1)}[\\theta_2]` indicate the temporal average parameters of the two networks in\nthe previous iteration :math:`(T-1)`, the initial temporal average parameters are\n:math:`E^{(0)}[\\theta_1]=\\theta_1,E^{(0)}[\\theta_2]=\\theta_2` and :math:`\\alpha` is the momentum.\n\nThese two networks cooperate with each other in three ways:\n\n- When running clustering algorithm, we average features produced by :math:`\\text{ensemble}(f_1)` and\n :math:`\\text{ensemble}(f_2)` instead of only considering one of them.\n- A **soft triplet loss** is optimized between :math:`f_1` and :math:`\\text{ensemble}(f_2)` and vice versa\n to force one network to learn from temporally average of another network.\n- A **cross entropy loss** is optimized between :math:`f_1` and :math:`\\text{ensemble}(f_2)` and vice versa\n to force one network to learn from temporally average of another network.\n\nThe above mentioned loss functions are listed below, more details can be found in training scripts.\n\n.. autoclass:: common.vision.models.reid.loss.SoftTripletLoss\n\n.. autoclass:: common.vision.models.reid.loss.CrossEntropyLoss\n\n\n=======================================\nOther Methods\n=======================================\n\n.. _AFN:\n\nAFN: Adaptive Feature Norm\n-----------------------------\n\n.. autoclass:: dalib.adaptation.afn.AdaptiveFeatureNorm\n\n.. autoclass:: dalib.adaptation.afn.Block\n\n.. autoclass:: dalib.adaptation.afn.ImageClassifier\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/docs/dalib/adaptation/statistics_matching.rst",
"content": "Statistics Matching\n===================================\n\n\n.. _DAN:\n\nDAN: Deep Adaptation Network\n-----------------------------\n\n.. autoclass:: dalib.adaptation.dan.MultipleKernelMaximumMeanDiscrepancy\n\n\n.. _JAN:\n\nJAN: Joint Adaptation Network\n------------------------------\n\n.. autoclass:: dalib.adaptation.jan.JointMultipleKernelMaximumMeanDiscrepancy\n\n\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/docs/dalib/benchmarks/image_classification.rst",
"content": "===============================\nImage Classification\n===============================\n\nWe provide benchmarks of different domain adaptation algorithms on `Digits`_, `Office-31`_ , `Office-Home`_, `VisDA-2017`_ and `DomainNet`_.\nThose domain adaptation algorithms includes:\n\n- :ref:`DANN`\n- :ref:`DAN`\n- :ref:`JAN`\n- :ref:`ADDA`\n- :ref:`CDAN`\n- :ref:`MCD`\n- :ref:`AFN`\n- :ref:`BSP`\n- :ref:`MDD`\n- :ref:`MCC`\n\n.. note::\n\n - ``Origin`` means the accuracy reported by the original paper.\n - ``Avg`` is the accuracy reported by Trasnfer-Learn.\n - ``Source Only`` refers to the model trained with data from the source domain.\n - ``Oracle`` refers to the model trained with data from the target domain.\n\n.. note::\n\n We found that the accuracies of adversarial methods (including DANN, ADDA, CDAN, MCD, BSP and MDD) are not stable\n even after the random seed is fixed, thus we repeat running adversarial methods on *Office-31* and *VisDA-2017*\n for three times and report their average accuracy.\n\n.. note::\n `ADDA` with gradient reverse layer is frequently benchmarked in the literature. Therefore, we implement this\n baseline and use `ADDAgrl` to denote it below.\n\n\n.. _Digits:\n\n--------------------------------\nDigits accuracy\n--------------------------------\n\n=========== =========== =========== ===========\nMethods SVHN2MNIST MNIST2USPS USPS2MNIST\nSource Only 74.1 82.1 74.6\nDANN 90.8 91.7 95.2\nDAN 82.1 86.0 89.5\nJAN 90.3 84.0 86.8\nADDA 93.3 94.5 98.3\nCDAN 93.8 96.0 97.7\nMCD 90.6 94.1 97.6\nAFN 88.2 88.6 97.2\nBSP+DANN 84.2 95.7 97.8\nMDD 88.4 94.8 97.8\nLDD 93.5 95.5 96.3\nMCC 76.6 95.1 94.6\n=========== =========== =========== ===========\n\n\n.. _Office-31:\n\n--------------------------------\nOffice-31 accuracy on ResNet-50\n--------------------------------\n\n=========== ====== ====== ====== ====== ====== ====== ====== ======\nMethods Origin Avg A → W D → W W → D A → D D → A W → A\nSource Only 76.1 79.5 75.8 95.5 99.0 79.3 63.6 63.8\nDANN 82.2 86.1 91.4 97.9 100.0 83.6 73.3 70.4\nDAN 80.4 83.7 84.2 98.4 100.0 87.3 66.9 65.2\nJAN 84.3 87.0 93.7 98.4 100.0 89.4 69.2 71.0\nADDA / 86.5 91.2 98.5 100.0 84.3 73.7 71.2\nADDAgrl / 87.3 94.6 97.5 99.7 90.0 69.6 72.5\nCDAN 87.7 87.7 93.8 98.5 100.0 89.9 73.4 70.4\nMCD / 85.4 90.4 98.5 100.0 87.3 68.3 67.6\nAFN 85.7 88.6 94.0 98.9 100.0 94.4 72.9 71.1\nBSP+DANN 87.7 87.8 92.7 97.9 100.0 88.2 74.1 73.8\nMDD 88.9 89.6 95.6 98.6 100.0 94.4 76.6 72.2\nMCC 89.4 89.6 94.1 98.4 99.8 95.6 75.5 74.2\n=========== ====== ====== ====== ====== ====== ====== ====== ======\n\n\n.. _Office-Home:\n\n-----------------------------------\nOffice-Home accuracy on ResNet-50\n-----------------------------------\n\n=========== ======= ======= ======= ======= ======= ======= ======= ======= ======= ======= ======= ======= ======= =======\nMethods Origin Avg Ar → Cl Ar → Pr Ar → Rw Cl → Ar Cl → Pr Cl → Rw Pr → Ar Pr → Cl Pr → Rw Rw → Ar Rw → Cl Rw → Pr\nSource Only 46.1 58.4 41.1 65.9 73.7 53.1 60.1 63.3 52.2 36.7 71.8 64.8 42.6 75.2\nDANN 57.6 65.2 53.8 62.6 74.0 55.8 67.3 67.3 55.8 55.1 77.9 71.1 60.7 81.1\nDAN 56.3 61.4 45.6 67.7 73.9 57.7 63.8 66.0 54.9 40.0 74.5 66.2 49.1 77.9\nJAN 58.3 65.9 50.8 71.9 76.5 60.6 68.3 68.7 60.5 49.6 76.9 71.0 55.9 80.5\nADDA / 62.5 47.4 63.9 72.6 53.1 62.6 64.3 56.0 49.1 76.3 68.1 56.5 80.3\nADDAgrl / 65.6 52.6 62.9 74.0 59.7 68.0 68.8 61.4 52.5 77.6 71.1 58.6 80.2\nCDAN 65.8 68.8 55.2 72.4 77.6 62.0 69.7 70.9 62.4 54.3 80.5 75.5 61.0 83.8\nMCD / 67.8 51.7 72.2 78.2 63.7 69.5 70.8 61.5 52.8 78.0 74.5 58.4 81.8\nAFN 67.3 68.2 53.2 72.7 76.8 65.0 71.3 72.3 65.0 51.4 77.9 72.3 57.8 82.4\nBSP+DANN 64.9 67.6 54.7 67.7 76.2 61.0 69.4 70.9 60.9 55.2 80.2 73.4 60.3 81.2\nMDD 68.1 69.7 56.2 75.4 79.6 63.5 72.1 73.8 62.5 54.8 79.9 73.5 60.9 84.5\nMCC / 72.4 58.4 79.6 83.0 67.5 77.0 78.5 66.6 54.8 81.8 74.4 61.4 85.6\n=========== ======= ======= ======= ======= ======= ======= ======= ======= ======= ======= ======= ======= ======= =======\n\n\n.. _VisDA-2017:\n\n-----------------------------------\nVisDA-2017 accuracy ResNet-101\n-----------------------------------\n\n.. note::\n - ``Origin`` means the accuracy reported by the original paper.\n - ``Mean`` refers to the accuracy average over ``classes``\n - ``Avg`` refers to accuracy average over ``samples``.\n\n=========== ========== ======= ======= ======= ======= ======= ======= ======= ======= ======= ======= ======= ======= ======= =======\nMethods Origin Mean plane bcycl bus car horse knife mcycl person plant sktbrd train truck Avg\nSource Only 52.4 51.7 63.6 35.3 50.6 78.2 74.6 18.7 82.1 16.0 84.2 35.5 77.4 4.7 56.9\nDANN 57.4 79.5 93.5 74.3 83.4 50.7 87.2 90.2 89.9 76.1 88.1 91.4 89.7 39.8 74.9\nDAN 61.1 66.4 89.2 37.2 77.7 61.8 81.7 64.3 90.6 61.4 79.9 37.7 88.1 27.4 67.2\nJAN / 73.4 96.3 66.0 82.0 44.1 86.4 70.3 87.9 74.6 83.0 64.6 84.5 41.3 70.3\nADDA / 79.3 93.6 70.8 83.2 63.5 90.6 93.2 89.0 75.3 88.4 79.3 87.4 37.2 76.4\nADDAgrl / 77.5 95.6 70.8 84.4 54.0 87.8 75.8 88.4 69.3 84.1 86.2 85.0 48.0 74.3\nCDAN / 80.1 94.0 69.2 78.9 57.0 89.8 94.9 91.9 80.3 86.8 84.9 85.0 48.5 76.5\nMCD 71.9 77.7 87.8 75.7 84.2 78.1 91.6 95.3 88.1 78.3 83.4 64.5 84.8 20.9 76.7\nAFN 76.1 75.0 95.6 56.2 81.3 69.8 93.0 81.0 93.4 74.1 91.7 55.0 90.6 18.1 74.4\nBSP+DANN 75.9 80.5 95.7 75.6 82.8 54.5 89.2 96.5 91.3 72.2 88.9 88.7 88.0 43.4 76.2\nMDD / 82.0 88.3 62.8 85.2 69.9 91.9 95.1 94.4 81.2 93.8 89.8 84.1 47.9 79.8\nMCC 78.8 83.6 95.3 85.8 77.1 68.0 93.9 92.9 84.5 79.5 93.6 93.7 85.3 53.8 80.4\n=========== ========== ======= ======= ======= ======= ======= ======= ======= ======= ======= ======= ======= ======= ======= =======\n\n.. _DomainNet:\n\n-----------------------------------\nDomainNet accuracy on ResNet-101\n-----------------------------------\n\n=========== ====== ====== ====== ====== ====== ====== ====== ====== ====== ====== ====== ====== ======\nMethods c->p c->r c->s p->c p->r p->s r->c r->p r->s s->c s->p s->r Avg\nSource Only 32.7 50.6 39.4 41.1 56.8 35.0 48.6 48.8 36.1 49.0 34.8 46.1 43.3\nDANN 37.9 54.3 44.4 41.7 55.6 36.8 50.7 50.8 40.1 55.0 45.0 54.5 47.2\nDAN 38.8 55.2 43.9 45.9 59.0 40.8 50.8 49.8 38.9 56.1 45.9 55.5 48.4\nJAN 40.5 56.7 45.1 47.2 59.9 43.0 54.2 52.6 41.9 56.6 46.2 55.5 50.0\nADDA 38.4 54.1 44.1 43.5 56.7 39.2 52.8 51.3 40.9 55.0 45.4 54.5 48.0\nCDAN 40.4 56.8 46.1 45.1 58.4 40.5 55.6 53.6 43.0 57.2 46.4 55.7 49.9\nMCD 37.5 52.9 44.0 44.6 54.5 41.6 52.0 51.5 39.7 55.5 44.6 52.0 47.5\nMDD 42.9 59.5 47.5 48.6 59.4 42.6 58.3 53.7 46.2 58.7 46.5 57.7 51.8\nMCC 37.7 55.7 42.6 45.4 59.8 39.9 54.4 53.1 37.0 58.1 46.3 56.2 48.9\n=========== ====== ====== ====== ====== ====== ====== ====== ====== ====== ====== ====== ====== ======\n\nOracle DomainNet accuracy on ResNet-101\n--------------------------------------------\n\n=========== ====== ====== ====== ====== ====== ======\nOracle clp\t inf\t pnt\t real skt Avg\n/ 78.2 40.7 71.6 83.8 70.6 69.0\n=========== ====== ====== ====== ====== ====== ======\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/docs/dalib/benchmarks/image_regression.rst",
"content": "===============================================\nImage Regression\n===============================================\n\nWe provide benchmarks of different domain adaptation algorithms on `dSprites`_ and `MPI3D`_ .\nThose domain adaptation algorithms includes:\n\n- :ref:`MDD`\n\n.. note::\n\n - ``Origin`` means the accuracy reported by the original paper.\n - ``Avg`` is the accuracy reported by Transfer-Learn.\n - ``Source Only`` refers to the model trained with data from the source domain.\n - ``Oracle`` refers to the model trained with data from the target domain.\n\n\n.. note::\n\n Labels are all normalized to [0, 1] to eliminate the effects of diverse scale in regression values.\n\n We repeat experiments on DD for three times and report the average error of the ``final`` epoch.\n\n.. _dSprites:\n\ndSprites error on ResNet-18\n---------------------------------\n=========== ====== ====== ====== ====== ====== ====== ======\nMethods Avg C → N C → S N → C N → S S → C S → N\nSource Only 0.157 0.232 0.271 0.081 0.220 0.038 0.092\nDD 0.057 0.047 0.080 0.030 0.095 0.053 0.037\n=========== ====== ====== ====== ====== ====== ====== ======\n\n\n.. _MPI3D:\n\nMPI3D error on ResNet-18\n---------------------------------\n=========== ======== ======== ======== ======== ======== ======== ========\nMethods Avg RL → RC RL → T RC → RL RC → T T → RL T → RC\nSource Only 0.176 0.232 0.271 0.081 0.220 0.038 0.092\nDD 0.030 0.086 0.029 0.057 0.189 0.131 0.087\n=========== ======== ======== ======== ======== ======== ======== ========\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/docs/dalib/benchmarks/keypoint_detection.rst",
"content": "==========================================\nKeypoint Detection\n==========================================\n\nWe provide benchmarks of different keypoint detection domain adaptation algorithms on as follows.\nThose domain adaptation algorithms includes:\n\n- :ref:`RegDA`\n\n.. _RHD2H3D:\n\n--------------------------------\nRHD->H3D accuracy on ResNet-101\n--------------------------------\n\n=========== ====== ====== ====== ========= ======\nMethods MCP PIP DIP Fingertip Avg\nSource Only 67.4\t64.2\t63.3\t54.8\t 61.8\nRegDA 79.6\t74.4\t71.2\t62.9\t 72.5\nOracle 97.7\t97.2\t95.7\t92.5\t 95.8\n=========== ====== ====== ====== ========= ======\n\n\n.. _Surreal2Human36M:\n\n-----------------------------------------\nSurreal->Human3.6M accuracy on ResNet-101\n-----------------------------------------\n\n=========== ======== ====== ====== ===== ===== ===== =====\nMethods Shoulder Elbow Wrist Hip Knee Ankle Avg\nSource Only 69.4 75.4 66.4 37.9 77.3 77.7 67.3\nRegDA 73.3 86.4 72.8 54.8 82.0 84.4 75.6\nOracle 95.3 91.8 86.9 95.6 94.1 93.6 92.9\n=========== ======== ====== ====== ===== ===== ===== =====\n\n\n.. _Surreal2LSP:\n\n-----------------------------------\nSurreal->LSP accuracy on ResNet-101\n-----------------------------------\n\n=========== ======== ====== ====== ===== ===== ===== =====\nMethods Shoulder Elbow Wrist Hip Knee Ankle Avg\nSource Only 51.5\t 65.0 62.9 68.0 68.7 67.4 63.9\nRegDA 62.7\t 76.7 71.1 81.0 80.3 75.3 74.6\n=========== ======== ====== ====== ===== ===== ===== =====\n\n\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/docs/dalib/benchmarks/multi_source_domain_adaptation.rst",
"content": "==========================================\nMulti Source Domain Adaptation\n==========================================\n\nWe provide benchmarks of different domain adaptation algorithms on `Office-Home`_ and `DomainNet`_.\nThose domain adaptation algorithms includes:\n\n- :ref:`MDD`\n\n.. note::\n\n - ``Origin`` means the accuracy reported by the original paper.\n - ``Avg`` is the accuracy reported by Transfer-Learn.\n - ``Source Only`` refers to the model trained with data from the source domain.\n - ``Oracle`` refers to the model trained with data from the target domain.\n\n.. _Office-Home:\n\nOffice-Home accuracy on ResNet-50\n---------------------------------\n\n=========== ====== ====== ====== ====== ======\nMethods Avg :Ar :Cl :Pr :Rw\nSource Only 69.5 67.3 50.8 78.8 81.0\nMDD 77.2 74.7 64.6 85.2 84.1\n=========== ====== ====== ====== ====== ======\n\n.. _DomainNet:\n\nDomainNet accuracy on ResNet-50\n-----------------------------------\n\n=========== ======= ======= ======= ======= ======= ======= ======= =======\nMethods Origin Avg :c :i :p :q :r :s\nSource Only 32.9 47.0 64.9 25.2 54.4 16.9 68.2 52.3\nMDD / 48.8 68.7 29.7 58.2 9.7\t 69.4 56.9\nOracle 63.0 69.1 78.2 40.7 71.6 69.7 83.8 70.6\n=========== ======= ======= ======= ======= ======= ======= ======= =======\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/docs/dalib/benchmarks/openset_domain_adaptation.rst",
"content": "==========================================\nOpenset Domain Adaptation\n==========================================\n\nWe provide benchmarks of different domain adaptation algorithms on `Office-31`_ , `Office-Home`_ and `VisDA-2017`_ as follows.\nThose domain adaptation algorithms includes:\n\n- :ref:`DANN`\n- :ref:`OSBP`\n\n\n.. note::\n - ``Source Only`` refers to the model trained with data from the source domain.\n - ``OS`` means normalized accuracy for all classes including the unknown as one class.\n - ``OS*`` means normalized accuracy only on known classes.\n - ``UNK`` is the accuracy of unknown samples.\n\n In ``OS``, the accuracy of each common class has the same contribution\n as the whole ``unknown`` class. Thus we report ``HOS`` used in `ROS (ECCV 2020)`_\n to better measure the abilities of different open set domain adaptation algorithms.\n\n .. math::\n \\textit{HOS} = 2 \\cdot \\dfrac{ \\textit{OS*} \\cdot \\textit{UNK} }{ \\textit{OS*} + \\textit{UNK} }\n\n The new evaluation metric is high only when both the ``OS*`` and ``UNK`` are high.\n\n.. note::\n We report the best ``HOS`` in all epochs.\n\n DANN (baseline model) will degrade performance as training progresses, thus the\n final ``HOS`` will be much lower than reported.\n In contrast, OSBP will improve performance stably.\n\n.. _Office-31:\n\nOffice-31 H-Score on ResNet-50\n---------------------------------\n\n.. note::\n We conduct `21` class classification experiments in this setting (follows :ref:`OSBP`).\n\n=========== ===== ====== ====== ====== ====== ====== ======\nMethods Avg A → W D → W W → D A → D D → A W → A\nSource Only 75.9 67.7 85.7 91.4 72.1 68.4 67.8\nDANN 80.4 81.4 89.1 92.0 82.5 66.7 70.4\nOSBP 87.8 90.7 96.4 97.5 88.7 77.0 76.7\n=========== ===== ====== ====== ====== ====== ====== ======\n\n.. _Office-Home:\n\n.. note::\n\n - ``Origin`` means the accuracy reported by the related paper.\n - ``Avg`` is the accuracy reported by DALIB.\n\nOffice-Home HOS on ResNet-50\n-----------------------------------\n\n=========== ====== ====== ======= ======= ======= ======= ======= ======= ======= ======= ======= ======= ======= =======\nMethods Origin Avg Ar → Cl Ar → Pr Ar → Rw Cl → Ar Cl → Pr Cl → Rw Pr → Ar Pr → Cl Pr → Rw Rw → Ar Rw → Cl Rw → Pr\nSource Only / 59.8 55.2 65.2 71.4 52.8 59.6 65.2 55.8 44.8 68.0 63.8 49.4 68.0\nDANN / 64.8 55.2 65.2 71.4 52.8 59.6 65.2 55.8 44.8 68.0 63.8 49.4 68.0\nOSBP 64.7 68.6 62.0 70.8 76.5 66.4 68.8 73.8 65.8 57.1 75.4 70.6 60.6 75.9\n=========== ====== ====== ======= ======= ======= ======= ======= ======= ======= ======= ======= ======= ======= =======\n\n.. _VisDA-2017:\n\nVisDA-2017 performance on ResNet-50\n-----------------------------------\n\n=========== ======== ====== ===== ==== ======= ======= ======= ======= ======= =======\nMethods HOS OS OS* UNK bcycl bus car mcycl train truck\nSource Only 42.6 37.6 34.7 55.1 42.6 6.4 30.5 67.1 84.0 0.2\nDANN 57.8 50.4 45.6 78.9 20.1\t71.4\t29.5\t74.4\t67.8\t10.4\nOSBP 75.4 67.3 62.9 94.3 63.7\t75.9\t49.6\t74.4\t86.2\t27.3\n=========== ======== ====== ===== ==== ======= ======= ======= ======= ======= =======\n\n\n.. _ROS (ECCV 2020): http://www.ecva.net/papers/eccv_2020/papers_ECCV/papers/123610409.pdf\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/docs/dalib/benchmarks/partial_domain_adaptation.rst",
"content": "==========================================\nPartial Domain Adaptation\n==========================================\n\nWe provide benchmarks of different domain adaptation algorithms on `Office-31`_ , `Office-Home`_, `VisDA-2017`_ and `ImageNet-Caltech`_.\nThose domain adaptation algorithms includes:\n\n- :ref:`DANN`\n- :ref:`PADA`\n- :ref:`IWAN`\n- :ref:`AFN`\n\n.. note::\n\n - ``Origin`` means the accuracy reported by the original paper.\n - ``Avg`` is the accuracy reported by Transfer-Learn.\n - ``Source Only`` refers to the model trained with data from the source domain.\n\n.. note::\n\n We found that the accuracies of adversarial methods are not stable even after the random seed is fixed, thus\n we repeat running adversarial methods on *Office-31* and *VisDA-2017* for three times and report their average accuracy.\n\n.. _Office-31:\n\nOffice-31 accuracy on ResNet-50\n---------------------------------\n\n=========== ====== ====== ====== ====== ====== ====== ====== ======\nMethods Origin Avg A → W D → W W → D A → D D → A W → A\nSource Only 75.6 90.1 78.3\t98.3\t99.4\t87.3\t88.5\t88.8\nDANN 43.4 82.4 60.0\t94.9\t98.1\t71.3\t84.9\t85.0\nPADA 92.7 93.8 86.4\t100.0\t100.0\t87.3\t93.8\t95.4\nIWAN 94.7 94.8 91.2\t99.7\t99.4\t89.8\t94.2\t94.3\nAFN / 93.1 87.8\t95.6\t99.4\t87.9\t93.9\t94.1\n=========== ====== ====== ====== ====== ====== ====== ====== ======\n\n.. _Office-Home:\n\nOffice-Home accuracy on ResNet-50\n-----------------------------------\n\n=========== ======= ======= ======= ======= ======= ======= ======= ======= ======= ======= ======= ======= ======= =======\nMethods Origin Avg Ar → Cl Ar → Pr Ar → Rw Cl → Ar Cl → Pr Cl → Rw Pr → Ar Pr → Cl Pr → Rw Rw → Ar Rw → Cl Rw → Pr\nSource Only 53.7 60.1 42.0 66.9 78.5 56.4 55.2 65.4 57.9 36.0 75.5 68.7 43.6 74.8\nDANN 47.4 57.0 46.2 59.3 76.9 47.0 47.4 56.4 51.6 38.8 72.1 68.0 46.1 74.2\nPADA 62.1 65.9 52.9 69.3 82.8 59.0 57.5 66.4 66.0 41.7 82.5 78.0 50.2 84.1\nIWAN 63.6 71.3 59.2 76.6 84.0 67.8 66.7 69.2 73.3 55.0 83.9 79.0 58.3 82.2\nAFN 71.8 72.6 59.2 76.7 82.8 72.5 74.5 76.8 72.5 56.7 80.8 77.0 60.5 81.6\n=========== ======= ======= ======= ======= ======= ======= ======= ======= ======= ======= ======= ======= ======= =======\n\n.. _ImageNet-Caltech:\n\nImageNet-Caltech accuracy on ResNet-50\n--------------------------------------\n\n=========== ======= ======= ==== ====\nMethods Origin Avg I→C C→I\nSource Only 68.9 73.3 71.8\t74.8\nDANN 60.8 73.1 71.6\t74.5\nPADA 72.8 79.2 79.2\t79.1\nIWAN 75.7 78.9 77.5 75.7\n=========== ======= ======= ==== ====\n\n.. _VisDA-2017:\n\nVisDA-2017 accuracy on ResNet-50\n-----------------------------------\n\nNote that `Origin` means the accuracy reported by the original paper,\n`Mean` refers to the accuracy average over classes, while `Avg` refers to accuracy average over samples.\n\n=========== ========== ======= ======= ======= ======= ======= ======= ======= =======\nMethods Origin Mean plane bcycl bus car horse knife Avg\nSource Only 45.3 50.9\t59.2\t31.3\t68.7\t73.2\t69.3\t3.4\t 60.0\nDANN 51.0 55.9\t88.4\t34.1\t72.1\t50.7\t61.9\t27.8\t57.1\nPADA 53.5 60.5\t89.4\t35.1\t72.5\t69.2\t86.7\t10.1\t66.8\nIWAN / 61.5\t89.2\t57.0\t61.5\t55.2\t80.1\t25.7\t66.8\nAFN 67.6 61.0 79.1\t62.7\t73.9\t49.6\t79.6\t21.0 64.1\n=========== ========== ======= ======= ======= ======= ======= ======= ======= =======\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/docs/dalib/benchmarks/re_identification.rst",
"content": "===========================================\nRe-Identification\n===========================================\n\nWe provide benchmarks of different domain adaptation algorithms. Currently three datasets are supported:\n`Market1501 `_,\n`DukeMTMC `_, `MSMT17 `_.\nThose domain adaptation algorithms includes:\n\n- :ref:`IBN`\n- :ref:`SPGAN`\n- :ref:`MMT`\n\n\nWe adopt cross dataset setting (another one is cross camera setting). The model is first trained on source dataset, then we evaluate it on target dataset and report `mAP` (mean average precision) on target dataset.\n\nFor a fair comparison, our model is trained with standard cross entropy loss and triplet loss. We adopt modified resnet architecture from `Mutual Mean-Teaching: Pseudo Label Refinery for Unsupervised Domain Adaptation on Person Re-identification (ICLR 2020) `_.\n\nAs we are given unlabeled samples from target domain, we can utilize clustering algorithms to produce pseudo labels on target domain and then use them as supervision signals to perform self-training. This simple method turns out to be a strong baseline. We use ``Baseline_Cluster`` to represent this baseline in our results.\n\n.. note::\n\n - ``Avg`` means the average mAP across these tasks reported by Transfer-Learn.\n\n-----------------------------------\nCross dataset mAP on ResNet-50\n-----------------------------------\n========================= ======= ============= ============= ============= ============= =========== ===========\nMethods Avg Market2Duke Duke2Market Market2MSMT MSMT2Market Duke2MSMT MSMT2Duke\nBaseline 27.1 32.4 31.4 8.2 36.7 11.0 43.1\nIBN 30.0 35.2 36.5 11.3 38.7 14.1 44.3\nSPGAN 30.7 34.4 35.4 14.1 40.2 16.1 43.8\nBaseline_Cluster(kmeans) 45.1 52.8 59.5 19.0 62.6 20.3 56.2\nBaseline_Cluster(dbscan) 54.9 62.5 73.5 25.2 77.9 25.3 65.0\nMMT(kmeans) 55.4 63.7 72.5 26.2 75.8 28.0 66.1\nMMT(dbscan) 60.0 68.2 80.0 28.2 82.5 31.2 70.0\n========================= ======= ============= ============= ============= ============= =========== ===========\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/docs/dalib/benchmarks/semantic_segmentation.rst",
"content": "==========================================\nSemantic Segmentation\n==========================================\n\nWe provide benchmarks of different segmentation domain adaptation algorithms on `GTA5->Cityscapes`_ and `Synthia->Cityscapes`_ as follows.\nThose domain adaptation algorithms includes:\n\n- :ref:`CycleGAN`\n- :ref:`Cycada`\n- :ref:`ADVENT`\n- :ref:`FDA`\n\n\n.. note::\n\n - ``Origin`` means the accuracy reported by the original paper.\n - ``mIoU`` is the mean IoU reported by Transfer-Learn.\n - ``Src Only`` refers to the model trained with data from the source domain.\n - ``Oracle`` refers to the model trained with data from the target domain.\n\n.. _GTA5->Cityscapes:\n\nGTA5->Cityscapes mIoU on deeplabv2 (ResNet-101)\n-----------------------------------------------\n\n=========== ====== ======= ======= ======== ======== ======= ======= ======= ======= ======= =======\nMethods Origin mIoU road sidewalk building wall fence pole light sign veg\nSrc Only 27.1 37.3 66.5 17.4 73.3 13.4 21.5 22.8 30.1 17.1 82.2\nCycleGAN / 47.0\t88.4\t41.9\t 83.6 34.4\t23.9\t32.9\t35.5\t26.0\t83.1\nCycada 42.7\t47.4\t87.3\t35.7\t 83.7 31.3\t24.0\t32.2\t35.8\t30.3\t82.7\nADVENT 43.8 43.8 89.3 33.9 80.3 24.0 25.2 27.8 36.7 18.2 84.3\nFDA 44.6 45.6 85.5 31.7 81.8 27.1 24.9 28.9 38.1 23.2 83.7\nOracle 65.1 70.5 97.4 79.7 90.1 53.0 50.0 48.0 55.5 67.2 90.2\n=========== ====== ======= ======= ======== ======== ======= ======= ======= ======= ======= =======\n\n\n=========== ======= ======= ======= ======= ======= ======= ======= ======= ======= =======\nMethods terrain sky person rider car truck bus train mbike bike\nSrc Only 7.1 73.6 57.4 28.4 78.6 36.1 13.4 1.5 31.9 36.2\nCycleGAN 36.8 82.3 59.9 27.0 83.4 31.6 42.3 11.0 28.2 40.5\nCycada 32.0 85.7 60.8 31.5 85.6 39.8 43.3 5.4 29.5 44.6\nADVENT 33.9 81.3 59.8 28.4 84.3 34.1 44.4 0.1 33.2 12.9\nFDA 40.3 80.6 60.5 30.3 79.1 32.8 45.1 5.0\t 32.4 35.2\nOracle 60.0 93.0 72.7 55.2 92.7 76.5 78.5 56.0 54.6 68.8\n=========== ======= ======= ======= ======= ======= ======= ======= ======= ======= =======\n\n.. _Synthia->Cityscapes:\n\nSynthia->Cityscapes mIoU on deeplabv2 (ResNet-101)\n--------------------------------------------------\n\n========= ====== ==== ==== ======== ======== ===== ==== ==== ==== ====== ===== ==== ==== ===== ====\nMethods Origin mIoU road sidewalk building light sign veg sky person rider car bus mbike bike\nSrc Only 22.1 41.5 59.6 21.1 77.4 7.7 17.6 78.0 84.5 53.2 16.9 65.9 24.9 8.5 24.8\nADVENT 47.6 47.9 88.3 44.9 80.5 4.5 9.1 81.3 86.2 52.9 21.0 82.0 30.3 11.9 30.2\nFDA / 43.9 62.5 23.7 78.5 9.4 15.7 78.3 81.1 52.3 18.7 79.8 32.5 8.7 29.6\nOracle 71.7 76.6 97.4 79.7 90.1 55.5 67.2 90.2 93.0 72.7 55.2 92.7 78.5 54.6 68.8\n========= ====== ==== ==== ======== ======== ===== ==== ==== ==== ====== ===== ==== ==== ===== ====\n\n\nCityscapes->Foggy Cityscapes mIoU on deeplabv2 (ResNet-101)\n--------------------------------------------------------------\n\n=========== ======= ======= ======== ======== ======= ======= ======= ======= ======= =======\nMethods mIoU road sidewalk building wall fence pole light sign veg\nSrc Only 51.2\t95.3\t70.2\t 64.1 31.9\t35.2\t30.7\t33.3\t51.1\t42.3\nCycleGAN 66.0\t97.1\t77.6 84.3 42.7\t46.3\t42.8\t47.5\t61.0\t84.0\nCycada 63.3\t96.8\t75.5 79.1 38.0\t40.3\t42.1\t48.2\t61.2\t76.9\nADVENT 61.8\t96.8\t75.1\t 76.4 46.2 42.6\t39.3\t43.6\t58.9\t74.3\nFDA 61.9\t96.9\t77.2 75.3 46.5\t42.0\t39.8\t47.1\t61.0\t72.7\nOracle 66.9\t97.4\t78.6 88.1 50.7\t50.5\t46.2\t51.3\t64.4\t88.1\n=========== ======= ======= ======== ======== ======= ======= ======= ======= ======= =======\n\n\n=========== ======= ======= ======= ======= ======= ======= ======= ======= ======= =======\nMethods terrain sky person rider car truck bus train mbike bike\nSrc Only 44.0 32.1 64.4 47.0 86.0 64.4 56.4 21.1 43.1 60.8\nCycleGAN 55.2 83.4 69.4 51.8 90.7 73.7 76.2 54.2 50.7 65.6\nCycada 52.1 77.6 68.6 51.7 90.4 71.7 70.4 43.3 52.6 65.7\nADVENT 50.1 75.9 67.3 51.0 89.4 70.5 64.7 39.9 47.9 65.0\nFDA 54.6 63.8 68.4 50.1 90.1 72.8 68.0 35.5 50.8 64.2\nOracle 55.3 87.4 70.9 52.7 91.6 72.4 73.2 31.8 52.2 67.4\n=========== ======= ======= ======= ======= ======= ======= ======= ======= ======= =======\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/docs/dalib/modules.rst",
"content": "Common Adaptation Modules\n===========================\n\n\nDomain Discriminator\n-------------------------------\n.. autoclass:: dalib.modules.domain_discriminator.DomainDiscriminator\n :members:\n\nGRL\n------------------------\n.. autoclass:: dalib.modules.grl.WarmStartGradientReverseLayer\n :members:\n\nKernels\n------------------------\n.. autoclass:: dalib.modules.kernels.GaussianKernel\n\n\nEntropy\n------------------------\n.. autofunction:: dalib.modules.entropy.entropy\n\n\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/docs/dglib/benchmarks/image_classification.rst",
"content": "===============================\nImage Classification\n===============================\n\nWe provide benchmarks of different domain generalization algorithms on `PACS`_, `Office-Home`_,\n`iWildCam-Wilds`_, `Camelyon17-Wilds`_, `FMoW-Wilds`_.\nThose domain generalization algorithms includes:\n\n- :ref:`IBN`\n- :ref:`MIXSTYLE`\n- :ref:`MLDG`\n- :ref:`IRM`\n- :ref:`VREX`\n- :ref:`GroupDRO`\n- :ref:`CORAL`\n\n.. note::\n\n `DomainBed `_ proposed three model selection methods, our hyper\n parameter is selected based on model's performance on `training-domain validation set` (first rule proposed).\n Concretely, we select model with highest accuracy on `training-domain validation set` during whole training\n process and use selected checkpoint to test on target domain.\n\n.. note::\n\n Different from `DomainBed `_, we do not freeze `BatchNorm2d` layers\n and do not insert additional `Dropout` layer except for `PACS` dataset. Besides, we use `SGD` with momentum as\n default optimizer and find it usually achieves better results than `Adam`. During training, a cosine learning rate\n decay strategy is used.\n\n.. note::\n - ``ERM`` refers to the model trained with `ERM `_, which is a strong baseline.\n - ``Avg`` is the average accuracy.\n - ``Acc1`` is the top-1 accuracy on `OOD` test set for Wilds datasets.\n\n.. _PACS:\n\n-----------------------------------\nPACS accuracy on ResNet-50\n-----------------------------------\n\n======== ===== ===== ===== ===== =====\nMethods avg A C P S\nERM 86.4 88.5 78.4 97.2 81.4\nIBN 87.8 88.2 84.5 97.1 81.4\nMixStyle 87.4 87.8 82.3 95.0 84.5\nMLDG 87.2 88.2 81.4 96.6 82.5\nIRM 86.9 88.0 82.5 98.0 79.0\nVREx 87.0 87.2 82.3 97.4 81.0\nGroupDRO 87.3 88.9 81.7 97.8 80.8\nCORAL 86.4 89.1 80.0 97.4 79.1\n======== ===== ===== ===== ===== =====\n\n.. _Office-Home:\n\n-----------------------------------\nOffice-Home accuracy on ResNet-50\n-----------------------------------\n\n======== ===== ===== ===== ===== =====\nMethods avg A C P R\nERM 70.8 68.3 55.9 78.9 80.0\nIBN 69.9 67.4 55.2 77.3 79.6\nMixStyle 71.7 66.8 58.1 78.0 79.9\nMLDG 70.3 65.9 57.6 78.2 79.6\nIRM 70.3 66.7 54.8 78.6 80.9\nVREx 70.2 66.9 54.9 78.2 80.9\nGroupDRO 70.0 66.7 55.2 78.8 79.9\nCORAL 70.9 68.3 55.4 78.8 81.0\n======== ===== ===== ===== ===== =====\n\n.. _iWildCam-Wilds:\n\n----------------------------------------\niWildCam-Wilds accuracy on ResNet-50\n----------------------------------------\n\n======== ======\nMethods acc1\nERM 75.4\nIBN 77.3\nMixStyle 71.0\nIRM 75.5\nVREx 71.5\nGroupDRO 28.0\nCORAL 71.0\n======== ======\n\n.. _Camelyon17-Wilds:\n\n----------------------------------------\nCamelyon17-Wilds accuracy on ResNet-50\n----------------------------------------\n\n======== ======\nMethods acc1\nERM 94.6\nIBN 96.1\nMixStyle 94.2\nMLDG 91.2\nIRM 94.9\nVREx 88.2\nGroupDRO 93.1\nCORAL 90.6\n======== ======\n\n.. _FMoW-Wilds:\n\n--------------------------------------\nFMoW-Wilds accuracy on DenseNet-121\n--------------------------------------\n\n======== ======\nMethods acc1\nERM 53.0\nMLDG 47.4\nIRM 48.1\nVREx 50.4\nGroupDRO 47.5\nCORAL 50.0\n======== ======\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/docs/dglib/benchmarks/re_identification.rst",
"content": "===============================\nRe-Identification\n===============================\n\nWe provide benchmarks of different domain generalization algorithms. Currently three datasets are supported:\n`Market1501 `_,\n`DukeMTMC `_, `MSMT17 `_.\nThose domain generalization algorithms includes:\n\n- :ref:`IBN`\n- :ref:`MIXSTYLE`\n\n.. note::\n\n We adopt cross dataset setting (another one is cross camera setting). The model is first trained on source dataset,\n then we evaluate it on target dataset and report `mAP` (mean average precision) on target dataset.\n\n.. note::\n For a fair comparison, our model is trained with standard cross entropy loss and triplet loss. We adopt modified\n resnet architecture from `Mutual Mean-Teaching: Pseudo Label Refinery for Unsupervised\n Domain Adaptation on Person Re-identification (ICLR 2020) `_.\n\n-----------------------------------\nCross dataset mAP on ResNet-50\n-----------------------------------\n======== ======= ============= ============= ============= ============= =========== ===========\nMethods Avg Market2Duke Duke2Market Market2MSMT MSMT2Market Duke2MSMT MSMT2Duke\nBaseline 23.5 25.6 29.6 6.3 31.7 10.1 37.8\nIBN 27.0 31.5 33.3 10.4 33.6 13.7 40.0\nMixStyle 25.5 27.2 31.6 8.2 33.9 12.4 39.9\n======== ======= ============= ============= ============= ============= =========== ===========\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/docs/dglib/generalization/architecture_design.rst",
"content": "=======================================\nArchitecture Design\n=======================================\n\n.. _IBN:\n\nIBN-Net: Instance-Batch Normalization Network\n------------------------------------------------\n\n.. autoclass:: common.vision.models.ibn.IBN\n\n.. autoclass:: common.vision.models.ibn.ResNet_IBN\n :members:\n\n.. automodule:: common.vision.models.ibn\n :members:\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/docs/dglib/generalization/data_manipulation.rst",
"content": "=======================================\nData Manipulation\n=======================================\n\n.. _MIXSTYLE:\n\nDomain Generalization with MixStyle (MixStyle)\n-------------------------------------------------\n\n.. autoclass:: dglib.generalization.mixstyle.models.mixstyle.MixStyle\n\n.. note::\n MixStyle is only activated during `training` stage, with some probability :math:`p`.\n\n.. automodule:: dglib.generalization.mixstyle.models.resnet\n :members:\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/docs/dglib/generalization/learning_strategy.rst",
"content": "=======================================\nLearning Strategy\n=======================================\n\n.. _GroupDRO:\n\nGroup Distributionally robust optimization (GroupDRO)\n------------------------------------------------------\n\n.. autoclass:: dglib.generalization.groupdro.AutomaticUpdateDomainWeightModule\n :members:\n\n.. _IRM:\n\nInvariant Risk Minimization (IRM)\n----------------------------------\n\n.. autoclass:: dglib.generalization.irm.InvariancePenaltyLoss\n\n.. _MLDG:\n\nMeta Learning for Domain Generalization (MLDG)\n------------------------------------------------\n`Learning to Generalize: Meta-Learning for Domain Generalization (AAAI 2018) `_\n\nConsider there are :math:`S` source domains, at each learning iteration MLDG splits the\noriginal :math:`S` source domains into `meta-train` domains :math:`S_1` and `meta-test` domains :math:`S_2`. The `inner`\nobjective is cross entropy loss on `meta-train` domains :math:`S_1`. The `outer` (meta-optimization)\nobjective contains two terms. The first one (which is the same as inner objective) is cross entropy loss on `meta-train`\ndomains :math:`S_1` with current model parameters :math:`\\theta`\n\n.. math::\n \\mathbb{E}_{(x,y) \\in S_1} l(f(\\theta, x), y)\nwhere :math:`l` denotes cross entropy loss, :math:`f(\\theta, x)` denotes predictions from model.\nThe second term is cross entropy loss on `meta-test` domains :math:`S_2` with inner optimized model parameters\n:math:`\\theta_{updated}`\n\n.. math::\n \\mathbb{E}_{(x,y) \\in S_2} l(f(\\theta_{updated}, x), y)\n\nIn this way, MLDG simulates train/test domain shift during training by synthesizing virtual testing domains within\neach mini-batch. The outer objective forces that steps to improve training domain performance should\nalso improve testing domain performance.\n\n.. note::\n Because we need to compute second-order gradient, this full optimization process may take a long time and have\n heavy budget on GPU resource. A first order approximation implementation can be found at\n `DomainBed `_.\n\n.. _VREX:\n\nVariance Risk Extrapolation (VREx)\n------------------------------------------------\n`Out-of-Distribution Generalization via Risk Extrapolation (ICML 2021) `_\n\nVREx shows that reducing differences in risk across training domains can reduce a model’s sensitivity to a wide range\nof extreme distributional shifts. Consider there are :math:`S` source domains. At each learning iteration VREx first\ncomputes cross entropy loss on each source domain separately, producing a loss vector :math:`l \\in R^S`. The `ERM`\n(vanilla cross entropy) loss can be computed as\n\n.. math::\n l_{\\text{ERM}} = \\frac{1}{S}\\sum_{i=1}^S l_i\n\nAnd the penalty loss is\n\n.. math::\n penalty = \\frac{1}{S} \\sum_{i=1}^S {(l_i - l_{\\text{ERM}})}^2\n\nThe final objective is then\n\n.. math::\n objective = l_{\\text{ERM}} + \\beta * penalty\n\nwhere :math:`\\beta` is the trade off hyper parameter.\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/docs/dglib/generalization/representation_learning.rst",
"content": "=======================================\nRepresentation Learning\n=======================================\n\n.. _CORAL:\n\nCorrelation Alignment for Deep Domain Adaptation (Deep CORAL)\n--------------------------------------------------------------\n\n.. autoclass:: dglib.generalization.coral.CorrelationAlignmentLoss\n\n.. note::\n Under domain generalization setting, we can't access to samples from target domain during training. Therefore,\n two different domains from source domains are treated as source domain and target domain respectively to calculate\n `Correlation Alignment Loss`.\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/docs/dglib/modules.rst",
"content": "Common Generalization Modules\n==============================\n\nClassifier\n-----------------------------\n\n.. autoclass:: dglib.modules.classifier.ImageClassifier\n\nSampler\n-----------------------------\n\n.. autoclass:: dglib.modules.sampler.DefaultSampler\n\n.. autoclass:: dglib.modules.sampler.RandomDomainSampler\n\n.. autoclass:: dglib.generalization.mixstyle.sampler.RandomDomainMultiInstanceSampler\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/docs/get_started/faq.rst",
"content": "*************\nFAQ\n*************\n\n1. How to build the doc?\n=========================\n\nSometimes, the online doc is not consistent with the latest code. In this case, you can build the doc by yourself.\nFirst, you need to install sphinx, which is used to build doc.\n\n.. code-block:: shell\n\n pip install -U sphinx\n\nSecond, you need to download a pytorch-style `html.zip `_\ninto directory ``docs/build`` and unzip it. Some browsers may identify it as malicious file. You can safely ignore those warnings.\n\nThen, in the directory ``docs`` run the following command\n\n.. code-block:: shell\n\n make html\n\nAlso, warnings during ``make`` process doesn't matter and can be ignored.\n\nFinally, you can open the docs in ``docs/build/html/index.html``\n\n2. How to customize model?\n===============================\nA typical classifier has 3 components.\n\n- backbone: usually ``ResNet`` network to extract feature maps.\n- bottleneck: random initialized bottleneck layers between ``backbone`` and ``head`` to increase model capacity\n- head: classifier head which outputs final predictions.\n\nThe way we construct an image classifier is as follows.\n\n.. code-block:: python\n\n import common.modules.Classifier\n\n # define classifier\n classifier = Classifier(backbone=your_backbone, num_classes=num_classes, bottleneck=your_bottleneck,\n bottleneck_dim=your_bottleneck_dim, finetune=True)\n\nBelow are some examples on how to change these components.\n\n2.1 Define a new backbone\n'''''''''''''''''''''''''''\n.. code-block:: python\n\n import torch.nn as nn\n\n\n class FeatureExtractor(nn.Module):\n\n def __init__(self):\n pass\n\n def forward(self, x):\n pass\n\n @property\n def out_features(self):\n pass\n\n your_backbone = FeatureExtractor()\n\nNotice your ``FeatureExtractor`` should implement `out_features` method, which returns the dimension of output features.\n\n2.2 Define a new bottleneck\n'''''''''''''''''''''''''''\n.. code-block:: python\n\n import torch.nn as nn\n\n your_bottleneck = nn.Sequential(\n nn.AdaptiveAvgPool2d(output_size=(1, 1)), # for 2d images only\n nn.Flatten(),\n nn.Linear(512, 512),\n nn.BatchNorm1d(512),\n nn.ReLU()\n )\n\n\n2.3 Override get_parameters method\n'''''''''''''''''''''''''''''''''''\nIn ``DA`` settings, we usually use smaller learning rates for ``backbone`` parameters, typically ``0.1x`` compared to other layers.\nIf you want to use other strategies such as different momentum, different weight_decay for different parts, you should inherit `Classifier` and\noverride ``get_parameters`` method.\n\nBelow we use same learning rate for ``backbone``, ``bottleneck`` and ``head``.\nAnd momentum factor of ``0.9`` is used for ``bottleneck`` and ``head``.\n\n.. code-block:: python\n\n import torch.nn as nn\n import common.modules.Classifier\n\n class ImageClassifier(Classifier):\n\n def __init__(self):\n pass\n\n def get_parameters(self):\n params = [\n {\"params\": self.backbone.parameters()},\n {\"params\": self.bottleneck.parameters(), \"momentum\": 0.9},\n {\"params\": self.head.parameters(), \"momentum\": 0.9},\n ]\n return params\n\n3. How to customize datasets?\n==================================\n\nIf you want to implement your own vision datasets, you can use ``torchvision.datasets.VisionDataset``\nor ``common.vision.datasets.ImageList``.\n\nBefore using ``ImageList``, you need to prepare a txt file ``dog_cat.txt``.\nIn this file, each line should has the following format::\n\n path/to/dog/0.jpg 0\n path/to/cat/1.jpg 1\n\nwhere the first part is a relative file path, and the second part is an integer label.\n\nThe way to construct an dog-cat dataset is\n\n.. code-block:: python\n\n import torch.nn as nn\n import common.vision.datasets.ImageList\n\n dataset = ImageList(root=\"your_root\", classes=(\"dog\", \"cat\"), data_list_file=\"dog_cat.txt\")\n\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/docs/get_started/installing.rst",
"content": "\n************\nInstallation\n************\n\nVia Git or Download\n===================\n\nFor flexible use and modification,\nplease git clone `Transfer Learning Library `_.\n\nCheck that you have install all the dependency.\n\n.. code-block::\n\n pip install -r requirements.txt\n\nIt's recommended to use pytorch==1.7.1 and torchvision==0.8.2 in order to better reproduce the benchmark results.\n\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/docs/get_started/introduction.rst",
"content": "************\nIntroduction\n************\n\nCurrently, **Trans-Learn** provide algorithms for **domain adaptation (DA)**, **task adaptation (TA)**, and **domain generalization (DG)**.\nWe implement training codes (in directory ``examples``) in a unified way, which allows quantitative, fair, and reproducible comparisons between different algorithms.\n\n\nComparison with Existing Library\n===================================\n\nThere exist several open-source transfer learning libraries,\nbut most of them only focus on domain adaptation, especially statistics matching and adversarial adaptation, while ignoring other adaptation methods.\n\nIn contrast, Transfer-Learn covers more types of adaptation algorithms.\n\n- For more convenient algorithm selection, we provide the most comprehensive benchmark among all those libraries.\n- For faster algorithm reproduction, we provide training scripts for all the results in the benchmark.\n- For better application of the library, we provide detailed documentation and examples in many downstream tasks.\n\nAs far as we know, Transfer-Learn is currently the only one open source transfer learning library that can meet all the above requirements.\n\n=========================== ================= ==================== =============== ============== ======================\nFeatures Transfer-Learn transferlearning.xyz salad Dassl deep transfer learning\nTask Adaptation :math:`\\surd` :math:`\\times` :math:`\\times` :math:`\\times` :math:`\\times`\nStatistics Matching :math:`\\surd` :math:`\\surd` :math:`\\surd` :math:`\\surd` :math:`\\surd`\nAdversarial Adaptation :math:`\\surd` :math:`\\surd` :math:`\\surd` :math:`\\surd` :math:`\\surd`\nDisparity Discrepancy :math:`\\surd` :math:`\\times` :math:`\\times` :math:`\\times` :math:`\\times`\nTranslation :math:`\\surd` :math:`\\times` :math:`\\times` :math:`\\times` :math:`\\times`\nBenchmark :math:`\\surd` :math:`\\surd` :math:`\\surd` :math:`\\times` :math:`\\surd`\nReproducible Scripts :math:`\\surd` :math:`\\surd` :math:`\\surd` :math:`\\times` :math:`\\times`\nDetailed Documentation :math:`\\surd` :math:`\\times` :math:`\\surd` :math:`\\times` :math:`\\times`\nApplication :math:`\\surd` :math:`\\times` :math:`\\times` :math:`\\times` :math:`\\surd`\n=========================== ================= ==================== =============== ============== ======================\n\nLibrary Usage\n=============\nHere we give a short description on how to use Transfer-Learn using `DANN `_ as an instance.\n\nIn the original implementation of DANN, the domain adversarial loss, domain discriminator, feature extractor, and classifier are tightly coupled together in one *nn.Module*, which cause the difficulty of reuse, e.g., the entire algorithm needs re-implementation when the input data is changed from image to text. Yet in this case, the domain adversarial loss and the domain discriminator remains unchanged and can be reused.\n\nThus, in Transfer-Learn, models and loss functions are decoupled.\nWhen using DANN, users need to initialize a domain discriminator and pass it to the domain adversarial loss module, and then use this module the same way as the cross-entropy loss module defined in the Pytorch,\n\n\n.. code:: python\n\n >>> # define the domain discriminator\n >>> from dalib.modules.domain_discriminator import DomainDiscriminator\n >>> discriminator = DomainDiscriminator(in_feature=1024, hidden_size=1024)\n >>> # define the domain adversarial loss module\n >>> from dalib.adptation.dann import DomainAdversarialLoss\n >>> dann = DomainAdversarialLoss(discriminator, reduction='mean')\n >>> # features from the source and target domain\n >>> f_s, f_t = torch.randn(20, 1024), torch.randn(20, 1024)\n >>> # calculate the final loss\n >>> loss = dann(f_s, f_t)\n\n\nTransfer-Learn provide similar API for other transfer learning algorithms. Detailed usages of these algorithms can be found at the documentation.\n\nLibrary philosophy\n=====================\n\nTransfer-Learn is designed to be *extensible* by researchers and *simple* for practitioners.\nCurrently, there are mainly two types of algorithm implementations.\n\n- One is to encapsulate each algorithm into a Trainer, whose typical representative is *Pytorch-lighting*. The user only needs to feed the training data to it, and does not need to care about the specific training process.\n- Another strategy is to encapsulate the core loss function in each algorithm, and the user needs to implement the complete training process themselves. A typical representative is *Pytorch*.\n\nAlthough the former method is easier to use, it is less extensible. Since it's often necessary to adjust training process in different transfer learning scenarios, Transfer-Learn adopts the latter method for better extensibility.\n\nFollowing Pytorch, we provide training examples of different transfer algorithms in different scenarios, which allows users to quickly adapt to Transfer-Learn as long as they have learned Pytorch before.\n\nWe try our best to make Transfer-Learn easy to start with, e.g., we support the automatic download of most common transfer learning datasets so that users do not need to spend time on data preprocessing.\n\n\nHow to Choose Algorithms?\n======================================\n\nAlthough there are many methods proposed for transfer learning each year, unfortunately, no method is universal.\nEach method has its advantages and disadvantages and applicable scenarios.\n\nTherefore, we provide benchmarks under different settings. They can serve as a reference when you choose an algorithm.\n\nYou are suggested to read related papers if you want to have a deeper understanding of an algorithm.\n\nAfter completing the above things, trying is still the most important.\nIn addition to the final accuracy, you need to pay attention to the output of the program throughout the training process.\nYou can also use the tools in ``common.utils.analysis`` to visualize the results.\n\n\n\n\n\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/docs/get_started/quickstart.rst",
"content": "************\nQuick Start\n************\n\nIn this section, we will use `DANN `_ as an example,\nand show you how to reproduce the benchmark results.\n\nBefore going deeper, please **make sure** you have installed all the dependency.\n\nStep1: Find it\n===================\nOur code for domain adaptation is in the directory ``examples/domain_adaptation``.\n\nDANN is designed for close set domain adaptation tasks. You can find the training code in\n``examples/domain_adaptation/image_classification``. This directory contains implementations for other algorithms such as ``CDAN``, ``MDD``.\nFor now, you only need to care about two files: ``dann.py`` and ``dann.sh``.\n\nStep2: Run it\n===================\n``dann.py`` is an executable python file. And you can find all the necessary running scripts to\nreproduce the benchmarks with specified hyper-parameters in ``dann.sh``.\n\nFor instance, running the following command will start training on ``Office-31`` dataset.\n\n.. code-block:: shell\n\n CUDA_VISIBLE_DEVICES=0 python dann.py data/office31 -d Office31 -s A -t W -a resnet50 --epochs 20 --seed 1 --log logs/dann/Office31_A2W\n\nNote that ``-s`` specifies the source domain, ``-t`` specifies the target domain,\nand ``--log`` specifies where to store results.\n\nAfter running the above command, it will download ``Office-31`` datasets from the Internet if it's the\nfirst time you run the code. Directory that stores datasets will be named as\n``examples/domain_adaptation/image_classification/data/``.\n\nStep3: Analysis it\n===================\nIf everything works fine, you will see results in following format::\n\n Epoch: [1][ 900/1000]\tTime 0.60 ( 0.69)\tData 0.22 ( 0.31)\tLoss 0.74 ( 0.85)\tCls Acc 96.9 (95.1)\tDomain Acc 64.1 (62.6)\n\n``Trans-Learn`` provides detailed statistics to monitor during training.\n\nHere your algorithm is running in epoch 2 (index starts from 0), has been trained for 900 iterations.\nThe loss comes down to 0.74, on source domain your classifier achieves 96.9% accuracy\nand your domain discriminator has an accuracy of 64.1%.\nDifferent algorithms may show you different statistics.\n\nYou can also watch these results in a log file.\nThey are located in directory ``logs/dann/Office31_A2W/log.txt``.\n\nDuring training, ``latest`` and ``best`` model checkpoints will be saved in directory ``logs/dann/Office31_A2W/checkpoints``.\nResuming from checkpoint is also **supported**.\n\nAfter training, you can test your algorithm's performance by passing in ``--phase test``.\n\n.. code-block:: shell\n\n CUDA_VISIBLE_DEVICES=0 python dann.py data/office31 -d Office31 -s A -t W -a resnet50 --epochs 20 --seed 1 --log logs/dann/Office31_A2W --phase test\n\nIn next section, we will introduce how to visualize the results in details."
},
{
"path": "domain_generalization/Transfer-Learning-Library/docs/get_started/visualization.rst",
"content": "*************\nVisualization\n*************\n\nHow to visualize the representations using t-SNE?\n===================================================================\n\nMany algorithms aim at aligning feature representations between ``source`` and ``target`` domain.\nThrough visualization, you can find and analysis the mis-alignment between different domains. \n\nAfter training `DANN`, in directory ``examples/domain_adaptation/image_classification``, run the following command\n\n.. code-block:: shell\n\n CUDA_VISIBLE_DEVICES=0 python dann.py data/office31 -d Office31 -s A -t W -a resnet50 --epochs 20 --seed 1 --log logs/dann/Office31_A2W --phase analysis\n\nIt may take a while, then in directory ``logs/dann/Office31_A2W/visualize``, you can find\n``TSNE.png``.\n\n.. figure:: ../_static/images/resnet_A2W.png\n :width: 300\n\n t-SNE of representations from ResNet50 trained on source domain.\n\n.. figure:: ../_static/images/dann_A2W.png\n :width: 300\n\n t-SNE of representations from DANN.\n\n\nHow to visualize the segmentation predictions?\n===================================================================\nFor each segmentation algorithms, we've implemented the visualization code. All you need to do is set ``--debug`` during training.\nFor instance, in the directory ``examples/domain_adaptation/semantic_segmentation``,\n\n.. code-block:: shell\n\n CUDA_VISIBLE_DEVICES=0 python source_only.py data/GTA5 data/Cityscapes -s GTA5 -t Cityscapes --log logs/src_only/gtav2cityscapes --debug\n\nThen you can find visualization images in directory ``logs/src_only/gtav2cityscapes/visualize/``.\n\n.. figure:: ../_static/images/visualization/segmentation_image.png\n :width: 400\n\n Cityscapes image.\n\n.. figure:: ../_static/images/visualization/segmentation_pred.png\n :width: 400\n\n Segmentation predictions.\n\n.. figure:: ../_static/images/visualization/segmentation_label.png\n :width: 400\n\n Segmentation labels.\n\n\nTranslation model such as CycleGAN will save images by default. Here is the translation results from source style to target style.\n\n\n.. figure:: ../_static/images/visualization/cyclegan_real_S.png\n :width: 400\n\n Source images.\n\n.. figure:: ../_static/images/visualization/cyclegan_fake_T.png\n :width: 400\n\n Source image in target style.\n\n\n\nHow to visualize the keypoint detection predictions?\n===================================================================\nFor each keypoint detection algorithms, we've implemented the visualization code. All you need to do is set ``--debug`` during training.\nFor instance, in the directory ``examples/domain_adaptation/keypoint_detection``,\n\n.. code-block:: shell\n\n CUDA_VISIBLE_DEVICES=0 python source_only.py data/RHD data/H3D_crop -s RenderedHandPose -t Hand3DStudio --log logs/baseline/rhd2h3d --debug --seed 0\n\nThen you can find visualization images in directory ``logs/baseline/rhd2h3d/visualize/``.\n\n.. figure:: ../_static/images/visualization/keypoint_detection.jpg\n :width: 300\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/docs/index.rst",
"content": "=====================================\nTransfer Learning\n=====================================\n\n.. toctree::\n :caption: Get Started for Transfer Learning Library\n :maxdepth: 2\n :titlesonly:\n\n get_started/installing\n get_started/introduction\n get_started/quickstart\n get_started/visualization\n get_started/faq\n\n\n.. toctree::\n :maxdepth: 2\n :caption: Domain Adaptation Settings\n :titlesonly:\n\n dalib/benchmarks/image_classification\n dalib/benchmarks/image_regression\n dalib/benchmarks/semantic_segmentation\n dalib/benchmarks/keypoint_detection\n dalib/benchmarks/re_identification\n dalib/benchmarks/partial_domain_adaptation\n dalib/benchmarks/openset_domain_adaptation\n dalib/benchmarks/multi_source_domain_adaptation\n\n.. toctree::\n :maxdepth: 2\n :caption: Task Adaptation Settings\n :titlesonly:\n\n talib/benchmarks/image_classification\n\n\n\n.. toctree::\n :maxdepth: 2\n :caption: Domain Generalization Settings\n :titlesonly:\n\n dglib/benchmarks/image_classification\n dglib/benchmarks/re_identification\n\n\n.. toctree::\n :maxdepth: 1\n :caption: Domain Adaptation Methods\n :titlesonly:\n\n dalib/adaptation/statistics_matching.rst\n dalib/adaptation/domain_adversarial.rst\n dalib/adaptation/hypothesis_adversarial.rst\n dalib/adaptation/domain_translation.rst\n dalib/adaptation/others.rst\n dalib/modules.rst\n\n\n\n.. toctree::\n :maxdepth: 1\n :caption: Task Adaptation Methods\n :titlesonly:\n\n talib/finetune.rst\n\n\n\n.. toctree::\n :maxdepth: 1\n :caption: Domain Generalization Methods\n :titlesonly:\n\n dglib/generalization/data_manipulation.rst\n dglib/generalization/architecture_design.rst\n dglib/generalization/representation_learning.rst\n dglib/generalization/learning_strategy.rst\n dglib/modules.rst\n\n\n\n.. toctree::\n :maxdepth: 1\n :caption: Common Packages\n :titlesonly:\n\n common/modules\n common/loss\n common/vision/datasets\n common/vision/models\n common/vision/transforms\n common/utils/base\n common/utils/metric\n common/utils/analysis\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/docs/make.bat",
"content": "@ECHO OFF\n\npushd %~dp0\n\nREM Command file for Sphinx documentation\n\nif \"%SPHINXBUILD%\" == \"\" (\n\tset SPHINXBUILD=python -msphinx\n)\nset SPHINXOPTS=\nset SPHINXBUILD=sphinx-build\nset SOURCEDIR=.\nset BUILDDIR=build\nset SPHINXPROJ=PyTorchSphinxTheme\n\nif \"%1\" == \"\" goto help\n\n%SPHINXBUILD% >NUL 2>NUL\nif errorlevel 9009 (\n\techo.\n\techo.The Sphinx module was not found. Make sure you have Sphinx installed,\n\techo.then set the SPHINXBUILD environment variable to point to the full\n\techo.path of the 'sphinx-build' executable. Alternatively you may add the\n\techo.Sphinx directory to PATH.\n\techo.\n\techo.If you don't have Sphinx installed, grab it from\n\techo.http://sphinx-doc.org/\n\texit /b 1\n)\n\n%SPHINXBUILD% -M %1 %SOURCEDIR% %BUILDDIR% %SPHINXOPTS%\ngoto end\n\n:help\n%SPHINXBUILD% -M help %SOURCEDIR% %BUILDDIR% %SPHINXOPTS%\n\n:end\npopd\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/docs/requirements.txt",
"content": "sphinxcontrib-httpdomain\nsphinx\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/docs/talib/benchmarks/image_classification.rst",
"content": "========================================================\nImage Classification\n========================================================\n\nWe provide benchmarks of different task adaptation algorithms on fine-grained classification datasets `CUB-200-2011`_, `StanfordCars`_,\n`Aircraft`_, and specialized datasets `Resisc45`_, `PatchCamelyon`_.\n\nThose task adaptation algorithms includes:\n\n- :ref:`LWF`\n- :ref:`L2SP`\n- :ref:`BSS`\n- :ref:`DELTA`\n- :ref:`CoTuning`\n- :ref:`StochNorm`\n- :ref:`BiTuning`\n\nWe follow the common practice in the community as described in :ref:`BSS`.\nTraining iterations and data augmentations are kept the same for different task-adaptation\nmethods for a fair comparison.\nHyper-parameters of each method are selected by the performance on target validation data.\n\n\n.. _CUB-200-2011:\n\n------------------------------------------------------------------------\nCUB-200-2011 on ResNet-50 (Supervised Pre-trained)\n------------------------------------------------------------------------\n\n=========== ====== ====== ====== ====== ======\nSample rate 15% 30% 50% 100% Avg\nBaseline 51.2\t64.6\t74.6\t81.8 68.1\nLWF 56.7\t66.8\t73.4\t81.5\t69.6\nBSS 53.4\t66.7\t76.0\t82.0\t69.5\nDELTA 54.8\t67.3\t76.3\t82.3\t70.2\nStochNorm 54.8\t66.8\t75.8\t82.2\t69.9\nCo-Tuning 57.6\t70.1\t77.3\t82.5\t71.9\nBi-Tuning 55.8\t69.3\t77.2\t83.1\t71.4\n=========== ====== ====== ====== ====== ======\n\n.. _StanfordCars:\n\n------------------------------------------------------------------------\nStanford Cars on ResNet-50 (Supervised Pre-trained)\n------------------------------------------------------------------------\n\n=========== ====== ====== ====== ====== ======\nSample rate 15% 30% 50% 100% Avg\nBaseline 41.1\t65.9\t78.4\t87.8\t68.3\nLWF 44.9\t67.0\t77.6\t87.5\t69.3\nBSS 43.3\t67.6\t79.6\t88.0\t69.6\nDELTA 45.0\t68.4\t79.6\t88.4\t70.4\nStochNorm 44.4\t68.1\t79.3\t87.9\t69.9\nCo-Tuning 49.0\t70.6\t81.9\t89.1\t72.7\nBi-Tuning 48.3\t72.8\t83.3\t90.2\t73.7\n=========== ====== ====== ====== ====== ======\n\n.. _Aircraft:\n\n------------------------------------------------------------------------\nFGVC Aircraft on ResNet-50 (Supervised Pre-trained)\n------------------------------------------------------------------------\n\n=========== ====== ====== ====== ====== ======\nSample rate 15% 30% 50% 100% Avg\nBaseline 41.6\t57.8\t68.7\t80.2\t62.1\nLWF 44.1\t60.6\t68.7\t82.4\t64.0\nBSS 43.6\t59.5\t69.6\t81.2\t63.5\nDELTA 44.4\t61.9\t71.4\t82.7\t65.1\nStochNorm 44.3\t60.6\t70.1\t81.5\t64.1\nCo-Tuning 45.9\t61.2\t71.3\t82.2\t65.2\nBi-Tuning 47.2\t64.3\t73.7\t84.3\t67.4\n=========== ====== ====== ====== ====== ======\n\nResisc45 and PatchCamelyon have very different data distributions from ImageNet,\nthus some regularization-based fine-tuning methods lead to negative transfer on these two benchmarks.\n\n.. _Resisc45:\n\n------------------------------------------------------------------------\nResisc45 on ResNet-50 (Supervised Pre-trained)\n------------------------------------------------------------------------\n\n================= ====== ====== ====== ====== ======\n#samples/#classes 10 20 40 80 Avg\nBaseline 74.0 81.9 86.5 90.1 83.1\nLWF 74.2 81.7 86.1 89.9 83.0\nBSS 73.5 80.9 85.7 90.0 82.5\nDELTA 73.9 81.7 86.0 90.1 82.9\nStochNorm 73.9 82.1 87.0 90.2 83.3\nCo-Tuning 75.0 82.7 87.3 91.4 84.1\nBi-Tuning 75.3 83.2 87.4 91.4 84.3\n================= ====== ====== ====== ====== ======\n\n\n.. _PatchCamelyon:\n\n------------------------------------------------------------------------\nPatchCamelyon on ResNet-50 (Supervised Pre-trained)\n------------------------------------------------------------------------\n\n================= ====== ====== ====== ====== ======\n#samples/#classes 40 80 160 320 Avg\nBaseline 75.5 75.9 80.4 81.2 78.3\nLWF 75.3 77.7 80.6 82.5 79.0\nBSS 78.0 78.2 80.4 80.4 79.3\nDELTA 74.1 76.4 80.0 81.9 78.1\nStochNorm 75.9 77.1 78.2 81.3 78.1\nCo-Tuning 75.1 76.2 80.7 81.8 78.5\nBi-Tuning 75.1 77.6 80.6 81.4 78.7\n================= ====== ====== ====== ====== ======\n\nWe further evaluate task adaptation algorithms when the downstream tasks are different\nfrom the pre-training tasks. The pre-training task is MoCo unsupervised pre-training, and\nthe downstream tasks are still fine-grained classification.\nIn this scenario, some regularization-based fine-tuning methods also lead to negative transfer.\n\n.. _CUB-200-2011_MoCo:\n\n------------------------------------------------------------------------\nCUB-200-2011 on ResNet-50 (MoCo Pre-trained)\n------------------------------------------------------------------------\n\n=========== ====== ====== ====== ====== ======\nSample rate 15% 30% 50% 100% Avg\nBaseline 28.0\t48.2\t62.7\t75.6\t53.6\nLWF 28.8\t50.1\t62.8\t76.2\t54.5\nBSS 30.9\t50.3\t63.7\t75.8\t55.2\nDELTA 27.9\t51.4\t65.9\t74.6\t55.0\nStochNorm 20.8\t44.9\t60.1\t72.8\t49.7\nCo-Tuning 29.1\t50.1\t63.8\t75.9\t54.7\nBi-Tuning 32.4\t51.8\t65.7\t76.1\t56.5\n=========== ====== ====== ====== ====== ======\n\n.. _StanfordCars_MoCo:\n\n------------------------------------------------------------------------\nStanford Cars on ResNet-50 (MoCo Pre-trained)\n------------------------------------------------------------------------\n\n=========== ====== ====== ====== ====== ======\nSample rate 15% 30% 50% 100% Avg\nBaseline 42.5\t71.2\t83.0\t90.1\t71.7\nLWF 44.2\t71.7\t82.9\t90.5\t72.3\nBSS 45.0\t71.5\t83.8\t90.1\t72.6\nDELTA 45.9\t72.9\t82.5\t88.9\t72.6\nStochNorm 40.3\t66.2\t78.0\t86.2\t67.7\nCo-Tuning 44.2\t72.6\t83.3\t90.3\t72.6\nBi-Tuning 45.6\t72.8\t83.2\t90.8\t73.1\n=========== ====== ====== ====== ====== ======\n\n.. _Aircraft_MoCo:\n\n------------------------------------------------------------------------\nFGVC Aircraft on ResNet-50 (MoCo Pre-trained)\n------------------------------------------------------------------------\n\n=========== ====== ====== ====== ====== ======\nSample rate 15% 30% 50% 100% Avg\nBaseline 45.8\t67.6\t78.8\t88.0\t70.1\nLWF 48.5\t68.5\t78.0\t87.9\t70.7\nBSS 47.7\t69.1\t79.2\t88.0\t71.0\nDELTA \\- \\- \\- \\- \\-\nStochNorm 45.4\t68.8\t76.7\t86.1\t69.3\nCo-Tuning 48.2\t68.5\t78.7\t87.3\t70.7\nBi-Tuning 46.4\t69.6\t79.4\t87.9\t70.8\n=========== ====== ====== ====== ====== ======\n\n.. note::\n \\- indicates that the training cannot converge.\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/docs/talib/finetune.rst",
"content": "=======================================\nFine-tuning\n=======================================\n\nRegularization-based Fine-tuning\n===========================================\n\n.. _L2:\n\nL2\n------\n\n.. autoclass:: talib.finetune.delta.L2Regularization\n\n\n.. _L2SP:\n\nL2-SP\n------\n\n.. autoclass:: talib.finetune.delta.SPRegularization\n\n\n.. _DELTA:\n\nDELTA: DEep Learning Transfer using Feature Map with Attention\n-------------------------------------------------------------------------------------\n\n.. autoclass:: talib.finetune.delta.BehavioralRegularization\n\n.. autoclass:: talib.finetune.delta.AttentionBehavioralRegularization\n\n.. autoclass:: talib.finetune.delta.IntermediateLayerGetter\n\n\n.. _LWF:\n\nLWF: Learning without Forgetting\n------------------------------------------\n\n.. autoclass:: talib.finetune.lwf.Classifier\n\n\n\n.. _CoTuning:\n\nCo-Tuning\n------------------------------------------\n\n.. autoclass:: talib.finetune.co_tuning.CoTuningLoss\n\n.. autoclass:: talib.finetune.co_tuning.Relationship\n\n\n.. _StochNorm:\n\nStochNorm: Stochastic Normalization\n------------------------------------------\n\n.. autoclass:: talib.finetune.stochnorm.StochNorm1d\n\n.. autoclass:: talib.finetune.stochnorm.StochNorm2d\n\n.. autoclass:: talib.finetune.stochnorm.StochNorm3d\n\n.. autofunction:: talib.finetune.stochnorm.convert_model\n\n\nAdaptive Fine-tuning\n===========================================\n\n.. _BiTuning:\n\nBi-Tuning\n------------------------------------------\n\n.. autoclass:: talib.finetune.bi_tuning.BiTuning\n\n\n\nRejecting Untransferable Information\n===========================================\n\n.. _BSS:\n\nBSS: Batch Spectral Shrinkage\n------------------------------------------\n\n.. autoclass:: talib.finetune.bss.BatchSpectralShrinkage\n\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/examples/domain_adaptation/image_classification/README.md",
"content": "# Unsupervised Domain Adaptation for Image Classification\n\n## Installation\nExample scripts can deal with [WILDS datasets](https://wilds.stanford.edu/).\nYou should first install ``wilds`` before using these scripts.\n\n```\npip install wilds\n```\n\nExample scripts also support all models in [PyTorch-Image-Models](https://github.com/rwightman/pytorch-image-models).\nYou also need to install timm to use PyTorch-Image-Models.\n\n```\npip install timm\n```\n\n## Dataset\n\nFollowing datasets can be downloaded automatically:\n\n- [MNIST](http://yann.lecun.com/exdb/mnist/), [SVHN](http://ufldl.stanford.edu/housenumbers/), [USPS](https://www.csie.ntu.edu.tw/~cjlin/libsvmtools/datasets/multiclass.html#usps) \n- [Office31](https://www.cc.gatech.edu/~judy/domainadapt/)\n- [OfficeCaltech](https://www.cc.gatech.edu/~judy/domainadapt/)\n- [OfficeHome](https://www.hemanthdv.org/officeHomeDataset.html)\n- [VisDA2017](http://ai.bu.edu/visda-2017/)\n- [DomainNet](http://ai.bu.edu/M3SDA/)\n- [iwildcam (WILDS)](https://wilds.stanford.edu/datasets/)\n- [camelyon17 (WILDS)](https://wilds.stanford.edu/datasets/)\n- [fmow (WILDS)](https://wilds.stanford.edu/datasets/)\n\nYou need to prepare following datasets manually if you want to use them:\n- [ImageNet](https://www.image-net.org/)\n- [ImageNetR](https://github.com/hendrycks/imagenet-r)\n- [ImageNet-Sketch](https://github.com/HaohanWang/ImageNet-Sketch)\n\nand prepare them following [Documentation for ImageNetR](/common/vision/datasets/imagenet_r.py) and [ImageNet-Sketch](/common/vision/datasets/imagenet_sketch.py).\n\n## Supported Methods\n\nSupported methods include:\n\n- [Domain Adversarial Neural Network (DANN)](https://arxiv.org/abs/1505.07818)\n- [Deep Adaptation Network (DAN)](https://arxiv.org/pdf/1502.02791)\n- [Joint Adaptation Network (JAN)](https://arxiv.org/abs/1605.06636)\n- [Adversarial Discriminative Domain Adaptation (ADDA)](https://arxiv.org/pdf/1702.05464.pdf)\n- [Conditional Domain Adversarial Network (CDAN)](https://arxiv.org/abs/1705.10667)\n- [Maximum Classifier Discrepancy (MCD)](https://arxiv.org/abs/1712.02560)\n- [Adaptive Feature Norm (AFN)](https://arxiv.org/pdf/1811.07456v2.pdf)\n- [Batch Spectral Penalization (BSP)](http://ise.thss.tsinghua.edu.cn/~mlong/doc/batch-spectral-penalization-icml19.pdf)\n- [Margin Disparity Discrepancy (MDD)](https://arxiv.org/abs/1904.05801)\n- [Minimum Class Confusion (MCC)](https://arxiv.org/abs/1912.03699)\n\n## Experiment and Results\n\nThe shell files give the script to reproduce the [benchmarks](/docs/dalib/benchmarks/image_classification.rst) with specified hyper-parameters.\nFor example, if you want to train DANN on Office31, use the following script\n\n```shell script\n# Train a DANN on Office-31 Amazon -> Webcam task using ResNet 50.\n# Assume you have put the datasets under the path `data/office-31`, \n# or you are glad to download the datasets automatically from the Internet to this path\nCUDA_VISIBLE_DEVICES=0 python dann.py data/office31 -d Office31 -s A -t W -a resnet50 --epochs 20 --seed 1 --log logs/dann/Office31_A2W\n```\n\nFor more information please refer to [Get Started](/docs/get_started/quickstart.rst) for help.\n\n## TODO\nSupport methods: AdaBN/TransNorm, CycleGAN, CyCADA\n\n## Citation\nIf you use these methods in your research, please consider citing.\n\n```\n@inproceedings{DANN,\n author = {Ganin, Yaroslav and Lempitsky, Victor},\n Booktitle = {ICML},\n Title = {Unsupervised domain adaptation by backpropagation},\n Year = {2015}\n}\n\n@inproceedings{DAN,\n author = {Mingsheng Long and\n Yue Cao and\n Jianmin Wang and\n Michael I. Jordan},\n title = {Learning Transferable Features with Deep Adaptation Networks},\n booktitle = {ICML},\n year = {2015},\n}\n\n@inproceedings{JAN,\n title={Deep transfer learning with joint adaptation networks},\n author={Long, Mingsheng and Zhu, Han and Wang, Jianmin and Jordan, Michael I},\n booktitle={ICML},\n year={2017},\n}\n\n@inproceedings{ADDA,\n title={Adversarial discriminative domain adaptation},\n author={Tzeng, Eric and Hoffman, Judy and Saenko, Kate and Darrell, Trevor},\n booktitle={CVPR},\n year={2017}\n}\n\n@inproceedings{CDAN,\n author = {Mingsheng Long and\n Zhangjie Cao and\n Jianmin Wang and\n Michael I. Jordan},\n title = {Conditional Adversarial Domain Adaptation},\n booktitle = {NeurIPS},\n year = {2018}\n}\n\n@inproceedings{MCD,\n title={Maximum classifier discrepancy for unsupervised domain adaptation},\n author={Saito, Kuniaki and Watanabe, Kohei and Ushiku, Yoshitaka and Harada, Tatsuya},\n booktitle={CVPR},\n year={2018}\n}\n\n@InProceedings{AFN,\n author = {Xu, Ruijia and Li, Guanbin and Yang, Jihan and Lin, Liang},\n title = {Larger Norm More Transferable: An Adaptive Feature Norm Approach for Unsupervised Domain Adaptation},\n booktitle = {ICCV},\n year = {2019}\n}\n\n@inproceedings{MDD,\n title={Bridging theory and algorithm for domain adaptation},\n author={Zhang, Yuchen and Liu, Tianle and Long, Mingsheng and Jordan, Michael},\n booktitle={ICML},\n year={2019},\n}\n\n@inproceedings{BSP,\n title={Transferability vs. discriminability: Batch spectral penalization for adversarial domain adaptation},\n author={Chen, Xinyang and Wang, Sinan and Long, Mingsheng and Wang, Jianmin},\n booktitle={ICML},\n year={2019},\n}\n\n@inproceedings{MCC,\n author = {Ying Jin and\n Ximei Wang and\n Mingsheng Long and\n Jianmin Wang},\n title = {Less Confusion More Transferable: Minimum Class Confusion for Versatile\n Domain Adaptation},\n year={2020},\n booktitle={ECCV},\n}\n```\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/examples/domain_adaptation/image_classification/adda.py",
"content": "\"\"\"\n@author: Baixu Chen\n@contact: cbx_99_hasta@outlook.com\nNote: Our implementation is different from ADDA paper in several respects. We do not use separate networks for\nsource and target domain, nor fix classifier head. Besides, we do not adopt asymmetric objective loss function\nof the feature extractor. We achieve promising results on digits datasets (reported by ADDA paper).\nBut on other benchmarks, ADDA-grl may achieve better results.\n\"\"\"\nimport random\nimport time\nimport warnings\nimport sys\nimport argparse\nimport shutil\nimport os.path as osp\n\nimport torch\nimport torch.nn as nn\nimport torch.backends.cudnn as cudnn\nfrom torch.optim import SGD\nfrom torch.optim.lr_scheduler import LambdaLR\nfrom torch.utils.data import DataLoader\nimport torch.nn.functional as F\n\nsys.path.append('../../..')\nfrom dalib.modules.domain_discriminator import DomainDiscriminator\nfrom dalib.adaptation.adda import ImageClassifier, DomainAdversarialLoss\nfrom dalib.modules.gl import WarmStartGradientLayer\nfrom dalib.translation.cyclegan.util import set_requires_grad\nfrom common.utils.data import ForeverDataIterator\nfrom common.utils.metric import accuracy, binary_accuracy\nfrom common.utils.meter import AverageMeter, ProgressMeter\nfrom common.utils.logger import CompleteLogger\nfrom common.utils.analysis import collect_feature, tsne, a_distance\n\nsys.path.append('.')\nimport utils\n\ndevice = torch.device(\"cuda\" if torch.cuda.is_available() else \"cpu\")\n\n\ndef main(args: argparse.Namespace):\n logger = CompleteLogger(args.log, args.phase)\n print(args)\n\n if args.seed is not None:\n random.seed(args.seed)\n torch.manual_seed(args.seed)\n cudnn.deterministic = True\n warnings.warn('You have chosen to seed training. '\n 'This will turn on the CUDNN deterministic setting, '\n 'which can slow down your training considerably! '\n 'You may see unexpected behavior when restarting '\n 'from checkpoints.')\n\n cudnn.benchmark = True\n\n # Data loading code\n train_transform = utils.get_train_transform(args.train_resizing, random_horizontal_flip=not args.no_hflip,\n random_color_jitter=False, resize_size=args.resize_size,\n norm_mean=args.norm_mean, norm_std=args.norm_std)\n val_transform = utils.get_val_transform(args.val_resizing, resize_size=args.resize_size,\n norm_mean=args.norm_mean, norm_std=args.norm_std)\n print(\"train_transform: \", train_transform)\n print(\"val_transform: \", val_transform)\n\n train_source_dataset, train_target_dataset, val_dataset, test_dataset, num_classes, args.class_names = \\\n utils.get_dataset(args.data, args.root, args.source, args.target, train_transform, val_transform)\n train_source_loader = DataLoader(train_source_dataset, batch_size=args.batch_size,\n shuffle=True, num_workers=args.workers, drop_last=True)\n train_target_loader = DataLoader(train_target_dataset, batch_size=args.batch_size,\n shuffle=True, num_workers=args.workers, drop_last=True)\n val_loader = DataLoader(val_dataset, batch_size=args.batch_size, shuffle=False, num_workers=args.workers)\n test_loader = DataLoader(test_dataset, batch_size=args.batch_size, shuffle=False, num_workers=args.workers)\n\n train_source_iter = ForeverDataIterator(train_source_loader)\n train_target_iter = ForeverDataIterator(train_target_loader)\n\n # create model\n print(\"=> using model '{}'\".format(args.arch))\n backbone = utils.get_model(args.arch, pretrain=not args.scratch)\n pool_layer = nn.Identity() if args.no_pool else None\n classifier = ImageClassifier(backbone, num_classes, bottleneck_dim=args.bottleneck_dim,\n pool_layer=pool_layer, finetune=not args.scratch).to(device)\n domain_discri = DomainDiscriminator(in_feature=classifier.features_dim, hidden_size=1024).to(device)\n\n # define loss function\n domain_adv = DomainAdversarialLoss().to(device)\n gl = WarmStartGradientLayer(alpha=1., lo=0., hi=1., max_iters=1000, auto_step=True)\n\n # define optimizer and lr scheduler\n optimizer = SGD(classifier.get_parameters(), args.lr, momentum=args.momentum, weight_decay=args.weight_decay,\n nesterov=True)\n optimizer_d = SGD(domain_discri.get_parameters(), args.lr_d, momentum=args.momentum, weight_decay=args.weight_decay,\n nesterov=True)\n lr_scheduler = LambdaLR(optimizer, lambda x: args.lr * (1. + args.lr_gamma * float(x)) ** (-args.lr_decay))\n lr_scheduler_d = LambdaLR(optimizer_d, lambda x: args.lr_d * (1. + args.lr_gamma * float(x)) ** (-args.lr_decay))\n\n # resume from the best checkpoint\n if args.phase != 'train':\n checkpoint = torch.load(logger.get_checkpoint_path('best'), map_location='cpu')\n classifier.load_state_dict(checkpoint)\n\n # analysis the model\n if args.phase == 'analysis':\n # extract features from both domains\n feature_extractor = nn.Sequential(classifier.backbone, classifier.pool_layer, classifier.bottleneck).to(device)\n source_feature = collect_feature(train_source_loader, feature_extractor, device)\n target_feature = collect_feature(train_target_loader, feature_extractor, device)\n # plot t-SNE\n tSNE_filename = osp.join(logger.visualize_directory, 'TSNE.pdf')\n tsne.visualize(source_feature, target_feature, tSNE_filename)\n print(\"Saving t-SNE to\", tSNE_filename)\n # calculate A-distance, which is a measure for distribution discrepancy\n A_distance = a_distance.calculate(source_feature, target_feature, device)\n print(\"A-distance =\", A_distance)\n return\n\n if args.phase == 'test':\n acc1 = utils.validate(test_loader, classifier, args, device)\n print(acc1)\n return\n\n # start training\n best_acc1 = 0.\n for epoch in range(args.epochs):\n print(\"lr classifier:\", lr_scheduler.get_lr())\n print(\"lr discriminator:\", lr_scheduler_d.get_lr())\n # train for one epoch\n train(train_source_iter, train_target_iter, classifier, domain_discri, domain_adv, gl, optimizer,\n lr_scheduler, optimizer_d, lr_scheduler_d, epoch, args)\n\n # evaluate on validation set\n acc1 = utils.validate(val_loader, classifier, args, device)\n\n # remember best acc@1 and save checkpoint\n torch.save(classifier.state_dict(), logger.get_checkpoint_path('latest'))\n if acc1 > best_acc1:\n shutil.copy(logger.get_checkpoint_path('latest'), logger.get_checkpoint_path('best'))\n best_acc1 = max(acc1, best_acc1)\n\n print(\"best_acc1 = {:3.1f}\".format(best_acc1))\n\n # evaluate on test set\n classifier.load_state_dict(torch.load(logger.get_checkpoint_path('best')))\n acc1 = utils.validate(test_loader, classifier, args, device)\n print(\"test_acc1 = {:3.1f}\".format(acc1))\n\n logger.close()\n\n\ndef train(train_source_iter: ForeverDataIterator, train_target_iter: ForeverDataIterator, model: ImageClassifier,\n domain_discri: DomainDiscriminator, domain_adv: DomainAdversarialLoss, gl,\n optimizer: SGD, lr_scheduler: LambdaLR, optimizer_d: SGD, lr_scheduler_d: LambdaLR,\n epoch: int, args: argparse.Namespace):\n batch_time = AverageMeter('Time', ':5.2f')\n data_time = AverageMeter('Data', ':5.2f')\n losses_s = AverageMeter('Cls Loss', ':6.2f')\n losses_transfer = AverageMeter('Transfer Loss', ':6.2f')\n losses_discriminator = AverageMeter('Discriminator Loss', ':6.2f')\n cls_accs = AverageMeter('Cls Acc', ':3.1f')\n domain_accs = AverageMeter('Domain Acc', ':3.1f')\n progress = ProgressMeter(\n args.iters_per_epoch,\n [batch_time, data_time, losses_s, losses_transfer, losses_discriminator, cls_accs, domain_accs],\n prefix=\"Epoch: [{}]\".format(epoch))\n\n end = time.time()\n for i in range(args.iters_per_epoch):\n x_s, labels_s = next(train_source_iter)\n x_t, _ = next(train_target_iter)\n\n x_s = x_s.to(device)\n x_t = x_t.to(device)\n labels_s = labels_s.to(device)\n\n # measure data loading time\n data_time.update(time.time() - end)\n\n # Step 1: Train the classifier, freeze the discriminator\n model.train()\n domain_discri.eval()\n set_requires_grad(model, True)\n set_requires_grad(domain_discri, False)\n x = torch.cat((x_s, x_t), dim=0)\n y, f = model(x)\n y_s, y_t = y.chunk(2, dim=0)\n loss_s = F.cross_entropy(y_s, labels_s)\n\n # adversarial training to fool the discriminator\n d = domain_discri(gl(f))\n d_s, d_t = d.chunk(2, dim=0)\n loss_transfer = 0.5 * (domain_adv(d_s, 'target') + domain_adv(d_t, 'source'))\n\n optimizer.zero_grad()\n (loss_s + loss_transfer * args.trade_off).backward()\n optimizer.step()\n lr_scheduler.step()\n\n # Step 2: Train the discriminator\n model.eval()\n domain_discri.train()\n set_requires_grad(model, False)\n set_requires_grad(domain_discri, True)\n d = domain_discri(f.detach())\n d_s, d_t = d.chunk(2, dim=0)\n loss_discriminator = 0.5 * (domain_adv(d_s, 'source') + domain_adv(d_t, 'target'))\n\n optimizer_d.zero_grad()\n loss_discriminator.backward()\n optimizer_d.step()\n lr_scheduler_d.step()\n\n losses_s.update(loss_s.item(), x_s.size(0))\n losses_transfer.update(loss_transfer.item(), x_s.size(0))\n losses_discriminator.update(loss_discriminator.item(), x_s.size(0))\n\n cls_acc = accuracy(y_s, labels_s)[0]\n cls_accs.update(cls_acc.item(), x_s.size(0))\n domain_acc = 0.5 * (binary_accuracy(d_s, torch.ones_like(d_s)) + binary_accuracy(d_t, torch.zeros_like(d_t)))\n domain_accs.update(domain_acc.item(), x_s.size(0))\n\n # measure elapsed time\n batch_time.update(time.time() - end)\n end = time.time()\n\n if i % args.print_freq == 0:\n progress.display(i)\n\n\nif __name__ == '__main__':\n parser = argparse.ArgumentParser(description='ADDA for Unsupervised Domain Adaptation')\n # dataset parameters\n parser.add_argument('root', metavar='DIR',\n help='root path of dataset')\n parser.add_argument('-d', '--data', metavar='DATA', default='Office31', choices=utils.get_dataset_names(),\n help='dataset: ' + ' | '.join(utils.get_dataset_names()) +\n ' (default: Office31)')\n parser.add_argument('-s', '--source', help='source domain(s)', nargs='+')\n parser.add_argument('-t', '--target', help='target domain(s)', nargs='+')\n parser.add_argument('--train-resizing', type=str, default='default')\n parser.add_argument('--val-resizing', type=str, default='default')\n parser.add_argument('--resize-size', type=int, default=224,\n help='the image size after resizing')\n parser.add_argument('--no-hflip', action='store_true',\n help='no random horizontal flipping during training')\n parser.add_argument('--norm-mean', type=float, nargs='+',\n default=(0.485, 0.456, 0.406), help='normalization mean')\n parser.add_argument('--norm-std', type=float, nargs='+',\n default=(0.229, 0.224, 0.225), help='normalization std')\n # model parameters\n parser.add_argument('-a', '--arch', metavar='ARCH', default='resnet18',\n choices=utils.get_model_names(),\n help='backbone architecture: ' +\n ' | '.join(utils.get_model_names()) +\n ' (default: resnet18)')\n parser.add_argument('--bottleneck-dim', default=256, type=int,\n help='Dimension of bottleneck')\n parser.add_argument('--no-pool', action='store_true',\n help='no pool layer after the feature extractor.')\n parser.add_argument('--scratch', action='store_true', help='whether train from scratch.')\n parser.add_argument('--trade-off', default=0.1, type=float,\n help='the trade-off hyper-parameter for transfer loss')\n # training parameters\n parser.add_argument('-b', '--batch-size', default=32, type=int,\n metavar='N',\n help='mini-batch size (default: 32)')\n parser.add_argument('--lr', '--learning-rate', default=0.01, type=float,\n metavar='LR', help='initial learning rate of the classifier', dest='lr')\n parser.add_argument('--lr-d', default=0.01, type=float,\n help='initial learning rate of the domain discriminator')\n parser.add_argument('--lr-gamma', default=0.001, type=float, help='parameter for lr scheduler')\n parser.add_argument('--lr-decay', default=0.75, type=float, help='parameter for lr scheduler')\n parser.add_argument('--momentum', default=0.9, type=float, metavar='M',\n help='momentum')\n parser.add_argument('--wd', '--weight-decay', default=1e-3, type=float,\n metavar='W', help='weight decay (default: 1e-3)',\n dest='weight_decay')\n parser.add_argument('-j', '--workers', default=2, type=int, metavar='N',\n help='number of data loading workers (default: 4)')\n parser.add_argument('--epochs', default=20, type=int, metavar='N',\n help='number of total epochs to run')\n parser.add_argument('-i', '--iters-per-epoch', default=1000, type=int,\n help='Number of iterations per epoch')\n parser.add_argument('-p', '--print-freq', default=100, type=int,\n metavar='N', help='print frequency (default: 100)')\n parser.add_argument('--seed', default=None, type=int,\n help='seed for initializing training. ')\n parser.add_argument('--per-class-eval', action='store_true',\n help='whether output per-class accuracy during evaluation')\n parser.add_argument(\"--log\", type=str, default='adda',\n help=\"Where to save logs, checkpoints and debugging images.\")\n parser.add_argument(\"--phase\", type=str, default='train', choices=['train', 'test', 'analysis'],\n help=\"When phase is 'test', only test the model.\"\n \"When phase is 'analysis', only analysis the model.\")\n args = parser.parse_args()\n main(args)\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/examples/domain_adaptation/image_classification/adda.sh",
"content": "#!/usr/bin/env bash\n# ResNet50, Office31, Single Source\nCUDA_VISIBLE_DEVICES=0 python adda.py data/office31 -d Office31 -s A -t W -a resnet50 --epochs 20 --seed 1 --log logs/adda/Office31_A2W\nCUDA_VISIBLE_DEVICES=0 python adda.py data/office31 -d Office31 -s D -t W -a resnet50 --epochs 20 --seed 1 --log logs/adda/Office31_D2W\nCUDA_VISIBLE_DEVICES=0 python adda.py data/office31 -d Office31 -s W -t D -a resnet50 --epochs 20 --seed 1 --log logs/adda/Office31_W2D\nCUDA_VISIBLE_DEVICES=0 python adda.py data/office31 -d Office31 -s A -t D -a resnet50 --epochs 20 --seed 1 --log logs/adda/Office31_A2D\nCUDA_VISIBLE_DEVICES=0 python adda.py data/office31 -d Office31 -s D -t A -a resnet50 --epochs 20 --seed 1 --log logs/adda/Office31_D2A\nCUDA_VISIBLE_DEVICES=0 python adda.py data/office31 -d Office31 -s W -t A -a resnet50 --epochs 20 --seed 1 --log logs/adda/Office31_W2A\n\n# ResNet50, Office-Home, Single Source\nCUDA_VISIBLE_DEVICES=0 python adda.py data/office-home -d OfficeHome -s Ar -t Cl -a resnet50 --epochs 30 --seed 0 --log logs/adda/OfficeHome_Ar2Cl\nCUDA_VISIBLE_DEVICES=0 python adda.py data/office-home -d OfficeHome -s Ar -t Pr -a resnet50 --epochs 30 --seed 0 --log logs/adda/OfficeHome_Ar2Pr\nCUDA_VISIBLE_DEVICES=0 python adda.py data/office-home -d OfficeHome -s Ar -t Rw -a resnet50 --epochs 30 --seed 0 --log logs/adda/OfficeHome_Ar2Rw\nCUDA_VISIBLE_DEVICES=0 python adda.py data/office-home -d OfficeHome -s Cl -t Ar -a resnet50 --epochs 30 --seed 0 --log logs/adda/OfficeHome_Cl2Ar\nCUDA_VISIBLE_DEVICES=0 python adda.py data/office-home -d OfficeHome -s Cl -t Pr -a resnet50 --epochs 30 --seed 0 --log logs/adda/OfficeHome_Cl2Pr\nCUDA_VISIBLE_DEVICES=0 python adda.py data/office-home -d OfficeHome -s Cl -t Rw -a resnet50 --epochs 30 --seed 0 --log logs/adda/OfficeHome_Cl2Rw\nCUDA_VISIBLE_DEVICES=0 python adda.py data/office-home -d OfficeHome -s Pr -t Ar -a resnet50 --epochs 30 --seed 0 --log logs/adda/OfficeHome_Pr2Ar\nCUDA_VISIBLE_DEVICES=0 python adda.py data/office-home -d OfficeHome -s Pr -t Cl -a resnet50 --epochs 30 --seed 0 --log logs/adda/OfficeHome_Pr2Cl\nCUDA_VISIBLE_DEVICES=0 python adda.py data/office-home -d OfficeHome -s Pr -t Rw -a resnet50 --epochs 30 --seed 0 --log logs/adda/OfficeHome_Pr2Rw\nCUDA_VISIBLE_DEVICES=0 python adda.py data/office-home -d OfficeHome -s Rw -t Ar -a resnet50 --epochs 30 --seed 0 --log logs/adda/OfficeHome_Rw2Ar\nCUDA_VISIBLE_DEVICES=0 python adda.py data/office-home -d OfficeHome -s Rw -t Cl -a resnet50 --epochs 30 --seed 0 --log logs/adda/OfficeHome_Rw2Cl\nCUDA_VISIBLE_DEVICES=0 python adda.py data/office-home -d OfficeHome -s Rw -t Pr -a resnet50 --epochs 30 --seed 0 --log logs/adda/OfficeHome_Rw2Pr\n\n# ResNet101, VisDA-2017, Single Source\nCUDA_VISIBLE_DEVICES=0 python adda.py data/visda-2017 -d VisDA2017 -s Synthetic -t Real -a resnet101 \\\n --epochs 30 --seed 0 --per-class-eval --train-resizing cen.crop --log logs/adda/VisDA2017\n\n# ResNet101, DomainNet, Single Source\nCUDA_VISIBLE_DEVICES=0 python adda.py data/domainnet -d DomainNet -s c -t p -a resnet101 --bottleneck-dim 1024 --epochs 30 -i 2500 -p 500 --seed 0 --log logs/adda/DomainNet_c2p\nCUDA_VISIBLE_DEVICES=0 python adda.py data/domainnet -d DomainNet -s c -t r -a resnet101 --bottleneck-dim 1024 --epochs 30 -i 2500 -p 500 --seed 0 --log logs/adda/DomainNet_c2r\nCUDA_VISIBLE_DEVICES=0 python adda.py data/domainnet -d DomainNet -s c -t s -a resnet101 --bottleneck-dim 1024 --epochs 30 -i 2500 -p 500 --seed 0 --log logs/adda/DomainNet_c2s\nCUDA_VISIBLE_DEVICES=0 python adda.py data/domainnet -d DomainNet -s p -t c -a resnet101 --bottleneck-dim 1024 --epochs 30 -i 2500 -p 500 --seed 0 --log logs/adda/DomainNet_p2c\nCUDA_VISIBLE_DEVICES=0 python adda.py data/domainnet -d DomainNet -s p -t r -a resnet101 --bottleneck-dim 1024 --epochs 30 -i 2500 -p 500 --seed 0 --log logs/adda/DomainNet_p2r\nCUDA_VISIBLE_DEVICES=0 python adda.py data/domainnet -d DomainNet -s p -t s -a resnet101 --bottleneck-dim 1024 --epochs 30 -i 2500 -p 500 --seed 0 --log logs/adda/DomainNet_p2s\nCUDA_VISIBLE_DEVICES=0 python adda.py data/domainnet -d DomainNet -s r -t c -a resnet101 --bottleneck-dim 1024 --epochs 30 -i 2500 -p 500 --seed 0 --log logs/adda/DomainNet_r2c\nCUDA_VISIBLE_DEVICES=0 python adda.py data/domainnet -d DomainNet -s r -t p -a resnet101 --bottleneck-dim 1024 --epochs 30 -i 2500 -p 500 --seed 0 --log logs/adda/DomainNet_r2p\nCUDA_VISIBLE_DEVICES=0 python adda.py data/domainnet -d DomainNet -s r -t s -a resnet101 --bottleneck-dim 1024 --epochs 30 -i 2500 -p 500 --seed 0 --log logs/adda/DomainNet_r2s\nCUDA_VISIBLE_DEVICES=0 python adda.py data/domainnet -d DomainNet -s s -t c -a resnet101 --bottleneck-dim 1024 --epochs 30 -i 2500 -p 500 --seed 0 --log logs/adda/DomainNet_s2c\nCUDA_VISIBLE_DEVICES=0 python adda.py data/domainnet -d DomainNet -s s -t p -a resnet101 --bottleneck-dim 1024 --epochs 30 -i 2500 -p 500 --seed 0 --log logs/adda/DomainNet_s2p\nCUDA_VISIBLE_DEVICES=0 python adda.py data/domainnet -d DomainNet -s s -t r -a resnet101 --bottleneck-dim 1024 --epochs 30 -i 2500 -p 500 --seed 0 --log logs/adda/DomainNet_s2r\n\n# ResNet50, ImageNet200 -> ImageNetR\nCUDA_VISIBLE_DEVICES=0 python adda.py data/ImageNetR -d ImageNetR -s IN -t INR -a resnet50 --epochs 30 -i 2500 -p 500 --seed 0 --log logs/adda/ImageNet_IN2INR\n\n# ig_resnext101_32x8d, ImageNet -> ImageNetSketch\nCUDA_VISIBLE_DEVICES=0 python adda.py data/imagenet-sketch -d ImageNetSketch -s IN -t sketch -a ig_resnext101_32x8d --epochs 30 -i 2500 -p 500 --bottleneck-dim 1024 --log logs/dann_ig_resnext101_32x8d/ImageNet_IN2sketch\n\n# Vision Transformer, Office-Home, Single Source\nCUDA_VISIBLE_DEVICES=0 python adda.py data/office-home -d OfficeHome -s Ar -t Cl -a vit_base_patch16_224 --epochs 30 --seed 0 -b 24 --no-pool --log logs/dann_vit/OfficeHome_Ar2Cl\nCUDA_VISIBLE_DEVICES=0 python adda.py data/office-home -d OfficeHome -s Ar -t Pr -a vit_base_patch16_224 --epochs 30 --seed 0 -b 24 --no-pool --log logs/dann_vit/OfficeHome_Ar2Pr\nCUDA_VISIBLE_DEVICES=0 python adda.py data/office-home -d OfficeHome -s Ar -t Rw -a vit_base_patch16_224 --epochs 30 --seed 0 -b 24 --no-pool --log logs/dann_vit/OfficeHome_Ar2Rw\nCUDA_VISIBLE_DEVICES=0 python adda.py data/office-home -d OfficeHome -s Cl -t Ar -a vit_base_patch16_224 --epochs 30 --seed 0 -b 24 --no-pool --log logs/dann_vit/OfficeHome_Cl2Ar\nCUDA_VISIBLE_DEVICES=0 python adda.py data/office-home -d OfficeHome -s Cl -t Pr -a vit_base_patch16_224 --epochs 30 --seed 0 -b 24 --no-pool --log logs/dann_vit/OfficeHome_Cl2Pr\nCUDA_VISIBLE_DEVICES=0 python adda.py data/office-home -d OfficeHome -s Cl -t Rw -a vit_base_patch16_224 --epochs 30 --seed 0 -b 24 --no-pool --log logs/dann_vit/OfficeHome_Cl2Rw\nCUDA_VISIBLE_DEVICES=0 python adda.py data/office-home -d OfficeHome -s Pr -t Ar -a vit_base_patch16_224 --epochs 30 --seed 0 -b 24 --no-pool --log logs/dann_vit/OfficeHome_Pr2Ar\nCUDA_VISIBLE_DEVICES=0 python adda.py data/office-home -d OfficeHome -s Pr -t Cl -a vit_base_patch16_224 --epochs 30 --seed 0 -b 24 --no-pool --log logs/dann_vit/OfficeHome_Pr2Cl\nCUDA_VISIBLE_DEVICES=0 python adda.py data/office-home -d OfficeHome -s Pr -t Rw -a vit_base_patch16_224 --epochs 30 --seed 0 -b 24 --no-pool --log logs/dann_vit/OfficeHome_Pr2Rw\nCUDA_VISIBLE_DEVICES=0 python adda.py data/office-home -d OfficeHome -s Rw -t Ar -a vit_base_patch16_224 --epochs 30 --seed 0 -b 24 --no-pool --log logs/dann_vit/OfficeHome_Rw2Ar\nCUDA_VISIBLE_DEVICES=0 python adda.py data/office-home -d OfficeHome -s Rw -t Cl -a vit_base_patch16_224 --epochs 30 --seed 0 -b 24 --no-pool --log logs/dann_vit/OfficeHome_Rw2Cl\nCUDA_VISIBLE_DEVICES=0 python adda.py data/office-home -d OfficeHome -s Rw -t Pr -a vit_base_patch16_224 --epochs 30 --seed 0 -b 24 --no-pool --log logs/dann_vit/OfficeHome_Rw2Pr\n\n# ResNet50, Office-Home, Multi Source\nCUDA_VISIBLE_DEVICES=0 python adda.py data/office-home -d OfficeHome -s Cl Pr Rw -t Ar -a resnet50 --epochs 30 --seed 0 --log logs/adda/OfficeHome_:2Ar\nCUDA_VISIBLE_DEVICES=0 python adda.py data/office-home -d OfficeHome -s Ar Pr Rw -t Cl -a resnet50 --epochs 30 --seed 0 --log logs/adda/OfficeHome_:2Cl\nCUDA_VISIBLE_DEVICES=0 python adda.py data/office-home -d OfficeHome -s Ar Cl Rw -t Pr -a resnet50 --epochs 30 --seed 0 --log logs/adda/OfficeHome_:2Pr\nCUDA_VISIBLE_DEVICES=0 python adda.py data/office-home -d OfficeHome -s Ar Cl Pr -t Rw -a resnet50 --epochs 30 --seed 0 --log logs/adda/OfficeHome_:2Rw\n\n# ResNet101, DomainNet, Multi Source\nCUDA_VISIBLE_DEVICES=0 python adda.py data/domainnet -d DomainNet -s i p q r s -t c -a resnet101 --bottleneck-dim 1024 --epochs 40 -i 5000 -p 500 --seed 0 --log logs/adda/DomainNet_:2c\nCUDA_VISIBLE_DEVICES=0 python adda.py data/domainnet -d DomainNet -s c p q r s -t i -a resnet101 --bottleneck-dim 1024 --epochs 40 -i 5000 -p 500 --seed 0 --log logs/adda/DomainNet_:2i\nCUDA_VISIBLE_DEVICES=0 python adda.py data/domainnet -d DomainNet -s c i q r s -t p -a resnet101 --bottleneck-dim 1024 --epochs 40 -i 5000 -p 500 --seed 0 --log logs/adda/DomainNet_:2p\nCUDA_VISIBLE_DEVICES=0 python adda.py data/domainnet -d DomainNet -s c i p r s -t q -a resnet101 --bottleneck-dim 1024 --epochs 40 -i 5000 -p 500 --seed 0 --log logs/adda/DomainNet_:2q\nCUDA_VISIBLE_DEVICES=0 python adda.py data/domainnet -d DomainNet -s c i p q s -t r -a resnet101 --bottleneck-dim 1024 --epochs 40 -i 5000 -p 500 --seed 0 --log logs/adda/DomainNet_:2r\nCUDA_VISIBLE_DEVICES=0 python adda.py data/domainnet -d DomainNet -s c i p q r -t s -a resnet101 --bottleneck-dim 1024 --epochs 40 -i 5000 -p 500 --seed 0 --log logs/adda/DomainNet_:2s\n\n# ResNet50, Wilds Dataset\nCUDA_VISIBLE_DEVICES=0 python adda.py data/wilds -d iwildcam -a resnet50 --epochs 30 -i 1000 --seed 0 --log logs/adda/iwildcam\n\n# DenseNet121, Wilds Dataset\nCUDA_VISIBLE_DEVICES=0 python adda.py data/wilds -d camelyon17 --train-resizing 'res.' --val-resizing 'res.' --resize-size 96 \\\n -a densenet121 --scratch --epochs 10 -i 1000 --lr 0.01 --seed 0 --log logs/adda/camelyon17\nCUDA_VISIBLE_DEVICES=0 python adda.py data/wilds -d fmow --train-resizing 'res.' --val-resizing 'res.' \\\n -a densenet121 --epochs 10 -i 1000 --lr 0.01 --seed 0 --log logs/adda/fmow\n\n# Digits\nCUDA_VISIBLE_DEVICES=0 python adda.py data/digits -d Digits -s MNIST -t USPS --train-resizing 'res.' --val-resizing 'res.' \\\n --resize-size 28 --no-hflip --norm-mean 0.5 --norm-std 0.5 -a lenet --no-pool --lr 0.01 -b 128 -i 2500 --scratch --seed 0 --log logs/adda/MNIST2USPS\nCUDA_VISIBLE_DEVICES=0 python adda.py data/digits -d Digits -s USPS -t MNIST --train-resizing 'res.' --val-resizing 'res.' \\\n --resize-size 28 --no-hflip --norm-mean 0.5 --norm-std 0.5 -a lenet --no-pool --lr 0.1 -b 128 -i 2500 --scratch --seed 0 --log logs/adda/USPS2MNIST\nCUDA_VISIBLE_DEVICES=3 python adda.py data/digits -d Digits -s SVHNRGB -t MNISTRGB --train-resizing 'res.' --val-resizing 'res.' \\\n --resize-size 32 --no-hflip --norm-mean 0.5 0.5 0.5 --norm-std 0.5 0.5 0.5 -a dtn --no-pool --trade-off 0.3 --lr 0.03 --lr-d 0.03 -b 128 -i 2500 --scratch --seed 0 --log logs/adda/SVHN2MNIST\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/examples/domain_adaptation/image_classification/addagrl.py",
"content": "\"\"\"\n@author: Baixu Chen\n@contact: cbx_99_hasta@outlook.com\nNote: ADDA with gradient reverse layer\n\"\"\"\nimport random\nimport time\nimport warnings\nimport sys\nimport copy\nimport argparse\nimport shutil\nimport os.path as osp\n\nimport torch\nimport torch.nn as nn\nimport torch.backends.cudnn as cudnn\nfrom torch.optim import SGD\nfrom torch.optim.lr_scheduler import LambdaLR\nfrom torch.utils.data import DataLoader\n\nsys.path.append('../../..')\nfrom dalib.modules.domain_discriminator import DomainDiscriminator\nfrom dalib.adaptation.adda import ImageClassifier\nfrom dalib.adaptation.dann import DomainAdversarialLoss\nfrom dalib.translation.cyclegan.util import set_requires_grad\nfrom dalib.modules.grl import WarmStartGradientReverseLayer\nfrom common.utils.data import ForeverDataIterator\nfrom common.utils.meter import AverageMeter, ProgressMeter\nfrom common.utils.logger import CompleteLogger\nfrom common.utils.analysis import collect_feature, tsne, a_distance\n\nsys.path.append('../classification')\nimport utils\n\ndevice = torch.device(\"cuda\" if torch.cuda.is_available() else \"cpu\")\n\n\ndef main(args: argparse.Namespace):\n logger = CompleteLogger(args.log, args.phase)\n print(args)\n\n if args.seed is not None:\n random.seed(args.seed)\n torch.manual_seed(args.seed)\n cudnn.deterministic = True\n warnings.warn('You have chosen to seed training. '\n 'This will turn on the CUDNN deterministic setting, '\n 'which can slow down your training considerably! '\n 'You may see unexpected behavior when restarting '\n 'from checkpoints.')\n\n cudnn.benchmark = True\n\n # Data loading code\n train_transform = utils.get_train_transform(args.train_resizing, random_horizontal_flip=not args.no_hflip,\n random_color_jitter=False, resize_size=args.resize_size,\n norm_mean=args.norm_mean, norm_std=args.norm_std)\n val_transform = utils.get_val_transform(args.val_resizing, resize_size=args.resize_size,\n norm_mean=args.norm_mean, norm_std=args.norm_std)\n print(\"train_transform: \", train_transform)\n print(\"val_transform: \", val_transform)\n\n train_source_dataset, train_target_dataset, val_dataset, test_dataset, num_classes, args.class_names = \\\n utils.get_dataset(args.data, args.root, args.source, args.target, train_transform, val_transform)\n train_source_loader = DataLoader(train_source_dataset, batch_size=args.batch_size,\n shuffle=True, num_workers=args.workers, drop_last=True)\n train_target_loader = DataLoader(train_target_dataset, batch_size=args.batch_size,\n shuffle=True, num_workers=args.workers, drop_last=True)\n val_loader = DataLoader(val_dataset, batch_size=args.batch_size, shuffle=False, num_workers=args.workers)\n test_loader = DataLoader(test_dataset, batch_size=args.batch_size, shuffle=False, num_workers=args.workers)\n\n train_source_iter = ForeverDataIterator(train_source_loader)\n train_target_iter = ForeverDataIterator(train_target_loader)\n\n # create model\n print(\"=> using model '{}'\".format(args.arch))\n backbone = utils.get_model(args.arch, pretrain=not args.scratch)\n pool_layer = nn.Identity() if args.no_pool else None\n source_classifier = ImageClassifier(backbone, num_classes, bottleneck_dim=args.bottleneck_dim,\n pool_layer=pool_layer, finetune=not args.scratch).to(device)\n\n if args.phase == 'train' and args.pretrain is None:\n # first pretrain the classifier wish source data\n print(\"Pretraining the model on source domain.\")\n args.pretrain = logger.get_checkpoint_path('pretrain')\n pretrain_model = ImageClassifier(backbone, num_classes, bottleneck_dim=args.bottleneck_dim,\n pool_layer=pool_layer, finetune=not args.scratch).to(device)\n pretrain_optimizer = SGD(pretrain_model.get_parameters(), args.pretrain_lr, momentum=args.momentum,\n weight_decay=args.weight_decay, nesterov=True)\n pretrain_lr_scheduler = LambdaLR(pretrain_optimizer,\n lambda x: args.pretrain_lr * (1. + args.lr_gamma * float(x)) ** (\n -args.lr_decay))\n # start pretraining\n for epoch in range(args.pretrain_epochs):\n print(\"lr:\", pretrain_lr_scheduler.get_lr())\n # pretrain for one epoch\n utils.pretrain(train_source_iter, pretrain_model, pretrain_optimizer, pretrain_lr_scheduler, epoch, args,\n device)\n # validate to show pretrain process\n utils.validate(val_loader, pretrain_model, args, device)\n\n torch.save(pretrain_model.state_dict(), args.pretrain)\n print(\"Pretraining process is done.\")\n\n checkpoint = torch.load(args.pretrain, map_location='cpu')\n source_classifier.load_state_dict(checkpoint)\n target_classifier = copy.deepcopy(source_classifier)\n\n # freeze source classifier\n set_requires_grad(source_classifier, False)\n source_classifier.freeze_bn()\n\n domain_discri = DomainDiscriminator(in_feature=source_classifier.features_dim, hidden_size=1024).to(device)\n\n # define loss function\n grl = WarmStartGradientReverseLayer(alpha=1., lo=0., hi=2., max_iters=1000, auto_step=True)\n domain_adv = DomainAdversarialLoss(domain_discri, grl=grl).to(device)\n\n # define optimizer and lr scheduler\n # note that we only optimize target feature extractor\n optimizer = SGD(target_classifier.get_parameters(optimize_head=False) + domain_discri.get_parameters(), args.lr,\n momentum=args.momentum, weight_decay=args.weight_decay, nesterov=True)\n lr_scheduler = LambdaLR(optimizer, lambda x: args.lr * (1. + args.lr_gamma * float(x)) ** (-args.lr_decay))\n\n # resume from the best checkpoint\n if args.phase != 'train':\n checkpoint = torch.load(logger.get_checkpoint_path('best'), map_location='cpu')\n target_classifier.load_state_dict(checkpoint)\n\n # analysis the model\n if args.phase == 'analysis':\n # extract features from both domains\n feature_extractor = nn.Sequential(target_classifier.backbone, target_classifier.pool_layer,\n target_classifier.bottleneck).to(device)\n source_feature = collect_feature(train_source_loader, feature_extractor, device)\n target_feature = collect_feature(train_target_loader, feature_extractor, device)\n # plot t-SNE\n tSNE_filename = osp.join(logger.visualize_directory, 'TSNE.pdf')\n tsne.visualize(source_feature, target_feature, tSNE_filename)\n print(\"Saving t-SNE to\", tSNE_filename)\n # calculate A-distance, which is a measure for distribution discrepancy\n A_distance = a_distance.calculate(source_feature, target_feature, device)\n print(\"A-distance =\", A_distance)\n return\n\n if args.phase == 'test':\n acc1 = utils.validate(test_loader, target_classifier, args, device)\n print(acc1)\n return\n\n # start training\n best_acc1 = 0.\n for epoch in range(args.epochs):\n print(lr_scheduler.get_lr())\n # train for one epoch\n train(train_source_iter, train_target_iter, source_classifier, target_classifier, domain_adv,\n optimizer, lr_scheduler, epoch, args)\n\n # evaluate on validation set\n acc1 = utils.validate(val_loader, target_classifier, args, device)\n\n # remember best acc@1 and save checkpoint\n torch.save(target_classifier.state_dict(), logger.get_checkpoint_path('latest'))\n if acc1 > best_acc1:\n shutil.copy(logger.get_checkpoint_path('latest'), logger.get_checkpoint_path('best'))\n best_acc1 = max(acc1, best_acc1)\n\n print(\"best_acc1 = {:3.1f}\".format(best_acc1))\n\n # evaluate on test set\n target_classifier.load_state_dict(torch.load(logger.get_checkpoint_path('best')))\n acc1 = utils.validate(test_loader, target_classifier, args, device)\n print(\"test_acc1 = {:3.1f}\".format(acc1))\n\n logger.close()\n\n\ndef train(train_source_iter: ForeverDataIterator, train_target_iter: ForeverDataIterator,\n source_model: ImageClassifier, target_model: ImageClassifier, domain_adv: DomainAdversarialLoss,\n optimizer: SGD, lr_scheduler: LambdaLR, epoch: int, args: argparse.Namespace):\n batch_time = AverageMeter('Time', ':5.2f')\n data_time = AverageMeter('Data', ':5.2f')\n losses_transfer = AverageMeter('Transfer Loss', ':6.2f')\n domain_accs = AverageMeter('Domain Acc', ':3.1f')\n progress = ProgressMeter(\n args.iters_per_epoch,\n [batch_time, data_time, losses_transfer, domain_accs],\n prefix=\"Epoch: [{}]\".format(epoch))\n\n # switch to train mode\n target_model.train()\n domain_adv.train()\n\n end = time.time()\n for i in range(args.iters_per_epoch):\n x_s, _ = next(train_source_iter)\n x_t, _ = next(train_target_iter)\n\n x_s = x_s.to(device)\n x_t = x_t.to(device)\n\n # measure data loading time\n data_time.update(time.time() - end)\n\n _, f_s = source_model(x_s)\n _, f_t = target_model(x_t)\n loss_transfer = domain_adv(f_s, f_t)\n\n # Compute gradient and do SGD step\n optimizer.zero_grad()\n loss_transfer.backward()\n optimizer.step()\n lr_scheduler.step()\n\n losses_transfer.update(loss_transfer.item(), x_s.size(0))\n domain_acc = domain_adv.domain_discriminator_accuracy\n domain_accs.update(domain_acc.item(), x_s.size(0))\n\n # measure elapsed time\n batch_time.update(time.time() - end)\n end = time.time()\n\n if i % args.print_freq == 0:\n progress.display(i)\n\n\nif __name__ == '__main__':\n parser = argparse.ArgumentParser(description='ADDA for Unsupervised Domain Adaptation')\n # dataset parameters\n parser.add_argument('root', metavar='DIR',\n help='root path of dataset')\n parser.add_argument('-d', '--data', metavar='DATA', default='Office31', choices=utils.get_dataset_names(),\n help='dataset: ' + ' | '.join(utils.get_dataset_names()) +\n ' (default: Office31)')\n parser.add_argument('-s', '--source', help='source domain(s)', nargs='+')\n parser.add_argument('-t', '--target', help='target domain(s)', nargs='+')\n parser.add_argument('--train-resizing', type=str, default='default')\n parser.add_argument('--val-resizing', type=str, default='default')\n parser.add_argument('--resize-size', type=int, default=224,\n help='the image size after resizing')\n parser.add_argument('--no-hflip', action='store_true',\n help='no random horizontal flipping during training')\n parser.add_argument('--norm-mean', type=float, nargs='+',\n default=(0.485, 0.456, 0.406), help='normalization mean')\n parser.add_argument('--norm-std', type=float, nargs='+',\n default=(0.229, 0.224, 0.225), help='normalization std')\n # model parameters\n parser.add_argument('-a', '--arch', metavar='ARCH', default='resnet18',\n choices=utils.get_model_names(),\n help='backbone architecture: ' +\n ' | '.join(utils.get_model_names()) +\n ' (default: resnet18)')\n parser.add_argument('--pretrain', type=str, default=None,\n help='pretrain checkpoint for classification model')\n parser.add_argument('--bottleneck-dim', default=256, type=int,\n help='Dimension of bottleneck')\n parser.add_argument('--no-pool', action='store_true',\n help='no pool layer after the feature extractor.')\n parser.add_argument('--scratch', action='store_true', help='whether train from scratch.')\n # training parameters\n parser.add_argument('-b', '--batch-size', default=32, type=int,\n metavar='N',\n help='mini-batch size (default: 32)')\n parser.add_argument('--lr', '--learning-rate', default=0.001, type=float,\n metavar='LR', help='initial learning rate of the classifier', dest='lr')\n parser.add_argument('--pretrain-lr', default=0.001, type=float, help='initial pretrain learning rate')\n parser.add_argument('--lr-gamma', default=0.0003, type=float, help='parameter for lr scheduler')\n parser.add_argument('--lr-decay', default=0.75, type=float, help='parameter for lr scheduler')\n parser.add_argument('--momentum', default=0.9, type=float, metavar='M',\n help='momentum')\n parser.add_argument('--wd', '--weight-decay', default=1e-3, type=float,\n metavar='W', help='weight decay (default: 1e-3)',\n dest='weight_decay')\n parser.add_argument('-j', '--workers', default=2, type=int, metavar='N',\n help='number of data loading workers (default: 4)')\n parser.add_argument('--epochs', default=20, type=int, metavar='N',\n help='number of total epochs to run')\n parser.add_argument('--pretrain-epochs', default=3, type=int, metavar='N',\n help='number of total epochs(pretrain) to run')\n parser.add_argument('-i', '--iters-per-epoch', default=1000, type=int,\n help='Number of iterations per epoch')\n parser.add_argument('-p', '--print-freq', default=100, type=int,\n metavar='N', help='print frequency (default: 100)')\n parser.add_argument('--seed', default=None, type=int,\n help='seed for initializing training. ')\n parser.add_argument('--per-class-eval', action='store_true',\n help='whether output per-class accuracy during evaluation')\n parser.add_argument(\"--log\", type=str, default='addagrl',\n help=\"Where to save logs, checkpoints and debugging images.\")\n parser.add_argument(\"--phase\", type=str, default='train', choices=['train', 'test', 'analysis'],\n help=\"When phase is 'test', only test the model.\"\n \"When phase is 'analysis', only analysis the model.\")\n args = parser.parse_args()\n main(args)\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/examples/domain_adaptation/image_classification/addagrl.sh",
"content": "#!/usr/bin/env bash\n# ResNet50, Office31, Single Source\nCUDA_VISIBLE_DEVICES=0 python addagrl.py data/office31 -d Office31 -s A -t W -a resnet50 --epochs 20 --seed 1 --log logs/addagrl/Office31_A2W\nCUDA_VISIBLE_DEVICES=0 python addagrl.py data/office31 -d Office31 -s D -t W -a resnet50 --epochs 20 --seed 1 --log logs/addagrl/Office31_D2W\nCUDA_VISIBLE_DEVICES=0 python addagrl.py data/office31 -d Office31 -s W -t D -a resnet50 --epochs 20 --seed 1 --log logs/addagrl/Office31_W2D\nCUDA_VISIBLE_DEVICES=0 python addagrl.py data/office31 -d Office31 -s A -t D -a resnet50 --epochs 20 --seed 1 --log logs/addagrl/Office31_A2D\nCUDA_VISIBLE_DEVICES=0 python addagrl.py data/office31 -d Office31 -s D -t A -a resnet50 --epochs 20 --seed 1 --log logs/addagrl/Office31_D2A\nCUDA_VISIBLE_DEVICES=0 python addagrl.py data/office31 -d Office31 -s W -t A -a resnet50 --epochs 20 --seed 1 --log logs/addagrl/Office31_W2A\n\n# ResNet50, Office-Home, Single Source\nCUDA_VISIBLE_DEVICES=0 python addagrl.py data/office-home -d OfficeHome -s Ar -t Cl -a resnet50 --epochs 30 --seed 0 --log logs/addagrl/OfficeHome_Ar2Cl\nCUDA_VISIBLE_DEVICES=0 python addagrl.py data/office-home -d OfficeHome -s Ar -t Pr -a resnet50 --epochs 30 --seed 0 --log logs/addagrl/OfficeHome_Ar2Pr\nCUDA_VISIBLE_DEVICES=0 python addagrl.py data/office-home -d OfficeHome -s Ar -t Rw -a resnet50 --epochs 30 --seed 0 --log logs/addagrl/OfficeHome_Ar2Rw\nCUDA_VISIBLE_DEVICES=0 python addagrl.py data/office-home -d OfficeHome -s Cl -t Ar -a resnet50 --epochs 30 --seed 0 --log logs/addagrl/OfficeHome_Cl2Ar\nCUDA_VISIBLE_DEVICES=0 python addagrl.py data/office-home -d OfficeHome -s Cl -t Pr -a resnet50 --epochs 30 --seed 0 --log logs/addagrl/OfficeHome_Cl2Pr\nCUDA_VISIBLE_DEVICES=0 python addagrl.py data/office-home -d OfficeHome -s Cl -t Rw -a resnet50 --epochs 30 --seed 0 --log logs/addagrl/OfficeHome_Cl2Rw\nCUDA_VISIBLE_DEVICES=0 python addagrl.py data/office-home -d OfficeHome -s Pr -t Ar -a resnet50 --epochs 30 --seed 0 --log logs/addagrl/OfficeHome_Pr2Ar\nCUDA_VISIBLE_DEVICES=0 python addagrl.py data/office-home -d OfficeHome -s Pr -t Cl -a resnet50 --epochs 30 --seed 0 --log logs/addagrl/OfficeHome_Pr2Cl\nCUDA_VISIBLE_DEVICES=0 python addagrl.py data/office-home -d OfficeHome -s Pr -t Rw -a resnet50 --epochs 30 --seed 0 --log logs/addagrl/OfficeHome_Pr2Rw\nCUDA_VISIBLE_DEVICES=0 python addagrl.py data/office-home -d OfficeHome -s Rw -t Ar -a resnet50 --epochs 30 --seed 0 --log logs/addagrl/OfficeHome_Rw2Ar\nCUDA_VISIBLE_DEVICES=0 python addagrl.py data/office-home -d OfficeHome -s Rw -t Cl -a resnet50 --epochs 30 --seed 0 --log logs/addagrl/OfficeHome_Rw2Cl\nCUDA_VISIBLE_DEVICES=0 python addagrl.py data/office-home -d OfficeHome -s Rw -t Pr -a resnet50 --epochs 30 --seed 0 --log logs/addagrl/OfficeHome_Rw2Pr\n\n# ResNet101, VisDA-2017, Single Source\nCUDA_VISIBLE_DEVICES=0 python addagrl.py data/visda-2017 -d VisDA2017 -s Synthetic -t Real -a resnet101 \\\n --epochs 30 --seed 0 --per-class-eval --train-resizing cen.crop --log logs/addagrl/VisDA2017\n\n# ResNet101, DomainNet, Single Source\nCUDA_VISIBLE_DEVICES=0 python addagrl.py data/domainnet -d DomainNet -s c -t p -a resnet101 --bottleneck-dim 1024 --epochs 30 -i 2500 -p 500 --seed 0 --log logs/addagrl/DomainNet_c2p\nCUDA_VISIBLE_DEVICES=0 python addagrl.py data/domainnet -d DomainNet -s c -t r -a resnet101 --bottleneck-dim 1024 --epochs 30 -i 2500 -p 500 --seed 0 --log logs/addagrl/DomainNet_c2r\nCUDA_VISIBLE_DEVICES=0 python addagrl.py data/domainnet -d DomainNet -s c -t s -a resnet101 --bottleneck-dim 1024 --epochs 30 -i 2500 -p 500 --seed 0 --log logs/addagrl/DomainNet_c2s\nCUDA_VISIBLE_DEVICES=0 python addagrl.py data/domainnet -d DomainNet -s p -t c -a resnet101 --bottleneck-dim 1024 --epochs 30 -i 2500 -p 500 --seed 0 --log logs/addagrl/DomainNet_p2c\nCUDA_VISIBLE_DEVICES=0 python addagrl.py data/domainnet -d DomainNet -s p -t r -a resnet101 --bottleneck-dim 1024 --epochs 30 -i 2500 -p 500 --seed 0 --log logs/addagrl/DomainNet_p2r\nCUDA_VISIBLE_DEVICES=0 python addagrl.py data/domainnet -d DomainNet -s p -t s -a resnet101 --bottleneck-dim 1024 --epochs 30 -i 2500 -p 500 --seed 0 --log logs/addagrl/DomainNet_p2s\nCUDA_VISIBLE_DEVICES=0 python addagrl.py data/domainnet -d DomainNet -s r -t c -a resnet101 --bottleneck-dim 1024 --epochs 30 -i 2500 -p 500 --seed 0 --log logs/addagrl/DomainNet_r2c\nCUDA_VISIBLE_DEVICES=0 python addagrl.py data/domainnet -d DomainNet -s r -t p -a resnet101 --bottleneck-dim 1024 --epochs 30 -i 2500 -p 500 --seed 0 --log logs/addagrl/DomainNet_r2p\nCUDA_VISIBLE_DEVICES=0 python addagrl.py data/domainnet -d DomainNet -s r -t s -a resnet101 --bottleneck-dim 1024 --epochs 30 -i 2500 -p 500 --seed 0 --log logs/addagrl/DomainNet_r2s\nCUDA_VISIBLE_DEVICES=0 python addagrl.py data/domainnet -d DomainNet -s s -t c -a resnet101 --bottleneck-dim 1024 --epochs 30 -i 2500 -p 500 --seed 0 --log logs/addagrl/DomainNet_s2c\nCUDA_VISIBLE_DEVICES=0 python addagrl.py data/domainnet -d DomainNet -s s -t p -a resnet101 --bottleneck-dim 1024 --epochs 30 -i 2500 -p 500 --seed 0 --log logs/addagrl/DomainNet_s2p\nCUDA_VISIBLE_DEVICES=0 python addagrl.py data/domainnet -d DomainNet -s s -t r -a resnet101 --bottleneck-dim 1024 --epochs 30 -i 2500 -p 500 --seed 0 --log logs/addagrl/DomainNet_s2r\n\n# ResNet50, ImageNet200 -> ImageNetR\nCUDA_VISIBLE_DEVICES=0 python addagrl.py data/ImageNetR -d ImageNetR -s IN -t INR -a resnet50 --epochs 30 -i 2500 -p 500 --seed 0 --log logs/addagrl/ImageNet_IN2INR\n\n# ig_resnext101_32x8d, ImageNet -> ImageNetSketch\nCUDA_VISIBLE_DEVICES=0 python addagrl.py data/imagenet-sketch -d ImageNetSketch -s IN -t sketch -a ig_resnext101_32x8d --epochs 30 -i 2500 -p 500 --bottleneck-dim 1024 --log logs/dann_ig_resnext101_32x8d/ImageNet_IN2sketch\n\n# Vision Transformer, Office-Home, Single Source\nCUDA_VISIBLE_DEVICES=0 python addagrl.py data/office-home -d OfficeHome -s Ar -t Cl -a vit_base_patch16_224 --epochs 30 --seed 0 -b 24 --no-pool --log logs/dann_vit/OfficeHome_Ar2Cl\nCUDA_VISIBLE_DEVICES=0 python addagrl.py data/office-home -d OfficeHome -s Ar -t Pr -a vit_base_patch16_224 --epochs 30 --seed 0 -b 24 --no-pool --log logs/dann_vit/OfficeHome_Ar2Pr\nCUDA_VISIBLE_DEVICES=0 python addagrl.py data/office-home -d OfficeHome -s Ar -t Rw -a vit_base_patch16_224 --epochs 30 --seed 0 -b 24 --no-pool --log logs/dann_vit/OfficeHome_Ar2Rw\nCUDA_VISIBLE_DEVICES=0 python addagrl.py data/office-home -d OfficeHome -s Cl -t Ar -a vit_base_patch16_224 --epochs 30 --seed 0 -b 24 --no-pool --log logs/dann_vit/OfficeHome_Cl2Ar\nCUDA_VISIBLE_DEVICES=0 python addagrl.py data/office-home -d OfficeHome -s Cl -t Pr -a vit_base_patch16_224 --epochs 30 --seed 0 -b 24 --no-pool --log logs/dann_vit/OfficeHome_Cl2Pr\nCUDA_VISIBLE_DEVICES=0 python addagrl.py data/office-home -d OfficeHome -s Cl -t Rw -a vit_base_patch16_224 --epochs 30 --seed 0 -b 24 --no-pool --log logs/dann_vit/OfficeHome_Cl2Rw\nCUDA_VISIBLE_DEVICES=0 python addagrl.py data/office-home -d OfficeHome -s Pr -t Ar -a vit_base_patch16_224 --epochs 30 --seed 0 -b 24 --no-pool --log logs/dann_vit/OfficeHome_Pr2Ar\nCUDA_VISIBLE_DEVICES=0 python addagrl.py data/office-home -d OfficeHome -s Pr -t Cl -a vit_base_patch16_224 --epochs 30 --seed 0 -b 24 --no-pool --log logs/dann_vit/OfficeHome_Pr2Cl\nCUDA_VISIBLE_DEVICES=0 python addagrl.py data/office-home -d OfficeHome -s Pr -t Rw -a vit_base_patch16_224 --epochs 30 --seed 0 -b 24 --no-pool --log logs/dann_vit/OfficeHome_Pr2Rw\nCUDA_VISIBLE_DEVICES=0 python addagrl.py data/office-home -d OfficeHome -s Rw -t Ar -a vit_base_patch16_224 --epochs 30 --seed 0 -b 24 --no-pool --log logs/dann_vit/OfficeHome_Rw2Ar\nCUDA_VISIBLE_DEVICES=0 python addagrl.py data/office-home -d OfficeHome -s Rw -t Cl -a vit_base_patch16_224 --epochs 30 --seed 0 -b 24 --no-pool --log logs/dann_vit/OfficeHome_Rw2Cl\nCUDA_VISIBLE_DEVICES=0 python addagrl.py data/office-home -d OfficeHome -s Rw -t Pr -a vit_base_patch16_224 --epochs 30 --seed 0 -b 24 --no-pool --log logs/dann_vit/OfficeHome_Rw2Pr\n\n# ResNet50, Office-Home, Multi Source\nCUDA_VISIBLE_DEVICES=0 python addagrl.py data/office-home -d OfficeHome -s Cl Pr Rw -t Ar -a resnet50 --epochs 30 --seed 0 --log logs/addagrl/OfficeHome_:2Ar\nCUDA_VISIBLE_DEVICES=0 python addagrl.py data/office-home -d OfficeHome -s Ar Pr Rw -t Cl -a resnet50 --epochs 30 --seed 0 --log logs/addagrl/OfficeHome_:2Cl\nCUDA_VISIBLE_DEVICES=0 python addagrl.py data/office-home -d OfficeHome -s Ar Cl Rw -t Pr -a resnet50 --epochs 30 --seed 0 --log logs/addagrl/OfficeHome_:2Pr\nCUDA_VISIBLE_DEVICES=0 python addagrl.py data/office-home -d OfficeHome -s Ar Cl Pr -t Rw -a resnet50 --epochs 30 --seed 0 --log logs/addagrl/OfficeHome_:2Rw\n\n# ResNet101, DomainNet, Multi Source\nCUDA_VISIBLE_DEVICES=0 python addagrl.py data/domainnet -d DomainNet -s i p q r s -t c -a resnet101 --bottleneck-dim 1024 --epochs 40 -i 5000 -p 500 --seed 0 --log logs/addagrl/DomainNet_:2c\nCUDA_VISIBLE_DEVICES=0 python addagrl.py data/domainnet -d DomainNet -s c p q r s -t i -a resnet101 --bottleneck-dim 1024 --epochs 40 -i 5000 -p 500 --seed 0 --log logs/addagrl/DomainNet_:2i\nCUDA_VISIBLE_DEVICES=0 python addagrl.py data/domainnet -d DomainNet -s c i q r s -t p -a resnet101 --bottleneck-dim 1024 --epochs 40 -i 5000 -p 500 --seed 0 --log logs/addagrl/DomainNet_:2p\nCUDA_VISIBLE_DEVICES=0 python addagrl.py data/domainnet -d DomainNet -s c i p r s -t q -a resnet101 --bottleneck-dim 1024 --epochs 40 -i 5000 -p 500 --seed 0 --log logs/addagrl/DomainNet_:2q\nCUDA_VISIBLE_DEVICES=0 python addagrl.py data/domainnet -d DomainNet -s c i p q s -t r -a resnet101 --bottleneck-dim 1024 --epochs 40 -i 5000 -p 500 --seed 0 --log logs/addagrl/DomainNet_:2r\nCUDA_VISIBLE_DEVICES=0 python addagrl.py data/domainnet -d DomainNet -s c i p q r -t s -a resnet101 --bottleneck-dim 1024 --epochs 40 -i 5000 -p 500 --seed 0 --log logs/addagrl/DomainNet_:2s\n\n# ResNet50, Wilds Dataset\nCUDA_VISIBLE_DEVICES=0 python addagrl.py data/wilds -d iwildcam -a resnet50 --epochs 30 -i 1000 --seed 0 --log logs/addagrl/iwildcam\n\n# DenseNet121, Wilds Dataset\nCUDA_VISIBLE_DEVICES=0 python addagrl.py data/wilds -d camelyon17 --train-resizing 'res.' --val-resizing 'res.' --resize-size 96 \\\n -a densenet121 --scratch --epochs 10 -i 1000 --lr 0.01 --seed 0 --log logs/addagrl/camelyon17\nCUDA_VISIBLE_DEVICES=0 python addagrl.py data/wilds -d fmow --train-resizing 'res.' --val-resizing 'res.' \\\n -a densenet121 --epochs 10 -i 1000 --lr 0.01 --seed 0 --log logs/addagrl/fmow\n\n# Digits\nCUDA_VISIBLE_DEVICES=0 python addagrl.py data/digits -d Digits -s MNIST -t USPS --train-resizing 'res.' --val-resizing 'res.' \\\n --resize-size 28 --no-hflip --norm-mean 0.5 --norm-std 0.5 -a lenet --no-pool --lr 0.01 -b 128 -i 2500 --scratch --seed 0 --log logs/addagrl/MNIST2USPS\nCUDA_VISIBLE_DEVICES=0 python addagrl.py data/digits -d Digits -s USPS -t MNIST --train-resizing 'res.' --val-resizing 'res.' \\\n --resize-size 28 --no-hflip --norm-mean 0.5 --norm-std 0.5 -a lenet --no-pool --lr 0.1 -b 128 -i 2500 --scratch --seed 0 --log logs/addagrl/USPS2MNIST\nCUDA_VISIBLE_DEVICES=0 python addagrl.py data/digits -d Digits -s SVHNRGB -t MNISTRGB --train-resizing 'res.' --val-resizing 'res.' \\\n --resize-size 32 --no-hflip --norm-mean 0.5 0.5 0.5 --norm-std 0.5 0.5 0.5 -a dtn --no-pool --lr 0.03 --lr-d 0.03 -b 128 -i 2500 --scratch --seed 0 --log logs/addagrl/SVHN2MNIST\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/examples/domain_adaptation/image_classification/afn.py",
"content": "\"\"\"\n@author: Baixu Chen\n@contact: cbx_99_hasta@outlook.com\n\"\"\"\nimport random\nimport time\nimport warnings\nimport sys\nimport argparse\nimport shutil\nimport os.path as osp\n\nimport torch\nimport torch.nn as nn\nimport torch.backends.cudnn as cudnn\nfrom torch.optim import SGD\nfrom torch.utils.data import DataLoader\nimport torch.nn.functional as F\n\nsys.path.append('../../..')\nfrom dalib.adaptation.afn import AdaptiveFeatureNorm, ImageClassifier\nfrom dalib.modules.entropy import entropy\nfrom common.utils.data import ForeverDataIterator\nfrom common.utils.metric import accuracy\nfrom common.utils.meter import AverageMeter, ProgressMeter\nfrom common.utils.logger import CompleteLogger\nfrom common.utils.analysis import collect_feature, tsne, a_distance\n\nsys.path.append('.')\nimport utils\n\ndevice = torch.device(\"cuda\" if torch.cuda.is_available() else \"cpu\")\n\n\ndef main(args: argparse.Namespace):\n logger = CompleteLogger(args.log, args.phase)\n print(args)\n\n if args.seed is not None:\n random.seed(args.seed)\n torch.manual_seed(args.seed)\n cudnn.deterministic = True\n warnings.warn('You have chosen to seed training. '\n 'This will turn on the CUDNN deterministic setting, '\n 'which can slow down your training considerably! '\n 'You may see unexpected behavior when restarting '\n 'from checkpoints.')\n\n cudnn.benchmark = True\n\n # Data loading code\n train_transform = utils.get_train_transform(args.train_resizing, random_horizontal_flip=not args.no_hflip,\n random_color_jitter=False, resize_size=args.resize_size,\n norm_mean=args.norm_mean, norm_std=args.norm_std)\n val_transform = utils.get_val_transform(args.val_resizing, resize_size=args.resize_size,\n norm_mean=args.norm_mean, norm_std=args.norm_std)\n print(\"train_transform: \", train_transform)\n print(\"val_transform: \", val_transform)\n\n train_source_dataset, train_target_dataset, val_dataset, test_dataset, num_classes, args.class_names = \\\n utils.get_dataset(args.data, args.root, args.source, args.target, train_transform, val_transform)\n train_source_loader = DataLoader(train_source_dataset, batch_size=args.batch_size,\n shuffle=True, num_workers=args.workers, drop_last=True)\n train_target_loader = DataLoader(train_target_dataset, batch_size=args.batch_size,\n shuffle=True, num_workers=args.workers, drop_last=True)\n val_loader = DataLoader(val_dataset, batch_size=args.batch_size, shuffle=False, num_workers=args.workers)\n test_loader = DataLoader(test_dataset, batch_size=args.batch_size, shuffle=False, num_workers=args.workers)\n\n train_source_iter = ForeverDataIterator(train_source_loader)\n train_target_iter = ForeverDataIterator(train_target_loader)\n\n # create model\n print(\"=> using model '{}'\".format(args.arch))\n backbone = utils.get_model(args.arch, pretrain=not args.scratch)\n pool_layer = nn.Identity() if args.no_pool else None\n classifier = ImageClassifier(backbone, num_classes, args.num_blocks,\n bottleneck_dim=args.bottleneck_dim, dropout_p=args.dropout_p,\n pool_layer=pool_layer, finetune=not args.scratch).to(device)\n adaptive_feature_norm = AdaptiveFeatureNorm(args.delta).to(device)\n\n # define optimizer\n # the learning rate is fixed according to origin paper\n optimizer = SGD(classifier.get_parameters(), args.lr, weight_decay=args.weight_decay)\n\n # resume from the best checkpoint\n if args.phase != 'train':\n checkpoint = torch.load(logger.get_checkpoint_path('best'), map_location='cpu')\n classifier.load_state_dict(checkpoint)\n\n # analysis the model\n if args.phase == 'analysis':\n # extract features from both domains\n feature_extractor = nn.Sequential(classifier.backbone, classifier.pool_layer, classifier.bottleneck).to(device)\n source_feature = collect_feature(train_source_loader, feature_extractor, device)\n target_feature = collect_feature(train_target_loader, feature_extractor, device)\n # plot t-SNE\n tSNE_filename = osp.join(logger.visualize_directory, 'TSNE.pdf')\n tsne.visualize(source_feature, target_feature, tSNE_filename)\n print(\"Saving t-SNE to\", tSNE_filename)\n # calculate A-distance, which is a measure for distribution discrepancy\n A_distance = a_distance.calculate(source_feature, target_feature, device)\n print(\"A-distance =\", A_distance)\n return\n\n if args.phase == 'test':\n acc1 = utils.validate(test_loader, classifier, args, device)\n print(acc1)\n return\n\n # start training\n best_acc1 = 0.\n for epoch in range(args.epochs):\n # train for one epoch\n train(train_source_iter, train_target_iter, classifier, adaptive_feature_norm, optimizer, epoch, args)\n\n # evaluate on validation set\n acc1 = utils.validate(val_loader, classifier, args, device)\n\n # remember best acc@1 and save checkpoint\n torch.save(classifier.state_dict(), logger.get_checkpoint_path('latest'))\n if acc1 > best_acc1:\n shutil.copy(logger.get_checkpoint_path('latest'), logger.get_checkpoint_path('best'))\n best_acc1 = max(acc1, best_acc1)\n\n print(\"best_acc1 = {:3.1f}\".format(best_acc1))\n\n # evaluate on test set\n classifier.load_state_dict(torch.load(logger.get_checkpoint_path('best')))\n acc1 = utils.validate(test_loader, classifier, args, device)\n print(\"test_acc1 = {:3.1f}\".format(acc1))\n\n logger.close()\n\n\ndef train(train_source_iter: ForeverDataIterator, train_target_iter: ForeverDataIterator, model: ImageClassifier,\n adaptive_feature_norm: AdaptiveFeatureNorm, optimizer: SGD, epoch: int, args: argparse.Namespace):\n batch_time = AverageMeter('Time', ':3.1f')\n data_time = AverageMeter('Data', ':3.1f')\n cls_losses = AverageMeter('Cls Loss', ':3.2f')\n norm_losses = AverageMeter('Norm Loss', ':3.2f')\n src_feature_norm = AverageMeter('Source Feature Norm', ':3.2f')\n tgt_feature_norm = AverageMeter('Target Feature Norm', ':3.2f')\n cls_accs = AverageMeter('Cls Acc', ':3.1f')\n tgt_accs = AverageMeter('Tgt Acc', ':3.1f')\n\n progress = ProgressMeter(\n args.iters_per_epoch,\n [batch_time, data_time, cls_losses, norm_losses, src_feature_norm, tgt_feature_norm, cls_accs, tgt_accs],\n prefix=\"Epoch: [{}]\".format(epoch))\n\n # switch to train mode\n model.train()\n\n end = time.time()\n for i in range(args.iters_per_epoch):\n x_s, labels_s = next(train_source_iter)\n x_t, labels_t = next(train_target_iter)\n\n x_s = x_s.to(device)\n x_t = x_t.to(device)\n labels_s = labels_s.to(device)\n labels_t = labels_t.to(device)\n\n # measure data loading time\n data_time.update(time.time() - end)\n\n # compute output\n y_s, f_s = model(x_s)\n y_t, f_t = model(x_t)\n\n # classification loss\n cls_loss = F.cross_entropy(y_s, labels_s)\n # norm loss\n norm_loss = adaptive_feature_norm(f_s) + adaptive_feature_norm(f_t)\n loss = cls_loss + norm_loss * args.trade_off_norm\n\n # using entropy minimization\n if args.trade_off_entropy:\n y_t = F.softmax(y_t, dim=1)\n entropy_loss = entropy(y_t, reduction='mean')\n loss += entropy_loss * args.trade_off_entropy\n\n # compute gradient and do SGD step\n optimizer.zero_grad()\n loss.backward()\n optimizer.step()\n\n # update statistics\n cls_acc = accuracy(y_s, labels_s)[0]\n tgt_acc = accuracy(y_t, labels_t)[0]\n\n cls_losses.update(cls_loss.item(), x_s.size(0))\n norm_losses.update(norm_loss.item(), x_s.size(0))\n src_feature_norm.update(f_s.norm(p=2, dim=1).mean().item(), x_s.size(0))\n tgt_feature_norm.update(f_t.norm(p=2, dim=1).mean().item(), x_s.size(0))\n cls_accs.update(cls_acc.item(), x_s.size(0))\n tgt_accs.update(tgt_acc.item(), x_s.size(0))\n\n # measure elapsed time\n batch_time.update(time.time() - end)\n end = time.time()\n\n if i % args.print_freq == 0:\n progress.display(i)\n\n\nif __name__ == '__main__':\n parser = argparse.ArgumentParser(description='DANN for Unsupervised Domain Adaptation')\n # dataset parameters\n parser.add_argument('root', metavar='DIR',\n help='root path of dataset')\n parser.add_argument('-d', '--data', metavar='DATA', default='Office31', choices=utils.get_dataset_names(),\n help='dataset: ' + ' | '.join(utils.get_dataset_names()) +\n ' (default: Office31)')\n parser.add_argument('-s', '--source', help='source domain(s)', nargs='+')\n parser.add_argument('-t', '--target', help='target domain(s)', nargs='+')\n parser.add_argument('--train-resizing', type=str, default='ran.crop')\n parser.add_argument('--val-resizing', type=str, default='default')\n parser.add_argument('--resize-size', type=int, default=224,\n help='the image size after resizing')\n parser.add_argument('--no-hflip', action='store_true',\n help='no random horizontal flipping during training')\n parser.add_argument('--norm-mean', type=float, nargs='+',\n default=(0.485, 0.456, 0.406), help='normalization mean')\n parser.add_argument('--norm-std', type=float, nargs='+',\n default=(0.229, 0.224, 0.225), help='normalization std')\n # model parameters\n parser.add_argument('-a', '--arch', metavar='ARCH', default='resnet18',\n choices=utils.get_model_names(),\n help='backbone architecture: ' +\n ' | '.join(utils.get_model_names()) +\n ' (default: resnet18)')\n parser.add_argument('--no-pool', action='store_true',\n help='no pool layer after the feature extractor.')\n parser.add_argument('--scratch', action='store_true', help='whether train from scratch.')\n parser.add_argument('-n', '--num-blocks', default=1, type=int, help='Number of basic blocks for classifier')\n parser.add_argument('--bottleneck-dim', default=1000, type=int, help='Dimension of bottleneck')\n parser.add_argument('--dropout-p', default=0.5, type=float,\n help='Dropout probability')\n # training parameters\n parser.add_argument('-b', '--batch-size', default=32, type=int,\n metavar='N',\n help='mini-batch size (default: 32)')\n parser.add_argument('--lr', '--learning-rate', default=0.001, type=float,\n metavar='LR', help='initial learning rate', dest='lr')\n parser.add_argument('--momentum', default=0.9, type=float, metavar='M',\n help='momentum')\n parser.add_argument('--wd', '--weight-decay', default=5e-4, type=float,\n metavar='W', help='weight decay (default: 5e-4)',\n dest='weight_decay')\n parser.add_argument('--trade-off-norm', default=0.05, type=float,\n help='the trade-off hyper-parameter for norm loss')\n parser.add_argument('--trade-off-entropy', default=None, type=float,\n help='the trade-off hyper-parameter for entropy loss')\n parser.add_argument('-r', '--delta', default=1, type=float, help='Increment for L2 norm')\n parser.add_argument('-j', '--workers', default=2, type=int, metavar='N',\n help='number of data loading workers (default: 2)')\n parser.add_argument('--epochs', default=20, type=int, metavar='N',\n help='number of total epochs to run')\n parser.add_argument('-i', '--iters-per-epoch', default=500, type=int,\n help='Number of iterations per epoch')\n parser.add_argument('-p', '--print-freq', default=100, type=int,\n metavar='N', help='print frequency (default: 100)')\n parser.add_argument('--seed', default=None, type=int,\n help='seed for initializing training. ')\n parser.add_argument('--per-class-eval', action='store_true',\n help='whether output per-class accuracy during evaluation')\n parser.add_argument(\"--log\", type=str, default='afn',\n help=\"Where to save logs, checkpoints and debugging images.\")\n parser.add_argument(\"--phase\", type=str, default='train', choices=['train', 'test', 'analysis'],\n help=\"When phase is 'test', only test the model.\"\n \"When phase is 'analysis', only analysis the model.\")\n args = parser.parse_args()\n main(args)\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/examples/domain_adaptation/image_classification/afn.sh",
"content": "#!/usr/bin/env bash\n# ResNet50, Office31, Single Source\nCUDA_VISIBLE_DEVICES=0 python afn.py data/office31 -d Office31 -s A -t W -a resnet50 --trade-off-entropy 0.1 --epochs 20 --seed 1 --log logs/afn/Office31_A2W\nCUDA_VISIBLE_DEVICES=0 python afn.py data/office31 -d Office31 -s D -t W -a resnet50 --trade-off-entropy 0.1 --epochs 20 --seed 1 --log logs/afn/Office31_D2W\nCUDA_VISIBLE_DEVICES=0 python afn.py data/office31 -d Office31 -s W -t D -a resnet50 --trade-off-entropy 0.1 --epochs 20 --seed 1 --log logs/afn/Office31_W2D\nCUDA_VISIBLE_DEVICES=0 python afn.py data/office31 -d Office31 -s A -t D -a resnet50 --trade-off-entropy 0.1 --epochs 20 --seed 1 --log logs/afn/Office31_A2D\nCUDA_VISIBLE_DEVICES=0 python afn.py data/office31 -d Office31 -s D -t A -a resnet50 --trade-off-entropy 0.1 --epochs 20 --seed 1 --log logs/afn/Office31_D2A\nCUDA_VISIBLE_DEVICES=0 python afn.py data/office31 -d Office31 -s W -t A -a resnet50 --trade-off-entropy 0.1 --epochs 20 --seed 1 --log logs/afn/Office31_W2A\n\n# ResNet50, Office-Home, Single Source\nCUDA_VISIBLE_DEVICES=0 python afn.py data/office-home -d OfficeHome -s Ar -t Cl -a resnet50 --epochs 20 --seed 0 --log logs/afn/OfficeHome_Ar2Cl\nCUDA_VISIBLE_DEVICES=0 python afn.py data/office-home -d OfficeHome -s Ar -t Pr -a resnet50 --epochs 20 --seed 0 --log logs/afn/OfficeHome_Ar2Pr\nCUDA_VISIBLE_DEVICES=0 python afn.py data/office-home -d OfficeHome -s Ar -t Rw -a resnet50 --epochs 20 --seed 0 --log logs/afn/OfficeHome_Ar2Rw\nCUDA_VISIBLE_DEVICES=0 python afn.py data/office-home -d OfficeHome -s Cl -t Ar -a resnet50 --epochs 20 --seed 0 --log logs/afn/OfficeHome_Cl2Ar\nCUDA_VISIBLE_DEVICES=0 python afn.py data/office-home -d OfficeHome -s Cl -t Pr -a resnet50 --epochs 20 --seed 0 --log logs/afn/OfficeHome_Cl2Pr\nCUDA_VISIBLE_DEVICES=0 python afn.py data/office-home -d OfficeHome -s Cl -t Rw -a resnet50 --epochs 20 --seed 0 --log logs/afn/OfficeHome_Cl2Rw\nCUDA_VISIBLE_DEVICES=0 python afn.py data/office-home -d OfficeHome -s Pr -t Ar -a resnet50 --epochs 20 --seed 0 --log logs/afn/OfficeHome_Pr2Ar\nCUDA_VISIBLE_DEVICES=0 python afn.py data/office-home -d OfficeHome -s Pr -t Cl -a resnet50 --epochs 20 --seed 0 --log logs/afn/OfficeHome_Pr2Cl\nCUDA_VISIBLE_DEVICES=0 python afn.py data/office-home -d OfficeHome -s Pr -t Rw -a resnet50 --epochs 20 --seed 0 --log logs/afn/OfficeHome_Pr2Rw\nCUDA_VISIBLE_DEVICES=0 python afn.py data/office-home -d OfficeHome -s Rw -t Ar -a resnet50 --epochs 20 --seed 0 --log logs/afn/OfficeHome_Rw2Ar\nCUDA_VISIBLE_DEVICES=0 python afn.py data/office-home -d OfficeHome -s Rw -t Cl -a resnet50 --epochs 20 --seed 0 --log logs/afn/OfficeHome_Rw2Cl\nCUDA_VISIBLE_DEVICES=0 python afn.py data/office-home -d OfficeHome -s Rw -t Pr -a resnet50 --epochs 20 --seed 0 --log logs/afn/OfficeHome_Rw2Pr\n\n# ResNet101, VisDA-2017, Single Source\nCUDA_VISIBLE_DEVICES=0 python afn.py data/visda-2017 -d VisDA2017 -s Synthetic -t Real -a resnet101 -r 0.3 -b 36 \\\n --epochs 10 -i 1000 --seed 0 --per-class-eval --train-resizing cen.crop --log logs/afn/VisDA2017\n\n# ResNet101, DomainNet, Single Source\nCUDA_VISIBLE_DEVICES=0 python afn.py data/domainnet -d DomainNet -s c -t p -a resnet101 --bottleneck-dim 1024 --epochs 30 -i 2500 -p 500 --seed 0 --trade-off-norm 0.01 --lr 0.002 --log logs/afn/DomainNet_c2p\nCUDA_VISIBLE_DEVICES=0 python afn.py data/domainnet -d DomainNet -s c -t r -a resnet101 --bottleneck-dim 1024 --epochs 30 -i 2500 -p 500 --seed 0 --trade-off-norm 0.01 --lr 0.002 --log logs/afn/DomainNet_c2r\nCUDA_VISIBLE_DEVICES=0 python afn.py data/domainnet -d DomainNet -s c -t s -a resnet101 --bottleneck-dim 1024 --epochs 30 -i 2500 -p 500 --seed 0 --trade-off-norm 0.01 --lr 0.002 --log logs/afn/DomainNet_c2s\nCUDA_VISIBLE_DEVICES=0 python afn.py data/domainnet -d DomainNet -s p -t c -a resnet101 --bottleneck-dim 1024 --epochs 30 -i 2500 -p 500 --seed 0 --trade-off-norm 0.01 --lr 0.002 --log logs/afn/DomainNet_p2c\nCUDA_VISIBLE_DEVICES=0 python afn.py data/domainnet -d DomainNet -s p -t r -a resnet101 --bottleneck-dim 1024 --epochs 30 -i 2500 -p 500 --seed 0 --trade-off-norm 0.01 --lr 0.002 --log logs/afn/DomainNet_p2r\nCUDA_VISIBLE_DEVICES=0 python afn.py data/domainnet -d DomainNet -s p -t s -a resnet101 --bottleneck-dim 1024 --epochs 30 -i 2500 -p 500 --seed 0 --trade-off-norm 0.01 --lr 0.002 --log logs/afn/DomainNet_p2s\nCUDA_VISIBLE_DEVICES=0 python afn.py data/domainnet -d DomainNet -s r -t c -a resnet101 --bottleneck-dim 1024 --epochs 30 -i 2500 -p 500 --seed 0 --trade-off-norm 0.01 --lr 0.002 --log logs/afn/DomainNet_r2c\nCUDA_VISIBLE_DEVICES=0 python afn.py data/domainnet -d DomainNet -s r -t p -a resnet101 --bottleneck-dim 1024 --epochs 30 -i 2500 -p 500 --seed 0 --trade-off-norm 0.01 --lr 0.002 --log logs/afn/DomainNet_r2p\nCUDA_VISIBLE_DEVICES=0 python afn.py data/domainnet -d DomainNet -s r -t s -a resnet101 --bottleneck-dim 1024 --epochs 30 -i 2500 -p 500 --seed 0 --trade-off-norm 0.01 --lr 0.002 --log logs/afn/DomainNet_r2s\nCUDA_VISIBLE_DEVICES=0 python afn.py data/domainnet -d DomainNet -s s -t c -a resnet101 --bottleneck-dim 1024 --epochs 30 -i 2500 -p 500 --seed 0 --trade-off-norm 0.01 --lr 0.002 --log logs/afn/DomainNet_s2c\nCUDA_VISIBLE_DEVICES=0 python afn.py data/domainnet -d DomainNet -s s -t p -a resnet101 --bottleneck-dim 1024 --epochs 30 -i 2500 -p 500 --seed 0 --trade-off-norm 0.01 --lr 0.002 --log logs/afn/DomainNet_s2p\nCUDA_VISIBLE_DEVICES=0 python afn.py data/domainnet -d DomainNet -s s -t r -a resnet101 --bottleneck-dim 1024 --epochs 30 -i 2500 -p 500 --seed 0 --trade-off-norm 0.01 --lr 0.002 --log logs/afn/DomainNet_s2r\n\n# ResNet50, ImageNet200 -> ImageNetR\nCUDA_VISIBLE_DEVICES=0 python afn.py data/ImageNetR -d ImageNetR -s IN -t INR -a resnet50 --epochs 20 -i 2500 --seed 0 --log logs/afn/ImageNet_IN2INR\n\n# ig_resnext101_32x8d, ImageNet -> ImageNetSketch\nCUDA_VISIBLE_DEVICES=0 python afn.py data/imagenet-sketch -d ImageNetSketch -s IN -t sketch -a ig_resnext101_32x8d --epochs 20 -i 2500 --seed 0 --log logs/afn_ig_resnext101_32x8d/ImageNet_IN2sketch\n\n# Vision Transformer, Office-Home, Single Source\nCUDA_VISIBLE_DEVICES=0 python afn.py data/office-home -d OfficeHome -s Ar -t Cl -a vit_base_patch16_224 --no-pool --epochs 30 --seed 0 -b 24 --log logs/afn_vit/OfficeHome_Ar2Cl\nCUDA_VISIBLE_DEVICES=0 python afn.py data/office-home -d OfficeHome -s Ar -t Pr -a vit_base_patch16_224 --no-pool --epochs 30 --seed 0 -b 24 --log logs/afn_vit/OfficeHome_Ar2Pr\nCUDA_VISIBLE_DEVICES=0 python afn.py data/office-home -d OfficeHome -s Ar -t Rw -a vit_base_patch16_224 --no-pool --epochs 30 --seed 0 -b 24 --log logs/afn_vit/OfficeHome_Ar2Rw\nCUDA_VISIBLE_DEVICES=0 python afn.py data/office-home -d OfficeHome -s Cl -t Ar -a vit_base_patch16_224 --no-pool --epochs 30 --seed 0 -b 24 --log logs/afn_vit/OfficeHome_Cl2Ar\nCUDA_VISIBLE_DEVICES=0 python afn.py data/office-home -d OfficeHome -s Cl -t Pr -a vit_base_patch16_224 --no-pool --epochs 30 --seed 0 -b 24 --log logs/afn_vit/OfficeHome_Cl2Pr\nCUDA_VISIBLE_DEVICES=0 python afn.py data/office-home -d OfficeHome -s Cl -t Rw -a vit_base_patch16_224 --no-pool --epochs 30 --seed 0 -b 24 --log logs/afn_vit/OfficeHome_Cl2Rw\nCUDA_VISIBLE_DEVICES=0 python afn.py data/office-home -d OfficeHome -s Pr -t Ar -a vit_base_patch16_224 --no-pool --epochs 30 --seed 0 -b 24 --log logs/afn_vit/OfficeHome_Pr2Ar\nCUDA_VISIBLE_DEVICES=0 python afn.py data/office-home -d OfficeHome -s Pr -t Cl -a vit_base_patch16_224 --no-pool --epochs 30 --seed 0 -b 24 --log logs/afn_vit/OfficeHome_Pr2Cl\nCUDA_VISIBLE_DEVICES=0 python afn.py data/office-home -d OfficeHome -s Pr -t Rw -a vit_base_patch16_224 --no-pool --epochs 30 --seed 0 -b 24 --log logs/afn_vit/OfficeHome_Pr2Rw\nCUDA_VISIBLE_DEVICES=0 python afn.py data/office-home -d OfficeHome -s Rw -t Ar -a vit_base_patch16_224 --no-pool --epochs 30 --seed 0 -b 24 --log logs/afn_vit/OfficeHome_Rw2Ar\nCUDA_VISIBLE_DEVICES=0 python afn.py data/office-home -d OfficeHome -s Rw -t Cl -a vit_base_patch16_224 --no-pool --epochs 30 --seed 0 -b 24 --log logs/afn_vit/OfficeHome_Rw2Cl\nCUDA_VISIBLE_DEVICES=0 python afn.py data/office-home -d OfficeHome -s Rw -t Pr -a vit_base_patch16_224 --no-pool --epochs 30 --seed 0 -b 24 --log logs/afn_vit/OfficeHome_Rw2Pr\n\n# ResNet50, Office-Home, Multi Source\nCUDA_VISIBLE_DEVICES=0 python afn.py data/office-home -d OfficeHome -s Cl Pr Rw -t Ar -a resnet50 --epochs 30 --seed 0 --log logs/afn/OfficeHome_:2Ar\nCUDA_VISIBLE_DEVICES=0 python afn.py data/office-home -d OfficeHome -s Ar Pr Rw -t Cl -a resnet50 --epochs 30 --seed 0 --log logs/afn/OfficeHome_:2Cl\nCUDA_VISIBLE_DEVICES=0 python afn.py data/office-home -d OfficeHome -s Ar Cl Rw -t Pr -a resnet50 --epochs 30 --seed 0 --log logs/afn/OfficeHome_:2Pr\nCUDA_VISIBLE_DEVICES=0 python afn.py data/office-home -d OfficeHome -s Ar Cl Pr -t Rw -a resnet50 --epochs 30 --seed 0 --log logs/afn/OfficeHome_:2Rw\n\n# ResNet101, DomainNet, Multi Source\nCUDA_VISIBLE_DEVICES=0 python afn.py data/domainnet -d DomainNet -s i p q r s -t c -a resnet101 --bottleneck-dim 1024 --epochs 40 -i 5000 -p 500 --seed 0 --log logs/afn/DomainNet_:2c\nCUDA_VISIBLE_DEVICES=0 python afn.py data/domainnet -d DomainNet -s c p q r s -t i -a resnet101 --bottleneck-dim 1024 --epochs 40 -i 5000 -p 500 --seed 0 --log logs/afn/DomainNet_:2i\nCUDA_VISIBLE_DEVICES=0 python afn.py data/domainnet -d DomainNet -s c i q r s -t p -a resnet101 --bottleneck-dim 1024 --epochs 40 -i 5000 -p 500 --seed 0 --log logs/afn/DomainNet_:2p\nCUDA_VISIBLE_DEVICES=0 python afn.py data/domainnet -d DomainNet -s c i p r s -t q -a resnet101 --bottleneck-dim 1024 --epochs 40 -i 5000 -p 500 --seed 0 --log logs/afn/DomainNet_:2q\nCUDA_VISIBLE_DEVICES=0 python afn.py data/domainnet -d DomainNet -s c i p q s -t r -a resnet101 --bottleneck-dim 1024 --epochs 40 -i 5000 -p 500 --seed 0 --log logs/afn/DomainNet_:2r\nCUDA_VISIBLE_DEVICES=0 python afn.py data/domainnet -d DomainNet -s c i p q r -t s -a resnet101 --bottleneck-dim 1024 --epochs 40 -i 5000 -p 500 --seed 0 --log logs/afn/DomainNet_:2s\n\n# ResNet50, Wilds Dataset\nCUDA_VISIBLE_DEVICES=0 python afn.py data/wilds -d iwildcam -a resnet50 --epochs 30 -i 1000 --seed 0 --log logs/afn/iwildcam\n\n# DenseNet121, Wilds Dataset\nCUDA_VISIBLE_DEVICES=0 python afn.py data/wilds -d camelyon17 --train-resizing 'res.' --val-resizing 'res.' --resize-size 96 \\\n -a densenet121 --scratch --epochs 10 -i 1000 --lr 0.01 --seed 0 --log logs/afn/camelyon17\nCUDA_VISIBLE_DEVICES=0 python afn.py data/wilds -d fmow --train-resizing 'res.' --val-resizing 'res.' \\\n -a densenet121 --epochs 10 -i 1000 --lr 0.01 --seed 0 --log logs/afn/fmow\n\n# Digits\nCUDA_VISIBLE_DEVICES=0 python afn.py data/digits -d Digits -s MNIST -t USPS --train-resizing 'res.' --val-resizing 'res.' \\\n --resize-size 28 --no-hflip --norm-mean 0.5 --norm-std 0.5 -a lenet --no-pool -r 0.3 --lr 0.01 --trade-off-entropy 0.03 -b 128 -i 2500 --scratch --seed 0 --log logs/afn/MNIST2USPS\nCUDA_VISIBLE_DEVICES=0 python afn.py data/digits -d Digits -s USPS -t MNIST --train-resizing 'res.' --val-resizing 'res.' \\\n --resize-size 28 --no-hflip --norm-mean 0.5 --norm-std 0.5 -a lenet --no-pool -r 0.1 --lr 0.03 --trade-off-entropy 0.03 -b 128 -i 2500 --scratch --seed 0 --log logs/afn/USPS2MNIST\nCUDA_VISIBLE_DEVICES=0 python afn.py data/digits -d Digits -s SVHNRGB -t MNISTRGB --train-resizing 'res.' --val-resizing 'res.' \\\n --resize-size 32 --no-hflip --norm-mean 0.5 0.5 0.5 --norm-std 0.5 0.5 0.5 -a dtn --no-pool -r 0.1 --lr 0.1 --trade-off-entropy 0.03 -b 128 -i 2500 --scratch --seed 0 --log logs/afn/SVHN2MNIST\n\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/examples/domain_adaptation/image_classification/bsp.py",
"content": "\"\"\"\n@author: Baixu Chen\n@contact: cbx_99_hasta@outlook.com\n\"\"\"\nimport random\nimport time\nimport warnings\nimport sys\nimport argparse\nimport shutil\nimport os.path as osp\n\nimport torch\nimport torch.nn as nn\nimport torch.backends.cudnn as cudnn\nfrom torch.optim import SGD\nfrom torch.optim.lr_scheduler import LambdaLR\nfrom torch.utils.data import DataLoader\nimport torch.nn.functional as F\n\nsys.path.append('../../..')\nfrom dalib.modules.domain_discriminator import DomainDiscriminator\nfrom dalib.adaptation.dann import DomainAdversarialLoss\nfrom dalib.adaptation.bsp import BatchSpectralPenalizationLoss, ImageClassifier\nfrom common.utils.data import ForeverDataIterator\nfrom common.utils.metric import accuracy\nfrom common.utils.meter import AverageMeter, ProgressMeter\nfrom common.utils.logger import CompleteLogger\nfrom common.utils.analysis import collect_feature, tsne, a_distance\n\nsys.path.append('.')\nimport utils\n\ndevice = torch.device(\"cuda\" if torch.cuda.is_available() else \"cpu\")\n\n\ndef main(args: argparse.Namespace):\n logger = CompleteLogger(args.log, args.phase)\n print(args)\n\n if args.seed is not None:\n random.seed(args.seed)\n torch.manual_seed(args.seed)\n cudnn.deterministic = True\n warnings.warn('You have chosen to seed training. '\n 'This will turn on the CUDNN deterministic setting, '\n 'which can slow down your training considerably! '\n 'You may see unexpected behavior when restarting '\n 'from checkpoints.')\n\n cudnn.benchmark = True\n\n # Data loading code\n train_transform = utils.get_train_transform(args.train_resizing, random_horizontal_flip=not args.no_hflip,\n random_color_jitter=False, resize_size=args.resize_size,\n norm_mean=args.norm_mean, norm_std=args.norm_std)\n val_transform = utils.get_val_transform(args.val_resizing, resize_size=args.resize_size,\n norm_mean=args.norm_mean, norm_std=args.norm_std)\n print(\"train_transform: \", train_transform)\n print(\"val_transform: \", val_transform)\n\n train_source_dataset, train_target_dataset, val_dataset, test_dataset, num_classes, args.class_names = \\\n utils.get_dataset(args.data, args.root, args.source, args.target, train_transform, val_transform)\n train_source_loader = DataLoader(train_source_dataset, batch_size=args.batch_size,\n shuffle=True, num_workers=args.workers, drop_last=True)\n train_target_loader = DataLoader(train_target_dataset, batch_size=args.batch_size,\n shuffle=True, num_workers=args.workers, drop_last=True)\n val_loader = DataLoader(val_dataset, batch_size=args.batch_size, shuffle=False, num_workers=args.workers)\n test_loader = DataLoader(test_dataset, batch_size=args.batch_size, shuffle=False, num_workers=args.workers)\n\n train_source_iter = ForeverDataIterator(train_source_loader)\n train_target_iter = ForeverDataIterator(train_target_loader)\n\n # create model\n print(\"=> using model '{}'\".format(args.arch))\n backbone = utils.get_model(args.arch, pretrain=not args.scratch)\n pool_layer = nn.Identity() if args.no_pool else None\n classifier = ImageClassifier(backbone, num_classes, bottleneck_dim=args.bottleneck_dim,\n pool_layer=pool_layer, finetune=not args.scratch).to(device)\n domain_discri = DomainDiscriminator(in_feature=classifier.features_dim, hidden_size=1024).to(device)\n\n # define optimizer and lr scheduler\n optimizer = SGD(classifier.get_parameters() + domain_discri.get_parameters(),\n args.lr, momentum=args.momentum, weight_decay=args.weight_decay, nesterov=True)\n lr_scheduler = LambdaLR(optimizer, lambda x: args.lr * (1. + args.lr_gamma * float(x)) ** (-args.lr_decay))\n\n # define loss function\n domain_adv = DomainAdversarialLoss(domain_discri).to(device)\n bsp_penalty = BatchSpectralPenalizationLoss().to(device)\n\n # resume from the best checkpoint\n if args.phase != 'train':\n checkpoint = torch.load(logger.get_checkpoint_path('best'), map_location='cpu')\n classifier.load_state_dict(checkpoint)\n\n # analysis the model\n if args.phase == 'analysis':\n # extract features from both domains\n feature_extractor = nn.Sequential(classifier.backbone, classifier.pool_layer, classifier.bottleneck).to(device)\n source_feature = collect_feature(train_source_loader, feature_extractor, device)\n target_feature = collect_feature(train_target_loader, feature_extractor, device)\n # plot t-SNE\n tSNE_filename = osp.join(logger.visualize_directory, 'TSNE.pdf')\n tsne.visualize(source_feature, target_feature, tSNE_filename)\n print(\"Saving t-SNE to\", tSNE_filename)\n # calculate A-distance, which is a measure for distribution discrepancy\n A_distance = a_distance.calculate(source_feature, target_feature, device)\n print(\"A-distance =\", A_distance)\n return\n\n if args.phase == 'test':\n acc1 = utils.validate(test_loader, classifier, args, device)\n print(acc1)\n return\n\n if args.pretrain is None:\n # first pretrain the classifier wish source data\n print(\"Pretraining the model on source domain.\")\n args.pretrain = logger.get_checkpoint_path('pretrain')\n pretrain_model = ImageClassifier(backbone, num_classes, bottleneck_dim=args.bottleneck_dim,\n pool_layer=pool_layer, finetune=not args.scratch).to(device)\n pretrain_optimizer = SGD(pretrain_model.get_parameters(), args.pretrain_lr, momentum=args.momentum,\n weight_decay=args.weight_decay, nesterov=True)\n pretrain_lr_scheduler = LambdaLR(pretrain_optimizer,\n lambda x: args.pretrain_lr * (1. + args.lr_gamma * float(x)) ** (\n -args.lr_decay))\n # start pretraining\n for epoch in range(args.pretrain_epochs):\n print(\"lr:\", pretrain_lr_scheduler.get_lr())\n # pretrain for one epoch\n utils.pretrain(train_source_iter, pretrain_model, pretrain_optimizer, pretrain_lr_scheduler, epoch, args,\n device)\n # validate to show pretrain process\n utils.validate(val_loader, pretrain_model, args, device)\n\n torch.save(pretrain_model.state_dict(), args.pretrain)\n print(\"Pretraining process is done.\")\n\n checkpoint = torch.load(args.pretrain, map_location='cpu')\n classifier.load_state_dict(checkpoint)\n\n # start training\n best_acc1 = 0.\n for epoch in range(args.epochs):\n print(\"lr:\", lr_scheduler.get_last_lr()[0])\n # train for one epoch\n train(train_source_iter, train_target_iter, classifier, domain_adv, bsp_penalty, optimizer,\n lr_scheduler, epoch, args)\n\n # evaluate on validation set\n acc1 = utils.validate(val_loader, classifier, args, device)\n\n # remember best acc@1 and save checkpoint\n torch.save(classifier.state_dict(), logger.get_checkpoint_path('latest'))\n if acc1 > best_acc1:\n shutil.copy(logger.get_checkpoint_path('latest'), logger.get_checkpoint_path('best'))\n best_acc1 = max(acc1, best_acc1)\n\n print(\"best_acc1 = {:3.1f}\".format(best_acc1))\n\n # evaluate on test set\n classifier.load_state_dict(torch.load(logger.get_checkpoint_path('best')))\n acc1 = utils.validate(test_loader, classifier, args, device)\n print(\"test_acc1 = {:3.1f}\".format(acc1))\n\n logger.close()\n\n\ndef train(train_source_iter: ForeverDataIterator, train_target_iter: ForeverDataIterator,\n model: ImageClassifier, domain_adv: DomainAdversarialLoss, bsp_penalty: BatchSpectralPenalizationLoss,\n optimizer: SGD, lr_scheduler: LambdaLR, epoch: int, args: argparse.Namespace):\n batch_time = AverageMeter('Time', ':5.2f')\n data_time = AverageMeter('Data', ':5.2f')\n losses = AverageMeter('Loss', ':6.2f')\n cls_accs = AverageMeter('Cls Acc', ':3.1f')\n domain_accs = AverageMeter('Domain Acc', ':3.1f')\n progress = ProgressMeter(\n args.iters_per_epoch,\n [batch_time, data_time, losses, cls_accs, domain_accs],\n prefix=\"Epoch: [{}]\".format(epoch))\n\n # switch to train mode\n model.train()\n domain_adv.train()\n\n end = time.time()\n for i in range(args.iters_per_epoch):\n x_s, labels_s = next(train_source_iter)\n x_t, _ = next(train_target_iter)\n\n x_s = x_s.to(device)\n x_t = x_t.to(device)\n labels_s = labels_s.to(device)\n\n # measure data loading time\n data_time.update(time.time() - end)\n\n # compute output\n x = torch.cat((x_s, x_t), dim=0)\n y, f = model(x)\n y_s, y_t = y.chunk(2, dim=0)\n f_s, f_t = f.chunk(2, dim=0)\n\n cls_loss = F.cross_entropy(y_s, labels_s)\n transfer_loss = domain_adv(f_s, f_t)\n bsp_loss = bsp_penalty(f_s, f_t)\n domain_acc = domain_adv.domain_discriminator_accuracy\n loss = cls_loss + transfer_loss * args.trade_off + bsp_loss * args.trade_off_bsp\n\n cls_acc = accuracy(y_s, labels_s)[0]\n\n losses.update(loss.item(), x_s.size(0))\n cls_accs.update(cls_acc.item(), x_s.size(0))\n domain_accs.update(domain_acc.item(), x_s.size(0))\n\n # compute gradient and do SGD step\n optimizer.zero_grad()\n loss.backward()\n optimizer.step()\n lr_scheduler.step()\n\n # measure elapsed time\n batch_time.update(time.time() - end)\n end = time.time()\n\n if i % args.print_freq == 0:\n progress.display(i)\n\n\nif __name__ == '__main__':\n parser = argparse.ArgumentParser(description='DANN for Unsupervised Domain Adaptation')\n # dataset parameters\n parser.add_argument('root', metavar='DIR',\n help='root path of dataset')\n parser.add_argument('-d', '--data', metavar='DATA', default='Office31', choices=utils.get_dataset_names(),\n help='dataset: ' + ' | '.join(utils.get_dataset_names()) +\n ' (default: Office31)')\n parser.add_argument('-s', '--source', help='source domain(s)', nargs='+')\n parser.add_argument('-t', '--target', help='target domain(s)', nargs='+')\n parser.add_argument('--train-resizing', type=str, default='default')\n parser.add_argument('--val-resizing', type=str, default='default')\n parser.add_argument('--resize-size', type=int, default=224,\n help='the image size after resizing')\n parser.add_argument('--no-hflip', action='store_true',\n help='no random horizontal flipping during training')\n parser.add_argument('--norm-mean', type=float, nargs='+',\n default=(0.485, 0.456, 0.406), help='normalization mean')\n parser.add_argument('--norm-std', type=float, nargs='+',\n default=(0.229, 0.224, 0.225), help='normalization std')\n # model parameters\n parser.add_argument('-a', '--arch', metavar='ARCH', default='resnet18',\n choices=utils.get_model_names(),\n help='backbone architecture: ' +\n ' | '.join(utils.get_model_names()) +\n ' (default: resnet18)')\n parser.add_argument('--pretrain', type=str, default=None,\n help='pretrain checkpoint for classification model')\n parser.add_argument('--bottleneck-dim', default=256, type=int,\n help='Dimension of bottleneck')\n parser.add_argument('--no-pool', action='store_true',\n help='no pool layer after the feature extractor.')\n parser.add_argument('--scratch', action='store_true', help='whether train from scratch.')\n parser.add_argument('--trade-off', default=1., type=float,\n help='the trade-off hyper-parameter for transfer loss')\n parser.add_argument('--trade-off-bsp', default=2e-4, type=float,\n help='the trade-off hyper-parameter for bsp loss')\n # training parameters\n parser.add_argument('-b', '--batch-size', default=32, type=int,\n metavar='N',\n help='mini-batch size (default: 32)')\n parser.add_argument('--lr', '--learning-rate', default=0.003, type=float,\n metavar='LR', help='initial learning rate', dest='lr')\n parser.add_argument('--pretrain-lr', default=0.001, type=float, help='initial pretrain learning rate')\n parser.add_argument('--lr-gamma', default=0.001, type=float, help='parameter for lr scheduler')\n parser.add_argument('--lr-decay', default=0.75, type=float, help='parameter for lr scheduler')\n parser.add_argument('--momentum', default=0.9, type=float, metavar='M',\n help='momentum')\n parser.add_argument('--wd', '--weight-decay', default=1e-3, type=float,\n metavar='W', help='weight decay (default: 1e-3)',\n dest='weight_decay')\n parser.add_argument('-j', '--workers', default=2, type=int, metavar='N',\n help='number of data loading workers (default: 2)')\n parser.add_argument('--epochs', default=20, type=int, metavar='N',\n help='number of total epochs to run')\n parser.add_argument('--pretrain-epochs', default=3, type=int, metavar='N',\n help='number of total epochs(pretrain) to run')\n parser.add_argument('-i', '--iters-per-epoch', default=1000, type=int,\n help='Number of iterations per epoch')\n parser.add_argument('-p', '--print-freq', default=100, type=int,\n metavar='N', help='print frequency (default: 100)')\n parser.add_argument('--seed', default=None, type=int,\n help='seed for initializing training. ')\n parser.add_argument('--per-class-eval', action='store_true',\n help='whether output per-class accuracy during evaluation')\n parser.add_argument(\"--log\", type=str, default='bsp',\n help=\"Where to save logs, checkpoints and debugging images.\")\n parser.add_argument(\"--phase\", type=str, default='train', choices=['train', 'test', 'analysis'],\n help=\"When phase is 'test', only test the model.\"\n \"When phase is 'analysis', only analysis the model.\")\n args = parser.parse_args()\n main(args)\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/examples/domain_adaptation/image_classification/bsp.sh",
"content": "#!/usr/bin/env bash\n# ResNet50, Office31, Single Source\nCUDA_VISIBLE_DEVICES=0 python bsp.py data/office31 -d Office31 -s A -t W -a resnet50 --epochs 20 --seed 1 --log logs/bsp/Office31_A2W\nCUDA_VISIBLE_DEVICES=0 python bsp.py data/office31 -d Office31 -s D -t W -a resnet50 --epochs 20 --seed 1 --log logs/bsp/Office31_D2W\nCUDA_VISIBLE_DEVICES=0 python bsp.py data/office31 -d Office31 -s W -t D -a resnet50 --epochs 20 --seed 1 --log logs/bsp/Office31_W2D\nCUDA_VISIBLE_DEVICES=0 python bsp.py data/office31 -d Office31 -s A -t D -a resnet50 --epochs 20 --seed 1 --log logs/bsp/Office31_A2D\nCUDA_VISIBLE_DEVICES=0 python bsp.py data/office31 -d Office31 -s D -t A -a resnet50 --epochs 20 --seed 1 --log logs/bsp/Office31_D2A\nCUDA_VISIBLE_DEVICES=0 python bsp.py data/office31 -d Office31 -s W -t A -a resnet50 --epochs 20 --seed 1 --log logs/bsp/Office31_W2A\n\n# ResNet50, Office-Home, Single Source\nCUDA_VISIBLE_DEVICES=0 python bsp.py data/office-home -d OfficeHome -s Ar -t Cl -a resnet50 --epochs 30 --seed 0 --log logs/bsp/OfficeHome_Ar2Cl\nCUDA_VISIBLE_DEVICES=0 python bsp.py data/office-home -d OfficeHome -s Ar -t Pr -a resnet50 --epochs 30 --seed 0 --log logs/bsp/OfficeHome_Ar2Pr\nCUDA_VISIBLE_DEVICES=0 python bsp.py data/office-home -d OfficeHome -s Ar -t Rw -a resnet50 --epochs 30 --seed 0 --log logs/bsp/OfficeHome_Ar2Rw\nCUDA_VISIBLE_DEVICES=0 python bsp.py data/office-home -d OfficeHome -s Cl -t Ar -a resnet50 --epochs 30 --seed 0 --log logs/bsp/OfficeHome_Cl2Ar\nCUDA_VISIBLE_DEVICES=0 python bsp.py data/office-home -d OfficeHome -s Cl -t Pr -a resnet50 --epochs 30 --seed 0 --log logs/bsp/OfficeHome_Cl2Pr\nCUDA_VISIBLE_DEVICES=0 python bsp.py data/office-home -d OfficeHome -s Cl -t Rw -a resnet50 --epochs 30 --seed 0 --log logs/bsp/OfficeHome_Cl2Rw\nCUDA_VISIBLE_DEVICES=0 python bsp.py data/office-home -d OfficeHome -s Pr -t Ar -a resnet50 --epochs 30 --seed 0 --log logs/bsp/OfficeHome_Pr2Ar\nCUDA_VISIBLE_DEVICES=0 python bsp.py data/office-home -d OfficeHome -s Pr -t Cl -a resnet50 --epochs 30 --seed 0 --log logs/bsp/OfficeHome_Pr2Cl\nCUDA_VISIBLE_DEVICES=0 python bsp.py data/office-home -d OfficeHome -s Pr -t Rw -a resnet50 --epochs 30 --seed 0 --log logs/bsp/OfficeHome_Pr2Rw\nCUDA_VISIBLE_DEVICES=0 python bsp.py data/office-home -d OfficeHome -s Rw -t Ar -a resnet50 --epochs 30 --seed 0 --log logs/bsp/OfficeHome_Rw2Ar\nCUDA_VISIBLE_DEVICES=0 python bsp.py data/office-home -d OfficeHome -s Rw -t Cl -a resnet50 --epochs 30 --seed 0 --log logs/bsp/OfficeHome_Rw2Cl\nCUDA_VISIBLE_DEVICES=0 python bsp.py data/office-home -d OfficeHome -s Rw -t Pr -a resnet50 --epochs 30 --seed 0 --log logs/bsp/OfficeHome_Rw2Pr\n\n# ResNet101, VisDA-2017, Single Source\nCUDA_VISIBLE_DEVICES=0 python bsp.py data/visda-2017 -d VisDA2017 -s Synthetic -t Real -a resnet101 \\\n --epochs 30 --seed 0 --per-class-eval --train-resizing cen.crop --log logs/bsp/VisDA2017\n\n# ResNet101, DomainNet, Single Source\nCUDA_VISIBLE_DEVICES=0 python bsp.py data/domainnet -d DomainNet -s c -t p -a resnet101 --bottleneck-dim 1024 --epochs 30 -i 2500 -p 500 --seed 0 --log logs/bsp/DomainNet_c2p\nCUDA_VISIBLE_DEVICES=0 python bsp.py data/domainnet -d DomainNet -s c -t r -a resnet101 --bottleneck-dim 1024 --epochs 30 -i 2500 -p 500 --seed 0 --log logs/bsp/DomainNet_c2r\nCUDA_VISIBLE_DEVICES=0 python bsp.py data/domainnet -d DomainNet -s c -t s -a resnet101 --bottleneck-dim 1024 --epochs 30 -i 2500 -p 500 --seed 0 --log logs/bsp/DomainNet_c2s\nCUDA_VISIBLE_DEVICES=0 python bsp.py data/domainnet -d DomainNet -s p -t c -a resnet101 --bottleneck-dim 1024 --epochs 30 -i 2500 -p 500 --seed 0 --log logs/bsp/DomainNet_p2c\nCUDA_VISIBLE_DEVICES=0 python bsp.py data/domainnet -d DomainNet -s p -t r -a resnet101 --bottleneck-dim 1024 --epochs 30 -i 2500 -p 500 --seed 0 --log logs/bsp/DomainNet_p2r\nCUDA_VISIBLE_DEVICES=0 python bsp.py data/domainnet -d DomainNet -s p -t s -a resnet101 --bottleneck-dim 1024 --epochs 30 -i 2500 -p 500 --seed 0 --log logs/bsp/DomainNet_p2s\nCUDA_VISIBLE_DEVICES=0 python bsp.py data/domainnet -d DomainNet -s r -t c -a resnet101 --bottleneck-dim 1024 --epochs 30 -i 2500 -p 500 --seed 0 --log logs/bsp/DomainNet_r2c\nCUDA_VISIBLE_DEVICES=0 python bsp.py data/domainnet -d DomainNet -s r -t p -a resnet101 --bottleneck-dim 1024 --epochs 30 -i 2500 -p 500 --seed 0 --log logs/bsp/DomainNet_r2p\nCUDA_VISIBLE_DEVICES=0 python bsp.py data/domainnet -d DomainNet -s r -t s -a resnet101 --bottleneck-dim 1024 --epochs 30 -i 2500 -p 500 --seed 0 --log logs/bsp/DomainNet_r2s\nCUDA_VISIBLE_DEVICES=0 python bsp.py data/domainnet -d DomainNet -s s -t c -a resnet101 --bottleneck-dim 1024 --epochs 30 -i 2500 -p 500 --seed 0 --log logs/bsp/DomainNet_s2c\nCUDA_VISIBLE_DEVICES=0 python bsp.py data/domainnet -d DomainNet -s s -t p -a resnet101 --bottleneck-dim 1024 --epochs 30 -i 2500 -p 500 --seed 0 --log logs/bsp/DomainNet_s2p\nCUDA_VISIBLE_DEVICES=0 python bsp.py data/domainnet -d DomainNet -s s -t r -a resnet101 --bottleneck-dim 1024 --epochs 30 -i 2500 -p 500 --seed 0 --log logs/bsp/DomainNet_s2r\n\n# ResNet50, ImageNet200 -> ImageNetR\nCUDA_VISIBLE_DEVICES=0 python bsp.py data/ImageNetR -d ImageNetR -s IN -t INR -a resnet50 --epochs 30 -i 2500 -p 500 --seed 0 --log logs/bsp/ImageNet_IN2INR\n\n# ig_resnext101_32x8d, ImageNet -> ImageNetSketch\nCUDA_VISIBLE_DEVICES=0 python bsp.py data/imagenet-sketch -d ImageNetSketch -s IN -t sketch -a ig_resnext101_32x8d --epochs 30 -i 2500 -p 500 --bottleneck-dim 1024 --log logs/dann_ig_resnext101_32x8d/ImageNet_IN2sketch\n\n# Vision Transformer, Office-Home, Single Source\nCUDA_VISIBLE_DEVICES=0 python bsp.py data/office-home -d OfficeHome -s Ar -t Cl -a vit_base_patch16_224 --epochs 30 --seed 0 -b 24 --no-pool --log logs/dann_vit/OfficeHome_Ar2Cl\nCUDA_VISIBLE_DEVICES=0 python bsp.py data/office-home -d OfficeHome -s Ar -t Pr -a vit_base_patch16_224 --epochs 30 --seed 0 -b 24 --no-pool --log logs/dann_vit/OfficeHome_Ar2Pr\nCUDA_VISIBLE_DEVICES=0 python bsp.py data/office-home -d OfficeHome -s Ar -t Rw -a vit_base_patch16_224 --epochs 30 --seed 0 -b 24 --no-pool --log logs/dann_vit/OfficeHome_Ar2Rw\nCUDA_VISIBLE_DEVICES=0 python bsp.py data/office-home -d OfficeHome -s Cl -t Ar -a vit_base_patch16_224 --epochs 30 --seed 0 -b 24 --no-pool --log logs/dann_vit/OfficeHome_Cl2Ar\nCUDA_VISIBLE_DEVICES=0 python bsp.py data/office-home -d OfficeHome -s Cl -t Pr -a vit_base_patch16_224 --epochs 30 --seed 0 -b 24 --no-pool --log logs/dann_vit/OfficeHome_Cl2Pr\nCUDA_VISIBLE_DEVICES=0 python bsp.py data/office-home -d OfficeHome -s Cl -t Rw -a vit_base_patch16_224 --epochs 30 --seed 0 -b 24 --no-pool --log logs/dann_vit/OfficeHome_Cl2Rw\nCUDA_VISIBLE_DEVICES=0 python bsp.py data/office-home -d OfficeHome -s Pr -t Ar -a vit_base_patch16_224 --epochs 30 --seed 0 -b 24 --no-pool --log logs/dann_vit/OfficeHome_Pr2Ar\nCUDA_VISIBLE_DEVICES=0 python bsp.py data/office-home -d OfficeHome -s Pr -t Cl -a vit_base_patch16_224 --epochs 30 --seed 0 -b 24 --no-pool --log logs/dann_vit/OfficeHome_Pr2Cl\nCUDA_VISIBLE_DEVICES=0 python bsp.py data/office-home -d OfficeHome -s Pr -t Rw -a vit_base_patch16_224 --epochs 30 --seed 0 -b 24 --no-pool --log logs/dann_vit/OfficeHome_Pr2Rw\nCUDA_VISIBLE_DEVICES=0 python bsp.py data/office-home -d OfficeHome -s Rw -t Ar -a vit_base_patch16_224 --epochs 30 --seed 0 -b 24 --no-pool --log logs/dann_vit/OfficeHome_Rw2Ar\nCUDA_VISIBLE_DEVICES=0 python bsp.py data/office-home -d OfficeHome -s Rw -t Cl -a vit_base_patch16_224 --epochs 30 --seed 0 -b 24 --no-pool --log logs/dann_vit/OfficeHome_Rw2Cl\nCUDA_VISIBLE_DEVICES=0 python bsp.py data/office-home -d OfficeHome -s Rw -t Pr -a vit_base_patch16_224 --epochs 30 --seed 0 -b 24 --no-pool --log logs/dann_vit/OfficeHome_Rw2Pr\n\n# ResNet50, Office-Home, Multi Source\nCUDA_VISIBLE_DEVICES=0 python bsp.py data/office-home -d OfficeHome -s Cl Pr Rw -t Ar -a resnet50 --epochs 30 --seed 0 --log logs/bsp/OfficeHome_:2Ar\nCUDA_VISIBLE_DEVICES=0 python bsp.py data/office-home -d OfficeHome -s Ar Pr Rw -t Cl -a resnet50 --epochs 30 --seed 0 --log logs/bsp/OfficeHome_:2Cl\nCUDA_VISIBLE_DEVICES=0 python bsp.py data/office-home -d OfficeHome -s Ar Cl Rw -t Pr -a resnet50 --epochs 30 --seed 0 --log logs/bsp/OfficeHome_:2Pr\nCUDA_VISIBLE_DEVICES=0 python bsp.py data/office-home -d OfficeHome -s Ar Cl Pr -t Rw -a resnet50 --epochs 30 --seed 0 --log logs/bsp/OfficeHome_:2Rw\n\n# ResNet101, DomainNet, Multi Source\nCUDA_VISIBLE_DEVICES=0 python bsp.py data/domainnet -d DomainNet -s i p q r s -t c -a resnet101 --bottleneck-dim 1024 --epochs 40 -i 5000 -p 500 --seed 0 --log logs/bsp/DomainNet_:2c\nCUDA_VISIBLE_DEVICES=0 python bsp.py data/domainnet -d DomainNet -s c p q r s -t i -a resnet101 --bottleneck-dim 1024 --epochs 40 -i 5000 -p 500 --seed 0 --log logs/bsp/DomainNet_:2i\nCUDA_VISIBLE_DEVICES=0 python bsp.py data/domainnet -d DomainNet -s c i q r s -t p -a resnet101 --bottleneck-dim 1024 --epochs 40 -i 5000 -p 500 --seed 0 --log logs/bsp/DomainNet_:2p\nCUDA_VISIBLE_DEVICES=0 python bsp.py data/domainnet -d DomainNet -s c i p r s -t q -a resnet101 --bottleneck-dim 1024 --epochs 40 -i 5000 -p 500 --seed 0 --log logs/bsp/DomainNet_:2q\nCUDA_VISIBLE_DEVICES=0 python bsp.py data/domainnet -d DomainNet -s c i p q s -t r -a resnet101 --bottleneck-dim 1024 --epochs 40 -i 5000 -p 500 --seed 0 --log logs/bsp/DomainNet_:2r\nCUDA_VISIBLE_DEVICES=0 python bsp.py data/domainnet -d DomainNet -s c i p q r -t s -a resnet101 --bottleneck-dim 1024 --epochs 40 -i 5000 -p 500 --seed 0 --log logs/bsp/DomainNet_:2s\n\n# ResNet50, Wilds Dataset\nCUDA_VISIBLE_DEVICES=0 python bsp.py data/wilds -d iwildcam -a resnet50 --epochs 30 -i 1000 --seed 0 --log logs/bsp/iwildcam\n\n# DenseNet121, Wilds Dataset\nCUDA_VISIBLE_DEVICES=0 python bsp.py data/wilds -d camelyon17 --train-resizing 'res.' --val-resizing 'res.' --resize-size 96 \\\n -a densenet121 --scratch --epochs 10 -i 1000 --lr 0.01 --seed 0 --log logs/bsp/camelyon17\nCUDA_VISIBLE_DEVICES=0 python bsp.py data/wilds -d fmow --train-resizing 'res.' --val-resizing 'res.' \\\n -a densenet121 --epochs 10 -i 1000 --lr 0.01 --seed 0 --log logs/bsp/fmow\n\n# Digits\nCUDA_VISIBLE_DEVICES=0 python bsp.py data/digits -d Digits -s MNIST -t USPS --train-resizing 'res.' --val-resizing 'res.' \\\n --resize-size 28 --no-hflip --norm-mean 0.5 --norm-std 0.5 -a lenet --no-pool --lr 0.01 --trade-off-bsp 0.0001 -b 128 -i 2500 --scratch --seed 0 --log logs/bsp/MNIST2USPS\nCUDA_VISIBLE_DEVICES=0 python bsp.py data/digits -d Digits -s USPS -t MNIST --train-resizing 'res.' --val-resizing 'res.' \\\n --resize-size 28 --no-hflip --norm-mean 0.5 --norm-std 0.5 -a lenet --no-pool --lr 0.1 --trade-off-bsp 0.0001 -b 128 -i 2500 --scratch --seed 0 --log logs/bsp/USPS2MNIST\nCUDA_VISIBLE_DEVICES=0 python bsp.py data/digits -d Digits -s SVHNRGB -t MNISTRGB --train-resizing 'res.' --val-resizing 'res.' \\\n --resize-size 32 --no-hflip --norm-mean 0.5 0.5 0.5 --norm-std 0.5 0.5 0.5 -a dtn --no-pool --lr 0.1 --trade-off-bsp 0.0001 -b 128 -i 2500 --scratch --seed 0 --log logs/bsp/SVHN2MNIST --pretrain-epochs 1\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/examples/domain_adaptation/image_classification/cdan.py",
"content": "\"\"\"\n@author: Junguang Jiang\n@contact: JiangJunguang1123@outlook.com\n\"\"\"\nimport random\nimport time\nimport warnings\nimport sys\nimport argparse\nimport shutil\nimport os.path as osp\n\nimport torch\nimport torch.nn as nn\nimport torch.backends.cudnn as cudnn\nfrom torch.optim import SGD\nfrom torch.optim.lr_scheduler import LambdaLR\nfrom torch.utils.data import DataLoader\nimport torch.nn.functional as F\n\nsys.path.append('../../..')\nfrom dalib.modules.domain_discriminator import DomainDiscriminator\nfrom dalib.adaptation.cdan import ConditionalDomainAdversarialLoss, ImageClassifier\nfrom common.utils.data import ForeverDataIterator\nfrom common.utils.metric import accuracy\nfrom common.utils.meter import AverageMeter, ProgressMeter\nfrom common.utils.logger import CompleteLogger\nfrom common.utils.analysis import collect_feature, tsne, a_distance\n\nsys.path.append('.')\nimport utils\n\ndevice = torch.device(\"cuda\" if torch.cuda.is_available() else \"cpu\")\n\n\ndef main(args: argparse.Namespace):\n logger = CompleteLogger(args.log, args.phase)\n print(args)\n\n if args.seed is not None:\n random.seed(args.seed)\n torch.manual_seed(args.seed)\n cudnn.deterministic = True\n warnings.warn('You have chosen to seed training. '\n 'This will turn on the CUDNN deterministic setting, '\n 'which can slow down your training considerably! '\n 'You may see unexpected behavior when restarting '\n 'from checkpoints.')\n\n cudnn.benchmark = True\n\n # Data loading code\n train_transform = utils.get_train_transform(args.train_resizing, random_horizontal_flip=not args.no_hflip,\n random_color_jitter=False, resize_size=args.resize_size,\n norm_mean=args.norm_mean, norm_std=args.norm_std)\n val_transform = utils.get_val_transform(args.val_resizing, resize_size=args.resize_size,\n norm_mean=args.norm_mean, norm_std=args.norm_std)\n print(\"train_transform: \", train_transform)\n print(\"val_transform: \", val_transform)\n\n train_source_dataset, train_target_dataset, val_dataset, test_dataset, num_classes, args.class_names = \\\n utils.get_dataset(args.data, args.root, args.source, args.target, train_transform, val_transform)\n train_source_loader = DataLoader(train_source_dataset, batch_size=args.batch_size,\n shuffle=True, num_workers=args.workers, drop_last=True)\n train_target_loader = DataLoader(train_target_dataset, batch_size=args.batch_size,\n shuffle=True, num_workers=args.workers, drop_last=True)\n val_loader = DataLoader(val_dataset, batch_size=args.batch_size, shuffle=False, num_workers=args.workers)\n test_loader = DataLoader(test_dataset, batch_size=args.batch_size, shuffle=False, num_workers=args.workers)\n\n train_source_iter = ForeverDataIterator(train_source_loader)\n train_target_iter = ForeverDataIterator(train_target_loader)\n\n # create model\n print(\"=> using model '{}'\".format(args.arch))\n backbone = utils.get_model(args.arch, pretrain=not args.scratch)\n pool_layer = nn.Identity() if args.no_pool else None\n classifier = ImageClassifier(backbone, num_classes, bottleneck_dim=args.bottleneck_dim,\n pool_layer=pool_layer, finetune=not args.scratch).to(device)\n classifier_feature_dim = classifier.features_dim\n\n if args.randomized:\n domain_discri = DomainDiscriminator(args.randomized_dim, hidden_size=1024).to(device)\n else:\n domain_discri = DomainDiscriminator(classifier_feature_dim * num_classes, hidden_size=1024).to(device)\n\n all_parameters = classifier.get_parameters() + domain_discri.get_parameters()\n # define optimizer and lr scheduler\n optimizer = SGD(all_parameters, args.lr, momentum=args.momentum, weight_decay=args.weight_decay, nesterov=True)\n lr_scheduler = LambdaLR(optimizer, lambda x: args.lr * (1. + args.lr_gamma * float(x)) ** (-args.lr_decay))\n\n # define loss function\n domain_adv = ConditionalDomainAdversarialLoss(\n domain_discri, entropy_conditioning=args.entropy,\n num_classes=num_classes, features_dim=classifier_feature_dim, randomized=args.randomized,\n randomized_dim=args.randomized_dim\n ).to(device)\n\n # resume from the best checkpoint\n if args.phase != 'train':\n checkpoint = torch.load(logger.get_checkpoint_path('best'), map_location='cpu')\n classifier.load_state_dict(checkpoint)\n\n # analysis the model\n if args.phase == 'analysis':\n # extract features from both domains\n feature_extractor = nn.Sequential(classifier.backbone, classifier.pool_layer, classifier.bottleneck).to(device)\n source_feature = collect_feature(train_source_loader, feature_extractor, device)\n target_feature = collect_feature(train_target_loader, feature_extractor, device)\n # plot t-SNE\n tSNE_filename = osp.join(logger.visualize_directory, 'TSNE.pdf')\n tsne.visualize(source_feature, target_feature, tSNE_filename)\n print(\"Saving t-SNE to\", tSNE_filename)\n # calculate A-distance, which is a measure for distribution discrepancy\n A_distance = a_distance.calculate(source_feature, target_feature, device)\n print(\"A-distance =\", A_distance)\n return\n\n if args.phase == 'test':\n acc1 = utils.validate(test_loader, classifier, args, device)\n print(acc1)\n return\n\n # start training\n best_acc1 = 0.\n for epoch in range(args.epochs):\n print(\"lr:\", lr_scheduler.get_last_lr()[0])\n # train for one epoch\n train(train_source_iter, train_target_iter, classifier, domain_adv, optimizer,\n lr_scheduler, epoch, args)\n\n # evaluate on validation set\n acc1 = utils.validate(val_loader, classifier, args, device)\n\n # remember best acc@1 and save checkpoint\n torch.save(classifier.state_dict(), logger.get_checkpoint_path('latest'))\n if acc1 > best_acc1:\n shutil.copy(logger.get_checkpoint_path('latest'), logger.get_checkpoint_path('best'))\n best_acc1 = max(acc1, best_acc1)\n\n print(\"best_acc1 = {:3.1f}\".format(best_acc1))\n\n # evaluate on test set\n classifier.load_state_dict(torch.load(logger.get_checkpoint_path('best')))\n acc1 = utils.validate(test_loader, classifier, args, device)\n print(\"test_acc1 = {:3.1f}\".format(acc1))\n\n logger.close()\n\n\ndef train(train_source_iter: ForeverDataIterator, train_target_iter: ForeverDataIterator, model: ImageClassifier,\n domain_adv: ConditionalDomainAdversarialLoss, optimizer: SGD,\n lr_scheduler: LambdaLR, epoch: int, args: argparse.Namespace):\n batch_time = AverageMeter('Time', ':3.1f')\n data_time = AverageMeter('Data', ':3.1f')\n losses = AverageMeter('Loss', ':3.2f')\n trans_losses = AverageMeter('Trans Loss', ':3.2f')\n cls_accs = AverageMeter('Cls Acc', ':3.1f')\n domain_accs = AverageMeter('Domain Acc', ':3.1f')\n progress = ProgressMeter(\n args.iters_per_epoch,\n [batch_time, data_time, losses, trans_losses, cls_accs, domain_accs],\n prefix=\"Epoch: [{}]\".format(epoch))\n\n # switch to train mode\n model.train()\n domain_adv.train()\n\n end = time.time()\n for i in range(args.iters_per_epoch):\n x_s, labels_s = next(train_source_iter)\n x_t, _ = next(train_target_iter)\n\n x_s = x_s.to(device)\n x_t = x_t.to(device)\n labels_s = labels_s.to(device)\n\n # measure data loading time\n data_time.update(time.time() - end)\n\n # compute output\n x = torch.cat((x_s, x_t), dim=0)\n y, f = model(x)\n y_s, y_t = y.chunk(2, dim=0)\n f_s, f_t = f.chunk(2, dim=0)\n\n cls_loss = F.cross_entropy(y_s, labels_s)\n transfer_loss = domain_adv(y_s, f_s, y_t, f_t)\n domain_acc = domain_adv.domain_discriminator_accuracy\n loss = cls_loss + transfer_loss * args.trade_off\n\n cls_acc = accuracy(y_s, labels_s)[0]\n\n losses.update(loss.item(), x_s.size(0))\n cls_accs.update(cls_acc, x_s.size(0))\n domain_accs.update(domain_acc, x_s.size(0))\n trans_losses.update(transfer_loss.item(), x_s.size(0))\n\n # compute gradient and do SGD step\n optimizer.zero_grad()\n loss.backward()\n optimizer.step()\n lr_scheduler.step()\n\n # measure elapsed time\n batch_time.update(time.time() - end)\n end = time.time()\n\n if i % args.print_freq == 0:\n progress.display(i)\n\n\nif __name__ == '__main__':\n parser = argparse.ArgumentParser(description='CDAN for Unsupervised Domain Adaptation')\n # dataset parameters\n parser.add_argument('root', metavar='DIR',\n help='root path of dataset')\n parser.add_argument('-d', '--data', metavar='DATA', default='Office31', choices=utils.get_dataset_names(),\n help='dataset: ' + ' | '.join(utils.get_dataset_names()) +\n ' (default: Office31)')\n parser.add_argument('-s', '--source', help='source domain(s)', nargs='+')\n parser.add_argument('-t', '--target', help='target domain(s)', nargs='+')\n parser.add_argument('--train-resizing', type=str, default='default')\n parser.add_argument('--val-resizing', type=str, default='default')\n parser.add_argument('--resize-size', type=int, default=224,\n help='the image size after resizing')\n parser.add_argument('--no-hflip', action='store_true',\n help='no random horizontal flipping during training')\n parser.add_argument('--norm-mean', type=float, nargs='+',\n default=(0.485, 0.456, 0.406), help='normalization mean')\n parser.add_argument('--norm-std', type=float, nargs='+',\n default=(0.229, 0.224, 0.225), help='normalization std')\n # model parameters\n parser.add_argument('-a', '--arch', metavar='ARCH', default='resnet18',\n choices=utils.get_model_names(),\n help='backbone architecture: ' +\n ' | '.join(utils.get_model_names()) +\n ' (default: resnet18)')\n parser.add_argument('--bottleneck-dim', default=256, type=int,\n help='Dimension of bottleneck')\n parser.add_argument('--no-pool', action='store_true',\n help='no pool layer after the feature extractor.')\n parser.add_argument('--scratch', action='store_true', help='whether train from scratch.')\n parser.add_argument('-r', '--randomized', action='store_true',\n help='using randomized multi-linear-map (default: False)')\n parser.add_argument('-rd', '--randomized-dim', default=1024, type=int,\n help='randomized dimension when using randomized multi-linear-map (default: 1024)')\n parser.add_argument('--entropy', default=False, action='store_true', help='use entropy conditioning')\n parser.add_argument('--trade-off', default=1., type=float,\n help='the trade-off hyper-parameter for transfer loss')\n # training parameters\n parser.add_argument('-b', '--batch-size', default=32, type=int,\n metavar='N',\n help='mini-batch size (default: 32)')\n parser.add_argument('--lr', '--learning-rate', default=0.01, type=float,\n metavar='LR', help='initial learning rate', dest='lr')\n parser.add_argument('--lr-gamma', default=0.001, type=float, help='parameter for lr scheduler')\n parser.add_argument('--lr-decay', default=0.75, type=float, help='parameter for lr scheduler')\n parser.add_argument('--momentum', default=0.9, type=float, metavar='M', help='momentum')\n parser.add_argument('--wd', '--weight-decay', default=1e-3, type=float,\n metavar='W', help='weight decay (default: 1e-3)',\n dest='weight_decay')\n parser.add_argument('-j', '--workers', default=2, type=int, metavar='N',\n help='number of data loading workers (default: 2)')\n parser.add_argument('--epochs', default=20, type=int, metavar='N',\n help='number of total epochs to run')\n parser.add_argument('-i', '--iters-per-epoch', default=1000, type=int,\n help='Number of iterations per epoch')\n parser.add_argument('-p', '--print-freq', default=100, type=int,\n metavar='N', help='print frequency (default: 100)')\n parser.add_argument('--seed', default=None, type=int,\n help='seed for initializing training. ')\n parser.add_argument('--per-class-eval', action='store_true',\n help='whether output per-class accuracy during evaluation')\n parser.add_argument(\"--log\", type=str, default='cdan',\n help=\"Where to save logs, checkpoints and debugging images.\")\n parser.add_argument(\"--phase\", type=str, default='train', choices=['train', 'test', 'analysis'],\n help=\"When phase is 'test', only test the model.\"\n \"When phase is 'analysis', only analysis the model.\")\n args = parser.parse_args()\n main(args)\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/examples/domain_adaptation/image_classification/cdan.sh",
"content": "#!/usr/bin/env bash\n# ResNet50, Office31, Single Source\nCUDA_VISIBLE_DEVICES=0 python cdan.py data/office31 -d Office31 -s A -t W -a resnet50 --epochs 20 --seed 2 --log logs/cdan/Office31_A2W\nCUDA_VISIBLE_DEVICES=0 python cdan.py data/office31 -d Office31 -s D -t W -a resnet50 --epochs 20 --seed 2 --log logs/cdan/Office31_D2W\nCUDA_VISIBLE_DEVICES=0 python cdan.py data/office31 -d Office31 -s W -t D -a resnet50 --epochs 20 --seed 2 --log logs/cdan/Office31_W2D\nCUDA_VISIBLE_DEVICES=0 python cdan.py data/office31 -d Office31 -s A -t D -a resnet50 --epochs 20 --seed 2 --log logs/cdan/Office31_A2D\nCUDA_VISIBLE_DEVICES=0 python cdan.py data/office31 -d Office31 -s D -t A -a resnet50 --epochs 20 --seed 2 --log logs/cdan/Office31_D2A\nCUDA_VISIBLE_DEVICES=0 python cdan.py data/office31 -d Office31 -s W -t A -a resnet50 --epochs 20 --seed 2 --log logs/cdan/Office31_W2A\n\n# ResNet50, Office-Home, Single Source\nCUDA_VISIBLE_DEVICES=0 python cdan.py data/office-home -d OfficeHome -s Ar -t Cl -a resnet50 --epochs 30 --seed 0 --log logs/cdan/OfficeHome_Ar2Cl\nCUDA_VISIBLE_DEVICES=0 python cdan.py data/office-home -d OfficeHome -s Ar -t Pr -a resnet50 --epochs 30 --seed 0 --log logs/cdan/OfficeHome_Ar2Pr\nCUDA_VISIBLE_DEVICES=0 python cdan.py data/office-home -d OfficeHome -s Ar -t Rw -a resnet50 --epochs 30 --seed 0 --log logs/cdan/OfficeHome_Ar2Rw\nCUDA_VISIBLE_DEVICES=0 python cdan.py data/office-home -d OfficeHome -s Cl -t Ar -a resnet50 --epochs 30 --seed 0 --log logs/cdan/OfficeHome_Cl2Ar\nCUDA_VISIBLE_DEVICES=0 python cdan.py data/office-home -d OfficeHome -s Cl -t Pr -a resnet50 --epochs 30 --seed 0 --log logs/cdan/OfficeHome_Cl2Pr\nCUDA_VISIBLE_DEVICES=0 python cdan.py data/office-home -d OfficeHome -s Cl -t Rw -a resnet50 --epochs 30 --seed 0 --log logs/cdan/OfficeHome_Cl2Rw\nCUDA_VISIBLE_DEVICES=0 python cdan.py data/office-home -d OfficeHome -s Pr -t Ar -a resnet50 --epochs 30 --seed 0 --log logs/cdan/OfficeHome_Pr2Ar\nCUDA_VISIBLE_DEVICES=0 python cdan.py data/office-home -d OfficeHome -s Pr -t Cl -a resnet50 --epochs 30 --seed 0 --log logs/cdan/OfficeHome_Pr2Cl\nCUDA_VISIBLE_DEVICES=0 python cdan.py data/office-home -d OfficeHome -s Pr -t Rw -a resnet50 --epochs 30 --seed 0 --log logs/cdan/OfficeHome_Pr2Rw\nCUDA_VISIBLE_DEVICES=0 python cdan.py data/office-home -d OfficeHome -s Rw -t Ar -a resnet50 --epochs 30 --seed 0 --log logs/cdan/OfficeHome_Rw2Ar\nCUDA_VISIBLE_DEVICES=0 python cdan.py data/office-home -d OfficeHome -s Rw -t Cl -a resnet50 --epochs 30 --seed 0 --log logs/cdan/OfficeHome_Rw2Cl\nCUDA_VISIBLE_DEVICES=0 python cdan.py data/office-home -d OfficeHome -s Rw -t Pr -a resnet50 --epochs 30 --seed 0 --log logs/cdan/OfficeHome_Rw2Pr\n \n# ResNet101, VisDA-2017, Single Source\nCUDA_VISIBLE_DEVICES=0 python cdan.py data/visda-2017 -d VisDA2017 -s Synthetic -t Real -a resnet101 \\\n --epochs 30 --seed 0 --per-class-eval --train-resizing cen.crop --log logs/cdan/VisDA2017\n\n# ResNet101, DomainNet, Single Source\n# Use randomized multi-linear-map to decrease GPU memory usage\nCUDA_VISIBLE_DEVICES=0 python cdan.py data/domainnet -d DomainNet -s c -t p -a resnet101 --bottleneck-dim 1024 -r -rd 51200 --epochs 30 -i 2500 -p 500 --seed 0 --log logs/cdan/DomainNet_c2p\nCUDA_VISIBLE_DEVICES=0 python cdan.py data/domainnet -d DomainNet -s c -t r -a resnet101 --bottleneck-dim 1024 -r -rd 51200 --epochs 30 -i 2500 -p 500 --seed 0 --log logs/cdan/DomainNet_c2r\nCUDA_VISIBLE_DEVICES=0 python cdan.py data/domainnet -d DomainNet -s c -t s -a resnet101 --bottleneck-dim 1024 -r -rd 51200 --epochs 30 -i 2500 -p 500 --seed 0 --log logs/cdan/DomainNet_c2s\nCUDA_VISIBLE_DEVICES=0 python cdan.py data/domainnet -d DomainNet -s p -t c -a resnet101 --bottleneck-dim 1024 -r -rd 51200 --epochs 30 -i 2500 -p 500 --seed 0 --log logs/cdan/DomainNet_p2c\nCUDA_VISIBLE_DEVICES=0 python cdan.py data/domainnet -d DomainNet -s p -t r -a resnet101 --bottleneck-dim 1024 -r -rd 51200 --epochs 30 -i 2500 -p 500 --seed 0 --log logs/cdan/DomainNet_p2r\nCUDA_VISIBLE_DEVICES=0 python cdan.py data/domainnet -d DomainNet -s p -t s -a resnet101 --bottleneck-dim 1024 -r -rd 51200 --epochs 30 -i 2500 -p 500 --seed 0 --log logs/cdan/DomainNet_p2s\nCUDA_VISIBLE_DEVICES=0 python cdan.py data/domainnet -d DomainNet -s r -t c -a resnet101 --bottleneck-dim 1024 -r -rd 51200 --epochs 30 -i 2500 -p 500 --seed 0 --log logs/cdan/DomainNet_r2c\nCUDA_VISIBLE_DEVICES=0 python cdan.py data/domainnet -d DomainNet -s r -t p -a resnet101 --bottleneck-dim 1024 -r -rd 51200 --epochs 30 -i 2500 -p 500 --seed 0 --log logs/cdan/DomainNet_r2p\nCUDA_VISIBLE_DEVICES=0 python cdan.py data/domainnet -d DomainNet -s r -t s -a resnet101 --bottleneck-dim 1024 -r -rd 51200 --epochs 30 -i 2500 -p 500 --seed 0 --log logs/cdan/DomainNet_r2s\nCUDA_VISIBLE_DEVICES=0 python cdan.py data/domainnet -d DomainNet -s s -t c -a resnet101 --bottleneck-dim 1024 -r -rd 51200 --epochs 30 -i 2500 -p 500 --seed 0 --log logs/cdan/DomainNet_s2c\nCUDA_VISIBLE_DEVICES=0 python cdan.py data/domainnet -d DomainNet -s s -t p -a resnet101 --bottleneck-dim 1024 -r -rd 51200 --epochs 30 -i 2500 -p 500 --seed 0 --log logs/cdan/DomainNet_s2p\nCUDA_VISIBLE_DEVICES=0 python cdan.py data/domainnet -d DomainNet -s s -t r -a resnet101 --bottleneck-dim 1024 -r -rd 51200 --epochs 30 -i 2500 -p 500 --seed 0 --log logs/cdan/DomainNet_s2r\n\n# ResNet50, ImageNet200 -> ImageNetR\nCUDA_VISIBLE_DEVICES=0 python cdan.py data/ImageNetR -d ImageNetR -s IN -t INR -a resnet50 --epochs 30 -i 2500 -p 500 --seed 0 --log logs/cdan/ImageNet_IN2INR\n\n# ig_resnext101_32x8d, ImageNet -> ImageNetSketch\nCUDA_VISIBLE_DEVICES=0 python cdan.py data/imagenet-sketch -d ImageNetSketch -s IN -t sketch -a ig_resnext101_32x8d --bottleneck-dim 1024 --epochs 30 -i 2500 -p 500 --log logs/cdan_ig_resnext101_32x8d/ImageNet_IN2sketch\n\n# Vision Transformer, Office-Home, Single Source\nCUDA_VISIBLE_DEVICES=0 python cdan.py data/office-home -d OfficeHome -s Ar -t Cl -a vit_base_patch16_224 --epochs 30 --seed 0 -b 24 --no-pool --log logs/cdan_vit/OfficeHome_Ar2Cl\nCUDA_VISIBLE_DEVICES=0 python cdan.py data/office-home -d OfficeHome -s Ar -t Pr -a vit_base_patch16_224 --epochs 30 --seed 0 -b 24 --no-pool --log logs/cdan_vit/OfficeHome_Ar2Pr\nCUDA_VISIBLE_DEVICES=0 python cdan.py data/office-home -d OfficeHome -s Ar -t Rw -a vit_base_patch16_224 --epochs 30 --seed 0 -b 24 --no-pool --log logs/cdan_vit/OfficeHome_Ar2Rw\nCUDA_VISIBLE_DEVICES=0 python cdan.py data/office-home -d OfficeHome -s Cl -t Ar -a vit_base_patch16_224 --epochs 30 --seed 0 -b 24 --no-pool --log logs/cdan_vit/OfficeHome_Cl2Ar\nCUDA_VISIBLE_DEVICES=0 python cdan.py data/office-home -d OfficeHome -s Cl -t Pr -a vit_base_patch16_224 --epochs 30 --seed 0 -b 24 --no-pool --log logs/cdan_vit/OfficeHome_Cl2Pr\nCUDA_VISIBLE_DEVICES=0 python cdan.py data/office-home -d OfficeHome -s Cl -t Rw -a vit_base_patch16_224 --epochs 30 --seed 0 -b 24 --no-pool --log logs/cdan_vit/OfficeHome_Cl2Rw\nCUDA_VISIBLE_DEVICES=0 python cdan.py data/office-home -d OfficeHome -s Pr -t Ar -a vit_base_patch16_224 --epochs 30 --seed 0 -b 24 --no-pool --log logs/cdan_vit/OfficeHome_Pr2Ar\nCUDA_VISIBLE_DEVICES=0 python cdan.py data/office-home -d OfficeHome -s Pr -t Cl -a vit_base_patch16_224 --epochs 30 --seed 0 -b 24 --no-pool --log logs/cdan_vit/OfficeHome_Pr2Cl\nCUDA_VISIBLE_DEVICES=0 python cdan.py data/office-home -d OfficeHome -s Pr -t Rw -a vit_base_patch16_224 --epochs 30 --seed 0 -b 24 --no-pool --log logs/cdan_vit/OfficeHome_Pr2Rw\nCUDA_VISIBLE_DEVICES=0 python cdan.py data/office-home -d OfficeHome -s Rw -t Ar -a vit_base_patch16_224 --epochs 30 --seed 0 -b 24 --no-pool --log logs/cdan_vit/OfficeHome_Rw2Ar\nCUDA_VISIBLE_DEVICES=0 python cdan.py data/office-home -d OfficeHome -s Rw -t Cl -a vit_base_patch16_224 --epochs 30 --seed 0 -b 24 --no-pool --log logs/cdan_vit/OfficeHome_Rw2Cl\nCUDA_VISIBLE_DEVICES=0 python cdan.py data/office-home -d OfficeHome -s Rw -t Pr -a vit_base_patch16_224 --epochs 30 --seed 0 -b 24 --no-pool --log logs/cdan_vit/OfficeHome_Rw2Pr\n\n# ResNet50, Office-Home, Multi Source\nCUDA_VISIBLE_DEVICES=0 python cdan.py data/office-home -d OfficeHome -s Cl Pr Rw -t Ar -a resnet50 --epochs 30 --seed 0 --log logs/cdan/OfficeHome_:2Ar\nCUDA_VISIBLE_DEVICES=0 python cdan.py data/office-home -d OfficeHome -s Ar Pr Rw -t Cl -a resnet50 --epochs 30 --seed 0 --log logs/cdan/OfficeHome_:2Cl\nCUDA_VISIBLE_DEVICES=0 python cdan.py data/office-home -d OfficeHome -s Ar Cl Rw -t Pr -a resnet50 --epochs 30 --seed 0 --log logs/cdan/OfficeHome_:2Pr\nCUDA_VISIBLE_DEVICES=0 python cdan.py data/office-home -d OfficeHome -s Ar Cl Pr -t Rw -a resnet50 --epochs 30 --seed 0 --log logs/cdan/OfficeHome_:2Rw\n\n# ResNet101, DomainNet, Multi Source\nCUDA_VISIBLE_DEVICES=0 python cdan.py data/domainnet -d DomainNet -s i p q r s -t c -a resnet101 --bottleneck-dim 1024 -r -rd 51200 --epochs 40 -i 5000 -p 500 --seed 0 --log logs/cdan/DomainNet_:2c\nCUDA_VISIBLE_DEVICES=0 python cdan.py data/domainnet -d DomainNet -s c p q r s -t i -a resnet101 --bottleneck-dim 1024 -r -rd 51200 --epochs 40 -i 5000 -p 500 --seed 0 --log logs/cdan/DomainNet_:2i\nCUDA_VISIBLE_DEVICES=0 python cdan.py data/domainnet -d DomainNet -s c i q r s -t p -a resnet101 --bottleneck-dim 1024 -r -rd 51200 --epochs 40 -i 5000 -p 500 --seed 0 --log logs/cdan/DomainNet_:2p\nCUDA_VISIBLE_DEVICES=0 python cdan.py data/domainnet -d DomainNet -s c i p r s -t q -a resnet101 --bottleneck-dim 1024 -r -rd 51200 --epochs 40 -i 5000 -p 500 --seed 0 --log logs/cdan/DomainNet_:2q\nCUDA_VISIBLE_DEVICES=0 python cdan.py data/domainnet -d DomainNet -s c i p q s -t r -a resnet101 --bottleneck-dim 1024 -r -rd 51200 --epochs 40 -i 5000 -p 500 --seed 0 --log logs/cdan/DomainNet_:2r\nCUDA_VISIBLE_DEVICES=0 python cdan.py data/domainnet -d DomainNet -s c i p q r -t s -a resnet101 --bottleneck-dim 1024 -r -rd 51200 --epochs 40 -i 5000 -p 500 --seed 0 --log logs/cdan/DomainNet_:2s\n\n# ResNet50, Wilds Dataset\nCUDA_VISIBLE_DEVICES=0 python cdan.py data/wilds -d iwildcam -a resnet50 --epochs 30 -i 1000 --seed 0 --log logs/cdan/iwildcam\n\n# DenseNet121, Wilds Dataset\nCUDA_VISIBLE_DEVICES=0 python cdan.py data/wilds -d camelyon17 --train-resizing 'res.' --val-resizing 'res.' --resize-size 96 \\\n -a densenet121 --scratch --epochs 10 -i 1000 --lr 0.01 --seed 0 --log logs/cdan/camelyon17\nCUDA_VISIBLE_DEVICES=0 python cdan.py data/wilds -d fmow --train-resizing 'res.' --val-resizing 'res.' \\\n -a densenet121 --epochs 10 -i 1000 --lr 0.01 --seed 0 --log logs/cdan/fmow\n\n# Digits\nCUDA_VISIBLE_DEVICES=0 python cdan.py data/digits -d Digits -s MNIST -t USPS --train-resizing 'res.' --val-resizing 'res.' \\\n --resize-size 28 --no-hflip --norm-mean 0.5 --norm-std 0.5 -a lenet --no-pool --lr 0.01 -b 128 -i 2500 --scratch --seed 0 --log logs/cdan/MNIST2USPS\nCUDA_VISIBLE_DEVICES=0 python cdan.py data/digits -d Digits -s USPS -t MNIST --train-resizing 'res.' --val-resizing 'res.' \\\n --resize-size 28 --no-hflip --norm-mean 0.5 --norm-std 0.5 -a lenet --no-pool --lr 0.1 -b 128 -i 2500 --scratch --seed 0 --log logs/cdan/USPS2MNIST\nCUDA_VISIBLE_DEVICES=0 python cdan.py data/digits -d Digits -s SVHNRGB -t MNISTRGB --train-resizing 'res.' --val-resizing 'res.' \\\n --resize-size 32 --no-hflip --norm-mean 0.5 0.5 0.5 --norm-std 0.5 0.5 0.5 -a dtn --no-pool --lr 0.01 -b 128 -i 2500 --scratch --seed 0 --log logs/cdan/SVHN2MNIST\n\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/examples/domain_adaptation/image_classification/dan.py",
"content": "\"\"\"\n@author: Junguang Jiang\n@contact: JiangJunguang1123@outlook.com\n\"\"\"\nimport random\nimport time\nimport warnings\nimport sys\nimport argparse\nimport shutil\nimport os.path as osp\n\nimport torch\nimport torch.nn as nn\nimport torch.backends.cudnn as cudnn\nfrom torch.optim import SGD\nfrom torch.optim.lr_scheduler import LambdaLR\nfrom torch.utils.data import DataLoader\nimport torch.nn.functional as F\n\nsys.path.append('../../..')\nfrom dalib.adaptation.dan import MultipleKernelMaximumMeanDiscrepancy, ImageClassifier\nfrom dalib.modules.kernels import GaussianKernel\nfrom common.utils.data import ForeverDataIterator\nfrom common.utils.metric import accuracy\nfrom common.utils.meter import AverageMeter, ProgressMeter\nfrom common.utils.logger import CompleteLogger\nfrom common.utils.analysis import collect_feature, tsne, a_distance\n\nsys.path.append('.')\nimport utils\n\ndevice = torch.device(\"cuda\" if torch.cuda.is_available() else \"cpu\")\n\nclass TransformerDecorator(torch.nn.Module):\n def __init__(self, backbone=None, add_bt=3, dim=2048, eval_global=0):\n super(TransformerDecorator, self).__init__()\n self.encoder_layers = torch.nn.TransformerEncoderLayer(dim, 4, dim, 0.5)\n self.eval_global = eval_global\n self.add_bt = add_bt\n self.backbone = backbone\n\n def forward_feats(self, feature):\n feature = self.backbone(feature)\n if self.training and self.add_bt > 0:\n pre_feature = feature\n feature = feature.unsqueeze(1)\n feature = self.encoder_layers(feature)\n # feature = self.encoder_layers(feature, feature, feature)\n feature = feature.squeeze(1)\n if self.add_bt >= 1:\n return torch.cat([pre_feature, feature], dim=0)\n return feature\n\n def forward(self, feature):\n feature = self.forward_feats(feature)\n return feature\n\n\ndef main(args: argparse.Namespace):\n logger = CompleteLogger(args.log, args.phase)\n print(args)\n\n if args.seed is not None:\n random.seed(args.seed)\n torch.manual_seed(args.seed)\n cudnn.deterministic = True\n warnings.warn('You have chosen to seed training. '\n 'This will turn on the CUDNN deterministic setting, '\n 'which can slow down your training considerably! '\n 'You may see unexpected behavior when restarting '\n 'from checkpoints.')\n\n cudnn.benchmark = True\n\n # Data loading code\n train_transform = utils.get_train_transform(args.train_resizing, random_horizontal_flip=not args.no_hflip,\n random_color_jitter=False, resize_size=args.resize_size,\n norm_mean=args.norm_mean, norm_std=args.norm_std)\n val_transform = utils.get_val_transform(args.val_resizing, resize_size=args.resize_size,\n norm_mean=args.norm_mean, norm_std=args.norm_std)\n print(\"train_transform: \", train_transform)\n print(\"val_transform: \", val_transform)\n\n train_source_dataset, train_target_dataset, val_dataset, test_dataset, num_classes, args.class_names = \\\n utils.get_dataset(args.data, args.root, args.source, args.target, train_transform, val_transform)\n train_source_loader = DataLoader(train_source_dataset, batch_size=args.batch_size,\n shuffle=True, num_workers=args.workers, drop_last=True)\n train_target_loader = DataLoader(train_target_dataset, batch_size=args.batch_size,\n shuffle=True, num_workers=args.workers, drop_last=True)\n val_loader = DataLoader(val_dataset, batch_size=args.batch_size, shuffle=False, num_workers=args.workers)\n test_loader = DataLoader(test_dataset, batch_size=args.batch_size, shuffle=False, num_workers=args.workers)\n\n train_source_iter = ForeverDataIterator(train_source_loader)\n train_target_iter = ForeverDataIterator(train_target_loader)\n\n # create model\n print(\"=> using model '{}'\".format(args.arch))\n backbone = utils.get_model(args.arch, pretrain=not args.scratch)\n # pool_layer = nn.Identity() if args.no_pool else None\n pool_layer = nn.Sequential(\n nn.AdaptiveAvgPool2d(output_size=(1, 1)),\n nn.Flatten(),\n )\n pool_layer = TransformerDecorator(pool_layer, add_bt=args.add_bt, dim=backbone.out_features)\n classifier = ImageClassifier(backbone, num_classes, bottleneck_dim=args.bottleneck_dim,\n pool_layer=pool_layer, finetune=not args.scratch).to(device)\n\n # define optimizer and lr scheduler\n optimizer = SGD(classifier.get_parameters(), args.lr, momentum=args.momentum, weight_decay=args.wd, nesterov=True)\n lr_scheduler = LambdaLR(optimizer, lambda x: args.lr * (1. + args.lr_gamma * float(x)) ** (-args.lr_decay))\n\n # define loss function\n mkmmd_loss = MultipleKernelMaximumMeanDiscrepancy(\n kernels=[GaussianKernel(alpha=2 ** k) for k in range(-3, 2)],\n linear=not args.non_linear\n )\n\n # resume from the best checkpoint\n if args.phase != 'train':\n checkpoint = torch.load(logger.get_checkpoint_path('best'), map_location='cpu')\n classifier.load_state_dict(checkpoint)\n\n # analysis the model\n if args.phase == 'analysis':\n # extract features from both domains\n feature_extractor = nn.Sequential(classifier.backbone, classifier.pool_layer, classifier.bottleneck).to(device)\n source_feature = collect_feature(train_source_loader, feature_extractor, device)\n target_feature = collect_feature(train_target_loader, feature_extractor, device)\n # plot t-SNE\n tSNE_filename = osp.join(logger.visualize_directory, 'TSNE.pdf')\n tsne.visualize(source_feature, target_feature, tSNE_filename)\n print(\"Saving t-SNE to\", tSNE_filename)\n # calculate A-distance, which is a measure for distribution discrepancy\n A_distance = a_distance.calculate(source_feature, target_feature, device)\n print(\"A-distance =\", A_distance)\n return\n\n if args.phase == 'test':\n acc1 = utils.validate(test_loader, classifier, args, device)\n print(acc1)\n return\n\n # start training\n best_acc1 = 0.\n for epoch in range(args.epochs):\n # train for one epoch\n train(train_source_iter, train_target_iter, classifier, mkmmd_loss, optimizer,\n lr_scheduler, epoch, args)\n\n # evaluate on validation set\n acc1 = utils.validate(val_loader, classifier, args, device)\n\n # remember best acc@1 and save checkpoint\n torch.save(classifier.state_dict(), logger.get_checkpoint_path('latest'))\n if acc1 > best_acc1:\n shutil.copy(logger.get_checkpoint_path('latest'), logger.get_checkpoint_path('best'))\n best_acc1 = max(acc1, best_acc1)\n\n print(\"best_acc1 = {:3.1f}\".format(best_acc1))\n\n # evaluate on test set\n classifier.load_state_dict(torch.load(logger.get_checkpoint_path('best')))\n acc1 = utils.validate(test_loader, classifier, args, device)\n print(\"test_acc1 = {:3.1f}\".format(acc1))\n\n logger.close()\n\n\ndef train(train_source_iter: ForeverDataIterator, train_target_iter: ForeverDataIterator, model: ImageClassifier,\n mkmmd_loss: MultipleKernelMaximumMeanDiscrepancy, optimizer: SGD,\n lr_scheduler: LambdaLR, epoch: int, args: argparse.Namespace):\n batch_time = AverageMeter('Time', ':4.2f')\n data_time = AverageMeter('Data', ':3.1f')\n losses = AverageMeter('Loss', ':3.2f')\n trans_losses = AverageMeter('Trans Loss', ':5.4f')\n cls_accs = AverageMeter('Cls Acc', ':3.1f')\n tgt_accs = AverageMeter('Tgt Acc', ':3.1f')\n\n progress = ProgressMeter(\n args.iters_per_epoch,\n [batch_time, data_time, losses, trans_losses, cls_accs, tgt_accs],\n prefix=\"Epoch: [{}]\".format(epoch))\n\n # switch to train mode\n model.train()\n mkmmd_loss.train()\n\n end = time.time()\n for i in range(args.iters_per_epoch):\n x_s, labels_s = next(train_source_iter)\n x_t, labels_t = next(train_target_iter)\n\n x_s = x_s.to(device)\n x_t = x_t.to(device)\n labels_s = labels_s.to(device)\n labels_t = labels_t.to(device)\n\n # measure data loading time\n data_time.update(time.time() - end)\n\n # compute output\n y_s, f_s = model(x_s)\n y_t, f_t = model(x_t)\n\n if len(y_s) > len(labels_s):\n labels_s = torch.cat([labels_s, labels_s], dim=0)\n if len(y_t) > len(labels_t):\n labels_t = torch.cat([labels_t, labels_t], dim=0)\n cls_loss = F.cross_entropy(y_s, labels_s)\n transfer_loss = mkmmd_loss(f_s, f_t)\n loss = cls_loss + transfer_loss * args.trade_off\n\n cls_acc = accuracy(y_s, labels_s)[0]\n tgt_acc = accuracy(y_t, labels_t)[0]\n\n losses.update(loss.item(), x_s.size(0))\n cls_accs.update(cls_acc.item(), x_s.size(0))\n tgt_accs.update(tgt_acc.item(), x_t.size(0))\n trans_losses.update(transfer_loss.item(), x_s.size(0))\n\n # compute gradient and do SGD step\n optimizer.zero_grad()\n loss.backward()\n optimizer.step()\n lr_scheduler.step()\n\n # measure elapsed time\n batch_time.update(time.time() - end)\n end = time.time()\n\n if i % args.print_freq == 0:\n progress.display(i)\n\n\nif __name__ == '__main__':\n parser = argparse.ArgumentParser(description='DAN for Unsupervised Domain Adaptation')\n # dataset parameters\n parser.add_argument('root', metavar='DIR',\n help='root path of dataset')\n parser.add_argument('-d', '--data', metavar='DATA', default='Office31', choices=utils.get_dataset_names(),\n help='dataset: ' + ' | '.join(utils.get_dataset_names()) +\n ' (default: Office31)')\n parser.add_argument('-s', '--source', help='source domain(s)', nargs='+')\n parser.add_argument('-t', '--target', help='target domain(s)', nargs='+')\n parser.add_argument('--train-resizing', type=str, default='default')\n parser.add_argument('--val-resizing', type=str, default='default')\n parser.add_argument('--resize-size', type=int, default=224,\n help='the image size after resizing')\n parser.add_argument('--no-hflip', action='store_true',\n help='no random horizontal flipping during training')\n parser.add_argument('--norm-mean', type=float, nargs='+',\n default=(0.485, 0.456, 0.406), help='normalization mean')\n parser.add_argument('--norm-std', type=float, nargs='+',\n default=(0.229, 0.224, 0.225), help='normalization std')\n # model parameters\n parser.add_argument('-a', '--arch', metavar='ARCH', default='resnet18',\n choices=utils.get_model_names(),\n help='backbone architecture: ' +\n ' | '.join(utils.get_model_names()) +\n ' (default: resnet18)')\n parser.add_argument('--bottleneck-dim', default=256, type=int,\n help='Dimension of bottleneck')\n parser.add_argument('--no-pool', action='store_true',\n help='no pool layer after the feature extractor.')\n parser.add_argument('--scratch', action='store_true', help='whether train from scratch.')\n parser.add_argument('--non-linear', default=False, action='store_true',\n help='whether not use the linear version')\n parser.add_argument('--trade-off', default=1., type=float,\n help='the trade-off hyper-parameter for transfer loss')\n # training parameters\n parser.add_argument('-b', '--batch-size', default=32, type=int,\n metavar='N',\n help='mini-batch size (default: 32)')\n parser.add_argument('--lr', '--learning-rate', default=0.003, type=float,\n metavar='LR', help='initial learning rate', dest='lr')\n parser.add_argument('--lr-gamma', default=0.0003, type=float, help='parameter for lr scheduler')\n parser.add_argument('--lr-decay', default=0.75, type=float, help='parameter for lr scheduler')\n parser.add_argument('--momentum', default=0.9, type=float, metavar='M',\n help='momentum')\n parser.add_argument('--wd', '--weight-decay', default=0.0005, type=float,\n metavar='W', help='weight decay (default: 5e-4)')\n parser.add_argument('-j', '--workers', default=2, type=int, metavar='N',\n help='number of data loading workers (default: 2)')\n parser.add_argument('--epochs', default=20, type=int, metavar='N',\n help='number of total epochs to run')\n parser.add_argument('-i', '--iters-per-epoch', default=500, type=int,\n help='Number of iterations per epoch')\n parser.add_argument('-p', '--print-freq', default=100, type=int,\n metavar='N', help='print frequency (default: 100)')\n parser.add_argument('--seed', default=None, type=int,\n help='seed for initializing training. ')\n parser.add_argument('--per-class-eval', action='store_true',\n help='whether output per-class accuracy during evaluation')\n parser.add_argument(\"--log\", type=str, default='dan',\n help=\"Where to save logs, checkpoints and debugging images.\")\n parser.add_argument(\"--phase\", type=str, default='train', choices=['train', 'test', 'analysis'],\n help=\"When phase is 'test', only test the model.\"\n \"When phase is 'analysis', only analysis the model.\")\n parser.add_argument(\"--add_bt\", type=int, default=0,\n help=\"add_bt\")\n args = parser.parse_args()\n main(args)\n\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/examples/domain_adaptation/image_classification/dan.sh",
"content": "#!/usr/bin/env bash\n# ResNet50, Office31, Single Source\nCUDA_VISIBLE_DEVICES=0 python dan.py data/office31 -d Office31 -s D -t A -a resnet50 --epochs 20 --seed 0 --log logs/dan/Office31_D2A\nCUDA_VISIBLE_DEVICES=0 python dan.py data/office31 -d Office31 -s W -t A -a resnet50 --epochs 20 --seed 0 --log logs/dan/Office31_W2A\nCUDA_VISIBLE_DEVICES=0 python dan.py data/office31 -d Office31 -s A -t W -a resnet50 --epochs 20 --seed 0 --log logs/dan/Office31_A2W\nCUDA_VISIBLE_DEVICES=0 python dan.py data/office31 -d Office31 -s A -t D -a resnet50 --epochs 20 --seed 0 --log logs/dan/Office31_A2D\nCUDA_VISIBLE_DEVICES=0 python dan.py data/office31 -d Office31 -s D -t W -a resnet50 --epochs 20 --seed 0 --log logs/dan/Office31_D2W\nCUDA_VISIBLE_DEVICES=0 python dan.py data/office31 -d Office31 -s W -t D -a resnet50 --epochs 20 --seed 0 --log logs/dan/Office31_W2D\n\n# ResNet50, Office-Home, Single Source\nCUDA_VISIBLE_DEVICES=0 python dan.py data/office-home -d OfficeHome -s Ar -t Cl -a resnet50 --epochs 20 -i 500 --seed 0 --log logs/dan/OfficeHome_Ar2Cl\nCUDA_VISIBLE_DEVICES=0 python dan.py data/office-home -d OfficeHome -s Ar -t Pr -a resnet50 --epochs 20 -i 500 --seed 0 --log logs/dan/OfficeHome_Ar2Pr\nCUDA_VISIBLE_DEVICES=0 python dan.py data/office-home -d OfficeHome -s Ar -t Rw -a resnet50 --epochs 20 -i 500 --seed 0 --log logs/dan/OfficeHome_Ar2Rw\nCUDA_VISIBLE_DEVICES=0 python dan.py data/office-home -d OfficeHome -s Cl -t Ar -a resnet50 --epochs 20 -i 500 --seed 0 --log logs/dan/OfficeHome_Cl2Ar\nCUDA_VISIBLE_DEVICES=0 python dan.py data/office-home -d OfficeHome -s Cl -t Pr -a resnet50 --epochs 20 -i 500 --seed 0 --log logs/dan/OfficeHome_Cl2Pr\nCUDA_VISIBLE_DEVICES=0 python dan.py data/office-home -d OfficeHome -s Cl -t Rw -a resnet50 --epochs 20 -i 500 --seed 0 --log logs/dan/OfficeHome_Cl2Rw\nCUDA_VISIBLE_DEVICES=0 python dan.py data/office-home -d OfficeHome -s Pr -t Ar -a resnet50 --epochs 20 -i 500 --seed 0 --log logs/dan/OfficeHome_Pr2Ar\nCUDA_VISIBLE_DEVICES=0 python dan.py data/office-home -d OfficeHome -s Pr -t Cl -a resnet50 --epochs 20 -i 500 --seed 0 --log logs/dan/OfficeHome_Pr2Cl\nCUDA_VISIBLE_DEVICES=0 python dan.py data/office-home -d OfficeHome -s Pr -t Rw -a resnet50 --epochs 20 -i 500 --seed 0 --log logs/dan/OfficeHome_Pr2Rw\nCUDA_VISIBLE_DEVICES=0 python dan.py data/office-home -d OfficeHome -s Rw -t Ar -a resnet50 --epochs 20 -i 500 --seed 0 --log logs/dan/OfficeHome_Rw2Ar\nCUDA_VISIBLE_DEVICES=0 python dan.py data/office-home -d OfficeHome -s Rw -t Cl -a resnet50 --epochs 20 -i 500 --seed 0 --log logs/dan/OfficeHome_Rw2Cl\nCUDA_VISIBLE_DEVICES=0 python dan.py data/office-home -d OfficeHome -s Rw -t Pr -a resnet50 --epochs 20 -i 500 --seed 0 --log logs/dan/OfficeHome_Rw2Pr\n\n# ResNet101, VisDA-2017, Single Source\nCUDA_VISIBLE_DEVICES=0 python dan.py data/visda-2017 -d VisDA2017 -s Synthetic -t Real -a resnet101 \\\n --epochs 20 -i 500 --seed 0 --per-class-eval --train-resizing cen.crop --log logs/dan/VisDA2017\n\n# ResNet101, DomainNet, Single Source\nCUDA_VISIBLE_DEVICES=0 python dan.py data/domainnet -d DomainNet -s c -t p -a resnet101 --bottleneck-dim 1024 --epochs 30 -i 2500 -p 500 --seed 0 --log logs/dan/DomainNet_c2p\nCUDA_VISIBLE_DEVICES=0 python dan.py data/domainnet -d DomainNet -s c -t r -a resnet101 --bottleneck-dim 1024 --epochs 30 -i 2500 -p 500 --seed 0 --log logs/dan/DomainNet_c2r\nCUDA_VISIBLE_DEVICES=0 python dan.py data/domainnet -d DomainNet -s c -t s -a resnet101 --bottleneck-dim 1024 --epochs 30 -i 2500 -p 500 --seed 0 --log logs/dan/DomainNet_c2s\nCUDA_VISIBLE_DEVICES=0 python dan.py data/domainnet -d DomainNet -s p -t c -a resnet101 --bottleneck-dim 1024 --epochs 30 -i 2500 -p 500 --seed 0 --log logs/dan/DomainNet_p2c\nCUDA_VISIBLE_DEVICES=0 python dan.py data/domainnet -d DomainNet -s p -t r -a resnet101 --bottleneck-dim 1024 --epochs 30 -i 2500 -p 500 --seed 0 --log logs/dan/DomainNet_p2r\nCUDA_VISIBLE_DEVICES=0 python dan.py data/domainnet -d DomainNet -s p -t s -a resnet101 --bottleneck-dim 1024 --epochs 30 -i 2500 -p 500 --seed 0 --log logs/dan/DomainNet_p2s\nCUDA_VISIBLE_DEVICES=0 python dan.py data/domainnet -d DomainNet -s r -t c -a resnet101 --bottleneck-dim 1024 --epochs 30 -i 2500 -p 500 --seed 0 --log logs/dan/DomainNet_r2c\nCUDA_VISIBLE_DEVICES=0 python dan.py data/domainnet -d DomainNet -s r -t p -a resnet101 --bottleneck-dim 1024 --epochs 30 -i 2500 -p 500 --seed 0 --log logs/dan/DomainNet_r2p\nCUDA_VISIBLE_DEVICES=0 python dan.py data/domainnet -d DomainNet -s r -t s -a resnet101 --bottleneck-dim 1024 --epochs 30 -i 2500 -p 500 --seed 0 --log logs/dan/DomainNet_r2s\nCUDA_VISIBLE_DEVICES=0 python dan.py data/domainnet -d DomainNet -s s -t c -a resnet101 --bottleneck-dim 1024 --epochs 30 -i 2500 -p 500 --seed 0 --log logs/dan/DomainNet_s2c\nCUDA_VISIBLE_DEVICES=0 python dan.py data/domainnet -d DomainNet -s s -t p -a resnet101 --bottleneck-dim 1024 --epochs 30 -i 2500 -p 500 --seed 0 --log logs/dan/DomainNet_s2p\nCUDA_VISIBLE_DEVICES=0 python dan.py data/domainnet -d DomainNet -s s -t r -a resnet101 --bottleneck-dim 1024 --epochs 30 -i 2500 -p 500 --seed 0 --log logs/dan/DomainNet_s2r\n\n# ResNet50, ImageNet200 -> ImageNetR\nCUDA_VISIBLE_DEVICES=0 python dan.py data/ImageNetR -d ImageNetR -s IN -t INR -a resnet50 --epochs 30 -i 2500 -p 500 --seed 0 --log logs/dan/ImageNet_IN2INR\n\n# ig_resnext101_32x8d, ImageNet -> ImageNetSketch\nCUDA_VISIBLE_DEVICES=0 python dan.py data/imagenet-sketch -d ImageNetSketch -s IN -t sketch -a ig_resnext101_32x8d --bottleneck-dim 1024 --epochs 30 -i 2500 -p 500 --log logs/dan_ig_resnext101_32x8d/ImageNet_IN2sketch\n\n# Vision Transformer, Office-Home, Single Source\nCUDA_VISIBLE_DEVICES=0 python dan.py data/office-home -d OfficeHome -s Ar -t Cl -a vit_base_patch16_224 --no-pool --epochs 30 --seed 0 -b 24 --log logs/dan_vit/OfficeHome_Ar2Cl\nCUDA_VISIBLE_DEVICES=0 python dan.py data/office-home -d OfficeHome -s Ar -t Pr -a vit_base_patch16_224 --no-pool --epochs 30 --seed 0 -b 24 --log logs/dan_vit/OfficeHome_Ar2Pr\nCUDA_VISIBLE_DEVICES=0 python dan.py data/office-home -d OfficeHome -s Ar -t Rw -a vit_base_patch16_224 --no-pool --epochs 30 --seed 0 -b 24 --log logs/dan_vit/OfficeHome_Ar2Rw\nCUDA_VISIBLE_DEVICES=0 python dan.py data/office-home -d OfficeHome -s Cl -t Ar -a vit_base_patch16_224 --no-pool --epochs 30 --seed 0 -b 24 --log logs/dan_vit/OfficeHome_Cl2Ar\nCUDA_VISIBLE_DEVICES=0 python dan.py data/office-home -d OfficeHome -s Cl -t Pr -a vit_base_patch16_224 --no-pool --epochs 30 --seed 0 -b 24 --log logs/dan_vit/OfficeHome_Cl2Pr\nCUDA_VISIBLE_DEVICES=0 python dan.py data/office-home -d OfficeHome -s Cl -t Rw -a vit_base_patch16_224 --no-pool --epochs 30 --seed 0 -b 24 --log logs/dan_vit/OfficeHome_Cl2Rw\nCUDA_VISIBLE_DEVICES=0 python dan.py data/office-home -d OfficeHome -s Pr -t Ar -a vit_base_patch16_224 --no-pool --epochs 30 --seed 0 -b 24 --log logs/dan_vit/OfficeHome_Pr2Ar\nCUDA_VISIBLE_DEVICES=0 python dan.py data/office-home -d OfficeHome -s Pr -t Cl -a vit_base_patch16_224 --no-pool --epochs 30 --seed 0 -b 24 --log logs/dan_vit/OfficeHome_Pr2Cl\nCUDA_VISIBLE_DEVICES=0 python dan.py data/office-home -d OfficeHome -s Pr -t Rw -a vit_base_patch16_224 --no-pool --epochs 30 --seed 0 -b 24 --log logs/dan_vit/OfficeHome_Pr2Rw\nCUDA_VISIBLE_DEVICES=0 python dan.py data/office-home -d OfficeHome -s Rw -t Ar -a vit_base_patch16_224 --no-pool --epochs 30 --seed 0 -b 24 --log logs/dan_vit/OfficeHome_Rw2Ar\nCUDA_VISIBLE_DEVICES=0 python dan.py data/office-home -d OfficeHome -s Rw -t Cl -a vit_base_patch16_224 --no-pool --epochs 30 --seed 0 -b 24 --log logs/dan_vit/OfficeHome_Rw2Cl\nCUDA_VISIBLE_DEVICES=0 python dan.py data/office-home -d OfficeHome -s Rw -t Pr -a vit_base_patch16_224 --no-pool --epochs 30 --seed 0 -b 24 --log logs/dan_vit/OfficeHome_Rw2Pr\n\n# ResNet50, Office-Home, Multi Source\nCUDA_VISIBLE_DEVICES=0 python dan.py data/office-home -d OfficeHome -s Cl Pr Rw -t Ar -a resnet50 --epochs 30 --seed 0 --log logs/dan/OfficeHome_:2Ar\nCUDA_VISIBLE_DEVICES=0 python dan.py data/office-home -d OfficeHome -s Ar Pr Rw -t Cl -a resnet50 --epochs 30 --seed 0 --log logs/dan/OfficeHome_:2Cl\nCUDA_VISIBLE_DEVICES=0 python dan.py data/office-home -d OfficeHome -s Ar Cl Rw -t Pr -a resnet50 --epochs 30 --seed 0 --log logs/dan/OfficeHome_:2Pr\nCUDA_VISIBLE_DEVICES=0 python dan.py data/office-home -d OfficeHome -s Ar Cl Pr -t Rw -a resnet50 --epochs 30 --seed 0 --log logs/dan/OfficeHome_:2Rw\n\n# ResNet101, DomainNet, Multi Source\nCUDA_VISIBLE_DEVICES=0 python dan.py data/domainnet -d DomainNet -s i p q r s -t c -a resnet101 --bottleneck-dim 1024 --epochs 40 -i 5000 -p 500 --seed 0 --log logs/dan/DomainNet_:2c\nCUDA_VISIBLE_DEVICES=0 python dan.py data/domainnet -d DomainNet -s c p q r s -t i -a resnet101 --bottleneck-dim 1024 --epochs 40 -i 5000 -p 500 --seed 0 --log logs/dan/DomainNet_:2i\nCUDA_VISIBLE_DEVICES=0 python dan.py data/domainnet -d DomainNet -s c i q r s -t p -a resnet101 --bottleneck-dim 1024 --epochs 40 -i 5000 -p 500 --seed 0 --log logs/dan/DomainNet_:2p\nCUDA_VISIBLE_DEVICES=0 python dan.py data/domainnet -d DomainNet -s c i p r s -t q -a resnet101 --bottleneck-dim 1024 --epochs 40 -i 5000 -p 500 --seed 0 --log logs/dan/DomainNet_:2q\nCUDA_VISIBLE_DEVICES=0 python dan.py data/domainnet -d DomainNet -s c i p q s -t r -a resnet101 --bottleneck-dim 1024 --epochs 40 -i 5000 -p 500 --seed 0 --log logs/dan/DomainNet_:2r\nCUDA_VISIBLE_DEVICES=0 python dan.py data/domainnet -d DomainNet -s c i p q r -t s -a resnet101 --bottleneck-dim 1024 --epochs 40 -i 5000 -p 500 --seed 0 --log logs/dan/DomainNet_:2s\n\n# ResNet50, Wilds Dataset\nCUDA_VISIBLE_DEVICES=0 python dan.py data/wilds -d iwildcam -a resnet50 --epochs 30 -i 1000 --seed 0 --log logs/dan/iwildcam\n\n# DenseNet121, Wilds Dataset\nCUDA_VISIBLE_DEVICES=0 python dan.py data/wilds -d camelyon17 --train-resizing 'res.' --val-resizing 'res.' --resize-size 96 \\\n -a densenet121 --scratch --epochs 10 -i 1000 --lr 0.01 --seed 0 --log logs/dan/camelyon17\nCUDA_VISIBLE_DEVICES=0 python dan.py data/wilds -d fmow --train-resizing 'res.' --val-resizing 'res.' \\\n -a densenet121 --epochs 10 -i 1000 --lr 0.01 --seed 0 --log logs/dan/fmow\n\n# Digits\nCUDA_VISIBLE_DEVICES=0 python dan.py data/digits -d Digits -s MNIST -t USPS --train-resizing 'res.' --val-resizing 'res.' \\\n --resize-size 28 --no-hflip --norm-mean 0.5 --norm-std 0.5 -a lenet --no-pool --lr 0.01 -b 128 -i 2500 --scratch --seed 0 --log logs/dan/MNIST2USPS\nCUDA_VISIBLE_DEVICES=0 python dan.py data/digits -d Digits -s USPS -t MNIST --train-resizing 'res.' --val-resizing 'res.' \\\n --resize-size 28 --no-hflip --norm-mean 0.5 --norm-std 0.5 -a lenet --no-pool --lr 0.1 -b 128 -i 2500 --scratch --seed 0 --log logs/dan/USPS2MNIST\nCUDA_VISIBLE_DEVICES=0 python dan.py data/digits -d Digits -s SVHNRGB -t MNISTRGB --train-resizing 'res.' --val-resizing 'res.' \\\n --resize-size 32 --no-hflip --norm-mean 0.5 0.5 0.5 --norm-std 0.5 0.5 0.5 -a dtn --no-pool --lr 0.01 -b 128 -i 2500 --scratch --seed 0 --log logs/dan/SVHN2MNIST\n\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/examples/domain_adaptation/image_classification/dann.py",
"content": "\"\"\"\n@author: Junguang Jiang\n@contact: JiangJunguang1123@outlook.com\n\"\"\"\nimport random\nimport time\nimport warnings\nimport sys\nimport argparse\nimport shutil\nimport os.path as osp\n\nimport torch\nimport torch.nn as nn\nimport torch.backends.cudnn as cudnn\nfrom torch.optim import SGD\nfrom torch.optim.lr_scheduler import LambdaLR\nfrom torch.utils.data import DataLoader\nimport torch.nn.functional as F\n\nsys.path.append('../../..')\nfrom dalib.modules.domain_discriminator import DomainDiscriminator\nfrom dalib.adaptation.dann import DomainAdversarialLoss, ImageClassifier\nfrom common.utils.data import ForeverDataIterator\nfrom common.utils.metric import accuracy\nfrom common.utils.meter import AverageMeter, ProgressMeter\nfrom common.utils.logger import CompleteLogger\nfrom common.utils.analysis import collect_feature, tsne, a_distance\n\nsys.path.append('.')\nimport utils\n\ndevice = torch.device(\"cuda\" if torch.cuda.is_available() else \"cpu\")\n\nclass TransformerDecorator(torch.nn.Module):\n def __init__(self, backbone=None, add_bt=3, dim=2048, eval_global=0):\n super(TransformerDecorator, self).__init__()\n self.encoder_layers = torch.nn.TransformerEncoderLayer(dim, 4, dim, 0.5)\n self.eval_global = eval_global\n self.add_bt = add_bt\n self.backbone = backbone\n\n def forward_feats(self, feature):\n feature = self.backbone(feature)\n if self.training and self.add_bt > 0:\n pre_feature = feature\n feature = feature.unsqueeze(1)\n feature = self.encoder_layers(feature)\n # feature = self.encoder_layers(feature, feature, feature)\n feature = feature.squeeze(1)\n if self.add_bt >= 1:\n return torch.cat([pre_feature, feature], dim=0)\n return feature\n\n def forward(self, feature):\n feature = self.forward_feats(feature)\n return feature\n\n\ndef main(args: argparse.Namespace):\n logger = CompleteLogger(args.log, args.phase)\n print(args)\n\n if args.seed is not None:\n random.seed(args.seed)\n torch.manual_seed(args.seed)\n cudnn.deterministic = True\n warnings.warn('You have chosen to seed training. '\n 'This will turn on the CUDNN deterministic setting, '\n 'which can slow down your training considerably! '\n 'You may see unexpected behavior when restarting '\n 'from checkpoints.')\n\n cudnn.benchmark = True\n\n # Data loading code\n train_transform = utils.get_train_transform(args.train_resizing, random_horizontal_flip=not args.no_hflip,\n random_color_jitter=False, resize_size=args.resize_size,\n norm_mean=args.norm_mean, norm_std=args.norm_std)\n val_transform = utils.get_val_transform(args.val_resizing, resize_size=args.resize_size,\n norm_mean=args.norm_mean, norm_std=args.norm_std)\n print(\"train_transform: \", train_transform)\n print(\"val_transform: \", val_transform)\n\n train_source_dataset, train_target_dataset, val_dataset, test_dataset, num_classes, args.class_names = \\\n utils.get_dataset(args.data, args.root, args.source, args.target, train_transform, val_transform)\n train_source_loader = DataLoader(train_source_dataset, batch_size=args.batch_size,\n shuffle=True, num_workers=args.workers, drop_last=True)\n train_target_loader = DataLoader(train_target_dataset, batch_size=args.batch_size,\n shuffle=True, num_workers=args.workers, drop_last=True)\n val_loader = DataLoader(val_dataset, batch_size=args.batch_size, shuffle=False, num_workers=args.workers)\n test_loader = DataLoader(test_dataset, batch_size=args.batch_size, shuffle=False, num_workers=args.workers)\n\n train_source_iter = ForeverDataIterator(train_source_loader)\n train_target_iter = ForeverDataIterator(train_target_loader)\n\n # create model\n print(\"=> using model '{}'\".format(args.arch))\n backbone = utils.get_model(args.arch, pretrain=not args.scratch)\n # pool_layer = nn.Identity() if args.no_pool else None\n pool_layer = nn.Sequential(\n nn.AdaptiveAvgPool2d(output_size=(1, 1)),\n nn.Flatten(),\n )\n pool_layer = TransformerDecorator(pool_layer, add_bt=args.add_bt, dim=backbone.out_features)\n classifier = ImageClassifier(backbone, num_classes, bottleneck_dim=args.bottleneck_dim,\n pool_layer=pool_layer, finetune=not args.scratch).to(device)\n domain_discri = DomainDiscriminator(in_feature=classifier.features_dim, hidden_size=1024).to(device)\n\n # define optimizer and lr scheduler\n optimizer = SGD(classifier.get_parameters() + domain_discri.get_parameters(),\n args.lr, momentum=args.momentum, weight_decay=args.weight_decay, nesterov=True)\n lr_scheduler = LambdaLR(optimizer, lambda x: args.lr * (1. + args.lr_gamma * float(x)) ** (-args.lr_decay))\n\n # define loss function\n domain_adv = DomainAdversarialLoss(domain_discri).to(device)\n\n # resume from the best checkpoint\n if args.phase != 'train':\n checkpoint = torch.load(logger.get_checkpoint_path('best'), map_location='cpu')\n classifier.load_state_dict(checkpoint)\n\n # analysis the model\n if args.phase == 'analysis':\n # extract features from both domains\n feature_extractor = nn.Sequential(classifier.backbone, classifier.pool_layer, classifier.bottleneck).to(device)\n source_feature = collect_feature(train_source_loader, feature_extractor, device)\n target_feature = collect_feature(train_target_loader, feature_extractor, device)\n # plot t-SNE\n tSNE_filename = osp.join(logger.visualize_directory, 'TSNE.pdf')\n tsne.visualize(source_feature, target_feature, tSNE_filename)\n print(\"Saving t-SNE to\", tSNE_filename)\n # calculate A-distance, which is a measure for distribution discrepancy\n A_distance = a_distance.calculate(source_feature, target_feature, device)\n print(\"A-distance =\", A_distance)\n return\n\n if args.phase == 'test':\n acc1 = utils.validate(test_loader, classifier, args, device)\n print(acc1)\n return\n\n # start training\n best_acc1 = 0.\n for epoch in range(args.epochs):\n print(\"lr:\", lr_scheduler.get_last_lr()[0])\n # train for one epoch\n train(train_source_iter, train_target_iter, classifier, domain_adv, optimizer,\n lr_scheduler, epoch, args)\n\n # evaluate on validation set\n acc1 = utils.validate(val_loader, classifier, args, device)\n\n # remember best acc@1 and save checkpoint\n torch.save(classifier.state_dict(), logger.get_checkpoint_path('latest'))\n if acc1 > best_acc1:\n shutil.copy(logger.get_checkpoint_path('latest'), logger.get_checkpoint_path('best'))\n best_acc1 = max(acc1, best_acc1)\n\n print(\"best_acc1 = {:3.1f}\".format(best_acc1))\n\n # evaluate on test set\n classifier.load_state_dict(torch.load(logger.get_checkpoint_path('best')))\n acc1 = utils.validate(test_loader, classifier, args, device)\n print(\"test_acc1 = {:3.1f}\".format(acc1))\n\n logger.close()\n\n\ndef train(train_source_iter: ForeverDataIterator, train_target_iter: ForeverDataIterator,\n model: ImageClassifier, domain_adv: DomainAdversarialLoss, optimizer: SGD,\n lr_scheduler: LambdaLR, epoch: int, args: argparse.Namespace):\n batch_time = AverageMeter('Time', ':5.2f')\n data_time = AverageMeter('Data', ':5.2f')\n losses = AverageMeter('Loss', ':6.2f')\n cls_accs = AverageMeter('Cls Acc', ':3.1f')\n domain_accs = AverageMeter('Domain Acc', ':3.1f')\n progress = ProgressMeter(\n args.iters_per_epoch,\n [batch_time, data_time, losses, cls_accs, domain_accs],\n prefix=\"Epoch: [{}]\".format(epoch))\n\n # switch to train mode\n model.train()\n domain_adv.train()\n\n end = time.time()\n for i in range(args.iters_per_epoch):\n x_s, labels_s = next(train_source_iter)\n x_t, _ = next(train_target_iter)\n\n x_s = x_s.to(device)\n x_t = x_t.to(device)\n labels_s = labels_s.to(device)\n\n # measure data loading time\n data_time.update(time.time() - end)\n\n # compute output\n x = torch.cat((x_s, x_t), dim=0)\n y, f = model(x)\n if len(y) > len(x):\n y_s, y_t, y_s1, y_t1 = y.chunk(4, dim=0)\n y_s = torch.cat([y_s, y_s1], dim=0)\n y_t = torch.cat([y_t, y_t1], dim=0)\n f_s, f_t, f_s1, f_t1 = f.chunk(4, dim=0)\n f_s = torch.cat([f_s, f_s1], dim=0)\n f_t = torch.cat([f_t, f_t1], dim=0)\n else:\n y_s, y_t = y.chunk(2, dim=0)\n f_s, f_t = f.chunk(2, dim=0)\n if len(y_s) > len(labels_s):\n labels_s = torch.cat([labels_s, labels_s], dim=0)\n\n cls_loss = F.cross_entropy(y_s, labels_s)\n transfer_loss = domain_adv(f_s, f_t)\n domain_acc = domain_adv.domain_discriminator_accuracy\n loss = cls_loss + transfer_loss * args.trade_off\n\n cls_acc = accuracy(y_s, labels_s)[0]\n\n losses.update(loss.item(), x_s.size(0))\n cls_accs.update(cls_acc.item(), x_s.size(0))\n domain_accs.update(domain_acc.item(), x_s.size(0))\n\n # compute gradient and do SGD step\n optimizer.zero_grad()\n loss.backward()\n optimizer.step()\n lr_scheduler.step()\n\n # measure elapsed time\n batch_time.update(time.time() - end)\n end = time.time()\n\n if i % args.print_freq == 0:\n progress.display(i)\n\n\nif __name__ == '__main__':\n parser = argparse.ArgumentParser(description='DANN for Unsupervised Domain Adaptation')\n # dataset parameters\n parser.add_argument('root', metavar='DIR',\n help='root path of dataset')\n parser.add_argument('-d', '--data', metavar='DATA', default='Office31', choices=utils.get_dataset_names(),\n help='dataset: ' + ' | '.join(utils.get_dataset_names()) +\n ' (default: Office31)')\n parser.add_argument('-s', '--source', help='source domain(s)', nargs='+')\n parser.add_argument('-t', '--target', help='target domain(s)', nargs='+')\n parser.add_argument('--train-resizing', type=str, default='default')\n parser.add_argument('--val-resizing', type=str, default='default')\n parser.add_argument('--resize-size', type=int, default=224,\n help='the image size after resizing')\n parser.add_argument('--no-hflip', action='store_true',\n help='no random horizontal flipping during training')\n parser.add_argument('--norm-mean', type=float, nargs='+',\n default=(0.485, 0.456, 0.406), help='normalization mean')\n parser.add_argument('--norm-std', type=float, nargs='+',\n default=(0.229, 0.224, 0.225), help='normalization std')\n # model parameters\n parser.add_argument('-a', '--arch', metavar='ARCH', default='resnet18',\n choices=utils.get_model_names(),\n help='backbone architecture: ' +\n ' | '.join(utils.get_model_names()) +\n ' (default: resnet18)')\n parser.add_argument('--bottleneck-dim', default=256, type=int,\n help='Dimension of bottleneck')\n parser.add_argument('--no-pool', action='store_true',\n help='no pool layer after the feature extractor.')\n parser.add_argument('--scratch', action='store_true', help='whether train from scratch.')\n parser.add_argument('--trade-off', default=1., type=float,\n help='the trade-off hyper-parameter for transfer loss')\n # training parameters\n parser.add_argument('-b', '--batch-size', default=32, type=int,\n metavar='N',\n help='mini-batch size (default: 32)')\n parser.add_argument('--lr', '--learning-rate', default=0.01, type=float,\n metavar='LR', help='initial learning rate', dest='lr')\n parser.add_argument('--lr-gamma', default=0.001, type=float, help='parameter for lr scheduler')\n parser.add_argument('--lr-decay', default=0.75, type=float, help='parameter for lr scheduler')\n parser.add_argument('--momentum', default=0.9, type=float, metavar='M',\n help='momentum')\n parser.add_argument('--wd', '--weight-decay',default=1e-3, type=float,\n metavar='W', help='weight decay (default: 1e-3)',\n dest='weight_decay')\n parser.add_argument('-j', '--workers', default=2, type=int, metavar='N',\n help='number of data loading workers (default: 2)')\n parser.add_argument('--epochs', default=20, type=int, metavar='N',\n help='number of total epochs to run')\n parser.add_argument('-i', '--iters-per-epoch', default=1000, type=int,\n help='Number of iterations per epoch')\n parser.add_argument('-p', '--print-freq', default=100, type=int,\n metavar='N', help='print frequency (default: 100)')\n parser.add_argument('--seed', default=None, type=int,\n help='seed for initializing training. ')\n parser.add_argument('--per-class-eval', action='store_true',\n help='whether output per-class accuracy during evaluation')\n parser.add_argument(\"--log\", type=str, default='dann',\n help=\"Where to save logs, checkpoints and debugging images.\")\n parser.add_argument(\"--phase\", type=str, default='train', choices=['train', 'test', 'analysis'],\n help=\"When phase is 'test', only test the model.\"\n \"When phase is 'analysis', only analysis the model.\")\n parser.add_argument(\"--add_bt\", type=int, default=0,\n help=\"add_bt\")\n args = parser.parse_args()\n main(args)\n\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/examples/domain_adaptation/image_classification/dann.sh",
"content": "#!/usr/bin/env bash\n# ResNet50, Office31, Single Source\nCUDA_VISIBLE_DEVICES=0 python dann.py data/office31 -d Office31 -s A -t W -a resnet50 --epochs 20 --seed 1 --log logs/dann/Office31_A2W\nCUDA_VISIBLE_DEVICES=0 python dann.py data/office31 -d Office31 -s D -t W -a resnet50 --epochs 20 --seed 1 --log logs/dann/Office31_D2W\nCUDA_VISIBLE_DEVICES=0 python dann.py data/office31 -d Office31 -s W -t D -a resnet50 --epochs 20 --seed 1 --log logs/dann/Office31_W2D\nCUDA_VISIBLE_DEVICES=0 python dann.py data/office31 -d Office31 -s A -t D -a resnet50 --epochs 20 --seed 1 --log logs/dann/Office31_A2D\nCUDA_VISIBLE_DEVICES=0 python dann.py data/office31 -d Office31 -s D -t A -a resnet50 --epochs 20 --seed 1 --log logs/dann/Office31_D2A\nCUDA_VISIBLE_DEVICES=0 python dann.py data/office31 -d Office31 -s W -t A -a resnet50 --epochs 20 --seed 1 --log logs/dann/Office31_W2A\n\n# ResNet50, Office-Home, Single Source\nCUDA_VISIBLE_DEVICES=0 python dann.py data/office-home -d OfficeHome -s Ar -t Cl -a resnet50 --epochs 30 --seed 0 --log logs/dann/OfficeHome_Ar2Cl\nCUDA_VISIBLE_DEVICES=0 python dann.py data/office-home -d OfficeHome -s Ar -t Pr -a resnet50 --epochs 30 --seed 0 --log logs/dann/OfficeHome_Ar2Pr\nCUDA_VISIBLE_DEVICES=0 python dann.py data/office-home -d OfficeHome -s Ar -t Rw -a resnet50 --epochs 30 --seed 0 --log logs/dann/OfficeHome_Ar2Rw\nCUDA_VISIBLE_DEVICES=0 python dann.py data/office-home -d OfficeHome -s Cl -t Ar -a resnet50 --epochs 30 --seed 0 --log logs/dann/OfficeHome_Cl2Ar\nCUDA_VISIBLE_DEVICES=0 python dann.py data/office-home -d OfficeHome -s Cl -t Pr -a resnet50 --epochs 30 --seed 0 --log logs/dann/OfficeHome_Cl2Pr\nCUDA_VISIBLE_DEVICES=0 python dann.py data/office-home -d OfficeHome -s Cl -t Rw -a resnet50 --epochs 30 --seed 0 --log logs/dann/OfficeHome_Cl2Rw\nCUDA_VISIBLE_DEVICES=0 python dann.py data/office-home -d OfficeHome -s Pr -t Ar -a resnet50 --epochs 30 --seed 0 --log logs/dann/OfficeHome_Pr2Ar\nCUDA_VISIBLE_DEVICES=0 python dann.py data/office-home -d OfficeHome -s Pr -t Cl -a resnet50 --epochs 30 --seed 0 --log logs/dann/OfficeHome_Pr2Cl\nCUDA_VISIBLE_DEVICES=0 python dann.py data/office-home -d OfficeHome -s Pr -t Rw -a resnet50 --epochs 30 --seed 0 --log logs/dann/OfficeHome_Pr2Rw\nCUDA_VISIBLE_DEVICES=0 python dann.py data/office-home -d OfficeHome -s Rw -t Ar -a resnet50 --epochs 30 --seed 0 --log logs/dann/OfficeHome_Rw2Ar\nCUDA_VISIBLE_DEVICES=0 python dann.py data/office-home -d OfficeHome -s Rw -t Cl -a resnet50 --epochs 30 --seed 0 --log logs/dann/OfficeHome_Rw2Cl\nCUDA_VISIBLE_DEVICES=0 python dann.py data/office-home -d OfficeHome -s Rw -t Pr -a resnet50 --epochs 30 --seed 0 --log logs/dann/OfficeHome_Rw2Pr\n\n# ResNet101, VisDA-2017, Single Source\nCUDA_VISIBLE_DEVICES=0 python dann.py data/visda-2017 -d VisDA2017 -s Synthetic -t Real -a resnet101 \\\n --epochs 30 --seed 0 --per-class-eval --train-resizing cen.crop --log logs/dann/VisDA2017\n\n# ResNet101, DomainNet, Single Source\nCUDA_VISIBLE_DEVICES=0 python dann.py data/domainnet -d DomainNet -s c -t p -a resnet101 --bottleneck-dim 1024 --epochs 30 -i 2500 -p 500 --seed 0 --log logs/dann/DomainNet_c2p\nCUDA_VISIBLE_DEVICES=0 python dann.py data/domainnet -d DomainNet -s c -t r -a resnet101 --bottleneck-dim 1024 --epochs 30 -i 2500 -p 500 --seed 0 --log logs/dann/DomainNet_c2r\nCUDA_VISIBLE_DEVICES=0 python dann.py data/domainnet -d DomainNet -s c -t s -a resnet101 --bottleneck-dim 1024 --epochs 30 -i 2500 -p 500 --seed 0 --log logs/dann/DomainNet_c2s\nCUDA_VISIBLE_DEVICES=0 python dann.py data/domainnet -d DomainNet -s p -t c -a resnet101 --bottleneck-dim 1024 --epochs 30 -i 2500 -p 500 --seed 0 --log logs/dann/DomainNet_p2c\nCUDA_VISIBLE_DEVICES=0 python dann.py data/domainnet -d DomainNet -s p -t r -a resnet101 --bottleneck-dim 1024 --epochs 30 -i 2500 -p 500 --seed 0 --log logs/dann/DomainNet_p2r\nCUDA_VISIBLE_DEVICES=0 python dann.py data/domainnet -d DomainNet -s p -t s -a resnet101 --bottleneck-dim 1024 --epochs 30 -i 2500 -p 500 --seed 0 --log logs/dann/DomainNet_p2s\nCUDA_VISIBLE_DEVICES=0 python dann.py data/domainnet -d DomainNet -s r -t c -a resnet101 --bottleneck-dim 1024 --epochs 30 -i 2500 -p 500 --seed 0 --log logs/dann/DomainNet_r2c\nCUDA_VISIBLE_DEVICES=0 python dann.py data/domainnet -d DomainNet -s r -t p -a resnet101 --bottleneck-dim 1024 --epochs 30 -i 2500 -p 500 --seed 0 --log logs/dann/DomainNet_r2p\nCUDA_VISIBLE_DEVICES=0 python dann.py data/domainnet -d DomainNet -s r -t s -a resnet101 --bottleneck-dim 1024 --epochs 30 -i 2500 -p 500 --seed 0 --log logs/dann/DomainNet_r2s\nCUDA_VISIBLE_DEVICES=0 python dann.py data/domainnet -d DomainNet -s s -t c -a resnet101 --bottleneck-dim 1024 --epochs 30 -i 2500 -p 500 --seed 0 --log logs/dann/DomainNet_s2c\nCUDA_VISIBLE_DEVICES=0 python dann.py data/domainnet -d DomainNet -s s -t p -a resnet101 --bottleneck-dim 1024 --epochs 30 -i 2500 -p 500 --seed 0 --log logs/dann/DomainNet_s2p\nCUDA_VISIBLE_DEVICES=0 python dann.py data/domainnet -d DomainNet -s s -t r -a resnet101 --bottleneck-dim 1024 --epochs 30 -i 2500 -p 500 --seed 0 --log logs/dann/DomainNet_s2r\n\n# ResNet50, ImageNet200 -> ImageNetR\nCUDA_VISIBLE_DEVICES=0 python dann.py data/ImageNetR -d ImageNetR -s IN -t INR -a resnet50 --epochs 30 -i 2500 -p 500 --seed 0 --log logs/dann/ImageNet_IN2INR\n\n# ig_resnext101_32x8d, ImageNet -> ImageNetSketch\nCUDA_VISIBLE_DEVICES=0 python dann.py data/imagenet-sketch -d ImageNetSketch -s IN -t sketch -a ig_resnext101_32x8d --epochs 30 -i 2500 -p 500 --bottleneck-dim 1024 --log logs/dann_ig_resnext101_32x8d/ImageNet_IN2sketch\n\n# Vision Transformer, Office-Home, Single Source\nCUDA_VISIBLE_DEVICES=0 python dann.py data/office-home -d OfficeHome -s Ar -t Cl -a vit_base_patch16_224 --epochs 30 --seed 0 -b 24 --no-pool --log logs/dann_vit/OfficeHome_Ar2Cl\nCUDA_VISIBLE_DEVICES=0 python dann.py data/office-home -d OfficeHome -s Ar -t Pr -a vit_base_patch16_224 --epochs 30 --seed 0 -b 24 --no-pool --log logs/dann_vit/OfficeHome_Ar2Pr\nCUDA_VISIBLE_DEVICES=0 python dann.py data/office-home -d OfficeHome -s Ar -t Rw -a vit_base_patch16_224 --epochs 30 --seed 0 -b 24 --no-pool --log logs/dann_vit/OfficeHome_Ar2Rw\nCUDA_VISIBLE_DEVICES=0 python dann.py data/office-home -d OfficeHome -s Cl -t Ar -a vit_base_patch16_224 --epochs 30 --seed 0 -b 24 --no-pool --log logs/dann_vit/OfficeHome_Cl2Ar\nCUDA_VISIBLE_DEVICES=0 python dann.py data/office-home -d OfficeHome -s Cl -t Pr -a vit_base_patch16_224 --epochs 30 --seed 0 -b 24 --no-pool --log logs/dann_vit/OfficeHome_Cl2Pr\nCUDA_VISIBLE_DEVICES=0 python dann.py data/office-home -d OfficeHome -s Cl -t Rw -a vit_base_patch16_224 --epochs 30 --seed 0 -b 24 --no-pool --log logs/dann_vit/OfficeHome_Cl2Rw\nCUDA_VISIBLE_DEVICES=0 python dann.py data/office-home -d OfficeHome -s Pr -t Ar -a vit_base_patch16_224 --epochs 30 --seed 0 -b 24 --no-pool --log logs/dann_vit/OfficeHome_Pr2Ar\nCUDA_VISIBLE_DEVICES=0 python dann.py data/office-home -d OfficeHome -s Pr -t Cl -a vit_base_patch16_224 --epochs 30 --seed 0 -b 24 --no-pool --log logs/dann_vit/OfficeHome_Pr2Cl\nCUDA_VISIBLE_DEVICES=0 python dann.py data/office-home -d OfficeHome -s Pr -t Rw -a vit_base_patch16_224 --epochs 30 --seed 0 -b 24 --no-pool --log logs/dann_vit/OfficeHome_Pr2Rw\nCUDA_VISIBLE_DEVICES=0 python dann.py data/office-home -d OfficeHome -s Rw -t Ar -a vit_base_patch16_224 --epochs 30 --seed 0 -b 24 --no-pool --log logs/dann_vit/OfficeHome_Rw2Ar\nCUDA_VISIBLE_DEVICES=0 python dann.py data/office-home -d OfficeHome -s Rw -t Cl -a vit_base_patch16_224 --epochs 30 --seed 0 -b 24 --no-pool --log logs/dann_vit/OfficeHome_Rw2Cl\nCUDA_VISIBLE_DEVICES=0 python dann.py data/office-home -d OfficeHome -s Rw -t Pr -a vit_base_patch16_224 --epochs 30 --seed 0 -b 24 --no-pool --log logs/dann_vit/OfficeHome_Rw2Pr\n\n# ResNet50, Office-Home, Multi Source\nCUDA_VISIBLE_DEVICES=0 python dann.py data/office-home -d OfficeHome -s Cl Pr Rw -t Ar -a resnet50 --epochs 30 --seed 0 --log logs/dann/OfficeHome_:2Ar\nCUDA_VISIBLE_DEVICES=0 python dann.py data/office-home -d OfficeHome -s Ar Pr Rw -t Cl -a resnet50 --epochs 30 --seed 0 --log logs/dann/OfficeHome_:2Cl\nCUDA_VISIBLE_DEVICES=0 python dann.py data/office-home -d OfficeHome -s Ar Cl Rw -t Pr -a resnet50 --epochs 30 --seed 0 --log logs/dann/OfficeHome_:2Pr\nCUDA_VISIBLE_DEVICES=0 python dann.py data/office-home -d OfficeHome -s Ar Cl Pr -t Rw -a resnet50 --epochs 30 --seed 0 --log logs/dann/OfficeHome_:2Rw\n\n# ResNet101, DomainNet, Multi Source\nCUDA_VISIBLE_DEVICES=0 python dann.py data/domainnet -d DomainNet -s i p q r s -t c -a resnet101 --bottleneck-dim 1024 --epochs 40 -i 5000 -p 500 --seed 0 --log logs/dann/DomainNet_:2c\nCUDA_VISIBLE_DEVICES=0 python dann.py data/domainnet -d DomainNet -s c p q r s -t i -a resnet101 --bottleneck-dim 1024 --epochs 40 -i 5000 -p 500 --seed 0 --log logs/dann/DomainNet_:2i\nCUDA_VISIBLE_DEVICES=0 python dann.py data/domainnet -d DomainNet -s c i q r s -t p -a resnet101 --bottleneck-dim 1024 --epochs 40 -i 5000 -p 500 --seed 0 --log logs/dann/DomainNet_:2p\nCUDA_VISIBLE_DEVICES=0 python dann.py data/domainnet -d DomainNet -s c i p r s -t q -a resnet101 --bottleneck-dim 1024 --epochs 40 -i 5000 -p 500 --seed 0 --log logs/dann/DomainNet_:2q\nCUDA_VISIBLE_DEVICES=0 python dann.py data/domainnet -d DomainNet -s c i p q s -t r -a resnet101 --bottleneck-dim 1024 --epochs 40 -i 5000 -p 500 --seed 0 --log logs/dann/DomainNet_:2r\nCUDA_VISIBLE_DEVICES=0 python dann.py data/domainnet -d DomainNet -s c i p q r -t s -a resnet101 --bottleneck-dim 1024 --epochs 40 -i 5000 -p 500 --seed 0 --log logs/dann/DomainNet_:2s\n\n# ResNet50, Wilds Dataset\nCUDA_VISIBLE_DEVICES=0 python dann.py data/wilds -d iwildcam -a resnet50 --epochs 30 -i 1000 --seed 0 --log logs/dann/iwildcam\n\n# DenseNet121, Wilds Dataset\nCUDA_VISIBLE_DEVICES=0 python dann.py data/wilds -d camelyon17 --train-resizing 'res.' --val-resizing 'res.' --resize-size 96 \\\n -a densenet121 --scratch --epochs 10 -i 1000 --lr 0.01 --seed 0 --log logs/dann/camelyon17\nCUDA_VISIBLE_DEVICES=0 python dann.py data/wilds -d fmow --train-resizing 'res.' --val-resizing 'res.' \\\n -a densenet121 --epochs 10 -i 1000 --lr 0.01 --seed 0 --log logs/dann/fmow\n\n# Digits\nCUDA_VISIBLE_DEVICES=0 python dann.py data/digits -d Digits -s MNIST -t USPS --train-resizing 'res.' --val-resizing 'res.' \\\n --resize-size 28 --no-hflip --norm-mean 0.5 --norm-std 0.5 -a lenet --no-pool --lr 0.01 -b 128 -i 2500 --scratch --seed 0 --log logs/dann/MNIST2USPS\nCUDA_VISIBLE_DEVICES=0 python dann.py data/digits -d Digits -s USPS -t MNIST --train-resizing 'res.' --val-resizing 'res.' \\\n --resize-size 28 --no-hflip --norm-mean 0.5 --norm-std 0.5 -a lenet --no-pool --lr 0.1 -b 128 -i 2500 --scratch --seed 0 --log logs/dann/USPS2MNIST\nCUDA_VISIBLE_DEVICES=0 python dann.py data/digits -d Digits -s SVHNRGB -t MNISTRGB --train-resizing 'res.' --val-resizing 'res.' \\\n --resize-size 32 --no-hflip --norm-mean 0.5 0.5 0.5 --norm-std 0.5 0.5 0.5 -a dtn --no-pool --lr 0.03 -b 128 -i 2500 --scratch --seed 0 --log logs/dann/SVHN2MNIST\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/examples/domain_adaptation/image_classification/gpu_dann.pbs",
"content": "#!/bin/bash\n#PBS -P ZHIHOU\n#PBS -l select=1:ngpus=1:ncpus=6:mem=28GB\n#PBS -l walltime=48:00:00\n##PBS -q alloc-dt\nsource activate pyt\n\n#env\n\ncd \"$PBS_O_WORKDIR\"\n\n#\n## ResNet50, Office31, Single Source\n#CUDA_VISIBLE_DEVICES=0 python dann.py data/office31 -d Office31 -s A -t W -a resnet50 --epochs 20 --seed 1 --log logs/dann/Office31_A2W_g16 --add_bt 16\n#CUDA_VISIBLE_DEVICES=0 python dann.py data/office31 -d Office31 -s D -t W -a resnet50 --epochs 20 --seed 1 --log logs/dann/Office31_D2W_g16 --add_bt 16\n#CUDA_VISIBLE_DEVICES=0 python dann.py data/office31 -d Office31 -s W -t D -a resnet50 --epochs 20 --seed 1 --log logs/dann/Office31_W2D_g16 --add_bt 16\n#CUDA_VISIBLE_DEVICES=0 python dann.py data/office31 -d Office31 -s A -t D -a resnet50 --epochs 20 --seed 1 --log logs/dann/Office31_A2D_g16 --add_bt 16\n#CUDA_VISIBLE_DEVICES=0 python dann.py data/office31 -d Office31 -s D -t A -a resnet50 --epochs 20 --seed 1 --log logs/dann/Office31_D2A_g16 --add_bt 16\n#CUDA_VISIBLE_DEVICES=0 python dann.py data/office31 -d Office31 -s W -t A -a resnet50 --epochs 20 --seed 1 --log logs/dann/Office31_W2A_g16 --add_bt 16\n#\n## ResNet50, Office-Home, Single Source\n#CUDA_VISIBLE_DEVICES=0 python dann.py data/office-home -d OfficeHome -s Ar -t Cl -a resnet50 --epochs 30 --seed 0 --log logs/dann/OfficeHome_Ar2Cl_g16 --add_bt 16\n#CUDA_VISIBLE_DEVICES=0 python dann.py data/office-home -d OfficeHome -s Ar -t Pr -a resnet50 --epochs 30 --seed 0 --log logs/dann/OfficeHome_Ar2Pr_g16 --add_bt 16\n#CUDA_VISIBLE_DEVICES=0 python dann.py data/office-home -d OfficeHome -s Ar -t Rw -a resnet50 --epochs 30 --seed 0 --log logs/dann/OfficeHome_Ar2Rw_g16 --add_bt 16\n#CUDA_VISIBLE_DEVICES=0 python dann.py data/office-home -d OfficeHome -s Cl -t Ar -a resnet50 --epochs 30 --seed 0 --log logs/dann/OfficeHome_Cl2Ar_g16 --add_bt 16\n#CUDA_VISIBLE_DEVICES=0 python dann.py data/office-home -d OfficeHome -s Cl -t Pr -a resnet50 --epochs 30 --seed 0 --log logs/dann/OfficeHome_Cl2Pr_g16 --add_bt 16\n#CUDA_VISIBLE_DEVICES=0 python dann.py data/office-home -d OfficeHome -s Cl -t Rw -a resnet50 --epochs 30 --seed 0 --log logs/dann/OfficeHome_Cl2Rw_g16 --add_bt 16\nCUDA_VISIBLE_DEVICES=0 python dann.py data/office-home -d OfficeHome -s Pr -t Ar -a resnet50 --epochs 30 --seed 0 --log logs/dann/OfficeHome_Pr2Ar_g16 --add_bt 16\nCUDA_VISIBLE_DEVICES=0 python dann.py data/office-home -d OfficeHome -s Pr -t Cl -a resnet50 --epochs 30 --seed 0 --log logs/dann/OfficeHome_Pr2Cl_g16 --add_bt 16\nCUDA_VISIBLE_DEVICES=0 python dann.py data/office-home -d OfficeHome -s Pr -t Rw -a resnet50 --epochs 30 --seed 0 --log logs/dann/OfficeHome_Pr2Rw_g16 --add_bt 16\nCUDA_VISIBLE_DEVICES=0 python dann.py data/office-home -d OfficeHome -s Rw -t Ar -a resnet50 --epochs 30 --seed 0 --log logs/dann/OfficeHome_Rw2Ar_g16 --add_bt 16\nCUDA_VISIBLE_DEVICES=0 python dann.py data/office-home -d OfficeHome -s Rw -t Cl -a resnet50 --epochs 30 --seed 0 --log logs/dann/OfficeHome_Rw2Cl_g16 --add_bt 16\nCUDA_VISIBLE_DEVICES=0 python dann.py data/office-home -d OfficeHome -s Rw -t Pr -a resnet50 --epochs 30 --seed 0 --log logs/dann/OfficeHome_Rw2Pr_g16 --add_bt 16\n##\n### ResNet101, VisDA-2017, Single Source\n#CUDA_VISIBLE_DEVICES=0 python dann.py data/visda-2017 -d VisDA2017 -s Synthetic -t Real -a resnet101 \\\n# --epochs 30 --seed 0 --per-class-eval --train-resizing cen.crop --log logs/dann/VisDA2017_g16 --add_bt 16\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/examples/domain_adaptation/image_classification/gpu_mcc.pbs",
"content": "#!/bin/bash\n#PBS -P ZHIHOU\n#PBS -l select=1:ngpus=1:ncpus=6:mem=28GB\n#PBS -l walltime=48:00:00\n#PBS -q alloc-dt\nsource activate pyt\n\n#env\n\ncd \"$PBS_O_WORKDIR\"\n\n# ResNet50, Office31, Single Source\n#CUDA_VISIBLE_DEVICES=0 python mcc.py data/office31 -d Office31 -s A -t W -a resnet18 --epochs 20 -i 500 --seed 1 --bottleneck-dim 1024 --log logs/mcc/Office31_A2W\n#CUDA_VISIBLE_DEVICES=0 python mcc.py data/office31 -d Office31 -s D -t W -a resnet18 --epochs 20 -i 500 --seed 1 --bottleneck-dim 1024 --log logs/mcc/Office31_D2W\n#CUDA_VISIBLE_DEVICES=0 python mcc.py data/office31 -d Office31 -s W -t D -a resnet18 --epochs 20 -i 500 --seed 1 --bottleneck-dim 1024 --log logs/mcc/Office31_W2D\n#CUDA_VISIBLE_DEVICES=0 python mcc.py data/office31 -d Office31 -s A -t D -a resnet18 --epochs 20 -i 500 --seed 1 --bottleneck-dim 1024 --log logs/mcc/Office31_A2D\n#CUDA_VISIBLE_DEVICES=0 python mcc.py data/office31 -d Office31 -s D -t A -a resnet18 --epochs 20 -i 500 --seed 1 --bottleneck-dim 1024 --log logs/mcc/Office31_D2A\n#CUDA_VISIBLE_DEVICES=0 python mcc.py data/office31 -d Office31 -s W -t A -a resnet18 --epochs 20 -i 500 --seed 1 --bottleneck-dim 1024 --log logs/mcc/Office31_W2A\n\n## ResNet50, Office-Home, Single Source\nCUDA_VISIBLE_DEVICES=0 python mcc.py data/office-home -d OfficeHome -s Ar -t Cl -a resnet18 --epochs 30 --seed 2 --bottleneck-dim 2048 --log logs/mcc_r18/OfficeHome_Ar2Cl\nCUDA_VISIBLE_DEVICES=0 python mcc.py data/office-home -d OfficeHome -s Ar -t Pr -a resnet18 --epochs 30 --seed 2 --bottleneck-dim 2048 --log logs/mcc_r18/OfficeHome_Ar2Pr\nCUDA_VISIBLE_DEVICES=0 python mcc.py data/office-home -d OfficeHome -s Ar -t Rw -a resnet18 --epochs 30 --seed 2 --bottleneck-dim 2048 --log logs/mcc_r18/OfficeHome_Ar2Rw\nCUDA_VISIBLE_DEVICES=0 python mcc.py data/office-home -d OfficeHome -s Cl -t Ar -a resnet18 --epochs 30 --seed 2 --bottleneck-dim 2048 --log logs/mcc_r18/OfficeHome_Cl2Ar\nCUDA_VISIBLE_DEVICES=0 python mcc.py data/office-home -d OfficeHome -s Cl -t Pr -a resnet18 --epochs 30 --seed 2 --bottleneck-dim 2048 --log logs/mcc_r18/OfficeHome_Cl2Pr\nCUDA_VISIBLE_DEVICES=0 python mcc.py data/office-home -d OfficeHome -s Cl -t Rw -a resnet18 --epochs 30 --seed 2 --bottleneck-dim 2048 --log logs/mcc_r18/OfficeHome_Cl2Rw\nCUDA_VISIBLE_DEVICES=0 python mcc.py data/office-home -d OfficeHome -s Pr -t Ar -a resnet18 --epochs 30 --seed 2 --bottleneck-dim 2048 --log logs/mcc_r18/OfficeHome_Pr2Ar\nCUDA_VISIBLE_DEVICES=0 python mcc.py data/office-home -d OfficeHome -s Pr -t Cl -a resnet18 --epochs 30 --seed 2 --bottleneck-dim 2048 --log logs/mcc_r18/OfficeHome_Pr2Cl\nCUDA_VISIBLE_DEVICES=0 python mcc.py data/office-home -d OfficeHome -s Pr -t Rw -a resnet18 --epochs 30 --seed 2 --bottleneck-dim 2048 --log logs/mcc_r18/OfficeHome_Pr2Rw\nCUDA_VISIBLE_DEVICES=0 python mcc.py data/office-home -d OfficeHome -s Rw -t Ar -a resnet18 --epochs 30 --seed 2 --bottleneck-dim 2048 --log logs/mcc_r18/OfficeHome_Rw2Ar\nCUDA_VISIBLE_DEVICES=0 python mcc.py data/office-home -d OfficeHome -s Rw -t Cl -a resnet18 --epochs 30 --seed 2 --bottleneck-dim 2048 --log logs/mcc_r18/OfficeHome_Rw2Cl\nCUDA_VISIBLE_DEVICES=0 python mcc.py data/office-home -d OfficeHome -s Rw -t Pr -a resnet18 --epochs 30 --seed 2 --bottleneck-dim 2048 --log logs/mcc_r18/OfficeHome_Rw2Pr\n\n## ResNet101, VisDA-2017, Single Source\n#CUDA_VISIBLE_DEVICES=5 python mcc.py data/visda-2017 -d VisDA2017 -s Synthetic -t Real -a resnet101 \\\n# --epochs 30 --seed 0 --lr 0.002 --per-class-eval --temperature 3.0 --train-resizing cen.crop --log logs/mcc/VisDA2017\n\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/examples/domain_adaptation/image_classification/gpu_mdd.pbs",
"content": "#!/bin/bash\n#PBS -P ZHIHOU\n#PBS -l select=1:ngpus=1:ncpus=6:mem=28GB\n#PBS -l walltime=48:00:00\n#PBS -q alloc-dt\nsource activate pyt\n\n#env\n\ncd \"$PBS_O_WORKDIR\"\n# ResNet50, Office31, Single Source\n#CUDA_VISIBLE_DEVICES=0 python mdd.py data/office31 -d Office31 -s A -t W -a resnet50 --epochs 20 --bottleneck-dim 1024 --seed 1 --log logs/mdd/Office31_A2W\n#CUDA_VISIBLE_DEVICES=0 python mdd.py data/office31 -d Office31 -s D -t W -a resnet50 --epochs 20 --bottleneck-dim 1024 --seed 1 --log logs/mdd/Office31_D2W\n#CUDA_VISIBLE_DEVICES=0 python mdd.py data/office31 -d Office31 -s W -t D -a resnet50 --epochs 20 --bottleneck-dim 1024 --seed 1 --log logs/mdd/Office31_W2D\n#CUDA_VISIBLE_DEVICES=0 python mdd.py data/office31 -d Office31 -s A -t D -a resnet50 --epochs 20 --bottleneck-dim 1024 --seed 1 --log logs/mdd/Office31_A2D\n#CUDA_VISIBLE_DEVICES=0 python mdd.py data/office31 -d Office31 -s D -t A -a resnet50 --epochs 20 --bottleneck-dim 1024 --seed 1 --log logs/mdd/Office31_D2A\n#CUDA_VISIBLE_DEVICES=0 python mdd.py data/office31 -d Office31 -s W -t A -a resnet50 --epochs 20 --bottleneck-dim 1024 --seed 1 --log logs/mdd/Office31_W2A\n#\n## ResNet50, Office-Home, Single Source\n#CUDA_VISIBLE_DEVICES=0 python mdd.py data/office-home -d OfficeHome -s Ar -t Cl -a resnet50 --epochs 30 --bottleneck-dim 2048 --seed 0 --log logs/mdd/OfficeHome_Ar2Cl_g16 --add_bt 16\n#CUDA_VISIBLE_DEVICES=0 python mdd.py data/office-home -d OfficeHome -s Ar -t Pr -a resnet50 --epochs 30 --bottleneck-dim 2048 --seed 0 --log logs/mdd/OfficeHome_Ar2Pr_g16 --add_bt 16\n#CUDA_VISIBLE_DEVICES=0 python mdd.py data/office-home -d OfficeHome -s Ar -t Rw -a resnet50 --epochs 30 --bottleneck-dim 2048 --seed 0 --log logs/mdd/OfficeHome_Ar2Rw_g16 --add_bt 16\n#CUDA_VISIBLE_DEVICES=0 python mdd.py data/office-home -d OfficeHome -s Cl -t Ar -a resnet50 --epochs 30 --bottleneck-dim 2048 --seed 0 --log logs/mdd/OfficeHome_Cl2Ar_g16 --add_bt 16\n#CUDA_VISIBLE_DEVICES=0 python mdd.py data/office-home -d OfficeHome -s Cl -t Pr -a resnet50 --epochs 30 --bottleneck-dim 2048 --seed 0 --log logs/mdd/OfficeHome_Cl2Pr_g16 --add_bt 16\n#CUDA_VISIBLE_DEVICES=0 python mdd.py data/office-home -d OfficeHome -s Cl -t Rw -a resnet50 --epochs 30 --bottleneck-dim 2048 --seed 0 --log logs/mdd/OfficeHome_Cl2Rw_g16 --add_bt 16\n#CUDA_VISIBLE_DEVICES=0 python mdd.py data/office-home -d OfficeHome -s Pr -t Ar -a resnet50 --epochs 30 --bottleneck-dim 2048 --seed 0 --log logs/mdd/OfficeHome_Pr2Ar_g16 --add_bt 16\n#CUDA_VISIBLE_DEVICES=0 python mdd.py data/office-home -d OfficeHome -s Pr -t Cl -a resnet50 --epochs 30 --bottleneck-dim 2048 --seed 0 --log logs/mdd/OfficeHome_Pr2Cl_g16 --add_bt 16\n#CUDA_VISIBLE_DEVICES=0 python mdd.py data/office-home -d OfficeHome -s Pr -t Rw -a resnet50 --epochs 30 --bottleneck-dim 2048 --seed 0 --log logs/mdd/OfficeHome_Pr2Rw_g16 --add_bt 16\n#CUDA_VISIBLE_DEVICES=0 python mdd.py data/office-home -d OfficeHome -s Rw -t Ar -a resnet50 --epochs 30 --bottleneck-dim 2048 --seed 0 --log logs/mdd/OfficeHome_Rw2Ar_g16 --add_bt 16\n#CUDA_VISIBLE_DEVICES=0 python mdd.py data/office-home -d OfficeHome -s Rw -t Cl -a resnet50 --epochs 30 --bottleneck-dim 2048 --seed 0 --log logs/mdd/OfficeHome_Rw2Cl_g16 --add_bt 16\n#CUDA_VISIBLE_DEVICES=0 python mdd.py data/office-home -d OfficeHome -s Rw -t Pr -a resnet50 --epochs 30 --bottleneck-dim 2048 --seed 0 --log logs/mdd/OfficeHome_Rw2Pr_g16 --add_bt 16\n#\n## ResNet101, VisDA-2017, Single Source\nCUDA_VISIBLE_DEVICES=0 python mdd.py data/visda-2017 -d VisDA2017 -s Synthetic -t Real -a resnet101 --epochs 30 \\\n --bottleneck-dim 1024 --seed 0 --train-resizing cen.crop --per-class-eval -b 36 --log logs/mdd/VisDA2017\n\nCUDA_VISIBLE_DEVICES=0 python mdd.py data/visda-2017 -d VisDA2017 -s Synthetic -t Real -a resnet101 --epochs 30 \\\n --bottleneck-dim 1024 --seed 0 --train-resizing cen.crop --per-class-eval -b 36 --log logs/mdd/VisDA2017_g16 --add_bt 16\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/examples/domain_adaptation/image_classification/jan.py",
"content": "\"\"\"\n@author: Junguang Jiang\n@contact: JiangJunguang1123@outlook.com\n\"\"\"\nimport random\nimport time\nimport warnings\nimport sys\nimport argparse\nimport shutil\nimport os.path as osp\n\nimport torch\nimport torch.nn as nn\nimport torch.backends.cudnn as cudnn\nfrom torch.optim import SGD\nfrom torch.optim.lr_scheduler import LambdaLR\nfrom torch.utils.data import DataLoader\nimport torch.nn.functional as F\n\nsys.path.append('../../..')\nfrom dalib.adaptation.jan import JointMultipleKernelMaximumMeanDiscrepancy, ImageClassifier, Theta\nfrom dalib.modules.kernels import GaussianKernel\nfrom common.utils.data import ForeverDataIterator\nfrom common.utils.metric import accuracy\nfrom common.utils.meter import AverageMeter, ProgressMeter\nfrom common.utils.logger import CompleteLogger\nfrom common.utils.analysis import collect_feature, tsne, a_distance\n\nsys.path.append('.')\nimport utils\n\ndevice = torch.device(\"cuda\" if torch.cuda.is_available() else \"cpu\")\n\n\ndef main(args: argparse.Namespace):\n logger = CompleteLogger(args.log, args.phase)\n print(args)\n\n if args.seed is not None:\n random.seed(args.seed)\n torch.manual_seed(args.seed)\n cudnn.deterministic = True\n warnings.warn('You have chosen to seed training. '\n 'This will turn on the CUDNN deterministic setting, '\n 'which can slow down your training considerably! '\n 'You may see unexpected behavior when restarting '\n 'from checkpoints.')\n\n cudnn.benchmark = True\n\n # Data loading code\n train_transform = utils.get_train_transform(args.train_resizing, random_horizontal_flip=not args.no_hflip,\n random_color_jitter=False, resize_size=args.resize_size,\n norm_mean=args.norm_mean, norm_std=args.norm_std)\n val_transform = utils.get_val_transform(args.val_resizing, resize_size=args.resize_size,\n norm_mean=args.norm_mean, norm_std=args.norm_std)\n print(\"train_transform: \", train_transform)\n print(\"val_transform: \", val_transform)\n\n train_source_dataset, train_target_dataset, val_dataset, test_dataset, num_classes, args.class_names = \\\n utils.get_dataset(args.data, args.root, args.source, args.target, train_transform, val_transform)\n train_source_loader = DataLoader(train_source_dataset, batch_size=args.batch_size,\n shuffle=True, num_workers=args.workers, drop_last=True)\n train_target_loader = DataLoader(train_target_dataset, batch_size=args.batch_size,\n shuffle=True, num_workers=args.workers, drop_last=True)\n val_loader = DataLoader(val_dataset, batch_size=args.batch_size, shuffle=False, num_workers=args.workers)\n test_loader = DataLoader(test_dataset, batch_size=args.batch_size, shuffle=False, num_workers=args.workers)\n\n train_source_iter = ForeverDataIterator(train_source_loader)\n train_target_iter = ForeverDataIterator(train_target_loader)\n\n # create model\n print(\"=> using model '{}'\".format(args.arch))\n backbone = utils.get_model(args.arch, pretrain=not args.scratch)\n pool_layer = nn.Identity() if args.no_pool else None\n classifier = ImageClassifier(backbone, num_classes, bottleneck_dim=args.bottleneck_dim,\n pool_layer=pool_layer, finetune=not args.scratch).to(device)\n\n # define loss function\n if args.adversarial:\n thetas = [Theta(dim).to(device) for dim in (classifier.features_dim, num_classes)]\n else:\n thetas = None\n jmmd_loss = JointMultipleKernelMaximumMeanDiscrepancy(\n kernels=(\n [GaussianKernel(alpha=2 ** k) for k in range(-3, 2)],\n (GaussianKernel(sigma=0.92, track_running_stats=False),)\n ),\n linear=args.linear, thetas=thetas\n ).to(device)\n\n parameters = classifier.get_parameters()\n if thetas is not None:\n parameters += [{\"params\": theta.parameters(), 'lr': 0.1} for theta in thetas]\n\n # define optimizer\n optimizer = SGD(parameters, args.lr, momentum=args.momentum, weight_decay=args.wd, nesterov=True)\n lr_scheduler = LambdaLR(optimizer, lambda x: args.lr * (1. + args.lr_gamma * float(x)) ** (-args.lr_decay))\n\n # resume from the best checkpoint\n if args.phase != 'train':\n checkpoint = torch.load(logger.get_checkpoint_path('best'), map_location='cpu')\n classifier.load_state_dict(checkpoint)\n\n # analysis the model\n if args.phase == 'analysis':\n # extract features from both domains\n feature_extractor = nn.Sequential(classifier.backbone, classifier.pool_layer, classifier.bottleneck).to(device)\n source_feature = collect_feature(train_source_loader, feature_extractor, device)\n target_feature = collect_feature(train_target_loader, feature_extractor, device)\n # plot t-SNE\n tSNE_filename = osp.join(logger.visualize_directory, 'TSNE.pdf')\n tsne.visualize(source_feature, target_feature, tSNE_filename)\n print(\"Saving t-SNE to\", tSNE_filename)\n # calculate A-distance, which is a measure for distribution discrepancy\n A_distance = a_distance.calculate(source_feature, target_feature, device)\n print(\"A-distance =\", A_distance)\n return\n\n if args.phase == 'test':\n acc1 = utils.validate(test_loader, classifier, args, device)\n print(acc1)\n return\n\n # start training\n best_acc1 = 0.\n for epoch in range(args.epochs):\n # train for one epoch\n train(train_source_iter, train_target_iter, classifier, jmmd_loss, optimizer,\n lr_scheduler, epoch, args)\n\n # evaluate on validation set\n acc1 = utils.validate(val_loader, classifier, args, device)\n\n # remember best acc@1 and save checkpoint\n torch.save(classifier.state_dict(), logger.get_checkpoint_path('latest'))\n if acc1 > best_acc1:\n shutil.copy(logger.get_checkpoint_path('latest'), logger.get_checkpoint_path('best'))\n best_acc1 = max(acc1, best_acc1)\n\n print(\"best_acc1 = {:3.1f}\".format(best_acc1))\n\n # evaluate on test set\n classifier.load_state_dict(torch.load(logger.get_checkpoint_path('best')))\n acc1 = utils.validate(test_loader, classifier, args, device)\n print(\"test_acc1 = {:3.1f}\".format(acc1))\n\n logger.close()\n\n\ndef train(train_source_iter: ForeverDataIterator, train_target_iter: ForeverDataIterator, model: ImageClassifier,\n jmmd_loss: JointMultipleKernelMaximumMeanDiscrepancy, optimizer: SGD,\n lr_scheduler: LambdaLR, epoch: int, args: argparse.Namespace):\n batch_time = AverageMeter('Time', ':4.2f')\n data_time = AverageMeter('Data', ':3.1f')\n losses = AverageMeter('Loss', ':3.2f')\n trans_losses = AverageMeter('Trans Loss', ':5.4f')\n cls_accs = AverageMeter('Cls Acc', ':3.1f')\n tgt_accs = AverageMeter('Tgt Acc', ':3.1f')\n\n progress = ProgressMeter(\n args.iters_per_epoch,\n [batch_time, data_time, losses, trans_losses, cls_accs, tgt_accs],\n prefix=\"Epoch: [{}]\".format(epoch))\n\n # switch to train mode\n model.train()\n jmmd_loss.train()\n\n end = time.time()\n for i in range(args.iters_per_epoch):\n x_s, labels_s = next(train_source_iter)\n x_t, labels_t = next(train_target_iter)\n\n x_s = x_s.to(device)\n x_t = x_t.to(device)\n labels_s = labels_s.to(device)\n labels_t = labels_t.to(device)\n\n # measure data loading time\n data_time.update(time.time() - end)\n\n # compute output\n x = torch.cat((x_s, x_t), dim=0)\n y, f = model(x)\n y_s, y_t = y.chunk(2, dim=0)\n f_s, f_t = f.chunk(2, dim=0)\n\n cls_loss = F.cross_entropy(y_s, labels_s)\n transfer_loss = jmmd_loss(\n (f_s, F.softmax(y_s, dim=1)),\n (f_t, F.softmax(y_t, dim=1))\n )\n loss = cls_loss + transfer_loss * args.trade_off\n\n cls_acc = accuracy(y_s, labels_s)[0]\n tgt_acc = accuracy(y_t, labels_t)[0]\n\n losses.update(loss.item(), x_s.size(0))\n cls_accs.update(cls_acc.item(), x_s.size(0))\n tgt_accs.update(tgt_acc.item(), x_t.size(0))\n trans_losses.update(transfer_loss.item(), x_s.size(0))\n\n # compute gradient and do SGD step\n optimizer.zero_grad()\n loss.backward()\n optimizer.step()\n lr_scheduler.step()\n\n # measure elapsed time\n batch_time.update(time.time() - end)\n end = time.time()\n\n if i % args.print_freq == 0:\n progress.display(i)\n\n\nif __name__ == '__main__':\n parser = argparse.ArgumentParser(description='JAN for Unsupervised Domain Adaptation')\n # dataset parameters\n parser.add_argument('root', metavar='DIR',\n help='root path of dataset')\n parser.add_argument('-d', '--data', metavar='DATA', default='Office31', choices=utils.get_dataset_names(),\n help='dataset: ' + ' | '.join(utils.get_dataset_names()) +\n ' (default: Office31)')\n parser.add_argument('-s', '--source', help='source domain(s)', nargs='+')\n parser.add_argument('-t', '--target', help='target domain(s)', nargs='+')\n parser.add_argument('--train-resizing', type=str, default='default')\n parser.add_argument('--val-resizing', type=str, default='default')\n parser.add_argument('--resize-size', type=int, default=224,\n help='the image size after resizing')\n parser.add_argument('--no-hflip', action='store_true',\n help='no random horizontal flipping during training')\n parser.add_argument('--norm-mean', type=float, nargs='+',\n default=(0.485, 0.456, 0.406), help='normalization mean')\n parser.add_argument('--norm-std', type=float, nargs='+',\n default=(0.229, 0.224, 0.225), help='normalization std')\n # model parameters\n parser.add_argument('-a', '--arch', metavar='ARCH', default='resnet18',\n choices=utils.get_model_names(),\n help='backbone architecture: ' +\n ' | '.join(utils.get_model_names()) +\n ' (default: resnet18)')\n parser.add_argument('--bottleneck-dim', default=256, type=int,\n help='Dimension of bottleneck')\n parser.add_argument('--no-pool', action='store_true',\n help='no pool layer after the feature extractor.')\n parser.add_argument('--scratch', action='store_true', help='whether train from scratch.')\n parser.add_argument('--linear', default=False, action='store_true',\n help='whether use the linear version')\n parser.add_argument('--adversarial', default=False, action='store_true',\n help='whether use adversarial theta')\n parser.add_argument('--trade-off', default=1., type=float,\n help='the trade-off hyper-parameter for transfer loss')\n # training parameters\n parser.add_argument('-b', '--batch-size', default=32, type=int,\n metavar='N',\n help='mini-batch size (default: 32)')\n parser.add_argument('--lr', '--learning-rate', default=0.003, type=float,\n metavar='LR', help='initial learning rate', dest='lr')\n parser.add_argument('--lr-gamma', default=0.0003, type=float, help='parameter for lr scheduler')\n parser.add_argument('--lr-decay', default=0.75, type=float, help='parameter for lr scheduler')\n parser.add_argument('--momentum', default=0.9, type=float, metavar='M',\n help='momentum')\n parser.add_argument('--wd', '--weight-decay', default=0.0005, type=float,\n metavar='W', help='weight decay (default: 5e-4)')\n parser.add_argument('-j', '--workers', default=2, type=int, metavar='N',\n help='number of data loading workers (default: 2)')\n parser.add_argument('--epochs', default=20, type=int, metavar='N',\n help='number of total epochs to run')\n parser.add_argument('-i', '--iters-per-epoch', default=500, type=int,\n help='Number of iterations per epoch')\n parser.add_argument('-p', '--print-freq', default=100, type=int,\n metavar='N', help='print frequency (default: 100)')\n parser.add_argument('--seed', default=None, type=int,\n help='seed for initializing training. ')\n parser.add_argument('--per-class-eval', action='store_true',\n help='whether output per-class accuracy during evaluation')\n parser.add_argument(\"--log\", type=str, default='jan',\n help=\"Where to save logs, checkpoints and debugging images.\")\n parser.add_argument(\"--phase\", type=str, default='train', choices=['train', 'test', 'analysis'],\n help=\"When phase is 'test', only test the model.\"\n \"When phase is 'analysis', only analysis the model.\")\n args = parser.parse_args()\n main(args)\n\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/examples/domain_adaptation/image_classification/jan.sh",
"content": "#!/usr/bin/env bash\n# ResNet50, Office31, Single Source\nCUDA_VISIBLE_DEVICES=0 python jan.py data/office31 -d Office31 -s D -t A -a resnet50 --epochs 20 --seed 0 --log logs/jan/Office31_D2A\nCUDA_VISIBLE_DEVICES=0 python jan.py data/office31 -d Office31 -s W -t A -a resnet50 --epochs 20 --seed 0 --log logs/jan/Office31_W2A\nCUDA_VISIBLE_DEVICES=0 python jan.py data/office31 -d Office31 -s A -t W -a resnet50 --epochs 20 --seed 0 --log logs/jan/Office31_A2W\nCUDA_VISIBLE_DEVICES=0 python jan.py data/office31 -d Office31 -s A -t D -a resnet50 --epochs 20 --seed 0 --log logs/jan/Office31_A2D\nCUDA_VISIBLE_DEVICES=0 python jan.py data/office31 -d Office31 -s D -t W -a resnet50 --epochs 20 --seed 0 --log logs/jan/Office31_D2W\nCUDA_VISIBLE_DEVICES=0 python jan.py data/office31 -d Office31 -s W -t D -a resnet50 --epochs 20 --seed 0 --log logs/jan/Office31_W2D\n\n# ResNet50, Office-Home, Single Source\nCUDA_VISIBLE_DEVICES=0 python jan.py data/office-home -d OfficeHome -s Ar -t Cl -a resnet50 --epochs 20 --seed 0 --log logs/jan/OfficeHome_Ar2Cl\nCUDA_VISIBLE_DEVICES=0 python jan.py data/office-home -d OfficeHome -s Ar -t Pr -a resnet50 --epochs 20 --seed 0 --log logs/jan/OfficeHome_Ar2Pr\nCUDA_VISIBLE_DEVICES=0 python jan.py data/office-home -d OfficeHome -s Ar -t Rw -a resnet50 --epochs 20 --seed 0 --log logs/jan/OfficeHome_Ar2Rw\nCUDA_VISIBLE_DEVICES=0 python jan.py data/office-home -d OfficeHome -s Cl -t Ar -a resnet50 --epochs 20 --seed 0 --log logs/jan/OfficeHome_Cl2Ar\nCUDA_VISIBLE_DEVICES=0 python jan.py data/office-home -d OfficeHome -s Cl -t Pr -a resnet50 --epochs 20 --seed 0 --log logs/jan/OfficeHome_Cl2Pr\nCUDA_VISIBLE_DEVICES=0 python jan.py data/office-home -d OfficeHome -s Cl -t Rw -a resnet50 --epochs 20 --seed 0 --log logs/jan/OfficeHome_Cl2Rw\nCUDA_VISIBLE_DEVICES=0 python jan.py data/office-home -d OfficeHome -s Pr -t Ar -a resnet50 --epochs 20 --seed 0 --log logs/jan/OfficeHome_Pr2Ar\nCUDA_VISIBLE_DEVICES=0 python jan.py data/office-home -d OfficeHome -s Pr -t Cl -a resnet50 --epochs 20 --seed 0 --log logs/jan/OfficeHome_Pr2Cl\nCUDA_VISIBLE_DEVICES=0 python jan.py data/office-home -d OfficeHome -s Pr -t Rw -a resnet50 --epochs 20 --seed 0 --log logs/jan/OfficeHome_Pr2Rw\nCUDA_VISIBLE_DEVICES=0 python jan.py data/office-home -d OfficeHome -s Rw -t Ar -a resnet50 --epochs 20 --seed 0 --log logs/jan/OfficeHome_Rw2Ar\nCUDA_VISIBLE_DEVICES=0 python jan.py data/office-home -d OfficeHome -s Rw -t Cl -a resnet50 --epochs 20 --seed 0 --log logs/jan/OfficeHome_Rw2Cl\nCUDA_VISIBLE_DEVICES=0 python jan.py data/office-home -d OfficeHome -s Rw -t Pr -a resnet50 --epochs 20 --seed 0 --log logs/jan/OfficeHome_Rw2Pr\n\n# ResNet101, VisDA-2017, Single Source\nCUDA_VISIBLE_DEVICES=0 python jan.py data/visda-2017 -d VisDA2017 -s Synthetic -t Real -a resnet101 \\\n --epochs 20 -i 500 --seed 0 --per-class-eval --train-resizing cen.crop --log logs/jan/VisDA2017\n\n# ResNet101, DomainNet, Single Source\nCUDA_VISIBLE_DEVICES=0 python jan.py data/domainnet -d DomainNet -s c -t p -a resnet101 --bottleneck-dim 1024 --epochs 30 -i 2500 -p 500 --seed 0 --log logs/jan/DomainNet_c2p\nCUDA_VISIBLE_DEVICES=0 python jan.py data/domainnet -d DomainNet -s c -t r -a resnet101 --bottleneck-dim 1024 --epochs 30 -i 2500 -p 500 --seed 0 --log logs/jan/DomainNet_c2r\nCUDA_VISIBLE_DEVICES=0 python jan.py data/domainnet -d DomainNet -s c -t s -a resnet101 --bottleneck-dim 1024 --epochs 30 -i 2500 -p 500 --seed 0 --log logs/jan/DomainNet_c2s\nCUDA_VISIBLE_DEVICES=0 python jan.py data/domainnet -d DomainNet -s p -t c -a resnet101 --bottleneck-dim 1024 --epochs 30 -i 2500 -p 500 --seed 0 --log logs/jan/DomainNet_p2c\nCUDA_VISIBLE_DEVICES=0 python jan.py data/domainnet -d DomainNet -s p -t r -a resnet101 --bottleneck-dim 1024 --epochs 30 -i 2500 -p 500 --seed 0 --log logs/jan/DomainNet_p2r\nCUDA_VISIBLE_DEVICES=0 python jan.py data/domainnet -d DomainNet -s p -t s -a resnet101 --bottleneck-dim 1024 --epochs 30 -i 2500 -p 500 --seed 0 --log logs/jan/DomainNet_p2s\nCUDA_VISIBLE_DEVICES=0 python jan.py data/domainnet -d DomainNet -s r -t c -a resnet101 --bottleneck-dim 1024 --epochs 30 -i 2500 -p 500 --seed 0 --log logs/jan/DomainNet_r2c\nCUDA_VISIBLE_DEVICES=0 python jan.py data/domainnet -d DomainNet -s r -t p -a resnet101 --bottleneck-dim 1024 --epochs 30 -i 2500 -p 500 --seed 0 --log logs/jan/DomainNet_r2p\nCUDA_VISIBLE_DEVICES=0 python jan.py data/domainnet -d DomainNet -s r -t s -a resnet101 --bottleneck-dim 1024 --epochs 30 -i 2500 -p 500 --seed 0 --log logs/jan/DomainNet_r2s\nCUDA_VISIBLE_DEVICES=0 python jan.py data/domainnet -d DomainNet -s s -t c -a resnet101 --bottleneck-dim 1024 --epochs 30 -i 2500 -p 500 --seed 0 --log logs/jan/DomainNet_s2c\nCUDA_VISIBLE_DEVICES=0 python jan.py data/domainnet -d DomainNet -s s -t p -a resnet101 --bottleneck-dim 1024 --epochs 30 -i 2500 -p 500 --seed 0 --log logs/jan/DomainNet_s2p\nCUDA_VISIBLE_DEVICES=0 python jan.py data/domainnet -d DomainNet -s s -t r -a resnet101 --bottleneck-dim 1024 --epochs 30 -i 2500 -p 500 --seed 0 --log logs/jan/DomainNet_s2r\n\n# ResNet50, ImageNet200 -> ImageNetR\nCUDA_VISIBLE_DEVICES=0 python jan.py data/ImageNetR -d ImageNetR -s IN -t INR -a resnet50 --epochs 30 -i 2500 -p 500 --seed 0 --log logs/jan/ImageNet_IN2INR\n\n# ig_resnext101_32x8d, ImageNet -> ImageNetSketch\nCUDA_VISIBLE_DEVICES=0 python jan.py data/imagenet-sketch -d ImageNetSketch -s IN -t sketch -a ig_resnext101_32x8d --epochs 30 -i 2500 -p 500 --log logs/jan_ig_resnext101_32x8d/ImageNet_IN2sketch\n\n# Vision Transformer, Office-Home, Single Source\nCUDA_VISIBLE_DEVICES=0 python jan.py data/office-home -d OfficeHome -s Ar -t Cl -a vit_base_patch16_224 --no-pool --epochs 30 --seed 0 -b 24 --log logs/jan_vit/OfficeHome_Ar2Cl\nCUDA_VISIBLE_DEVICES=0 python jan.py data/office-home -d OfficeHome -s Ar -t Pr -a vit_base_patch16_224 --no-pool --epochs 30 --seed 0 -b 24 --log logs/jan_vit/OfficeHome_Ar2Pr\nCUDA_VISIBLE_DEVICES=0 python jan.py data/office-home -d OfficeHome -s Ar -t Rw -a vit_base_patch16_224 --no-pool --epochs 30 --seed 0 -b 24 --log logs/jan_vit/OfficeHome_Ar2Rw\nCUDA_VISIBLE_DEVICES=0 python jan.py data/office-home -d OfficeHome -s Cl -t Ar -a vit_base_patch16_224 --no-pool --epochs 30 --seed 0 -b 24 --log logs/jan_vit/OfficeHome_Cl2Ar\nCUDA_VISIBLE_DEVICES=0 python jan.py data/office-home -d OfficeHome -s Cl -t Pr -a vit_base_patch16_224 --no-pool --epochs 30 --seed 0 -b 24 --log logs/jan_vit/OfficeHome_Cl2Pr\nCUDA_VISIBLE_DEVICES=0 python jan.py data/office-home -d OfficeHome -s Cl -t Rw -a vit_base_patch16_224 --no-pool --epochs 30 --seed 0 -b 24 --log logs/jan_vit/OfficeHome_Cl2Rw\nCUDA_VISIBLE_DEVICES=0 python jan.py data/office-home -d OfficeHome -s Pr -t Ar -a vit_base_patch16_224 --no-pool --epochs 30 --seed 0 -b 24 --log logs/jan_vit/OfficeHome_Pr2Ar\nCUDA_VISIBLE_DEVICES=0 python jan.py data/office-home -d OfficeHome -s Pr -t Cl -a vit_base_patch16_224 --no-pool --epochs 30 --seed 0 -b 24 --log logs/jan_vit/OfficeHome_Pr2Cl\nCUDA_VISIBLE_DEVICES=0 python jan.py data/office-home -d OfficeHome -s Pr -t Rw -a vit_base_patch16_224 --no-pool --epochs 30 --seed 0 -b 24 --log logs/jan_vit/OfficeHome_Pr2Rw\nCUDA_VISIBLE_DEVICES=0 python jan.py data/office-home -d OfficeHome -s Rw -t Ar -a vit_base_patch16_224 --no-pool --epochs 30 --seed 0 -b 24 --log logs/jan_vit/OfficeHome_Rw2Ar\nCUDA_VISIBLE_DEVICES=0 python jan.py data/office-home -d OfficeHome -s Rw -t Cl -a vit_base_patch16_224 --no-pool --epochs 30 --seed 0 -b 24 --log logs/jan_vit/OfficeHome_Rw2Cl\nCUDA_VISIBLE_DEVICES=0 python jan.py data/office-home -d OfficeHome -s Rw -t Pr -a vit_base_patch16_224 --no-pool --epochs 30 --seed 0 -b 24 --log logs/jan_vit/OfficeHome_Rw2Pr\n\n# ResNet50, Office-Home, Multi Source\nCUDA_VISIBLE_DEVICES=0 python jan.py data/office-home -d OfficeHome -s Cl Pr Rw -t Ar -a resnet50 --epochs 30 --seed 0 --log logs/jan/OfficeHome_:2Ar\nCUDA_VISIBLE_DEVICES=0 python jan.py data/office-home -d OfficeHome -s Ar Pr Rw -t Cl -a resnet50 --epochs 30 --seed 0 --log logs/jan/OfficeHome_:2Cl\nCUDA_VISIBLE_DEVICES=0 python jan.py data/office-home -d OfficeHome -s Ar Cl Rw -t Pr -a resnet50 --epochs 30 --seed 0 --log logs/jan/OfficeHome_:2Pr\nCUDA_VISIBLE_DEVICES=0 python jan.py data/office-home -d OfficeHome -s Ar Cl Pr -t Rw -a resnet50 --epochs 30 --seed 0 --log logs/jan/OfficeHome_:2Rw\n\n# ResNet101, DomainNet, Multi Source\nCUDA_VISIBLE_DEVICES=0 python jan.py data/domainnet -d DomainNet -s i p q r s -t c -a resnet101 --bottleneck-dim 1024 --epochs 40 -i 5000 -p 500 --seed 0 --log logs/jan/DomainNet_:2c\nCUDA_VISIBLE_DEVICES=0 python jan.py data/domainnet -d DomainNet -s c p q r s -t i -a resnet101 --bottleneck-dim 1024 --epochs 40 -i 5000 -p 500 --seed 0 --log logs/jan/DomainNet_:2i\nCUDA_VISIBLE_DEVICES=0 python jan.py data/domainnet -d DomainNet -s c i q r s -t p -a resnet101 --bottleneck-dim 1024 --epochs 40 -i 5000 -p 500 --seed 0 --log logs/jan/DomainNet_:2p\nCUDA_VISIBLE_DEVICES=0 python jan.py data/domainnet -d DomainNet -s c i p r s -t q -a resnet101 --bottleneck-dim 1024 --epochs 40 -i 5000 -p 500 --seed 0 --log logs/jan/DomainNet_:2q\nCUDA_VISIBLE_DEVICES=0 python jan.py data/domainnet -d DomainNet -s c i p q s -t r -a resnet101 --bottleneck-dim 1024 --epochs 40 -i 5000 -p 500 --seed 0 --log logs/jan/DomainNet_:2r\nCUDA_VISIBLE_DEVICES=0 python jan.py data/domainnet -d DomainNet -s c i p q r -t s -a resnet101 --bottleneck-dim 1024 --epochs 40 -i 5000 -p 500 --seed 0 --log logs/jan/DomainNet_:2s\n\n# ResNet50, Wilds Dataset\nCUDA_VISIBLE_DEVICES=0 python jan.py data/wilds -d iwildcam -a resnet50 --epochs 30 -i 1000 --seed 0 --log logs/jan/iwildcam\n\n# DenseNet121, Wilds Dataset\nCUDA_VISIBLE_DEVICES=0 python jan.py data/wilds -d camelyon17 --train-resizing 'res.' --val-resizing 'res.' --resize-size 96 \\\n -a densenet121 --scratch --epochs 10 -i 1000 --lr 0.01 --seed 0 --log logs/jan/camelyon17\nCUDA_VISIBLE_DEVICES=0 python jan.py data/wilds -d fmow --train-resizing 'res.' --val-resizing 'res.' \\\n -a densenet121 --epochs 10 -i 1000 --lr 0.01 --seed 0 --log logs/jan/fmow\n\n# Digits\nCUDA_VISIBLE_DEVICES=0 python jan.py data/digits -d Digits -s MNIST -t USPS --train-resizing 'res.' --val-resizing 'res.' \\\n --resize-size 28 --no-hflip --norm-mean 0.5 --norm-std 0.5 -a lenet --no-pool --lr 0.01 -b 128 -i 2500 --scratch --seed 0 --log logs/jan/MNIST2USPS\nCUDA_VISIBLE_DEVICES=0 python jan.py data/digits -d Digits -s USPS -t MNIST --train-resizing 'res.' --val-resizing 'res.' \\\n --resize-size 28 --no-hflip --norm-mean 0.5 --norm-std 0.5 -a lenet --no-pool --lr 0.1 -b 128 -i 2500 --scratch --seed 0 --log logs/jan/USPS2MNIST\nCUDA_VISIBLE_DEVICES=4 python jan.py data/digits -d Digits -s SVHNRGB -t MNISTRGB --train-resizing 'res.' --val-resizing 'res.' \\\n --resize-size 32 --no-hflip --norm-mean 0.5 0.5 0.5 --norm-std 0.5 0.5 0.5 -a dtn --no-pool --lr 0.03 -b 128 -i 2500 --scratch --seed 0 --log logs/jan/SVHN2MNIST\n\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/examples/domain_adaptation/image_classification/mcc.py",
"content": "\"\"\"\n@author: Ying Jin\n@contact: sherryying003@gmail.com\n\"\"\"\nimport random\nimport time\nimport warnings\nimport sys\nimport argparse\nimport shutil\nimport os.path as osp\n\nimport torch\nimport torch.nn as nn\nimport torch.backends.cudnn as cudnn\nfrom torch.optim import SGD\nfrom torch.optim.lr_scheduler import LambdaLR\nfrom torch.utils.data import DataLoader\nimport torch.nn.functional as F\n\nsys.path.append('../../..')\nfrom dalib.adaptation.mcc import MinimumClassConfusionLoss, ImageClassifier\nfrom common.utils.data import ForeverDataIterator\nfrom common.utils.metric import accuracy\nfrom common.utils.meter import AverageMeter, ProgressMeter\nfrom common.utils.logger import CompleteLogger\nfrom common.utils.analysis import collect_feature, tsne, a_distance\n\nsys.path.append('.')\nimport utils\n\ndevice = torch.device(\"cuda\" if torch.cuda.is_available() else \"cpu\")\n\nclass TransformerDecorator(torch.nn.Module):\n def __init__(self, backbone=None, add_bt=3, dim=2048, eval_global=0):\n super(TransformerDecorator, self).__init__()\n self.encoder_layers = torch.nn.TransformerEncoderLayer(dim, 4, dim, 0.5)\n self.eval_global = eval_global\n self.add_bt = add_bt\n self.backbone = backbone\n\n def forward_feats(self, feature):\n feature = self.backbone(feature)\n if self.training and self.add_bt > 0:\n pre_feature = feature\n feature = feature.unsqueeze(1)\n feature = self.encoder_layers(feature)\n # feature = self.encoder_layers(feature, feature, feature)\n feature = feature.squeeze(1)\n if self.add_bt >= 1:\n return torch.cat([pre_feature, feature], dim=0)\n return feature\n\n def forward(self, feature):\n feature = self.forward_feats(feature)\n return feature\n\n\ndef main(args: argparse.Namespace):\n logger = CompleteLogger(args.log, args.phase)\n print(args)\n\n if args.seed is not None:\n random.seed(args.seed)\n torch.manual_seed(args.seed)\n cudnn.deterministic = True\n warnings.warn('You have chosen to seed training. '\n 'This will turn on the CUDNN deterministic setting, '\n 'which can slow down your training considerably! '\n 'You may see unexpected behavior when restarting '\n 'from checkpoints.')\n\n cudnn.benchmark = True\n\n # Data loading code\n train_transform = utils.get_train_transform(args.train_resizing, random_horizontal_flip=not args.no_hflip,\n random_color_jitter=False, resize_size=args.resize_size,\n norm_mean=args.norm_mean, norm_std=args.norm_std)\n val_transform = utils.get_val_transform(args.val_resizing, resize_size=args.resize_size,\n norm_mean=args.norm_mean, norm_std=args.norm_std)\n print(\"train_transform: \", train_transform)\n print(\"val_transform: \", val_transform)\n\n train_source_dataset, train_target_dataset, val_dataset, test_dataset, num_classes, args.class_names = \\\n utils.get_dataset(args.data, args.root, args.source, args.target, train_transform, val_transform)\n train_source_loader = DataLoader(train_source_dataset, batch_size=args.batch_size,\n shuffle=True, num_workers=args.workers, drop_last=True)\n train_target_loader = DataLoader(train_target_dataset, batch_size=args.batch_size,\n shuffle=True, num_workers=args.workers, drop_last=True)\n val_loader = DataLoader(val_dataset, batch_size=args.batch_size, shuffle=False, num_workers=args.workers)\n test_loader = DataLoader(test_dataset, batch_size=args.batch_size, shuffle=False, num_workers=args.workers)\n\n train_source_iter = ForeverDataIterator(train_source_loader)\n train_target_iter = ForeverDataIterator(train_target_loader)\n\n # create model\n print(\"=> using model '{}'\".format(args.arch))\n backbone = utils.get_model(args.arch, pretrain=not args.scratch)\n # pool_layer = nn.Identity() if args.no_pool else None\n pool_layer = nn.Sequential(\n nn.AdaptiveAvgPool2d(output_size=(1, 1)),\n nn.Flatten(),\n )\n pool_layer = TransformerDecorator(pool_layer, add_bt=args.add_bt, dim=backbone.out_features)\n classifier = ImageClassifier(backbone, num_classes, bottleneck_dim=args.bottleneck_dim,\n pool_layer=pool_layer, finetune=not args.scratch).to(device)\n\n # define optimizer and lr scheduler\n optimizer = SGD(classifier.get_parameters(), args.lr, momentum=args.momentum, weight_decay=args.weight_decay, nesterov=True)\n lr_scheduler = LambdaLR(optimizer, lambda x: args.lr * (1. + args.lr_gamma * float(x)) ** (-args.lr_decay))\n\n # define loss function\n mcc_loss = MinimumClassConfusionLoss(temperature=args.temperature)\n\n # resume from the best checkpoint\n if args.phase != 'train':\n checkpoint = torch.load(logger.get_checkpoint_path('best'), map_location='cpu')\n classifier.load_state_dict(checkpoint)\n\n # analysis the model\n if args.phase == 'analysis':\n # extract features from both domains\n feature_extractor = nn.Sequential(classifier.backbone, classifier.pool_layer, classifier.bottleneck).to(device)\n source_feature = collect_feature(train_source_loader, feature_extractor, device)\n target_feature = collect_feature(train_target_loader, feature_extractor, device)\n # plot t-SNE\n tSNE_filename = osp.join(logger.visualize_directory, 'TSNE.pdf')\n tsne.visualize(source_feature, target_feature, tSNE_filename)\n print(\"Saving t-SNE to\", tSNE_filename)\n # calculate A-distance, which is a measure for distribution discrepancy\n A_distance = a_distance.calculate(source_feature, target_feature, device)\n print(\"A-distance =\", A_distance)\n return\n\n if args.phase == 'test':\n acc1 = utils.validate(test_loader, classifier, args, device)\n print(acc1)\n return\n\n # start training\n best_acc1 = 0.\n for epoch in range(args.epochs):\n print(\"lr:\", lr_scheduler.get_last_lr()[0])\n # train for one epoch\n train(train_source_iter, train_target_iter, classifier, mcc_loss, optimizer,\n lr_scheduler, epoch, args)\n\n # evaluate on validation set\n acc1 = utils.validate(val_loader, classifier, args, device)\n\n # remember best acc@1 and save checkpoint\n torch.save(classifier.state_dict(), logger.get_checkpoint_path('latest'))\n if acc1 > best_acc1:\n shutil.copy(logger.get_checkpoint_path('latest'), logger.get_checkpoint_path('best'))\n best_acc1 = max(acc1, best_acc1)\n\n print(\"best_acc1 = {:3.1f}\".format(best_acc1))\n\n # evaluate on test set\n classifier.load_state_dict(torch.load(logger.get_checkpoint_path('best')))\n acc1 = utils.validate(test_loader, classifier, args, device)\n print(\"test_acc1 = {:3.1f}\".format(acc1))\n\n logger.close()\n\n\ndef train(train_source_iter: ForeverDataIterator, train_target_iter: ForeverDataIterator,\n model: ImageClassifier, mcc: MinimumClassConfusionLoss, optimizer: SGD,\n lr_scheduler: LambdaLR, epoch: int, args: argparse.Namespace):\n batch_time = AverageMeter('Time', ':3.1f')\n data_time = AverageMeter('Data', ':3.1f')\n losses = AverageMeter('Loss', ':3.2f')\n trans_losses = AverageMeter('Trans Loss', ':3.2f')\n cls_accs = AverageMeter('Cls Acc', ':3.1f')\n\n progress = ProgressMeter(\n args.iters_per_epoch,\n [batch_time, data_time, losses, trans_losses, cls_accs],\n prefix=\"Epoch: [{}]\".format(epoch))\n\n # switch to train mode\n model.train()\n\n end = time.time()\n for i in range(args.iters_per_epoch):\n x_s, labels_s = next(train_source_iter)\n x_t, _ = next(train_target_iter)\n\n x_s = x_s.to(device)\n x_t = x_t.to(device)\n labels_s = labels_s.to(device)\n\n # measure data loading time\n data_time.update(time.time() - end)\n\n # compute output\n x = torch.cat((x_s, x_t), dim=0)\n y, f = model(x)\n if len(y) > len(x):\n y_s, y_t, y_s1, y_t1 = y.chunk(4, dim=0)\n y_s = torch.cat([y_s, y_s1], dim=0)\n y_t = torch.cat([y_t, y_t1], dim=0)\n else:\n y_s, y_t = y.chunk(2, dim=0)\n if len(y_s) > len(labels_s):\n labels_s = torch.cat([labels_s, labels_s], dim=0)\n cls_loss = F.cross_entropy(y_s, labels_s)\n transfer_loss = mcc(y_t)\n loss = cls_loss + transfer_loss * args.trade_off\n\n cls_acc = accuracy(y_s, labels_s)[0]\n\n losses.update(loss.item(), x_s.size(0))\n cls_accs.update(cls_acc.item(), x_s.size(0))\n trans_losses.update(transfer_loss.item(), x_s.size(0))\n\n # compute gradient and do SGD step\n optimizer.zero_grad()\n loss.backward()\n optimizer.step()\n lr_scheduler.step()\n\n # measure elapsed time\n batch_time.update(time.time() - end)\n end = time.time()\n\n if i % args.print_freq == 0:\n progress.display(i)\n\n\nif __name__ == '__main__':\n parser = argparse.ArgumentParser(description='MCC for Unsupervised Domain Adaptation')\n # dataset parameters\n parser.add_argument('root', metavar='DIR',\n help='root path of dataset')\n parser.add_argument('-d', '--data', metavar='DATA', default='Office31', choices=utils.get_dataset_names(),\n help='dataset: ' + ' | '.join(utils.get_dataset_names()) +\n ' (default: Office31)')\n parser.add_argument('-s', '--source', help='source domain(s)', nargs='+')\n parser.add_argument('-t', '--target', help='target domain(s)', nargs='+')\n parser.add_argument('--train-resizing', type=str, default='default')\n parser.add_argument('--val-resizing', type=str, default='default')\n parser.add_argument('--resize-size', type=int, default=224,\n help='the image size after resizing')\n parser.add_argument('--no-hflip', action='store_true',\n help='no random horizontal flipping during training')\n parser.add_argument('--norm-mean', type=float, nargs='+',\n default=(0.485, 0.456, 0.406), help='normalization mean')\n parser.add_argument('--norm-std', type=float, nargs='+',\n default=(0.229, 0.224, 0.225), help='normalization std')\n # model parameters\n parser.add_argument('-a', '--arch', metavar='ARCH', default='resnet18',\n choices=utils.get_model_names(),\n help='backbone architecture: ' +\n ' | '.join(utils.get_model_names()) +\n ' (default: resnet18)')\n parser.add_argument('--bottleneck-dim', default=256, type=int,\n help='Dimension of bottleneck')\n parser.add_argument('--no-pool', action='store_true',\n help='no pool layer after the feature extractor.')\n parser.add_argument('--scratch', action='store_true', help='whether train from scratch.')\n parser.add_argument('--temperature', default=2.5, type=float, help='parameter temperature scaling')\n parser.add_argument('--trade-off', default=1., type=float,\n help='the trade-off hyper-parameter for transfer loss')\n # training parameters\n parser.add_argument('-b', '--batch-size', default=36, type=int,\n metavar='N',\n help='mini-batch size (default: 36)')\n parser.add_argument('--lr', '--learning-rate', default=0.005, type=float,\n metavar='LR', help='initial learning rate', dest='lr')\n parser.add_argument('--lr-gamma', default=0.001, type=float, help='parameter for lr scheduler')\n parser.add_argument('--lr-decay', default=0.75, type=float, help='parameter for lr scheduler')\n parser.add_argument('--momentum', default=0.9, type=float, metavar='M', help='momentum')\n parser.add_argument('--wd', '--weight-decay', default=1e-3, type=float,\n metavar='W', help='weight decay (default: 1e-3)',\n dest='weight_decay')\n parser.add_argument('-j', '--workers', default=2, type=int, metavar='N',\n help='number of data loading workers (default: 2)')\n parser.add_argument('--epochs', default=20, type=int, metavar='N',\n help='number of total epochs to run')\n parser.add_argument('-i', '--iters-per-epoch', default=1000, type=int,\n help='Number of iterations per epoch')\n parser.add_argument('-p', '--print-freq', default=100, type=int,\n metavar='N', help='print frequency (default: 100)')\n parser.add_argument('--seed', default=None, type=int,\n help='seed for initializing training. ')\n parser.add_argument('--per-class-eval', action='store_true',\n help='whether output per-class accuracy during evaluation')\n parser.add_argument(\"--log\", type=str, default='mcc',\n help=\"Where to save logs, checkpoints and debugging images.\")\n parser.add_argument(\"--phase\", type=str, default='train', choices=['train', 'test', 'analysis'],\n help=\"When phase is 'test', only test the model.\"\n \"When phase is 'analysis', only analysis the model.\")\n parser.add_argument(\"--add_bt\", type=int, default=0,\n help=\"add_bt\")\n args = parser.parse_args()\n main(args)\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/examples/domain_adaptation/image_classification/mcc.sh",
"content": "#!/usr/bin/env bash\n# ResNet50, Office31, Single Source\nCUDA_VISIBLE_DEVICES=0 python mcc.py data/office31 -d Office31 -s A -t W -a resnet50 --epochs 20 -i 500 --seed 2 --bottleneck-dim 1024 --log logs/mcc/Office31_A2W\nCUDA_VISIBLE_DEVICES=0 python mcc.py data/office31 -d Office31 -s D -t W -a resnet50 --epochs 20 -i 500 --seed 2 --bottleneck-dim 1024 --log logs/mcc/Office31_D2W\nCUDA_VISIBLE_DEVICES=0 python mcc.py data/office31 -d Office31 -s W -t D -a resnet50 --epochs 20 -i 500 --seed 2 --bottleneck-dim 1024 --log logs/mcc/Office31_W2D\nCUDA_VISIBLE_DEVICES=0 python mcc.py data/office31 -d Office31 -s A -t D -a resnet50 --epochs 20 -i 500 --seed 2 --bottleneck-dim 1024 --log logs/mcc/Office31_A2D\nCUDA_VISIBLE_DEVICES=0 python mcc.py data/office31 -d Office31 -s D -t A -a resnet50 --epochs 20 -i 500 --seed 2 --bottleneck-dim 1024 --log logs/mcc/Office31_D2A\nCUDA_VISIBLE_DEVICES=0 python mcc.py data/office31 -d Office31 -s W -t A -a resnet50 --epochs 20 -i 500 --seed 2 --bottleneck-dim 1024 --log logs/mcc/Office31_W2A\n\n# ResNet50, Office-Home, Single Source\nCUDA_VISIBLE_DEVICES=0 python mcc.py data/office-home -d OfficeHome -s Ar -t Cl -a resnet50 --epochs 30 --seed 0 --bottleneck-dim 2048 --log logs/mcc/OfficeHome_Ar2Cl\nCUDA_VISIBLE_DEVICES=0 python mcc.py data/office-home -d OfficeHome -s Ar -t Pr -a resnet50 --epochs 30 --seed 0 --bottleneck-dim 2048 --log logs/mcc/OfficeHome_Ar2Pr\nCUDA_VISIBLE_DEVICES=0 python mcc.py data/office-home -d OfficeHome -s Ar -t Rw -a resnet50 --epochs 30 --seed 0 --bottleneck-dim 2048 --log logs/mcc/OfficeHome_Ar2Rw\nCUDA_VISIBLE_DEVICES=0 python mcc.py data/office-home -d OfficeHome -s Cl -t Ar -a resnet50 --epochs 30 --seed 0 --bottleneck-dim 2048 --log logs/mcc/OfficeHome_Cl2Ar\nCUDA_VISIBLE_DEVICES=0 python mcc.py data/office-home -d OfficeHome -s Cl -t Pr -a resnet50 --epochs 30 --seed 0 --bottleneck-dim 2048 --log logs/mcc/OfficeHome_Cl2Pr\nCUDA_VISIBLE_DEVICES=0 python mcc.py data/office-home -d OfficeHome -s Cl -t Rw -a resnet50 --epochs 30 --seed 0 --bottleneck-dim 2048 --log logs/mcc/OfficeHome_Cl2Rw\nCUDA_VISIBLE_DEVICES=0 python mcc.py data/office-home -d OfficeHome -s Pr -t Ar -a resnet50 --epochs 30 --seed 0 --bottleneck-dim 2048 --log logs/mcc/OfficeHome_Pr2Ar\nCUDA_VISIBLE_DEVICES=0 python mcc.py data/office-home -d OfficeHome -s Pr -t Cl -a resnet50 --epochs 30 --seed 0 --bottleneck-dim 2048 --log logs/mcc/OfficeHome_Pr2Cl\nCUDA_VISIBLE_DEVICES=0 python mcc.py data/office-home -d OfficeHome -s Pr -t Rw -a resnet50 --epochs 30 --seed 0 --bottleneck-dim 2048 --log logs/mcc/OfficeHome_Pr2Rw\nCUDA_VISIBLE_DEVICES=0 python mcc.py data/office-home -d OfficeHome -s Rw -t Ar -a resnet50 --epochs 30 --seed 0 --bottleneck-dim 2048 --log logs/mcc/OfficeHome_Rw2Ar\nCUDA_VISIBLE_DEVICES=0 python mcc.py data/office-home -d OfficeHome -s Rw -t Cl -a resnet50 --epochs 30 --seed 0 --bottleneck-dim 2048 --log logs/mcc/OfficeHome_Rw2Cl\nCUDA_VISIBLE_DEVICES=0 python mcc.py data/office-home -d OfficeHome -s Rw -t Pr -a resnet50 --epochs 30 --seed 0 --bottleneck-dim 2048 --log logs/mcc/OfficeHome_Rw2Pr\n\n# ResNet101, VisDA-2017, Single Source\nCUDA_VISIBLE_DEVICES=5 python mcc.py data/visda-2017 -d VisDA2017 -s Synthetic -t Real -a resnet101 \\\n --epochs 30 --seed 0 --lr 0.002 --per-class-eval --temperature 3.0 --train-resizing cen.crop --log logs/mcc/VisDA2017\n\n# ResNet101, DomainNet, Single Source\nCUDA_VISIBLE_DEVICES=0 python mcc.py data/domainnet -d DomainNet -s c -t p -a resnet101 --epochs 30 -b 32 -i 2500 -p 500 --temperature 2.0 --lr 0.005 --bottleneck-dim 2048 --trade-off 10.0 --seed 0 --log logs/mcc/DomainNet_c2p\nCUDA_VISIBLE_DEVICES=0 python mcc.py data/domainnet -d DomainNet -s c -t r -a resnet101 --epochs 30 -b 32 -i 2500 -p 500 --temperature 2.0 --lr 0.005 --bottleneck-dim 2048 --trade-off 10.0 --seed 0 --log logs/mcc/DomainNet_c2r\nCUDA_VISIBLE_DEVICES=0 python mcc.py data/domainnet -d DomainNet -s c -t s -a resnet101 --epochs 30 -b 32 -i 2500 -p 500 --temperature 2.0 --lr 0.005 --bottleneck-dim 2048 --trade-off 10.0 --seed 0 --log logs/mcc/DomainNet_c2s\nCUDA_VISIBLE_DEVICES=0 python mcc.py data/domainnet -d DomainNet -s p -t c -a resnet101 --epochs 30 -b 32 -i 2500 -p 500 --temperature 2.0 --lr 0.005 --bottleneck-dim 2048 --trade-off 10.0 --seed 0 --log logs/mcc/DomainNet_p2c\nCUDA_VISIBLE_DEVICES=0 python mcc.py data/domainnet -d DomainNet -s p -t r -a resnet101 --epochs 30 -b 32 -i 2500 -p 500 --temperature 2.0 --lr 0.005 --bottleneck-dim 2048 --trade-off 10.0 --seed 0 --log logs/mcc/DomainNet_p2r\nCUDA_VISIBLE_DEVICES=0 python mcc.py data/domainnet -d DomainNet -s p -t s -a resnet101 --epochs 30 -b 32 -i 2500 -p 500 --temperature 2.0 --lr 0.005 --bottleneck-dim 2048 --trade-off 10.0 --seed 0 --log logs/mcc/DomainNet_p2s\nCUDA_VISIBLE_DEVICES=0 python mcc.py data/domainnet -d DomainNet -s r -t c -a resnet101 --epochs 30 -b 32 -i 2500 -p 500 --temperature 2.0 --lr 0.005 --bottleneck-dim 2048 --trade-off 10.0 --seed 0 --log logs/mcc/DomainNet_r2c\nCUDA_VISIBLE_DEVICES=0 python mcc.py data/domainnet -d DomainNet -s r -t p -a resnet101 --epochs 30 -b 32 -i 2500 -p 500 --temperature 2.0 --lr 0.005 --bottleneck-dim 2048 --trade-off 10.0 --seed 0 --log logs/mcc/DomainNet_r2p\nCUDA_VISIBLE_DEVICES=0 python mcc.py data/domainnet -d DomainNet -s r -t s -a resnet101 --epochs 30 -b 32 -i 2500 -p 500 --temperature 2.0 --lr 0.005 --bottleneck-dim 2048 --trade-off 10.0 --seed 0 --log logs/mcc/DomainNet_r2s\nCUDA_VISIBLE_DEVICES=0 python mcc.py data/domainnet -d DomainNet -s s -t c -a resnet101 --epochs 30 -b 32 -i 2500 -p 500 --temperature 2.0 --lr 0.005 --bottleneck-dim 2048 --trade-off 10.0 --seed 0 --log logs/mcc/DomainNet_s2c\nCUDA_VISIBLE_DEVICES=0 python mcc.py data/domainnet -d DomainNet -s s -t p -a resnet101 --epochs 30 -b 32 -i 2500 -p 500 --temperature 2.0 --lr 0.005 --bottleneck-dim 2048 --trade-off 10.0 --seed 0 --log logs/mcc/DomainNet_s2p\nCUDA_VISIBLE_DEVICES=0 python mcc.py data/domainnet -d DomainNet -s s -t r -a resnet101 --epochs 30 -b 32 -i 2500 -p 500 --temperature 2.0 --lr 0.005 --bottleneck-dim 2048 --trade-off 10.0 --seed 0 --log logs/mcc/DomainNet_s2r\n\n# ResNet50, ImageNet200 -> ImageNetR\nCUDA_VISIBLE_DEVICES=0 python mcc.py data/ImageNetR -d ImageNetR -s IN -t INR -a resnet50 --epochs 30 --seed 0 --temperature 2.5 --bottleneck-dim 2048 --log logs/mcc/ImageNet_IN2INR\n\n# ig_resnext101_32x8d, ImageNet -> ImageNetSketch\nCUDA_VISIBLE_DEVICES=0 python mcc.py data/imagenet-sketch -d ImageNetSketch -s IN -t sketch -a ig_resnext101_32x8d --epochs 30 -i 2500 -p 500 --log logs/mcc_ig_resnext101_32x8d/ImageNet_IN2sketch\n\n# Vision Transformer, Office-Home, Single Source\nCUDA_VISIBLE_DEVICES=0 python mcc.py data/office-home -d OfficeHome -s Ar -t Cl -a vit_base_patch16_224 --no-pool --bottleneck-dim 2048 --epochs 30 --seed 0 -b 24 --log logs/mcc_vit/OfficeHome_Ar2Cl\nCUDA_VISIBLE_DEVICES=0 python mcc.py data/office-home -d OfficeHome -s Ar -t Pr -a vit_base_patch16_224 --no-pool --bottleneck-dim 2048 --epochs 30 --seed 0 -b 24 --log logs/mcc_vit/OfficeHome_Ar2Pr\nCUDA_VISIBLE_DEVICES=0 python mcc.py data/office-home -d OfficeHome -s Ar -t Rw -a vit_base_patch16_224 --no-pool --bottleneck-dim 2048 --epochs 30 --seed 0 -b 24 --log logs/mcc_vit/OfficeHome_Ar2Rw\nCUDA_VISIBLE_DEVICES=0 python mcc.py data/office-home -d OfficeHome -s Cl -t Ar -a vit_base_patch16_224 --no-pool --bottleneck-dim 2048 --epochs 30 --seed 0 -b 24 --log logs/mcc_vit/OfficeHome_Cl2Ar\nCUDA_VISIBLE_DEVICES=0 python mcc.py data/office-home -d OfficeHome -s Cl -t Pr -a vit_base_patch16_224 --no-pool --bottleneck-dim 2048 --epochs 30 --seed 0 -b 24 --log logs/mcc_vit/OfficeHome_Cl2Pr\nCUDA_VISIBLE_DEVICES=0 python mcc.py data/office-home -d OfficeHome -s Cl -t Rw -a vit_base_patch16_224 --no-pool --bottleneck-dim 2048 --epochs 30 --seed 0 -b 24 --log logs/mcc_vit/OfficeHome_Cl2Rw\nCUDA_VISIBLE_DEVICES=0 python mcc.py data/office-home -d OfficeHome -s Pr -t Ar -a vit_base_patch16_224 --no-pool --bottleneck-dim 2048 --epochs 30 --seed 0 -b 24 --log logs/mcc_vit/OfficeHome_Pr2Ar\nCUDA_VISIBLE_DEVICES=0 python mcc.py data/office-home -d OfficeHome -s Pr -t Cl -a vit_base_patch16_224 --no-pool --bottleneck-dim 2048 --epochs 30 --seed 0 -b 24 --log logs/mcc_vit/OfficeHome_Pr2Cl\nCUDA_VISIBLE_DEVICES=0 python mcc.py data/office-home -d OfficeHome -s Pr -t Rw -a vit_base_patch16_224 --no-pool --bottleneck-dim 2048 --epochs 30 --seed 0 -b 24 --log logs/mcc_vit/OfficeHome_Pr2Rw\nCUDA_VISIBLE_DEVICES=0 python mcc.py data/office-home -d OfficeHome -s Rw -t Ar -a vit_base_patch16_224 --no-pool --bottleneck-dim 2048 --epochs 30 --seed 0 -b 24 --log logs/mcc_vit/OfficeHome_Rw2Ar\nCUDA_VISIBLE_DEVICES=0 python mcc.py data/office-home -d OfficeHome -s Rw -t Cl -a vit_base_patch16_224 --no-pool --bottleneck-dim 2048 --epochs 30 --seed 0 -b 24 --log logs/mcc_vit/OfficeHome_Rw2Cl\nCUDA_VISIBLE_DEVICES=0 python mcc.py data/office-home -d OfficeHome -s Rw -t Pr -a vit_base_patch16_224 --no-pool --bottleneck-dim 2048 --epochs 30 --seed 0 -b 24 --log logs/mcc_vit/OfficeHome_Rw2Pr\n\n# ResNet50, Office-Home, Multi Source\nCUDA_VISIBLE_DEVICES=0 python mcc.py data/office-home -d OfficeHome -s Cl Pr Rw -t Ar -a resnet50 --bottleneck-dim 2048 --epochs 30 --seed 0 --log logs/mcc/OfficeHome_:2Ar\nCUDA_VISIBLE_DEVICES=0 python mcc.py data/office-home -d OfficeHome -s Ar Pr Rw -t Cl -a resnet50 --bottleneck-dim 2048 --epochs 30 --seed 0 --log logs/mcc/OfficeHome_:2Cl\nCUDA_VISIBLE_DEVICES=0 python mcc.py data/office-home -d OfficeHome -s Ar Cl Rw -t Pr -a resnet50 --bottleneck-dim 2048 --epochs 30 --seed 0 --log logs/mcc/OfficeHome_:2Pr\nCUDA_VISIBLE_DEVICES=0 python mcc.py data/office-home -d OfficeHome -s Ar Cl Pr -t Rw -a resnet50 --bottleneck-dim 2048 --epochs 30 --seed 0 --log logs/mcc/OfficeHome_:2Rw\n\n# ResNet101, DomainNet, Multi Source\nCUDA_VISIBLE_DEVICES=0 python mcc.py data/domainnet -d DomainNet -s i p q r s -t c -a resnet101 --bottleneck-dim 2048 --epochs 40 -i 5000 -p 500 --seed 0 --log logs/mcc/DomainNet_:2c\nCUDA_VISIBLE_DEVICES=0 python mcc.py data/domainnet -d DomainNet -s c p q r s -t i -a resnet101 --bottleneck-dim 2048 --epochs 40 -i 5000 -p 500 --seed 0 --log logs/mcc/DomainNet_:2i\nCUDA_VISIBLE_DEVICES=0 python mcc.py data/domainnet -d DomainNet -s c i q r s -t p -a resnet101 --bottleneck-dim 2048 --epochs 40 -i 5000 -p 500 --seed 0 --log logs/mcc/DomainNet_:2p\nCUDA_VISIBLE_DEVICES=0 python mcc.py data/domainnet -d DomainNet -s c i p r s -t q -a resnet101 --bottleneck-dim 2048 --epochs 40 -i 5000 -p 500 --seed 0 --log logs/mcc/DomainNet_:2q\nCUDA_VISIBLE_DEVICES=0 python mcc.py data/domainnet -d DomainNet -s c i p q s -t r -a resnet101 --bottleneck-dim 2048 --epochs 40 -i 5000 -p 500 --seed 0 --log logs/mcc/DomainNet_:2r\nCUDA_VISIBLE_DEVICES=0 python mcc.py data/domainnet -d DomainNet -s c i p q r -t s -a resnet101 --bottleneck-dim 2048 --epochs 40 -i 5000 -p 500 --seed 0 --log logs/mcc/DomainNet_:2s\n\n# ResNet50, Wilds Dataset\nCUDA_VISIBLE_DEVICES=0 python mcc.py data/wilds -d iwildcam -a resnet50 --epochs 30 -i 1000 --seed 0 --bottleneck-dim 2048 --log logs/mcc/iwildcam\n\n# DenseNet121, Wilds Dataset\nCUDA_VISIBLE_DEVICES=0 python mcc.py data/wilds -d camelyon17 --train-resizing 'res.' --val-resizing 'res.' --resize-size 96 \\\n -a densenet121 --scratch --epochs 10 -i 1000 --lr 0.01 --seed 0 --log logs/mcc/camelyon17\nCUDA_VISIBLE_DEVICES=0 python mcc.py data/wilds -d fmow --train-resizing 'res.' --val-resizing 'res.' \\\n -a densenet121 --epochs 10 -i 1000 --lr 0.01 --seed 0 --bottleneck-dim 2048 --log logs/mcc/fmow\n\n# Digits\nCUDA_VISIBLE_DEVICES=0 python mcc.py data/digits -d Digits -s MNIST -t USPS --train-resizing 'res.' --val-resizing 'res.' \\\n --resize-size 28 --no-hflip --norm-mean 0.5 --norm-std 0.5 -a lenet --no-pool --lr 0.01 -b 128 -i 2500 --scratch --seed 0 --log logs/mcc/MNIST2USPS\nCUDA_VISIBLE_DEVICES=0 python mcc.py data/digits -d Digits -s USPS -t MNIST --train-resizing 'res.' --val-resizing 'res.' \\\n --resize-size 28 --no-hflip --norm-mean 0.5 --norm-std 0.5 -a lenet --no-pool --lr 0.1 -b 128 -i 2500 --scratch --seed 0 --log logs/mcc/USPS2MNIST\nCUDA_VISIBLE_DEVICES=0 python mcc.py data/digits -d Digits -s SVHNRGB -t MNISTRGB --train-resizing 'res.' --val-resizing 'res.' \\\n --resize-size 32 --no-hflip --norm-mean 0.5 0.5 0.5 --norm-std 0.5 0.5 0.5 -a dtn --no-pool --lr 0.01 -b 128 -i 2500 --scratch --seed 0 --log logs/mcc/SVHN2MNIST\n\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/examples/domain_adaptation/image_classification/mcd.py",
"content": "\"\"\"\n@author: Junguang Jiang\n@contact: JiangJunguang1123@outlook.com\n\"\"\"\nimport random\nimport time\nimport warnings\nimport sys\nimport argparse\nimport shutil\nimport os.path as osp\nfrom typing import Tuple\n\nimport torch\nimport torch.nn as nn\nimport torch.backends.cudnn as cudnn\nfrom torch.optim import SGD\nimport torch.utils.data\nfrom torch.utils.data import DataLoader\nimport torch.nn.functional as F\n\nsys.path.append('../../..')\nfrom dalib.adaptation.mcd import ImageClassifierHead, entropy, classifier_discrepancy\nfrom common.utils.data import ForeverDataIterator\nfrom common.utils.metric import accuracy, ConfusionMatrix\nfrom common.utils.meter import AverageMeter, ProgressMeter\nfrom common.utils.logger import CompleteLogger\nfrom common.utils.analysis import collect_feature, tsne, a_distance\n\nsys.path.append('.')\nimport utils\n\ndevice = torch.device(\"cuda\" if torch.cuda.is_available() else \"cpu\")\n\n\ndef main(args: argparse.Namespace):\n logger = CompleteLogger(args.log, args.phase)\n print(args)\n\n if args.seed is not None:\n random.seed(args.seed)\n torch.manual_seed(args.seed)\n cudnn.deterministic = True\n warnings.warn('You have chosen to seed training. '\n 'This will turn on the CUDNN deterministic setting, '\n 'which can slow down your training considerably! '\n 'You may see unexpected behavior when restarting '\n 'from checkpoints.')\n\n cudnn.benchmark = True\n\n # Data loading code\n train_transform = utils.get_train_transform(args.train_resizing, random_horizontal_flip=not args.no_hflip,\n random_color_jitter=False, resize_size=args.resize_size,\n norm_mean=args.norm_mean, norm_std=args.norm_std)\n val_transform = utils.get_val_transform(args.val_resizing, resize_size=args.resize_size,\n norm_mean=args.norm_mean, norm_std=args.norm_std)\n print(\"train_transform: \", train_transform)\n print(\"val_transform: \", val_transform)\n\n train_source_dataset, train_target_dataset, val_dataset, test_dataset, num_classes, args.class_names = \\\n utils.get_dataset(args.data, args.root, args.source, args.target, train_transform, val_transform)\n train_source_loader = DataLoader(train_source_dataset, batch_size=args.batch_size,\n shuffle=True, num_workers=args.workers, drop_last=True)\n train_target_loader = DataLoader(train_target_dataset, batch_size=args.batch_size,\n shuffle=True, num_workers=args.workers, drop_last=True)\n val_loader = DataLoader(val_dataset, batch_size=args.batch_size, shuffle=False, num_workers=args.workers)\n test_loader = DataLoader(test_dataset, batch_size=args.batch_size, shuffle=False, num_workers=args.workers)\n\n train_source_iter = ForeverDataIterator(train_source_loader)\n train_target_iter = ForeverDataIterator(train_target_loader)\n\n # create model\n print(\"=> using model '{}'\".format(args.arch))\n G = utils.get_model(args.arch, pretrain=not args.scratch).to(device) # feature extractor\n # two image classifier heads\n pool_layer = nn.Identity() if args.no_pool else None\n F1 = ImageClassifierHead(G.out_features, num_classes, args.bottleneck_dim, pool_layer).to(device)\n F2 = ImageClassifierHead(G.out_features, num_classes, args.bottleneck_dim, pool_layer).to(device)\n\n # define optimizer\n # the learning rate is fixed according to origin paper\n optimizer_g = SGD(G.parameters(), lr=args.lr, weight_decay=0.0005)\n optimizer_f = SGD([\n {\"params\": F1.parameters()},\n {\"params\": F2.parameters()},\n ], momentum=0.9, lr=args.lr, weight_decay=0.0005)\n\n # resume from the best checkpoint\n if args.phase != 'train':\n checkpoint = torch.load(logger.get_checkpoint_path('best'), map_location='cpu')\n G.load_state_dict(checkpoint['G'])\n F1.load_state_dict(checkpoint['F1'])\n F2.load_state_dict(checkpoint['F2'])\n\n # analysis the model\n if args.phase == 'analysis':\n # extract features from both domains\n feature_extractor = nn.Sequential(G, F1.pool_layer).to(device)\n source_feature = collect_feature(train_source_loader, feature_extractor, device)\n target_feature = collect_feature(train_target_loader, feature_extractor, device)\n # plot t-SNE\n tSNE_filename = osp.join(logger.visualize_directory, 'TSNE.pdf')\n tsne.visualize(source_feature, target_feature, tSNE_filename)\n print(\"Saving t-SNE to\", tSNE_filename)\n # calculate A-distance, which is a measure for distribution discrepancy\n A_distance = a_distance.calculate(source_feature, target_feature, device)\n print(\"A-distance =\", A_distance)\n return\n\n if args.phase == 'test':\n acc1 = validate(test_loader, G, F1, F2, args)\n print(acc1)\n return\n\n # start training\n best_acc1 = 0.\n best_results = None\n for epoch in range(args.epochs):\n # train for one epoch\n train(train_source_iter, train_target_iter, G, F1, F2, optimizer_g, optimizer_f, epoch, args)\n\n # evaluate on validation set\n results = validate(val_loader, G, F1, F2, args)\n\n # remember best acc@1 and save checkpoint\n torch.save({\n 'G': G.state_dict(),\n 'F1': F1.state_dict(),\n 'F2': F2.state_dict()\n }, logger.get_checkpoint_path('latest'))\n if max(results) > best_acc1:\n shutil.copy(logger.get_checkpoint_path('latest'), logger.get_checkpoint_path('best'))\n best_acc1 = max(results)\n best_results = results\n\n print(\"best_acc1 = {:3.1f}, results = {}\".format(best_acc1, best_results))\n\n # evaluate on test set\n checkpoint = torch.load(logger.get_checkpoint_path('best'), map_location='cpu')\n G.load_state_dict(checkpoint['G'])\n F1.load_state_dict(checkpoint['F1'])\n F2.load_state_dict(checkpoint['F2'])\n results = validate(test_loader, G, F1, F2, args)\n print(\"test_acc1 = {:3.1f}\".format(max(results)))\n\n logger.close()\n\n\ndef train(train_source_iter: ForeverDataIterator, train_target_iter: ForeverDataIterator,\n G: nn.Module, F1: ImageClassifierHead, F2: ImageClassifierHead,\n optimizer_g: SGD, optimizer_f: SGD, epoch: int, args: argparse.Namespace):\n batch_time = AverageMeter('Time', ':3.1f')\n data_time = AverageMeter('Data', ':3.1f')\n losses = AverageMeter('Loss', ':3.2f')\n trans_losses = AverageMeter('Trans Loss', ':3.2f')\n cls_accs = AverageMeter('Cls Acc', ':3.1f')\n tgt_accs = AverageMeter('Tgt Acc', ':3.1f')\n\n progress = ProgressMeter(\n args.iters_per_epoch,\n [batch_time, data_time, losses, trans_losses, cls_accs, tgt_accs],\n prefix=\"Epoch: [{}]\".format(epoch))\n\n # switch to train mode\n G.train()\n F1.train()\n F2.train()\n\n end = time.time()\n for i in range(args.iters_per_epoch):\n x_s, labels_s = next(train_source_iter)\n x_t, labels_t = next(train_target_iter)\n\n x_s = x_s.to(device)\n x_t = x_t.to(device)\n labels_s = labels_s.to(device)\n labels_t = labels_t.to(device)\n x = torch.cat((x_s, x_t), dim=0)\n assert x.requires_grad is False\n\n # measure data loading time\n data_time.update(time.time() - end)\n\n # Step A train all networks to minimize loss on source domain\n optimizer_g.zero_grad()\n optimizer_f.zero_grad()\n\n g = G(x)\n y_1 = F1(g)\n y_2 = F2(g)\n y1_s, y1_t = y_1.chunk(2, dim=0)\n y2_s, y2_t = y_2.chunk(2, dim=0)\n\n y1_t, y2_t = F.softmax(y1_t, dim=1), F.softmax(y2_t, dim=1)\n loss = F.cross_entropy(y1_s, labels_s) + F.cross_entropy(y2_s, labels_s) + \\\n (entropy(y1_t) + entropy(y2_t)) * args.trade_off_entropy\n loss.backward()\n optimizer_g.step()\n optimizer_f.step()\n\n # Step B train classifier to maximize discrepancy\n optimizer_g.zero_grad()\n optimizer_f.zero_grad()\n\n g = G(x)\n y_1 = F1(g)\n y_2 = F2(g)\n y1_s, y1_t = y_1.chunk(2, dim=0)\n y2_s, y2_t = y_2.chunk(2, dim=0)\n y1_t, y2_t = F.softmax(y1_t, dim=1), F.softmax(y2_t, dim=1)\n loss = F.cross_entropy(y1_s, labels_s) + F.cross_entropy(y2_s, labels_s) + \\\n (entropy(y1_t) + entropy(y2_t)) * args.trade_off_entropy - \\\n classifier_discrepancy(y1_t, y2_t) * args.trade_off\n loss.backward()\n optimizer_f.step()\n\n # Step C train genrator to minimize discrepancy\n for k in range(args.num_k):\n optimizer_g.zero_grad()\n g = G(x)\n y_1 = F1(g)\n y_2 = F2(g)\n y1_s, y1_t = y_1.chunk(2, dim=0)\n y2_s, y2_t = y_2.chunk(2, dim=0)\n y1_t, y2_t = F.softmax(y1_t, dim=1), F.softmax(y2_t, dim=1)\n mcd_loss = classifier_discrepancy(y1_t, y2_t) * args.trade_off\n mcd_loss.backward()\n optimizer_g.step()\n\n cls_acc = accuracy(y1_s, labels_s)[0]\n tgt_acc = accuracy(y1_t, labels_t)[0]\n\n losses.update(loss.item(), x_s.size(0))\n cls_accs.update(cls_acc.item(), x_s.size(0))\n tgt_accs.update(tgt_acc.item(), x_t.size(0))\n trans_losses.update(mcd_loss.item(), x_s.size(0))\n\n # measure elapsed time\n batch_time.update(time.time() - end)\n end = time.time()\n\n if i % args.print_freq == 0:\n progress.display(i)\n\n\ndef validate(val_loader: DataLoader, G: nn.Module, F1: ImageClassifierHead,\n F2: ImageClassifierHead, args: argparse.Namespace) -> Tuple[float, float]:\n batch_time = AverageMeter('Time', ':6.3f')\n top1_1 = AverageMeter('Acc_1', ':6.2f')\n top1_2 = AverageMeter('Acc_2', ':6.2f')\n progress = ProgressMeter(\n len(val_loader),\n [batch_time, top1_1, top1_2],\n prefix='Test: ')\n\n # switch to evaluate mode\n G.eval()\n F1.eval()\n F2.eval()\n\n if args.per_class_eval:\n confmat = ConfusionMatrix(len(args.class_names))\n else:\n confmat = None\n\n with torch.no_grad():\n end = time.time()\n for i, (images, target) in enumerate(val_loader):\n images = images.to(device)\n target = target.to(device)\n\n # compute output\n g = G(images)\n y1, y2 = F1(g), F2(g)\n\n # measure accuracy and record loss\n acc1, = accuracy(y1, target)\n acc2, = accuracy(y2, target)\n if confmat:\n confmat.update(target, y1.argmax(1))\n top1_1.update(acc1.item(), images.size(0))\n top1_2.update(acc2.item(), images.size(0))\n\n # measure elapsed time\n batch_time.update(time.time() - end)\n end = time.time()\n\n if i % args.print_freq == 0:\n progress.display(i)\n\n print(' * Acc1 {top1_1.avg:.3f} Acc2 {top1_2.avg:.3f}'\n .format(top1_1=top1_1, top1_2=top1_2))\n if confmat:\n print(confmat.format(args.class_names))\n\n return top1_1.avg, top1_2.avg\n\n\nif __name__ == '__main__':\n parser = argparse.ArgumentParser(description='MCD for Unsupervised Domain Adaptation')\n # dataset parameters\n parser.add_argument('root', metavar='DIR',\n help='root path of dataset')\n parser.add_argument('-d', '--data', metavar='DATA', default='Office31', choices=utils.get_dataset_names(),\n help='dataset: ' + ' | '.join(utils.get_dataset_names()) +\n ' (default: Office31)')\n parser.add_argument('-s', '--source', help='source domain(s)', nargs='+')\n parser.add_argument('-t', '--target', help='target domain(s)', nargs='+')\n parser.add_argument('--train-resizing', type=str, default='default')\n parser.add_argument('--val-resizing', type=str, default='default')\n parser.add_argument('--resize-size', type=int, default=224,\n help='the image size after resizing')\n parser.add_argument('--no-hflip', action='store_true',\n help='no random horizontal flipping during training')\n parser.add_argument('--norm-mean', type=float, nargs='+',\n default=(0.485, 0.456, 0.406), help='normalization mean')\n parser.add_argument('--norm-std', type=float, nargs='+',\n default=(0.229, 0.224, 0.225), help='normalization std')\n # model parameters\n parser.add_argument('-a', '--arch', metavar='ARCH', default='resnet18',\n choices=utils.get_model_names(),\n help='backbone architecture: ' +\n ' | '.join(utils.get_model_names()) +\n ' (default: resnet18)')\n parser.add_argument('--bottleneck-dim', default=1024, type=int)\n parser.add_argument('--no-pool', action='store_true',\n help='no pool layer after the feature extractor.')\n parser.add_argument('--scratch', action='store_true', help='whether train from scratch.')\n parser.add_argument('--trade-off', default=1., type=float,\n help='the trade-off hyper-parameter for transfer loss')\n parser.add_argument('--trade-off-entropy', default=0.01, type=float,\n help='the trade-off hyper-parameter for entropy loss')\n parser.add_argument('--num-k', type=int, default=4, metavar='K',\n help='how many steps to repeat the generator update')\n # training parameters\n parser.add_argument('-b', '--batch-size', default=32, type=int,\n metavar='N',\n help='mini-batch size (default: 32)')\n parser.add_argument('--lr', '--learning-rate', default=0.001, type=float,\n metavar='LR', help='initial learning rate', dest='lr')\n parser.add_argument('-j', '--workers', default=2, type=int, metavar='N',\n help='number of data loading workers (default: 2)')\n parser.add_argument('--epochs', default=20, type=int, metavar='N',\n help='number of total epochs to run')\n parser.add_argument('-i', '--iters-per-epoch', default=1000, type=int,\n help='Number of iterations per epoch')\n parser.add_argument('-p', '--print-freq', default=100, type=int,\n metavar='N', help='print frequency (default: 100)')\n parser.add_argument('--seed', default=None, type=int,\n help='seed for initializing training. ')\n parser.add_argument('--per-class-eval', action='store_true',\n help='whether output per-class accuracy during evaluation')\n parser.add_argument(\"--log\", type=str, default='mcd',\n help=\"Where to save logs, checkpoints and debugging images.\")\n parser.add_argument(\"--phase\", type=str, default='train', choices=['train', 'test', 'analysis'],\n help=\"When phase is 'test', only test the model.\"\n \"When phase is 'analysis', only analysis the model.\")\n args = parser.parse_args()\n main(args)\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/examples/domain_adaptation/image_classification/mcd.sh",
"content": "#!/usr/bin/env bash\n# ResNet50, Office31, Single Source\n# We found MCD loss is sensitive to class number,\n# thus, when the class number increase, please increase trade-off correspondingly.\nCUDA_VISIBLE_DEVICES=0 python mcd.py data/office31 -d Office31 -s D -t A -a resnet50 --epochs 20 --seed 0 -i 500 --trade-off 10.0 --log logs/mcd/Office31_D2A\nCUDA_VISIBLE_DEVICES=0 python mcd.py data/office31 -d Office31 -s W -t A -a resnet50 --epochs 20 --seed 0 -i 500 --trade-off 10.0 --log logs/mcd/Office31_W2A\nCUDA_VISIBLE_DEVICES=0 python mcd.py data/office31 -d Office31 -s A -t W -a resnet50 --epochs 20 --seed 0 -i 500 --trade-off 10.0 --log logs/mcd/Office31_A2W\nCUDA_VISIBLE_DEVICES=0 python mcd.py data/office31 -d Office31 -s A -t D -a resnet50 --epochs 20 --seed 0 -i 500 --trade-off 10.0 --log logs/mcd/Office31_A2D\nCUDA_VISIBLE_DEVICES=0 python mcd.py data/office31 -d Office31 -s D -t W -a resnet50 --epochs 20 --seed 0 -i 500 --trade-off 10.0 --log logs/mcd/Office31_D2W\nCUDA_VISIBLE_DEVICES=0 python mcd.py data/office31 -d Office31 -s W -t D -a resnet50 --epochs 20 --seed 0 -i 500 --trade-off 10.0 --log logs/mcd/Office31_W2D\n\n# ResNet50, Office-Home, Single Source\nCUDA_VISIBLE_DEVICES=0 python mcd.py data/office-home -d OfficeHome -s Ar -t Cl -a resnet50 --epochs 20 -i 500 --seed 0 --trade-off 30.0 --log logs/mcd/OfficeHome_Ar2Cl\nCUDA_VISIBLE_DEVICES=0 python mcd.py data/office-home -d OfficeHome -s Ar -t Pr -a resnet50 --epochs 20 -i 500 --seed 0 --trade-off 30.0 --log logs/mcd/OfficeHome_Ar2Pr\nCUDA_VISIBLE_DEVICES=0 python mcd.py data/office-home -d OfficeHome -s Ar -t Rw -a resnet50 --epochs 20 -i 500 --seed 0 --trade-off 30.0 --log logs/mcd/OfficeHome_Ar2Rw\nCUDA_VISIBLE_DEVICES=0 python mcd.py data/office-home -d OfficeHome -s Cl -t Ar -a resnet50 --epochs 20 -i 500 --seed 0 --trade-off 30.0 --log logs/mcd/OfficeHome_Cl2Ar\nCUDA_VISIBLE_DEVICES=0 python mcd.py data/office-home -d OfficeHome -s Cl -t Pr -a resnet50 --epochs 20 -i 500 --seed 0 --trade-off 30.0 --log logs/mcd/OfficeHome_Cl2Pr\nCUDA_VISIBLE_DEVICES=0 python mcd.py data/office-home -d OfficeHome -s Cl -t Rw -a resnet50 --epochs 20 -i 500 --seed 0 --trade-off 30.0 --log logs/mcd/OfficeHome_Cl2Rw\nCUDA_VISIBLE_DEVICES=0 python mcd.py data/office-home -d OfficeHome -s Pr -t Ar -a resnet50 --epochs 20 -i 500 --seed 0 --trade-off 30.0 --log logs/mcd/OfficeHome_Pr2Ar\nCUDA_VISIBLE_DEVICES=0 python mcd.py data/office-home -d OfficeHome -s Pr -t Cl -a resnet50 --epochs 20 -i 500 --seed 0 --trade-off 30.0 --log logs/mcd/OfficeHome_Pr2Cl\nCUDA_VISIBLE_DEVICES=0 python mcd.py data/office-home -d OfficeHome -s Pr -t Rw -a resnet50 --epochs 20 -i 500 --seed 0 --trade-off 30.0 --log logs/mcd/OfficeHome_Pr2Rw\nCUDA_VISIBLE_DEVICES=0 python mcd.py data/office-home -d OfficeHome -s Rw -t Ar -a resnet50 --epochs 20 -i 500 --seed 0 --trade-off 30.0 --log logs/mcd/OfficeHome_Rw2Ar\nCUDA_VISIBLE_DEVICES=0 python mcd.py data/office-home -d OfficeHome -s Rw -t Cl -a resnet50 --epochs 20 -i 500 --seed 0 --trade-off 30.0 --log logs/mcd/OfficeHome_Rw2Cl\nCUDA_VISIBLE_DEVICES=0 python mcd.py data/office-home -d OfficeHome -s Rw -t Pr -a resnet50 --epochs 20 -i 500 --seed 0 --trade-off 30.0 --log logs/mcd/OfficeHome_Rw2Pr\n\n# ResNet101, VisDA-2017, Single Source\nCUDA_VISIBLE_DEVICES=0 python mcd.py data/visda-2017 -d VisDA2017 -s Synthetic -t Real -a resnet101 \\\n --epochs 20 --center-crop --seed 0 -i 500 --per-class-eval --train-resizing cen.crop --log logs/mcd/VisDA2017\n\n# ResNet101, DomainNet, Single Source\nCUDA_VISIBLE_DEVICES=0 python mcd.py data/domainnet -d DomainNet -s c -t p -a resnet101 --bottleneck-dim 1024 --epochs 30 -i 2500 -p 500 --seed 0 --trade-off 120.0 --log logs/mcd/DomainNet_c2p\nCUDA_VISIBLE_DEVICES=0 python mcd.py data/domainnet -d DomainNet -s c -t r -a resnet101 --bottleneck-dim 1024 --epochs 30 -i 2500 -p 500 --seed 0 --trade-off 120.0 --log logs/mcd/DomainNet_c2r\nCUDA_VISIBLE_DEVICES=0 python mcd.py data/domainnet -d DomainNet -s c -t s -a resnet101 --bottleneck-dim 1024 --epochs 30 -i 2500 -p 500 --seed 0 --trade-off 120.0 --log logs/mcd/DomainNet_c2s\nCUDA_VISIBLE_DEVICES=0 python mcd.py data/domainnet -d DomainNet -s p -t c -a resnet101 --bottleneck-dim 1024 --epochs 30 -i 2500 -p 500 --seed 0 --trade-off 120.0 --log logs/mcd/DomainNet_p2c\nCUDA_VISIBLE_DEVICES=0 python mcd.py data/domainnet -d DomainNet -s p -t i -a resnet101 --bottleneck-dim 1024 --epochs 30 -i 2500 -p 500 --seed 0 --trade-off 120.0 --log logs/mcd/DomainNet_p2i\nCUDA_VISIBLE_DEVICES=0 python mcd.py data/domainnet -d DomainNet -s p -t r -a resnet101 --bottleneck-dim 1024 --epochs 30 -i 2500 -p 500 --seed 0 --trade-off 120.0 --log logs/mcd/DomainNet_p2r\nCUDA_VISIBLE_DEVICES=0 python mcd.py data/domainnet -d DomainNet -s p -t s -a resnet101 --bottleneck-dim 1024 --epochs 30 -i 2500 -p 500 --seed 0 --trade-off 120.0 --log logs/mcd/DomainNet_p2s\nCUDA_VISIBLE_DEVICES=0 python mcd.py data/domainnet -d DomainNet -s r -t c -a resnet101 --bottleneck-dim 1024 --epochs 30 -i 2500 -p 500 --seed 0 --trade-off 120.0 --log logs/mcd/DomainNet_r2c\nCUDA_VISIBLE_DEVICES=0 python mcd.py data/domainnet -d DomainNet -s r -t p -a resnet101 --bottleneck-dim 1024 --epochs 30 -i 2500 -p 500 --seed 0 --trade-off 120.0 --log logs/mcd/DomainNet_r2p\nCUDA_VISIBLE_DEVICES=0 python mcd.py data/domainnet -d DomainNet -s r -t s -a resnet101 --bottleneck-dim 1024 --epochs 30 -i 2500 -p 500 --seed 0 --trade-off 120.0 --log logs/mcd/DomainNet_r2s\nCUDA_VISIBLE_DEVICES=0 python mcd.py data/domainnet -d DomainNet -s s -t c -a resnet101 --bottleneck-dim 1024 --epochs 30 -i 2500 -p 500 --seed 0 --trade-off 120.0 --log logs/mcd/DomainNet_s2c\nCUDA_VISIBLE_DEVICES=0 python mcd.py data/domainnet -d DomainNet -s s -t p -a resnet101 --bottleneck-dim 1024 --epochs 30 -i 2500 -p 500 --seed 0 --trade-off 120.0 --log logs/mcd/DomainNet_s2p\nCUDA_VISIBLE_DEVICES=0 python mcd.py data/domainnet -d DomainNet -s s -t r -a resnet101 --bottleneck-dim 1024 --epochs 30 -i 2500 -p 500 --seed 0 --trade-off 120.0 --log logs/mcd/DomainNet_s2r\n\n# ResNet50, ImageNet200 -> ImageNetR\nCUDA_VISIBLE_DEVICES=0 python mcd.py data/ImageNetR -d ImageNetR -s IN -t INR -a resnet50 --epochs 30 -i 2500 -p 500 --seed 0 --trade-off 100.0 --log logs/mcd/ImageNet_IN2INR\n\n# ig_resnext101_32x8d, ImageNet -> ImageNetSketch\nCUDA_VISIBLE_DEVICES=0 python mcd.py data/imagenet-sketch -d ImageNetSketch -s IN -t sketch -a ig_resnext101_32x8d --epochs 30 -i 2500 -p 500 --trade-off 500.0 --log logs/mcd_ig_resnext101_32x8d/ImageNet_IN2sketch\n\n# Vision Transformer, Office-Home, Single Source\nCUDA_VISIBLE_DEVICES=0 python mcd.py data/office-home -d OfficeHome -s Ar -t Cl -a vit_base_patch16_224 --epochs 30 --seed 0 -b 24 --no-pool --trade-off 30.0 --log logs/mcd_vit/OfficeHome_Ar2Cl\nCUDA_VISIBLE_DEVICES=0 python mcd.py data/office-home -d OfficeHome -s Ar -t Pr -a vit_base_patch16_224 --epochs 30 --seed 0 -b 24 --no-pool --trade-off 30.0 --log logs/mcd_vit/OfficeHome_Ar2Pr\nCUDA_VISIBLE_DEVICES=0 python mcd.py data/office-home -d OfficeHome -s Ar -t Rw -a vit_base_patch16_224 --epochs 30 --seed 0 -b 24 --no-pool --trade-off 30.0 --log logs/mcd_vit/OfficeHome_Ar2Rw\nCUDA_VISIBLE_DEVICES=0 python mcd.py data/office-home -d OfficeHome -s Cl -t Ar -a vit_base_patch16_224 --epochs 30 --seed 0 -b 24 --no-pool --trade-off 30.0 --log logs/mcd_vit/OfficeHome_Cl2Ar\nCUDA_VISIBLE_DEVICES=0 python mcd.py data/office-home -d OfficeHome -s Cl -t Pr -a vit_base_patch16_224 --epochs 30 --seed 0 -b 24 --no-pool --trade-off 30.0 --log logs/mcd_vit/OfficeHome_Cl2Pr\nCUDA_VISIBLE_DEVICES=0 python mcd.py data/office-home -d OfficeHome -s Cl -t Rw -a vit_base_patch16_224 --epochs 30 --seed 0 -b 24 --no-pool --trade-off 30.0 --log logs/mcd_vit/OfficeHome_Cl2Rw\nCUDA_VISIBLE_DEVICES=0 python mcd.py data/office-home -d OfficeHome -s Pr -t Ar -a vit_base_patch16_224 --epochs 30 --seed 0 -b 24 --no-pool --trade-off 30.0 --log logs/mcd_vit/OfficeHome_Pr2Ar\nCUDA_VISIBLE_DEVICES=0 python mcd.py data/office-home -d OfficeHome -s Pr -t Cl -a vit_base_patch16_224 --epochs 30 --seed 0 -b 24 --no-pool --trade-off 30.0 --log logs/mcd_vit/OfficeHome_Pr2Cl\nCUDA_VISIBLE_DEVICES=0 python mcd.py data/office-home -d OfficeHome -s Pr -t Rw -a vit_base_patch16_224 --epochs 30 --seed 0 -b 24 --no-pool --trade-off 30.0 --log logs/mcd_vit/OfficeHome_Pr2Rw\nCUDA_VISIBLE_DEVICES=0 python mcd.py data/office-home -d OfficeHome -s Rw -t Ar -a vit_base_patch16_224 --epochs 30 --seed 0 -b 24 --no-pool --trade-off 30.0 --log logs/mcd_vit/OfficeHome_Rw2Ar\nCUDA_VISIBLE_DEVICES=0 python mcd.py data/office-home -d OfficeHome -s Rw -t Cl -a vit_base_patch16_224 --epochs 30 --seed 0 -b 24 --no-pool --trade-off 30.0 --log logs/mcd_vit/OfficeHome_Rw2Cl\nCUDA_VISIBLE_DEVICES=0 python mcd.py data/office-home -d OfficeHome -s Rw -t Pr -a vit_base_patch16_224 --epochs 30 --seed 0 -b 24 --no-pool --trade-off 30.0 --log logs/mcd_vit/OfficeHome_Rw2Pr\n\n# ResNet50, Wilds Dataset\nCUDA_VISIBLE_DEVICES=0 python mcd.py data/wilds -d iwildcam -a resnet50 --epochs 30 -i 1000 --seed 0 --trade-off 100.0 --log logs/mcd/iwildcam\n\n# DenseNet121, Wilds Dataset\nCUDA_VISIBLE_DEVICES=0 python mcd.py data/wilds -d camelyon17 --train-resizing 'res.' --val-resizing 'res.' --resize-size 96 \\\n -a densenet121 --scratch --epochs 10 -i 1000 --lr 0.01 --seed 0 --log logs/mcd/camelyon17\nCUDA_VISIBLE_DEVICES=0 python mcd.py data/wilds -d fmow --train-resizing 'res.' --val-resizing 'res.' \\\n -a densenet121 --epochs 10 -i 1000 --lr 0.01 --seed 0 --trade-off 30.0 --log logs/mcd/fmow\n\n# Digits\nCUDA_VISIBLE_DEVICES=0 python mcd.py data/digits -d Digits -s MNIST -t USPS --train-resizing 'res.' --val-resizing 'res.' \\\n --resize-size 28 --no-hflip --norm-mean 0.5 --norm-std 0.5 -a lenet --no-pool --lr 0.01 --trade-off 0.3 --trade-off-entropy 0.03 -b 128 -i 2500 --scratch --seed 0 --log logs/mcd/MNIST2USPS\nCUDA_VISIBLE_DEVICES=0 python mcd.py data/digits -d Digits -s USPS -t MNIST --train-resizing 'res.' --val-resizing 'res.' \\\n --resize-size 28 --no-hflip --norm-mean 0.5 --norm-std 0.5 -a lenet --no-pool --lr 0.1 --trade-off 0.3 --trade-off-entropy 0.03 -b 128 -i 2500 --scratch --seed 0 --log logs/mcd/USPS2MNIST\nCUDA_VISIBLE_DEVICES=0 python mcd.py data/digits -d Digits -s SVHNRGB -t MNISTRGB --train-resizing 'res.' --val-resizing 'res.' \\\n --resize-size 32 --no-hflip --norm-mean 0.5 0.5 0.5 --norm-std 0.5 0.5 0.5 -a dtn --no-pool --lr 0.03 --trade-off 0.3 --trade-off-entropy 0.03 -b 128 -i 2500 --scratch --seed 0 --log logs/mcd/SVHN2MNIST\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/examples/domain_adaptation/image_classification/mdd.py",
"content": "\"\"\"\n@author: Junguang Jiang\n@contact: JiangJunguang1123@outlook.com\n\"\"\"\nimport random\nimport time\nimport warnings\nimport sys\nimport argparse\nimport os.path as osp\nimport shutil\n\nimport torch\nimport torch.nn as nn\nimport torch.backends.cudnn as cudnn\nfrom torch.optim import SGD\nfrom torch.optim.lr_scheduler import LambdaLR\nfrom torch.utils.data import DataLoader\nimport torch.nn.functional as F\n\nsys.path.append('../../..')\nfrom dalib.adaptation.mdd import ClassificationMarginDisparityDiscrepancy\\\n as MarginDisparityDiscrepancy, ImageClassifier\nfrom common.utils.data import ForeverDataIterator\nfrom common.utils.metric import accuracy\nfrom common.utils.meter import AverageMeter, ProgressMeter\nfrom common.utils.logger import CompleteLogger\nfrom common.utils.analysis import collect_feature, tsne, a_distance\n\nsys.path.append('.')\nimport utils\n\ndevice = torch.device(\"cuda\" if torch.cuda.is_available() else \"cpu\")\n\nclass TransformerDecorator(torch.nn.Module):\n def __init__(self, backbone=None, add_bt=3, dim=2048, eval_global=0):\n super(TransformerDecorator, self).__init__()\n self.encoder_layers = torch.nn.TransformerEncoderLayer(dim, 4, dim, 0.5)\n self.eval_global = eval_global\n self.add_bt = add_bt\n self.backbone = backbone\n\n def forward_feats(self, feature):\n feature = self.backbone(feature)\n if self.training and self.add_bt > 0:\n pre_feature = feature\n feature = feature.unsqueeze(1)\n feature = self.encoder_layers(feature)\n # feature = self.encoder_layers(feature, feature, feature)\n feature = feature.squeeze(1)\n if self.add_bt >= 1:\n return torch.cat([pre_feature, feature], dim=0)\n return feature\n\n def forward(self, feature):\n feature = self.forward_feats(feature)\n return feature\n\n\ndef main(args: argparse.Namespace):\n logger = CompleteLogger(args.log, args.phase)\n print(args)\n\n if args.seed is not None:\n random.seed(args.seed)\n torch.manual_seed(args.seed)\n cudnn.deterministic = True\n warnings.warn('You have chosen to seed training. '\n 'This will turn on the CUDNN deterministic setting, '\n 'which can slow down your training considerably! '\n 'You may see unexpected behavior when restarting '\n 'from checkpoints.')\n\n cudnn.benchmark = True\n\n # Data loading code\n train_transform = utils.get_train_transform(args.train_resizing, random_horizontal_flip=not args.no_hflip,\n random_color_jitter=False, resize_size=args.resize_size,\n norm_mean=args.norm_mean, norm_std=args.norm_std)\n val_transform = utils.get_val_transform(args.val_resizing, resize_size=args.resize_size,\n norm_mean=args.norm_mean, norm_std=args.norm_std)\n print(\"train_transform: \", train_transform)\n print(\"val_transform: \", val_transform)\n\n train_source_dataset, train_target_dataset, val_dataset, test_dataset, num_classes, args.class_names = \\\n utils.get_dataset(args.data, args.root, args.source, args.target, train_transform, val_transform)\n train_source_loader = DataLoader(train_source_dataset, batch_size=args.batch_size,\n shuffle=True, num_workers=args.workers, drop_last=True)\n train_target_loader = DataLoader(train_target_dataset, batch_size=args.batch_size,\n shuffle=True, num_workers=args.workers, drop_last=True)\n val_loader = DataLoader(val_dataset, batch_size=args.batch_size, shuffle=False, num_workers=args.workers)\n test_loader = DataLoader(test_dataset, batch_size=args.batch_size, shuffle=False, num_workers=args.workers)\n\n train_source_iter = ForeverDataIterator(train_source_loader)\n train_target_iter = ForeverDataIterator(train_target_loader)\n\n # create model\n print(\"=> using model '{}'\".format(args.arch))\n backbone = utils.get_model(args.arch, pretrain=not args.scratch)\n # pool_layer = nn.Identity() if args.no_pool else None\n pool_layer = nn.Sequential(\n nn.AdaptiveAvgPool2d(output_size=(1, 1)),\n nn.Flatten(),\n )\n pool_layer = TransformerDecorator(pool_layer, add_bt=args.add_bt, dim=backbone.out_features)\n classifier = ImageClassifier(backbone, num_classes, bottleneck_dim=args.bottleneck_dim,\n width=args.bottleneck_dim, pool_layer=pool_layer).to(device)\n mdd = MarginDisparityDiscrepancy(args.margin).to(device)\n\n # define optimizer and lr_scheduler\n # The learning rate of the classifiers are set 10 times to that of the feature extractor by default.\n optimizer = SGD(classifier.get_parameters(), args.lr, momentum=args.momentum, weight_decay=args.wd, nesterov=True)\n lr_scheduler = LambdaLR(optimizer, lambda x: args.lr * (1. + args.lr_gamma * float(x)) ** (-args.lr_decay))\n\n # resume from the best checkpoint\n if args.phase != 'train':\n checkpoint = torch.load(logger.get_checkpoint_path('best'), map_location='cpu')\n classifier.load_state_dict(checkpoint)\n\n # analysis the model\n if args.phase == 'analysis':\n # extract features from both domains\n feature_extractor = nn.Sequential(classifier.backbone, classifier.bottleneck).to(device)\n source_feature = collect_feature(train_source_loader, feature_extractor, device)\n target_feature = collect_feature(train_target_loader, feature_extractor, device)\n # plot t-SNE\n tSNE_filename = osp.join(logger.visualize_directory, 'TSNE.pdf')\n tsne.visualize(source_feature, target_feature, tSNE_filename)\n print(\"Saving t-SNE to\", tSNE_filename)\n # calculate A-distance, which is a measure for distribution discrepancy\n A_distance = a_distance.calculate(source_feature, target_feature, device)\n print(\"A-distance =\", A_distance)\n return\n\n if args.phase == 'test':\n acc1 = utils.validate(test_loader, classifier, args, device)\n print(acc1)\n return\n\n # start training\n best_acc1 = 0.\n for epoch in range(args.epochs):\n print(lr_scheduler.get_lr())\n # train for one epoch\n train(train_source_iter, train_target_iter, classifier, mdd, optimizer,\n lr_scheduler, epoch, args)\n\n # evaluate on validation set\n acc1 = utils.validate(val_loader, classifier, args, device)\n\n # remember best acc@1 and save checkpoint\n torch.save(classifier.state_dict(), logger.get_checkpoint_path('latest'))\n if acc1 > best_acc1:\n shutil.copy(logger.get_checkpoint_path('latest'), logger.get_checkpoint_path('best'))\n best_acc1 = max(acc1, best_acc1)\n\n print(\"best_acc1 = {:3.1f}\".format(best_acc1))\n\n # evaluate on test set\n classifier.load_state_dict(torch.load(logger.get_checkpoint_path('best')))\n acc1 = utils.validate(test_loader, classifier, args, device)\n print(\"test_acc1 = {:3.1f}\".format(acc1))\n\n logger.close()\n\n\ndef train(train_source_iter: ForeverDataIterator, train_target_iter: ForeverDataIterator,\n classifier: ImageClassifier, mdd: MarginDisparityDiscrepancy, optimizer: SGD,\n lr_scheduler: LambdaLR, epoch: int, args: argparse.Namespace):\n batch_time = AverageMeter('Time', ':3.1f')\n data_time = AverageMeter('Data', ':3.1f')\n losses = AverageMeter('Loss', ':3.2f')\n trans_losses = AverageMeter('Trans Loss', ':3.2f')\n cls_accs = AverageMeter('Cls Acc', ':3.1f')\n tgt_accs = AverageMeter('Tgt Acc', ':3.1f')\n\n progress = ProgressMeter(\n args.iters_per_epoch,\n [batch_time, data_time, losses, trans_losses, cls_accs, tgt_accs],\n prefix=\"Epoch: [{}]\".format(epoch))\n\n # switch to train mode\n classifier.train()\n mdd.train()\n\n end = time.time()\n for i in range(args.iters_per_epoch):\n optimizer.zero_grad()\n\n x_s, labels_s = next(train_source_iter)\n x_t, labels_t = next(train_target_iter)\n\n x_s = x_s.to(device)\n x_t = x_t.to(device)\n labels_s = labels_s.to(device)\n labels_t = labels_t.to(device)\n\n # measure data loading time\n data_time.update(time.time() - end)\n\n # compute output\n x = torch.cat((x_s, x_t), dim=0)\n outputs, outputs_adv = classifier(x)\n if len(outputs) > len(x):\n y_s, y_t, y_s1, y_t1 = outputs.chunk(4, dim=0)\n y_s = torch.cat([y_s, y_s1], dim=0)\n y_t = torch.cat([y_t, y_t1], dim=0)\n y_s_adv, y_t_adv, y_s_adv1, y_t_adv1 = outputs_adv.chunk(4, dim=0)\n y_s_adv = torch.cat([y_s_adv, y_s_adv1], dim=0)\n y_t_adv = torch.cat([y_t_adv, y_t_adv1], dim=0)\n else:\n y_s, y_t = outputs.chunk(2, dim=0)\n y_s_adv, y_t_adv = outputs_adv.chunk(2, dim=0)\n if len(y_s) > len(labels_s):\n labels_s = torch.cat([labels_s, labels_s], dim=0)\n if len(y_t) > len(labels_t):\n labels_t = torch.cat([labels_t, labels_t], dim=0)\n # compute cross entropy loss on source domain\n cls_loss = F.cross_entropy(y_s, labels_s)\n # compute margin disparity discrepancy between domains\n # for adversarial classifier, minimize negative mdd is equal to maximize mdd\n transfer_loss = -mdd(y_s, y_s_adv, y_t, y_t_adv)\n loss = cls_loss + transfer_loss * args.trade_off\n classifier.step()\n\n cls_acc = accuracy(y_s, labels_s)[0]\n tgt_acc = accuracy(y_t, labels_t)[0]\n\n losses.update(loss.item(), x_s.size(0))\n cls_accs.update(cls_acc.item(), x_s.size(0))\n tgt_accs.update(tgt_acc.item(), x_t.size(0))\n trans_losses.update(transfer_loss.item(), x_s.size(0))\n\n # compute gradient and do SGD step\n loss.backward()\n optimizer.step()\n lr_scheduler.step()\n\n # measure elapsed time\n batch_time.update(time.time() - end)\n end = time.time()\n\n if i % args.print_freq == 0:\n progress.display(i)\n\n\nif __name__ == '__main__':\n parser = argparse.ArgumentParser(description='MDD for Unsupervised Domain Adaptation')\n # dataset parameters\n parser.add_argument('root', metavar='DIR',\n help='root path of dataset')\n parser.add_argument('-d', '--data', metavar='DATA', default='Office31', choices=utils.get_dataset_names(),\n help='dataset: ' + ' | '.join(utils.get_dataset_names()) +\n ' (default: Office31)')\n parser.add_argument('-s', '--source', help='source domain(s)', nargs='+')\n parser.add_argument('-t', '--target', help='target domain(s)', nargs='+')\n parser.add_argument('--train-resizing', type=str, default='default')\n parser.add_argument('--val-resizing', type=str, default='default')\n parser.add_argument('--resize-size', type=int, default=224,\n help='the image size after resizing')\n parser.add_argument('--no-hflip', action='store_true', help='no random horizontal flipping during training')\n parser.add_argument('--norm-mean', type=float, nargs='+', default=(0.485, 0.456, 0.406), help='normalization mean')\n parser.add_argument('--norm-std', type=float, nargs='+', default=(0.229, 0.224, 0.225), help='normalization std')\n # model parameters\n parser.add_argument('-a', '--arch', metavar='ARCH', default='resnet18',\n choices=utils.get_model_names(),\n help='backbone architecture: ' +\n ' | '.join(utils.get_model_names()) +\n ' (default: resnet18)')\n parser.add_argument('--bottleneck-dim', default=1024, type=int)\n parser.add_argument('--no-pool', action='store_true',\n help='no pool layer after the feature extractor.')\n parser.add_argument('--scratch', action='store_true', help='whether train from scratch.')\n parser.add_argument('--margin', type=float, default=4., help=\"margin gamma\")\n parser.add_argument('--trade-off', default=1., type=float,\n help='the trade-off hyper-parameter for transfer loss')\n # training parameters\n parser.add_argument('-b', '--batch-size', default=32, type=int,\n metavar='N',\n help='mini-batch size (default: 32)')\n parser.add_argument('--lr', '--learning-rate', default=0.004, type=float,\n metavar='LR', help='initial learning rate', dest='lr')\n parser.add_argument('--lr-gamma', default=0.0002, type=float)\n parser.add_argument('--lr-decay', default=0.75, type=float, help='parameter for lr scheduler')\n parser.add_argument('--momentum', default=0.9, type=float, metavar='M', help='momentum')\n parser.add_argument('--wd', '--weight-decay', default=0.0005, type=float,\n metavar='W', help='weight decay (default: 5e-4)')\n parser.add_argument('-j', '--workers', default=2, type=int, metavar='N',\n help='number of data loading workers (default: 4)')\n parser.add_argument('--epochs', default=20, type=int, metavar='N',\n help='number of total epochs to run')\n parser.add_argument('-i', '--iters-per-epoch', default=1000, type=int,\n help='Number of iterations per epoch')\n parser.add_argument('-p', '--print-freq', default=100, type=int,\n metavar='N', help='print frequency (default: 100)')\n parser.add_argument('--seed', default=None, type=int,\n help='seed for initializing training. ')\n parser.add_argument('--per-class-eval', action='store_true',\n help='whether output per-class accuracy during evaluation')\n parser.add_argument(\"--log\", type=str, default='mdd',\n help=\"Where to save logs, checkpoints and debugging images.\")\n parser.add_argument(\"--phase\", type=str, default='train', choices=['train', 'test', 'analysis'],\n help=\"When phase is 'test', only test the model.\"\n \"When phase is 'analysis', only analysis the model.\")\n parser.add_argument(\"--add_bt\", type=int, default=0,\n help=\"add_bt\")\n args = parser.parse_args()\n main(args)\n\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/examples/domain_adaptation/image_classification/mdd.sh",
"content": "#!/usr/bin/env bash\n# ResNet50, Office31, Single Source\nCUDA_VISIBLE_DEVICES=0 python mdd.py data/office31 -d Office31 -s A -t W -a resnet50 --epochs 20 --bottleneck-dim 1024 --seed 1 --log logs/mdd/Office31_A2W\nCUDA_VISIBLE_DEVICES=0 python mdd.py data/office31 -d Office31 -s D -t W -a resnet50 --epochs 20 --bottleneck-dim 1024 --seed 1 --log logs/mdd/Office31_D2W\nCUDA_VISIBLE_DEVICES=0 python mdd.py data/office31 -d Office31 -s W -t D -a resnet50 --epochs 20 --bottleneck-dim 1024 --seed 1 --log logs/mdd/Office31_W2D\nCUDA_VISIBLE_DEVICES=0 python mdd.py data/office31 -d Office31 -s A -t D -a resnet50 --epochs 20 --bottleneck-dim 1024 --seed 1 --log logs/mdd/Office31_A2D\nCUDA_VISIBLE_DEVICES=0 python mdd.py data/office31 -d Office31 -s D -t A -a resnet50 --epochs 20 --bottleneck-dim 1024 --seed 1 --log logs/mdd/Office31_D2A\nCUDA_VISIBLE_DEVICES=0 python mdd.py data/office31 -d Office31 -s W -t A -a resnet50 --epochs 20 --bottleneck-dim 1024 --seed 1 --log logs/mdd/Office31_W2A\n\n# ResNet50, Office-Home, Single Source\nCUDA_VISIBLE_DEVICES=0 python mdd.py data/office-home -d OfficeHome -s Ar -t Cl -a resnet50 --epochs 30 --bottleneck-dim 2048 --seed 0 --log logs/mdd/OfficeHome_Ar2Cl\nCUDA_VISIBLE_DEVICES=0 python mdd.py data/office-home -d OfficeHome -s Ar -t Pr -a resnet50 --epochs 30 --bottleneck-dim 2048 --seed 0 --log logs/mdd/OfficeHome_Ar2Pr\nCUDA_VISIBLE_DEVICES=0 python mdd.py data/office-home -d OfficeHome -s Ar -t Rw -a resnet50 --epochs 30 --bottleneck-dim 2048 --seed 0 --log logs/mdd/OfficeHome_Ar2Rw\nCUDA_VISIBLE_DEVICES=0 python mdd.py data/office-home -d OfficeHome -s Cl -t Ar -a resnet50 --epochs 30 --bottleneck-dim 2048 --seed 0 --log logs/mdd/OfficeHome_Cl2Ar\nCUDA_VISIBLE_DEVICES=0 python mdd.py data/office-home -d OfficeHome -s Cl -t Pr -a resnet50 --epochs 30 --bottleneck-dim 2048 --seed 0 --log logs/mdd/OfficeHome_Cl2Pr\nCUDA_VISIBLE_DEVICES=0 python mdd.py data/office-home -d OfficeHome -s Cl -t Rw -a resnet50 --epochs 30 --bottleneck-dim 2048 --seed 0 --log logs/mdd/OfficeHome_Cl2Rw\nCUDA_VISIBLE_DEVICES=0 python mdd.py data/office-home -d OfficeHome -s Pr -t Ar -a resnet50 --epochs 30 --bottleneck-dim 2048 --seed 0 --log logs/mdd/OfficeHome_Pr2Ar\nCUDA_VISIBLE_DEVICES=0 python mdd.py data/office-home -d OfficeHome -s Pr -t Cl -a resnet50 --epochs 30 --bottleneck-dim 2048 --seed 0 --log logs/mdd/OfficeHome_Pr2Cl\nCUDA_VISIBLE_DEVICES=0 python mdd.py data/office-home -d OfficeHome -s Pr -t Rw -a resnet50 --epochs 30 --bottleneck-dim 2048 --seed 0 --log logs/mdd/OfficeHome_Pr2Rw\nCUDA_VISIBLE_DEVICES=0 python mdd.py data/office-home -d OfficeHome -s Rw -t Ar -a resnet50 --epochs 30 --bottleneck-dim 2048 --seed 0 --log logs/mdd/OfficeHome_Rw2Ar\nCUDA_VISIBLE_DEVICES=0 python mdd.py data/office-home -d OfficeHome -s Rw -t Cl -a resnet50 --epochs 30 --bottleneck-dim 2048 --seed 0 --log logs/mdd/OfficeHome_Rw2Cl\nCUDA_VISIBLE_DEVICES=0 python mdd.py data/office-home -d OfficeHome -s Rw -t Pr -a resnet50 --epochs 30 --bottleneck-dim 2048 --seed 0 --log logs/mdd/OfficeHome_Rw2Pr\n\n# ResNet101, VisDA-2017, Single Source\nCUDA_VISIBLE_DEVICES=0 python mdd.py data/visda-2017 -d VisDA2017 -s Synthetic -t Real -a resnet101 --epochs 30 \\\n --bottleneck-dim 1024 --seed 0 --train-resizing cen.crop --per-class-eval -b 36 --log logs/mdd/VisDA2017\n\n# ResNet101, DomainNet, Single Source\nCUDA_VISIBLE_DEVICES=0 python mdd.py data/domainnet -d DomainNet -s c -t p -a resnet101 --epochs 40 -i 5000 -p 500 --bottleneck-dim 2048 --seed 0 --lr 0.004 --log logs/mdd/DomainNet_c2p\nCUDA_VISIBLE_DEVICES=0 python mdd.py data/domainnet -d DomainNet -s c -t r -a resnet101 --epochs 40 -i 5000 -p 500 --bottleneck-dim 2048 --seed 0 --lr 0.004 --log logs/mdd/DomainNet_c2r\nCUDA_VISIBLE_DEVICES=0 python mdd.py data/domainnet -d DomainNet -s c -t s -a resnet101 --epochs 40 -i 5000 -p 500 --bottleneck-dim 2048 --seed 0 --lr 0.004 --log logs/mdd/DomainNet_c2s\nCUDA_VISIBLE_DEVICES=0 python mdd.py data/domainnet -d DomainNet -s p -t c -a resnet101 --epochs 40 -i 5000 -p 500 --bottleneck-dim 2048 --seed 0 --lr 0.004 --log logs/mdd/DomainNet_p2c\nCUDA_VISIBLE_DEVICES=0 python mdd.py data/domainnet -d DomainNet -s p -t r -a resnet101 --epochs 40 -i 5000 -p 500 --bottleneck-dim 2048 --seed 0 --lr 0.004 --log logs/mdd/DomainNet_p2r\nCUDA_VISIBLE_DEVICES=0 python mdd.py data/domainnet -d DomainNet -s p -t s -a resnet101 --epochs 40 -i 5000 -p 500 --bottleneck-dim 2048 --seed 0 --lr 0.004 --log logs/mdd/DomainNet_p2s\nCUDA_VISIBLE_DEVICES=0 python mdd.py data/domainnet -d DomainNet -s r -t c -a resnet101 --epochs 40 -i 5000 -p 500 --bottleneck-dim 2048 --seed 0 --lr 0.004 --log logs/mdd/DomainNet_r2c\nCUDA_VISIBLE_DEVICES=0 python mdd.py data/domainnet -d DomainNet -s r -t p -a resnet101 --epochs 40 -i 5000 -p 500 --bottleneck-dim 2048 --seed 0 --lr 0.004 --log logs/mdd/DomainNet_r2p\nCUDA_VISIBLE_DEVICES=0 python mdd.py data/domainnet -d DomainNet -s r -t s -a resnet101 --epochs 40 -i 5000 -p 500 --bottleneck-dim 2048 --seed 0 --lr 0.004 --log logs/mdd/DomainNet_r2s\nCUDA_VISIBLE_DEVICES=0 python mdd.py data/domainnet -d DomainNet -s s -t c -a resnet101 --epochs 40 -i 5000 -p 500 --bottleneck-dim 2048 --seed 0 --lr 0.004 --log logs/mdd/DomainNet_s2c\nCUDA_VISIBLE_DEVICES=0 python mdd.py data/domainnet -d DomainNet -s s -t p -a resnet101 --epochs 40 -i 5000 -p 500 --bottleneck-dim 2048 --seed 0 --lr 0.004 --log logs/mdd/DomainNet_s2p\nCUDA_VISIBLE_DEVICES=0 python mdd.py data/domainnet -d DomainNet -s s -t r -a resnet101 --epochs 40 -i 5000 -p 500 --bottleneck-dim 2048 --seed 0 --lr 0.004 --log logs/mdd/DomainNet_s2r\n\n# ResNet50, ImageNet200 -> ImageNetR\nCUDA_VISIBLE_DEVICES=0 python mdd.py data/ImageNetR -d ImageNetR -s IN -t INR -a resnet50 --epochs 40 -i 2500 -p 500 \\\n --bottleneck-dim 2048 --seed 0 --lr 0.004 --train-resizing cen.crop --log logs/mdd/ImageNet_IN2INR\n\n# ig_resnext101_32x8d, ImageNet -> ImageNetSketch\nCUDA_VISIBLE_DEVICES=0 python mdd.py data/imagenet-sketch -d ImageNetSketch -s IN -t sketch -a ig_resnext101_32x8d \\\n --epochs 40 -i 2500 -p 500 --bottleneck-dim 2048 --margin 2. --seed 0 --lr 0.004 --train-resizing cen.crop \\\n --log logs/mdd_ig_resnext101_32x8d/ImageNet_IN2sketch\n\n# Vision Transformer, Office-Home, Single Source\nCUDA_VISIBLE_DEVICES=0 python mdd.py data/office-home -d OfficeHome -s Ar -t Cl -a vit_base_patch16_224 --epochs 40 --bottleneck-dim 2048 --seed 0 -b 24 --no-pool --log logs/mdd_vit/OfficeHome_Ar2Cl\nCUDA_VISIBLE_DEVICES=0 python mdd.py data/office-home -d OfficeHome -s Ar -t Pr -a vit_base_patch16_224 --epochs 40 --bottleneck-dim 2048 --seed 0 -b 24 --no-pool --log logs/mdd_vit/OfficeHome_Ar2Pr\nCUDA_VISIBLE_DEVICES=0 python mdd.py data/office-home -d OfficeHome -s Ar -t Rw -a vit_base_patch16_224 --epochs 40 --bottleneck-dim 2048 --seed 0 -b 24 --no-pool --log logs/mdd_vit/OfficeHome_Ar2Rw\nCUDA_VISIBLE_DEVICES=0 python mdd.py data/office-home -d OfficeHome -s Cl -t Ar -a vit_base_patch16_224 --epochs 40 --bottleneck-dim 2048 --seed 0 -b 24 --no-pool --log logs/mdd_vit/OfficeHome_Cl2Ar\nCUDA_VISIBLE_DEVICES=0 python mdd.py data/office-home -d OfficeHome -s Cl -t Pr -a vit_base_patch16_224 --epochs 40 --bottleneck-dim 2048 --seed 0 -b 24 --no-pool --log logs/mdd_vit/OfficeHome_Cl2Pr\nCUDA_VISIBLE_DEVICES=0 python mdd.py data/office-home -d OfficeHome -s Cl -t Rw -a vit_base_patch16_224 --epochs 40 --bottleneck-dim 2048 --seed 0 -b 24 --no-pool --log logs/mdd_vit/OfficeHome_Cl2Rw\nCUDA_VISIBLE_DEVICES=0 python mdd.py data/office-home -d OfficeHome -s Pr -t Ar -a vit_base_patch16_224 --epochs 40 --bottleneck-dim 2048 --seed 0 -b 24 --no-pool --log logs/mdd_vit/OfficeHome_Pr2Ar\nCUDA_VISIBLE_DEVICES=0 python mdd.py data/office-home -d OfficeHome -s Pr -t Cl -a vit_base_patch16_224 --epochs 40 --bottleneck-dim 2048 --seed 0 -b 24 --no-pool --log logs/mdd_vit/OfficeHome_Pr2Cl\nCUDA_VISIBLE_DEVICES=0 python mdd.py data/office-home -d OfficeHome -s Pr -t Rw -a vit_base_patch16_224 --epochs 40 --bottleneck-dim 2048 --seed 0 -b 24 --no-pool --log logs/mdd_vit/OfficeHome_Pr2Rw\nCUDA_VISIBLE_DEVICES=0 python mdd.py data/office-home -d OfficeHome -s Rw -t Ar -a vit_base_patch16_224 --epochs 40 --bottleneck-dim 2048 --seed 0 -b 24 --no-pool --log logs/mdd_vit/OfficeHome_Rw2Ar\nCUDA_VISIBLE_DEVICES=0 python mdd.py data/office-home -d OfficeHome -s Rw -t Cl -a vit_base_patch16_224 --epochs 40 --bottleneck-dim 2048 --seed 0 -b 24 --no-pool --log logs/mdd_vit/OfficeHome_Rw2Cl\nCUDA_VISIBLE_DEVICES=0 python mdd.py data/office-home -d OfficeHome -s Rw -t Pr -a vit_base_patch16_224 --epochs 40 --bottleneck-dim 2048 --seed 0 -b 24 --no-pool --log logs/mdd_vit/OfficeHome_Rw2Pr\n\n# ResNet50, Office-Home, Multi Source\nCUDA_VISIBLE_DEVICES=0 python mdd.py data/office-home -d OfficeHome -s Cl Pr Rw -t Ar -a resnet50 --epochs 30 --bottleneck-dim 2048 --seed 0 --log logs/mdd/OfficeHome_:2Ar\nCUDA_VISIBLE_DEVICES=0 python mdd.py data/office-home -d OfficeHome -s Ar Pr Rw -t Cl -a resnet50 --epochs 30 --bottleneck-dim 2048 --seed 0 --log logs/mdd/OfficeHome_:2Cl\nCUDA_VISIBLE_DEVICES=0 python mdd.py data/office-home -d OfficeHome -s Ar Cl Rw -t Pr -a resnet50 --epochs 30 --bottleneck-dim 2048 --seed 0 --log logs/mdd/OfficeHome_:2Pr\nCUDA_VISIBLE_DEVICES=0 python mdd.py data/office-home -d OfficeHome -s Ar Cl Pr -t Rw -a resnet50 --epochs 30 --bottleneck-dim 2048 --seed 0 --log logs/mdd/OfficeHome_:2Rw\n\n# ResNet101, DomainNet, Multi Source\nCUDA_VISIBLE_DEVICES=0 python mdd.py data/domainnet -d DomainNet -s i p q r s -t c -a resnet101 --epochs 40 -i 5000 -p 500 --bottleneck-dim 2048 --seed 0 --lr 0.004 --log logs/mdd/DomainNet_:2c\nCUDA_VISIBLE_DEVICES=0 python mdd.py data/domainnet -d DomainNet -s c p q r s -t i -a resnet101 --epochs 40 -i 5000 -p 500 --bottleneck-dim 2048 --seed 0 --lr 0.004 --log logs/mdd/DomainNet_:2i\nCUDA_VISIBLE_DEVICES=0 python mdd.py data/domainnet -d DomainNet -s c i q r s -t p -a resnet101 --epochs 40 -i 5000 -p 500 --bottleneck-dim 2048 --seed 0 --lr 0.004 --log logs/mdd/DomainNet_:2p\nCUDA_VISIBLE_DEVICES=0 python mdd.py data/domainnet -d DomainNet -s c i p r s -t q -a resnet101 --epochs 40 -i 5000 -p 500 --bottleneck-dim 2048 --seed 0 --lr 0.004 --log logs/mdd/DomainNet_:2q\nCUDA_VISIBLE_DEVICES=0 python mdd.py data/domainnet -d DomainNet -s c i p q s -t r -a resnet101 --epochs 40 -i 5000 -p 500 --bottleneck-dim 2048 --seed 0 --lr 0.004 --log logs/mdd/DomainNet_:2r\nCUDA_VISIBLE_DEVICES=0 python mdd.py data/domainnet -d DomainNet -s c i p q r -t s -a resnet101 --epochs 40 -i 5000 -p 500 --bottleneck-dim 2048 --seed 0 --lr 0.004 --log logs/mdd/DomainNet_:2s\n\n# ResNet50, Wilds Dataset\nCUDA_VISIBLE_DEVICES=0 python mdd.py data/wilds -d iwildcam --train-resizing 'res.' --val-resizing 'res.' \\\n -a resnet50 --bottleneck-dim 2048 --epochs 30 -i 1000 --seed 0 --log logs/mdd/iwildcam\n\n# DenseNet121, Wilds Dataset\nCUDA_VISIBLE_DEVICES=0 python mdd.py data/wilds -d camelyon17 --train-resizing 'res.' --val-resizing 'res.' --resize-size 96 \\\n -a densenet121 --scratch --epochs 30 -i 1000 --lr 0.01 --seed 0 --log logs/mdd/camelyon17\nCUDA_VISIBLE_DEVICES=0 python mdd.py data/wilds -d fmow --train-resizing 'res.' --val-resizing 'res.' \\\n -a densenet121 --bottleneck-dim 2048 --epochs 30 -i 1000 --lr 0.01 --seed 0 --log logs/mdd/fmow\n\n# Digits\nCUDA_VISIBLE_DEVICES=0 python mdd.py data/digits -d Digits -s MNIST -t USPS --train-resizing 'res.' --val-resizing 'res.' \\\n --resize-size 28 --no-hflip --norm-mean 0.5 --norm-std 0.5 -a lenet --no-pool --lr 0.01 -b 128 -i 2500 --scratch --seed 0 --log logs/mdd/MNIST2USPS\nCUDA_VISIBLE_DEVICES=0 python mdd.py data/digits -d Digits -s USPS -t MNIST --train-resizing 'res.' --val-resizing 'res.' \\\n --resize-size 28 --no-hflip --norm-mean 0.5 --norm-std 0.5 -a lenet --no-pool --lr 0.01 -b 128 -i 2500 --scratch --seed 0 --log logs/mdd/USPS2MNIST\nCUDA_VISIBLE_DEVICES=0 python mdd.py data/digits -d Digits -s SVHNRGB -t MNISTRGB --train-resizing 'res.' --val-resizing 'res.' \\\n --resize-size 32 --no-hflip --norm-mean 0.5 0.5 0.5 --norm-std 0.5 0.5 0.5 -a dtn --no-pool --lr 0.01 -b 128 -i 2500 --scratch --seed 0 --log logs/mdd/SVHN2MNIST\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/examples/domain_adaptation/image_classification/requirements.txt",
"content": "wilds\ntimm"
},
{
"path": "domain_generalization/Transfer-Learning-Library/examples/domain_adaptation/image_classification/self_ensemble.py",
"content": "\"\"\"\n@author: Baixu Chen\n@contact: cbx_99_hasta@outlook.com\n\"\"\"\nimport random\nimport time\nimport warnings\nimport sys\nimport argparse\nimport shutil\nimport os.path as osp\n\nimport torch\nimport torch.nn as nn\nimport torch.backends.cudnn as cudnn\nfrom torch.optim import Adam\nfrom torch.optim.lr_scheduler import LambdaLR\nfrom torch.utils.data import DataLoader\nimport torchvision.transforms as T\nimport torch.nn.functional as F\n\nsys.path.append('../../..')\nfrom dalib.adaptation.self_ensemble import EmaTeacher, L2ConsistencyLoss, ClassBalanceLoss, ImageClassifier\nfrom common.vision.transforms import ResizeImage, MultipleApply\nfrom common.utils.data import ForeverDataIterator\nfrom common.utils.metric import accuracy\nfrom common.utils.meter import AverageMeter, ProgressMeter\nfrom common.utils.logger import CompleteLogger\nfrom common.utils.analysis import collect_feature, tsne, a_distance\n\nsys.path.append('.')\nimport utils\n\ndevice = torch.device(\"cuda\" if torch.cuda.is_available() else \"cpu\")\n\n\ndef main(args: argparse.Namespace):\n logger = CompleteLogger(args.log, args.phase)\n print(args)\n\n if args.seed is not None:\n random.seed(args.seed)\n torch.manual_seed(args.seed)\n cudnn.deterministic = True\n warnings.warn('You have chosen to seed training. '\n 'This will turn on the CUDNN deterministic setting, '\n 'which can slow down your training considerably! '\n 'You may see unexpected behavior when restarting '\n 'from checkpoints.')\n\n cudnn.benchmark = True\n\n # Data loading code\n normalize = T.Normalize(mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225])\n train_transform = T.Compose([\n ResizeImage(256),\n T.RandomCrop(224),\n T.RandomHorizontalFlip(),\n T.ColorJitter(brightness=0.7, contrast=0.7, saturation=0.7, hue=0.5),\n T.RandomGrayscale(),\n T.ToTensor(),\n normalize\n ])\n val_transform = T.Compose([\n ResizeImage(256),\n T.CenterCrop(224),\n T.ToTensor(),\n normalize\n ])\n\n train_source_dataset, train_target_dataset, val_dataset, test_dataset, num_classes, args.class_names = \\\n utils.get_dataset(args.data, args.root, args.source, args.target,\n train_transform, val_transform, MultipleApply([train_transform, val_transform]))\n train_source_loader = DataLoader(train_source_dataset, batch_size=args.batch_size,\n shuffle=True, num_workers=args.workers, drop_last=True)\n train_target_loader = DataLoader(train_target_dataset, batch_size=args.batch_size,\n shuffle=True, num_workers=args.workers, drop_last=True)\n val_loader = DataLoader(val_dataset, batch_size=args.batch_size, shuffle=False, num_workers=args.workers)\n test_loader = DataLoader(test_dataset, batch_size=args.batch_size, shuffle=False, num_workers=args.workers)\n\n train_source_iter = ForeverDataIterator(train_source_loader)\n train_target_iter = ForeverDataIterator(train_target_loader)\n\n # create model\n print(\"=> using model '{}'\".format(args.arch))\n backbone = utils.get_model(args.arch, pretrain=not args.scratch)\n pool_layer = nn.Identity() if args.no_pool else None\n classifier = ImageClassifier(backbone, num_classes, bottleneck_dim=args.bottleneck_dim,\n pool_layer=pool_layer, finetune=not args.scratch).to(device)\n\n # define optimizer and lr scheduler\n optimizer = Adam(classifier.get_parameters(), args.lr)\n lr_scheduler = LambdaLR(optimizer, lambda x: args.lr * (1. + args.lr_gamma * float(x)) ** (-args.lr_decay))\n\n # resume from the best checkpoint\n if args.phase != 'train':\n checkpoint = torch.load(logger.get_checkpoint_path('best'), map_location='cpu')\n classifier.load_state_dict(checkpoint)\n\n # analysis the model\n if args.phase == 'analysis':\n # extract features from both domains\n feature_extractor = nn.Sequential(classifier.backbone, classifier.pool_layer, classifier.bottleneck).to(device)\n source_feature = collect_feature(train_source_loader, feature_extractor, device)\n target_feature = collect_feature(train_target_loader, feature_extractor, device)\n # plot t-SNE\n tSNE_filename = osp.join(logger.visualize_directory, 'TSNE.pdf')\n tsne.visualize(source_feature, target_feature, tSNE_filename)\n print(\"Saving t-SNE to\", tSNE_filename)\n # calculate A-distance, which is a measure for distribution discrepancy\n A_distance = a_distance.calculate(source_feature, target_feature, device)\n print(\"A-distance =\", A_distance)\n return\n\n if args.phase == 'test':\n acc1 = utils.validate(test_loader, classifier, args, device)\n print(acc1)\n return\n\n if args.pretrain is None:\n # first pretrain the classifier wish source data\n print(\"Pretraining the model on source domain.\")\n args.pretrain = logger.get_checkpoint_path('pretrain')\n pretrain_model = ImageClassifier(backbone, num_classes, bottleneck_dim=args.bottleneck_dim,\n pool_layer=pool_layer, finetune=not args.scratch).to(device)\n pretrain_optimizer = Adam(pretrain_model.get_parameters(), args.pretrain_lr)\n pretrain_lr_scheduler = LambdaLR(pretrain_optimizer,\n lambda x: args.pretrain_lr * (1. + args.lr_gamma * float(x)) ** (\n -args.lr_decay))\n\n # start pretraining\n for epoch in range(args.pretrain_epochs):\n # pretrain for one epoch\n utils.pretrain(train_source_iter, pretrain_model, pretrain_optimizer, pretrain_lr_scheduler, epoch, args,\n device)\n # validate to show pretrain process\n utils.validate(val_loader, pretrain_model, args, device)\n\n torch.save(pretrain_model.state_dict(), args.pretrain)\n print(\"Pretraining process is done.\")\n\n checkpoint = torch.load(args.pretrain, map_location='cpu')\n classifier.load_state_dict(checkpoint)\n teacher = EmaTeacher(classifier, alpha=args.alpha)\n consistent_loss = L2ConsistencyLoss().to(device)\n class_balance_loss = ClassBalanceLoss(num_classes).to(device)\n\n # start training\n best_acc1 = 0.\n for epoch in range(args.epochs):\n print(lr_scheduler.get_lr())\n # train for one epoch\n train(train_source_iter, train_target_iter, classifier, teacher, consistent_loss, class_balance_loss, optimizer,\n lr_scheduler, epoch, args)\n\n # evaluate on validation set\n acc1 = utils.validate(val_loader, classifier, args, device)\n\n # remember best acc@1 and save checkpoint\n torch.save(classifier.state_dict(), logger.get_checkpoint_path('latest'))\n if acc1 > best_acc1:\n shutil.copy(logger.get_checkpoint_path('latest'), logger.get_checkpoint_path('best'))\n best_acc1 = max(acc1, best_acc1)\n\n print(\"best_acc1 = {:3.1f}\".format(best_acc1))\n\n # evaluate on test set\n classifier.load_state_dict(torch.load(logger.get_checkpoint_path('best')))\n acc1 = utils.validate(test_loader, classifier, args, device)\n print(\"test_acc1 = {:3.1f}\".format(acc1))\n\n logger.close()\n\n\ndef train(train_source_iter: ForeverDataIterator, train_target_iter: ForeverDataIterator, model: ImageClassifier,\n teacher: EmaTeacher, consistent_loss, class_balance_loss,\n optimizer: Adam, lr_scheduler: LambdaLR, epoch: int, args: argparse.Namespace):\n batch_time = AverageMeter('Time', ':3.1f')\n data_time = AverageMeter('Data', ':3.1f')\n cls_losses = AverageMeter('Cls Loss', ':3.2f')\n cons_losses = AverageMeter('Cons Loss', ':3.2f')\n cls_accs = AverageMeter('Cls Acc', ':3.1f')\n tgt_accs = AverageMeter('Tgt Acc', ':3.1f')\n\n progress = ProgressMeter(\n args.iters_per_epoch,\n [batch_time, data_time, cls_losses, cons_losses, cls_accs, tgt_accs],\n prefix=\"Epoch: [{}]\".format(epoch))\n\n # switch to train mode\n model.train()\n teacher.train()\n\n end = time.time()\n for i in range(args.iters_per_epoch):\n x_s, labels_s = next(train_source_iter)\n (x_t1, x_t2), labels_t = next(train_target_iter)\n\n x_s = x_s.to(device)\n x_t1 = x_t1.to(device)\n x_t2 = x_t2.to(device)\n labels_s = labels_s.to(device)\n labels_t = labels_t.to(device)\n\n # measure data loading time\n data_time.update(time.time() - end)\n\n # compute output\n y_s, _ = model(x_s)\n y_t, _ = model(x_t1)\n y_t_teacher, _ = teacher(x_t2)\n\n # classification loss\n cls_loss = F.cross_entropy(y_s, labels_s)\n # compute output and mask\n y_t = F.softmax(y_t, dim=1)\n y_t_teacher = F.softmax(y_t_teacher, dim=1)\n max_prob, _ = y_t_teacher.max(dim=1)\n mask = (max_prob > args.threshold).float()\n\n # consistent loss\n cons_loss = consistent_loss(y_t, y_t_teacher, mask)\n # balance loss\n balance_loss = class_balance_loss(y_t) * mask.mean()\n\n loss = cls_loss + args.trade_off_cons * cons_loss + args.trade_off_balance * balance_loss\n\n # compute gradient and do SGD step\n optimizer.zero_grad()\n loss.backward()\n optimizer.step()\n lr_scheduler.step()\n\n # update teacher\n teacher.update()\n\n # update statistics\n cls_acc = accuracy(y_s, labels_s)[0]\n tgt_acc = accuracy(y_t, labels_t)[0]\n cls_losses.update(cls_loss.item(), x_s.size(0))\n cons_losses.update(cons_loss.item(), x_s.size(0))\n cls_accs.update(cls_acc.item(), x_s.size(0))\n tgt_accs.update(tgt_acc.item(), x_s.size(0))\n\n # measure elapsed time\n batch_time.update(time.time() - end)\n end = time.time()\n\n if i % args.print_freq == 0:\n progress.display(i)\n\n\nif __name__ == '__main__':\n parser = argparse.ArgumentParser(description='Self Ensemble for Unsupervised Domain Adaptation')\n # dataset parameters\n parser.add_argument('root', metavar='DIR',\n help='root path of dataset')\n parser.add_argument('-d', '--data', metavar='DATA', default='Office31', choices=utils.get_dataset_names(),\n help='dataset: ' + ' | '.join(utils.get_dataset_names()) +\n ' (default: Office31)')\n parser.add_argument('-s', '--source', help='source domain(s)', nargs='+')\n parser.add_argument('-t', '--target', help='target domain(s)', nargs='+')\n parser.add_argument('--train-resizing', type=str, default='default')\n parser.add_argument('--val-resizing', type=str, default='default')\n parser.add_argument('--resize-size', type=int, default=224,\n help='the image size after resizing')\n parser.add_argument('--no-hflip', action='store_true',\n help='no random horizontal flipping during training')\n parser.add_argument('--norm-mean', type=float, nargs='+',\n default=(0.485, 0.456, 0.406), help='normalization mean')\n parser.add_argument('--norm-std', type=float, nargs='+',\n default=(0.229, 0.224, 0.225), help='normalization std')\n # model parameters\n parser.add_argument('-a', '--arch', metavar='ARCH', default='resnet18',\n choices=utils.get_model_names(),\n help='backbone architecture: ' +\n ' | '.join(utils.get_model_names()) +\n ' (default: resnet18)')\n parser.add_argument('--pretrain', type=str, default=None,\n help='pretrain checkpoint for classification model')\n parser.add_argument('--bottleneck-dim', default=256, type=int,\n help='Dimension of bottleneck')\n parser.add_argument('--no-pool', action='store_true',\n help='no pool layer after the feature extractor.')\n parser.add_argument('--scratch', action='store_true', help='whether train from scratch.')\n # training parameters\n parser.add_argument('-b', '--batch-size', default=36, type=int,\n metavar='N',\n help='mini-batch size (default: 36)')\n parser.add_argument('--lr', '--learning-rate', default=1e-4, type=float,\n metavar='LR', help='initial learning rate', dest='lr')\n parser.add_argument('--pretrain-lr', '--pretrain-learning-rate', default=3e-5, type=float,\n help='initial pretrain learning rate', dest='pretrain_lr')\n parser.add_argument('--lr-gamma', default=0.001, type=float, help='parameter for lr scheduler')\n parser.add_argument('--lr-decay', default=0.75, type=float, help='parameter for lr scheduler')\n parser.add_argument('--alpha', default=0.99, type=float, help='ema decay rate (default: 0.99)')\n parser.add_argument('--threshold', default=0.8, type=float, help='confidence threshold')\n parser.add_argument('--trade-off-cons', default=3, type=float, help='trade off parameter for consistent loss')\n parser.add_argument('--trade-off-balance', default=0.01, type=float,\n help='trade off parameter for class balance loss')\n parser.add_argument('-j', '--workers', default=2, type=int, metavar='N',\n help='number of data loading workers (default: 2)')\n parser.add_argument('--pretrain-epochs', default=0, type=int, metavar='N',\n help='number of total epochs(pretrain) to run')\n parser.add_argument('--epochs', default=10, type=int, metavar='N',\n help='number of total epochs to run')\n parser.add_argument('-i', '--iters-per-epoch', default=1000, type=int,\n help='Number of iterations per epoch')\n parser.add_argument('-p', '--print-freq', default=100, type=int,\n metavar='N', help='print frequency (default: 100)')\n parser.add_argument('--seed', default=None, type=int,\n help='seed for initializing training. ')\n parser.add_argument('--per-class-eval', action='store_true',\n help='whether output per-class accuracy during evaluation')\n parser.add_argument(\"--log\", type=str, default='self_ensemble',\n help=\"Where to save logs, checkpoints and debugging images.\")\n parser.add_argument(\"--phase\", type=str, default='train', choices=['train', 'test', 'analysis'],\n help=\"When phase is 'test', only test the model.\"\n \"When phase is 'analysis', only analysis the model.\")\n args = parser.parse_args()\n main(args)\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/examples/domain_adaptation/image_classification/self_ensemble.sh",
"content": "#!/usr/bin/env bash\n# ResNet50, Office31, Single Source\nCUDA_VISIBLE_DEVICES=0 python self_ensemble.py data/office31 -d Office31 -s A -t W -a resnet50 --seed 1 --log logs/self_ensemble/Office31_A2W\nCUDA_VISIBLE_DEVICES=0 python self_ensemble.py data/office31 -d Office31 -s D -t W -a resnet50 --seed 1 --log logs/self_ensemble/Office31_D2W\nCUDA_VISIBLE_DEVICES=0 python self_ensemble.py data/office31 -d Office31 -s W -t D -a resnet50 --seed 1 --log logs/self_ensemble/Office31_W2D\nCUDA_VISIBLE_DEVICES=0 python self_ensemble.py data/office31 -d Office31 -s A -t D -a resnet50 --seed 1 --log logs/self_ensemble/Office31_A2D\nCUDA_VISIBLE_DEVICES=0 python self_ensemble.py data/office31 -d Office31 -s D -t A -a resnet50 --seed 1 --log logs/self_ensemble/Office31_D2A\nCUDA_VISIBLE_DEVICES=0 python self_ensemble.py data/office31 -d Office31 -s W -t A -a resnet50 --seed 1 --log logs/self_ensemble/Office31_W2A\n\n# ResNet50, Office-Home, Single Source\nCUDA_VISIBLE_DEVICES=0 python self_ensemble.py data/office-home -d OfficeHome -s Ar -t Cl -a resnet50 --seed 0 --log logs/self_ensemble/OfficeHome_Ar2Cl\nCUDA_VISIBLE_DEVICES=0 python self_ensemble.py data/office-home -d OfficeHome -s Ar -t Pr -a resnet50 --seed 0 --log logs/self_ensemble/OfficeHome_Ar2Pr\nCUDA_VISIBLE_DEVICES=0 python self_ensemble.py data/office-home -d OfficeHome -s Ar -t Rw -a resnet50 --seed 0 --log logs/self_ensemble/OfficeHome_Ar2Rw\nCUDA_VISIBLE_DEVICES=0 python self_ensemble.py data/office-home -d OfficeHome -s Cl -t Ar -a resnet50 --seed 0 --log logs/self_ensemble/OfficeHome_Cl2Ar\nCUDA_VISIBLE_DEVICES=0 python self_ensemble.py data/office-home -d OfficeHome -s Cl -t Pr -a resnet50 --seed 0 --log logs/self_ensemble/OfficeHome_Cl2Pr\nCUDA_VISIBLE_DEVICES=0 python self_ensemble.py data/office-home -d OfficeHome -s Cl -t Rw -a resnet50 --seed 0 --log logs/self_ensemble/OfficeHome_Cl2Rw\nCUDA_VISIBLE_DEVICES=0 python self_ensemble.py data/office-home -d OfficeHome -s Pr -t Ar -a resnet50 --seed 0 --log logs/self_ensemble/OfficeHome_Pr2Ar\nCUDA_VISIBLE_DEVICES=0 python self_ensemble.py data/office-home -d OfficeHome -s Pr -t Cl -a resnet50 --seed 0 --log logs/self_ensemble/OfficeHome_Pr2Cl\nCUDA_VISIBLE_DEVICES=0 python self_ensemble.py data/office-home -d OfficeHome -s Pr -t Rw -a resnet50 --seed 0 --log logs/self_ensemble/OfficeHome_Pr2Rw\nCUDA_VISIBLE_DEVICES=0 python self_ensemble.py data/office-home -d OfficeHome -s Rw -t Ar -a resnet50 --seed 0 --log logs/self_ensemble/OfficeHome_Rw2Ar\nCUDA_VISIBLE_DEVICES=0 python self_ensemble.py data/office-home -d OfficeHome -s Rw -t Cl -a resnet50 --seed 0 --log logs/self_ensemble/OfficeHome_Rw2Cl\nCUDA_VISIBLE_DEVICES=0 python self_ensemble.py data/office-home -d OfficeHome -s Rw -t Pr -a resnet50 --seed 0 --log logs/self_ensemble/OfficeHome_Rw2Pr\n\n# ResNet101, VisDA-2017, Single Source\nCUDA_VISIBLE_DEVICES=0 python self_ensemble.py data/visda-2017 -d VisDA2017 -s Synthetic -t Real -a resnet101 \\\n --epochs 20 --seed 0 --per-class-eval --log logs/self_ensemble/VisDA2017 --lr-gamma 0.0002 -b 32\n\n# Office-Home on Vision Transformer\nCUDA_VISIBLE_DEVICES=0 python self_ensemble.py data/office-home -d OfficeHome -s Ar -t Cl -a vit_base_patch16_224 --no-pool --epochs 30 --seed 0 -b 24 --log logs/self_ensemble_vit/OfficeHome_Ar2Cl\nCUDA_VISIBLE_DEVICES=0 python self_ensemble.py data/office-home -d OfficeHome -s Ar -t Pr -a vit_base_patch16_224 --no-pool --epochs 30 --seed 0 -b 24 --log logs/self_ensemble_vit/OfficeHome_Ar2Pr\nCUDA_VISIBLE_DEVICES=0 python self_ensemble.py data/office-home -d OfficeHome -s Ar -t Rw -a vit_base_patch16_224 --no-pool --epochs 30 --seed 0 -b 24 --log logs/self_ensemble_vit/OfficeHome_Ar2Rw\nCUDA_VISIBLE_DEVICES=0 python self_ensemble.py data/office-home -d OfficeHome -s Cl -t Ar -a vit_base_patch16_224 --no-pool --epochs 30 --seed 0 -b 24 --log logs/self_ensemble_vit/OfficeHome_Cl2Ar\nCUDA_VISIBLE_DEVICES=0 python self_ensemble.py data/office-home -d OfficeHome -s Cl -t Pr -a vit_base_patch16_224 --no-pool --epochs 30 --seed 0 -b 24 --log logs/self_ensemble_vit/OfficeHome_Cl2Pr\nCUDA_VISIBLE_DEVICES=0 python self_ensemble.py data/office-home -d OfficeHome -s Cl -t Rw -a vit_base_patch16_224 --no-pool --epochs 30 --seed 0 -b 24 --log logs/self_ensemble_vit/OfficeHome_Cl2Rw\nCUDA_VISIBLE_DEVICES=0 python self_ensemble.py data/office-home -d OfficeHome -s Pr -t Ar -a vit_base_patch16_224 --no-pool --epochs 30 --seed 0 -b 24 --log logs/self_ensemble_vit/OfficeHome_Pr2Ar\nCUDA_VISIBLE_DEVICES=0 python self_ensemble.py data/office-home -d OfficeHome -s Pr -t Cl -a vit_base_patch16_224 --no-pool --epochs 30 --seed 0 -b 24 --log logs/self_ensemble_vit/OfficeHome_Pr2Cl\nCUDA_VISIBLE_DEVICES=0 python self_ensemble.py data/office-home -d OfficeHome -s Pr -t Rw -a vit_base_patch16_224 --no-pool --epochs 30 --seed 0 -b 24 --log logs/self_ensemble_vit/OfficeHome_Pr2Rw\nCUDA_VISIBLE_DEVICES=0 python self_ensemble.py data/office-home -d OfficeHome -s Rw -t Ar -a vit_base_patch16_224 --no-pool --epochs 30 --seed 0 -b 24 --log logs/self_ensemble_vit/OfficeHome_Rw2Ar\nCUDA_VISIBLE_DEVICES=0 python self_ensemble.py data/office-home -d OfficeHome -s Rw -t Cl -a vit_base_patch16_224 --no-pool --epochs 30 --seed 0 -b 24 --log logs/self_ensemble_vit/OfficeHome_Rw2Cl\nCUDA_VISIBLE_DEVICES=0 python self_ensemble.py data/office-home -d OfficeHome -s Rw -t Pr -a vit_base_patch16_224 --no-pool --epochs 30 --seed 0 -b 24 --log logs/self_ensemble_vit/OfficeHome_Rw2Pr\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/examples/domain_adaptation/image_classification/source_only.py",
"content": "\"\"\"\n@author: Junguang Jiang\n@contact: JiangJunguang1123@outlook.com\n\"\"\"\nimport random\nimport time\nimport warnings\nimport sys\nimport argparse\nimport shutil\nimport os.path as osp\n\nimport torch\nimport torch.nn as nn\nimport torch.backends.cudnn as cudnn\nfrom torch.optim import SGD\nfrom torch.optim.lr_scheduler import LambdaLR\nfrom torch.utils.data import DataLoader\nimport torch.nn.functional as F\n\nsys.path.append('../../..')\nfrom common.modules.classifier import Classifier\nfrom common.utils.data import ForeverDataIterator\nfrom common.utils.metric import accuracy\nfrom common.utils.meter import AverageMeter, ProgressMeter\nfrom common.utils.logger import CompleteLogger\nfrom common.utils.analysis import collect_feature, tsne, a_distance\n\nsys.path.append('.')\nimport utils\n\ndevice = torch.device(\"cuda\" if torch.cuda.is_available() else \"cpu\")\n\nclass TransformerDecorator(torch.nn.Module):\n def __init__(self, backbone=None, add_bt=3, dim=2048, eval_global=0):\n super(TransformerDecorator, self).__init__()\n self.encoder_layers = torch.nn.TransformerEncoderLayer(dim, 4, dim, 0.5)\n self.eval_global = eval_global\n self.add_bt = add_bt\n self.backbone = backbone\n\n def forward_feats(self, feature):\n feature = self.backbone(feature)\n if self.training and self.add_bt > 0:\n pre_feature = feature\n feature = feature.unsqueeze(1)\n feature = self.encoder_layers(feature)\n # feature = self.encoder_layers(feature, feature, feature)\n feature = feature.squeeze(1)\n if self.add_bt >= 1:\n return torch.cat([pre_feature, feature], dim=0)\n return feature\n\n def forward(self, feature):\n feature = self.forward_feats(feature)\n return feature\n\n\ndef main(args: argparse.Namespace):\n logger = CompleteLogger(args.log, args.phase)\n print(args)\n\n if args.seed is not None:\n random.seed(args.seed)\n torch.manual_seed(args.seed)\n cudnn.deterministic = True\n warnings.warn('You have chosen to seed training. '\n 'This will turn on the CUDNN deterministic setting, '\n 'which can slow down your training considerably! '\n 'You may see unexpected behavior when restarting '\n 'from checkpoints.')\n\n cudnn.benchmark = True\n\n # Data loading code\n train_transform = utils.get_train_transform(args.train_resizing, random_horizontal_flip=not args.no_hflip,\n random_color_jitter=False, resize_size=args.resize_size,\n norm_mean=args.norm_mean, norm_std=args.norm_std)\n val_transform = utils.get_val_transform(args.val_resizing, resize_size=args.resize_size,\n norm_mean=args.norm_mean, norm_std=args.norm_std)\n print(\"train_transform: \", train_transform)\n print(\"val_transform: \", val_transform)\n\n train_source_dataset, train_target_dataset, val_dataset, test_dataset, num_classes, args.class_names = \\\n utils.get_dataset(args.data, args.root, args.source, args.target, train_transform, val_transform)\n train_source_loader = DataLoader(train_source_dataset, batch_size=args.batch_size,\n shuffle=True, num_workers=args.workers, drop_last=True)\n train_target_loader = DataLoader(train_target_dataset, batch_size=args.batch_size,\n shuffle=True, num_workers=args.workers, drop_last=True)\n val_loader = DataLoader(val_dataset, batch_size=args.batch_size, shuffle=False, num_workers=args.workers)\n test_loader = DataLoader(test_dataset, batch_size=args.batch_size, shuffle=False, num_workers=args.workers)\n\n train_source_iter = ForeverDataIterator(train_source_loader)\n\n # create model\n print(\"=> using model '{}'\".format(args.arch))\n backbone = utils.get_model(args.arch, pretrain=not args.scratch)\n pool_layer = nn.Identity() if args.no_pool else None\n pool_layer = nn.Sequential(\n nn.AdaptiveAvgPool2d(output_size=(1, 1)),\n nn.Flatten(),\n )\n pool_layer = TransformerDecorator(pool_layer, add_bt=args.add_bt, dim=backbone.out_features)\n classifier = Classifier(backbone, num_classes, pool_layer=pool_layer, finetune=not args.scratch).to(device)\n\n # define optimizer and lr scheduler\n optimizer = SGD(classifier.get_parameters(), args.lr, momentum=args.momentum, weight_decay=args.wd, nesterov=True)\n lr_scheduler = LambdaLR(optimizer, lambda x: args.lr * (1. + args.lr_gamma * float(x)) ** (-args.lr_decay))\n\n # resume from the best checkpoint\n if args.phase != 'train':\n checkpoint = torch.load(logger.get_checkpoint_path('best'), map_location='cpu')\n classifier.load_state_dict(checkpoint)\n\n # analysis the model\n if args.phase == 'analysis':\n # extract features from both domains\n feature_extractor = nn.Sequential(classifier.backbone, classifier.pool_layer, classifier.bottleneck).to(device)\n source_feature = collect_feature(train_source_loader, feature_extractor, device)\n target_feature = collect_feature(train_target_loader, feature_extractor, device)\n # plot t-SNE\n tSNE_filename = osp.join(logger.visualize_directory, 'TSNE.pdf')\n tsne.visualize(source_feature, target_feature, tSNE_filename)\n print(\"Saving t-SNE to\", tSNE_filename)\n # calculate A-distance, which is a measure for distribution discrepancy\n A_distance = a_distance.calculate(source_feature, target_feature, device)\n print(\"A-distance =\", A_distance)\n return\n\n if args.phase == 'test':\n acc1 = utils.validate(test_loader, classifier, args, device)\n print(acc1)\n return\n\n # start training\n best_acc1 = 0.\n for epoch in range(args.epochs):\n print(lr_scheduler.get_lr())\n # train for one epoch\n utils.pretrain(train_source_iter, classifier, optimizer, lr_scheduler, epoch, args, device)\n\n # evaluate on validation set\n acc1 = utils.validate(val_loader, classifier, args, device)\n\n # remember best acc@1 and save checkpoint\n torch.save(classifier.state_dict(), logger.get_checkpoint_path('latest'))\n if acc1 > best_acc1:\n shutil.copy(logger.get_checkpoint_path('latest'), logger.get_checkpoint_path('best'))\n best_acc1 = max(acc1, best_acc1)\n\n print(\"best_acc1 = {:3.1f}\".format(best_acc1))\n\n # evaluate on test set\n classifier.load_state_dict(torch.load(logger.get_checkpoint_path('best')))\n acc1 = utils.validate(test_loader, classifier, args, device)\n print(\"test_acc1 = {:3.1f}\".format(acc1))\n\n logger.close()\n\n\nif __name__ == '__main__':\n parser = argparse.ArgumentParser(description='Source Only for Unsupervised Domain Adaptation')\n # dataset parameters\n parser.add_argument('root', metavar='DIR',\n help='root path of dataset')\n parser.add_argument('-d', '--data', metavar='DATA', default='Office31', choices=utils.get_dataset_names(),\n help='dataset: ' + ' | '.join(utils.get_dataset_names()) +\n ' (default: Office31)')\n parser.add_argument('-s', '--source', help='source domain(s)', nargs='+')\n parser.add_argument('-t', '--target', help='target domain(s)', nargs='+')\n parser.add_argument('--train-resizing', type=str, default='default')\n parser.add_argument('--val-resizing', type=str, default='default')\n parser.add_argument('--resize-size', type=int, default=224,\n help='the image size after resizing')\n parser.add_argument('--no-hflip', action='store_true',\n help='no random horizontal flipping during training')\n parser.add_argument('--norm-mean', type=float, nargs='+',\n default=(0.485, 0.456, 0.406), help='normalization mean')\n parser.add_argument('--norm-std', type=float, nargs='+',\n default=(0.229, 0.224, 0.225), help='normalization std')\n # model parameters\n parser.add_argument('-a', '--arch', metavar='ARCH', default='resnet18',\n choices=utils.get_model_names(),\n help='backbone architecture: ' +\n ' | '.join(utils.get_model_names()) +\n ' (default: resnet18)')\n parser.add_argument('--no-pool', action='store_true',\n help='no pool layer after the feature extractor.')\n parser.add_argument('--scratch', action='store_true', help='whether train from scratch.')\n # training parameters\n parser.add_argument('-b', '--batch-size', default=32, type=int,\n metavar='N',\n help='mini-batch size (default: 32)')\n parser.add_argument('--lr', '--learning-rate', default=0.001, type=float,\n metavar='LR', help='initial learning rate', dest='lr')\n parser.add_argument('--lr-gamma', default=0.0003, type=float, help='parameter for lr scheduler')\n parser.add_argument('--lr-decay', default=0.75, type=float, help='parameter for lr scheduler')\n parser.add_argument('--momentum', default=0.9, type=float, metavar='M',\n help='momentum')\n parser.add_argument('--wd', '--weight-decay', default=0.0005, type=float,\n metavar='W', help='weight decay (default: 5e-4)')\n parser.add_argument('-j', '--workers', default=2, type=int, metavar='N',\n help='number of data loading workers (default: 2)')\n parser.add_argument('--epochs', default=20, type=int, metavar='N',\n help='number of total epochs to run')\n parser.add_argument('-i', '--iters-per-epoch', default=500, type=int,\n help='Number of iterations per epoch')\n parser.add_argument('-p', '--print-freq', default=100, type=int,\n metavar='N', help='print frequency (default: 100)')\n parser.add_argument('--seed', default=None, type=int,\n help='seed for initializing training. ')\n parser.add_argument('--per-class-eval', action='store_true',\n help='whether output per-class accuracy during evaluation')\n parser.add_argument(\"--log\", type=str, default='src_only',\n help=\"Where to save logs, checkpoints and debugging images.\")\n parser.add_argument(\"--phase\", type=str, default='train', choices=['train', 'test', 'analysis'],\n help=\"When phase is 'test', only test the model.\"\n \"When phase is 'analysis', only analysis the model.\")\n parser.add_argument(\"--add_bt\", type=int, default=0,\n help=\"add_bt\")\n args = parser.parse_args()\n main(args)\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/examples/domain_adaptation/image_classification/source_only.sh",
"content": "#!/usr/bin/env bash\n# ResNet50, Office31, Single Source\nCUDA_VISIBLE_DEVICES=0 python source_only.py data/office31 -d Office31 -s A -t W -a resnet50 --epochs 5 --seed 0 --log logs/src_only/Office31_A2W\nCUDA_VISIBLE_DEVICES=0 python source_only.py data/office31 -d Office31 -s D -t W -a resnet50 --epochs 5 --seed 0 --log logs/src_only/Office31_D2W\nCUDA_VISIBLE_DEVICES=0 python source_only.py data/office31 -d Office31 -s W -t D -a resnet50 --epochs 5 --seed 0 --log logs/src_only/Office31_W2D\nCUDA_VISIBLE_DEVICES=0 python source_only.py data/office31 -d Office31 -s A -t D -a resnet50 --epochs 5 --seed 0 --log logs/src_only/Office31_A2D\nCUDA_VISIBLE_DEVICES=0 python source_only.py data/office31 -d Office31 -s D -t A -a resnet50 --epochs 5 --seed 0 --log logs/src_only/Office31_D2A\nCUDA_VISIBLE_DEVICES=0 python source_only.py data/office31 -d Office31 -s W -t A -a resnet50 --epochs 5 --seed 0 --log logs/src_only/Office31_W2A\n\n# ResNet50, Office-Home, Single Source\nCUDA_VISIBLE_DEVICES=0 python source_only.py data/office-home -d OfficeHome -s Ar -t Cl -a resnet50 --epochs 5 -i 500 --seed 0 --log logs/src_only/OfficeHome_Ar2Cl\nCUDA_VISIBLE_DEVICES=0 python source_only.py data/office-home -d OfficeHome -s Ar -t Pr -a resnet50 --epochs 5 -i 500 --seed 0 --log logs/src_only/OfficeHome_Ar2Pr\nCUDA_VISIBLE_DEVICES=0 python source_only.py data/office-home -d OfficeHome -s Ar -t Rw -a resnet50 --epochs 5 -i 500 --seed 0 --log logs/src_only/OfficeHome_Ar2Rw\nCUDA_VISIBLE_DEVICES=0 python source_only.py data/office-home -d OfficeHome -s Cl -t Ar -a resnet50 --epochs 5 -i 500 --seed 0 --log logs/src_only/OfficeHome_Cl2Ar\nCUDA_VISIBLE_DEVICES=0 python source_only.py data/office-home -d OfficeHome -s Cl -t Pr -a resnet50 --epochs 5 -i 500 --seed 0 --log logs/src_only/OfficeHome_Cl2Pr\nCUDA_VISIBLE_DEVICES=0 python source_only.py data/office-home -d OfficeHome -s Cl -t Rw -a resnet50 --epochs 5 -i 500 --seed 0 --log logs/src_only/OfficeHome_Cl2Rw\nCUDA_VISIBLE_DEVICES=0 python source_only.py data/office-home -d OfficeHome -s Pr -t Ar -a resnet50 --epochs 5 -i 500 --seed 0 --log logs/src_only/OfficeHome_Pr2Ar\nCUDA_VISIBLE_DEVICES=0 python source_only.py data/office-home -d OfficeHome -s Pr -t Cl -a resnet50 --epochs 5 -i 500 --seed 0 --log logs/src_only/OfficeHome_Pr2Cl\nCUDA_VISIBLE_DEVICES=0 python source_only.py data/office-home -d OfficeHome -s Pr -t Rw -a resnet50 --epochs 5 -i 500 --seed 0 --log logs/src_only/OfficeHome_Pr2Rw\nCUDA_VISIBLE_DEVICES=0 python source_only.py data/office-home -d OfficeHome -s Rw -t Ar -a resnet50 --epochs 5 -i 500 --seed 0 --log logs/src_only/OfficeHome_Rw2Ar\nCUDA_VISIBLE_DEVICES=0 python source_only.py data/office-home -d OfficeHome -s Rw -t Cl -a resnet50 --epochs 5 -i 500 --seed 0 --log logs/src_only/OfficeHome_Rw2Cl\nCUDA_VISIBLE_DEVICES=0 python source_only.py data/office-home -d OfficeHome -s Rw -t Pr -a resnet50 --epochs 5 -i 500 --seed 0 --log logs/src_only/OfficeHome_Rw2Pr\n\n# ResNet101, VisDA-2017, Single Source\nCUDA_VISIBLE_DEVICES=0 python source_only.py data/visda-2017 -d VisDA2017 -s Synthetic -t Real -a resnet101 \\\n --epochs 20 -i 1000 --seed 0 --per-class-eval --train-resizing cen.crop --log logs/src_only/VisDA2017\n\n# ResNet101, DomainNet, Oracle\nCUDA_VISIBLE_DEVICES=0 python source_only.py data/domainnet -d DomainNet -s c -t c -a resnet101 --epochs 20 -i 2500 --seed 0 --lr 0.01 --log logs/oracle/DomainNet_c\nCUDA_VISIBLE_DEVICES=0 python source_only.py data/domainnet -d DomainNet -s i -t i -a resnet101 --epochs 20 -i 2500 --seed 0 --lr 0.01 --log logs/oracle/DomainNet_i\nCUDA_VISIBLE_DEVICES=0 python source_only.py data/domainnet -d DomainNet -s p -t p -a resnet101 --epochs 20 -i 2500 --seed 0 --lr 0.01 --log logs/oracle/DomainNet_p\nCUDA_VISIBLE_DEVICES=0 python source_only.py data/domainnet -d DomainNet -s q -t q -a resnet101 --epochs 20 -i 2500 --seed 0 --lr 0.01 --log logs/oracle/DomainNet_q\nCUDA_VISIBLE_DEVICES=0 python source_only.py data/domainnet -d DomainNet -s r -t r -a resnet101 --epochs 20 -i 2500 --seed 0 --lr 0.01 --log logs/oracle/DomainNet_r\nCUDA_VISIBLE_DEVICES=0 python source_only.py data/domainnet -d DomainNet -s s -t s -a resnet101 --epochs 20 -i 2500 --seed 0 --lr 0.01 --log logs/oracle/DomainNet_s\n\n# ResNet101, DomainNet, Single Source\nCUDA_VISIBLE_DEVICES=0 python source_only.py data/domainnet -d DomainNet -s c -t p -a resnet101 --epochs 20 -i 2500 --seed 0 --lr 0.01 --log logs/src_only/DomainNet_c2p\nCUDA_VISIBLE_DEVICES=0 python source_only.py data/domainnet -d DomainNet -s c -t r -a resnet101 --epochs 20 -i 2500 --seed 0 --lr 0.01 --log logs/src_only/DomainNet_c2r\nCUDA_VISIBLE_DEVICES=0 python source_only.py data/domainnet -d DomainNet -s c -t s -a resnet101 --epochs 20 -i 2500 --seed 0 --lr 0.01 --log logs/src_only/DomainNet_c2s\nCUDA_VISIBLE_DEVICES=0 python source_only.py data/domainnet -d DomainNet -s p -t c -a resnet101 --epochs 20 -i 2500 --seed 0 --lr 0.01 --log logs/src_only/DomainNet_p2c\nCUDA_VISIBLE_DEVICES=0 python source_only.py data/domainnet -d DomainNet -s p -t r -a resnet101 --epochs 20 -i 2500 --seed 0 --lr 0.01 --log logs/src_only/DomainNet_p2r\nCUDA_VISIBLE_DEVICES=0 python source_only.py data/domainnet -d DomainNet -s p -t s -a resnet101 --epochs 20 -i 2500 --seed 0 --lr 0.01 --log logs/src_only/DomainNet_p2s\nCUDA_VISIBLE_DEVICES=0 python source_only.py data/domainnet -d DomainNet -s r -t c -a resnet101 --epochs 20 -i 2500 --seed 0 --lr 0.01 --log logs/src_only/DomainNet_r2c\nCUDA_VISIBLE_DEVICES=0 python source_only.py data/domainnet -d DomainNet -s r -t p -a resnet101 --epochs 20 -i 2500 --seed 0 --lr 0.01 --log logs/src_only/DomainNet_r2p\nCUDA_VISIBLE_DEVICES=0 python source_only.py data/domainnet -d DomainNet -s r -t s -a resnet101 --epochs 20 -i 2500 --seed 0 --lr 0.01 --log logs/src_only/DomainNet_r2s\nCUDA_VISIBLE_DEVICES=0 python source_only.py data/domainnet -d DomainNet -s s -t c -a resnet101 --epochs 20 -i 2500 --seed 0 --lr 0.01 --log logs/src_only/DomainNet_s2c\nCUDA_VISIBLE_DEVICES=0 python source_only.py data/domainnet -d DomainNet -s s -t p -a resnet101 --epochs 20 -i 2500 --seed 0 --lr 0.01 --log logs/src_only/DomainNet_s2p\nCUDA_VISIBLE_DEVICES=0 python source_only.py data/domainnet -d DomainNet -s s -t r -a resnet101 --epochs 20 -i 2500 --seed 0 --lr 0.01 --log logs/src_only/DomainNet_s2r\n\n# ResNet50, ImageNet200 -> ImageNetR\nCUDA_VISIBLE_DEVICES=0 python source_only.py data/ImageNetR -d ImageNetR -s IN -t INR -a resnet50 --epochs 20 -i 2500 --seed 0 --log logs/src_only/ImageNet_IN2INR\n\n# ig_resnext101_32x8d, ImageNet -> ImageNetSketch\nCUDA_VISIBLE_DEVICES=0 python source_only.py data/imagenet-sketch -d ImageNetSketch -s IN -t sketch -a ig_resnext101_32x8d --epochs 20 -i 2500 --seed 0 --log logs/src_only_ig_resnext101_32x8d/ImageNet_IN2sketch\n\n# Vision Transformer, Office-Home, Single Source\nCUDA_VISIBLE_DEVICES=0 python source_only.py data/office-home -d OfficeHome -s Ar -t Cl -a vit_base_patch16_224 --no-pool --epochs 20 -i 1000 -b 24 --seed 0 --log logs/src_only_vit/OfficeHome_Ar2Cl\nCUDA_VISIBLE_DEVICES=0 python source_only.py data/office-home -d OfficeHome -s Ar -t Pr -a vit_base_patch16_224 --no-pool --epochs 20 -i 1000 -b 24 --seed 0 --log logs/src_only_vit/OfficeHome_Ar2Pr\nCUDA_VISIBLE_DEVICES=0 python source_only.py data/office-home -d OfficeHome -s Ar -t Rw -a vit_base_patch16_224 --no-pool --epochs 20 -i 1000 -b 24 --seed 0 --log logs/src_only_vit/OfficeHome_Ar2Rw\nCUDA_VISIBLE_DEVICES=0 python source_only.py data/office-home -d OfficeHome -s Cl -t Ar -a vit_base_patch16_224 --no-pool --epochs 20 -i 1000 -b 24 --seed 0 --log logs/src_only_vit/OfficeHome_Cl2Ar\nCUDA_VISIBLE_DEVICES=0 python source_only.py data/office-home -d OfficeHome -s Cl -t Pr -a vit_base_patch16_224 --no-pool --epochs 20 -i 1000 -b 24 --seed 0 --log logs/src_only_vit/OfficeHome_Cl2Pr\nCUDA_VISIBLE_DEVICES=0 python source_only.py data/office-home -d OfficeHome -s Cl -t Rw -a vit_base_patch16_224 --no-pool --epochs 20 -i 1000 -b 24 --seed 0 --log logs/src_only_vit/OfficeHome_Cl2Rw\nCUDA_VISIBLE_DEVICES=0 python source_only.py data/office-home -d OfficeHome -s Pr -t Ar -a vit_base_patch16_224 --no-pool --epochs 20 -i 1000 -b 24 --seed 0 --log logs/src_only_vit/OfficeHome_Pr2Ar\nCUDA_VISIBLE_DEVICES=0 python source_only.py data/office-home -d OfficeHome -s Pr -t Cl -a vit_base_patch16_224 --no-pool --epochs 20 -i 1000 -b 24 --seed 0 --log logs/src_only_vit/OfficeHome_Pr2Cl\nCUDA_VISIBLE_DEVICES=0 python source_only.py data/office-home -d OfficeHome -s Pr -t Rw -a vit_base_patch16_224 --no-pool --epochs 20 -i 1000 -b 24 --seed 0 --log logs/src_only_vit/OfficeHome_Pr2Rw\nCUDA_VISIBLE_DEVICES=0 python source_only.py data/office-home -d OfficeHome -s Rw -t Ar -a vit_base_patch16_224 --no-pool --epochs 20 -i 1000 -b 24 --seed 0 --log logs/src_only_vit/OfficeHome_Rw2Ar\nCUDA_VISIBLE_DEVICES=0 python source_only.py data/office-home -d OfficeHome -s Rw -t Cl -a vit_base_patch16_224 --no-pool --epochs 20 -i 1000 -b 24 --seed 0 --log logs/src_only_vit/OfficeHome_Rw2Cl\nCUDA_VISIBLE_DEVICES=0 python source_only.py data/office-home -d OfficeHome -s Rw -t Pr -a vit_base_patch16_224 --no-pool --epochs 20 -i 1000 -b 24 --seed 0 --log logs/src_only_vit/OfficeHome_Rw2Pr\n\n# ResNet50, Office-Home, Multi Source\nCUDA_VISIBLE_DEVICES=0 python source_only.py data/office-home -d OfficeHome -s Cl Pr Rw -t Ar -a resnet50 --epochs 10 -i 1000 --seed 0 --log logs/src_only/OfficeHome_:2Ar\nCUDA_VISIBLE_DEVICES=0 python source_only.py data/office-home -d OfficeHome -s Ar Pr Rw -t Cl -a resnet50 --epochs 10 -i 1000 --seed 0 --log logs/src_only/OfficeHome_:2Cl\nCUDA_VISIBLE_DEVICES=0 python source_only.py data/office-home -d OfficeHome -s Ar Cl Rw -t Pr -a resnet50 --epochs 10 -i 1000 --seed 0 --log logs/src_only/OfficeHome_:2Pr\nCUDA_VISIBLE_DEVICES=0 python source_only.py data/office-home -d OfficeHome -s Ar Cl Pr -t Rw -a resnet50 --epochs 10 -i 1000 --seed 0 --log logs/src_only/OfficeHome_:2Rw\n\n# ResNet101, DomainNet, Multi Source\nCUDA_VISIBLE_DEVICES=0 python source_only.py data/domainnet -d DomainNet -s i p q r s -t c -a resnet101 --epochs 20 -i 2500 --seed 0 --lr 0.01 --log logs/src_only/DomainNet_:2c\nCUDA_VISIBLE_DEVICES=0 python source_only.py data/domainnet -d DomainNet -s c p q r s -t i -a resnet101 --epochs 20 -i 2500 --seed 0 --lr 0.01 --log logs/src_only/DomainNet_:2i\nCUDA_VISIBLE_DEVICES=0 python source_only.py data/domainnet -d DomainNet -s c i q r s -t p -a resnet101 --epochs 20 -i 2500 --seed 0 --lr 0.01 --log logs/src_only/DomainNet_:2p\nCUDA_VISIBLE_DEVICES=0 python source_only.py data/domainnet -d DomainNet -s c i p r s -t q -a resnet101 --epochs 20 -i 2500 --seed 0 --lr 0.01 --log logs/src_only/DomainNet_:2q\nCUDA_VISIBLE_DEVICES=0 python source_only.py data/domainnet -d DomainNet -s c i p q s -t r -a resnet101 --epochs 20 -i 2500 --seed 0 --lr 0.01 --log logs/src_only/DomainNet_:2r\nCUDA_VISIBLE_DEVICES=0 python source_only.py data/domainnet -d DomainNet -s c i p q r -t s -a resnet101 --epochs 20 -i 2500 --seed 0 --lr 0.01 --log logs/src_only/DomainNet_:2s\n\n# ResNet50, Wilds Dataset\nCUDA_VISIBLE_DEVICES=0 python source_only.py data/wilds -d iwildcam --train-resizing 'res.' --val-resizing 'res.' \\\n -a resnet50 --epochs 10 -i 1000 --seed 0 --log logs/src_only/iwildcam\n\n# DenseNet121, Wilds Dataset\nCUDA_VISIBLE_DEVICES=0 python source_only.py data/wilds -d camelyon17 --train-resizing 'res.' --val-resizing 'res.' --resize-size 96 \\\n -a densenet121 --scratch --epochs 10 -i 1000 --lr 0.01 --seed 0 --log logs/src_only/camelyon17\nCUDA_VISIBLE_DEVICES=0 python source_only.py data/wilds -d fmow --train-resizing 'res.' --val-resizing 'res.' \\\n -a densenet121 --epochs 10 -i 1000 --lr 0.01 --seed 0 --log logs/src_only/fmow\n\n# Digits\nCUDA_VISIBLE_DEVICES=0 python source_only.py data/digits -d Digits -s MNIST -t USPS --train-resizing 'res.' --val-resizing 'res.' \\\n --resize-size 28 --no-hflip --norm-mean 0.5 --norm-std 0.5 -a lenet --no-pool --lr 0.01 -b 128 -i 2500 --scratch --seed 0 --log logs/src_only/MNIST2USPS\nCUDA_VISIBLE_DEVICES=0 python source_only.py data/digits -d Digits -s USPS -t MNIST --train-resizing 'res.' --val-resizing 'res.' \\\n --resize-size 28 --no-hflip --norm-mean 0.5 --norm-std 0.5 -a lenet --no-pool --lr 0.1 -b 128 -i 2500 --scratch --seed 0 --log logs/src_only/USPS2MNIST\nCUDA_VISIBLE_DEVICES=0 python source_only.py data/digits -d Digits -s SVHNRGB -t MNISTRGB --train-resizing 'res.' --val-resizing 'res.' \\\n --resize-size 32 --no-hflip --norm-mean 0.5 0.5 0.5 --norm-std 0.5 0.5 0.5 -a dtn --no-pool --lr 0.1 -b 128 -i 2500 --scratch --seed 0 --log logs/src_only/SVHN2MNIST\n\n\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/examples/domain_adaptation/image_classification/utils.py",
"content": "\"\"\"\n@author: Junguang Jiang, Baixu Chen\n@contact: JiangJunguang1123@outlook.com, cbx_99_hasta@outlook.com\n\"\"\"\nimport sys\nimport os.path as osp\nimport time\nimport timm\nimport torch\nimport torch.nn as nn\nimport torch.nn.functional as F\nimport torchvision.transforms as T\nfrom torch.utils.data import ConcatDataset\nimport wilds\n\nsys.path.append('../../..')\nimport common.vision.datasets as datasets\nimport common.vision.models as models\nfrom common.vision.transforms import ResizeImage\nfrom common.utils.metric import accuracy, ConfusionMatrix\nfrom common.utils.meter import AverageMeter, ProgressMeter\n\n\ndef get_model_names():\n return sorted(\n name for name in models.__dict__\n if name.islower() and not name.startswith(\"__\")\n and callable(models.__dict__[name])\n ) + timm.list_models()\n\n\ndef get_model(model_name, pretrain=True):\n if model_name in models.__dict__:\n # load models from common.vision.models\n backbone = models.__dict__[model_name](pretrained=pretrain)\n else:\n # load models from pytorch-image-models\n backbone = timm.create_model(model_name, pretrained=pretrain)\n try:\n backbone.out_features = backbone.get_classifier().in_features\n backbone.reset_classifier(0, '')\n except:\n backbone.out_features = backbone.head.in_features\n backbone.head = nn.Identity()\n return backbone\n\n\ndef convert_from_wilds_dataset(wild_dataset):\n class Dataset:\n def __init__(self):\n self.dataset = wild_dataset\n\n def __getitem__(self, idx):\n x, y, metadata = self.dataset[idx]\n return x, y\n\n def __len__(self):\n return len(self.dataset)\n\n return Dataset()\n\n\ndef get_dataset_names():\n return sorted(\n name for name in datasets.__dict__\n if not name.startswith(\"__\") and callable(datasets.__dict__[name])\n ) + wilds.supported_datasets + ['Digits']\n\n\ndef get_dataset(dataset_name, root, source, target, train_source_transform, val_transform, train_target_transform=None):\n if train_target_transform is None:\n train_target_transform = train_source_transform\n if dataset_name == \"Digits\":\n train_source_dataset = datasets.__dict__[source[0]](osp.join(root, source[0]), download=True,\n transform=train_source_transform)\n train_target_dataset = datasets.__dict__[target[0]](osp.join(root, target[0]), download=True,\n transform=train_target_transform)\n val_dataset = test_dataset = datasets.__dict__[target[0]](osp.join(root, target[0]), split='test',\n download=True, transform=val_transform)\n class_names = datasets.MNIST.get_classes()\n num_classes = len(class_names)\n elif dataset_name in datasets.__dict__:\n # load datasets from common.vision.datasets\n dataset = datasets.__dict__[dataset_name]\n\n def concat_dataset(tasks, **kwargs):\n return ConcatDataset([dataset(task=task, **kwargs) for task in tasks])\n\n train_source_dataset = concat_dataset(root=root, tasks=source, download=True, transform=train_source_transform)\n train_target_dataset = concat_dataset(root=root, tasks=target, download=True, transform=train_target_transform)\n val_dataset = concat_dataset(root=root, tasks=target, download=True, transform=val_transform)\n if dataset_name == 'DomainNet':\n test_dataset = concat_dataset(root=root, tasks=target, split='test', download=True, transform=val_transform)\n else:\n test_dataset = val_dataset\n class_names = train_source_dataset.datasets[0].classes\n num_classes = len(class_names)\n else:\n # load datasets from wilds\n dataset = wilds.get_dataset(dataset_name, root_dir=root, download=True)\n num_classes = dataset.n_classes\n class_names = None\n train_source_dataset = convert_from_wilds_dataset(dataset.get_subset('train', transform=train_source_transform))\n train_target_dataset = convert_from_wilds_dataset(dataset.get_subset('test', transform=train_target_transform))\n val_dataset = test_dataset = convert_from_wilds_dataset(dataset.get_subset('test', transform=val_transform))\n return train_source_dataset, train_target_dataset, val_dataset, test_dataset, num_classes, class_names\n\n\ndef validate(val_loader, model, args, device) -> float:\n batch_time = AverageMeter('Time', ':6.3f')\n losses = AverageMeter('Loss', ':.4e')\n top1 = AverageMeter('Acc@1', ':6.2f')\n progress = ProgressMeter(\n len(val_loader),\n [batch_time, losses, top1],\n prefix='Test: ')\n\n # switch to evaluate mode\n model.eval()\n if args.per_class_eval:\n confmat = ConfusionMatrix(len(args.class_names))\n else:\n confmat = None\n\n with torch.no_grad():\n end = time.time()\n for i, (images, target) in enumerate(val_loader):\n images = images.to(device)\n target = target.to(device)\n\n # compute output\n output = model(images)\n loss = F.cross_entropy(output, target)\n\n # measure accuracy and record loss\n acc1, = accuracy(output, target, topk=(1,))\n if confmat:\n confmat.update(target, output.argmax(1))\n losses.update(loss.item(), images.size(0))\n top1.update(acc1.item(), images.size(0))\n\n # measure elapsed time\n batch_time.update(time.time() - end)\n end = time.time()\n\n if i % args.print_freq == 0:\n progress.display(i)\n\n print(' * Acc@1 {top1.avg:.3f}'.format(top1=top1))\n if confmat:\n print(confmat.format(args.class_names))\n\n return top1.avg\n\n\ndef get_train_transform(resizing='default', random_horizontal_flip=True, random_color_jitter=False,\n resize_size=224, norm_mean=(0.485, 0.456, 0.406), norm_std=(0.229, 0.224, 0.225)):\n \"\"\"\n resizing mode:\n - default: resize the image to 256 and take a random resized crop of size 224;\n - cen.crop: resize the image to 256 and take the center crop of size 224;\n - res: resize the image to 224;\n \"\"\"\n if resizing == 'default':\n transform = T.Compose([\n ResizeImage(256),\n T.RandomResizedCrop(224)\n ])\n elif resizing == 'cen.crop':\n transform = T.Compose([\n ResizeImage(256),\n T.CenterCrop(224)\n ])\n elif resizing == 'ran.crop':\n transform = T.Compose([\n ResizeImage(256),\n T.RandomCrop(224)\n ])\n elif resizing == 'res.':\n transform = ResizeImage(resize_size)\n else:\n raise NotImplementedError(resizing)\n transforms = [transform]\n if random_horizontal_flip:\n transforms.append(T.RandomHorizontalFlip())\n if random_color_jitter:\n transforms.append(T.ColorJitter(brightness=0.5, contrast=0.5, saturation=0.5, hue=0.5))\n transforms.extend([\n T.ToTensor(),\n T.Normalize(mean=norm_mean, std=norm_std)\n ])\n return T.Compose(transforms)\n\n\ndef get_val_transform(resizing='default', resize_size=224,\n norm_mean=(0.485, 0.456, 0.406), norm_std=(0.229, 0.224, 0.225)):\n \"\"\"\n resizing mode:\n - default: resize the image to 256 and take the center crop of size 224;\n – res.: resize the image to 224\n \"\"\"\n if resizing == 'default':\n transform = T.Compose([\n ResizeImage(256),\n T.CenterCrop(224),\n ])\n elif resizing == 'res.':\n transform = ResizeImage(resize_size)\n else:\n raise NotImplementedError(resizing)\n return T.Compose([\n transform,\n T.ToTensor(),\n T.Normalize(mean=norm_mean, std=norm_std)\n ])\n\n\ndef pretrain(train_source_iter, model, optimizer, lr_scheduler, epoch, args, device):\n batch_time = AverageMeter('Time', ':3.1f')\n data_time = AverageMeter('Data', ':3.1f')\n losses = AverageMeter('Loss', ':3.2f')\n cls_accs = AverageMeter('Cls Acc', ':3.1f')\n\n progress = ProgressMeter(\n args.iters_per_epoch,\n [batch_time, data_time, losses, cls_accs],\n prefix=\"Epoch: [{}]\".format(epoch))\n\n # switch to train mode\n model.train()\n\n end = time.time()\n for i in range(args.iters_per_epoch):\n x_s, labels_s = next(train_source_iter)\n x_s = x_s.to(device)\n labels_s = labels_s.to(device)\n\n # measure data loading time\n data_time.update(time.time() - end)\n\n # compute output\n y_s, f_s = model(x_s)\n if len(y_s) > len(labels_s):\n labels_s = torch.cat([labels_s, labels_s], dim=0)\n\n cls_loss = F.cross_entropy(y_s, labels_s)\n loss = cls_loss\n\n cls_acc = accuracy(y_s, labels_s)[0]\n\n losses.update(loss.item(), x_s.size(0))\n cls_accs.update(cls_acc.item(), x_s.size(0))\n\n # compute gradient and do SGD step\n optimizer.zero_grad()\n loss.backward()\n optimizer.step()\n lr_scheduler.step()\n\n # measure elapsed time\n batch_time.update(time.time() - end)\n end = time.time()\n\n if i % args.print_freq == 0:\n progress.display(i)\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/examples/domain_adaptation/image_regression/README.md",
"content": "# Unsupervised Domain Adaptation for Image Regression Tasks\n\n## Dataset\n\nFollowing datasets can be downloaded automatically:\n\n- [DSprites](https://github.com/deepmind/dsprites-dataset)\n- [MPI3D](https://github.com/rr-learning/disentanglement_dataset)\n\n## Supported Methods\n\nSupported methods include:\n\n- [Disparity Discrepancy (DD)](https://arxiv.org/abs/1904.05801)\n- [Representation Subspace Distance (RSD)](http://ise.thss.tsinghua.edu.cn/~mlong/doc/Representation-Subspace-Distance-for-Domain-Adaptation-Regression-icml21.pdf)\n\n## Experiment and Results\n\nThe shell files give the script to reproduce the [benchmarks](/docs/dalib/benchmarks/image_regression.rst) with specified hyper-parameters.\nFor example, if you want to train DD on Office31, use the following script\n\n```shell script\n# Train a DD on DSprites C->N task using ResNet 18.\n# Assume you have put the datasets under the path `data/dSprites`, \n# or you are glad to download the datasets automatically from the Internet to this path\nCUDA_VISIBLE_DEVICES=0 python dd.py data/dSprites -d DSprites -s C -t N -a resnet18 --epochs 40 --seed 0 -b 128 --log logs/mdd/dSprites_C2N --wd 0.0005\n```\n\nFor more information please refer to [Get Started](/docs/get_started/quickstart.rst) for help.\n\n## Citation\nIf you use these methods in your research, please consider citing.\n\n```\n@inproceedings{MDD,\n title={Bridging theory and algorithm for domain adaptation},\n author={Zhang, Yuchen and Liu, Tianle and Long, Mingsheng and Jordan, Michael},\n booktitle={ICML},\n year={2019},\n}\n\n@inproceedings{RSD,\n title={Representation Subspace Distance for Domain Adaptation Regression}, \n author={Chen, Xinyang and Wang, Sinan and Wang, Jianmin and Long, Mingsheng}, \n booktitle={ICML}, \n year={2021} \n}\n```\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/examples/domain_adaptation/image_regression/dann.py",
"content": "\"\"\"\n@author: Junguang Jiang\n@contact: JiangJunguang1123@outlook.com\n\"\"\"\nimport random\nimport time\nimport warnings\nimport sys\nimport argparse\nimport shutil\nimport os.path as osp\n\nimport torch\nimport torch.nn as nn\nimport torch.backends.cudnn as cudnn\nfrom torch.optim import SGD\nfrom torch.optim.lr_scheduler import LambdaLR\nfrom torch.utils.data import DataLoader\nimport torchvision.transforms as T\nimport torch.nn.functional as F\n\nsys.path.append('../../..')\nfrom common.modules.regressor import Regressor\nfrom dalib.adaptation.dann import DomainAdversarialLoss\nfrom dalib.modules.domain_discriminator import DomainDiscriminator\nimport common.vision.datasets.regression as datasets\nimport common.vision.models as models\nfrom common.utils.data import ForeverDataIterator\nfrom common.utils.meter import AverageMeter, ProgressMeter\nfrom common.utils.logger import CompleteLogger\nfrom common.utils.analysis import collect_feature, tsne, a_distance\n\nsys.path.append('.')\nfrom utils import convert_model, validate\n\ndevice = torch.device(\"cuda\" if torch.cuda.is_available() else \"cpu\")\n\n\ndef main(args: argparse.Namespace):\n logger = CompleteLogger(args.log, args.phase)\n print(args)\n\n if args.seed is not None:\n random.seed(args.seed)\n torch.manual_seed(args.seed)\n cudnn.deterministic = True\n warnings.warn('You have chosen to seed training. '\n 'This will turn on the CUDNN deterministic setting, '\n 'which can slow down your training considerably! '\n 'You may see unexpected behavior when restarting '\n 'from checkpoints.')\n\n cudnn.benchmark = True\n\n # Data loading code\n normalize = T.Normalize(mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225])\n train_transform = T.Compose([\n T.Resize(args.resize_size),\n T.ToTensor(),\n normalize\n ])\n val_transform = T.Compose([\n T.Resize(args.resize_size),\n T.ToTensor(),\n normalize\n ])\n\n dataset = datasets.__dict__[args.data]\n train_source_dataset = dataset(root=args.root, task=args.source, split='train', download=True, transform=train_transform)\n train_source_loader = DataLoader(train_source_dataset, batch_size=args.batch_size,\n shuffle=True, num_workers=args.workers, drop_last=True)\n train_target_dataset = dataset(root=args.root, task=args.target, split='train', download=True, transform=train_transform)\n train_target_loader = DataLoader(train_target_dataset, batch_size=args.batch_size,\n shuffle=True, num_workers=args.workers, drop_last=True)\n val_dataset = dataset(root=args.root, task=args.target, split='test', download=True, transform=val_transform)\n val_loader = DataLoader(val_dataset, batch_size=args.batch_size, shuffle=False, num_workers=args.workers)\n\n train_source_iter = ForeverDataIterator(train_source_loader)\n train_target_iter = ForeverDataIterator(train_target_loader)\n\n # create model\n print(\"=> using pre-trained model '{}'\".format(args.arch))\n backbone = models.__dict__[args.arch](pretrained=True)\n if args.normalization == 'IN':\n backbone = convert_model(backbone)\n num_factors = train_source_dataset.num_factors\n bottleneck = nn.Sequential(\n nn.AdaptiveAvgPool2d(output_size=(1, 1)),\n nn.Flatten(),\n nn.Linear(backbone.out_features, 256),\n nn.ReLU()\n )\n regressor = Regressor(backbone=backbone, num_factors=num_factors, bottleneck=bottleneck, bottleneck_dim=256).to(device)\n print(regressor)\n domain_discri = DomainDiscriminator(in_feature=regressor.features_dim, hidden_size=1024).to(device)\n\n # define optimizer and lr scheduler\n optimizer = SGD(regressor.get_parameters() + domain_discri.get_parameters(), args.lr, momentum=args.momentum, weight_decay=args.wd, nesterov=True)\n lr_scheduler = LambdaLR(optimizer, lambda x: args.lr * (1. + args.lr_gamma * float(x)) ** (-args.lr_decay))\n\n # define loss function\n dann = DomainAdversarialLoss(domain_discri).to(device)\n\n # resume from the best checkpoint\n if args.phase != 'train':\n checkpoint = torch.load(logger.get_checkpoint_path('best'), map_location='cpu')\n regressor.load_state_dict(checkpoint)\n\n # analysis the model\n if args.phase == 'analysis':\n train_source_loader = DataLoader(train_source_dataset, batch_size=args.batch_size,\n shuffle=True, num_workers=args.workers, drop_last=True)\n train_target_loader = DataLoader(train_target_dataset, batch_size=args.batch_size,\n shuffle=True, num_workers=args.workers, drop_last=True)\n # extract features from both domains\n feature_extractor = nn.Sequential(regressor.backbone, regressor.bottleneck).to(device)\n source_feature = collect_feature(train_source_loader, feature_extractor, device)\n target_feature = collect_feature(train_target_loader, feature_extractor, device)\n # plot t-SNE\n tSNE_filename = osp.join(logger.visualize_directory, 'TSNE.pdf')\n tsne.visualize(source_feature, target_feature, tSNE_filename)\n print(\"Saving t-SNE to\", tSNE_filename)\n # calculate A-distance, which is a measure for distribution discrepancy\n A_distance = a_distance.calculate(source_feature, target_feature, device)\n print(\"A-distance =\", A_distance)\n return\n\n if args.phase == 'test':\n mae = validate(val_loader, regressor, args, train_source_dataset.factors, device)\n print(mae)\n return\n\n # start training\n best_mae = 100000.\n for epoch in range(args.epochs):\n # train for one epoch\n print(\"lr\", lr_scheduler.get_lr())\n train(train_source_iter, train_target_iter, regressor, dann, optimizer,\n lr_scheduler, epoch, args)\n\n # evaluate on validation set\n mae = validate(val_loader, regressor, args, train_source_dataset.factors, device)\n\n # remember best mae and save checkpoint\n torch.save(regressor.state_dict(), logger.get_checkpoint_path('latest'))\n if mae < best_mae:\n shutil.copy(logger.get_checkpoint_path('latest'), logger.get_checkpoint_path('best'))\n best_mae = min(mae, best_mae)\n print(\"mean MAE {:6.3f} best MAE {:6.3f}\".format(mae, best_mae))\n\n print(\"best_mae = {:6.3f}\".format(best_mae))\n\n logger.close()\n\n\ndef train(train_source_iter: ForeverDataIterator, train_target_iter: ForeverDataIterator,\n model: Regressor, domain_adv: DomainAdversarialLoss, optimizer: SGD,\n lr_scheduler: LambdaLR, epoch: int, args: argparse.Namespace):\n batch_time = AverageMeter('Time', ':4.2f')\n data_time = AverageMeter('Data', ':3.1f')\n mse_losses = AverageMeter('MSE Loss', ':6.3f')\n dann_losses = AverageMeter('DANN Loss', ':6.3f')\n domain_accs = AverageMeter('Domain Acc', ':3.1f')\n mae_losses_s = AverageMeter('MAE Loss (s)', ':6.3f')\n mae_losses_t = AverageMeter('MAE Loss (t)', ':6.3f')\n\n progress = ProgressMeter(\n args.iters_per_epoch,\n [batch_time, data_time, mse_losses, dann_losses, mae_losses_s, mae_losses_t, domain_accs],\n prefix=\"Epoch: [{}]\".format(epoch))\n\n # switch to train mode\n model.train()\n domain_adv.train()\n\n end = time.time()\n for i in range(args.iters_per_epoch):\n optimizer.zero_grad()\n\n x_s, labels_s = next(train_source_iter)\n x_s = x_s.to(device)\n labels_s = labels_s.to(device).float()\n x_t, labels_t = next(train_target_iter)\n x_t = x_t.to(device)\n labels_t = labels_t.to(device).float()\n\n # measure data loading time\n data_time.update(time.time() - end)\n\n # compute output\n y_s, f_s = model(x_s)\n y_t, f_t = model(x_t)\n\n mse_loss = F.mse_loss(y_s, labels_s)\n mae_loss_s = F.l1_loss(y_s, labels_s)\n mae_loss_t = F.l1_loss(y_t, labels_t)\n transfer_loss = domain_adv(f_s, f_t)\n loss = mse_loss + transfer_loss * args.trade_off\n domain_acc = domain_adv.domain_discriminator_accuracy\n\n mse_losses.update(mse_loss.item(), x_s.size(0))\n dann_losses.update(transfer_loss.item(), x_s.size(0))\n mae_losses_s.update(mae_loss_s.item(), x_s.size(0))\n mae_losses_t.update(mae_loss_t.item(), x_s.size(0))\n domain_accs.update(domain_acc.item(), x_s.size(0))\n\n # compute gradient and do SGD step\n loss.backward()\n optimizer.step()\n lr_scheduler.step()\n\n # measure elapsed time\n batch_time.update(time.time() - end)\n end = time.time()\n\n if i % args.print_freq == 0:\n progress.display(i)\n\n\nif __name__ == '__main__':\n architecture_names = sorted(\n name for name in models.__dict__\n if name.islower() and not name.startswith(\"__\")\n and callable(models.__dict__[name])\n )\n dataset_names = sorted(\n name for name in datasets.__dict__\n if not name.startswith(\"__\") and callable(datasets.__dict__[name])\n )\n\n parser = argparse.ArgumentParser(description='DANN for Regression Domain Adaptation')\n # dataset parameters\n parser.add_argument('root', metavar='DIR',\n help='root path of dataset')\n parser.add_argument('-d', '--data', metavar='DATA', default='DSprites',\n help='dataset: ' + ' | '.join(dataset_names) +\n ' (default: Office31)')\n parser.add_argument('-s', '--source', help='source domain(s)')\n parser.add_argument('-t', '--target', help='target domain(s)')\n parser.add_argument('--resize-size', type=int, default=128)\n # model parameters\n parser.add_argument('-a', '--arch', metavar='ARCH', default='resnet18',\n choices=architecture_names,\n help='backbone architecture: ' +\n ' | '.join(architecture_names) +\n ' (default: resnet18)')\n parser.add_argument('--normalization', default='BN', type=str, choices=[\"BN\", \"IN\"])\n parser.add_argument('--trade-off', default=1., type=float,\n help='the trade-off hyper-parameter for transfer loss')\n # training parameters\n parser.add_argument('-b', '--batch-size', default=36, type=int,\n metavar='N',\n help='mini-batch size (default: 32)')\n parser.add_argument('--lr', '--learning-rate', default=0.1, type=float,\n metavar='LR', help='initial learning rate', dest='lr')\n parser.add_argument('--lr-gamma', default=0.0001, type=float, help='parameter for lr scheduler')\n parser.add_argument('--lr-decay', default=0.75, type=float, help='parameter for lr scheduler')\n parser.add_argument('--momentum', default=0.9, type=float, metavar='M',\n help='momentum')\n parser.add_argument('--wd', '--weight-decay', default=0.001, type=float,\n metavar='W', help='weight decay (default: 5e-4)')\n parser.add_argument('-j', '--workers', default=2, type=int, metavar='N',\n help='number of data loading workers (default: 4)')\n parser.add_argument('--epochs', default=20, type=int, metavar='N',\n help='number of total epochs to run')\n parser.add_argument('-i', '--iters-per-epoch', default=500, type=int,\n help='Number of iterations per epoch')\n parser.add_argument('-p', '--print-freq', default=100, type=int,\n metavar='N', help='print frequency (default: 100)')\n parser.add_argument('--seed', default=None, type=int,\n help='seed for initializing training. ')\n parser.add_argument(\"--log\", type=str, default='dann',\n help=\"Where to save logs, checkpoints and debugging images.\")\n parser.add_argument(\"--phase\", type=str, default='train', choices=['train', 'test', 'analysis'],\n help=\"When phase is 'test', only test the model.\"\n \"When phase is 'analysis', only analysis the model.\")\n args = parser.parse_args()\n main(args)\n\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/examples/domain_adaptation/image_regression/dann.sh",
"content": "# DSprites\nCUDA_VISIBLE_DEVICES=0 python dann.py data/dSprites -d DSprites -s C -t N -a resnet18 --epochs 40 --seed 0 --log logs/dann/DSprites_C2N\nCUDA_VISIBLE_DEVICES=0 python dann.py data/dSprites -d DSprites -s C -t S -a resnet18 --epochs 40 --seed 0 --log logs/dann/DSprites_C2S\nCUDA_VISIBLE_DEVICES=0 python dann.py data/dSprites -d DSprites -s N -t C -a resnet18 --epochs 40 --seed 0 --log logs/dann/DSprites_N2C\nCUDA_VISIBLE_DEVICES=0 python dann.py data/dSprites -d DSprites -s N -t S -a resnet18 --epochs 40 --seed 0 --log logs/dann/DSprites_N2S\nCUDA_VISIBLE_DEVICES=0 python dann.py data/dSprites -d DSprites -s S -t C -a resnet18 --epochs 40 --seed 0 --log logs/dann/DSprites_S2C\nCUDA_VISIBLE_DEVICES=0 python dann.py data/dSprites -d DSprites -s S -t N -a resnet18 --epochs 40 --seed 0 --log logs/dann/DSprites_S2N\n\n# MPI3D\nCUDA_VISIBLE_DEVICES=0 python dann.py data/mpi3d -d MPI3D -s RL -t RC -a resnet18 --epochs 40 --seed 0 --log logs/dann/MPI3D_RL2RC --resize-size 224\nCUDA_VISIBLE_DEVICES=0 python dann.py data/mpi3d -d MPI3D -s RL -t T -a resnet18 --epochs 40 --seed 0 --log logs/dann/MPI3D_RL2T --resize-size 224\nCUDA_VISIBLE_DEVICES=0 python dann.py data/mpi3d -d MPI3D -s RC -t RL -a resnet18 --epochs 40 --seed 0 --log logs/dann/MPI3D_RC2RL --resize-size 224\nCUDA_VISIBLE_DEVICES=0 python dann.py data/mpi3d -d MPI3D -s RC -t T -a resnet18 --epochs 40 --seed 0 --log logs/dann/MPI3D_RC2T --resize-size 224\nCUDA_VISIBLE_DEVICES=0 python dann.py data/mpi3d -d MPI3D -s T -t RL -a resnet18 --epochs 40 --seed 0 --log logs/dann/MPI3D_T2RL --resize-size 224\nCUDA_VISIBLE_DEVICES=0 python dann.py data/mpi3d -d MPI3D -s T -t RC -a resnet18 --epochs 40 --seed 0 --log logs/dann/MPI3D_T2RC --resize-size 224\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/examples/domain_adaptation/image_regression/dd.py",
"content": "\"\"\"\n@author: Junguang Jiang\n@contact: JiangJunguang1123@outlook.com\n\"\"\"\nimport random\nimport time\nimport warnings\nimport sys\nimport argparse\nimport shutil\nimport os.path as osp\n\nimport torch\nimport torch.backends.cudnn as cudnn\nfrom torch.optim import SGD\nfrom torch.optim.lr_scheduler import LambdaLR\nfrom torch.utils.data import DataLoader\nimport torchvision.transforms as T\nimport torch.nn.functional as F\nimport torch.nn as nn\n\nsys.path.append('../../..')\nfrom dalib.adaptation.mdd import RegressionMarginDisparityDiscrepancy as MarginDisparityDiscrepancy, ImageRegressor\nimport common.vision.datasets.regression as datasets\nimport common.vision.models as models\nfrom common.utils.data import ForeverDataIterator\nfrom common.utils.meter import AverageMeter, ProgressMeter\nfrom common.utils.logger import CompleteLogger\nfrom common.utils.analysis import collect_feature, tsne, a_distance\n\nsys.path.append('.')\nfrom utils import convert_model, validate\n\ndevice = torch.device(\"cuda\" if torch.cuda.is_available() else \"cpu\")\n\n\ndef main(args: argparse.Namespace):\n logger = CompleteLogger(args.log, args.phase)\n print(args)\n\n if args.seed is not None:\n random.seed(args.seed)\n torch.manual_seed(args.seed)\n cudnn.deterministic = True\n warnings.warn('You have chosen to seed training. '\n 'This will turn on the CUDNN deterministic setting, '\n 'which can slow down your training considerably! '\n 'You may see unexpected behavior when restarting '\n 'from checkpoints.')\n\n cudnn.benchmark = True\n\n # Data loading code\n normalize = T.Normalize(mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225])\n train_transform = T.Compose([\n T.Resize(args.resize_size),\n T.ToTensor(),\n normalize\n ])\n val_transform = T.Compose([\n T.Resize(args.resize_size),\n T.ToTensor(),\n normalize\n ])\n\n dataset = datasets.__dict__[args.data]\n train_source_dataset = dataset(root=args.root, task=args.source, split='train', download=True, transform=train_transform)\n train_source_loader = DataLoader(train_source_dataset, batch_size=args.batch_size,\n shuffle=True, num_workers=args.workers, drop_last=True)\n train_target_dataset = dataset(root=args.root, task=args.target, split='train', download=True, transform=train_transform)\n train_target_loader = DataLoader(train_target_dataset, batch_size=args.batch_size,\n shuffle=True, num_workers=args.workers, drop_last=True)\n val_dataset = dataset(root=args.root, task=args.target, split='test', download=True, transform=val_transform)\n val_loader = DataLoader(val_dataset, batch_size=args.batch_size, shuffle=False, num_workers=args.workers)\n\n train_source_iter = ForeverDataIterator(train_source_loader)\n train_target_iter = ForeverDataIterator(train_target_loader)\n\n # create model\n print(\"=> using pre-trained model '{}'\".format(args.arch))\n num_factors = train_source_dataset.num_factors\n backbone = models.__dict__[args.arch](pretrained=True)\n bottleneck_dim = args.bottleneck_dim\n if args.normalization == 'IN':\n backbone = convert_model(backbone)\n bottleneck = nn.Sequential(\n nn.Conv2d(backbone.out_features, bottleneck_dim, kernel_size=3, stride=1, padding=1),\n nn.InstanceNorm2d(bottleneck_dim),\n nn.ReLU(),\n )\n head = nn.Sequential(\n nn.Conv2d(bottleneck_dim, bottleneck_dim, kernel_size=3, stride=1, padding=1),\n nn.InstanceNorm2d(bottleneck_dim),\n nn.ReLU(),\n nn.Conv2d(bottleneck_dim, bottleneck_dim, kernel_size=3, stride=1, padding=1),\n nn.InstanceNorm2d(bottleneck_dim),\n nn.ReLU(),\n nn.AdaptiveAvgPool2d(output_size=(1, 1)),\n nn.Flatten(),\n nn.Linear(bottleneck_dim, num_factors),\n nn.Sigmoid()\n )\n for layer in head:\n if isinstance(layer, nn.Conv2d) or isinstance(layer, nn.Linear):\n nn.init.normal_(layer.weight, 0, 0.01)\n nn.init.constant_(layer.bias, 0)\n adv_head = nn.Sequential(\n nn.Conv2d(bottleneck_dim, bottleneck_dim, kernel_size=3, stride=1, padding=1),\n nn.InstanceNorm2d(bottleneck_dim),\n nn.ReLU(),\n nn.Conv2d(bottleneck_dim, bottleneck_dim, kernel_size=3, stride=1, padding=1),\n nn.InstanceNorm2d(bottleneck_dim),\n nn.ReLU(),\n nn.AdaptiveAvgPool2d(output_size=(1, 1)),\n nn.Flatten(),\n nn.Linear(bottleneck_dim, num_factors),\n nn.Sigmoid()\n )\n for layer in adv_head:\n if isinstance(layer, nn.Conv2d) or isinstance(layer, nn.Linear):\n nn.init.normal_(layer.weight, 0, 0.01)\n nn.init.constant_(layer.bias, 0)\n regressor = ImageRegressor(backbone, num_factors, bottleneck=bottleneck, head=head, adv_head=adv_head,\n bottleneck_dim=bottleneck_dim, width=bottleneck_dim)\n else:\n regressor = ImageRegressor(backbone, num_factors,\n bottleneck_dim=bottleneck_dim, width=bottleneck_dim)\n\n regressor = regressor.to(device)\n print(regressor)\n mdd = MarginDisparityDiscrepancy(args.margin).to(device)\n\n # define optimizer and lr scheduler\n optimizer = SGD(regressor.get_parameters(), args.lr, momentum=args.momentum, weight_decay=args.wd, nesterov=True)\n lr_scheduler = LambdaLR(optimizer, lambda x: args.lr * (1. + args.lr_gamma * float(x)) ** (-args.lr_decay))\n\n # resume from the best checkpoint\n if args.phase != 'train':\n checkpoint = torch.load(logger.get_checkpoint_path('best'), map_location='cpu')\n regressor.load_state_dict(checkpoint)\n\n # analysis the model\n if args.phase == 'analysis':\n train_source_loader = DataLoader(train_source_dataset, batch_size=args.batch_size,\n shuffle=True, num_workers=args.workers, drop_last=True)\n train_target_loader = DataLoader(train_target_dataset, batch_size=args.batch_size,\n shuffle=True, num_workers=args.workers, drop_last=True)\n # extract features from both domains\n feature_extractor = nn.Sequential(regressor.backbone, regressor.bottleneck, regressor.head[:-2]).to(device)\n source_feature = collect_feature(train_source_loader, feature_extractor, device)\n target_feature = collect_feature(train_target_loader, feature_extractor, device)\n # plot t-SNE\n tSNE_filename = osp.join(logger.visualize_directory, 'TSNE.pdf')\n tsne.visualize(source_feature, target_feature, tSNE_filename)\n print(\"Saving t-SNE to\", tSNE_filename)\n # calculate A-distance, which is a measure for distribution discrepancy\n A_distance = a_distance.calculate(source_feature, target_feature, device)\n print(\"A-distance =\", A_distance)\n return\n\n if args.phase == 'test':\n mae = validate(val_loader, regressor, args, train_source_dataset.factors, device)\n print(mae)\n return\n\n # start training\n best_mae = 100000.\n for epoch in range(args.epochs):\n # train for one epoch\n print(\"lr\", lr_scheduler.get_lr())\n train(train_source_iter, train_target_iter, regressor, mdd, optimizer,\n lr_scheduler, epoch, args)\n\n # evaluate on validation set\n mae = validate(val_loader, regressor, args, train_source_dataset.factors, device)\n\n # remember best mae and save checkpoint\n torch.save(regressor.state_dict(), logger.get_checkpoint_path('latest'))\n if mae < best_mae:\n shutil.copy(logger.get_checkpoint_path('latest'), logger.get_checkpoint_path('best'))\n best_mae = min(mae, best_mae)\n print(\"mean MAE {:6.3f} best MAE {:6.3f}\".format(mae, best_mae))\n\n print(\"best_mae = {:6.3f}\".format(best_mae))\n\n logger.close()\n\n\ndef train(train_source_iter: ForeverDataIterator, train_target_iter: ForeverDataIterator,\n model, mdd: MarginDisparityDiscrepancy, optimizer: SGD,\n lr_scheduler: LambdaLR, epoch: int, args: argparse.Namespace):\n batch_time = AverageMeter('Time', ':4.2f')\n data_time = AverageMeter('Data', ':3.1f')\n source_losses = AverageMeter('Source Loss', ':6.3f')\n trans_losses = AverageMeter('Trans Loss', ':6.3f')\n mae_losses_s = AverageMeter('MAE Loss (s)', ':6.3f')\n mae_losses_t = AverageMeter('MAE Loss (t)', ':6.3f')\n\n progress = ProgressMeter(\n args.iters_per_epoch,\n [batch_time, data_time, source_losses, trans_losses, mae_losses_s, mae_losses_t],\n prefix=\"Epoch: [{}]\".format(epoch))\n\n # switch to train mode\n model.train()\n mdd.train()\n\n end = time.time()\n for i in range(args.iters_per_epoch):\n optimizer.zero_grad()\n\n x_s, labels_s = next(train_source_iter)\n x_s = x_s.to(device)\n labels_s = labels_s.to(device).float()\n x_t, labels_t = next(train_target_iter)\n x_t = x_t.to(device)\n labels_t = labels_t.to(device).float()\n\n # measure data loading time\n data_time.update(time.time() - end)\n\n # compute output\n x = torch.cat([x_s, x_t], dim=0)\n outputs, outputs_adv = model(x)\n y_s, y_t = outputs.chunk(2, dim=0)\n y_s_adv, y_t_adv = outputs_adv.chunk(2, dim=0)\n\n # compute mean square loss on source domain\n mse_loss = F.mse_loss(y_s, labels_s)\n\n # compute margin disparity discrepancy between domains\n transfer_loss = mdd(y_s, y_s_adv, y_t, y_t_adv)\n # for adversarial classifier, minimize negative mdd is equal to maximize mdd\n loss = mse_loss - transfer_loss * args.trade_off\n model.step()\n\n mae_loss_s = F.l1_loss(y_s, labels_s)\n mae_loss_t = F.l1_loss(y_t, labels_t)\n\n source_losses.update(mse_loss.item(), x_s.size(0))\n trans_losses.update(transfer_loss.item(), x_s.size(0))\n mae_losses_s.update(mae_loss_s.item(), x_s.size(0))\n mae_losses_t.update(mae_loss_t.item(), x_s.size(0))\n\n # compute gradient and do SGD step\n loss.backward()\n optimizer.step()\n lr_scheduler.step()\n\n # measure elapsed time\n batch_time.update(time.time() - end)\n end = time.time()\n\n if i % args.print_freq == 0:\n progress.display(i)\n\n\nif __name__ == '__main__':\n architecture_names = sorted(\n name for name in models.__dict__\n if name.islower() and not name.startswith(\"__\")\n and callable(models.__dict__[name])\n )\n dataset_names = sorted(\n name for name in datasets.__dict__\n if not name.startswith(\"__\") and callable(datasets.__dict__[name])\n )\n\n parser = argparse.ArgumentParser(description='DD for Regression Domain Adaptation')\n # dataset parameters\n parser.add_argument('root', metavar='DIR',\n help='root path of dataset')\n parser.add_argument('-d', '--data', metavar='DATA', default='DSprites',\n help='dataset: ' + ' | '.join(dataset_names) +\n ' (default: Office31)')\n parser.add_argument('-s', '--source', help='source domain(s)')\n parser.add_argument('-t', '--target', help='target domain(s)')\n parser.add_argument('--resize-size', type=int, default=128)\n # model parameters\n parser.add_argument('-a', '--arch', metavar='ARCH', default='resnet18',\n choices=architecture_names,\n help='backbone architecture: ' +\n ' | '.join(architecture_names) +\n ' (default: resnet18)')\n parser.add_argument('--bottleneck-dim', default=512, type=int)\n parser.add_argument('--normalization', default='BN', type=str, choices=['BN', 'IN'])\n parser.add_argument('--margin', type=float, default=1., help=\"margin gamma\")\n parser.add_argument('--trade-off', default=1., type=float,\n help='the trade-off hyper-parameter for transfer loss')\n # training parameters\n parser.add_argument('-b', '--batch-size', default=36, type=int,\n metavar='N',\n help='mini-batch size (default: 32)')\n parser.add_argument('--lr', '--learning-rate', default=0.1, type=float,\n metavar='LR', help='initial learning rate', dest='lr')\n parser.add_argument('--lr-gamma', default=0.0001, type=float, help='parameter for lr scheduler')\n parser.add_argument('--lr-decay', default=0.75, type=float, help='parameter for lr scheduler')\n parser.add_argument('--momentum', default=0.9, type=float, metavar='M',\n help='momentum')\n parser.add_argument('--wd', '--weight-decay', default=0.0005, type=float,\n metavar='W', help='weight decay (default: 5e-4)')\n parser.add_argument('-j', '--workers', default=2, type=int, metavar='N',\n help='number of data loading workers (default: 4)')\n parser.add_argument('--epochs', default=20, type=int, metavar='N',\n help='number of total epochs to run')\n parser.add_argument('-i', '--iters-per-epoch', default=500, type=int,\n help='Number of iterations per epoch')\n parser.add_argument('-p', '--print-freq', default=100, type=int,\n metavar='N', help='print frequency (default: 100)')\n parser.add_argument('--seed', default=None, type=int,\n help='seed for initializing training. ')\n parser.add_argument(\"--log\", type=str, default='dd',\n help=\"Where to save logs, checkpoints and debugging images.\")\n parser.add_argument(\"--phase\", type=str, default='train', choices=['train', 'test', 'analysis'],\n help=\"When phase is 'test', only test the model.\" \n \"When phase is 'analysis', only analysis the model.\")\n args = parser.parse_args()\n main(args)\n\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/examples/domain_adaptation/image_regression/dd.sh",
"content": "# DSprites\nCUDA_VISIBLE_DEVICES=0 python dd.py data/dSprites -d DSprites -s C -t N -a resnet18 --epochs 40 --seed 0 -b 128 --log logs/dd/dSprites_C2N --wd 0.0005\nCUDA_VISIBLE_DEVICES=0 python dd.py data/dSprites -d DSprites -s C -t S -a resnet18 --epochs 40 --seed 0 -b 128 --log logs/dd/dSprites_C2S --wd 0.0005\nCUDA_VISIBLE_DEVICES=0 python dd.py data/dSprites -d DSprites -s N -t C -a resnet18 --epochs 40 --seed 0 -b 128 --log logs/dd/dSprites_N2C --wd 0.0005\nCUDA_VISIBLE_DEVICES=0 python dd.py data/dSprites -d DSprites -s N -t S -a resnet18 --epochs 40 --seed 0 -b 128 --log logs/dd/dSprites_N2S --wd 0.0005\nCUDA_VISIBLE_DEVICES=0 python dd.py data/dSprites -d DSprites -s S -t C -a resnet18 --epochs 40 --seed 0 -b 128 --log logs/dd/dSprites_S2C --wd 0.0005\nCUDA_VISIBLE_DEVICES=0 python dd.py data/dSprites -d DSprites -s S -t N -a resnet18 --epochs 40 --seed 0 -b 128 --log logs/dd/dSprites_S2N --wd 0.0005\n\n# MPI3D\nCUDA_VISIBLE_DEVICES=0 python dd.py data/mpi3d -d MPI3D -s RL -t RC -a resnet18 --epochs 60 --seed 0 -b 36 --log logs/dd/MPI3D_RL2RC --normalization IN --resize-size 224 --weight-decay 0.001\nCUDA_VISIBLE_DEVICES=0 python dd.py data/mpi3d -d MPI3D -s RL -t T -a resnet18 --epochs 60 --seed 0 -b 36 --log logs/dd/MPI3D_RL2T --normalization IN --resize-size 224 --weight-decay 0.001\nCUDA_VISIBLE_DEVICES=0 python dd.py data/mpi3d -d MPI3D -s RC -t RL -a resnet18 --epochs 60 --seed 0 -b 36 --log logs/dd/MPI3D_RC2RL --normalization IN --resize-size 224 --weight-decay 0.001\nCUDA_VISIBLE_DEVICES=0 python dd.py data/mpi3d -d MPI3D -s RC -t T -a resnet18 --epochs 60 --seed 0 -b 36 --log logs/dd/MPI3D_RC2T --normalization IN --resize-size 224 --weight-decay 0.001\nCUDA_VISIBLE_DEVICES=0 python dd.py data/mpi3d -d MPI3D -s T -t RL -a resnet18 --epochs 60 --seed 0 -b 36 --log logs/dd/MPI3D_T2RL --normalization IN --resize-size 224 --weight-decay 0.001\nCUDA_VISIBLE_DEVICES=0 python dd.py data/mpi3d -d MPI3D -s T -t RC -a resnet18 --epochs 60 --seed 0 -b 36 --log logs/dd/MPI3D_T2RC --normalization IN --resize-size 224 --weight-decay 0.001\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/examples/domain_adaptation/image_regression/rsd.py",
"content": "\"\"\"\n@author: Junguang Jiang\n@contact: JiangJunguang1123@outlook.com\n\"\"\"\nimport random\nimport time\nimport warnings\nimport sys\nimport argparse\nimport shutil\nimport os.path as osp\n\nimport torch\nimport torch.nn as nn\nimport torch.backends.cudnn as cudnn\nfrom torch.optim import SGD\nfrom torch.optim.lr_scheduler import LambdaLR\nfrom torch.utils.data import DataLoader\nimport torchvision.transforms as T\nimport torch.nn.functional as F\n\nsys.path.append('../../..')\nfrom common.modules.regressor import Regressor\nfrom dalib.adaptation.rsd import RepresentationSubspaceDistance\nimport common.vision.datasets.regression as datasets\nimport common.vision.models as models\nfrom common.utils.data import ForeverDataIterator\nfrom common.utils.meter import AverageMeter, ProgressMeter\nfrom common.utils.logger import CompleteLogger\nfrom common.utils.analysis import collect_feature, tsne, a_distance\n\nsys.path.append('.')\nfrom utils import convert_model, validate\n\ndevice = torch.device(\"cuda\" if torch.cuda.is_available() else \"cpu\")\n\n\ndef main(args: argparse.Namespace):\n logger = CompleteLogger(args.log, args.phase)\n print(args)\n\n if args.seed is not None:\n random.seed(args.seed)\n torch.manual_seed(args.seed)\n cudnn.deterministic = True\n warnings.warn('You have chosen to seed training. '\n 'This will turn on the CUDNN deterministic setting, '\n 'which can slow down your training considerably! '\n 'You may see unexpected behavior when restarting '\n 'from checkpoints.')\n\n cudnn.benchmark = True\n\n # Data loading code\n normalize = T.Normalize(mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225])\n train_transform = T.Compose([\n T.Resize(args.resize_size),\n T.ToTensor(),\n normalize\n ])\n val_transform = T.Compose([\n T.Resize(args.resize_size),\n T.ToTensor(),\n normalize\n ])\n\n dataset = datasets.__dict__[args.data]\n train_source_dataset = dataset(root=args.root, task=args.source, split='train', download=True, transform=train_transform)\n train_source_loader = DataLoader(train_source_dataset, batch_size=args.batch_size,\n shuffle=True, num_workers=args.workers, drop_last=True)\n train_target_dataset = dataset(root=args.root, task=args.target, split='train', download=True, transform=train_transform)\n train_target_loader = DataLoader(train_target_dataset, batch_size=args.batch_size,\n shuffle=True, num_workers=args.workers, drop_last=True)\n val_dataset = dataset(root=args.root, task=args.target, split='test', download=True, transform=val_transform)\n val_loader = DataLoader(val_dataset, batch_size=args.batch_size, shuffle=False, num_workers=args.workers)\n\n train_source_iter = ForeverDataIterator(train_source_loader)\n train_target_iter = ForeverDataIterator(train_target_loader)\n\n # create model\n print(\"=> using pre-trained model '{}'\".format(args.arch))\n backbone = models.__dict__[args.arch](pretrained=True)\n if args.normalization == 'IN':\n backbone = convert_model(backbone)\n num_factors = train_source_dataset.num_factors\n bottleneck = nn.Sequential(\n nn.AdaptiveAvgPool2d(output_size=(1, 1)),\n nn.Flatten(),\n nn.Linear(backbone.out_features, 256),\n nn.ReLU()\n )\n regressor = Regressor(backbone=backbone, num_factors=num_factors, bottleneck=bottleneck, bottleneck_dim=backbone.out_features).to(device)\n print(regressor)\n\n # define optimizer and lr scheduler\n optimizer = SGD(regressor.get_parameters(), args.lr, momentum=args.momentum, weight_decay=args.wd, nesterov=True)\n lr_scheduler = LambdaLR(optimizer, lambda x: args.lr * (1. + args.lr_gamma * float(x)) ** (-args.lr_decay))\n\n # define loss function\n rsd = RepresentationSubspaceDistance(args.trade_off_bmp)\n\n # resume from the best checkpoint\n if args.phase != 'train':\n checkpoint = torch.load(logger.get_checkpoint_path('best'), map_location='cpu')\n regressor.load_state_dict(checkpoint)\n\n # analysis the model\n if args.phase == 'analysis':\n train_source_loader = DataLoader(train_source_dataset, batch_size=args.batch_size,\n shuffle=True, num_workers=args.workers, drop_last=True)\n train_target_loader = DataLoader(train_target_dataset, batch_size=args.batch_size,\n shuffle=True, num_workers=args.workers, drop_last=True)\n # extract features from both domains\n feature_extractor = nn.Sequential(regressor.backbone, regressor.bottleneck).to(device)\n source_feature = collect_feature(train_source_loader, feature_extractor, device)\n target_feature = collect_feature(train_target_loader, feature_extractor, device)\n # plot t-SNE\n tSNE_filename = osp.join(logger.visualize_directory, 'TSNE.pdf')\n tsne.visualize(source_feature, target_feature, tSNE_filename)\n print(\"Saving t-SNE to\", tSNE_filename)\n # calculate A-distance, which is a measure for distribution discrepancy\n A_distance = a_distance.calculate(source_feature, target_feature, device)\n print(\"A-distance =\", A_distance)\n return\n\n if args.phase == 'test':\n mae = validate(val_loader, regressor, args, train_source_dataset.factors, device)\n print(mae)\n return\n\n # start training\n best_mae = 100000.\n for epoch in range(args.epochs):\n # train for one epoch\n print(\"lr\", lr_scheduler.get_lr())\n train(train_source_iter, train_target_iter, regressor, rsd, optimizer,\n lr_scheduler, epoch, args)\n\n # evaluate on validation set\n mae = validate(val_loader, regressor, args, train_source_dataset.factors, device)\n\n # remember best mae and save checkpoint\n torch.save(regressor.state_dict(), logger.get_checkpoint_path('latest'))\n if mae < best_mae:\n shutil.copy(logger.get_checkpoint_path('latest'), logger.get_checkpoint_path('best'))\n best_mae = min(mae, best_mae)\n print(\"mean MAE {:6.3f} best MAE {:6.3f}\".format(mae, best_mae))\n\n print(\"best_mae = {:6.3f}\".format(best_mae))\n\n logger.close()\n\n\ndef train(train_source_iter: ForeverDataIterator, train_target_iter: ForeverDataIterator,\n model: Regressor, rsd, optimizer: SGD,\n lr_scheduler: LambdaLR, epoch: int, args: argparse.Namespace):\n batch_time = AverageMeter('Time', ':4.2f')\n data_time = AverageMeter('Data', ':3.1f')\n mse_losses = AverageMeter('MSE Loss', ':6.3f')\n rsd_losses = AverageMeter('RSD Loss', ':6.3f')\n mae_losses_s = AverageMeter('MAE Loss (s)', ':6.3f')\n mae_losses_t = AverageMeter('MAE Loss (t)', ':6.3f')\n\n progress = ProgressMeter(\n args.iters_per_epoch,\n [batch_time, data_time, mse_losses, rsd_losses, mae_losses_s, mae_losses_t],\n prefix=\"Epoch: [{}]\".format(epoch))\n\n # switch to train mode\n model.train()\n\n end = time.time()\n for i in range(args.iters_per_epoch):\n optimizer.zero_grad()\n\n x_s, labels_s = next(train_source_iter)\n x_s = x_s.to(device)\n labels_s = labels_s.to(device).float()\n x_t, labels_t = next(train_target_iter)\n x_t = x_t.to(device)\n labels_t = labels_t.to(device).float()\n\n # measure data loading time\n data_time.update(time.time() - end)\n\n # compute output\n y_s, f_s = model(x_s)\n y_t, f_t = model(x_t)\n\n mse_loss = F.mse_loss(y_s, labels_s)\n mae_loss_s = F.l1_loss(y_s, labels_s)\n mae_loss_t = F.l1_loss(y_t, labels_t)\n rsd_loss = rsd(f_s, f_t)\n loss = mse_loss + rsd_loss * args.trade_off\n\n mse_losses.update(mse_loss.item(), x_s.size(0))\n rsd_losses.update(rsd_loss.item(), x_s.size(0))\n mae_losses_s.update(mae_loss_s.item(), x_s.size(0))\n mae_losses_t.update(mae_loss_t.item(), x_s.size(0))\n\n # compute gradient and do SGD step\n loss.backward()\n optimizer.step()\n lr_scheduler.step()\n\n # measure elapsed time\n batch_time.update(time.time() - end)\n end = time.time()\n\n if i % args.print_freq == 0:\n progress.display(i)\n\n\nif __name__ == '__main__':\n architecture_names = sorted(\n name for name in models.__dict__\n if name.islower() and not name.startswith(\"__\")\n and callable(models.__dict__[name])\n )\n dataset_names = sorted(\n name for name in datasets.__dict__\n if not name.startswith(\"__\") and callable(datasets.__dict__[name])\n )\n\n parser = argparse.ArgumentParser(description='RSD for Regression Domain Adaptation')\n # dataset parameters\n parser.add_argument('root', metavar='DIR',\n help='root path of dataset')\n parser.add_argument('-d', '--data', metavar='DATA', default='DSprites',\n help='dataset: ' + ' | '.join(dataset_names) +\n ' (default: Office31)')\n parser.add_argument('-s', '--source', help='source domain(s)')\n parser.add_argument('-t', '--target', help='target domain(s)')\n parser.add_argument('--resize-size', type=int, default=128)\n # model parameters\n parser.add_argument('-a', '--arch', metavar='ARCH', default='resnet18',\n choices=architecture_names,\n help='backbone architecture: ' +\n ' | '.join(architecture_names) +\n ' (default: resnet18)')\n parser.add_argument('--normalization', default='BN', type=str, choices=[\"BN\", \"IN\"])\n parser.add_argument('--trade-off', default=0.001, type=float)\n parser.add_argument('--trade-off-bmp', default=0.1, type=float)\n # training parameters\n parser.add_argument('-b', '--batch-size', default=36, type=int,\n metavar='N',\n help='mini-batch size (default: 32)')\n parser.add_argument('--lr', '--learning-rate', default=0.1, type=float,\n metavar='LR', help='initial learning rate', dest='lr')\n parser.add_argument('--lr-gamma', default=0.0001, type=float, help='parameter for lr scheduler')\n parser.add_argument('--lr-decay', default=0.75, type=float, help='parameter for lr scheduler')\n parser.add_argument('--momentum', default=0.9, type=float, metavar='M',\n help='momentum')\n parser.add_argument('--wd', '--weight-decay', default=0.001, type=float,\n metavar='W', help='weight decay (default: 5e-4)')\n parser.add_argument('-j', '--workers', default=2, type=int, metavar='N',\n help='number of data loading workers (default: 4)')\n parser.add_argument('--epochs', default=20, type=int, metavar='N',\n help='number of total epochs to run')\n parser.add_argument('-i', '--iters-per-epoch', default=500, type=int,\n help='Number of iterations per epoch')\n parser.add_argument('-p', '--print-freq', default=100, type=int,\n metavar='N', help='print frequency (default: 100)')\n parser.add_argument('--seed', default=None, type=int,\n help='seed for initializing training. ')\n parser.add_argument(\"--log\", type=str, default='rsd',\n help=\"Where to save logs, checkpoints and debugging images.\")\n parser.add_argument(\"--phase\", type=str, default='train', choices=['train', 'test', 'analysis'],\n help=\"When phase is 'test', only test the model.\" \n \"When phase is 'analysis', only analysis the model.\")\n args = parser.parse_args()\n main(args)\n\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/examples/domain_adaptation/image_regression/rsd.sh",
"content": "# DSprites\nCUDA_VISIBLE_DEVICES=0 python rsd.py data/dSprites -d DSprites -s C -t N -a resnet18 --epochs 40 --seed 0 --log logs/rsd/DSprites_C2N\nCUDA_VISIBLE_DEVICES=0 python rsd.py data/dSprites -d DSprites -s C -t S -a resnet18 --epochs 40 --seed 0 --log logs/rsd/DSprites_C2S\nCUDA_VISIBLE_DEVICES=0 python rsd.py data/dSprites -d DSprites -s N -t C -a resnet18 --epochs 40 --seed 0 --log logs/rsd/DSprites_N2C\nCUDA_VISIBLE_DEVICES=0 python rsd.py data/dSprites -d DSprites -s N -t S -a resnet18 --epochs 40 --seed 0 --log logs/rsd/DSprites_N2S\nCUDA_VISIBLE_DEVICES=0 python rsd.py data/dSprites -d DSprites -s S -t C -a resnet18 --epochs 40 --seed 0 --log logs/rsd/DSprites_S2C\nCUDA_VISIBLE_DEVICES=0 python rsd.py data/dSprites -d DSprites -s S -t N -a resnet18 --epochs 40 --seed 0 --log logs/rsd/DSprites_S2N\n\n# MPI3D\nCUDA_VISIBLE_DEVICES=0 python rsd.py data/mpi3d -d MPI3D -s RL -t RC -a resnet18 --epochs 40 --seed 0 --log logs/rsd/MPI3D_RL2RC --resize-size 224\nCUDA_VISIBLE_DEVICES=0 python rsd.py data/mpi3d -d MPI3D -s RL -t T -a resnet18 --epochs 40 --seed 0 --log logs/rsd/MPI3D_RL2T --resize-size 224\nCUDA_VISIBLE_DEVICES=0 python rsd.py data/mpi3d -d MPI3D -s RC -t RL -a resnet18 --epochs 40 --seed 0 --log logs/rsd/MPI3D_RC2RL --resize-size 224\nCUDA_VISIBLE_DEVICES=0 python rsd.py data/mpi3d -d MPI3D -s RC -t T -a resnet18 --epochs 40 --seed 0 --log logs/rsd/MPI3D_RC2T --resize-size 224\nCUDA_VISIBLE_DEVICES=0 python rsd.py data/mpi3d -d MPI3D -s T -t RL -a resnet18 --epochs 40 --seed 0 --log logs/rsd/MPI3D_T2RL --resize-size 224\nCUDA_VISIBLE_DEVICES=0 python rsd.py data/mpi3d -d MPI3D -s T -t RC -a resnet18 --epochs 40 --seed 0 --log logs/rsd/MPI3D_T2RC --resize-size 224\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/examples/domain_adaptation/image_regression/source_only.py",
"content": "\"\"\"\n@author: Junguang Jiang\n@contact: JiangJunguang1123@outlook.com\n\"\"\"\nimport random\nimport time\nimport warnings\nimport sys\nimport argparse\nimport shutil\nimport os.path as osp\n\nimport torch\nimport torch.nn as nn\nimport torch.backends.cudnn as cudnn\nfrom torch.optim import SGD\nfrom torch.optim.lr_scheduler import LambdaLR\nfrom torch.utils.data import DataLoader\nimport torchvision.transforms as T\nimport torch.nn.functional as F\n\nsys.path.append('../../..')\nfrom common.modules.regressor import Regressor\nimport common.vision.datasets.regression as datasets\nimport common.vision.models as models\nfrom common.utils.data import ForeverDataIterator\nfrom common.utils.meter import AverageMeter, ProgressMeter\nfrom common.utils.logger import CompleteLogger\nfrom common.utils.analysis import collect_feature, tsne, a_distance\n\nsys.path.append('.')\nfrom utils import convert_model, validate\n\ndevice = torch.device(\"cuda\" if torch.cuda.is_available() else \"cpu\")\n\n\ndef main(args: argparse.Namespace):\n logger = CompleteLogger(args.log, args.phase)\n print(args)\n\n if args.seed is not None:\n random.seed(args.seed)\n torch.manual_seed(args.seed)\n cudnn.deterministic = True\n warnings.warn('You have chosen to seed training. '\n 'This will turn on the CUDNN deterministic setting, '\n 'which can slow down your training considerably! '\n 'You may see unexpected behavior when restarting '\n 'from checkpoints.')\n\n cudnn.benchmark = True\n\n # Data loading code\n normalize = T.Normalize(mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225])\n train_transform = T.Compose([\n T.Resize(args.resize_size),\n T.ToTensor(),\n normalize\n ])\n val_transform = T.Compose([\n T.Resize(args.resize_size),\n T.ToTensor(),\n normalize\n ])\n\n dataset = datasets.__dict__[args.data]\n train_source_dataset = dataset(root=args.root, task=args.source, split='train', download=True, transform=train_transform)\n train_source_loader = DataLoader(train_source_dataset, batch_size=args.batch_size,\n shuffle=True, num_workers=args.workers, drop_last=True)\n train_target_dataset = dataset(root=args.root, task=args.target, split='train', download=True, transform=train_transform)\n train_target_loader = DataLoader(train_target_dataset, batch_size=args.batch_size,\n shuffle=True, num_workers=args.workers, drop_last=True)\n val_dataset = dataset(root=args.root, task=args.target, split='test', download=True, transform=val_transform)\n val_loader = DataLoader(val_dataset, batch_size=args.batch_size, shuffle=False, num_workers=args.workers)\n\n train_source_iter = ForeverDataIterator(train_source_loader)\n train_target_iter = ForeverDataIterator(train_target_loader)\n\n # create model\n print(\"=> using pre-trained model '{}'\".format(args.arch))\n backbone = models.__dict__[args.arch](pretrained=True)\n if args.normalization == 'IN':\n backbone = convert_model(backbone)\n num_factors = train_source_dataset.num_factors\n regressor = Regressor(backbone=backbone, num_factors=num_factors).to(device)\n print(regressor)\n # define optimizer and lr scheduler\n optimizer = SGD(regressor.get_parameters(), args.lr, momentum=args.momentum, weight_decay=args.wd, nesterov=True)\n lr_scheduler = LambdaLR(optimizer, lambda x: args.lr * (1. + args.lr_gamma * float(x)) ** (-args.lr_decay))\n\n # resume from the best checkpoint\n if args.phase != 'train':\n checkpoint = torch.load(logger.get_checkpoint_path('best'), map_location='cpu')\n regressor.load_state_dict(checkpoint)\n\n # analysis the model\n if args.phase == 'analysis':\n train_source_loader = DataLoader(train_source_dataset, batch_size=args.batch_size,\n shuffle=True, num_workers=args.workers, drop_last=True)\n train_target_loader = DataLoader(train_target_dataset, batch_size=args.batch_size,\n shuffle=True, num_workers=args.workers, drop_last=True)\n # extract features from both domains\n feature_extractor = nn.Sequential(regressor.backbone, regressor.bottleneck).to(device)\n source_feature = collect_feature(train_source_loader, feature_extractor, device)\n target_feature = collect_feature(train_target_loader, feature_extractor, device)\n # plot t-SNE\n tSNE_filename = osp.join(logger.visualize_directory, 'TSNE.pdf')\n tsne.visualize(source_feature, target_feature, tSNE_filename)\n print(\"Saving t-SNE to\", tSNE_filename)\n # calculate A-distance, which is a measure for distribution discrepancy\n A_distance = a_distance.calculate(source_feature, target_feature, device)\n print(\"A-distance =\", A_distance)\n return\n\n if args.phase == 'test':\n mae = validate(val_loader, regressor, args, train_source_dataset.factors, device)\n print(mae)\n return\n\n # start training\n best_mae = 100000.\n for epoch in range(args.epochs):\n # train for one epoch\n print(\"lr\", lr_scheduler.get_lr())\n train(train_source_iter, train_target_iter, regressor, optimizer,\n lr_scheduler, epoch, args)\n\n # evaluate on validation set\n mae = validate(val_loader, regressor, args, train_source_dataset.factors, device)\n\n # remember best mae and save checkpoint\n torch.save(regressor.state_dict(), logger.get_checkpoint_path('latest'))\n if mae < best_mae:\n shutil.copy(logger.get_checkpoint_path('latest'), logger.get_checkpoint_path('best'))\n best_mae = min(mae, best_mae)\n print(\"mean MAE {:6.3f} best MAE {:6.3f}\".format(mae, best_mae))\n\n print(\"best_mae = {:6.3f}\".format(best_mae))\n\n logger.close()\n\n\ndef train(train_source_iter: ForeverDataIterator, train_target_iter: ForeverDataIterator,\n model: Regressor, optimizer: SGD,\n lr_scheduler: LambdaLR, epoch: int, args: argparse.Namespace):\n batch_time = AverageMeter('Time', ':4.2f')\n data_time = AverageMeter('Data', ':3.1f')\n mse_losses = AverageMeter('MSE Loss', ':6.3f')\n mae_losses_s = AverageMeter('MAE Loss (s)', ':6.3f')\n mae_losses_t = AverageMeter('MAE Loss (t)', ':6.3f')\n\n progress = ProgressMeter(\n args.iters_per_epoch,\n [batch_time, data_time, mse_losses, mae_losses_s, mae_losses_t],\n prefix=\"Epoch: [{}]\".format(epoch))\n\n # switch to train mode\n model.train()\n\n end = time.time()\n for i in range(args.iters_per_epoch):\n optimizer.zero_grad()\n\n x_s, labels_s = next(train_source_iter)\n x_s = x_s.to(device)\n labels_s = labels_s.to(device).float()\n x_t, labels_t = next(train_target_iter)\n x_t = x_t.to(device)\n labels_t = labels_t.to(device).float()\n\n # measure data loading time\n data_time.update(time.time() - end)\n\n # compute output\n y_s, _ = model(x_s)\n y_t, _ = model(x_t)\n\n mse_loss = F.mse_loss(y_s, labels_s)\n mae_loss_s = F.l1_loss(y_s, labels_s)\n mae_loss_t = F.l1_loss(y_t, labels_t)\n loss = mse_loss\n\n mse_losses.update(mse_loss.item(), x_s.size(0))\n mae_losses_s.update(mae_loss_s.item(), x_s.size(0))\n mae_losses_t.update(mae_loss_t.item(), x_s.size(0))\n\n # compute gradient and do SGD step\n loss.backward()\n optimizer.step()\n lr_scheduler.step()\n\n # measure elapsed time\n batch_time.update(time.time() - end)\n end = time.time()\n\n if i % args.print_freq == 0:\n progress.display(i)\n\n\nif __name__ == '__main__':\n architecture_names = sorted(\n name for name in models.__dict__\n if name.islower() and not name.startswith(\"__\")\n and callable(models.__dict__[name])\n )\n dataset_names = sorted(\n name for name in datasets.__dict__\n if not name.startswith(\"__\") and callable(datasets.__dict__[name])\n )\n\n parser = argparse.ArgumentParser(description='Source Only for Regression Domain Adaptation')\n # dataset parameters\n parser.add_argument('root', metavar='DIR',\n help='root path of dataset')\n parser.add_argument('-d', '--data', metavar='DATA', default='DSprites',\n help='dataset: ' + ' | '.join(dataset_names) +\n ' (default: Office31)')\n parser.add_argument('-s', '--source', help='source domain(s)')\n parser.add_argument('-t', '--target', help='target domain(s)')\n parser.add_argument('--resize-size', type=int, default=128)\n # model parameters\n parser.add_argument('-a', '--arch', metavar='ARCH', default='resnet18',\n choices=architecture_names,\n help='backbone architecture: ' +\n ' | '.join(architecture_names) +\n ' (default: resnet18)')\n parser.add_argument('--normalization', default='BN', type=str, choices=[\"IN\", \"BN\"])\n # training parameters\n parser.add_argument('-b', '--batch-size', default=36, type=int,\n metavar='N',\n help='mini-batch size (default: 32)')\n parser.add_argument('--lr', '--learning-rate', default=0.1, type=float,\n metavar='LR', help='initial learning rate', dest='lr')\n parser.add_argument('--lr-gamma', default=0.0001, type=float, help='parameter for lr scheduler')\n parser.add_argument('--lr-decay', default=0.75, type=float, help='parameter for lr scheduler')\n parser.add_argument('--momentum', default=0.9, type=float, metavar='M',\n help='momentum')\n parser.add_argument('--wd', '--weight-decay', default=0.0005, type=float,\n metavar='W', help='weight decay (default: 5e-4)')\n parser.add_argument('-j', '--workers', default=2, type=int, metavar='N',\n help='number of data loading workers (default: 4)')\n parser.add_argument('--epochs', default=20, type=int, metavar='N',\n help='number of total epochs to run')\n parser.add_argument('-i', '--iters-per-epoch', default=500, type=int,\n help='Number of iterations per epoch')\n parser.add_argument('-p', '--print-freq', default=100, type=int,\n metavar='N', help='print frequency (default: 100)')\n parser.add_argument('--seed', default=None, type=int,\n help='seed for initializing training. ')\n parser.add_argument(\"--log\", type=str, default='src_only',\n help=\"Where to save logs, checkpoints and debugging images.\")\n parser.add_argument(\"--phase\", type=str, default='train', choices=['train', 'test', 'analysis'],\n help=\"When phase is 'test', only test the model.\" \n \"When phase is 'analysis', only analysis the model.\")\n args = parser.parse_args()\n main(args)\n\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/examples/domain_adaptation/image_regression/source_only.sh",
"content": "# DSprites\nCUDA_VISIBLE_DEVICES=0 python source_only.py data/dSprites -d DSprites -s C -t N -a resnet18 --epochs 20 --seed 0 -b 128 --log logs/src_only/DSprites_C2N\nCUDA_VISIBLE_DEVICES=0 python source_only.py data/dSprites -d DSprites -s C -t S -a resnet18 --epochs 20 --seed 0 -b 128 --log logs/src_only/DSprites_C2S\nCUDA_VISIBLE_DEVICES=0 python source_only.py data/dSprites -d DSprites -s N -t C -a resnet18 --epochs 20 --seed 0 -b 128 --log logs/src_only/DSprites_N2C\nCUDA_VISIBLE_DEVICES=0 python source_only.py data/dSprites -d DSprites -s N -t S -a resnet18 --epochs 20 --seed 0 -b 128 --log logs/src_only/DSprites_N2S\nCUDA_VISIBLE_DEVICES=0 python source_only.py data/dSprites -d DSprites -s S -t C -a resnet18 --epochs 20 --seed 0 -b 128 --log logs/src_only/DSprites_S2C\nCUDA_VISIBLE_DEVICES=0 python source_only.py data/dSprites -d DSprites -s S -t N -a resnet18 --epochs 20 --seed 0 -b 128 --log logs/src_only/DSprites_S2N\n\n# MPI3D\nCUDA_VISIBLE_DEVICES=0 python source_only.py data/mpi3d -d MPI3D -s RL -t RC -a resnet18 --epochs 40 --seed 0 -b 36 --log logs/src_only/MPI3D_RL2RC\nCUDA_VISIBLE_DEVICES=0 python source_only.py data/mpi3d -d MPI3D -s RL -t T -a resnet18 --epochs 40 --seed 0 -b 36 --log logs/src_only/MPI3D_RL2T\nCUDA_VISIBLE_DEVICES=0 python source_only.py data/mpi3d -d MPI3D -s RC -t RL -a resnet18 --epochs 40 --seed 0 -b 36 --log logs/src_only/MPI3D_RC2RL\nCUDA_VISIBLE_DEVICES=0 python source_only.py data/mpi3d -d MPI3D -s RC -t T -a resnet18 --epochs 40 --seed 0 -b 36 --log logs/src_only/MPI3D_RC2T\nCUDA_VISIBLE_DEVICES=0 python source_only.py data/mpi3d -d MPI3D -s T -t RL -a resnet18 --epochs 40 --seed 0 -b 36 --log logs/src_only/MPI3D_T2RL\nCUDA_VISIBLE_DEVICES=0 python source_only.py data/mpi3d -d MPI3D -s T -t RC -a resnet18 --epochs 40 --seed 0 -b 36 --log logs/src_only/MPI3D_T2RC\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/examples/domain_adaptation/image_regression/utils.py",
"content": "\"\"\"\n@author: Junguang Jiang\n@contact: JiangJunguang1123@outlook.com\n\"\"\"\nimport sys\nimport time\nimport torch\nimport torch.nn.functional as F\n\nsys.path.append('../../..')\nfrom common.utils.meter import AverageMeter, ProgressMeter\nfrom torch.nn.modules.batchnorm import BatchNorm1d, BatchNorm2d, BatchNorm3d\nfrom torch.nn.modules.instancenorm import InstanceNorm1d, InstanceNorm2d, InstanceNorm3d\n\n\ndef convert_model(module):\n \"\"\"convert BatchNorms in the `module` into InstanceNorms\"\"\"\n source_modules = (BatchNorm1d, BatchNorm2d, BatchNorm3d)\n target_modules = (InstanceNorm1d, InstanceNorm2d, InstanceNorm3d)\n for src_module, tgt_module in zip(source_modules, target_modules):\n if isinstance(module, src_module):\n mod = tgt_module(module.num_features, module.eps, module.momentum, module.affine)\n module = mod\n\n for name, child in module.named_children():\n module.add_module(name, convert_model(child))\n\n return module\n\n\ndef validate(val_loader, model, args, factors, device):\n batch_time = AverageMeter('Time', ':6.3f')\n mae_losses = [AverageMeter('mae {}'.format(factor), ':6.3f') for factor in factors]\n progress = ProgressMeter(\n len(val_loader),\n [batch_time] + mae_losses,\n prefix='Test: ')\n\n # switch to evaluate mode\n model.eval()\n\n with torch.no_grad():\n end = time.time()\n for i, (images, target) in enumerate(val_loader):\n images = images.to(device)\n target = target.to(device)\n\n # compute output\n output = model(images)\n for j in range(len(factors)):\n mae_loss = F.l1_loss(output[:, j], target[:, j])\n mae_losses[j].update(mae_loss.item(), images.size(0))\n\n # measure elapsed time\n batch_time.update(time.time() - end)\n end = time.time()\n\n if i % args.print_freq == 0:\n progress.display(i)\n\n for i, factor in enumerate(factors):\n print(\"{} MAE {mae.avg:6.3f}\".format(factor, mae=mae_losses[i]))\n mean_mae = sum(l.avg for l in mae_losses) / len(factors)\n return mean_mae\n\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/examples/domain_adaptation/keypoint_detection/README.md",
"content": "# Unsupervised Domain Adaptation for Keypoint Detection\n\n## Dataset\nFollowing datasets can be downloaded automatically:\n\n- [Rendered Handpose Dataset](https://lmb.informatik.uni-freiburg.de/resources/datasets/RenderedHandposeDataset.en.html)\n- [Hand-3d-Studio Dataset](https://www.yangangwang.com/papers/ZHAO-H3S-2020-02.html)\n- [FreiHAND Dataset](https://lmb.informatik.uni-freiburg.de/projects/freihand/)\n- [Surreal Dataset](https://www.di.ens.fr/willow/research/surreal/data/)\n- [LSP Dataset](http://sam.johnson.io/research/lsp.html)\n\nYou need to prepare following datasets manually if you want to use them:\n- [Human3.6M Dataset](http://vision.imar.ro/human3.6m/description.php)\n\nand prepare them following [Documentations for Human3.6M Dataset](/common/vision/datasets/keypoint_detection/human36m.py).\n\n## Supported Methods\n\nSupported methods include:\n\n- [Regressive Domain Adaptation for Unsupervised Keypoint Detection (RegDA, CVPR 2021)](http://ise.thss.tsinghua.edu.cn/~mlong/doc/regressive-domain-adaptation-cvpr21.pdf)\n\n## Experiment and Results\n\nThe shell files give the script to reproduce the [benchmarks](/docs/dalib/benchmarks/keypoint_detection.rst) with specified hyper-parameters.\nFor example, if you want to train DANN on Office31, use the following script\n\n```shell script\n# Train a RegDA on RHD -> H3D task using PoseResNet.\n# Assume you have put the datasets under the path `data/RHD` and `data/H3D_crop`, \n# or you are glad to download the datasets automatically from the Internet to this path\nCUDA_VISIBLE_DEVICES=0 python regda.py data/RHD data/H3D_crop \\\n -s RenderedHandPose -t Hand3DStudio --finetune --seed 0 --debug --log logs/regda/rhd2h3d\n```\n\nFor more information please refer to [Get Started](/docs/get_started/quickstart.rst) for help.\n\n## TODO\nSupport methods: CycleGAN\n\n\n## Citation\nIf you use these methods in your research, please consider citing.\n\n```\n@InProceedings{RegDA,\n author = {Junguang Jiang and\n Yifei Ji and\n Ximei Wang and\n Yufeng Liu and\n Jianmin Wang and\n Mingsheng Long},\n title = {Regressive Domain Adaptation for Unsupervised Keypoint Detection},\n booktitle = {CVPR},\n year = {2021}\n}\n\n```\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/examples/domain_adaptation/keypoint_detection/regda.py",
"content": "\"\"\"\n@author: Junguang Jiang\n@contact: JiangJunguang1123@outlook.com\n\"\"\"\nimport random\nimport time\nimport warnings\nimport sys\nimport argparse\nimport shutil\n\nimport torch\nimport torch.backends.cudnn as cudnn\nfrom torch.optim import SGD\nfrom torch.optim.lr_scheduler import LambdaLR, MultiStepLR\nfrom torch.utils.data import DataLoader\nfrom torchvision.transforms import Compose, ToPILImage\n\nsys.path.append('../../..')\nfrom dalib.adaptation.regda import PoseResNet2d as RegDAPoseResNet, \\\n PseudoLabelGenerator2d, RegressionDisparity\nimport common.vision.models as models\nfrom common.vision.models.keypoint_detection.pose_resnet import Upsampling, PoseResNet\nfrom common.vision.models.keypoint_detection.loss import JointsKLLoss\nimport common.vision.datasets.keypoint_detection as datasets\nimport common.vision.transforms.keypoint_detection as T\nfrom common.vision.transforms import Denormalize\nfrom common.utils.data import ForeverDataIterator\nfrom common.utils.meter import AverageMeter, ProgressMeter, AverageMeterDict\nfrom common.utils.metric.keypoint_detection import accuracy\nfrom common.utils.logger import CompleteLogger\n\ndevice = torch.device(\"cuda\" if torch.cuda.is_available() else \"cpu\")\n\n\ndef main(args: argparse.Namespace):\n logger = CompleteLogger(args.log, args.phase)\n print(args)\n\n if args.seed is not None:\n random.seed(args.seed)\n torch.manual_seed(args.seed)\n cudnn.deterministic = True\n warnings.warn('You have chosen to seed training. '\n 'This will turn on the CUDNN deterministic setting, '\n 'which can slow down your training considerably! '\n 'You may see unexpected behavior when restarting '\n 'from checkpoints.')\n\n cudnn.benchmark = True\n\n # Data loading code\n normalize = T.Normalize([0.485, 0.456, 0.406], [0.229, 0.224, 0.225])\n train_transform = T.Compose([\n T.RandomRotation(args.rotation),\n T.RandomResizedCrop(size=args.image_size, scale=args.resize_scale),\n T.ColorJitter(brightness=0.25, contrast=0.25, saturation=0.25),\n T.GaussianBlur(),\n T.ToTensor(),\n normalize\n ])\n val_transform = T.Compose([\n T.Resize(args.image_size),\n T.ToTensor(),\n normalize\n ])\n image_size = (args.image_size, args.image_size)\n heatmap_size = (args.heatmap_size, args.heatmap_size)\n source_dataset = datasets.__dict__[args.source]\n train_source_dataset = source_dataset(root=args.source_root, transforms=train_transform,\n image_size=image_size, heatmap_size=heatmap_size)\n train_source_loader = DataLoader(train_source_dataset, batch_size=args.batch_size,\n shuffle=True, num_workers=args.workers, pin_memory=True, drop_last=True)\n val_source_dataset = source_dataset(root=args.source_root, split='test', transforms=val_transform,\n image_size=image_size, heatmap_size=heatmap_size)\n val_source_loader = DataLoader(val_source_dataset, batch_size=args.batch_size, shuffle=False, pin_memory=True)\n\n target_dataset = datasets.__dict__[args.target]\n train_target_dataset = target_dataset(root=args.target_root, transforms=train_transform,\n image_size=image_size, heatmap_size=heatmap_size)\n train_target_loader = DataLoader(train_target_dataset, batch_size=args.batch_size,\n shuffle=True, num_workers=args.workers, pin_memory=True, drop_last=True)\n val_target_dataset = target_dataset(root=args.target_root, split='test', transforms=val_transform,\n image_size=image_size, heatmap_size=heatmap_size)\n val_target_loader = DataLoader(val_target_dataset, batch_size=args.batch_size, shuffle=False, pin_memory=True)\n\n print(\"Source train:\", len(train_source_loader))\n print(\"Target train:\", len(train_target_loader))\n print(\"Source test:\", len(val_source_loader))\n print(\"Target test:\", len(val_target_loader))\n\n train_source_iter = ForeverDataIterator(train_source_loader)\n train_target_iter = ForeverDataIterator(train_target_loader)\n\n # create model\n backbone = models.__dict__[args.arch](pretrained=True)\n upsampling = Upsampling(backbone.out_features)\n num_keypoints = train_source_dataset.num_keypoints\n model = RegDAPoseResNet(backbone, upsampling, 256, num_keypoints, num_head_layers=args.num_head_layers, finetune=True).to(device)\n # define loss function\n criterion = JointsKLLoss()\n pseudo_label_generator = PseudoLabelGenerator2d(num_keypoints, args.heatmap_size, args.heatmap_size)\n regression_disparity = RegressionDisparity(pseudo_label_generator, JointsKLLoss(epsilon=1e-7))\n\n # define optimizer and lr scheduler\n optimizer_f = SGD([\n {'params': backbone.parameters(), 'lr': 0.1},\n {'params': upsampling.parameters(), 'lr': 0.1},\n ], lr=0.1, momentum=args.momentum, weight_decay=args.wd, nesterov=True)\n optimizer_h = SGD(model.head.parameters(), lr=1., momentum=args.momentum, weight_decay=args.wd, nesterov=True)\n optimizer_h_adv = SGD(model.head_adv.parameters(), lr=1., momentum=args.momentum, weight_decay=args.wd, nesterov=True)\n lr_decay_function = lambda x: args.lr * (1. + args.lr_gamma * float(x)) ** (-args.lr_decay)\n lr_scheduler_f = LambdaLR(optimizer_f, lr_decay_function)\n lr_scheduler_h = LambdaLR(optimizer_h, lr_decay_function)\n lr_scheduler_h_adv = LambdaLR(optimizer_h_adv, lr_decay_function)\n start_epoch = 0\n\n if args.resume is None:\n if args.pretrain is None:\n # first pretrain the backbone and upsampling\n print(\"Pretraining the model on source domain.\")\n args.pretrain = logger.get_checkpoint_path('pretrain')\n pretrained_model = PoseResNet(backbone, upsampling, 256, num_keypoints, True).to(device)\n optimizer = SGD(pretrained_model.get_parameters(lr=args.lr), momentum=args.momentum, weight_decay=args.wd, nesterov=True)\n lr_scheduler = MultiStepLR(optimizer, args.lr_step, args.lr_factor)\n best_acc = 0\n for epoch in range(args.pretrain_epochs):\n lr_scheduler.step()\n print(lr_scheduler.get_lr())\n\n pretrain(train_source_iter, pretrained_model, criterion, optimizer, epoch, args)\n source_val_acc = validate(val_source_loader, pretrained_model, criterion, None, args)\n\n # remember best acc and save checkpoint\n if source_val_acc['all'] > best_acc:\n best_acc = source_val_acc['all']\n torch.save(\n {\n 'model': pretrained_model.state_dict()\n }, args.pretrain\n )\n print(\"Source: {} best: {}\".format(source_val_acc['all'], best_acc))\n\n # load from the pretrained checkpoint\n pretrained_dict = torch.load(args.pretrain, map_location='cpu')['model']\n model_dict = model.state_dict()\n # remove keys from pretrained dict that doesn't appear in model dict\n pretrained_dict = {k: v for k, v in pretrained_dict.items() if k in model_dict}\n model.load_state_dict(pretrained_dict, strict=False)\n else:\n # optionally resume from a checkpoint\n checkpoint = torch.load(args.resume, map_location='cpu')\n model.load_state_dict(checkpoint['model'])\n optimizer_f.load_state_dict(checkpoint['optimizer_f'])\n optimizer_h.load_state_dict(checkpoint['optimizer_h'])\n optimizer_h_adv.load_state_dict(checkpoint['optimizer_h_adv'])\n lr_scheduler_f.load_state_dict(checkpoint['lr_scheduler_f'])\n lr_scheduler_h.load_state_dict(checkpoint['lr_scheduler_h'])\n lr_scheduler_h_adv.load_state_dict(checkpoint['lr_scheduler_h_adv'])\n start_epoch = checkpoint['epoch'] + 1\n\n # define visualization function\n tensor_to_image = Compose([\n Denormalize([0.485, 0.456, 0.406], [0.229, 0.224, 0.225]),\n ToPILImage()\n ])\n\n def visualize(image, keypoint2d, name, heatmaps=None):\n \"\"\"\n Args:\n image (tensor): image in shape 3 x H x W\n keypoint2d (tensor): keypoints in shape K x 2\n name: name of the saving image\n \"\"\"\n train_source_dataset.visualize(tensor_to_image(image),\n keypoint2d, logger.get_image_path(\"{}.jpg\".format(name)))\n\n if args.phase == 'test':\n # evaluate on validation set\n source_val_acc = validate(val_source_loader, model, criterion, None, args)\n target_val_acc = validate(val_target_loader, model, criterion, visualize, args)\n print(\"Source: {:4.3f} Target: {:4.3f}\".format(source_val_acc['all'], target_val_acc['all']))\n for name, acc in target_val_acc.items():\n print(\"{}: {:4.3f}\".format(name, acc))\n return\n\n # start training\n best_acc = 0\n print(\"Start regression domain adaptation.\")\n for epoch in range(start_epoch, args.epochs):\n logger.set_epoch(epoch)\n print(lr_scheduler_f.get_lr(), lr_scheduler_h.get_lr(), lr_scheduler_h_adv.get_lr())\n\n # train for one epoch\n train(train_source_iter, train_target_iter, model, criterion, regression_disparity,\n optimizer_f, optimizer_h, optimizer_h_adv, lr_scheduler_f, lr_scheduler_h, lr_scheduler_h_adv,\n epoch, visualize if args.debug else None, args)\n\n # evaluate on validation set\n source_val_acc = validate(val_source_loader, model, criterion, None, args)\n target_val_acc = validate(val_target_loader, model, criterion, visualize if args.debug else None, args)\n\n # remember best acc and save checkpoint\n torch.save(\n {\n 'model': model.state_dict(),\n 'optimizer_f': optimizer_f.state_dict(),\n 'optimizer_h': optimizer_h.state_dict(),\n 'optimizer_h_adv': optimizer_h_adv.state_dict(),\n 'lr_scheduler_f': lr_scheduler_f.state_dict(),\n 'lr_scheduler_h': lr_scheduler_h.state_dict(),\n 'lr_scheduler_h_adv': lr_scheduler_h_adv.state_dict(),\n 'epoch': epoch,\n 'args': args\n }, logger.get_checkpoint_path(epoch)\n )\n if target_val_acc['all'] > best_acc:\n shutil.copy(logger.get_checkpoint_path(epoch), logger.get_checkpoint_path('best'))\n best_acc = target_val_acc['all']\n print(\"Source: {:4.3f} Target: {:4.3f} Target(best): {:4.3f}\".format(source_val_acc['all'], target_val_acc['all'], best_acc))\n for name, acc in target_val_acc.items():\n print(\"{}: {:4.3f}\".format(name, acc))\n\n logger.close()\n\n\ndef pretrain(train_source_iter, model, criterion, optimizer,\n epoch: int, args: argparse.Namespace):\n batch_time = AverageMeter('Time', ':4.2f')\n data_time = AverageMeter('Data', ':3.1f')\n losses_s = AverageMeter('Loss (s)', \":.2e\")\n acc_s = AverageMeter(\"Acc (s)\", \":3.2f\")\n\n progress = ProgressMeter(\n args.iters_per_epoch,\n [batch_time, data_time, losses_s, acc_s],\n prefix=\"Epoch: [{}]\".format(epoch))\n\n # switch to train mode\n model.train()\n\n end = time.time()\n for i in range(args.iters_per_epoch):\n optimizer.zero_grad()\n\n x_s, label_s, weight_s, meta_s = next(train_source_iter)\n\n x_s = x_s.to(device)\n label_s = label_s.to(device)\n weight_s = weight_s.to(device)\n\n # measure data loading time\n data_time.update(time.time() - end)\n\n # compute output\n y_s = model(x_s)\n loss_s = criterion(y_s, label_s, weight_s)\n\n # compute gradient and do SGD step\n loss_s.backward()\n optimizer.step()\n\n # measure accuracy and record loss\n _, avg_acc_s, cnt_s, pred_s = accuracy(y_s.detach().cpu().numpy(),\n label_s.detach().cpu().numpy())\n acc_s.update(avg_acc_s, cnt_s)\n losses_s.update(loss_s, cnt_s)\n\n # measure elapsed time\n batch_time.update(time.time() - end)\n end = time.time()\n\n if i % args.print_freq == 0:\n progress.display(i)\n\n\ndef train(train_source_iter, train_target_iter, model, criterion,regression_disparity,\n optimizer_f, optimizer_h, optimizer_h_adv, lr_scheduler_f, lr_scheduler_h, lr_scheduler_h_adv,\n epoch: int, visualize, args: argparse.Namespace):\n batch_time = AverageMeter('Time', ':4.2f')\n data_time = AverageMeter('Data', ':3.1f')\n losses_s = AverageMeter('Loss (s)', \":.2e\")\n losses_gf = AverageMeter('Loss (t, false)', \":.2e\")\n losses_gt = AverageMeter('Loss (t, truth)', \":.2e\")\n acc_s = AverageMeter(\"Acc (s)\", \":3.2f\")\n acc_t = AverageMeter(\"Acc (t)\", \":3.2f\")\n acc_s_adv = AverageMeter(\"Acc (s, adv)\", \":3.2f\")\n acc_t_adv = AverageMeter(\"Acc (t, adv)\", \":3.2f\")\n\n progress = ProgressMeter(\n args.iters_per_epoch,\n [batch_time, data_time, losses_s, losses_gf, losses_gt, acc_s, acc_t, acc_s_adv, acc_t_adv],\n prefix=\"Epoch: [{}]\".format(epoch))\n\n # switch to train mode\n model.train()\n\n end = time.time()\n for i in range(args.iters_per_epoch):\n x_s, label_s, weight_s, meta_s = next(train_source_iter)\n x_t, label_t, weight_t, meta_t = next(train_target_iter)\n\n x_s = x_s.to(device)\n label_s = label_s.to(device)\n weight_s = weight_s.to(device)\n\n x_t = x_t.to(device)\n label_t = label_t.to(device)\n weight_t = weight_t.to(device)\n\n # measure data loading time\n data_time.update(time.time() - end)\n\n # Step A train all networks to minimize loss on source domain\n optimizer_f.zero_grad()\n optimizer_h.zero_grad()\n optimizer_h_adv.zero_grad()\n\n y_s, y_s_adv = model(x_s)\n loss_s = criterion(y_s, label_s, weight_s) + \\\n args.margin * args.trade_off * regression_disparity(y_s, y_s_adv, weight_s, mode='min')\n loss_s.backward()\n optimizer_f.step()\n optimizer_h.step()\n optimizer_h_adv.step()\n\n # Step B train adv regressor to maximize regression disparity\n optimizer_h_adv.zero_grad()\n y_t, y_t_adv = model(x_t)\n loss_ground_false = args.trade_off * regression_disparity(y_t, y_t_adv, weight_t, mode='max')\n loss_ground_false.backward()\n optimizer_h_adv.step()\n\n # Step C train feature extractor to minimize regression disparity\n optimizer_f.zero_grad()\n y_t, y_t_adv = model(x_t)\n loss_ground_truth = args.trade_off * regression_disparity(y_t, y_t_adv, weight_t, mode='min')\n loss_ground_truth.backward()\n optimizer_f.step()\n\n # do update step\n model.step()\n lr_scheduler_f.step()\n lr_scheduler_h.step()\n lr_scheduler_h_adv.step()\n\n # measure accuracy and record loss\n _, avg_acc_s, cnt_s, pred_s = accuracy(y_s.detach().cpu().numpy(),\n label_s.detach().cpu().numpy())\n acc_s.update(avg_acc_s, cnt_s)\n _, avg_acc_t, cnt_t, pred_t = accuracy(y_t.detach().cpu().numpy(),\n label_t.detach().cpu().numpy())\n acc_t.update(avg_acc_t, cnt_t)\n _, avg_acc_s_adv, cnt_s_adv, pred_s_adv = accuracy(y_s_adv.detach().cpu().numpy(),\n label_s.detach().cpu().numpy())\n acc_s_adv.update(avg_acc_s_adv, cnt_s)\n _, avg_acc_t_adv, cnt_t_adv, pred_t_adv = accuracy(y_t_adv.detach().cpu().numpy(),\n label_t.detach().cpu().numpy())\n acc_t_adv.update(avg_acc_t_adv, cnt_t)\n losses_s.update(loss_s, cnt_s)\n losses_gf.update(loss_ground_false, cnt_s)\n losses_gt.update(loss_ground_truth, cnt_s)\n\n # measure elapsed time\n batch_time.update(time.time() - end)\n end = time.time()\n\n if i % args.print_freq == 0:\n progress.display(i)\n if visualize is not None:\n visualize(x_s[0], pred_s[0] * args.image_size / args.heatmap_size, \"source_{}_pred\".format(i))\n visualize(x_s[0], meta_s['keypoint2d'][0], \"source_{}_label\".format(i))\n visualize(x_t[0], pred_t[0] * args.image_size / args.heatmap_size, \"target_{}_pred\".format(i))\n visualize(x_t[0], meta_t['keypoint2d'][0], \"target_{}_label\".format(i))\n visualize(x_s[0], pred_s_adv[0] * args.image_size / args.heatmap_size, \"source_adv_{}_pred\".format(i))\n visualize(x_t[0], pred_t_adv[0] * args.image_size / args.heatmap_size, \"target_adv_{}_pred\".format(i))\n\n\ndef validate(val_loader, model, criterion, visualize, args: argparse.Namespace):\n batch_time = AverageMeter('Time', ':6.3f')\n losses = AverageMeter('Loss', ':.2e')\n acc = AverageMeterDict(val_loader.dataset.keypoints_group.keys(), \":3.2f\")\n progress = ProgressMeter(\n len(val_loader),\n [batch_time, losses, acc['all']],\n prefix='Test: ')\n\n # switch to evaluate mode\n model.eval()\n\n with torch.no_grad():\n end = time.time()\n for i, (x, label, weight, meta) in enumerate(val_loader):\n x = x.to(device)\n label = label.to(device)\n weight = weight.to(device)\n\n # compute output\n y = model(x)\n loss = criterion(y, label, weight)\n\n # measure accuracy and record loss\n losses.update(loss.item(), x.size(0))\n acc_per_points, avg_acc, cnt, pred = accuracy(y.cpu().numpy(),\n label.cpu().numpy())\n\n group_acc = val_loader.dataset.group_accuracy(acc_per_points)\n acc.update(group_acc, x.size(0))\n\n # measure elapsed time\n batch_time.update(time.time() - end)\n end = time.time()\n\n if i % args.print_freq == 0:\n progress.display(i)\n if visualize is not None:\n visualize(x[0], pred[0] * args.image_size / args.heatmap_size, \"val_{}_pred.jpg\".format(i))\n visualize(x[0], meta['keypoint2d'][0], \"val_{}_label.jpg\".format(i))\n\n return acc.average()\n\n\nif __name__ == '__main__':\n architecture_names = sorted(\n name for name in models.__dict__\n if name.islower() and not name.startswith(\"__\")\n and callable(models.__dict__[name])\n )\n dataset_names = sorted(\n name for name in datasets.__dict__\n if not name.startswith(\"__\") and callable(datasets.__dict__[name])\n )\n\n parser = argparse.ArgumentParser(description='RegDA for Keypoint Detection Domain Adaptation')\n # dataset parameters\n parser.add_argument('source_root', help='root path of the source dataset')\n parser.add_argument('target_root', help='root path of the target dataset')\n parser.add_argument('-s', '--source', help='source domain(s)')\n parser.add_argument('-t', '--target', help='target domain(s)')\n parser.add_argument('--resize-scale', nargs='+', type=float, default=(0.6, 1.3),\n help='scale range for the RandomResizeCrop augmentation')\n parser.add_argument('--rotation', type=int, default=180,\n help='rotation range of the RandomRotation augmentation')\n parser.add_argument('--image-size', type=int, default=256,\n help='input image size')\n parser.add_argument('--heatmap-size', type=int, default=64,\n help='output heatmap size')\n # model parameters\n parser.add_argument('-a', '--arch', metavar='ARCH', default='resnet101',\n choices=architecture_names,\n help='backbone architecture: ' +\n ' | '.join(architecture_names) +\n ' (default: resnet101)')\n parser.add_argument(\"--pretrain\", type=str, default=None,\n help=\"Where restore pretrained model parameters from.\")\n parser.add_argument(\"--resume\", type=str, default=None,\n help=\"where restore model parameters from.\")\n parser.add_argument('--num-head-layers', type=int, default=2)\n parser.add_argument('--margin', type=float, default=4., help=\"margin gamma\")\n parser.add_argument('--trade-off', default=1., type=float,\n help='the trade-off hyper-parameter for transfer loss')\n # training parameters\n parser.add_argument('-b', '--batch-size', default=32, type=int,\n metavar='N',\n help='mini-batch size (default: 32)')\n parser.add_argument('--lr', '--learning-rate', default=0.1, type=float,\n metavar='LR', help='initial learning rate', dest='lr')\n parser.add_argument('--momentum', default=0.9, type=float, metavar='M',\n help='momentum')\n parser.add_argument('--wd', '--weight-decay', default=0.0001, type=float,\n metavar='W', help='weight decay (default: 1e-4)')\n parser.add_argument('--lr-gamma', default=0.0001, type=float)\n parser.add_argument('--lr-decay', default=0.75, type=float, help='parameter for lr scheduler')\n parser.add_argument('--lr-step', default=[45, 60], type=tuple, help='parameter for lr scheduler')\n parser.add_argument('--lr-factor', default=0.1, type=float, help='parameter for lr scheduler')\n parser.add_argument('-j', '--workers', default=4, type=int, metavar='N',\n help='number of data loading workers (default: 4)')\n parser.add_argument('--pretrain_epochs', default=70, type=int, metavar='N',\n help='number of total epochs to run')\n parser.add_argument('--epochs', default=30, type=int, metavar='N',\n help='number of total epochs to run')\n parser.add_argument('-i', '--iters-per-epoch', default=500, type=int,\n help='Number of iterations per epoch')\n parser.add_argument('-p', '--print-freq', default=100, type=int,\n metavar='N', help='print frequency (default: 100)')\n parser.add_argument('--seed', default=None, type=int,\n help='seed for initializing training. ')\n parser.add_argument(\"--log\", type=str, default='regda',\n help=\"Where to save logs, checkpoints and debugging images.\")\n parser.add_argument(\"--phase\", type=str, default='train', choices=['train', 'test'],\n help=\"When phase is 'test', only test the model.\")\n parser.add_argument('--debug', action=\"store_true\",\n help='In the debug mode, save images and predictions')\n args = parser.parse_args()\n main(args)\n\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/examples/domain_adaptation/keypoint_detection/regda.sh",
"content": "# Hands Dataset\nCUDA_VISIBLE_DEVICES=0 python regda.py data/RHD data/H3D_crop \\\n -s RenderedHandPose -t Hand3DStudio --seed 0 --debug --log logs/regda/rhd2h3d\nCUDA_VISIBLE_DEVICES=0 python regda.py data/FreiHand data/RHD \\\n -s FreiHand -t RenderedHandPose --seed 0 --debug --log logs/regda/freihand2rhd\n\n# Body Dataset\nCUDA_VISIBLE_DEVICES=0 python regda.py data/surreal_processed data/Human36M \\\n -s SURREAL -t Human36M --finetune --seed 0 --debug --rotation 30 --epochs 10 --log logs/regda/surreal2human36m\nCUDA_VISIBLE_DEVICES=0 python regda.py data/surreal_processed data/lsp \\\n -s SURREAL -t LSP --finetune --seed 0 --debug --rotation 30 --log logs/regda/surreal2lsp\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/examples/domain_adaptation/keypoint_detection/regda_fast.py",
"content": "\"\"\"\n@author: Junguang Jiang\n@contact: JiangJunguang1123@outlook.com\n\"\"\"\nimport random\nimport time\nimport warnings\nimport sys\nimport argparse\nimport shutil\n\nimport torch\nimport torch.backends.cudnn as cudnn\nfrom torch.optim import SGD\nfrom torch.optim.lr_scheduler import LambdaLR, MultiStepLR\nfrom torch.utils.data import DataLoader\nfrom torchvision.transforms import Compose, ToPILImage\n\nsys.path.append('../../..')\nfrom dalib.adaptation.regda import PoseResNet2d as RegDAPoseResNet, \\\n FastPseudoLabelGenerator2d, RegressionDisparity\nimport common.vision.models as models\nfrom common.vision.models.keypoint_detection.pose_resnet import Upsampling, PoseResNet\nfrom common.vision.models.keypoint_detection.loss import JointsKLLoss\nimport common.vision.datasets.keypoint_detection as datasets\nimport common.vision.transforms.keypoint_detection as T\nfrom common.vision.transforms import Denormalize\nfrom common.utils.data import ForeverDataIterator\nfrom common.utils.meter import AverageMeter, ProgressMeter, AverageMeterDict\nfrom common.utils.metric.keypoint_detection import accuracy\nfrom common.utils.logger import CompleteLogger\n\ndevice = torch.device(\"cuda\" if torch.cuda.is_available() else \"cpu\")\n\n\ndef main(args: argparse.Namespace):\n logger = CompleteLogger(args.log, args.phase)\n print(args)\n\n if args.seed is not None:\n random.seed(args.seed)\n torch.manual_seed(args.seed)\n cudnn.deterministic = True\n warnings.warn('You have chosen to seed training. '\n 'This will turn on the CUDNN deterministic setting, '\n 'which can slow down your training considerably! '\n 'You may see unexpected behavior when restarting '\n 'from checkpoints.')\n\n cudnn.benchmark = True\n\n # Data loading code\n normalize = T.Normalize([0.485, 0.456, 0.406], [0.229, 0.224, 0.225])\n train_transform = T.Compose([\n T.RandomRotation(args.rotation),\n T.RandomResizedCrop(size=args.image_size, scale=args.resize_scale),\n T.ColorJitter(brightness=0.25, contrast=0.25, saturation=0.25),\n T.GaussianBlur(),\n T.ToTensor(),\n normalize\n ])\n val_transform = T.Compose([\n T.Resize(args.image_size),\n T.ToTensor(),\n normalize\n ])\n image_size = (args.image_size, args.image_size)\n heatmap_size = (args.heatmap_size, args.heatmap_size)\n source_dataset = datasets.__dict__[args.source]\n train_source_dataset = source_dataset(root=args.source_root, transforms=train_transform,\n image_size=image_size, heatmap_size=heatmap_size)\n train_source_loader = DataLoader(train_source_dataset, batch_size=args.batch_size,\n shuffle=True, num_workers=args.workers, pin_memory=True, drop_last=True)\n val_source_dataset = source_dataset(root=args.source_root, split='test', transforms=val_transform,\n image_size=image_size, heatmap_size=heatmap_size)\n val_source_loader = DataLoader(val_source_dataset, batch_size=args.batch_size, shuffle=False, pin_memory=True)\n\n target_dataset = datasets.__dict__[args.target]\n train_target_dataset = target_dataset(root=args.target_root, transforms=train_transform,\n image_size=image_size, heatmap_size=heatmap_size)\n train_target_loader = DataLoader(train_target_dataset, batch_size=args.batch_size,\n shuffle=True, num_workers=args.workers, pin_memory=True, drop_last=True)\n val_target_dataset = target_dataset(root=args.target_root, split='test', transforms=val_transform,\n image_size=image_size, heatmap_size=heatmap_size)\n val_target_loader = DataLoader(val_target_dataset, batch_size=args.batch_size, shuffle=False, pin_memory=True)\n\n print(\"Source train:\", len(train_source_loader))\n print(\"Target train:\", len(train_target_loader))\n print(\"Source test:\", len(val_source_loader))\n print(\"Target test:\", len(val_target_loader))\n\n train_source_iter = ForeverDataIterator(train_source_loader)\n train_target_iter = ForeverDataIterator(train_target_loader)\n\n # create model\n backbone = models.__dict__[args.arch](pretrained=True)\n upsampling = Upsampling(backbone.out_features)\n num_keypoints = train_source_dataset.num_keypoints\n model = RegDAPoseResNet(backbone, upsampling, 256, num_keypoints, num_head_layers=args.num_head_layers, finetune=True).to(device)\n # define loss function\n criterion = JointsKLLoss()\n pseudo_label_generator = FastPseudoLabelGenerator2d()\n regression_disparity = RegressionDisparity(pseudo_label_generator, JointsKLLoss(epsilon=1e-7))\n\n # define optimizer and lr scheduler\n optimizer_f = SGD([\n {'params': backbone.parameters(), 'lr': 0.1},\n {'params': upsampling.parameters(), 'lr': 0.1},\n ], lr=0.1, momentum=args.momentum, weight_decay=args.wd, nesterov=True)\n optimizer_h = SGD(model.head.parameters(), lr=1., momentum=args.momentum, weight_decay=args.wd, nesterov=True)\n optimizer_h_adv = SGD(model.head_adv.parameters(), lr=1., momentum=args.momentum, weight_decay=args.wd, nesterov=True)\n lr_decay_function = lambda x: args.lr * (1. + args.lr_gamma * float(x)) ** (-args.lr_decay)\n lr_scheduler_f = LambdaLR(optimizer_f, lr_decay_function)\n lr_scheduler_h = LambdaLR(optimizer_h, lr_decay_function)\n lr_scheduler_h_adv = LambdaLR(optimizer_h_adv, lr_decay_function)\n start_epoch = 0\n\n if args.resume is None:\n if args.pretrain is None:\n # first pretrain the backbone and upsampling\n print(\"Pretraining the model on source domain.\")\n args.pretrain = logger.get_checkpoint_path('pretrain')\n pretrained_model = PoseResNet(backbone, upsampling, 256, num_keypoints, True).to(device)\n optimizer = SGD(pretrained_model.get_parameters(lr=args.lr), momentum=args.momentum, weight_decay=args.wd, nesterov=True)\n lr_scheduler = MultiStepLR(optimizer, args.lr_step, args.lr_factor)\n best_acc = 0\n for epoch in range(args.pretrain_epochs):\n lr_scheduler.step()\n print(lr_scheduler.get_lr())\n\n pretrain(train_source_iter, pretrained_model, criterion, optimizer, epoch, args)\n source_val_acc = validate(val_source_loader, pretrained_model, criterion, None, args)\n\n # remember best acc and save checkpoint\n if source_val_acc['all'] > best_acc:\n best_acc = source_val_acc['all']\n torch.save(\n {\n 'model': pretrained_model.state_dict()\n }, args.pretrain\n )\n print(\"Source: {} best: {}\".format(source_val_acc['all'], best_acc))\n\n # load from the pretrained checkpoint\n pretrained_dict = torch.load(args.pretrain, map_location='cpu')['model']\n model_dict = model.state_dict()\n # remove keys from pretrained dict that doesn't appear in model dict\n pretrained_dict = {k: v for k, v in pretrained_dict.items() if k in model_dict}\n model.load_state_dict(pretrained_dict, strict=False)\n else:\n # optionally resume from a checkpoint\n checkpoint = torch.load(args.resume, map_location='cpu')\n model.load_state_dict(checkpoint['model'])\n optimizer_f.load_state_dict(checkpoint['optimizer_f'])\n optimizer_h.load_state_dict(checkpoint['optimizer_h'])\n optimizer_h_adv.load_state_dict(checkpoint['optimizer_h_adv'])\n lr_scheduler_f.load_state_dict(checkpoint['lr_scheduler_f'])\n lr_scheduler_h.load_state_dict(checkpoint['lr_scheduler_h'])\n lr_scheduler_h_adv.load_state_dict(checkpoint['lr_scheduler_h_adv'])\n start_epoch = checkpoint['epoch'] + 1\n\n # define visualization function\n tensor_to_image = Compose([\n Denormalize([0.485, 0.456, 0.406], [0.229, 0.224, 0.225]),\n ToPILImage()\n ])\n\n def visualize(image, keypoint2d, name, heatmaps=None):\n \"\"\"\n Args:\n image (tensor): image in shape 3 x H x W\n keypoint2d (tensor): keypoints in shape K x 2\n name: name of the saving image\n \"\"\"\n train_source_dataset.visualize(tensor_to_image(image),\n keypoint2d, logger.get_image_path(\"{}.jpg\".format(name)))\n\n if args.phase == 'test':\n # evaluate on validation set\n source_val_acc = validate(val_source_loader, model, criterion, None, args)\n target_val_acc = validate(val_target_loader, model, criterion, visualize, args)\n print(\"Source: {:4.3f} Target: {:4.3f}\".format(source_val_acc['all'], target_val_acc['all']))\n for name, acc in target_val_acc.items():\n print(\"{}: {:4.3f}\".format(name, acc))\n return\n\n # start training\n best_acc = 0\n print(\"Start regression domain adaptation.\")\n for epoch in range(start_epoch, args.epochs):\n logger.set_epoch(epoch)\n print(lr_scheduler_f.get_lr(), lr_scheduler_h.get_lr(), lr_scheduler_h_adv.get_lr())\n\n # train for one epoch\n train(train_source_iter, train_target_iter, model, criterion, regression_disparity,\n optimizer_f, optimizer_h, optimizer_h_adv, lr_scheduler_f, lr_scheduler_h, lr_scheduler_h_adv,\n epoch, visualize if args.debug else None, args)\n\n # evaluate on validation set\n source_val_acc = validate(val_source_loader, model, criterion, None, args)\n target_val_acc = validate(val_target_loader, model, criterion, visualize if args.debug else None, args)\n\n # remember best acc and save checkpoint\n torch.save(\n {\n 'model': model.state_dict(),\n 'optimizer_f': optimizer_f.state_dict(),\n 'optimizer_h': optimizer_h.state_dict(),\n 'optimizer_h_adv': optimizer_h_adv.state_dict(),\n 'lr_scheduler_f': lr_scheduler_f.state_dict(),\n 'lr_scheduler_h': lr_scheduler_h.state_dict(),\n 'lr_scheduler_h_adv': lr_scheduler_h_adv.state_dict(),\n 'epoch': epoch,\n 'args': args\n }, logger.get_checkpoint_path(epoch)\n )\n if target_val_acc['all'] > best_acc:\n shutil.copy(logger.get_checkpoint_path(epoch), logger.get_checkpoint_path('best'))\n best_acc = target_val_acc['all']\n print(\"Source: {:4.3f} Target: {:4.3f} Target(best): {:4.3f}\".format(source_val_acc['all'], target_val_acc['all'], best_acc))\n for name, acc in target_val_acc.items():\n print(\"{}: {:4.3f}\".format(name, acc))\n\n logger.close()\n\n\ndef pretrain(train_source_iter, model, criterion, optimizer,\n epoch: int, args: argparse.Namespace):\n batch_time = AverageMeter('Time', ':4.2f')\n data_time = AverageMeter('Data', ':3.1f')\n losses_s = AverageMeter('Loss (s)', \":.2e\")\n acc_s = AverageMeter(\"Acc (s)\", \":3.2f\")\n\n progress = ProgressMeter(\n args.iters_per_epoch,\n [batch_time, data_time, losses_s, acc_s],\n prefix=\"Epoch: [{}]\".format(epoch))\n\n # switch to train mode\n model.train()\n\n end = time.time()\n for i in range(args.iters_per_epoch):\n optimizer.zero_grad()\n\n x_s, label_s, weight_s, meta_s = next(train_source_iter)\n\n x_s = x_s.to(device)\n label_s = label_s.to(device)\n weight_s = weight_s.to(device)\n\n # measure data loading time\n data_time.update(time.time() - end)\n\n # compute output\n y_s = model(x_s)\n loss_s = criterion(y_s, label_s, weight_s)\n\n # compute gradient and do SGD step\n loss_s.backward()\n optimizer.step()\n\n # measure accuracy and record loss\n _, avg_acc_s, cnt_s, pred_s = accuracy(y_s.detach().cpu().numpy(),\n label_s.detach().cpu().numpy())\n acc_s.update(avg_acc_s, cnt_s)\n losses_s.update(loss_s, cnt_s)\n\n # measure elapsed time\n batch_time.update(time.time() - end)\n end = time.time()\n\n if i % args.print_freq == 0:\n progress.display(i)\n\n\ndef train(train_source_iter, train_target_iter, model, criterion,regression_disparity,\n optimizer_f, optimizer_h, optimizer_h_adv, lr_scheduler_f, lr_scheduler_h, lr_scheduler_h_adv,\n epoch: int, visualize, args: argparse.Namespace):\n batch_time = AverageMeter('Time', ':4.2f')\n data_time = AverageMeter('Data', ':3.1f')\n losses_s = AverageMeter('Loss (s)', \":.2e\")\n losses_gf = AverageMeter('Loss (t, false)', \":.2e\")\n losses_gt = AverageMeter('Loss (t, truth)', \":.2e\")\n acc_s = AverageMeter(\"Acc (s)\", \":3.2f\")\n acc_t = AverageMeter(\"Acc (t)\", \":3.2f\")\n acc_s_adv = AverageMeter(\"Acc (s, adv)\", \":3.2f\")\n acc_t_adv = AverageMeter(\"Acc (t, adv)\", \":3.2f\")\n\n progress = ProgressMeter(\n args.iters_per_epoch,\n [batch_time, data_time, losses_s, losses_gf, losses_gt, acc_s, acc_t, acc_s_adv, acc_t_adv],\n prefix=\"Epoch: [{}]\".format(epoch))\n\n # switch to train mode\n model.train()\n\n end = time.time()\n for i in range(args.iters_per_epoch):\n x_s, label_s, weight_s, meta_s = next(train_source_iter)\n x_t, label_t, weight_t, meta_t = next(train_target_iter)\n\n x_s = x_s.to(device)\n label_s = label_s.to(device)\n weight_s = weight_s.to(device)\n\n x_t = x_t.to(device)\n label_t = label_t.to(device)\n weight_t = weight_t.to(device)\n\n # measure data loading time\n data_time.update(time.time() - end)\n\n # Step A train all networks to minimize loss on source domain\n optimizer_f.zero_grad()\n optimizer_h.zero_grad()\n optimizer_h_adv.zero_grad()\n\n y_s, y_s_adv = model(x_s)\n loss_s = criterion(y_s, label_s, weight_s) + \\\n args.margin * args.trade_off * regression_disparity(y_s, y_s_adv, weight_s, mode='min')\n loss_s.backward()\n optimizer_f.step()\n optimizer_h.step()\n optimizer_h_adv.step()\n\n # Step B train adv regressor to maximize regression disparity\n optimizer_h_adv.zero_grad()\n y_t, y_t_adv = model(x_t)\n loss_ground_false = args.trade_off * regression_disparity(y_t, y_t_adv, weight_t, mode='max')\n loss_ground_false.backward()\n optimizer_h_adv.step()\n\n # Step C train feature extractor to minimize regression disparity\n optimizer_f.zero_grad()\n y_t, y_t_adv = model(x_t)\n loss_ground_truth = args.trade_off * regression_disparity(y_t, y_t_adv, weight_t, mode='min')\n loss_ground_truth.backward()\n optimizer_f.step()\n\n # do update step\n model.step()\n lr_scheduler_f.step()\n lr_scheduler_h.step()\n lr_scheduler_h_adv.step()\n\n # measure accuracy and record loss\n _, avg_acc_s, cnt_s, pred_s = accuracy(y_s.detach().cpu().numpy(),\n label_s.detach().cpu().numpy())\n acc_s.update(avg_acc_s, cnt_s)\n _, avg_acc_t, cnt_t, pred_t = accuracy(y_t.detach().cpu().numpy(),\n label_t.detach().cpu().numpy())\n acc_t.update(avg_acc_t, cnt_t)\n _, avg_acc_s_adv, cnt_s_adv, pred_s_adv = accuracy(y_s_adv.detach().cpu().numpy(),\n label_s.detach().cpu().numpy())\n acc_s_adv.update(avg_acc_s_adv, cnt_s)\n _, avg_acc_t_adv, cnt_t_adv, pred_t_adv = accuracy(y_t_adv.detach().cpu().numpy(),\n label_t.detach().cpu().numpy())\n acc_t_adv.update(avg_acc_t_adv, cnt_t)\n losses_s.update(loss_s, cnt_s)\n losses_gf.update(loss_ground_false, cnt_s)\n losses_gt.update(loss_ground_truth, cnt_s)\n\n # measure elapsed time\n batch_time.update(time.time() - end)\n end = time.time()\n\n if i % args.print_freq == 0:\n progress.display(i)\n if visualize is not None:\n visualize(x_s[0], pred_s[0] * args.image_size / args.heatmap_size, \"source_{}_pred\".format(i))\n visualize(x_s[0], meta_s['keypoint2d'][0], \"source_{}_label\".format(i))\n visualize(x_t[0], pred_t[0] * args.image_size / args.heatmap_size, \"target_{}_pred\".format(i))\n visualize(x_t[0], meta_t['keypoint2d'][0], \"target_{}_label\".format(i))\n visualize(x_s[0], pred_s_adv[0] * args.image_size / args.heatmap_size, \"source_adv_{}_pred\".format(i))\n visualize(x_t[0], pred_t_adv[0] * args.image_size / args.heatmap_size, \"target_adv_{}_pred\".format(i))\n\n\ndef validate(val_loader, model, criterion, visualize, args: argparse.Namespace):\n batch_time = AverageMeter('Time', ':6.3f')\n losses = AverageMeter('Loss', ':.2e')\n acc = AverageMeterDict(val_loader.dataset.keypoints_group.keys(), \":3.2f\")\n progress = ProgressMeter(\n len(val_loader),\n [batch_time, losses, acc['all']],\n prefix='Test: ')\n\n # switch to evaluate mode\n model.eval()\n\n with torch.no_grad():\n end = time.time()\n for i, (x, label, weight, meta) in enumerate(val_loader):\n x = x.to(device)\n label = label.to(device)\n weight = weight.to(device)\n\n # compute output\n y = model(x)\n loss = criterion(y, label, weight)\n\n # measure accuracy and record loss\n losses.update(loss.item(), x.size(0))\n acc_per_points, avg_acc, cnt, pred = accuracy(y.cpu().numpy(),\n label.cpu().numpy())\n\n group_acc = val_loader.dataset.group_accuracy(acc_per_points)\n acc.update(group_acc, x.size(0))\n\n # measure elapsed time\n batch_time.update(time.time() - end)\n end = time.time()\n\n if i % args.print_freq == 0:\n progress.display(i)\n if visualize is not None:\n visualize(x[0], pred[0] * args.image_size / args.heatmap_size, \"val_{}_pred.jpg\".format(i))\n visualize(x[0], meta['keypoint2d'][0], \"val_{}_label.jpg\".format(i))\n\n return acc.average()\n\n\nif __name__ == '__main__':\n architecture_names = sorted(\n name for name in models.__dict__\n if name.islower() and not name.startswith(\"__\")\n and callable(models.__dict__[name])\n )\n dataset_names = sorted(\n name for name in datasets.__dict__\n if not name.startswith(\"__\") and callable(datasets.__dict__[name])\n )\n\n parser = argparse.ArgumentParser(description='RegDA (fast) for Keypoint Detection Domain Adaptation')\n # dataset parameters\n parser.add_argument('source_root', help='root path of the source dataset')\n parser.add_argument('target_root', help='root path of the target dataset')\n parser.add_argument('-s', '--source', help='source domain(s)')\n parser.add_argument('-t', '--target', help='target domain(s)')\n parser.add_argument('--resize-scale', nargs='+', type=float, default=(0.6, 1.3),\n help='scale range for the RandomResizeCrop augmentation')\n parser.add_argument('--rotation', type=int, default=180,\n help='rotation range of the RandomRotation augmentation')\n parser.add_argument('--image-size', type=int, default=256,\n help='input image size')\n parser.add_argument('--heatmap-size', type=int, default=64,\n help='output heatmap size')\n # model parameters\n parser.add_argument('-a', '--arch', metavar='ARCH', default='resnet101',\n choices=architecture_names,\n help='backbone architecture: ' +\n ' | '.join(architecture_names) +\n ' (default: resnet101)')\n parser.add_argument(\"--pretrain\", type=str, default=None,\n help=\"Where restore pretrained model parameters from.\")\n parser.add_argument(\"--resume\", type=str, default=None,\n help=\"where restore model parameters from.\")\n parser.add_argument('--num-head-layers', type=int, default=2)\n parser.add_argument('--margin', type=float, default=4., help=\"margin gamma\")\n parser.add_argument('--trade-off', default=1., type=float,\n help='the trade-off hyper-parameter for transfer loss')\n # training parameters\n parser.add_argument('-b', '--batch-size', default=32, type=int,\n metavar='N',\n help='mini-batch size (default: 32)')\n parser.add_argument('--lr', '--learning-rate', default=0.1, type=float,\n metavar='LR', help='initial learning rate', dest='lr')\n parser.add_argument('--momentum', default=0.9, type=float, metavar='M',\n help='momentum')\n parser.add_argument('--wd', '--weight-decay', default=0.0001, type=float,\n metavar='W', help='weight decay (default: 1e-4)')\n parser.add_argument('--lr-gamma', default=0.0001, type=float)\n parser.add_argument('--lr-decay', default=0.75, type=float, help='parameter for lr scheduler')\n parser.add_argument('--lr-step', default=[45, 60], type=tuple, help='parameter for lr scheduler')\n parser.add_argument('--lr-factor', default=0.1, type=float, help='parameter for lr scheduler')\n parser.add_argument('-j', '--workers', default=4, type=int, metavar='N',\n help='number of data loading workers (default: 4)')\n parser.add_argument('--pretrain_epochs', default=70, type=int, metavar='N',\n help='number of total epochs to run')\n parser.add_argument('--epochs', default=30, type=int, metavar='N',\n help='number of total epochs to run')\n parser.add_argument('-i', '--iters-per-epoch', default=500, type=int,\n help='Number of iterations per epoch')\n parser.add_argument('-p', '--print-freq', default=100, type=int,\n metavar='N', help='print frequency (default: 100)')\n parser.add_argument('--seed', default=None, type=int,\n help='seed for initializing training. ')\n parser.add_argument(\"--log\", type=str, default='regda_fast',\n help=\"Where to save logs, checkpoints and debugging images.\")\n parser.add_argument(\"--phase\", type=str, default='train', choices=['train', 'test'],\n help=\"When phase is 'test', only test the model.\")\n parser.add_argument('--debug', action=\"store_true\",\n help='In the debug mode, save images and predictions')\n args = parser.parse_args()\n main(args)\n\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/examples/domain_adaptation/keypoint_detection/regda_fast.sh",
"content": "# regda_fast is provided by https://github.com/YouJiacheng?tab=repositories\n# On single V100(16G), overall adversarial training time is reduced by about 40%.\n# yet the PCK might drop 1% for each dataset.\n# Hands Dataset\nCUDA_VISIBLE_DEVICES=0 python regda_fast.py data/RHD data/H3D_crop \\\n -s RenderedHandPose -t Hand3DStudio --seed 0 --debug --log logs/regda_fast/rhd2h3d\nCUDA_VISIBLE_DEVICES=0 python regda_fast.py data/FreiHand data/RHD \\\n -s FreiHand -t RenderedHandPose --seed 0 --debug --log logs/regda_fast/freihand2rhd\n\n# Body Dataset\nCUDA_VISIBLE_DEVICES=0 python regda_fast.py data/surreal_processed data/Human36M \\\n -s SURREAL -t Human36M --seed 0 --debug --rotation 30 --epochs 10 --log logs/regda_fast/surreal2human36m\nCUDA_VISIBLE_DEVICES=0 python regda_fast.py data/surreal_processed data/lsp \\\n -s SURREAL -t LSP --seed 0 --debug --rotation 30 --log logs/regda_fast/surreal2lsp\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/examples/domain_adaptation/keypoint_detection/source_only.py",
"content": "\"\"\"\n@author: Junguang Jiang\n@contact: JiangJunguang1123@outlook.com\n\"\"\"\nimport random\nimport time\nimport warnings\nimport sys\nimport argparse\nimport shutil\n\nimport torch\nimport torch.backends.cudnn as cudnn\nfrom torch.optim import Adam\nfrom torch.optim.lr_scheduler import MultiStepLR\nfrom torch.utils.data import DataLoader\nfrom torchvision.transforms import Compose, ToPILImage\n\nsys.path.append('../../..')\nimport common.vision.models.keypoint_detection as models\nfrom common.vision.models.keypoint_detection.loss import JointsMSELoss\nimport common.vision.datasets.keypoint_detection as datasets\nimport common.vision.transforms.keypoint_detection as T\nfrom common.vision.transforms import Denormalize\nfrom common.utils.data import ForeverDataIterator\nfrom common.utils.meter import AverageMeter, ProgressMeter, AverageMeterDict\nfrom common.utils.metric.keypoint_detection import accuracy\nfrom common.utils.logger import CompleteLogger\n\n\ndevice = torch.device(\"cuda\" if torch.cuda.is_available() else \"cpu\")\n\n\ndef main(args: argparse.Namespace):\n logger = CompleteLogger(args.log, args.phase)\n print(args)\n\n if args.seed is not None:\n random.seed(args.seed)\n torch.manual_seed(args.seed)\n cudnn.deterministic = True\n warnings.warn('You have chosen to seed training. '\n 'This will turn on the CUDNN deterministic setting, '\n 'which can slow down your training considerably! '\n 'You may see unexpected behavior when restarting '\n 'from checkpoints.')\n\n cudnn.benchmark = True\n\n # Data loading code\n normalize = T.Normalize([0.485, 0.456, 0.406], [0.229, 0.224, 0.225])\n train_transform = T.Compose([\n T.RandomRotation(args.rotation),\n T.RandomResizedCrop(size=args.image_size, scale=args.resize_scale),\n T.ColorJitter(brightness=0.25, contrast=0.25, saturation=0.25),\n T.GaussianBlur(),\n T.ToTensor(),\n normalize\n ])\n val_transform = T.Compose([\n T.Resize(args.image_size),\n T.ToTensor(),\n normalize\n ])\n image_size = (args.image_size, args.image_size)\n heatmap_size = (args.heatmap_size, args.heatmap_size)\n source_dataset = datasets.__dict__[args.source]\n train_source_dataset = source_dataset(root=args.source_root, transforms=train_transform,\n image_size=image_size, heatmap_size=heatmap_size)\n train_source_loader = DataLoader(train_source_dataset, batch_size=args.batch_size,\n shuffle=True, num_workers=args.workers, pin_memory=True, drop_last=True)\n val_source_dataset = source_dataset(root=args.source_root, split='test', transforms=val_transform,\n image_size=image_size, heatmap_size=heatmap_size)\n val_source_loader = DataLoader(val_source_dataset, batch_size=args.batch_size, shuffle=False, pin_memory=True)\n\n target_dataset = datasets.__dict__[args.target]\n train_target_dataset = target_dataset(root=args.target_root, transforms=train_transform,\n image_size=image_size, heatmap_size=heatmap_size)\n train_target_loader = DataLoader(train_target_dataset, batch_size=args.batch_size,\n shuffle=True, num_workers=args.workers, pin_memory=True, drop_last=True)\n val_target_dataset = target_dataset(root=args.target_root, split='test', transforms=val_transform,\n image_size=image_size, heatmap_size=heatmap_size)\n val_target_loader = DataLoader(val_target_dataset, batch_size=args.batch_size, shuffle=False, pin_memory=True)\n\n print(\"Source train:\", len(train_source_loader))\n print(\"Target train:\", len(train_target_loader))\n print(\"Source test:\", len(val_source_loader))\n print(\"Target test:\", len(val_target_loader))\n\n train_source_iter = ForeverDataIterator(train_source_loader)\n train_target_iter = ForeverDataIterator(train_target_loader)\n\n # create model\n model = models.__dict__[args.arch](num_keypoints=train_source_dataset.num_keypoints).to(device)\n criterion = JointsMSELoss()\n\n # define optimizer and lr scheduler\n optimizer = Adam(model.get_parameters(lr=args.lr))\n lr_scheduler = MultiStepLR(optimizer, args.lr_step, args.lr_factor)\n\n # optionally resume from a checkpoint\n start_epoch = 0\n if args.resume:\n checkpoint = torch.load(args.resume, map_location='cpu')\n model.load_state_dict(checkpoint['model'])\n optimizer.load_state_dict(checkpoint['optimizer'])\n lr_scheduler.load_state_dict(checkpoint['lr_scheduler'])\n start_epoch = checkpoint['epoch'] + 1\n\n # define visualization function\n tensor_to_image = Compose([\n Denormalize([0.485, 0.456, 0.406], [0.229, 0.224, 0.225]),\n ToPILImage()\n ])\n\n def visualize(image, keypoint2d, name):\n \"\"\"\n Args:\n image (tensor): image in shape 3 x H x W\n keypoint2d (tensor): keypoints in shape K x 2\n name: name of the saving image\n \"\"\"\n train_source_dataset.visualize(tensor_to_image(image),\n keypoint2d, logger.get_image_path(\"{}.jpg\".format(name)))\n\n if args.phase == 'test':\n # evaluate on validation set\n source_val_acc = validate(val_source_loader, model, criterion, None, args)\n target_val_acc = validate(val_target_loader, model, criterion, visualize, args)\n print(\"Source: {:4.3f} Target: {:4.3f}\".format(source_val_acc['all'], target_val_acc['all']))\n for name, acc in target_val_acc.items():\n print(\"{}: {:4.3f}\".format(name, acc))\n return\n\n # start training\n best_acc = 0\n for epoch in range(start_epoch, args.epochs):\n logger.set_epoch(epoch)\n lr_scheduler.step()\n\n # train for one epoch\n train(train_source_iter, train_target_iter, model, criterion, optimizer, epoch,\n visualize if args.debug else None, args)\n\n # evaluate on validation set\n source_val_acc = validate(val_source_loader, model, criterion, None, args)\n target_val_acc = validate(val_target_loader, model, criterion, visualize if args.debug else None, args)\n\n # remember best acc and save checkpoint\n torch.save(\n {\n 'model': model.state_dict(),\n 'optimizer': optimizer.state_dict(),\n 'lr_scheduler': lr_scheduler.state_dict(),\n 'epoch': epoch,\n 'args': args\n }, logger.get_checkpoint_path(epoch)\n )\n if target_val_acc['all'] > best_acc:\n shutil.copy(logger.get_checkpoint_path(epoch), logger.get_checkpoint_path('best'))\n best_acc = target_val_acc['all']\n print(\"Source: {:4.3f} Target: {:4.3f} Target(best): {:4.3f}\".format(source_val_acc['all'], target_val_acc['all'], best_acc))\n for name, acc in target_val_acc.items():\n print(\"{}: {:4.3f}\".format(name, acc))\n\n logger.close()\n\n\ndef train(train_source_iter, train_target_iter, model, criterion,\n optimizer, epoch: int, visualize, args: argparse.Namespace):\n batch_time = AverageMeter('Time', ':4.2f')\n data_time = AverageMeter('Data', ':3.1f')\n losses_s = AverageMeter('Loss (s)', \":.2e\")\n acc_s = AverageMeter(\"Acc (s)\", \":3.2f\")\n\n progress = ProgressMeter(\n args.iters_per_epoch,\n [batch_time, data_time, losses_s, acc_s],\n prefix=\"Epoch: [{}]\".format(epoch))\n\n # switch to train mode\n model.train()\n\n end = time.time()\n for i in range(args.iters_per_epoch):\n optimizer.zero_grad()\n\n x_s, label_s, weight_s, meta_s = next(train_source_iter)\n\n x_s = x_s.to(device)\n label_s = label_s.to(device)\n weight_s = weight_s.to(device)\n\n # measure data loading time\n data_time.update(time.time() - end)\n\n # compute output\n y_s = model(x_s)\n loss_s = criterion(y_s, label_s, weight_s)\n\n # compute gradient and do SGD step\n loss_s.backward()\n optimizer.step()\n\n # measure accuracy and record loss\n _, avg_acc_s, cnt_s, pred_s = accuracy(y_s.detach().cpu().numpy(),\n label_s.detach().cpu().numpy())\n acc_s.update(avg_acc_s, cnt_s)\n losses_s.update(loss_s, cnt_s)\n\n # measure elapsed time\n batch_time.update(time.time() - end)\n end = time.time()\n\n if i % args.print_freq == 0:\n progress.display(i)\n if visualize is not None:\n visualize(x_s[0], pred_s[0] * args.image_size / args.heatmap_size, \"source_{}_pred.jpg\".format(i))\n visualize(x_s[0], meta_s['keypoint2d'][0], \"source_{}_label.jpg\".format(i))\n\n\ndef validate(val_loader, model, criterion, visualize, args: argparse.Namespace):\n batch_time = AverageMeter('Time', ':6.3f')\n losses = AverageMeter('Loss', ':.2e')\n acc = AverageMeterDict(val_loader.dataset.keypoints_group.keys(), \":3.2f\")\n progress = ProgressMeter(\n len(val_loader),\n [batch_time, losses, acc['all']],\n prefix='Test: ')\n\n # switch to evaluate mode\n model.eval()\n\n with torch.no_grad():\n end = time.time()\n for i, (x, label, weight, meta) in enumerate(val_loader):\n x = x.to(device)\n label = label.to(device)\n weight = weight.to(device)\n\n # compute output\n y = model(x)\n loss = criterion(y, label, weight)\n\n # measure accuracy and record loss\n losses.update(loss.item(), x.size(0))\n acc_per_points, avg_acc, cnt, pred = accuracy(y.cpu().numpy(),\n label.cpu().numpy())\n\n group_acc = val_loader.dataset.group_accuracy(acc_per_points)\n acc.update(group_acc, x.size(0))\n\n # measure elapsed time\n batch_time.update(time.time() - end)\n end = time.time()\n\n if i % args.print_freq == 0:\n progress.display(i)\n if visualize is not None:\n visualize(x[0], pred[0] * args.image_size / args.heatmap_size, \"val_{}_pred.jpg\".format(i))\n visualize(x[0], meta['keypoint2d'][0], \"val_{}_label.jpg\".format(i))\n\n return acc.average()\n\n\nif __name__ == '__main__':\n architecture_names = sorted(\n name for name in models.__dict__\n if name.islower() and not name.startswith(\"__\")\n and callable(models.__dict__[name])\n )\n dataset_names = sorted(\n name for name in datasets.__dict__\n if not name.startswith(\"__\") and callable(datasets.__dict__[name])\n )\n\n parser = argparse.ArgumentParser(description='Source Only for Keypoint Detection Domain Adaptation')\n # dataset parameters\n parser.add_argument('source_root', help='root path of the source dataset')\n parser.add_argument('target_root', help='root path of the target dataset')\n parser.add_argument('-s', '--source', help='source domain(s)')\n parser.add_argument('-t', '--target', help='target domain(s)')\n parser.add_argument('--resize-scale', nargs='+', type=float, default=(0.6, 1.3),\n help='scale range for the RandomResizeCrop augmentation')\n parser.add_argument('--rotation', type=int, default=180,\n help='rotation range of the RandomRotation augmentation')\n parser.add_argument('--image-size', type=int, default=256,\n help='input image size')\n parser.add_argument('--heatmap-size', type=int, default=64,\n help='output heatmap size')\n # model parameters\n parser.add_argument('-a', '--arch', metavar='ARCH', default='pose_resnet101',\n choices=architecture_names,\n help='backbone architecture: ' +\n ' | '.join(architecture_names) +\n ' (default: pose_resnet101)')\n parser.add_argument(\"--resume\", type=str, default=None,\n help=\"where restore model parameters from.\")\n # training parameters\n parser.add_argument('-b', '--batch-size', default=32, type=int,\n metavar='N',\n help='mini-batch size (default: 32)')\n parser.add_argument('--lr', '--learning-rate', default=0.001, type=float,\n metavar='LR', help='initial learning rate', dest='lr')\n parser.add_argument('--lr-step', default=[45, 60], type=tuple, help='parameter for lr scheduler')\n parser.add_argument('--lr-factor', default=0.1, type=float, help='parameter for lr scheduler')\n parser.add_argument('-j', '--workers', default=4, type=int, metavar='N',\n help='number of data loading workers (default: 4)')\n parser.add_argument('--epochs', default=70, type=int, metavar='N',\n help='number of total epochs to run')\n parser.add_argument('-i', '--iters-per-epoch', default=500, type=int,\n help='Number of iterations per epoch')\n parser.add_argument('-p', '--print-freq', default=100, type=int,\n metavar='N', help='print frequency (default: 100)')\n parser.add_argument('--seed', default=None, type=int,\n help='seed for initializing training. ')\n parser.add_argument(\"--log\", type=str, default='src_only',\n help=\"Where to save logs, checkpoints and debugging images.\")\n parser.add_argument(\"--phase\", type=str, default='train', choices=['train', 'test'],\n help=\"When phase is 'test', only test the model.\")\n parser.add_argument('--debug', action=\"store_true\",\n help='In the debug mode, save images and predictions')\n args = parser.parse_args()\n main(args)\n\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/examples/domain_adaptation/keypoint_detection/source_only.sh",
"content": "# Source Only\n# Hands Dataset\nCUDA_VISIBLE_DEVICES=0 python source_only.py data/RHD data/H3D_crop \\\n -s RenderedHandPose -t Hand3DStudio --log logs/baseline/rhd2h3d --debug --seed 0\nCUDA_VISIBLE_DEVICES=0 python source_only.py data/FreiHand data/RHD \\\n -s FreiHand -t RenderedHandPose --log logs/baseline/freihand2rhd --debug --seed 0\n\n# Body Dataset\nCUDA_VISIBLE_DEVICES=0 python source_only.py data/surreal_processed data/Human36M \\\n -s SURREAL -t Human36M --log logs/baseline/surreal2human36m --debug --seed 0 --rotation 30\nCUDA_VISIBLE_DEVICES=0 python source_only.py data/surreal_processed data/lsp \\\n -s SURREAL -t LSP --log logs/baseline/surreal2lsp --debug --seed 0 --rotation 30\n\n# Oracle Results\nCUDA_VISIBLE_DEVICES=0 python source_only.py data/H3D_crop data/H3D_crop \\\n -s Hand3DStudio -t Hand3DStudio --log logs/oracle/h3d --debug --seed 0\nCUDA_VISIBLE_DEVICES=0 python source_only.py data/Human36M data/Human36M \\\n -s Human36M -t Human36M --log logs/oracle/human36m --debug --seed 0 --rotation 30\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/examples/domain_adaptation/openset_domain_adaptation/README.md",
"content": "# Open-set Domain Adaptation for Image Classification\n\n## Installation\nExample scripts also support all models in [PyTorch-Image-Models](https://github.com/rwightman/pytorch-image-models).\nYou also need to install timm to use PyTorch-Image-Models.\n\n```\npip install timm\n```\n\n## Dataset\n\nFollowing datasets can be downloaded automatically:\n\n- [Office31](https://www.cc.gatech.edu/~judy/domainadapt/)\n- [OfficeHome](https://www.hemanthdv.org/officeHomeDataset.html)\n- [VisDA2017](http://ai.bu.edu/visda-2017/)\n\n## Supported Methods\n\nSupported methods include:\n\n- [Open Set Domain Adaptation (OSBP)](https://arxiv.org/abs/1804.10427)\n\n## Experiment and Results\n\nThe shell files give the script to reproduce the [benchmarks](/docs/dalib/benchmarks/openset_domain_adaptation.rst) with specified hyper-parameters.\nFor example, if you want to train DANN on Office31, use the following script\n\n```shell script\n# Train a DANN on Office-31 Amazon -> Webcam task using ResNet 50.\n# Assume you have put the datasets under the path `data/office-31`, \n# or you are glad to download the datasets automatically from the Internet to this path\nCUDA_VISIBLE_DEVICES=0 python dann.py data/office31 -d Office31 -s A -t W -a resnet50 --epochs 20 --seed 1 --log logs/dann/Office31_A2W\n```\n\nFor more information please refer to [Get Started](/docs/get_started/quickstart.rst) for help.\n\n## Citation\nIf you use these methods in your research, please consider citing.\n\n```\n@InProceedings{OSBP,\n author = {Saito, Kuniaki and Yamamoto, Shohei and Ushiku, Yoshitaka and Harada, Tatsuya},\n title = {Open Set Domain Adaptation by Backpropagation},\n booktitle = {ECCV},\n year = {2018}\n}\n```\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/examples/domain_adaptation/openset_domain_adaptation/dann.py",
"content": "\"\"\"\n@author: Junguang Jiang\n@contact: JiangJunguang1123@outlook.com\n\"\"\"\nimport random\nimport time\nimport warnings\nimport sys\nimport argparse\nimport shutil\nimport os.path as osp\n\nimport torch\nimport torch.nn as nn\nimport torch.backends.cudnn as cudnn\nfrom torch.optim import SGD\nfrom torch.optim.lr_scheduler import LambdaLR\nfrom torch.utils.data import DataLoader\nimport torch.nn.functional as F\n\nsys.path.append('../../..')\nfrom dalib.modules.domain_discriminator import DomainDiscriminator\nfrom common.modules.classifier import Classifier\nfrom dalib.adaptation.dann import DomainAdversarialLoss, ImageClassifier\nfrom common.utils.data import ForeverDataIterator\nfrom common.utils.metric import accuracy, ConfusionMatrix\nfrom common.utils.meter import AverageMeter, ProgressMeter\nfrom common.utils.logger import CompleteLogger\nfrom common.utils.analysis import collect_feature, tsne, a_distance\n\nsys.path.append('.')\nimport utils\n\ndevice = torch.device(\"cuda\" if torch.cuda.is_available() else \"cpu\")\n\n\ndef main(args: argparse.Namespace):\n logger = CompleteLogger(args.log, args.phase)\n print(args)\n\n if args.seed is not None:\n random.seed(args.seed)\n torch.manual_seed(args.seed)\n cudnn.deterministic = True\n warnings.warn('You have chosen to seed training. '\n 'This will turn on the CUDNN deterministic setting, '\n 'which can slow down your training considerably! '\n 'You may see unexpected behavior when restarting '\n 'from checkpoints.')\n\n cudnn.benchmark = True\n\n # Data loading code\n train_transform = utils.get_train_transform(args.train_resizing, random_horizontal_flip=True,\n random_color_jitter=False)\n val_transform = utils.get_val_transform(args.val_resizing)\n print(\"train_transform: \", train_transform)\n print(\"val_transform: \", val_transform)\n\n train_source_dataset, train_target_dataset, val_dataset, test_dataset, num_classes, args.class_names = \\\n utils.get_dataset(args.data, args.root, args.source, args.target, train_transform, val_transform)\n train_source_loader = DataLoader(train_source_dataset, batch_size=args.batch_size,\n shuffle=True, num_workers=args.workers, drop_last=True)\n train_target_loader = DataLoader(train_target_dataset, batch_size=args.batch_size,\n shuffle=True, num_workers=args.workers, drop_last=True)\n val_loader = DataLoader(val_dataset, batch_size=args.batch_size, shuffle=False, num_workers=args.workers)\n test_loader = DataLoader(test_dataset, batch_size=args.batch_size, shuffle=False, num_workers=args.workers)\n\n train_source_iter = ForeverDataIterator(train_source_loader)\n train_target_iter = ForeverDataIterator(train_target_loader)\n\n # create model\n print(\"=> using pre-trained model '{}'\".format(args.arch))\n backbone = utils.get_model(args.arch)\n pool_layer = nn.Identity() if args.no_pool else None\n classifier = ImageClassifier(backbone, num_classes, bottleneck_dim=args.bottleneck_dim, pool_layer=pool_layer).to(device)\n domain_discri = DomainDiscriminator(in_feature=classifier.features_dim, hidden_size=1024).to(device)\n\n # define optimizer and lr scheduler\n optimizer = SGD(classifier.get_parameters() + domain_discri.get_parameters(),\n args.lr, momentum=args.momentum, weight_decay=args.weight_decay, nesterov=True)\n lr_scheduler = LambdaLR(optimizer, lambda x: args.lr * (1. + args.lr_gamma * float(x)) ** (-args.lr_decay))\n\n # define loss function\n domain_adv = DomainAdversarialLoss(domain_discri).to(device)\n\n # analysis the model\n if args.phase == 'analysis':\n # extract features from both domains\n feature_extractor = nn.Sequential(classifier.backbone, classifier.pool_layer, classifier.bottleneck).to(device)\n source_feature = collect_feature(train_source_loader, feature_extractor, device)\n target_feature = collect_feature(train_target_loader, feature_extractor, device)\n # plot t-SNE\n tSNE_filename = osp.join(logger.visualize_directory, 'TSNE.png')\n tsne.visualize(source_feature, target_feature, tSNE_filename)\n print(\"Saving t-SNE to\", tSNE_filename)\n # calculate A-distance, which is a measure for distribution discrepancy\n A_distance = a_distance.calculate(source_feature, target_feature, device)\n print(\"A-distance =\", A_distance)\n return\n\n if args.phase == 'test':\n acc1 = validate(test_loader, classifier, args)\n print(acc1)\n return\n\n # start training\n best_h_score = 0.\n for epoch in range(args.epochs):\n # train for one epoch\n train(train_source_iter, train_target_iter, classifier, domain_adv, optimizer,\n lr_scheduler, epoch, args)\n\n # evaluate on validation set\n h_score = validate(val_loader, classifier, args)\n\n # remember best acc@1 and save checkpoint\n torch.save(classifier.state_dict(), logger.get_checkpoint_path('latest'))\n if h_score > best_h_score:\n shutil.copy(logger.get_checkpoint_path('latest'), logger.get_checkpoint_path('best'))\n best_h_score = max(h_score, best_h_score)\n\n print(\"best_h_score = {:3.1f}\".format(best_h_score))\n\n # evaluate on test set\n classifier.load_state_dict(torch.load(logger.get_checkpoint_path('best')))\n h_score = validate(test_loader, classifier, args)\n print(\"test_h_score = {:3.1f}\".format(h_score))\n\n logger.close()\n\n\ndef train(train_source_iter: ForeverDataIterator, train_target_iter: ForeverDataIterator,\n model: ImageClassifier, domain_adv: DomainAdversarialLoss, optimizer: SGD,\n lr_scheduler: LambdaLR, epoch: int, args: argparse.Namespace):\n batch_time = AverageMeter('Time', ':5.2f')\n data_time = AverageMeter('Data', ':5.2f')\n losses = AverageMeter('Loss', ':6.2f')\n cls_accs = AverageMeter('Cls Acc', ':3.1f')\n tgt_accs = AverageMeter('Tgt Acc', ':3.1f')\n domain_accs = AverageMeter('Domain Acc', ':3.1f')\n progress = ProgressMeter(\n args.iters_per_epoch,\n [batch_time, data_time, losses, cls_accs, tgt_accs, domain_accs],\n prefix=\"Epoch: [{}]\".format(epoch))\n\n # switch to train mode\n model.train()\n domain_adv.train()\n\n end = time.time()\n for i in range(args.iters_per_epoch):\n x_s, labels_s = next(train_source_iter)\n x_t, labels_t = next(train_target_iter)\n\n x_s = x_s.to(device)\n x_t = x_t.to(device)\n labels_s = labels_s.to(device)\n labels_t = labels_t.to(device)\n\n # measure data loading time\n data_time.update(time.time() - end)\n\n # compute output\n x = torch.cat((x_s, x_t), dim=0)\n y, f = model(x)\n y_s, y_t = y.chunk(2, dim=0)\n f_s, f_t = f.chunk(2, dim=0)\n\n cls_loss = F.cross_entropy(y_s, labels_s)\n transfer_loss = domain_adv(f_s, f_t)\n domain_acc = domain_adv.domain_discriminator_accuracy\n loss = cls_loss + transfer_loss * args.trade_off\n\n cls_acc = accuracy(y_s, labels_s)[0]\n tgt_acc = accuracy(y_t, labels_t)[0]\n\n losses.update(loss.item(), x_s.size(0))\n cls_accs.update(cls_acc.item(), x_s.size(0))\n tgt_accs.update(tgt_acc.item(), x_s.size(0))\n domain_accs.update(domain_acc.item(), x_s.size(0))\n\n # compute gradient and do SGD step\n optimizer.zero_grad()\n loss.backward()\n optimizer.step()\n lr_scheduler.step()\n\n # measure elapsed time\n batch_time.update(time.time() - end)\n end = time.time()\n\n if i % args.print_freq == 0:\n progress.display(i)\n\n\ndef validate(val_loader: DataLoader, model: Classifier, args: argparse.Namespace) -> float:\n batch_time = AverageMeter('Time', ':6.3f')\n classes = val_loader.dataset.classes\n confmat = ConfusionMatrix(len(classes))\n progress = ProgressMeter(\n len(val_loader),\n [batch_time],\n prefix='Test: ')\n\n # switch to evaluate mode\n model.eval()\n\n with torch.no_grad():\n end = time.time()\n for i, (images, target) in enumerate(val_loader):\n images = images.to(device)\n target = target.to(device)\n\n # compute output\n output = model(images)\n softmax_output = F.softmax(output, dim=1)\n softmax_output[:, -1] = args.threshold\n\n # measure accuracy and record loss\n confmat.update(target, softmax_output.argmax(1))\n\n # measure elapsed time\n batch_time.update(time.time() - end)\n end = time.time()\n\n if i % args.print_freq == 0:\n progress.display(i)\n\n acc_global, accs, iu = confmat.compute()\n all_acc = torch.mean(accs).item() * 100\n known = torch.mean(accs[:-1]).item() * 100\n unknown = accs[-1].item() * 100\n h_score = 2 * known * unknown / (known + unknown)\n if args.per_class_eval:\n print(confmat.format(classes))\n print(' * All {all:.3f} Known {known:.3f} Unknown {unknown:.3f} H-score {h_score:.3f}'\n .format(all=all_acc, known=known, unknown=unknown, h_score=h_score))\n\n return h_score\n\n\nif __name__ == '__main__':\n parser = argparse.ArgumentParser(description='DANN for Openset Domain Adaptation')\n # dataset parameters\n parser.add_argument('root', metavar='DIR',\n help='root path of dataset')\n parser.add_argument('-d', '--data', metavar='DATA', default='Office31', choices=utils.get_dataset_names(),\n help='dataset: ' + ' | '.join(utils.get_dataset_names()) +\n ' (default: Office31)')\n parser.add_argument('-s', '--source', help='source domain')\n parser.add_argument('-t', '--target', help='target domain')\n parser.add_argument('--train-resizing', type=str, default='default')\n parser.add_argument('--val-resizing', type=str, default='default')\n # model parameters\n parser.add_argument('-a', '--arch', metavar='ARCH', default='resnet18',\n choices=utils.get_model_names(),\n help='backbone architecture: ' +\n ' | '.join(utils.get_model_names()) +\n ' (default: resnet18)')\n parser.add_argument('--no-pool', action='store_true',\n help='no pool layer after the feature extractor.')\n parser.add_argument('--bottleneck-dim', default=256, type=int,\n help='Dimension of bottleneck')\n parser.add_argument('--threshold', default=0.8, type=float,\n help='When class confidence is less than the given threshold, '\n 'model will output \"unknown\" (default: 0.5)')\n parser.add_argument('--trade-off', default=1., type=float,\n help='the trade-off hyper-parameter for transfer loss')\n # training parameters\n parser.add_argument('-b', '--batch-size', default=32, type=int,\n metavar='N',\n help='mini-batch size (default: 32)')\n parser.add_argument('--lr', '--learning-rate', default=0.002, type=float,\n metavar='LR', help='initial learning rate', dest='lr')\n parser.add_argument('--lr-gamma', default=0.001, type=float, help='parameter for lr scheduler')\n parser.add_argument('--lr-decay', default=0.75, type=float, help='parameter for lr scheduler')\n parser.add_argument('--momentum', default=0.9, type=float, metavar='M',\n help='momentum')\n parser.add_argument('--wd', '--weight-decay',default=1e-3, type=float,\n metavar='W', help='weight decay (default: 1e-3)',\n dest='weight_decay')\n parser.add_argument('-j', '--workers', default=2, type=int, metavar='N',\n help='number of data loading workers (default: 4)')\n parser.add_argument('--epochs', default=20, type=int, metavar='N',\n help='number of total epochs to run')\n parser.add_argument('-i', '--iters-per-epoch', default=500, type=int,\n help='Number of iterations per epoch')\n parser.add_argument('-p', '--print-freq', default=100, type=int,\n metavar='N', help='print frequency (default: 100)')\n parser.add_argument('--seed', default=None, type=int,\n help='seed for initializing training. ')\n parser.add_argument('--per-class-eval', action='store_true',\n help='whether output per-class accuracy during evaluation')\n parser.add_argument(\"--log\", type=str, default='dann',\n help=\"Where to save logs, checkpoints and debugging images.\")\n parser.add_argument(\"--phase\", type=str, default='train', choices=['train', 'test', 'analysis'],\n help=\"When phase is 'test', only test the model.\"\n \"When phase is 'analysis', only analysis the model.\")\n args = parser.parse_args()\n main(args)\n\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/examples/domain_adaptation/openset_domain_adaptation/dann.sh",
"content": "#!/usr/bin/env bash\n# Office31\nCUDA_VISIBLE_DEVICES=0 python dann.py data/office31 -d Office31 -s A -t W -a resnet50 --epochs 20 --seed 0 --threshold 0.9 --log logs/dann/Office31_A2W\nCUDA_VISIBLE_DEVICES=0 python dann.py data/office31 -d Office31 -s D -t W -a resnet50 --epochs 20 --seed 0 --threshold 0.9 --log logs/dann/Office31_D2W\nCUDA_VISIBLE_DEVICES=0 python dann.py data/office31 -d Office31 -s W -t D -a resnet50 --epochs 20 --seed 0 --threshold 0.9 --log logs/dann/Office31_W2D\nCUDA_VISIBLE_DEVICES=0 python dann.py data/office31 -d Office31 -s A -t D -a resnet50 --epochs 20 --seed 0 --threshold 0.9 --log logs/dann/Office31_A2D\nCUDA_VISIBLE_DEVICES=0 python dann.py data/office31 -d Office31 -s D -t A -a resnet50 --epochs 20 --seed 0 --threshold 0.9 --log logs/dann/Office31_D2A\nCUDA_VISIBLE_DEVICES=0 python dann.py data/office31 -d Office31 -s W -t A -a resnet50 --epochs 20 --seed 0 --threshold 0.9 --log logs/dann/Office31_W2A\n\n# Office-Home\nCUDA_VISIBLE_DEVICES=0 python dann.py data/office-home -d OfficeHome -s Ar -t Cl -a resnet50 --epochs 30 --seed 0 --log logs/dann/OfficeHome_Ar2Cl\nCUDA_VISIBLE_DEVICES=0 python dann.py data/office-home -d OfficeHome -s Ar -t Pr -a resnet50 --epochs 30 --seed 0 --log logs/dann/OfficeHome_Ar2Pr\nCUDA_VISIBLE_DEVICES=0 python dann.py data/office-home -d OfficeHome -s Ar -t Rw -a resnet50 --epochs 30 --seed 0 --log logs/dann/OfficeHome_Ar2Rw\nCUDA_VISIBLE_DEVICES=0 python dann.py data/office-home -d OfficeHome -s Cl -t Ar -a resnet50 --epochs 30 --seed 0 --log logs/dann/OfficeHome_Cl2Ar\nCUDA_VISIBLE_DEVICES=0 python dann.py data/office-home -d OfficeHome -s Cl -t Pr -a resnet50 --epochs 30 --seed 0 --log logs/dann/OfficeHome_Cl2Pr\nCUDA_VISIBLE_DEVICES=0 python dann.py data/office-home -d OfficeHome -s Cl -t Rw -a resnet50 --epochs 30 --seed 0 --log logs/dann/OfficeHome_Cl2Rw\nCUDA_VISIBLE_DEVICES=0 python dann.py data/office-home -d OfficeHome -s Pr -t Ar -a resnet50 --epochs 30 --seed 0 --log logs/dann/OfficeHome_Pr2Ar\nCUDA_VISIBLE_DEVICES=0 python dann.py data/office-home -d OfficeHome -s Pr -t Cl -a resnet50 --epochs 30 --seed 0 --log logs/dann/OfficeHome_Pr2Cl\nCUDA_VISIBLE_DEVICES=0 python dann.py data/office-home -d OfficeHome -s Pr -t Rw -a resnet50 --epochs 30 --seed 0 --log logs/dann/OfficeHome_Pr2Rw\nCUDA_VISIBLE_DEVICES=0 python dann.py data/office-home -d OfficeHome -s Rw -t Ar -a resnet50 --epochs 30 --seed 0 --log logs/dann/OfficeHome_Rw2Ar\nCUDA_VISIBLE_DEVICES=0 python dann.py data/office-home -d OfficeHome -s Rw -t Cl -a resnet50 --epochs 30 --seed 0 --log logs/dann/OfficeHome_Rw2Cl\nCUDA_VISIBLE_DEVICES=0 python dann.py data/office-home -d OfficeHome -s Rw -t Pr -a resnet50 --epochs 30 --seed 0 --log logs/dann/OfficeHome_Rw2Pr\n\n# VisDA-2017\nCUDA_VISIBLE_DEVICES=0 python dann.py data/visda-2017 -d VisDA2017 -s Synthetic -t Real -a resnet50 \\\n --epochs 30 --seed 0 --train-resizing cen.crop --per-class-eval --log logs/dann/VisDA2017_S2R\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/examples/domain_adaptation/openset_domain_adaptation/osbp.py",
"content": "\"\"\"\n@author: Junguang Jiang\n@contact: JiangJunguang1123@outlook.com\n\"\"\"\nimport random\nimport time\nimport warnings\nimport sys\nimport argparse\nimport shutil\nimport os.path as osp\n\nimport torch\nimport torch.nn as nn\nimport torch.backends.cudnn as cudnn\nfrom torch.optim import SGD\nfrom torch.optim.lr_scheduler import LambdaLR\nfrom torch.utils.data import DataLoader\nimport torch.nn.functional as F\n\nsys.path.append('../../..')\nfrom dalib.adaptation.osbp import ImageClassifier as Classifier, UnknownClassBinaryCrossEntropy\nfrom common.utils.data import ForeverDataIterator\nfrom common.utils.metric import accuracy, ConfusionMatrix\nfrom common.utils.meter import AverageMeter, ProgressMeter\nfrom common.utils.logger import CompleteLogger\nfrom common.utils.analysis import collect_feature, tsne, a_distance\n\nsys.path.append('.')\nimport utils\n\ndevice = torch.device(\"cuda\" if torch.cuda.is_available() else \"cpu\")\n\n\ndef main(args: argparse.Namespace):\n logger = CompleteLogger(args.log, args.phase)\n print(args)\n\n if args.seed is not None:\n random.seed(args.seed)\n torch.manual_seed(args.seed)\n cudnn.deterministic = True\n warnings.warn('You have chosen to seed training. '\n 'This will turn on the CUDNN deterministic setting, '\n 'which can slow down your training considerably! '\n 'You may see unexpected behavior when restarting '\n 'from checkpoints.')\n\n cudnn.benchmark = True\n\n # Data loading code\n train_transform = utils.get_train_transform(args.train_resizing, random_horizontal_flip=True,\n random_color_jitter=False)\n val_transform = utils.get_val_transform(args.val_resizing)\n print(\"train_transform: \", train_transform)\n print(\"val_transform: \", val_transform)\n\n train_source_dataset, train_target_dataset, val_dataset, test_dataset, num_classes, args.class_names = \\\n utils.get_dataset(args.data, args.root, args.source, args.target, train_transform, val_transform)\n train_source_loader = DataLoader(train_source_dataset, batch_size=args.batch_size,\n shuffle=True, num_workers=args.workers, drop_last=True)\n train_target_loader = DataLoader(train_target_dataset, batch_size=args.batch_size,\n shuffle=True, num_workers=args.workers, drop_last=True)\n val_loader = DataLoader(val_dataset, batch_size=args.batch_size, shuffle=False, num_workers=args.workers)\n test_loader = DataLoader(test_dataset, batch_size=args.batch_size, shuffle=False, num_workers=args.workers)\n\n train_source_iter = ForeverDataIterator(train_source_loader)\n train_target_iter = ForeverDataIterator(train_target_loader)\n\n # create model\n print(\"=> using pre-trained model '{}'\".format(args.arch))\n backbone = utils.get_model(args.arch)\n pool_layer = nn.Identity() if args.no_pool else None\n classifier = Classifier(backbone, num_classes, bottleneck_dim=args.bottleneck_dim, pool_layer=pool_layer).to(device)\n print(classifier)\n unknown_bce = UnknownClassBinaryCrossEntropy(t=0.5)\n\n # define optimizer and lr scheduler\n optimizer = SGD(classifier.get_parameters(), args.lr, momentum=args.momentum, weight_decay=args.wd, nesterov=True)\n lr_scheduler = LambdaLR(optimizer, lambda x: args.lr * (1. + args.lr_gamma * float(x)) ** (-args.lr_decay))\n\n # analysis the model\n if args.phase == 'analysis':\n # extract features from both domains\n feature_extractor = nn.Sequential(classifier.backbone, classifier.pool_layer, classifier.bottleneck).to(device)\n source_feature = collect_feature(train_source_loader, feature_extractor, device)\n target_feature = collect_feature(train_target_loader, feature_extractor, device)\n # plot t-SNE\n tSNE_filename = osp.join(logger.visualize_directory, 'TSNE.png')\n tsne.visualize(source_feature, target_feature, tSNE_filename)\n print(\"Saving t-SNE to\", tSNE_filename)\n # calculate A-distance, which is a measure for distribution discrepancy\n A_distance = a_distance.calculate(source_feature, target_feature, device)\n print(\"A-distance =\", A_distance)\n return\n\n if args.phase == 'test':\n acc1 = validate(test_loader, classifier, args)\n print(acc1)\n return\n\n # start training\n best_h_score = 0.\n for epoch in range(args.epochs):\n # train for one epoch\n train(train_source_iter, train_target_iter, classifier, unknown_bce, optimizer,\n lr_scheduler, epoch, args)\n\n # evaluate on validation set\n h_score = validate(val_loader, classifier, args)\n\n # remember best acc@1 and save checkpoint\n torch.save(classifier.state_dict(), logger.get_checkpoint_path('latest'))\n if h_score > best_h_score:\n shutil.copy(logger.get_checkpoint_path('latest'), logger.get_checkpoint_path('best'))\n best_h_score = max(h_score, best_h_score)\n\n print(\"best_h_score = {:3.1f}\".format(best_h_score))\n\n # evaluate on test set\n classifier.load_state_dict(torch.load(logger.get_checkpoint_path('best')))\n h_score = validate(test_loader, classifier, args)\n print(\"test_h_score = {:3.1f}\".format(h_score))\n\n logger.close()\n\n\ndef train(train_source_iter: ForeverDataIterator, train_target_iter: ForeverDataIterator,model: Classifier, unknown_bce: UnknownClassBinaryCrossEntropy, optimizer: SGD,\n lr_scheduler: LambdaLR, epoch: int, args: argparse.Namespace):\n batch_time = AverageMeter('Time', ':4.2f')\n data_time = AverageMeter('Data', ':3.1f')\n losses = AverageMeter('Loss', ':3.2f')\n cls_accs = AverageMeter('Cls Acc', ':3.1f')\n tgt_accs = AverageMeter('Tgt Acc', ':3.1f')\n trans_losses = AverageMeter('Trans Loss', ':3.2f')\n\n progress = ProgressMeter(\n args.iters_per_epoch,\n [batch_time, data_time, losses, trans_losses, cls_accs, tgt_accs],\n prefix=\"Epoch: [{}]\".format(epoch))\n\n # switch to train mode\n model.train()\n\n end = time.time()\n for i in range(args.iters_per_epoch):\n x_s, labels_s = next(train_source_iter)\n x_t, labels_t = next(train_target_iter)\n\n x_s = x_s.to(device)\n x_t = x_t.to(device)\n labels_s = labels_s.to(device)\n labels_t = labels_t.to(device)\n\n # measure data loading time\n data_time.update(time.time() - end)\n\n # compute output\n y_s, _ = model(x_s, grad_reverse=False)\n y_t, _ = model(x_t, grad_reverse=True)\n\n cls_loss = F.cross_entropy(y_s, labels_s)\n trans_loss = unknown_bce(y_t)\n loss = cls_loss + trans_loss\n\n cls_acc = accuracy(y_s, labels_s)[0]\n tgt_acc = accuracy(y_t, labels_t)[0]\n\n losses.update(loss.item(), x_s.size(0))\n trans_losses.update(trans_loss.item(), x_s.size(0))\n cls_accs.update(cls_acc.item(), x_s.size(0))\n tgt_accs.update(tgt_acc.item(), x_t.size(0))\n\n # compute gradient and do SGD step\n optimizer.zero_grad()\n loss.backward()\n optimizer.step()\n lr_scheduler.step()\n\n # measure elapsed time\n batch_time.update(time.time() - end)\n end = time.time()\n\n if i % args.print_freq == 0:\n progress.display(i)\n\n\ndef validate(val_loader: DataLoader, model: Classifier, args: argparse.Namespace) -> float:\n batch_time = AverageMeter('Time', ':6.3f')\n classes = val_loader.dataset.classes\n confmat = ConfusionMatrix(len(classes))\n progress = ProgressMeter(\n len(val_loader),\n [batch_time],\n prefix='Test: ')\n\n # switch to evaluate mode\n model.eval()\n\n with torch.no_grad():\n end = time.time()\n for i, (images, target) in enumerate(val_loader):\n images = images.to(device)\n target = target.to(device)\n\n # compute output\n output = model(images)\n\n # measure accuracy and record loss\n confmat.update(target, output.argmax(1))\n\n # measure elapsed time\n batch_time.update(time.time() - end)\n end = time.time()\n\n if i % args.print_freq == 0:\n progress.display(i)\n\n acc_global, accs, iu = confmat.compute()\n all_acc = torch.mean(accs).item() * 100\n known = torch.mean(accs[:-1]).item() * 100\n unknown = accs[-1].item() * 100\n h_score = 2 * known * unknown / (known + unknown)\n if args.per_class_eval:\n print(confmat.format(classes))\n print(' * All {all:.3f} Known {known:.3f} Unknown {unknown:.3f} H-score {h_score:.3f}'\n .format(all=all_acc, known=known, unknown=unknown, h_score=h_score))\n\n return h_score\n\n\nif __name__ == '__main__':\n parser = argparse.ArgumentParser(description='OSBP for Openset Domain Adaptation')\n # dataset parameters\n parser.add_argument('root', metavar='DIR',\n help='root path of dataset')\n parser.add_argument('-d', '--data', metavar='DATA', default='Office31', choices=utils.get_dataset_names(),\n help='dataset: ' + ' | '.join(utils.get_dataset_names()) +\n ' (default: Office31)')\n parser.add_argument('-s', '--source', help='source domain')\n parser.add_argument('-t', '--target', help='target domain')\n parser.add_argument('--train-resizing', type=str, default='default')\n parser.add_argument('--val-resizing', type=str, default='default')\n # model parameters\n parser.add_argument('-a', '--arch', metavar='ARCH', default='resnet18',\n choices=utils.get_model_names(),\n help='backbone architecture: ' +\n ' | '.join(utils.get_model_names()) +\n ' (default: resnet18)')\n parser.add_argument('--no-pool', action='store_true',\n help='no pool layer after the feature extractor.')\n parser.add_argument('--bottleneck-dim', default=256, type=int,\n help='Dimension of bottleneck')\n # training parameters\n parser.add_argument('-b', '--batch-size', default=32, type=int,\n metavar='N',\n help='mini-batch size (default: 32)')\n parser.add_argument('--lr', '--learning-rate', default=0.001, type=float,\n metavar='LR', help='initial learning rate', dest='lr')\n parser.add_argument('--lr-gamma', default=0.0003, type=float, help='parameter for lr scheduler')\n parser.add_argument('--lr-decay', default=0.75, type=float, help='parameter for lr scheduler')\n parser.add_argument('--momentum', default=0.9, type=float, metavar='M',\n help='momentum')\n parser.add_argument('--wd', '--weight-decay', default=0.0005, type=float,\n metavar='W', help='weight decay (default: 5e-4)')\n parser.add_argument('-j', '--workers', default=2, type=int, metavar='N',\n help='number of data loading workers (default: 4)')\n parser.add_argument('--epochs', default=20, type=int, metavar='N',\n help='number of total epochs to run')\n parser.add_argument('-i', '--iters-per-epoch', default=500, type=int,\n help='Number of iterations per epoch')\n parser.add_argument('-p', '--print-freq', default=100, type=int,\n metavar='N', help='print frequency (default: 100)')\n parser.add_argument('--seed', default=None, type=int,\n help='seed for initializing training. ')\n parser.add_argument('--per-class-eval', action='store_true',\n help='whether output per-class accuracy during evaluation')\n parser.add_argument(\"--log\", type=str, default='osbp',\n help=\"Where to save logs, checkpoints and debugging images.\")\n parser.add_argument(\"--phase\", type=str, default='train', choices=['train', 'test', 'analysis'],\n help=\"When phase is 'test', only test the model.\"\n \"When phase is 'analysis', only analysis the model.\")\n args = parser.parse_args()\n main(args)\n\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/examples/domain_adaptation/openset_domain_adaptation/osbp.sh",
"content": "#!/usr/bin/env bash\n# Office31\nCUDA_VISIBLE_DEVICES=0 python osbp.py data/office31 -d Office31 -s A -t W -a resnet50 --epochs 20 --seed 0 --log logs/osbp/Office31_A2W\nCUDA_VISIBLE_DEVICES=0 python osbp.py data/office31 -d Office31 -s D -t W -a resnet50 --epochs 20 --seed 0 --log logs/osbp/Office31_D2W\nCUDA_VISIBLE_DEVICES=0 python osbp.py data/office31 -d Office31 -s W -t D -a resnet50 --epochs 20 --seed 0 --log logs/osbp/Office31_W2D\nCUDA_VISIBLE_DEVICES=0 python osbp.py data/office31 -d Office31 -s A -t D -a resnet50 --epochs 20 --seed 0 --log logs/osbp/Office31_A2D\nCUDA_VISIBLE_DEVICES=0 python osbp.py data/office31 -d Office31 -s D -t A -a resnet50 --epochs 20 --seed 0 --log logs/osbp/Office31_D2A\nCUDA_VISIBLE_DEVICES=0 python osbp.py data/office31 -d Office31 -s W -t A -a resnet50 --epochs 20 --seed 0 --log logs/osbp/Office31_W2A\n\n# Office-Home\nCUDA_VISIBLE_DEVICES=0 python osbp.py data/office-home -d OfficeHome -s Ar -t Cl -a resnet50 --epochs 30 --seed 0 --log logs/osbp/OfficeHome_Ar2Cl\nCUDA_VISIBLE_DEVICES=0 python osbp.py data/office-home -d OfficeHome -s Ar -t Pr -a resnet50 --epochs 30 --seed 0 --log logs/osbp/OfficeHome_Ar2Pr\nCUDA_VISIBLE_DEVICES=0 python osbp.py data/office-home -d OfficeHome -s Ar -t Rw -a resnet50 --epochs 30 --seed 0 --log logs/osbp/OfficeHome_Ar2Rw\nCUDA_VISIBLE_DEVICES=0 python osbp.py data/office-home -d OfficeHome -s Cl -t Ar -a resnet50 --epochs 30 --seed 0 --log logs/osbp/OfficeHome_Cl2Ar\nCUDA_VISIBLE_DEVICES=0 python osbp.py data/office-home -d OfficeHome -s Cl -t Pr -a resnet50 --epochs 30 --seed 0 --log logs/osbp/OfficeHome_Cl2Pr\nCUDA_VISIBLE_DEVICES=0 python osbp.py data/office-home -d OfficeHome -s Cl -t Rw -a resnet50 --epochs 30 --seed 0 --log logs/osbp/OfficeHome_Cl2Rw\nCUDA_VISIBLE_DEVICES=0 python osbp.py data/office-home -d OfficeHome -s Pr -t Ar -a resnet50 --epochs 30 --seed 0 --log logs/osbp/OfficeHome_Pr2Ar\nCUDA_VISIBLE_DEVICES=0 python osbp.py data/office-home -d OfficeHome -s Pr -t Cl -a resnet50 --epochs 30 --seed 0 --log logs/osbp/OfficeHome_Pr2Cl\nCUDA_VISIBLE_DEVICES=0 python osbp.py data/office-home -d OfficeHome -s Pr -t Rw -a resnet50 --epochs 30 --seed 0 --log logs/osbp/OfficeHome_Pr2Rw\nCUDA_VISIBLE_DEVICES=0 python osbp.py data/office-home -d OfficeHome -s Rw -t Ar -a resnet50 --epochs 30 --seed 0 --log logs/osbp/OfficeHome_Rw2Ar\nCUDA_VISIBLE_DEVICES=0 python osbp.py data/office-home -d OfficeHome -s Rw -t Cl -a resnet50 --epochs 30 --seed 0 --log logs/osbp/OfficeHome_Rw2Cl\nCUDA_VISIBLE_DEVICES=0 python osbp.py data/office-home -d OfficeHome -s Rw -t Pr -a resnet50 --epochs 30 --seed 0 --log logs/osbp/OfficeHome_Rw2Pr\n\n# VisDA-2017\nCUDA_VISIBLE_DEVICES=0 python osbp.py data/visda-2017 -d VisDA2017 -s Synthetic -t Real -a resnet50 \\\n --epochs 30 -i 1000 --seed 0 --train-resizing cen.crop --per-class-eval --log logs/osbp/VisDA2017_S2R"
},
{
"path": "domain_generalization/Transfer-Learning-Library/examples/domain_adaptation/openset_domain_adaptation/source_only.py",
"content": "\"\"\"\n@author: Junguang Jiang\n@contact: JiangJunguang1123@outlook.com\n\"\"\"\nimport random\nimport time\nimport warnings\nimport sys\nimport argparse\nimport shutil\nimport os.path as osp\n\nimport torch\nimport torch.nn as nn\nimport torch.backends.cudnn as cudnn\nfrom torch.optim import SGD\nfrom torch.optim.lr_scheduler import LambdaLR\nfrom torch.utils.data import DataLoader\nimport torch.nn.functional as F\n\nsys.path.append('../../..')\nfrom common.modules.classifier import Classifier\nfrom common.utils.data import ForeverDataIterator\nfrom common.utils.metric import accuracy, ConfusionMatrix\nfrom common.utils.meter import AverageMeter, ProgressMeter\nfrom common.utils.logger import CompleteLogger\nfrom common.utils.analysis import collect_feature, tsne, a_distance\n\nsys.path.append('.')\nimport utils\n\ndevice = torch.device(\"cuda\" if torch.cuda.is_available() else \"cpu\")\n\n\ndef main(args: argparse.Namespace):\n logger = CompleteLogger(args.log, args.phase)\n print(args)\n\n if args.seed is not None:\n random.seed(args.seed)\n torch.manual_seed(args.seed)\n cudnn.deterministic = True\n warnings.warn('You have chosen to seed training. '\n 'This will turn on the CUDNN deterministic setting, '\n 'which can slow down your training considerably! '\n 'You may see unexpected behavior when restarting '\n 'from checkpoints.')\n\n cudnn.benchmark = True\n\n # Data loading code\n # Data loading code\n train_transform = utils.get_train_transform(args.train_resizing, random_horizontal_flip=True, random_color_jitter=False)\n val_transform = utils.get_val_transform(args.val_resizing)\n print(\"train_transform: \", train_transform)\n print(\"val_transform: \", val_transform)\n\n train_source_dataset, _, val_dataset, test_dataset, num_classes, args.class_names = \\\n utils.get_dataset(args.data, args.root, args.source, args.target, train_transform, val_transform)\n train_source_loader = DataLoader(train_source_dataset, batch_size=args.batch_size,\n shuffle=True, num_workers=args.workers, drop_last=True)\n val_loader = DataLoader(val_dataset, batch_size=args.batch_size, shuffle=False, num_workers=args.workers)\n test_loader = DataLoader(test_dataset, batch_size=args.batch_size, shuffle=False, num_workers=args.workers)\n\n train_source_iter = ForeverDataIterator(train_source_loader)\n\n # create model\n print(\"=> using pre-trained model '{}'\".format(args.arch))\n backbone = utils.get_model(args.arch)\n pool_layer = nn.Identity() if args.no_pool else None\n classifier = Classifier(backbone, num_classes, pool_layer=pool_layer).to(device)\n\n # define optimizer and lr scheduler\n optimizer = SGD(classifier.get_parameters(), args.lr, momentum=args.momentum, weight_decay=args.wd, nesterov=True)\n lr_scheduler = LambdaLR(optimizer, lambda x: args.lr * (1. + args.lr_gamma * float(x)) ** (-args.lr_decay))\n\n # analysis the model\n if args.phase == 'analysis':\n # using shuffled val loader\n val_loader = DataLoader(val_dataset, batch_size=args.batch_size, shuffle=True, num_workers=args.workers)\n # extract features from both domains\n feature_extractor = nn.Sequential(classifier.backbone, classifier.pool_layer, classifier.bottleneck).to(device)\n source_feature = collect_feature(train_source_loader, feature_extractor, device)\n target_feature = collect_feature(val_loader, feature_extractor, device)\n # plot t-SNE\n tSNE_filename = osp.join(logger.visualize_directory, 'TSNE.png')\n tsne.visualize(source_feature, target_feature, tSNE_filename)\n print(\"Saving t-SNE to\", tSNE_filename)\n # calculate A-distance, which is a measure for distribution discrepancy\n A_distance = a_distance.calculate(source_feature, target_feature, device)\n print(\"A-distance =\", A_distance)\n return\n\n if args.phase == 'test':\n acc1 = validate(test_loader, classifier, args)\n print(acc1)\n return\n\n # start training\n best_h_score = 0.\n for epoch in range(args.epochs):\n # train for one epoch\n train(train_source_iter, classifier, optimizer,\n lr_scheduler, epoch, args)\n\n # evaluate on validation set\n h_score = validate(val_loader, classifier, args)\n\n # remember best acc@1 and save checkpoint\n torch.save(classifier.state_dict(), logger.get_checkpoint_path('latest'))\n if h_score > best_h_score:\n shutil.copy(logger.get_checkpoint_path('latest'), logger.get_checkpoint_path('best'))\n best_h_score = max(h_score, best_h_score)\n\n print(\"best_h_score = {:3.1f}\".format(best_h_score))\n\n # evaluate on test set\n classifier.load_state_dict(torch.load(logger.get_checkpoint_path('best')))\n h_score = validate(test_loader, classifier, args)\n print(\"test_h_score = {:3.1f}\".format(h_score))\n\n logger.close()\n\n\ndef train(train_source_iter: ForeverDataIterator, model: Classifier, optimizer: SGD,\n lr_scheduler: LambdaLR, epoch: int, args: argparse.Namespace):\n batch_time = AverageMeter('Time', ':4.2f')\n data_time = AverageMeter('Data', ':3.1f')\n losses = AverageMeter('Loss', ':3.2f')\n cls_accs = AverageMeter('Cls Acc', ':3.1f')\n\n progress = ProgressMeter(\n args.iters_per_epoch,\n [batch_time, data_time, losses, cls_accs],\n prefix=\"Epoch: [{}]\".format(epoch))\n\n # switch to train mode\n model.train()\n\n end = time.time()\n for i in range(args.iters_per_epoch):\n x_s, labels_s = next(train_source_iter)\n x_s = x_s.to(device)\n labels_s = labels_s.to(device)\n\n # measure data loading time\n data_time.update(time.time() - end)\n\n # compute output\n y_s, f_s = model(x_s)\n\n cls_loss = F.cross_entropy(y_s, labels_s)\n loss = cls_loss\n\n cls_acc = accuracy(y_s, labels_s)[0]\n\n losses.update(loss.item(), x_s.size(0))\n cls_accs.update(cls_acc.item(), x_s.size(0))\n\n # compute gradient and do SGD step\n optimizer.zero_grad()\n loss.backward()\n optimizer.step()\n lr_scheduler.step()\n\n # measure elapsed time\n batch_time.update(time.time() - end)\n end = time.time()\n\n if i % args.print_freq == 0:\n progress.display(i)\n\n\ndef validate(val_loader: DataLoader, model: Classifier, args: argparse.Namespace) -> float:\n batch_time = AverageMeter('Time', ':6.3f')\n classes = val_loader.dataset.classes\n confmat = ConfusionMatrix(len(classes))\n progress = ProgressMeter(\n len(val_loader),\n [batch_time],\n prefix='Test: ')\n\n # switch to evaluate mode\n model.eval()\n\n with torch.no_grad():\n end = time.time()\n for i, (images, target) in enumerate(val_loader):\n images = images.to(device)\n target = target.to(device)\n\n # compute output\n output = model(images)\n softmax_output = F.softmax(output, dim=1)\n softmax_output[:, -1] = args.threshold\n\n # measure accuracy and record loss\n confmat.update(target, softmax_output.argmax(1))\n\n # measure elapsed time\n batch_time.update(time.time() - end)\n end = time.time()\n\n if i % args.print_freq == 0:\n progress.display(i)\n\n acc_global, accs, iu = confmat.compute()\n all_acc = torch.mean(accs).item() * 100\n known = torch.mean(accs[:-1]).item() * 100\n unknown = accs[-1].item() * 100\n h_score = 2 * known * unknown / (known + unknown)\n if args.per_class_eval:\n print(confmat.format(classes))\n print(' * All {all:.3f} Known {known:.3f} Unknown {unknown:.3f} H-score {h_score:.3f}'\n .format(all=all_acc, known=known, unknown=unknown, h_score=h_score))\n\n return h_score\n\n\nif __name__ == '__main__':\n parser = argparse.ArgumentParser(description='Source Only for Openset Domain Adaptation')\n # dataset parameters\n parser.add_argument('root', metavar='DIR',\n help='root path of dataset')\n parser.add_argument('-d', '--data', metavar='DATA', default='Office31', choices=utils.get_dataset_names(),\n help='dataset: ' + ' | '.join(utils.get_dataset_names()) +\n ' (default: Office31)')\n parser.add_argument('-s', '--source', help='source domain')\n parser.add_argument('-t', '--target', help='target domain')\n parser.add_argument('--train-resizing', type=str, default='default')\n parser.add_argument('--val-resizing', type=str, default='default')\n # model parameters\n parser.add_argument('-a', '--arch', metavar='ARCH', default='resnet18',\n choices=utils.get_model_names(),\n help='backbone architecture: ' +\n ' | '.join(utils.get_model_names()) +\n ' (default: resnet18)')\n parser.add_argument('--no-pool', action='store_true',\n help='no pool layer after the feature extractor.')\n parser.add_argument('--threshold', default=0.8, type=float,\n help='When class confidence is less than the given threshold, '\n 'model will output \"unknown\" (default: 0.5)')\n # training parameters\n parser.add_argument('-b', '--batch-size', default=32, type=int,\n metavar='N',\n help='mini-batch size (default: 32)')\n parser.add_argument('--lr', '--learning-rate', default=0.001, type=float,\n metavar='LR', help='initial learning rate', dest='lr')\n parser.add_argument('--lr-gamma', default=0.0003, type=float, help='parameter for lr scheduler')\n parser.add_argument('--lr-decay', default=0.75, type=float, help='parameter for lr scheduler')\n parser.add_argument('--momentum', default=0.9, type=float, metavar='M',\n help='momentum')\n parser.add_argument('--wd', '--weight-decay', default=0.0005, type=float,\n metavar='W', help='weight decay (default: 5e-4)')\n parser.add_argument('-j', '--workers', default=2, type=int, metavar='N',\n help='number of data loading workers (default: 4)')\n parser.add_argument('--epochs', default=20, type=int, metavar='N',\n help='number of total epochs to run')\n parser.add_argument('-i', '--iters-per-epoch', default=500, type=int,\n help='Number of iterations per epoch')\n parser.add_argument('-p', '--print-freq', default=100, type=int,\n metavar='N', help='print frequency (default: 100)')\n parser.add_argument('--seed', default=None, type=int,\n help='seed for initializing training. ')\n parser.add_argument('--per-class-eval', action='store_true',\n help='whether output per-class accuracy during evaluation')\n parser.add_argument(\"--log\", type=str, default='src_only',\n help=\"Where to save logs, checkpoints and debugging images.\")\n parser.add_argument(\"--phase\", type=str, default='train', choices=['train', 'test', 'analysis'],\n help=\"When phase is 'test', only test the model.\"\n \"When phase is 'analysis', only analysis the model.\")\n args = parser.parse_args()\n main(args)\n\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/examples/domain_adaptation/openset_domain_adaptation/source_only.sh",
"content": "#!/usr/bin/env bash\n# Office31\nCUDA_VISIBLE_DEVICES=0 python source_only.py data/office31 -d Office31 -s A -t W -a resnet50 --epochs 20 --seed 0 --log logs/source_only/Office31_A2W\nCUDA_VISIBLE_DEVICES=0 python source_only.py data/office31 -d Office31 -s D -t W -a resnet50 --epochs 20 --seed 0 --log logs/source_only/Office31_D2W\nCUDA_VISIBLE_DEVICES=0 python source_only.py data/office31 -d Office31 -s W -t D -a resnet50 --epochs 20 --seed 0 --log logs/source_only/Office31_W2D\nCUDA_VISIBLE_DEVICES=0 python source_only.py data/office31 -d Office31 -s A -t D -a resnet50 --epochs 20 --seed 0 --log logs/source_only/Office31_A2D\nCUDA_VISIBLE_DEVICES=0 python source_only.py data/office31 -d Office31 -s D -t A -a resnet50 --epochs 20 --seed 0 --log logs/source_only/Office31_D2A\nCUDA_VISIBLE_DEVICES=0 python source_only.py data/office31 -d Office31 -s W -t A -a resnet50 --epochs 20 --seed 0 --log logs/source_only/Office31_W2A\n\n# Office-Home\nCUDA_VISIBLE_DEVICES=0 python source_only.py data/office-home -d OfficeHome -s Ar -t Cl -a resnet50 --epochs 30 --seed 0 --log logs/source_only/OfficeHome_Ar2Cl\nCUDA_VISIBLE_DEVICES=0 python source_only.py data/office-home -d OfficeHome -s Ar -t Pr -a resnet50 --epochs 30 --seed 0 --log logs/source_only/OfficeHome_Ar2Pr\nCUDA_VISIBLE_DEVICES=0 python source_only.py data/office-home -d OfficeHome -s Ar -t Rw -a resnet50 --epochs 30 --seed 0 --log logs/source_only/OfficeHome_Ar2Rw\nCUDA_VISIBLE_DEVICES=0 python source_only.py data/office-home -d OfficeHome -s Cl -t Ar -a resnet50 --epochs 30 --seed 0 --log logs/source_only/OfficeHome_Cl2Ar\nCUDA_VISIBLE_DEVICES=0 python source_only.py data/office-home -d OfficeHome -s Cl -t Pr -a resnet50 --epochs 30 --seed 0 --log logs/source_only/OfficeHome_Cl2Pr\nCUDA_VISIBLE_DEVICES=0 python source_only.py data/office-home -d OfficeHome -s Cl -t Rw -a resnet50 --epochs 30 --seed 0 --log logs/source_only/OfficeHome_Cl2Rw\nCUDA_VISIBLE_DEVICES=0 python source_only.py data/office-home -d OfficeHome -s Pr -t Ar -a resnet50 --epochs 30 --seed 0 --log logs/source_only/OfficeHome_Pr2Ar\nCUDA_VISIBLE_DEVICES=0 python source_only.py data/office-home -d OfficeHome -s Pr -t Cl -a resnet50 --epochs 30 --seed 0 --log logs/source_only/OfficeHome_Pr2Cl\nCUDA_VISIBLE_DEVICES=0 python source_only.py data/office-home -d OfficeHome -s Pr -t Rw -a resnet50 --epochs 30 --seed 0 --log logs/source_only/OfficeHome_Pr2Rw\nCUDA_VISIBLE_DEVICES=0 python source_only.py data/office-home -d OfficeHome -s Rw -t Ar -a resnet50 --epochs 30 --seed 0 --log logs/source_only/OfficeHome_Rw2Ar\nCUDA_VISIBLE_DEVICES=0 python source_only.py data/office-home -d OfficeHome -s Rw -t Cl -a resnet50 --epochs 30 --seed 0 --log logs/source_only/OfficeHome_Rw2Cl\nCUDA_VISIBLE_DEVICES=0 python source_only.py data/office-home -d OfficeHome -s Rw -t Pr -a resnet50 --epochs 30 --seed 0 --log logs/source_only/OfficeHome_Rw2Pr\n\n# VisDA-2017\nCUDA_VISIBLE_DEVICES=0 python source_only.py data/visda-2017 -d VisDA2017 -s Synthetic -t Real -a resnet50 \\\n --epochs 30 -i 500 --seed 0 --train-resizing cen.crop --per-class-eval --log logs/source_only/VisDA2017_S2R\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/examples/domain_adaptation/openset_domain_adaptation/utils.py",
"content": "import sys\nimport time\nimport timm\nimport torch\nimport torch.nn as nn\nimport torch.nn.functional as F\nimport torchvision.transforms as T\n\nsys.path.append('../../..')\nimport common.vision.datasets.openset as datasets\nfrom common.vision.datasets.openset import default_open_set as open_set\nimport common.vision.models as models\nfrom common.vision.transforms import ResizeImage\nfrom common.utils.metric import accuracy, ConfusionMatrix\nfrom common.utils.meter import AverageMeter, ProgressMeter\n\n\ndef get_model_names():\n return sorted(\n name for name in models.__dict__\n if name.islower() and not name.startswith(\"__\")\n and callable(models.__dict__[name])\n ) + timm.list_models()\n\n\ndef get_model(model_name):\n if model_name in models.__dict__:\n # load models from common.vision.models\n backbone = models.__dict__[model_name](pretrained=True)\n else:\n # load models from pytorch-image-models\n backbone = timm.create_model(model_name, pretrained=True)\n try:\n backbone.out_features = backbone.get_classifier().in_features\n backbone.reset_classifier(0, '')\n backbone.copy_head = backbone.get_classifier\n except:\n backbone.out_features = backbone.head.in_features\n backbone.head = nn.Identity()\n backbone.copy_head = lambda x: x.head\n return backbone\n\n\ndef get_dataset_names():\n return sorted(\n name for name in datasets.__dict__\n if not name.startswith(\"__\") and callable(datasets.__dict__[name])\n )\n\n\ndef get_dataset(dataset_name, root, source, target, train_source_transform, val_transform, train_target_transform=None):\n if train_target_transform is None:\n train_target_transform = train_source_transform\n # load datasets from common.vision.datasets\n dataset = datasets.__dict__[dataset_name]\n source_dataset = open_set(dataset, source=True)\n target_dataset = open_set(dataset, source=False)\n\n train_source_dataset = source_dataset(root=root, task=source, download=True, transform=train_source_transform)\n train_target_dataset = target_dataset(root=root, task=target, download=True, transform=train_target_transform)\n val_dataset = target_dataset(root=root, task=target, download=True, transform=val_transform)\n if dataset_name == 'DomainNet':\n test_dataset = target_dataset(root=root, task=target, split='test', download=True, transform=val_transform)\n else:\n test_dataset = val_dataset\n class_names = train_source_dataset.classes\n num_classes = len(class_names)\n return train_source_dataset, train_target_dataset, val_dataset, test_dataset, num_classes, class_names\n\n\ndef get_train_transform(resizing='default', random_horizontal_flip=True, random_color_jitter=False):\n \"\"\"\n resizing mode:\n - default: resize the image to 256 and take a random resized crop of size 224;\n - cen.crop: resize the image to 256 and take the center crop of size 224;\n - res: resize the image to 224;\n - res.|crop: resize the image to 256 and take a random crop of size 224;\n - res.sma|crop: resize the image keeping its aspect ratio such that the\n smaller side is 256, then take a random crop of size 224;\n – inc.crop: “inception crop” from (Szegedy et al., 2015);\n – cif.crop: resize the image to 224, zero-pad it by 28 on each side, then take a random crop of size 224.\n \"\"\"\n if resizing == 'default':\n transform = T.Compose([\n ResizeImage(256),\n T.RandomResizedCrop(224)\n ])\n elif resizing == 'cen.crop':\n transform = T.Compose([\n ResizeImage(256),\n T.CenterCrop(224)\n ])\n elif resizing == 'ran.crop':\n transform = T.Compose([\n ResizeImage(256),\n T.RandomCrop(224)\n ])\n elif resizing == 'res.':\n transform = T.Resize(224)\n elif resizing == 'res.|crop':\n transform = T.Compose([\n T.Resize((256, 256)),\n T.RandomCrop(224)\n ])\n elif resizing == \"res.sma|crop\":\n transform = T.Compose([\n T.Resize(256),\n T.RandomCrop(224)\n ])\n elif resizing == 'inc.crop':\n transform = T.RandomResizedCrop(224)\n elif resizing == 'cif.crop':\n transform = T.Compose([\n T.Resize((224, 224)),\n T.Pad(28),\n T.RandomCrop(224),\n ])\n else:\n raise NotImplementedError(resizing)\n transforms = [transform]\n if random_horizontal_flip:\n transforms.append(T.RandomHorizontalFlip())\n if random_color_jitter:\n transforms.append(T.ColorJitter(brightness=0.5, contrast=0.5, saturation=0.5, hue=0.5))\n transforms.extend([\n T.ToTensor(),\n T.Normalize(mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225])\n ])\n return T.Compose(transforms)\n\n\ndef get_val_transform(resizing='default'):\n \"\"\"\n resizing mode:\n - default: resize the image to 256 and take the center crop of size 224;\n – res.: resize the image to 224\n – res.|crop: resize the image such that the smaller side is of size 256 and\n then take a central crop of size 224.\n \"\"\"\n if resizing == 'default':\n transform = T.Compose([\n ResizeImage(256),\n T.CenterCrop(224),\n ])\n elif resizing == 'res.':\n transform = T.Resize((224, 224))\n elif resizing == 'res.|crop':\n transform = T.Compose([\n T.Resize(256),\n T.CenterCrop(224),\n ])\n else:\n raise NotImplementedError(resizing)\n return T.Compose([\n transform,\n T.ToTensor(),\n T.Normalize(mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225])\n ])\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/examples/domain_adaptation/partial_domain_adaptation/README.md",
"content": "# Partial Domain Adaptation for Image Classification\n\n## Installation\nExample scripts also support all models in [PyTorch-Image-Models](https://github.com/rwightman/pytorch-image-models).\nYou also need to install timm to use PyTorch-Image-Models.\n\n```\npip install timm\n```\n\n## Dataset\n\nFollowing datasets can be downloaded automatically:\n\n- [Office31](https://www.cc.gatech.edu/~judy/domainadapt/)\n- [OfficeHome](https://www.hemanthdv.org/officeHomeDataset.html)\n- [VisDA2017](http://ai.bu.edu/visda-2017/)\n\nYou need to prepare following datasets manually if you want to use them:\n- [ImageNet](https://www.image-net.org/)\n\nand prepare them following [Documentation for ImageNetCaltech](/common/vision/datasets/partial/imagenet_caltech.py) and [CaltechImageNet](/common/vision/datasets/partial/caltech_imagenet.py).\n\n## Supported Methods\n\nSupported methods include:\n\n- [Domain Adversarial Neural Network (DANN)](https://arxiv.org/abs/1505.07818)\n- [Partial Adversarial Domain Adaptation (PADA)](https://arxiv.org/abs/1808.04205)\n- [Importance Weighted Adversarial Nets (IWAN)](https://arxiv.org/abs/1803.09210)\n- [Adaptive Feature Norm (AFN)](https://arxiv.org/pdf/1811.07456v2.pdf)\n\n## Experiment and Results\n\nThe shell files give the script to reproduce the [benchmarks](/docs/dalib/benchmarks/partial_domain_adaptation.rst) with specified hyper-parameters.\nFor example, if you want to train DANN on Office31, use the following script\n\n```shell script\n# Train a DANN on Office-31 Amazon -> Webcam task using ResNet 50.\n# Assume you have put the datasets under the path `data/office-31`, \n# or you are glad to download the datasets automatically from the Internet to this path\nCUDA_VISIBLE_DEVICES=0 python dann.py data/office31 -d Office31 -s A -t W -a resnet50 --epochs 20 --seed 1 --log logs/dann/Office31_A2W\n```\n\nFor more information please refer to [Get Started](/docs/get_started/quickstart.rst) for help.\n\n## Citation\nIf you use these methods in your research, please consider citing.\n\n```\n@inproceedings{DANN,\n author = {Ganin, Yaroslav and Lempitsky, Victor},\n Booktitle = {ICML},\n Title = {Unsupervised domain adaptation by backpropagation},\n Year = {2015}\n}\n\n@InProceedings{PADA,\n author = {Zhangjie Cao and\n Lijia Ma and\n Mingsheng Long and\n Jianmin Wang},\n title = {Partial Adversarial Domain Adaptation},\n booktitle = {ECCV},\n year = {2018}\n}\n\n@InProceedings{IWAN,\n author = {Jing Zhang and\n Zewei Ding and\n Wanqing Li and\n Philip Ogunbona},\n title = {Importance Weighted Adversarial Nets for Partial Domain Adaptation},\n booktitle = {CVPR},\n year = {2018}\n}\n\n@InProceedings{AFN,\n author = {Xu, Ruijia and Li, Guanbin and Yang, Jihan and Lin, Liang},\n title = {Larger Norm More Transferable: An Adaptive Feature Norm Approach for Unsupervised Domain Adaptation},\n booktitle = {ICCV},\n year = {2019}\n}\n```\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/examples/domain_adaptation/partial_domain_adaptation/afn.py",
"content": "\"\"\"\n@author: Baixu Chen\n@contact: cbx_99_hasta@outlook.com\n\"\"\"\nimport random\nimport time\nimport warnings\nimport sys\nimport argparse\nimport shutil\nimport os.path as osp\n\nimport torch\nimport torch.nn as nn\nimport torch.backends.cudnn as cudnn\nfrom torch.optim import SGD\nfrom torch.utils.data import DataLoader\nimport torch.nn.functional as F\n\nsys.path.append('../../..')\nfrom dalib.adaptation.afn import AdaptiveFeatureNorm, ImageClassifier\nfrom dalib.modules.entropy import entropy\nimport common.vision.models as models\nfrom common.utils.data import ForeverDataIterator\nfrom common.utils.metric import accuracy\nfrom common.utils.meter import AverageMeter, ProgressMeter\nfrom common.utils.logger import CompleteLogger\nfrom common.utils.analysis import collect_feature, tsne, a_distance\n\nsys.path.append('.')\nimport utils\n\ndevice = torch.device(\"cuda\" if torch.cuda.is_available() else \"cpu\")\n\n\ndef main(args: argparse.Namespace):\n logger = CompleteLogger(args.log, args.phase)\n print(args)\n\n if args.seed is not None:\n random.seed(args.seed)\n torch.manual_seed(args.seed)\n cudnn.deterministic = True\n warnings.warn('You have chosen to seed training. '\n 'This will turn on the CUDNN deterministic setting, '\n 'which can slow down your training considerably! '\n 'You may see unexpected behavior when restarting '\n 'from checkpoints.')\n\n cudnn.benchmark = True\n\n # Data loading code\n train_transform = utils.get_train_transform(args.train_resizing, random_horizontal_flip=True,\n random_color_jitter=False)\n val_transform = utils.get_val_transform(args.val_resizing)\n print(\"train_transform: \", train_transform)\n print(\"val_transform: \", val_transform)\n\n train_source_dataset, train_target_dataset, val_dataset, test_dataset, num_classes, args.class_names = \\\n utils.get_dataset(args.data, args.root, args.source, args.target, train_transform, val_transform)\n train_source_loader = DataLoader(train_source_dataset, batch_size=args.batch_size,\n shuffle=True, num_workers=args.workers, drop_last=True)\n train_target_loader = DataLoader(train_target_dataset, batch_size=args.batch_size,\n shuffle=True, num_workers=args.workers, drop_last=True)\n val_loader = DataLoader(val_dataset, batch_size=args.batch_size, shuffle=False, num_workers=args.workers)\n test_loader = DataLoader(test_dataset, batch_size=args.batch_size, shuffle=False, num_workers=args.workers)\n\n train_source_iter = ForeverDataIterator(train_source_loader)\n train_target_iter = ForeverDataIterator(train_target_loader)\n\n # create model\n print(\"=> using pre-trained model '{}'\".format(args.arch))\n backbone = utils.get_model(args.arch)\n pool_layer = nn.Identity() if args.no_pool else None\n backbone = models.__dict__[args.arch](pretrained=True)\n classifier = ImageClassifier(backbone, train_source_dataset.num_classes, args.num_blocks,\n bottleneck_dim=args.bottleneck_dim, dropout_p=args.dropout_p, pool_layer=pool_layer).to(device)\n adaptive_feature_norm = AdaptiveFeatureNorm(args.delta).to(device)\n\n # define optimizer\n # the learning rate is fixed according to origin paper\n optimizer = SGD(classifier.get_parameters(), args.lr, weight_decay=args.weight_decay)\n\n # resume from the best checkpoint\n if args.phase != 'train':\n checkpoint = torch.load(logger.get_checkpoint_path('best'), map_location='cpu')\n classifier.load_state_dict(checkpoint)\n\n # analysis the model\n if args.phase == 'analysis':\n # extract features from both domains\n feature_extractor = nn.Sequential(classifier.backbone, classifier.pool_layer, classifier.bottleneck).to(device)\n source_feature = collect_feature(train_source_loader, feature_extractor, device)\n target_feature = collect_feature(train_target_loader, feature_extractor, device)\n # plot t-SNE\n tSNE_filename = osp.join(logger.visualize_directory, 'TSNE.png')\n tsne.visualize(source_feature, target_feature, tSNE_filename)\n print(\"Saving t-SNE to\", tSNE_filename)\n # calculate A-distance, which is a measure for distribution discrepancy\n A_distance = a_distance.calculate(source_feature, target_feature, device)\n print(\"A-distance =\", A_distance)\n return\n\n if args.phase == 'test':\n acc1 = utils.validate(test_loader, classifier, args, device)\n print(acc1)\n return\n\n # start training\n best_acc1 = 0.\n for epoch in range(args.epochs):\n # train for one epoch\n train(train_source_iter, train_target_iter, classifier, adaptive_feature_norm, optimizer, epoch, args)\n\n # evaluate on validation set\n acc1 = utils.validate(val_loader, classifier, args, device)\n\n # remember best acc@1 and save checkpoint\n torch.save(classifier.state_dict(), logger.get_checkpoint_path('latest'))\n if acc1 > best_acc1:\n shutil.copy(logger.get_checkpoint_path('latest'), logger.get_checkpoint_path('best'))\n best_acc1 = max(acc1, best_acc1)\n\n print(\"best_acc1 = {:3.1f}\".format(best_acc1))\n\n # evaluate on test set\n classifier.load_state_dict(torch.load(logger.get_checkpoint_path('best')))\n acc1 = utils.validate(test_loader, classifier, args, device)\n print(\"test_acc1 = {:3.1f}\".format(acc1))\n\n logger.close()\n\n\ndef train(train_source_iter: ForeverDataIterator, train_target_iter: ForeverDataIterator, model: ImageClassifier,\n adaptive_feature_norm: AdaptiveFeatureNorm, optimizer: SGD, epoch: int, args: argparse.Namespace):\n batch_time = AverageMeter('Time', ':3.1f')\n data_time = AverageMeter('Data', ':3.1f')\n cls_losses = AverageMeter('Cls Loss', ':3.2f')\n norm_losses = AverageMeter('Norm Loss', ':3.2f')\n src_feature_norm = AverageMeter('Source Feature Norm', ':3.2f')\n tgt_feature_norm = AverageMeter('Target Feature Norm', ':3.2f')\n cls_accs = AverageMeter('Cls Acc', ':3.1f')\n tgt_accs = AverageMeter('Tgt Acc', ':3.1f')\n\n progress = ProgressMeter(\n args.iters_per_epoch,\n [batch_time, data_time, cls_losses, norm_losses, src_feature_norm, tgt_feature_norm, cls_accs, tgt_accs],\n prefix=\"Epoch: [{}]\".format(epoch))\n\n # switch to train mode\n model.train()\n\n end = time.time()\n for i in range(args.iters_per_epoch):\n x_s, labels_s = next(train_source_iter)\n x_t, labels_t = next(train_target_iter)\n\n x_s = x_s.to(device)\n x_t = x_t.to(device)\n labels_s = labels_s.to(device)\n labels_t = labels_t.to(device)\n\n # measure data loading time\n data_time.update(time.time() - end)\n\n # compute output\n y_s, f_s = model(x_s)\n y_t, f_t = model(x_t)\n\n # classification loss\n cls_loss = F.cross_entropy(y_s, labels_s)\n # norm loss\n norm_loss = adaptive_feature_norm(f_s) + adaptive_feature_norm(f_t)\n\n loss = cls_loss + norm_loss * args.trade_off_norm\n\n # using entropy minimization\n if args.trade_off_entropy:\n y_t = F.softmax(y_t, dim=1)\n entropy_loss = entropy(y_t, reduction='mean')\n loss += entropy_loss * args.trade_off_entropy\n\n # compute gradient and do SGD step\n optimizer.zero_grad()\n loss.backward()\n optimizer.step()\n\n # update statistics\n cls_acc = accuracy(y_s, labels_s)[0]\n tgt_acc = accuracy(y_t, labels_t)[0]\n\n cls_losses.update(cls_loss.item(), x_s.size(0))\n norm_losses.update(norm_loss.item(), x_s.size(0))\n src_feature_norm.update(f_s.norm(p=2, dim=1).mean().item(), x_s.size(0))\n tgt_feature_norm.update(f_t.norm(p=2, dim=1).mean().item(), x_s.size(0))\n cls_accs.update(cls_acc.item(), x_s.size(0))\n tgt_accs.update(tgt_acc.item(), x_s.size(0))\n\n # measure elapsed time\n batch_time.update(time.time() - end)\n end = time.time()\n\n if i % args.print_freq == 0:\n progress.display(i)\n\n\nif __name__ == '__main__':\n parser = argparse.ArgumentParser(description='AFN for Partial Domain Adaptation')\n # dataset parameters\n parser.add_argument('root', metavar='DIR',\n help='root path of dataset')\n parser.add_argument('-d', '--data', metavar='DATA', default='Office31', choices=utils.get_dataset_names(),\n help='dataset: ' + ' | '.join(utils.get_dataset_names()) +\n ' (default: Office31)')\n parser.add_argument('-s', '--source', help='source domain')\n parser.add_argument('-t', '--target', help='target domain')\n parser.add_argument('--train-resizing', type=str, default='ran.crop')\n parser.add_argument('--val-resizing', type=str, default='default')\n # model parameters\n parser.add_argument('-a', '--arch', metavar='ARCH', default='resnet18',\n choices=utils.get_model_names(),\n help='backbone architecture: ' +\n ' | '.join(utils.get_model_names()) +\n ' (default: resnet18)')\n parser.add_argument('--no-pool', action='store_true',\n help='no pool layer after the feature extractor.')\n parser.add_argument('-n', '--num-blocks', default=1, type=int, help='Number of basic blocks for classifier')\n parser.add_argument('--bottleneck-dim', default=1000, type=int, help='Dimension of bottleneck')\n parser.add_argument('--dropout-p', default=0.5, type=float,\n help='Dropout probability')\n # training parameters\n parser.add_argument('-b', '--batch-size', default=32, type=int,\n metavar='N',\n help='mini-batch size (default: 32)')\n parser.add_argument('--lr', '--learning-rate', default=0.001, type=float,\n metavar='LR', help='initial learning rate', dest='lr')\n parser.add_argument('--momentum', default=0.9, type=float, metavar='M',\n help='momentum')\n parser.add_argument('--wd', '--weight-decay', default=5e-4, type=float,\n metavar='W', help='weight decay (default: 5e-4)',\n dest='weight_decay')\n parser.add_argument('--trade-off-norm', default=0.05, type=float,\n help='the trade-off hyper-parameter for norm loss')\n parser.add_argument('--trade-off-entropy', default=None, type=float,\n help='the trade-off hyper-parameter for entropy loss')\n parser.add_argument('-r', '--delta', default=1, type=float, help='Increment for L2 norm')\n parser.add_argument('-j', '--workers', default=4, type=int, metavar='N',\n help='number of data loading workers (default: 4)')\n parser.add_argument('--epochs', default=20, type=int, metavar='N',\n help='number of total epochs to run')\n parser.add_argument('-i', '--iters-per-epoch', default=500, type=int,\n help='Number of iterations per epoch')\n parser.add_argument('-p', '--print-freq', default=100, type=int,\n metavar='N', help='print frequency (default: 100)')\n parser.add_argument('--seed', default=None, type=int,\n help='seed for initializing training. ')\n parser.add_argument('--per-class-eval', action='store_true',\n help='whether output per-class accuracy during evaluation')\n parser.add_argument(\"--log\", type=str, default='afn',\n help=\"Where to save logs, checkpoints and debugging images.\")\n parser.add_argument(\"--phase\", type=str, default='train', choices=['train', 'test', 'analysis'],\n help=\"When phase is 'test', only test the model.\"\n \"When phase is 'analysis', only analysis the model.\")\n args = parser.parse_args()\n main(args)\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/examples/domain_adaptation/partial_domain_adaptation/afn.sh",
"content": "#!/usr/bin/env bash\n# Office31\nCUDA_VISIBLE_DEVICES=0 python afn.py data/office31 -d Office31 -s A -t W -a resnet50 --trade-off-entropy 0.1 --epochs 20 --seed 1 --log logs/afn/Office31_A2W\nCUDA_VISIBLE_DEVICES=0 python afn.py data/office31 -d Office31 -s D -t W -a resnet50 --trade-off-entropy 0.1 --epochs 20 --seed 1 --log logs/afn/Office31_D2W\nCUDA_VISIBLE_DEVICES=0 python afn.py data/office31 -d Office31 -s W -t D -a resnet50 --trade-off-entropy 0.1 --epochs 20 --seed 1 --log logs/afn/Office31_W2D\nCUDA_VISIBLE_DEVICES=0 python afn.py data/office31 -d Office31 -s A -t D -a resnet50 --trade-off-entropy 0.1 --epochs 20 --seed 1 --log logs/afn/Office31_A2D\nCUDA_VISIBLE_DEVICES=0 python afn.py data/office31 -d Office31 -s D -t A -a resnet50 --trade-off-entropy 0.1 --epochs 20 --seed 1 --log logs/afn/Office31_D2A\nCUDA_VISIBLE_DEVICES=0 python afn.py data/office31 -d Office31 -s W -t A -a resnet50 --trade-off-entropy 0.1 --epochs 20 --seed 1 --log logs/afn/Office31_W2A\n\n# Office-Home\nCUDA_VISIBLE_DEVICES=0 python afn.py data/office-home -d OfficeHome -s Ar -t Cl -a resnet50 --epochs 30 --seed 0 --log logs/afn/OfficeHome_Ar2Cl\nCUDA_VISIBLE_DEVICES=0 python afn.py data/office-home -d OfficeHome -s Ar -t Pr -a resnet50 --epochs 30 --seed 0 --log logs/afn/OfficeHome_Ar2Pr\nCUDA_VISIBLE_DEVICES=0 python afn.py data/office-home -d OfficeHome -s Ar -t Rw -a resnet50 --epochs 30 --seed 0 --log logs/afn/OfficeHome_Ar2Rw\nCUDA_VISIBLE_DEVICES=0 python afn.py data/office-home -d OfficeHome -s Cl -t Ar -a resnet50 --epochs 30 --seed 0 --log logs/afn/OfficeHome_Cl2Ar\nCUDA_VISIBLE_DEVICES=0 python afn.py data/office-home -d OfficeHome -s Cl -t Pr -a resnet50 --epochs 30 --seed 0 --log logs/afn/OfficeHome_Cl2Pr\nCUDA_VISIBLE_DEVICES=0 python afn.py data/office-home -d OfficeHome -s Cl -t Rw -a resnet50 --epochs 30 --seed 0 --log logs/afn/OfficeHome_Cl2Rw\nCUDA_VISIBLE_DEVICES=0 python afn.py data/office-home -d OfficeHome -s Pr -t Ar -a resnet50 --epochs 30 --seed 0 --log logs/afn/OfficeHome_Pr2Ar\nCUDA_VISIBLE_DEVICES=0 python afn.py data/office-home -d OfficeHome -s Pr -t Cl -a resnet50 --epochs 30 --seed 0 --log logs/afn/OfficeHome_Pr2Cl\nCUDA_VISIBLE_DEVICES=0 python afn.py data/office-home -d OfficeHome -s Pr -t Rw -a resnet50 --epochs 30 --seed 0 --log logs/afn/OfficeHome_Pr2Rw\nCUDA_VISIBLE_DEVICES=0 python afn.py data/office-home -d OfficeHome -s Rw -t Ar -a resnet50 --epochs 30 --seed 0 --log logs/afn/OfficeHome_Rw2Ar\nCUDA_VISIBLE_DEVICES=0 python afn.py data/office-home -d OfficeHome -s Rw -t Cl -a resnet50 --epochs 30 --seed 0 --log logs/afn/OfficeHome_Rw2Cl\nCUDA_VISIBLE_DEVICES=0 python afn.py data/office-home -d OfficeHome -s Rw -t Pr -a resnet50 --epochs 30 --seed 0 --log logs/afn/OfficeHome_Rw2Pr\n\n# VisDA-2017\nCUDA_VISIBLE_DEVICES=0 python afn.py data/visda-2017 -d VisDA2017 -s Synthetic -t Real -a resnet50 -r 0.3 -b 36 \\\n --epochs 30 -i 1000 --seed 0 --per-class-eval --train-resizing cen.crop --log logs/afn/VisDA2017"
},
{
"path": "domain_generalization/Transfer-Learning-Library/examples/domain_adaptation/partial_domain_adaptation/dann.py",
"content": "\"\"\"\n@author: Junguang Jiang\n@contact: JiangJunguang1123@outlook.com\n\"\"\"\nimport random\nimport time\nimport warnings\nimport sys\nimport argparse\nimport shutil\nimport os.path as osp\n\nimport torch\nimport torch.nn as nn\nimport torch.backends.cudnn as cudnn\nfrom torch.optim import SGD\nfrom torch.optim.lr_scheduler import LambdaLR\nfrom torch.utils.data import DataLoader\nimport torch.nn.functional as F\n\nsys.path.append('../../..')\nfrom dalib.modules.domain_discriminator import DomainDiscriminator\nfrom common.modules.classifier import Classifier\nfrom dalib.adaptation.dann import DomainAdversarialLoss, ImageClassifier\nfrom common.utils.data import ForeverDataIterator\nfrom common.utils.metric import accuracy\nfrom common.utils.meter import AverageMeter, ProgressMeter\nfrom common.utils.logger import CompleteLogger\nfrom common.utils.analysis import collect_feature, tsne, a_distance\n\nsys.path.append('.')\nimport utils\n\ndevice = torch.device(\"cuda\" if torch.cuda.is_available() else \"cpu\")\n\n\ndef main(args: argparse.Namespace):\n logger = CompleteLogger(args.log, args.phase)\n print(args)\n\n if args.seed is not None:\n random.seed(args.seed)\n torch.manual_seed(args.seed)\n cudnn.deterministic = True\n warnings.warn('You have chosen to seed training. '\n 'This will turn on the CUDNN deterministic setting, '\n 'which can slow down your training considerably! '\n 'You may see unexpected behavior when restarting '\n 'from checkpoints.')\n\n cudnn.benchmark = True\n\n # Data loading code\n train_transform = utils.get_train_transform(args.train_resizing, random_horizontal_flip=True,\n random_color_jitter=False)\n val_transform = utils.get_val_transform(args.val_resizing)\n print(\"train_transform: \", train_transform)\n print(\"val_transform: \", val_transform)\n\n train_source_dataset, train_target_dataset, val_dataset, test_dataset, num_classes, args.class_names = \\\n utils.get_dataset(args.data, args.root, args.source, args.target, train_transform, val_transform)\n train_source_loader = DataLoader(train_source_dataset, batch_size=args.batch_size,\n shuffle=True, num_workers=args.workers, drop_last=True)\n train_target_loader = DataLoader(train_target_dataset, batch_size=args.batch_size,\n shuffle=True, num_workers=args.workers, drop_last=True)\n val_loader = DataLoader(val_dataset, batch_size=args.batch_size, shuffle=False, num_workers=args.workers)\n test_loader = DataLoader(test_dataset, batch_size=args.batch_size, shuffle=False, num_workers=args.workers)\n\n train_source_iter = ForeverDataIterator(train_source_loader)\n train_target_iter = ForeverDataIterator(train_target_loader)\n\n # create model\n print(\"=> using pre-trained model '{}'\".format(args.arch))\n backbone = utils.get_model(args.arch)\n pool_layer = nn.Identity() if args.no_pool else None\n if args.data == 'ImageNetCaltech':\n classifier = Classifier(backbone, num_classes, head=backbone.copy_head(), pool_layer=pool_layer).to(device)\n else:\n classifier = ImageClassifier(backbone, num_classes, args.bottleneck_dim, pool_layer=pool_layer).to(device)\n\n domain_discri = DomainDiscriminator(in_feature=classifier.features_dim, hidden_size=1024).to(device)\n\n # define optimizer and lr scheduler\n optimizer = SGD(classifier.get_parameters() + domain_discri.get_parameters(),\n args.lr, momentum=args.momentum, weight_decay=args.weight_decay, nesterov=True)\n lr_scheduler = LambdaLR(optimizer, lambda x: args.lr * (1. + args.lr_gamma * float(x)) ** (-args.lr_decay))\n\n # define loss function\n domain_adv = DomainAdversarialLoss(domain_discri).to(device)\n\n # resume from the best checkpoint\n if args.phase != 'train':\n checkpoint = torch.load(logger.get_checkpoint_path('best'), map_location='cpu')\n classifier.load_state_dict(checkpoint)\n\n # analysis the model\n if args.phase == 'analysis':\n # extract features from both domains\n feature_extractor = nn.Sequential(classifier.backbone, classifier.pool_layer, classifier.bottleneck).to(device)\n source_feature = collect_feature(train_source_loader, feature_extractor, device)\n target_feature = collect_feature(train_target_loader, feature_extractor, device)\n # plot t-SNE\n tSNE_filename = osp.join(logger.visualize_directory, 'TSNE.png')\n tsne.visualize(source_feature, target_feature, tSNE_filename)\n print(\"Saving t-SNE to\", tSNE_filename)\n # calculate A-distance, which is a measure for distribution discrepancy\n A_distance = a_distance.calculate(source_feature, target_feature, device)\n print(\"A-distance =\", A_distance)\n return\n\n if args.phase == 'test':\n acc1 = utils.validate(test_loader, classifier, args, device)\n print(acc1)\n return\n\n # start training\n best_acc1 = 0.\n for epoch in range(args.epochs):\n # train for one epoch\n train(train_source_iter, train_target_iter, classifier, domain_adv, optimizer,\n lr_scheduler, epoch, args)\n\n # evaluate on validation set\n acc1 = utils.validate(val_loader, classifier, args, device)\n\n # remember best acc@1 and save checkpoint\n torch.save(classifier.state_dict(), logger.get_checkpoint_path('latest'))\n if acc1 > best_acc1:\n shutil.copy(logger.get_checkpoint_path('latest'), logger.get_checkpoint_path('best'))\n best_acc1 = max(acc1, best_acc1)\n\n print(\"best_acc1 = {:3.1f}\".format(best_acc1))\n\n # evaluate on test set\n classifier.load_state_dict(torch.load(logger.get_checkpoint_path('best')))\n acc1 = utils.validate(test_loader, classifier, args, device)\n print(\"test_acc1 = {:3.1f}\".format(acc1))\n\n logger.close()\n\n\ndef train(train_source_iter: ForeverDataIterator, train_target_iter: ForeverDataIterator,\n model: ImageClassifier, domain_adv: DomainAdversarialLoss, optimizer: SGD,\n lr_scheduler: LambdaLR, epoch: int, args: argparse.Namespace):\n batch_time = AverageMeter('Time', ':5.2f')\n data_time = AverageMeter('Data', ':5.2f')\n losses = AverageMeter('Loss', ':6.2f')\n cls_accs = AverageMeter('Cls Acc', ':3.1f')\n tgt_accs = AverageMeter('Tgt Acc', ':3.1f')\n domain_accs = AverageMeter('Domain Acc', ':3.1f')\n progress = ProgressMeter(\n args.iters_per_epoch,\n [batch_time, data_time, losses, cls_accs, tgt_accs, domain_accs],\n prefix=\"Epoch: [{}]\".format(epoch))\n\n # switch to train mode\n model.train()\n domain_adv.train()\n\n end = time.time()\n for i in range(args.iters_per_epoch):\n x_s, labels_s = next(train_source_iter)\n x_t, labels_t = next(train_target_iter)\n\n x_s = x_s.to(device)\n x_t = x_t.to(device)\n labels_s = labels_s.to(device)\n labels_t = labels_t.to(device)\n\n # measure data loading time\n data_time.update(time.time() - end)\n\n # compute output\n x = torch.cat((x_s, x_t), dim=0)\n y, f = model(x)\n y_s, y_t = y.chunk(2, dim=0)\n f_s, f_t = f.chunk(2, dim=0)\n\n cls_loss = F.cross_entropy(y_s, labels_s)\n transfer_loss = domain_adv(f_s, f_t)\n domain_acc = domain_adv.domain_discriminator_accuracy\n loss = cls_loss + transfer_loss * args.trade_off\n\n cls_acc = accuracy(y_s, labels_s)[0]\n tgt_acc = accuracy(y_t, labels_t)[0]\n\n losses.update(loss.item(), x_s.size(0))\n cls_accs.update(cls_acc.item(), x_s.size(0))\n tgt_accs.update(tgt_acc.item(), x_s.size(0))\n domain_accs.update(domain_acc.item(), x_s.size(0))\n\n # compute gradient and do SGD step\n optimizer.zero_grad()\n loss.backward()\n optimizer.step()\n lr_scheduler.step()\n\n # measure elapsed time\n batch_time.update(time.time() - end)\n end = time.time()\n\n if i % args.print_freq == 0:\n progress.display(i)\n\n\nif __name__ == '__main__':\n parser = argparse.ArgumentParser(description='DANN for Partial Domain Adaptation')\n # dataset parameters\n parser.add_argument('root', metavar='DIR',\n help='root path of dataset')\n parser.add_argument('-d', '--data', metavar='DATA', default='Office31', choices=utils.get_dataset_names(),\n help='dataset: ' + ' | '.join(utils.get_dataset_names()) +\n ' (default: Office31)')\n parser.add_argument('-s', '--source', help='source domain')\n parser.add_argument('-t', '--target', help='target domain')\n parser.add_argument('--train-resizing', type=str, default='default')\n parser.add_argument('--val-resizing', type=str, default='default')\n # model parameters\n parser.add_argument('-a', '--arch', metavar='ARCH', default='resnet18',\n choices=utils.get_model_names(),\n help='backbone architecture: ' +\n ' | '.join(utils.get_model_names()) +\n ' (default: resnet18)')\n parser.add_argument('--no-pool', action='store_true',\n help='no pool layer after the feature extractor.')\n parser.add_argument('--bottleneck-dim', default=256, type=int,\n help='Dimension of bottleneck')\n parser.add_argument('--trade-off', default=1., type=float,\n help='the trade-off hyper-parameter for transfer loss')\n # training parameters\n parser.add_argument('-b', '--batch-size', default=36, type=int,\n metavar='N',\n help='mini-batch size (default: 36)')\n parser.add_argument('--lr', '--learning-rate', default=0.002, type=float,\n metavar='LR', help='initial learning rate', dest='lr')\n parser.add_argument('--lr-gamma', default=0.001, type=float, help='parameter for lr scheduler')\n parser.add_argument('--lr-decay', default=0.75, type=float, help='parameter for lr scheduler')\n parser.add_argument('--momentum', default=0.9, type=float, metavar='M',\n help='momentum')\n parser.add_argument('--wd', '--weight-decay',default=1e-3, type=float,\n metavar='W', help='weight decay (default: 1e-3)',\n dest='weight_decay')\n parser.add_argument('-j', '--workers', default=2, type=int, metavar='N',\n help='number of data loading workers (default: 4)')\n parser.add_argument('--epochs', default=20, type=int, metavar='N',\n help='number of total epochs to run')\n parser.add_argument('-i', '--iters-per-epoch', default=1000, type=int,\n help='Number of iterations per epoch')\n parser.add_argument('-p', '--print-freq', default=100, type=int,\n metavar='N', help='print frequency (default: 100)')\n parser.add_argument('--seed', default=None, type=int,\n help='seed for initializing training. ')\n parser.add_argument('--per-class-eval', action='store_true',\n help='whether output per-class accuracy during evaluation')\n parser.add_argument(\"--log\", type=str, default='dann',\n help=\"Where to save logs, checkpoints and debugging images.\")\n parser.add_argument(\"--phase\", type=str, default='train', choices=['train', 'test', 'analysis'],\n help=\"When phase is 'test', only test the model.\"\n \"When phase is 'analysis', only analysis the model.\")\n args = parser.parse_args()\n main(args)\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/examples/domain_adaptation/partial_domain_adaptation/dann.sh",
"content": "#!/usr/bin/env bash\n# Office31\nCUDA_VISIBLE_DEVICES=0 python dann.py data/office31 -d Office31 -s A -t W -a resnet50 --epochs 5 --seed 0 --log logs/dann/Office31_A2W\nCUDA_VISIBLE_DEVICES=0 python dann.py data/office31 -d Office31 -s D -t W -a resnet50 --epochs 5 --seed 0 --log logs/dann/Office31_D2W\nCUDA_VISIBLE_DEVICES=0 python dann.py data/office31 -d Office31 -s W -t D -a resnet50 --epochs 5 --seed 0 --log logs/dann/Office31_W2D\nCUDA_VISIBLE_DEVICES=0 python dann.py data/office31 -d Office31 -s A -t D -a resnet50 --epochs 5 --seed 0 --log logs/dann/Office31_A2D\nCUDA_VISIBLE_DEVICES=0 python dann.py data/office31 -d Office31 -s D -t A -a resnet50 --epochs 5 --seed 0 --log logs/dann/Office31_D2A\nCUDA_VISIBLE_DEVICES=0 python dann.py data/office31 -d Office31 -s W -t A -a resnet50 --epochs 5 --seed 0 --log logs/dann/Office31_W2A\n\n# Office-Home\nCUDA_VISIBLE_DEVICES=0 python dann.py data/office-home -d OfficeHome -s Ar -t Cl -a resnet50 --epochs 5 --seed 0 --log logs/dann/OfficeHome_Ar2Cl\nCUDA_VISIBLE_DEVICES=0 python dann.py data/office-home -d OfficeHome -s Ar -t Pr -a resnet50 --epochs 5 --seed 0 --log logs/dann/OfficeHome_Ar2Pr\nCUDA_VISIBLE_DEVICES=0 python dann.py data/office-home -d OfficeHome -s Ar -t Rw -a resnet50 --epochs 5 --seed 0 --log logs/dann/OfficeHome_Ar2Rw\nCUDA_VISIBLE_DEVICES=0 python dann.py data/office-home -d OfficeHome -s Cl -t Ar -a resnet50 --epochs 5 --seed 0 --log logs/dann/OfficeHome_Cl2Ar\nCUDA_VISIBLE_DEVICES=0 python dann.py data/office-home -d OfficeHome -s Cl -t Pr -a resnet50 --epochs 5 --seed 0 --log logs/dann/OfficeHome_Cl2Pr\nCUDA_VISIBLE_DEVICES=0 python dann.py data/office-home -d OfficeHome -s Cl -t Rw -a resnet50 --epochs 5 --seed 0 --log logs/dann/OfficeHome_Cl2Rw\nCUDA_VISIBLE_DEVICES=0 python dann.py data/office-home -d OfficeHome -s Pr -t Ar -a resnet50 --epochs 5 --seed 0 --log logs/dann/OfficeHome_Pr2Ar\nCUDA_VISIBLE_DEVICES=0 python dann.py data/office-home -d OfficeHome -s Pr -t Cl -a resnet50 --epochs 5 --seed 0 --log logs/dann/OfficeHome_Pr2Cl\nCUDA_VISIBLE_DEVICES=0 python dann.py data/office-home -d OfficeHome -s Pr -t Rw -a resnet50 --epochs 5 --seed 0 --log logs/dann/OfficeHome_Pr2Rw\nCUDA_VISIBLE_DEVICES=0 python dann.py data/office-home -d OfficeHome -s Rw -t Ar -a resnet50 --epochs 5 --seed 0 --log logs/dann/OfficeHome_Rw2Ar\nCUDA_VISIBLE_DEVICES=0 python dann.py data/office-home -d OfficeHome -s Rw -t Cl -a resnet50 --epochs 5 --seed 0 --log logs/dann/OfficeHome_Rw2Cl\nCUDA_VISIBLE_DEVICES=0 python dann.py data/office-home -d OfficeHome -s Rw -t Pr -a resnet50 --epochs 5 --seed 0 --log logs/dann/OfficeHome_Rw2Pr\n\n# VisDA-2017\nCUDA_VISIBLE_DEVICES=0 python dann.py data/visda-2017 -d VisDA2017 -s Synthetic -t Real -a resnet50 \\\n --epochs 5 --seed 0 --train-resizing cen.crop --per-class-eval --log logs/dann/VisDA2017_S2R\n\n# ImageNet-Caltech\nCUDA_VISIBLE_DEVICES=0 python dann.py data/ImageNetCaltech -d ImageNetCaltech -s I -t C -a resnet50 \\\n --epochs 5 --seed 0 --log logs/dann/I2C\nCUDA_VISIBLE_DEVICES=0 python dann.py data/ImageNetCaltech -d CaltechImageNet -s C -t I -a resnet50 \\\n --epochs 5 --seed 0 --bottleneck-dim 2048 --log logs/dann/C2I\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/examples/domain_adaptation/partial_domain_adaptation/iwan.py",
"content": "\"\"\"\n@author: Baixu Chen\n@contact: cbx_99_hasta@outlook.com\n\"\"\"\nimport random\nimport time\nimport warnings\nimport sys\nimport argparse\nimport shutil\nimport os.path as osp\n\nimport torch\nimport torch.nn as nn\nimport torch.backends.cudnn as cudnn\nfrom torch.optim import SGD\nfrom torch.optim.lr_scheduler import LambdaLR\nfrom torch.utils.data import DataLoader\nimport torch.nn.functional as F\n\nsys.path.append('../../..')\nfrom common.modules.classifier import Classifier\nfrom dalib.modules.entropy import entropy\nfrom dalib.modules.domain_discriminator import DomainDiscriminator\nfrom dalib.adaptation.iwan import ImportanceWeightModule, ImageClassifier\nfrom dalib.adaptation.dann import DomainAdversarialLoss\nfrom common.utils.data import ForeverDataIterator\nfrom common.utils.metric import accuracy\nfrom common.utils.meter import AverageMeter, ProgressMeter\nfrom common.utils.logger import CompleteLogger\nfrom common.utils.analysis import collect_feature, tsne, a_distance\n\nsys.path.append('.')\nimport utils\n\ndevice = torch.device(\"cuda\" if torch.cuda.is_available() else \"cpu\")\n\n\ndef main(args: argparse.Namespace):\n logger = CompleteLogger(args.log, args.phase)\n print(args)\n\n if args.seed is not None:\n random.seed(args.seed)\n torch.manual_seed(args.seed)\n cudnn.deterministic = True\n warnings.warn('You have chosen to seed training. '\n 'This will turn on the CUDNN deterministic setting, '\n 'which can slow down your training considerably! '\n 'You may see unexpected behavior when restarting '\n 'from checkpoints.')\n\n cudnn.benchmark = True\n\n # Data loading code\n train_transform = utils.get_train_transform(args.train_resizing, random_horizontal_flip=True,\n random_color_jitter=False)\n val_transform = utils.get_val_transform(args.val_resizing)\n print(\"train_transform: \", train_transform)\n print(\"val_transform: \", val_transform)\n\n train_source_dataset, train_target_dataset, val_dataset, test_dataset, num_classes, args.class_names = \\\n utils.get_dataset(args.data, args.root, args.source, args.target, train_transform, val_transform)\n train_source_loader = DataLoader(train_source_dataset, batch_size=args.batch_size,\n shuffle=True, num_workers=args.workers, drop_last=True)\n train_target_loader = DataLoader(train_target_dataset, batch_size=args.batch_size,\n shuffle=True, num_workers=args.workers, drop_last=True)\n val_loader = DataLoader(val_dataset, batch_size=args.batch_size, shuffle=False, num_workers=args.workers)\n test_loader = DataLoader(test_dataset, batch_size=args.batch_size, shuffle=False, num_workers=args.workers)\n\n train_source_iter = ForeverDataIterator(train_source_loader)\n train_target_iter = ForeverDataIterator(train_target_loader)\n\n # create model\n print(\"=> using pre-trained model '{}'\".format(args.arch))\n backbone = utils.get_model(args.arch)\n pool_layer = nn.Identity() if args.no_pool else None\n if args.data == 'ImageNetCaltech':\n classifier = Classifier(backbone, num_classes, head=backbone.copy_head(), pool_layer=pool_layer).to(device)\n else:\n classifier = ImageClassifier(backbone, num_classes, args.bottleneck_dim, pool_layer=pool_layer).to(device)\n\n # define domain classifier D, D_0\n D = DomainDiscriminator(in_feature=classifier.features_dim, hidden_size=1024, batch_norm=False).to(device)\n D_0 = DomainDiscriminator(in_feature=classifier.features_dim, hidden_size=1024, batch_norm=False).to(device)\n\n # define optimizer and lr scheduler\n optimizer = SGD(classifier.get_parameters() + D.get_parameters() + D_0.get_parameters(),\n args.lr, momentum=args.momentum, weight_decay=args.weight_decay, nesterov=True)\n lr_scheduler = LambdaLR(optimizer, lambda x: args.lr * (1. + args.lr_gamma * float(x)) ** (-args.lr_decay))\n\n # define loss function\n domain_adv_D = DomainAdversarialLoss(D).to(device)\n domain_adv_D_0 = DomainAdversarialLoss(D_0).to(device)\n # define importance weight module\n importance_weight_module = ImportanceWeightModule(D, train_target_dataset.partial_classes_idx)\n\n # resume from the best checkpoint\n if args.phase != 'train':\n checkpoint = torch.load(logger.get_checkpoint_path('best'), map_location='cpu')\n classifier.load_state_dict(checkpoint)\n\n # analysis the model\n if args.phase == 'analysis':\n # extract features from both domains\n feature_extractor = nn.Sequential(classifier.backbone, classifier.pool_layer, classifier.bottleneck).to(device)\n source_feature = collect_feature(train_source_loader, feature_extractor, device)\n target_feature = collect_feature(train_target_loader, feature_extractor, device)\n # plot t-SNE\n tSNE_filename = osp.join(logger.visualize_directory, 'TSNE.png')\n tsne.visualize(source_feature, target_feature, tSNE_filename)\n print(\"Saving t-SNE to\", tSNE_filename)\n # calculate A-distance, which is a measure for distribution discrepancy\n A_distance = a_distance.calculate(source_feature, target_feature, device)\n print(\"A-distance =\", A_distance)\n return\n\n if args.phase == 'test':\n acc1 = utils.validate(test_loader, classifier, args, device)\n print(acc1)\n return\n\n # start training\n best_acc1 = 0.\n for epoch in range(args.epochs):\n # train for one epoch\n train(train_source_iter, train_target_iter, classifier, domain_adv_D, domain_adv_D_0,\n importance_weight_module, optimizer, lr_scheduler, epoch, args)\n\n # evaluate on validation set\n acc1 = utils.validate(val_loader, classifier, args, device)\n\n # remember best acc@1 and save checkpoint\n torch.save(classifier.state_dict(), logger.get_checkpoint_path('latest'))\n if acc1 > best_acc1:\n shutil.copy(logger.get_checkpoint_path('latest'), logger.get_checkpoint_path('best'))\n best_acc1 = max(acc1, best_acc1)\n\n print(\"best_acc1 = {:3.1f}\".format(best_acc1))\n\n # evaluate on test set\n classifier.load_state_dict(torch.load(logger.get_checkpoint_path('best')))\n acc1 = utils.validate(test_loader, classifier, args, device)\n print(\"test_acc1 = {:3.1f}\".format(acc1))\n\n logger.close()\n\n\ndef train(train_source_iter: ForeverDataIterator, train_target_iter: ForeverDataIterator, model: ImageClassifier,\n domain_adv_D: DomainAdversarialLoss, domain_adv_D_0: DomainAdversarialLoss,\n importance_weight_module, optimizer: SGD, lr_scheduler: LambdaLR, epoch: int, args: argparse.Namespace):\n batch_time = AverageMeter('Time', ':5.2f')\n data_time = AverageMeter('Data', ':5.2f')\n losses = AverageMeter('Loss', ':6.2f')\n cls_accs = AverageMeter('Cls Acc', ':3.1f')\n tgt_accs = AverageMeter('Tgt Acc', ':3.1f')\n domain_accs_D = AverageMeter('Domain Acc for D', ':3.1f')\n domain_accs_D_0 = AverageMeter('Domain Acc for D_0', ':3.1f')\n partial_classes_weights = AverageMeter('Partial Weight', ':3.2f')\n non_partial_classes_weights = AverageMeter('Non-Partial Weight', ':3.2f')\n\n progress = ProgressMeter(\n args.iters_per_epoch,\n [batch_time, data_time, losses, cls_accs, tgt_accs,\n domain_accs_D, domain_accs_D_0, partial_classes_weights, non_partial_classes_weights],\n prefix=\"Epoch: [{}]\".format(epoch))\n\n # switch to train mode\n model.train()\n domain_adv_D.train()\n domain_adv_D_0.train()\n\n end = time.time()\n for i in range(args.iters_per_epoch):\n x_s, labels_s = next(train_source_iter)\n x_t, labels_t = next(train_target_iter)\n\n x_s = x_s.to(device)\n x_t = x_t.to(device)\n labels_s = labels_s.to(device)\n labels_t = labels_t.to(device)\n\n # measure data loading time\n data_time.update(time.time() - end)\n\n # compute output\n x = torch.cat((x_s, x_t), dim=0)\n y, f = model(x)\n y_s, y_t = y.chunk(2, dim=0)\n f_s, f_t = f.chunk(2, dim=0)\n\n # classification loss\n cls_loss = F.cross_entropy(y_s, labels_s)\n\n # domain adversarial loss for D\n adv_loss_D = domain_adv_D(f_s.detach(), f_t.detach())\n\n # get importance weights\n w_s = importance_weight_module.get_importance_weight(f_s)\n # domain adversarial loss for D_0\n adv_loss_D_0 = domain_adv_D_0(f_s, f_t, w_s=w_s)\n\n # entropy loss\n y_t = F.softmax(y_t, dim=1)\n entropy_loss = entropy(y_t, reduction='mean')\n\n loss = cls_loss + 1.5 * args.trade_off * adv_loss_D + \\\n args.trade_off * adv_loss_D_0 + args.gamma * entropy_loss\n\n # compute gradient and do SGD step\n optimizer.zero_grad()\n loss.backward()\n optimizer.step()\n lr_scheduler.step()\n\n cls_acc = accuracy(y_s, labels_s)[0]\n tgt_acc = accuracy(y_t, labels_t)[0]\n\n losses.update(loss.item(), x_s.size(0))\n cls_accs.update(cls_acc.item(), x_s.size(0))\n tgt_accs.update(tgt_acc.item(), x_s.size(0))\n domain_accs_D.update(domain_adv_D.domain_discriminator_accuracy, x_s.size(0))\n domain_accs_D_0.update(domain_adv_D_0.domain_discriminator_accuracy, x_s.size(0))\n\n # debug: output class weight averaged on the partial classes and non-partial classes respectively\n partial_class_weight, non_partial_classes_weight = \\\n importance_weight_module.get_partial_classes_weight(w_s, labels_s)\n partial_classes_weights.update(partial_class_weight.item(), x_s.size(0))\n non_partial_classes_weights.update(non_partial_classes_weight.item(), x_s.size(0))\n\n batch_time.update(time.time() - end)\n end = time.time()\n\n if i % args.print_freq == 0:\n progress.display(i)\n\n\nif __name__ == '__main__':\n parser = argparse.ArgumentParser(description='IWAN for Partial Domain Adaptation')\n # dataset parameters\n parser.add_argument('root', metavar='DIR',\n help='root path of source (and target) dataset')\n parser.add_argument('-d', '--data', metavar='DATA', default='Office31', choices=utils.get_dataset_names(),\n help='dataset: ' + ' | '.join(utils.get_dataset_names()) +\n ' (default: Office31)')\n parser.add_argument('-s', '--source', help='source domain')\n parser.add_argument('-t', '--target', help='target domain')\n parser.add_argument('--train-resizing', type=str, default='default')\n parser.add_argument('--val-resizing', type=str, default='default')\n # model parameters\n parser.add_argument('-a', '--arch', metavar='ARCH', default='resnet18',\n choices=utils.get_model_names(),\n help='backbone architecture: ' +\n ' | '.join(utils.get_model_names()) +\n ' (default: resnet18)')\n parser.add_argument('--no-pool', action='store_true',\n help='no pool layer after the feature extractor.')\n parser.add_argument('--bottleneck-dim', default=256, type=int,\n help='Dimension of bottleneck')\n parser.add_argument('--gamma', default=0.1, type=float,\n help='the trade-off hyper-parameter for entropy loss(default: 0.1)')\n parser.add_argument('--trade-off', default=3, type=float,\n help='the trade-off hyper-parameter for transfer loss(default: 3))')\n # training parameters\n parser.add_argument('-b', '--batch-size', default=36, type=int,\n metavar='N',\n help='mini-batch size (default: 36)')\n parser.add_argument('--lr', '--learning-rate', default=0.001, type=float,\n metavar='LR', help='initial learning rate', dest='lr')\n parser.add_argument('--lr-gamma', default=0.001, type=float, help='parameter for lr scheduler')\n parser.add_argument('--lr-decay', default=0.75, type=float, help='parameter for lr scheduler')\n parser.add_argument('--momentum', default=0.9, type=float, metavar='M',\n help='momentum')\n parser.add_argument('--wd', '--weight-decay', default=1e-3, type=float,\n metavar='W', help='weight decay (default: 1e-3)',\n dest='weight_decay')\n parser.add_argument('-j', '--workers', default=2, type=int, metavar='N',\n help='number of data loading workers (default: 2)')\n parser.add_argument('--epochs', default=10, type=int, metavar='N',\n help='number of total epochs to run')\n parser.add_argument('-i', '--iters-per-epoch', default=1000, type=int,\n help='Number of iterations per epoch')\n parser.add_argument('-p', '--print-freq', default=100, type=int,\n metavar='N', help='print frequency (default: 100)')\n parser.add_argument('--seed', default=None, type=int,\n help='seed for initializing training. ')\n parser.add_argument('--per-class-eval', action='store_true',\n help='whether output per-class accuracy during evaluation')\n parser.add_argument(\"--log\", type=str, default='iwan',\n help=\"Where to save logs, checkpoints and debugging images.\")\n parser.add_argument(\"--phase\", type=str, default='train', choices=['train', 'test', 'analysis'],\n help=\"When phase is 'test', only test the model.\"\n \"When phase is 'analysis', only analysis the model.\")\n args = parser.parse_args()\n main(args)\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/examples/domain_adaptation/partial_domain_adaptation/iwan.sh",
"content": "#!/usr/bin/env bash\n# Office31\nCUDA_VISIBLE_DEVICES=0 python iwan.py data/office31 -d Office31 -s A -t W -a resnet50 --lr 0.0003 --seed 0 --log logs/iwan/Office31_A2W\nCUDA_VISIBLE_DEVICES=0 python iwan.py data/office31 -d Office31 -s D -t W -a resnet50 --lr 0.0003 --seed 0 --log logs/iwan/Office31_D2W\nCUDA_VISIBLE_DEVICES=0 python iwan.py data/office31 -d Office31 -s W -t D -a resnet50 --lr 0.0003 --seed 0 --log logs/iwan/Office31_W2D\nCUDA_VISIBLE_DEVICES=0 python iwan.py data/office31 -d Office31 -s A -t D -a resnet50 --lr 0.0003 --seed 0 --log logs/iwan/Office31_A2D\nCUDA_VISIBLE_DEVICES=0 python iwan.py data/office31 -d Office31 -s D -t A -a resnet50 --lr 0.0003 --seed 0 --log logs/iwan/Office31_D2A\nCUDA_VISIBLE_DEVICES=0 python iwan.py data/office31 -d Office31 -s W -t A -a resnet50 --lr 0.0003 --seed 0 --log logs/iwan/Office31_W2A\n\n# Office-Home\nCUDA_VISIBLE_DEVICES=0 python iwan.py data/office-home -d OfficeHome -s Ar -t Cl -a resnet50 --epochs 30 --seed 0 --log logs/iwan/OfficeHome_Ar2Cl\nCUDA_VISIBLE_DEVICES=0 python iwan.py data/office-home -d OfficeHome -s Ar -t Pr -a resnet50 --epochs 30 --seed 0 --log logs/iwan/OfficeHome_Ar2Pr\nCUDA_VISIBLE_DEVICES=0 python iwan.py data/office-home -d OfficeHome -s Ar -t Rw -a resnet50 --epochs 30 --seed 0 --log logs/iwan/OfficeHome_Ar2Rw\nCUDA_VISIBLE_DEVICES=0 python iwan.py data/office-home -d OfficeHome -s Cl -t Ar -a resnet50 --epochs 30 --seed 0 --log logs/iwan/OfficeHome_Cl2Ar\nCUDA_VISIBLE_DEVICES=0 python iwan.py data/office-home -d OfficeHome -s Cl -t Pr -a resnet50 --epochs 30 --seed 0 --log logs/iwan/OfficeHome_Cl2Pr\nCUDA_VISIBLE_DEVICES=0 python iwan.py data/office-home -d OfficeHome -s Cl -t Rw -a resnet50 --epochs 30 --seed 0 --log logs/iwan/OfficeHome_Cl2Rw\nCUDA_VISIBLE_DEVICES=0 python iwan.py data/office-home -d OfficeHome -s Pr -t Ar -a resnet50 --epochs 30 --seed 0 --log logs/iwan/OfficeHome_Pr2Ar\nCUDA_VISIBLE_DEVICES=0 python iwan.py data/office-home -d OfficeHome -s Pr -t Cl -a resnet50 --epochs 30 --seed 0 --log logs/iwan/OfficeHome_Pr2Cl\nCUDA_VISIBLE_DEVICES=0 python iwan.py data/office-home -d OfficeHome -s Pr -t Rw -a resnet50 --epochs 30 --seed 0 --log logs/iwan/OfficeHome_Pr2Rw\nCUDA_VISIBLE_DEVICES=0 python iwan.py data/office-home -d OfficeHome -s Rw -t Ar -a resnet50 --epochs 30 --seed 0 --log logs/iwan/OfficeHome_Rw2Ar\nCUDA_VISIBLE_DEVICES=0 python iwan.py data/office-home -d OfficeHome -s Rw -t Cl -a resnet50 --epochs 30 --seed 0 --log logs/iwan/OfficeHome_Rw2Cl\nCUDA_VISIBLE_DEVICES=0 python iwan.py data/office-home -d OfficeHome -s Rw -t Pr -a resnet50 --epochs 30 --seed 0 --log logs/iwan/OfficeHome_Rw2Pr\n\n# VisDA-2017\nCUDA_VISIBLE_DEVICES=0 python iwan.py data/visda-2017 -d VisDA2017 -s Synthetic -t Real -a resnet50 \\\n --lr 0.0003 --seed 0 --train-resizing cen.crop --per-class-eval --log logs/iwan/VisDA2017_S2R\n\n# ImageNet-Caltech\nCUDA_VISIBLE_DEVICES=0 python iwan.py data/ImageNetCaltech -d ImageNetCaltech -s I -t C -a resnet50 \\\n --seed 0 --log logs/iwan/I2C\nCUDA_VISIBLE_DEVICES=0 python iwan.py data/ImageNetCaltech -d CaltechImageNet -s C -t I -a resnet50 \\\n --seed 0 --bottleneck-dim 2048 --log logs/iwan/C2I\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/examples/domain_adaptation/partial_domain_adaptation/pada.py",
"content": "\"\"\"\n@author: Junguang Jiang\n@contact: JiangJunguang1123@outlook.com\n\"\"\"\nimport random\nimport time\nimport warnings\nimport sys\nimport argparse\nimport shutil\nimport os.path as osp\n\nimport torch\nimport torch.nn as nn\nimport torch.backends.cudnn as cudnn\nfrom torch.optim import SGD\nfrom torch.optim.lr_scheduler import LambdaLR\nfrom torch.utils.data import DataLoader\nimport torch.nn.functional as F\n\nsys.path.append('../../..')\nfrom dalib.modules.domain_discriminator import DomainDiscriminator\nfrom common.modules.classifier import Classifier\nfrom dalib.adaptation.dann import DomainAdversarialLoss, ImageClassifier\nfrom dalib.adaptation.pada import AutomaticUpdateClassWeightModule\nfrom common.utils.data import ForeverDataIterator\nfrom common.utils.metric import accuracy\nfrom common.utils.meter import AverageMeter, ProgressMeter\nfrom common.utils.logger import CompleteLogger\nfrom common.utils.analysis import collect_feature, tsne, a_distance\n\nsys.path.append('.')\nimport utils\n\ndevice = torch.device(\"cuda\" if torch.cuda.is_available() else \"cpu\")\n\n\ndef main(args: argparse.Namespace):\n logger = CompleteLogger(args.log, args.phase)\n print(args)\n\n if args.seed is not None:\n random.seed(args.seed)\n torch.manual_seed(args.seed)\n cudnn.deterministic = True\n warnings.warn('You have chosen to seed training. '\n 'This will turn on the CUDNN deterministic setting, '\n 'which can slow down your training considerably! '\n 'You may see unexpected behavior when restarting '\n 'from checkpoints.')\n\n cudnn.benchmark = True\n\n # Data loading code\n train_transform = utils.get_train_transform(args.train_resizing, random_horizontal_flip=True,\n random_color_jitter=False)\n val_transform = utils.get_val_transform(args.val_resizing)\n print(\"train_transform: \", train_transform)\n print(\"val_transform: \", val_transform)\n\n train_source_dataset, train_target_dataset, val_dataset, test_dataset, num_classes, args.class_names = \\\n utils.get_dataset(args.data, args.root, args.source, args.target, train_transform, val_transform)\n train_source_loader = DataLoader(train_source_dataset, batch_size=args.batch_size,\n shuffle=True, num_workers=args.workers, drop_last=True)\n train_target_loader = DataLoader(train_target_dataset, batch_size=args.batch_size,\n shuffle=True, num_workers=args.workers, drop_last=True)\n val_loader = DataLoader(val_dataset, batch_size=args.batch_size, shuffle=False, num_workers=args.workers)\n test_loader = DataLoader(test_dataset, batch_size=args.batch_size, shuffle=False, num_workers=args.workers)\n\n train_source_iter = ForeverDataIterator(train_source_loader)\n train_target_iter = ForeverDataIterator(train_target_loader)\n\n # create model\n print(\"=> using pre-trained model '{}'\".format(args.arch))\n backbone = utils.get_model(args.arch)\n pool_layer = nn.Identity() if args.no_pool else None\n if args.data == 'ImageNetCaltech':\n classifier = Classifier(backbone, num_classes, pool_layer=pool_layer, head=backbone.copy_head()).to(device)\n else:\n classifier = ImageClassifier(backbone, num_classes, args.bottleneck_dim, pool_layer=pool_layer).to(device)\n domain_discri = DomainDiscriminator(in_feature=classifier.features_dim, hidden_size=1024).to(device)\n class_weight_module = AutomaticUpdateClassWeightModule(args.class_weight_update_steps, train_target_loader,\n classifier, num_classes, device, args.temperature,\n train_target_dataset.partial_classes_idx)\n # define optimizer and lr scheduler\n optimizer = SGD(classifier.get_parameters() + domain_discri.get_parameters(),\n args.lr, momentum=args.momentum, weight_decay=args.weight_decay, nesterov=True)\n lr_scheduler = LambdaLR(optimizer, lambda x: args.lr * (1. + args.lr_gamma * float(x)) ** (-args.lr_decay))\n\n # define loss function\n domain_adv = DomainAdversarialLoss(domain_discri).to(device)\n\n # resume from the best checkpoint\n if args.phase != 'train':\n checkpoint = torch.load(logger.get_checkpoint_path('best'), map_location='cpu')\n classifier.load_state_dict(checkpoint)\n\n # analysis the model\n if args.phase == 'analysis':\n # extract features from both domains\n feature_extractor = nn.Sequential(classifier.backbone, classifier.pool_layer, classifier.bottleneck).to(device)\n source_feature = collect_feature(train_source_loader, feature_extractor, device)\n target_feature = collect_feature(train_target_loader, feature_extractor, device)\n # plot t-SNE\n tSNE_filename = osp.join(logger.visualize_directory, 'TSNE.png')\n tsne.visualize(source_feature, target_feature, tSNE_filename)\n print(\"Saving t-SNE to\", tSNE_filename)\n # calculate A-distance, which is a measure for distribution discrepancy\n A_distance = a_distance.calculate(source_feature, target_feature, device)\n print(\"A-distance =\", A_distance)\n return\n\n if args.phase == 'test':\n acc1 = utils.validate(test_loader, classifier, args, device)\n print(acc1)\n return\n\n # start training\n best_acc1 = 0.\n for epoch in range(args.epochs):\n # train for one epoch\n train(train_source_iter, train_target_iter, classifier, domain_adv, class_weight_module,\n optimizer, lr_scheduler, epoch, args)\n\n # evaluate on validation set\n acc1 = utils.validate(val_loader, classifier, args, device)\n\n # remember best acc@1 and save checkpoint\n torch.save(classifier.state_dict(), logger.get_checkpoint_path('latest'))\n if acc1 > best_acc1:\n shutil.copy(logger.get_checkpoint_path('latest'), logger.get_checkpoint_path('best'))\n best_acc1 = max(acc1, best_acc1)\n\n print(\"best_acc1 = {:3.1f}\".format(best_acc1))\n\n # evaluate on test set\n classifier.load_state_dict(torch.load(logger.get_checkpoint_path('best')))\n acc1 = utils.validate(test_loader, classifier, args, device)\n print(\"test_acc1 = {:3.1f}\".format(acc1))\n\n logger.close()\n\n\ndef train(train_source_iter: ForeverDataIterator, train_target_iter: ForeverDataIterator, model: ImageClassifier,\n domain_adv: DomainAdversarialLoss, class_weight_module: AutomaticUpdateClassWeightModule,\n optimizer: SGD, lr_scheduler: LambdaLR, epoch: int, args: argparse.Namespace):\n batch_time = AverageMeter('Time', ':5.2f')\n data_time = AverageMeter('Data', ':5.2f')\n losses = AverageMeter('Loss', ':6.2f')\n cls_accs = AverageMeter('Cls Acc', ':3.1f')\n domain_accs = AverageMeter('Domain Acc', ':3.1f')\n tgt_accs = AverageMeter('Tgt Acc', ':3.1f')\n partial_classes_weights = AverageMeter('Partial Weight', ':3.1f')\n non_partial_classes_weights = AverageMeter('Non-partial Weight', ':3.1f')\n\n progress = ProgressMeter(\n args.iters_per_epoch,\n [batch_time, data_time, losses, cls_accs, domain_accs, tgt_accs, partial_classes_weights, non_partial_classes_weights],\n prefix=\"Epoch: [{}]\".format(epoch))\n\n # switch to train mode\n model.train()\n domain_adv.train()\n\n end = time.time()\n for i in range(args.iters_per_epoch):\n x_s, labels_s = next(train_source_iter)\n x_t, labels_t = next(train_target_iter)\n\n x_s = x_s.to(device)\n x_t = x_t.to(device)\n labels_s = labels_s.to(device)\n labels_t = labels_t.to(device)\n\n # measure data loading time\n data_time.update(time.time() - end)\n\n # compute output\n x = torch.cat((x_s, x_t), dim=0)\n y, f = model(x)\n y_s, y_t = y.chunk(2, dim=0)\n f_s, f_t = f.chunk(2, dim=0)\n\n cls_loss = F.cross_entropy(y_s, labels_s, class_weight_module.get_class_weight_for_cross_entropy_loss())\n w_s, w_t = class_weight_module.get_class_weight_for_adversarial_loss(labels_s)\n transfer_loss = domain_adv(f_s, f_t, w_s, w_t)\n class_weight_module.step()\n partial_classes_weight, non_partial_classes_weight = class_weight_module.get_partial_classes_weight()\n domain_acc = domain_adv.domain_discriminator_accuracy\n loss = cls_loss + transfer_loss * args.trade_off\n\n cls_acc = accuracy(y_s, labels_s)[0]\n tgt_acc = accuracy(y_t, labels_t)[0]\n\n losses.update(loss.item(), x_s.size(0))\n cls_accs.update(cls_acc.item(), x_s.size(0))\n domain_accs.update(domain_acc.item(), x_s.size(0))\n tgt_accs.update(tgt_acc.item(), x_s.size(0))\n partial_classes_weights.update(partial_classes_weight.item(), x_s.size(0))\n non_partial_classes_weights.update(non_partial_classes_weight.item(), x_s.size(0))\n\n # compute gradient and do SGD step\n optimizer.zero_grad()\n loss.backward()\n optimizer.step()\n lr_scheduler.step()\n\n # measure elapsed time\n batch_time.update(time.time() - end)\n end = time.time()\n\n if i % args.print_freq == 0:\n progress.display(i)\n\n\nif __name__ == '__main__':\n parser = argparse.ArgumentParser(description='PADA for Partial Domain Adaptation')\n # dataset parameters\n parser.add_argument('root', metavar='DIR',\n help='root path of source (and target) dataset')\n parser.add_argument('-d', '--data', metavar='DATA', default='Office31', choices=utils.get_dataset_names(),\n help='dataset: ' + ' | '.join(utils.get_dataset_names()) +\n ' (default: Office31)')\n parser.add_argument('-s', '--source', help='source domain')\n parser.add_argument('-t', '--target', help='target domain')\n parser.add_argument('--train-resizing', type=str, default='default')\n parser.add_argument('--val-resizing', type=str, default='default')\n # model parameters\n parser.add_argument('-a', '--arch', metavar='ARCH', default='resnet18',\n choices=utils.get_model_names(),\n help='backbone architecture: ' +\n ' | '.join(utils.get_model_names()) +\n ' (default: resnet18)')\n parser.add_argument('--no-pool', action='store_true',\n help='no pool layer after the feature extractor.')\n parser.add_argument('--bottleneck-dim', default=256, type=int,\n help='Dimension of bottleneck')\n parser.add_argument('-u', '--class-weight-update-steps', default=500, type=int,\n help='Number of steps to update class weight once')\n parser.add_argument('--temperature', default=0.1, type=float,\n help='temperature for softmax when calculating class weight')\n parser.add_argument('--trade-off', default=1., type=float,\n help='the trade-off hyper-parameter for transfer loss')\n # training parameters\n parser.add_argument('-b', '--batch-size', default=36, type=int,\n metavar='N',\n help='mini-batch size (default: 36)')\n parser.add_argument('--lr', '--learning-rate', default=0.002, type=float,\n metavar='LR', help='initial learning rate', dest='lr')\n parser.add_argument('--lr-gamma', default=0.001, type=float, help='parameter for lr scheduler')\n parser.add_argument('--lr-decay', default=0.75, type=float, help='parameter for lr scheduler')\n parser.add_argument('--momentum', default=0.9, type=float, metavar='M',\n help='momentum')\n parser.add_argument('--wd', '--weight-decay',default=1e-3, type=float,\n metavar='W', help='weight decay (default: 1e-3)',\n dest='weight_decay')\n parser.add_argument('-j', '--workers', default=2, type=int, metavar='N',\n help='number of data loading workers (default: 4)')\n parser.add_argument('--epochs', default=20, type=int, metavar='N',\n help='number of total epochs to run')\n parser.add_argument('-i', '--iters-per-epoch', default=1000, type=int,\n help='Number of iterations per epoch')\n parser.add_argument('-p', '--print-freq', default=100, type=int,\n metavar='N', help='print frequency (default: 100)')\n parser.add_argument('--seed', default=None, type=int,\n help='seed for initializing training. ')\n parser.add_argument('--per-class-eval', action='store_true',\n help='whether output per-class accuracy during evaluation')\n parser.add_argument(\"--log\", type=str, default='pada',\n help=\"Where to save logs, checkpoints and debugging images.\")\n parser.add_argument(\"--phase\", type=str, default='train', choices=['train', 'test', 'analysis'],\n help=\"When phase is 'test', only test the model.\"\n \"When phase is 'analysis', only analysis the model.\")\n args = parser.parse_args()\n main(args)\n\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/examples/domain_adaptation/partial_domain_adaptation/pada.sh",
"content": "#!/usr/bin/env bash\n# Office31\nCUDA_VISIBLE_DEVICES=0 python pada.py data/office31 -d Office31 -s A -t W -a resnet50 --epochs 20 --seed 0 --log logs/pada/Office31_A2W\nCUDA_VISIBLE_DEVICES=0 python pada.py data/office31 -d Office31 -s D -t W -a resnet50 --epochs 20 --seed 0 --log logs/pada/Office31_D2W\nCUDA_VISIBLE_DEVICES=0 python pada.py data/office31 -d Office31 -s W -t D -a resnet50 --epochs 20 --seed 0 --log logs/pada/Office31_W2D\nCUDA_VISIBLE_DEVICES=0 python pada.py data/office31 -d Office31 -s A -t D -a resnet50 --epochs 20 --seed 0 --log logs/pada/Office31_A2D\nCUDA_VISIBLE_DEVICES=0 python pada.py data/office31 -d Office31 -s D -t A -a resnet50 --epochs 20 --seed 0 --log logs/pada/Office31_D2A\nCUDA_VISIBLE_DEVICES=0 python pada.py data/office31 -d Office31 -s W -t A -a resnet50 --epochs 20 --seed 0 --log logs/pada/Office31_W2A\n\n# Office-Home\nCUDA_VISIBLE_DEVICES=0 python pada.py data/office-home -d OfficeHome -s Ar -t Cl -a resnet50 --epochs 30 --temperature 0.01 --seed 0 --log logs/pada/OfficeHome_Ar2Cl\nCUDA_VISIBLE_DEVICES=0 python pada.py data/office-home -d OfficeHome -s Ar -t Pr -a resnet50 --epochs 30 --temperature 0.01 --seed 0 --log logs/pada/OfficeHome_Ar2Pr\nCUDA_VISIBLE_DEVICES=0 python pada.py data/office-home -d OfficeHome -s Ar -t Rw -a resnet50 --epochs 30 --temperature 0.01 --seed 0 --log logs/pada/OfficeHome_Ar2Rw\nCUDA_VISIBLE_DEVICES=0 python pada.py data/office-home -d OfficeHome -s Cl -t Ar -a resnet50 --epochs 30 --temperature 0.01 --seed 0 --log logs/pada/OfficeHome_Cl2Ar\nCUDA_VISIBLE_DEVICES=0 python pada.py data/office-home -d OfficeHome -s Cl -t Pr -a resnet50 --epochs 30 --temperature 0.01 --seed 0 --log logs/pada/OfficeHome_Cl2Pr\nCUDA_VISIBLE_DEVICES=0 python pada.py data/office-home -d OfficeHome -s Cl -t Rw -a resnet50 --epochs 30 --temperature 0.01 --seed 0 --log logs/pada/OfficeHome_Cl2Rw\nCUDA_VISIBLE_DEVICES=0 python pada.py data/office-home -d OfficeHome -s Pr -t Ar -a resnet50 --epochs 30 --temperature 0.01 --seed 0 --log logs/pada/OfficeHome_Pr2Ar\nCUDA_VISIBLE_DEVICES=0 python pada.py data/office-home -d OfficeHome -s Pr -t Cl -a resnet50 --epochs 30 --temperature 0.01 --seed 0 --log logs/pada/OfficeHome_Pr2Cl\nCUDA_VISIBLE_DEVICES=0 python pada.py data/office-home -d OfficeHome -s Pr -t Rw -a resnet50 --epochs 30 --temperature 0.01 --seed 0 --log logs/pada/OfficeHome_Pr2Rw\nCUDA_VISIBLE_DEVICES=0 python pada.py data/office-home -d OfficeHome -s Rw -t Ar -a resnet50 --epochs 30 --temperature 0.01 --seed 0 --log logs/pada/OfficeHome_Rw2Ar\nCUDA_VISIBLE_DEVICES=0 python pada.py data/office-home -d OfficeHome -s Rw -t Cl -a resnet50 --epochs 30 --temperature 0.01 --seed 0 --log logs/pada/OfficeHome_Rw2Cl\nCUDA_VISIBLE_DEVICES=0 python pada.py data/office-home -d OfficeHome -s Rw -t Pr -a resnet50 --epochs 30 --temperature 0.01 --seed 0 --log logs/pada/OfficeHome_Rw2Pr\n\n\n# VisDA-2017\nCUDA_VISIBLE_DEVICES=0 python pada.py data/visda-2017 -d VisDA2017 -s Synthetic -t Real -a resnet50 \\\n --epochs 20 --seed 0 -u 500 -i 500 --train-resizing cen.crop --trade-off 0.4 --per-class-eval --log logs/pada/VisDA2017_S2R\n\n# ImageNet-Caltech\nCUDA_VISIBLE_DEVICES=0 python pada.py data/ImageNetCaltech -d ImageNetCaltech -s I -t C -a resnet50 \\\n --epochs 20 --seed 0 --lr 0.003 --temperature 0.01 -u 2000 -i 2000 --log logs/pada/I2C\nCUDA_VISIBLE_DEVICES=0 python pada.py data/ImageNetCaltech -d CaltechImageNet -s C -t I -a resnet50 \\\n --epochs 20 --seed 0 --lr 0.003 --temperature 0.01 -u 2000 -i 2000 --bottleneck-dim 2048 --log logs/pada/C2I"
},
{
"path": "domain_generalization/Transfer-Learning-Library/examples/domain_adaptation/partial_domain_adaptation/requirements.txt",
"content": "timm"
},
{
"path": "domain_generalization/Transfer-Learning-Library/examples/domain_adaptation/partial_domain_adaptation/source_only.py",
"content": "\"\"\"\n@author: Junguang Jiang\n@contact: JiangJunguang1123@outlook.com\n\"\"\"\nimport random\nimport time\nimport warnings\nimport sys\nimport argparse\nimport shutil\nimport os.path as osp\n\nimport torch\nimport torch.nn as nn\nimport torch.backends.cudnn as cudnn\nfrom torch.optim import SGD\nfrom torch.optim.lr_scheduler import LambdaLR\nfrom torch.utils.data import DataLoader\nimport torch.nn.functional as F\n\nsys.path.append('../../..')\nfrom common.modules.classifier import Classifier\nfrom common.utils.data import ForeverDataIterator\nfrom common.utils.metric import accuracy\nfrom common.utils.meter import AverageMeter, ProgressMeter\nfrom common.utils.logger import CompleteLogger\nfrom common.utils.analysis import collect_feature, tsne, a_distance\n\nsys.path.append('.')\nimport utils\n\ndevice = torch.device(\"cuda\" if torch.cuda.is_available() else \"cpu\")\n\n\ndef main(args: argparse.Namespace):\n logger = CompleteLogger(args.log, args.phase)\n print(args)\n\n if args.seed is not None:\n random.seed(args.seed)\n torch.manual_seed(args.seed)\n cudnn.deterministic = True\n warnings.warn('You have chosen to seed training. '\n 'This will turn on the CUDNN deterministic setting, '\n 'which can slow down your training considerably! '\n 'You may see unexpected behavior when restarting '\n 'from checkpoints.')\n\n cudnn.benchmark = True\n\n # Data loading code\n train_transform = utils.get_train_transform(args.train_resizing, random_horizontal_flip=True, random_color_jitter=False)\n val_transform = utils.get_val_transform(args.val_resizing)\n print(\"train_transform: \", train_transform)\n print(\"val_transform: \", val_transform)\n\n train_source_dataset, _, val_dataset, test_dataset, num_classes, args.class_names = \\\n utils.get_dataset(args.data, args.root, args.source, args.target, train_transform, val_transform)\n train_source_loader = DataLoader(train_source_dataset, batch_size=args.batch_size,\n shuffle=True, num_workers=args.workers, drop_last=True)\n val_loader = DataLoader(val_dataset, batch_size=args.batch_size, shuffle=False, num_workers=args.workers)\n test_loader = DataLoader(test_dataset, batch_size=args.batch_size, shuffle=False, num_workers=args.workers)\n\n train_source_iter = ForeverDataIterator(train_source_loader)\n\n # create model\n print(\"=> using pre-trained model '{}'\".format(args.arch))\n backbone = utils.get_model(args.arch)\n pool_layer = nn.Identity() if args.no_pool else None\n head = backbone.copy_head() if args.data == 'ImageNetCaltech' else None\n classifier = Classifier(backbone, num_classes, pool_layer=pool_layer, head=head).to(device)\n\n # define optimizer and lr scheduler\n optimizer = SGD(classifier.get_parameters(), args.lr, momentum=args.momentum, weight_decay=args.wd, nesterov=True)\n lr_scheduler = LambdaLR(optimizer, lambda x: args.lr * (1. + args.lr_gamma * float(x)) ** (-args.lr_decay))\n\n # resume from the best checkpoint\n if args.phase != 'train':\n checkpoint = torch.load(logger.get_checkpoint_path('best'), map_location='cpu')\n classifier.load_state_dict(checkpoint)\n\n # analysis the model\n if args.phase == 'analysis':\n # using shuffled val loader\n val_loader = DataLoader(val_dataset, batch_size=args.batch_size, shuffle=True, num_workers=args.workers)\n # extract features from both domains\n feature_extractor = nn.Sequential(classifier.backbone, classifier.pool_layer, classifier.bottleneck).to(device)\n source_feature = collect_feature(train_source_loader, feature_extractor, device)\n target_feature = collect_feature(val_loader, feature_extractor, device)\n # plot t-SNE\n tSNE_filename = osp.join(logger.visualize_directory, 'TSNE.png')\n tsne.visualize(source_feature, target_feature, tSNE_filename)\n print(\"Saving t-SNE to\", tSNE_filename)\n # calculate A-distance, which is a measure for distribution discrepancy\n A_distance = a_distance.calculate(source_feature, target_feature, device)\n print(\"A-distance =\", A_distance)\n return\n\n if args.phase == 'test':\n acc1 = utils.validate(test_loader, classifier, args, device)\n print(acc1)\n return\n\n # start training\n best_acc1 = 0.\n for epoch in range(args.epochs):\n # train for one epoch\n train(train_source_iter, classifier, optimizer,\n lr_scheduler, epoch, args)\n\n # evaluate on validation set\n acc1 = utils.validate(val_loader, classifier, args, device)\n\n # remember best acc@1 and save checkpoint\n torch.save(classifier.state_dict(), logger.get_checkpoint_path('latest'))\n if acc1 > best_acc1:\n shutil.copy(logger.get_checkpoint_path('latest'), logger.get_checkpoint_path('best'))\n best_acc1 = max(acc1, best_acc1)\n\n print(\"best_acc1 = {:3.1f}\".format(best_acc1))\n\n # evaluate on test set\n classifier.load_state_dict(torch.load(logger.get_checkpoint_path('best')))\n acc1 = utils.validate(test_loader, classifier, args, device)\n print(\"test_acc1 = {:3.1f}\".format(acc1))\n\n logger.close()\n\n\ndef train(train_source_iter: ForeverDataIterator, model: Classifier, optimizer: SGD,\n lr_scheduler: LambdaLR, epoch: int, args: argparse.Namespace):\n batch_time = AverageMeter('Time', ':4.2f')\n data_time = AverageMeter('Data', ':3.1f')\n losses = AverageMeter('Loss', ':3.2f')\n cls_accs = AverageMeter('Cls Acc', ':3.1f')\n\n progress = ProgressMeter(\n args.iters_per_epoch,\n [batch_time, data_time, losses, cls_accs],\n prefix=\"Epoch: [{}]\".format(epoch))\n\n # switch to train mode\n model.train()\n\n end = time.time()\n for i in range(args.iters_per_epoch):\n x_s, labels_s = next(train_source_iter)\n x_s = x_s.to(device)\n labels_s = labels_s.to(device)\n\n # measure data loading time\n data_time.update(time.time() - end)\n\n # compute output\n y_s, f_s = model(x_s)\n\n cls_loss = F.cross_entropy(y_s, labels_s)\n loss = cls_loss\n\n cls_acc = accuracy(y_s, labels_s)[0]\n\n losses.update(loss.item(), x_s.size(0))\n cls_accs.update(cls_acc.item(), x_s.size(0))\n\n # compute gradient and do SGD step\n optimizer.zero_grad()\n loss.backward()\n optimizer.step()\n lr_scheduler.step()\n\n # measure elapsed time\n batch_time.update(time.time() - end)\n end = time.time()\n\n if i % args.print_freq == 0:\n progress.display(i)\n\n\nif __name__ == '__main__':\n parser = argparse.ArgumentParser(description='Source Only for Partial Domain Adaptation')\n # dataset parameters\n parser.add_argument('root', metavar='DIR',\n help='root path of dataset')\n parser.add_argument('-d', '--data', metavar='DATA', default='Office31', choices=utils.get_dataset_names(),\n help='dataset: ' + ' | '.join(utils.get_dataset_names()) +\n ' (default: Office31)')\n parser.add_argument('-s', '--source', help='source domain')\n parser.add_argument('-t', '--target', help='target domain')\n parser.add_argument('--train-resizing', type=str, default='default')\n parser.add_argument('--val-resizing', type=str, default='default')\n # model parameters\n parser.add_argument('-a', '--arch', metavar='ARCH', default='resnet18',\n choices=utils.get_model_names(),\n help='backbone architecture: ' +\n ' | '.join(utils.get_model_names()) +\n ' (default: resnet18)')\n parser.add_argument('--no-pool', action='store_true',\n help='no pool layer after the feature extractor.')\n # training parameters\n parser.add_argument('-b', '--batch-size', default=36, type=int,\n metavar='N',\n help='mini-batch size (default: 36)')\n parser.add_argument('--lr', '--learning-rate', default=0.001, type=float,\n metavar='LR', help='initial learning rate', dest='lr')\n parser.add_argument('--lr-gamma', default=0.0003, type=float, help='parameter for lr scheduler')\n parser.add_argument('--lr-decay', default=0.75, type=float, help='parameter for lr scheduler')\n parser.add_argument('--momentum', default=0.9, type=float, metavar='M',\n help='momentum')\n parser.add_argument('--wd', '--weight-decay', default=0.0005, type=float,\n metavar='W', help='weight decay (default: 5e-4)')\n parser.add_argument('-j', '--workers', default=2, type=int, metavar='N',\n help='number of data loading workers (default: 4)')\n parser.add_argument('--epochs', default=20, type=int, metavar='N',\n help='number of total epochs to run')\n parser.add_argument('-i', '--iters-per-epoch', default=500, type=int,\n help='Number of iterations per epoch')\n parser.add_argument('-p', '--print-freq', default=100, type=int,\n metavar='N', help='print frequency (default: 100)')\n parser.add_argument('--seed', default=None, type=int,\n help='seed for initializing training. ')\n parser.add_argument('--per-class-eval', action='store_true',\n help='whether output per-class accuracy during evaluation')\n parser.add_argument(\"--log\", type=str, default='src_only',\n help=\"Where to save logs, checkpoints and debugging images.\")\n parser.add_argument(\"--phase\", type=str, default='train', choices=['train', 'test', 'analysis'],\n help=\"When phase is 'test', only test the model.\"\n \"When phase is 'analysis', only analysis the model.\")\n args = parser.parse_args()\n main(args)\n\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/examples/domain_adaptation/partial_domain_adaptation/source_only.sh",
"content": "#!/usr/bin/env bash\n# Office31\nCUDA_VISIBLE_DEVICES=0 python source_only.py data/office31 -d Office31 -s A -t W -a resnet50 --epochs 5 --seed 0 --log logs/src_only/Office31_A2W\nCUDA_VISIBLE_DEVICES=0 python source_only.py data/office31 -d Office31 -s D -t W -a resnet50 --epochs 5 --seed 0 --log logs/src_only/Office31_D2W\nCUDA_VISIBLE_DEVICES=0 python source_only.py data/office31 -d Office31 -s W -t D -a resnet50 --epochs 5 --seed 0 --log logs/src_only/Office31_W2D\nCUDA_VISIBLE_DEVICES=0 python source_only.py data/office31 -d Office31 -s A -t D -a resnet50 --epochs 5 --seed 0 --log logs/src_only/Office31_A2D\nCUDA_VISIBLE_DEVICES=0 python source_only.py data/office31 -d Office31 -s D -t A -a resnet50 --epochs 5 --seed 0 --log logs/src_only/Office31_D2A\nCUDA_VISIBLE_DEVICES=0 python source_only.py data/office31 -d Office31 -s W -t A -a resnet50 --epochs 5 --seed 0 --log logs/src_only/Office31_W2A\n\n# Office-Home\nCUDA_VISIBLE_DEVICES=0 python source_only.py data/office-home -d OfficeHome -s Ar -t Cl -a resnet50 --epochs 5 -i 500 --seed 0 --log logs/src_only/OfficeHome_Ar2Cl\nCUDA_VISIBLE_DEVICES=0 python source_only.py data/office-home -d OfficeHome -s Ar -t Pr -a resnet50 --epochs 5 -i 500 --seed 0 --log logs/src_only/OfficeHome_Ar2Pr\nCUDA_VISIBLE_DEVICES=0 python source_only.py data/office-home -d OfficeHome -s Ar -t Rw -a resnet50 --epochs 5 -i 500 --seed 0 --log logs/src_only/OfficeHome_Ar2Rw\nCUDA_VISIBLE_DEVICES=0 python source_only.py data/office-home -d OfficeHome -s Cl -t Ar -a resnet50 --epochs 5 -i 500 --seed 0 --log logs/src_only/OfficeHome_Cl2Ar\nCUDA_VISIBLE_DEVICES=0 python source_only.py data/office-home -d OfficeHome -s Cl -t Pr -a resnet50 --epochs 5 -i 500 --seed 0 --log logs/src_only/OfficeHome_Cl2Pr\nCUDA_VISIBLE_DEVICES=0 python source_only.py data/office-home -d OfficeHome -s Cl -t Rw -a resnet50 --epochs 5 -i 500 --seed 0 --log logs/src_only/OfficeHome_Cl2Rw\nCUDA_VISIBLE_DEVICES=0 python source_only.py data/office-home -d OfficeHome -s Pr -t Ar -a resnet50 --epochs 5 -i 500 --seed 0 --log logs/src_only/OfficeHome_Pr2Ar\nCUDA_VISIBLE_DEVICES=0 python source_only.py data/office-home -d OfficeHome -s Pr -t Cl -a resnet50 --epochs 5 -i 500 --seed 0 --log logs/src_only/OfficeHome_Pr2Cl\nCUDA_VISIBLE_DEVICES=0 python source_only.py data/office-home -d OfficeHome -s Pr -t Rw -a resnet50 --epochs 5 -i 500 --seed 0 --log logs/src_only/OfficeHome_Pr2Rw\nCUDA_VISIBLE_DEVICES=0 python source_only.py data/office-home -d OfficeHome -s Rw -t Ar -a resnet50 --epochs 5 -i 500 --seed 0 --log logs/src_only/OfficeHome_Rw2Ar\nCUDA_VISIBLE_DEVICES=0 python source_only.py data/office-home -d OfficeHome -s Rw -t Cl -a resnet50 --epochs 5 -i 500 --seed 0 --log logs/src_only/OfficeHome_Rw2Cl\nCUDA_VISIBLE_DEVICES=0 python source_only.py data/office-home -d OfficeHome -s Rw -t Pr -a resnet50 --epochs 5 -i 500 --seed 0 --log logs/src_only/OfficeHome_Rw2Pr\n\n# VisDA-2017\nCUDA_VISIBLE_DEVICES=0 python source_only.py data/visda-2017 -d VisDA2017 -s Synthetic -t Real -a resnet50 \\\n --epochs 10 -i 500 --seed 0 --per-class-eval --log logs/src_only/VisDA2017_S2R\n\n# ImageNet-Caltech\nCUDA_VISIBLE_DEVICES=0 python source_only.py data/ImageNetCaltech -d ImageNetCaltech -s I -t C -a resnet50 \\\n --epochs 20 --seed 0 -i 2000 --log logs/src_only/I2C\nCUDA_VISIBLE_DEVICES=0 python source_only.py data/ImageNetCaltech -d CaltechImageNet -s C -t I -a resnet50 \\\n --epochs 20 --seed 0 -i 2000 --log logs/src_only/C2I\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/examples/domain_adaptation/partial_domain_adaptation/utils.py",
"content": "import sys\nimport time\nimport timm\nimport torch\nimport torch.nn as nn\nimport torch.nn.functional as F\nimport torchvision.transforms as T\n\nsys.path.append('../../..')\nimport common.vision.datasets.partial as datasets\nfrom common.vision.datasets.partial import default_partial as partial\nimport common.vision.models as models\nfrom common.vision.transforms import ResizeImage\nfrom common.utils.metric import accuracy, ConfusionMatrix\nfrom common.utils.meter import AverageMeter, ProgressMeter\n\n\ndef get_model_names():\n return sorted(\n name for name in models.__dict__\n if name.islower() and not name.startswith(\"__\")\n and callable(models.__dict__[name])\n ) + timm.list_models()\n\n\ndef get_model(model_name):\n if model_name in models.__dict__:\n # load models from common.vision.models\n backbone = models.__dict__[model_name](pretrained=True)\n else:\n # load models from pytorch-image-models\n backbone = timm.create_model(model_name, pretrained=True)\n try:\n backbone.out_features = backbone.get_classifier().in_features\n backbone.reset_classifier(0, '')\n backbone.copy_head = backbone.get_classifier\n except:\n backbone.out_features = backbone.head.in_features\n backbone.head = nn.Identity()\n backbone.copy_head = lambda x: x.head\n return backbone\n\n\ndef get_dataset_names():\n return sorted(\n name for name in datasets.__dict__\n if not name.startswith(\"__\") and callable(datasets.__dict__[name])\n )\n\n\ndef get_dataset(dataset_name, root, source, target, train_source_transform, val_transform, train_target_transform=None):\n if train_target_transform is None:\n train_target_transform = train_source_transform\n # load datasets from common.vision.datasets\n dataset = datasets.__dict__[dataset_name]\n partial_dataset = partial(dataset)\n\n train_source_dataset = dataset(root=root, task=source, download=True, transform=train_source_transform)\n train_target_dataset = partial_dataset(root=root, task=target, download=True, transform=train_target_transform)\n val_dataset = partial_dataset(root=root, task=target, download=True, transform=val_transform)\n if dataset_name == 'DomainNet':\n test_dataset = partial_dataset(root=root, task=target, split='test', download=True, transform=val_transform)\n else:\n test_dataset = val_dataset\n class_names = train_source_dataset.classes\n num_classes = len(class_names)\n return train_source_dataset, train_target_dataset, val_dataset, test_dataset, num_classes, class_names\n\n\ndef validate(val_loader, model, args, device) -> float:\n batch_time = AverageMeter('Time', ':6.3f')\n losses = AverageMeter('Loss', ':.4e')\n top1 = AverageMeter('Acc@1', ':6.2f')\n progress = ProgressMeter(\n len(val_loader),\n [batch_time, losses, top1],\n prefix='Test: ')\n\n # switch to evaluate mode\n model.eval()\n if args.per_class_eval:\n confmat = ConfusionMatrix(len(args.class_names))\n else:\n confmat = None\n\n with torch.no_grad():\n end = time.time()\n for i, (images, target) in enumerate(val_loader):\n images = images.to(device)\n target = target.to(device)\n\n # compute output\n output = model(images)\n loss = F.cross_entropy(output, target)\n\n # measure accuracy and record loss\n acc1, = accuracy(output, target, topk=(1,))\n if confmat:\n confmat.update(target, output.argmax(1))\n losses.update(loss.item(), images.size(0))\n top1.update(acc1.item(), images.size(0))\n\n # measure elapsed time\n batch_time.update(time.time() - end)\n end = time.time()\n\n if i % args.print_freq == 0:\n progress.display(i)\n\n print(' * Acc@1 {top1.avg:.3f}'.format(top1=top1))\n if confmat:\n print(confmat.format(args.class_names))\n\n return top1.avg\n\n\ndef get_train_transform(resizing='default', random_horizontal_flip=True, random_color_jitter=False):\n \"\"\"\n resizing mode:\n - default: resize the image to 256 and take a random resized crop of size 224;\n - cen.crop: resize the image to 256 and take the center crop of size 224;\n - res: resize the image to 224;\n - res.|crop: resize the image to 256 and take a random crop of size 224;\n - res.sma|crop: resize the image keeping its aspect ratio such that the\n smaller side is 256, then take a random crop of size 224;\n – inc.crop: “inception crop” from (Szegedy et al., 2015);\n – cif.crop: resize the image to 224, zero-pad it by 28 on each side, then take a random crop of size 224.\n \"\"\"\n if resizing == 'default':\n transform = T.Compose([\n ResizeImage(256),\n T.RandomResizedCrop(224)\n ])\n elif resizing == 'cen.crop':\n transform = T.Compose([\n ResizeImage(256),\n T.CenterCrop(224)\n ])\n elif resizing == 'res.':\n transform = T.Resize(224)\n elif resizing == 'res.|crop':\n transform = T.Compose([\n T.Resize((256, 256)),\n T.RandomCrop(224)\n ])\n elif resizing == \"res.sma|crop\":\n transform = T.Compose([\n T.Resize(256),\n T.RandomCrop(224)\n ])\n elif resizing == 'inc.crop':\n transform = T.RandomResizedCrop(224)\n elif resizing == 'cif.crop':\n transform = T.Compose([\n T.Resize((224, 224)),\n T.Pad(28),\n T.RandomCrop(224),\n ])\n else:\n raise NotImplementedError(resizing)\n transforms = [transform]\n if random_horizontal_flip:\n transforms.append(T.RandomHorizontalFlip())\n if random_color_jitter:\n transforms.append(T.ColorJitter(brightness=0.5, contrast=0.5, saturation=0.5, hue=0.5))\n transforms.extend([\n T.ToTensor(),\n T.Normalize(mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225])\n ])\n return T.Compose(transforms)\n\n\ndef get_val_transform(resizing='default'):\n \"\"\"\n resizing mode:\n - default: resize the image to 256 and take the center crop of size 224;\n – res.: resize the image to 224\n – res.|crop: resize the image such that the smaller side is of size 256 and\n then take a central crop of size 224.\n \"\"\"\n if resizing == 'default':\n transform = T.Compose([\n ResizeImage(256),\n T.CenterCrop(224),\n ])\n elif resizing == 'res.':\n transform = T.Resize((224, 224))\n elif resizing == 'res.|crop':\n transform = T.Compose([\n T.Resize(256),\n T.CenterCrop(224),\n ])\n else:\n raise NotImplementedError(resizing)\n return T.Compose([\n transform,\n T.ToTensor(),\n T.Normalize(mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225])\n ])\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/examples/domain_adaptation/re_identification/README.md",
"content": "# Domain Adaptation for Person Re-Identification\n\n## Installation\nExample scripts support all models in [PyTorch-Image-Models](https://github.com/rwightman/pytorch-image-models).\nYou also need to install timm to use PyTorch-Image-Models.\n\n```\npip install timm\n```\n\n## Dataset \nFollowing datasets can be downloaded automatically:\n\n- [Market1501](http://zheng-lab.cecs.anu.edu.au/Project/project_reid.html)\n- [DukeMTMC](https://exposing.ai/duke_mtmc/)\n- [MSMT17](https://arxiv.org/pdf/1711.08565.pdf)\n\n## Supported Methods\n\nSupported methods include:\n\n- [Two at Once: Enhancing Learning and Generalization Capacities via IBN-Net (IBN-Net, 2018 ECCV)](https://openaccess.thecvf.com/content_ECCV_2018/papers/Xingang_Pan_Two_at_Once_ECCV_2018_paper.pdf)\n- [Mutual Mean-Teaching: Pseudo Label Refinery for Unsupervised Domain Adaptation on Person Re-identification (MMT, 2020 ICLR)](https://arxiv.org/abs/2001.01526)\n- [Similarity Preserving Generative Adversarial Network (SPGAN, 2018 CVPR)](https://arxiv.org/pdf/1811.10551.pdf)\n\n## Experiment and Results\n\nThe shell files give the script to reproduce the [benchmarks](/docs/dalib/benchmarks/re_identification.rst) with specified hyper-parameters.\nFor example, if you want to reproduce MMT on Market1501 -> DukeMTMC task, use the following script\n\n```shell script\n# Train MMT on Market1501 -> DukeMTMC task using ResNet 50.\n# Assume you have put the datasets under the path `data/market1501` and `data/dukemtmc`, \n# or you are glad to download the datasets automatically from the Internet to this path\n\n# MMT involves two training steps:\n# step1: pretrain\nCUDA_VISIBLE_DEVICES=0 python baseline.py data -s Market1501 -t DukeMTMC -a reid_resnet50 \\\n--iters-per-epoch 800 --print-freq 80 --finetune --seed 0 --log logs/baseline/Market2DukeSeed0\nCUDA_VISIBLE_DEVICES=0 python baseline.py data -s Market1501 -t DukeMTMC -a reid_resnet50 \\\n--iters-per-epoch 800 --print-freq 80 --finetune --seed 1 --log logs/baseline/Market2DukeSeed1\n\n# step2: train mmt\nCUDA_VISIBLE_DEVICES=0,1,2,3 python mmt.py data -t DukeMTMC -a reid_resnet50 \\\n--pretrained-model-1-path logs/baseline/Market2DukeSeed0/checkpoints/best.pth \\\n--pretrained-model-2-path logs/baseline/Market2DukeSeed1/checkpoints/best.pth \\\n--finetune --seed 0 --log logs/mmt/Market2Duke\n```\n\nFor more information please refer to [Get Started](/docs/get_started/quickstart.rst) for help.\n\n## Citation\nIf you use these methods in your research, please consider citing.\n\n```\n@inproceedings{IBN-Net, \n author = {Xingang Pan, Ping Luo, Jianping Shi, and Xiaoou Tang}, \n title = {Two at Once: Enhancing Learning and Generalization Capacities via IBN-Net}, \n booktitle = {ECCV}, \n year = {2018} \n}\n\n@inproceedings{SPGAN,\n title={Image-image domain adaptation with preserved self-similarity and domain-dissimilarity for person re-identification},\n author={Deng, Weijian and Zheng, Liang and Ye, Qixiang and Kang, Guoliang and Yang, Yi and Jiao, Jianbin},\n booktitle={CVPR},\n year={2018}\n}\n\n@inproceedings{\n MMT,\n title={Mutual Mean-Teaching: Pseudo Label Refinery for Unsupervised Domain Adaptation on Person Re-identification},\n author={Yixiao Ge and Dapeng Chen and Hongsheng Li},\n booktitle={ICLR},\n year={2020},\n}\n```"
},
{
"path": "domain_generalization/Transfer-Learning-Library/examples/domain_adaptation/re_identification/baseline.py",
"content": "\"\"\"\n@author: Baixu Chen\n@contact: cbx_99_hasta@outlook.com\n\"\"\"\nimport random\nimport time\nimport warnings\nimport sys\nimport argparse\nimport shutil\nimport os.path as osp\n\nimport numpy as np\nimport torch\nimport torch.nn as nn\nfrom torch.nn import DataParallel\nimport torch.backends.cudnn as cudnn\nfrom torch.optim import Adam\nfrom torch.utils.data import DataLoader\n\nsys.path.append('../../..')\nfrom common.vision.models.reid.loss import CrossEntropyLossWithLabelSmooth, SoftTripletLoss\nfrom common.vision.models.reid.identifier import ReIdentifier\nimport common.vision.datasets.reid as datasets\nfrom common.vision.datasets.reid.convert import convert_to_pytorch_dataset\nfrom common.utils.scheduler import WarmupMultiStepLR\nfrom common.utils.metric.reid import validate, visualize_ranked_results\nfrom common.utils.data import ForeverDataIterator, RandomMultipleGallerySampler\nfrom common.utils.metric import accuracy\nfrom common.utils.meter import AverageMeter, ProgressMeter\nfrom common.utils.logger import CompleteLogger\n\ndevice = torch.device(\"cuda\" if torch.cuda.is_available() else \"cpu\")\n\nsys.path.append('.')\nimport utils\n\n\ndef main(args: argparse.Namespace):\n logger = CompleteLogger(args.log, args.phase)\n print(args)\n\n if args.seed is not None:\n random.seed(args.seed)\n np.random.seed(args.seed)\n torch.manual_seed(args.seed)\n cudnn.deterministic = True\n warnings.warn('You have chosen to seed training. '\n 'This will turn on the CUDNN deterministic setting, '\n 'which can slow down your training considerably! '\n 'You may see unexpected behavior when restarting '\n 'from checkpoints.')\n\n cudnn.benchmark = True\n\n # Data loading code\n train_transform = utils.get_train_transform(args.height, args.width, args.train_resizing,\n random_horizontal_flip=True, random_color_jitter=False,\n random_gray_scale=False, random_erasing=False)\n val_transform = utils.get_val_transform(args.height, args.width)\n print(\"train_transform: \", train_transform)\n print(\"val_transform: \", val_transform)\n\n working_dir = osp.dirname(osp.abspath(__file__))\n source_root = osp.join(working_dir, args.source_root)\n target_root = osp.join(working_dir, args.target_root)\n\n # source dataset\n source_dataset = datasets.__dict__[args.source](root=osp.join(source_root, args.source.lower()))\n sampler = RandomMultipleGallerySampler(source_dataset.train, args.num_instances)\n train_source_loader = DataLoader(\n convert_to_pytorch_dataset(source_dataset.train, root=source_dataset.images_dir, transform=train_transform),\n batch_size=args.batch_size, num_workers=args.workers, sampler=sampler, pin_memory=True, drop_last=True)\n train_source_iter = ForeverDataIterator(train_source_loader)\n val_loader = DataLoader(\n convert_to_pytorch_dataset(list(set(source_dataset.query) | set(source_dataset.gallery)),\n root=source_dataset.images_dir,\n transform=val_transform),\n batch_size=args.batch_size, num_workers=args.workers, shuffle=False, pin_memory=True)\n\n # target dataset\n target_dataset = datasets.__dict__[args.target](root=osp.join(target_root, args.target.lower()))\n train_target_loader = DataLoader(\n convert_to_pytorch_dataset(target_dataset.train, root=target_dataset.images_dir, transform=train_transform),\n batch_size=args.batch_size, num_workers=args.workers, shuffle=True, pin_memory=True, drop_last=True)\n train_target_iter = ForeverDataIterator(train_target_loader)\n test_loader = DataLoader(\n convert_to_pytorch_dataset(list(set(target_dataset.query) | set(target_dataset.gallery)),\n root=target_dataset.images_dir,\n transform=val_transform),\n batch_size=args.batch_size, num_workers=args.workers, shuffle=False, pin_memory=True)\n\n # create model\n num_classes = source_dataset.num_train_pids\n backbone = utils.get_model(args.arch)\n pool_layer = nn.Identity() if args.no_pool else None\n model = ReIdentifier(backbone, num_classes, finetune=args.finetune, pool_layer=pool_layer).to(device)\n model = DataParallel(model)\n\n # define optimizer and lr scheduler\n optimizer = Adam(model.module.get_parameters(base_lr=args.lr, rate=args.rate), args.lr,\n weight_decay=args.weight_decay)\n lr_scheduler = WarmupMultiStepLR(optimizer, args.milestones, gamma=0.1, warmup_factor=0.1,\n warmup_steps=args.warmup_steps)\n\n # resume from the best checkpoint\n if args.phase != 'train':\n checkpoint = torch.load(logger.get_checkpoint_path('best'), map_location='cpu')\n model.load_state_dict(checkpoint)\n\n # analysis the model\n if args.phase == 'analysis':\n # plot t-SNE\n utils.visualize_tsne(source_loader=val_loader, target_loader=test_loader, model=model,\n filename=osp.join(logger.visualize_directory, 'analysis', 'TSNE.pdf'), device=device)\n # visualize ranked results\n visualize_ranked_results(test_loader, model, target_dataset.query, target_dataset.gallery, device,\n visualize_dir=logger.visualize_directory, width=args.width, height=args.height,\n rerank=args.rerank)\n return\n\n if args.phase == 'test':\n print(\"Test on source domain:\")\n validate(val_loader, model, source_dataset.query, source_dataset.gallery, device, cmc_flag=True,\n rerank=args.rerank)\n print(\"Test on target domain:\")\n validate(test_loader, model, target_dataset.query, target_dataset.gallery, device, cmc_flag=True,\n rerank=args.rerank)\n return\n\n # define loss function\n criterion_ce = CrossEntropyLossWithLabelSmooth(num_classes).to(device)\n criterion_triplet = SoftTripletLoss(margin=args.margin).to(device)\n\n # start training\n best_val_mAP = 0.\n best_test_mAP = 0.\n for epoch in range(args.epochs):\n # print learning rate\n print(lr_scheduler.get_lr())\n\n # train for one epoch\n train(train_source_iter, train_target_iter, model, criterion_ce, criterion_triplet, optimizer, epoch, args)\n\n # update learning rate\n lr_scheduler.step()\n\n if (epoch + 1) % args.eval_step == 0 or (epoch == args.epochs - 1):\n\n # evaluate on validation set\n print(\"Validation on source domain...\")\n _, val_mAP = validate(val_loader, model, source_dataset.query, source_dataset.gallery, device,\n cmc_flag=True)\n\n # remember best mAP and save checkpoint\n torch.save(model.state_dict(), logger.get_checkpoint_path('latest'))\n if val_mAP > best_val_mAP:\n shutil.copy(logger.get_checkpoint_path('latest'), logger.get_checkpoint_path('best'))\n best_val_mAP = max(val_mAP, best_val_mAP)\n\n # evaluate on test set\n print(\"Test on target domain...\")\n _, test_mAP = validate(test_loader, model, target_dataset.query, target_dataset.gallery, device,\n cmc_flag=True, rerank=args.rerank)\n best_test_mAP = max(test_mAP, best_test_mAP)\n\n # evaluate on test set\n model.load_state_dict(torch.load(logger.get_checkpoint_path('best')))\n print(\"Test on target domain:\")\n _, test_mAP = validate(test_loader, model, target_dataset.query, target_dataset.gallery, device,\n cmc_flag=True, rerank=args.rerank)\n print(\"test mAP on target = {}\".format(test_mAP))\n print(\"oracle mAP on target = {}\".format(best_test_mAP))\n logger.close()\n\n\ndef train(train_source_iter: ForeverDataIterator, train_target_iter: ForeverDataIterator, model,\n criterion_ce: CrossEntropyLossWithLabelSmooth, criterion_triplet: SoftTripletLoss, optimizer: Adam,\n epoch: int, args: argparse.Namespace):\n batch_time = AverageMeter('Time', ':4.2f')\n data_time = AverageMeter('Data', ':3.1f')\n losses_ce = AverageMeter('CeLoss', ':3.2f')\n losses_triplet = AverageMeter('TripletLoss', ':3.2f')\n losses = AverageMeter('Loss', ':3.2f')\n cls_accs = AverageMeter('Cls Acc', ':3.1f')\n\n progress = ProgressMeter(\n args.iters_per_epoch,\n [batch_time, data_time, losses_ce, losses_triplet, losses, cls_accs],\n prefix=\"Epoch: [{}]\".format(epoch))\n\n # switch to train mode\n model.train()\n\n end = time.time()\n\n for i in range(args.iters_per_epoch):\n x_s, _, labels_s, _ = next(train_source_iter)\n x_t, _, _, _ = next(train_target_iter)\n\n x_s = x_s.to(device)\n x_t = x_t.to(device)\n labels_s = labels_s.to(device)\n\n # measure data loading time\n data_time.update(time.time() - end)\n\n # compute output\n y_s, f_s = model(x_s)\n y_t, f_t = model(x_t)\n\n # cross entropy loss\n loss_ce = criterion_ce(y_s, labels_s)\n # triplet loss\n loss_triplet = criterion_triplet(f_s, f_s, labels_s)\n loss = loss_ce + loss_triplet * args.trade_off\n\n cls_acc = accuracy(y_s, labels_s)[0]\n losses_ce.update(loss_ce.item(), x_s.size(0))\n losses_triplet.update(loss_triplet.item(), x_s.size(0))\n losses.update(loss.item(), x_s.size(0))\n cls_accs.update(cls_acc.item(), x_s.size(0))\n\n # compute gradient and do SGD step\n optimizer.zero_grad()\n loss.backward()\n optimizer.step()\n\n # measure elapsed time\n batch_time.update(time.time() - end)\n end = time.time()\n\n if i % args.print_freq == 0:\n progress.display(i)\n\n\nif __name__ == '__main__':\n dataset_names = sorted(\n name for name in datasets.__dict__\n if not name.startswith(\"__\") and callable(datasets.__dict__[name])\n )\n parser = argparse.ArgumentParser(description=\"Baseline for Domain Adaptative ReID\")\n # dataset parameters\n parser.add_argument('source_root', help='root path of the source dataset')\n parser.add_argument('target_root', help='root path of the target dataset')\n parser.add_argument('-s', '--source', type=str, help='source domain')\n parser.add_argument('-t', '--target', type=str, help='target domain')\n parser.add_argument('--train-resizing', type=str, default='default')\n # model parameters\n parser.add_argument('-a', '--arch', metavar='ARCH', default='reid_resnet50',\n choices=utils.get_model_names(),\n help='backbone architecture: ' +\n ' | '.join(utils.get_model_names()) +\n ' (default: reid_resnet50)')\n parser.add_argument('--no-pool', action='store_true', help='no pool layer after the feature extractor.')\n parser.add_argument('--finetune', action='store_true', help='whether use 10x smaller lr for backbone')\n parser.add_argument('--rate', type=float, default=0.2)\n # training parameters\n parser.add_argument('--trade-off', type=float, default=1,\n help='trade-off hyper parameter between cross entropy loss and triplet loss')\n parser.add_argument('--margin', type=float, default=0.0, help='margin for the triplet loss with batch hard')\n parser.add_argument('-j', '--workers', type=int, default=4)\n parser.add_argument('-b', '--batch-size', type=int, default=16)\n parser.add_argument('--height', type=int, default=256, help=\"input height\")\n parser.add_argument('--width', type=int, default=128, help=\"input width\")\n parser.add_argument('--num-instances', type=int, default=4,\n help=\"each minibatch consist of \"\n \"(batch_size // num_instances) identities, and \"\n \"each identity has num_instances instances, \"\n \"default: 4\")\n parser.add_argument('--lr', type=float, default=0.00035,\n help=\"initial learning rate\")\n parser.add_argument('--weight-decay', type=float, default=5e-4)\n parser.add_argument('--epochs', type=int, default=80)\n parser.add_argument('--warmup-steps', type=int, default=10, help='number of warm-up steps')\n parser.add_argument('--milestones', nargs='+', type=int, default=[40, 70],\n help='milestones for the learning rate decay')\n parser.add_argument('--eval-step', type=int, default=40)\n parser.add_argument('--iters-per-epoch', type=int, default=400)\n parser.add_argument('--print-freq', type=int, default=40)\n parser.add_argument('--seed', default=None, type=int, help='seed for initializing training.')\n parser.add_argument('--rerank', action='store_true', help=\"evaluation only\")\n parser.add_argument(\"--log\", type=str, default='baseline',\n help=\"Where to save logs, checkpoints and debugging images.\")\n parser.add_argument(\"--phase\", type=str, default='train', choices=['train', 'test', 'analysis'],\n help=\"When phase is 'test', only test the model.\"\n \"When phase is 'analysis', only analysis the model.\")\n args = parser.parse_args()\n main(args)\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/examples/domain_adaptation/re_identification/baseline.sh",
"content": "#!/usr/bin/env bash\n# Market1501 -> Duke\nCUDA_VISIBLE_DEVICES=0 python baseline.py data data -s Market1501 -t DukeMTMC -a reid_resnet50 \\\n--iters-per-epoch 800 --print-freq 80 --finetune --seed 0 --log logs/baseline/Market2Duke\n\n# Duke -> Market1501\nCUDA_VISIBLE_DEVICES=0 python baseline.py data data -s DukeMTMC -t Market1501 -a reid_resnet50 \\\n--iters-per-epoch 800 --print-freq 80 --finetune --seed 0 --log logs/baseline/Duke2Market\n\n# Market1501 -> MSMT\nCUDA_VISIBLE_DEVICES=0 python baseline.py data data -s Market1501 -t MSMT17 -a reid_resnet50 \\\n--iters-per-epoch 800 --print-freq 80 --finetune --seed 0 --log logs/baseline/Market2MSMT\n\n# MSMT -> Market1501\nCUDA_VISIBLE_DEVICES=0 python baseline.py data data -s MSMT17 -t Market1501 -a reid_resnet50 \\\n--iters-per-epoch 800 --print-freq 80 --finetune --seed 0 --log logs/baseline/MSMT2Market\n\n# Duke -> MSMT\nCUDA_VISIBLE_DEVICES=0 python baseline.py data data -s DukeMTMC -t MSMT17 -a reid_resnet50 \\\n--iters-per-epoch 800 --print-freq 80 --finetune --seed 0 --log logs/baseline/Duke2MSMT\n\n# MSMT -> Duke\nCUDA_VISIBLE_DEVICES=0 python baseline.py data data -s MSMT17 -t DukeMTMC -a reid_resnet50 \\\n--iters-per-epoch 800 --print-freq 80 --finetune --seed 0 --log logs/baseline/MSMT2Duke\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/examples/domain_adaptation/re_identification/baseline_cluster.py",
"content": "\"\"\"\n@author: Baixu Chen\n@contact: cbx_99_hasta@outlook.com\n\"\"\"\nimport random\nimport time\nimport warnings\nimport sys\nimport argparse\nimport shutil\nimport os.path as osp\n\nimport numpy as np\nimport torch\nimport torch.nn as nn\nimport torch.nn.functional as F\nfrom torch.nn import DataParallel\nimport torch.backends.cudnn as cudnn\nfrom torch.optim import Adam\nfrom torch.utils.data import DataLoader\nfrom sklearn.cluster import KMeans, DBSCAN\n\nsys.path.append('../../..')\nimport common.vision.datasets.reid as datasets\nfrom common.vision.datasets.reid.convert import convert_to_pytorch_dataset\nfrom common.vision.models.reid.identifier import ReIdentifier\nfrom common.vision.models.reid.loss import CrossEntropyLossWithLabelSmooth, SoftTripletLoss\nfrom common.utils.metric.reid import extract_reid_feature, validate, visualize_ranked_results\nfrom common.utils.data import ForeverDataIterator, RandomMultipleGallerySampler\nfrom common.utils.metric import accuracy\nfrom common.utils.meter import AverageMeter, ProgressMeter\nfrom common.utils.logger import CompleteLogger\n\nsys.path.append('.')\nimport utils\n\ndevice = torch.device(\"cuda\" if torch.cuda.is_available() else \"cpu\")\n\n\ndef main(args: argparse.Namespace):\n logger = CompleteLogger(args.log, args.phase)\n print(args)\n\n if args.seed is not None:\n random.seed(args.seed)\n np.random.seed(args.seed)\n torch.manual_seed(args.seed)\n cudnn.deterministic = True\n warnings.warn('You have chosen to seed training. '\n 'This will turn on the CUDNN deterministic setting, '\n 'which can slow down your training considerably! '\n 'You may see unexpected behavior when restarting '\n 'from checkpoints.')\n\n cudnn.benchmark = True\n\n # Data loading code\n train_transform = utils.get_train_transform(args.height, args.width, args.train_resizing,\n random_horizontal_flip=True, random_color_jitter=False,\n random_gray_scale=False, random_erasing=True)\n val_transform = utils.get_val_transform(args.height, args.width)\n print(\"train_transform: \", train_transform)\n print(\"val_transform: \", val_transform)\n\n working_dir = osp.dirname(osp.abspath(__file__))\n source_root = osp.join(working_dir, args.source_root)\n target_root = osp.join(working_dir, args.target_root)\n\n # source dataset\n source_dataset = datasets.__dict__[args.source](root=osp.join(source_root, args.source.lower()))\n val_loader = DataLoader(\n convert_to_pytorch_dataset(list(set(source_dataset.query) | set(source_dataset.gallery)),\n root=source_dataset.images_dir,\n transform=val_transform),\n batch_size=args.batch_size, num_workers=args.workers, shuffle=False, pin_memory=True)\n\n # target dataset\n target_dataset = datasets.__dict__[args.target](root=osp.join(target_root, args.target.lower()))\n cluster_loader = DataLoader(\n convert_to_pytorch_dataset(target_dataset.train, root=target_dataset.images_dir, transform=val_transform),\n batch_size=args.batch_size, num_workers=args.workers, shuffle=False, pin_memory=True)\n test_loader = DataLoader(\n convert_to_pytorch_dataset(list(set(target_dataset.query) | set(target_dataset.gallery)),\n root=target_dataset.images_dir,\n transform=val_transform),\n batch_size=args.batch_size, num_workers=args.workers, shuffle=False, pin_memory=True)\n\n # create model\n num_classes = args.num_clusters\n backbone = utils.get_model(args.arch)\n pool_layer = nn.Identity() if args.no_pool else None\n model = ReIdentifier(backbone, num_classes, finetune=args.finetune, pool_layer=pool_layer).to(device)\n model = DataParallel(model)\n\n # load pretrained weights\n pretrained_model = torch.load(args.pretrained_model_path)\n utils.copy_state_dict(model, pretrained_model)\n\n # resume from the best checkpoint\n if args.phase != 'train':\n checkpoint = torch.load(logger.get_checkpoint_path('best'), map_location='cpu')\n utils.copy_state_dict(model, checkpoint['model'])\n\n # analysis the model\n if args.phase == 'analysis':\n # plot t-SNE\n utils.visualize_tsne(source_loader=val_loader, target_loader=test_loader, model=model,\n filename=osp.join(logger.visualize_directory, 'analysis', 'TSNE.pdf'), device=device)\n # visualize ranked results\n visualize_ranked_results(test_loader, model, target_dataset.query, target_dataset.gallery, device,\n visualize_dir=logger.visualize_directory, width=args.width, height=args.height,\n rerank=args.rerank)\n return\n\n if args.phase == 'test':\n print(\"Test on Source domain:\")\n validate(val_loader, model, source_dataset.query, source_dataset.gallery, device, cmc_flag=True,\n rerank=args.rerank)\n print(\"Test on target domain:\")\n validate(test_loader, model, target_dataset.query, target_dataset.gallery, device, cmc_flag=True,\n rerank=args.rerank)\n return\n\n # define loss function\n criterion_ce = CrossEntropyLossWithLabelSmooth(num_classes).to(device)\n criterion_triplet = SoftTripletLoss(margin=args.margin).to(device)\n\n # optionally resume from a checkpoint\n if args.resume:\n checkpoint = torch.load(args.resume, map_location='cpu')\n utils.copy_state_dict(model, checkpoint['model'])\n args.start_epoch = checkpoint['epoch'] + 1\n\n # start training\n best_test_mAP = 0.\n for epoch in range(args.start_epoch, args.epochs):\n # run clustering algorithm and generate pseudo labels\n if args.clustering_algorithm == 'kmeans':\n train_target_iter = run_kmeans(cluster_loader, model, target_dataset, train_transform, args)\n elif args.clustering_algorithm == 'dbscan':\n train_target_iter, num_classes = run_dbscan(cluster_loader, model, target_dataset, train_transform, args)\n\n # define cross entropy loss with current number of classes\n criterion_ce = CrossEntropyLossWithLabelSmooth(num_classes).to(device)\n\n # define optimizer\n optimizer = Adam(model.module.get_parameters(base_lr=args.lr, rate=args.rate), args.lr,\n weight_decay=args.weight_decay)\n\n # train for one epoch\n train(train_target_iter, model, optimizer, criterion_ce, criterion_triplet, epoch, args)\n\n if (epoch + 1) % args.eval_step == 0 or (epoch == args.epochs - 1):\n # remember best mAP and save checkpoint\n torch.save(\n {\n 'model': model.state_dict(),\n 'epoch': epoch\n }, logger.get_checkpoint_path(epoch)\n )\n print(\"Test on target domain...\")\n _, test_mAP = validate(test_loader, model, target_dataset.query, target_dataset.gallery, device,\n cmc_flag=True, rerank=args.rerank)\n if test_mAP > best_test_mAP:\n shutil.copy(logger.get_checkpoint_path(epoch), logger.get_checkpoint_path('best'))\n best_test_mAP = max(test_mAP, best_test_mAP)\n\n print(\"best mAP on target = {}\".format(best_test_mAP))\n logger.close()\n\n\ndef run_kmeans(cluster_loader: DataLoader, model: DataParallel, target_dataset, train_transform,\n args: argparse.Namespace):\n # run kmeans clustering algorithm\n print('Clustering into {} classes'.format(args.num_clusters))\n feature_dict = extract_reid_feature(cluster_loader, model, device, normalize=True)\n feature = torch.stack(list(feature_dict.values())).cpu().numpy()\n km = KMeans(n_clusters=args.num_clusters, random_state=args.seed).fit(feature)\n cluster_labels = km.labels_\n cluster_centers = km.cluster_centers_\n print('Clustering finished')\n\n # normalize cluster centers and convert to pytorch tensor\n cluster_centers = torch.from_numpy(cluster_centers).float().to(device)\n cluster_centers = F.normalize(cluster_centers, dim=1)\n # reinitialize classifier head\n model.module.head.weight.data.copy_(cluster_centers)\n\n # generate training set with pseudo labels\n target_train_set = []\n for (fname, _, cid), label in zip(target_dataset.train, cluster_labels):\n target_train_set.append((fname, int(label), cid))\n\n sampler = RandomMultipleGallerySampler(target_train_set, args.num_instances)\n train_target_loader = DataLoader(\n convert_to_pytorch_dataset(target_train_set, root=target_dataset.images_dir, transform=train_transform),\n batch_size=args.batch_size, num_workers=args.workers, sampler=sampler, pin_memory=True, drop_last=True)\n train_target_iter = ForeverDataIterator(train_target_loader)\n\n return train_target_iter\n\n\ndef run_dbscan(cluster_loader: DataLoader, model: DataParallel, target_dataset, train_transform,\n args: argparse.Namespace):\n # run dbscan clustering algorithm\n feature_dict = extract_reid_feature(cluster_loader, model, device, normalize=True)\n feature = torch.stack(list(feature_dict.values())).cpu()\n rerank_dist = utils.compute_rerank_dist(feature).numpy()\n\n print('Clustering with dbscan algorithm')\n dbscan = DBSCAN(eps=0.6, min_samples=4, metric='precomputed', n_jobs=-1)\n cluster_labels = dbscan.fit_predict(rerank_dist)\n print('Clustering finished')\n\n # generate training set with pseudo labels and calculate cluster centers\n target_train_set = []\n cluster_centers = {}\n for i, ((fname, _, cid), label) in enumerate(zip(target_dataset.train, cluster_labels)):\n if label == -1:\n continue\n target_train_set.append((fname, label, cid))\n\n if label not in cluster_centers:\n cluster_centers[label] = []\n cluster_centers[label].append(feature[i])\n\n cluster_centers = [torch.stack(cluster_centers[idx]).mean(0) for idx in sorted(cluster_centers.keys())]\n cluster_centers = torch.stack(cluster_centers)\n # normalize cluster centers\n cluster_centers = F.normalize(cluster_centers, dim=1).float().to(device)\n\n # reinitialize classifier head\n features_dim = model.module.features_dim\n num_clusters = len(set(cluster_labels)) - (1 if -1 in cluster_labels else 0)\n model.module.head = nn.Linear(features_dim, num_clusters, bias=False).to(device)\n model.module.head.weight.data.copy_(cluster_centers)\n\n sampler = RandomMultipleGallerySampler(target_train_set, args.num_instances)\n train_target_loader = DataLoader(\n convert_to_pytorch_dataset(target_train_set, root=target_dataset.images_dir, transform=train_transform),\n batch_size=args.batch_size, num_workers=args.workers, sampler=sampler, pin_memory=True, drop_last=True)\n train_target_iter = ForeverDataIterator(train_target_loader)\n\n return train_target_iter, num_clusters\n\n\ndef train(train_target_iter: ForeverDataIterator, model, optimizer, criterion_ce: CrossEntropyLossWithLabelSmooth,\n criterion_triplet: SoftTripletLoss, epoch: int, args: argparse.Namespace):\n # train with pseudo labels\n batch_time = AverageMeter('Time', ':4.2f')\n data_time = AverageMeter('Data', ':3.1f')\n losses_ce = AverageMeter('CeLoss', ':3.2f')\n losses_triplet = AverageMeter('TripletLoss', ':3.2f')\n losses = AverageMeter('Loss', ':3.2f')\n cls_accs = AverageMeter('Cls Acc', ':3.1f')\n\n progress = ProgressMeter(\n args.iters_per_epoch,\n [batch_time, data_time, losses_ce, losses_triplet, losses, cls_accs],\n prefix=\"Epoch: [{}]\".format(epoch))\n\n # switch to train mode\n model.train()\n\n end = time.time()\n\n for i in range(args.iters_per_epoch):\n x_t, _, labels_t, _ = next(train_target_iter)\n\n x_t = x_t.to(device)\n labels_t = labels_t.to(device)\n\n # measure data loading time\n data_time.update(time.time() - end)\n\n # compute output\n y_t, f_t = model(x_t)\n\n # cross entropy loss\n loss_ce = criterion_ce(y_t, labels_t)\n # triplet loss\n loss_triplet = criterion_triplet(f_t, f_t, labels_t)\n loss = loss_ce + loss_triplet * args.trade_off\n\n cls_acc = accuracy(y_t, labels_t)[0]\n losses_ce.update(loss_ce.item(), x_t.size(0))\n losses_triplet.update(loss_triplet.item(), x_t.size(0))\n losses.update(loss.item(), x_t.size(0))\n cls_accs.update(cls_acc.item(), x_t.size(0))\n\n # compute gradient and do SGD step\n optimizer.zero_grad()\n loss.backward()\n optimizer.step()\n\n # measure elapsed time\n batch_time.update(time.time() - end)\n end = time.time()\n\n if i % args.print_freq == 0:\n progress.display(i)\n\n\nif __name__ == '__main__':\n dataset_names = sorted(\n name for name in datasets.__dict__\n if not name.startswith(\"__\") and callable(datasets.__dict__[name])\n )\n parser = argparse.ArgumentParser(description=\"Cluster Baseline for Domain Adaptative ReID\")\n # dataset parameters\n parser.add_argument('source_root', help='root path of the source dataset')\n parser.add_argument('target_root', help='root path of the target dataset')\n parser.add_argument('-s', '--source', type=str, help='source domain')\n parser.add_argument('-t', '--target', type=str, help='target domain')\n parser.add_argument('--train-resizing', type=str, default='default')\n # model parameters\n parser.add_argument('-a', '--arch', metavar='ARCH', default='reid_resnet50',\n choices=utils.get_model_names(),\n help='backbone architecture: ' +\n ' | '.join(utils.get_model_names()) +\n ' (default: reid_resnet50)')\n parser.add_argument('--num-clusters', type=int, default=500)\n parser.add_argument('--no-pool', action='store_true', help='no pool layer after the feature extractor.')\n parser.add_argument('--finetune', action='store_true', help='whether use 10x smaller lr for backbone')\n parser.add_argument('--rate', type=float, default=0.2)\n # training parameters\n parser.add_argument('--clustering-algorithm', type=str, default='dbscan', choices=['kmeans', 'dbscan'],\n help='clustering algorithm to run, currently supported method: [\"kmeans\", \"dbscan\"]')\n parser.add_argument('--resume', type=str, default=None,\n help=\"Where restore model parameters from.\")\n parser.add_argument('--pretrained-model-path', type=str, help='path to pretrained (source-only) model')\n parser.add_argument('--trade-off', type=float, default=1,\n help='trade-off hyper parameter between cross entropy loss and triplet loss')\n parser.add_argument('--margin', type=float, default=0.0, help='margin for the triplet loss with batch hard')\n parser.add_argument('-j', '--workers', type=int, default=4)\n parser.add_argument('-b', '--batch-size', type=int, default=64)\n parser.add_argument('--height', type=int, default=256, help=\"input height\")\n parser.add_argument('--width', type=int, default=128, help=\"input width\")\n parser.add_argument('--num-instances', type=int, default=4,\n help=\"each minibatch consist of \"\n \"(batch_size // num_instances) identities, and \"\n \"each identity has num_instances instances, \"\n \"default: 4\")\n parser.add_argument('--lr', type=float, default=0.00035,\n help=\"learning rate\")\n parser.add_argument('--weight-decay', type=float, default=5e-4)\n parser.add_argument('--epochs', type=int, default=40)\n parser.add_argument('--start-epoch', default=0, type=int, help='start epoch')\n parser.add_argument('--eval-step', type=int, default=1)\n parser.add_argument('--iters-per-epoch', type=int, default=400)\n parser.add_argument('--print-freq', type=int, default=40)\n parser.add_argument('--seed', default=None, type=int, help='seed for initializing training.')\n parser.add_argument('--rerank', action='store_true', help=\"evaluation only\")\n parser.add_argument(\"--log\", type=str, default='baseline_cluster',\n help=\"Where to save logs, checkpoints and debugging images.\")\n parser.add_argument(\"--phase\", type=str, default='train', choices=['train', 'test', 'analysis'],\n help=\"When phase is 'test', only test the model.\"\n \"When phase is 'analysis', only analysis the model.\")\n args = parser.parse_args()\n main(args)\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/examples/domain_adaptation/re_identification/baseline_cluster.sh",
"content": "#!/usr/bin/env bash\n# Market1501 -> Duke\n# step1: pretrain\nCUDA_VISIBLE_DEVICES=0 python baseline.py data data -s Market1501 -t DukeMTMC -a reid_resnet50 \\\n--iters-per-epoch 800 --print-freq 80 --finetune --seed 0 --log logs/baseline/Market2Duke\n# step2: train with pseudo labels assigned by cluster algorithm\nCUDA_VISIBLE_DEVICES=0,1,2,3 python baseline_cluster.py data data -s Market1501 -t DukeMTMC -a reid_resnet50 \\\n--pretrained-model-path logs/baseline/Market2Duke/checkpoints/best.pth \\\n--finetune --seed 0 --log logs/baseline_cluster/Market2Duke\n\n# Duke -> Market1501\n# step1: pretrain\nCUDA_VISIBLE_DEVICES=0 python baseline.py data data -s DukeMTMC -t Market1501 -a reid_resnet50 \\\n--iters-per-epoch 800 --print-freq 80 --finetune --seed 0 --log logs/baseline/Duke2Market\n# step2: train with pseudo labels assigned by cluster algorithm\nCUDA_VISIBLE_DEVICES=0,1,2,3 python baseline_cluster.py data data -s DukeMTMC -t Market1501 -a reid_resnet50 \\\n--pretrained-model-path logs/baseline/Duke2Market/checkpoints/best.pth \\\n--finetune --seed 0 --log logs/baseline_cluster/Duke2Market\n\n# Market1501 -> MSMT\n# step1: pretrain\nCUDA_VISIBLE_DEVICES=0 python baseline.py data data -s Market1501 -t MSMT17 -a reid_resnet50 \\\n--iters-per-epoch 800 --print-freq 80 --finetune --seed 0 --log logs/baseline/Market2MSMT\n# step2: train with pseudo labels assigned by cluster algorithm\nCUDA_VISIBLE_DEVICES=0,1,2,3 python baseline_cluster.py data data -s Market1501 -t MSMT17 -a reid_resnet50 \\\n--pretrained-model-path logs/baseline/Market2MSMT/checkpoints/best.pth \\\n--num-clusters 1000 --finetune --seed 0 --log logs/baseline_cluster/Market2MSMT\n\n# MSMT -> Market1501\n# step1: pretrain\nCUDA_VISIBLE_DEVICES=0 python baseline.py data data -s MSMT17 -t Market1501 -a reid_resnet50 \\\n--iters-per-epoch 800 --print-freq 80 --finetune --seed 0 --log logs/baseline/MSMT2Market\n# step2: train with pseudo labels assigned by cluster algorithm\nCUDA_VISIBLE_DEVICES=0,1,2,3 python baseline_cluster.py data data -s MSMT17 -t Market1501 -a reid_resnet50 \\\n--pretrained-model-path logs/baseline/MSMT2Market/checkpoints/best.pth \\\n--finetune --seed 0 --log logs/baseline_cluster/MSMT2Market\n\n# Duke -> MSMT\n# step1: pretrain\nCUDA_VISIBLE_DEVICES=0 python baseline.py data data -s DukeMTMC -t MSMT17 -a reid_resnet50 \\\n--iters-per-epoch 800 --print-freq 80 --finetune --seed 0 --log logs/baseline/Duke2MSMT\n# step2: train with pseudo labels assigned by cluster algorithm\nCUDA_VISIBLE_DEVICES=0,1,2,3 python baseline_cluster.py data data -s DukeMTMC -t MSMT17 -a reid_resnet50 \\\n--pretrained-model-path logs/baseline/Duke2MSMT/checkpoints/best.pth \\\n--num-clusters 1000 --finetune --seed 0 --log logs/baseline_cluster/Duke2MSMT\n\n# MSMT -> Duke\n# step1: pretrain\nCUDA_VISIBLE_DEVICES=0 python baseline.py data data -s MSMT17 -t DukeMTMC -a reid_resnet50 \\\n--iters-per-epoch 800 --print-freq 80 --finetune --seed 0 --log logs/baseline/MSMT2Duke\n# step2: train with pseudo labels assigned by cluster algorithm\nCUDA_VISIBLE_DEVICES=0,1,2,3 python baseline_cluster.py data data -s MSMT17 -t DukeMTMC -a reid_resnet50 \\\n--pretrained-model-path logs/baseline/MSMT2Duke/checkpoints/best.pth \\\n--finetune --seed 0 --log logs/baseline_cluster/MSMT2Duke\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/examples/domain_adaptation/re_identification/ibn.sh",
"content": "#!/usr/bin/env bash\n# Market1501 -> Duke\nCUDA_VISIBLE_DEVICES=0 python baseline.py data data -s Market1501 -t DukeMTMC -a resnet50_ibn_a \\\n--iters-per-epoch 800 --print-freq 80 --finetune --seed 0 --log logs/ibn/Market2Duke\nCUDA_VISIBLE_DEVICES=0 python baseline.py data data -s Market1501 -t DukeMTMC -a resnet50_ibn_b \\\n--iters-per-epoch 800 --print-freq 80 --finetune --seed 0 --log logs/ibn/Market2Duke\n\n# Duke -> Market1501\nCUDA_VISIBLE_DEVICES=0 python baseline.py data data -s DukeMTMC -t Market1501 -a resnet50_ibn_a \\\n--iters-per-epoch 800 --print-freq 80 --finetune --seed 0 --log logs/ibn/Duke2Market\nCUDA_VISIBLE_DEVICES=0 python baseline.py data data -s DukeMTMC -t Market1501 -a resnet50_ibn_b \\\n--iters-per-epoch 800 --print-freq 80 --finetune --seed 0 --log logs/ibn/Duke2Market\n\n# Market1501 -> MSMT\nCUDA_VISIBLE_DEVICES=0 python baseline.py data data -s Market1501 -t MSMT17 -a resnet50_ibn_a \\\n--iters-per-epoch 800 --print-freq 80 --finetune --seed 0 --log logs/ibn/Market2MSMT\nCUDA_VISIBLE_DEVICES=0 python baseline.py data data -s Market1501 -t MSMT17 -a resnet50_ibn_b \\\n--iters-per-epoch 800 --print-freq 80 --finetune --seed 0 --log logs/ibn/Market2MSMT\n\n# MSMT -> Market1501\nCUDA_VISIBLE_DEVICES=0 python baseline.py data data -s MSMT17 -t Market1501 -a resnet50_ibn_a \\\n--iters-per-epoch 800 --print-freq 80 --finetune --seed 0 --log logs/ibn/MSMT2Market\nCUDA_VISIBLE_DEVICES=0 python baseline.py data data -s MSMT17 -t Market1501 -a resnet50_ibn_b \\\n--iters-per-epoch 800 --print-freq 80 --finetune --seed 0 --log logs/ibn/MSMT2Market\n\n# Duke -> MSMT\nCUDA_VISIBLE_DEVICES=0 python baseline.py data data -s DukeMTMC -t MSMT17 -a resnet50_ibn_a \\\n--iters-per-epoch 800 --print-freq 80 --finetune --seed 0 --log logs/ibn/Duke2MSMT\nCUDA_VISIBLE_DEVICES=0 python baseline.py data data -s DukeMTMC -t MSMT17 -a resnet50_ibn_b \\\n--iters-per-epoch 800 --print-freq 80 --finetune --seed 0 --log logs/ibn/Duke2MSMT\n\n# MSMT -> Duke\nCUDA_VISIBLE_DEVICES=0 python baseline.py data data -s MSMT17 -t DukeMTMC -a resnet50_ibn_a \\\n--iters-per-epoch 800 --print-freq 80 --finetune --seed 0 --log logs/ibn/MSMT2Duke\nCUDA_VISIBLE_DEVICES=0 python baseline.py data data -s MSMT17 -t DukeMTMC -a resnet50_ibn_b \\\n--iters-per-epoch 800 --print-freq 80 --finetune --seed 0 --log logs/ibn/MSMT2Duke\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/examples/domain_adaptation/re_identification/idm.py",
"content": "import random\nimport time\nimport warnings\nimport sys\nimport argparse\nimport shutil\nimport os.path as osp\n\nimport numpy as np\nimport torch\nimport torch.nn as nn\nimport torch.nn.functional as F\nfrom torch.nn import DataParallel\nimport torch.backends.cudnn as cudnn\nfrom torch.optim import Adam\nfrom torch.optim.lr_scheduler import StepLR\nfrom torch.utils.data import DataLoader\nfrom sklearn.cluster import DBSCAN\n\nsys.path.append('../../..')\nimport dalib.adaptation.idm.models as models\nfrom dalib.adaptation.idm.models.identifier import ReIdentifier\nfrom dalib.adaptation.idm.xbm import XBM\nfrom dalib.adaptation.idm.loss import BridgeFeatLoss, BridgeProbLoss, DivLoss\nfrom dalib.adaptation.idm.utils import filter_layers, convert_dsbn_idm\nimport common.vision.datasets.reid as datasets\nfrom common.vision.datasets.reid.convert import convert_to_pytorch_dataset\nfrom common.utils.metric.reid import extract_reid_feature, validate, visualize_ranked_results\nfrom common.vision.models.reid.loss import TripletLoss, TripletLossXBM\nfrom common.utils.data import ForeverDataIterator, RandomMultipleGallerySampler\nfrom common.utils.metric import accuracy\nfrom common.utils.meter import AverageMeter, ProgressMeter\nfrom common.utils.logger import CompleteLogger\n\nsys.path.append('.')\nimport utils\n\ndevice = torch.device(\"cuda\" if torch.cuda.is_available() else \"cpu\")\n\n\ndef main(args: argparse.Namespace):\n logger = CompleteLogger(args.log, args.phase)\n print(args)\n\n if args.seed is not None:\n random.seed(args.seed)\n np.random.seed(args.seed)\n torch.manual_seed(args.seed)\n cudnn.deterministic = True\n warnings.warn('You have chosen to seed training. '\n 'This will turn on the CUDNN deterministic setting, '\n 'which can slow down your training considerably! '\n 'You may see unexpected behavior when restarting '\n 'from checkpoints.')\n\n cudnn.benchmark = True\n\n # Data loading code\n train_transform = utils.get_train_transform(args.height, args.width, args.train_resizing,\n random_horizontal_flip=True, random_color_jitter=False,\n random_gray_scale=False, random_erasing=True)\n val_transform = utils.get_val_transform(args.height, args.width)\n print(\"train_transform: \", train_transform)\n print(\"val_transform: \", val_transform)\n\n working_dir = osp.dirname(osp.abspath(__file__))\n source_root = osp.join(working_dir, args.source_root)\n target_root = osp.join(working_dir, args.target_root)\n\n # source dataset\n source_dataset = datasets.__dict__[args.source](root=osp.join(source_root, args.source.lower()))\n sampler = RandomMultipleGallerySampler(source_dataset.train, args.num_instances)\n train_source_loader = DataLoader(\n convert_to_pytorch_dataset(source_dataset.train, root=source_dataset.images_dir, transform=train_transform),\n batch_size=args.batch_size, num_workers=args.workers, sampler=sampler, pin_memory=True, drop_last=True)\n train_source_iter = ForeverDataIterator(train_source_loader)\n cluster_source_loader = DataLoader(\n convert_to_pytorch_dataset(source_dataset.train, root=source_dataset.images_dir, transform=val_transform),\n batch_size=args.batch_size, num_workers=args.workers, shuffle=False, pin_memory=True)\n val_loader = DataLoader(\n convert_to_pytorch_dataset(list(set(source_dataset.query) | set(source_dataset.gallery)),\n root=source_dataset.images_dir, transform=val_transform),\n batch_size=args.batch_size, num_workers=args.workers, shuffle=False, pin_memory=True)\n\n # target dataset\n target_dataset = datasets.__dict__[args.target](root=osp.join(target_root, args.target.lower()))\n cluster_target_loader = DataLoader(\n convert_to_pytorch_dataset(target_dataset.train, root=target_dataset.images_dir, transform=val_transform),\n batch_size=args.batch_size, num_workers=args.workers, shuffle=False, pin_memory=True)\n test_loader = DataLoader(\n convert_to_pytorch_dataset(list(set(target_dataset.query) | set(target_dataset.gallery)),\n root=target_dataset.images_dir, transform=val_transform),\n batch_size=args.batch_size, num_workers=args.workers, shuffle=False, pin_memory=True)\n\n n_s_classes = source_dataset.num_train_pids\n args.n_classes = n_s_classes + len(target_dataset.train)\n args.n_s_classes = n_s_classes\n args.n_t_classes = len(target_dataset.train)\n\n # create model\n backbone = models.__dict__[args.arch](pretrained=True)\n pool_layer = nn.Identity() if args.no_pool else None\n model = ReIdentifier(backbone, args.n_classes, finetune=args.finetune, pool_layer=pool_layer)\n features_dim = model.features_dim\n\n idm_bn_names = filter_layers(args.stage)\n convert_dsbn_idm(model, idm_bn_names, idm=False)\n\n model = model.to(device)\n model = DataParallel(model)\n\n # resume from the best checkpoint\n if args.phase != 'train':\n checkpoint = torch.load(logger.get_checkpoint_path('best'), map_location='cpu')\n utils.copy_state_dict(model, checkpoint['model'])\n\n # analysis the model\n if args.phase == 'analysis':\n # plot t-SNE\n utils.visualize_tsne(source_loader=val_loader, target_loader=test_loader, model=model,\n filename=osp.join(logger.visualize_directory, 'analysis', 'TSNE.pdf'), device=device)\n # visualize ranked results\n visualize_ranked_results(test_loader, model, target_dataset.query, target_dataset.gallery, device,\n visualize_dir=logger.visualize_directory, width=args.width, height=args.height,\n rerank=args.rerank)\n return\n\n if args.phase == 'test':\n print(\"Test on target domain:\")\n validate(test_loader, model, target_dataset.query, target_dataset.gallery, device, cmc_flag=True,\n rerank=args.rerank)\n return\n\n # create XBM\n dataset_size = len(source_dataset.train) + len(target_dataset.train)\n memory_size = int(args.ratio * dataset_size)\n xbm = XBM(memory_size, features_dim)\n\n # initialize source-domain class centroids\n source_feature_dict = extract_reid_feature(cluster_source_loader, model, device, normalize=True)\n source_features_per_id = {}\n for f, pid, _ in source_dataset.train:\n if pid not in source_features_per_id:\n source_features_per_id[pid] = []\n source_features_per_id[pid].append(source_feature_dict[f].unsqueeze(0))\n source_centers = [torch.cat(source_features_per_id[pid], 0).mean(0) for pid in\n sorted(source_features_per_id.keys())]\n source_centers = torch.stack(source_centers, 0)\n source_centers = F.normalize(source_centers, dim=1)\n model.module.head.weight.data[0: n_s_classes].copy_(source_centers.to(device))\n\n # save memory\n del source_centers, cluster_source_loader, source_features_per_id\n\n # define optimizer and lr scheduler\n optimizer = Adam(model.module.get_parameters(base_lr=args.lr, rate=args.rate), args.lr,\n weight_decay=args.weight_decay)\n lr_scheduler = StepLR(optimizer, step_size=args.step_size, gamma=0.1)\n\n if args.resume:\n checkpoint = torch.load(args.resume, map_location='cpu')\n utils.copy_state_dict(model, checkpoint['model'])\n optimizer.load_state_dict(checkpoint['optimizer'])\n lr_scheduler.load_state_dict(checkpoint['lr_scheduler'])\n args.start_epoch = checkpoint['epoch'] + 1\n\n # start training\n best_test_mAP = 0.\n for epoch in range(args.start_epoch, args.epochs):\n # run clustering algorithm and generate pseudo labels\n train_target_iter = run_dbscan(cluster_target_loader, model, target_dataset, train_transform, args)\n\n # train for one epoch\n print(lr_scheduler.get_lr())\n train(train_source_iter, train_target_iter, model, optimizer, xbm, epoch, args)\n\n if (epoch + 1) % args.eval_step == 0 or (epoch == args.epochs - 1):\n # remember best mAP and save checkpoint\n torch.save(\n {\n 'model': model.state_dict(),\n 'optimizer': optimizer.state_dict(),\n 'lr_scheduler': lr_scheduler.state_dict(),\n 'epoch': epoch\n }, logger.get_checkpoint_path(epoch)\n )\n print(\"Test on target domain...\")\n _, test_mAP = validate(test_loader, model, target_dataset.query, target_dataset.gallery, device,\n cmc_flag=True, rerank=args.rerank)\n if test_mAP > best_test_mAP:\n shutil.copy(logger.get_checkpoint_path(epoch), logger.get_checkpoint_path('best'))\n best_test_mAP = max(test_mAP, best_test_mAP)\n\n # update lr\n lr_scheduler.step()\n\n print(\"best mAP on target = {}\".format(best_test_mAP))\n logger.close()\n\n\ndef run_dbscan(cluster_loader: DataLoader, model: DataParallel, target_dataset, train_transform,\n args: argparse.Namespace):\n # run dbscan clustering algorithm\n feature_dict = extract_reid_feature(cluster_loader, model, device, normalize=True)\n feature = torch.stack(list(feature_dict.values())).cpu()\n rerank_dist = utils.compute_rerank_dist(feature).numpy()\n\n print('Clustering with dbscan algorithm')\n dbscan = DBSCAN(eps=args.eps, min_samples=4, metric='precomputed', n_jobs=-1)\n cluster_labels = dbscan.fit_predict(rerank_dist)\n print('Clustering finished')\n\n # generate training set with pseudo labels and calculate cluster centers\n target_train_set = []\n cluster_centers = {}\n for i, ((fname, _, cid), label) in enumerate(zip(target_dataset.train, cluster_labels)):\n if label == -1:\n continue\n target_train_set.append((fname, args.n_s_classes + label, cid))\n\n if label not in cluster_centers:\n cluster_centers[label] = []\n cluster_centers[label].append(feature[i])\n\n cluster_centers = [torch.stack(cluster_centers[idx]).mean(0) for idx in sorted(cluster_centers.keys())]\n cluster_centers = torch.stack(cluster_centers)\n # normalize cluster centers\n cluster_centers = F.normalize(cluster_centers, dim=1).float().to(device)\n\n # reinitialize classifier head\n num_clusters = len(set(cluster_labels)) - (1 if -1 in cluster_labels else 0)\n args.n_t_classes = num_clusters\n model.module.head.weight.data[args.n_s_classes: args.n_s_classes + args.n_t_classes].copy_(cluster_centers)\n\n sampler = RandomMultipleGallerySampler(target_train_set, args.num_instances)\n train_target_loader = DataLoader(\n convert_to_pytorch_dataset(target_train_set, root=target_dataset.images_dir, transform=train_transform),\n batch_size=args.batch_size, num_workers=args.workers, sampler=sampler, pin_memory=True, drop_last=True)\n train_target_iter = ForeverDataIterator(train_target_loader)\n\n return train_target_iter\n\n\ndef train(train_source_iter: ForeverDataIterator, train_target_iter: ForeverDataIterator, model, optimizer: Adam,\n xbm: XBM, epoch: int, args: argparse.Namespace):\n # define loss function\n criterion_ce = BridgeProbLoss(args.n_s_classes + args.n_t_classes).to(device)\n criterion_triplet = TripletLoss(margin=args.margin).to(device)\n criterion_triplet_xbm = TripletLossXBM(margin=args.margin).to(device)\n criterion_bridge_feat = BridgeFeatLoss().to(device)\n criterion_diverse = DivLoss().to(device)\n\n batch_time = AverageMeter('Time', ':4.2f')\n data_time = AverageMeter('Data', ':3.1f')\n losses_ce = AverageMeter('CeLoss', ':3.2f')\n losses_triplet = AverageMeter('TripletLoss', ':3.2f')\n losses_triplet_xbm = AverageMeter('XBMTripletLoss', ':3.2f')\n losses_bridge_prob = AverageMeter('BridgeProbLoss', ':3.2f')\n losses_bridge_feat = AverageMeter('BridgeFeatLoss', ':3.2f')\n losses_diverse = AverageMeter('DiverseLoss', ':3.2f')\n losses = AverageMeter('Loss', ':3.2f')\n\n cls_accs_s = AverageMeter('Src Cls Acc', ':3.1f')\n cls_accs_t = AverageMeter('Tgt Cls Acc', ':3.1f')\n\n progress = ProgressMeter(\n args.iters_per_epoch,\n [batch_time, data_time, losses_ce, losses_triplet, losses_triplet_xbm, losses_bridge_prob, losses_bridge_feat,\n losses_diverse, losses, cls_accs_s, cls_accs_t],\n prefix=\"Epoch: [{}]\".format(epoch))\n\n # switch to train mode\n model.train()\n\n end = time.time()\n\n for i in range(args.iters_per_epoch):\n x_s, _, labels_s, _ = next(train_source_iter)\n x_t, _, labels_t, _ = next(train_target_iter)\n\n x_s = x_s.to(device)\n x_t = x_t.to(device)\n labels_s = labels_s.to(device)\n labels_t = labels_t.to(device)\n\n # measure data loading time\n data_time.update(time.time() - end)\n\n # arrange batch for domain-specific BN\n device_num = torch.cuda.device_count()\n B, C, H, W = x_s.size()\n\n def reshape(tensor):\n return tensor.view(device_num, -1, C, H, W)\n\n x_s, x_t = reshape(x_s), reshape(x_t)\n x = torch.cat((x_s, x_t), 1).view(-1, C, H, W)\n\n labels = torch.cat((labels_s.view(device_num, -1), labels_t.view(device_num, -1)), 1)\n labels = labels.view(-1)\n\n # compute output\n y, f, attention_lam = model(x, stage=args.stage)\n y = y[:, 0:args.n_s_classes + args.n_t_classes] # only (n_s_classes + n_t_classes) classes are meaningful\n\n # split feats\n ori_f = f.view(device_num, -1, f.size(-1))\n f_s, f_t, f_mixed = ori_f.split(ori_f.size(1) // 3, dim=1)\n ori_f = torch.cat((f_s, f_t), 1).view(-1, ori_f.size(-1))\n\n # cross entropy loss\n loss_ce, loss_bridge_prob = criterion_ce(y, labels, attention_lam[:, 0].detach(), device_num)\n # triplet loss\n loss_triplet = criterion_triplet(ori_f, labels)\n # diverse loss\n loss_diverse = criterion_diverse(attention_lam)\n # bridge feature loss\n f_s = f_s.contiguous().view(-1, f.size(-1))\n f_t = f_t.contiguous().view(-1, f.size(-1))\n f_mixed = f_mixed.contiguous().view(-1, f.size(-1))\n loss_bridge_feat = criterion_bridge_feat(f_s, f_t, f_mixed, attention_lam)\n # xbm triplet loss\n xbm.enqueue_dequeue(ori_f.detach(), labels.detach())\n xbm_f, xbm_labels = xbm.get()\n loss_triplet_xbm = criterion_triplet_xbm(ori_f, labels, xbm_f, xbm_labels)\n\n loss = (1. - args.mu1) * loss_ce + loss_triplet + loss_triplet_xbm + \\\n args.mu1 * loss_bridge_prob + args.mu2 * loss_bridge_feat + args.mu3 * loss_diverse\n\n # compute gradient and do SGD step\n optimizer.zero_grad()\n loss.backward()\n optimizer.step()\n\n ori_y = y.view(device_num, -1, y.size(-1))\n y_s, y_t, _ = ori_y.split(ori_y.size(1) // 3, dim=1)\n cls_acc_s = accuracy(y_s.reshape(-1, y_s.size(-1)), labels_s)[0]\n cls_acc_t = accuracy(y_t.reshape(-1, y_t.size(-1)), labels_t)[0]\n\n # update statistics\n losses_ce.update(loss_ce.item(), x_s.size(0))\n losses_triplet.update(loss_triplet.item(), x_s.size(0))\n losses_triplet_xbm.update(loss_triplet_xbm.item(), x_s.size(0))\n losses_bridge_prob.update(loss_bridge_prob.item(), x_s.size(0))\n losses_bridge_feat.update(loss_bridge_feat.item(), x_s.size(0))\n losses_diverse.update(loss_diverse.item(), x_s.size(0))\n losses.update(loss.item(), x_s.size(0))\n\n cls_accs_s.update(cls_acc_s.item(), x_s.size(0))\n cls_accs_t.update(cls_acc_t.item(), x_s.size(0))\n\n # measure elapsed time\n batch_time.update(time.time() - end)\n end = time.time()\n\n if i % args.print_freq == 0:\n progress.display(i)\n\n\nif __name__ == '__main__':\n dataset_names = sorted(\n name for name in datasets.__dict__\n if not name.startswith(\"__\") and callable(datasets.__dict__[name])\n )\n architecture_names = sorted(\n name for name in models.__dict__\n if name.islower() and not name.startswith(\"__\")\n and callable(models.__dict__[name])\n )\n parser = argparse.ArgumentParser(description=\"IDM for Domain Adaptative ReID\")\n # dataset parameters\n parser.add_argument('source_root', help='root path of the source dataset')\n parser.add_argument('target_root', help='root path of the target dataset')\n parser.add_argument('-s', '--source', type=str, help='source domain')\n parser.add_argument('-t', '--target', type=str, help='target domain')\n parser.add_argument('--train-resizing', type=str, default='default')\n # model parameters\n parser.add_argument('-a', '--arch', metavar='ARCH', default='reid_resnet50',\n choices=architecture_names,\n help='backbone architecture: ' +\n ' | '.join(architecture_names) +\n ' (default: reid_resnet50)')\n parser.add_argument('--no-pool', action='store_true', help='no pool layer after the feature extractor.')\n parser.add_argument('--finetune', action='store_true', help='whether use 10x smaller lr for backbone')\n parser.add_argument('--rate', type=float, default=0.2)\n parser.add_argument('--n-classes', type=int, default=1000, help='number of classes')\n parser.add_argument('--n-s-classes', type=int, default=1000, help=\"number of source classes\")\n parser.add_argument('--n-t-classes', type=int, default=1000, help=\"number of target classes\")\n # training parameters\n parser.add_argument('--resume', type=str, default=None, help=\"Where to restore model parameters from.\")\n parser.add_argument('--eps', type=float, default=0.6, help=\"max neighbor distance for DBSCAN\")\n parser.add_argument('--margin', type=float, default=0.3, help='margin for triplet loss')\n parser.add_argument('--mu1', type=float, default=0.7, help=\"weight for prediction bridge loss\")\n parser.add_argument('--mu2', type=float, default=0.1, help=\"weight for feature bridge loss\")\n parser.add_argument('--mu3', type=float, default=1, help=\"weight for diverse loss\")\n parser.add_argument('--ratio', type=float, default=1, help='ratio of dataset to store in memory bank')\n parser.add_argument('--stage', type=int, default=0, choices=range(5))\n parser.add_argument('-j', '--workers', type=int, default=4)\n parser.add_argument('-b', '--batch-size', type=int, default=64)\n parser.add_argument('--height', type=int, default=256, help=\"input height\")\n parser.add_argument('--width', type=int, default=128, help=\"input width\")\n parser.add_argument('--num-instances', type=int, default=4,\n help=\"each minibatch consist of \"\n \"(batch_size // num_instances) identities, and \"\n \"each identity has num_instances instances, \"\n \"default: 4\")\n parser.add_argument('--lr', type=float, default=0.00035, help=\"learning rate\")\n parser.add_argument('--weight-decay', type=float, default=5e-4)\n parser.add_argument('--epochs', type=int, default=50)\n parser.add_argument('--step-size', type=int, default=20)\n parser.add_argument('--start-epoch', default=0, type=int, help='start epoch')\n parser.add_argument('--eval-step', type=int, default=1)\n parser.add_argument('--iters-per-epoch', type=int, default=400)\n parser.add_argument('--print-freq', type=int, default=40)\n parser.add_argument('--seed', default=None, type=int, help='seed for initializing training.')\n parser.add_argument('--rerank', action='store_true', help=\"evaluation only\")\n parser.add_argument(\"--log\", type=str, default='idm',\n help=\"Where to save logs, checkpoints and debugging images.\")\n parser.add_argument(\"--phase\", type=str, default='train', choices=['train', 'test', 'analysis'],\n help=\"When phase is 'test', only test the model.\"\n \"When phase is 'analysis', only analysis the model.\")\n args = parser.parse_args()\n main(args)\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/examples/domain_adaptation/re_identification/idm.sh",
"content": "#!/usr/bin/env bash\n# Market1501 -> Duke\nCUDA_VISIBLE_DEVICES=0,1,2,3 python idm.py data data -s Market1501 -t DukeMTMC -a reid_resnet50 \\\n--seed 0 --log logs/idm/Market2Duke\n\n# Duke -> Market1501\nCUDA_VISIBLE_DEVICES=0,1,2,3 python idm.py data data -s DukeMTMC -t Market1501 -a reid_resnet50 \\\n--seed 0 --log logs/idm/Duke2Market\n\n# Market1501 -> MSMT\nCUDA_VISIBLE_DEVICES=0,1,2,3 python idm.py data data -s Market1501 -t MSMT17 -a reid_resnet50 \\\n--seed 0 --log logs/idm/Market2MSMT\n\n# MSMT -> Market1501\nCUDA_VISIBLE_DEVICES=0,1,2,3 python idm.py data data -s MSMT17 -t Market1501 -a reid_resnet50 \\\n--seed 0 --log logs/idm/MSMT2Market\n\n# Duke -> MSMT\nCUDA_VISIBLE_DEVICES=0,1,2,3 python idm.py data data -s DukeMTMC -t MSMT17 -a reid_resnet50 \\\n--seed 1 --log logs/idm/Duke2MSMT\n\n# MSMT -> Duke\nCUDA_VISIBLE_DEVICES=0,1,2,3 python idm.py data data -s MSMT17 -t DukeMTMC -a reid_resnet50 \\\n--seed 0 --log logs/idm/MSMT2Duke\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/examples/domain_adaptation/re_identification/mmt.py",
"content": "\"\"\"\n@author: Baixu Chen\n@contact: cbx_99_hasta@outlook.com\n\"\"\"\nimport random\nimport time\nimport warnings\nimport sys\nimport argparse\nimport os.path as osp\n\nimport numpy as np\nimport torch\nimport torch.nn as nn\nimport torch.nn.functional as F\nfrom torch.nn import DataParallel\nimport torch.backends.cudnn as cudnn\nfrom torch.optim import Adam\nfrom torch.utils.data import DataLoader\nfrom sklearn.cluster import KMeans, DBSCAN\n\nsys.path.append('../../..')\nimport common.vision.datasets.reid as datasets\nfrom common.vision.datasets.reid.convert import convert_to_pytorch_dataset\nfrom common.vision.models.reid.identifier import ReIdentifier\nfrom common.vision.models.reid.loss import CrossEntropyLossWithLabelSmooth, SoftTripletLoss, CrossEntropyLoss\nfrom dalib.adaptation.self_ensemble import EmaTeacher\nfrom common.vision.transforms import MultipleApply\nfrom common.utils.metric.reid import extract_reid_feature, validate, visualize_ranked_results\nfrom common.utils.data import ForeverDataIterator, RandomMultipleGallerySampler\nfrom common.utils.metric import accuracy\nfrom common.utils.meter import AverageMeter, ProgressMeter\nfrom common.utils.logger import CompleteLogger\n\nsys.path.append('.')\nimport utils\n\ndevice = torch.device(\"cuda\" if torch.cuda.is_available() else \"cpu\")\n\n\ndef main(args: argparse.Namespace):\n logger = CompleteLogger(args.log, args.phase)\n print(args)\n\n if args.seed is not None:\n random.seed(args.seed)\n np.random.seed(args.seed)\n torch.manual_seed(args.seed)\n cudnn.deterministic = True\n warnings.warn('You have chosen to seed training. '\n 'This will turn on the CUDNN deterministic setting, '\n 'which can slow down your training considerably! '\n 'You may see unexpected behavior when restarting '\n 'from checkpoints.')\n\n cudnn.benchmark = True\n\n # Data loading code\n train_transform = utils.get_train_transform(args.height, args.width, args.train_resizing,\n random_horizontal_flip=True, random_color_jitter=False,\n random_gray_scale=False, random_erasing=True)\n val_transform = utils.get_val_transform(args.height, args.width)\n print(\"train_transform: \", train_transform)\n print(\"val_transform: \", val_transform)\n\n working_dir = osp.dirname(osp.abspath(__file__))\n source_root = osp.join(working_dir, args.source_root)\n target_root = osp.join(working_dir, args.target_root)\n\n # source dataset\n source_dataset = datasets.__dict__[args.source](root=osp.join(source_root, args.source.lower()))\n val_loader = DataLoader(\n convert_to_pytorch_dataset(list(set(source_dataset.query) | set(source_dataset.gallery)),\n root=source_dataset.images_dir,\n transform=val_transform),\n batch_size=args.batch_size, num_workers=args.workers, shuffle=False, pin_memory=True)\n\n # target dataset\n target_dataset = datasets.__dict__[args.target](root=osp.join(target_root, args.target.lower()))\n cluster_loader = DataLoader(\n convert_to_pytorch_dataset(target_dataset.train, root=target_dataset.images_dir, transform=val_transform),\n batch_size=args.batch_size, num_workers=args.workers, shuffle=False, pin_memory=True)\n test_loader = DataLoader(\n convert_to_pytorch_dataset(list(set(target_dataset.query) | set(target_dataset.gallery)),\n root=target_dataset.images_dir, transform=val_transform),\n batch_size=args.batch_size, num_workers=args.workers, shuffle=False, pin_memory=True)\n\n # create model\n model_1, model_1_ema = create_model(args, args.pretrained_model_1_path)\n model_2, model_2_ema = create_model(args, args.pretrained_model_2_path)\n\n # resume from the best checkpoint\n if args.phase != 'train':\n checkpoint = torch.load(logger.get_checkpoint_path('best'), map_location='cpu')\n utils.copy_state_dict(model_1_ema, checkpoint)\n\n # analysis the model\n if args.phase == 'analysis':\n # plot t-SNE\n utils.visualize_tsne(source_loader=val_loader, target_loader=test_loader, model=model_1_ema,\n filename=osp.join(logger.visualize_directory, 'analysis', 'TSNE.pdf'), device=device)\n # visualize ranked results\n visualize_ranked_results(test_loader, model_1_ema, target_dataset.query, target_dataset.gallery, device,\n visualize_dir=logger.visualize_directory, width=args.width, height=args.height,\n rerank=args.rerank)\n return\n\n if args.phase == 'test':\n print(\"Test on Source domain:\")\n validate(val_loader, model_1_ema, source_dataset.query, source_dataset.gallery, device, cmc_flag=True,\n rerank=args.rerank)\n print(\"Test on target domain:\")\n validate(test_loader, model_1_ema, target_dataset.query, target_dataset.gallery, device, cmc_flag=True,\n rerank=args.rerank)\n return\n\n # define loss function\n num_classes = args.num_clusters\n criterion_ce = CrossEntropyLossWithLabelSmooth(num_classes).to(device)\n criterion_ce_soft = CrossEntropyLoss().to(device)\n criterion_triplet = SoftTripletLoss(margin=0.0).to(device)\n criterion_triplet_soft = SoftTripletLoss(margin=None).to(device)\n\n # optionally resume from a checkpoint\n if args.resume:\n checkpoint = torch.load(args.resume, map_location='cpu')\n utils.copy_state_dict(model_1, checkpoint['model_1'])\n utils.copy_state_dict(model_1_ema, checkpoint['model_1_ema'])\n utils.copy_state_dict(model_2, checkpoint['model_2'])\n utils.copy_state_dict(model_2_ema, checkpoint['model_2_ema'])\n args.start_epoch = checkpoint['epoch'] + 1\n\n # start training\n best_test_mAP = 0.\n for epoch in range(args.start_epoch, args.epochs):\n # run clustering algorithm and generate pseudo labels\n if args.clustering_algorithm == 'kmeans':\n train_target_iter = run_kmeans(cluster_loader, model_1, model_2, model_1_ema, model_2_ema, target_dataset,\n train_transform, args)\n elif args.clustering_algorithm == 'dbscan':\n train_target_iter, num_classes = run_dbscan(cluster_loader, model_1, model_2, model_1_ema, model_2_ema,\n target_dataset, train_transform, args)\n\n # define cross entropy loss with current number of classes\n criterion_ce = CrossEntropyLossWithLabelSmooth(num_classes).to(device)\n\n # define optimizer\n optimizer = Adam(model_1.module.get_parameters(base_lr=args.lr, rate=args.rate) + model_2.module.get_parameters(\n base_lr=args.lr, rate=args.rate), args.lr, weight_decay=args.weight_decay)\n\n # train for one epoch\n train(train_target_iter, model_1, model_1_ema, model_2, model_2_ema, optimizer, criterion_ce, criterion_ce_soft,\n criterion_triplet, criterion_triplet_soft, epoch, args)\n\n if (epoch + 1) % args.eval_step == 0 or (epoch == args.epochs - 1):\n # save checkpoint and remember best mAP\n torch.save(\n {\n 'model_1': model_1.state_dict(),\n 'model_1_ema': model_1_ema.state_dict(),\n 'model_2': model_2.state_dict(),\n 'model_2_ema': model_2_ema.state_dict(),\n 'epoch': epoch\n }, logger.get_checkpoint_path(epoch)\n )\n print(\"Test model_1 on target domain...\")\n _, test_mAP_1 = validate(test_loader, model_1_ema, target_dataset.query, target_dataset.gallery,\n device, cmc_flag=True, rerank=args.rerank)\n print(\"Test model_2 on target domain...\")\n _, test_mAP_2 = validate(test_loader, model_2_ema, target_dataset.query, target_dataset.gallery,\n device, cmc_flag=True, rerank=args.rerank)\n if test_mAP_1 > test_mAP_2 and test_mAP_1 > best_test_mAP:\n torch.save(model_1_ema.state_dict(), logger.get_checkpoint_path('best'))\n best_test_mAP = test_mAP_1\n if test_mAP_2 > test_mAP_1 and test_mAP_2 > best_test_mAP:\n torch.save(model_2_ema.state_dict(), logger.get_checkpoint_path('best'))\n best_test_mAP = test_mAP_2\n\n print(\"best mAP on target = {}\".format(best_test_mAP))\n logger.close()\n\n\ndef create_model(args: argparse.Namespace, pretrained_model_path: str):\n num_classes = args.num_clusters\n backbone = utils.get_model(args.arch)\n pool_layer = nn.Identity() if args.no_pool else None\n model = ReIdentifier(backbone, num_classes, finetune=args.finetune, pool_layer=pool_layer).to(device)\n model = DataParallel(model)\n\n # load pretrained weights\n pretrained_model = torch.load(pretrained_model_path)\n utils.copy_state_dict(model, pretrained_model)\n\n # EMA model\n model_ema = EmaTeacher(model, args.alpha)\n return model, model_ema\n\n\ndef run_kmeans(cluster_loader: DataLoader, model_1: DataParallel, model_2: DataParallel, model_1_ema: EmaTeacher,\n model_2_ema: EmaTeacher, target_dataset, train_transform, args: argparse.Namespace):\n # run kmeans clustering algorithm\n print('Clustering into {} classes'.format(args.num_clusters))\n # collect feature with different ema teachers\n feature_dict_1 = extract_reid_feature(cluster_loader, model_1_ema, device, normalize=True)\n feature_1 = torch.stack(list(feature_dict_1.values())).cpu().numpy()\n feature_dict_2 = extract_reid_feature(cluster_loader, model_2_ema, device, normalize=True)\n feature_2 = torch.stack(list(feature_dict_2.values())).cpu().numpy()\n # average feature_1, feature_2 to create final feature\n feature = (feature_1 + feature_2) / 2\n\n km = KMeans(n_clusters=args.num_clusters, random_state=args.seed).fit(feature)\n cluster_labels = km.labels_\n cluster_centers = km.cluster_centers_\n print('Clustering finished')\n\n # normalize cluster centers and convert to pytorch tensor\n cluster_centers = torch.from_numpy(cluster_centers).float().to(device)\n cluster_centers = F.normalize(cluster_centers, dim=1)\n\n # reinitialize classifier head\n model_1.module.head.weight.data.copy_(cluster_centers)\n model_2.module.head.weight.data.copy_(cluster_centers)\n model_1_ema.module.head.weight.data.copy_(cluster_centers)\n model_2_ema.module.head.weight.data.copy_(cluster_centers)\n\n # generate training set with pseudo labels\n target_train_set = []\n for (fname, _, cid), label in zip(target_dataset.train, cluster_labels):\n target_train_set.append((fname, int(label), cid))\n\n sampler = RandomMultipleGallerySampler(target_train_set, args.num_instances)\n train_target_loader = DataLoader(\n convert_to_pytorch_dataset(target_train_set, root=target_dataset.images_dir,\n transform=MultipleApply([train_transform, train_transform])),\n batch_size=args.batch_size, num_workers=args.workers, sampler=sampler, pin_memory=True, drop_last=True)\n train_target_iter = ForeverDataIterator(train_target_loader)\n\n return train_target_iter\n\n\ndef run_dbscan(cluster_loader: DataLoader, model_1: DataParallel, model_2: DataParallel, model_1_ema: EmaTeacher,\n model_2_ema: EmaTeacher, target_dataset, train_transform, args: argparse.Namespace):\n # run dbscan clustering algorithm\n\n # collect feature with different ema teachers\n feature_dict_1 = extract_reid_feature(cluster_loader, model_1_ema, device, normalize=True)\n feature_1 = torch.stack(list(feature_dict_1.values())).cpu()\n feature_dict_2 = extract_reid_feature(cluster_loader, model_2_ema, device, normalize=True)\n feature_2 = torch.stack(list(feature_dict_2.values())).cpu()\n # average feature_1, feature_2 to create final feature\n feature = (feature_1 + feature_2) / 2\n feature = F.normalize(feature, dim=1)\n rerank_dist = utils.compute_rerank_dist(feature).numpy()\n\n print('Clustering with dbscan algorithm')\n dbscan = DBSCAN(eps=0.7, min_samples=4, metric='precomputed', n_jobs=-1)\n cluster_labels = dbscan.fit_predict(rerank_dist)\n print('Clustering finished')\n\n # generate training set with pseudo labels and calculate cluster centers\n target_train_set = []\n cluster_centers = {}\n for i, ((fname, _, cid), label) in enumerate(zip(target_dataset.train, cluster_labels)):\n if label == -1:\n continue\n target_train_set.append((fname, label, cid))\n\n if label not in cluster_centers:\n cluster_centers[label] = []\n cluster_centers[label].append(feature[i])\n\n cluster_centers = [torch.stack(cluster_centers[idx]).mean(0) for idx in sorted(cluster_centers.keys())]\n cluster_centers = torch.stack(cluster_centers)\n # normalize cluster centers\n cluster_centers = F.normalize(cluster_centers, dim=1).float().to(device)\n\n # reinitialize classifier head\n features_dim = model_1.module.features_dim\n num_clusters = len(set(cluster_labels)) - (1 if -1 in cluster_labels else 0)\n\n model_1.module.head = nn.Linear(features_dim, num_clusters, bias=False).to(device)\n model_2.module.head = nn.Linear(features_dim, num_clusters, bias=False).to(device)\n model_1_ema.module.head = nn.Linear(features_dim, num_clusters, bias=False).to(device)\n model_2_ema.module.head = nn.Linear(features_dim, num_clusters, bias=False).to(device)\n\n model_1.module.head.weight.data.copy_(cluster_centers)\n model_2.module.head.weight.data.copy_(cluster_centers)\n model_1_ema.module.head.weight.data.copy_(cluster_centers)\n model_2_ema.module.head.weight.data.copy_(cluster_centers)\n\n sampler = RandomMultipleGallerySampler(target_train_set, args.num_instances)\n train_target_loader = DataLoader(\n convert_to_pytorch_dataset(target_train_set, root=target_dataset.images_dir,\n transform=MultipleApply([train_transform, train_transform])),\n batch_size=args.batch_size, num_workers=args.workers, sampler=sampler, pin_memory=True, drop_last=True)\n train_target_iter = ForeverDataIterator(train_target_loader)\n\n return train_target_iter, num_clusters\n\n\ndef train(train_target_iter: ForeverDataIterator, model_1: DataParallel, model_1_ema: EmaTeacher, model_2: DataParallel,\n model_2_ema: EmaTeacher, optimizer: Adam, criterion_ce: CrossEntropyLossWithLabelSmooth,\n criterion_ce_soft: CrossEntropyLoss, criterion_triplet: SoftTripletLoss,\n criterion_triplet_soft: SoftTripletLoss, epoch: int, args: argparse.Namespace):\n # train with pseudo labels\n batch_time = AverageMeter('Time', ':4.2f')\n data_time = AverageMeter('Data', ':3.1f')\n # statistics for model_1\n losses_ce_1 = AverageMeter('Model_1 CELoss', ':3.2f')\n losses_triplet_1 = AverageMeter('Model_1 TripletLoss', ':3.2f')\n cls_accs_1 = AverageMeter('Model_1 Cls Acc', ':3.1f')\n # statistics for model_2\n losses_ce_2 = AverageMeter('Model_2 CELoss', ':3.2f')\n losses_triplet_2 = AverageMeter('Model_2 TripletLoss', ':3.2f')\n cls_accs_2 = AverageMeter('Model_2 Cls Acc', ':3.1f')\n\n losses_ce_soft = AverageMeter('Soft CELoss', ':3.2f')\n losses_triplet_soft = AverageMeter('Soft TripletLoss', ':3.2f')\n losses = AverageMeter('Loss', ':3.2f')\n\n progress = ProgressMeter(\n args.iters_per_epoch,\n [batch_time, data_time, losses_ce_1, losses_triplet_1, cls_accs_1, losses_ce_2, losses_triplet_2, cls_accs_2,\n losses_ce_soft, losses_triplet_soft, losses],\n prefix=\"Epoch: [{}]\".format(epoch))\n\n # switch to train mode\n model_1.train()\n model_2.train()\n model_1_ema.train()\n model_2_ema.train()\n\n end = time.time()\n\n for i in range(args.iters_per_epoch):\n # below we ignore subscript `t` and use `x_1`, `x_2` to denote different augmented versions of origin samples\n # `x_t` from target domain\n (x_1, x_2), _, labels, _ = next(train_target_iter)\n\n x_1 = x_1.to(device)\n x_2 = x_2.to(device)\n labels = labels.to(device)\n\n # measure data loading time\n data_time.update(time.time() - end)\n\n # compute output\n y_1, f_1 = model_1(x_1)\n y_2, f_2 = model_2(x_2)\n # compute output by ema-teacher\n y_1_teacher, f_1_teacher = model_1_ema(x_1)\n y_2_teacher, f_2_teacher = model_2_ema(x_2)\n\n # cross entropy loss\n loss_ce_1 = criterion_ce(y_1, labels)\n loss_ce_2 = criterion_ce(y_2, labels)\n # triplet loss\n loss_triplet_1 = criterion_triplet(f_1, f_1, labels)\n loss_triplet_2 = criterion_triplet(f_2, f_2, labels)\n # soft cross entropy loss\n loss_ce_soft = criterion_ce_soft(y_1, y_2_teacher) + \\\n criterion_ce_soft(y_2, y_1_teacher)\n # soft triplet loss\n loss_triplet_soft = criterion_triplet_soft(f_1, f_2_teacher, labels) + \\\n criterion_triplet_soft(f_2, f_1_teacher, labels)\n # final objective\n loss = (loss_ce_1 + loss_ce_2) * (1 - args.trade_off_ce_soft) + \\\n (loss_triplet_1 + loss_triplet_2) * (1 - args.trade_off_triplet_soft) + \\\n loss_ce_soft * args.trade_off_ce_soft + \\\n loss_triplet_soft * args.trade_off_triplet_soft\n\n # update statistics\n batch_size = args.batch_size\n cls_acc_1 = accuracy(y_1, labels)[0]\n cls_acc_2 = accuracy(y_2, labels)[0]\n # model 1\n losses_ce_1.update(loss_ce_1.item(), batch_size)\n losses_triplet_1.update(loss_triplet_1.item(), batch_size)\n cls_accs_1.update(cls_acc_1.item(), batch_size)\n # model 2\n losses_ce_2.update(loss_ce_2.item(), batch_size)\n losses_triplet_2.update(loss_triplet_2.item(), batch_size)\n cls_accs_2.update(cls_acc_2.item(), batch_size)\n\n losses_ce_soft.update(loss_ce_soft.item(), batch_size)\n losses_triplet_soft.update(loss_triplet_soft.item(), batch_size)\n losses.update(loss.item(), batch_size)\n\n # compute gradient and do SGD step\n optimizer.zero_grad()\n loss.backward()\n optimizer.step()\n\n # update teacher\n global_step = epoch * args.iters_per_epoch + i + 1\n model_1_ema.set_alpha(min(args.alpha, 1 - 1 / global_step))\n model_2_ema.set_alpha(min(args.alpha, 1 - 1 / global_step))\n model_1_ema.update()\n model_2_ema.update()\n\n # measure elapsed time\n batch_time.update(time.time() - end)\n end = time.time()\n\n if i % args.print_freq == 0:\n progress.display(i)\n\n\nif __name__ == '__main__':\n dataset_names = sorted(\n name for name in datasets.__dict__\n if not name.startswith(\"__\") and callable(datasets.__dict__[name])\n )\n parser = argparse.ArgumentParser(description=\"MMT for Domain Adaptative ReID\")\n # dataset parameters\n parser.add_argument('source_root', help='root path of the source dataset')\n parser.add_argument('target_root', help='root path of the target dataset')\n parser.add_argument('-s', '--source', type=str, help='source domain')\n parser.add_argument('-t', '--target', type=str, help='target domain')\n parser.add_argument('--train-resizing', type=str, default='default')\n # model parameters\n parser.add_argument('-a', '--arch', metavar='ARCH', default='reid_resnet50',\n choices=utils.get_model_names(),\n help='backbone architecture: ' +\n ' | '.join(utils.get_model_names()) +\n ' (default: reid_resnet50)')\n parser.add_argument('--num-clusters', type=int, default=500)\n parser.add_argument('--no-pool', action='store_true', help='no pool layer after the feature extractor.')\n parser.add_argument('--alpha', type=float, default=0.999, help='ema alpha')\n parser.add_argument('--finetune', action='store_true', help='whether use 10x smaller lr for backbone')\n parser.add_argument('--rate', type=float, default=0.2)\n # training parameters\n parser.add_argument('--clustering-algorithm', type=str, default='dbscan', choices=['kmeans', 'dbscan'],\n help='clustering algorithm to run, currently supported method: [\"kmeans\", \"dbscan\"]')\n parser.add_argument('--resume', type=str, default=None,\n help=\"Where restore model parameters from.\")\n parser.add_argument('--pretrained-model-1-path', type=str, help='path to pretrained (source-only) model_1')\n parser.add_argument('--pretrained-model-2-path', type=str, help='path to pretrained (source-only) model_2')\n parser.add_argument('--trade-off-ce-soft', type=float, default=0.5,\n help='the trade off hyper parameter between cross entropy loss and soft cross entropy loss')\n parser.add_argument('--trade-off-triplet-soft', type=float, default=0.8,\n help='the trade off hyper parameter between triplet loss and soft triplet loss')\n parser.add_argument('-j', '--workers', type=int, default=4)\n parser.add_argument('-b', '--batch-size', type=int, default=64)\n parser.add_argument('--height', type=int, default=256, help=\"input height\")\n parser.add_argument('--width', type=int, default=128, help=\"input width\")\n parser.add_argument('--num-instances', type=int, default=4,\n help=\"each minibatch consist of \"\n \"(batch_size // num_instances) identities, and \"\n \"each identity has num_instances instances, \"\n \"default: 4\")\n parser.add_argument('--lr', type=float, default=0.00035,\n help=\"learning rate\")\n parser.add_argument('--weight-decay', type=float, default=5e-4)\n parser.add_argument('--epochs', type=int, default=40)\n parser.add_argument('--start-epoch', default=0, type=int, help='start epoch')\n parser.add_argument('--eval-step', type=int, default=1)\n parser.add_argument('--iters-per-epoch', type=int, default=400)\n parser.add_argument('--print-freq', type=int, default=40)\n parser.add_argument('--seed', default=None, type=int, help='seed for initializing training.')\n parser.add_argument('--rerank', action='store_true', help=\"evaluation only\")\n parser.add_argument(\"--log\", type=str, default='mmt',\n help=\"Where to save logs, checkpoints and debugging images.\")\n parser.add_argument(\"--phase\", type=str, default='train', choices=['train', 'test', 'analysis'],\n help=\"When phase is 'test', only test the model.\"\n \"When phase is 'analysis', only analysis the model.\")\n args = parser.parse_args()\n main(args)\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/examples/domain_adaptation/re_identification/mmt.sh",
"content": "#!/usr/bin/env bash\n# Market1501 -> Duke\n# step1: pretrain\nCUDA_VISIBLE_DEVICES=0 python baseline.py data data -s Market1501 -t DukeMTMC -a reid_resnet50 \\\n--iters-per-epoch 800 --print-freq 80 --finetune --seed 0 --log logs/baseline/Market2DukeSeed0\nCUDA_VISIBLE_DEVICES=0 python baseline.py data data -s Market1501 -t DukeMTMC -a reid_resnet50 \\\n--iters-per-epoch 800 --print-freq 80 --finetune --seed 1 --log logs/baseline/Market2DukeSeed1\n# step2: train mmt\nCUDA_VISIBLE_DEVICES=0,1,2,3 python mmt.py data data -s Market1501 -t DukeMTMC -a reid_resnet50 \\\n--pretrained-model-1-path logs/baseline/Market2DukeSeed0/checkpoints/best.pth \\\n--pretrained-model-2-path logs/baseline/Market2DukeSeed1/checkpoints/best.pth \\\n--finetune --seed 0 --log logs/mmt/Market2Duke\n\n# Duke -> Market1501\n# step1: pretrain\nCUDA_VISIBLE_DEVICES=0 python baseline.py data data -s DukeMTMC -t Market1501 -a reid_resnet50 \\\n--iters-per-epoch 800 --print-freq 80 --finetune --seed 0 --log logs/baseline/Duke2MarketSeed0\nCUDA_VISIBLE_DEVICES=0 python baseline.py data data -s DukeMTMC -t Market1501 -a reid_resnet50 \\\n--iters-per-epoch 800 --print-freq 80 --finetune --seed 1 --log logs/baseline/Duke2MarketSeed1\n# step2: train mmt\nCUDA_VISIBLE_DEVICES=0,1,2,3 python mmt.py data data -s DukeMTMC -t Market1501 -a reid_resnet50 \\\n--pretrained-model-1-path logs/baseline/Duke2MarketSeed0/checkpoints/best.pth \\\n--pretrained-model-2-path logs/baseline/Duke2MarketSeed1/checkpoints/best.pth \\\n--finetune --seed 0 --log logs/mmt/Duke2Market\n\n# Market1501 -> MSMT\n# step1: pretrain\nCUDA_VISIBLE_DEVICES=0 python baseline.py data data -s Market1501 -t MSMT17 -a reid_resnet50 \\\n--iters-per-epoch 800 --print-freq 80 --finetune --seed 0 --log logs/baseline/Market2MSMTSeed0\nCUDA_VISIBLE_DEVICES=0 python baseline.py data data -s Market1501 -t MSMT17 -a reid_resnet50 \\\n--iters-per-epoch 800 --print-freq 80 --finetune --seed 1 --log logs/baseline/Market2MSMTSeed1\n# step2: train mmt\nCUDA_VISIBLE_DEVICES=0,1,2,3 python mmt.py data data -s Market1501 -t MSMT17 -a reid_resnet50 \\\n--pretrained-model-1-path logs/baseline/Market2MSMTSeed0/checkpoints/best.pth \\\n--pretrained-model-2-path logs/baseline/Market2MSMTSeed1/checkpoints/best.pth \\\n--num-clusters 1000 --finetune --seed 0 --log logs/mmt/Market2MSMT\n\n# MSMT -> Market1501\n# step1: pretrain\nCUDA_VISIBLE_DEVICES=0 python baseline.py data data -s MSMT17 -t Market1501 -a reid_resnet50 \\\n--iters-per-epoch 800 --print-freq 80 --finetune --seed 0 --log logs/baseline/MSMT2MarketSeed0\nCUDA_VISIBLE_DEVICES=0 python baseline.py data data -s MSMT17 -t Market1501 -a reid_resnet50 \\\n--iters-per-epoch 800 --print-freq 80 --finetune --seed 1 --log logs/baseline/MSMT2MarketSeed1\n# step2: train mmt\nCUDA_VISIBLE_DEVICES=0,1,2,3 python mmt.py data data -s MSMT17 -t Market1501 -a reid_resnet50 \\\n--pretrained-model-1-path logs/baseline/MSMT2MarketSeed0/checkpoints/best.pth \\\n--pretrained-model-2-path logs/baseline/MSMT2MarketSeed1/checkpoints/best.pth \\\n--finetune --seed 0 --log logs/mmt/MSMT2Market\n\n# Duke -> MSMT\n# step1: pretrain\nCUDA_VISIBLE_DEVICES=0 python baseline.py data data -s DukeMTMC -t MSMT17 -a reid_resnet50 \\\n--iters-per-epoch 800 --print-freq 80 --finetune --seed 0 --log logs/baseline/Duke2MSMTSeed0\nCUDA_VISIBLE_DEVICES=0 python baseline.py data data -s DukeMTMC -t MSMT17 -a reid_resnet50 \\\n--iters-per-epoch 800 --print-freq 80 --finetune --seed 1 --log logs/baseline/Duke2MSMTSeed1\n# step2: train mmt\nCUDA_VISIBLE_DEVICES=0,1,2,3 python mmt.py data data -s DukeMTMC -t MSMT17 -a reid_resnet50 \\\n--pretrained-model-1-path logs/baseline/Duke2MSMTSeed0/checkpoints/best.pth \\\n--pretrained-model-2-path logs/baseline/Duke2MSMTSeed1/checkpoints/best.pth \\\n--num-clusters 1000 --finetune --seed 0 --log logs/mmt/Duke2MSMT\n\n# MSMT -> Duke\n# step1: pretrain\nCUDA_VISIBLE_DEVICES=0 python baseline.py data data -s MSMT17 -t DukeMTMC -a reid_resnet50 \\\n--iters-per-epoch 800 --print-freq 80 --finetune --seed 0 --log logs/baseline/MSMT2DukeSeed0\nCUDA_VISIBLE_DEVICES=0 python baseline.py data data -s MSMT17 -t DukeMTMC -a reid_resnet50 \\\n--iters-per-epoch 800 --print-freq 80 --finetune --seed 1 --log logs/baseline/MSMT2DukeSeed1\n# step2: train mmt\nCUDA_VISIBLE_DEVICES=0,1,2,3 python mmt.py data data -s MSMT17 -t DukeMTMC -a reid_resnet50 \\\n--pretrained-model-1-path logs/baseline/MSMT2DukeSeed0/checkpoints/best.pth \\\n--pretrained-model-2-path logs/baseline/MSMT2DukeSeed1/checkpoints/best.pth \\\n--finetune --seed 0 --log logs/mmt/MSMT2Duke\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/examples/domain_adaptation/re_identification/requirements.txt",
"content": "timm\nopencv-python"
},
{
"path": "domain_generalization/Transfer-Learning-Library/examples/domain_adaptation/re_identification/spgan.py",
"content": "\"\"\"\n@author: Baixu Chen\n@contact: cbx_99_hasta@outlook.com\n\"\"\"\nimport random\nimport time\nimport warnings\nimport sys\nimport argparse\nimport itertools\nimport os.path as osp\nfrom PIL import Image\n\nimport torch\nimport torch.nn as nn\nimport torch.backends.cudnn as cudnn\nfrom torch.optim import Adam\nfrom torch.optim.lr_scheduler import LambdaLR\nfrom torch.utils.data import DataLoader\nimport torchvision.transforms as T\n\nsys.path.append('../../..')\nimport dalib.translation.cyclegan as cyclegan\nimport dalib.translation.spgan as spgan\nfrom dalib.translation.cyclegan.util import ImagePool, set_requires_grad\nimport common.vision.datasets.reid as datasets\nfrom common.vision.datasets.reid.convert import convert_to_pytorch_dataset\nfrom common.vision.transforms import Denormalize\nfrom common.utils.data import ForeverDataIterator\nfrom common.utils.meter import AverageMeter, ProgressMeter\nfrom common.utils.logger import CompleteLogger\n\ndevice = torch.device(\"cuda\" if torch.cuda.is_available() else \"cpu\")\n\n\ndef main(args):\n logger = CompleteLogger(args.log, args.phase)\n print(args)\n\n if args.seed is not None:\n random.seed(args.seed)\n torch.manual_seed(args.seed)\n cudnn.deterministic = True\n warnings.warn('You have chosen to seed training. '\n 'This will turn on the CUDNN deterministic setting, '\n 'which can slow down your training considerably! '\n 'You may see unexpected behavior when restarting '\n 'from checkpoints.')\n\n cudnn.benchmark = True\n\n # Data loading code\n train_transform = T.Compose([\n T.Resize(args.load_size, Image.BICUBIC),\n T.RandomCrop(args.input_size),\n T.RandomHorizontalFlip(),\n T.ToTensor(),\n T.Normalize((0.5, 0.5, 0.5), (0.5, 0.5, 0.5))\n ])\n\n working_dir = osp.dirname(osp.abspath(__file__))\n root = osp.join(working_dir, args.root)\n\n source_dataset = datasets.__dict__[args.source](root=osp.join(root, args.source.lower()))\n train_source_loader = DataLoader(\n convert_to_pytorch_dataset(source_dataset.train, root=source_dataset.images_dir, transform=train_transform),\n batch_size=args.batch_size, num_workers=args.workers, shuffle=True, pin_memory=True, drop_last=True)\n\n target_dataset = datasets.__dict__[args.target](root=osp.join(root, args.target.lower()))\n train_target_loader = DataLoader(\n convert_to_pytorch_dataset(target_dataset.train, root=target_dataset.images_dir, transform=train_transform),\n batch_size=args.batch_size, num_workers=args.workers, shuffle=True, pin_memory=True, drop_last=True)\n\n train_source_iter = ForeverDataIterator(train_source_loader)\n train_target_iter = ForeverDataIterator(train_target_loader)\n\n # define networks (generators, discriminators and siamese network)\n netG_S2T = cyclegan.generator.__dict__[args.netG](ngf=args.ngf, norm=args.norm, use_dropout=False).to(device)\n netG_T2S = cyclegan.generator.__dict__[args.netG](ngf=args.ngf, norm=args.norm, use_dropout=False).to(device)\n netD_S = cyclegan.discriminator.__dict__[args.netD](ndf=args.ndf, norm=args.norm).to(device)\n netD_T = cyclegan.discriminator.__dict__[args.netD](ndf=args.ndf, norm=args.norm).to(device)\n siamese_net = spgan.SiameseNetwork(nsf=args.nsf).to(device)\n\n # create image buffer to store previously generated images\n fake_S_pool = ImagePool(args.pool_size)\n fake_T_pool = ImagePool(args.pool_size)\n\n # define optimizer and lr scheduler\n optimizer_G = Adam(itertools.chain(netG_S2T.parameters(), netG_T2S.parameters()), lr=args.lr,\n betas=(args.beta1, 0.999))\n optimizer_D = Adam(itertools.chain(netD_S.parameters(), netD_T.parameters()), lr=args.lr, betas=(args.beta1, 0.999))\n optimizer_siamese = Adam(siamese_net.parameters(), lr=args.lr, betas=(args.beta1, 0.999))\n\n lr_decay_function = lambda epoch: 1.0 - max(0, epoch - args.epochs) / float(args.epochs_decay)\n lr_scheduler_G = LambdaLR(optimizer_G, lr_lambda=lr_decay_function)\n lr_scheduler_D = LambdaLR(optimizer_D, lr_lambda=lr_decay_function)\n lr_scheduler_siamese = LambdaLR(optimizer_siamese, lr_lambda=lr_decay_function)\n\n # optionally resume from a checkpoint\n if args.resume:\n print(\"Resume from\", args.resume)\n checkpoint = torch.load(args.resume, map_location='cpu')\n\n netG_S2T.load_state_dict(checkpoint['netG_S2T'])\n netG_T2S.load_state_dict(checkpoint['netG_T2S'])\n netD_S.load_state_dict(checkpoint['netD_S'])\n netD_T.load_state_dict(checkpoint['netD_T'])\n siamese_net.load_state_dict(checkpoint['siamese_net'])\n\n optimizer_G.load_state_dict(checkpoint['optimizer_G'])\n optimizer_D.load_state_dict(checkpoint['optimizer_D'])\n optimizer_siamese.load_state_dict(checkpoint['optimizer_siamese'])\n lr_scheduler_G.load_state_dict(checkpoint['lr_scheduler_G'])\n lr_scheduler_D.load_state_dict(checkpoint['lr_scheduler_D'])\n lr_scheduler_siamese.load_state_dict(checkpoint['lr_scheduler_siamese'])\n\n args.start_epoch = checkpoint['epoch'] + 1\n\n if args.phase == 'test':\n transform = T.Compose([\n T.Resize(args.test_input_size, Image.BICUBIC),\n cyclegan.transform.Translation(netG_S2T, device)\n ])\n source_dataset.translate(transform, osp.join(args.translated_root, args.source.lower()))\n return\n\n # define loss function\n criterion_gan = cyclegan.LeastSquaresGenerativeAdversarialLoss()\n criterion_cycle = nn.L1Loss()\n criterion_identity = nn.L1Loss()\n criterion_contrastive = spgan.ContrastiveLoss(margin=args.margin)\n\n # define visualization function\n tensor_to_image = T.Compose([\n Denormalize((0.5, 0.5, 0.5), (0.5, 0.5, 0.5)),\n T.ToPILImage()\n ])\n\n def visualize(image, name):\n \"\"\"\n Args:\n image (tensor): image in shape 3 x H x W\n name: name of the saving image\n \"\"\"\n tensor_to_image(image).save(logger.get_image_path(\"{}.png\".format(name)))\n\n # start training\n for epoch in range(args.start_epoch, args.epochs + args.epochs_decay):\n logger.set_epoch(epoch)\n print(lr_scheduler_G.get_lr())\n\n # train for one epoch\n train(train_source_iter, train_target_iter, netG_S2T, netG_T2S, netD_S, netD_T, siamese_net,\n criterion_gan, criterion_cycle, criterion_identity, criterion_contrastive,\n optimizer_G, optimizer_D, optimizer_siamese,\n fake_S_pool, fake_T_pool, epoch, visualize, args)\n\n # update learning rates\n lr_scheduler_G.step()\n lr_scheduler_D.step()\n lr_scheduler_siamese.step()\n\n # save checkpoint\n torch.save(\n {\n 'netG_S2T': netG_S2T.state_dict(),\n 'netG_T2S': netG_T2S.state_dict(),\n 'netD_S': netD_S.state_dict(),\n 'netD_T': netD_T.state_dict(),\n 'siamese_net': siamese_net.state_dict(),\n 'optimizer_G': optimizer_G.state_dict(),\n 'optimizer_D': optimizer_D.state_dict(),\n 'optimizer_siamese': optimizer_siamese.state_dict(),\n 'lr_scheduler_G': lr_scheduler_G.state_dict(),\n 'lr_scheduler_D': lr_scheduler_D.state_dict(),\n 'lr_scheduler_siamese': lr_scheduler_siamese.state_dict(),\n 'epoch': epoch,\n 'args': args\n }, logger.get_checkpoint_path(epoch)\n )\n\n if args.translated_root is not None:\n transform = T.Compose([\n T.Resize(args.test_input_size, Image.BICUBIC),\n cyclegan.transform.Translation(netG_S2T, device)\n ])\n source_dataset.translate(transform, osp.join(args.translated_root, args.source.lower()))\n\n logger.close()\n\n\ndef train(train_source_iter: ForeverDataIterator, train_target_iter: ForeverDataIterator,\n netG_S2T, netG_T2S, netD_S, netD_T, siamese_net: spgan.SiameseNetwork,\n criterion_gan: cyclegan.LeastSquaresGenerativeAdversarialLoss,\n criterion_cycle: nn.L1Loss, criterion_identity: nn.L1Loss,\n criterion_contrastive: spgan.ContrastiveLoss,\n optimizer_G: Adam, optimizer_D: Adam, optimizer_siamese: Adam,\n fake_S_pool: ImagePool, fake_T_pool: ImagePool, epoch: int, visualize, args: argparse.Namespace):\n batch_time = AverageMeter('Time', ':4.2f')\n data_time = AverageMeter('Data', ':3.1f')\n losses_G_S2T = AverageMeter('G_S2T', ':3.2f')\n losses_G_T2S = AverageMeter('G_T2S', ':3.2f')\n losses_D_S = AverageMeter('D_S', ':3.2f')\n losses_D_T = AverageMeter('D_T', ':3.2f')\n losses_cycle_S = AverageMeter('cycle_S', ':3.2f')\n losses_cycle_T = AverageMeter('cycle_T', ':3.2f')\n losses_identity_S = AverageMeter('idt_S', ':3.2f')\n losses_identity_T = AverageMeter('idt_T', ':3.2f')\n losses_contrastive_G = AverageMeter('contrastive_G', ':3.2f')\n losses_contrastive_siamese = AverageMeter('contrastive_siamese', ':3.2f')\n\n progress = ProgressMeter(\n args.iters_per_epoch,\n [batch_time, data_time, losses_G_S2T, losses_G_T2S, losses_D_S, losses_D_T,\n losses_cycle_S, losses_cycle_T, losses_identity_S, losses_identity_T,\n losses_contrastive_G, losses_contrastive_siamese],\n prefix=\"Epoch: [{}]\".format(epoch))\n\n end = time.time()\n\n for i in range(args.iters_per_epoch):\n real_S, _, _, _ = next(train_source_iter)\n real_T, _, _, _ = next(train_target_iter)\n\n real_S = real_S.to(device)\n real_T = real_T.to(device)\n\n # measure data loading time\n data_time.update(time.time() - end)\n\n # Compute fake images and reconstruction images.\n fake_T = netG_S2T(real_S)\n rec_S = netG_T2S(fake_T)\n fake_S = netG_T2S(real_T)\n rec_T = netG_S2T(fake_S)\n\n # ===============================================\n # train the generators (every two iterations)\n # ===============================================\n if i % 2 == 0:\n # save memory\n set_requires_grad(netD_S, False)\n set_requires_grad(netD_T, False)\n set_requires_grad(siamese_net, False)\n # GAN loss D_T(G_S2T(S))\n loss_G_S2T = criterion_gan(netD_T(fake_T), real=True)\n # GAN loss D_S(G_T2S(B))\n loss_G_T2S = criterion_gan(netD_S(fake_S), real=True)\n # Cycle loss || G_T2S(G_S2T(S)) - S||\n loss_cycle_S = criterion_cycle(rec_S, real_S) * args.trade_off_cycle\n # Cycle loss || G_S2T(G_T2S(T)) - T||\n loss_cycle_T = criterion_cycle(rec_T, real_T) * args.trade_off_cycle\n # Identity loss\n # G_S2T should be identity if real_T is fed: ||G_S2T(real_T) - real_T||\n identity_T = netG_S2T(real_T)\n loss_identity_T = criterion_identity(identity_T, real_T) * args.trade_off_identity\n # G_T2S should be identity if real_S is fed: ||G_T2S(real_S) - real_S||\n identity_S = netG_T2S(real_S)\n loss_identity_S = criterion_identity(identity_S, real_S) * args.trade_off_identity\n\n # siamese network output\n f_real_S = siamese_net(real_S)\n f_fake_T = siamese_net(fake_T)\n f_real_T = siamese_net(real_T)\n f_fake_S = siamese_net(fake_S)\n\n # positive pair\n loss_contrastive_p_G = criterion_contrastive(f_real_S, f_fake_T, 0) + \\\n criterion_contrastive(f_real_T, f_fake_S, 0)\n # negative pair\n loss_contrastive_n_G = criterion_contrastive(f_fake_T, f_real_T, 1) + \\\n criterion_contrastive(f_fake_S, f_real_S, 1) + \\\n criterion_contrastive(f_real_S, f_real_T, 1)\n # contrastive loss\n loss_contrastive_G = (loss_contrastive_p_G + 0.5 * loss_contrastive_n_G) / 4 * args.trade_off_contrastive\n\n # combined loss and calculate gradients\n loss_G = loss_G_S2T + loss_G_T2S + loss_cycle_S + loss_cycle_T + loss_identity_S + loss_identity_T\n if epoch > 1:\n loss_G += loss_contrastive_G\n netG_S2T.zero_grad()\n netG_T2S.zero_grad()\n loss_G.backward()\n optimizer_G.step()\n\n # update corresponding statistics\n losses_G_S2T.update(loss_G_S2T.item(), real_S.size(0))\n losses_G_T2S.update(loss_G_T2S.item(), real_S.size(0))\n losses_cycle_S.update(loss_cycle_S.item(), real_S.size(0))\n losses_cycle_T.update(loss_cycle_T.item(), real_S.size(0))\n losses_identity_S.update(loss_identity_S.item(), real_S.size(0))\n losses_identity_T.update(loss_identity_T.item(), real_S.size(0))\n if epoch > 1:\n losses_contrastive_G.update(loss_contrastive_G, real_S.size(0))\n\n # ===============================================\n # train the siamese network (when epoch > 0)\n # ===============================================\n if epoch > 0:\n set_requires_grad(siamese_net, True)\n # siamese network output\n f_real_S = siamese_net(real_S)\n f_fake_T = siamese_net(fake_T.detach())\n f_real_T = siamese_net(real_T)\n f_fake_S = siamese_net(fake_S.detach())\n\n # positive pair\n loss_contrastive_p_siamese = criterion_contrastive(f_real_S, f_fake_T, 0) + \\\n criterion_contrastive(f_real_T, f_fake_S, 0)\n # negative pair\n loss_contrastive_n_siamese = criterion_contrastive(f_real_S, f_real_T, 1)\n # contrastive loss\n loss_contrastive_siamese = (loss_contrastive_p_siamese + 2 * loss_contrastive_n_siamese) / 3\n\n # update siamese network\n siamese_net.zero_grad()\n loss_contrastive_siamese.backward()\n optimizer_siamese.step()\n\n # update corresponding statistics\n losses_contrastive_siamese.update(loss_contrastive_siamese, real_S.size(0))\n\n # ===============================================\n # train the discriminators\n # ===============================================\n\n set_requires_grad(netD_S, True)\n set_requires_grad(netD_T, True)\n # Calculate GAN loss for discriminator D_S\n fake_S_ = fake_S_pool.query(fake_S.detach())\n loss_D_S = 0.5 * (criterion_gan(netD_S(real_S), True) + criterion_gan(netD_S(fake_S_), False))\n # Calculate GAN loss for discriminator D_T\n fake_T_ = fake_T_pool.query(fake_T.detach())\n loss_D_T = 0.5 * (criterion_gan(netD_T(real_T), True) + criterion_gan(netD_T(fake_T_), False))\n\n # update discriminators\n netD_S.zero_grad()\n netD_T.zero_grad()\n loss_D_S.backward()\n loss_D_T.backward()\n optimizer_D.step()\n\n # update corresponding statistics\n losses_D_S.update(loss_D_S.item(), real_S.size(0))\n losses_D_T.update(loss_D_T.item(), real_S.size(0))\n\n # measure elapsed time\n batch_time.update(time.time() - end)\n end = time.time()\n\n if i % args.print_freq == 0:\n progress.display(i)\n\n for tensor, name in zip([real_S, real_T, fake_S, fake_T, rec_S, rec_T, identity_S, identity_T],\n [\"real_S\", \"real_T\", \"fake_S\", \"fake_T\", \"rec_S\",\n \"rec_T\", \"identity_S\", \"identity_T\"]):\n visualize(tensor[0], \"{}_{}\".format(i, name))\n\n\nif __name__ == '__main__':\n dataset_names = sorted(\n name for name in datasets.__dict__\n if not name.startswith(\"__\") and callable(datasets.__dict__[name])\n )\n parser = argparse.ArgumentParser(description='SPGAN for Domain Adaptative ReID')\n # dataset parameters\n parser.add_argument('root', metavar='DIR',\n help='root path of dataset')\n parser.add_argument('-s', '--source', type=str, help='source domain')\n parser.add_argument('-t', '--target', type=str, help='target domain')\n parser.add_argument('--load-size', nargs='+', type=int, default=(286, 144), help='loading image size')\n parser.add_argument('--input-size', nargs='+', type=int, default=(256, 128),\n help='the input and output image size during training')\n # model parameters\n parser.add_argument('--ngf', type=int, default=64, help='# of gen filters in the last conv layer')\n parser.add_argument('--ndf', type=int, default=64, help='# of discrim filters in the first conv layer')\n parser.add_argument('--nsf', type=int, default=64, help='# of sianet filters int the first conv layer')\n parser.add_argument('--netD', type=str, default='patch',\n help='specify discriminator architecture [patch | pixel]. The basic model is a 70x70 PatchGAN.')\n parser.add_argument('--netG', type=str, default='resnet_9',\n help='specify generator architecture [resnet_9 | resnet_6 | unet_256 | unet_128]')\n parser.add_argument('--norm', type=str, default='instance',\n help='instance normalization or batch normalization [instance | batch | none]')\n # training parameters\n parser.add_argument(\"--resume\", type=str, default=None,\n help=\"Where restore model parameters from.\")\n parser.add_argument('--trade-off-cycle', type=float, default=10.0, help='trade off for cycle loss')\n parser.add_argument('--trade-off-identity', type=float, default=5.0, help='trade off for identity loss')\n parser.add_argument('--trade-off-contrastive', type=float, default=2.0, help='trade off for contrastive loss')\n parser.add_argument('--margin', type=float, default=2,\n help='margin for contrastive loss')\n parser.add_argument('-b', '--batch-size', default=8, type=int,\n metavar='N', help='mini-batch size (default: 8)')\n parser.add_argument('--lr', type=float, default=0.0002, help='initial learning rate for adam')\n parser.add_argument('--beta1', type=float, default=0.5, help='momentum term of adam')\n parser.add_argument('-j', '--workers', default=4, type=int, metavar='N',\n help='number of data loading workers (default: 4)')\n parser.add_argument('--epochs', default=15, type=int, metavar='N',\n help='number of total epochs to run')\n parser.add_argument('--epochs-decay', type=int, default=15,\n help='number of epochs to linearly decay learning rate to zero')\n parser.add_argument('--start-epoch', default=0, type=int, metavar='N',\n help='start epoch')\n parser.add_argument('-i', '--iters-per-epoch', default=2000, type=int,\n help='Number of iterations per epoch')\n parser.add_argument('--pool-size', type=int, default=50,\n help='the size of image buffer that stores previously generated images')\n parser.add_argument('-p', '--print-freq', default=500, type=int,\n metavar='N', help='print frequency (default: 500)')\n parser.add_argument('--seed', default=None, type=int,\n help='seed for initializing training. ')\n parser.add_argument(\"--log\", type=str, default='spgan',\n help=\"Where to save logs, checkpoints and debugging images.\")\n # test parameters\n parser.add_argument(\"--phase\", type=str, default='train', choices=['train', 'test'],\n help=\"When phase is 'test', only test the model.\")\n parser.add_argument('--translated-root', type=str, default=None,\n help=\"The root to put the translated dataset\")\n parser.add_argument('--test-input-size', nargs='+', type=int, default=(256, 128),\n help='the input image size during testing')\n args = parser.parse_args()\n main(args)\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/examples/domain_adaptation/re_identification/spgan.sh",
"content": "# Market1501 -> Duke\n# step1: train SPGAN\nCUDA_VISIBLE_DEVICES=0 python spgan.py data -s Market1501 -t DukeMTMC \\\n--log logs/spgan/Market2Duke --translated-root data/spganM2D --seed 0\n# step2: train baseline on translated source dataset\nCUDA_VISIBLE_DEVICES=0 python baseline.py data/spganM2D data -s Market1501 -t DukeMTMC -a reid_resnet50 \\\n--iters-per-epoch 800 --print-freq 80 --finetune --seed 0 --log logs/spgan/Market2Duke\n\n# Duke -> Market1501\n# step1: train SPGAN\nCUDA_VISIBLE_DEVICES=0 python spgan.py data -s DukeMTMC -t Market1501 \\\n--log logs/spgan/Duke2Market --translated-root data/spganD2M --seed 0\n# step2: train baseline on translated source dataset\nCUDA_VISIBLE_DEVICES=0 python baseline.py data/spganD2M data -s DukeMTMC -t Market1501 -a reid_resnet50 \\\n--iters-per-epoch 800 --print-freq 80 --finetune --seed 0 --log logs/spgan/Duke2Market\n\n# Market1501 -> MSMT17\n# step1: train SPGAN\nCUDA_VISIBLE_DEVICES=0 python spgan.py data -s Market1501 -t MSMT17 \\\n--log logs/spgan/Market2MSMT --translated-root data/spganM2S --seed 0\n# step2: train baseline on translated source dataset\nCUDA_VISIBLE_DEVICES=0 python baseline.py data/spganM2S data -s Market1501 -t MSMT17 -a reid_resnet50 \\\n--iters-per-epoch 800 --print-freq 80 --finetune --seed 0 --log logs/spgan/Market2MSMT\n\n# MSMT -> Market1501\n# step1: train SPGAN\nCUDA_VISIBLE_DEVICES=0 python spgan.py data -s MSMT17 -t Market1501 \\\n--log logs/spgan/MSMT2Market --translated-root data/spganS2M --seed 0\n# step2: train baseline on translated source dataset\nCUDA_VISIBLE_DEVICES=0 python baseline.py data/spganS2M data -s MSMT17 -t Market1501 -a reid_resnet50 \\\n--iters-per-epoch 800 --print-freq 80 --finetune --seed 0 --log logs/spgan/MSMT2Market\n\n# Duke -> MSMT\n# step1: train SPGAN\nCUDA_VISIBLE_DEVICES=0 python spgan.py data -s DukeMTMC -t MSMT17 \\\n--log logs/spgan/Duke2MSMT --translated-root data/spganD2S --seed 0\n# step2: train baseline on translated source dataset\nCUDA_VISIBLE_DEVICES=0 python baseline.py data/spganD2S data -s DukeMTMC -t MSMT17 -a reid_resnet50 \\\n--iters-per-epoch 800 --print-freq 80 --finetune --seed 0 --log logs/spgan/Duke2MSMT\n\n# MSMT -> Duke\n# step1: train SPGAN\nCUDA_VISIBLE_DEVICES=0 python spgan.py data -s MSMT17 -t DukeMTMC \\\n--log logs/spgan/MSMT2Duke --translated-root data/spganS2D --seed 0\n# step2: train baseline on translated source dataset\nCUDA_VISIBLE_DEVICES=0 python baseline.py data/spganS2D data -s MSMT17 -t DukeMTMC -a reid_resnet50 \\\n--iters-per-epoch 800 --print-freq 80 --finetune --seed 0 --log logs/spgan/MSMT2Duke\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/examples/domain_adaptation/re_identification/utils.py",
"content": "\"\"\"\n@author: Baixu Chen\n@contact: cbx_99_hasta@outlook.com\n\"\"\"\nimport sys\nimport timm\nimport numpy as np\nimport torch\nimport torch.nn as nn\nfrom torch.nn import Parameter\nimport torchvision.transforms as T\n\nsys.path.append('../../..')\nfrom common.utils.metric.reid import extract_reid_feature\nfrom common.utils.analysis import tsne\nfrom common.vision.transforms import RandomErasing\nimport common.vision.models.reid as models\n\n\ndef copy_state_dict(model, state_dict, strip=None):\n \"\"\"Copy state dict into the passed in ReID model. As we are using classification loss, which means we need to output\n different number of classes(identities) for different datasets, we will not copy the parameters of last `fc` layer.\n \"\"\"\n tgt_state = model.state_dict()\n copied_names = set()\n for name, param in state_dict.items():\n if strip is not None and name.startswith(strip):\n name = name[len(strip):]\n if name not in tgt_state:\n continue\n if isinstance(param, Parameter):\n param = param.data\n if param.size() != tgt_state[name].size():\n print('mismatch:', name, param.size(), tgt_state[name].size())\n continue\n tgt_state[name].copy_(param)\n copied_names.add(name)\n\n missing = set(tgt_state.keys()) - copied_names\n if len(missing) > 0:\n print(\"missing keys in state_dict:\", missing)\n\n return model\n\n\ndef get_model_names():\n return sorted(\n name for name in models.__dict__\n if name.islower() and not name.startswith(\"__\")\n and callable(models.__dict__[name])\n ) + timm.list_models()\n\n\ndef get_model(model_name):\n if model_name in models.__dict__:\n # load models from common.vision.models\n backbone = models.__dict__[model_name](pretrained=True)\n else:\n # load models from pytorch-image-models\n backbone = timm.create_model(model_name, pretrained=True)\n try:\n backbone.out_features = backbone.get_classifier().in_features\n backbone.reset_classifier(0, '')\n except:\n backbone.out_features = backbone.head.in_features\n backbone.head = nn.Identity()\n return backbone\n\n\ndef get_train_transform(height, width, resizing='default', random_horizontal_flip=True, random_color_jitter=False,\n random_gray_scale=False, random_erasing=False):\n \"\"\"\n resizing mode:\n - default: resize the image to (height, width), zero-pad it by 10 on each size, the take a random crop of\n (height, width)\n - res: resize the image to(height, width)\n \"\"\"\n if resizing == 'default':\n transform = T.Compose([\n T.Resize((height, width), interpolation=3),\n T.Pad(10),\n T.RandomCrop((height, width))\n ])\n elif resizing == 'res':\n transform = T.Resize((height, width), interpolation=3)\n else:\n raise NotImplementedError(resizing)\n transforms = [transform]\n if random_horizontal_flip:\n transforms.append(T.RandomHorizontalFlip())\n if random_color_jitter:\n transforms.append(T.ColorJitter(brightness=0.3, contrast=0.3, saturation=0.3, hue=0.3))\n if random_gray_scale:\n transforms.append(T.RandomGrayscale())\n transforms.extend([\n T.ToTensor(),\n T.Normalize(mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225])\n ])\n if random_erasing:\n transforms.append(RandomErasing(probability=0.5, mean=[0.485, 0.456, 0.406]))\n return T.Compose(transforms)\n\n\ndef get_val_transform(height, width):\n return T.Compose([\n T.Resize((height, width), interpolation=3),\n T.ToTensor(),\n T.Normalize(mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225])\n ])\n\n\ndef visualize_tsne(source_loader, target_loader, model, filename, device, n_data_points_per_domain=3000):\n \"\"\"Visualize features from different domains using t-SNE. As we can have very large number of samples in each\n domain, only `n_data_points_per_domain` number of samples are randomly selected in each domain.\n \"\"\"\n source_feature_dict = extract_reid_feature(source_loader, model, device, normalize=True)\n source_feature = torch.stack(list(source_feature_dict.values())).cpu()\n source_feature = source_feature[torch.randperm(len(source_feature))]\n source_feature = source_feature[:n_data_points_per_domain]\n\n target_feature_dict = extract_reid_feature(target_loader, model, device, normalize=True)\n target_feature = torch.stack(list(target_feature_dict.values())).cpu()\n target_feature = target_feature[torch.randperm(len(target_feature))]\n target_feature = target_feature[:n_data_points_per_domain]\n\n tsne.visualize(source_feature, target_feature, filename, source_color='cornflowerblue', target_color='darkorange')\n print('T-SNE process is done, figure is saved to {}'.format(filename))\n\n\ndef k_reciprocal_neigh(initial_rank, i, k1):\n \"\"\"Compute k-reciprocal neighbors of i-th sample. Two samples f_i, f_j are k reciprocal-neighbors if and only if\n each one of them is among the k-nearest samples of another sample.\n \"\"\"\n forward_k_neigh_index = initial_rank[i, :k1 + 1]\n backward_k_neigh_index = initial_rank[forward_k_neigh_index, :k1 + 1]\n fi = torch.nonzero(backward_k_neigh_index == i)[:, 0]\n return forward_k_neigh_index[fi]\n\n\ndef compute_rerank_dist(target_features, k1=30, k2=6):\n \"\"\"Compute distance according to `Re-ranking Person Re-identification with k-reciprocal Encoding\n (CVPR 2017) `_.\n \"\"\"\n n = target_features.size(0)\n original_dist = torch.pow(target_features, 2).sum(dim=1, keepdim=True) * 2\n original_dist = original_dist.expand(n, n) - 2 * torch.mm(target_features, target_features.t())\n original_dist /= original_dist.max(0)[0]\n original_dist = original_dist.t()\n initial_rank = torch.argsort(original_dist, dim=-1)\n all_num = gallery_num = original_dist.size(0)\n\n del target_features\n\n nn_k1 = []\n nn_k1_half = []\n for i in range(all_num):\n nn_k1.append(k_reciprocal_neigh(initial_rank, i, k1))\n nn_k1_half.append(k_reciprocal_neigh(initial_rank, i, int(np.around(k1 / 2))))\n\n V = torch.zeros(all_num, all_num)\n for i in range(all_num):\n k_reciprocal_index = nn_k1[i]\n k_reciprocal_expansion_index = k_reciprocal_index\n for candidate in k_reciprocal_index:\n candidate_k_reciprocal_index = nn_k1_half[candidate]\n if (len(np.intersect1d(candidate_k_reciprocal_index, k_reciprocal_index)) > 2 / 3 * len(\n candidate_k_reciprocal_index)):\n k_reciprocal_expansion_index = torch.cat((k_reciprocal_expansion_index, candidate_k_reciprocal_index))\n\n k_reciprocal_expansion_index = torch.unique(k_reciprocal_expansion_index)\n weight = torch.exp(-original_dist[i, k_reciprocal_expansion_index])\n V[i, k_reciprocal_expansion_index] = weight / torch.sum(weight)\n\n if k2 != 1:\n k2_rank = initial_rank[:, :k2].clone().view(-1)\n V_qe = V[k2_rank]\n V_qe = V_qe.view(initial_rank.size(0), k2, -1).sum(1)\n V_qe /= k2\n V = V_qe\n del V_qe\n del initial_rank\n\n invIndex = []\n for i in range(gallery_num):\n invIndex.append(torch.nonzero(V[:, i])[:, 0])\n\n jaccard_dist = torch.zeros_like(original_dist)\n for i in range(all_num):\n temp_min = torch.zeros(1, gallery_num)\n indNonZero = torch.nonzero(V[i, :])[:, 0]\n indImages = [invIndex[ind] for ind in indNonZero]\n for j in range(len(indNonZero)):\n temp_min[0, indImages[j]] = temp_min[0, indImages[j]] + \\\n torch.min(V[i, indNonZero[j]], V[indImages[j], indNonZero[j]])\n jaccard_dist[i] = 1 - temp_min / (2 - temp_min)\n del invIndex\n del V\n\n pos_bool = (jaccard_dist < 0)\n jaccard_dist[pos_bool] = 0.0\n return jaccard_dist\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/examples/domain_adaptation/semantic_segmentation/README.md",
"content": "# Unsupervised Domain Adaptation for Semantic Segmentation\n\n## Dataset\n\nYou need to prepare following datasets manually if you want to use them:\n- [Cityscapes](https://www.cityscapes-dataset.com/)\n- [GTA5](https://download.visinf.tu-darmstadt.de/data/from_games/)\n- [Synthia](https://synthia-dataset.net/)\n\nand prepare them following [Documentations for Cityscapes](/common/vision/datasets/segmentation/cityscapes.py), [GTA5](/common/vision/datasets/segmentation/gta5.py) and [Synthia](/common/vision/datasets/segmentation/synthia.py), \n\n## Supported Methods\n\nSupported methods include:\n\n- [Cycle-Consistent Adversarial Networks (CycleGAN)](https://arxiv.org/pdf/1703.10593.pdf)\n- [CyCADA: Cycle-Consistent Adversarial Domain Adaptation](https://arxiv.org/abs/1711.03213)\n- [Adversarial Entropy Minimization (ADVENT)](https://arxiv.org/abs/1811.12833)\n- [Fourier Domain Adaptation (FDA)](https://arxiv.org/abs/2004.05498)\n\n## Experiment and Results\n\nThe shell files give the script to reproduce the [benchmarks](/docs/dalib/benchmarks/semantic_segmentation.rst) with specified hyper-parameters.\nFor example, if you want to train ADVENT on Office31, use the following script\n\n```shell script\n# Train a ADVENT on GTA5 to Cityscapes.\n# Assume you have put the datasets under the path `data/GTA5` and `data/Cityscapes`, \nCUDA_VISIBLE_DEVICES=0 python advent.py data/GTA5 data/Cityscapes -s GTA5 -t Cityscapes \\\n --log logs/advent/gtav2cityscapes\n```\n\nFor more information please refer to [Get Started](/docs/get_started/quickstart.rst) for help.\n\n## TODO\nSupport methods: AdaptSeg\n\n## Citation\nIf you use these methods in your research, please consider citing.\n\n```\n@inproceedings{CycleGAN,\n title={Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks},\n author={Zhu, Jun-Yan and Park, Taesung and Isola, Phillip and Efros, Alexei A},\n booktitle={ICCV},\n year={2017}\n}\n\n@inproceedings{cycada,\n title={Cycada: Cycle-consistent adversarial domain adaptation},\n author={Hoffman, Judy and Tzeng, Eric and Park, Taesung and Zhu, Jun-Yan and Isola, Phillip and Saenko, Kate and Efros, Alexei and Darrell, Trevor},\n booktitle={ICML},\n year={2018},\n}\n\n@inproceedings{Advent,\n author = {Vu, Tuan-Hung and Jain, Himalaya and Bucher, Maxime and Cord, Matthieu and Perez, Patrick},\n title = {ADVENT: Adversarial Entropy Minimization for Domain Adaptation in Semantic Segmentation},\n booktitle = {CVPR},\n year = {2019}\n}\n\n@inproceedings{FDA,\n author = {Yanchao Yang and\n Stefano Soatto},\n title = {{FDA:} Fourier Domain Adaptation for Semantic Segmentation},\n booktitle = {CVPR},\n year = {2020}\n}\n```\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/examples/domain_adaptation/semantic_segmentation/advent.py",
"content": "\"\"\"\n@author: Junguang Jiang\n@contact: JiangJunguang1123@outlook.com\n\"\"\"\nimport random\nimport time\nimport warnings\nimport sys\nimport argparse\nfrom PIL import Image\nimport numpy as np\nimport shutil\n\nimport torch\nimport torch.nn as nn\nimport torch.backends.cudnn as cudnn\nfrom torch.optim import SGD, Adam\nfrom torch.optim.lr_scheduler import LambdaLR\nfrom torch.utils.data import DataLoader\n\nsys.path.append('../../..')\nfrom dalib.adaptation.advent import Discriminator, DomainAdversarialEntropyLoss\nimport common.vision.models.segmentation as models\nimport common.vision.datasets.segmentation as datasets\nimport common.vision.transforms.segmentation as T\nfrom common.vision.transforms import DeNormalizeAndTranspose\nfrom common.utils.data import ForeverDataIterator\nfrom common.utils.metric import ConfusionMatrix\nfrom common.utils.meter import AverageMeter, ProgressMeter, Meter\nfrom common.utils.logger import CompleteLogger\n\n\ndevice = torch.device(\"cuda\" if torch.cuda.is_available() else \"cpu\")\n\n\ndef main(args: argparse.Namespace):\n logger = CompleteLogger(args.log, args.phase)\n print(args)\n\n if args.seed is not None:\n random.seed(args.seed)\n torch.manual_seed(args.seed)\n cudnn.deterministic = True\n warnings.warn('You have chosen to seed training. '\n 'This will turn on the CUDNN deterministic setting, '\n 'which can slow down your training considerably! '\n 'You may see unexpected behavior when restarting '\n 'from checkpoints.')\n\n cudnn.benchmark = True\n\n # Data loading code\n source_dataset = datasets.__dict__[args.source]\n train_source_dataset = source_dataset(\n root=args.source_root,\n transforms=T.Compose([\n T.RandomResizedCrop(size=args.train_size, ratio=args.resize_ratio, scale=(0.5, 1.)),\n T.ColorJitter(brightness=0.3, contrast=0.3),\n T.RandomHorizontalFlip(),\n T.NormalizeAndTranspose(),\n ]),\n )\n train_source_loader = DataLoader(train_source_dataset, batch_size=args.batch_size,\n shuffle=True, num_workers=args.workers, pin_memory=True, drop_last=True)\n\n target_dataset = datasets.__dict__[args.target]\n train_target_dataset = target_dataset(\n root=args.target_root,\n transforms=T.Compose([\n T.RandomResizedCrop(size=args.train_size, ratio=(2., 2.), scale=(0.5, 1.)),\n T.RandomHorizontalFlip(),\n T.NormalizeAndTranspose(),\n ]),\n )\n train_target_loader = DataLoader(train_target_dataset, batch_size=args.batch_size,\n shuffle=True, num_workers=args.workers, pin_memory=True, drop_last=True)\n val_target_dataset = target_dataset(\n root=args.target_root, split='val',\n transforms=T.Compose([\n T.Resize(image_size=args.test_input_size, label_size=args.test_output_size),\n T.NormalizeAndTranspose(),\n ]),\n )\n val_target_loader = DataLoader(val_target_dataset, batch_size=1, shuffle=False, pin_memory=True)\n\n train_source_iter = ForeverDataIterator(train_source_loader)\n train_target_iter = ForeverDataIterator(train_target_loader)\n\n # create model\n num_classes = train_source_dataset.num_classes\n model = models.__dict__[args.arch](num_classes=num_classes).to(device)\n discriminator = Discriminator(num_classes=num_classes).to(device)\n\n # define optimizer and lr scheduler\n optimizer = SGD(model.get_parameters(), lr=args.lr, momentum=args.momentum, weight_decay=args.weight_decay)\n optimizer_d = Adam(discriminator.parameters(), lr=args.lr_d, betas=(0.9, 0.99))\n lr_scheduler = LambdaLR(optimizer, lambda x: args.lr * (1. - float(x) / args.epochs / args.iters_per_epoch) ** (args.lr_power))\n lr_scheduler_d = LambdaLR(optimizer_d, lambda x: (1. - float(x) / args.epochs / args.iters_per_epoch) ** (args.lr_power))\n\n # optionally resume from a checkpoint\n if args.resume:\n checkpoint = torch.load(args.resume, map_location='cpu')\n model.load_state_dict(checkpoint['model'])\n discriminator.load_state_dict(checkpoint['discriminator'])\n optimizer.load_state_dict(checkpoint['optimizer'])\n lr_scheduler.load_state_dict(checkpoint['lr_scheduler'])\n optimizer_d.load_state_dict(checkpoint['optimizer_d'])\n lr_scheduler_d.load_state_dict(checkpoint['lr_scheduler_d'])\n args.start_epoch = checkpoint['epoch'] + 1\n\n # define loss function (criterion)\n criterion = torch.nn.CrossEntropyLoss(ignore_index=args.ignore_label).to(device)\n dann = DomainAdversarialEntropyLoss(discriminator)\n interp_train = nn.Upsample(size=args.train_size[::-1], mode='bilinear', align_corners=True)\n interp_val = nn.Upsample(size=args.test_output_size[::-1], mode='bilinear', align_corners=True)\n\n # define visualization function\n decode = train_source_dataset.decode_target\n\n def visualize(image, pred, label, prefix):\n \"\"\"\n Args:\n image (tensor): 3 x H x W\n pred (tensor): C x H x W\n label (tensor): H x W\n prefix: prefix of the saving image\n \"\"\"\n image = image.detach().cpu().numpy()\n pred = pred.detach().max(dim=0)[1].cpu().numpy()\n label = label.cpu().numpy()\n for tensor, name in [\n (Image.fromarray(np.uint8(DeNormalizeAndTranspose()(image))), \"image\"),\n (decode(label), \"label\"),\n (decode(pred), \"pred\")\n ]:\n tensor.save(logger.get_image_path(\"{}_{}.png\".format(prefix, name)))\n\n if args.phase == 'test':\n confmat = validate(val_target_loader, model, interp_val, criterion, visualize, args)\n print(confmat)\n return\n\n # start training\n best_iou = 0.\n for epoch in range(args.start_epoch, args.epochs):\n logger.set_epoch(epoch)\n print(lr_scheduler.get_lr(), lr_scheduler_d.get_lr())\n # train for one epoch\n train(train_source_iter, train_target_iter, model, interp_train, criterion, dann, optimizer,\n lr_scheduler, optimizer_d, lr_scheduler_d, epoch, visualize if args.debug else None, args)\n\n # evaluate on validation set\n confmat = validate(val_target_loader, model, interp_val, criterion, None, args)\n print(confmat.format(train_source_dataset.classes))\n acc_global, acc, iu = confmat.compute()\n\n # calculate the mean iou over partial classes\n indexes = [train_source_dataset.classes.index(name) for name\n in train_source_dataset.evaluate_classes]\n iu = iu[indexes]\n mean_iou = iu.mean()\n\n # remember best acc@1 and save checkpoint\n torch.save(\n {\n 'model': model.state_dict(),\n 'discriminator': discriminator.state_dict(),\n 'optimizer': optimizer.state_dict(),\n 'optimizer_d': optimizer_d.state_dict(),\n 'lr_scheduler': lr_scheduler.state_dict(),\n 'lr_scheduler_d': lr_scheduler_d.state_dict(),\n 'epoch': epoch,\n 'args': args\n }, logger.get_checkpoint_path(epoch)\n )\n if mean_iou > best_iou:\n shutil.copy(logger.get_checkpoint_path(epoch), logger.get_checkpoint_path('best'))\n best_iou = max(best_iou, mean_iou)\n print(\"Target: {} Best: {}\".format(mean_iou, best_iou))\n\n logger.close()\n\n\ndef train(train_source_iter: ForeverDataIterator, train_target_iter: ForeverDataIterator,\n model, interp, criterion, dann,\n optimizer: SGD, lr_scheduler: LambdaLR, optimizer_d: SGD, lr_scheduler_d: LambdaLR,\n epoch: int, visualize, args: argparse.Namespace):\n batch_time = AverageMeter('Time', ':4.2f')\n data_time = AverageMeter('Data', ':3.1f')\n losses_s = AverageMeter('Loss (s)', ':3.2f')\n losses_transfer = AverageMeter('Loss (transfer)', ':3.2f')\n losses_discriminator = AverageMeter('Loss (discriminator)', ':3.2f')\n accuracies_s = Meter('Acc (s)', ':3.2f')\n accuracies_t = Meter('Acc (t)', ':3.2f')\n iou_s = Meter('IoU (s)', ':3.2f')\n iou_t = Meter('IoU (t)', ':3.2f')\n\n confmat_s = ConfusionMatrix(model.num_classes)\n confmat_t = ConfusionMatrix(model.num_classes)\n progress = ProgressMeter(\n args.iters_per_epoch,\n [batch_time, data_time, losses_s, losses_transfer, losses_discriminator,\n accuracies_s, accuracies_t, iou_s, iou_t],\n prefix=\"Epoch: [{}]\".format(epoch))\n\n # switch to train mode\n model.train()\n\n end = time.time()\n\n for i in range(args.iters_per_epoch):\n x_s, label_s = next(train_source_iter)\n x_t, label_t = next(train_target_iter)\n\n x_s = x_s.to(device)\n label_s = label_s.long().to(device)\n x_t = x_t.to(device)\n label_t = label_t.long().to(device)\n\n # measure data loading time\n data_time.update(time.time() - end)\n\n optimizer.zero_grad()\n optimizer_d.zero_grad()\n\n # Step 1: Train the segmentation network, freeze the discriminator\n dann.eval()\n y_s = model(x_s)\n pred_s = interp(y_s)\n loss_cls_s = criterion(pred_s, label_s)\n loss_cls_s.backward()\n\n # adversarial training to fool the discriminator\n y_t = model(x_t)\n pred_t = interp(y_t)\n loss_transfer = dann(pred_t, 'source')\n (loss_transfer * args.trade_off).backward()\n\n # Step 2: Train the discriminator\n dann.train()\n loss_discriminator = 0.5 * (dann(pred_s.detach(), 'source') + dann(pred_t.detach(), 'target'))\n loss_discriminator.backward()\n\n # compute gradient and do SGD step\n optimizer.step()\n optimizer_d.step()\n lr_scheduler.step()\n lr_scheduler_d.step()\n\n # measure accuracy and record loss\n losses_s.update(loss_cls_s.item(), x_s.size(0))\n losses_transfer.update(loss_transfer.item(), x_s.size(0))\n losses_discriminator.update(loss_discriminator.item(), x_s.size(0))\n\n confmat_s.update(label_s.flatten(), pred_s.argmax(1).flatten())\n confmat_t.update(label_t.flatten(), pred_t.argmax(1).flatten())\n acc_global_s, acc_s, iu_s = confmat_s.compute()\n acc_global_t, acc_t, iu_t = confmat_t.compute()\n accuracies_s.update(acc_s.mean().item())\n accuracies_t.update(acc_t.mean().item())\n iou_s.update(iu_s.mean().item())\n iou_t.update(iu_t.mean().item())\n\n # measure elapsed time\n batch_time.update(time.time() - end)\n end = time.time()\n\n if i % args.print_freq == 0:\n progress.display(i)\n\n if visualize is not None:\n visualize(x_s[0], pred_s[0], label_s[0], \"source_{}\".format(i))\n visualize(x_t[0], pred_t[0], label_t[0], \"target_{}\".format(i))\n\n\ndef validate(val_loader: DataLoader, model, interp, criterion, visualize, args: argparse.Namespace):\n batch_time = AverageMeter('Time', ':6.3f')\n losses = AverageMeter('Loss', ':.4e')\n acc = Meter('Acc', ':3.2f')\n iou = Meter('IoU', ':3.2f')\n progress = ProgressMeter(\n len(val_loader),\n [batch_time, losses, acc, iou],\n prefix='Test: ')\n\n # switch to evaluate mode\n model.eval()\n confmat = ConfusionMatrix(model.num_classes)\n\n with torch.no_grad():\n end = time.time()\n for i, (x, label) in enumerate(val_loader):\n x = x.to(device)\n label = label.long().to(device)\n\n # compute output\n output = interp(model(x))\n loss = criterion(output, label)\n\n # measure accuracy and record loss\n losses.update(loss.item(), x.size(0))\n confmat.update(label.flatten(), output.argmax(1).flatten())\n acc_global, accs, iu = confmat.compute()\n acc.update(accs.mean().item())\n iou.update(iu.mean().item())\n\n # measure elapsed time\n batch_time.update(time.time() - end)\n end = time.time()\n\n if i % args.print_freq == 0:\n progress.display(i)\n\n if visualize is not None:\n visualize(x[0], output[0], label[0], \"val_{}\".format(i))\n\n return confmat\n\n\nif __name__ == '__main__':\n architecture_names = sorted(\n name for name in models.__dict__\n if name.islower() and not name.startswith(\"__\")\n and callable(models.__dict__[name])\n )\n dataset_names = sorted(\n name for name in datasets.__dict__\n if not name.startswith(\"__\") and callable(datasets.__dict__[name])\n )\n\n parser = argparse.ArgumentParser(description='ADVENT for Segmentation Domain Adaptation')\n # dataset parameters\n parser.add_argument('source_root', help='root path of the source dataset')\n parser.add_argument('target_root', help='root path of the target dataset')\n parser.add_argument('-s', '--source', help='source domain(s)')\n parser.add_argument('-t', '--target', help='target domain(s)')\n parser.add_argument('--resize-ratio', nargs='+', type=float, default=(1.5, 8 / 3.),\n help='the resize ratio for the random resize crop')\n parser.add_argument('--train-size', nargs='+', type=int, default=(1024, 512),\n help='the input and output image size during training')\n parser.add_argument('--test-input-size', nargs='+', type=int, default=(1024, 512),\n help='the input image size during test')\n parser.add_argument('--test-output-size', nargs='+', type=int, default=(2048, 1024),\n help='the output image size during test')\n # model parameters\n parser.add_argument('-a', '--arch', metavar='ARCH', default='deeplabv2_resnet101',\n choices=architecture_names,\n help='backbone architecture: ' +\n ' | '.join(architecture_names) +\n ' (default: deeplabv2_resnet101)')\n parser.add_argument(\"--resume\", type=str, default=None,\n help=\"Where restore model parameters from.\")\n parser.add_argument('--trade-off', type=float, default=0.001,\n help='trade-off parameter for the advent loss')\n # training parameters\n parser.add_argument('-b', '--batch-size', default=2, type=int,\n metavar='N',\n help='mini-batch size (default: 2)')\n parser.add_argument('--lr', '--learning-rate', default=2.5e-3, type=float,\n metavar='LR', help='initial learning rate', dest='lr')\n parser.add_argument(\"--momentum\", type=float, default=0.9, help=\"Momentum component of the optimiser.\")\n parser.add_argument(\"--weight-decay\", type=float, default=0.0005, help=\"Regularisation parameter for L2-loss.\")\n parser.add_argument(\"--lr-power\", type=float, default=0.9,\n help=\"Decay parameter to compute the learning rate (only for deeplab).\")\n parser.add_argument(\"--lr-d\", default=1e-4, type=float,\n metavar='LR', help='initial learning rate for discriminator')\n parser.add_argument('-j', '--workers', default=4, type=int, metavar='N',\n help='number of data loading workers (default: 4)')\n parser.add_argument('--epochs', default=60, type=int, metavar='N',\n help='number of total epochs to run')\n parser.add_argument('--start-epoch', default=0, type=int, metavar='N',\n help='start epoch')\n parser.add_argument('-i', '--iters-per-epoch', default=2500, type=int,\n help='Number of iterations per epoch')\n parser.add_argument('-p', '--print-freq', default=100, type=int,\n metavar='N', help='print frequency (default: 100)')\n parser.add_argument('--seed', default=None, type=int,\n help='seed for initializing training. ')\n parser.add_argument(\"--ignore-label\", type=int, default=255,\n help=\"The index of the label to ignore during the training.\")\n parser.add_argument(\"--log\", type=str, default='advent',\n help=\"Where to save logs, checkpoints and debugging images.\")\n parser.add_argument(\"--phase\", type=str, default='train', choices=['train', 'test'],\n help=\"When phase is 'test', only test the model.\")\n parser.add_argument('--debug', action=\"store_true\",\n help='In the debug mode, save images and predictions during training')\n args = parser.parse_args()\n main(args)\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/examples/domain_adaptation/semantic_segmentation/advent.sh",
"content": "# GTA5 to Cityscapes\nCUDA_VISIBLE_DEVICES=0 python advent.py data/GTA5 data/Cityscapes -s GTA5 -t Cityscapes \\\n --log logs/advent/gtav2cityscapes\n\n# Synthia to Cityscapes\nCUDA_VISIBLE_DEVICES=0 python advent.py data/synthia data/Cityscapes -s Synthia -t Cityscapes \\\n --log logs/advent/synthia2cityscapes\n\n# Cityscapes to Foggy\nCUDA_VISIBLE_DEVICES=0 python advent.py data/Cityscapes data/Cityscapes -s Cityscapes -t FoggyCityscapes \\\n --log logs/advent/cityscapes2foggy"
},
{
"path": "domain_generalization/Transfer-Learning-Library/examples/domain_adaptation/semantic_segmentation/cycada.py",
"content": "\"\"\"\n@author: Junguang Jiang\n@contact: JiangJunguang1123@outlook.com\n\"\"\"\nimport random\nimport time\nimport warnings\nimport sys\nimport argparse\nimport itertools\n\nimport torch\nimport torch.nn as nn\nimport torch.backends.cudnn as cudnn\nfrom torch.optim import Adam\nfrom torch.optim.lr_scheduler import LambdaLR\nfrom torch.utils.data import DataLoader\nfrom torchvision.transforms import ToPILImage, Compose, Lambda\n\nsys.path.append('../../..')\nimport dalib.translation.cyclegan as cyclegan\nfrom dalib.translation.cyclegan.util import ImagePool, set_requires_grad\nfrom dalib.translation.cycada import SemanticConsistency\nimport common.vision.models.segmentation as models\nimport common.vision.datasets.segmentation as datasets\nfrom common.vision.transforms import Denormalize, NormalizeAndTranspose\nimport common.vision.transforms.segmentation as T\nfrom common.utils.data import ForeverDataIterator\nfrom common.utils.meter import AverageMeter, ProgressMeter\nfrom common.utils.logger import CompleteLogger\n\n\ndevice = torch.device(\"cuda\" if torch.cuda.is_available() else \"cpu\")\n\n\ndef main(args):\n logger = CompleteLogger(args.log, args.phase)\n print(args)\n\n if args.seed is not None:\n random.seed(args.seed)\n torch.manual_seed(args.seed)\n cudnn.deterministic = True\n warnings.warn('You have chosen to seed training. '\n 'This will turn on the CUDNN deterministic setting, '\n 'which can slow down your training considerably! '\n 'You may see unexpected behavior when restarting '\n 'from checkpoints.')\n\n cudnn.benchmark = True\n\n # Data loading code\n train_transform = T.Compose([\n T.RandomResizedCrop(size=args.train_size, ratio=args.resize_ratio, scale=(0.5, 1.)),\n T.RandomHorizontalFlip(),\n T.ToTensor(),\n T.Normalize((0.5, 0.5, 0.5), (0.5, 0.5, 0.5))\n ])\n source_dataset = datasets.__dict__[args.source]\n train_source_dataset = source_dataset(root=args.source_root, transforms=train_transform)\n train_source_loader = DataLoader(train_source_dataset, batch_size=args.batch_size,\n shuffle=True, num_workers=args.workers, pin_memory=True, drop_last=True)\n\n target_dataset = datasets.__dict__[args.target]\n train_target_dataset = target_dataset(root=args.target_root, transforms=train_transform)\n train_target_loader = DataLoader(train_target_dataset, batch_size=args.batch_size,\n shuffle=True, num_workers=args.workers, pin_memory=True, drop_last=True)\n\n train_source_iter = ForeverDataIterator(train_source_loader)\n train_target_iter = ForeverDataIterator(train_target_loader)\n\n # define networks (both generators and discriminators)\n netG_S2T = cyclegan.generator.__dict__[args.netG](ngf=args.ngf, norm=args.norm, use_dropout=False).to(device)\n netG_T2S = cyclegan.generator.__dict__[args.netG](ngf=args.ngf, norm=args.norm, use_dropout=False).to(device)\n netD_S = cyclegan.discriminator.__dict__[args.netD](ndf=args.ndf, norm=args.norm).to(device)\n netD_T = cyclegan.discriminator.__dict__[args.netD](ndf=args.ndf, norm=args.norm).to(device)\n\n # create image buffer to store previously generated images\n fake_S_pool = ImagePool(args.pool_size)\n fake_T_pool = ImagePool(args.pool_size)\n\n # define optimizer and lr scheduler\n optimizer_G = Adam(itertools.chain(netG_S2T.parameters(), netG_T2S.parameters()), lr=args.lr, betas=(args.beta1, 0.999))\n optimizer_D = Adam(itertools.chain(netD_S.parameters(), netD_T.parameters()), lr=args.lr, betas=(args.beta1, 0.999))\n lr_decay_function = lambda epoch: 1.0 - max(0, epoch - args.epochs) / float(args.epochs_decay)\n lr_scheduler_G = LambdaLR(optimizer_G, lr_lambda=lr_decay_function)\n lr_scheduler_D = LambdaLR(optimizer_D, lr_lambda=lr_decay_function)\n\n # optionally resume from a checkpoint\n if args.resume:\n print(\"Resume from\", args.resume)\n checkpoint = torch.load(args.resume, map_location='cpu')\n netG_S2T.load_state_dict(checkpoint['netG_S2T'])\n netG_T2S.load_state_dict(checkpoint['netG_T2S'])\n netD_S.load_state_dict(checkpoint['netD_S'])\n netD_T.load_state_dict(checkpoint['netD_T'])\n optimizer_G.load_state_dict(checkpoint['optimizer_G'])\n optimizer_D.load_state_dict(checkpoint['optimizer_D'])\n lr_scheduler_G.load_state_dict(checkpoint['lr_scheduler_G'])\n lr_scheduler_D.load_state_dict(checkpoint['lr_scheduler_D'])\n args.start_epoch = checkpoint['epoch'] + 1\n\n if args.phase == 'test':\n transform = T.Compose([\n T.Resize(image_size=args.test_input_size),\n T.wrapper(cyclegan.transform.Translation)(netG_S2T, device),\n ])\n train_source_dataset.translate(transform, args.translated_root)\n return\n\n # define loss function\n criterion_gan = cyclegan.LeastSquaresGenerativeAdversarialLoss()\n criterion_cycle = nn.L1Loss()\n criterion_identity = nn.L1Loss()\n criterion_semantic = SemanticConsistency(ignore_index=[args.ignore_label]+train_source_dataset.ignore_classes).to(device)\n interp_train = nn.Upsample(size=args.train_size[::-1], mode='bilinear', align_corners=True)\n\n # define segmentation model and predict function\n model = models.__dict__[args.arch](num_classes=train_source_dataset.num_classes).to(device)\n if args.pretrain:\n print(\"Loading pretrain segmentation model from\", args.pretrain)\n checkpoint = torch.load(args.pretrain, map_location='cpu')\n model.load_state_dict(checkpoint['model'])\n model.eval()\n\n cycle_gan_tensor_to_segmentation_tensor = Compose([\n Denormalize((0.5, 0.5, 0.5), (0.5, 0.5, 0.5)),\n Lambda(lambda image: image.mul(255).permute((1, 2, 0))),\n NormalizeAndTranspose(),\n ])\n\n def predict(image):\n image = cycle_gan_tensor_to_segmentation_tensor(image.squeeze())\n image = image.unsqueeze(dim=0).to(device)\n prediction = model(image)\n return interp_train(prediction)\n\n # define visualization function\n tensor_to_image = Compose([\n Denormalize((0.5, 0.5, 0.5), (0.5, 0.5, 0.5)),\n ToPILImage()\n ])\n decode = train_source_dataset.decode_target\n\n def visualize(image, name, pred=None):\n \"\"\"\n Args:\n image (tensor): image in shape 3 x H x W\n name: name of the saving image\n pred (tensor): predictions in shape C x H x W\n \"\"\"\n tensor_to_image(image).save(logger.get_image_path(\"{}.png\".format(name)))\n if pred is not None:\n pred = pred.detach().max(dim=0).indices.cpu().numpy()\n pred = decode(pred)\n pred.save(logger.get_image_path(\"pred_{}.png\".format(name)))\n\n # start training\n for epoch in range(args.start_epoch, args.epochs+args.epochs_decay):\n logger.set_epoch(epoch)\n print(lr_scheduler_G.get_lr())\n\n # train for one epoch\n train(train_source_iter, train_target_iter, netG_S2T, netG_T2S, netD_S, netD_T, predict,\n criterion_gan, criterion_cycle, criterion_identity, criterion_semantic, optimizer_G, optimizer_D,\n fake_S_pool, fake_T_pool, epoch, visualize, args)\n\n # update learning rates\n lr_scheduler_G.step()\n lr_scheduler_D.step()\n\n # save checkpoint\n torch.save(\n {\n 'netG_S2T': netG_S2T.state_dict(),\n 'netG_T2S': netG_T2S.state_dict(),\n 'netD_S': netD_S.state_dict(),\n 'netD_T': netD_T.state_dict(),\n 'optimizer_G': optimizer_G.state_dict(),\n 'optimizer_D': optimizer_D.state_dict(),\n 'lr_scheduler_G': lr_scheduler_G.state_dict(),\n 'lr_scheduler_D': lr_scheduler_D.state_dict(),\n 'epoch': epoch,\n 'args': args\n }, logger.get_checkpoint_path(epoch)\n )\n\n if args.translated_root is not None:\n transform = T.Compose([\n T.Resize(image_size=args.test_input_size),\n T.wrapper(cyclegan.transform.Translation)(netG_S2T, device),\n ])\n train_source_dataset.translate(transform, args.translated_root)\n\n logger.close()\n\n\ndef train(train_source_iter, train_target_iter, netG_S2T, netG_T2S, netD_S, netD_T, predict,\n criterion_gan, criterion_cycle, criterion_identity, criterion_semantic,\n optimizer_G, optimizer_D, fake_S_pool, fake_T_pool,\n epoch: int, visualize, args: argparse.Namespace):\n batch_time = AverageMeter('Time', ':4.2f')\n data_time = AverageMeter('Data', ':3.1f')\n losses_G_S2T = AverageMeter('G_S2T', ':3.2f')\n losses_G_T2S = AverageMeter('G_T2S', ':3.2f')\n losses_D_S = AverageMeter('D_S', ':3.2f')\n losses_D_T = AverageMeter('D_T', ':3.2f')\n losses_cycle_S = AverageMeter('cycle_S', ':3.2f')\n losses_cycle_T = AverageMeter('cycle_T', ':3.2f')\n losses_identity_S = AverageMeter('idt_S', ':3.2f')\n losses_identity_T = AverageMeter('idt_T', ':3.2f')\n losses_semantic_S2T = AverageMeter('sem_S2T', ':3.2f')\n losses_semantic_T2S = AverageMeter('sem_T2S', ':3.2f')\n\n progress = ProgressMeter(\n args.iters_per_epoch,\n [batch_time, data_time, losses_G_S2T, losses_G_T2S, losses_D_S, losses_D_T,\n losses_cycle_S, losses_cycle_T, losses_identity_S, losses_identity_T,\n losses_semantic_S2T, losses_semantic_T2S],\n prefix=\"Epoch: [{}]\".format(epoch))\n\n end = time.time()\n\n for i in range(args.iters_per_epoch):\n real_S, label_s = next(train_source_iter)\n real_T, _ = next(train_target_iter)\n\n real_S = real_S.to(device)\n real_T = real_T.to(device)\n label_s = label_s.to(device)\n\n # measure data loading time\n data_time.update(time.time() - end)\n\n # Compute fake images and reconstruction images.\n fake_T = netG_S2T(real_S)\n rec_S = netG_T2S(fake_T)\n fake_S = netG_T2S(real_T)\n rec_T = netG_S2T(fake_S)\n\n # Optimizing generators\n # discriminators require no gradients\n set_requires_grad(netD_S, False)\n set_requires_grad(netD_T, False)\n\n optimizer_G.zero_grad()\n # GAN loss D_T(G_S2T(S))\n loss_G_S2T = criterion_gan(netD_T(fake_T), real=True)\n # GAN loss D_S(G_T2S(B))\n loss_G_T2S = criterion_gan(netD_S(fake_S), real=True)\n # Cycle loss || G_T2S(G_S2T(S)) - S||\n loss_cycle_S = criterion_cycle(rec_S, real_S) * args.trade_off_cycle\n # Cycle loss || G_S2T(G_T2S(T)) - T||\n loss_cycle_T = criterion_cycle(rec_T, real_T) * args.trade_off_cycle\n # Identity loss\n # G_S2T should be identity if real_T is fed: ||G_S2T(real_T) - real_T||\n identity_T = netG_S2T(real_T)\n loss_identity_T = criterion_identity(identity_T, real_T) * args.trade_off_identity\n # G_T2S should be identity if real_S is fed: ||G_T2S(real_S) - real_S||\n identity_S = netG_T2S(real_S)\n loss_identity_S = criterion_identity(identity_S, real_S) * args.trade_off_identity\n # Semantic loss\n pred_fake_T = predict(fake_T)\n pred_real_S = predict(real_S)\n loss_semantic_S2T = criterion_semantic(pred_fake_T, label_s) * args.trade_off_semantic\n pred_fake_S = predict(fake_S)\n pred_real_T = predict(real_T)\n loss_semantic_T2S = criterion_semantic(pred_fake_S, pred_real_T.max(1).indices) * args.trade_off_semantic\n # combined loss and calculate gradients\n loss_G = loss_G_S2T + loss_G_T2S + loss_cycle_S + loss_cycle_T + \\\n loss_identity_S + loss_identity_T + loss_semantic_S2T + loss_semantic_T2S\n loss_G.backward()\n optimizer_G.step()\n\n # Optimize discriminator\n set_requires_grad(netD_S, True)\n set_requires_grad(netD_T, True)\n optimizer_D.zero_grad()\n # Calculate GAN loss for discriminator D_S\n fake_S_ = fake_S_pool.query(fake_S.detach())\n loss_D_S = 0.5 * (criterion_gan(netD_S(real_S), True) + criterion_gan(netD_S(fake_S_), False))\n loss_D_S.backward()\n # Calculate GAN loss for discriminator D_T\n fake_T_ = fake_T_pool.query(fake_T.detach())\n loss_D_T = 0.5 * (criterion_gan(netD_T(real_T), True) + criterion_gan(netD_T(fake_T_), False))\n loss_D_T.backward()\n optimizer_D.step()\n\n # measure elapsed time\n losses_G_S2T.update(loss_G_S2T.item(), real_S.size(0))\n losses_G_T2S.update(loss_G_T2S.item(), real_S.size(0))\n losses_D_S.update(loss_D_S.item(), real_S.size(0))\n losses_D_T.update(loss_D_T.item(), real_S.size(0))\n losses_cycle_S.update(loss_cycle_S.item(), real_S.size(0))\n losses_cycle_T.update(loss_cycle_T.item(), real_S.size(0))\n losses_identity_S.update(loss_identity_S.item(), real_S.size(0))\n losses_identity_T.update(loss_identity_T.item(), real_S.size(0))\n losses_semantic_S2T.update(loss_semantic_S2T.item(), real_S.size(0))\n losses_semantic_T2S.update(loss_semantic_T2S.item(), real_S.size(0))\n batch_time.update(time.time() - end)\n end = time.time()\n\n if i % args.print_freq == 0:\n progress.display(i)\n for image, prediction, name in zip([real_S, real_T, fake_S, fake_T],\n [pred_real_S, pred_real_T, pred_fake_S, pred_fake_T],\n [\"real_S\", \"real_T\", \"fake_S\", \"fake_T\"]):\n visualize(image[0], \"{}_{}\".format(i, name), prediction[0])\n for image, name in zip([rec_S, rec_T, identity_S, identity_T],\n [\"rec_S\", \"rec_T\", \"identity_S\", \"identity_T\"]):\n visualize(image[0], \"{}_{}\".format(i, name))\n\n\nif __name__ == '__main__':\n architecture_names = sorted(\n name for name in models.__dict__\n if name.islower() and not name.startswith(\"__\")\n and callable(models.__dict__[name])\n )\n dataset_names = sorted(\n name for name in datasets.__dict__\n if not name.startswith(\"__\") and callable(datasets.__dict__[name])\n )\n # dataset parameters\n parser = argparse.ArgumentParser(description='Cycada for Segmentation Domain Adaptation')\n parser.add_argument('source_root', help='root path of the source dataset')\n parser.add_argument('target_root', help='root path of the target dataset')\n parser.add_argument('-s', '--source', help='source domain(s)')\n parser.add_argument('-t', '--target', help='target domain(s)')\n parser.add_argument('--resize-ratio', nargs='+', type=float, default=(1.5, 8 / 3.),\n help='the resize ratio for the random resize crop')\n parser.add_argument('--train-size', nargs='+', type=int, default=(512, 256),\n help='the input and output image size during training')\n # model parameters\n parser.add_argument('-a', '--arch', metavar='ARCH', default='deeplabv2_resnet101',\n choices=architecture_names,\n help='backbone architecture: ' +\n ' | '.join(architecture_names) +\n ' (default: deeplabv2_resnet101)')\n parser.add_argument('--pretrain', type=str, default=None,\n help='pretrain checkpoints for segementation model')\n parser.add_argument('--ngf', type=int, default=64, help='# of gen filters in the last conv layer')\n parser.add_argument('--ndf', type=int, default=64, help='# of discrim filters in the first conv layer')\n parser.add_argument('--netD', type=str, default='patch',\n help='specify discriminator architecture [patch | pixel]. The basic model is a 70x70 PatchGAN.')\n parser.add_argument('--netG', type=str, default='unet_256',\n help='specify generator architecture [resnet_9 | resnet_6 | unet_256 | unet_128]')\n parser.add_argument('--norm', type=str, default='instance',\n help='instance normalization or batch normalization [instance | batch | none]')\n parser.add_argument(\"--resume\", type=str, default=None,\n help=\"Where restore cyclegan model parameters from.\")\n parser.add_argument('--trade-off-cycle', type=float, default=10.0, help='trade off for cycle loss')\n parser.add_argument('--trade-off-identity', type=float, default=5.0, help='trade off for identity loss')\n parser.add_argument('--trade-off-semantic', type=float, default=1.0, help='trade off for semantic loss')\n # training parameters\n parser.add_argument('-b', '--batch-size', default=1, type=int,\n metavar='N',\n help='mini-batch size (default: 1)')\n parser.add_argument('--lr', type=float, default=0.0002, help='initial learning rate for adam')\n parser.add_argument('--beta1', type=float, default=0.5, help='momentum term of adam')\n parser.add_argument('-j', '--workers', default=4, type=int, metavar='N',\n help='number of data loading workers (default: 4)')\n parser.add_argument('--epochs', default=20, type=int, metavar='N',\n help='number of total epochs to run')\n parser.add_argument('--epochs-decay', type=int, default=20,\n help='number of epochs to linearly decay learning rate to zero')\n parser.add_argument('--start-epoch', default=0, type=int, metavar='N',\n help='start epoch')\n parser.add_argument('-i', '--iters-per-epoch', default=5000, type=int,\n help='Number of iterations per epoch')\n parser.add_argument('--pool-size', type=int, default=50,\n help='the size of image buffer that stores previously generated images')\n parser.add_argument('-p', '--print-freq', default=400, type=int,\n metavar='N', help='print frequency (default: 100)')\n parser.add_argument('--seed', default=None, type=int,\n help='seed for initializing training. ')\n parser.add_argument(\"--ignore-label\", type=int, default=255,\n help=\"The index of the label to ignore during the training.\")\n parser.add_argument(\"--log\", type=str, default='cycada',\n help=\"Where to save logs, checkpoints and debugging images.\")\n # test parameters\n parser.add_argument(\"--phase\", type=str, default='train', choices=['train', 'test'],\n help=\"When phase is 'test', only test the model.\")\n parser.add_argument('--translated-root', type=str, default=None,\n help=\"The root to put the translated dataset\")\n parser.add_argument('--test-input-size', nargs='+', type=int, default=(1024, 512),\n help='the input image size during test')\n args = parser.parse_args()\n main(args)\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/examples/domain_adaptation/semantic_segmentation/cycada.sh",
"content": "# GTA5 to Cityscapes\n# First, train the CycleGAN\nCUDA_VISIBLE_DEVICES=0 python cycada.py data/GTA5 data/Cityscapes -s GTA5 -t Cityscapes \\\n --log logs/cycada/gtav2cityscapes --pretrain logs/src_only/gtav2cityscapes/checkpoints/59.pth \\\n --translated-root data/GTA52Cityscapes/cycada_39\n# Then, train the src_only model on the translated source dataset\nCUDA_VISIBLE_DEVICES=0 python source_only.py data/GTA52Cityscapes/cycada_39 data/Cityscapes \\\n -s GTA5 -t Cityscapes --log logs/cycada_src_only/gtav2cityscapes\n\n\n## Synthia to Cityscapes\n# First, train the Cycada\nCUDA_VISIBLE_DEVICES=0 python cycada.py data/synthia data/Cityscapes -s Synthia -t Cityscapes \\\n --log logs/cycada/synthia2cityscapes --pretrain logs/src_only/synthia2cityscapes/checkpoints/59.pth \\\n --translated-root data/Synthia2Cityscapes/cycada_39\n# Then, train the src_only model on the translated source dataset\nCUDA_VISIBLE_DEVICES=0 python source_only.py data/Synthia2Cityscapes/cycada_39 data/Cityscapes \\\n -s Synthia -t Cityscapes --log logs/cycada_src_only/synthia2cityscapes\n\n\n# Cityscapes to FoggyCityscapes\n# First, train the CycleGAN\nCUDA_VISIBLE_DEVICES=0 python cycada.py data/Cityscapes data/Cityscapes -s Cityscapes -t FoggyCityscapes \\\n --log logs/cycada/cityscapes2foggy --pretrain logs/src_only/cityscapes2foggy/checkpoints/59.pth \\\n --translated-root data/Cityscapes2Foggy/cycada_39\n# Then, train the src_only model on the translated source dataset\nCUDA_VISIBLE_DEVICES=0 python source_only.py data/Cityscapes2Foggy/cycada_39 data/Cityscapes \\\n -s Cityscapes -t FoggyCityscapes --log logs/cycada_src_only/cityscapes2foggy\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/examples/domain_adaptation/semantic_segmentation/cycle_gan.py",
"content": "\"\"\"\n@author: Junguang Jiang\n@contact: JiangJunguang1123@outlook.com\n\"\"\"\nimport random\nimport time\nimport warnings\nimport sys\nimport argparse\nimport itertools\n\nimport torch\nimport torch.nn as nn\nimport torch.backends.cudnn as cudnn\nfrom torch.optim import Adam\nfrom torch.optim.lr_scheduler import LambdaLR\nfrom torch.utils.data import DataLoader\nfrom torchvision.transforms import ToPILImage, Compose\n\nsys.path.append('../../..')\nimport dalib.translation.cyclegan as cyclegan\nfrom dalib.translation.cyclegan.util import ImagePool, set_requires_grad\nimport common.vision.datasets.segmentation as datasets\nfrom common.vision.transforms import Denormalize\nimport common.vision.transforms.segmentation as T\nfrom common.utils.data import ForeverDataIterator\nfrom common.utils.meter import AverageMeter, ProgressMeter\nfrom common.utils.logger import CompleteLogger\n\n\ndevice = torch.device(\"cuda\" if torch.cuda.is_available() else \"cpu\")\n\n\ndef main(args):\n logger = CompleteLogger(args.log, args.phase)\n print(args)\n\n if args.seed is not None:\n random.seed(args.seed)\n torch.manual_seed(args.seed)\n cudnn.deterministic = True\n warnings.warn('You have chosen to seed training. '\n 'This will turn on the CUDNN deterministic setting, '\n 'which can slow down your training considerably! '\n 'You may see unexpected behavior when restarting '\n 'from checkpoints.')\n\n cudnn.benchmark = True\n\n # Data loading code\n train_transform = T.Compose([\n T.RandomResizedCrop(size=args.train_size, ratio=args.resize_ratio, scale=(0.5, 1.)),\n T.RandomHorizontalFlip(),\n T.ToTensor(),\n T.Normalize((0.5, 0.5, 0.5), (0.5, 0.5, 0.5))\n ])\n source_dataset = datasets.__dict__[args.source]\n train_source_dataset = source_dataset(root=args.source_root, transforms=train_transform)\n train_source_loader = DataLoader(train_source_dataset, batch_size=args.batch_size,\n shuffle=True, num_workers=args.workers, pin_memory=True, drop_last=True)\n\n target_dataset = datasets.__dict__[args.target]\n train_target_dataset = target_dataset(root=args.target_root, transforms=train_transform)\n train_target_loader = DataLoader(train_target_dataset, batch_size=args.batch_size,\n shuffle=True, num_workers=args.workers, pin_memory=True, drop_last=True)\n\n train_source_iter = ForeverDataIterator(train_source_loader)\n train_target_iter = ForeverDataIterator(train_target_loader)\n\n # define networks (both generators and discriminators)\n netG_S2T = cyclegan.generator.__dict__[args.netG](ngf=args.ngf, norm=args.norm, use_dropout=False).to(device)\n netG_T2S = cyclegan.generator.__dict__[args.netG](ngf=args.ngf, norm=args.norm, use_dropout=False).to(device)\n netD_S = cyclegan.discriminator.__dict__[args.netD](ndf=args.ndf, norm=args.norm).to(device)\n netD_T = cyclegan.discriminator.__dict__[args.netD](ndf=args.ndf, norm=args.norm).to(device)\n\n # create image buffer to store previously generated images\n fake_S_pool = ImagePool(args.pool_size)\n fake_T_pool = ImagePool(args.pool_size)\n\n # define optimizer and lr scheduler\n optimizer_G = Adam(itertools.chain(netG_S2T.parameters(), netG_T2S.parameters()), lr=args.lr, betas=(args.beta1, 0.999))\n optimizer_D = Adam(itertools.chain(netD_S.parameters(), netD_T.parameters()), lr=args.lr, betas=(args.beta1, 0.999))\n lr_decay_function = lambda epoch: 1.0 - max(0, epoch - args.epochs) / float(args.epochs_decay)\n lr_scheduler_G = LambdaLR(optimizer_G, lr_lambda=lr_decay_function)\n lr_scheduler_D = LambdaLR(optimizer_D, lr_lambda=lr_decay_function)\n\n # optionally resume from a checkpoint\n if args.resume:\n print(\"Resume from\", args.resume)\n checkpoint = torch.load(args.resume, map_location='cpu')\n netG_S2T.load_state_dict(checkpoint['netG_S2T'])\n netG_T2S.load_state_dict(checkpoint['netG_T2S'])\n netD_S.load_state_dict(checkpoint['netD_S'])\n netD_T.load_state_dict(checkpoint['netD_T'])\n optimizer_G.load_state_dict(checkpoint['optimizer_G'])\n optimizer_D.load_state_dict(checkpoint['optimizer_D'])\n lr_scheduler_G.load_state_dict(checkpoint['lr_scheduler_G'])\n lr_scheduler_D.load_state_dict(checkpoint['lr_scheduler_D'])\n args.start_epoch = checkpoint['epoch'] + 1\n\n if args.phase == 'test':\n transform = T.Compose([\n T.Resize(image_size=args.test_input_size),\n T.wrapper(cyclegan.transform.Translation)(netG_S2T, device),\n ])\n train_source_dataset.translate(transform, args.translated_root)\n return\n\n # define loss function\n criterion_gan = cyclegan.LeastSquaresGenerativeAdversarialLoss()\n criterion_cycle = nn.L1Loss()\n criterion_identity = nn.L1Loss()\n\n # define visualization function\n tensor_to_image = Compose([\n Denormalize((0.5, 0.5, 0.5), (0.5, 0.5, 0.5)),\n ToPILImage()\n ])\n\n def visualize(image, name):\n \"\"\"\n Args:\n image (tensor): image in shape 3 x H x W\n name: name of the saving image\n \"\"\"\n tensor_to_image(image).save(logger.get_image_path(\"{}.png\".format(name)))\n\n # start training\n for epoch in range(args.start_epoch, args.epochs+args.epochs_decay):\n logger.set_epoch(epoch)\n print(lr_scheduler_G.get_lr())\n\n # train for one epoch\n train(train_source_iter, train_target_iter, netG_S2T, netG_T2S, netD_S, netD_T,\n criterion_gan, criterion_cycle, criterion_identity, optimizer_G, optimizer_D,\n fake_S_pool, fake_T_pool, epoch, visualize, args)\n\n # update learning rates\n lr_scheduler_G.step()\n lr_scheduler_D.step()\n\n # save checkpoint\n torch.save(\n {\n 'netG_S2T': netG_S2T.state_dict(),\n 'netG_T2S': netG_T2S.state_dict(),\n 'netD_S': netD_S.state_dict(),\n 'netD_T': netD_T.state_dict(),\n 'optimizer_G': optimizer_G.state_dict(),\n 'optimizer_D': optimizer_D.state_dict(),\n 'lr_scheduler_G': lr_scheduler_G.state_dict(),\n 'lr_scheduler_D': lr_scheduler_D.state_dict(),\n 'epoch': epoch,\n 'args': args\n }, logger.get_checkpoint_path(epoch)\n )\n\n if args.translated_root is not None:\n transform = T.Compose([\n T.Resize(image_size=args.test_input_size),\n T.wrapper(cyclegan.transform.Translation)(netG_S2T, device),\n ])\n train_source_dataset.translate(transform, args.translated_root)\n\n logger.close()\n\n\ndef train(train_source_iter, train_target_iter, netG_S2T, netG_T2S, netD_S, netD_T,\n criterion_gan, criterion_cycle, criterion_identity, optimizer_G, optimizer_D,\n fake_S_pool, fake_T_pool, epoch: int, visualize, args: argparse.Namespace):\n batch_time = AverageMeter('Time', ':4.2f')\n data_time = AverageMeter('Data', ':3.1f')\n losses_G_S2T = AverageMeter('G_S2T', ':3.2f')\n losses_G_T2S = AverageMeter('G_T2S', ':3.2f')\n losses_D_S = AverageMeter('D_S', ':3.2f')\n losses_D_T = AverageMeter('D_T', ':3.2f')\n losses_cycle_S = AverageMeter('cycle_S', ':3.2f')\n losses_cycle_T = AverageMeter('cycle_T', ':3.2f')\n losses_identity_S = AverageMeter('idt_S', ':3.2f')\n losses_identity_T = AverageMeter('idt_T', ':3.2f')\n\n progress = ProgressMeter(\n args.iters_per_epoch,\n [batch_time, data_time, losses_G_S2T, losses_G_T2S, losses_D_S, losses_D_T,\n losses_cycle_S, losses_cycle_T, losses_identity_S, losses_identity_T],\n prefix=\"Epoch: [{}]\".format(epoch))\n\n end = time.time()\n\n for i in range(args.iters_per_epoch):\n real_S, _ = next(train_source_iter)\n real_T, _ = next(train_target_iter)\n\n real_S = real_S.to(device)\n real_T = real_T.to(device)\n\n # measure data loading time\n data_time.update(time.time() - end)\n\n # Compute fake images and reconstruction images.\n fake_T = netG_S2T(real_S)\n rec_S = netG_T2S(fake_T)\n fake_S = netG_T2S(real_T)\n rec_T = netG_S2T(fake_S)\n\n # Optimizing generators\n # discriminators require no gradients\n set_requires_grad(netD_S, False)\n set_requires_grad(netD_T, False)\n\n optimizer_G.zero_grad()\n # GAN loss D_T(G_S2T(S))\n loss_G_S2T = criterion_gan(netD_T(fake_T), real=True)\n # GAN loss D_S(G_T2S(B))\n loss_G_T2S = criterion_gan(netD_S(fake_S), real=True)\n # Cycle loss || G_T2S(G_S2T(S)) - S||\n loss_cycle_S = criterion_cycle(rec_S, real_S) * args.trade_off_cycle\n # Cycle loss || G_S2T(G_T2S(T)) - T||\n loss_cycle_T = criterion_cycle(rec_T, real_T) * args.trade_off_cycle\n # Identity loss\n # G_S2T should be identity if real_T is fed: ||G_S2T(real_T) - real_T||\n identity_T = netG_S2T(real_T)\n loss_identity_T = criterion_identity(identity_T, real_T) * args.trade_off_identity\n # G_T2S should be identity if real_S is fed: ||G_T2S(real_S) - real_S||\n identity_S = netG_T2S(real_S)\n loss_identity_S = criterion_identity(identity_S, real_S) * args.trade_off_identity\n # combined loss and calculate gradients\n loss_G = loss_G_S2T + loss_G_T2S + loss_cycle_S + loss_cycle_T + loss_identity_S + loss_identity_T\n loss_G.backward()\n optimizer_G.step()\n\n # Optimize discriminator\n set_requires_grad(netD_S, True)\n set_requires_grad(netD_T, True)\n optimizer_D.zero_grad()\n # Calculate GAN loss for discriminator D_S\n fake_S_ = fake_S_pool.query(fake_S.detach())\n loss_D_S = 0.5 * (criterion_gan(netD_S(real_S), True) + criterion_gan(netD_S(fake_S_), False))\n loss_D_S.backward()\n # Calculate GAN loss for discriminator D_T\n fake_T_ = fake_T_pool.query(fake_T.detach())\n loss_D_T = 0.5 * (criterion_gan(netD_T(real_T), True) + criterion_gan(netD_T(fake_T_), False))\n loss_D_T.backward()\n optimizer_D.step()\n\n # measure elapsed time\n losses_G_S2T.update(loss_G_S2T.item(), real_S.size(0))\n losses_G_T2S.update(loss_G_T2S.item(), real_S.size(0))\n losses_D_S.update(loss_D_S.item(), real_S.size(0))\n losses_D_T.update(loss_D_T.item(), real_S.size(0))\n losses_cycle_S.update(loss_cycle_S.item(), real_S.size(0))\n losses_cycle_T.update(loss_cycle_T.item(), real_S.size(0))\n losses_identity_S.update(loss_identity_S.item(), real_S.size(0))\n losses_identity_T.update(loss_identity_T.item(), real_S.size(0))\n batch_time.update(time.time() - end)\n end = time.time()\n\n if i % args.print_freq == 0:\n progress.display(i)\n\n for tensor, name in zip([real_S, real_T, fake_S, fake_T, rec_S, rec_T, identity_S, identity_T],\n [\"real_S\", \"real_T\", \"fake_S\", \"fake_T\", \"rec_S\",\n \"rec_T\", \"identity_S\", \"identity_T\"]):\n visualize(tensor[0], \"{}_{}\".format(i, name))\n\n\nif __name__ == '__main__':\n dataset_names = sorted(\n name for name in datasets.__dict__\n if not name.startswith(\"__\") and callable(datasets.__dict__[name])\n )\n parser = argparse.ArgumentParser(description='CycleGAN for Segmentation Domain Adaptation')\n # dataset parameters\n parser.add_argument('source_root', help='root path of the source dataset')\n parser.add_argument('target_root', help='root path of the target dataset')\n parser.add_argument('-s', '--source', help='source domain(s)')\n parser.add_argument('-t', '--target', help='target domain(s)')\n parser.add_argument('--resize-ratio', nargs='+', type=float, default=(1.5, 8 / 3.),\n help='the resize ratio for the random resize crop')\n parser.add_argument('--train-size', nargs='+', type=int, default=(1024, 512),\n help='the input and output image size during training')\n # model parameters\n parser.add_argument('--ngf', type=int, default=64, help='# of gen filters in the last conv layer')\n parser.add_argument('--ndf', type=int, default=64, help='# of discrim filters in the first conv layer')\n parser.add_argument('--netD', type=str, default='patch',\n help='specify discriminator architecture [patch | pixel]. The basic model is a 70x70 PatchGAN.')\n parser.add_argument('--netG', type=str, default='unet_256',\n help='specify generator architecture [resnet_9 | resnet_6 | unet_256 | unet_128]')\n parser.add_argument('--norm', type=str, default='instance',\n help='instance normalization or batch normalization [instance | batch | none]')\n parser.add_argument(\"--resume\", type=str, default=None,\n help=\"Where restore model parameters from.\")\n parser.add_argument('--trade-off-cycle', type=float, default=10.0, help='trade off for cycle loss')\n parser.add_argument('--trade-off-identity', type=float, default=5.0, help='trade off for identity loss')\n # training parameters\n parser.add_argument('-b', '--batch-size', default=1, type=int,\n metavar='N',\n help='mini-batch size (default: 1)')\n parser.add_argument('--lr', type=float, default=0.0002, help='initial learning rate for adam')\n parser.add_argument('--beta1', type=float, default=0.5, help='momentum term of adam')\n parser.add_argument('-j', '--workers', default=4, type=int, metavar='N',\n help='number of data loading workers (default: 4)')\n parser.add_argument('--epochs', default=20, type=int, metavar='N',\n help='number of total epochs to run')\n parser.add_argument('--epochs-decay', type=int, default=20,\n help='number of epochs to linearly decay learning rate to zero')\n parser.add_argument('--start-epoch', default=0, type=int, metavar='N',\n help='start epoch')\n parser.add_argument('-i', '--iters-per-epoch', default=5000, type=int,\n help='Number of iterations per epoch')\n parser.add_argument('--pool-size', type=int, default=50,\n help='the size of image buffer that stores previously generated images')\n parser.add_argument('-p', '--print-freq', default=400, type=int,\n metavar='N', help='print frequency (default: 100)')\n parser.add_argument('--seed', default=None, type=int,\n help='seed for initializing training. ')\n parser.add_argument(\"--log\", type=str, default='cyclegan',\n help=\"Where to save logs, checkpoints and debugging images.\")\n # test parameters\n parser.add_argument(\"--phase\", type=str, default='train', choices=['train', 'test'],\n help=\"When phase is 'test', only test the model.\")\n parser.add_argument('--translated-root', type=str, default=None,\n help=\"The root to put the translated dataset\")\n parser.add_argument('--test-input-size', nargs='+', type=int, default=(1024, 512),\n help='the input image size during test')\n args = parser.parse_args()\n main(args)\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/examples/domain_adaptation/semantic_segmentation/cycle_gan.sh",
"content": "# GTA5 to Cityscapes\n# First, train the CycleGAN\nCUDA_VISIBLE_DEVICES=0 python cycle_gan.py data/GTA5 data/Cityscapes -s GTA5 -t Cityscapes \\\n --log logs/cyclegan/gtav2cityscapes --translated-root data/GTA52Cityscapes/CycleGAN_39\n# Then, train the src_only model on the translated source dataset\nCUDA_VISIBLE_DEVICES=0 python source_only.py data/GTA52Cityscapes/CycleGAN_39 data/Cityscapes \\\n -s GTA5 -t Cityscapes --log logs/cyclegan_src_only/gtav2cityscapes\n\n\n# Cityscapes to FoggyCityscapes\n# First, train the CycleGAN\nCUDA_VISIBLE_DEVICES=0 python cycle_gan.py data/Cityscapes data/Cityscapes -s Cityscapes -t FoggyCityscapes \\\n --log logs/cyclegan/cityscapes2foggy --translated-root data/Cityscapes2Foggy/CycleGAN_39\n# Then, train the src_only model on the translated source dataset\nCUDA_VISIBLE_DEVICES=0 python source_only.py data/Cityscapes2Foggy/CycleGAN_39 data/Cityscapes \\\n -s Cityscapes -t FoggyCityscapes --log logs/cyclegan_src_only/cityscapes2foggy\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/examples/domain_adaptation/semantic_segmentation/fda.py",
"content": "\"\"\"\n@author: Junguang Jiang\n@contact: JiangJunguang1123@outlook.com\n\"\"\"\nimport random\nimport time\nimport warnings\nimport sys\nimport argparse\nfrom PIL import Image\nimport numpy as np\nimport os\nimport shutil\n\nimport torch\nimport torch.nn as nn\nimport torch.backends.cudnn as cudnn\nfrom torch.optim import SGD\nfrom torch.optim.lr_scheduler import LambdaLR\nfrom torch.utils.data import DataLoader\n\nsys.path.append('../../..')\nfrom dalib.translation.fourier_transform import FourierTransform\nfrom dalib.adaptation.fda import robust_entropy\nimport common.vision.models.segmentation as models\nimport common.vision.datasets.segmentation as datasets\nimport common.vision.transforms.segmentation as T\nfrom common.vision.transforms import DeNormalizeAndTranspose\nfrom common.utils.data import ForeverDataIterator\nfrom common.utils.metric import ConfusionMatrix\nfrom common.utils.meter import AverageMeter, ProgressMeter, Meter\nfrom common.utils.logger import CompleteLogger\n\n\ndevice = torch.device(\"cuda\" if torch.cuda.is_available() else \"cpu\")\n\n\ndef main(args: argparse.Namespace):\n logger = CompleteLogger(args.log, args.phase)\n print(args)\n\n if args.seed is not None:\n random.seed(args.seed)\n torch.manual_seed(args.seed)\n cudnn.deterministic = True\n warnings.warn('You have chosen to seed training. '\n 'This will turn on the CUDNN deterministic setting, '\n 'which can slow down your training considerably! '\n 'You may see unexpected behavior when restarting '\n 'from checkpoints.')\n\n cudnn.benchmark = True\n\n # Data loading code\n target_dataset = datasets.__dict__[args.target]\n train_target_dataset = target_dataset(\n root=args.target_root,\n transforms=T.Compose([\n T.Resize(image_size=args.train_size),\n T.NormalizeAndTranspose(),\n ]),\n )\n train_target_loader = DataLoader(train_target_dataset, batch_size=args.batch_size,\n shuffle=True, num_workers=args.workers, pin_memory=True, drop_last=True)\n val_target_dataset = target_dataset(\n root=args.target_root, split='val',\n transforms=T.Compose([\n T.Resize(image_size=args.test_input_size, label_size=args.test_output_size),\n T.NormalizeAndTranspose(),\n ])\n )\n val_target_loader = DataLoader(val_target_dataset, batch_size=1, shuffle=False, pin_memory=True)\n\n # collect the absolute paths of all images in the target dataset\n target_image_list = train_target_dataset.collect_image_paths()\n # build a fourier transform that translate source images to the target style\n fourier_transform = T.wrapper(FourierTransform)(target_image_list, os.path.join(logger.root, \"amplitudes\"),\n rebuild=False, beta=args.beta)\n\n source_dataset = datasets.__dict__[args.source]\n train_source_dataset = source_dataset(\n root=args.source_root,\n transforms=T.Compose([\n T.Resize((2048, 1024)), # convert source image to the size of the target image before fourier transform\n fourier_transform,\n T.RandomResizedCrop(size=args.train_size, ratio=args.resize_ratio, scale=(0.5, 1.)),\n T.ColorJitter(brightness=0.3, contrast=0.3),\n T.RandomHorizontalFlip(),\n T.NormalizeAndTranspose(),\n ]),\n )\n train_source_loader = DataLoader(train_source_dataset, batch_size=args.batch_size,\n shuffle=True, num_workers=args.workers, pin_memory=True, drop_last=True)\n\n train_source_iter = ForeverDataIterator(train_source_loader)\n train_target_iter = ForeverDataIterator(train_target_loader)\n\n # create model\n model = models.__dict__[args.arch](num_classes=train_source_dataset.num_classes).to(device)\n # define optimizer and lr scheduler\n optimizer = SGD(model.get_parameters(), lr=args.lr, momentum=args.momentum, weight_decay=args.weight_decay)\n lr_scheduler = LambdaLR(optimizer,\n lambda x: args.lr * (1. - float(x) / args.epochs / args.iters_per_epoch) ** (args.lr_power))\n\n # optionally resume from a checkpoint\n if args.resume:\n checkpoint = torch.load(args.resume, map_location='cpu')\n model.load_state_dict(checkpoint['model'])\n optimizer.load_state_dict(checkpoint['optimizer'])\n lr_scheduler.load_state_dict(checkpoint['lr_scheduler'])\n args.start_epoch = checkpoint['epoch'] + 1\n\n # define loss function (criterion)\n criterion = torch.nn.CrossEntropyLoss(ignore_index=args.ignore_label).to(device)\n interp_train = nn.Upsample(size=args.train_size[::-1], mode='bilinear', align_corners=True)\n interp_val = nn.Upsample(size=args.test_output_size[::-1], mode='bilinear', align_corners=True)\n\n # define visualization function\n decode = train_source_dataset.decode_target\n def visualize(image, pred, label, prefix):\n \"\"\"\n Args:\n image (tensor): 3 x H x W\n pred (tensor): C x H x W\n label (tensor): H x W\n prefix: prefix of the saving image\n \"\"\"\n image = image.detach().cpu().numpy()\n pred = pred.detach().max(dim=0)[1].cpu().numpy()\n label = label.cpu().numpy()\n for tensor, name in [\n (Image.fromarray(np.uint8(DeNormalizeAndTranspose()(image))), \"image\"),\n (decode(label), \"label\"),\n (decode(pred), \"pred\")\n ]:\n tensor.save(logger.get_image_path(\"{}_{}.png\".format(prefix, name)))\n\n if args.phase == 'test':\n confmat = validate(val_target_loader, model, interp_val, criterion, visualize, args)\n print(confmat)\n return\n\n # start training\n best_iou = 0.\n for epoch in range(args.start_epoch, args.epochs):\n logger.set_epoch(epoch)\n print(lr_scheduler.get_lr())\n\n # train for one epoch\n train(train_source_iter, train_target_iter, model, interp_train, criterion, optimizer,\n lr_scheduler, epoch, visualize if args.debug else None, args)\n\n # evaluate on validation set\n confmat = validate(val_target_loader, model, interp_val, criterion, None, args)\n print(confmat.format(train_source_dataset.classes))\n acc_global, acc, iu = confmat.compute()\n\n # calculate the mean iou over partial classes\n indexes = [train_source_dataset.classes.index(name) for name\n in train_source_dataset.evaluate_classes]\n iu = iu[indexes]\n mean_iou = iu.mean()\n\n # remember best acc@1 and save checkpoint\n torch.save(\n {\n 'model': model.state_dict(),\n 'optimizer': optimizer.state_dict(),\n 'lr_scheduler': lr_scheduler.state_dict(),\n 'epoch': epoch,\n 'args': args\n }, logger.get_checkpoint_path(epoch)\n )\n if mean_iou > best_iou:\n shutil.copy(logger.get_checkpoint_path(epoch), logger.get_checkpoint_path('best'))\n best_iou = max(best_iou, mean_iou)\n print(\"Target: {} Best: {}\".format(mean_iou, best_iou))\n\n logger.close()\n\n\ndef train(train_source_iter: ForeverDataIterator, train_target_iter: ForeverDataIterator,\n model, interp, criterion, optimizer: SGD,\n lr_scheduler: LambdaLR, epoch: int, visualize, args: argparse.Namespace):\n batch_time = AverageMeter('Time', ':4.2f')\n data_time = AverageMeter('Data', ':3.1f')\n losses_s = AverageMeter('Loss (s)', ':3.2f')\n losses_t = AverageMeter('Loss (t)', ':3.2f')\n losses_entropy_t = AverageMeter('Entropy (t)', ':3.2f')\n accuracies_s = Meter('Acc (s)', ':3.2f')\n accuracies_t = Meter('Acc (t)', ':3.2f')\n iou_s = Meter('IoU (s)', ':3.2f')\n iou_t = Meter('IoU (t)', ':3.2f')\n\n confmat_s = ConfusionMatrix(model.num_classes)\n confmat_t = ConfusionMatrix(model.num_classes)\n progress = ProgressMeter(\n args.iters_per_epoch,\n [batch_time, data_time, losses_s, losses_t, losses_entropy_t,\n accuracies_s, accuracies_t, iou_s, iou_t],\n prefix=\"Epoch: [{}]\".format(epoch))\n\n # switch to train mode\n model.train()\n\n end = time.time()\n for i in range(args.iters_per_epoch):\n optimizer.zero_grad()\n\n x_s, label_s = next(train_source_iter)\n x_t, label_t = next(train_target_iter)\n\n x_s = x_s.to(device)\n label_s = label_s.long().to(device)\n x_t = x_t.to(device)\n label_t = label_t.long().to(device)\n\n # measure data loading time\n data_time.update(time.time() - end)\n\n # compute output\n y_s = model(x_s)\n pred_s = interp(y_s)\n loss_cls_s = criterion(pred_s, label_s)\n loss_cls_s.backward()\n\n y_t = model(x_t)\n pred_t = interp(y_t)\n loss_cls_t = criterion(pred_t, label_t)\n loss_entropy_t = robust_entropy(y_t, args.ita)\n (args.entropy_weight * loss_entropy_t).backward()\n\n # compute gradient and do SGD step\n optimizer.step()\n lr_scheduler.step()\n\n # measure accuracy and record loss\n losses_s.update(loss_cls_s.item(), x_s.size(0))\n losses_t.update(loss_cls_t.item(), x_s.size(0))\n losses_entropy_t.update(loss_entropy_t.item(), x_s.size(0))\n\n confmat_s.update(label_s.flatten(), pred_s.argmax(1).flatten())\n confmat_t.update(label_t.flatten(), pred_t.argmax(1).flatten())\n acc_global_s, acc_s, iu_s = confmat_s.compute()\n acc_global_t, acc_t, iu_t = confmat_t.compute()\n accuracies_s.update(acc_s.mean().item())\n accuracies_t.update(acc_t.mean().item())\n iou_s.update(iu_s.mean().item())\n iou_t.update(iu_t.mean().item())\n\n # measure elapsed time\n batch_time.update(time.time() - end)\n end = time.time()\n\n if i % args.print_freq == 0:\n progress.display(i)\n\n if visualize is not None:\n visualize(x_s[0], pred_s[0], label_s[0], \"source_{}\".format(i))\n visualize(x_t[0], pred_t[0], label_t[0], \"target_{}\".format(i))\n\n\ndef validate(val_loader: DataLoader, model, interp, criterion, visualize, args: argparse.Namespace):\n batch_time = AverageMeter('Time', ':6.3f')\n losses = AverageMeter('Loss', ':.4e')\n acc = Meter('Acc', ':3.2f')\n iou = Meter('IoU', ':3.2f')\n progress = ProgressMeter(\n len(val_loader),\n [batch_time, losses, acc, iou],\n prefix='Test: ')\n\n # switch to evaluate mode\n model.eval()\n confmat = ConfusionMatrix(model.num_classes)\n\n with torch.no_grad():\n end = time.time()\n for i, (x, label) in enumerate(val_loader):\n x = x.to(device)\n label = label.long().to(device)\n\n # compute output\n output = interp(model(x))\n loss = criterion(output, label)\n\n # measure accuracy and record loss\n losses.update(loss.item(), x.size(0))\n confmat.update(label.flatten(), output.argmax(1).flatten())\n acc_global, accs, iu = confmat.compute()\n acc.update(accs.mean().item())\n iou.update(iu.mean().item())\n\n # measure elapsed time\n batch_time.update(time.time() - end)\n end = time.time()\n\n if i % args.print_freq == 0:\n progress.display(i)\n\n if visualize is not None:\n visualize(x[0], output[0], label[0], \"val_{}\".format(i))\n\n return confmat\n\n\nif __name__ == '__main__':\n architecture_names = sorted(\n name for name in models.__dict__\n if name.islower() and not name.startswith(\"__\")\n and callable(models.__dict__[name])\n )\n dataset_names = sorted(\n name for name in datasets.__dict__\n if not name.startswith(\"__\") and callable(datasets.__dict__[name])\n )\n\n parser = argparse.ArgumentParser(description='FDA for Segmentation Domain Adaptation')\n # dataset parameters\n parser.add_argument('source_root', help='root path of the source dataset')\n parser.add_argument('target_root', help='root path of the target dataset')\n parser.add_argument('-s', '--source', help='source domain(s)')\n parser.add_argument('-t', '--target', help='target domain(s)')\n parser.add_argument('--resize-ratio', nargs='+', type=float, default=(1.5, 8 / 3.),\n help='the resize ratio for the random resize crop')\n parser.add_argument('--train-size', nargs='+', type=int, default=(1024, 512),\n help='the input and output image size during training')\n parser.add_argument('--test-input-size', nargs='+', type=int, default=(1024, 512),\n help='the input image size during test')\n parser.add_argument('--test-output-size', nargs='+', type=int, default=(2048, 1024),\n help='the output image size during test')\n # model parameters\n parser.add_argument('-a', '--arch', metavar='ARCH', default='deeplabv2_resnet101',\n choices=architecture_names,\n help='backbone architecture: ' +\n ' | '.join(architecture_names) +\n ' (default: deeplabv2_resnet101)')\n parser.add_argument(\"--entropy-weight\", type=float, default=0., help=\"weight for entropy\")\n parser.add_argument(\"--ita\", type=float, default=2.0, help=\"ita for robust entropy\")\n parser.add_argument(\"--beta\", type=int, default=1, help=\"beta for FDA\")\n parser.add_argument(\"--resume\", type=str, default=None,\n help=\"Where restore model parameters from.\")\n # training parameters\n parser.add_argument('-b', '--batch-size', default=2, type=int,\n metavar='N',\n help='mini-batch size (default: 2)')\n parser.add_argument('--lr', '--learning-rate', default=2.5e-3, type=float,\n metavar='LR', help='initial learning rate', dest='lr')\n parser.add_argument(\"--momentum\", type=float, default=0.9, help=\"Momentum component of the optimiser.\")\n parser.add_argument(\"--weight-decay\", type=float, default=0.0005, help=\"Regularisation parameter for L2-loss.\")\n parser.add_argument(\"--lr-power\", type=float, default=0.9,\n help=\"Decay parameter to compute the learning rate (only for deeplab).\")\n parser.add_argument('-j', '--workers', default=4, type=int, metavar='N',\n help='number of data loading workers (default: 4)')\n parser.add_argument('--epochs', default=60, type=int, metavar='N',\n help='number of total epochs to run')\n parser.add_argument('--start-epoch', default=0, type=int, metavar='N',\n help='start epoch')\n parser.add_argument('-i', '--iters-per-epoch', default=2500, type=int,\n help='Number of iterations per epoch')\n parser.add_argument('-p', '--print-freq', default=100, type=int,\n metavar='N', help='print frequency (default: 100)')\n parser.add_argument('--seed', default=None, type=int,\n help='seed for initializing training. ')\n parser.add_argument(\"--ignore-label\", type=int, default=255,\n help=\"The index of the label to ignore during the training.\")\n parser.add_argument(\"--log\", type=str, default='fda',\n help=\"Where to save logs, checkpoints and debugging images.\")\n parser.add_argument(\"--phase\", type=str, default='train', choices=['train', 'test'],\n help=\"When phase is 'test', only test the model.\")\n parser.add_argument('--debug', action=\"store_true\",\n help='In the debug mode, save images and predictions during training')\n args = parser.parse_args()\n main(args)\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/examples/domain_adaptation/semantic_segmentation/fda.sh",
"content": "# GTA5 to Cityscapes\nCUDA_VISIBLE_DEVICES=0 python fda.py data/GTA5 data/Cityscapes -s GTA5 -t Cityscapes \\\n --log logs/fda/gtav2cityscapes --debug\n\n# Synthia to Cityscapes\nCUDA_VISIBLE_DEVICES=0 python fda.py data/synthia data/Cityscapes -s Synthia -t Cityscapes \\\n --log logs/fda/synthia2cityscapes --debug\n\n# Cityscapes to FoggyCityscapes\nCUDA_VISIBLE_DEVICES=0 python fda.py data/Cityscapes data/Cityscapes -s Cityscapes -t FoggyCityscapes \\\n --log logs/fda/cityscapes2foggy --debug\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/examples/domain_adaptation/semantic_segmentation/source_only.py",
"content": "\"\"\"\n@author: Junguang Jiang\n@contact: JiangJunguang1123@outlook.com\n\"\"\"\nimport random\nimport time\nimport warnings\nimport sys\nimport argparse\nfrom PIL import Image\nimport numpy as np\nimport shutil\n\nimport torch\nimport torch.nn as nn\nimport torch.backends.cudnn as cudnn\nfrom torch.optim import SGD\nfrom torch.optim.lr_scheduler import LambdaLR\nfrom torch.utils.data import DataLoader\n\nsys.path.append('../../..')\nimport common.vision.models.segmentation as models\nimport common.vision.datasets.segmentation as datasets\nimport common.vision.transforms.segmentation as T\nfrom common.vision.transforms import DeNormalizeAndTranspose\nfrom common.utils.data import ForeverDataIterator\nfrom common.utils.metric import ConfusionMatrix\nfrom common.utils.meter import AverageMeter, ProgressMeter, Meter\nfrom common.utils.logger import CompleteLogger\n\ndevice = torch.device(\"cuda\" if torch.cuda.is_available() else \"cpu\")\n\n\ndef main(args: argparse.Namespace):\n logger = CompleteLogger(args.log, args.phase)\n print(args)\n\n if args.seed is not None:\n random.seed(args.seed)\n torch.manual_seed(args.seed)\n cudnn.deterministic = True\n warnings.warn('You have chosen to seed training. '\n 'This will turn on the CUDNN deterministic setting, '\n 'which can slow down your training considerably! '\n 'You may see unexpected behavior when restarting '\n 'from checkpoints.')\n\n cudnn.benchmark = True\n\n # Data loading code\n source_dataset = datasets.__dict__[args.source]\n train_source_dataset = source_dataset(\n root=args.source_root,\n transforms=T.Compose([\n T.RandomResizedCrop(size=args.train_size, ratio=args.resize_ratio, scale=args.resize_scale),\n T.ColorJitter(brightness=0.3, contrast=0.3),\n T.RandomHorizontalFlip(),\n T.NormalizeAndTranspose(),\n ]),\n )\n train_source_loader = DataLoader(train_source_dataset, batch_size=args.batch_size,\n shuffle=True, num_workers=args.workers, pin_memory=True, drop_last=True)\n\n target_dataset = datasets.__dict__[args.target]\n val_target_dataset = target_dataset(\n root=args.target_root, split='val',\n transforms=T.Compose([\n T.Resize(image_size=args.test_input_size, label_size=args.test_output_size),\n T.NormalizeAndTranspose(),\n ]),\n )\n val_target_loader = DataLoader(val_target_dataset, batch_size=1, shuffle=False, pin_memory=True)\n\n train_source_iter = ForeverDataIterator(train_source_loader)\n\n # create model\n model = models.__dict__[args.arch](num_classes=train_source_dataset.num_classes).to(device)\n # define optimizer and lr scheduler\n optimizer = SGD(model.get_parameters(), lr=args.lr, momentum=args.momentum, weight_decay=args.weight_decay)\n lr_scheduler = LambdaLR(optimizer, lambda x: args.lr * (1. - float(x) / args.epochs / args.iters_per_epoch)\n ** (args.lr_power))\n\n # optionally resume from a checkpoint\n if args.resume:\n checkpoint = torch.load(args.resume, map_location='cpu')\n model.load_state_dict(checkpoint['model'])\n optimizer.load_state_dict(checkpoint['optimizer'])\n lr_scheduler.load_state_dict(checkpoint['lr_scheduler'])\n args.start_epoch = checkpoint['epoch'] + 1\n\n # define loss function (criterion)\n criterion = torch.nn.CrossEntropyLoss(ignore_index=args.ignore_label).to(device)\n interp_train = nn.Upsample(size=args.train_size[::-1], mode='bilinear', align_corners=True)\n interp_val = nn.Upsample(size=args.test_output_size[::-1], mode='bilinear', align_corners=True)\n\n # define visualization function\n decode = train_source_dataset.decode_target\n\n def visualize(image, pred, label, prefix):\n \"\"\"\n Args:\n image (tensor): 3 x H x W\n pred (tensor): C x H x W\n label (tensor): H x W\n prefix: prefix of the saving image\n \"\"\"\n image = image.detach().cpu().numpy()\n pred = pred.detach().max(dim=0)[1].cpu().numpy()\n label = label.cpu().numpy()\n for tensor, name in [\n (Image.fromarray(np.uint8(DeNormalizeAndTranspose()(image))), \"image\"),\n (decode(label), \"label\"),\n (decode(pred), \"pred\")\n ]:\n tensor.save(logger.get_image_path(\"{}_{}.png\".format(prefix, name)))\n\n if args.phase == 'test':\n confmat = validate(val_target_loader, model, interp_val, criterion, visualize, args)\n print(confmat)\n return\n\n # start training\n best_iou = 0.\n for epoch in range(args.start_epoch, args.epochs):\n logger.set_epoch(epoch)\n print(lr_scheduler.get_lr())\n\n # train for one epoch\n train(train_source_iter, model, interp_train, criterion, optimizer,\n lr_scheduler, epoch, visualize if args.debug else None, args)\n\n # evaluate on validation set\n confmat = validate(val_target_loader, model, interp_val, criterion, visualize if args.debug else None, args)\n print(confmat.format(train_source_dataset.classes))\n acc_global, acc, iu = confmat.compute()\n\n # calculate the mean iou over partial classes\n indexes = [train_source_dataset.classes.index(name) for name\n in train_source_dataset.evaluate_classes]\n iu = iu[indexes]\n mean_iou = iu.mean()\n\n # remember best iou and save checkpoint\n torch.save(\n {\n 'model': model.state_dict(),\n 'optimizer': optimizer.state_dict(),\n 'lr_scheduler': lr_scheduler.state_dict(),\n 'epoch': epoch,\n 'args': args\n }, logger.get_checkpoint_path(epoch)\n )\n if mean_iou > best_iou:\n shutil.copy(logger.get_checkpoint_path(epoch), logger.get_checkpoint_path('best'))\n best_iou = max(best_iou, mean_iou)\n print(\"Target: {} Best: {}\".format(mean_iou, best_iou))\n\n logger.close()\n\n\ndef train(train_source_iter: ForeverDataIterator, model, interp, criterion, optimizer: SGD,\n lr_scheduler: LambdaLR, epoch: int, visualize, args: argparse.Namespace):\n batch_time = AverageMeter('Time', ':4.2f')\n data_time = AverageMeter('Data', ':3.1f')\n losses_s = AverageMeter('Loss (s)', ':3.2f')\n accuracies_s = Meter('Acc (s)', ':3.2f')\n iou_s = Meter('IoU (s)', ':3.2f')\n\n confmat_s = ConfusionMatrix(model.num_classes)\n progress = ProgressMeter(\n args.iters_per_epoch,\n [batch_time, data_time, losses_s,\n accuracies_s, iou_s],\n prefix=\"Epoch: [{}]\".format(epoch))\n\n # switch to train mode\n model.train()\n\n end = time.time()\n for i in range(args.iters_per_epoch):\n optimizer.zero_grad()\n\n x_s, label_s = next(train_source_iter)\n x_s = x_s.to(device)\n label_s = label_s.long().to(device)\n # measure data loading time\n data_time.update(time.time() - end)\n\n # compute output\n y_s = model(x_s)\n pred_s = interp(y_s)\n loss_cls_s = criterion(pred_s, label_s)\n loss_cls_s.backward()\n\n # compute gradient and do SGD step\n optimizer.step()\n lr_scheduler.step()\n\n # measure accuracy and record loss\n losses_s.update(loss_cls_s.item(), x_s.size(0))\n confmat_s.update(label_s.flatten(), pred_s.argmax(1).flatten())\n acc_global_s, acc_s, iu_s = confmat_s.compute()\n accuracies_s.update(acc_s.mean().item())\n iou_s.update(iu_s.mean().item())\n\n # measure elapsed time\n batch_time.update(time.time() - end)\n end = time.time()\n\n if i % args.print_freq == 0:\n progress.display(i)\n\n if visualize is not None:\n visualize(x_s[0], pred_s[0], label_s[0], \"source_{}\".format(i))\n\n\ndef validate(val_loader: DataLoader, model, interp, criterion, visualize, args: argparse.Namespace):\n batch_time = AverageMeter('Time', ':6.3f')\n losses = AverageMeter('Loss', ':.4e')\n acc = Meter('Acc', ':3.2f')\n iou = Meter('IoU', ':3.2f')\n progress = ProgressMeter(\n len(val_loader),\n [batch_time, losses, acc, iou],\n prefix='Test: ')\n\n # switch to evaluate mode\n model.eval()\n confmat = ConfusionMatrix(model.num_classes)\n\n with torch.no_grad():\n end = time.time()\n for i, (x, label) in enumerate(val_loader):\n x = x.to(device)\n label = label.long().to(device)\n\n # compute output\n output = interp(model(x))\n loss = criterion(output, label)\n\n # measure accuracy and record loss\n losses.update(loss.item(), x.size(0))\n confmat.update(label.flatten(), output.argmax(1).flatten())\n acc_global, accs, iu = confmat.compute()\n acc.update(accs.mean().item())\n iou.update(iu.mean().item())\n\n # measure elapsed time\n batch_time.update(time.time() - end)\n end = time.time()\n\n if i % args.print_freq == 0:\n progress.display(i)\n\n if visualize is not None:\n visualize(x[0], output[0], label[0], \"val_{}\".format(i))\n\n return confmat\n\n\nif __name__ == '__main__':\n architecture_names = sorted(\n name for name in models.__dict__\n if name.islower() and not name.startswith(\"__\")\n and callable(models.__dict__[name])\n )\n dataset_names = sorted(\n name for name in datasets.__dict__\n if not name.startswith(\"__\") and callable(datasets.__dict__[name])\n )\n\n parser = argparse.ArgumentParser(description='Source Only for Segmentation Domain Adaptation')\n # dataset parameters\n parser.add_argument('source_root', help='root path of the source dataset')\n parser.add_argument('target_root', help='root path of the target dataset')\n parser.add_argument('-s', '--source', help='source domain(s)')\n parser.add_argument('-t', '--target', help='target domain(s)')\n parser.add_argument('--resize-ratio', nargs='+', type=float, default=(1.5, 8 / 3.),\n help='the resize ratio for the random resize crop')\n parser.add_argument('--resize-scale', nargs='+', type=float, default=(0.5, 1.),\n help='the resize scale for the random resize crop')\n parser.add_argument('--train-size', nargs='+', type=int, default=(1024, 512),\n help='the input and output image size during training')\n parser.add_argument('--test-input-size', nargs='+', type=int, default=(1024, 512),\n help='the input image size during test')\n parser.add_argument('--test-output-size', nargs='+', type=int, default=(2048, 1024),\n help='the output image size during test')\n # model parameters\n parser.add_argument('-a', '--arch', metavar='ARCH', default='deeplabv2_resnet101',\n choices=architecture_names,\n help='backbone architecture: ' +\n ' | '.join(architecture_names) +\n ' (default: deeplabv2_resnet101)')\n parser.add_argument(\"--resume\", type=str, default=None,\n help=\"Where restore model parameters from.\")\n # training parameters\n parser.add_argument('-b', '--batch-size', default=2, type=int,\n metavar='N',\n help='mini-batch size (default: 2)')\n parser.add_argument('--lr', '--learning-rate', default=2.5e-3, type=float,\n metavar='LR', help='initial learning rate', dest='lr')\n parser.add_argument(\"--momentum\", type=float, default=0.9, help=\"Momentum component of the optimiser.\")\n parser.add_argument(\"--weight-decay\", type=float, default=0.0005,\n help=\"Regularisation parameter for L2-loss.\")\n parser.add_argument(\"--lr-power\", type=float, default=0.9,\n help=\"Decay parameter to compute the learning rate (only for deeplab).\")\n parser.add_argument('-j', '--workers', default=4, type=int, metavar='N',\n help='number of data loading workers (default: 4)')\n parser.add_argument('--epochs', default=60, type=int, metavar='N',\n help='number of total epochs to run')\n parser.add_argument('--start-epoch', default=0, type=int, metavar='N',\n help='start epoch')\n parser.add_argument('-i', '--iters-per-epoch', default=2500, type=int,\n help='Number of iterations per epoch')\n parser.add_argument('-p', '--print-freq', default=100, type=int,\n metavar='N', help='print frequency (default: 100)')\n parser.add_argument('--seed', default=None, type=int,\n help='seed for initializing training. ')\n parser.add_argument(\"--ignore-label\", type=int, default=255,\n help=\"The index of the label to ignore during the training.\")\n parser.add_argument(\"--log\", type=str, default='src_only',\n help=\"Where to save logs, checkpoints and debugging images.\")\n parser.add_argument(\"--phase\", type=str, default='train', choices=['train', 'test'],\n help=\"When phase is 'test', only test the model.\")\n parser.add_argument('--debug', action=\"store_true\",\n help='In the debug mode, save images and predictions during training')\n args = parser.parse_args()\n main(args)\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/examples/domain_adaptation/semantic_segmentation/source_only.sh",
"content": "# Source Only\n# GTA5 to Cityscapes\nCUDA_VISIBLE_DEVICES=0 python source_only.py data/GTA5 data/Cityscapes \\\n -s GTA5 -t Cityscapes --log logs/src_only/gtav2cityscapes\n\n# Synthia to Cityscapes\nCUDA_VISIBLE_DEVICES=0 python source_only.py data/synthia data/Cityscapes \\\n -s Synthia -t Cityscapes --log logs/src_only/synthia2cityscapes\n\n# Cityscapes to FoggyCityscapes\nCUDA_VISIBLE_DEVICES=0 python source_only.py data/Cityscapes data/Cityscapes \\\n -s Cityscapes -t FoggyCityscapes --log logs/src_only/cityscapes2foggy\n\n# Oracle\n# Oracle Results on Cityscapes\nCUDA_VISIBLE_DEVICES=0 python source_only.py data/Cityscapes data/Cityscapes \\\n -s Cityscapes -t Cityscapes --log logs/oracle/cityscapes\n\n# Oracle Results on Foggy Cityscapes\nCUDA_VISIBLE_DEVICES=0 python source_only.py data/Cityscapes data/Cityscapes \\\n -s FoggyCityscapes -t FoggyCityscapes --log logs/oracle/foggy_cityscapes\n\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/examples/domain_generalization/image_classification/README.md",
"content": "# Domain Generalization for Image Classification\n\n## Installation\nExample scripts can deal with [WILDS datasets](https://wilds.stanford.edu/).\nYou should first install ``wilds`` before using these scripts.\n\n```\npip install wilds\n```\n\nExample scripts also support all models in [PyTorch-Image-Models](https://github.com/rwightman/pytorch-image-models).\nYou also need to install timm to use PyTorch-Image-Models.\n\n```\npip install timm\n```\n\n## Dataset\n\nFollowing datasets can be downloaded automatically:\n\n- [Office31](https://www.cc.gatech.edu/~judy/domainadapt/)\n- [OfficeHome](https://www.hemanthdv.org/officeHomeDataset.html)\n- [DomainNet](http://ai.bu.edu/M3SDA/)\n- [PACS](https://domaingeneralization.github.io/#data)\n- [iwildcam (WILDS)](https://wilds.stanford.edu/datasets/)\n- [camelyon17 (WILDS)](https://wilds.stanford.edu/datasets/)\n- [fmow (WILDS)](https://wilds.stanford.edu/datasets/)\n\n## Supported Methods\n\n- [Two at Once: Enhancing Learning and Generalization Capacities via IBN-Net (IBN-Net, 2018 ECCV)](https://openaccess.thecvf.com/content_ECCV_2018/papers/Xingang_Pan_Two_at_Once_ECCV_2018_paper.pdf)\n- [Domain Generalization with MixStyle (MixStyle, 2021 ICLR)](https://arxiv.org/abs/2104.02008)\n- [Learning to Generalize: Meta-Learning for Domain Generalization (MLDG, 2018 AAAI)](https://arxiv.org/pdf/1710.03463.pdf)\n- [Invariant Risk Minimization (IRM)](https://arxiv.org/abs/1907.02893)\n- [Out-of-Distribution Generalization via Risk Extrapolation (VREx, 2021 ICML)](https://arxiv.org/abs/2003.00688)\n- [Distributionally Robust Neural Networks for Group Shifts: On the Importance of Regularization for Worst-Case Generalization (GroupDRO)](https://arxiv.org/abs/1911.08731)\n- [Deep CORAL: Correlation Alignment for Deep Domain Adaptation (Deep Coral, 2016 ECCV)](https://arxiv.org/abs/1607.01719)\n\n## Experiment and Results\n\nThe shell files give the script to reproduce the [benchmarks](/docs/dglib/benchmarks/image_classification.rst) with specified hyper-parameters.\nFor example, if you want to reproduce IRM on Office-Home, use the following script\n\n```shell script\n# Train with IRM on Office-Home Ar Cl Rw -> Pr task using ResNet 50.\n# Assume you have put the datasets under the path `data/office-home`, \n# or you are glad to download the datasets automatically from the Internet to this path\nCUDA_VISIBLE_DEVICES=0 python irm.py data/office-home -d OfficeHome -s Ar Cl Rw -t Pr -a resnet50 --seed 0 --log logs/irm/OfficeHome_Pr\n```\n\nFor more information please refer to [Get Started](/docs/get_started/quickstart.rst) for help.\n\n## Citation\nIf you use these methods in your research, please consider citing.\n\n```\n@inproceedings{IBN-Net, \n author = {Xingang Pan, Ping Luo, Jianping Shi, and Xiaoou Tang}, \n title = {Two at Once: Enhancing Learning and Generalization Capacities via IBN-Net}, \n booktitle = {ECCV}, \n year = {2018} \n}\n\n@inproceedings{mixstyle,\n title={Domain Generalization with MixStyle},\n author={Zhou, Kaiyang and Yang, Yongxin and Qiao, Yu and Xiang, Tao},\n booktitle={ICLR},\n year={2021}\n}\n\n@inproceedings{MLDG,\n title={Learning to Generalize: Meta-Learning for Domain Generalization},\n author={Li, Da and Yang, Yongxin and Song, Yi-Zhe and Hospedales, Timothy},\n booktitle={AAAI},\n year={2018}\n}\n \n@misc{IRM,\n title={Invariant Risk Minimization}, \n author={Martin Arjovsky and Léon Bottou and Ishaan Gulrajani and David Lopez-Paz},\n year={2020},\n eprint={1907.02893},\n archivePrefix={arXiv},\n primaryClass={stat.ML}\n}\n\n@inproceedings{VREx,\n title={Out-of-Distribution Generalization via Risk Extrapolation (REx)}, \n author={David Krueger and Ethan Caballero and Joern-Henrik Jacobsen and Amy Zhang and Jonathan Binas and Dinghuai Zhang and Remi Le Priol and Aaron Courville},\n year={2021},\n booktitle={ICML},\n}\n\n@inproceedings{GroupDRO,\n title={Distributionally Robust Neural Networks for Group Shifts: On the Importance of Regularization for Worst-Case Generalization}, \n author={Shiori Sagawa and Pang Wei Koh and Tatsunori B. Hashimoto and Percy Liang},\n year={2020},\n booktitle={ICLR}\n}\n\n@inproceedings{deep_coral,\n title={Deep coral: Correlation alignment for deep domain adaptation},\n author={Sun, Baochen and Saenko, Kate},\n booktitle={ECCV},\n year={2016},\n}\n```"
},
{
"path": "domain_generalization/Transfer-Learning-Library/examples/domain_generalization/image_classification/baseline.py",
"content": "\"\"\"\n@author: Baixu Chen\n@contact: cbx_99_hasta@outlook.com\n\"\"\"\nimport random\nimport time\nimport warnings\nimport sys\nimport argparse\nimport shutil\nimport os.path as osp\n\nimport torch\nimport torch.nn as nn\nimport torch.backends.cudnn as cudnn\nfrom torch.optim import SGD\nfrom torch.optim.lr_scheduler import CosineAnnealingLR\nfrom torch.utils.data import DataLoader\nimport torch.nn.functional as F\n\nsys.path.append('../../..')\nfrom dglib.modules.classifier import ImageClassifier as Classifier\nfrom common.utils.data import ForeverDataIterator\nfrom common.utils.metric import accuracy\nfrom common.utils.meter import AverageMeter, ProgressMeter\nfrom common.utils.logger import CompleteLogger\nfrom common.utils.analysis import tsne, a_distance\n\nsys.path.append('.')\nimport utils\n\ndevice = torch.device(\"cuda\" if torch.cuda.is_available() else \"cpu\")\n\nclass TransformerDecorator(torch.nn.Module):\n def __init__(self, backbone=None, add_bt=3, dim=2048, eval_global=0):\n super(TransformerDecorator, self).__init__()\n self.encoder_layers = torch.nn.TransformerEncoderLayer(dim, 4, dim, 0.5)\n self.eval_global = eval_global\n self.add_bt = add_bt\n self.backbone = backbone\n\n def forward_feats(self, feature):\n feature = self.backbone(feature)\n if self.training and self.add_bt > 0:\n pre_feature = feature\n feature = feature.unsqueeze(1)\n feature = self.encoder_layers(feature)\n # feature = self.encoder_layers(feature, feature, feature)\n feature = feature.squeeze(1)\n if self.add_bt:\n return torch.cat([pre_feature, feature], dim=0)\n return feature\n\n def forward(self, feature):\n feature = self.forward_feats(feature)\n return feature\n\n\ndef main(args: argparse.Namespace):\n logger = CompleteLogger(args.log, args.phase)\n print(args)\n\n if args.seed is not None:\n random.seed(args.seed)\n torch.manual_seed(args.seed)\n cudnn.deterministic = True\n warnings.warn('You have chosen to seed training. '\n 'This will turn on the CUDNN deterministic setting, '\n 'which can slow down your training considerably! '\n 'You may see unexpected behavior when restarting '\n 'from checkpoints.')\n\n cudnn.benchmark = True\n\n # Data loading code\n train_transform = utils.get_train_transform(args.train_resizing, random_horizontal_flip=True,\n random_color_jitter=True, random_gray_scale=True)\n val_transform = utils.get_val_transform(args.val_resizing)\n print(\"train_transform: \", train_transform)\n print(\"val_transform: \", val_transform)\n\n train_dataset, num_classes = utils.get_dataset(dataset_name=args.data, root=args.root, task_list=args.sources,\n split='train', download=True, transform=train_transform,\n seed=args.seed)\n train_loader = DataLoader(train_dataset, batch_size=args.batch_size,\n shuffle=True, num_workers=args.workers, drop_last=True)\n val_dataset, _ = utils.get_dataset(dataset_name=args.data, root=args.root, task_list=args.sources, split='val',\n download=True, transform=val_transform, seed=args.seed)\n val_loader = DataLoader(val_dataset, batch_size=args.batch_size, shuffle=False, num_workers=args.workers)\n test_dataset, _ = utils.get_dataset(dataset_name=args.data, root=args.root, task_list=args.targets, split='test',\n download=True, transform=val_transform, seed=args.seed)\n test_loader = DataLoader(test_dataset, batch_size=args.batch_size, shuffle=False, num_workers=args.workers)\n\n print(\"train_dataset_size: \", len(train_dataset))\n print('val_dataset_size: ', len(val_dataset))\n print(\"test_dataset_size: \", len(test_dataset))\n train_iter = ForeverDataIterator(train_loader)\n\n # create model\n print(\"=> using pre-trained model '{}'\".format(args.arch))\n backbone = utils.get_model(args.arch)\n pool_layer = nn.Identity() if args.no_pool else None\n pool_layer = nn.Sequential(\n nn.AdaptiveAvgPool2d(output_size=(1, 1)),\n nn.Flatten(),\n )\n pool_layer = TransformerDecorator(pool_layer, add_bt=args.add_bt, dim=backbone.out_features)\n classifier = Classifier(backbone, num_classes, freeze_bn=args.freeze_bn, dropout_p=args.dropout_p,\n finetune=args.finetune, pool_layer=pool_layer).to(device)\n\n # define optimizer and lr scheduler\n optimizer = SGD(classifier.get_parameters(base_lr=args.lr), args.lr, momentum=args.momentum, weight_decay=args.wd,\n nesterov=True)\n lr_scheduler = CosineAnnealingLR(optimizer, args.epochs * args.iters_per_epoch)\n\n # resume from the best checkpoint\n if args.phase != 'train':\n checkpoint = torch.load(logger.get_checkpoint_path('best'), map_location='cpu')\n classifier.load_state_dict(checkpoint)\n\n # analysis the model\n if args.phase == 'analysis':\n # extract features from both domains\n feature_extractor = nn.Sequential(classifier.backbone, classifier.pool_layer, classifier.bottleneck).to(device)\n source_feature = utils.collect_feature(val_loader, feature_extractor, device, max_num_features=100)\n target_feature = utils.collect_feature(test_loader, feature_extractor, device, max_num_features=100)\n print(len(source_feature), len(target_feature))\n # plot t-SNE\n tSNE_filename = osp.join(logger.visualize_directory, 'TSNE.png')\n tsne.visualize(source_feature, target_feature, tSNE_filename)\n print(\"Saving t-SNE to\", tSNE_filename)\n # calculate A-distance, which is a measure for distribution discrepancy\n A_distance = a_distance.calculate(source_feature, target_feature, device)\n print(\"A-distance =\", A_distance)\n return\n\n if args.phase == 'test':\n acc1 = utils.validate(test_loader, classifier, args, device)\n print(acc1)\n return\n\n # start training\n best_val_acc1 = 0.\n best_test_acc1 = 0.\n for epoch in range(args.epochs):\n print(lr_scheduler.get_lr())\n # train for one epoch\n train(train_iter, classifier, optimizer, lr_scheduler, epoch, args)\n\n # evaluate on validation set\n print(\"Evaluate on validation set...\")\n acc1 = utils.validate(val_loader, classifier, args, device)\n\n # remember best acc@1 and save checkpoint\n torch.save(classifier.state_dict(), logger.get_checkpoint_path('latest'))\n if acc1 > best_val_acc1:\n shutil.copy(logger.get_checkpoint_path('latest'), logger.get_checkpoint_path('best'))\n best_val_acc1 = max(acc1, best_val_acc1)\n\n # evaluate on test set\n print(\"Evaluate on test set...\")\n best_test_acc1 = max(best_test_acc1, utils.validate(test_loader, classifier, args, device))\n\n # evaluate on test set\n classifier.load_state_dict(torch.load(logger.get_checkpoint_path('best')))\n acc1 = utils.validate(test_loader, classifier, args, device)\n print(\"test acc on test set = {}\".format(acc1))\n print(\"oracle acc on test set = {}\".format(best_test_acc1))\n logger.close()\n\n\ndef train(train_iter: ForeverDataIterator, model: Classifier, optimizer,\n lr_scheduler: CosineAnnealingLR, epoch: int, args: argparse.Namespace):\n batch_time = AverageMeter('Time', ':4.2f')\n data_time = AverageMeter('Data', ':3.1f')\n losses = AverageMeter('Loss', ':3.2f')\n cls_accs = AverageMeter('Cls Acc', ':3.1f')\n\n progress = ProgressMeter(\n args.iters_per_epoch,\n [batch_time, data_time, losses, cls_accs],\n prefix=\"Epoch: [{}]\".format(epoch))\n\n # switch to train mode\n model.train()\n\n end = time.time()\n for i in range(args.iters_per_epoch):\n x, labels, _ = next(train_iter)\n x = x.to(device)\n labels = labels.to(device)\n\n # measure data loading time\n data_time.update(time.time() - end)\n\n # compute output\n y, _ = model(x)\n if len(y) > len(labels):\n labels = torch.cat([labels, labels], dim=0)\n\n loss = F.cross_entropy(y, labels)\n\n cls_acc = accuracy(y, labels)[0]\n losses.update(loss.item(), x.size(0))\n cls_accs.update(cls_acc.item(), x.size(0))\n\n # compute gradient and do SGD step\n optimizer.zero_grad()\n loss.backward()\n optimizer.step()\n lr_scheduler.step()\n\n # measure elapsed time\n batch_time.update(time.time() - end)\n end = time.time()\n\n if i % args.print_freq == 0:\n progress.display(i)\n\n\nif __name__ == '__main__':\n parser = argparse.ArgumentParser(description='Baseline for Domain Generalization')\n # dataset parameters\n parser.add_argument('root', metavar='DIR',\n help='root path of dataset')\n parser.add_argument('-d', '--data', metavar='DATA', default='PACS',\n help='dataset: ' + ' | '.join(utils.get_dataset_names()) +\n ' (default: PACS)')\n parser.add_argument('-s', '--sources', nargs='+', default=None,\n help='source domain(s)')\n parser.add_argument('-t', '--targets', nargs='+', default=None,\n help='target domain(s)')\n parser.add_argument('--train-resizing', type=str, default='default')\n parser.add_argument('--val-resizing', type=str, default='default')\n # model parameters\n parser.add_argument('-a', '--arch', metavar='ARCH', default='resnet50',\n choices=utils.get_model_names(),\n help='backbone architecture: ' +\n ' | '.join(utils.get_model_names()) +\n ' (default: resnet50)')\n parser.add_argument('--no-pool', action='store_true', help='no pool layer after the feature extractor.')\n parser.add_argument('--finetune', action='store_true', help='whether use 10x smaller lr for backbone')\n parser.add_argument('--freeze-bn', action='store_true', help='whether freeze all bn layers')\n parser.add_argument('--dropout-p', type=float, default=0.1, help='only activated when freeze-bn is True')\n # training parameters\n parser.add_argument('-b', '--batch-size', default=36, type=int,\n metavar='N',\n help='mini-batch size (default: 36)')\n parser.add_argument('--lr', '--learning-rate', default=1e-3, type=float,\n metavar='LR', help='initial learning rate', dest='lr')\n parser.add_argument('--momentum', default=0.9, type=float, metavar='M',\n help='momentum')\n parser.add_argument('--wd', '--weight-decay', default=0.0005, type=float,\n metavar='W', help='weight decay (default: 5e-4)')\n parser.add_argument('-j', '--workers', default=4, type=int, metavar='N',\n help='number of data loading workers (default: 4)')\n parser.add_argument('--epochs', default=20, type=int, metavar='N',\n help='number of total epochs to run')\n parser.add_argument('-i', '--iters-per-epoch', default=500, type=int,\n help='Number of iterations per epoch')\n parser.add_argument('-p', '--print-freq', default=100, type=int,\n metavar='N', help='print frequency (default: 100)')\n parser.add_argument('--seed', default=0, type=int,\n help='seed for initializing training. ')\n parser.add_argument(\"--log\", type=str, default='baseline',\n help=\"Where to save logs, checkpoints and debugging images.\")\n parser.add_argument(\"--phase\", type=str, default='train', choices=['train', 'test', 'analysis'],\n help=\"When phase is 'test', only test the model.\"\n \"When phase is 'analysis', only analysis the model.\")\n parser.add_argument(\"--add_bt\", type=int, default=0,\n help=\"add_bt\")\n args = parser.parse_args()\n main(args)\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/examples/domain_generalization/image_classification/baseline.sh",
"content": "#!/usr/bin/env bash\n# ResNet50, PACS\nCUDA_VISIBLE_DEVICES=0 python baseline.py data/PACS -d PACS -s A C S -t P -a resnet50 --freeze-bn --seed 0 --log logs/baseline/PACS_P\nCUDA_VISIBLE_DEVICES=0 python baseline.py data/PACS -d PACS -s P C S -t A -a resnet50 --freeze-bn --seed 0 --log logs/baseline/PACS_A\nCUDA_VISIBLE_DEVICES=0 python baseline.py data/PACS -d PACS -s P A S -t C -a resnet50 --freeze-bn --seed 0 --log logs/baseline/PACS_C\nCUDA_VISIBLE_DEVICES=0 python baseline.py data/PACS -d PACS -s P A C -t S -a resnet50 --freeze-bn --seed 0 --log logs/baseline/PACS_S\n\n# ResNet50, Office-Home\nCUDA_VISIBLE_DEVICES=0 python baseline.py data/office-home -d OfficeHome -s Ar Cl Rw -t Pr -a resnet50 --seed 0 --log logs/baseline/OfficeHome_Pr\nCUDA_VISIBLE_DEVICES=0 python baseline.py data/office-home -d OfficeHome -s Ar Cl Pr -t Rw -a resnet50 --seed 0 --log logs/baseline/OfficeHome_Rw\nCUDA_VISIBLE_DEVICES=0 python baseline.py data/office-home -d OfficeHome -s Ar Rw Pr -t Cl -a resnet50 --seed 0 --log logs/baseline/OfficeHome_Cl\nCUDA_VISIBLE_DEVICES=0 python baseline.py data/office-home -d OfficeHome -s Cl Rw Pr -t Ar -a resnet50 --seed 0 --log logs/baseline/OfficeHome_Ar\n\n# ResNet50, DomainNet\nCUDA_VISIBLE_DEVICES=0 python baseline.py data/domainnet -d DomainNet -s i p q r s -t c -a resnet50 -i 2500 --lr 0.01 --seed 0 --log logs/baseline/DomainNet_c\nCUDA_VISIBLE_DEVICES=0 python baseline.py data/domainnet -d DomainNet -s c p q r s -t i -a resnet50 -i 2500 --lr 0.01 --seed 0 --log logs/baseline/DomainNet_i\nCUDA_VISIBLE_DEVICES=0 python baseline.py data/domainnet -d DomainNet -s c i q r s -t p -a resnet50 -i 2500 --lr 0.01 --seed 0 --log logs/baseline/DomainNet_p\nCUDA_VISIBLE_DEVICES=0 python baseline.py data/domainnet -d DomainNet -s c i p r s -t q -a resnet50 -i 2500 --lr 0.01 --seed 0 --log logs/baseline/DomainNet_q\nCUDA_VISIBLE_DEVICES=0 python baseline.py data/domainnet -d DomainNet -s c i p q s -t r -a resnet50 -i 2500 --lr 0.01 --seed 0 --log logs/baseline/DomainNet_r\nCUDA_VISIBLE_DEVICES=0 python baseline.py data/domainnet -d DomainNet -s c i p q r -t s -a resnet50 -i 2500 --lr 0.01 --seed 0 --log logs/baseline/DomainNet_s\n\n# ResNet50, Wilds Dataset\nCUDA_VISIBLE_DEVICES=0 python baseline.py data/wilds -d iwildcam --train-resizing 'res2x' --val-resizing 'res2x' \\\n -a resnet50 -b 16 --epochs 60 -i 1000 --lr 0.001 --finetune --seed 0 --log logs/baseline/iwildcam\nCUDA_VISIBLE_DEVICES=0 python baseline.py data/wilds -d camelyon17 -a resnet50 -b 36 --epochs 20 -i 1000 --lr 0.01 \\\n --seed 0 --log logs/baseline/camelyon17\n\n# DenseNet121, Wilds Dataset\nCUDA_VISIBLE_DEVICES=0 python baseline.py data/wilds -d fmow --train-resizing 'cen.crop' -a densenet121 \\\n -b 36 --epochs 60 -i 1000 --lr 0.003 --seed 0 --log logs/baseline/fmow\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/examples/domain_generalization/image_classification/coral.py",
"content": "\"\"\"\n@author: Baixu Chen\n@contact: cbx_99_hasta@outlook.com\n\"\"\"\nimport random\nimport time\nimport warnings\nimport sys\nimport argparse\nimport shutil\nimport os.path as osp\n\nimport torch\nimport torch.nn as nn\nimport torch.backends.cudnn as cudnn\nfrom torch.optim import SGD\nfrom torch.optim.lr_scheduler import CosineAnnealingLR\nfrom torch.utils.data import DataLoader\nimport torch.nn.functional as F\n\nsys.path.append('../../..')\nfrom dglib.generalization.coral import CorrelationAlignmentLoss\nfrom dglib.modules.sampler import RandomDomainSampler\nfrom dglib.modules.classifier import ImageClassifier as Classifier\nfrom common.utils.data import ForeverDataIterator\nfrom common.utils.metric import accuracy\nfrom common.utils.meter import AverageMeter, ProgressMeter\nfrom common.utils.logger import CompleteLogger\nfrom common.utils.analysis import tsne, a_distance\n\nsys.path.append('.')\nimport utils\n\ndevice = torch.device(\"cuda\" if torch.cuda.is_available() else \"cpu\")\n\nclass TransformerDecorator(torch.nn.Module):\n def __init__(self, backbone=None, add_bt=3, dim=2048, eval_global=0):\n super(TransformerDecorator, self).__init__()\n self.encoder_layers = torch.nn.TransformerEncoderLayer(dim, 4, dim, 0.5)\n self.eval_global = eval_global\n self.add_bt = add_bt\n self.backbone = backbone\n\n def forward_feats(self, feature):\n feature = self.backbone(feature)\n if self.training and self.add_bt > 0:\n pre_feature = feature\n feature = feature.unsqueeze(1)\n feature = self.encoder_layers(feature)\n # feature = self.encoder_layers(feature, feature, feature)\n feature = feature.squeeze(1)\n if self.add_bt:\n return torch.cat([pre_feature, feature], dim=0)\n return feature\n\n def forward(self, feature):\n feature = self.forward_feats(feature)\n return feature\n\n\ndef main(args: argparse.Namespace):\n logger = CompleteLogger(args.log, args.phase)\n print(args)\n\n if args.seed is not None:\n random.seed(args.seed)\n torch.manual_seed(args.seed)\n cudnn.deterministic = True\n warnings.warn('You have chosen to seed training. '\n 'This will turn on the CUDNN deterministic setting, '\n 'which can slow down your training considerably! '\n 'You may see unexpected behavior when restarting '\n 'from checkpoints.')\n\n cudnn.benchmark = True\n\n # Data loading code\n train_transform = utils.get_train_transform(args.train_resizing, random_horizontal_flip=True,\n random_color_jitter=True, random_gray_scale=True)\n val_transform = utils.get_val_transform(args.val_resizing)\n print(\"train_transform: \", train_transform)\n print(\"val_transform: \", val_transform)\n\n train_dataset, num_classes = utils.get_dataset(dataset_name=args.data, root=args.root, task_list=args.sources,\n split='train', download=True, transform=train_transform,\n seed=args.seed)\n sampler = RandomDomainSampler(train_dataset, args.batch_size, n_domains_per_batch=args.n_domains_per_batch)\n train_loader = DataLoader(train_dataset, batch_size=args.batch_size, num_workers=args.workers,\n sampler=sampler, drop_last=True)\n val_dataset, _ = utils.get_dataset(dataset_name=args.data, root=args.root, task_list=args.sources, split='val',\n download=True, transform=val_transform, seed=args.seed)\n val_loader = DataLoader(val_dataset, batch_size=args.batch_size, shuffle=False, num_workers=args.workers)\n test_dataset, _ = utils.get_dataset(dataset_name=args.data, root=args.root, task_list=args.targets, split='test',\n download=True, transform=val_transform, seed=args.seed)\n test_loader = DataLoader(test_dataset, batch_size=args.batch_size, shuffle=False, num_workers=args.workers)\n print(\"train_dataset_size: \", len(train_dataset))\n print('val_dataset_size: ', len(val_dataset))\n print(\"test_dataset_size: \", len(test_dataset))\n train_iter = ForeverDataIterator(train_loader)\n\n # create model\n print(\"=> using pre-trained model '{}'\".format(args.arch))\n backbone = utils.get_model(args.arch)\n pool_layer = nn.Identity() if args.no_pool else None\n pool_layer = nn.Sequential(\n nn.AdaptiveAvgPool2d(output_size=(1, 1)),\n nn.Flatten(),\n )\n pool_layer = TransformerDecorator(pool_layer, add_bt=args.add_bt, dim=backbone.out_features)\n classifier = Classifier(backbone, num_classes, freeze_bn=args.freeze_bn, dropout_p=args.dropout_p,\n finetune=args.finetune, pool_layer=pool_layer).to(device)\n\n # define optimizer and lr scheduler\n optimizer = SGD(classifier.get_parameters(base_lr=args.lr), args.lr, momentum=args.momentum, weight_decay=args.wd,\n nesterov=True)\n lr_scheduler = CosineAnnealingLR(optimizer, args.epochs * args.iters_per_epoch)\n\n # define loss function\n correlation_alignment_loss = CorrelationAlignmentLoss().to(device)\n\n # resume from the best checkpoint\n if args.phase != 'train':\n checkpoint = torch.load(logger.get_checkpoint_path('best'), map_location='cpu')\n classifier.load_state_dict(checkpoint)\n\n # analysis the model\n if args.phase == 'analysis':\n # extract features from both domains\n feature_extractor = nn.Sequential(classifier.backbone, classifier.pool_layer, classifier.bottleneck).to(device)\n source_feature = utils.collect_feature(val_loader, feature_extractor, device, max_num_features=100)\n target_feature = utils.collect_feature(test_loader, feature_extractor, device, max_num_features=100)\n print(len(source_feature), len(target_feature))\n # plot t-SNE\n tSNE_filename = osp.join(logger.visualize_directory, 'TSNE.png')\n tsne.visualize(source_feature, target_feature, tSNE_filename)\n print(\"Saving t-SNE to\", tSNE_filename)\n # calculate A-distance, which is a measure for distribution discrepancy\n A_distance = a_distance.calculate(source_feature, target_feature, device)\n print(\"A-distance =\", A_distance)\n return\n\n if args.phase == 'test':\n acc1 = utils.validate(test_loader, classifier, args, device)\n print(acc1)\n return\n\n # start training\n best_val_acc1 = 0.\n best_test_acc1 = 0.\n for epoch in range(args.epochs):\n\n print(lr_scheduler.get_lr())\n # train for one epoch\n train(train_iter, classifier, optimizer, lr_scheduler, correlation_alignment_loss, args.n_domains_per_batch,\n epoch, args)\n\n # evaluate on validation set\n print(\"Evaluate on validation set...\")\n acc1 = utils.validate(val_loader, classifier, args, device)\n\n # remember best acc@1 and save checkpoint\n torch.save(classifier.state_dict(), logger.get_checkpoint_path('latest'))\n if acc1 > best_val_acc1:\n shutil.copy(logger.get_checkpoint_path('latest'), logger.get_checkpoint_path('best'))\n best_val_acc1 = max(acc1, best_val_acc1)\n\n # evaluate on test set\n print(\"Evaluate on test set...\")\n best_test_acc1 = max(best_test_acc1, utils.validate(test_loader, classifier, args, device))\n\n # evaluate on test set\n classifier.load_state_dict(torch.load(logger.get_checkpoint_path('best')))\n acc1 = utils.validate(test_loader, classifier, args, device)\n print(\"test acc on test set = {}\".format(acc1))\n print(\"oracle acc on test set = {}\".format(best_test_acc1))\n logger.close()\n\n\ndef train(train_iter: ForeverDataIterator, model: Classifier, optimizer, lr_scheduler: CosineAnnealingLR,\n correlation_alignment_loss: CorrelationAlignmentLoss, n_domains_per_batch: int, epoch: int,\n args: argparse.Namespace):\n batch_time = AverageMeter('Time', ':4.2f')\n data_time = AverageMeter('Data', ':3.1f')\n losses = AverageMeter('Loss', ':3.2f')\n losses_ce = AverageMeter('CELoss', ':3.2f')\n losses_penalty = AverageMeter('Penalty Loss', ':3.2f')\n cls_accs = AverageMeter('Cls Acc', ':3.1f')\n\n progress = ProgressMeter(\n args.iters_per_epoch,\n [batch_time, data_time, losses, losses_ce, losses_penalty, cls_accs],\n prefix=\"Epoch: [{}]\".format(epoch))\n\n # switch to train mode\n model.train()\n\n end = time.time()\n for i in range(args.iters_per_epoch):\n x_all, labels_all, _ = next(train_iter)\n x_all = x_all.to(device)\n labels_all = labels_all.to(device)\n\n # compute output\n y_all, f_all = model(x_all)\n\n # measure data loading time\n data_time.update(time.time() - end)\n if len(y_all) > len(labels_all):\n labels_all = torch.cat([labels_all, labels_all], dim=0)\n y_all_ = y_all.chunk(n_domains_per_batch*2, dim=0)\n f_all_ = f_all.chunk(n_domains_per_batch*2, dim=0)\n labels_all_ = labels_all.chunk(n_domains_per_batch*2, dim=0)\n y_all = []\n f_all = []\n labels_all = []\n for i in range(n_domains_per_batch):\n y_all.append(torch.cat([y_all_[i], y_all_[i+n_domains_per_batch]], dim=0))\n f_all.append(torch.cat([f_all_[i], f_all_[i+n_domains_per_batch]], dim=0))\n labels_all.append(torch.cat([labels_all_[i], labels_all_[i+n_domains_per_batch]], dim=0))\n else:\n # separate into different domains\n y_all = y_all.chunk(n_domains_per_batch, dim=0)\n f_all = f_all.chunk(n_domains_per_batch, dim=0)\n labels_all = labels_all.chunk(n_domains_per_batch, dim=0)\n\n loss_ce = 0\n loss_penalty = 0\n cls_acc = 0\n for domain_i in range(n_domains_per_batch):\n # cls loss\n y_i, labels_i = y_all[domain_i], labels_all[domain_i]\n loss_ce += F.cross_entropy(y_i, labels_i)\n # update acc\n cls_acc += accuracy(y_i, labels_i)[0] / n_domains_per_batch\n # correlation alignment loss\n for domain_j in range(domain_i + 1, n_domains_per_batch):\n f_i = f_all[domain_i]\n f_j = f_all[domain_j]\n loss_penalty += correlation_alignment_loss(f_i, f_j)\n\n # normalize loss\n loss_ce /= n_domains_per_batch\n loss_penalty /= n_domains_per_batch * (n_domains_per_batch - 1) / 2\n\n loss = loss_ce + loss_penalty * args.trade_off\n\n losses.update(loss.item(), x_all.size(0))\n losses_ce.update(loss_ce.item(), x_all.size(0))\n losses_penalty.update(loss_penalty.item(), x_all.size(0))\n cls_accs.update(cls_acc.item(), x_all.size(0))\n\n # compute gradient and do SGD step\n optimizer.zero_grad()\n loss.backward()\n optimizer.step()\n lr_scheduler.step()\n\n # measure elapsed time\n batch_time.update(time.time() - end)\n end = time.time()\n\n if i % args.print_freq == 0:\n progress.display(i)\n\n\nif __name__ == '__main__':\n parser = argparse.ArgumentParser(description='CORAL for Domain Generalization')\n # dataset parameters\n parser.add_argument('root', metavar='DIR',\n help='root path of dataset')\n parser.add_argument('-d', '--data', metavar='DATA', default='PACS',\n help='dataset: ' + ' | '.join(utils.get_dataset_names()) +\n ' (default: PACS)')\n parser.add_argument('-s', '--sources', nargs='+', default=None,\n help='source domain(s)')\n parser.add_argument('-t', '--targets', nargs='+', default=None,\n help='target domain(s)')\n parser.add_argument('--train-resizing', type=str, default='default')\n parser.add_argument('--val-resizing', type=str, default='default')\n # model parameters\n parser.add_argument('-a', '--arch', metavar='ARCH', default='resnet50',\n choices=utils.get_model_names(),\n help='backbone architecture: ' +\n ' | '.join(utils.get_model_names()) +\n ' (default: resnet50)')\n parser.add_argument('--no-pool', action='store_true', help='no pool layer after the feature extractor.')\n parser.add_argument('--finetune', action='store_true', help='whether use 10x smaller lr for backbone')\n parser.add_argument('--freeze-bn', action='store_true', help='whether freeze all bn layers')\n parser.add_argument('--dropout-p', type=float, default=0.1, help='only activated when freeze-bn is True')\n # training parameters\n parser.add_argument('--trade-off', default=1, type=float,\n help='the trade off hyper parameter for correlation alignment loss')\n parser.add_argument('-b', '--batch-size', default=36, type=int,\n metavar='N',\n help='mini-batch size (default: 36)')\n parser.add_argument('--n-domains-per-batch', default=3, type=int,\n help='number of domains in each mini-batch')\n parser.add_argument('--lr', '--learning-rate', default=5e-4, type=float,\n metavar='LR', help='initial learning rate', dest='lr')\n parser.add_argument('--momentum', default=0.9, type=float, metavar='M',\n help='momentum')\n parser.add_argument('--wd', '--weight-decay', default=0.0005, type=float,\n metavar='W', help='weight decay (default: 5e-4)')\n parser.add_argument('-j', '--workers', default=4, type=int, metavar='N',\n help='number of data loading workers (default: 4)')\n parser.add_argument('--epochs', default=20, type=int, metavar='N',\n help='number of total epochs to run')\n parser.add_argument('-i', '--iters-per-epoch', default=500, type=int,\n help='Number of iterations per epoch')\n parser.add_argument('-p', '--print-freq', default=100, type=int,\n metavar='N', help='print frequency (default: 100)')\n parser.add_argument('--seed', default=None, type=int,\n help='seed for initializing training. ')\n parser.add_argument(\"--log\", type=str, default='coral',\n help=\"Where to save logs, checkpoints and debugging images.\")\n parser.add_argument(\"--phase\", type=str, default='train', choices=['train', 'test', 'analysis'],\n help=\"When phase is 'test', only test the model.\"\n \"When phase is 'analysis', only analysis the model.\")\n parser.add_argument(\"--add_bt\", type=int, default=0,\n help=\"add_bt\")\n args = parser.parse_args()\n main(args)\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/examples/domain_generalization/image_classification/coral.sh",
"content": "#!/usr/bin/env bash\n# ResNet50, PACS\nCUDA_VISIBLE_DEVICES=0 python coral.py data/PACS -d PACS -s A C S -t P -a resnet50 --freeze-bn --seed 0 --log logs/coral/PACS_P\nCUDA_VISIBLE_DEVICES=0 python coral.py data/PACS -d PACS -s P C S -t A -a resnet50 --freeze-bn --seed 0 --log logs/coral/PACS_A\nCUDA_VISIBLE_DEVICES=0 python coral.py data/PACS -d PACS -s P A S -t C -a resnet50 --freeze-bn --seed 0 --log logs/coral/PACS_C\nCUDA_VISIBLE_DEVICES=0 python coral.py data/PACS -d PACS -s P A C -t S -a resnet50 --freeze-bn --seed 0 --log logs/coral/PACS_S\n\n# ResNet50, Office-Home\nCUDA_VISIBLE_DEVICES=0 python coral.py data/office-home -d OfficeHome -s Ar Cl Rw -t Pr -a resnet50 --seed 0 --log logs/coral/OfficeHome_Pr\nCUDA_VISIBLE_DEVICES=0 python coral.py data/office-home -d OfficeHome -s Ar Cl Pr -t Rw -a resnet50 --seed 0 --log logs/coral/OfficeHome_Rw\nCUDA_VISIBLE_DEVICES=0 python coral.py data/office-home -d OfficeHome -s Ar Rw Pr -t Cl -a resnet50 --seed 0 --log logs/coral/OfficeHome_Cl\nCUDA_VISIBLE_DEVICES=0 python coral.py data/office-home -d OfficeHome -s Cl Rw Pr -t Ar -a resnet50 --seed 0 --log logs/coral/OfficeHome_Ar\n\n# ResNet50, DomainNet\nCUDA_VISIBLE_DEVICES=0 python coral.py data/domainnet -d DomainNet -s i p q r s -t c -a resnet50 -i 2500 --lr 0.01 --seed 0 --log logs/coral/DomainNet_c\nCUDA_VISIBLE_DEVICES=0 python coral.py data/domainnet -d DomainNet -s c p q r s -t i -a resnet50 -i 2500 --lr 0.01 --seed 0 --log logs/coral/DomainNet_i\nCUDA_VISIBLE_DEVICES=0 python coral.py data/domainnet -d DomainNet -s c i q r s -t p -a resnet50 -i 2500 --lr 0.01 --seed 0 --log logs/coral/DomainNet_p\nCUDA_VISIBLE_DEVICES=0 python coral.py data/domainnet -d DomainNet -s c i p r s -t q -a resnet50 -i 2500 --lr 0.01 --seed 0 --log logs/coral/DomainNet_q\nCUDA_VISIBLE_DEVICES=0 python coral.py data/domainnet -d DomainNet -s c i p q s -t r -a resnet50 -i 2500 --lr 0.01 --seed 0 --log logs/coral/DomainNet_r\nCUDA_VISIBLE_DEVICES=0 python coral.py data/domainnet -d DomainNet -s c i p q r -t s -a resnet50 -i 2500 --lr 0.01 --seed 0 --log logs/coral/DomainNet_s\n\n# ResNet50, Wilds Dataset\nCUDA_VISIBLE_DEVICES=0 python coral.py data/wilds -d iwildcam --train-resizing 'res2x' --val-resizing 'res2x' \\\n -a resnet50 -b 16 --n-domains-per-batch 2 --epochs 60 -i 1000 --lr 0.001 --finetune --trade-off 10 \\\n --seed 0 --log logs/coral/iwildcam\nCUDA_VISIBLE_DEVICES=0 python coral.py data/wilds -d camelyon17 -a resnet50 -b 36 --n-domains-per-batch 3 \\\n --epochs 20 -i 1000 --lr 0.01 --trade-off 0.1 --seed 0 --log logs/coral/camelyon17\n\n# DenseNet121, Wilds Dataset\nCUDA_VISIBLE_DEVICES=0 python coral.py data/wilds -d fmow --train-resizing 'cen.crop' -a densenet121 \\\n -b 36 --n-domains-per-batch 4 --epochs 60 -i 1000 --lr 0.003 --trade-off 0.1 --seed 0 --log logs/coral/fmow\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/examples/domain_generalization/image_classification/gpu_base.pbs",
"content": "#!/bin/bash\n#PBS -P ZHIHOU\n#PBS -l select=1:ngpus=1:ncpus=6:mem=28GB\n#PBS -l walltime=24:00:00\n#PBS -q alloc-xxx\nsource activate pyt\n\n#env\n\ncd \"$PBS_O_WORKDIR\"\n# ResNet50, PACS\n#CUDA_VISIBLE_DEVICES=0 python baseline.py data/PACS -d PACS -s A C S -t P -a resnet50 --freeze-bn --seed 0 --log logs/baseline/PACS_P\n#CUDA_VISIBLE_DEVICES=0 python baseline.py data/PACS -d PACS -s P C S -t A -a resnet50 --freeze-bn --seed 0 --log logs/baseline/PACS_A\n#CUDA_VISIBLE_DEVICES=0 python baseline.py data/PACS -d PACS -s P A S -t C -a resnet50 --freeze-bn --seed 0 --log logs/baseline/PACS_C\n#CUDA_VISIBLE_DEVICES=0 python baseline.py data/PACS -d PACS -s P A C -t S -a resnet50 --freeze-bn --seed 0 --log logs/baseline/PACS_S\n\n#CUDA_VISIBLE_DEVICES=0 python baseline.py data/PACS -d PACS -s A C S -t P -a resnet50 --freeze-bn --seed 1 --log logs/baseline/PACS_P\n#CUDA_VISIBLE_DEVICES=0 python baseline.py data/PACS -d PACS -s P C S -t A -a resnet50 --freeze-bn --seed 1 --log logs/baseline/PACS_A\n#CUDA_VISIBLE_DEVICES=0 python baseline.py data/PACS -d PACS -s P A S -t C -a resnet50 --freeze-bn --seed 1 --log logs/baseline/PACS_C\n#CUDA_VISIBLE_DEVICES=0 python baseline.py data/PACS -d PACS -s P A C -t S -a resnet50 --freeze-bn --seed 1 --log logs/baseline/PACS_S\n#\n#CUDA_VISIBLE_DEVICES=0 python baseline.py data/PACS -d PACS -s A C S -t P -a resnet50 --freeze-bn --seed 2 --log logs/baseline/PACS_P\n#CUDA_VISIBLE_DEVICES=0 python baseline.py data/PACS -d PACS -s P C S -t A -a resnet50 --freeze-bn --seed 2 --log logs/baseline/PACS_A\n#CUDA_VISIBLE_DEVICES=0 python baseline.py data/PACS -d PACS -s P A S -t C -a resnet50 --freeze-bn --seed 2 --log logs/baseline/PACS_C\n#CUDA_VISIBLE_DEVICES=0 python baseline.py data/PACS -d PACS -s P A C -t S -a resnet50 --freeze-bn --seed 2 --log logs/baseline/PACS_S\n\nCUDA_VISIBLE_DEVICES=0 python baseline.py data/PACS -d PACS -s P A C -t S -a resnet50 --freeze-bn --seed 0 --log logs/baseline/PACS_S_g16 --add_bt 16\nCUDA_VISIBLE_DEVICES=0 python baseline.py data/PACS -d PACS -s P A C -t S -a resnet50 --freeze-bn --seed 1 --log logs/baseline/PACS_S_g16 --add_bt 16\nCUDA_VISIBLE_DEVICES=0 python baseline.py data/PACS -d PACS -s P A C -t S -a resnet50 --freeze-bn --seed 2 --log logs/baseline/PACS_S_g16 --add_bt 16\n#\n#\n#CUDA_VISIBLE_DEVICES=0 python baseline.py data/PACS -d PACS -s A C S -t P -a resnet18 --freeze-bn --seed 0 --log logs/baseline_r18/PACS_P_g16 --add_bt 16\n#CUDA_VISIBLE_DEVICES=0 python baseline.py data/PACS -d PACS -s P C S -t A -a resnet18 --freeze-bn --seed 0 --log logs/baseline_r18/PACS_A_g16 --add_bt 16\n#CUDA_VISIBLE_DEVICES=0 python baseline.py data/PACS -d PACS -s P A S -t C -a resnet18 --freeze-bn --seed 0 --log logs/baseline_r18/PACS_C_g16 --add_bt 16\n#CUDA_VISIBLE_DEVICES=0 python baseline.py data/PACS -d PACS -s P A C -t S -a resnet18 --freeze-bn --seed 0 --log logs/baseline_r18/PACS_S_g16 --add_bt 16\n#\n#CUDA_VISIBLE_DEVICES=0 python baseline.py data/PACS -d PACS -s A C S -t P -a resnet18 --freeze-bn --seed 1 --log logs/baseline_r18/PACS_P_g16 --add_bt 16\n#CUDA_VISIBLE_DEVICES=0 python baseline.py data/PACS -d PACS -s P C S -t A -a resnet18 --freeze-bn --seed 1 --log logs/baseline_r18/PACS_A_g16 --add_bt 16\n#CUDA_VISIBLE_DEVICES=0 python baseline.py data/PACS -d PACS -s P A S -t C -a resnet18 --freeze-bn --seed 1 --log logs/baseline_r18/PACS_C_g16 --add_bt 16\n#CUDA_VISIBLE_DEVICES=0 python baseline.py data/PACS -d PACS -s P A C -t S -a resnet18 --freeze-bn --seed 1 --log logs/baseline_r18/PACS_S_g16 --add_bt 16\n###\n#CUDA_VISIBLE_DEVICES=0 python baseline.py data/PACS -d PACS -s A C S -t P -a resnet18 --freeze-bn --seed 2 --log logs/baseline_r18/PACS_P_g16 --add_bt 16\n#CUDA_VISIBLE_DEVICES=0 python baseline.py data/PACS -d PACS -s P C S -t A -a resnet18 --freeze-bn --seed 2 --log logs/baseline_r18/PACS_A_g16 --add_bt 16\n#CUDA_VISIBLE_DEVICES=0 python baseline.py data/PACS -d PACS -s P A S -t C -a resnet18 --freeze-bn --seed 2 --log logs/baseline_r18/PACS_C_g16 --add_bt 16\n#CUDA_VISIBLE_DEVICES=0 python baseline.py data/PACS -d PACS -s P A C -t S -a resnet18 --freeze-bn --seed 2 --log logs/baseline_r18/PACS_S_g16 --add_bt 16\n\n\n\n#CUDA_VISIBLE_DEVICES=0 python baseline.py data/PACS -d PACS -s A C S -t P -a resnet50 --freeze-bn --seed 0 --log logs/baseline/PACS_P_g16 --add_bt 16\n#CUDA_VISIBLE_DEVICES=0 python baseline.py data/PACS -d PACS -s P C S -t A -a resnet50 --freeze-bn --seed 0 --log logs/baseline/PACS_A_g16 --add_bt 16\n#CUDA_VISIBLE_DEVICES=0 python baseline.py data/PACS -d PACS -s P A S -t C -a resnet50 --freeze-bn --seed 0 --log logs/baseline/PACS_C_g16 --add_bt 16\n#CUDA_VISIBLE_DEVICES=0 python baseline.py data/PACS -d PACS -s P A C -t S -a resnet50 --freeze-bn --seed 0 --log logs/baseline/PACS_S_g16 --add_bt 16\n#\n#CUDA_VISIBLE_DEVICES=0 python baseline.py data/PACS -d PACS -s A C S -t P -a resnet50 --freeze-bn --seed 1 --log logs/baseline/PACS_P_g16 --add_bt 16\n#CUDA_VISIBLE_DEVICES=0 python baseline.py data/PACS -d PACS -s P C S -t A -a resnet50 --freeze-bn --seed 1 --log logs/baseline/PACS_A_g16 --add_bt 16\n#CUDA_VISIBLE_DEVICES=0 python baseline.py data/PACS -d PACS -s P A S -t C -a resnet50 --freeze-bn --seed 1 --log logs/baseline/PACS_C_g16 --add_bt 16\n#CUDA_VISIBLE_DEVICES=0 python baseline.py data/PACS -d PACS -s P A C -t S -a resnet50 --freeze-bn --seed 1 --log logs/baseline/PACS_S_g16 --add_bt 16\n#\n#CUDA_VISIBLE_DEVICES=0 python baseline.py data/PACS -d PACS -s A C S -t P -a resnet50 --freeze-bn --seed 2 --log logs/baseline/PACS_P_g16 --add_bt 16\n#CUDA_VISIBLE_DEVICES=0 python baseline.py data/PACS -d PACS -s P C S -t A -a resnet50 --freeze-bn --seed 2 --log logs/baseline/PACS_A_g16 --add_bt 16\n#CUDA_VISIBLE_DEVICES=0 python baseline.py data/PACS -d PACS -s P A S -t C -a resnet50 --freeze-bn --seed 2 --log logs/baseline/PACS_C_g16 --add_bt 16\n#CUDA_VISIBLE_DEVICES=0 python baseline.py data/PACS -d PACS -s P A C -t S -a resnet50 --freeze-bn --seed 2 --log logs/baseline/PACS_S_g16 --add_bt 16\n\n#\n#\n## ResNet50, Office-Home\n#CUDA_VISIBLE_DEVICES=0 python baseline.py data/office-home -d OfficeHome -s Ar Cl Rw -t Pr -a resnet50 --seed 0 --log logs/baseline/OfficeHome_Pr_g16 --add_bt 16\n#CUDA_VISIBLE_DEVICES=0 python baseline.py data/office-home -d OfficeHome -s Ar Cl Pr -t Rw -a resnet50 --seed 0 --log logs/baseline/OfficeHome_Rw_g16 --add_bt 16\n#CUDA_VISIBLE_DEVICES=0 python baseline.py data/office-home -d OfficeHome -s Ar Rw Pr -t Cl -a resnet50 --seed 0 --log logs/baseline/OfficeHome_Cl_g16 --add_bt 16\n#CUDA_VISIBLE_DEVICES=0 python baseline.py data/office-home -d OfficeHome -s Cl Rw Pr -t Ar -a resnet50 --seed 0 --log logs/baseline/OfficeHome_Ar_g16 --add_bt 16\n#\n#CUDA_VISIBLE_DEVICES=0 python baseline.py data/office-home -d OfficeHome -s Ar Cl Rw -t Pr -a resnet50 --seed 1 --log logs/baseline/OfficeHome_Pr_g16 --add_bt 16;\\\n#CUDA_VISIBLE_DEVICES=0 python baseline.py data/office-home -d OfficeHome -s Ar Cl Pr -t Rw -a resnet50 --seed 1 --log logs/baseline/OfficeHome_Rw_g16 --add_bt 16;\\\n#CUDA_VISIBLE_DEVICES=0 python baseline.py data/office-home -d OfficeHome -s Ar Rw Pr -t Cl -a resnet50 --seed 1 --log logs/baseline/OfficeHome_Cl_g16 --add_bt 16;\\\n#CUDA_VISIBLE_DEVICES=0 python baseline.py data/office-home -d OfficeHome -s Cl Rw Pr -t Ar -a resnet50 --seed 1 --log logs/baseline/OfficeHome_Ar_g16 --add_bt 16\n#\n#\n#CUDA_VISIBLE_DEVICES=0 python baseline.py data/office-home -d OfficeHome -s Ar Cl Rw -t Pr -a resnet50 --seed 2 --log logs/baseline/OfficeHome_Pr_g16 --add_bt 16;\\\n#CUDA_VISIBLE_DEVICES=0 python baseline.py data/office-home -d OfficeHome -s Ar Cl Pr -t Rw -a resnet50 --seed 2 --log logs/baseline/OfficeHome_Rw_g16 --add_bt 16;\\\n#CUDA_VISIBLE_DEVICES=0 python baseline.py data/office-home -d OfficeHome -s Ar Rw Pr -t Cl -a resnet50 --seed 2 --log logs/baseline/OfficeHome_Cl_g16 --add_bt 16;\\\n#CUDA_VISIBLE_DEVICES=0 python baseline.py data/office-home -d OfficeHome -s Cl Rw Pr -t Ar -a resnet50 --seed 2 --log logs/baseline/OfficeHome_Ar_g16 --add_bt 16\n\n\n\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/examples/domain_generalization/image_classification/gpu_coral.pbs",
"content": "#!/bin/bash\n#PBS -P ZHIHOU\n#PBS -l select=1:ngpus=1:ncpus=6:mem=28GB\n#PBS -l walltime=24:00:00\n#PBS -q alloc-xxx\nsource activate pyt\n\n#env\n\ncd \"$PBS_O_WORKDIR\"\n\n# ResNet50, PACS\n#CUDA_VISIBLE_DEVICES=0 python coral.py data/PACS -d PACS -s A C S -t P -a resnet50 --freeze-bn --seed 0 --log logs/coral/PACS_P_g16 --add_bt 16\n#CUDA_VISIBLE_DEVICES=0 python coral.py data/PACS -d PACS -s P C S -t A -a resnet50 --freeze-bn --seed 0 --log logs/coral/PACS_A_g16 --add_bt 16\n#CUDA_VISIBLE_DEVICES=0 python coral.py data/PACS -d PACS -s P A S -t C -a resnet50 --freeze-bn --seed 0 --log logs/coral/PACS_C_g16 --add_bt 16\n#CUDA_VISIBLE_DEVICES=0 python coral.py data/PACS -d PACS -s P A C -t S -a resnet50 --freeze-bn --seed 0 --log logs/coral/PACS_S_g16 --add_bt 16\n#\n#CUDA_VISIBLE_DEVICES=0 python coral.py data/PACS -d PACS -s A C S -t P -a resnet50 --freeze-bn --seed 1 --log logs/coral/PACS_P_g16 --add_bt 16\n#CUDA_VISIBLE_DEVICES=0 python coral.py data/PACS -d PACS -s P C S -t A -a resnet50 --freeze-bn --seed 1 --log logs/coral/PACS_A_g16 --add_bt 16\n#CUDA_VISIBLE_DEVICES=0 python coral.py data/PACS -d PACS -s P A S -t C -a resnet50 --freeze-bn --seed 1 --log logs/coral/PACS_C_g16 --add_bt 16\n#CUDA_VISIBLE_DEVICES=0 python coral.py data/PACS -d PACS -s P A C -t S -a resnet50 --freeze-bn --seed 1 --log logs/coral/PACS_S_g16 --add_bt 16\n#\n#CUDA_VISIBLE_DEVICES=0 python coral.py data/PACS -d PACS -s A C S -t P -a resnet50 --freeze-bn --seed 2 --log logs/coral/PACS_P_g16 --add_bt 16\n#CUDA_VISIBLE_DEVICES=0 python coral.py data/PACS -d PACS -s P C S -t A -a resnet50 --freeze-bn --seed 2 --log logs/coral/PACS_A_g16 --add_bt 16\n#CUDA_VISIBLE_DEVICES=0 python coral.py data/PACS -d PACS -s P A S -t C -a resnet50 --freeze-bn --seed 2 --log logs/coral/PACS_C_g16 --add_bt 16\n#CUDA_VISIBLE_DEVICES=0 python coral.py data/PACS -d PACS -s P A C -t S -a resnet50 --freeze-bn --seed 2 --log logs/coral/PACS_S_g16 --add_bt 16\n\n# ResNet50, Office-Home\n#CUDA_VISIBLE_DEVICES=0 python coral.py data/office-home -d OfficeHome -s Ar Cl Rw -t Pr -a resnet50 --seed 0 --log logs/coral/OfficeHome_Pr_g16 --add_bt 16\n#CUDA_VISIBLE_DEVICES=0 python coral.py data/office-home -d OfficeHome -s Ar Cl Pr -t Rw -a resnet50 --seed 0 --log logs/coral/OfficeHome_Rw_g16 --add_bt 16\n#CUDA_VISIBLE_DEVICES=0 python coral.py data/office-home -d OfficeHome -s Ar Rw Pr -t Cl -a resnet50 --seed 0 --log logs/coral/OfficeHome_Cl_g16 --add_bt 16\n#CUDA_VISIBLE_DEVICES=0 python coral.py data/office-home -d OfficeHome -s Cl Rw Pr -t Ar -a resnet50 --seed 0 --log logs/coral/OfficeHome_Ar_g16 --add_bt 16\n#\n#CUDA_VISIBLE_DEVICES=0 python coral.py data/office-home -d OfficeHome -s Ar Cl Rw -t Pr -a resnet50 --seed 1 --log logs/coral/OfficeHome_Pr\n#CUDA_VISIBLE_DEVICES=0 python coral.py data/office-home -d OfficeHome -s Ar Cl Pr -t Rw -a resnet50 --seed 1 --log logs/coral/OfficeHome_Rw\n#CUDA_VISIBLE_DEVICES=0 python coral.py data/office-home -d OfficeHome -s Ar Rw Pr -t Cl -a resnet50 --seed 1 --log logs/coral/OfficeHome_Cl\n#CUDA_VISIBLE_DEVICES=0 python coral.py data/office-home -d OfficeHome -s Cl Rw Pr -t Ar -a resnet50 --seed 1 --log logs/coral/OfficeHome_Ar\n\n##\n#CUDA_VISIBLE_DEVICES=0 python coral.py data/office-home -d OfficeHome -s Ar Cl Rw -t Pr -a resnet50 --seed 2 --log logs/coral/OfficeHome_Pr\n#CUDA_VISIBLE_DEVICES=0 python coral.py data/office-home -d OfficeHome -s Ar Cl Pr -t Rw -a resnet50 --seed 2 --log logs/coral/OfficeHome_Rw\n#CUDA_VISIBLE_DEVICES=0 python coral.py data/office-home -d OfficeHome -s Ar Rw Pr -t Cl -a resnet50 --seed 2 --log logs/coral/OfficeHome_Cl\n#CUDA_VISIBLE_DEVICES=0 python coral.py data/office-home -d OfficeHome -s Cl Rw Pr -t Ar -a resnet50 --seed 2 --log logs/coral/OfficeHome_Ar\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/examples/domain_generalization/image_classification/gpu_irm.pbs",
"content": "#!/bin/bash\n#PBS -P ZHIHOU\n#PBS -l select=1:ngpus=1:ncpus=6:mem=28GB\n#PBS -l walltime=24:00:00\n#PBS -q alloc-xxx\nsource activate pyt\n\n#env\n\ncd \"$PBS_O_WORKDIR\"\n#\n#CUDA_VISIBLE_DEVICES=0 python irm.py data/PACS -d PACS -s A C S -t P -a resnet50 --freeze-bn --seed 0 --log logs/irm/PACS_P_g16 --add_bt 16\n#CUDA_VISIBLE_DEVICES=0 python irm.py data/PACS -d PACS -s P C S -t A -a resnet50 --freeze-bn --seed 0 --log logs/irm/PACS_A_g16 --add_bt 16\n#CUDA_VISIBLE_DEVICES=0 python irm.py data/PACS -d PACS -s P A S -t C -a resnet50 --freeze-bn --seed 0 --log logs/irm/PACS_C_g16 --add_bt 16\n#CUDA_VISIBLE_DEVICES=0 python irm.py data/PACS -d PACS -s P A C -t S -a resnet50 --freeze-bn --seed 0 --log logs/irm/PACS_S_g16 --add_bt 16\n#\n#CUDA_VISIBLE_DEVICES=0 python irm.py data/PACS -d PACS -s A C S -t P -a resnet50 --freeze-bn --seed 1 --log logs/irm/PACS_P_g16 --add_bt 16\n#CUDA_VISIBLE_DEVICES=0 python irm.py data/PACS -d PACS -s P C S -t A -a resnet50 --freeze-bn --seed 1 --log logs/irm/PACS_A_g16 --add_bt 16\n#CUDA_VISIBLE_DEVICES=0 python irm.py data/PACS -d PACS -s P A S -t C -a resnet50 --freeze-bn --seed 1 --log logs/irm/PACS_C_g16 --add_bt 16\n#CUDA_VISIBLE_DEVICES=0 python irm.py data/PACS -d PACS -s P A C -t S -a resnet50 --freeze-bn --seed 1 --log logs/irm/PACS_S_g16 --add_bt 16\n#\n#CUDA_VISIBLE_DEVICES=0 python irm.py data/PACS -d PACS -s A C S -t P -a resnet50 --freeze-bn --seed 2 --log logs/irm/PACS_P_g16 --add_bt 16\n#CUDA_VISIBLE_DEVICES=0 python irm.py data/PACS -d PACS -s P C S -t A -a resnet50 --freeze-bn --seed 2 --log logs/irm/PACS_A_g16 --add_bt 16\n#CUDA_VISIBLE_DEVICES=0 python irm.py data/PACS -d PACS -s P A S -t C -a resnet50 --freeze-bn --seed 2 --log logs/irm/PACS_C_g16 --add_bt 16\n#CUDA_VISIBLE_DEVICES=0 python irm.py data/PACS -d PACS -s P A C -t S -a resnet50 --freeze-bn --seed 2 --log logs/irm/PACS_S_g16 --add_bt 16\n#\npython irm.py data/PACS -d PACS -s P A C -t S -a resnet50 --freeze-bn --seed 0 --log logs/irm/PACS_S\npython irm.py data/PACS -d PACS -s P A C -t S -a resnet50 --freeze-bn --seed 1 --log logs/irm/PACS_S\npython irm.py data/PACS -d PACS -s P A C -t S -a resnet50 --freeze-bn --seed 2 --log logs/irm/PACS_S\n\n### ResNet50, Office-Home\n#CUDA_VISIBLE_DEVICES=0 python irm.py data/office-home -d OfficeHome -s Ar Cl Rw -t Pr -a resnet50 --seed 0 --log logs/irm/OfficeHome_Pr_g16 --add_bt 16\n#CUDA_VISIBLE_DEVICES=0 python irm.py data/office-home -d OfficeHome -s Ar Cl Pr -t Rw -a resnet50 --seed 0 --log logs/irm/OfficeHome_Rw_g16 --add_bt 16\n#CUDA_VISIBLE_DEVICES=0 python irm.py data/office-home -d OfficeHome -s Ar Rw Pr -t Cl -a resnet50 --seed 0 --log logs/irm/OfficeHome_Cl_g16 --add_bt 16\n#CUDA_VISIBLE_DEVICES=0 python irm.py data/office-home -d OfficeHome -s Cl Rw Pr -t Ar -a resnet50 --seed 0 --log logs/irm/OfficeHome_Ar_g16 --add_bt 16\n#\n#CUDA_VISIBLE_DEVICES=0 python irm.py data/office-home -d OfficeHome -s Ar Cl Rw -t Pr -a resnet50 --seed 1 --log logs/irm/OfficeHome_Pr_g16 --add_bt 16\n#CUDA_VISIBLE_DEVICES=0 python irm.py data/office-home -d OfficeHome -s Ar Cl Pr -t Rw -a resnet50 --seed 1 --log logs/irm/OfficeHome_Rw_g16 --add_bt 16\n#CUDA_VISIBLE_DEVICES=0 python irm.py data/office-home -d OfficeHome -s Ar Rw Pr -t Cl -a resnet50 --seed 1 --log logs/irm/OfficeHome_Cl_g16 --add_bt 16\n#CUDA_VISIBLE_DEVICES=0 python irm.py data/office-home -d OfficeHome -s Cl Rw Pr -t Ar -a resnet50 --seed 1 --log logs/irm/OfficeHome_Ar_g16 --add_bt 16\n#\n#CUDA_VISIBLE_DEVICES=0 python irm.py data/office-home -d OfficeHome -s Ar Cl Rw -t Pr -a resnet50 --seed 2 --log logs/irm/OfficeHome_Pr\n#CUDA_VISIBLE_DEVICES=0 python irm.py data/office-home -d OfficeHome -s Cl Rw Pr -t Ar -a resnet50 --seed 2 --log logs/irm/OfficeHome_Ar\n#CUDA_VISIBLE_DEVICES=0 python irm.py data/office-home -d OfficeHome -s Ar Cl Pr -t Rw -a resnet50 --seed 2 --log logs/irm/OfficeHome_Rw\n#CUDA_VISIBLE_DEVICES=0 python irm.py data/office-home -d OfficeHome -s Ar Rw Pr -t Cl -a resnet50 --seed 2 --log logs/irm/OfficeHome_Cl\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/examples/domain_generalization/image_classification/groupdro.py",
"content": "\"\"\"\n@author: Baixu Chen\n@contact: cbx_99_hasta@outlook.com\n\"\"\"\nimport random\nimport time\nimport warnings\nimport sys\nimport argparse\nimport shutil\nimport os.path as osp\n\nimport torch\nimport torch.nn as nn\nimport torch.backends.cudnn as cudnn\nfrom torch.optim import SGD\nfrom torch.optim.lr_scheduler import CosineAnnealingLR\nfrom torch.utils.data import DataLoader\nimport torch.nn.functional as F\n\nsys.path.append('../../..')\nfrom dglib.generalization.groupdro import AutomaticUpdateDomainWeightModule\nfrom dglib.modules.sampler import RandomDomainSampler\nfrom dglib.modules.classifier import ImageClassifier as Classifier\nfrom common.utils.data import ForeverDataIterator\nfrom common.utils.metric import accuracy\nfrom common.utils.meter import AverageMeter, ProgressMeter\nfrom common.utils.logger import CompleteLogger\nfrom common.utils.analysis import tsne, a_distance\n\nsys.path.append('.')\nimport utils\n\ndevice = torch.device(\"cuda\" if torch.cuda.is_available() else \"cpu\")\n\n\ndef main(args: argparse.Namespace):\n logger = CompleteLogger(args.log, args.phase)\n print(args)\n\n if args.seed is not None:\n random.seed(args.seed)\n torch.manual_seed(args.seed)\n cudnn.deterministic = True\n warnings.warn('You have chosen to seed training. '\n 'This will turn on the CUDNN deterministic setting, '\n 'which can slow down your training considerably! '\n 'You may see unexpected behavior when restarting '\n 'from checkpoints.')\n\n cudnn.benchmark = True\n\n # Data loading code\n train_transform = utils.get_train_transform(args.train_resizing, random_horizontal_flip=True,\n random_color_jitter=True, random_gray_scale=True)\n val_transform = utils.get_val_transform(args.val_resizing)\n print(\"train_transform: \", train_transform)\n print(\"val_transform: \", val_transform)\n\n train_dataset, num_classes = utils.get_dataset(dataset_name=args.data, root=args.root, task_list=args.sources,\n split='train', download=True, transform=train_transform,\n seed=args.seed)\n sampler = RandomDomainSampler(train_dataset, args.batch_size, args.n_domains_per_batch)\n train_loader = DataLoader(train_dataset, batch_size=args.batch_size, num_workers=args.workers,\n sampler=sampler, drop_last=True)\n val_dataset, _ = utils.get_dataset(dataset_name=args.data, root=args.root, task_list=args.sources, split='val',\n download=True, transform=val_transform, seed=args.seed)\n val_loader = DataLoader(val_dataset, batch_size=args.batch_size, shuffle=False, num_workers=args.workers)\n test_dataset, _ = utils.get_dataset(dataset_name=args.data, root=args.root, task_list=args.targets, split='test',\n download=True, transform=val_transform, seed=args.seed)\n test_loader = DataLoader(test_dataset, batch_size=args.batch_size, shuffle=False, num_workers=args.workers)\n print(\"train_dataset_size: \", len(train_dataset))\n print('val_dataset_size: ', len(val_dataset))\n print(\"test_dataset_size: \", len(test_dataset))\n train_iter = ForeverDataIterator(train_loader)\n\n # create model\n print(\"=> using pre-trained model '{}'\".format(args.arch))\n backbone = utils.get_model(args.arch)\n pool_layer = nn.Identity() if args.no_pool else None\n classifier = Classifier(backbone, num_classes, freeze_bn=args.freeze_bn, dropout_p=args.dropout_p,\n finetune=args.finetune, pool_layer=pool_layer).to(device)\n num_all_domains = len(train_dataset.datasets)\n\n # define optimizer and lr scheduler\n optimizer = SGD(classifier.get_parameters(base_lr=args.lr), args.lr, momentum=args.momentum, weight_decay=args.wd,\n nesterov=True)\n lr_scheduler = CosineAnnealingLR(optimizer, args.epochs * args.iters_per_epoch)\n domain_weight_module = AutomaticUpdateDomainWeightModule(num_all_domains, args.eta, device)\n\n # resume from the best checkpoint\n if args.phase != 'train':\n checkpoint = torch.load(logger.get_checkpoint_path('best'), map_location='cpu')\n classifier.load_state_dict(checkpoint)\n\n # analysis the model\n if args.phase == 'analysis':\n # extract features from both domains\n feature_extractor = nn.Sequential(classifier.backbone, classifier.pool_layer, classifier.bottleneck).to(device)\n source_feature = utils.collect_feature(val_loader, feature_extractor, device, max_num_features=100)\n target_feature = utils.collect_feature(test_loader, feature_extractor, device, max_num_features=100)\n print(len(source_feature), len(target_feature))\n # plot t-SNE\n tSNE_filename = osp.join(logger.visualize_directory, 'TSNE.png')\n tsne.visualize(source_feature, target_feature, tSNE_filename)\n print(\"Saving t-SNE to\", tSNE_filename)\n # calculate A-distance, which is a measure for distribution discrepancy\n A_distance = a_distance.calculate(source_feature, target_feature, device)\n print(\"A-distance =\", A_distance)\n return\n\n if args.phase == 'test':\n acc1 = utils.validate(test_loader, classifier, args, device)\n print(acc1)\n return\n\n # start training\n best_val_acc1 = 0.\n best_test_acc1 = 0.\n for epoch in range(args.epochs):\n print(lr_scheduler.get_lr())\n # train for one epoch\n train(train_iter, classifier, optimizer, lr_scheduler, domain_weight_module, args.n_domains_per_batch, epoch,\n args)\n\n # evaluate on validation set\n print(\"Evaluate on validation set...\")\n acc1 = utils.validate(val_loader, classifier, args, device)\n\n # remember best acc@1 and save checkpoint\n torch.save(classifier.state_dict(), logger.get_checkpoint_path('latest'))\n if acc1 > best_val_acc1:\n shutil.copy(logger.get_checkpoint_path('latest'), logger.get_checkpoint_path('best'))\n best_val_acc1 = max(acc1, best_val_acc1)\n\n # evaluate on test set\n print(\"Evaluate on test set...\")\n best_test_acc1 = max(best_test_acc1, utils.validate(test_loader, classifier, args, device))\n\n # evaluate on test set\n classifier.load_state_dict(torch.load(logger.get_checkpoint_path('best')))\n acc1 = utils.validate(test_loader, classifier, args, device)\n print(\"test acc on test set = {}\".format(acc1))\n print(\"oracle acc on test set = {}\".format(best_test_acc1))\n logger.close()\n\n\ndef train(train_iter: ForeverDataIterator, model: Classifier, optimizer, lr_scheduler: CosineAnnealingLR,\n domain_weight_module: AutomaticUpdateDomainWeightModule, n_domains_per_batch: int, epoch: int,\n args: argparse.Namespace):\n batch_time = AverageMeter('Time', ':4.2f')\n data_time = AverageMeter('Data', ':3.1f')\n losses = AverageMeter('Loss', ':3.2f')\n cls_accs = AverageMeter('Cls Acc', ':3.1f')\n\n progress = ProgressMeter(\n args.iters_per_epoch,\n [batch_time, data_time, losses, cls_accs],\n prefix=\"Epoch: [{}]\".format(epoch))\n\n # switch to train mode\n model.train()\n\n end = time.time()\n for i in range(args.iters_per_epoch):\n x_all, labels_all, domain_labels = next(train_iter)\n x_all = x_all.to(device)\n labels_all = labels_all.to(device)\n domain_labels = domain_labels.to(device)\n\n # get selected domain idxes\n domain_labels = domain_labels.chunk(n_domains_per_batch, dim=0)\n sampled_domain_idxes = [domain_labels[i][0].item() for i in range(n_domains_per_batch)]\n\n # measure data loading time\n data_time.update(time.time() - end)\n\n loss_per_domain = torch.zeros(n_domains_per_batch).to(device)\n cls_acc = 0\n for domain_id, (x_per_domain, labels_per_domain) in enumerate(\n zip(x_all.chunk(n_domains_per_batch, dim=0), labels_all.chunk(n_domains_per_batch, dim=0))):\n y_per_domain, _ = model(x_per_domain)\n loss_per_domain[domain_id] = F.cross_entropy(y_per_domain, labels_per_domain)\n cls_acc += accuracy(y_per_domain, labels_per_domain)[0] / n_domains_per_batch\n\n # update domain weight\n domain_weight_module.update(loss_per_domain, sampled_domain_idxes)\n domain_weight = domain_weight_module.get_domain_weight(sampled_domain_idxes)\n\n # weighted cls loss\n loss = (loss_per_domain * domain_weight).sum()\n\n losses.update(loss.item(), x_all.size(0))\n cls_accs.update(cls_acc.item(), x_all.size(0))\n\n # compute gradient and do SGD step\n optimizer.zero_grad()\n loss.backward()\n optimizer.step()\n lr_scheduler.step()\n\n # measure elapsed time\n batch_time.update(time.time() - end)\n end = time.time()\n\n if i % args.print_freq == 0:\n progress.display(i)\n\n\nif __name__ == '__main__':\n parser = argparse.ArgumentParser(description='GroupDRO for Domain Generalization')\n # dataset parameters\n parser.add_argument('root', metavar='DIR',\n help='root path of dataset')\n parser.add_argument('-d', '--data', metavar='DATA', default='PACS',\n help='dataset: ' + ' | '.join(utils.get_dataset_names()) +\n ' (default: PACS)')\n parser.add_argument('-s', '--sources', nargs='+', default=None,\n help='source domain(s)')\n parser.add_argument('-t', '--targets', nargs='+', default=None,\n help='target domain(s)')\n parser.add_argument('--train-resizing', type=str, default='default')\n parser.add_argument('--val-resizing', type=str, default='default')\n # model parameters\n parser.add_argument('-a', '--arch', metavar='ARCH', default='resnet50',\n choices=utils.get_model_names(),\n help='backbone architecture: ' +\n ' | '.join(utils.get_model_names()) +\n ' (default: resnet50)')\n parser.add_argument('--no-pool', action='store_true', help='no pool layer after the feature extractor.')\n parser.add_argument('--finetune', action='store_true', help='whether use 10x smaller lr for backbone')\n parser.add_argument('--freeze-bn', action='store_true', help='whether freeze all bn layers')\n parser.add_argument('--dropout-p', type=float, default=0.1, help='only activated when freeze-bn is True')\n # training parameters\n parser.add_argument('--eta', default=1e-2, type=float,\n help='the eta hyper parameter')\n parser.add_argument('-b', '--batch-size', default=36, type=int,\n metavar='N',\n help='mini-batch size (default: 36)')\n parser.add_argument('--n-domains-per-batch', default=3, type=int,\n help='number of domains in each mini-batch')\n parser.add_argument('--lr', '--learning-rate', default=5e-4, type=float,\n metavar='LR', help='initial learning rate', dest='lr')\n parser.add_argument('--momentum', default=0.9, type=float, metavar='M',\n help='momentum')\n parser.add_argument('--wd', '--weight-decay', default=0.0005, type=float,\n metavar='W', help='weight decay (default: 5e-4)')\n parser.add_argument('-j', '--workers', default=4, type=int, metavar='N',\n help='number of data loading workers (default: 4)')\n parser.add_argument('--epochs', default=20, type=int, metavar='N',\n help='number of total epochs to run')\n parser.add_argument('-i', '--iters-per-epoch', default=500, type=int,\n help='Number of iterations per epoch')\n parser.add_argument('-p', '--print-freq', default=100, type=int,\n metavar='N', help='print frequency (default: 100)')\n parser.add_argument('--seed', default=None, type=int,\n help='seed for initializing training. ')\n parser.add_argument(\"--log\", type=str, default='groupdro',\n help=\"Where to save logs, checkpoints and debugging images.\")\n parser.add_argument(\"--phase\", type=str, default='train', choices=['train', 'test', 'analysis'],\n help=\"When phase is 'test', only test the model.\"\n \"When phase is 'analysis', only analysis the model.\")\n args = parser.parse_args()\n main(args)\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/examples/domain_generalization/image_classification/groupdro.sh",
"content": "#!/usr/bin/env bash\n# ResNet50, PACS\nCUDA_VISIBLE_DEVICES=0 python groupdro.py data/PACS -d PACS -s A C S -t P -a resnet50 --freeze-bn --seed 0 --log logs/groupdro/PACS_P\nCUDA_VISIBLE_DEVICES=0 python groupdro.py data/PACS -d PACS -s P C S -t A -a resnet50 --freeze-bn --seed 0 --log logs/groupdro/PACS_A\nCUDA_VISIBLE_DEVICES=0 python groupdro.py data/PACS -d PACS -s P A S -t C -a resnet50 --freeze-bn --seed 0 --log logs/groupdro/PACS_C\nCUDA_VISIBLE_DEVICES=0 python groupdro.py data/PACS -d PACS -s P A C -t S -a resnet50 --freeze-bn --seed 0 --log logs/groupdro/PACS_S\n\n# ResNet50, Office-Home\nCUDA_VISIBLE_DEVICES=0 python groupdro.py data/office-home -d OfficeHome -s Ar Cl Rw -t Pr -a resnet50 --seed 0 --log logs/groupdro/OfficeHome_Pr\nCUDA_VISIBLE_DEVICES=0 python groupdro.py data/office-home -d OfficeHome -s Ar Cl Pr -t Rw -a resnet50 --seed 0 --log logs/groupdro/OfficeHome_Rw\nCUDA_VISIBLE_DEVICES=0 python groupdro.py data/office-home -d OfficeHome -s Ar Rw Pr -t Cl -a resnet50 --seed 0 --log logs/groupdro/OfficeHome_Cl\nCUDA_VISIBLE_DEVICES=0 python groupdro.py data/office-home -d OfficeHome -s Cl Rw Pr -t Ar -a resnet50 --seed 0 --log logs/groupdro/OfficeHome_Ar\n\n# ResNet50, DomainNet\nCUDA_VISIBLE_DEVICES=0 python groupdro.py data/domainnet -d DomainNet -s i p q r s -t c -a resnet50 -i 4000 --lr 0.005 --seed 0 --log logs/groupdro/DomainNet_c\nCUDA_VISIBLE_DEVICES=0 python groupdro.py data/domainnet -d DomainNet -s c p q r s -t i -a resnet50 -i 4000 --lr 0.005 --seed 0 --log logs/groupdro/DomainNet_i\nCUDA_VISIBLE_DEVICES=0 python groupdro.py data/domainnet -d DomainNet -s c i q r s -t p -a resnet50 -i 4000 --lr 0.005 --seed 0 --log logs/groupdro/DomainNet_p\nCUDA_VISIBLE_DEVICES=0 python groupdro.py data/domainnet -d DomainNet -s c i p r s -t q -a resnet50 -i 4000 --lr 0.005 --seed 0 --log logs/groupdro/DomainNet_q\nCUDA_VISIBLE_DEVICES=0 python groupdro.py data/domainnet -d DomainNet -s c i p q s -t r -a resnet50 -i 4000 --lr 0.005 --seed 0 --log logs/groupdro/DomainNet_r\nCUDA_VISIBLE_DEVICES=0 python groupdro.py data/domainnet -d DomainNet -s c i p q r -t s -a resnet50 -i 4000 --lr 0.005 --seed 0 --log logs/groupdro/DomainNet_s\n\n# ResNet50, Wilds Dataset\nCUDA_VISIBLE_DEVICES=0 python groupdro.py data/wilds -d iwildcam --train-resizing 'res2x' --val-resizing 'res2x' \\\n -a resnet50 -b 16 --n-domains-per-batch 2 --epochs 60 -i 1000 --lr 0.001 --finetune --seed 0 --log logs/groupdro/iwildcam\nCUDA_VISIBLE_DEVICES=0 python groupdro.py data/wilds -d camelyon17 -a resnet50 -b 36 --n-domains-per-batch 3 \\\n --epochs 20 -i 1000 --lr 0.01 --seed 0 --log logs/groupdro/camelyon17\n\n# DenseNet121, Wilds Dataset\nCUDA_VISIBLE_DEVICES=0 python groupdro.py data/wilds -d fmow --train-resizing 'cen.crop' -a densenet121 \\\n -b 36 --n-domains-per-batch 4 --epochs 60 -i 1000 --lr 0.003 --seed 0 --log logs/groupdro/fmow\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/examples/domain_generalization/image_classification/ibn.sh",
"content": "#!/usr/bin/env bash\n# IBN_ResNet50_b, PACS\nCUDA_VISIBLE_DEVICES=0 python baseline.py data/PACS -d PACS -s A C S -t P -a resnet50_ibn_b --freeze-bn --seed 0 --log logs/baseline/PACS_P\nCUDA_VISIBLE_DEVICES=0 python baseline.py data/PACS -d PACS -s P C S -t A -a resnet50_ibn_b --freeze-bn --seed 0 --log logs/baseline/PACS_A\nCUDA_VISIBLE_DEVICES=2 python baseline.py data/PACS -d PACS -s P A S -t C -a resnet50_ibn_b --freeze-bn --seed 0 --log logs/baseline/PACS_C\nCUDA_VISIBLE_DEVICES=0 python baseline.py data/PACS -d PACS -s P A C -t S -a resnet50_ibn_b --freeze-bn --seed 0 --log logs/baseline/PACS_S\n\n# IBN_ResNet50_b, Office-Home\nCUDA_VISIBLE_DEVICES=0 python baseline.py data/office-home -d OfficeHome -s Ar Cl Rw -t Pr -a resnet50_ibn_b --seed 0 --log logs/baseline/OfficeHome_Pr\nCUDA_VISIBLE_DEVICES=0 python baseline.py data/office-home -d OfficeHome -s Ar Cl Pr -t Rw -a resnet50_ibn_b --seed 0 --log logs/baseline/OfficeHome_Rw\nCUDA_VISIBLE_DEVICES=0 python baseline.py data/office-home -d OfficeHome -s Ar Rw Pr -t Cl -a resnet50_ibn_b --seed 0 --log logs/baseline/OfficeHome_Cl\nCUDA_VISIBLE_DEVICES=0 python baseline.py data/office-home -d OfficeHome -s Cl Rw Pr -t Ar -a resnet50_ibn_b --seed 0 --log logs/baseline/OfficeHome_Ar\n\n# IBN_ResNet50_b, DomainNet\nCUDA_VISIBLE_DEVICES=0 python baseline.py data/domainnet -d DomainNet -s i p q r s -t c -a resnet50_ibn_b -i 2500 --lr 0.01 --seed 0 --log logs/baseline/DomainNet_c\nCUDA_VISIBLE_DEVICES=0 python baseline.py data/domainnet -d DomainNet -s c p q r s -t i -a resnet50_ibn_b -i 2500 --lr 0.01 --seed 0 --log logs/baseline/DomainNet_i\nCUDA_VISIBLE_DEVICES=0 python baseline.py data/domainnet -d DomainNet -s c i q r s -t p -a resnet50_ibn_b -i 2500 --lr 0.01 --seed 0 --log logs/baseline/DomainNet_p\nCUDA_VISIBLE_DEVICES=0 python baseline.py data/domainnet -d DomainNet -s c i p r s -t q -a resnet50_ibn_b -i 2500 --lr 0.01 --seed 0 --log logs/baseline/DomainNet_q\nCUDA_VISIBLE_DEVICES=0 python baseline.py data/domainnet -d DomainNet -s c i p q s -t r -a resnet50_ibn_b -i 2500 --lr 0.01 --seed 0 --log logs/baseline/DomainNet_r\nCUDA_VISIBLE_DEVICES=0 python baseline.py data/domainnet -d DomainNet -s c i p q r -t s -a resnet50_ibn_b -i 2500 --lr 0.01 --seed 0 --log logs/baseline/DomainNet_s\n\n# IBN_ResNet50_b, Wilds Dataset\nCUDA_VISIBLE_DEVICES=0 python baseline.py data/wilds -d iwildcam --train-resizing 'res2x' --val-resizing 'res2x' \\\n -a resnet50_ibn_b -b 16 --epochs 60 -i 1000 --lr 0.001 --finetune --seed 0 --log logs/baseline/iwildcam\nCUDA_VISIBLE_DEVICES=0 python baseline.py data/wilds -d camelyon17 -a resnet50_ibn_b -b 36 --epochs 20 -i 1000 --lr 0.01 \\\n --seed 0 --log logs/baseline/camelyon17\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/examples/domain_generalization/image_classification/irm.py",
"content": "\"\"\"\n@author: Baixu Chen\n@contact: cbx_99_hasta@outlook.com\n\"\"\"\nimport random\nimport time\nimport warnings\nimport sys\nimport argparse\nimport shutil\nimport os.path as osp\n\nimport torch\nimport torch.nn as nn\nimport torch.backends.cudnn as cudnn\nfrom torch.optim import SGD\nfrom torch.optim.lr_scheduler import CosineAnnealingLR\nfrom torch.utils.data import DataLoader\nimport torch.nn.functional as F\n\nsys.path.append('../../..')\nfrom dglib.generalization.irm import InvariancePenaltyLoss\nfrom dglib.modules.sampler import RandomDomainSampler\nfrom dglib.modules.classifier import ImageClassifier as Classifier\nfrom common.utils.data import ForeverDataIterator\nfrom common.utils.metric import accuracy\nfrom common.utils.meter import AverageMeter, ProgressMeter\nfrom common.utils.logger import CompleteLogger\nfrom common.utils.analysis import tsne, a_distance\n\nsys.path.append('.')\nimport utils\n\ndevice = torch.device(\"cuda\" if torch.cuda.is_available() else \"cpu\")\n\nclass TransformerDecorator(torch.nn.Module):\n def __init__(self, backbone=None, add_bt=3, dim=2048, eval_global=0):\n super(TransformerDecorator, self).__init__()\n self.eval_global = eval_global\n self.add_bt = add_bt\n self.backbone = backbone\n if self.add_bt > 0:\n self.encoder_layers = torch.nn.TransformerEncoderLayer(dim, 4, dim, 0.5)\n\n def forward(self, feature):\n feature = self.backbone(feature)\n if self.training and self.add_bt > 0:\n pre_feature = feature\n feature = feature.unsqueeze(1)\n feature = self.encoder_layers(feature)\n # feature = self.encoder_layers(feature, feature, feature)\n feature = feature.squeeze(1)\n if self.add_bt:\n return torch.cat([pre_feature, feature], dim=0)\n return feature\n\n\ndef main(args: argparse.Namespace):\n logger = CompleteLogger(args.log, args.phase)\n print(args)\n\n if args.seed is not None:\n random.seed(args.seed)\n torch.manual_seed(args.seed)\n cudnn.deterministic = True\n warnings.warn('You have chosen to seed training. '\n 'This will turn on the CUDNN deterministic setting, '\n 'which can slow down your training considerably! '\n 'You may see unexpected behavior when restarting '\n 'from checkpoints.')\n\n cudnn.benchmark = True\n\n # Data loading code\n train_transform = utils.get_train_transform(args.train_resizing, random_horizontal_flip=True,\n random_color_jitter=True, random_gray_scale=True)\n val_transform = utils.get_val_transform(args.val_resizing)\n print(\"train_transform: \", train_transform)\n print(\"val_transform: \", val_transform)\n\n train_dataset, num_classes = utils.get_dataset(dataset_name=args.data, root=args.root, task_list=args.sources,\n split='train', download=True, transform=train_transform,\n seed=args.seed)\n sampler = RandomDomainSampler(train_dataset, args.batch_size, n_domains_per_batch=args.n_domains_per_batch)\n train_loader = DataLoader(train_dataset, batch_size=args.batch_size, num_workers=args.workers,\n sampler=sampler, drop_last=True)\n val_dataset, _ = utils.get_dataset(dataset_name=args.data, root=args.root, task_list=args.sources, split='val',\n download=True, transform=val_transform, seed=args.seed)\n val_loader = DataLoader(val_dataset, batch_size=args.batch_size, shuffle=False, num_workers=args.workers)\n test_dataset, _ = utils.get_dataset(dataset_name=args.data, root=args.root, task_list=args.targets, split='test',\n download=True, transform=val_transform, seed=args.seed)\n test_loader = DataLoader(test_dataset, batch_size=args.batch_size, shuffle=False, num_workers=args.workers)\n print(\"train_dataset_size: \", len(train_dataset))\n print('val_dataset_size: ', len(val_dataset))\n print(\"test_dataset_size: \", len(test_dataset))\n train_iter = ForeverDataIterator(train_loader)\n\n # create model\n print(\"=> using pre-trained model '{}'\".format(args.arch))\n backbone = utils.get_model(args.arch)\n pool_layer = nn.Identity() if args.no_pool else None\n pool_layer = nn.Sequential(\n nn.AdaptiveAvgPool2d(output_size=(1, 1)),\n nn.Flatten(),\n )\n pool_layer = TransformerDecorator(pool_layer, add_bt=args.add_bt, dim=backbone.out_features)\n classifier = Classifier(backbone, num_classes, freeze_bn=args.freeze_bn, dropout_p=args.dropout_p,\n finetune=args.finetune, pool_layer=pool_layer).to(device)\n\n # define optimizer and lr scheduler\n optimizer = SGD(classifier.get_parameters(base_lr=args.lr), args.lr, momentum=args.momentum, weight_decay=args.wd,\n nesterov=True)\n lr_scheduler = CosineAnnealingLR(optimizer, args.epochs * args.iters_per_epoch)\n\n # define loss function\n invariance_penalty_loss = InvariancePenaltyLoss().to(device)\n\n # for simplicity\n assert args.anneal_iters % args.iters_per_epoch == 0\n\n # resume from the best checkpoint\n if args.phase != 'train':\n checkpoint = torch.load(logger.get_checkpoint_path('best'), map_location='cpu')\n classifier.load_state_dict(checkpoint)\n\n # analysis the model\n if args.phase == 'analysis':\n # extract features from both domains\n feature_extractor = nn.Sequential(classifier.backbone, classifier.pool_layer, classifier.bottleneck).to(device)\n source_feature = utils.collect_feature(val_loader, feature_extractor, device, max_num_features=100)\n target_feature = utils.collect_feature(test_loader, feature_extractor, device, max_num_features=100)\n print(len(source_feature), len(target_feature))\n # plot t-SNE\n tSNE_filename = osp.join(logger.visualize_directory, 'TSNE.png')\n tsne.visualize(source_feature, target_feature, tSNE_filename)\n print(\"Saving t-SNE to\", tSNE_filename)\n # calculate A-distance, which is a measure for distribution discrepancy\n A_distance = a_distance.calculate(source_feature, target_feature, device)\n print(\"A-distance =\", A_distance)\n return\n\n if args.phase == 'test':\n acc1 = utils.validate(test_loader, classifier, args, device)\n print(acc1)\n return\n\n # start training\n best_val_acc1 = 0.\n best_test_acc1 = 0.\n for epoch in range(args.epochs):\n if epoch * args.iters_per_epoch == args.anneal_iters:\n # reset optimizer to avoid sharp jump in gradient magnitudes\n optimizer = SGD(classifier.get_parameters(base_lr=args.lr), args.lr, momentum=args.momentum,\n weight_decay=args.wd, nesterov=True)\n lr_scheduler = CosineAnnealingLR(optimizer, args.epochs * args.iters_per_epoch - args.anneal_iters)\n\n print(lr_scheduler.get_lr())\n # train for one epoch\n train(train_iter, classifier, optimizer, lr_scheduler, invariance_penalty_loss, args.n_domains_per_batch, epoch,\n args)\n\n # evaluate on validation set\n print(\"Evaluate on validation set...\")\n acc1 = utils.validate(val_loader, classifier, args, device)\n\n # remember best acc@1 and save checkpoint\n torch.save(classifier.state_dict(), logger.get_checkpoint_path('latest'))\n if acc1 > best_val_acc1:\n shutil.copy(logger.get_checkpoint_path('latest'), logger.get_checkpoint_path('best'))\n best_val_acc1 = max(acc1, best_val_acc1)\n\n # evaluate on test set\n print(\"Evaluate on test set...\")\n best_test_acc1 = max(best_test_acc1, utils.validate(test_loader, classifier, args, device))\n\n # evaluate on test set\n classifier.load_state_dict(torch.load(logger.get_checkpoint_path('best')))\n acc1 = utils.validate(test_loader, classifier, args, device)\n print(\"test acc on test set = {}\".format(acc1))\n print(\"oracle acc on test set = {}\".format(best_test_acc1))\n logger.close()\n\n\ndef train(train_iter: ForeverDataIterator, model: Classifier, optimizer, lr_scheduler: CosineAnnealingLR,\n invariance_penalty_loss: InvariancePenaltyLoss, n_domains_per_batch: int, epoch: int,\n args: argparse.Namespace):\n batch_time = AverageMeter('Time', ':4.2f')\n data_time = AverageMeter('Data', ':3.1f')\n losses = AverageMeter('Loss', ':3.2f')\n losses_ce = AverageMeter('CELoss', ':3.2f')\n losses_penalty = AverageMeter('Penalty Loss', ':3.2f')\n cls_accs = AverageMeter('Cls Acc', ':3.1f')\n\n progress = ProgressMeter(\n args.iters_per_epoch,\n [batch_time, data_time, losses, losses_ce, losses_penalty, cls_accs],\n prefix=\"Epoch: [{}]\".format(epoch))\n\n # switch to train mode\n model.train()\n\n end = time.time()\n for i in range(args.iters_per_epoch):\n x_all, labels_all, _ = next(train_iter)\n x_all = x_all.to(device)\n labels_all = labels_all.to(device)\n\n # measure data loading time\n data_time.update(time.time() - end)\n\n # compute output\n y_all, _ = model(x_all)\n if len(y_all) > len(labels_all):\n labels_all = torch.cat([labels_all, labels_all], dim=0)\n # cls loss\n loss_ce = F.cross_entropy(y_all, labels_all)\n # penalty loss\n loss_penalty = 0\n for y_per_domain, labels_per_domain in zip(y_all.chunk(n_domains_per_batch, dim=0),\n labels_all.chunk(n_domains_per_batch, dim=0)):\n # normalize loss by domain num\n loss_penalty += invariance_penalty_loss(y_per_domain, labels_per_domain) / n_domains_per_batch\n\n global_iter = epoch * args.iters_per_epoch + i\n if global_iter >= args.anneal_iters:\n trade_off = args.trade_off\n else:\n trade_off = 1\n loss = loss_ce + loss_penalty * trade_off\n cls_acc = accuracy(y_all, labels_all)[0]\n\n losses.update(loss.item(), x_all.size(0))\n losses_ce.update(loss_ce.item(), x_all.size(0))\n losses_penalty.update(loss_penalty.item(), x_all.size(0))\n cls_accs.update(cls_acc.item(), x_all.size(0))\n\n # compute gradient and do SGD step\n optimizer.zero_grad()\n loss.backward()\n optimizer.step()\n lr_scheduler.step()\n\n # measure elapsed time\n batch_time.update(time.time() - end)\n end = time.time()\n\n if i % args.print_freq == 0:\n progress.display(i)\n\n\nif __name__ == '__main__':\n parser = argparse.ArgumentParser(description='IRM for Domain Generalization')\n # dataset parameters\n parser.add_argument('root', metavar='DIR',\n help='root path of dataset')\n parser.add_argument('-d', '--data', metavar='DATA', default='PACS',\n help='dataset: ' + ' | '.join(utils.get_dataset_names()) +\n ' (default: PACS)')\n parser.add_argument('-s', '--sources', nargs='+', default=None,\n help='source domain(s)')\n parser.add_argument('-t', '--targets', nargs='+', default=None,\n help='target domain(s)')\n parser.add_argument('--train-resizing', type=str, default='default')\n parser.add_argument('--val-resizing', type=str, default='default')\n # model parameters\n parser.add_argument('-a', '--arch', metavar='ARCH', default='resnet50',\n choices=utils.get_model_names(),\n help='backbone architecture: ' +\n ' | '.join(utils.get_model_names()) +\n ' (default: resnet50)')\n parser.add_argument('--no-pool', action='store_true', help='no pool layer after the feature extractor.')\n parser.add_argument('--finetune', action='store_true', help='whether use 10x smaller lr for backbone')\n parser.add_argument('--freeze-bn', action='store_true', help='whether freeze all bn layers')\n parser.add_argument('--dropout-p', type=float, default=0.1, help='only activated when freeze-bn is True')\n # training parameters\n parser.add_argument('--trade-off', default=1, type=float,\n help='the trade off hyper parameter for irm penalty')\n parser.add_argument('--anneal-iters', default=500, type=int,\n help='anneal iterations (trade off is set to 1 during these iterations)')\n parser.add_argument('-b', '--batch-size', default=36, type=int,\n metavar='N',\n help='mini-batch size (default: 36)')\n parser.add_argument('--n-domains-per-batch', default=3, type=int,\n help='number of domains in each mini-batch')\n parser.add_argument('--lr', '--learning-rate', default=5e-4, type=float,\n metavar='LR', help='initial learning rate', dest='lr')\n parser.add_argument('--momentum', default=0.9, type=float, metavar='M',\n help='momentum')\n parser.add_argument('--wd', '--weight-decay', default=0.0005, type=float,\n metavar='W', help='weight decay (default: 5e-4)')\n parser.add_argument('-j', '--workers', default=4, type=int, metavar='N',\n help='number of data loading workers (default: 4)')\n parser.add_argument('--epochs', default=20, type=int, metavar='N',\n help='number of total epochs to run')\n parser.add_argument('-i', '--iters-per-epoch', default=500, type=int,\n help='Number of iterations per epoch')\n parser.add_argument('-p', '--print-freq', default=100, type=int,\n metavar='N', help='print frequency (default: 100)')\n parser.add_argument('--seed', default=None, type=int,\n help='seed for initializing training. ')\n parser.add_argument(\"--log\", type=str, default='irm',\n help=\"Where to save logs, checkpoints and debugging images.\")\n parser.add_argument(\"--phase\", type=str, default='train', choices=['train', 'test', 'analysis'],\n help=\"When phase is 'test', only test the model.\"\n \"When phase is 'analysis', only analysis the model.\")\n parser.add_argument(\"--add_bt\", type=int, default=0,\n help=\"add_bt\")\n args = parser.parse_args()\n main(args)\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/examples/domain_generalization/image_classification/irm.sh",
"content": "#!/usr/bin/env bash\n# ResNet50, PACS\nCUDA_VISIBLE_DEVICES=0 python irm.py data/PACS -d PACS -s A C S -t P -a resnet50 --freeze-bn --seed 0 --log logs/irm/PACS_P\nCUDA_VISIBLE_DEVICES=0 python irm.py data/PACS -d PACS -s P C S -t A -a resnet50 --freeze-bn --seed 0 --log logs/irm/PACS_A\nCUDA_VISIBLE_DEVICES=0 python irm.py data/PACS -d PACS -s P A S -t C -a resnet50 --freeze-bn --seed 0 --log logs/irm/PACS_C\nCUDA_VISIBLE_DEVICES=0 python irm.py data/PACS -d PACS -s P A C -t S -a resnet50 --freeze-bn --seed 0 --log logs/irm/PACS_S\n\n# ResNet50, Office-Home\nCUDA_VISIBLE_DEVICES=0 python irm.py data/office-home -d OfficeHome -s Ar Cl Rw -t Pr -a resnet50 --seed 0 --log logs/irm/OfficeHome_Pr\nCUDA_VISIBLE_DEVICES=0 python irm.py data/office-home -d OfficeHome -s Ar Cl Pr -t Rw -a resnet50 --seed 0 --log logs/irm/OfficeHome_Rw\nCUDA_VISIBLE_DEVICES=0 python irm.py data/office-home -d OfficeHome -s Ar Rw Pr -t Cl -a resnet50 --seed 0 --log logs/irm/OfficeHome_Cl\nCUDA_VISIBLE_DEVICES=0 python irm.py data/office-home -d OfficeHome -s Cl Rw Pr -t Ar -a resnet50 --seed 0 --log logs/irm/OfficeHome_Ar\n\n# ResNet50, DomainNet\nCUDA_VISIBLE_DEVICES=0 python irm.py data/domainnet -d DomainNet -s i p q r s -t c -a resnet50 -i 4000 --anneal-iters 4000 --lr 0.005 --seed 0 --log logs/irm/DomainNet_c\nCUDA_VISIBLE_DEVICES=0 python irm.py data/domainnet -d DomainNet -s c p q r s -t i -a resnet50 -i 4000 --anneal-iters 4000 --lr 0.005 --seed 0 --log logs/irm/DomainNet_i\nCUDA_VISIBLE_DEVICES=0 python irm.py data/domainnet -d DomainNet -s c i q r s -t p -a resnet50 -i 4000 --anneal-iters 4000 --lr 0.005 --seed 0 --log logs/irm/DomainNet_p\nCUDA_VISIBLE_DEVICES=0 python irm.py data/domainnet -d DomainNet -s c i p r s -t q -a resnet50 -i 4000 --anneal-iters 4000 --lr 0.005 --seed 0 --log logs/irm/DomainNet_q\nCUDA_VISIBLE_DEVICES=0 python irm.py data/domainnet -d DomainNet -s c i p q s -t r -a resnet50 -i 4000 --anneal-iters 4000 --lr 0.005 --seed 0 --log logs/irm/DomainNet_r\nCUDA_VISIBLE_DEVICES=0 python irm.py data/domainnet -d DomainNet -s c i p q r -t s -a resnet50 -i 4000 --anneal-iters 4000 --lr 0.005 --seed 0 --log logs/irm/DomainNet_s\n\n# ResNet50, Wilds Dataset\nCUDA_VISIBLE_DEVICES=0 python irm.py data/wilds -d iwildcam --train-resizing 'res2x' --val-resizing 'res2x' \\\n -a resnet50 -b 16 --n-domains-per-batch 2 --epochs 60 -i 1000 --anneal-iters 1000 --lr 0.0003 --finetune --trade-off 0.1 \\\n --seed 0 --log logs/irm/iwildcam\nCUDA_VISIBLE_DEVICES=0 python irm.py data/wilds -d camelyon17 -a resnet50 -b 36 --n-domains-per-batch 3 \\\n --epochs 20 -i 1000 --anneal-iters 1000 --lr 0.001 --trade-off 0.1 --seed 0 --log logs/irm/camelyon17\n\n# DenseNet121, Wilds Dataset\nCUDA_VISIBLE_DEVICES=0 python irm.py data/wilds -d fmow --train-resizing 'cen.crop' -a densenet121 \\\n -b 36 --n-domains-per-batch 4 --epochs 60 -i 1000 --anneal-iters 1000 --lr 0.003 --trade-off 0.1 --seed 0 --log logs/irm/fmow\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/examples/domain_generalization/image_classification/mixstyle.py",
"content": "\"\"\"\n@author: Baixu Chen\n@contact: cbx_99_hasta@outlook.com\n\"\"\"\nimport random\nimport time\nimport warnings\nimport sys\nimport argparse\nimport shutil\nimport os.path as osp\n\nimport torch\nimport torch.nn as nn\nimport torch.backends.cudnn as cudnn\nfrom torch.optim import SGD\nfrom torch.optim.lr_scheduler import CosineAnnealingLR\nfrom torch.utils.data import DataLoader\nimport torch.nn.functional as F\n\nsys.path.append('../../..')\nimport dglib.generalization.mixstyle.models as models\nfrom dglib.modules.sampler import RandomDomainSampler\nfrom dglib.modules.classifier import ImageClassifier as Classifier\nfrom common.utils.data import ForeverDataIterator\nfrom common.utils.metric import accuracy\nfrom common.utils.meter import AverageMeter, ProgressMeter\nfrom common.utils.logger import CompleteLogger\nfrom common.utils.analysis import tsne, a_distance\n\nsys.path.append('.')\nimport utils\n\ndevice = torch.device(\"cuda\" if torch.cuda.is_available() else \"cpu\")\n\nclass TransformerDecorator(torch.nn.Module):\n def __init__(self, backbone=None, add_bt=3, dim=2048, eval_global=0):\n super(TransformerDecorator, self).__init__()\n self.eval_global = eval_global\n self.add_bt = add_bt\n self.backbone = backbone\n self.encoder_layers = torch.nn.TransformerEncoderLayer(dim, 4, dim, 0.5)\n\n def forward(self, feature):\n feature = self.backbone(feature)\n if self.training and self.add_bt > 0:\n pre_feature = feature\n feature = feature.unsqueeze(1)\n feature = self.encoder_layers(feature)\n # feature = self.encoder_layers(feature, feature, feature)\n feature = feature.squeeze(1)\n if self.add_bt:\n return torch.cat([pre_feature, feature], dim=0)\n return feature\n\n\ndef main(args: argparse.Namespace):\n logger = CompleteLogger(args.log, args.phase)\n print(args)\n\n if args.seed is not None:\n random.seed(args.seed)\n torch.manual_seed(args.seed)\n cudnn.deterministic = True\n warnings.warn('You have chosen to seed training. '\n 'This will turn on the CUDNN deterministic setting, '\n 'which can slow down your training considerably! '\n 'You may see unexpected behavior when restarting '\n 'from checkpoints.')\n\n cudnn.benchmark = True\n\n # Data loading code\n train_transform = utils.get_train_transform(args.train_resizing, random_horizontal_flip=True,\n random_color_jitter=True, random_gray_scale=True)\n val_transform = utils.get_val_transform(args.val_resizing)\n print(\"train_transform: \", train_transform)\n print(\"val_transform: \", val_transform)\n\n train_dataset, num_classes = utils.get_dataset(dataset_name=args.data, root=args.root, task_list=args.sources,\n split='train', download=True, transform=train_transform,\n seed=args.seed)\n sampler = RandomDomainSampler(train_dataset, args.batch_size, n_domains_per_batch=2)\n train_loader = DataLoader(train_dataset, batch_size=args.batch_size, num_workers=args.workers,\n sampler=sampler, drop_last=True)\n val_dataset, _ = utils.get_dataset(dataset_name=args.data, root=args.root, task_list=args.sources, split='val',\n download=True, transform=val_transform, seed=args.seed)\n val_loader = DataLoader(val_dataset, batch_size=args.batch_size, shuffle=False, num_workers=args.workers)\n test_dataset, _ = utils.get_dataset(dataset_name=args.data, root=args.root, task_list=args.targets, split='test',\n download=True, transform=val_transform, seed=args.seed)\n test_loader = DataLoader(test_dataset, batch_size=args.batch_size, shuffle=False, num_workers=args.workers)\n\n print(\"train_dataset_size: \", len(train_dataset))\n print('val_dataset_size: ', len(val_dataset))\n print(\"test_dataset_size: \", len(test_dataset))\n train_iter = ForeverDataIterator(train_loader)\n\n # create model\n print(\"=> using pre-trained model '{}'\".format(args.arch))\n backbone = models.__dict__[args.arch](mix_layers=args.mix_layers, mix_p=args.mix_p, mix_alpha=args.mix_alpha,\n pretrained=True)\n pool_layer = nn.Identity() if args.no_pool else None\n pool_layer = nn.Sequential(\n nn.AdaptiveAvgPool2d(output_size=(1, 1)),\n nn.Flatten(),\n )\n pool_layer = TransformerDecorator(pool_layer, add_bt=args.add_bt, dim=backbone.out_features)\n classifier = Classifier(backbone, num_classes, freeze_bn=args.freeze_bn, dropout_p=args.dropout_p,\n finetune=args.finetune, pool_layer=pool_layer).to(device)\n\n # define optimizer and lr scheduler\n optimizer = SGD(classifier.get_parameters(base_lr=args.lr), args.lr, momentum=args.momentum, weight_decay=args.wd,\n nesterov=True)\n lr_scheduler = CosineAnnealingLR(optimizer, args.epochs * args.iters_per_epoch)\n\n # resume from the best checkpoint\n if args.phase != 'train':\n checkpoint = torch.load(logger.get_checkpoint_path('best'), map_location='cpu')\n classifier.load_state_dict(checkpoint)\n\n # analysis the model\n if args.phase == 'analysis':\n # extract features from both domains\n feature_extractor = nn.Sequential(classifier.backbone, classifier.pool_layer, classifier.bottleneck).to(device)\n source_feature = utils.collect_feature(val_loader, feature_extractor, device, max_num_features=100)\n target_feature = utils.collect_feature(test_loader, feature_extractor, device, max_num_features=100)\n print(len(source_feature), len(target_feature))\n # plot t-SNE\n tSNE_filename = osp.join(logger.visualize_directory, 'TSNE.png')\n tsne.visualize(source_feature, target_feature, tSNE_filename)\n print(\"Saving t-SNE to\", tSNE_filename)\n # calculate A-distance, which is a measure for distribution discrepancy\n A_distance = a_distance.calculate(source_feature, target_feature, device)\n print(\"A-distance =\", A_distance)\n return\n\n if args.phase == 'test':\n acc1 = utils.validate(test_loader, classifier, args, device)\n print(acc1)\n return\n\n # start training\n best_val_acc1 = 0.\n best_test_acc1 = 0.\n for epoch in range(args.epochs):\n print(lr_scheduler.get_lr())\n # train for one epoch\n train(train_iter, classifier, optimizer, lr_scheduler, epoch, args)\n\n # evaluate on validation set\n print(\"Evaluate on validation set...\")\n acc1 = utils.validate(val_loader, classifier, args, device)\n\n # remember best acc@1 and save checkpoint\n torch.save(classifier.state_dict(), logger.get_checkpoint_path('latest'))\n if acc1 > best_val_acc1:\n shutil.copy(logger.get_checkpoint_path('latest'), logger.get_checkpoint_path('best'))\n best_val_acc1 = max(acc1, best_val_acc1)\n\n # evaluate on test set\n print(\"Evaluate on test set...\")\n best_test_acc1 = max(best_test_acc1, utils.validate(test_loader, classifier, args, device))\n\n # evaluate on test set\n classifier.load_state_dict(torch.load(logger.get_checkpoint_path('best')))\n acc1 = utils.validate(test_loader, classifier, args, device)\n print(\"test acc on test set = {}\".format(acc1))\n print(\"oracle acc on test set = {}\".format(best_test_acc1))\n logger.close()\n\n\ndef train(train_iter: ForeverDataIterator, model: Classifier, optimizer,\n lr_scheduler: CosineAnnealingLR, epoch: int, args: argparse.Namespace):\n batch_time = AverageMeter('Time', ':4.2f')\n data_time = AverageMeter('Data', ':3.1f')\n losses = AverageMeter('Loss', ':3.2f')\n cls_accs = AverageMeter('Cls Acc', ':3.1f')\n\n progress = ProgressMeter(\n args.iters_per_epoch,\n [batch_time, data_time, losses, cls_accs],\n prefix=\"Epoch: [{}]\".format(epoch))\n\n # switch to train mode\n model.train()\n\n end = time.time()\n for i in range(args.iters_per_epoch):\n x, labels, _ = next(train_iter)\n x = x.to(device)\n labels = labels.to(device)\n\n # measure data loading time\n data_time.update(time.time() - end)\n\n # compute output\n y, _ = model(x)\n if len(y) > len(labels):\n labels = torch.cat([labels, labels], dim=0)\n cls_loss = F.cross_entropy(y, labels)\n loss = cls_loss\n\n cls_acc = accuracy(y, labels)[0]\n\n losses.update(loss.item(), x.size(0))\n cls_accs.update(cls_acc.item(), x.size(0))\n\n # compute gradient and do SGD step\n optimizer.zero_grad()\n loss.backward()\n optimizer.step()\n lr_scheduler.step()\n\n # measure elapsed time\n batch_time.update(time.time() - end)\n end = time.time()\n\n if i % args.print_freq == 0:\n progress.display(i)\n\n\nif __name__ == '__main__':\n architecture_names = sorted(\n name for name in models.__dict__\n if name.islower() and not name.startswith(\"__\")\n and callable(models.__dict__[name])\n )\n parser = argparse.ArgumentParser(description='MixStyle for Domain Generalization')\n # dataset parameters\n parser.add_argument('root', metavar='DIR',\n help='root path of dataset')\n parser.add_argument('-d', '--data', metavar='DATA', default='PACS',\n help='dataset: ' + ' | '.join(utils.get_dataset_names()) +\n ' (default: PACS)')\n parser.add_argument('-s', '--sources', nargs='+', default=None,\n help='source domain(s)')\n parser.add_argument('-t', '--targets', nargs='+', default=None,\n help='target domain(s)')\n parser.add_argument('--train-resizing', type=str, default='default')\n parser.add_argument('--val-resizing', type=str, default='default')\n # model parameters\n parser.add_argument('-a', '--arch', metavar='ARCH', default='resnet50',\n choices=architecture_names,\n help='backbone architecture: ' +\n ' | '.join(architecture_names) +\n ' (default: resnet50)')\n parser.add_argument('--no-pool', action='store_true', help='no pool layer after the feature extractor.')\n parser.add_argument('--mix-layers', nargs='+', help='layers to apply MixStyle')\n parser.add_argument('--mix-p', default=0.5, type=float, help='probability to apply MixStyle')\n parser.add_argument('--mix-alpha', default=0.1, type=float, help='parameter alpha for beta distribution')\n parser.add_argument('--finetune', action='store_true', help='whether use 10x smaller lr for backbone')\n parser.add_argument('--freeze-bn', action='store_true', help='whether freeze all bn layers')\n parser.add_argument('--dropout-p', type=float, default=0.1, help='only activated when freeze-bn is True')\n # training parameters\n parser.add_argument('-b', '--batch-size', default=36, type=int,\n metavar='N',\n help='mini-batch size (default: 36)')\n parser.add_argument('--lr', '--learning-rate', default=1e-3, type=float,\n metavar='LR', help='initial learning rate', dest='lr')\n parser.add_argument('--momentum', default=0.9, type=float, metavar='M',\n help='momentum')\n parser.add_argument('--wd', '--weight-decay', default=0.0005, type=float,\n metavar='W', help='weight decay (default: 5e-4)')\n parser.add_argument('-j', '--workers', default=4, type=int, metavar='N',\n help='number of data loading workers (default: 4)')\n parser.add_argument('--epochs', default=20, type=int, metavar='N',\n help='number of total epochs to run')\n parser.add_argument('-i', '--iters-per-epoch', default=500, type=int,\n help='Number of iterations per epoch')\n parser.add_argument('-p', '--print-freq', default=100, type=int,\n metavar='N', help='print frequency (default: 100)')\n parser.add_argument('--seed', default=None, type=int,\n help='seed for initializing training. ')\n parser.add_argument(\"--log\", type=str, default='mixstyle',\n help=\"Where to save logs, checkpoints and debugging images.\")\n parser.add_argument(\"--phase\", type=str, default='train', choices=['train', 'test', 'analysis'],\n help=\"When phase is 'test', only test the model.\"\n \"When phase is 'analysis', only analysis the model.\")\n parser.add_argument(\"--add_bt\", type=int, default=0,\n help=\"add_bt\")\n args = parser.parse_args()\n main(args)\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/examples/domain_generalization/image_classification/mixstyle.sh",
"content": "#!/usr/bin/env bash\n# ResNet50, PACS\nCUDA_VISIBLE_DEVICES=0 python mixstyle.py data/PACS -d PACS -s A C S -t P -a resnet50 --mix-layers layer1 layer2 layer3 --freeze-bn --seed 0 --log logs/mixstyle/PACS_P\nCUDA_VISIBLE_DEVICES=0 python mixstyle.py data/PACS -d PACS -s P C S -t A -a resnet50 --mix-layers layer1 layer2 layer3 --freeze-bn --seed 0 --log logs/mixstyle/PACS_A\nCUDA_VISIBLE_DEVICES=0 python mixstyle.py data/PACS -d PACS -s P A S -t C -a resnet50 --mix-layers layer1 layer2 layer3 --freeze-bn --seed 0 --log logs/mixstyle/PACS_C\nCUDA_VISIBLE_DEVICES=0 python mixstyle.py data/PACS -d PACS -s P A C -t S -a resnet50 --mix-layers layer1 layer2 layer3 --freeze-bn --seed 0 --log logs/mixstyle/PACS_S\n\n# ResNet50, Office-Home\nCUDA_VISIBLE_DEVICES=0 python mixstyle.py data/office-home -d OfficeHome -s Ar Cl Rw -t Pr -a resnet50 --mix-layers layer1 layer2 --seed 0 --log logs/mixstyle/OfficeHome_Pr\nCUDA_VISIBLE_DEVICES=0 python mixstyle.py data/office-home -d OfficeHome -s Ar Cl Pr -t Rw -a resnet50 --mix-layers layer1 layer2 --seed 0 --log logs/mixstyle/OfficeHome_Rw\nCUDA_VISIBLE_DEVICES=0 python mixstyle.py data/office-home -d OfficeHome -s Ar Rw Pr -t Cl -a resnet50 --mix-layers layer1 layer2 --seed 0 --log logs/mixstyle/OfficeHome_Cl\nCUDA_VISIBLE_DEVICES=0 python mixstyle.py data/office-home -d OfficeHome -s Cl Rw Pr -t Ar -a resnet50 --mix-layers layer1 layer2 --seed 0 --log logs/mixstyle/OfficeHome_Ar\n\n# ResNet50, DomainNet\nCUDA_VISIBLE_DEVICES=0 python mixstyle.py data/domainnet -d DomainNet -s i p q r s -t c -a resnet50 -i 2500 --lr 0.01 --seed 0 --log logs/mixstyle/DomainNet_c\nCUDA_VISIBLE_DEVICES=0 python mixstyle.py data/domainnet -d DomainNet -s c p q r s -t i -a resnet50 -i 2500 --lr 0.01 --seed 0 --log logs/mixstyle/DomainNet_i\nCUDA_VISIBLE_DEVICES=0 python mixstyle.py data/domainnet -d DomainNet -s c i q r s -t p -a resnet50 -i 2500 --lr 0.01 --seed 0 --log logs/mixstyle/DomainNet_p\nCUDA_VISIBLE_DEVICES=0 python mixstyle.py data/domainnet -d DomainNet -s c i p r s -t q -a resnet50 -i 2500 --lr 0.01 --seed 0 --log logs/mixstyle/DomainNet_q\nCUDA_VISIBLE_DEVICES=0 python mixstyle.py data/domainnet -d DomainNet -s c i p q s -t r -a resnet50 -i 2500 --lr 0.01 --seed 0 --log logs/mixstyle/DomainNet_r\nCUDA_VISIBLE_DEVICES=0 python mixstyle.py data/domainnet -d DomainNet -s c i p q r -t s -a resnet50 -i 2500 --lr 0.01 --seed 0 --log logs/mixstyle/DomainNet_s\n\n# ResNet50, Wilds Dataset\nCUDA_VISIBLE_DEVICES=3 python mixstyle.py data/wilds -d iwildcam --train-resizing 'res2x' --val-resizing 'res2x' \\\n -a resnet50 --mix-layers layer1 layer2 -b 16 --epochs 60 -i 1000 --lr 0.001 --finetune --seed 0 --log logs/mixstyle/iwildcam\nCUDA_VISIBLE_DEVICES=0 python mixstyle.py data/wilds -d camelyon17 -a resnet50 --mix-layers layer1 layer2 \\\n -b 36 --epochs 20 -i 1000 --lr 0.01 --seed 0 --log logs/mixstyle/camelyon17\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/examples/domain_generalization/image_classification/mldg.py",
"content": "\"\"\"\n@author: Baixu Chen\n@contact: cbx_99_hasta@outlook.com\n\"\"\"\nimport random\nimport time\nimport warnings\nimport sys\nimport argparse\nimport shutil\nimport os.path as osp\n\nimport torch\nimport torch.nn as nn\nimport torch.backends.cudnn as cudnn\nfrom torch.optim import SGD\nfrom torch.optim.lr_scheduler import CosineAnnealingLR\nfrom torch.utils.data import DataLoader\nimport torch.nn.functional as F\nimport higher\n\nsys.path.append('../../..')\nfrom dglib.modules.sampler import RandomDomainSampler\nfrom dglib.modules.classifier import ImageClassifier as Classifier\nfrom common.utils.data import ForeverDataIterator\nfrom common.utils.metric import accuracy\nfrom common.utils.meter import AverageMeter, ProgressMeter\nfrom common.utils.logger import CompleteLogger\nfrom common.utils.analysis import tsne, a_distance\n\nsys.path.append('.')\nimport utils\n\ndevice = torch.device(\"cuda\" if torch.cuda.is_available() else \"cpu\")\n\n\ndef main(args: argparse.Namespace):\n logger = CompleteLogger(args.log, args.phase)\n print(args)\n\n if args.seed is not None:\n random.seed(args.seed)\n torch.manual_seed(args.seed)\n cudnn.deterministic = True\n warnings.warn('You have chosen to seed training. '\n 'This will turn on the CUDNN deterministic setting, '\n 'which can slow down your training considerably! '\n 'You may see unexpected behavior when restarting '\n 'from checkpoints.')\n\n cudnn.benchmark = True\n\n # Data loading code\n train_transform = utils.get_train_transform(args.train_resizing, random_horizontal_flip=True,\n random_color_jitter=True, random_gray_scale=True)\n val_transform = utils.get_val_transform(args.val_resizing)\n print(\"train_transform: \", train_transform)\n print(\"val_transform: \", val_transform)\n\n train_dataset, num_classes = utils.get_dataset(dataset_name=args.data, root=args.root, task_list=args.sources,\n split='train', download=True, transform=train_transform,\n seed=args.seed)\n n_domains_per_batch = args.n_support_domains + args.n_query_domains\n sampler = RandomDomainSampler(train_dataset, args.batch_size, n_domains_per_batch=n_domains_per_batch)\n train_loader = DataLoader(train_dataset, batch_size=args.batch_size, num_workers=args.workers,\n sampler=sampler, drop_last=True)\n val_dataset, _ = utils.get_dataset(dataset_name=args.data, root=args.root, task_list=args.sources, split='val',\n download=True, transform=val_transform, seed=args.seed)\n val_loader = DataLoader(val_dataset, batch_size=args.batch_size, shuffle=False, num_workers=args.workers)\n test_dataset, _ = utils.get_dataset(dataset_name=args.data, root=args.root, task_list=args.targets, split='test',\n download=True, transform=val_transform, seed=args.seed)\n test_loader = DataLoader(test_dataset, batch_size=args.batch_size, shuffle=False, num_workers=args.workers)\n print(\"train_dataset_size: \", len(train_dataset))\n print('val_dataset_size: ', len(val_dataset))\n print(\"test_dataset_size: \", len(test_dataset))\n train_iter = ForeverDataIterator(train_loader)\n\n # create model\n print(\"=> using pre-trained model '{}'\".format(args.arch))\n backbone = utils.get_model(args.arch)\n pool_layer = nn.Identity() if args.no_pool else None\n classifier = Classifier(backbone, num_classes, freeze_bn=args.freeze_bn, dropout_p=args.dropout_p,\n finetune=args.finetune, pool_layer=pool_layer).to(device)\n\n # define optimizer and lr scheduler\n optimizer = SGD(classifier.get_parameters(base_lr=args.lr), args.lr, momentum=args.momentum, weight_decay=args.wd,\n nesterov=True)\n lr_scheduler = CosineAnnealingLR(optimizer, args.epochs * args.iters_per_epoch)\n\n # resume from the best checkpoint\n if args.phase != 'train':\n checkpoint = torch.load(logger.get_checkpoint_path('best'), map_location='cpu')\n classifier.load_state_dict(checkpoint)\n\n # analysis the model\n if args.phase == 'analysis':\n # extract features from both domains\n feature_extractor = nn.Sequential(classifier.backbone, classifier.pool_layer, classifier.bottleneck).to(device)\n source_feature = utils.collect_feature(val_loader, feature_extractor, device, max_num_features=100)\n target_feature = utils.collect_feature(test_loader, feature_extractor, device, max_num_features=100)\n print(len(source_feature), len(target_feature))\n # plot t-SNE\n tSNE_filename = osp.join(logger.visualize_directory, 'TSNE.png')\n tsne.visualize(source_feature, target_feature, tSNE_filename)\n print(\"Saving t-SNE to\", tSNE_filename)\n # calculate A-distance, which is a measure for distribution discrepancy\n A_distance = a_distance.calculate(source_feature, target_feature, device)\n print(\"A-distance =\", A_distance)\n return\n\n if args.phase == 'test':\n acc1 = utils.validate(test_loader, classifier, args, device)\n print(acc1)\n return\n\n # start training\n best_val_acc1 = 0.\n best_test_acc1 = 0.\n for epoch in range(args.epochs):\n print(lr_scheduler.get_lr())\n # train for one epoch\n train(train_iter, classifier, optimizer, lr_scheduler, epoch, n_domains_per_batch, args)\n\n # evaluate on validation set\n print(\"Evaluate on validation set...\")\n acc1 = utils.validate(val_loader, classifier, args, device)\n\n # remember best acc@1 and save checkpoint\n torch.save(classifier.state_dict(), logger.get_checkpoint_path('latest'))\n if acc1 > best_val_acc1:\n shutil.copy(logger.get_checkpoint_path('latest'), logger.get_checkpoint_path('best'))\n best_val_acc1 = max(acc1, best_val_acc1)\n\n # evaluate on test set\n print(\"Evaluate on test set...\")\n best_test_acc1 = max(best_test_acc1, utils.validate(test_loader, classifier, args, device))\n\n # evaluate on test set\n classifier.load_state_dict(torch.load(logger.get_checkpoint_path('best')))\n acc1 = utils.validate(test_loader, classifier, args, device)\n print(\"test acc on test set = {}\".format(acc1))\n print(\"oracle acc on test set = {}\".format(best_test_acc1))\n logger.close()\n\n\ndef random_split(x_list, labels_list, n_domains_per_batch, n_support_domains):\n assert n_support_domains < n_domains_per_batch\n\n support_domain_idxes = random.sample(range(n_domains_per_batch), n_support_domains)\n support_domain_list = [(x_list[idx], labels_list[idx]) for idx in range(n_domains_per_batch) if\n idx in support_domain_idxes]\n query_domain_list = [(x_list[idx], labels_list[idx]) for idx in range(n_domains_per_batch) if\n idx not in support_domain_idxes]\n\n return support_domain_list, query_domain_list\n\n\ndef train(train_iter: ForeverDataIterator, model: Classifier, optimizer, lr_scheduler: CosineAnnealingLR, epoch: int,\n n_domains_per_batch: int, args: argparse.Namespace):\n batch_time = AverageMeter('Time', ':4.2f')\n data_time = AverageMeter('Data', ':3.1f')\n losses = AverageMeter('Loss', ':3.2f')\n cls_accs = AverageMeter('Cls Acc', ':3.1f')\n\n progress = ProgressMeter(\n args.iters_per_epoch,\n [batch_time, data_time, losses, cls_accs],\n prefix=\"Epoch: [{}]\".format(epoch))\n\n # switch to train mode\n model.train()\n\n end = time.time()\n for i in range(args.iters_per_epoch):\n x, labels, _ = next(train_iter)\n x = x.to(device)\n labels = labels.to(device)\n\n # measure data loading time\n data_time.update(time.time() - end)\n\n # split into support domain and query domain\n x_list = x.chunk(n_domains_per_batch, dim=0)\n labels_list = labels.chunk(n_domains_per_batch, dim=0)\n support_domain_list, query_domain_list = random_split(x_list, labels_list, n_domains_per_batch,\n args.n_support_domains)\n # clear grad\n optimizer.zero_grad()\n\n # compute output\n with higher.innerloop_ctx(model, optimizer, copy_initial_weights=False) as (inner_model, inner_optimizer):\n # perform inner optimization\n for _ in range(args.inner_iters):\n loss_inner = 0\n for (x_s, labels_s) in support_domain_list:\n y_s, _ = inner_model(x_s)\n # normalize loss by support domain num\n loss_inner += F.cross_entropy(y_s, labels_s) / args.n_support_domains\n\n inner_optimizer.step(loss_inner)\n\n # calculate outer loss\n loss_outer = 0\n cls_acc = 0\n\n # loss on support domains\n for (x_s, labels_s) in support_domain_list:\n y_s, _ = model(x_s)\n # normalize loss by support domain num\n loss_outer += F.cross_entropy(y_s, labels_s) / args.n_support_domains\n\n # loss on query domains\n for (x_q, labels_q) in query_domain_list:\n y_q, _ = inner_model(x_q)\n # normalize loss by query domain num\n loss_outer += F.cross_entropy(y_q, labels_q) * args.trade_off / args.n_query_domains\n cls_acc += accuracy(y_q, labels_q)[0] / args.n_query_domains\n\n # update statistics\n losses.update(loss_outer.item(), args.batch_size)\n cls_accs.update(cls_acc.item(), args.batch_size)\n\n # compute gradient and do SGD step\n loss_outer.backward()\n optimizer.step()\n lr_scheduler.step()\n\n # measure elapsed time\n batch_time.update(time.time() - end)\n end = time.time()\n\n if i % args.print_freq == 0:\n progress.display(i)\n\n\nif __name__ == '__main__':\n parser = argparse.ArgumentParser(description='Meta Learning for Domain Generalization')\n # dataset parameters\n parser.add_argument('root', metavar='DIR',\n help='root path of dataset')\n parser.add_argument('-d', '--data', metavar='DATA', default='PACS',\n help='dataset: ' + ' | '.join(utils.get_dataset_names()) +\n ' (default: PACS)')\n parser.add_argument('-s', '--sources', nargs='+', default=None,\n help='source domain(s)')\n parser.add_argument('-t', '--targets', nargs='+', default=None,\n help='target domain(s)')\n parser.add_argument('--train-resizing', type=str, default='default')\n parser.add_argument('--val-resizing', type=str, default='default')\n # model parameters\n parser.add_argument('-a', '--arch', metavar='ARCH', default='resnet50',\n choices=utils.get_model_names(),\n help='backbone architecture: ' +\n ' | '.join(utils.get_model_names()) +\n ' (default: resnet50)')\n parser.add_argument('--no-pool', action='store_true', help='no pool layer after the feature extractor.')\n parser.add_argument('--finetune', action='store_true', help='whether use 10x smaller lr for backbone')\n parser.add_argument('--freeze-bn', action='store_true', help='whether freeze all bn layers')\n parser.add_argument('--dropout-p', type=float, default=0.1, help='only activated when freeze-bn is True')\n # training parameters\n parser.add_argument('--n-support-domains', type=int, default=1,\n help='Number of support domains sampled in each iteration')\n parser.add_argument('--n-query-domains', type=int, default=2,\n help='Number of query domains in each iteration')\n parser.add_argument('--trade-off', type=float, default=1,\n help='hyper parameter beta')\n parser.add_argument('-b', '--batch-size', default=36, type=int,\n metavar='N',\n help='mini-batch size (default: 36)')\n parser.add_argument('--lr', '--learning-rate', default=5e-4, type=float,\n metavar='LR', help='initial learning rate', dest='lr')\n parser.add_argument('--momentum', default=0.9, type=float, metavar='M',\n help='momentum')\n parser.add_argument('--wd', '--weight-decay', default=0.0005, type=float,\n metavar='W', help='weight decay (default: 5e-4)')\n parser.add_argument('-j', '--workers', default=4, type=int, metavar='N',\n help='number of data loading workers (default: 4)')\n parser.add_argument('--epochs', default=20, type=int, metavar='N',\n help='number of total epochs to run')\n parser.add_argument('-i', '--iters-per-epoch', default=500, type=int,\n help='Number of iterations per epoch')\n parser.add_argument('--inner-iters', default=1, type=int,\n help='Number of iterations in inner loop')\n parser.add_argument('-p', '--print-freq', default=100, type=int,\n metavar='N', help='print frequency (default: 100)')\n parser.add_argument('--seed', default=None, type=int,\n help='seed for initializing training. ')\n parser.add_argument(\"--log\", type=str, default='mldg',\n help=\"Where to save logs, checkpoints and debugging images.\")\n parser.add_argument(\"--phase\", type=str, default='train', choices=['train', 'test', 'analysis'],\n help=\"When phase is 'test', only test the model.\"\n \"When phase is 'analysis', only analysis the model.\")\n args = parser.parse_args()\n main(args)\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/examples/domain_generalization/image_classification/mldg.sh",
"content": "#!/usr/bin/env bash\n# ResNet50, PACS\nCUDA_VISIBLE_DEVICES=0 python mldg.py data/PACS -d PACS -s A C S -t P -a resnet50 --freeze-bn --seed 0 --log logs/mldg/PACS_P\nCUDA_VISIBLE_DEVICES=0 python mldg.py data/PACS -d PACS -s P C S -t A -a resnet50 --freeze-bn --seed 0 --log logs/mldg/PACS_A\nCUDA_VISIBLE_DEVICES=0 python mldg.py data/PACS -d PACS -s P A S -t C -a resnet50 --freeze-bn --seed 0 --log logs/mldg/PACS_C\nCUDA_VISIBLE_DEVICES=0 python mldg.py data/PACS -d PACS -s P A C -t S -a resnet50 --freeze-bn --seed 0 --log logs/mldg/PACS_S\n\n# ResNet50, Office-Home\nCUDA_VISIBLE_DEVICES=0 python mldg.py data/office-home -d OfficeHome -s Ar Cl Rw -t Pr -a resnet50 --seed 0 --log logs/mldg/OfficeHome_Pr\nCUDA_VISIBLE_DEVICES=0 python mldg.py data/office-home -d OfficeHome -s Ar Cl Pr -t Rw -a resnet50 --seed 0 --log logs/mldg/OfficeHome_Rw\nCUDA_VISIBLE_DEVICES=0 python mldg.py data/office-home -d OfficeHome -s Ar Rw Pr -t Cl -a resnet50 --seed 0 --log logs/mldg/OfficeHome_Cl\nCUDA_VISIBLE_DEVICES=0 python mldg.py data/office-home -d OfficeHome -s Cl Rw Pr -t Ar -a resnet50 --seed 0 --log logs/mldg/OfficeHome_Ar\n\n# ResNet50, DomainNet\nCUDA_VISIBLE_DEVICES=0 python mldg.py data/domainnet -d DomainNet -s i p q r s -t c -a resnet50 -i 5000 --lr 0.005 --seed 0 --log logs/mldg/DomainNet_c\nCUDA_VISIBLE_DEVICES=0 python mldg.py data/domainnet -d DomainNet -s c p q r s -t i -a resnet50 -i 5000 --lr 0.005 --seed 0 --log logs/mldg/DomainNet_i\nCUDA_VISIBLE_DEVICES=0 python mldg.py data/domainnet -d DomainNet -s c i q r s -t p -a resnet50 -i 5000 --lr 0.005 --seed 0 --log logs/mldg/DomainNet_p\nCUDA_VISIBLE_DEVICES=0 python mldg.py data/domainnet -d DomainNet -s c i p r s -t q -a resnet50 -i 5000 --lr 0.005 --seed 0 --log logs/mldg/DomainNet_q\nCUDA_VISIBLE_DEVICES=0 python mldg.py data/domainnet -d DomainNet -s c i p q s -t r -a resnet50 -i 5000 --lr 0.005 --seed 0 --log logs/mldg/DomainNet_r\nCUDA_VISIBLE_DEVICES=0 python mldg.py data/domainnet -d DomainNet -s c i p q r -t s -a resnet50 -i 5000 --lr 0.005 --seed 0 --log logs/mldg/DomainNet_s\n\n# ResNet50, Wilds Dataset\nCUDA_VISIBLE_DEVICES=0 python mldg.py data/wilds -d camelyon17 -a resnet50 -b 36 \\\n --n-support-domains 1 --n-query-domains 2 --epochs 20 -i 1000 --lr 0.003 --seed 0 --log logs/mldg/camelyon17\n\n# DenseNet121, Wilds Dataset\nCUDA_VISIBLE_DEVICES=5 python mldg.py data/wilds -d fmow --train-resizing 'cen.crop' -a densenet121 \\\n -b 36 --n-support-domains 2 --n-query-domains 2 --epochs 60 -i 1000 --lr 0.001 --seed 0 --log logs/mldg/fmow\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/examples/domain_generalization/image_classification/requirements.txt",
"content": "timm\nwilds\nhigher"
},
{
"path": "domain_generalization/Transfer-Learning-Library/examples/domain_generalization/image_classification/utils.py",
"content": "\"\"\"\n@author: Baixu Chen\n@contact: cbx_99_hasta@outlook.com\n\"\"\"\nimport sys\nimport time\nimport timm\nimport tqdm\nimport torch\nimport torch.nn as nn\nimport torchvision.transforms as T\nimport torch.nn.functional as F\nimport numpy as np\nfrom torch.utils.data.dataset import Subset, ConcatDataset\nimport wilds\n\nsys.path.append('../../..')\nimport common.vision.datasets as datasets\nimport common.vision.models as models\nfrom common.vision.transforms import ResizeImage\nfrom common.utils.metric import accuracy\nfrom common.utils.meter import AverageMeter, ProgressMeter\n\n\ndef get_model_names():\n return sorted(\n name for name in models.__dict__\n if name.islower() and not name.startswith(\"__\")\n and callable(models.__dict__[name])\n ) + timm.list_models()\n\n\ndef get_model(model_name):\n if model_name in models.__dict__:\n # load models from common.vision.models\n backbone = models.__dict__[model_name](pretrained=True)\n else:\n # load models from pytorch-image-models\n backbone = timm.create_model(model_name, pretrained=True)\n try:\n backbone.out_features = backbone.get_classifier().in_features\n backbone.reset_classifier(0, '')\n except:\n backbone.out_features = backbone.head.in_features\n backbone.head = nn.Identity()\n return backbone\n\n\ndef get_dataset_names():\n return sorted(\n name for name in datasets.__dict__\n if not name.startswith(\"__\") and callable(datasets.__dict__[name])\n ) + wilds.supported_datasets\n\n\nclass ConcatDatasetWithDomainLabel(ConcatDataset):\n \"\"\"ConcatDataset with domain label\"\"\"\n\n def __init__(self, *args, **kwargs):\n super(ConcatDatasetWithDomainLabel, self).__init__(*args, **kwargs)\n self.index_to_domain_id = {}\n domain_id = 0\n start = 0\n for end in self.cumulative_sizes:\n for idx in range(start, end):\n self.index_to_domain_id[idx] = domain_id\n start = end\n domain_id += 1\n\n def __getitem__(self, index):\n img, target = super(ConcatDatasetWithDomainLabel, self).__getitem__(index)\n domain_id = self.index_to_domain_id[index]\n return img, target, domain_id\n\n\ndef convert_from_wilds_dataset(dataset_name, wild_dataset):\n metadata_array = wild_dataset.metadata_array\n sample_idxes_per_domain = {}\n for idx, metadata in enumerate(metadata_array):\n if dataset_name == 'iwildcam':\n # In iwildcam dataset, domain id is specified by location\n domain = metadata[0].item()\n elif dataset_name == 'camelyon17':\n # In camelyon17 dataset, domain id is specified by hospital\n domain = metadata[0].item()\n elif dataset_name == 'fmow':\n # In fmow dataset, domain id is specified by (region, year) tuple\n domain = (metadata[0].item(), metadata[1].item())\n\n if domain not in sample_idxes_per_domain:\n sample_idxes_per_domain[domain] = []\n sample_idxes_per_domain[domain].append(idx)\n\n class Dataset:\n def __init__(self):\n self.dataset = wild_dataset\n\n def __getitem__(self, idx):\n x, y, metadata = self.dataset[idx]\n return x, y\n\n def __len__(self):\n return len(self.dataset)\n\n dataset = Dataset()\n concat_dataset = ConcatDatasetWithDomainLabel(\n [Subset(dataset, sample_idxes_per_domain[domain]) for domain in sample_idxes_per_domain])\n return concat_dataset\n\n\ndef get_dataset(dataset_name, root, task_list, split='train', download=True, transform=None, seed=0):\n assert split in ['train', 'val', 'test']\n if dataset_name in datasets.__dict__:\n # load datasets from common.vision.datasets\n # currently only PACS, OfficeHome and DomainNet are supported\n supported_dataset = ['PACS', 'OfficeHome', 'DomainNet']\n assert dataset_name in supported_dataset\n\n dataset = datasets.__dict__[dataset_name]\n\n train_split_list = []\n val_split_list = []\n test_split_list = []\n # we follow DomainBed and split each dataset randomly into two parts, with 80% samples and 20% samples\n # respectively, the former (larger) will be used as training set, and the latter will be used as validation set.\n split_ratio = 0.8\n num_classes = 0\n\n # under domain generalization setting, we use all samples in target domain as test set\n for task in task_list:\n if dataset_name == 'PACS':\n all_split = dataset(root=root, task=task, split='all', download=download, transform=transform)\n num_classes = all_split.num_classes\n elif dataset_name == 'OfficeHome':\n all_split = dataset(root=root, task=task, download=download, transform=transform)\n num_classes = all_split.num_classes\n elif dataset_name == 'DomainNet':\n train_split = dataset(root=root, task=task, split='train', download=download, transform=transform)\n test_split = dataset(root=root, task=task, split='test', download=download, transform=transform)\n num_classes = train_split.num_classes\n all_split = ConcatDataset([train_split, test_split])\n\n train_split, val_split = split_dataset(all_split, int(len(all_split) * split_ratio), seed)\n\n train_split_list.append(train_split)\n val_split_list.append(val_split)\n test_split_list.append(all_split)\n\n train_dataset = ConcatDatasetWithDomainLabel(train_split_list)\n val_dataset = ConcatDatasetWithDomainLabel(val_split_list)\n test_dataset = ConcatDatasetWithDomainLabel(test_split_list)\n\n dataset_dict = {\n 'train': train_dataset,\n 'val': val_dataset,\n 'test': test_dataset\n }\n return dataset_dict[split], num_classes\n else:\n # load datasets from wilds\n # currently only iwildcam, camelyon17 and fmow are supported\n supported_dataset = ['iwildcam', 'camelyon17', 'fmow']\n assert dataset_name in supported_dataset\n\n dataset = wilds.get_dataset(dataset_name, root_dir=root, download=True)\n num_classes = dataset.n_classes\n return convert_from_wilds_dataset(dataset_name,\n dataset.get_subset(split=split, transform=transform)), num_classes\n\n\ndef split_dataset(dataset, n, seed=0):\n \"\"\"\n Return a pair of datasets corresponding to a random split of the given\n dataset, with n data points in the first dataset and the rest in the last,\n using the given random seed\n \"\"\"\n assert (n <= len(dataset))\n idxes = list(range(len(dataset)))\n np.random.RandomState(seed).shuffle(idxes)\n subset_1 = idxes[:n]\n subset_2 = idxes[n:]\n return Subset(dataset, subset_1), Subset(dataset, subset_2)\n\n\ndef validate(val_loader, model, args, device) -> float:\n batch_time = AverageMeter('Time', ':6.3f')\n losses = AverageMeter('Loss', ':.4e')\n top1 = AverageMeter('Acc@1', ':6.2f')\n progress = ProgressMeter(\n len(val_loader),\n [batch_time, losses, top1],\n prefix='Test: ')\n\n # switch to evaluate mode\n model.eval()\n\n with torch.no_grad():\n end = time.time()\n for i, (images, target, _) in enumerate(val_loader):\n images = images.to(device)\n target = target.to(device)\n\n # compute output\n output = model(images)\n loss = F.cross_entropy(output, target)\n\n # measure accuracy and record loss\n acc1 = accuracy(output, target)[0]\n losses.update(loss.item(), images.size(0))\n top1.update(acc1.item(), images.size(0))\n\n # measure elapsed time\n batch_time.update(time.time() - end)\n end = time.time()\n\n if i % args.print_freq == 0:\n progress.display(i)\n\n print(' * Acc@1 {top1.avg:.3f} '.format(top1=top1))\n\n return top1.avg\n\n\ndef get_train_transform(resizing='default', random_horizontal_flip=True, random_color_jitter=True,\n random_gray_scale=True):\n \"\"\"\n resizing mode:\n - default: random resized crop with scale factor(0.7, 1.0) and size 224;\n - cen.crop: take the center crop of 224;\n - res.|cen.crop: resize the image to 256 and take the center crop of size 224;\n - res: resize the image to 224;\n - res2x: resize the image to 448;\n - res.|crop: resize the image to 256 and take a random crop of size 224;\n - res.sma|crop: resize the image keeping its aspect ratio such that the\n smaller side is 256, then take a random crop of size 224;\n – inc.crop: “inception crop” from (Szegedy et al., 2015);\n – cif.crop: resize the image to 224, zero-pad it by 28 on each side, then take a random crop of size 224.\n \"\"\"\n if resizing == 'default':\n transform = T.RandomResizedCrop(224, scale=(0.7, 1.0))\n elif resizing == 'cen.crop':\n transform = T.CenterCrop(224)\n elif resizing == 'res.|cen.crop':\n transform = T.Compose([\n ResizeImage(256),\n T.CenterCrop(224)\n ])\n elif resizing == 'res':\n transform = ResizeImage(224)\n elif resizing == 'res2x':\n transform = ResizeImage(448)\n elif resizing == 'res.|crop':\n transform = T.Compose([\n T.Resize((256, 256)),\n T.RandomCrop(224)\n ])\n elif resizing == \"res.sma|crop\":\n transform = T.Compose([\n T.Resize(256),\n T.RandomCrop(224)\n ])\n elif resizing == 'inc.crop':\n transform = T.RandomResizedCrop(224)\n elif resizing == 'cif.crop':\n transform = T.Compose([\n T.Resize((224, 224)),\n T.Pad(28),\n T.RandomCrop(224),\n ])\n else:\n raise NotImplementedError(resizing)\n transforms = [transform]\n if random_horizontal_flip:\n transforms.append(T.RandomHorizontalFlip())\n if random_color_jitter:\n transforms.append(T.ColorJitter(brightness=0.3, contrast=0.3, saturation=0.3, hue=0.3))\n if random_gray_scale:\n transforms.append(T.RandomGrayscale())\n transforms.extend([\n T.ToTensor(),\n T.Normalize(mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225])\n ])\n return T.Compose(transforms)\n\n\ndef get_val_transform(resizing='default'):\n \"\"\"\n resizing mode:\n - default: resize the image to 224;\n - res2x: resize the image to 448;\n - res.|cen.crop: resize the image to 256 and take the center crop of size 224;\n \"\"\"\n if resizing == 'default':\n transform = ResizeImage(224)\n elif resizing == 'res2x':\n transform = ResizeImage(448)\n elif resizing == 'res.|cen.crop':\n transform = T.Compose([\n ResizeImage(256),\n T.CenterCrop(224),\n ])\n else:\n raise NotImplementedError(resizing)\n return T.Compose([\n transform,\n T.ToTensor(),\n T.Normalize(mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225])\n ])\n\n\ndef collect_feature(data_loader, feature_extractor: nn.Module, device: torch.device,\n max_num_features=None) -> torch.Tensor:\n \"\"\"\n Fetch data from `data_loader`, and then use `feature_extractor` to collect features. This function is\n specific for domain generalization because each element in data_loader is a tuple\n (images, labels, domain_labels).\n\n Args:\n data_loader (torch.utils.data.DataLoader): Data loader.\n feature_extractor (torch.nn.Module): A feature extractor.\n device (torch.device)\n max_num_features (int): The max number of features to return\n\n Returns:\n Features in shape (min(len(data_loader), max_num_features * mini-batch size), :math:`|\\mathcal{F}|`).\n \"\"\"\n feature_extractor.eval()\n all_features = []\n with torch.no_grad():\n for i, (images, target, domain_labels) in enumerate(tqdm.tqdm(data_loader)):\n if max_num_features is not None and i >= max_num_features:\n break\n images = images.to(device)\n feature = feature_extractor(images).cpu()\n all_features.append(feature)\n return torch.cat(all_features, dim=0)\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/examples/domain_generalization/image_classification/vrex.py",
"content": "\"\"\"\n@author: Baixu Chen\n@contact: cbx_99_hasta@outlook.com\n\"\"\"\nimport random\nimport time\nimport warnings\nimport sys\nimport argparse\nimport shutil\nimport os.path as osp\n\nimport torch\nimport torch.nn as nn\nimport torch.backends.cudnn as cudnn\nfrom torch.optim import SGD\nfrom torch.optim.lr_scheduler import CosineAnnealingLR\nfrom torch.utils.data import DataLoader\nimport torch.nn.functional as F\n\nsys.path.append('../../..')\nfrom dglib.modules.sampler import RandomDomainSampler\nfrom dglib.modules.classifier import ImageClassifier as Classifier\nfrom common.utils.data import ForeverDataIterator\nfrom common.utils.metric import accuracy\nfrom common.utils.meter import AverageMeter, ProgressMeter\nfrom common.utils.logger import CompleteLogger\nfrom common.utils.analysis import tsne, a_distance\n\nsys.path.append('.')\nimport utils\n\ndevice = torch.device(\"cuda\" if torch.cuda.is_available() else \"cpu\")\n\nclass TransformerDecorator(torch.nn.Module):\n def __init__(self, backbone=None, add_bt=3, dim=2048, eval_global=0):\n super(TransformerDecorator, self).__init__()\n self.eval_global = eval_global\n self.add_bt = add_bt\n self.backbone = backbone\n if self.add_bt > 0:\n self.encoder_layers = torch.nn.TransformerEncoderLayer(dim, 4, dim, 0.5)\n\n def forward(self, feature):\n feature = self.backbone(feature)\n if self.training and self.add_bt > 0:\n pre_feature = feature\n feature = feature.unsqueeze(1)\n feature = self.encoder_layers(feature)\n # feature = self.encoder_layers(feature, feature, feature)\n feature = feature.squeeze(1)\n if self.add_bt >= 1:\n return torch.cat([pre_feature, feature], dim=0)\n return feature\n\n\ndef main(args: argparse.Namespace):\n logger = CompleteLogger(args.log, args.phase)\n print(args)\n\n if args.seed is not None:\n random.seed(args.seed)\n torch.manual_seed(args.seed)\n cudnn.deterministic = True\n warnings.warn('You have chosen to seed training. '\n 'This will turn on the CUDNN deterministic setting, '\n 'which can slow down your training considerably! '\n 'You may see unexpected behavior when restarting '\n 'from checkpoints.')\n\n cudnn.benchmark = True\n\n # Data loading code\n train_transform = utils.get_train_transform(args.train_resizing, random_horizontal_flip=True,\n random_color_jitter=True, random_gray_scale=True)\n val_transform = utils.get_val_transform(args.val_resizing)\n print(\"train_transform: \", train_transform)\n print(\"val_transform: \", val_transform)\n\n train_dataset, num_classes = utils.get_dataset(dataset_name=args.data, root=args.root, task_list=args.sources,\n split='train', download=True, transform=train_transform,\n seed=args.seed)\n sampler = RandomDomainSampler(train_dataset, args.batch_size, args.n_domains_per_batch)\n train_loader = DataLoader(train_dataset, batch_size=args.batch_size, num_workers=args.workers,\n sampler=sampler, drop_last=True)\n val_dataset, _ = utils.get_dataset(dataset_name=args.data, root=args.root, task_list=args.sources, split='val',\n download=True, transform=val_transform, seed=args.seed)\n val_loader = DataLoader(val_dataset, batch_size=args.batch_size, shuffle=False, num_workers=args.workers)\n test_dataset, _ = utils.get_dataset(dataset_name=args.data, root=args.root, task_list=args.targets, split='test',\n download=True, transform=val_transform, seed=args.seed)\n test_loader = DataLoader(test_dataset, batch_size=args.batch_size, shuffle=False, num_workers=args.workers)\n print(\"train_dataset_size: \", len(train_dataset))\n print('val_dataset_size: ', len(val_dataset))\n print(\"test_dataset_size: \", len(test_dataset))\n train_iter = ForeverDataIterator(train_loader)\n\n # create model\n print(\"=> using pre-trained model '{}'\".format(args.arch))\n backbone = utils.get_model(args.arch)\n pool_layer = nn.Identity() if args.no_pool else None\n pool_layer = nn.Sequential(\n nn.AdaptiveAvgPool2d(output_size=(1, 1)),\n nn.Flatten(),\n )\n pool_layer = TransformerDecorator(pool_layer, add_bt=args.add_bt, dim=backbone.out_features)\n classifier = Classifier(backbone, num_classes, freeze_bn=args.freeze_bn, dropout_p=args.dropout_p,\n finetune=args.finetune, pool_layer=pool_layer).to(device)\n\n # define optimizer and lr scheduler\n optimizer = SGD(classifier.get_parameters(base_lr=args.lr), args.lr, momentum=args.momentum, weight_decay=args.wd,\n nesterov=True)\n lr_scheduler = CosineAnnealingLR(optimizer, args.epochs * args.iters_per_epoch)\n\n # for simplicity\n assert args.anneal_iters % args.iters_per_epoch == 0\n\n # resume from the best checkpoint\n if args.phase != 'train':\n checkpoint = torch.load(logger.get_checkpoint_path('best'), map_location='cpu')\n classifier.load_state_dict(checkpoint)\n\n # analysis the model\n if args.phase == 'analysis':\n # extract features from both domains\n feature_extractor = nn.Sequential(classifier.backbone, classifier.pool_layer, classifier.bottleneck).to(device)\n source_feature = utils.collect_feature(val_loader, feature_extractor, device, max_num_features=100)\n target_feature = utils.collect_feature(test_loader, feature_extractor, device, max_num_features=100)\n print(len(source_feature), len(target_feature))\n # plot t-SNE\n tSNE_filename = osp.join(logger.visualize_directory, 'TSNE.png')\n tsne.visualize(source_feature, target_feature, tSNE_filename)\n print(\"Saving t-SNE to\", tSNE_filename)\n # calculate A-distance, which is a measure for distribution discrepancy\n A_distance = a_distance.calculate(source_feature, target_feature, device)\n print(\"A-distance =\", A_distance)\n return\n\n if args.phase == 'test':\n acc1 = utils.validate(test_loader, classifier, args, device)\n print(acc1)\n return\n\n # start training\n best_val_acc1 = 0.\n best_test_acc1 = 0.\n for epoch in range(args.epochs):\n if epoch * args.iters_per_epoch == args.anneal_iters:\n # reset optimizer to avoid sharp jump in gradient magnitudes\n optimizer = SGD(classifier.get_parameters(base_lr=args.lr), args.lr, momentum=args.momentum,\n weight_decay=args.wd, nesterov=True)\n lr_scheduler = CosineAnnealingLR(optimizer, args.epochs * args.iters_per_epoch - args.anneal_iters)\n\n print(lr_scheduler.get_lr())\n # train for one epoch\n train(train_iter, classifier, optimizer, lr_scheduler, args.n_domains_per_batch, epoch, args)\n\n # evaluate on validation set\n print(\"Evaluate on validation set...\")\n acc1 = utils.validate(val_loader, classifier, args, device)\n\n # remember best acc@1 and save checkpoint\n torch.save(classifier.state_dict(), logger.get_checkpoint_path('latest'))\n if acc1 > best_val_acc1:\n shutil.copy(logger.get_checkpoint_path('latest'), logger.get_checkpoint_path('best'))\n best_val_acc1 = max(acc1, best_val_acc1)\n\n # evaluate on test set\n print(\"Evaluate on test set...\")\n best_test_acc1 = max(best_test_acc1, utils.validate(test_loader, classifier, args, device))\n\n # evaluate on test set\n classifier.load_state_dict(torch.load(logger.get_checkpoint_path('best')))\n acc1 = utils.validate(test_loader, classifier, args, device)\n print(\"test acc on test set = {}\".format(acc1))\n print(\"oracle acc on test set = {}\".format(best_test_acc1))\n logger.close()\n\n\ndef train(train_iter: ForeverDataIterator, model: Classifier, optimizer, lr_scheduler: CosineAnnealingLR,\n n_domains_per_batch: int, epoch: int, args: argparse.Namespace):\n batch_time = AverageMeter('Time', ':4.2f')\n data_time = AverageMeter('Data', ':3.1f')\n losses = AverageMeter('Loss', ':3.2f')\n losses_ce = AverageMeter('CELoss', ':3.2f')\n losses_penalty = AverageMeter('Penalty Loss', ':3.2f')\n cls_accs = AverageMeter('Cls Acc', ':3.1f')\n\n progress = ProgressMeter(\n args.iters_per_epoch,\n [batch_time, data_time, losses, losses_ce, losses_penalty, cls_accs],\n prefix=\"Epoch: [{}]\".format(epoch))\n\n # switch to train mode\n model.train()\n\n end = time.time()\n for i in range(args.iters_per_epoch):\n x_all, labels_all, _ = next(train_iter)\n x_all = x_all.to(device)\n labels_all = labels_all.to(device)\n\n # measure data loading time\n data_time.update(time.time() - end)\n\n # compute output\n y_all, _ = model(x_all)\n if len(y_all) > len(labels_all):\n all_shape = y_all.shape\n y_all = torch.unsqueeze(y_all, 1)\n y_all = torch.cat([y_all[:len(y_all)//2], y_all[len(y_all)//2:]], dim=1)\n y_all = torch.reshape(y_all, all_shape)\n # old_label_shape = labels_all.shape\n labels_all = torch.unsqueeze(labels_all, 1)\n labels_all = torch.cat([labels_all, labels_all], dim=1)\n labels_all = torch.reshape(labels_all, [-1])\n loss_ce_per_domain = torch.zeros(n_domains_per_batch).to(device)\n for domain_id, (y_per_domain, labels_per_domain) in enumerate(\n zip(y_all.chunk(n_domains_per_batch, dim=0), labels_all.chunk(n_domains_per_batch, dim=0))):\n loss_ce_per_domain[domain_id] = F.cross_entropy(y_per_domain, labels_per_domain)\n\n # cls loss\n loss_ce = loss_ce_per_domain.mean()\n # penalty loss\n loss_penalty = ((loss_ce_per_domain - loss_ce) ** 2).mean()\n\n global_iter = epoch * args.iters_per_epoch + i\n if global_iter >= args.anneal_iters:\n trade_off = args.trade_off\n else:\n trade_off = 1\n\n loss = loss_ce + loss_penalty * trade_off\n cls_acc = accuracy(y_all, labels_all)[0]\n\n losses.update(loss.item(), x_all.size(0))\n losses_ce.update(loss_ce.item(), x_all.size(0))\n losses_penalty.update(loss_penalty.item(), x_all.size(0))\n cls_accs.update(cls_acc.item(), x_all.size(0))\n\n # compute gradient and do SGD step\n optimizer.zero_grad()\n loss.backward()\n optimizer.step()\n lr_scheduler.step()\n\n # measure elapsed time\n batch_time.update(time.time() - end)\n end = time.time()\n\n if i % args.print_freq == 0:\n progress.display(i)\n\n\nif __name__ == '__main__':\n parser = argparse.ArgumentParser(description='VREx for Domain Generalization')\n # dataset parameters\n parser.add_argument('root', metavar='DIR',\n help='root path of dataset')\n parser.add_argument('-d', '--data', metavar='DATA', default='PACS',\n help='dataset: ' + ' | '.join(utils.get_dataset_names()) +\n ' (default: PACS)')\n parser.add_argument('-s', '--sources', nargs='+', default=None,\n help='source domain(s)')\n parser.add_argument('-t', '--targets', nargs='+', default=None,\n help='target domain(s)')\n parser.add_argument('--train-resizing', type=str, default='default')\n parser.add_argument('--val-resizing', type=str, default='default')\n # model parameters\n parser.add_argument('-a', '--arch', metavar='ARCH', default='resnet50',\n choices=utils.get_model_names(),\n help='backbone architecture: ' +\n ' | '.join(utils.get_model_names()) +\n ' (default: resnet50)')\n parser.add_argument('--no-pool', action='store_true', help='no pool layer after the feature extractor.')\n parser.add_argument('--finetune', action='store_true', help='whether use 10x smaller lr for backbone')\n parser.add_argument('--freeze-bn', action='store_true', help='whether freeze all bn layers')\n parser.add_argument('--dropout-p', type=float, default=0.1, help='only activated when freeze-bn is True')\n # training parameters\n parser.add_argument('--trade-off', default=3, type=float,\n help='the trade off hyper parameter for vrex penalty')\n parser.add_argument('--anneal-iters', default=500, type=int,\n help='anneal iterations (trade off is set to 1 during these iterations)')\n parser.add_argument('-b', '--batch-size', default=36, type=int,\n metavar='N',\n help='mini-batch size (default: 36)')\n parser.add_argument('--n-domains-per-batch', default=3, type=int,\n help='number of domains in each mini-batch')\n parser.add_argument('--lr', '--learning-rate', default=5e-4, type=float,\n metavar='LR', help='initial learning rate', dest='lr')\n parser.add_argument('--momentum', default=0.9, type=float, metavar='M',\n help='momentum')\n parser.add_argument('--wd', '--weight-decay', default=0.0005, type=float,\n metavar='W', help='weight decay (default: 5e-4)')\n parser.add_argument('-j', '--workers', default=4, type=int, metavar='N',\n help='number of data loading workers (default: 4)')\n parser.add_argument('--epochs', default=20, type=int, metavar='N',\n help='number of total epochs to run')\n parser.add_argument('-i', '--iters-per-epoch', default=500, type=int,\n help='Number of iterations per epoch')\n parser.add_argument('-p', '--print-freq', default=100, type=int,\n metavar='N', help='print frequency (default: 100)')\n parser.add_argument('--seed', default=None, type=int,\n help='seed for initializing training. ')\n parser.add_argument(\"--log\", type=str, default='vrex',\n help=\"Where to save logs, checkpoints and debugging images.\")\n parser.add_argument(\"--phase\", type=str, default='train', choices=['train', 'test', 'analysis'],\n help=\"When phase is 'test', only test the model.\"\n \"When phase is 'analysis', only analysis the model.\")\n parser.add_argument(\"--add_bt\", type=int, default=0,\n help=\"add_bt\")\n args = parser.parse_args()\n main(args)\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/examples/domain_generalization/image_classification/vrex.sh",
"content": "#!/usr/bin/env bash\n# ResNet50, PACS\nCUDA_VISIBLE_DEVICES=0 python vrex.py data/PACS -d PACS -s A C S -t P -a resnet50 --freeze-bn --seed 0 --log logs/vrex/PACS_P\nCUDA_VISIBLE_DEVICES=0 python vrex.py data/PACS -d PACS -s P C S -t A -a resnet50 --freeze-bn --seed 0 --log logs/vrex/PACS_A\nCUDA_VISIBLE_DEVICES=0 python vrex.py data/PACS -d PACS -s P A S -t C -a resnet50 --freeze-bn --seed 0 --log logs/vrex/PACS_C\nCUDA_VISIBLE_DEVICES=0 python vrex.py data/PACS -d PACS -s P A C -t S -a resnet50 --freeze-bn --seed 0 --log logs/vrex/PACS_S\n\n# ResNet50, Office-Home\nCUDA_VISIBLE_DEVICES=0 python vrex.py data/office-home -d OfficeHome -s Ar Cl Rw -t Pr -a resnet50 --seed 0 --log logs/vrex/OfficeHome_Pr\nCUDA_VISIBLE_DEVICES=0 python vrex.py data/office-home -d OfficeHome -s Ar Cl Pr -t Rw -a resnet50 --seed 0 --log logs/vrex/OfficeHome_Rw\nCUDA_VISIBLE_DEVICES=0 python vrex.py data/office-home -d OfficeHome -s Ar Rw Pr -t Cl -a resnet50 --seed 0 --log logs/vrex/OfficeHome_Cl\nCUDA_VISIBLE_DEVICES=0 python vrex.py data/office-home -d OfficeHome -s Cl Rw Pr -t Ar -a resnet50 --seed 0 --log logs/vrex/OfficeHome_Ar\n\n# ResNet50, DomainNet\nCUDA_VISIBLE_DEVICES=0 python vrex.py data/domainnet -d DomainNet -s i p q r s -t c -a resnet50 -i 4000 --anneal-iters 4000 --lr 0.005 --trade-off 1 --seed 0 --log logs/vrex/DomainNet_c\nCUDA_VISIBLE_DEVICES=0 python vrex.py data/domainnet -d DomainNet -s c p q r s -t i -a resnet50 -i 4000 --anneal-iters 4000 --lr 0.005 --trade-off 1 --seed 0 --log logs/vrex/DomainNet_i\nCUDA_VISIBLE_DEVICES=0 python vrex.py data/domainnet -d DomainNet -s c i q r s -t p -a resnet50 -i 4000 --anneal-iters 4000 --lr 0.005 --trade-off 1 --seed 0 --log logs/vrex/DomainNet_p\nCUDA_VISIBLE_DEVICES=0 python vrex.py data/domainnet -d DomainNet -s c i p r s -t q -a resnet50 -i 4000 --anneal-iters 4000 --lr 0.005 --trade-off 1 --seed 0 --log logs/vrex/DomainNet_q\nCUDA_VISIBLE_DEVICES=0 python vrex.py data/domainnet -d DomainNet -s c i p q s -t r -a resnet50 -i 4000 --anneal-iters 4000 --lr 0.005 --trade-off 1 --seed 0 --log logs/vrex/DomainNet_r\nCUDA_VISIBLE_DEVICES=0 python vrex.py data/domainnet -d DomainNet -s c i p q r -t s -a resnet50 -i 4000 --anneal-iters 4000 --lr 0.005 --trade-off 1 --seed 0 --log logs/vrex/DomainNet_s\n\n# ResNet50, Wilds Dataset\nCUDA_VISIBLE_DEVICES=0 python vrex.py data/wilds -d iwildcam --train-resizing 'res2x' --val-resizing 'res2x' \\\n -a resnet50 -b 16 --n-domains-per-batch 2 --epochs 60 -i 1000 --anneal-iters 1000 --lr 0.001 --finetune --trade-off 0.1 \\\n --seed 0 --log logs/vrex/iwildcam\nCUDA_VISIBLE_DEVICES=0 python vrex.py data/wilds -d camelyon17 -a resnet50 -b 36 --n-domains-per-batch 3 \\\n --epochs 20 -i 1000 --anneal-iters 1000 --lr 0.01 --trade-off 0.1 --seed 0 --log logs/vrex/camelyon17\n\n# DenseNet121, Wilds Dataset\nCUDA_VISIBLE_DEVICES=0 python vrex.py data/wilds -d fmow --train-resizing 'cen.crop' -a densenet121 \\\n -b 36 --n-domains-per-batch 4 --epochs 60 -i 1000 --anneal-iters 1000 --lr 0.003 --trade-off 0.1 --seed 0 --log logs/vrex/fmow\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/examples/domain_generalization/re_identification/README.md",
"content": "# Domain Generalization for Person Re-Identification\n\n## Installation\nExample scripts support all models in [PyTorch-Image-Models](https://github.com/rwightman/pytorch-image-models).\nYou also need to install timm to use PyTorch-Image-Models.\n\n```\npip install timm\n```\n\n## Dataset \nFollowing datasets can be downloaded automatically:\n\n- [Market1501](http://zheng-lab.cecs.anu.edu.au/Project/project_reid.html)\n- [DukeMTMC](https://exposing.ai/duke_mtmc/)\n- [MSMT17](https://arxiv.org/pdf/1711.08565.pdf)\n\n## Supported Methods\n\nSupported methods include:\n\n- [Two at Once: Enhancing Learning and Generalization Capacities via IBN-Net (IBN-Net, 2018 ECCV)](https://openaccess.thecvf.com/content_ECCV_2018/papers/Xingang_Pan_Two_at_Once_ECCV_2018_paper.pdf)\n- [Domain Generalization with MixStyle (MixStyle, 2021 ICLR)](https://arxiv.org/abs/2104.02008)\n\n## Experiment and Results\n\nThe shell files give the script to reproduce the [benchmarks](/docs/dglib/benchmarks/re_identification.rst) with specified hyper-parameters.\nFor example, if you want to reproduce MixStyle on Market1501 -> DukeMTMC task, use the following script\n\n```shell script\n# Train MixStyle on Market1501 -> DukeMTMC task using ResNet 50.\n# Assume you have put the datasets under the path `data/market1501` and `data/dukemtmc`, \n# or you are glad to download the datasets automatically from the Internet to this path\nCUDA_VISIBLE_DEVICES=0 python mixstyle.py data -s Market1501 -t DukeMTMC -a resnet50 \\\n--mix-layers layer1 layer2 --finetune --seed 0 --log logs/mixstyle/Market2Duke\n```\n\nFor more information please refer to [Get Started](/docs/get_started/quickstart.rst) for help.\n\n## Citation\nIf you use these methods in your research, please consider citing.\n\n```\n@inproceedings{IBN-Net, \n author = {Xingang Pan, Ping Luo, Jianping Shi, and Xiaoou Tang}, \n title = {Two at Once: Enhancing Learning and Generalization Capacities via IBN-Net}, \n booktitle = {ECCV}, \n year = {2018} \n}\n\n@inproceedings{mixstyle,\n title={Domain Generalization with MixStyle},\n author={Zhou, Kaiyang and Yang, Yongxin and Qiao, Yu and Xiang, Tao},\n booktitle={ICLR},\n year={2021}\n}\n```"
},
{
"path": "domain_generalization/Transfer-Learning-Library/examples/domain_generalization/re_identification/baseline.py",
"content": "\"\"\"\n@author: Baixu Chen\n@contact: cbx_99_hasta@outlook.com\n\"\"\"\nimport random\nimport time\nimport warnings\nimport sys\nimport argparse\nimport shutil\nimport os.path as osp\n\nimport numpy as np\nimport torch\nimport torch.nn as nn\nfrom torch.nn import DataParallel\nimport torch.backends.cudnn as cudnn\nfrom torch.optim import Adam\nfrom torch.utils.data import DataLoader\n\nsys.path.append('../../..')\nfrom common.vision.models.reid.loss import CrossEntropyLossWithLabelSmooth, SoftTripletLoss\nfrom common.vision.models.reid.identifier import ReIdentifier\nimport common.vision.datasets.reid as datasets\nfrom common.vision.datasets.reid.convert import convert_to_pytorch_dataset\nfrom common.utils.scheduler import WarmupMultiStepLR\nfrom common.utils.metric.reid import validate, visualize_ranked_results\nfrom common.utils.data import ForeverDataIterator, RandomMultipleGallerySampler\nfrom common.utils.metric import accuracy\nfrom common.utils.meter import AverageMeter, ProgressMeter\nfrom common.utils.logger import CompleteLogger\n\nsys.path.append('.')\nimport utils\n\ndevice = torch.device(\"cuda\" if torch.cuda.is_available() else \"cpu\")\n\n\ndef main(args: argparse.Namespace):\n logger = CompleteLogger(args.log, args.phase)\n print(args)\n\n if args.seed is not None:\n random.seed(args.seed)\n np.random.seed(args.seed)\n torch.manual_seed(args.seed)\n cudnn.deterministic = True\n warnings.warn('You have chosen to seed training. '\n 'This will turn on the CUDNN deterministic setting, '\n 'which can slow down your training considerably! '\n 'You may see unexpected behavior when restarting '\n 'from checkpoints.')\n\n cudnn.benchmark = True\n\n # Data loading code\n train_transform = utils.get_train_transform(args.height, args.width, args.train_resizing,\n random_horizontal_flip=True,\n random_color_jitter=False,\n random_gray_scale=False)\n val_transform = utils.get_val_transform(args.height, args.width)\n print(\"train_transform: \", train_transform)\n print(\"val_transform: \", val_transform)\n\n working_dir = osp.dirname(osp.abspath(__file__))\n root = osp.join(working_dir, args.root)\n\n # source dataset\n source_dataset = datasets.__dict__[args.source](root=osp.join(root, args.source.lower()))\n sampler = RandomMultipleGallerySampler(source_dataset.train, args.num_instances)\n train_loader = DataLoader(\n convert_to_pytorch_dataset(source_dataset.train, root=source_dataset.images_dir, transform=train_transform),\n batch_size=args.batch_size, num_workers=args.workers, sampler=sampler, pin_memory=True, drop_last=True)\n train_iter = ForeverDataIterator(train_loader)\n val_loader = DataLoader(\n convert_to_pytorch_dataset(list(set(source_dataset.query) | set(source_dataset.gallery)),\n root=source_dataset.images_dir,\n transform=val_transform),\n batch_size=args.batch_size, num_workers=args.workers, shuffle=False, pin_memory=True)\n\n # target dataset\n target_dataset = datasets.__dict__[args.target](root=osp.join(root, args.target.lower()))\n test_loader = DataLoader(\n convert_to_pytorch_dataset(list(set(target_dataset.query) | set(target_dataset.gallery)),\n root=target_dataset.images_dir,\n transform=val_transform),\n batch_size=args.batch_size, num_workers=args.workers, shuffle=False, pin_memory=True)\n\n # create model\n num_classes = source_dataset.num_train_pids\n backbone = utils.get_model(args.arch)\n pool_layer = nn.Identity() if args.no_pool else None\n model = ReIdentifier(backbone, num_classes, finetune=args.finetune, pool_layer=pool_layer).to(device)\n model = DataParallel(model)\n\n # define optimizer and learning rate scheduler\n optimizer = Adam(model.module.get_parameters(base_lr=args.lr, rate=args.rate), args.lr,\n weight_decay=args.weight_decay)\n lr_scheduler = WarmupMultiStepLR(optimizer, args.milestones, gamma=0.1, warmup_factor=0.1,\n warmup_steps=args.warmup_steps)\n\n # resume from the best checkpoint\n if args.phase != 'train':\n checkpoint = torch.load(logger.get_checkpoint_path('best'), map_location='cpu')\n model.load_state_dict(checkpoint)\n\n # analysis the model\n if args.phase == 'analysis':\n # plot t-SNE\n utils.visualize_tsne(source_loader=val_loader, target_loader=test_loader, model=model,\n filename=osp.join(logger.visualize_directory, 'analysis', 'TSNE.pdf'), device=device)\n # visualize ranked results\n visualize_ranked_results(test_loader, model, target_dataset.query, target_dataset.gallery, device,\n visualize_dir=logger.visualize_directory, width=args.width, height=args.height,\n rerank=args.rerank)\n return\n\n if args.phase == 'test':\n print(\"Test on source domain:\")\n validate(val_loader, model, source_dataset.query, source_dataset.gallery, device, cmc_flag=True,\n rerank=args.rerank)\n print(\"Test on target domain:\")\n validate(test_loader, model, target_dataset.query, target_dataset.gallery, device, cmc_flag=True,\n rerank=args.rerank)\n return\n\n # define loss function\n criterion_ce = CrossEntropyLossWithLabelSmooth(num_classes).to(device)\n criterion_triplet = SoftTripletLoss(margin=args.margin).to(device)\n\n # start training\n best_val_mAP = 0.\n best_test_mAP = 0.\n for epoch in range(args.epochs):\n # print learning rate\n print(lr_scheduler.get_lr())\n\n # train for one epoch\n train(train_iter, model, criterion_ce, criterion_triplet, optimizer, epoch, args)\n\n # update learning rate\n lr_scheduler.step()\n\n if (epoch + 1) % args.eval_step == 0 or (epoch == args.epochs - 1):\n\n # evaluate on validation set\n print(\"Validation on source domain...\")\n _, val_mAP = validate(val_loader, model, source_dataset.query, source_dataset.gallery, device,\n cmc_flag=True)\n\n # remember best mAP and save checkpoint\n torch.save(model.state_dict(), logger.get_checkpoint_path('latest'))\n if val_mAP > best_val_mAP:\n shutil.copy(logger.get_checkpoint_path('latest'), logger.get_checkpoint_path('best'))\n best_val_mAP = max(val_mAP, best_val_mAP)\n\n # evaluate on test set\n print(\"Test on target domain...\")\n _, test_mAP = validate(test_loader, model, target_dataset.query, target_dataset.gallery, device,\n cmc_flag=True, rerank=args.rerank)\n best_test_mAP = max(test_mAP, best_test_mAP)\n\n # evaluate on test set\n model.load_state_dict(torch.load(logger.get_checkpoint_path('best')))\n print(\"Test on target domain:\")\n _, test_mAP = validate(test_loader, model, target_dataset.query, target_dataset.gallery, device,\n cmc_flag=True, rerank=args.rerank)\n print(\"test mAP on target = {}\".format(test_mAP))\n print(\"oracle mAP on target = {}\".format(best_test_mAP))\n logger.close()\n\n\ndef train(train_iter: ForeverDataIterator, model, criterion_ce: CrossEntropyLossWithLabelSmooth,\n criterion_triplet: SoftTripletLoss, optimizer: Adam, epoch: int, args: argparse.Namespace):\n batch_time = AverageMeter('Time', ':4.2f')\n data_time = AverageMeter('Data', ':3.1f')\n losses_ce = AverageMeter('CeLoss', ':3.2f')\n losses_triplet = AverageMeter('TripletLoss', ':3.2f')\n losses = AverageMeter('Loss', ':3.2f')\n cls_accs = AverageMeter('Cls Acc', ':3.1f')\n\n progress = ProgressMeter(\n args.iters_per_epoch,\n [batch_time, data_time, losses_ce, losses_triplet, losses, cls_accs],\n prefix=\"Epoch: [{}]\".format(epoch))\n\n # switch to train mode\n model.train()\n\n end = time.time()\n\n for i in range(args.iters_per_epoch):\n x, _, labels, _ = next(train_iter)\n x = x.to(device)\n labels = labels.to(device)\n\n # measure data loading time\n data_time.update(time.time() - end)\n\n # compute output\n y, f = model(x)\n\n # cross entropy loss\n loss_ce = criterion_ce(y, labels)\n # triplet loss\n loss_triplet = criterion_triplet(f, f, labels)\n loss = loss_ce + loss_triplet * args.trade_off\n\n cls_acc = accuracy(y, labels)[0]\n losses_ce.update(loss_ce.item(), x.size(0))\n losses_triplet.update(loss_triplet.item(), x.size(0))\n losses.update(loss.item(), x.size(0))\n cls_accs.update(cls_acc.item(), x.size(0))\n\n # compute gradient and do SGD step\n optimizer.zero_grad()\n loss.backward()\n optimizer.step()\n\n # measure elapsed time\n batch_time.update(time.time() - end)\n end = time.time()\n\n if i % args.print_freq == 0:\n progress.display(i)\n\n\nif __name__ == '__main__':\n dataset_names = sorted(\n name for name in datasets.__dict__\n if not name.startswith(\"__\") and callable(datasets.__dict__[name])\n )\n parser = argparse.ArgumentParser(description=\"Baseline for Domain Generalizable ReID\")\n # dataset parameters\n parser.add_argument('root', metavar='DIR',\n help='root path of dataset')\n parser.add_argument('-s', '--source', type=str, help='source domain')\n parser.add_argument('-t', '--target', type=str, help='target domain')\n parser.add_argument('--train-resizing', type=str, default='default')\n # model parameters\n parser.add_argument('-a', '--arch', metavar='ARCH', default='reid_resnet50',\n choices=utils.get_model_names(),\n help='backbone architecture: ' +\n ' | '.join(utils.get_model_names()) +\n ' (default: reid_resnet50)')\n parser.add_argument('--no-pool', action='store_true', help='no pool layer after the feature extractor.')\n parser.add_argument('--finetune', action='store_true', help='whether use 10x smaller lr for backbone')\n parser.add_argument('--rate', type=float, default=0.2)\n # training parameters\n parser.add_argument('--trade-off', type=float, default=1,\n help='trade-off hyper parameter between cross entropy loss and triplet loss')\n parser.add_argument('--margin', type=float, default=0.0, help='margin for the triplet loss with batch hard')\n parser.add_argument('-j', '--workers', type=int, default=4)\n parser.add_argument('-b', '--batch-size', type=int, default=16)\n parser.add_argument('--height', type=int, default=256, help=\"input height\")\n parser.add_argument('--width', type=int, default=128, help=\"input width\")\n parser.add_argument('--num-instances', type=int, default=4,\n help=\"each minibatch consist of \"\n \"(batch_size // num_instances) identities, and \"\n \"each identity has num_instances instances, \"\n \"default: 4\")\n parser.add_argument('--lr', type=float, default=0.00035,\n help=\"initial learning rate\")\n parser.add_argument('--weight-decay', type=float, default=5e-4)\n parser.add_argument('--epochs', type=int, default=80)\n parser.add_argument('--warmup-steps', type=int, default=10, help='number of warp-up steps')\n parser.add_argument('--milestones', nargs='+', type=int, default=[40, 70],\n help='milestones for the learning rate decay')\n parser.add_argument('--eval-step', type=int, default=40)\n parser.add_argument('--iters-per-epoch', type=int, default=400)\n parser.add_argument('--print-freq', type=int, default=40)\n parser.add_argument('--seed', default=None, type=int, help='seed for initializing training.')\n parser.add_argument('--rerank', action='store_true', help=\"evaluation only\")\n parser.add_argument(\"--log\", type=str, default='baseline',\n help=\"Where to save logs, checkpoints and debugging images.\")\n parser.add_argument(\"--phase\", type=str, default='train', choices=['train', 'test', 'analysis'],\n help=\"When phase is 'test', only test the model.\"\n \"When phase is 'analysis', only analysis the model.\")\n args = parser.parse_args()\n main(args)\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/examples/domain_generalization/re_identification/baseline.sh",
"content": "#!/usr/bin/env bash\n# Market1501 -> Duke\nCUDA_VISIBLE_DEVICES=0 python baseline.py data -s Market1501 -t DukeMTMC -a reid_resnet50 \\\n--finetune --seed 0 --log logs/baseline/Market2Duke\n\n# Duke -> Market1501\nCUDA_VISIBLE_DEVICES=0 python baseline.py data -s DukeMTMC -t Market1501 -a reid_resnet50 \\\n--finetune --seed 0 --log logs/baseline/Duke2Market\n\n# Market1501 -> MSMT\nCUDA_VISIBLE_DEVICES=0 python baseline.py data -s Market1501 -t MSMT17 -a reid_resnet50 \\\n--finetune --seed 0 --log logs/baseline/Market2MSMT\n\n# MSMT -> Market1501\nCUDA_VISIBLE_DEVICES=0 python baseline.py data -s MSMT17 -t Market1501 -a reid_resnet50 \\\n--finetune --seed 0 --log logs/baseline/MSMT2Market\n\n# Duke -> MSMT\nCUDA_VISIBLE_DEVICES=0 python baseline.py data -s DukeMTMC -t MSMT17 -a reid_resnet50 \\\n--finetune --seed 0 --log logs/baseline/Duke2MSMT\n\n# MSMT -> Duke\nCUDA_VISIBLE_DEVICES=0 python baseline.py data -s MSMT17 -t DukeMTMC -a reid_resnet50 \\\n--finetune --seed 0 --log logs/baseline/MSMT2Duke\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/examples/domain_generalization/re_identification/ibn.sh",
"content": "#!/usr/bin/env bash\n# Market1501 -> Duke\nCUDA_VISIBLE_DEVICES=0 python baseline.py data -s Market1501 -t DukeMTMC -a resnet50_ibn_a \\\n--finetune --seed 0 --log logs/ibn/Market2Duke\nCUDA_VISIBLE_DEVICES=0 python baseline.py data -s Market1501 -t DukeMTMC -a resnet50_ibn_b \\\n--finetune --seed 0 --log logs/ibn/Market2Duke\n\n# Duke -> Market1501\nCUDA_VISIBLE_DEVICES=0 python baseline.py data -s DukeMTMC -t Market1501 -a resnet50_ibn_a \\\n--finetune --seed 0 --log logs/ibn/Duke2Market\nCUDA_VISIBLE_DEVICES=0 python baseline.py data -s DukeMTMC -t Market1501 -a resnet50_ibn_b \\\n--finetune --seed 0 --log logs/ibn/Duke2Market\n\n# Market1501 -> MSMT\nCUDA_VISIBLE_DEVICES=0 python baseline.py data -s Market1501 -t MSMT17 -a resnet50_ibn_a \\\n--finetune --seed 0 --log logs/ibn/Market2MSMT\nCUDA_VISIBLE_DEVICES=0 python baseline.py data -s Market1501 -t MSMT17 -a resnet50_ibn_b \\\n--finetune --seed 0 --log logs/ibn/Market2MSMT\n\n# MSMT -> Market1501\nCUDA_VISIBLE_DEVICES=0 python baseline.py data -s MSMT17 -t Market1501 -a resnet50_ibn_a \\\n--finetune --seed 0 --log logs/ibn/MSMT2Market\nCUDA_VISIBLE_DEVICES=0 python baseline.py data -s MSMT17 -t Market1501 -a resnet50_ibn_b \\\n--finetune --seed 0 --log logs/ibn/MSMT2Market\n\n# Duke -> MSMT\nCUDA_VISIBLE_DEVICES=0 python baseline.py data -s DukeMTMC -t MSMT17 -a resnet50_ibn_a \\\n--finetune --seed 0 --log logs/ibn/Duke2MSMT\nCUDA_VISIBLE_DEVICES=0 python baseline.py data -s DukeMTMC -t MSMT17 -a resnet50_ibn_b \\\n--finetune --seed 0 --log logs/ibn/Duke2MSMT\n\n# MSMT -> Duke\nCUDA_VISIBLE_DEVICES=0 python baseline.py data -s MSMT17 -t DukeMTMC -a resnet50_ibn_a \\\n--finetune --seed 0 --log logs/ibn/MSMT2Duke\nCUDA_VISIBLE_DEVICES=0 python baseline.py data -s MSMT17 -t DukeMTMC -a resnet50_ibn_b \\\n--finetune --seed 0 --log logs/ibn/MSMT2Duke\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/examples/domain_generalization/re_identification/mixstyle.py",
"content": "\"\"\"\n@author: Baixu Chen\n@contact: cbx_99_hasta@outlook.com\n\"\"\"\nimport random\nimport time\nimport warnings\nimport sys\nimport argparse\nimport shutil\nimport os.path as osp\n\nimport numpy as np\nimport torch\nfrom torch.nn import DataParallel\nimport torch.backends.cudnn as cudnn\nfrom torch.optim import Adam\nfrom torch.utils.data import DataLoader\n\nsys.path.append('../../..')\nfrom dglib.generalization.mixstyle.sampler import RandomDomainMultiInstanceSampler\nimport dglib.generalization.mixstyle.models as models\nfrom common.vision.models.reid.identifier import ReIdentifier\nfrom common.vision.models.reid.loss import CrossEntropyLossWithLabelSmooth, SoftTripletLoss\nimport common.vision.datasets.reid as datasets\nfrom common.vision.datasets.reid.convert import convert_to_pytorch_dataset\nfrom common.vision.models.reid.resnet import ReidResNet\nfrom common.utils.scheduler import WarmupMultiStepLR\nfrom common.utils.metric.reid import validate, visualize_ranked_results\nfrom common.utils.data import ForeverDataIterator\nfrom common.utils.metric import accuracy\nfrom common.utils.meter import AverageMeter, ProgressMeter\nfrom common.utils.logger import CompleteLogger\n\nsys.path.append('.')\nimport utils\n\ndevice = torch.device(\"cuda\" if torch.cuda.is_available() else \"cpu\")\n\n\ndef main(args: argparse.Namespace):\n logger = CompleteLogger(args.log, args.phase)\n print(args)\n\n if args.seed is not None:\n random.seed(args.seed)\n np.random.seed(args.seed)\n torch.manual_seed(args.seed)\n cudnn.deterministic = True\n warnings.warn('You have chosen to seed training. '\n 'This will turn on the CUDNN deterministic setting, '\n 'which can slow down your training considerably! '\n 'You may see unexpected behavior when restarting '\n 'from checkpoints.')\n\n cudnn.benchmark = True\n\n # Data loading code\n train_transform = utils.get_train_transform(args.height, args.width, args.train_resizing,\n random_horizontal_flip=True,\n random_color_jitter=False,\n random_gray_scale=False)\n val_transform = utils.get_val_transform(args.height, args.width)\n print(\"train_transform: \", train_transform)\n print(\"val_transform: \", val_transform)\n\n working_dir = osp.dirname(osp.abspath(__file__))\n root = osp.join(working_dir, args.root)\n\n # source dataset\n source_dataset = datasets.__dict__[args.source](root=osp.join(root, args.source.lower()))\n sampler = RandomDomainMultiInstanceSampler(source_dataset.train, batch_size=args.batch_size, n_domains_per_batch=2,\n num_instances=args.num_instances)\n train_loader = DataLoader(\n convert_to_pytorch_dataset(source_dataset.train, root=source_dataset.images_dir, transform=train_transform),\n batch_size=args.batch_size, num_workers=args.workers, sampler=sampler, pin_memory=True, drop_last=True)\n train_iter = ForeverDataIterator(train_loader)\n val_loader = DataLoader(\n convert_to_pytorch_dataset(list(set(source_dataset.query) | set(source_dataset.gallery)),\n root=source_dataset.images_dir,\n transform=val_transform),\n batch_size=args.batch_size, num_workers=args.workers, shuffle=False, pin_memory=True)\n\n # target dataset\n target_dataset = datasets.__dict__[args.target](root=osp.join(root, args.target.lower()))\n test_loader = DataLoader(\n convert_to_pytorch_dataset(list(set(target_dataset.query) | set(target_dataset.gallery)),\n root=target_dataset.images_dir,\n transform=val_transform),\n batch_size=args.batch_size, num_workers=args.workers, shuffle=False, pin_memory=True)\n\n # create model\n num_classes = source_dataset.num_train_pids\n backbone = models.__dict__[args.arch](mix_layers=args.mix_layers, mix_p=args.mix_p, mix_alpha=args.mix_alpha,\n resnet_class=ReidResNet, pretrained=True)\n model = ReIdentifier(backbone, num_classes, finetune=args.finetune).to(device)\n model = DataParallel(model)\n\n # define optimizer and learning rate scheduler\n optimizer = Adam(model.module.get_parameters(base_lr=args.lr, rate=args.rate), args.lr,\n weight_decay=args.weight_decay)\n lr_scheduler = WarmupMultiStepLR(optimizer, args.milestones, gamma=0.1, warmup_factor=0.1,\n warmup_steps=args.warmup_steps)\n\n # resume from the best checkpoint\n if args.phase != 'train':\n checkpoint = torch.load(logger.get_checkpoint_path('best'), map_location='cpu')\n model.load_state_dict(checkpoint)\n\n # analysis the model\n if args.phase == 'analysis':\n # plot t-SNE\n utils.visualize_tsne(source_loader=val_loader, target_loader=test_loader, model=model,\n filename=osp.join(logger.visualize_directory, 'analysis', 'TSNE.pdf'), device=device)\n # visualize ranked results\n visualize_ranked_results(test_loader, model, target_dataset.query, target_dataset.gallery, device,\n visualize_dir=logger.visualize_directory, width=args.width, height=args.height,\n rerank=args.rerank)\n return\n\n if args.phase == 'test':\n print(\"Test on source domain:\")\n validate(val_loader, model, source_dataset.query, source_dataset.gallery, device, cmc_flag=True,\n rerank=args.rerank)\n print(\"Test on target domain:\")\n validate(test_loader, model, target_dataset.query, target_dataset.gallery, device, cmc_flag=True,\n rerank=args.rerank)\n return\n\n # define loss function\n criterion_ce = CrossEntropyLossWithLabelSmooth(num_classes).to(device)\n criterion_triplet = SoftTripletLoss(margin=args.margin).to(device)\n\n # start training\n best_val_mAP = 0.\n best_test_mAP = 0.\n for epoch in range(args.epochs):\n # print learning rate\n print(lr_scheduler.get_lr())\n\n # train for one epoch\n train(train_iter, model, criterion_ce, criterion_triplet, optimizer, epoch, args)\n\n # update learning rate\n lr_scheduler.step()\n\n if (epoch + 1) % args.eval_step == 0 or (epoch == args.epochs - 1):\n\n # evaluate on validation set\n print(\"Validation on source domain...\")\n _, val_mAP = validate(val_loader, model, source_dataset.query, source_dataset.gallery, device,\n cmc_flag=True)\n\n # remember best mAP and save checkpoint\n torch.save(model.state_dict(), logger.get_checkpoint_path('latest'))\n if val_mAP > best_val_mAP:\n shutil.copy(logger.get_checkpoint_path('latest'), logger.get_checkpoint_path('best'))\n best_val_mAP = max(val_mAP, best_val_mAP)\n\n # evaluate on test set\n print(\"Test on target domain...\")\n _, test_mAP = validate(test_loader, model, target_dataset.query, target_dataset.gallery, device,\n cmc_flag=True, rerank=args.rerank)\n best_test_mAP = max(test_mAP, best_test_mAP)\n\n # evaluate on test set\n model.load_state_dict(torch.load(logger.get_checkpoint_path('best')))\n print(\"Test on target domain:\")\n _, test_mAP = validate(test_loader, model, target_dataset.query, target_dataset.gallery, device,\n cmc_flag=True, rerank=args.rerank)\n print(\"test mAP on target = {}\".format(test_mAP))\n print(\"oracle mAP on target = {}\".format(best_test_mAP))\n logger.close()\n\n\ndef train(train_iter: ForeverDataIterator, model, criterion_ce: CrossEntropyLossWithLabelSmooth,\n criterion_triplet: SoftTripletLoss, optimizer: Adam, epoch: int, args: argparse.Namespace):\n batch_time = AverageMeter('Time', ':4.2f')\n data_time = AverageMeter('Data', ':3.1f')\n losses_ce = AverageMeter('CeLoss', ':3.2f')\n losses_triplet = AverageMeter('TripletLoss', ':3.2f')\n losses = AverageMeter('Loss', ':3.2f')\n cls_accs = AverageMeter('Cls Acc', ':3.1f')\n\n progress = ProgressMeter(\n args.iters_per_epoch,\n [batch_time, data_time, losses_ce, losses_triplet, losses, cls_accs],\n prefix=\"Epoch: [{}]\".format(epoch))\n\n # switch to train mode\n model.train()\n\n end = time.time()\n\n for i in range(args.iters_per_epoch):\n x, _, labels, _ = next(train_iter)\n x = x.to(device)\n labels = labels.to(device)\n\n # measure data loading time\n data_time.update(time.time() - end)\n\n # compute output\n y, f = model(x)\n\n # cross entropy loss\n loss_ce = criterion_ce(y, labels)\n # triplet loss\n loss_triplet = criterion_triplet(f, f, labels)\n loss = loss_ce + loss_triplet * args.trade_off\n\n cls_acc = accuracy(y, labels)[0]\n losses_ce.update(loss_ce.item(), x.size(0))\n losses_triplet.update(loss_triplet.item(), x.size(0))\n losses.update(loss.item(), x.size(0))\n cls_accs.update(cls_acc.item(), x.size(0))\n\n # compute gradient and do SGD step\n optimizer.zero_grad()\n loss.backward()\n optimizer.step()\n\n # measure elapsed time\n batch_time.update(time.time() - end)\n end = time.time()\n\n if i % args.print_freq == 0:\n progress.display(i)\n\n\nif __name__ == '__main__':\n architecture_names = sorted(\n name for name in models.__dict__\n if name.islower() and not name.startswith(\"__\")\n and callable(models.__dict__[name])\n )\n dataset_names = sorted(\n name for name in datasets.__dict__\n if not name.startswith(\"__\") and callable(datasets.__dict__[name])\n )\n parser = argparse.ArgumentParser(description=\"MixStyle for Domain Generalizable ReID\")\n # dataset parameters\n parser.add_argument('root', metavar='DIR',\n help='root path of dataset')\n parser.add_argument('-s', '--source', type=str, help='source domain')\n parser.add_argument('-t', '--target', type=str, help='target domain')\n parser.add_argument('--train-resizing', type=str, default='default')\n # model parameters\n parser.add_argument('-a', '--arch', metavar='ARCH', default='resnet50',\n choices=architecture_names,\n help='backbone architecture: ' +\n ' | '.join(architecture_names) +\n ' (default: resnet50)')\n parser.add_argument('--finetune', action='store_true', help='whether use 10x smaller lr for backbone')\n parser.add_argument('--rate', type=float, default=0.2)\n parser.add_argument('--mix-layers', nargs='+', help='layers to apply MixStyle')\n parser.add_argument('--mix-p', default=0.5, type=float, help='probability to apply MixStyle')\n parser.add_argument('--mix-alpha', default=0.1, type=float, help='parameter alpha for beta distribution')\n # training parameters\n parser.add_argument('--trade-off', type=float, default=1,\n help='trade-off hyper parameter between cross entropy loss and triplet loss')\n parser.add_argument('--margin', type=float, default=0.0, help='margin for the triplet loss with batch hard')\n parser.add_argument('-j', '--workers', type=int, default=4)\n parser.add_argument('-b', '--batch-size', type=int, default=16)\n parser.add_argument('--height', type=int, default=256, help=\"input height\")\n parser.add_argument('--width', type=int, default=128, help=\"input width\")\n parser.add_argument('--num-instances', type=int, default=4,\n help=\"each minibatch consist of \"\n \"(batch_size // num_instances) identities, and \"\n \"each identity has num_instances instances, \"\n \"default: 4\")\n parser.add_argument('--lr', type=float, default=0.00035,\n help=\"learning rate of new parameters, for pretrained \")\n parser.add_argument('--weight-decay', type=float, default=5e-4)\n parser.add_argument('--epochs', type=int, default=80)\n parser.add_argument('--warmup-steps', type=int, default=10, help='number of warm-up steps')\n parser.add_argument('--milestones', nargs='+', type=int, default=[40, 70],\n help='milestones for the learning rate decay')\n parser.add_argument('--eval-step', type=int, default=40)\n parser.add_argument('--iters-per-epoch', type=int, default=400)\n parser.add_argument('--print-freq', type=int, default=40)\n parser.add_argument('--seed', default=None, type=int, help='seed for initializing training.')\n parser.add_argument('--rerank', action='store_true', help=\"evaluation only\")\n parser.add_argument(\"--log\", type=str, default='mixstyle',\n help=\"Where to save logs, checkpoints and debugging images.\")\n parser.add_argument(\"--phase\", type=str, default='train', choices=['train', 'test', 'analysis'],\n help=\"When phase is 'test', only test the model.\"\n \"When phase is 'analysis', only analysis the model.\")\n args = parser.parse_args()\n main(args)\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/examples/domain_generalization/re_identification/mixstyle.sh",
"content": "#!/usr/bin/env bash\n# Market1501 -> Duke\nCUDA_VISIBLE_DEVICES=0 python mixstyle.py data -s Market1501 -t DukeMTMC -a resnet50 \\\n--mix-layers layer1 layer2 --finetune --seed 0 --log logs/mixstyle/Market2Duke\n\n# Duke -> Market1501\nCUDA_VISIBLE_DEVICES=0 python mixstyle.py data -s DukeMTMC -t Market1501 -a resnet50 \\\n--mix-layers layer1 layer2 --finetune --seed 0 --log logs/mixstyle/Duke2Market\n\n# Market1501 -> MSMT\nCUDA_VISIBLE_DEVICES=0 python mixstyle.py data -s Market1501 -t MSMT17 -a resnet50 \\\n--mix-layers layer1 layer2 --finetune --seed 0 --log logs/mixstyle/Market2MSMT\n\n# MSMT -> Market1501\nCUDA_VISIBLE_DEVICES=0 python mixstyle.py data -s MSMT17 -t Market1501 -a resnet50 \\\n--mix-layers layer1 layer2 --finetune --seed 0 --log logs/mixstyle/MSMT2Market\n\n# Duke -> MSMT\nCUDA_VISIBLE_DEVICES=0 python mixstyle.py data -s DukeMTMC -t MSMT17 -a resnet50 \\\n--mix-layers layer1 layer2 --finetune --seed 0 --log logs/mixstyle/Duke2MSMT\n\n# MSMT -> Duke\nCUDA_VISIBLE_DEVICES=0 python mixstyle.py data -s MSMT17 -t DukeMTMC -a resnet50 \\\n--mix-layers layer1 layer2 --finetune --seed 0 --log logs/mixstyle/MSMT2Duke\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/examples/domain_generalization/re_identification/requirements.txt",
"content": "timm\nopencv-python"
},
{
"path": "domain_generalization/Transfer-Learning-Library/examples/domain_generalization/re_identification/utils.py",
"content": "\"\"\"\n@author: Baixu Chen\n@contact: cbx_99_hasta@outlook.com\n\"\"\"\nimport sys\nimport timm\nimport torch\nimport torch.nn as nn\nimport torchvision.transforms as T\n\nsys.path.append('../../..')\nfrom common.utils.metric.reid import extract_reid_feature\nfrom common.utils.analysis import tsne\nimport common.vision.models.reid as models\n\n\ndef get_model_names():\n return sorted(\n name for name in models.__dict__\n if name.islower() and not name.startswith(\"__\")\n and callable(models.__dict__[name])\n ) + timm.list_models()\n\n\ndef get_model(model_name):\n if model_name in models.__dict__:\n # load models from common.vision.models\n backbone = models.__dict__[model_name](pretrained=True)\n else:\n # load models from pytorch-image-models\n backbone = timm.create_model(model_name, pretrained=True)\n try:\n backbone.out_features = backbone.get_classifier().in_features\n backbone.reset_classifier(0, '')\n except:\n backbone.out_features = backbone.head.in_features\n backbone.head = nn.Identity()\n return backbone\n\n\ndef get_train_transform(height, width, resizing='default', random_horizontal_flip=True, random_color_jitter=False,\n random_gray_scale=False):\n \"\"\"\n resizing mode:\n - default: resize the image to (height, width), zero-pad it by 10 on each size, the take a random crop of\n (height, width)\n - res: resize the image to(height, width)\n \"\"\"\n if resizing == 'default':\n transform = T.Compose([\n T.Resize((height, width), interpolation=3),\n T.Pad(10),\n T.RandomCrop((height, width))\n ])\n elif resizing == 'res':\n transform = T.Resize((height, width), interpolation=3)\n else:\n raise NotImplementedError(resizing)\n transforms = [transform]\n if random_horizontal_flip:\n transforms.append(T.RandomHorizontalFlip())\n if random_color_jitter:\n transforms.append(T.ColorJitter(brightness=0.3, contrast=0.3, saturation=0.3, hue=0.3))\n if random_gray_scale:\n transforms.append(T.RandomGrayscale())\n transforms.extend([\n T.ToTensor(),\n T.Normalize(mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225])\n ])\n return T.Compose(transforms)\n\n\ndef get_val_transform(height, width):\n return T.Compose([\n T.Resize((height, width), interpolation=3),\n T.ToTensor(),\n T.Normalize(mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225])\n ])\n\n\ndef visualize_tsne(source_loader, target_loader, model, filename, device, n_data_points_per_domain=3000):\n \"\"\"Visualize features from different domains using t-SNE. As we can have very large number of samples in each\n domain, only `n_data_points_per_domain` number of samples are randomly selected in each domain.\n \"\"\"\n source_feature_dict = extract_reid_feature(source_loader, model, device, normalize=True)\n source_feature = torch.stack(list(source_feature_dict.values())).cpu()\n source_feature = source_feature[torch.randperm(len(source_feature))]\n source_feature = source_feature[:n_data_points_per_domain]\n\n target_feature_dict = extract_reid_feature(target_loader, model, device, normalize=True)\n target_feature = torch.stack(list(target_feature_dict.values())).cpu()\n target_feature = target_feature[torch.randperm(len(target_feature))]\n target_feature = target_feature[:n_data_points_per_domain]\n\n tsne.visualize(source_feature, target_feature, filename, source_color='cornflowerblue', target_color='darkorange')\n print('T-SNE process is done, figure is saved to {}'.format(filename))\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/examples/task_adaptation/image_classification/README.md",
"content": "# Task Adaptation for Image Classification\n\n## Installation\nExample scripts support all models in [PyTorch-Image-Models](https://github.com/rwightman/pytorch-image-models).\nYou need to install timm to use PyTorch-Image-Models.\n\n```\npip install timm\n```\n\n## Dataset\n\nFollowing datasets can be downloaded automatically:\n\n- [CUB200](http://www.vision.caltech.edu/visipedia/CUB-200-2011.html)\n- [StanfordCars](https://ai.stanford.edu/~jkrause/cars/car_dataset.html)\n- [Aircraft](https://www.robots.ox.ac.uk/~vgg/data/fgvc-aircraft/)\n- [StanfordDogs](http://vision.stanford.edu/aditya86/ImageNetDogs/)\n- [OxfordIIITPet](https://www.robots.ox.ac.uk/~vgg/data/pets/)\n- [OxfordFlowers102](https://www.robots.ox.ac.uk/~vgg/data/flowers/102/)\n- [DTD](https://www.robots.ox.ac.uk/~vgg/data/dtd/index.html)\n- [PatchCamelyon](https://patchcamelyon.grand-challenge.org/)\n- [EuroSAT](https://github.com/phelber/eurosat)\n\nYou need to prepare following datasets manually if you want to use them:\n- [Retinopathy](https://www.kaggle.com/c/diabetic-retinopathy-detection/data)\n- [Resisc45](http://www.escience.cn/people/JunweiHan/NWPU-RESISC45.html)\n\nand prepare them following [Documentation for Retinopathy](/common/vision/datasets/retinopathy.py) and [Resisc45](/common/vision/datasets/resisc45.py).\n\n## Supported Methods\n\nSupported methods include:\n\n- [Explicit inductive bias for transfer learning with convolutional networks\n (L2-SP, ICML 2018)](https://arxiv.org/abs/1802.01483)\n- [Catastrophic Forgetting Meets Negative Transfer: Batch Spectral Shrinkage for Safe Transfer Learning (BSS, NIPS 2019)](https://proceedings.neurips.cc/paper/2019/file/c6bff625bdb0393992c9d4db0c6bbe45-Paper.pdf)\n- [DEep Learning Transfer using Fea- ture Map with Attention for convolutional networks (DELTA, ICLR 2019)](https://openreview.net/pdf?id=rkgbwsAcYm)\n- [Co-Tuning for Transfer Learning (Co-Tuning, NIPS 2020)](http://ise.thss.tsinghua.edu.cn/~mlong/doc/co-tuning-for-transfer-learning-nips20.pdf)\n- [Stochastic Normalization (StochNorm, NIPS 2020)](https://papers.nips.cc/paper/2020/file/bc573864331a9e42e4511de6f678aa83-Paper.pdf)\n- [Learning Without Forgetting (LWF, ECCV 2016)](https://arxiv.org/abs/1606.09282)\n- [Bi-tuning of Pre-trained Representations (Bi-Tuning)](https://arxiv.org/abs/2011.06182?utm_source=feedburner&utm_medium=feed&utm_campaign=Feed%3A+arxiv%2FQSXk+%28ExcitingAds%21+cs+updates+on+arXiv.org%29)\n\n## Experiment and Results\n\n### Fine-tune the supervised pre-trained model\n\nThe shell files give the script to reproduce the [supervised pretrained benchmarks](/docs/talib/benchmarks/image_classification.rst) with specified hyper-parameters.\nFor example, if you want to use vanilla fine-tune on CUB200, use the following script\n\n```shell script\n# Fine-tune ResNet50 on CUB200.\n# Assume you have put the datasets under the path `data/cub200`, \n# or you are glad to download the datasets automatically from the Internet to this path\nCUDA_VISIBLE_DEVICES=0 python baseline.py data/cub200 -d CUB200 -sr 100 --seed 0 --finetune --log logs/baseline/cub200_100\n```\n\n\n### Fine-tune the unsupervised pre-trained model\nTake MoCo as an example. \n\n1. Download MoCo pretrained checkpoints from https://github.com/facebookresearch/moco\n2. Convert the format of the MoCo checkpoints to the standard format of pytorch\n```shell\nmkdir checkpoints\npython convert_moco_to_pretrained.py checkpoints/moco_v1_200ep_pretrain.pth.tar checkpoints/moco_v1_200ep_backbone.pth checkpoints/moco_v1_200ep_fc.pth\n```\n3. Start training\n```shell\nCUDA_VISIBLE_DEVICES=0 python bi_tuning.py data/cub200 -d CUB200 -sr 100 --seed 0 --lr 0.1 -i 2000 --lr-decay-epochs 3 6 9 --epochs 12 \\\n --log logs/moco_pretrain_bi_tuning/cub200_100 --pretrained checkpoints/moco_v1_200ep_backbone.pth\n```\nThe shell files als give the script to reproduce the [unsupervised pretrained benchmarks](/docs/talib/benchmarks/image_classification.rst#) with specified hyper-parameters.\n\n\n## Citation\nIf you use these methods in your research, please consider citing.\n\n```\n@inproceedings{LWF,\n author = {Zhizhong Li and\n Derek Hoiem},\n title = {Learning without Forgetting},\n booktitle={ECCV},\n year = {2016},\n}\n\n@inproceedings{L2SP,\n title={Explicit inductive bias for transfer learning with convolutional networks},\n author={Xuhong, LI and Grandvalet, Yves and Davoine, Franck},\n booktitle={ICML},\n year={2018},\n}\n\n@inproceedings{BSS,\n title={Catastrophic forgetting meets negative transfer: Batch spectral shrinkage for safe transfer learning},\n author={Chen, Xinyang and Wang, Sinan and Fu, Bo and Long, Mingsheng and Wang, Jianmin},\n booktitle={NeurIPS},\n year={2019}\n}\n\n@inproceedings{DELTA,\n title={Delta: Deep learning transfer using feature map with attention for convolutional networks},\n author={Li, Xingjian and Xiong, Haoyi and Wang, Hanchao and Rao, Yuxuan and Liu, Liping and Chen, Zeyu and Huan, Jun},\n booktitle={ICLR},\n year={2019}\n}\n\n@inproceedings{StocNorm,\n title={Stochastic Normalization},\n author={Kou, Zhi and You, Kaichao and Long, Mingsheng and Wang, Jianmin},\n booktitle={NeurIPS},\n year={2020}\n}\n\n@inproceedings{CoTuning,\n title={Co-Tuning for Transfer Learning},\n author={You, Kaichao and Kou, Zhi and Long, Mingsheng and Wang, Jianmin},\n booktitle={NeurIPS},\n year={2020}\n}\n\n@article{BiTuning,\n title={Bi-tuning of Pre-trained Representations},\n author={Zhong, Jincheng and Wang, Ximei and Kou, Zhi and Wang, Jianmin and Long, Mingsheng},\n journal={arXiv preprint arXiv:2011.06182},\n year={2020}\n}\n```\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/examples/task_adaptation/image_classification/baseline.py",
"content": "\"\"\"\n@author: Yifei Ji, Junguang Jiang\n@contact: jiyf990330@163.com, JiangJunguang1123@outlook.com\n\"\"\"\nimport random\nimport time\nimport warnings\nimport sys\nimport argparse\nimport shutil\n\nimport torch\nimport torch.nn as nn\nimport torch.backends.cudnn as cudnn\nfrom torch.optim import SGD\nfrom torch.utils.data import DataLoader\nimport torch.nn.functional as F\n\nsys.path.append('../../..')\nfrom common.modules.classifier import Classifier\nfrom common.utils.metric import accuracy\nfrom common.utils.meter import AverageMeter, ProgressMeter\nfrom common.utils.data import ForeverDataIterator\nfrom common.utils.logger import CompleteLogger\n\nsys.path.append('.')\nimport utils\n\ndevice = torch.device(\"cuda\" if torch.cuda.is_available() else \"cpu\")\n\n\ndef main(args: argparse.Namespace):\n logger = CompleteLogger(args.log, args.phase)\n print(args)\n\n if args.seed is not None:\n random.seed(args.seed)\n torch.manual_seed(args.seed)\n cudnn.deterministic = True\n warnings.warn('You have chosen to seed training. '\n 'This will turn on the CUDNN deterministic setting, '\n 'which can slow down your training considerably! '\n 'You may see unexpected behavior when restarting '\n 'from checkpoints.')\n\n cudnn.benchmark = True\n\n # Data loading code\n train_transform = utils.get_train_transform(args.train_resizing, not args.no_hflip, args.color_jitter)\n val_transform = utils.get_val_transform(args.val_resizing)\n print(\"train_transform: \", train_transform)\n print(\"val_transform: \", val_transform)\n\n train_dataset, val_dataset, num_classes = utils.get_dataset(args.data, args.root, train_transform,\n val_transform, args.sample_rate, args.num_samples_per_classes)\n train_loader = DataLoader(train_dataset, batch_size=args.batch_size, shuffle=True,\n num_workers=args.workers, drop_last=True)\n train_iter = ForeverDataIterator(train_loader)\n val_loader = DataLoader(val_dataset, batch_size=args.batch_size, shuffle=False, num_workers=args.workers)\n print(\"training dataset size: {} test dataset size: {}\".format(len(train_dataset), len(val_dataset)))\n\n # create model\n print(\"=> using pre-trained model '{}'\".format(args.arch))\n backbone = utils.get_model(args.arch, args.pretrained)\n pool_layer = nn.Identity() if args.no_pool else None\n classifier = Classifier(backbone, num_classes, pool_layer=pool_layer, finetune=args.finetune).to(device)\n\n # define optimizer and lr scheduler\n optimizer = SGD(classifier.get_parameters(args.lr), lr=args.lr, momentum=args.momentum, weight_decay=args.wd, nesterov=True)\n lr_scheduler = torch.optim.lr_scheduler.MultiStepLR(optimizer, args.lr_decay_epochs, gamma=args.lr_gamma)\n\n # resume from the best checkpoint\n if args.phase == 'test':\n checkpoint = torch.load(logger.get_checkpoint_path('best'), map_location='cpu')\n classifier.load_state_dict(checkpoint)\n acc1 = utils.validate(val_loader, classifier, args, device)\n print(acc1)\n return\n\n # start training\n best_acc1 = 0.0\n for epoch in range(args.epochs):\n logger.set_epoch(epoch)\n print(lr_scheduler.get_lr())\n # train for one epoch\n train(train_iter, classifier, optimizer, epoch, args)\n lr_scheduler.step()\n # evaluate on validation set\n acc1 = utils.validate(val_loader, classifier, args, device)\n\n # remember best acc@1 and save checkpoint\n torch.save(classifier.state_dict(), logger.get_checkpoint_path('latest'))\n if acc1 > best_acc1:\n shutil.copy(logger.get_checkpoint_path('latest'), logger.get_checkpoint_path('best'))\n best_acc1 = max(acc1, best_acc1)\n\n print(\"best_acc1 = {:3.1f}\".format(best_acc1))\n logger.close()\n\n\ndef train(train_iter: ForeverDataIterator, model: Classifier, optimizer: SGD,\n epoch: int, args: argparse.Namespace):\n batch_time = AverageMeter('Time', ':4.2f')\n data_time = AverageMeter('Data', ':3.1f')\n losses = AverageMeter('Loss', ':3.2f')\n cls_accs = AverageMeter('Cls Acc', ':3.1f')\n\n progress = ProgressMeter(\n args.iters_per_epoch,\n [batch_time, data_time, losses, cls_accs],\n prefix=\"Epoch: [{}]\".format(epoch))\n\n # switch to train mode\n model.train()\n\n end = time.time()\n for i in range(args.iters_per_epoch):\n x, labels = next(train_iter)\n\n x = x.to(device)\n label = labels.to(device)\n\n # measure data loading time\n data_time.update(time.time() - end)\n\n # compute output\n y, f = model(x)\n cls_loss = F.cross_entropy(y, label)\n loss = cls_loss\n\n # measure accuracy and record loss\n losses.update(loss.item(), x.size(0))\n cls_acc = accuracy(y, label)[0]\n cls_accs.update(cls_acc.item(), x.size(0))\n\n # compute gradient and do SGD step\n optimizer.zero_grad()\n loss.backward()\n optimizer.step()\n\n # measure elapsed time\n batch_time.update(time.time() - end)\n end = time.time()\n\n if i % args.print_freq == 0:\n progress.display(i)\n\n\nif __name__ == '__main__':\n parser = argparse.ArgumentParser(description='Baseline for Finetuning')\n # dataset parameters\n parser.add_argument('root', metavar='DIR',\n help='root path of dataset')\n parser.add_argument('-d', '--data', metavar='DATA')\n parser.add_argument('-sr', '--sample-rate', default=100, type=float,\n metavar='N',\n help='sample rate of training dataset (default: 100)')\n parser.add_argument('-sc', '--num-samples-per-classes', default=None, type=int,\n help='number of samples per classes.')\n parser.add_argument('--train-resizing', type=str, default='default')\n parser.add_argument('--val-resizing', type=str, default='default')\n parser.add_argument('--no-hflip', action='store_true', help='no random horizontal flipping during training')\n parser.add_argument('--color-jitter', action='store_true')\n # model parameters\n parser.add_argument('-a', '--arch', metavar='ARCH', default='resnet50',\n choices=utils.get_model_names(),\n help='backbone architecture: ' +\n ' | '.join(utils.get_model_names()) +\n ' (default: resnet50)')\n parser.add_argument('--no-pool', action='store_true',\n help='no pool layer after the feature extractor.')\n parser.add_argument('--finetune', action='store_true', help='whether use 10x smaller lr for backbone')\n parser.add_argument('--pretrained', default=None,\n help=\"pretrained checkpoint of the backbone. \"\n \"(default: None, use the ImageNet supervised pretrained backbone)\")\n # training parameters\n parser.add_argument('-b', '--batch-size', default=48, type=int,\n metavar='N',\n help='mini-batch size (default: 48)')\n parser.add_argument('--optimizer', type=str, default='SGD', choices=['SGD', 'Adam'])\n parser.add_argument('--lr', '--learning-rate', default=0.01, type=float,\n metavar='LR', help='initial learning rate', dest='lr')\n parser.add_argument('--lr-gamma', default=0.1, type=float, help='parameter for lr scheduler')\n parser.add_argument('--lr-decay-epochs', type=int, default=(12, ), nargs='+', help='epochs to decay lr')\n parser.add_argument('--momentum', default=0.9, type=float, metavar='M',\n help='momentum')\n parser.add_argument('--wd', '--weight-decay', default=0.0005, type=float,\n metavar='W', help='weight decay (default: 5e-4)')\n parser.add_argument('-j', '--workers', default=2, type=int, metavar='N',\n help='number of data loading workers (default: 4)')\n parser.add_argument('--epochs', default=20, type=int, metavar='N',\n help='number of total epochs to run')\n parser.add_argument('-i', '--iters-per-epoch', default=500, type=int,\n help='Number of iterations per epoch')\n parser.add_argument('-p', '--print-freq', default=100, type=int,\n metavar='N', help='print frequency (default: 100)')\n parser.add_argument('--seed', default=None, type=int,\n help='seed for initializing training. ')\n parser.add_argument(\"--log\", type=str, default='baseline',\n help=\"Where to save logs, checkpoints and debugging images.\")\n parser.add_argument(\"--phase\", type=str, default='train', choices=['train', 'test'],\n help=\"When phase is 'test', only test the model.\")\n args = parser.parse_args()\n main(args)\n\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/examples/task_adaptation/image_classification/baseline.sh",
"content": "#!/usr/bin/env bash\n# Supervised Pretraining\n# CUB-200-2011\n CUDA_VISIBLE_DEVICES=0 python baseline.py data/cub200 -d CUB200 -sr 100 --seed 0 --finetune --log logs/baseline/cub200_100\n CUDA_VISIBLE_DEVICES=0 python baseline.py data/cub200 -d CUB200 -sr 50 --seed 0 --finetune --log logs/baseline/cub200_50\n CUDA_VISIBLE_DEVICES=0 python baseline.py data/cub200 -d CUB200 -sr 30 --seed 0 --finetune --log logs/baseline/cub200_30\n CUDA_VISIBLE_DEVICES=0 python baseline.py data/cub200 -d CUB200 -sr 15 --seed 0 --finetune --log logs/baseline/cub200_15\n\n# Standford Cars\n CUDA_VISIBLE_DEVICES=0 python baseline.py data/stanford_cars -d StanfordCars -sr 100 --seed 0 --finetune --log logs/baseline/car_100\n CUDA_VISIBLE_DEVICES=0 python baseline.py data/stanford_cars -d StanfordCars -sr 50 --seed 0 --finetune --log logs/baseline/car_50\n CUDA_VISIBLE_DEVICES=0 python baseline.py data/stanford_cars -d StanfordCars -sr 30 --seed 0 --finetune --log logs/baseline/car_30\n CUDA_VISIBLE_DEVICES=0 python baseline.py data/stanford_cars -d StanfordCars -sr 15 --seed 0 --finetune --log logs/baseline/car_15\n\n# Aircrafts\n CUDA_VISIBLE_DEVICES=0 python baseline.py data/aircraft -d Aircraft -sr 100 --seed 0 --finetune --log logs/baseline/aircraft_100\n CUDA_VISIBLE_DEVICES=0 python baseline.py data/aircraft -d Aircraft -sr 50 --seed 0 --finetune --log logs/baseline/aircraft_50\n CUDA_VISIBLE_DEVICES=0 python baseline.py data/aircraft -d Aircraft -sr 30 --seed 0 --finetune --log logs/baseline/aircraft_30\n CUDA_VISIBLE_DEVICES=0 python baseline.py data/aircraft -d Aircraft -sr 15 --seed 0 --finetune --log logs/baseline/aircraft_15\n\n# Resisc45\nCUDA_VISIBLE_DEVICES=0 python baseline.py data/vtab/resisc45 -d Resisc45 -sc 10 --finetune --seed 0 \\\n --log logs/baseline/resisc45_10\nCUDA_VISIBLE_DEVICES=0 python baseline.py data/vtab/resisc45 -d Resisc45 -sc 20 --finetune --seed 0 \\\n --log logs/baseline/resisc45_20\nCUDA_VISIBLE_DEVICES=0 python baseline.py data/vtab/resisc45 -d Resisc45 -sc 40 --finetune --seed 0 \\\n --log logs/baseline/resisc45_40\nCUDA_VISIBLE_DEVICES=0 python baseline.py data/vtab/resisc45 -d Resisc45 -sc 80 --finetune --seed 0 \\\n --log logs/baseline/resisc45_80\n\n# Patch Camelyon\nCUDA_VISIBLE_DEVICES=0 python baseline.py data/vtab/patch_camelyon -d PatchCamelyon -sc 40 --finetune \\\n --seed 0 --log logs/baseline/patch_camelyon_40\nCUDA_VISIBLE_DEVICES=0 python baseline.py data/vtab/patch_camelyon -d PatchCamelyon -sc 80 --finetune \\\n --seed 0 --log logs/baseline/patch_camelyon_80\nCUDA_VISIBLE_DEVICES=0 python baseline.py data/vtab/patch_camelyon -d PatchCamelyon -sc 160 --finetune \\\n --seed 0 --log logs/baseline/patch_camelyon_160\nCUDA_VISIBLE_DEVICES=0 python baseline.py data/vtab/patch_camelyon -d PatchCamelyon -sc 320 --finetune \\\n --seed 0 --log logs/baseline/patch_camelyon_320\n\n# MoCo (Unsupervised Pretraining)\n#CUB-200-2011\n CUDA_VISIBLE_DEVICES=0 python baseline.py data/cub200 -d CUB200 -sr 100 --seed 0 --lr 0.1 --finetune -i 2000 --lr-decay-epochs 3 6 9 --epochs 12 \\\n --log logs/moco_pretrain_baseline/cub200_100 --pretrained checkpoints/moco_v1_200ep_backbone.pth\n CUDA_VISIBLE_DEVICES=0 python baseline.py data/cub200 -d CUB200 -sr 50 --seed 0 --lr 0.1 --finetune -i 2000 --lr-decay-epochs 3 6 9 --epochs 12 \\\n --log logs/moco_pretrain_baseline/cub200_50 --pretrained checkpoints/moco_v1_200ep_backbone.pth\n CUDA_VISIBLE_DEVICES=0 python baseline.py data/cub200 -d CUB200 -sr 30 --seed 0 --lr 0.1 --finetune -i 2000 --lr-decay-epochs 3 6 9 --epochs 12 \\\n --log logs/moco_pretrain_baseline/cub200_30 --pretrained checkpoints/moco_v1_200ep_backbone.pth\n CUDA_VISIBLE_DEVICES=0 python baseline.py data/cub200 -d CUB200 -sr 15 --seed 0 --lr 0.1 --finetune -i 2000 --lr-decay-epochs 3 6 9 --epochs 12 \\\n --log logs/moco_pretrain_baseline/cub200_15 --pretrained checkpoints/moco_v1_200ep_backbone.pth\n\n# Standford Cars\n CUDA_VISIBLE_DEVICES=0 python baseline.py data/stanford_cars -d StanfordCars -sr 100 --seed 0 --lr 0.1 --finetune -i 2000 --lr-decay-epochs 3 6 9 --epochs 12 \\\n --log logs/moco_pretrain_baseline/cars_100 --pretrained checkpoints/moco_v1_200ep_backbone.pth\n CUDA_VISIBLE_DEVICES=0 python baseline.py data/stanford_cars -d StanfordCars -sr 50 --seed 0 --lr 0.1 --finetune -i 2000 --lr-decay-epochs 3 6 9 --epochs 12 \\\n --log logs/moco_pretrain_baseline/cars_50 --pretrained checkpoints/moco_v1_200ep_backbone.pth\n CUDA_VISIBLE_DEVICES=0 python baseline.py data/stanford_cars -d StanfordCars -sr 30 --seed 0 --lr 0.1 --finetune -i 2000 --lr-decay-epochs 3 6 9 --epochs 12 \\\n --log logs/moco_pretrain_baseline/cars_30 --pretrained checkpoints/moco_v1_200ep_backbone.pth\n CUDA_VISIBLE_DEVICES=0 python baseline.py data/stanford_cars -d StanfordCars -sr 15 --seed 0 --lr 0.1 --finetune -i 2000 --lr-decay-epochs 3 6 9 --epochs 12 \\\n --log logs/moco_pretrain_baseline/cars_15 --pretrained checkpoints/moco_v1_200ep_backbone.pth\n\n# Aircrafts\n CUDA_VISIBLE_DEVICES=0 python baseline.py data/aircraft -d Aircraft -sr 100 --seed 0 --lr 0.1 --finetune -i 2000 --lr-decay-epochs 3 6 9 --epochs 12 \\\n --log logs/moco_pretrain_baseline/aircraft_100 --pretrained checkpoints/moco_v1_200ep_backbone.pth\n CUDA_VISIBLE_DEVICES=0 python baseline.py data/aircraft -d Aircraft -sr 50 --seed 0 --lr 0.1 --finetune -i 2000 --lr-decay-epochs 3 6 9 --epochs 12 \\\n --log logs/moco_pretrain_baseline/aircraft_50 --pretrained checkpoints/moco_v1_200ep_backbone.pth\n CUDA_VISIBLE_DEVICES=0 python baseline.py data/aircraft -d Aircraft -sr 30 --seed 0 --lr 0.1 --finetune -i 2000 --lr-decay-epochs 3 6 9 --epochs 12 \\\n --log logs/moco_pretrain_baseline/aircraft_30 --pretrained checkpoints/moco_v1_200ep_backbone.pth\n CUDA_VISIBLE_DEVICES=0 python baseline.py data/aircraft -d Aircraft -sr 15 --seed 0 --lr 0.1 --finetune -i 2000 --lr-decay-epochs 3 6 9 --epochs 12 \\\n --log logs/moco_pretrain_baseline/aircraft_15 --pretrained checkpoints/moco_v1_200ep_backbone.pth\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/examples/task_adaptation/image_classification/bi_tuning.py",
"content": "\"\"\"\n@author: Junguang Jiang\n@contact: JiangJunguang1123@outlook.com\n\"\"\"\nimport random\nimport time\nimport warnings\nimport sys\nimport argparse\nimport shutil\n\nimport torch\nimport torch.nn as nn\nimport torch.backends.cudnn as cudnn\nfrom torch.optim import SGD\nfrom torch.utils.data import DataLoader\n\nsys.path.append('../../..')\nfrom common.vision.transforms import MultipleApply\nfrom common.utils.metric import accuracy\nfrom common.utils.meter import AverageMeter, ProgressMeter\nfrom common.utils.data import ForeverDataIterator\nfrom common.utils.logger import CompleteLogger\nfrom talib.finetune.bi_tuning import Classifier, BiTuning\n\nsys.path.append('.')\nimport utils\n\ndevice = torch.device(\"cuda\" if torch.cuda.is_available() else \"cpu\")\n\n\ndef main(args: argparse.Namespace):\n logger = CompleteLogger(args.log, args.phase)\n print(args)\n\n if args.seed is not None:\n random.seed(args.seed)\n torch.manual_seed(args.seed)\n cudnn.deterministic = True\n warnings.warn('You have chosen to seed training. '\n 'This will turn on the CUDNN deterministic setting, '\n 'which can slow down your training considerably! '\n 'You may see unexpected behavior when restarting '\n 'from checkpoints.')\n\n cudnn.benchmark = True\n\n # Data loading code\n train_augmentation = utils.get_train_transform(args.train_resizing, not args.no_hflip, args.color_jitter)\n val_transform = utils.get_val_transform(args.val_resizing)\n train_transform = MultipleApply([train_augmentation, train_augmentation])\n print(\"train_transform: \", train_transform)\n print(\"val_transform: \", val_transform)\n\n train_dataset, val_dataset, num_classes = utils.get_dataset(args.data, args.root, train_transform,\n val_transform, args.sample_rate,\n args.num_samples_per_classes)\n train_loader = DataLoader(train_dataset, batch_size=args.batch_size, shuffle=True,\n num_workers=args.workers, drop_last=True)\n train_iter = ForeverDataIterator(train_loader)\n val_loader = DataLoader(val_dataset, batch_size=args.batch_size, shuffle=False, num_workers=args.workers)\n print(\"training dataset size: {} test dataset size: {}\".format(len(train_dataset), len(val_dataset)))\n\n # create model\n print(\"=> using pre-trained model '{}'\".format(args.arch))\n backbone_q = utils.get_model(args.arch, args.pretrained)\n pool_layer = nn.Identity() if args.no_pool else None\n classifier_q = Classifier(backbone_q, num_classes, pool_layer=pool_layer, projection_dim=args.projection_dim,\n finetune=args.finetune)\n if args.pretrained_fc:\n print(\"=> loading pre-trained fc from '{}'\".format(args.pretrained_fc))\n pretrained_fc_dict = torch.load(args.pretrained_fc)\n classifier_q.projector.load_state_dict(pretrained_fc_dict, strict=False)\n classifier_q = classifier_q.to(device)\n backbone_k = utils.get_model(args.arch)\n classifier_k = Classifier(backbone_k, num_classes, pool_layer=pool_layer).to(device)\n\n bituning = BiTuning(classifier_q, classifier_k, num_classes, K=args.K, m=args.m, T=args.T)\n\n # define optimizer and lr scheduler\n optimizer = SGD(classifier_q.get_parameters(args.lr), lr=args.lr, momentum=args.momentum, weight_decay=args.wd,\n nesterov=True)\n lr_scheduler = torch.optim.lr_scheduler.MultiStepLR(optimizer, args.lr_decay_epochs, gamma=args.lr_gamma)\n\n # resume from the best checkpoint\n if args.phase == 'test':\n checkpoint = torch.load(logger.get_checkpoint_path('best'), map_location='cpu')\n classifier_q.load_state_dict(checkpoint)\n acc1 = utils.validate(val_loader, classifier_q, args, device)\n print(acc1)\n return\n\n # start training\n best_acc1 = 0.0\n for epoch in range(args.epochs):\n print(lr_scheduler.get_lr())\n # train for one epoch\n train(train_iter, bituning, optimizer, epoch, args)\n lr_scheduler.step()\n\n # evaluate on validation set\n acc1 = utils.validate(val_loader, classifier_q, args, device)\n\n # remember best acc@1 and save checkpoint\n torch.save(classifier_q.state_dict(), logger.get_checkpoint_path('latest'))\n if acc1 > best_acc1:\n shutil.copy(logger.get_checkpoint_path('latest'), logger.get_checkpoint_path('best'))\n best_acc1 = max(acc1, best_acc1)\n\n print(\"best_acc1 = {:3.1f}\".format(best_acc1))\n logger.close()\n\n\ndef train(train_iter: ForeverDataIterator, bituning: BiTuning, optimizer: SGD, epoch: int, args: argparse.Namespace):\n batch_time = AverageMeter('Time', ':4.2f')\n data_time = AverageMeter('Data', ':3.1f')\n cls_losses = AverageMeter('Cls Loss', ':3.2f')\n contrastive_losses = AverageMeter('Contrastive Loss', ':3.2f')\n losses = AverageMeter('Loss', ':3.2f')\n cls_accs = AverageMeter('Cls Acc', ':3.1f')\n\n progress = ProgressMeter(\n args.iters_per_epoch,\n [batch_time, data_time, losses, cls_losses, contrastive_losses, cls_accs],\n prefix=\"Epoch: [{}]\".format(epoch))\n\n classifier_criterion = torch.nn.CrossEntropyLoss().to(device)\n contrastive_criterion = torch.nn.KLDivLoss(reduction='batchmean').to(device)\n\n # switch to train mode\n bituning.train()\n\n end = time.time()\n for i in range(args.iters_per_epoch):\n x, labels = next(train_iter)\n img_q, img_k = x[0], x[1]\n\n img_q = img_q.to(device)\n img_k = img_k.to(device)\n labels = labels.to(device)\n\n # measure data loading time\n data_time.update(time.time() - end)\n\n # compute output\n y, logits_z, logits_y, bituning_labels = bituning(img_q, img_k, labels)\n cls_loss = classifier_criterion(y, labels)\n contrastive_loss_z = contrastive_criterion(logits_z, bituning_labels)\n contrastive_loss_y = contrastive_criterion(logits_y, bituning_labels)\n contrastive_loss = (contrastive_loss_z + contrastive_loss_y)\n loss = cls_loss + contrastive_loss * args.trade_off\n\n # measure accuracy and record loss\n losses.update(loss.item(), x[0].size(0))\n cls_losses.update(cls_loss.item(), x[0].size(0))\n contrastive_losses.update(contrastive_loss.item(), x[0].size(0))\n\n cls_acc = accuracy(y, labels)[0]\n cls_accs.update(cls_acc.item(), x[0].size(0))\n\n # compute gradient and do SGD step\n optimizer.zero_grad()\n loss.backward()\n optimizer.step()\n\n # measure elapsed time\n batch_time.update(time.time() - end)\n end = time.time()\n\n if i % args.print_freq == 0:\n progress.display(i)\n\n\nif __name__ == '__main__':\n parser = argparse.ArgumentParser(description='Bi-tuning for Finetuning')\n # dataset parameters\n parser.add_argument('root', metavar='DIR',\n help='root path of dataset')\n parser.add_argument('-d', '--data', metavar='DATA')\n parser.add_argument('-sr', '--sample-rate', default=100, type=int,\n metavar='N',\n help='sample rate of training dataset (default: 100)')\n parser.add_argument('-sc', '--num-samples-per-classes', default=None, type=int,\n help='number of samples per classes.')\n parser.add_argument('--train-resizing', type=str, default='default', help='resize mode during training')\n parser.add_argument('--val-resizing', type=str, default='default', help='resize mode during validation')\n parser.add_argument('--no-hflip', action='store_true', help='no random horizontal flipping during training')\n parser.add_argument('--color-jitter', action='store_true', help='no color jitter during training')\n # model parameters\n parser.add_argument('-a', '--arch', metavar='ARCH', default='resnet50',\n choices=utils.get_model_names(),\n help='backbone architecture: ' +\n ' | '.join(utils.get_model_names()) +\n ' (default: resnet50)')\n parser.add_argument('--no-pool', action='store_true',\n help='no pool layer after the feature extractor. Used in models such as ViT.')\n parser.add_argument('--finetune', action='store_true', help='whether use 10x smaller lr for backbone')\n parser.add_argument('--pretrained', default=None,\n help=\"pretrained checkpoint of the backbone. \"\n \"(default: None, use the ImageNet supervised pretrained backbone)\")\n parser.add_argument('--pretrained-fc', default=None,\n help=\"pretrained checkpoint of the fc. \"\n \"(default: None)\")\n parser.add_argument('--T', default=0.07, type=float, help=\"temperature. (default: 0.07)\")\n parser.add_argument('--K', type=int, default=40, help=\"queue size. (default: 40)\")\n parser.add_argument('--m', type=float, default=0.999, help=\"momentum coefficient. (default: 0.999)\")\n parser.add_argument('--projection-dim', type=int, default=128,\n help=\"dimension of the projection head. (default: 128)\")\n parser.add_argument('--trade-off', type=float, default=1.0, help=\"trade-off parameters. (default: 1.0)\")\n # training parameters\n parser.add_argument('-b', '--batch-size', default=48, type=int,\n metavar='N',\n help='mini-batch size (default: 48)')\n parser.add_argument('--optimizer', type=str, default='SGD', choices=['SGD', 'Adam'])\n parser.add_argument('--lr', '--learning-rate', default=0.01, type=float,\n metavar='LR', help='initial learning rate', dest='lr')\n parser.add_argument('--lr-gamma', default=0.1, type=float, help='parameter for lr scheduler')\n parser.add_argument('--lr-decay-epochs', type=int, default=(12,), nargs='+', help='epochs to decay lr')\n parser.add_argument('--momentum', default=0.9, type=float, metavar='M',\n help='momentum')\n parser.add_argument('--wd', '--weight-decay', default=0.0005, type=float,\n metavar='W', help='weight decay (default: 5e-4)')\n parser.add_argument('-j', '--workers', default=2, type=int, metavar='N',\n help='number of data loading workers (default: 4)')\n parser.add_argument('--epochs', default=20, type=int, metavar='N',\n help='number of total epochs to run')\n parser.add_argument('-i', '--iters-per-epoch', default=500, type=int,\n help='Number of iterations per epoch')\n parser.add_argument('-p', '--print-freq', default=100, type=int,\n metavar='N', help='print frequency (default: 100)')\n parser.add_argument('--seed', default=None, type=int,\n help='seed for initializing training. ')\n parser.add_argument(\"--log\", type=str, default='bi_tuning',\n help=\"Where to save logs, checkpoints and debugging images.\")\n parser.add_argument(\"--phase\", type=str, default='train', choices=['train', 'test'],\n help=\"When phase is 'test', only test the model.\")\n args = parser.parse_args()\n main(args)\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/examples/task_adaptation/image_classification/bi_tuning.sh",
"content": "#!/usr/bin/env bash\n# Supervised Pretraining\n# CUB-200-2011\nCUDA_VISIBLE_DEVICES=0 python bi_tuning.py data/cub200 -d CUB200 -sr 100 --seed 0 --log logs/bi_tuning/cub200_100\nCUDA_VISIBLE_DEVICES=0 python bi_tuning.py data/cub200 -d CUB200 -sr 50 --seed 0 --log logs/bi_tuning/cub200_50\nCUDA_VISIBLE_DEVICES=0 python bi_tuning.py data/cub200 -d CUB200 -sr 30 --seed 0 --log logs/bi_tuning/cub200_30\nCUDA_VISIBLE_DEVICES=0 python bi_tuning.py data/cub200 -d CUB200 -sr 15 --seed 0 --log logs/bi_tuning/cub200_15\n\n# Standford Cars\nCUDA_VISIBLE_DEVICES=0 python bi_tuning.py data/stanford_cars -d StanfordCars -sr 100 --seed 0 --log logs/bi_tuning/car_100\nCUDA_VISIBLE_DEVICES=0 python bi_tuning.py data/stanford_cars -d StanfordCars -sr 50 --seed 0 --log logs/bi_tuning/car_50\nCUDA_VISIBLE_DEVICES=0 python bi_tuning.py data/stanford_cars -d StanfordCars -sr 30 --seed 0 --log logs/bi_tuning/car_30\nCUDA_VISIBLE_DEVICES=0 python bi_tuning.py data/stanford_cars -d StanfordCars -sr 15 --seed 0 --log logs/bi_tuning/car_15\n\n# Aircrafts\nCUDA_VISIBLE_DEVICES=0 python bi_tuning.py data/aircraft -d Aircraft -sr 100 --seed 0 --log logs/bi_tuning/aircraft_100\nCUDA_VISIBLE_DEVICES=0 python bi_tuning.py data/aircraft -d Aircraft -sr 50 --seed 0 --log logs/bi_tuning/aircraft_50\nCUDA_VISIBLE_DEVICES=0 python bi_tuning.py data/aircraft -d Aircraft -sr 30 --seed 0 --log logs/bi_tuning/aircraft_30\nCUDA_VISIBLE_DEVICES=0 python bi_tuning.py data/aircraft -d Aircraft -sr 15 --seed 0 --log logs/bi_tuning/aircraft_15\n\n# Resisc45\nCUDA_VISIBLE_DEVICES=0 python bi_tuning.py data/vtab/resisc45 -d Resisc45 -sc 10 --finetune --seed 0 \\\n --log logs/bi_tuning/resisc45_10\nCUDA_VISIBLE_DEVICES=0 python bi_tuning.py data/vtab/resisc45 -d Resisc45 -sc 20 --finetune --seed 0 \\\n --log logs/bi_tuning/resisc45_20\nCUDA_VISIBLE_DEVICES=0 python bi_tuning.py data/vtab/resisc45 -d Resisc45 -sc 40 --finetune --seed 0 \\\n --log logs/bi_tuning/resisc45_40\nCUDA_VISIBLE_DEVICES=0 python bi_tuning.py data/vtab/resisc45 -d Resisc45 -sc 80 --finetune --seed 0 \\\n --log logs/bi_tuning/resisc45_80\n\n# Patch Camelyon\nCUDA_VISIBLE_DEVICES=0 python bi_tuning.py data/vtab/patch_camelyon -d PatchCamelyon -sc 40 --finetune \\\n --seed 0 --log logs/bi_tuning/patch_camelyon_40\nCUDA_VISIBLE_DEVICES=0 python bi_tuning.py data/vtab/patch_camelyon -d PatchCamelyon -sc 80 --finetune \\\n --seed 0 --log logs/bi_tuning/patch_camelyon_80\nCUDA_VISIBLE_DEVICES=0 python bi_tuning.py data/vtab/patch_camelyon -d PatchCamelyon -sc 160 --finetune \\\n --seed 0 --log logs/bi_tuning/patch_camelyon_160\nCUDA_VISIBLE_DEVICES=0 python bi_tuning.py data/vtab/patch_camelyon -d PatchCamelyon -sc 320 --finetune \\\n --seed 0 --log logs/bi_tuning/patch_camelyon_320\n\n\n# MoCo (Unsupervised Pretraining)\n# CUB-200-2011\nCUDA_VISIBLE_DEVICES=0 python bi_tuning.py data/cub200 -d CUB200 -sr 100 --seed 0 --lr 0.1 -i 2000 --lr-decay-epochs 3 6 9 --epochs 12 \\\n --log logs/moco_pretrain_bi_tuning/cub200_100 --pretrained checkpoints/moco_v1_200ep_backbone.pth\nCUDA_VISIBLE_DEVICES=0 python bi_tuning.py data/cub200 -d CUB200 -sr 50 --seed 0 --lr 0.1 -i 2000 --lr-decay-epochs 3 6 9 --epochs 12 \\\n --log logs/moco_pretrain_bi_tuning/cub200_50 --pretrained checkpoints/moco_v1_200ep_backbone.pth\nCUDA_VISIBLE_DEVICES=0 python bi_tuning.py data/cub200 -d CUB200 -sr 30 --seed 0 --lr 0.1 -i 2000 --lr-decay-epochs 3 6 9 --epochs 12 \\\n --log logs/moco_pretrain_bi_tuning/cub200_30 --pretrained checkpoints/moco_v1_200ep_backbone.pth\nCUDA_VISIBLE_DEVICES=0 python bi_tuning.py data/cub200 -d CUB200 -sr 15 --seed 0 --lr 0.1 -i 2000 --lr-decay-epochs 3 6 9 --epochs 12 \\\n --log logs/moco_pretrain_bi_tuning/cub200_15 --pretrained checkpoints/moco_v1_200ep_backbone.pth\n\n# Standford Cars\nCUDA_VISIBLE_DEVICES=0 python bi_tuning.py data/stanford_cars -d StanfordCars -sr 100 --seed 0 --lr 0.1 -i 2000 --lr-decay-epochs 3 6 9 --epochs 12 \\\n --log logs/moco_pretrain_bi_tuning/cars_100 --pretrained checkpoints/moco_v1_200ep_backbone.pth\nCUDA_VISIBLE_DEVICES=0 python bi_tuning.py data/stanford_cars -d StanfordCars -sr 50 --seed 0 --lr 0.1 -i 2000 --lr-decay-epochs 3 6 9 --epochs 12 \\\n --log logs/moco_pretrain_bi_tuning/cars_50 --pretrained checkpoints/moco_v1_200ep_backbone.pth\nCUDA_VISIBLE_DEVICES=0 python bi_tuning.py data/stanford_cars -d StanfordCars -sr 30 --seed 0 --lr 0.1 -i 2000 --lr-decay-epochs 3 6 9 --epochs 12 \\\n --log logs/moco_pretrain_bi_tuning/cars_30 --pretrained checkpoints/moco_v1_200ep_backbone.pth\nCUDA_VISIBLE_DEVICES=0 python bi_tuning.py data/stanford_cars -d StanfordCars -sr 15 --seed 0 --lr 0.1 -i 2000 --lr-decay-epochs 3 6 9 --epochs 12 \\\n --log logs/moco_pretrain_bi_tuning/cars_15 --pretrained checkpoints/moco_v1_200ep_backbone.pth\n\n# Aircrafts\nCUDA_VISIBLE_DEVICES=0 python bi_tuning.py data/aircraft -d Aircraft -sr 100 --seed 0 --lr 0.1 -i 2000 --lr-decay-epochs 3 6 9 --epochs 12 \\\n --log logs/moco_pretrain_bi_tuning/aircraft_100 --pretrained checkpoints/moco_v1_200ep_backbone.pth\nCUDA_VISIBLE_DEVICES=0 python bi_tuning.py data/aircraft -d Aircraft -sr 50 --seed 0 --lr 0.1 -i 2000 --lr-decay-epochs 3 6 9 --epochs 12 \\\n --log logs/moco_pretrain_bi_tuning/aircraft_50 --pretrained checkpoints/moco_v1_200ep_backbone.pth\nCUDA_VISIBLE_DEVICES=0 python bi_tuning.py data/aircraft -d Aircraft -sr 30 --seed 0 --lr 0.1 -i 2000 --lr-decay-epochs 3 6 9 --epochs 12 \\\n --log logs/moco_pretrain_bi_tuning/aircraft_30 --pretrained checkpoints/moco_v1_200ep_backbone.pth\nCUDA_VISIBLE_DEVICES=0 python bi_tuning.py data/aircraft -d Aircraft -sr 15 --seed 0 --lr 0.1 -i 2000 --lr-decay-epochs 3 6 9 --epochs 12 \\\n --log logs/moco_pretrain_bi_tuning/aircraft_15 --pretrained checkpoints/moco_v1_200ep_backbone.pth"
},
{
"path": "domain_generalization/Transfer-Learning-Library/examples/task_adaptation/image_classification/bss.py",
"content": "\"\"\"\n@author: Yifei Ji, Junguang Jiang\n@contact: jiyf990330@163.com, JiangJunguang1123@outlook.com\n\"\"\"\nimport random\nimport time\nimport warnings\nimport sys\nimport argparse\nimport shutil\n\nimport torch\nimport torch.nn as nn\nimport torch.backends.cudnn as cudnn\nfrom torch.optim import SGD\nfrom torch.utils.data import DataLoader\nimport torch.nn.functional as F\n\nsys.path.append('../../..')\nfrom talib.finetune.bss import BatchSpectralShrinkage\nfrom common.modules.classifier import Classifier\nfrom common.utils.metric import accuracy\nfrom common.utils.meter import AverageMeter, ProgressMeter\nfrom common.utils.data import ForeverDataIterator\nfrom common.utils.logger import CompleteLogger\n\nsys.path.append('.')\nimport utils\n\ndevice = torch.device(\"cuda\" if torch.cuda.is_available() else \"cpu\")\n\n\ndef main(args: argparse.Namespace):\n logger = CompleteLogger(args.log, args.phase)\n print(args)\n\n if args.seed is not None:\n random.seed(args.seed)\n torch.manual_seed(args.seed)\n cudnn.deterministic = True\n warnings.warn('You have chosen to seed training. '\n 'This will turn on the CUDNN deterministic setting, '\n 'which can slow down your training considerably! '\n 'You may see unexpected behavior when restarting '\n 'from checkpoints.')\n\n cudnn.benchmark = True\n\n # Data loading code\n train_transform = utils.get_train_transform(args.train_resizing, not args.no_hflip, args.color_jitter)\n val_transform = utils.get_val_transform(args.val_resizing)\n print(\"train_transform: \", train_transform)\n print(\"val_transform: \", val_transform)\n\n train_dataset, val_dataset, num_classes = utils.get_dataset(args.data, args.root, train_transform,\n val_transform, args.sample_rate, args.num_samples_per_classes)\n train_loader = DataLoader(train_dataset, batch_size=args.batch_size, shuffle=True,\n num_workers=args.workers, drop_last=True)\n train_iter = ForeverDataIterator(train_loader)\n val_loader = DataLoader(val_dataset, batch_size=args.batch_size, shuffle=False, num_workers=args.workers)\n print(\"training dataset size: {} test dataset size: {}\".format(len(train_dataset), len(val_dataset)))\n\n # create model\n print(\"=> using pre-trained model '{}'\".format(args.arch))\n backbone = utils.get_model(args.arch, args.pretrained)\n pool_layer = nn.Identity() if args.no_pool else None\n classifier = Classifier(backbone, num_classes, pool_layer=pool_layer, finetune=args.finetune).to(device)\n bss_module = BatchSpectralShrinkage(k=args.k)\n\n # define optimizer and lr scheduler\n optimizer = SGD(classifier.get_parameters(args.lr), momentum=args.momentum, weight_decay=args.wd, nesterov=True)\n lr_scheduler = torch.optim.lr_scheduler.MultiStepLR(optimizer, args.lr_decay_epochs, gamma=args.lr_gamma)\n\n # resume from the best checkpoint\n if args.phase == 'test':\n checkpoint = torch.load(logger.get_checkpoint_path('best'), map_location='cpu')\n classifier.load_state_dict(checkpoint)\n acc1 = utils.validate(val_loader, classifier, args, device)\n print(acc1)\n return\n\n # start training\n best_acc1 = 0.0\n for epoch in range(args.epochs):\n print(lr_scheduler.get_lr())\n # train for one epoch\n train(train_iter, classifier, bss_module, optimizer, epoch, args)\n lr_scheduler.step()\n # evaluate on validation set\n acc1 = utils.validate(val_loader, classifier, args, device)\n\n # remember best acc@1 and save checkpoint\n torch.save(classifier.state_dict(), logger.get_checkpoint_path('latest'))\n if acc1 > best_acc1:\n shutil.copy(logger.get_checkpoint_path('latest'), logger.get_checkpoint_path('best'))\n best_acc1 = max(acc1, best_acc1)\n\n print(\"best_acc1 = {:3.1f}\".format(best_acc1))\n logger.close()\n\n\ndef train(train_iter: ForeverDataIterator, model: Classifier, bss_module, optimizer: SGD,\n epoch: int, args: argparse.Namespace):\n batch_time = AverageMeter('Time', ':4.2f')\n data_time = AverageMeter('Data', ':3.1f')\n losses = AverageMeter('Loss', ':3.2f')\n cls_accs = AverageMeter('Cls Acc', ':3.1f')\n\n progress = ProgressMeter(\n args.iters_per_epoch,\n [batch_time, data_time, losses, cls_accs],\n prefix=\"Epoch: [{}]\".format(epoch))\n\n # switch to train mode\n model.train()\n\n end = time.time()\n for i in range(args.iters_per_epoch):\n x, labels = next(train_iter)\n\n x = x.to(device)\n label = labels.to(device)\n\n # measure data loading time\n data_time.update(time.time() - end)\n\n # compute output\n y, f = model(x)\n cls_loss = F.cross_entropy(y, label)\n bss_loss = bss_module(f)\n loss = cls_loss + args.trade_off * bss_loss\n\n cls_acc = accuracy(y, label)[0]\n\n losses.update(loss.item(), x.size(0))\n cls_accs.update(cls_acc.item(), x.size(0))\n\n # compute gradient and do SGD step\n optimizer.zero_grad()\n loss.backward()\n optimizer.step()\n\n # measure elapsed time\n batch_time.update(time.time() - end)\n end = time.time()\n\n if i % args.print_freq == 0:\n progress.display(i)\n\n\nif __name__ == '__main__':\n parser = argparse.ArgumentParser(description='BSS for Finetuning')\n # dataset parameters\n parser.add_argument('root', metavar='DIR',\n help='root path of dataset')\n parser.add_argument('-d', '--data', metavar='DATA')\n parser.add_argument('-sr', '--sample-rate', default=100, type=int,\n metavar='N',\n help='sample rate of training dataset (default: 100)')\n parser.add_argument('-sc', '--num-samples-per-classes', default=None, type=int,\n help='number of samples per classes.')\n parser.add_argument('--train-resizing', type=str, default='default')\n parser.add_argument('--val-resizing', type=str, default='default')\n parser.add_argument('--no-hflip', action='store_true', help='no random horizontal flipping during training')\n parser.add_argument('--color-jitter', action='store_true')\n # model parameters\n parser.add_argument('-a', '--arch', metavar='ARCH', default='resnet50',\n choices=utils.get_model_names(),\n help='backbone architecture: ' +\n ' | '.join(utils.get_model_names()) +\n ' (default: resnet50)')\n parser.add_argument('--no-pool', action='store_true',\n help='no pool layer after the feature extractor.')\n parser.add_argument('--finetune', action='store_true', help='whether use 10x smaller lr for backbone')\n parser.add_argument('--pretrained', default=None,\n help=\"pretrained checkpoint of the backbone. \"\n \"(default: None, use the ImageNet supervised pretrained backbone)\")\n parser.add_argument('-k', '--k', default=1, type=int,\n metavar='N',\n help='hyper-parameter for BSS loss')\n parser.add_argument('--trade-off', default=0.001, type=float,\n metavar='P', help='trade-off weight of BSS loss')\n # training parameters\n parser.add_argument('-b', '--batch-size', default=48, type=int,\n metavar='N',\n help='mini-batch size (default: 48)')\n parser.add_argument('--lr', '--learning-rate', default=0.01, type=float,\n metavar='LR', help='initial learning rate', dest='lr')\n parser.add_argument('--lr-gamma', default=0.1, type=float, help='parameter for lr scheduler')\n parser.add_argument('--lr-decay-epochs', type=int, default=(12, ), nargs='+', help='epochs to decay lr')\n parser.add_argument('--momentum', default=0.9, type=float, metavar='M',\n help='momentum')\n parser.add_argument('--wd', '--weight-decay', default=0.0005, type=float,\n metavar='W', help='weight decay (default: 5e-4)')\n parser.add_argument('-j', '--workers', default=2, type=int, metavar='N',\n help='number of data loading workers (default: 4)')\n parser.add_argument('--epochs', default=20, type=int, metavar='N',\n help='number of total epochs to run')\n parser.add_argument('-i', '--iters-per-epoch', default=500, type=int,\n help='Number of iterations per epoch')\n parser.add_argument('-p', '--print-freq', default=100, type=int,\n metavar='N', help='print frequency (default: 100)')\n parser.add_argument('--seed', default=None, type=int,\n help='seed for initializing training. ')\n parser.add_argument(\"--log\", type=str, default='bss',\n help=\"Where to save logs, checkpoints and debugging images.\")\n parser.add_argument(\"--phase\", type=str, default='train', choices=['train', 'test'],\n help=\"When phase is 'test', only test the model.\")\n args = parser.parse_args()\n main(args)\n\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/examples/task_adaptation/image_classification/bss.sh",
"content": "#!/usr/bin/env bash\n# Supervised Pretraining\n#CUB-200-2011\nCUDA_VISIBLE_DEVICES=0 python bss.py data/cub200 -d CUB200 -sr 100 --seed 0 --finetune --log logs/bss/cub200_100\nCUDA_VISIBLE_DEVICES=0 python bss.py data/cub200 -d CUB200 -sr 50 --seed 0 --finetune --log logs/bss/cub200_50\nCUDA_VISIBLE_DEVICES=0 python bss.py data/cub200 -d CUB200 -sr 30 --seed 0 --finetune --log logs/bss/cub200_30\nCUDA_VISIBLE_DEVICES=0 python bss.py data/cub200 -d CUB200 -sr 15 --seed 0 --finetune --log logs/bss/cub200_15\n\n# Standford Cars\nCUDA_VISIBLE_DEVICES=0 python bss.py data/stanford_cars -d StanfordCars -sr 100 --seed 0 --finetune --log logs/bss/car_100\nCUDA_VISIBLE_DEVICES=0 python bss.py data/stanford_cars -d StanfordCars -sr 50 --seed 0 --finetune --log logs/bss/car_50\nCUDA_VISIBLE_DEVICES=0 python bss.py data/stanford_cars -d StanfordCars -sr 30 --seed 0 --finetune --log logs/bss/car_30\nCUDA_VISIBLE_DEVICES=0 python bss.py data/stanford_cars -d StanfordCars -sr 15 --seed 0 --finetune --log logs/bss/car_15\n\n# Aircrafts\nCUDA_VISIBLE_DEVICES=0 python bss.py data/aircraft -d Aircraft -sr 100 --seed 0 --finetune --log logs/bss/aircraft_100\nCUDA_VISIBLE_DEVICES=0 python bss.py data/aircraft -d Aircraft -sr 50 --seed 0 --finetune --log logs/bss/aircraft_50\nCUDA_VISIBLE_DEVICES=0 python bss.py data/aircraft -d Aircraft -sr 30 --seed 0 --finetune --log logs/bss/aircraft_30\nCUDA_VISIBLE_DEVICES=0 python bss.py data/aircraft -d Aircraft -sr 15 --seed 0 --finetune --log logs/bss/aircraft_15\n\n# Resisc45\nCUDA_VISIBLE_DEVICES=0 python bss.py data/vtab/resisc45 -d Resisc45 -sc 10 --finetune --seed 0 \\\n --log logs/bss/resisc45_10\nCUDA_VISIBLE_DEVICES=0 python bss.py data/vtab/resisc45 -d Resisc45 -sc 20 --finetune --seed 0 \\\n --log logs/bss/resisc45_20\nCUDA_VISIBLE_DEVICES=0 python bss.py data/vtab/resisc45 -d Resisc45 -sc 40 --finetune --seed 0 \\\n --log logs/bss/resisc45_40\nCUDA_VISIBLE_DEVICES=0 python bss.py data/vtab/resisc45 -d Resisc45 -sc 80 --finetune --seed 0 \\\n --log logs/bss/resisc45_80\n\n# Patch Camelyon\nCUDA_VISIBLE_DEVICES=0 python bss.py data/vtab/patch_camelyon -d PatchCamelyon -sc 40 --finetune \\\n --seed 0 --log logs/bss/patch_camelyon_40\nCUDA_VISIBLE_DEVICES=0 python bss.py data/vtab/patch_camelyon -d PatchCamelyon -sc 80 --finetune \\\n --seed 0 --log logs/bss/patch_camelyon_80\nCUDA_VISIBLE_DEVICES=0 python bss.py data/vtab/patch_camelyon -d PatchCamelyon -sc 160 --finetune \\\n --seed 0 --log logs/bss/patch_camelyon_160\nCUDA_VISIBLE_DEVICES=0 python bss.py data/vtab/patch_camelyon -d PatchCamelyon -sc 320 --finetune \\\n --seed 0 --log logs/bss/patch_camelyon_320\n\n# MoCo (Unsupervised Pretraining)\n#CUB-200-2011\nCUDA_VISIBLE_DEVICES=0 python bss.py data/cub200 -d CUB200 -sr 100 --seed 0 --lr 0.1 -i 2000 --lr-decay-epochs 3 6 9 --epochs 12 \\\n --log logs/moco_pretrain_bss/cub200_100 --pretrained checkpoints/moco_v1_200ep_backbone.pth\nCUDA_VISIBLE_DEVICES=0 python bss.py data/cub200 -d CUB200 -sr 50 --seed 0 --lr 0.1 -i 2000 --lr-decay-epochs 3 6 9 --epochs 12 \\\n --log logs/moco_pretrain_bss/cub200_50 --pretrained checkpoints/moco_v1_200ep_backbone.pth\nCUDA_VISIBLE_DEVICES=0 python bss.py data/cub200 -d CUB200 -sr 30 --seed 0 --lr 0.1 -i 2000 --lr-decay-epochs 3 6 9 --epochs 12 \\\n --log logs/moco_pretrain_bss/cub200_30 --pretrained checkpoints/moco_v1_200ep_backbone.pth\nCUDA_VISIBLE_DEVICES=0 python bss.py data/cub200 -d CUB200 -sr 15 --seed 0 --lr 0.1 -i 2000 --lr-decay-epochs 3 6 9 --epochs 12 \\\n --log logs/moco_pretrain_bss/cub200_15 --pretrained checkpoints/moco_v1_200ep_backbone.pth\n\n# Standford Cars\nCUDA_VISIBLE_DEVICES=0 python bss.py data/stanford_cars -d StanfordCars -sr 100 --seed 0 --lr 0.1 -i 2000 --lr-decay-epochs 3 6 9 --epochs 12 \\\n --log logs/moco_pretrain_bss/cars_100 --pretrained checkpoints/moco_v1_200ep_backbone.pth\nCUDA_VISIBLE_DEVICES=0 python bss.py data/stanford_cars -d StanfordCars -sr 50 --seed 0 --lr 0.1 -i 2000 --lr-decay-epochs 3 6 9 --epochs 12 \\\n --log logs/moco_pretrain_bss/cars_50 --pretrained checkpoints/moco_v1_200ep_backbone.pth\nCUDA_VISIBLE_DEVICES=0 python bss.py data/stanford_cars -d StanfordCars -sr 30 --seed 0 --lr 0.1 -i 2000 --lr-decay-epochs 3 6 9 --epochs 12 \\\n --log logs/moco_pretrain_bss/cars_30 --pretrained checkpoints/moco_v1_200ep_backbone.pth\nCUDA_VISIBLE_DEVICES=0 python bss.py data/stanford_cars -d StanfordCars -sr 15 --seed 0 --lr 0.1 -i 2000 --lr-decay-epochs 3 6 9 --epochs 12 \\\n --log logs/moco_pretrain_bss/cars_15 --pretrained checkpoints/moco_v1_200ep_backbone.pth\n\n# Aircrafts\nCUDA_VISIBLE_DEVICES=0 python bss.py data/aircraft -d Aircraft -sr 100 --seed 0 --lr 0.1 -i 2000 --lr-decay-epochs 3 6 9 --epochs 12 \\\n --log logs/moco_pretrain_bss/aircraft_100 --pretrained checkpoints/moco_v1_200ep_backbone.pth\nCUDA_VISIBLE_DEVICES=0 python bss.py data/aircraft -d Aircraft -sr 50 --seed 0 --lr 0.1 -i 2000 --lr-decay-epochs 3 6 9 --epochs 12 \\\n --log logs/moco_pretrain_bss/aircraft_50 --pretrained checkpoints/moco_v1_200ep_backbone.pth\nCUDA_VISIBLE_DEVICES=0 python bss.py data/aircraft -d Aircraft -sr 30 --seed 0 --lr 0.1 -i 2000 --lr-decay-epochs 3 6 9 --epochs 12 \\\n --log logs/moco_pretrain_bss/aircraft_30 --pretrained checkpoints/moco_v1_200ep_backbone.pth\nCUDA_VISIBLE_DEVICES=0 python bss.py data/aircraft -d Aircraft -sr 15 --seed 0 --lr 0.1 -i 2000 --lr-decay-epochs 3 6 9 --epochs 12 \\\n --log logs/moco_pretrain_bss/aircraft_15 --pretrained checkpoints/moco_v1_200ep_backbone.pth\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/examples/task_adaptation/image_classification/co_tuning.py",
"content": "\"\"\"\n@author: Yifei Ji, Junguang Jiang\n@contact: jiyf990330@163.com, JiangJunguang1123@outlook.com\n\"\"\"\nimport random\nimport time\nimport warnings\nimport sys\nimport argparse\nimport shutil\nimport os\n\nimport torch\nimport torch.nn as nn\nimport torch.backends.cudnn as cudnn\nfrom torch.optim import SGD\nfrom torch.utils.data import DataLoader\nimport torch.nn.functional as F\nfrom torch.utils.data import Subset\n\nsys.path.append('../../..')\nfrom talib.finetune.co_tuning import CoTuningLoss, Relationship, Classifier\nfrom common.utils.metric import accuracy\nfrom common.utils.meter import AverageMeter, ProgressMeter\nfrom common.utils.logger import CompleteLogger\nfrom common.utils.data import ForeverDataIterator\nimport common.vision.datasets as datasets\n\nsys.path.append('.')\nimport utils\n\ndevice = torch.device(\"cuda\" if torch.cuda.is_available() else \"cpu\")\n\n\ndef get_dataset(dataset_name, root, train_transform, val_transform, sample_rate=100, num_samples_per_classes=None):\n dataset = datasets.__dict__[dataset_name]\n if sample_rate < 100:\n train_dataset = dataset(root=root, split='train', sample_rate=sample_rate, download=True, transform=train_transform)\n determin_train_dataset = dataset(root=root, split='train', sample_rate=sample_rate, download=True, transform=val_transform)\n test_dataset = dataset(root=root, split='test', sample_rate=100, download=True, transform=val_transform)\n num_classes = train_dataset.num_classes\n else:\n train_dataset = dataset(root=root, split='train', transform=train_transform)\n determin_train_dataset = dataset(root=root, split='train', transform=val_transform)\n test_dataset = dataset(root=root, split='test', transform=val_transform)\n num_classes = train_dataset.num_classes\n if num_samples_per_classes is not None:\n samples = list(range(len(train_dataset)))\n random.shuffle(samples)\n samples_len = min(num_samples_per_classes * num_classes, len(train_dataset))\n train_dataset = Subset(train_dataset, samples[:samples_len])\n determin_train_dataset = Subset(determin_train_dataset, samples[:samples_len])\n return train_dataset, determin_train_dataset, test_dataset, num_classes\n\n\ndef main(args: argparse.Namespace):\n logger = CompleteLogger(args.log, args.phase)\n print(args)\n\n if args.seed is not None:\n random.seed(args.seed)\n torch.manual_seed(args.seed)\n cudnn.deterministic = True\n warnings.warn('You have chosen to seed training. '\n 'This will turn on the CUDNN deterministic setting, '\n 'which can slow down your training considerably! '\n 'You may see unexpected behavior when restarting '\n 'from checkpoints.')\n\n cudnn.benchmark = True\n\n # Data loading code\n train_transform = utils.get_train_transform(args.train_resizing, not args.no_hflip, args.color_jitter)\n val_transform = utils.get_val_transform(args.val_resizing)\n print(\"train_transform: \", train_transform)\n print(\"val_transform: \", val_transform)\n\n train_dataset, determin_train_dataset, val_dataset, num_classes = get_dataset(args.data, args.root, train_transform,\n val_transform, args.sample_rate, args.num_samples_per_classes)\n print(\"training dataset size: {} test dataset size: {}\".format(len(train_dataset), len(val_dataset)))\n\n train_loader = DataLoader(train_dataset, batch_size=args.batch_size, shuffle=True,\n num_workers=args.workers, drop_last=True)\n determin_train_loader = DataLoader(determin_train_dataset, batch_size=args.batch_size,\n shuffle=False, num_workers=args.workers, drop_last=False)\n train_iter = ForeverDataIterator(train_loader)\n val_loader = DataLoader(val_dataset, batch_size=args.batch_size, shuffle=False, num_workers=args.workers)\n\n # create model\n print(\"=> using pre-trained model '{}'\".format(args.arch))\n backbone = utils.get_model(args.arch, args.pretrained)\n pool_layer = nn.Identity() if args.no_pool else None\n classifier = Classifier(backbone, num_classes, head_source=backbone.copy_head(), pool_layer=pool_layer, finetune=args.finetune).to(device)\n\n # define optimizer and lr scheduler\n optimizer = SGD(classifier.get_parameters(args.lr), momentum=args.momentum, weight_decay=args.wd, nesterov=True)\n lr_scheduler = torch.optim.lr_scheduler.MultiStepLR(optimizer, args.lr_decay_epochs, gamma=args.lr_gamma)\n\n # resume from the best checkpoint\n if args.phase == 'test':\n checkpoint = torch.load(logger.get_checkpoint_path('best'), map_location='cpu')\n classifier.load_state_dict(checkpoint)\n acc1 = utils.validate(val_loader, classifier, args, device)\n print(acc1)\n return\n\n # build relationship between source classes and target classes\n source_classifier = nn.Sequential(classifier.backbone, classifier.pool_layer, classifier.head_source)\n relationship = Relationship(determin_train_loader, source_classifier, device, os.path.join(logger.root, args.relationship))\n co_tuning_loss = CoTuningLoss()\n\n # start training\n best_acc1 = 0.0\n for epoch in range(args.epochs):\n # train for one epoch\n train(train_iter, classifier, optimizer, epoch, relationship, co_tuning_loss, args)\n lr_scheduler.step()\n # evaluate on validation set\n acc1 = utils.validate(val_loader, classifier, args, device)\n\n # remember best acc@1 and save checkpoint\n torch.save(classifier.state_dict(), logger.get_checkpoint_path('latest'))\n if acc1 > best_acc1:\n shutil.copy(logger.get_checkpoint_path('latest'), logger.get_checkpoint_path('best'))\n best_acc1 = max(acc1, best_acc1)\n\n print(\"best_acc1 = {:3.1f}\".format(best_acc1))\n logger.close()\n\n\ndef train(train_iter: ForeverDataIterator, model: Classifier, optimizer: SGD,\n epoch: int, relationship, co_tuning_loss, args: argparse.Namespace):\n batch_time = AverageMeter('Time', ':4.2f')\n data_time = AverageMeter('Data', ':3.1f')\n losses = AverageMeter('Loss', ':3.2f')\n cls_accs = AverageMeter('Cls Acc', ':3.1f')\n\n progress = ProgressMeter(\n args.iters_per_epoch,\n [batch_time, data_time, losses, cls_accs],\n prefix=\"Epoch: [{}]\".format(epoch))\n\n # switch to train mode\n model.train()\n\n end = time.time()\n for i in range(args.iters_per_epoch):\n x, label_t = next(train_iter)\n\n x = x.to(device)\n label_s = torch.from_numpy(relationship[label_t]).cuda().float()\n label_t = label_t.to(device)\n\n # measure data loading time\n data_time.update(time.time() - end)\n\n # compute output\n y_s, y_t = model(x)\n tgt_loss = F.cross_entropy(y_t, label_t)\n src_loss = co_tuning_loss(y_s, label_s)\n loss = tgt_loss + args.trade_off * src_loss\n\n # measure accuracy and record loss\n losses.update(loss.item(), x.size(0))\n cls_acc = accuracy(y_t, label_t)[0]\n cls_accs.update(cls_acc.item(), x.size(0))\n\n # compute gradient and do SGD step\n optimizer.zero_grad()\n loss.backward()\n optimizer.step()\n\n # measure elapsed time\n batch_time.update(time.time() - end)\n end = time.time()\n\n if i % args.print_freq == 0:\n progress.display(i)\n\n\nif __name__ == '__main__':\n parser = argparse.ArgumentParser(description='Co-Tuning for Finetuning')\n # dataset parameters\n parser.add_argument('root', metavar='DIR',\n help='root path of dataset')\n parser.add_argument('-d', '--data', metavar='DATA')\n parser.add_argument('-sr', '--sample-rate', default=100, type=int,\n metavar='N',\n help='sample rate of training dataset (default: 100)')\n parser.add_argument('-sc', '--num-samples-per-classes', default=None, type=int,\n help='number of samples per classes.')\n parser.add_argument('--train-resizing', type=str, default='default')\n parser.add_argument('--val-resizing', type=str, default='default')\n parser.add_argument('--no-hflip', action='store_true', help='no random horizontal flipping during training')\n parser.add_argument('--color-jitter', action='store_true')\n # model parameters\n parser.add_argument('-a', '--arch', metavar='ARCH', default='resnet50',\n choices=utils.get_model_names(),\n help='backbone architecture: ' +\n ' | '.join(utils.get_model_names()) +\n ' (default: resnet50)')\n parser.add_argument('--no-pool', action='store_true',\n help='no pool layer after the feature extractor.')\n parser.add_argument('--finetune', action='store_true', help='whether use 10x smaller lr for backbone')\n parser.add_argument('--trade-off', default=2.3, type=float,\n metavar='P', help='the trade-off hyper-parameter for co-tuning loss')\n parser.add_argument(\"--relationship\", type=str, default='relationship.npy',\n help=\"Where to save relationship file.\")\n parser.add_argument('--pretrained', default=None,\n help=\"pretrained checkpoint of the backbone. \"\n \"(default: None, use the ImageNet supervised pretrained backbone)\")\n # training parameters\n parser.add_argument('-b', '--batch-size', default=48, type=int,\n metavar='N',\n help='mini-batch size (default: 48)')\n parser.add_argument('--lr', '--learning-rate', default=0.01, type=float,\n metavar='LR', help='initial learning rate', dest='lr')\n parser.add_argument('--lr-gamma', default=0.1, type=float, help='parameter for lr scheduler')\n parser.add_argument('--lr-decay-epochs', type=int, default=(12, ), nargs='+', help='epochs to decay lr')\n parser.add_argument('--momentum', default=0.9, type=float, metavar='M',\n help='momentum')\n parser.add_argument('--wd', '--weight-decay', default=0.0005, type=float,\n metavar='W', help='weight decay (default: 5e-4)')\n parser.add_argument('-j', '--workers', default=2, type=int, metavar='N',\n help='number of data loading workers (default: 4)')\n parser.add_argument('--epochs', default=20, type=int, metavar='N',\n help='number of total epochs to run')\n parser.add_argument('-i', '--iters-per-epoch', default=500, type=int,\n help='Number of iterations per epoch')\n parser.add_argument('-p', '--print-freq', default=100, type=int,\n metavar='N', help='print frequency (default: 100)')\n parser.add_argument('--seed', default=None, type=int,\n help='seed for initializing training. ')\n parser.add_argument(\"--log\", type=str, default='cotuning',\n help=\"Where to save logs, checkpoints and debugging images.\")\n parser.add_argument(\"--phase\", type=str, default='train', choices=['train', 'test'],\n help=\"When phase is 'test', only test the model.\")\n args = parser.parse_args()\n main(args)\n\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/examples/task_adaptation/image_classification/co_tuning.sh",
"content": "#!/usr/bin/env bash\n# Supervised Pretraining\n# CUB-200-2011\nCUDA_VISIBLE_DEVICES=0 python co_tuning.py data/cub200 -d CUB200 -sr 100 --seed 0 --finetune --log logs/co_tuning/cub200_100\nCUDA_VISIBLE_DEVICES=0 python co_tuning.py data/cub200 -d CUB200 -sr 50 --seed 0 --finetune --log logs/co_tuning/cub200_50\nCUDA_VISIBLE_DEVICES=0 python co_tuning.py data/cub200 -d CUB200 -sr 30 --seed 0 --finetune --log logs/co_tuning/cub200_30\nCUDA_VISIBLE_DEVICES=0 python co_tuning.py data/cub200 -d CUB200 -sr 15 --seed 0 --finetune --log logs/co_tuning/cub200_15\n\n# Standford Cars\nCUDA_VISIBLE_DEVICES=0 python co_tuning.py data/stanford_cars -d StanfordCars -sr 100 --seed 0 --finetune --log logs/co_tuning/car_100\nCUDA_VISIBLE_DEVICES=0 python co_tuning.py data/stanford_cars -d StanfordCars -sr 50 --seed 0 --finetune --log logs/co_tuning/car_50\nCUDA_VISIBLE_DEVICES=0 python co_tuning.py data/stanford_cars -d StanfordCars -sr 30 --seed 0 --finetune --log logs/co_tuning/car_30\nCUDA_VISIBLE_DEVICES=0 python co_tuning.py data/stanford_cars -d StanfordCars -sr 15 --seed 0 --finetune --log logs/co_tuning/car_15\n\n# Aircrafts\nCUDA_VISIBLE_DEVICES=0 python co_tuning.py data/aircraft -d Aircraft -sr 100 --seed 0 --finetune --log logs/co_tuning/aircraft_100\nCUDA_VISIBLE_DEVICES=0 python co_tuning.py data/aircraft -d Aircraft -sr 50 --seed 0 --finetune --log logs/co_tuning/aircraft_50\nCUDA_VISIBLE_DEVICES=0 python co_tuning.py data/aircraft -d Aircraft -sr 30 --seed 0 --finetune --log logs/co_tuning/aircraft_30\nCUDA_VISIBLE_DEVICES=0 python co_tuning.py data/aircraft -d Aircraft -sr 15 --seed 0 --finetune --log logs/co_tuning/aircraft_15\n\n# Resisc45\nCUDA_VISIBLE_DEVICES=0 python co_tuning.py data/vtab/resisc45 -d Resisc45 -sc 10 --finetune --seed 0 \\\n --log logs/co_tuning/resisc45_10\nCUDA_VISIBLE_DEVICES=0 python co_tuning.py data/vtab/resisc45 -d Resisc45 -sc 20 --finetune --seed 0 \\\n --log logs/co_tuning/resisc45_20\nCUDA_VISIBLE_DEVICES=0 python co_tuning.py data/vtab/resisc45 -d Resisc45 -sc 40 --finetune --seed 0 \\\n --log logs/co_tuning/resisc45_40\nCUDA_VISIBLE_DEVICES=0 python co_tuning.py data/vtab/resisc45 -d Resisc45 -sc 80 --finetune --seed 0 \\\n --log logs/co_tuning/resisc45_80\n\n# Patch Camelyon\nCUDA_VISIBLE_DEVICES=0 python co_tuning.py data/vtab/patch_camelyon -d PatchCamelyon -sc 40 --finetune \\\n --seed 0 --log logs/co_tuning/patch_camelyon_40\nCUDA_VISIBLE_DEVICES=0 python co_tuning.py data/vtab/patch_camelyon -d PatchCamelyon -sc 80 --finetune \\\n --seed 0 --log logs/co_tuning/patch_camelyon_80\nCUDA_VISIBLE_DEVICES=0 python co_tuning.py data/vtab/patch_camelyon -d PatchCamelyon -sc 160 --finetune \\\n --seed 0 --log logs/co_tuning/patch_camelyon_160\nCUDA_VISIBLE_DEVICES=0 python co_tuning.py data/vtab/patch_camelyon -d PatchCamelyon -sc 320 --finetune \\\n --seed 0 --log logs/co_tuning/patch_camelyon_320\n\n# MoCo (Unsupervised Pretraining)\n#CUB-200-2011\nCUDA_VISIBLE_DEVICES=0 python co_tuning.py data/cub200 -d CUB200 -sr 100 --seed 0 --lr 0.1 -i 2000 --lr-decay-epochs 3 6 9 --epochs 12 \\\n --log logs/moco_pretrain_co_tuning/cub200_100 --pretrained checkpoints/moco_v1_200ep_backbone.pth\nCUDA_VISIBLE_DEVICES=0 python co_tuning.py data/cub200 -d CUB200 -sr 50 --seed 0 --lr 0.1 -i 2000 --lr-decay-epochs 3 6 9 --epochs 12 \\\n --log logs/moco_pretrain_co_tuning/cub200_50 --pretrained checkpoints/moco_v1_200ep_backbone.pth\nCUDA_VISIBLE_DEVICES=0 python co_tuning.py data/cub200 -d CUB200 -sr 30 --seed 0 --lr 0.1 -i 2000 --lr-decay-epochs 3 6 9 --epochs 12 \\\n --log logs/moco_pretrain_co_tuning/cub200_30 --pretrained checkpoints/moco_v1_200ep_backbone.pth\nCUDA_VISIBLE_DEVICES=0 python co_tuning.py data/cub200 -d CUB200 -sr 15 --seed 0 --lr 0.1 -i 2000 --lr-decay-epochs 3 6 9 --epochs 12 \\\n --log logs/moco_pretrain_co_tuning/cub200_15 --pretrained checkpoints/moco_v1_200ep_backbone.pth\n\n# Standford Cars\nCUDA_VISIBLE_DEVICES=0 python co_tuning.py data/stanford_cars -d StanfordCars -sr 100 --seed 0 --lr 0.1 -i 2000 --lr-decay-epochs 3 6 9 --epochs 12 \\\n --log logs/moco_pretrain_co_tuning/cars_100 --pretrained checkpoints/moco_v1_200ep_backbone.pth\nCUDA_VISIBLE_DEVICES=0 python co_tuning.py data/stanford_cars -d StanfordCars -sr 50 --seed 0 --lr 0.1 -i 2000 --lr-decay-epochs 3 6 9 --epochs 12 \\\n --log logs/moco_pretrain_co_tuning/cars_50 --pretrained checkpoints/moco_v1_200ep_backbone.pth\nCUDA_VISIBLE_DEVICES=0 python co_tuning.py data/stanford_cars -d StanfordCars -sr 30 --seed 0 --lr 0.1 -i 2000 --lr-decay-epochs 3 6 9 --epochs 12 \\\n --log logs/moco_pretrain_co_tuning/cars_30 --pretrained checkpoints/moco_v1_200ep_backbone.pth\nCUDA_VISIBLE_DEVICES=0 python co_tuning.py data/stanford_cars -d StanfordCars -sr 15 --seed 0 --lr 0.1 -i 2000 --lr-decay-epochs 3 6 9 --epochs 12 \\\n --log logs/moco_pretrain_co_tuning/cars_15 --pretrained checkpoints/moco_v1_200ep_backbone.pth\n\n# Aircrafts\nCUDA_VISIBLE_DEVICES=0 python co_tuning.py data/aircraft -d Aircraft -sr 100 --seed 0 --lr 0.1 -i 2000 --lr-decay-epochs 3 6 9 --epochs 12 \\\n --log logs/moco_pretrain_co_tuning/aircraft_100 --pretrained checkpoints/moco_v1_200ep_backbone.pth\nCUDA_VISIBLE_DEVICES=0 python co_tuning.py data/aircraft -d Aircraft -sr 50 --seed 0 --lr 0.1 -i 2000 --lr-decay-epochs 3 6 9 --epochs 12 \\\n --log logs/moco_pretrain_co_tuning/aircraft_50 --pretrained checkpoints/moco_v1_200ep_backbone.pth\nCUDA_VISIBLE_DEVICES=0 python co_tuning.py data/aircraft -d Aircraft -sr 30 --seed 0 --lr 0.1 -i 2000 --lr-decay-epochs 3 6 9 --epochs 12 \\\n --log logs/moco_pretrain_co_tuning/aircraft_30 --pretrained checkpoints/moco_v1_200ep_backbone.pth\nCUDA_VISIBLE_DEVICES=0 python co_tuning.py data/aircraft -d Aircraft -sr 15 --seed 0 --lr 0.1 -i 2000 --lr-decay-epochs 3 6 9 --epochs 12 \\\n --log logs/moco_pretrain_co_tuning/aircraft_15 --pretrained checkpoints/moco_v1_200ep_backbone.pth\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/examples/task_adaptation/image_classification/convert_moco_to_pretrained.py",
"content": "\"\"\"\n@author: Junguang Jiang\n@contact: JiangJunguang1123@outlook.com\n\"\"\"\nimport sys\nimport torch\n\nif __name__ == \"__main__\":\n input = sys.argv[1]\n\n obj = torch.load(input, map_location=\"cpu\")\n obj = obj[\"state_dict\"]\n\n newmodel = {}\n fc = {}\n for k, v in obj.items():\n if not k.startswith(\"module.encoder_q.\"):\n continue\n old_k = k\n k = k.replace(\"module.encoder_q.\", \"\")\n if k.startswith(\"fc\"):\n print(k)\n fc[k] = v\n else:\n newmodel[k] = v\n\n with open(sys.argv[2], \"wb\") as f:\n torch.save(newmodel, f)\n\n with open(sys.argv[3], \"wb\") as f:\n torch.save(fc, f)\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/examples/task_adaptation/image_classification/delta.py",
"content": "\"\"\"\n@author: Yifei Ji, Junguang Jiang\n@contact: jiyf990330@163.com, JiangJunguang1123@outlook.com\n\"\"\"\nimport math\nimport os\nimport random\nimport time\nimport warnings\nimport sys\nimport argparse\nimport shutil\n\nimport numpy as np\nfrom tqdm import tqdm\nimport torch.backends.cudnn as cudnn\nfrom torch.optim import SGD\nfrom torch.utils.data import DataLoader\nimport torch.nn.functional as F\n\nsys.path.append('../../..')\nfrom talib.finetune.delta import *\nfrom common.modules.classifier import Classifier\nfrom common.utils.data import ForeverDataIterator\nfrom common.utils.metric import accuracy\nfrom common.utils.meter import AverageMeter, ProgressMeter\nfrom common.utils.logger import CompleteLogger\n\nsys.path.append('.')\nimport utils\n\ndevice = torch.device(\"cuda\" if torch.cuda.is_available() else \"cpu\")\n\n\ndef main(args: argparse.Namespace):\n logger = CompleteLogger(args.log, args.phase)\n print(args)\n\n if args.seed is not None:\n random.seed(args.seed)\n torch.manual_seed(args.seed)\n cudnn.deterministic = True\n warnings.warn('You have chosen to seed training. '\n 'This will turn on the CUDNN deterministic setting, '\n 'which can slow down your training considerably! '\n 'You may see unexpected behavior when restarting '\n 'from checkpoints.')\n\n cudnn.benchmark = True\n\n # Data loading code\n train_transform = utils.get_train_transform(args.train_resizing, not args.no_hflip, args.color_jitter)\n val_transform = utils.get_val_transform(args.val_resizing)\n print(\"train_transform: \", train_transform)\n print(\"val_transform: \", val_transform)\n\n train_dataset, val_dataset, num_classes = utils.get_dataset(args.data, args.root, train_transform,\n val_transform, args.sample_rate, args.num_samples_per_classes)\n train_loader = DataLoader(train_dataset, batch_size=args.batch_size, shuffle=True,\n num_workers=args.workers, drop_last=True)\n train_iter = ForeverDataIterator(train_loader)\n val_loader = DataLoader(val_dataset, batch_size=args.batch_size, shuffle=False, num_workers=args.workers)\n print(\"training dataset size: {} test dataset size: {}\".format(len(train_dataset), len(val_dataset)))\n\n # create model\n print(\"=> using pre-trained model '{}'\".format(args.arch))\n backbone = utils.get_model(args.arch, args.pretrained)\n backbone_source = utils.get_model(args.arch, args.pretrained)\n pool_layer = nn.Identity() if args.no_pool else None\n classifier = Classifier(backbone, num_classes, pool_layer=pool_layer, finetune=args.finetune).to(device)\n source_classifier = Classifier(backbone_source, num_classes=backbone_source.fc.out_features,\n head=backbone_source.copy_head(), pool_layer=pool_layer).to(device)\n for param in source_classifier.parameters():\n param.requires_grad = False\n source_classifier.eval()\n\n # define optimizer and lr scheduler\n optimizer = SGD(classifier.get_parameters(args.lr), momentum=args.momentum, weight_decay=args.wd, nesterov=True)\n lr_scheduler = torch.optim.lr_scheduler.MultiStepLR(optimizer, args.lr_decay_epochs, gamma=args.lr_gamma)\n\n # resume from the best checkpoint\n if args.phase == 'test':\n checkpoint = torch.load(logger.get_checkpoint_path('best'), map_location='cpu')\n classifier.load_state_dict(checkpoint)\n acc1 = utils.validate(val_loader, classifier, args, device)\n print(acc1)\n return\n\n # create intermediate layer getter\n if args.arch == 'resnet50':\n return_layers = ['backbone.layer1.2.conv3', 'backbone.layer2.3.conv3', 'backbone.layer3.5.conv3', 'backbone.layer4.2.conv3']\n elif args.arch == 'resnet101':\n return_layers = ['backbone.layer1.2.conv3', 'backbone.layer2.3.conv3', 'backbone.layer3.5.conv3', 'backbone.layer4.2.conv3']\n else:\n raise NotImplementedError(args.arch)\n source_getter = IntermediateLayerGetter(source_classifier, return_layers=return_layers)\n target_getter = IntermediateLayerGetter(classifier, return_layers=return_layers)\n\n # get regularization\n if args.regularization_type == 'l2_sp':\n backbone_regularization = SPRegularization(source_classifier.backbone, classifier.backbone)\n elif args.regularization_type == 'feature_map':\n backbone_regularization = BehavioralRegularization()\n elif args.regularization_type == 'attention_feature_map':\n attention_file = os.path.join(logger.root, args.attention_file)\n if not os.path.exists(attention_file):\n attention = calculate_channel_attention(train_dataset, return_layers, num_classes, args)\n torch.save(attention, attention_file)\n else:\n print(\"Loading channel attention from\", attention_file)\n attention = torch.load(attention_file)\n attention = [a.to(device) for a in attention]\n backbone_regularization = AttentionBehavioralRegularization(attention)\n else:\n raise NotImplementedError(args.regularization_type)\n\n head_regularization = L2Regularization(nn.ModuleList([classifier.head, classifier.bottleneck]))\n\n # start training\n best_acc1 = 0.0\n\n for epoch in range(args.epochs):\n print(lr_scheduler.get_lr())\n # train for one epoch\n train(train_iter, classifier, backbone_regularization, head_regularization, target_getter, source_getter, optimizer, epoch, args)\n lr_scheduler.step()\n\n # evaluate on validation set\n acc1 = utils.validate(val_loader, classifier, args, device)\n\n # remember best acc@1 and save checkpoint\n torch.save(classifier.state_dict(), logger.get_checkpoint_path('latest'))\n if acc1 > best_acc1:\n shutil.copy(logger.get_checkpoint_path('latest'), logger.get_checkpoint_path('best'))\n best_acc1 = max(acc1, best_acc1)\n\n print(\"best_acc1 = {:3.1f}\".format(best_acc1))\n logger.close()\n\n\ndef calculate_channel_attention(dataset, return_layers, num_classes, args):\n backbone = utils.get_model(args.arch)\n classifier = Classifier(backbone, num_classes).to(device)\n optimizer = SGD(classifier.get_parameters(args.lr), momentum=args.momentum, weight_decay=args.wd, nesterov=True)\n data_loader = DataLoader(dataset, batch_size=args.attention_batch_size, shuffle=True,\n num_workers=args.workers, drop_last=False)\n lr_scheduler = torch.optim.lr_scheduler.ExponentialLR(optimizer, gamma=math.exp(math.log(0.1) / args.attention_lr_decay_epochs))\n criterion = nn.CrossEntropyLoss()\n\n channel_weights = []\n for layer_id, name in enumerate(return_layers):\n layer = get_attribute(classifier, name)\n layer_channel_weight = [0] * layer.out_channels\n channel_weights.append(layer_channel_weight)\n\n # train the classifier\n classifier.train()\n classifier.backbone.requires_grad = False\n print(\"Pretrain a classifier to calculate channel attention.\")\n for epoch in range(args.attention_epochs):\n losses = AverageMeter('Loss', ':3.2f')\n cls_accs = AverageMeter('Cls Acc', ':3.1f')\n progress = ProgressMeter(\n len(data_loader),\n [losses, cls_accs],\n prefix=\"Epoch: [{}]\".format(epoch))\n\n for i, data in enumerate(data_loader):\n inputs, labels = data\n inputs = inputs.to(device)\n labels = labels.to(device)\n outputs, _ = classifier(inputs)\n loss = criterion(outputs, labels)\n optimizer.zero_grad()\n loss.backward()\n optimizer.step()\n\n cls_acc = accuracy(outputs, labels)[0]\n\n losses.update(loss.item(), inputs.size(0))\n cls_accs.update(cls_acc.item(), inputs.size(0))\n\n if i % args.print_freq == 0:\n progress.display(i)\n lr_scheduler.step()\n\n # calculate the channel attention\n print('Calculating channel attention.')\n classifier.eval()\n if args.attention_iteration_limit > 0:\n total_iteration = min(len(data_loader), args.attention_iteration_limit)\n else:\n total_iteration = len(args.data_loader)\n\n progress = ProgressMeter(\n total_iteration,\n [],\n prefix=\"Iteration: \")\n\n for i, data in enumerate(data_loader):\n if i >= total_iteration:\n break\n inputs, labels = data\n inputs = inputs.to(device)\n labels = labels.to(device)\n outputs = classifier(inputs)\n loss_0 = criterion(outputs, labels)\n progress.display(i)\n for layer_id, name in enumerate(tqdm(return_layers)):\n layer = get_attribute(classifier, name)\n for j in range(layer.out_channels):\n tmp = classifier.state_dict()[name + '.weight'][j,].clone()\n classifier.state_dict()[name + '.weight'][j,] = 0.0\n outputs = classifier(inputs)\n loss_1 = criterion(outputs, labels)\n difference = loss_1 - loss_0\n difference = difference.detach().cpu().numpy().item()\n history_value = channel_weights[layer_id][j]\n channel_weights[layer_id][j] = 1.0 * (i * history_value + difference) / (i + 1)\n classifier.state_dict()[name + '.weight'][j, ] = tmp\n\n channel_attention = []\n for weight in channel_weights:\n weight = np.array(weight)\n weight = (weight - np.mean(weight)) / np.std(weight)\n weight = torch.from_numpy(weight).float().to(device)\n channel_attention.append(F.softmax(weight / 5).detach())\n return channel_attention\n\n\ndef train(train_iter: ForeverDataIterator, model: Classifier, backbone_regularization:nn.Module, head_regularization:nn.Module,\n target_getter: IntermediateLayerGetter,\n source_getter: IntermediateLayerGetter,\n optimizer: SGD, epoch: int, args: argparse.Namespace):\n batch_time = AverageMeter('Time', ':4.2f')\n data_time = AverageMeter('Data', ':3.1f')\n losses = AverageMeter('Loss', ':3.2f')\n losses_reg_head = AverageMeter('Loss (reg, head)', ':3.2f')\n losses_reg_backbone = AverageMeter('Loss (reg, backbone)', ':3.2f')\n cls_accs = AverageMeter('Cls Acc', ':3.1f')\n\n progress = ProgressMeter(\n args.iters_per_epoch,\n [batch_time, data_time, losses, losses_reg_head, losses_reg_backbone, cls_accs],\n prefix=\"Epoch: [{}]\".format(epoch))\n\n # switch to train mode\n model.train()\n\n end = time.time()\n for i in range(args.iters_per_epoch):\n x, labels = next(train_iter)\n x = x.to(device)\n label = labels.to(device)\n\n # measure data loading time\n data_time.update(time.time() - end)\n\n # compute output\n intermediate_output_s, output_s = source_getter(x)\n intermediate_output_t, output_t = target_getter(x)\n y, f = output_t\n\n # measure accuracy and record loss\n cls_acc = accuracy(y, label)[0]\n cls_loss = F.cross_entropy(y, label)\n if args.regularization_type == 'feature_map':\n loss_reg_backbone = backbone_regularization(intermediate_output_s, intermediate_output_t)\n elif args.regularization_type == 'attention_feature_map':\n loss_reg_backbone = backbone_regularization(intermediate_output_s, intermediate_output_t)\n else:\n loss_reg_backbone = backbone_regularization()\n loss_reg_head = head_regularization()\n loss = cls_loss + args.trade_off_backbone * loss_reg_backbone + args.trade_off_head * loss_reg_head\n\n losses_reg_backbone.update(loss_reg_backbone.item() * args.trade_off_backbone, x.size(0))\n losses_reg_head.update(loss_reg_head.item() * args.trade_off_head, x.size(0))\n losses.update(loss.item(), x.size(0))\n cls_accs.update(cls_acc.item(), x.size(0))\n\n # compute gradient and do SGD step\n optimizer.zero_grad()\n loss.backward()\n optimizer.step()\n\n # measure elapsed time\n batch_time.update(time.time() - end)\n end = time.time()\n\n if i % args.print_freq == 0:\n progress.display(i)\n\n\nif __name__ == '__main__':\n parser = argparse.ArgumentParser(description='Delta for Finetuning')\n # dataset parameters\n parser.add_argument('root', metavar='DIR',\n help='root path of dataset')\n parser.add_argument('-d', '--data', metavar='DATA')\n parser.add_argument('-sr', '--sample-rate', default=100, type=int,\n metavar='N',\n help='sample rate of training dataset (default: 100)')\n parser.add_argument('-sc', '--num-samples-per-classes', default=None, type=int,\n help='number of samples per classes.')\n parser.add_argument('--train-resizing', type=str, default='default')\n parser.add_argument('--val-resizing', type=str, default='default')\n parser.add_argument('--no-hflip', action='store_true', help='no random horizontal flipping during training')\n parser.add_argument('--color-jitter', action='store_true')\n # model parameters\n parser.add_argument('-a', '--arch', metavar='ARCH', default='resnet50',\n choices=utils.get_model_names(),\n help='backbone architecture: ' +\n ' | '.join(utils.get_model_names()) +\n ' (default: resnet50)')\n parser.add_argument('--no-pool', action='store_true',\n help='no pool layer after the feature extractor.')\n parser.add_argument('--finetune', action='store_true', help='whether use 10x smaller lr for backbone')\n parser.add_argument('--pretrained', default=None,\n help=\"pretrained checkpoint of the backbone. \"\n \"(default: None, use the ImageNet supervised pretrained backbone)\")\n parser.add_argument('--regularization-type', choices=['l2_sp', 'feature_map', 'attention_feature_map'],\n default='attention_feature_map')\n parser.add_argument('--trade-off-backbone', default=0.01, type=float,\n help='trade-off for backbone regularization')\n parser.add_argument('--trade-off-head', default=0.01, type=float,\n help='trade-off for head regularization')\n # training parameters\n parser.add_argument('-b', '--batch-size', default=48, type=int,\n metavar='N',\n help='mini-batch size (default: 48)')\n parser.add_argument('--lr', '--learning-rate', default=0.01, type=float,\n metavar='LR', help='initial learning rate', dest='lr')\n parser.add_argument('--lr-gamma', default=0.1, type=float, help='parameter for lr scheduler')\n parser.add_argument('--lr-decay-epochs', type=int, default=(12, ), nargs='+', help='epochs to decay lr')\n parser.add_argument('--momentum', default=0.9, type=float, metavar='M',\n help='momentum')\n parser.add_argument('--wd', '--weight-decay', default=0.0, type=float,\n metavar='W', help='weight decay (default: 0.0d)')\n parser.add_argument('-j', '--workers', default=2, type=int, metavar='N',\n help='number of data loading workers (default: 4)')\n parser.add_argument('--epochs', default=20, type=int, metavar='N',\n help='number of total epochs to run')\n parser.add_argument('-i', '--iters-per-epoch', default=500, type=int,\n help='Number of iterations per epoch')\n parser.add_argument('-p', '--print-freq', default=100, type=int,\n metavar='N', help='print frequency (default: 100)')\n parser.add_argument('--seed', default=None, type=int,\n help='seed for initializing training. ')\n parser.add_argument(\"--log\", type=str, default='delta',\n help=\"Where to save logs, checkpoints and debugging images.\")\n parser.add_argument(\"--phase\", type=str, default='train', choices=['train', 'test'],\n help=\"When phase is 'test', only test the model.\"\n \"When phase is 'analysis', only analysis the model.\")\n\n # parameters for calculating channel attention\n parser.add_argument(\"--attention-file\", type=str, default='channel_attention.pt',\n help=\"Where to save and load channel attention file.\")\n parser.add_argument('--attention-batch-size', default=32, type=int,\n metavar='N',\n help='mini-batch size for calculating channel attention (default: 32)')\n parser.add_argument('--attention-epochs', default=10, type=int, metavar='N',\n help='number of epochs to train for training before calculating channel weight')\n parser.add_argument('--attention-lr-decay-epochs', default=6, type=int, metavar='N',\n help='epochs to decay lr for training before calculating channel weight')\n parser.add_argument('--attention-iteration-limit', default=10, type=int, metavar='N',\n help='iteration limits for calculating channel attention, -1 means no limits')\n\n args = parser.parse_args()\n main(args)\n\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/examples/task_adaptation/image_classification/delta.sh",
"content": "#!/usr/bin/env bash\n# CUB-200-2011\nCUDA_VISIBLE_DEVICES=0 python delta.py data/cub200 -d CUB200 -sr 100 --seed 0 --finetune --log logs/delta/cub200_100\nCUDA_VISIBLE_DEVICES=0 python delta.py data/cub200 -d CUB200 -sr 50 --seed 0 --finetune --log logs/delta/cub200_50\nCUDA_VISIBLE_DEVICES=0 python delta.py data/cub200 -d CUB200 -sr 30 --seed 0 --finetune --log logs/delta/cub200_30\nCUDA_VISIBLE_DEVICES=0 python delta.py data/cub200 -d CUB200 -sr 15 --seed 0 --finetune --log logs/delta/cub200_15\n\n# Stanford Cars\nCUDA_VISIBLE_DEVICES=0 python delta.py data/stanford_cars -d StanfordCars -sr 100 --seed 0 --finetune --log logs/delta/car_100\nCUDA_VISIBLE_DEVICES=0 python delta.py data/stanford_cars -d StanfordCars -sr 50 --seed 0 --finetune --log logs/delta/car_50\nCUDA_VISIBLE_DEVICES=0 python delta.py data/stanford_cars -d StanfordCars -sr 30 --seed 0 --finetune --log logs/delta/car_30\nCUDA_VISIBLE_DEVICES=0 python delta.py data/stanford_cars -d StanfordCars -sr 15 --seed 0 --finetune --log logs/delta/car_15\n\n# Aircrafts\nCUDA_VISIBLE_DEVICES=0 python delta.py data/aircraft -d Aircraft -sr 100 --seed 0 --finetune --log logs/delta/aircraft_100\nCUDA_VISIBLE_DEVICES=0 python delta.py data/aircraft -d Aircraft -sr 50 --seed 0 --finetune --log logs/delta/aircraft_50\nCUDA_VISIBLE_DEVICES=0 python delta.py data/aircraft -d Aircraft -sr 30 --seed 0 --finetune --log logs/delta/aircraft_30\nCUDA_VISIBLE_DEVICES=0 python delta.py data/aircraft -d Aircraft -sr 15 --seed 0 --finetune --log logs/delta/aircraft_15\n\n# Resisc45\nCUDA_VISIBLE_DEVICES=0 python delta.py data/vtab/resisc45 -d Resisc45 -sc 10 --finetune --seed 0 \\\n --log logs/delta/resisc45_10\nCUDA_VISIBLE_DEVICES=0 python delta.py data/vtab/resisc45 -d Resisc45 -sc 20 --finetune --seed 0 \\\n --log logs/delta/resisc45_20\nCUDA_VISIBLE_DEVICES=0 python delta.py data/vtab/resisc45 -d Resisc45 -sc 40 --finetune --seed 0 \\\n --log logs/delta/resisc45_40\nCUDA_VISIBLE_DEVICES=0 python delta.py data/vtab/resisc45 -d Resisc45 -sc 80 --finetune --seed 0 \\\n --log logs/delta/resisc45_80\n\n# Patch Camelyon\nCUDA_VISIBLE_DEVICES=0 python delta.py data/vtab/patch_camelyon -d PatchCamelyon -sc 40 --finetune \\\n --seed 0 --log logs/delta/patch_camelyon_40\nCUDA_VISIBLE_DEVICES=0 python delta.py data/vtab/patch_camelyon -d PatchCamelyon -sc 80 --finetune \\\n --seed 0 --log logs/delta/patch_camelyon_80\nCUDA_VISIBLE_DEVICES=0 python delta.py data/vtab/patch_camelyon -d PatchCamelyon -sc 160 --finetune \\\n --seed 0 --log logs/delta/patch_camelyon_160\nCUDA_VISIBLE_DEVICES=0 python delta.py data/vtab/patch_camelyon -d PatchCamelyon -sc 320 --finetune \\\n --seed 0 --log logs/delta/patch_camelyon_320\n\n# MoCo (Unsupervised Pretraining)\n#CUB-200-2011\nCUDA_VISIBLE_DEVICES=0 python delta.py data/cub200 -d CUB200 -sr 100 --seed 0 --lr 0.1 -i 2000 --lr-decay-epochs 3 6 9 --epochs 12 \\\n --log logs/moco_pretrain_delta/cub200_100 --pretrained checkpoints/moco_v1_200ep_pretrain.pth\nCUDA_VISIBLE_DEVICES=0 python delta.py data/cub200 -d CUB200 -sr 50 --seed 0 --lr 0.1 -i 2000 --lr-decay-epochs 3 6 9 --epochs 12 \\\n --log logs/moco_pretrain_delta/cub200_50 --pretrained checkpoints/moco_v1_200ep_pretrain.pth\nCUDA_VISIBLE_DEVICES=0 python delta.py data/cub200 -d CUB200 -sr 30 --seed 0 --lr 0.1 -i 2000 --lr-decay-epochs 3 6 9 --epochs 12 \\\n --log logs/moco_pretrain_delta/cub200_30 --pretrained checkpoints/moco_v1_200ep_pretrain.pth\nCUDA_VISIBLE_DEVICES=0 python delta.py data/cub200 -d CUB200 -sr 15 --seed 0 --lr 0.1 -i 2000 --lr-decay-epochs 3 6 9 --epochs 12 \\\n --log logs/moco_pretrain_delta/cub200_15 --pretrained checkpoints/moco_v1_200ep_pretrain.pth\n\n# Standford Cars\nCUDA_VISIBLE_DEVICES=0 python delta.py data/stanford_cars -d StanfordCars -sr 100 --seed 0 --lr 0.1 -i 2000 --lr-decay-epochs 3 6 9 --epochs 12 \\\n --log logs/moco_pretrain_delta/cars_100 --pretrained checkpoints/moco_v1_200ep_pretrain.pth\nCUDA_VISIBLE_DEVICES=0 python delta.py data/stanford_cars -d StanfordCars -sr 50 --seed 0 --lr 0.1 -i 2000 --lr-decay-epochs 3 6 9 --epochs 12 \\\n --log logs/moco_pretrain_delta/cars_50 --pretrained checkpoints/moco_v1_200ep_pretrain.pth\nCUDA_VISIBLE_DEVICES=0 python delta.py data/stanford_cars -d StanfordCars -sr 30 --seed 0 --lr 0.1 -i 2000 --lr-decay-epochs 3 6 9 --epochs 12 \\\n --log logs/moco_pretrain_delta/cars_30 --pretrained checkpoints/moco_v1_200ep_pretrain.pth\nCUDA_VISIBLE_DEVICES=0 python delta.py data/stanford_cars -d StanfordCars -sr 15 --seed 0 --lr 0.1 -i 2000 --lr-decay-epochs 3 6 9 --epochs 12 \\\n --log logs/moco_pretrain_delta/cars_15 --pretrained checkpoints/moco_v1_200ep_pretrain.pth\n\n# Aircrafts\nCUDA_VISIBLE_DEVICES=0 python delta.py data/aircraft -d Aircraft -sr 100 --seed 0 --lr 0.1 -i 2000 --lr-decay-epochs 3 6 9 --epochs 12 \\\n --log logs/moco_pretrain_delta/aircraft_100 --pretrained checkpoints/moco_v1_200ep_pretrain.pth\nCUDA_VISIBLE_DEVICES=0 python delta.py data/aircraft -d Aircraft -sr 50 --seed 0 --lr 0.1 -i 2000 --lr-decay-epochs 3 6 9 --epochs 12 \\\n --log logs/moco_pretrain_delta/aircraft_50 --pretrained checkpoints/moco_v1_200ep_pretrain.pth\nCUDA_VISIBLE_DEVICES=0 python delta.py data/aircraft -d Aircraft -sr 30 --seed 0 --lr 0.1 -i 2000 --lr-decay-epochs 3 6 9 --epochs 12 \\\n --log logs/moco_pretrain_delta/aircraft_30 --pretrained checkpoints/moco_v1_200ep_pretrain.pth\nCUDA_VISIBLE_DEVICES=0 python delta.py data/aircraft -d Aircraft -sr 15 --seed 0 --lr 0.1 -i 2000 --lr-decay-epochs 3 6 9 --epochs 12 \\\n --log logs/moco_pretrain_delta/aircraft_15 --pretrained checkpoints/moco_v1_200ep_pretrain.pth\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/examples/task_adaptation/image_classification/lwf.py",
"content": "\"\"\"\nLearning Without Forgetting in Finetune.\n@author: Junguang Jiang\n@contact: JiangJunguang1123@outlook.com\n\"\"\"\nimport random\nimport time\nimport warnings\nimport sys\nimport argparse\nimport shutil\n\nimport torch\nimport torch.nn as nn\nimport torch.backends.cudnn as cudnn\nfrom torch.optim import SGD\nfrom torch.utils.data import DataLoader, TensorDataset\nimport torch.nn.functional as F\n\nsys.path.append('../../..')\nfrom talib.finetune.lwf import collect_pretrain_labels, Classifier\nfrom common.loss import KnowledgeDistillationLoss\nfrom common.utils.metric import accuracy\nfrom common.utils.meter import AverageMeter, ProgressMeter\nfrom common.utils.logger import CompleteLogger\nfrom common.utils.data import ForeverDataIterator, CombineDataset\n\nsys.path.append('.')\nimport utils\n\ndevice = torch.device(\"cuda\" if torch.cuda.is_available() else \"cpu\")\n\n\ndef main(args: argparse.Namespace):\n logger = CompleteLogger(args.log, args.phase)\n print(args)\n\n if args.seed is not None:\n random.seed(args.seed)\n torch.manual_seed(args.seed)\n cudnn.deterministic = True\n warnings.warn('You have chosen to seed training. '\n 'This will turn on the CUDNN deterministic setting, '\n 'which can slow down your training considerably! '\n 'You may see unexpected behavior when restarting '\n 'from checkpoints.')\n\n cudnn.benchmark = True\n\n # Data loading code\n train_transform = utils.get_train_transform(args.train_resizing, not args.no_hflip, args.color_jitter)\n val_transform = utils.get_val_transform(args.val_resizing)\n print(\"train_transform: \", train_transform)\n print(\"val_transform: \", val_transform)\n\n train_dataset, val_dataset, num_classes = utils.get_dataset(args.data, args.root, train_transform,\n val_transform, args.sample_rate, args.num_samples_per_classes)\n train_loader = DataLoader(train_dataset, batch_size=args.batch_size, shuffle=False,\n num_workers=args.workers, drop_last=False)\n val_loader = DataLoader(val_dataset, batch_size=args.batch_size, shuffle=False, num_workers=args.workers)\n print(\"training dataset size: {} test dataset size: {}\".format(len(train_dataset), len(val_dataset)))\n\n # create model\n print(\"=> using pre-trained model '{}'\".format(args.arch))\n backbone = utils.get_model(args.arch, args.pretrained)\n pool_layer = nn.Identity() if args.no_pool else None\n classifier = Classifier(backbone, num_classes, head_source=backbone.copy_head(), pool_layer=pool_layer,\n finetune=args.finetune).to(device)\n kd = KnowledgeDistillationLoss(args.T)\n\n source_classifier = nn.Sequential(classifier.backbone, classifier.pool_layer, classifier.head_source)\n pretrain_labels = collect_pretrain_labels(train_loader, source_classifier, device)\n train_dataset = CombineDataset([train_dataset, TensorDataset(pretrain_labels)])\n train_loader = DataLoader(train_dataset, batch_size=args.batch_size,\n shuffle=True, num_workers=args.workers, drop_last=True)\n train_iter = ForeverDataIterator(train_loader)\n\n # define optimizer and lr scheduler\n optimizer = SGD(classifier.get_parameters(args.lr), momentum=args.momentum, weight_decay=args.wd, nesterov=True)\n lr_scheduler = torch.optim.lr_scheduler.MultiStepLR(optimizer, args.lr_decay_epochs, gamma=args.lr_gamma)\n\n # resume from the best checkpoint\n if args.phase == 'test':\n checkpoint = torch.load(logger.get_checkpoint_path('best'), map_location='cpu')\n classifier.load_state_dict(checkpoint)\n acc1 = utils.validate(val_loader, classifier, args, device)\n print(acc1)\n return\n\n # start training\n best_acc1 = 0.0\n for epoch in range(args.epochs):\n # train for one epoch\n train(train_iter, classifier, kd, optimizer, epoch, args)\n lr_scheduler.step()\n # evaluate on validation set\n acc1 = utils.validate(val_loader, classifier, args, device)\n\n # remember best acc@1 and save checkpoint\n torch.save(classifier.state_dict(), logger.get_checkpoint_path('latest'))\n if acc1 > best_acc1:\n shutil.copy(logger.get_checkpoint_path('latest'), logger.get_checkpoint_path('best'))\n best_acc1 = max(acc1, best_acc1)\n\n print(\"best_acc1 = {:3.1f}\".format(best_acc1))\n logger.close()\n\n\ndef train(train_iter: ForeverDataIterator, model: Classifier, kd, optimizer: SGD,\n epoch: int, args: argparse.Namespace):\n batch_time = AverageMeter('Time', ':4.2f')\n data_time = AverageMeter('Data', ':3.1f')\n losses = AverageMeter('Loss', ':3.2f')\n losses_kd = AverageMeter('Loss (KD)', ':5.4f')\n cls_accs = AverageMeter('Cls Acc', ':3.1f')\n\n progress = ProgressMeter(\n args.iters_per_epoch,\n [batch_time, data_time, losses, losses_kd, cls_accs],\n prefix=\"Epoch: [{}]\".format(epoch))\n\n # switch to train mode\n model.train()\n\n end = time.time()\n for i in range(args.iters_per_epoch):\n x, label_t, label_s = next(train_iter)\n\n x = x.to(device)\n label_s = label_s.to(device)\n label_t = label_t.to(device)\n\n # measure data loading time\n data_time.update(time.time() - end)\n\n # compute output\n y_s, y_t = model(x)\n tgt_loss = F.cross_entropy(y_t, label_t)\n src_loss = kd(y_s, label_s)\n loss = tgt_loss + args.trade_off * src_loss\n\n # measure accuracy and record loss\n losses.update(tgt_loss.item(), x.size(0))\n losses_kd.update(src_loss.item(), x.size(0))\n cls_acc = accuracy(y_t, label_t)[0]\n cls_accs.update(cls_acc.item(), x.size(0))\n\n # compute gradient and do SGD step\n optimizer.zero_grad()\n loss.backward()\n optimizer.step()\n\n # measure elapsed time\n batch_time.update(time.time() - end)\n end = time.time()\n\n if i % args.print_freq == 0:\n progress.display(i)\n\n\nif __name__ == '__main__':\n parser = argparse.ArgumentParser(description='LWF (Learning without Forgetting) for Finetuning')\n # dataset parameters\n parser.add_argument('root', metavar='DIR',\n help='root path of dataset')\n parser.add_argument('-d', '--data', metavar='DATA')\n parser.add_argument('-sr', '--sample-rate', default=100, type=int,\n metavar='N',\n help='sample rate of training dataset (default: 100)')\n parser.add_argument('-sc', '--num-samples-per-classes', default=None, type=int,\n help='number of samples per classes.')\n parser.add_argument('--train-resizing', type=str, default='default')\n parser.add_argument('--val-resizing', type=str, default='default')\n parser.add_argument('--no-hflip', action='store_true', help='no random horizontal flipping during training')\n parser.add_argument('--color-jitter', action='store_true')\n # model parameters\n parser.add_argument('-a', '--arch', metavar='ARCH', default='resnet50',\n choices=utils.get_model_names(),\n help='backbone architecture: ' +\n ' | '.join(utils.get_model_names()) +\n ' (default: resnet50)')\n parser.add_argument('--no-pool', action='store_true',\n help='no pool layer after the feature extractor.')\n parser.add_argument('--finetune', action='store_true', help='whether use 10x smaller lr for backbone')\n parser.add_argument('--trade-off', default=4, type=float,\n metavar='P', help='weight of pretrained loss')\n parser.add_argument(\"-T\", type=float, default=3,\n help=\"temperature for knowledge distillation\")\n parser.add_argument('--pretrained', default=None,\n help=\"pretrained checkpoint of the backbone. \"\n \"(default: None, use the ImageNet supervised pretrained backbone)\")\n # training parameters\n parser.add_argument('-b', '--batch-size', default=48, type=int,\n metavar='N',\n help='mini-batch size (default: 48)')\n parser.add_argument('--lr', '--learning-rate', default=0.01, type=float,\n metavar='LR', help='initial learning rate', dest='lr')\n parser.add_argument('--lr-gamma', default=0.1, type=float, help='parameter for lr scheduler')\n parser.add_argument('--lr-decay-epochs', type=int, default=(12, ), nargs='+', help='epochs to decay lr')\n parser.add_argument('--momentum', default=0.9, type=float, metavar='M',\n help='momentum')\n parser.add_argument('--wd', '--weight-decay', default=0.0005, type=float,\n metavar='W', help='weight decay (default: 5e-4)')\n parser.add_argument('-j', '--workers', default=2, type=int, metavar='N',\n help='number of data loading workers (default: 4)')\n parser.add_argument('--epochs', default=20, type=int, metavar='N',\n help='number of total epochs to run')\n parser.add_argument('-i', '--iters-per-epoch', default=500, type=int,\n help='Number of iterations per epoch')\n parser.add_argument('-p', '--print-freq', default=100, type=int,\n metavar='N', help='print frequency (default: 100)')\n parser.add_argument('--seed', default=None, type=int,\n help='seed for initializing training. ')\n parser.add_argument(\"--log\", type=str, default='lwf',\n help=\"Where to save logs, checkpoints and debugging images.\")\n parser.add_argument(\"--phase\", type=str, default='train', choices=['train', 'test'],\n help=\"When phase is 'test', only test the model.\")\n args = parser.parse_args()\n main(args)\n\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/examples/task_adaptation/image_classification/lwf.sh",
"content": "#!/usr/bin/env bash\n# CUB-200-2011\nCUDA_VISIBLE_DEVICES=0 python lwf.py data/cub200 -d CUB200 -sr 100 --seed 0 --finetune --log logs/lwf/cub200_100 --lr 0.01\nCUDA_VISIBLE_DEVICES=0 python lwf.py data/cub200 -d CUB200 -sr 50 --seed 0 --finetune --log logs/lwf/cub200_50 --lr 0.001\nCUDA_VISIBLE_DEVICES=0 python lwf.py data/cub200 -d CUB200 -sr 30 --seed 0 --finetune --log logs/lwf/cub200_30 --lr 0.001\nCUDA_VISIBLE_DEVICES=0 python lwf.py data/cub200 -d CUB200 -sr 15 --seed 0 --finetune --log logs/lwf/cub200_15 --lr 0.001\n\n# Standford Cars\nCUDA_VISIBLE_DEVICES=0 python lwf.py data/stanford_cars -d StanfordCars -sr 100 --seed 0 --finetune --log logs/lwf/car_100 --lr 0.01\nCUDA_VISIBLE_DEVICES=0 python lwf.py data/stanford_cars -d StanfordCars -sr 50 --seed 0 --finetune --log logs/lwf/car_50 --lr 0.01\nCUDA_VISIBLE_DEVICES=0 python lwf.py data/stanford_cars -d StanfordCars -sr 30 --seed 0 --finetune --log logs/lwf/car_30 --lr 0.01\nCUDA_VISIBLE_DEVICES=0 python lwf.py data/stanford_cars -d StanfordCars -sr 15 --seed 0 --finetune --log logs/lwf/car_15 --lr 0.01\n\n# Aircrafts\nCUDA_VISIBLE_DEVICES=0 python lwf.py data/aircraft -d Aircraft -sr 100 --seed 0 --finetune --log logs/lwf/aircraft_100 --lr 0.001\nCUDA_VISIBLE_DEVICES=0 python lwf.py data/aircraft -d Aircraft -sr 50 --seed 0 --finetune --log logs/lwf/aircraft_50 --lr 0.001\nCUDA_VISIBLE_DEVICES=0 python lwf.py data/aircraft -d Aircraft -sr 30 --seed 0 --finetune --log logs/lwf/aircraft_30 --lr 0.001\nCUDA_VISIBLE_DEVICES=0 python lwf.py data/aircraft -d Aircraft -sr 15 --seed 0 --finetune --log logs/lwf/aircraft_15 --lr 0.001\n\n# Resisc45\nCUDA_VISIBLE_DEVICES=0 python lwf.py data/vtab/resisc45 -d Resisc45 -sc 10 --finetune --seed 0 \\\n --log logs/lwf/resisc45_10\nCUDA_VISIBLE_DEVICES=0 python lwf.py data/vtab/resisc45 -d Resisc45 -sc 20 --finetune --seed 0 \\\n --log logs/lwf/resisc45_20\nCUDA_VISIBLE_DEVICES=0 python lwf.py data/vtab/resisc45 -d Resisc45 -sc 40 --finetune --seed 0 \\\n --log logs/lwf/resisc45_40\nCUDA_VISIBLE_DEVICES=0 python lwf.py data/vtab/resisc45 -d Resisc45 -sc 80 --finetune --seed 0 \\\n --log logs/lwf/resisc45_80\n\n# Patch Camelyon\nCUDA_VISIBLE_DEVICES=0 python lwf.py data/vtab/patch_camelyon -d PatchCamelyon -sc 40 --finetune \\\n --seed 0 --log logs/lwf/patch_camelyon_40\nCUDA_VISIBLE_DEVICES=0 python lwf.py data/vtab/patch_camelyon -d PatchCamelyon -sc 80 --finetune \\\n --seed 0 --log logs/lwf/patch_camelyon_80\nCUDA_VISIBLE_DEVICES=0 python lwf.py data/vtab/patch_camelyon -d PatchCamelyon -sc 160 --finetune \\\n --seed 0 --log logs/lwf/patch_camelyon_160\nCUDA_VISIBLE_DEVICES=0 python lwf.py data/vtab/patch_camelyon -d PatchCamelyon -sc 320 --finetune \\\n --seed 0 --log logs/lwf/patch_camelyon_320"
},
{
"path": "domain_generalization/Transfer-Learning-Library/examples/task_adaptation/image_classification/requirements.txt",
"content": "timm"
},
{
"path": "domain_generalization/Transfer-Learning-Library/examples/task_adaptation/image_classification/stochnorm.py",
"content": "\"\"\"\n@author: Yifei Ji, Junguang Jiang\n@contact: jiyf990330@163.com, JiangJunguang1123@outlook.com\n\"\"\"\nimport random\nimport time\nimport warnings\nimport sys\nimport argparse\nimport shutil\n\nimport torch\nimport torch.nn as nn\nimport torch.backends.cudnn as cudnn\nfrom torch.optim import SGD\nfrom torch.utils.data import DataLoader\nimport torch.nn.functional as F\n\nsys.path.append('../../..')\nfrom talib.finetune.stochnorm import convert_model\nfrom common.modules.classifier import Classifier\nfrom common.utils.metric import accuracy\nfrom common.utils.meter import AverageMeter, ProgressMeter\nfrom common.utils.data import ForeverDataIterator\nfrom common.utils.logger import CompleteLogger\n\nsys.path.append('.')\nimport utils\n\ndevice = torch.device(\"cuda\" if torch.cuda.is_available() else \"cpu\")\n\n\ndef main(args: argparse.Namespace):\n logger = CompleteLogger(args.log, args.phase)\n print(args)\n\n if args.seed is not None:\n random.seed(args.seed)\n torch.manual_seed(args.seed)\n cudnn.deterministic = True\n warnings.warn('You have chosen to seed training. '\n 'This will turn on the CUDNN deterministic setting, '\n 'which can slow down your training considerably! '\n 'You may see unexpected behavior when restarting '\n 'from checkpoints.')\n\n cudnn.benchmark = True\n\n # Data loading code\n train_transform = utils.get_train_transform(args.train_resizing, not args.no_hflip, args.color_jitter)\n val_transform = utils.get_val_transform(args.val_resizing)\n print(\"train_transform: \", train_transform)\n print(\"val_transform: \", val_transform)\n\n train_dataset, val_dataset, num_classes = utils.get_dataset(args.data, args.root, train_transform,\n val_transform, args.sample_rate, args.num_samples_per_classes)\n train_loader = DataLoader(train_dataset, batch_size=args.batch_size, shuffle=True,\n num_workers=args.workers, drop_last=True)\n train_iter = ForeverDataIterator(train_loader)\n val_loader = DataLoader(val_dataset, batch_size=args.batch_size, shuffle=False, num_workers=args.workers)\n print(\"training dataset size: {} test dataset size: {}\".format(len(train_dataset), len(val_dataset)))\n\n # create model\n print(\"=> using pre-trained model '{}'\".format(args.arch))\n backbone = utils.get_model(args.arch, args.pretrained)\n pool_layer = nn.Identity() if args.no_pool else None\n classifier = Classifier(backbone, num_classes, pool_layer=pool_layer, finetune=args.finetune).to(device)\n classifier = convert_model(classifier, p=args.prob)\n\n # define optimizer and lr scheduler\n optimizer = SGD(classifier.get_parameters(args.lr), lr=args.lr, momentum=args.momentum, weight_decay=args.wd, nesterov=True)\n lr_scheduler = torch.optim.lr_scheduler.MultiStepLR(optimizer, args.lr_decay_epochs, gamma=args.lr_gamma)\n\n # resume from the best checkpoint\n if args.phase == 'test':\n checkpoint = torch.load(logger.get_checkpoint_path('best'), map_location='cpu')\n classifier.load_state_dict(checkpoint)\n acc1 = utils.validate(val_loader, classifier, args, device)\n print(acc1)\n return\n\n # start training\n best_acc1 = 0.0\n for epoch in range(args.epochs):\n print(lr_scheduler.get_lr())\n # train for one epoch\n train(train_iter, classifier, optimizer, epoch, args)\n lr_scheduler.step()\n\n # evaluate on validation set\n acc1 = utils.validate(val_loader, classifier, args, device)\n\n # remember best acc@1 and save checkpoint\n torch.save(classifier.state_dict(), logger.get_checkpoint_path('latest'))\n if acc1 > best_acc1:\n shutil.copy(logger.get_checkpoint_path('latest'), logger.get_checkpoint_path('best'))\n best_acc1 = max(acc1, best_acc1)\n\n print(\"best_acc1 = {:3.1f}\".format(best_acc1))\n logger.close()\n\n\ndef train(train_iter: ForeverDataIterator, model: Classifier, optimizer: SGD,\n epoch: int, args: argparse.Namespace):\n batch_time = AverageMeter('Time', ':4.2f')\n data_time = AverageMeter('Data', ':3.1f')\n losses = AverageMeter('Loss', ':3.2f')\n cls_accs = AverageMeter('Cls Acc', ':3.1f')\n\n progress = ProgressMeter(\n args.iters_per_epoch,\n [batch_time, data_time, losses, cls_accs],\n prefix=\"Epoch: [{}]\".format(epoch))\n\n # switch to train mode\n model.train()\n\n end = time.time()\n for i in range(args.iters_per_epoch):\n x, labels = next(train_iter)\n\n x = x.to(device)\n label = labels.to(device)\n\n # measure data loading time\n data_time.update(time.time() - end)\n\n # compute output\n y, f = model(x)\n\n cls_loss = F.cross_entropy(y, label)\n loss = cls_loss\n\n cls_acc = accuracy(y, label)[0]\n\n losses.update(loss.item(), x.size(0))\n cls_accs.update(cls_acc.item(), x.size(0))\n\n # compute gradient and do SGD step\n optimizer.zero_grad()\n loss.backward()\n optimizer.step()\n\n # measure elapsed time\n batch_time.update(time.time() - end)\n end = time.time()\n\n if i % args.print_freq == 0:\n progress.display(i)\n\n\nif __name__ == '__main__':\n parser = argparse.ArgumentParser(description='StochNorm for Finetuning')\n # dataset parameters\n parser.add_argument('root', metavar='DIR',\n help='root path of dataset')\n parser.add_argument('-d', '--data', metavar='DATA')\n parser.add_argument('-sr', '--sample-rate', default=100, type=int,\n metavar='N',\n help='sample rate of training dataset (default: 100)')\n parser.add_argument('-sc', '--num-samples-per-classes', default=None, type=int,\n help='number of samples per classes.')\n parser.add_argument('--train-resizing', type=str, default='default')\n parser.add_argument('--val-resizing', type=str, default='default')\n parser.add_argument('--no-hflip', action='store_true', help='no random horizontal flipping during training')\n parser.add_argument('--color-jitter', action='store_true')\n # model parameters\n parser.add_argument('-a', '--arch', metavar='ARCH', default='resnet50',\n choices=utils.get_model_names(),\n help='backbone architecture: ' +\n ' | '.join(utils.get_model_names()) +\n ' (default: resnet50)')\n parser.add_argument('--no-pool', action='store_true',\n help='no pool layer after the feature extractor.')\n parser.add_argument('--finetune', action='store_true', help='whether use 10x smaller lr for backbone')\n parser.add_argument('--prob', '--probability', default=0.5, type=float,\n metavar='P', help='Probability for StochNorm layers')\n parser.add_argument('--pretrained', default=None,\n help=\"pretrained checkpoint of the backbone. \"\n \"(default: None, use the ImageNet supervised pretrained backbone)\")\n # training parameters\n parser.add_argument('-b', '--batch-size', default=48, type=int,\n metavar='N',\n help='mini-batch size (default: 48)')\n parser.add_argument('--lr', '--learning-rate', default=0.01, type=float,\n metavar='LR', help='initial learning rate', dest='lr')\n parser.add_argument('--lr-gamma', default=0.1, type=float, help='parameter for lr scheduler')\n parser.add_argument('--lr-decay-epochs', type=int, default=(12, ), nargs='+', help='epochs to decay lr')\n parser.add_argument('--momentum', default=0.9, type=float, metavar='M',\n help='momentum')\n parser.add_argument('--wd', '--weight-decay', default=0.0005, type=float,\n metavar='W', help='weight decay (default: 5e-4)')\n parser.add_argument('-j', '--workers', default=2, type=int, metavar='N',\n help='number of data loading workers (default: 4)')\n parser.add_argument('--epochs', default=20, type=int, metavar='N',\n help='number of total epochs to run')\n parser.add_argument('-i', '--iters-per-epoch', default=500, type=int,\n help='Number of iterations per epoch')\n parser.add_argument('-p', '--print-freq', default=100, type=int,\n metavar='N', help='print frequency (default: 100)')\n parser.add_argument('--seed', default=None, type=int,\n help='seed for initializing training. ')\n parser.add_argument(\"--log\", type=str, default='stochnorm',\n help=\"Where to save logs, checkpoints and debugging images.\")\n parser.add_argument(\"--phase\", type=str, default='train', choices=['train', 'test'],\n help=\"When phase is 'test', only test the model.\")\n args = parser.parse_args()\n main(args)\n\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/examples/task_adaptation/image_classification/stochnorm.sh",
"content": "#!/usr/bin/env bash\n#CUB-200-2011\nCUDA_VISIBLE_DEVICES=0 python stochnorm.py data/cub200 -d CUB200 -sr 100 --seed 0 --finetune --log logs/stochnorm/cub200_100\nCUDA_VISIBLE_DEVICES=0 python stochnorm.py data/cub200 -d CUB200 -sr 50 --seed 0 --finetune --log logs/stochnorm/cub200_50\nCUDA_VISIBLE_DEVICES=0 python stochnorm.py data/cub200 -d CUB200 -sr 30 --seed 0 --finetune --log logs/stochnorm/cub200_30\nCUDA_VISIBLE_DEVICES=0 python stochnorm.py data/cub200 -d CUB200 -sr 15 --seed 0 --finetune --log logs/stochnorm/cub200_15\n\n# Standford Cars\nCUDA_VISIBLE_DEVICES=0 python stochnorm.py data/stanford_cars -d StanfordCars -sr 100 --seed 0 --finetune --log logs/stochnorm/car_100\nCUDA_VISIBLE_DEVICES=0 python stochnorm.py data/stanford_cars -d StanfordCars -sr 50 --seed 0 --finetune --log logs/stochnorm/car_50\nCUDA_VISIBLE_DEVICES=0 python stochnorm.py data/stanford_cars -d StanfordCars -sr 30 --seed 0 --finetune --log logs/stochnorm/car_30\nCUDA_VISIBLE_DEVICES=0 python stochnorm.py data/stanford_cars -d StanfordCars -sr 15 --seed 0 --finetune --log logs/stochnorm/car_15\n\n# Aircrafts\nCUDA_VISIBLE_DEVICES=0 python stochnorm.py data/aircraft -d Aircraft -sr 100 --seed 0 --finetune --log logs/stochnorm/aircraft_100\nCUDA_VISIBLE_DEVICES=0 python stochnorm.py data/aircraft -d Aircraft -sr 50 --seed 0 --finetune --log logs/stochnorm/aircraft_50\nCUDA_VISIBLE_DEVICES=0 python stochnorm.py data/aircraft -d Aircraft -sr 30 --seed 0 --finetune --log logs/stochnorm/aircraft_30\nCUDA_VISIBLE_DEVICES=0 python stochnorm.py data/aircraft -d Aircraft -sr 15 --seed 0 --finetune --log logs/stochnorm/aircraft_15\n\n# Resisc45\nCUDA_VISIBLE_DEVICES=0 python stochnorm.py data/vtab/resisc45 -d Resisc45 -sc 10 --finetune --seed 0 \\\n --log logs/stochnorm/resisc45_10\nCUDA_VISIBLE_DEVICES=0 python stochnorm.py data/vtab/resisc45 -d Resisc45 -sc 20 --finetune --seed 0 \\\n --log logs/stochnorm/resisc45_20\nCUDA_VISIBLE_DEVICES=0 python stochnorm.py data/vtab/resisc45 -d Resisc45 -sc 40 --finetune --seed 0 \\\n --log logs/stochnorm/resisc45_40\nCUDA_VISIBLE_DEVICES=0 python stochnorm.py data/vtab/resisc45 -d Resisc45 -sc 80 --finetune --seed 0 \\\n --log logs/stochnorm/resisc45_80\n\n# Patch Camelyon\nCUDA_VISIBLE_DEVICES=0 python stochnorm.py data/vtab/patch_camelyon -d PatchCamelyon -sc 40 --finetune \\\n --seed 0 --log logs/stochnorm/patch_camelyon_40\nCUDA_VISIBLE_DEVICES=0 python stochnorm.py data/vtab/patch_camelyon -d PatchCamelyon -sc 80 --finetune \\\n --seed 0 --log logs/stochnorm/patch_camelyon_80\nCUDA_VISIBLE_DEVICES=0 python stochnorm.py data/vtab/patch_camelyon -d PatchCamelyon -sc 160 --finetune \\\n --seed 0 --log logs/stochnorm/patch_camelyon_160\nCUDA_VISIBLE_DEVICES=0 python stochnorm.py data/vtab/patch_camelyon -d PatchCamelyon -sc 320 --finetune \\\n --seed 0 --log logs/stochnorm/patch_camelyon_320\n\n# MoCo (Unsupervised Pretraining)\n# CUB-200-2011\nCUDA_VISIBLE_DEVICES=0 python stochnorm.py data/cub200 -d CUB200 -sr 100 --seed 0 --lr 0.1 -i 2000 --lr-decay-epochs 3 6 9 --epochs 12 \\\n --log logs/moco_pretrain_stochnorm/cub200_100 --pretrained checkpoints/moco_v1_200ep_backbone.pth\nCUDA_VISIBLE_DEVICES=0 python stochnorm.py data/cub200 -d CUB200 -sr 50 --seed 0 --lr 0.1 -i 2000 --lr-decay-epochs 3 6 9 --epochs 12 \\\n --log logs/moco_pretrain_stochnorm/cub200_50 --pretrained checkpoints/moco_v1_200ep_backbone.pth\nCUDA_VISIBLE_DEVICES=0 python stochnorm.py data/cub200 -d CUB200 -sr 30 --seed 0 --lr 0.1 -i 2000 --lr-decay-epochs 3 6 9 --epochs 12 \\\n --log logs/moco_pretrain_stochnorm/cub200_30 --pretrained checkpoints/moco_v1_200ep_backbone.pth\nCUDA_VISIBLE_DEVICES=0 python stochnorm.py data/cub200 -d CUB200 -sr 15 --seed 0 --lr 0.1 -i 2000 --lr-decay-epochs 3 6 9 --epochs 12 \\\n --log logs/moco_pretrain_stochnorm/cub200_15 --pretrained checkpoints/moco_v1_200ep_backbone.pth\n\n# Standford Cars\nCUDA_VISIBLE_DEVICES=0 python stochnorm.py data/stanford_cars -d StanfordCars -sr 100 --seed 0 --lr 0.1 -i 2000 --lr-decay-epochs 3 6 9 --epochs 12 \\\n --log logs/moco_pretrain_stochnorm/cars_100 --pretrained checkpoints/moco_v1_200ep_backbone.pth\nCUDA_VISIBLE_DEVICES=0 python stochnorm.py data/stanford_cars -d StanfordCars -sr 50 --seed 0 --lr 0.1 -i 2000 --lr-decay-epochs 3 6 9 --epochs 12 \\\n --log logs/moco_pretrain_stochnorm/cars_50 --pretrained checkpoints/moco_v1_200ep_backbone.pth\nCUDA_VISIBLE_DEVICES=0 python stochnorm.py data/stanford_cars -d StanfordCars -sr 30 --seed 0 --lr 0.1 -i 2000 --lr-decay-epochs 3 6 9 --epochs 12 \\\n --log logs/moco_pretrain_stochnorm/cars_30 --pretrained checkpoints/moco_v1_200ep_backbone.pth\nCUDA_VISIBLE_DEVICES=0 python stochnorm.py data/stanford_cars -d StanfordCars -sr 15 --seed 0 --lr 0.1 -i 2000 --lr-decay-epochs 3 6 9 --epochs 12 \\\n --log logs/moco_pretrain_stochnorm/cars_15 --pretrained checkpoints/moco_v1_200ep_backbone.pth\n\n# Aircrafts\nCUDA_VISIBLE_DEVICES=0 python stochnorm.py data/aircraft -d Aircraft -sr 100 --seed 0 --lr 0.1 -i 2000 --lr-decay-epochs 3 6 9 --epochs 12 \\\n --log logs/moco_pretrain_stochnorm/aircraft_100 --pretrained checkpoints/moco_v1_200ep_backbone.pth\nCUDA_VISIBLE_DEVICES=0 python stochnorm.py data/aircraft -d Aircraft -sr 50 --seed 0 --lr 0.1 -i 2000 --lr-decay-epochs 3 6 9 --epochs 12 \\\n --log logs/moco_pretrain_stochnorm/aircraft_50 --pretrained checkpoints/moco_v1_200ep_backbone.pth\nCUDA_VISIBLE_DEVICES=0 python stochnorm.py data/aircraft -d Aircraft -sr 30 --seed 0 --lr 0.1 -i 2000 --lr-decay-epochs 3 6 9 --epochs 12 \\\n --log logs/moco_pretrain_stochnorm/aircraft_30 --pretrained checkpoints/moco_v1_200ep_backbone.pth\nCUDA_VISIBLE_DEVICES=0 python stochnorm.py data/aircraft -d Aircraft -sr 15 --seed 0 --lr 0.1 -i 2000 --lr-decay-epochs 3 6 9 --epochs 12 \\\n --log logs/moco_pretrain_stochnorm/aircraft_15 --pretrained checkpoints/moco_v1_200ep_backbone.pth\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/examples/task_adaptation/image_classification/utils.py",
"content": "\"\"\"\n@author: Junguang Jiang\n@contact: JiangJunguang1123@outlook.com\n\"\"\"\nimport time\nfrom PIL import Image\nimport timm\nimport numpy as np\nimport random\n\nimport torch\nimport torch.nn as nn\nimport torch.nn.functional as F\nfrom torch.utils.data import Subset\nimport torchvision.transforms as T\nfrom torch.optim import SGD, Adam\n\nimport common.vision.datasets as datasets\nimport common.vision.models as models\nfrom common.utils.metric import accuracy\nfrom common.utils.meter import AverageMeter, ProgressMeter\nfrom common.vision.transforms import Denormalize\n\n\ndef get_model_names():\n return sorted(\n name for name in models.__dict__\n if name.islower() and not name.startswith(\"__\")\n and callable(models.__dict__[name])\n ) + timm.list_models()\n\n\ndef get_model(model_name, pretrained_checkpoint=None):\n if model_name in models.__dict__:\n # load models from common.vision.models\n backbone = models.__dict__[model_name](pretrained=True)\n else:\n # load models from pytorch-image-models\n backbone = timm.create_model(model_name, pretrained=True)\n try:\n backbone.out_features = backbone.get_classifier().in_features\n backbone.reset_classifier(0, '')\n backbone.copy_head = backbone.get_classifier\n except:\n backbone.out_features = backbone.head.in_features\n backbone.head = nn.Identity()\n backbone.copy_head = lambda x: x.head\n if pretrained_checkpoint:\n print(\"=> loading pre-trained model from '{}'\".format(pretrained_checkpoint))\n pretrained_dict = torch.load(pretrained_checkpoint)\n backbone.load_state_dict(pretrained_dict, strict=False)\n return backbone\n\n\ndef get_dataset(dataset_name, root, train_transform, val_transform, sample_rate=100, num_samples_per_classes=None):\n \"\"\"\n When sample_rate < 100, e.g. sample_rate = 50, use 50% data to train the model.\n Otherwise,\n if num_samples_per_classes is not None, e.g. 5, then sample 5 images for each class, and use them to train the model;\n otherwise, keep all the data.\n \"\"\"\n dataset = datasets.__dict__[dataset_name]\n if sample_rate < 100:\n train_dataset = dataset(root=root, split='train', sample_rate=sample_rate, download=True, transform=train_transform)\n test_dataset = dataset(root=root, split='test', sample_rate=100, download=True, transform=val_transform)\n num_classes = train_dataset.num_classes\n else:\n train_dataset = dataset(root=root, split='train', download=True, transform=train_transform)\n test_dataset = dataset(root=root, split='test', download=True, transform=val_transform)\n num_classes = train_dataset.num_classes\n if num_samples_per_classes is not None:\n samples = list(range(len(train_dataset)))\n random.shuffle(samples)\n samples_len = min(num_samples_per_classes * num_classes, len(train_dataset))\n print(\"Origin dataset:\", len(train_dataset), \"Sampled dataset:\", samples_len, \"Ratio:\", float(samples_len) / len(train_dataset))\n train_dataset = Subset(train_dataset, samples[:samples_len])\n return train_dataset, test_dataset, num_classes\n\n\ndef validate(val_loader, model, args, device, visualize=None) -> float:\n batch_time = AverageMeter('Time', ':6.3f')\n losses = AverageMeter('Loss', ':.4e')\n top1 = AverageMeter('Acc@1', ':6.2f')\n progress = ProgressMeter(\n len(val_loader),\n [batch_time, losses, top1],\n prefix='Test: ')\n\n # switch to evaluate mode\n model.eval()\n\n with torch.no_grad():\n end = time.time()\n for i, (images, target) in enumerate(val_loader):\n images = images.to(device)\n target = target.to(device)\n\n # compute output\n output = model(images)\n loss = F.cross_entropy(output, target)\n\n # measure accuracy and record loss\n acc1, = accuracy(output, target, topk=(1, ))\n losses.update(loss.item(), images.size(0))\n top1.update(acc1.item(), images.size(0))\n\n # measure elapsed time\n batch_time.update(time.time() - end)\n end = time.time()\n\n if i % args.print_freq == 0:\n progress.display(i)\n if visualize is not None:\n visualize(images[0], \"val_{}\".format(i))\n\n print(' * Acc@1 {top1.avg:.3f}'.format(top1=top1))\n\n return top1.avg\n\n\ndef get_train_transform(resizing='default', random_horizontal_flip=True, random_color_jitter=False):\n \"\"\"\n resizing mode:\n - default: take a random resized crop of size 224 with scale in [0.2, 1.];\n - res: resize the image to 224;\n - res.|crop: resize the image to 256 and take a random crop of size 224;\n - res.sma|crop: resize the image keeping its aspect ratio such that the\n smaller side is 256, then take a random crop of size 224;\n – inc.crop: “inception crop” from (Szegedy et al., 2015);\n – cif.crop: resize the image to 224, zero-pad it by 28 on each side, then take a random crop of size 224.\n \"\"\"\n if resizing == 'default':\n transform = T.RandomResizedCrop(224, scale=(0.2, 1.))\n elif resizing == 'res.':\n transform = T.Resize((224, 224))\n elif resizing == 'res.|crop':\n transform = T.Compose([\n T.Resize((256, 256)),\n T.RandomCrop(224)\n ])\n elif resizing == \"res.sma|crop\":\n transform = T.Compose([\n T.Resize(256),\n T.RandomCrop(224)\n ])\n elif resizing == 'inc.crop':\n transform = T.RandomResizedCrop(224)\n elif resizing == 'cif.crop':\n transform = T.Compose([\n T.Resize((224, 224)),\n T.Pad(28),\n T.RandomCrop(224),\n ])\n else:\n raise NotImplementedError(resizing)\n transforms = [transform]\n if random_horizontal_flip:\n transforms.append(T.RandomHorizontalFlip())\n if random_color_jitter:\n transforms.append(T.ColorJitter(brightness=0.5, contrast=0.5, saturation=0.5, hue=0.5))\n transforms.extend([\n T.ToTensor(),\n T.Normalize(mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225])\n ])\n return T.Compose(transforms)\n\n\ndef get_val_transform(resizing='default'):\n \"\"\"\n resizing mode:\n - default: resize the image to 256 and take the center crop of size 224;\n – res.: resize the image to 224\n – res.|crop: resize the image such that the smaller side is of size 256 and\n then take a central crop of size 224.\n \"\"\"\n if resizing == 'default':\n transform = T.Compose([\n T.Resize(256),\n T.CenterCrop(224),\n ])\n elif resizing == 'res.':\n transform = T.Resize((224, 224))\n elif resizing == 'res.|crop':\n transform = T.Compose([\n T.Resize((256, 256)),\n T.CenterCrop(224),\n ])\n else:\n raise NotImplementedError(resizing)\n return T.Compose([\n transform,\n T.ToTensor(),\n T.Normalize(mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225])\n ])\n\n\ndef get_optimizer(optimizer_name, params, lr, wd, momentum):\n '''\n Args:\n optimizer_name:\n - SGD\n - Adam\n params: iterable of parameters to optimize or dicts defining parameter groups\n lr: learning rate\n weight_decay: weight decay\n momentum: momentum factor for SGD\n '''\n if optimizer_name == 'SGD':\n optimizer = SGD(params=params, lr=lr, momentum=momentum, weight_decay=wd, nesterov=True)\n elif optimizer_name == 'Adam':\n optimizer = Adam(params=params, lr=lr, weight_decay=wd)\n else:\n raise NotImplementedError(optimizer_name)\n return optimizer\n\n\ndef visualize(image, filename):\n \"\"\"\n Args:\n image (tensor): 3 x H x W\n filename: filename of the saving image\n \"\"\"\n image = image.detach().cpu()\n image = Denormalize(mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225])(image)\n image = image.numpy().transpose((1, 2, 0)) * 255\n Image.fromarray(np.uint8(image)).save(filename)\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/requirements.txt",
"content": "torch>=1.7.0\ntorchvision\nnumpy\nprettytable\ntqdm\nscikit-learn\nwebcolors\nmatplotlib"
},
{
"path": "domain_generalization/Transfer-Learning-Library/setup.py",
"content": "from setuptools import setup, find_packages\nimport re\nfrom os import path\n\nhere = path.abspath(path.dirname(__file__))\n\n# Get the version string\nwith open(path.join(here, 'dalib', '__init__.py')) as f:\n version = re.search(r'__version__ = \\'(.*?)\\'', f.read()).group(1)\n\nif __name__ == '__main__':\n setup(\n name=\"dalib\", # Replace with your own username\n version=version,\n author=\"THUML\",\n author_email=\"JiangJunguang1123@outlook.com\",\n keywords=\"domain adaptation, task adaptation, domain generalization, \"\n \"transfer learning, deep learning, pytorch\",\n description=\"A Transfer Learning Library for Domain Adaptation, Task Adaptation, and Domain Generalization\",\n long_description=open('README.md', encoding='utf8').read(),\n long_description_content_type=\"text/markdown\",\n url=\"https://github.com/thuml/Transfer-Learning-Library\",\n packages=find_packages(exclude=['docs', 'examples']),\n classifiers=[\n # How mature is this project? Common values are\n # 3 - Alpha\n # 4 - Beta\n # 5 - Production/Stable\n 'Development Status :: 3 - Alpha',\n # Indicate who your project is intended for\n 'Intended Audience :: Science/Research',\n 'Topic :: Scientific/Engineering :: Artificial Intelligence',\n 'Topic :: Software Development :: Libraries :: Python Modules',\n # Pick your license as you wish (should match \"license\" above)\n 'License :: OSI Approved :: MIT License',\n # Specify the Python versions you support here. In particular, ensure\n # that you indicate whether you support Python 2, Python 3 or both.\n 'Programming Language :: Python :: 3.6',\n 'Programming Language :: Python :: 3.7',\n 'Programming Language :: Python :: 3.8',\n ],\n python_requires='>=3.6',\n install_requires=[\n 'torch>=1.7.0',\n 'torchvision>=0.5.0',\n 'numpy',\n 'prettytable',\n 'tqdm',\n 'scikit-learn',\n 'webcolors',\n 'matplotlib'\n ],\n extras_require={\n 'dev': [\n 'Sphinx',\n 'sphinx_rtd_theme',\n ]\n },\n )\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/talib/__init__.py",
"content": "__all__ = ['finetune']\n\n__version__ = '0.3'"
},
{
"path": "domain_generalization/Transfer-Learning-Library/talib/finetune/__init__.py",
"content": "from .stochnorm import *\nfrom .bss import *\nfrom .co_tuning import *\nfrom .delta import *\nfrom .bi_tuning import *\n\n__all__ = ['stochnorm', 'bss', 'co_tuning', 'delta', 'bi_tuning']"
},
{
"path": "domain_generalization/Transfer-Learning-Library/talib/finetune/bi_tuning.py",
"content": "\"\"\"\n@author: Junguang Jiang\n@contact: JiangJunguang1123@outlook.com\n\"\"\"\nimport torch\nimport torch.nn as nn\nfrom torch.nn.functional import normalize\nfrom common.modules.classifier import Classifier as ClassifierBase\n\n\nclass Classifier(ClassifierBase):\n \"\"\"Classifier class for Bi-Tuning.\n\n Args:\n backbone (torch.nn.Module): Any backbone to extract 2-d features from data\n num_classes (int): Number of classes\n projection_dim (int, optional): Dimension of the projector head. Default: 128\n finetune (bool): Whether finetune the classifier or train from scratch. Default: True\n\n .. note::\n The learning rate of this classifier is set 10 times to that of the feature extractor for better accuracy\n by default. If you have other optimization strategies, please over-ride :meth:`~Classifier.get_parameters`.\n\n Inputs:\n - x (tensor): input data fed to `backbone`\n\n Outputs:\n In the training mode,\n - y: classifier's predictions\n - z: projector's predictions\n - hn: normalized features after `bottleneck` layer and before `head` layer\n In the eval mode,\n - y: classifier's predictions\n\n Shape:\n - Inputs: (minibatch, *) where * means, any number of additional dimensions\n - y: (minibatch, `num_classes`)\n - z: (minibatch, `projection_dim`)\n - hn: (minibatch, `features_dim`)\n\n \"\"\"\n\n def __init__(self, backbone: nn.Module, num_classes: int, projection_dim=128, finetune=True, pool_layer=None):\n head = nn.Linear(backbone.out_features, num_classes)\n head.weight.data.normal_(0, 0.01)\n head.bias.data.fill_(0.0)\n super(Classifier, self).__init__(backbone, num_classes=num_classes, head=head, finetune=finetune,\n pool_layer=pool_layer)\n self.projector = nn.Linear(backbone.out_features, projection_dim)\n self.projection_dim = projection_dim\n\n def forward(self, x: torch.Tensor):\n batch_size = x.shape[0]\n h = self.backbone(x)\n h = self.pool_layer(h)\n h = self.bottleneck(h)\n y = self.head(h)\n z = normalize(self.projector(h), dim=1)\n hn = torch.cat([h, torch.ones(batch_size, 1, dtype=torch.float).to(h.device)], dim=1)\n hn = normalize(hn, dim=1)\n if self.training:\n return y, z, hn\n else:\n return y\n\n def get_parameters(self, base_lr=1.0):\n \"\"\"A parameter list which decides optimization hyper-parameters,\n such as the relative learning rate of each layer\n \"\"\"\n params = [\n {\"params\": self.backbone.parameters(), \"lr\": 0.1 * base_lr if self.finetune else 1.0 * base_lr},\n {\"params\": self.bottleneck.parameters(), \"lr\": 1.0 * base_lr},\n {\"params\": self.head.parameters(), \"lr\": 1.0 * base_lr},\n {\"params\": self.projector.parameters(), \"lr\": 0.1 * base_lr if self.finetune else 1.0 * base_lr},\n ]\n\n return params\n\n\nclass BiTuning(nn.Module):\n \"\"\"\n Bi-Tuning Module in `Bi-tuning of Pre-trained Representations `_.\n\n Args:\n encoder_q (Classifier): Query encoder.\n encoder_k (Classifier): Key encoder.\n num_classes (int): Number of classes\n K (int): Queue size. Default: 40\n m (float): Momentum coefficient. Default: 0.999\n T (float): Temperature. Default: 0.07\n\n Inputs:\n - im_q (tensor): input data fed to `encoder_q`\n - im_k (tensor): input data fed to `encoder_k`\n - labels (tensor): classification labels of input data\n\n Outputs: y_q, logits_z, logits_y, labels_c\n - y_q: query classifier's predictions\n - logits_z: projector's predictions on both positive and negative samples\n - logits_y: classifier's predictions on both positive and negative samples\n - labels_c: contrastive labels\n\n Shape:\n - im_q, im_k: (minibatch, *) where * means, any number of additional dimensions\n - labels: (minibatch, )\n - y_q: (minibatch, `num_classes`)\n - logits_z: (minibatch, 1 + `num_classes` x `K`, `projection_dim`)\n - logits_y: (minibatch, 1 + `num_classes` x `K`, `num_classes`)\n - labels_c: (minibatch, 1 + `num_classes` x `K`)\n \"\"\"\n\n def __init__(self, encoder_q: Classifier, encoder_k: Classifier, num_classes, K=40, m=0.999, T=0.07):\n super(BiTuning, self).__init__()\n self.K = K\n self.m = m\n self.T = T\n self.num_classes = num_classes\n\n # create the encoders\n # num_classes is the output fc dimension\n self.encoder_q = encoder_q\n self.encoder_k = encoder_k\n\n for param_q, param_k in zip(self.encoder_q.parameters(), self.encoder_k.parameters()):\n param_k.data.copy_(param_q.data) # initialize\n param_k.requires_grad = False # not update by gradient\n\n # create the queue\n self.register_buffer(\"queue_h\", torch.randn(encoder_q.features_dim + 1, num_classes, K))\n self.register_buffer(\"queue_z\", torch.randn(encoder_q.projection_dim, num_classes, K))\n self.queue_h = normalize(self.queue_h, dim=0)\n self.queue_z = normalize(self.queue_z, dim=0)\n\n self.register_buffer(\"queue_ptr\", torch.zeros(num_classes, dtype=torch.long))\n\n @torch.no_grad()\n def _momentum_update_key_encoder(self):\n \"\"\"\n Momentum update of the key encoder\n \"\"\"\n for param_q, param_k in zip(self.encoder_q.parameters(), self.encoder_k.parameters()):\n param_k.data = param_k.data * self.m + param_q.data * (1. - self.m)\n\n @torch.no_grad()\n def _dequeue_and_enqueue(self, h, z, label):\n batch_size = h.shape[0]\n assert self.K % batch_size == 0 # for simplicity\n\n ptr = int(self.queue_ptr[label])\n # replace the keys at ptr (dequeue and enqueue)\n self.queue_h[:, label, ptr: ptr + batch_size] = h.T\n self.queue_z[:, label, ptr: ptr + batch_size] = z.T\n\n # move pointer\n self.queue_ptr[label] = (ptr + batch_size) % self.K\n\n def forward(self, im_q, im_k, labels):\n batch_size = im_q.size(0)\n device = im_q.device\n # compute query features\n y_q, z_q, h_q = self.encoder_q(im_q)\n\n # compute key features\n with torch.no_grad(): # no gradient to keys\n self._momentum_update_key_encoder() # update the key encoder\n y_k, z_k, h_k = self.encoder_k(im_k)\n\n # compute logits for projection z\n # current positive logits: Nx1\n logits_z_cur = torch.einsum('nc,nc->n', [z_q, z_k]).unsqueeze(-1)\n queue_z = self.queue_z.clone().detach().to(device)\n # positive logits: N x K\n logits_z_pos = torch.Tensor([]).to(device)\n # negative logits: N x ((C-1) x K)\n logits_z_neg = torch.Tensor([]).to(device)\n\n for i in range(batch_size):\n c = labels[i]\n pos_samples = queue_z[:, c, :] # D x K\n neg_samples = torch.cat([queue_z[:, 0: c, :], queue_z[:, c + 1:, :]], dim=1).flatten(\n start_dim=1) # D x ((C-1)xK)\n ith_pos = torch.einsum('nc,ck->nk', [z_q[i: i + 1], pos_samples]) # 1 x D\n ith_neg = torch.einsum('nc,ck->nk', [z_q[i: i + 1], neg_samples]) # 1 x ((C-1)xK)\n logits_z_pos = torch.cat((logits_z_pos, ith_pos), dim=0)\n logits_z_neg = torch.cat((logits_z_neg, ith_neg), dim=0)\n\n self._dequeue_and_enqueue(h_k[i:i + 1], z_k[i:i + 1], labels[i])\n\n logits_z = torch.cat([logits_z_cur, logits_z_pos, logits_z_neg], dim=1) # Nx(1+C*K)\n\n # apply temperature\n logits_z /= self.T\n logits_z = nn.LogSoftmax(dim=1)(logits_z)\n\n # compute logits for classification y\n w = torch.cat([self.encoder_q.head.weight.data, self.encoder_q.head.bias.data.unsqueeze(-1)], dim=1)\n w = normalize(w, dim=1) # C x F\n\n # current positive logits: Nx1\n logits_y_cur = torch.einsum('nk,kc->nc', [h_q, w.T]) # N x C\n queue_y = self.queue_h.clone().detach().to(device).flatten(start_dim=1).T # (C * K) x F\n logits_y_queue = torch.einsum('nk,kc->nc', [queue_y, w.T]).reshape(self.num_classes, -1,\n self.num_classes) # C x K x C\n\n logits_y = torch.Tensor([]).to(device)\n\n for i in range(batch_size):\n c = labels[i]\n # calculate the ith sample in the batch\n cur_sample = logits_y_cur[i:i + 1, c] # 1\n pos_samples = logits_y_queue[c, :, c] # K\n neg_samples = torch.cat([logits_y_queue[0: c, :, c], logits_y_queue[c + 1:, :, c]], dim=0).view(\n -1) # (C-1)*K\n\n ith = torch.cat([cur_sample, pos_samples, neg_samples]) # 1+C*K\n logits_y = torch.cat([logits_y, ith.unsqueeze(dim=0)], dim=0)\n\n logits_y /= self.T\n logits_y = nn.LogSoftmax(dim=1)(logits_y)\n\n # contrastive labels\n labels_c = torch.zeros([batch_size, self.K * self.num_classes + 1]).to(device)\n labels_c[:, 0:self.K + 1].fill_(1.0 / (self.K + 1))\n return y_q, logits_z, logits_y, labels_c\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/talib/finetune/bss.py",
"content": "\"\"\"\n@author: Yifei Ji\n@contact: jiyf990330@163.com\n\"\"\"\nimport torch\nimport torch.nn as nn\n\n__all__ = ['BatchSpectralShrinkage']\n\n\nclass BatchSpectralShrinkage(nn.Module):\n r\"\"\"\n The regularization term in `Catastrophic Forgetting Meets Negative Transfer:\n Batch Spectral Shrinkage for Safe Transfer Learning (NIPS 2019) `_.\n\n\n The BSS regularization of feature matrix :math:`F` can be described as:\n\n .. math::\n L_{bss}(F) = \\sum_{i=1}^{k} \\sigma_{-i}^2 ,\n\n where :math:`k` is the number of singular values to be penalized, :math:`\\sigma_{-i}` is the :math:`i`-th smallest singular value of feature matrix :math:`F`.\n\n All the singular values of feature matrix :math:`F` are computed by `SVD`:\n\n .. math::\n F = U\\Sigma V^T,\n\n where the main diagonal elements of the singular value matrix :math:`\\Sigma` is :math:`[\\sigma_1, \\sigma_2, ..., \\sigma_b]`.\n\n\n Args:\n k (int): The number of singular values to be penalized. Default: 1\n\n Shape:\n - Input: :math:`(b, |\\mathcal{f}|)` where :math:`b` is the batch size and :math:`|\\mathcal{f}|` is feature dimension.\n - Output: scalar.\n\n \"\"\"\n def __init__(self, k=1):\n super(BatchSpectralShrinkage, self).__init__()\n self.k = k\n\n def forward(self, feature):\n result = 0\n u, s, v = torch.svd(feature.t())\n num = s.size(0)\n for i in range(self.k):\n result += torch.pow(s[num-1-i], 2)\n return result\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/talib/finetune/co_tuning.py",
"content": "\"\"\"\n@author: Yifei Ji\n@contact: jiyf990330@163.com\n\"\"\"\nfrom typing import Tuple, Optional, List, Dict\nimport os\nimport torch\nimport torch.nn as nn\nimport numpy as np\nimport torch.nn.functional as F\nimport tqdm\nfrom .lwf import Classifier as ClassifierBase\n\n__all__ = ['Classifier', 'CoTuningLoss', 'Relationship']\n\n\nclass CoTuningLoss(nn.Module):\n \"\"\"\n The Co-Tuning loss in `Co-Tuning for Transfer Learning (NIPS 2020)\n `_.\n\n Inputs:\n - input: p(y_s) predicted by source classifier.\n - target: p(y_s|y_t), where y_t is the ground truth class label in target dataset.\n\n Shape:\n - input: (b, N_p), where b is the batch size and N_p is the number of classes in source dataset\n - target: (b, N_p), where b is the batch size and N_p is the number of classes in source dataset\n - Outputs: scalar.\n \"\"\"\n\n def __init__(self):\n super(CoTuningLoss, self).__init__()\n\n def forward(self, input: torch.Tensor, target: torch.Tensor) -> torch.Tensor:\n y = - target * F.log_softmax(input, dim=-1)\n y = torch.mean(torch.sum(y, dim=-1))\n return y\n\n\nclass Relationship(object):\n \"\"\"Learns the category relationship p(y_s|y_t) between source dataset and target dataset.\n\n Args:\n data_loader (torch.utils.data.DataLoader): A data loader of target dataset.\n classifier (torch.nn.Module): A classifier for Co-Tuning.\n device (torch.nn.Module): The device to run classifier.\n cache (str, optional): Path to find and save the relationship file.\n\n \"\"\"\n def __init__(self, data_loader, classifier, device, cache=None):\n super(Relationship, self).__init__()\n self.data_loader = data_loader\n self.classifier = classifier\n self.device = device\n if cache is None or not os.path.exists(cache):\n source_predictions, target_labels = self.collect_labels()\n self.relationship = self.get_category_relationship(source_predictions, target_labels)\n if cache is not None:\n np.save(cache, self.relationship)\n else:\n self.relationship = np.load(cache)\n\n def __getitem__(self, category):\n return self.relationship[category]\n\n def collect_labels(self):\n \"\"\"\n Collects predictions of target dataset by source model and corresponding ground truth class labels.\n\n Returns:\n - source_probabilities, [N, N_p], where N_p is the number of classes in source dataset\n - target_labels, [N], where 0 <= each number < N_t, and N_t is the number of classes in target dataset\n \"\"\"\n\n print(\"Collecting labels to calculate relationship\")\n source_predictions = []\n target_labels = []\n\n self.classifier.eval()\n with torch.no_grad():\n for i, (x, label) in enumerate(tqdm.tqdm(self.data_loader)):\n x = x.to(self.device)\n y_s = self.classifier(x)\n\n source_predictions.append(F.softmax(y_s, dim=1).detach().cpu().numpy())\n target_labels.append(label)\n\n return np.concatenate(source_predictions, 0), np.concatenate(target_labels, 0)\n\n def get_category_relationship(self, source_probabilities, target_labels):\n \"\"\"\n The direct approach of learning category relationship p(y_s | y_t).\n\n Args:\n source_probabilities (numpy.array): [N, N_p], where N_p is the number of classes in source dataset\n target_labels (numpy.array): [N], where 0 <= each number < N_t, and N_t is the number of classes in target dataset\n\n Returns:\n Conditional probability, [N_c, N_p] matrix representing the conditional probability p(pre-trained class | target_class)\n \"\"\"\n N_t = np.max(target_labels) + 1 # the number of target classes\n conditional = []\n for i in range(N_t):\n this_class = source_probabilities[target_labels == i]\n average = np.mean(this_class, axis=0, keepdims=True)\n conditional.append(average)\n return np.concatenate(conditional)\n\n\nclass Classifier(ClassifierBase):\n \"\"\"A Classifier used in `Co-Tuning for Transfer Learning (NIPS 2020)\n `_..\n\n Args:\n backbone (torch.nn.Module): Any backbone to extract 2-d features from data.\n num_classes (int): Number of classes.\n head_source (torch.nn.Module): Classifier head of source model.\n head_target (torch.nn.Module, optional): Any classifier head. Use :class:`torch.nn.Linear` by default\n finetune (bool): Whether finetune the classifier or train from scratch. Default: True\n\n\n Inputs:\n - x (tensor): input data fed to backbone\n\n Outputs:\n - y_s: predictions of source classifier head\n - y_t: predictions of target classifier head\n\n Shape:\n - Inputs: (b, *) where b is the batch size and * means any number of additional dimensions\n - y_s: (b, N), where b is the batch size and N is the number of classes\n - y_t: (b, N), where b is the batch size and N is the number of classes\n\n \"\"\"\n def __init__(self, backbone: nn.Module, num_classes: int, head_source, **kwargs):\n super(Classifier, self).__init__(backbone, num_classes, head_source, **kwargs)\n\n def get_parameters(self, base_lr=1.0) -> List[Dict]:\n \"\"\"A parameter list which decides optimization hyper-parameters,\n such as the relative learning rate of each layer\n \"\"\"\n params = [\n {\"params\": self.backbone.parameters(), \"lr\": 0.1 * base_lr if self.finetune else 1.0 * base_lr},\n {\"params\": self.head_source.parameters(), \"lr\": 0.1 * base_lr if self.finetune else 1.0 * base_lr},\n {\"params\": self.bottleneck.parameters(), \"lr\": 1.0 * base_lr},\n {\"params\": self.head_target.parameters(), \"lr\": 1.0 * base_lr},\n ]\n return params\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/talib/finetune/delta.py",
"content": "\"\"\"\n@author: Yifei Ji\n@contact: jiyf990330@163.com\n\"\"\"\nimport torch\nimport torch.nn as nn\n\nimport functools\nfrom collections import OrderedDict\n\n\nclass L2Regularization(nn.Module):\n r\"\"\"The L2 regularization of parameters :math:`w` can be described as:\n\n .. math::\n {\\Omega} (w) = \\dfrac{1}{2} \\Vert w\\Vert_2^2 ,\n\n Args:\n model (torch.nn.Module): The model to apply L2 penalty.\n\n Shape:\n - Output: scalar.\n \"\"\"\n def __init__(self, model: nn.Module):\n super(L2Regularization, self).__init__()\n self.model = model\n\n def forward(self):\n output = 0.0\n for param in self.model.parameters():\n output += 0.5 * torch.norm(param) ** 2\n return output\n\n\nclass SPRegularization(nn.Module):\n r\"\"\"\n The SP (Starting Point) regularization from `Explicit inductive bias for transfer learning with convolutional networks\n (ICML 2018) `_\n\n The SP regularization of parameters :math:`w` can be described as:\n\n .. math::\n {\\Omega} (w) = \\dfrac{1}{2} \\Vert w-w_0\\Vert_2^2 ,\n\n where :math:`w_0` is the parameter vector of the model pretrained on the source problem, acting as the starting point (SP) in fine-tuning.\n\n\n Args:\n source_model (torch.nn.Module): The source (starting point) model.\n target_model (torch.nn.Module): The target (fine-tuning) model.\n\n Shape:\n - Output: scalar.\n \"\"\"\n def __init__(self, source_model: nn.Module, target_model: nn.Module):\n super(SPRegularization, self).__init__()\n self.target_model = target_model\n self.source_weight = {}\n for name, param in source_model.named_parameters():\n self.source_weight[name] = param.detach()\n\n def forward(self):\n output = 0.0\n for name, param in self.target_model.named_parameters():\n output += 0.5 * torch.norm(param - self.source_weight[name]) ** 2\n return output\n\n\nclass BehavioralRegularization(nn.Module):\n r\"\"\"\n The behavioral regularization from `DELTA:DEep Learning Transfer using Feature Map with Attention\n for convolutional networks (ICLR 2019) `_\n\n It can be described as:\n\n .. math::\n {\\Omega} (w) = \\sum_{j=1}^{N} \\Vert FM_j(w, \\boldsymbol x)-FM_j(w^0, \\boldsymbol x)\\Vert_2^2 ,\n\n where :math:`w^0` is the parameter vector of the model pretrained on the source problem, acting as the starting point (SP) in fine-tuning,\n :math:`FM_j(w, \\boldsymbol x)` is feature maps generated from the :math:`j`-th layer of the model parameterized with :math:`w`, given the input :math:`\\boldsymbol x`.\n\n\n Inputs:\n layer_outputs_source (OrderedDict): The dictionary for source model, where the keys are layer names and the values are feature maps correspondingly.\n\n layer_outputs_target (OrderedDict): The dictionary for target model, where the keys are layer names and the values are feature maps correspondingly.\n\n Shape:\n - Output: scalar.\n\n \"\"\"\n def __init__(self):\n super(BehavioralRegularization, self).__init__()\n\n def forward(self, layer_outputs_source, layer_outputs_target):\n output = 0.0\n for fm_src, fm_tgt in zip(layer_outputs_source.values(), layer_outputs_target.values()):\n output += 0.5 * (torch.norm(fm_tgt - fm_src.detach()) ** 2)\n return output\n\n\nclass AttentionBehavioralRegularization(nn.Module):\n r\"\"\"\n The behavioral regularization with attention from `DELTA:DEep Learning Transfer using Feature Map with Attention\n for convolutional networks (ICLR 2019) `_\n\n It can be described as:\n\n .. math::\n {\\Omega} (w) = \\sum_{j=1}^{N} W_j(w) \\Vert FM_j(w, \\boldsymbol x)-FM_j(w^0, \\boldsymbol x)\\Vert_2^2 ,\n\n where\n :math:`w^0` is the parameter vector of the model pretrained on the source problem, acting as the starting point (SP) in fine-tuning.\n :math:`FM_j(w, \\boldsymbol x)` is feature maps generated from the :math:`j`-th layer of the model parameterized with :math:`w`, given the input :math:`\\boldsymbol x`.\n :math:`W_j(w)` is the channel attention of the :math:`j`-th layer of the model parameterized with :math:`w`.\n\n Args:\n channel_attention (list): The channel attentions of feature maps generated by each selected layer. For the layer with C channels, the channel attention is a tensor of shape [C].\n\n Inputs:\n layer_outputs_source (OrderedDict): The dictionary for source model, where the keys are layer names and the values are feature maps correspondingly.\n\n layer_outputs_target (OrderedDict): The dictionary for target model, where the keys are layer names and the values are feature maps correspondingly.\n\n Shape:\n - Output: scalar.\n\n \"\"\"\n def __init__(self, channel_attention):\n super(AttentionBehavioralRegularization, self).__init__()\n self.channel_attention = channel_attention\n\n def forward(self, layer_outputs_source, layer_outputs_target):\n output = 0.0\n for i, (fm_src, fm_tgt) in enumerate(zip(layer_outputs_source.values(), layer_outputs_target.values())):\n b, c, h, w = fm_src.shape\n fm_src = fm_src.reshape(b, c, h * w)\n fm_tgt = fm_tgt.reshape(b, c, h * w)\n\n distance = torch.norm(fm_tgt - fm_src.detach(), 2, 2)\n distance = c * torch.mul(self.channel_attention[i], distance ** 2) / (h * w)\n output += 0.5 * torch.sum(distance)\n\n return output\n\n\ndef get_attribute(obj, attr, *args):\n def _getattr(obj, attr):\n return getattr(obj, attr, *args)\n return functools.reduce(_getattr, [obj] + attr.split('.'))\n\n\nclass IntermediateLayerGetter:\n r\"\"\"\n Wraps a model to get intermediate output values of selected layers.\n\n Args:\n model (torch.nn.Module): The model to collect intermediate layer feature maps.\n return_layers (list): The names of selected modules to return the output.\n keep_output (bool): If True, `model_output` contains the final model's output, else return None. Default: True\n\n Returns:\n - An OrderedDict of intermediate outputs. The keys are selected layer names in `return_layers` and the values are the feature map outputs. The order is the same as `return_layers`.\n - The model's final output. If `keep_output` is False, return None.\n\n \"\"\"\n def __init__(self, model, return_layers, keep_output=True):\n self._model = model\n self.return_layers = return_layers\n self.keep_output = keep_output\n\n def __call__(self, *args, **kwargs):\n ret = OrderedDict()\n handles = []\n for name in self.return_layers:\n layer = get_attribute(self._model, name)\n def hook(module, input, output, name=name):\n ret[name] = output\n try:\n h = layer.register_forward_hook(hook)\n except AttributeError as e:\n raise AttributeError(f'Module {name} not found')\n handles.append(h)\n\n if self.keep_output:\n output = self._model(*args, **kwargs)\n else:\n self._model(*args, **kwargs)\n output = None\n\n for h in handles:\n h.remove()\n\n return ret, output\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/talib/finetune/lwf.py",
"content": "\"\"\"\n@author: Junguang Jiang\n@contact: JiangJunguang1123@outlook.com\n\"\"\"\nfrom typing import Optional, List, Dict\nimport torch\nimport torch.nn as nn\nimport tqdm\n\n\ndef collect_pretrain_labels(data_loader, classifier, device):\n source_predictions = []\n\n classifier.eval()\n with torch.no_grad():\n for i, (x, label) in enumerate(tqdm.tqdm(data_loader)):\n x = x.to(device)\n y_s = classifier(x)\n source_predictions.append(y_s.detach().cpu())\n return torch.cat(source_predictions, dim=0)\n\n\nclass Classifier(nn.Module):\n \"\"\"A Classifier used in `Learning Without Forgetting (ECCV 2016)\n `_..\n\n Args:\n backbone (torch.nn.Module): Any backbone to extract 2-d features from data.\n num_classes (int): Number of classes.\n head_source (torch.nn.Module): Classifier head of source model.\n head_target (torch.nn.Module, optional): Any classifier head. Use :class:`torch.nn.Linear` by default\n finetune (bool): Whether finetune the classifier or train from scratch. Default: True\n\n\n Inputs:\n - x (tensor): input data fed to backbone\n\n Outputs:\n - y_s: predictions of source classifier head\n - y_t: predictions of target classifier head\n\n Shape:\n - Inputs: (b, *) where b is the batch size and * means any number of additional dimensions\n - y_s: (b, N), where b is the batch size and N is the number of classes\n - y_t: (b, N), where b is the batch size and N is the number of classes\n\n \"\"\"\n def __init__(self, backbone: nn.Module, num_classes: int, head_source,\n head_target: Optional[nn.Module] = None, bottleneck: Optional[nn.Module] = None,\n bottleneck_dim: Optional[int] = -1, finetune=True, pool_layer=None):\n super(Classifier, self).__init__()\n self.backbone = backbone\n self.num_classes = num_classes\n if pool_layer is None:\n self.pool_layer = nn.Sequential(\n nn.AdaptiveAvgPool2d(output_size=(1, 1)),\n nn.Flatten()\n )\n else:\n self.pool_layer = pool_layer\n if bottleneck is None:\n self.bottleneck = nn.Identity()\n self._features_dim = backbone.out_features\n else:\n self.bottleneck = bottleneck\n assert bottleneck_dim > 0\n self._features_dim = bottleneck_dim\n\n self.head_source = head_source\n if head_target is None:\n self.head_target = nn.Linear(self._features_dim, num_classes)\n else:\n self.head_target = head_target\n self.finetune = finetune\n\n @property\n def features_dim(self) -> int:\n \"\"\"The dimension of features before the final `head` layer\"\"\"\n return self._features_dim\n\n def forward(self, x: torch.Tensor):\n \"\"\"\"\"\"\n f = self.backbone(x)\n f = self.pool_layer(f)\n y_s = self.head_source(f)\n y_t = self.head_target(self.bottleneck(f))\n if self.training:\n return y_s, y_t\n else:\n return y_t\n\n def get_parameters(self, base_lr=1.0) -> List[Dict]:\n \"\"\"A parameter list which decides optimization hyper-parameters,\n such as the relative learning rate of each layer\n \"\"\"\n params = [\n {\"params\": self.backbone.parameters(), \"lr\": 0.1 * base_lr if self.finetune else 1.0 * base_lr},\n # {\"params\": self.head_source.parameters(), \"lr\": 0.1 * base_lr if self.finetune else 1.0 * base_lr},\n {\"params\": self.bottleneck.parameters(), \"lr\": 1.0 * base_lr},\n {\"params\": self.head_target.parameters(), \"lr\": 1.0 * base_lr},\n ]\n return params\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/talib/finetune/stochnorm.py",
"content": "\"\"\"\n@author: Yifei Ji\n@contact: jiyf990330@163.com\n\"\"\"\nimport numpy as np\nimport torch\nimport torch.nn as nn\nimport torch.nn.functional as F\n\nfrom torch.nn.parameter import Parameter\n\n__all__ = ['StochNorm1d', 'StochNorm2d', 'convert_model']\n\n\nclass _StochNorm(nn.Module):\n\n def __init__(self, num_features, eps=1e-5, momentum=0.1, affine=True, track_running_stats=True, p=0.5):\n super(_StochNorm, self).__init__()\n self.num_features = num_features\n self.eps = eps\n self.momentum = momentum\n self.affine = affine\n self.track_running_stats = track_running_stats\n self.p = p\n if self.affine:\n self.weight = Parameter(torch.Tensor(num_features))\n self.bias = Parameter(torch.Tensor(num_features))\n else:\n self.register_parameter('weight', None)\n self.register_parameter('bias', None)\n\n if self.track_running_stats:\n self.register_buffer('running_mean', torch.zeros(num_features))\n self.register_buffer('running_var', torch.ones(num_features))\n else:\n self.register_parameter('running_mean', None)\n self.register_parameter('running_var', None)\n self.reset_parameters()\n\n def reset_parameters(self):\n if self.track_running_stats:\n self.running_mean.zero_()\n self.running_var.fill_(1)\n if self.affine:\n self.weight.data.uniform_()\n self.bias.data.zero_()\n\n def _check_input_dim(self, input):\n return NotImplemented\n\n def forward(self, input):\n self._check_input_dim(input)\n\n if self.training:\n z_0 = F.batch_norm(\n input, self.running_mean, self.running_var, self.weight, self.bias,\n False, self.momentum, self.eps)\n\n z_1 = F.batch_norm(\n input, self.running_mean, self.running_var, self.weight, self.bias,\n True, self.momentum, self.eps)\n\n if input.dim() == 2:\n s = torch.from_numpy(\n np.random.binomial(n=1, p=self.p, size=self.num_features).reshape(1,\n self.num_features)).float().cuda()\n elif input.dim() == 3:\n s = torch.from_numpy(\n np.random.binomial(n=1, p=self.p, size=self.num_features).reshape(1, self.num_features,\n 1)).float().cuda()\n elif input.dim() == 4:\n s = torch.from_numpy(\n np.random.binomial(n=1, p=self.p, size=self.num_features).reshape(1, self.num_features, 1,\n 1)).float().cuda()\n else:\n raise BaseException()\n\n z = (1 - s) * z_0 + s * z_1\n else:\n z = F.batch_norm(\n input, self.running_mean, self.running_var, self.weight, self.bias,\n False, self.momentum, self.eps)\n\n return z\n\n\nclass StochNorm1d(_StochNorm):\n r\"\"\"Applies Stochastic Normalization over a 2D or 3D input (a mini-batch of 1D inputs with optional additional channel dimension)\n\n Stochastic Normalization is proposed in `Stochastic Normalization (NIPS 2020) `_\n\n .. math::\n\n \\hat{x}_{i,0} = \\frac{x_i - \\tilde{\\mu}}{ \\sqrt{\\tilde{\\sigma} + \\epsilon}}\n\n \\hat{x}_{i,1} = \\frac{x_i - \\mu}{ \\sqrt{\\sigma + \\epsilon}}\n\n \\hat{x}_i = (1-s)\\cdot \\hat{x}_{i,0} + s\\cdot \\hat{x}_{i,1}\n\n y_i = \\gamma \\hat{x}_i + \\beta\n\n where :math:`\\mu` and :math:`\\sigma` are mean and variance of current mini-batch data.\n\n :math:`\\tilde{\\mu}` and :math:`\\tilde{\\sigma}` are current moving statistics of training data.\n\n :math:`s` is a branch-selection variable generated from a Bernoulli distribution, where :math:`P(s=1)=p`.\n\n\n During training, there are two normalization branches. One uses mean and\n variance of current mini-batch data, while the other uses current moving\n statistics of the training data as usual batch normalization.\n\n During evaluation, the moving statistics is used for normalization.\n\n\n Args:\n num_features (int): :math:`c` from an expected input of size :math:`(b, c, l)` or :math:`l` from an expected input of size :math:`(b, l)`.\n eps (float): A value added to the denominator for numerical stability.\n Default: 1e-5\n momentum (float): The value used for the running_mean and running_var\n computation. Default: 0.1\n affine (bool): A boolean value that when set to ``True``, gives the layer learnable\n affine parameters. Default: ``True``\n track_running_stats (bool): A boolean value that when set to True, this module tracks\n the running mean and variance, and when set to False, this module does not\n track such statistics, and initializes statistics buffers running_mean and\n running_var as None. When these buffers are None, this module always uses\n batch statistics in both training and eval modes. Default: True\n p (float): The probability to choose the second branch (usual BN). Default: 0.5\n\n Shape:\n - Input: :math:`(b, l)` or :math:`(b, c, l)`\n - Output: :math:`(b, l)` or :math:`(b, c, l)` (same shape as input)\n \"\"\"\n\n def _check_input_dim(self, input):\n if input.dim() != 2 and input.dim() != 3:\n raise ValueError('expected 2D or 3D input (got {}D input)'\n .format(input.dim()))\n\n\nclass StochNorm2d(_StochNorm):\n r\"\"\"\n Applies Stochastic Normalization over a 4D input (a mini-batch of 2D inputs with additional channel dimension)\n\n Stochastic Normalization is proposed in `Stochastic Normalization (NIPS 2020) `_\n\n .. math::\n\n \\hat{x}_{i,0} = \\frac{x_i - \\tilde{\\mu}}{ \\sqrt{\\tilde{\\sigma} + \\epsilon}}\n\n \\hat{x}_{i,1} = \\frac{x_i - \\mu}{ \\sqrt{\\sigma + \\epsilon}}\n\n \\hat{x}_i = (1-s)\\cdot \\hat{x}_{i,0} + s\\cdot \\hat{x}_{i,1}\n\n y_i = \\gamma \\hat{x}_i + \\beta\n\n where :math:`\\mu` and :math:`\\sigma` are mean and variance of current mini-batch data.\n\n :math:`\\tilde{\\mu}` and :math:`\\tilde{\\sigma}` are current moving statistics of training data.\n\n :math:`s` is a branch-selection variable generated from a Bernoulli distribution, where :math:`P(s=1)=p`.\n\n\n During training, there are two normalization branches. One uses mean and\n variance of current mini-batch data, while the other uses current moving\n statistics of the training data as usual batch normalization.\n\n During evaluation, the moving statistics is used for normalization.\n\n\n Args:\n num_features (int): :math:`c` from an expected input of size :math:`(b, c, h, w)`.\n eps (float): A value added to the denominator for numerical stability.\n Default: 1e-5\n momentum (float): The value used for the running_mean and running_var\n computation. Default: 0.1\n affine (bool): A boolean value that when set to ``True``, gives the layer learnable\n affine parameters. Default: ``True``\n track_running_stats (bool): A boolean value that when set to True, this module tracks\n the running mean and variance, and when set to False, this module does not\n track such statistics, and initializes statistics buffers running_mean and\n running_var as None. When these buffers are None, this module always uses\n batch statistics in both training and eval modes. Default: True\n p (float): The probability to choose the second branch (usual BN). Default: 0.5\n\n Shape:\n - Input: :math:`(b, c, h, w)`\n - Output: :math:`(b, c, h, w)` (same shape as input)\n \"\"\"\n\n def _check_input_dim(self, input):\n if input.dim() != 4:\n raise ValueError('expected 4D input (got {}D input)'\n .format(input.dim()))\n\n\nclass StochNorm3d(_StochNorm):\n r\"\"\"\n Applies Stochastic Normalization over a 5D input (a mini-batch of 3D inputs with additional channel dimension)\n\n Stochastic Normalization is proposed in `Stochastic Normalization (NIPS 2020) `_\n\n .. math::\n\n \\hat{x}_{i,0} = \\frac{x_i - \\tilde{\\mu}}{ \\sqrt{\\tilde{\\sigma} + \\epsilon}}\n\n \\hat{x}_{i,1} = \\frac{x_i - \\mu}{ \\sqrt{\\sigma + \\epsilon}}\n\n \\hat{x}_i = (1-s)\\cdot \\hat{x}_{i,0} + s\\cdot \\hat{x}_{i,1}\n\n y_i = \\gamma \\hat{x}_i + \\beta\n\n where :math:`\\mu` and :math:`\\sigma` are mean and variance of current mini-batch data.\n\n :math:`\\tilde{\\mu}` and :math:`\\tilde{\\sigma}` are current moving statistics of training data.\n\n :math:`s` is a branch-selection variable generated from a Bernoulli distribution, where :math:`P(s=1)=p`.\n\n\n During training, there are two normalization branches. One uses mean and\n variance of current mini-batch data, while the other uses current moving\n statistics of the training data as usual batch normalization.\n\n During evaluation, the moving statistics is used for normalization.\n\n\n Args:\n num_features (int): :math:`c` from an expected input of size :math:`(b, c, d, h, w)`\n eps (float): A value added to the denominator for numerical stability.\n Default: 1e-5\n momentum (float): The value used for the running_mean and running_var\n computation. Default: 0.1\n affine (bool): A boolean value that when set to ``True``, gives the layer learnable\n affine parameters. Default: ``True``\n track_running_stats (bool): A boolean value that when set to True, this module tracks\n the running mean and variance, and when set to False, this module does not\n track such statistics, and initializes statistics buffers running_mean and\n running_var as None. When these buffers are None, this module always uses\n batch statistics in both training and eval modes. Default: True\n p (float): The probability to choose the second branch (usual BN). Default: 0.5\n\n Shape:\n - Input: :math:`(b, c, d, h, w)`\n - Output: :math:`(b, c, d, h, w)` (same shape as input)\n \"\"\"\n\n def _check_input_dim(self, input):\n if input.dim() != 5:\n raise ValueError('expected 4D input (got {}D input)'\n .format(input.dim()))\n\n\ndef convert_model(module, p):\n \"\"\"\n Traverses the input module and its child recursively and replaces all\n instance of BatchNorm to StochNorm.\n\n Args:\n module (torch.nn.Module): The input module needs to be convert to StochNorm model.\n p (float): The hyper-parameter for StochNorm layer.\n\n Returns:\n The module converted to StochNorm version.\n \"\"\"\n\n mod = module\n for pth_module, stoch_module in zip([torch.nn.modules.batchnorm.BatchNorm1d,\n torch.nn.modules.batchnorm.BatchNorm2d,\n torch.nn.modules.batchnorm.BatchNorm3d],\n [StochNorm1d,\n StochNorm2d,\n StochNorm3d]):\n if isinstance(module, pth_module):\n mod = stoch_module(module.num_features, module.eps, module.momentum, module.affine, p)\n mod.running_mean = module.running_mean\n mod.running_var = module.running_var\n\n if module.affine:\n mod.weight.data = module.weight.data.clone().detach()\n mod.bias.data = module.bias.data.clone().detach()\n\n for name, child in module.named_children():\n mod.add_module(name, convert_model(child, p))\n\n return mod\n"
},
{
"path": "domain_generalization/baseline.py",
"content": "\"\"\"\n@author: Baixu Chen\n@contact: cbx_99_hasta@outlook.com\n\"\"\"\nimport random\nimport time\nimport warnings\nimport sys\nimport argparse\nimport shutil\nimport os.path as osp\n\nimport torch\nimport torch.nn as nn\nimport torch.backends.cudnn as cudnn\nfrom torch.optim import SGD\nfrom torch.optim.lr_scheduler import CosineAnnealingLR\nfrom torch.utils.data import DataLoader\nimport torch.nn.functional as F\n\nsys.path.append('../../..')\nfrom dglib.modules.classifier import ImageClassifier as Classifier\nfrom common.utils.data import ForeverDataIterator\nfrom common.utils.metric import accuracy\nfrom common.utils.meter import AverageMeter, ProgressMeter\nfrom common.utils.logger import CompleteLogger\nfrom common.utils.analysis import tsne, a_distance\n\nsys.path.append('.')\nimport utils\n\ndevice = torch.device(\"cuda\" if torch.cuda.is_available() else \"cpu\")\n\nclass TransformerDecorator(torch.nn.Module):\n def __init__(self, backbone=None, add_bt=3, dim=2048, eval_global=0):\n super(TransformerDecorator, self).__init__()\n self.encoder_layers = torch.nn.TransformerEncoderLayer(dim, 4, dim, 0.5)\n self.eval_global = eval_global\n self.add_bt = add_bt\n self.backbone = backbone\n\n def forward_feats(self, feature):\n feature = self.backbone(feature)\n if self.training and self.add_bt > 0:\n pre_feature = feature\n feature = feature.unsqueeze(1)\n feature = self.encoder_layers(feature)\n # feature = self.encoder_layers(feature, feature, feature)\n feature = feature.squeeze(1)\n if self.add_bt:\n return torch.cat([pre_feature, feature], dim=0)\n return feature\n\n def forward(self, feature):\n feature = self.forward_feats(feature)\n return feature\n\n\ndef main(args: argparse.Namespace):\n logger = CompleteLogger(args.log, args.phase)\n print(args)\n\n if args.seed is not None:\n random.seed(args.seed)\n torch.manual_seed(args.seed)\n cudnn.deterministic = True\n warnings.warn('You have chosen to seed training. '\n 'This will turn on the CUDNN deterministic setting, '\n 'which can slow down your training considerably! '\n 'You may see unexpected behavior when restarting '\n 'from checkpoints.')\n\n cudnn.benchmark = True\n\n # Data loading code\n train_transform = utils.get_train_transform(args.train_resizing, random_horizontal_flip=True,\n random_color_jitter=True, random_gray_scale=True)\n val_transform = utils.get_val_transform(args.val_resizing)\n print(\"train_transform: \", train_transform)\n print(\"val_transform: \", val_transform)\n\n train_dataset, num_classes = utils.get_dataset(dataset_name=args.data, root=args.root, task_list=args.sources,\n split='train', download=True, transform=train_transform,\n seed=args.seed)\n train_loader = DataLoader(train_dataset, batch_size=args.batch_size,\n shuffle=True, num_workers=args.workers, drop_last=True)\n val_dataset, _ = utils.get_dataset(dataset_name=args.data, root=args.root, task_list=args.sources, split='val',\n download=True, transform=val_transform, seed=args.seed)\n val_loader = DataLoader(val_dataset, batch_size=args.batch_size, shuffle=False, num_workers=args.workers)\n test_dataset, _ = utils.get_dataset(dataset_name=args.data, root=args.root, task_list=args.targets, split='test',\n download=True, transform=val_transform, seed=args.seed)\n test_loader = DataLoader(test_dataset, batch_size=args.batch_size, shuffle=False, num_workers=args.workers)\n\n print(\"train_dataset_size: \", len(train_dataset))\n print('val_dataset_size: ', len(val_dataset))\n print(\"test_dataset_size: \", len(test_dataset))\n train_iter = ForeverDataIterator(train_loader)\n\n # create model\n print(\"=> using pre-trained model '{}'\".format(args.arch))\n backbone = utils.get_model(args.arch)\n pool_layer = nn.Identity() if args.no_pool else None\n pool_layer = nn.Sequential(\n nn.AdaptiveAvgPool2d(output_size=(1, 1)),\n nn.Flatten(),\n )\n pool_layer = TransformerDecorator(pool_layer, add_bt=args.add_bt, dim=backbone.out_features)\n classifier = Classifier(backbone, num_classes, freeze_bn=args.freeze_bn, dropout_p=args.dropout_p,\n finetune=args.finetune, pool_layer=pool_layer).to(device)\n\n # define optimizer and lr scheduler\n optimizer = SGD(classifier.get_parameters(base_lr=args.lr), args.lr, momentum=args.momentum, weight_decay=args.wd,\n nesterov=True)\n lr_scheduler = CosineAnnealingLR(optimizer, args.epochs * args.iters_per_epoch)\n\n # resume from the best checkpoint\n if args.phase != 'train':\n checkpoint = torch.load(logger.get_checkpoint_path('best'), map_location='cpu')\n classifier.load_state_dict(checkpoint)\n\n # analysis the model\n if args.phase == 'analysis':\n # extract features from both domains\n feature_extractor = nn.Sequential(classifier.backbone, classifier.pool_layer, classifier.bottleneck).to(device)\n source_feature = utils.collect_feature(val_loader, feature_extractor, device, max_num_features=100)\n target_feature = utils.collect_feature(test_loader, feature_extractor, device, max_num_features=100)\n print(len(source_feature), len(target_feature))\n # plot t-SNE\n tSNE_filename = osp.join(logger.visualize_directory, 'TSNE.png')\n tsne.visualize(source_feature, target_feature, tSNE_filename)\n print(\"Saving t-SNE to\", tSNE_filename)\n # calculate A-distance, which is a measure for distribution discrepancy\n A_distance = a_distance.calculate(source_feature, target_feature, device)\n print(\"A-distance =\", A_distance)\n return\n\n if args.phase == 'test':\n acc1 = utils.validate(test_loader, classifier, args, device)\n print(acc1)\n return\n\n # start training\n best_val_acc1 = 0.\n best_test_acc1 = 0.\n for epoch in range(args.epochs):\n print(lr_scheduler.get_lr())\n # train for one epoch\n train(train_iter, classifier, optimizer, lr_scheduler, epoch, args)\n\n # evaluate on validation set\n print(\"Evaluate on validation set...\")\n acc1 = utils.validate(val_loader, classifier, args, device)\n\n # remember best acc@1 and save checkpoint\n torch.save(classifier.state_dict(), logger.get_checkpoint_path('latest'))\n if acc1 > best_val_acc1:\n shutil.copy(logger.get_checkpoint_path('latest'), logger.get_checkpoint_path('best'))\n best_val_acc1 = max(acc1, best_val_acc1)\n\n # evaluate on test set\n print(\"Evaluate on test set...\")\n best_test_acc1 = max(best_test_acc1, utils.validate(test_loader, classifier, args, device))\n\n # evaluate on test set\n classifier.load_state_dict(torch.load(logger.get_checkpoint_path('best')))\n acc1 = utils.validate(test_loader, classifier, args, device)\n print(\"test acc on test set = {}\".format(acc1))\n print(\"oracle acc on test set = {}\".format(best_test_acc1))\n logger.close()\n\n\ndef train(train_iter: ForeverDataIterator, model: Classifier, optimizer,\n lr_scheduler: CosineAnnealingLR, epoch: int, args: argparse.Namespace):\n batch_time = AverageMeter('Time', ':4.2f')\n data_time = AverageMeter('Data', ':3.1f')\n losses = AverageMeter('Loss', ':3.2f')\n cls_accs = AverageMeter('Cls Acc', ':3.1f')\n\n progress = ProgressMeter(\n args.iters_per_epoch,\n [batch_time, data_time, losses, cls_accs],\n prefix=\"Epoch: [{}]\".format(epoch))\n\n # switch to train mode\n model.train()\n\n end = time.time()\n for i in range(args.iters_per_epoch):\n x, labels, _ = next(train_iter)\n x = x.to(device)\n labels = labels.to(device)\n\n # measure data loading time\n data_time.update(time.time() - end)\n\n # compute output\n y, _ = model(x)\n if len(y) > len(labels):\n labels = torch.cat([labels, labels], dim=0)\n\n loss = F.cross_entropy(y, labels)\n\n cls_acc = accuracy(y, labels)[0]\n losses.update(loss.item(), x.size(0))\n cls_accs.update(cls_acc.item(), x.size(0))\n\n # compute gradient and do SGD step\n optimizer.zero_grad()\n loss.backward()\n optimizer.step()\n lr_scheduler.step()\n\n # measure elapsed time\n batch_time.update(time.time() - end)\n end = time.time()\n\n if i % args.print_freq == 0:\n progress.display(i)\n\n\nif __name__ == '__main__':\n parser = argparse.ArgumentParser(description='Baseline for Domain Generalization')\n # dataset parameters\n parser.add_argument('root', metavar='DIR',\n help='root path of dataset')\n parser.add_argument('-d', '--data', metavar='DATA', default='PACS',\n help='dataset: ' + ' | '.join(utils.get_dataset_names()) +\n ' (default: PACS)')\n parser.add_argument('-s', '--sources', nargs='+', default=None,\n help='source domain(s)')\n parser.add_argument('-t', '--targets', nargs='+', default=None,\n help='target domain(s)')\n parser.add_argument('--train-resizing', type=str, default='default')\n parser.add_argument('--val-resizing', type=str, default='default')\n # model parameters\n parser.add_argument('-a', '--arch', metavar='ARCH', default='resnet50',\n choices=utils.get_model_names(),\n help='backbone architecture: ' +\n ' | '.join(utils.get_model_names()) +\n ' (default: resnet50)')\n parser.add_argument('--no-pool', action='store_true', help='no pool layer after the feature extractor.')\n parser.add_argument('--finetune', action='store_true', help='whether use 10x smaller lr for backbone')\n parser.add_argument('--freeze-bn', action='store_true', help='whether freeze all bn layers')\n parser.add_argument('--dropout-p', type=float, default=0.1, help='only activated when freeze-bn is True')\n # training parameters\n parser.add_argument('-b', '--batch-size', default=36, type=int,\n metavar='N',\n help='mini-batch size (default: 36)')\n parser.add_argument('--lr', '--learning-rate', default=1e-3, type=float,\n metavar='LR', help='initial learning rate', dest='lr')\n parser.add_argument('--momentum', default=0.9, type=float, metavar='M',\n help='momentum')\n parser.add_argument('--wd', '--weight-decay', default=0.0005, type=float,\n metavar='W', help='weight decay (default: 5e-4)')\n parser.add_argument('-j', '--workers', default=4, type=int, metavar='N',\n help='number of data loading workers (default: 4)')\n parser.add_argument('--epochs', default=20, type=int, metavar='N',\n help='number of total epochs to run')\n parser.add_argument('-i', '--iters-per-epoch', default=500, type=int,\n help='Number of iterations per epoch')\n parser.add_argument('-p', '--print-freq', default=100, type=int,\n metavar='N', help='print frequency (default: 100)')\n parser.add_argument('--seed', default=0, type=int,\n help='seed for initializing training. ')\n parser.add_argument(\"--log\", type=str, default='baseline',\n help=\"Where to save logs, checkpoints and debugging images.\")\n parser.add_argument(\"--phase\", type=str, default='train', choices=['train', 'test', 'analysis'],\n help=\"When phase is 'test', only test the model.\"\n \"When phase is 'analysis', only analysis the model.\")\n parser.add_argument(\"--add_bt\", type=int, default=0,\n help=\"add_bt\")\n args = parser.parse_args()\n main(args)\n"
},
{
"path": "domain_generalization/coral.py",
"content": "\"\"\"\n@author: Baixu Chen\n@contact: cbx_99_hasta@outlook.com\n\"\"\"\nimport random\nimport time\nimport warnings\nimport sys\nimport argparse\nimport shutil\nimport os.path as osp\n\nimport torch\nimport torch.nn as nn\nimport torch.backends.cudnn as cudnn\nfrom torch.optim import SGD\nfrom torch.optim.lr_scheduler import CosineAnnealingLR\nfrom torch.utils.data import DataLoader\nimport torch.nn.functional as F\n\nsys.path.append('../../..')\nfrom dglib.generalization.coral import CorrelationAlignmentLoss\nfrom dglib.modules.sampler import RandomDomainSampler\nfrom dglib.modules.classifier import ImageClassifier as Classifier\nfrom common.utils.data import ForeverDataIterator\nfrom common.utils.metric import accuracy\nfrom common.utils.meter import AverageMeter, ProgressMeter\nfrom common.utils.logger import CompleteLogger\nfrom common.utils.analysis import tsne, a_distance\n\nsys.path.append('.')\nimport utils\n\ndevice = torch.device(\"cuda\" if torch.cuda.is_available() else \"cpu\")\n\nclass TransformerDecorator(torch.nn.Module):\n def __init__(self, backbone=None, add_bt=3, dim=2048, eval_global=0):\n super(TransformerDecorator, self).__init__()\n self.encoder_layers = torch.nn.TransformerEncoderLayer(dim, 4, dim, 0.5)\n self.eval_global = eval_global\n self.add_bt = add_bt\n self.backbone = backbone\n\n def forward_feats(self, feature):\n feature = self.backbone(feature)\n if self.training and self.add_bt > 0:\n pre_feature = feature\n feature = feature.unsqueeze(1)\n feature = self.encoder_layers(feature)\n # feature = self.encoder_layers(feature, feature, feature)\n feature = feature.squeeze(1)\n if self.add_bt:\n return torch.cat([pre_feature, feature], dim=0)\n return feature\n\n def forward(self, feature):\n feature = self.forward_feats(feature)\n return feature\n\n\ndef main(args: argparse.Namespace):\n logger = CompleteLogger(args.log, args.phase)\n print(args)\n\n if args.seed is not None:\n random.seed(args.seed)\n torch.manual_seed(args.seed)\n cudnn.deterministic = True\n warnings.warn('You have chosen to seed training. '\n 'This will turn on the CUDNN deterministic setting, '\n 'which can slow down your training considerably! '\n 'You may see unexpected behavior when restarting '\n 'from checkpoints.')\n\n cudnn.benchmark = True\n\n # Data loading code\n train_transform = utils.get_train_transform(args.train_resizing, random_horizontal_flip=True,\n random_color_jitter=True, random_gray_scale=True)\n val_transform = utils.get_val_transform(args.val_resizing)\n print(\"train_transform: \", train_transform)\n print(\"val_transform: \", val_transform)\n\n train_dataset, num_classes = utils.get_dataset(dataset_name=args.data, root=args.root, task_list=args.sources,\n split='train', download=True, transform=train_transform,\n seed=args.seed)\n sampler = RandomDomainSampler(train_dataset, args.batch_size, n_domains_per_batch=args.n_domains_per_batch)\n train_loader = DataLoader(train_dataset, batch_size=args.batch_size, num_workers=args.workers,\n sampler=sampler, drop_last=True)\n val_dataset, _ = utils.get_dataset(dataset_name=args.data, root=args.root, task_list=args.sources, split='val',\n download=True, transform=val_transform, seed=args.seed)\n val_loader = DataLoader(val_dataset, batch_size=args.batch_size, shuffle=False, num_workers=args.workers)\n test_dataset, _ = utils.get_dataset(dataset_name=args.data, root=args.root, task_list=args.targets, split='test',\n download=True, transform=val_transform, seed=args.seed)\n test_loader = DataLoader(test_dataset, batch_size=args.batch_size, shuffle=False, num_workers=args.workers)\n print(\"train_dataset_size: \", len(train_dataset))\n print('val_dataset_size: ', len(val_dataset))\n print(\"test_dataset_size: \", len(test_dataset))\n train_iter = ForeverDataIterator(train_loader)\n\n # create model\n print(\"=> using pre-trained model '{}'\".format(args.arch))\n backbone = utils.get_model(args.arch)\n pool_layer = nn.Identity() if args.no_pool else None\n pool_layer = nn.Sequential(\n nn.AdaptiveAvgPool2d(output_size=(1, 1)),\n nn.Flatten(),\n )\n pool_layer = TransformerDecorator(pool_layer, add_bt=args.add_bt, dim=backbone.out_features)\n classifier = Classifier(backbone, num_classes, freeze_bn=args.freeze_bn, dropout_p=args.dropout_p,\n finetune=args.finetune, pool_layer=pool_layer).to(device)\n\n # define optimizer and lr scheduler\n optimizer = SGD(classifier.get_parameters(base_lr=args.lr), args.lr, momentum=args.momentum, weight_decay=args.wd,\n nesterov=True)\n lr_scheduler = CosineAnnealingLR(optimizer, args.epochs * args.iters_per_epoch)\n\n # define loss function\n correlation_alignment_loss = CorrelationAlignmentLoss().to(device)\n\n # resume from the best checkpoint\n if args.phase != 'train':\n checkpoint = torch.load(logger.get_checkpoint_path('best'), map_location='cpu')\n classifier.load_state_dict(checkpoint)\n\n # analysis the model\n if args.phase == 'analysis':\n # extract features from both domains\n feature_extractor = nn.Sequential(classifier.backbone, classifier.pool_layer, classifier.bottleneck).to(device)\n source_feature = utils.collect_feature(val_loader, feature_extractor, device, max_num_features=100)\n target_feature = utils.collect_feature(test_loader, feature_extractor, device, max_num_features=100)\n print(len(source_feature), len(target_feature))\n # plot t-SNE\n tSNE_filename = osp.join(logger.visualize_directory, 'TSNE.png')\n tsne.visualize(source_feature, target_feature, tSNE_filename)\n print(\"Saving t-SNE to\", tSNE_filename)\n # calculate A-distance, which is a measure for distribution discrepancy\n A_distance = a_distance.calculate(source_feature, target_feature, device)\n print(\"A-distance =\", A_distance)\n return\n\n if args.phase == 'test':\n acc1 = utils.validate(test_loader, classifier, args, device)\n print(acc1)\n return\n\n # start training\n best_val_acc1 = 0.\n best_test_acc1 = 0.\n for epoch in range(args.epochs):\n\n print(lr_scheduler.get_lr())\n # train for one epoch\n train(train_iter, classifier, optimizer, lr_scheduler, correlation_alignment_loss, args.n_domains_per_batch,\n epoch, args)\n\n # evaluate on validation set\n print(\"Evaluate on validation set...\")\n acc1 = utils.validate(val_loader, classifier, args, device)\n\n # remember best acc@1 and save checkpoint\n torch.save(classifier.state_dict(), logger.get_checkpoint_path('latest'))\n if acc1 > best_val_acc1:\n shutil.copy(logger.get_checkpoint_path('latest'), logger.get_checkpoint_path('best'))\n best_val_acc1 = max(acc1, best_val_acc1)\n\n # evaluate on test set\n print(\"Evaluate on test set...\")\n best_test_acc1 = max(best_test_acc1, utils.validate(test_loader, classifier, args, device))\n\n # evaluate on test set\n classifier.load_state_dict(torch.load(logger.get_checkpoint_path('best')))\n acc1 = utils.validate(test_loader, classifier, args, device)\n print(\"test acc on test set = {}\".format(acc1))\n print(\"oracle acc on test set = {}\".format(best_test_acc1))\n logger.close()\n\n\ndef train(train_iter: ForeverDataIterator, model: Classifier, optimizer, lr_scheduler: CosineAnnealingLR,\n correlation_alignment_loss: CorrelationAlignmentLoss, n_domains_per_batch: int, epoch: int,\n args: argparse.Namespace):\n batch_time = AverageMeter('Time', ':4.2f')\n data_time = AverageMeter('Data', ':3.1f')\n losses = AverageMeter('Loss', ':3.2f')\n losses_ce = AverageMeter('CELoss', ':3.2f')\n losses_penalty = AverageMeter('Penalty Loss', ':3.2f')\n cls_accs = AverageMeter('Cls Acc', ':3.1f')\n\n progress = ProgressMeter(\n args.iters_per_epoch,\n [batch_time, data_time, losses, losses_ce, losses_penalty, cls_accs],\n prefix=\"Epoch: [{}]\".format(epoch))\n\n # switch to train mode\n model.train()\n\n end = time.time()\n for i in range(args.iters_per_epoch):\n x_all, labels_all, _ = next(train_iter)\n x_all = x_all.to(device)\n labels_all = labels_all.to(device)\n\n # compute output\n y_all, f_all = model(x_all)\n\n # measure data loading time\n data_time.update(time.time() - end)\n if len(y_all) > len(labels_all):\n labels_all = torch.cat([labels_all, labels_all], dim=0)\n y_all_ = y_all.chunk(n_domains_per_batch*2, dim=0)\n f_all_ = f_all.chunk(n_domains_per_batch*2, dim=0)\n labels_all_ = labels_all.chunk(n_domains_per_batch*2, dim=0)\n y_all = []\n f_all = []\n labels_all = []\n for i in range(n_domains_per_batch):\n y_all.append(torch.cat([y_all_[i], y_all_[i+n_domains_per_batch]], dim=0))\n f_all.append(torch.cat([f_all_[i], f_all_[i+n_domains_per_batch]], dim=0))\n labels_all.append(torch.cat([labels_all_[i], labels_all_[i+n_domains_per_batch]], dim=0))\n else:\n # separate into different domains\n y_all = y_all.chunk(n_domains_per_batch, dim=0)\n f_all = f_all.chunk(n_domains_per_batch, dim=0)\n labels_all = labels_all.chunk(n_domains_per_batch, dim=0)\n\n loss_ce = 0\n loss_penalty = 0\n cls_acc = 0\n for domain_i in range(n_domains_per_batch):\n # cls loss\n y_i, labels_i = y_all[domain_i], labels_all[domain_i]\n loss_ce += F.cross_entropy(y_i, labels_i)\n # update acc\n cls_acc += accuracy(y_i, labels_i)[0] / n_domains_per_batch\n # correlation alignment loss\n for domain_j in range(domain_i + 1, n_domains_per_batch):\n f_i = f_all[domain_i]\n f_j = f_all[domain_j]\n loss_penalty += correlation_alignment_loss(f_i, f_j)\n\n # normalize loss\n loss_ce /= n_domains_per_batch\n loss_penalty /= n_domains_per_batch * (n_domains_per_batch - 1) / 2\n\n loss = loss_ce + loss_penalty * args.trade_off\n\n losses.update(loss.item(), x_all.size(0))\n losses_ce.update(loss_ce.item(), x_all.size(0))\n losses_penalty.update(loss_penalty.item(), x_all.size(0))\n cls_accs.update(cls_acc.item(), x_all.size(0))\n\n # compute gradient and do SGD step\n optimizer.zero_grad()\n loss.backward()\n optimizer.step()\n lr_scheduler.step()\n\n # measure elapsed time\n batch_time.update(time.time() - end)\n end = time.time()\n\n if i % args.print_freq == 0:\n progress.display(i)\n\n\nif __name__ == '__main__':\n parser = argparse.ArgumentParser(description='CORAL for Domain Generalization')\n # dataset parameters\n parser.add_argument('root', metavar='DIR',\n help='root path of dataset')\n parser.add_argument('-d', '--data', metavar='DATA', default='PACS',\n help='dataset: ' + ' | '.join(utils.get_dataset_names()) +\n ' (default: PACS)')\n parser.add_argument('-s', '--sources', nargs='+', default=None,\n help='source domain(s)')\n parser.add_argument('-t', '--targets', nargs='+', default=None,\n help='target domain(s)')\n parser.add_argument('--train-resizing', type=str, default='default')\n parser.add_argument('--val-resizing', type=str, default='default')\n # model parameters\n parser.add_argument('-a', '--arch', metavar='ARCH', default='resnet50',\n choices=utils.get_model_names(),\n help='backbone architecture: ' +\n ' | '.join(utils.get_model_names()) +\n ' (default: resnet50)')\n parser.add_argument('--no-pool', action='store_true', help='no pool layer after the feature extractor.')\n parser.add_argument('--finetune', action='store_true', help='whether use 10x smaller lr for backbone')\n parser.add_argument('--freeze-bn', action='store_true', help='whether freeze all bn layers')\n parser.add_argument('--dropout-p', type=float, default=0.1, help='only activated when freeze-bn is True')\n # training parameters\n parser.add_argument('--trade-off', default=1, type=float,\n help='the trade off hyper parameter for correlation alignment loss')\n parser.add_argument('-b', '--batch-size', default=36, type=int,\n metavar='N',\n help='mini-batch size (default: 36)')\n parser.add_argument('--n-domains-per-batch', default=3, type=int,\n help='number of domains in each mini-batch')\n parser.add_argument('--lr', '--learning-rate', default=5e-4, type=float,\n metavar='LR', help='initial learning rate', dest='lr')\n parser.add_argument('--momentum', default=0.9, type=float, metavar='M',\n help='momentum')\n parser.add_argument('--wd', '--weight-decay', default=0.0005, type=float,\n metavar='W', help='weight decay (default: 5e-4)')\n parser.add_argument('-j', '--workers', default=4, type=int, metavar='N',\n help='number of data loading workers (default: 4)')\n parser.add_argument('--epochs', default=20, type=int, metavar='N',\n help='number of total epochs to run')\n parser.add_argument('-i', '--iters-per-epoch', default=500, type=int,\n help='Number of iterations per epoch')\n parser.add_argument('-p', '--print-freq', default=100, type=int,\n metavar='N', help='print frequency (default: 100)')\n parser.add_argument('--seed', default=None, type=int,\n help='seed for initializing training. ')\n parser.add_argument(\"--log\", type=str, default='coral',\n help=\"Where to save logs, checkpoints and debugging images.\")\n parser.add_argument(\"--phase\", type=str, default='train', choices=['train', 'test', 'analysis'],\n help=\"When phase is 'test', only test the model.\"\n \"When phase is 'analysis', only analysis the model.\")\n parser.add_argument(\"--add_bt\", type=int, default=0,\n help=\"add_bt\")\n args = parser.parse_args()\n main(args)\n"
},
{
"path": "domain_generalization/irm.py",
"content": "\"\"\"\n@author: Baixu Chen\n@contact: cbx_99_hasta@outlook.com\n\"\"\"\nimport random\nimport time\nimport warnings\nimport sys\nimport argparse\nimport shutil\nimport os.path as osp\n\nimport torch\nimport torch.nn as nn\nimport torch.backends.cudnn as cudnn\nfrom torch.optim import SGD\nfrom torch.optim.lr_scheduler import CosineAnnealingLR\nfrom torch.utils.data import DataLoader\nimport torch.nn.functional as F\n\nsys.path.append('../../..')\nfrom dglib.generalization.irm import InvariancePenaltyLoss\nfrom dglib.modules.sampler import RandomDomainSampler\nfrom dglib.modules.classifier import ImageClassifier as Classifier\nfrom common.utils.data import ForeverDataIterator\nfrom common.utils.metric import accuracy\nfrom common.utils.meter import AverageMeter, ProgressMeter\nfrom common.utils.logger import CompleteLogger\nfrom common.utils.analysis import tsne, a_distance\n\nsys.path.append('.')\nimport utils\n\ndevice = torch.device(\"cuda\" if torch.cuda.is_available() else \"cpu\")\n\nclass TransformerDecorator(torch.nn.Module):\n def __init__(self, backbone=None, add_bt=3, dim=2048, eval_global=0):\n super(TransformerDecorator, self).__init__()\n self.eval_global = eval_global\n self.add_bt = add_bt\n self.backbone = backbone\n if self.add_bt > 0:\n self.encoder_layers = torch.nn.TransformerEncoderLayer(dim, 4, dim, 0.5)\n\n def forward(self, feature):\n feature = self.backbone(feature)\n if self.training and self.add_bt > 0:\n pre_feature = feature\n feature = feature.unsqueeze(1)\n feature = self.encoder_layers(feature)\n # feature = self.encoder_layers(feature, feature, feature)\n feature = feature.squeeze(1)\n if self.add_bt:\n return torch.cat([pre_feature, feature], dim=0)\n return feature\n\n\ndef main(args: argparse.Namespace):\n logger = CompleteLogger(args.log, args.phase)\n print(args)\n\n if args.seed is not None:\n random.seed(args.seed)\n torch.manual_seed(args.seed)\n cudnn.deterministic = True\n warnings.warn('You have chosen to seed training. '\n 'This will turn on the CUDNN deterministic setting, '\n 'which can slow down your training considerably! '\n 'You may see unexpected behavior when restarting '\n 'from checkpoints.')\n\n cudnn.benchmark = True\n\n # Data loading code\n train_transform = utils.get_train_transform(args.train_resizing, random_horizontal_flip=True,\n random_color_jitter=True, random_gray_scale=True)\n val_transform = utils.get_val_transform(args.val_resizing)\n print(\"train_transform: \", train_transform)\n print(\"val_transform: \", val_transform)\n\n train_dataset, num_classes = utils.get_dataset(dataset_name=args.data, root=args.root, task_list=args.sources,\n split='train', download=True, transform=train_transform,\n seed=args.seed)\n sampler = RandomDomainSampler(train_dataset, args.batch_size, n_domains_per_batch=args.n_domains_per_batch)\n train_loader = DataLoader(train_dataset, batch_size=args.batch_size, num_workers=args.workers,\n sampler=sampler, drop_last=True)\n val_dataset, _ = utils.get_dataset(dataset_name=args.data, root=args.root, task_list=args.sources, split='val',\n download=True, transform=val_transform, seed=args.seed)\n val_loader = DataLoader(val_dataset, batch_size=args.batch_size, shuffle=False, num_workers=args.workers)\n test_dataset, _ = utils.get_dataset(dataset_name=args.data, root=args.root, task_list=args.targets, split='test',\n download=True, transform=val_transform, seed=args.seed)\n test_loader = DataLoader(test_dataset, batch_size=args.batch_size, shuffle=False, num_workers=args.workers)\n print(\"train_dataset_size: \", len(train_dataset))\n print('val_dataset_size: ', len(val_dataset))\n print(\"test_dataset_size: \", len(test_dataset))\n train_iter = ForeverDataIterator(train_loader)\n\n # create model\n print(\"=> using pre-trained model '{}'\".format(args.arch))\n backbone = utils.get_model(args.arch)\n pool_layer = nn.Identity() if args.no_pool else None\n pool_layer = nn.Sequential(\n nn.AdaptiveAvgPool2d(output_size=(1, 1)),\n nn.Flatten(),\n )\n pool_layer = TransformerDecorator(pool_layer, add_bt=args.add_bt, dim=backbone.out_features)\n classifier = Classifier(backbone, num_classes, freeze_bn=args.freeze_bn, dropout_p=args.dropout_p,\n finetune=args.finetune, pool_layer=pool_layer).to(device)\n\n # define optimizer and lr scheduler\n optimizer = SGD(classifier.get_parameters(base_lr=args.lr), args.lr, momentum=args.momentum, weight_decay=args.wd,\n nesterov=True)\n lr_scheduler = CosineAnnealingLR(optimizer, args.epochs * args.iters_per_epoch)\n\n # define loss function\n invariance_penalty_loss = InvariancePenaltyLoss().to(device)\n\n # for simplicity\n assert args.anneal_iters % args.iters_per_epoch == 0\n\n # resume from the best checkpoint\n if args.phase != 'train':\n checkpoint = torch.load(logger.get_checkpoint_path('best'), map_location='cpu')\n classifier.load_state_dict(checkpoint)\n\n # analysis the model\n if args.phase == 'analysis':\n # extract features from both domains\n feature_extractor = nn.Sequential(classifier.backbone, classifier.pool_layer, classifier.bottleneck).to(device)\n source_feature = utils.collect_feature(val_loader, feature_extractor, device, max_num_features=100)\n target_feature = utils.collect_feature(test_loader, feature_extractor, device, max_num_features=100)\n print(len(source_feature), len(target_feature))\n # plot t-SNE\n tSNE_filename = osp.join(logger.visualize_directory, 'TSNE.png')\n tsne.visualize(source_feature, target_feature, tSNE_filename)\n print(\"Saving t-SNE to\", tSNE_filename)\n # calculate A-distance, which is a measure for distribution discrepancy\n A_distance = a_distance.calculate(source_feature, target_feature, device)\n print(\"A-distance =\", A_distance)\n return\n\n if args.phase == 'test':\n acc1 = utils.validate(test_loader, classifier, args, device)\n print(acc1)\n return\n\n # start training\n best_val_acc1 = 0.\n best_test_acc1 = 0.\n for epoch in range(args.epochs):\n if epoch * args.iters_per_epoch == args.anneal_iters:\n # reset optimizer to avoid sharp jump in gradient magnitudes\n optimizer = SGD(classifier.get_parameters(base_lr=args.lr), args.lr, momentum=args.momentum,\n weight_decay=args.wd, nesterov=True)\n lr_scheduler = CosineAnnealingLR(optimizer, args.epochs * args.iters_per_epoch - args.anneal_iters)\n\n print(lr_scheduler.get_lr())\n # train for one epoch\n train(train_iter, classifier, optimizer, lr_scheduler, invariance_penalty_loss, args.n_domains_per_batch, epoch,\n args)\n\n # evaluate on validation set\n print(\"Evaluate on validation set...\")\n acc1 = utils.validate(val_loader, classifier, args, device)\n\n # remember best acc@1 and save checkpoint\n torch.save(classifier.state_dict(), logger.get_checkpoint_path('latest'))\n if acc1 > best_val_acc1:\n shutil.copy(logger.get_checkpoint_path('latest'), logger.get_checkpoint_path('best'))\n best_val_acc1 = max(acc1, best_val_acc1)\n\n # evaluate on test set\n print(\"Evaluate on test set...\")\n best_test_acc1 = max(best_test_acc1, utils.validate(test_loader, classifier, args, device))\n\n # evaluate on test set\n classifier.load_state_dict(torch.load(logger.get_checkpoint_path('best')))\n acc1 = utils.validate(test_loader, classifier, args, device)\n print(\"test acc on test set = {}\".format(acc1))\n print(\"oracle acc on test set = {}\".format(best_test_acc1))\n logger.close()\n\n\ndef train(train_iter: ForeverDataIterator, model: Classifier, optimizer, lr_scheduler: CosineAnnealingLR,\n invariance_penalty_loss: InvariancePenaltyLoss, n_domains_per_batch: int, epoch: int,\n args: argparse.Namespace):\n batch_time = AverageMeter('Time', ':4.2f')\n data_time = AverageMeter('Data', ':3.1f')\n losses = AverageMeter('Loss', ':3.2f')\n losses_ce = AverageMeter('CELoss', ':3.2f')\n losses_penalty = AverageMeter('Penalty Loss', ':3.2f')\n cls_accs = AverageMeter('Cls Acc', ':3.1f')\n\n progress = ProgressMeter(\n args.iters_per_epoch,\n [batch_time, data_time, losses, losses_ce, losses_penalty, cls_accs],\n prefix=\"Epoch: [{}]\".format(epoch))\n\n # switch to train mode\n model.train()\n\n end = time.time()\n for i in range(args.iters_per_epoch):\n x_all, labels_all, _ = next(train_iter)\n x_all = x_all.to(device)\n labels_all = labels_all.to(device)\n\n # measure data loading time\n data_time.update(time.time() - end)\n\n # compute output\n y_all, _ = model(x_all)\n if len(y_all) > len(labels_all):\n labels_all = torch.cat([labels_all, labels_all], dim=0)\n # cls loss\n loss_ce = F.cross_entropy(y_all, labels_all)\n # penalty loss\n loss_penalty = 0\n for y_per_domain, labels_per_domain in zip(y_all.chunk(n_domains_per_batch, dim=0),\n labels_all.chunk(n_domains_per_batch, dim=0)):\n # normalize loss by domain num\n loss_penalty += invariance_penalty_loss(y_per_domain, labels_per_domain) / n_domains_per_batch\n\n global_iter = epoch * args.iters_per_epoch + i\n if global_iter >= args.anneal_iters:\n trade_off = args.trade_off\n else:\n trade_off = 1\n loss = loss_ce + loss_penalty * trade_off\n cls_acc = accuracy(y_all, labels_all)[0]\n\n losses.update(loss.item(), x_all.size(0))\n losses_ce.update(loss_ce.item(), x_all.size(0))\n losses_penalty.update(loss_penalty.item(), x_all.size(0))\n cls_accs.update(cls_acc.item(), x_all.size(0))\n\n # compute gradient and do SGD step\n optimizer.zero_grad()\n loss.backward()\n optimizer.step()\n lr_scheduler.step()\n\n # measure elapsed time\n batch_time.update(time.time() - end)\n end = time.time()\n\n if i % args.print_freq == 0:\n progress.display(i)\n\n\nif __name__ == '__main__':\n parser = argparse.ArgumentParser(description='IRM for Domain Generalization')\n # dataset parameters\n parser.add_argument('root', metavar='DIR',\n help='root path of dataset')\n parser.add_argument('-d', '--data', metavar='DATA', default='PACS',\n help='dataset: ' + ' | '.join(utils.get_dataset_names()) +\n ' (default: PACS)')\n parser.add_argument('-s', '--sources', nargs='+', default=None,\n help='source domain(s)')\n parser.add_argument('-t', '--targets', nargs='+', default=None,\n help='target domain(s)')\n parser.add_argument('--train-resizing', type=str, default='default')\n parser.add_argument('--val-resizing', type=str, default='default')\n # model parameters\n parser.add_argument('-a', '--arch', metavar='ARCH', default='resnet50',\n choices=utils.get_model_names(),\n help='backbone architecture: ' +\n ' | '.join(utils.get_model_names()) +\n ' (default: resnet50)')\n parser.add_argument('--no-pool', action='store_true', help='no pool layer after the feature extractor.')\n parser.add_argument('--finetune', action='store_true', help='whether use 10x smaller lr for backbone')\n parser.add_argument('--freeze-bn', action='store_true', help='whether freeze all bn layers')\n parser.add_argument('--dropout-p', type=float, default=0.1, help='only activated when freeze-bn is True')\n # training parameters\n parser.add_argument('--trade-off', default=1, type=float,\n help='the trade off hyper parameter for irm penalty')\n parser.add_argument('--anneal-iters', default=500, type=int,\n help='anneal iterations (trade off is set to 1 during these iterations)')\n parser.add_argument('-b', '--batch-size', default=36, type=int,\n metavar='N',\n help='mini-batch size (default: 36)')\n parser.add_argument('--n-domains-per-batch', default=3, type=int,\n help='number of domains in each mini-batch')\n parser.add_argument('--lr', '--learning-rate', default=5e-4, type=float,\n metavar='LR', help='initial learning rate', dest='lr')\n parser.add_argument('--momentum', default=0.9, type=float, metavar='M',\n help='momentum')\n parser.add_argument('--wd', '--weight-decay', default=0.0005, type=float,\n metavar='W', help='weight decay (default: 5e-4)')\n parser.add_argument('-j', '--workers', default=4, type=int, metavar='N',\n help='number of data loading workers (default: 4)')\n parser.add_argument('--epochs', default=20, type=int, metavar='N',\n help='number of total epochs to run')\n parser.add_argument('-i', '--iters-per-epoch', default=500, type=int,\n help='Number of iterations per epoch')\n parser.add_argument('-p', '--print-freq', default=100, type=int,\n metavar='N', help='print frequency (default: 100)')\n parser.add_argument('--seed', default=None, type=int,\n help='seed for initializing training. ')\n parser.add_argument(\"--log\", type=str, default='irm',\n help=\"Where to save logs, checkpoints and debugging images.\")\n parser.add_argument(\"--phase\", type=str, default='train', choices=['train', 'test', 'analysis'],\n help=\"When phase is 'test', only test the model.\"\n \"When phase is 'analysis', only analysis the model.\")\n parser.add_argument(\"--add_global\", type=int, default=0,\n help=\"add_global\")\n args = parser.parse_args()\n main(args)\n"
},
{
"path": "domain_generalization/mixstyle.py",
"content": "\"\"\"\n@author: Baixu Chen\n@contact: cbx_99_hasta@outlook.com\n\"\"\"\nimport random\nimport time\nimport warnings\nimport sys\nimport argparse\nimport shutil\nimport os.path as osp\n\nimport torch\nimport torch.nn as nn\nimport torch.backends.cudnn as cudnn\nfrom torch.optim import SGD\nfrom torch.optim.lr_scheduler import CosineAnnealingLR\nfrom torch.utils.data import DataLoader\nimport torch.nn.functional as F\n\nsys.path.append('../../..')\nimport dglib.generalization.mixstyle.models as models\nfrom dglib.modules.sampler import RandomDomainSampler\nfrom dglib.modules.classifier import ImageClassifier as Classifier\nfrom common.utils.data import ForeverDataIterator\nfrom common.utils.metric import accuracy\nfrom common.utils.meter import AverageMeter, ProgressMeter\nfrom common.utils.logger import CompleteLogger\nfrom common.utils.analysis import tsne, a_distance\n\nsys.path.append('.')\nimport utils\n\ndevice = torch.device(\"cuda\" if torch.cuda.is_available() else \"cpu\")\n\nclass TransformerDecorator(torch.nn.Module):\n def __init__(self, backbone=None, add_bt=3, dim=2048, eval_global=0):\n super(TransformerDecorator, self).__init__()\n self.eval_global = eval_global\n self.add_bt = add_bt\n self.backbone = backbone\n self.encoder_layers = torch.nn.TransformerEncoderLayer(dim, 4, dim, 0.5)\n\n def forward(self, feature):\n feature = self.backbone(feature)\n if self.training and self.add_bt > 0:\n pre_feature = feature\n feature = feature.unsqueeze(1)\n feature = self.encoder_layers(feature)\n # feature = self.encoder_layers(feature, feature, feature)\n feature = feature.squeeze(1)\n if self.add_bt:\n return torch.cat([pre_feature, feature], dim=0)\n return feature\n\n\ndef main(args: argparse.Namespace):\n logger = CompleteLogger(args.log, args.phase)\n print(args)\n\n if args.seed is not None:\n random.seed(args.seed)\n torch.manual_seed(args.seed)\n cudnn.deterministic = True\n warnings.warn('You have chosen to seed training. '\n 'This will turn on the CUDNN deterministic setting, '\n 'which can slow down your training considerably! '\n 'You may see unexpected behavior when restarting '\n 'from checkpoints.')\n\n cudnn.benchmark = True\n\n # Data loading code\n train_transform = utils.get_train_transform(args.train_resizing, random_horizontal_flip=True,\n random_color_jitter=True, random_gray_scale=True)\n val_transform = utils.get_val_transform(args.val_resizing)\n print(\"train_transform: \", train_transform)\n print(\"val_transform: \", val_transform)\n\n train_dataset, num_classes = utils.get_dataset(dataset_name=args.data, root=args.root, task_list=args.sources,\n split='train', download=True, transform=train_transform,\n seed=args.seed)\n sampler = RandomDomainSampler(train_dataset, args.batch_size, n_domains_per_batch=2)\n train_loader = DataLoader(train_dataset, batch_size=args.batch_size, num_workers=args.workers,\n sampler=sampler, drop_last=True)\n val_dataset, _ = utils.get_dataset(dataset_name=args.data, root=args.root, task_list=args.sources, split='val',\n download=True, transform=val_transform, seed=args.seed)\n val_loader = DataLoader(val_dataset, batch_size=args.batch_size, shuffle=False, num_workers=args.workers)\n test_dataset, _ = utils.get_dataset(dataset_name=args.data, root=args.root, task_list=args.targets, split='test',\n download=True, transform=val_transform, seed=args.seed)\n test_loader = DataLoader(test_dataset, batch_size=args.batch_size, shuffle=False, num_workers=args.workers)\n\n print(\"train_dataset_size: \", len(train_dataset))\n print('val_dataset_size: ', len(val_dataset))\n print(\"test_dataset_size: \", len(test_dataset))\n train_iter = ForeverDataIterator(train_loader)\n\n # create model\n print(\"=> using pre-trained model '{}'\".format(args.arch))\n backbone = models.__dict__[args.arch](mix_layers=args.mix_layers, mix_p=args.mix_p, mix_alpha=args.mix_alpha,\n pretrained=True)\n pool_layer = nn.Identity() if args.no_pool else None\n pool_layer = nn.Sequential(\n nn.AdaptiveAvgPool2d(output_size=(1, 1)),\n nn.Flatten(),\n )\n pool_layer = TransformerDecorator(pool_layer, add_bt=args.add_bt, dim=backbone.out_features)\n classifier = Classifier(backbone, num_classes, freeze_bn=args.freeze_bn, dropout_p=args.dropout_p,\n finetune=args.finetune, pool_layer=pool_layer).to(device)\n\n # define optimizer and lr scheduler\n optimizer = SGD(classifier.get_parameters(base_lr=args.lr), args.lr, momentum=args.momentum, weight_decay=args.wd,\n nesterov=True)\n lr_scheduler = CosineAnnealingLR(optimizer, args.epochs * args.iters_per_epoch)\n\n # resume from the best checkpoint\n if args.phase != 'train':\n checkpoint = torch.load(logger.get_checkpoint_path('best'), map_location='cpu')\n classifier.load_state_dict(checkpoint)\n\n # analysis the model\n if args.phase == 'analysis':\n # extract features from both domains\n feature_extractor = nn.Sequential(classifier.backbone, classifier.pool_layer, classifier.bottleneck).to(device)\n source_feature = utils.collect_feature(val_loader, feature_extractor, device, max_num_features=100)\n target_feature = utils.collect_feature(test_loader, feature_extractor, device, max_num_features=100)\n print(len(source_feature), len(target_feature))\n # plot t-SNE\n tSNE_filename = osp.join(logger.visualize_directory, 'TSNE.png')\n tsne.visualize(source_feature, target_feature, tSNE_filename)\n print(\"Saving t-SNE to\", tSNE_filename)\n # calculate A-distance, which is a measure for distribution discrepancy\n A_distance = a_distance.calculate(source_feature, target_feature, device)\n print(\"A-distance =\", A_distance)\n return\n\n if args.phase == 'test':\n acc1 = utils.validate(test_loader, classifier, args, device)\n print(acc1)\n return\n\n # start training\n best_val_acc1 = 0.\n best_test_acc1 = 0.\n for epoch in range(args.epochs):\n print(lr_scheduler.get_lr())\n # train for one epoch\n train(train_iter, classifier, optimizer, lr_scheduler, epoch, args)\n\n # evaluate on validation set\n print(\"Evaluate on validation set...\")\n acc1 = utils.validate(val_loader, classifier, args, device)\n\n # remember best acc@1 and save checkpoint\n torch.save(classifier.state_dict(), logger.get_checkpoint_path('latest'))\n if acc1 > best_val_acc1:\n shutil.copy(logger.get_checkpoint_path('latest'), logger.get_checkpoint_path('best'))\n best_val_acc1 = max(acc1, best_val_acc1)\n\n # evaluate on test set\n print(\"Evaluate on test set...\")\n best_test_acc1 = max(best_test_acc1, utils.validate(test_loader, classifier, args, device))\n\n # evaluate on test set\n classifier.load_state_dict(torch.load(logger.get_checkpoint_path('best')))\n acc1 = utils.validate(test_loader, classifier, args, device)\n print(\"test acc on test set = {}\".format(acc1))\n print(\"oracle acc on test set = {}\".format(best_test_acc1))\n logger.close()\n\n\ndef train(train_iter: ForeverDataIterator, model: Classifier, optimizer,\n lr_scheduler: CosineAnnealingLR, epoch: int, args: argparse.Namespace):\n batch_time = AverageMeter('Time', ':4.2f')\n data_time = AverageMeter('Data', ':3.1f')\n losses = AverageMeter('Loss', ':3.2f')\n cls_accs = AverageMeter('Cls Acc', ':3.1f')\n\n progress = ProgressMeter(\n args.iters_per_epoch,\n [batch_time, data_time, losses, cls_accs],\n prefix=\"Epoch: [{}]\".format(epoch))\n\n # switch to train mode\n model.train()\n\n end = time.time()\n for i in range(args.iters_per_epoch):\n x, labels, _ = next(train_iter)\n x = x.to(device)\n labels = labels.to(device)\n\n # measure data loading time\n data_time.update(time.time() - end)\n\n # compute output\n y, _ = model(x)\n if len(y) > len(labels):\n labels = torch.cat([labels, labels], dim=0)\n cls_loss = F.cross_entropy(y, labels)\n loss = cls_loss\n\n cls_acc = accuracy(y, labels)[0]\n\n losses.update(loss.item(), x.size(0))\n cls_accs.update(cls_acc.item(), x.size(0))\n\n # compute gradient and do SGD step\n optimizer.zero_grad()\n loss.backward()\n optimizer.step()\n lr_scheduler.step()\n\n # measure elapsed time\n batch_time.update(time.time() - end)\n end = time.time()\n\n if i % args.print_freq == 0:\n progress.display(i)\n\n\nif __name__ == '__main__':\n architecture_names = sorted(\n name for name in models.__dict__\n if name.islower() and not name.startswith(\"__\")\n and callable(models.__dict__[name])\n )\n parser = argparse.ArgumentParser(description='MixStyle for Domain Generalization')\n # dataset parameters\n parser.add_argument('root', metavar='DIR',\n help='root path of dataset')\n parser.add_argument('-d', '--data', metavar='DATA', default='PACS',\n help='dataset: ' + ' | '.join(utils.get_dataset_names()) +\n ' (default: PACS)')\n parser.add_argument('-s', '--sources', nargs='+', default=None,\n help='source domain(s)')\n parser.add_argument('-t', '--targets', nargs='+', default=None,\n help='target domain(s)')\n parser.add_argument('--train-resizing', type=str, default='default')\n parser.add_argument('--val-resizing', type=str, default='default')\n # model parameters\n parser.add_argument('-a', '--arch', metavar='ARCH', default='resnet50',\n choices=architecture_names,\n help='backbone architecture: ' +\n ' | '.join(architecture_names) +\n ' (default: resnet50)')\n parser.add_argument('--no-pool', action='store_true', help='no pool layer after the feature extractor.')\n parser.add_argument('--mix-layers', nargs='+', help='layers to apply MixStyle')\n parser.add_argument('--mix-p', default=0.5, type=float, help='probability to apply MixStyle')\n parser.add_argument('--mix-alpha', default=0.1, type=float, help='parameter alpha for beta distribution')\n parser.add_argument('--finetune', action='store_true', help='whether use 10x smaller lr for backbone')\n parser.add_argument('--freeze-bn', action='store_true', help='whether freeze all bn layers')\n parser.add_argument('--dropout-p', type=float, default=0.1, help='only activated when freeze-bn is True')\n # training parameters\n parser.add_argument('-b', '--batch-size', default=36, type=int,\n metavar='N',\n help='mini-batch size (default: 36)')\n parser.add_argument('--lr', '--learning-rate', default=1e-3, type=float,\n metavar='LR', help='initial learning rate', dest='lr')\n parser.add_argument('--momentum', default=0.9, type=float, metavar='M',\n help='momentum')\n parser.add_argument('--wd', '--weight-decay', default=0.0005, type=float,\n metavar='W', help='weight decay (default: 5e-4)')\n parser.add_argument('-j', '--workers', default=4, type=int, metavar='N',\n help='number of data loading workers (default: 4)')\n parser.add_argument('--epochs', default=20, type=int, metavar='N',\n help='number of total epochs to run')\n parser.add_argument('-i', '--iters-per-epoch', default=500, type=int,\n help='Number of iterations per epoch')\n parser.add_argument('-p', '--print-freq', default=100, type=int,\n metavar='N', help='print frequency (default: 100)')\n parser.add_argument('--seed', default=None, type=int,\n help='seed for initializing training. ')\n parser.add_argument(\"--log\", type=str, default='mixstyle',\n help=\"Where to save logs, checkpoints and debugging images.\")\n parser.add_argument(\"--phase\", type=str, default='train', choices=['train', 'test', 'analysis'],\n help=\"When phase is 'test', only test the model.\"\n \"When phase is 'analysis', only analysis the model.\")\n parser.add_argument(\"--add_bt\", type=int, default=0,\n help=\"add_bt\")\n args = parser.parse_args()\n main(args)\n"
},
{
"path": "domain_generalization/swad/.gitignore",
"content": "__pycache__\ntags\ntrain_output/\n.vim/\n.ipynb_checkpoints/\n*.log\n*.json\nswa_plane_results/\nfeatures/\nperturb_results/\n*.pdf\n*.sh\n*.ipynb\n"
},
{
"path": "domain_generalization/swad/LICENSE",
"content": "MIT License\n\nCopyright (c) 2021 Junbum Cha\n\nPermission is hereby granted, free of charge, to any person obtaining a copy\nof this software and associated documentation files (the \"Software\"), to deal\nin the Software without restriction, including without limitation the rights\nto use, copy, modify, merge, publish, distribute, sublicense, and/or sell\ncopies of the Software, and to permit persons to whom the Software is\nfurnished to do so, subject to the following conditions:\n\nThe above copyright notice and this permission notice shall be included in all\ncopies or substantial portions of the Software.\n\nTHE SOFTWARE IS PROVIDED \"AS IS\", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR\nIMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,\nFITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE\nAUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER\nLIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,\nOUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE\nSOFTWARE.\n"
},
{
"path": "domain_generalization/swad/README.md",
"content": "# BatchFormer for SWAD\n\n## Preparation\n\n### Dependencies\n\n```sh\npip install -r requirements.txt\n```\n\n### Datasets\n\n```sh\npython -m domainbed.scripts.download --data_dir=/my/datasets/path\n```\n\n### Environments\n\nEnvironment details used for our study.\n\n```\nPython: 3.8.6\nPyTorch: 1.7.0+cu92\nTorchvision: 0.8.1+cu92\nCUDA: 9.2\nCUDNN: 7603\nNumPy: 1.19.4\nPIL: 8.0.1\n```\n\n\n### How to run\n\nWe provide the instructions to reproduce the main results of the paper, Table 1 and 2.\nNote that the difference in a detailed environment or uncontrolled randomness may bring a little different result from the paper.\n\n- PACS\n\n```\npython train_all.py PACS0 --dataset PACS --deterministic --trial_seed 0 --checkpoint_freq 100 --data_dir /my/datasets/path --resnet18 --add_bt 16\npython train_all.py PACS1 --dataset PACS --deterministic --trial_seed 1 --checkpoint_freq 100 --data_dir /my/datasets/path --resnet18 --add_bt 16\npython train_all.py PACS2 --dataset PACS --deterministic --trial_seed 2 --checkpoint_freq 100 --data_dir /my/datasets/path --resnet18 --add_bt 16\n```\n\n**You can also run the code with \"--add_bt 1\" which do not include shared classifier. Meanwhile, you can also set a smaller learning rate with \"--lr_scale_bt 0.1\".\nThat is also beneficial for some datasets, especially when the backbone is ResNet50.** \n\n\n## License\nThis code is based on SWAD.\n\nThis source code is released under the MIT license, included [here](./LICENSE).\n\nThis project includes some code from [DomainBed](https://github.com/facebookresearch/DomainBed/tree/3fe9d7bb4bc14777a42b3a9be8dd887e709ec414), also MIT licensed.\n"
},
{
"path": "domain_generalization/swad/config.yaml",
"content": "# Default update config\n# Config order: hparams_registry -> config.yaml -> CLI\nscheduler: const\n\nswad: LossValley\nswad_kwargs:\n n_converge: 3\n n_tolerance: 6\n tolerance_ratio: 0.3\ntest_batchsize: 128\nindomain_test: False # use indomain test split or not\n"
},
{
"path": "domain_generalization/swad/domainbed/algorithms/__init__.py",
"content": "from .algorithms import *\n\n\ndef get_algorithm_class(algorithm_name):\n \"\"\"Return the algorithm class with the given name.\"\"\"\n if algorithm_name not in globals():\n raise NotImplementedError(\"Algorithm not found: {}\".format(algorithm_name))\n return globals()[algorithm_name]\n"
},
{
"path": "domain_generalization/swad/domainbed/algorithms/algorithms.py",
"content": "# Copyright (c) Facebook, Inc. and its affiliates. All Rights Reserved\n\nimport copy\nfrom typing import List\n\nimport torch\nimport torch.nn as nn\nimport torch.nn.functional as F\nimport torch.autograd as autograd\nimport numpy as np\n\n# import higher\n\nfrom domainbed import networks\nfrom domainbed.lib.misc import random_pairs_of_minibatches\nfrom domainbed.optimizers import get_optimizer\n\nfrom domainbed.models.resnet_mixstyle import (\n resnet18_mixstyle_L234_p0d5_a0d1,\n resnet50_mixstyle_L234_p0d5_a0d1,\n)\nfrom domainbed.models.resnet_mixstyle2 import (\n resnet18_mixstyle2_L234_p0d5_a0d1,\n resnet50_mixstyle2_L234_p0d5_a0d1,\n)\n\n\ndef to_minibatch(x, y):\n minibatches = list(zip(x, y))\n return minibatches\n\n\nclass Algorithm(torch.nn.Module):\n \"\"\"\n A subclass of Algorithm implements a domain generalization algorithm.\n Subclasses should implement the following:\n - update()\n - predict()\n \"\"\"\n\n transforms = {}\n\n def __init__(self, input_shape, num_classes, num_domains, hparams):\n super(Algorithm, self).__init__()\n self.input_shape = input_shape\n self.num_classes = num_classes\n self.num_domains = num_domains\n self.hparams = hparams\n\n def update(self, x, y, **kwargs):\n \"\"\"\n Perform one update step, given a list of (x, y) tuples for all\n environments.\n \"\"\"\n raise NotImplementedError\n\n def predict(self, x):\n raise NotImplementedError\n\n def forward(self, x):\n return self.predict(x)\n\n def new_optimizer(self, parameters):\n optimizer = get_optimizer(\n self.hparams[\"optimizer\"],\n parameters,\n lr=self.hparams[\"lr\"],\n weight_decay=self.hparams[\"weight_decay\"],\n )\n return optimizer\n\n def clone(self):\n clone = copy.deepcopy(self)\n clone.optimizer = self.new_optimizer(clone.network.parameters())\n clone.optimizer.load_state_dict(self.optimizer.state_dict())\n\n return clone\n\n\nclass ERM(Algorithm):\n \"\"\"\n Empirical Risk Minimization (ERM)\n \"\"\"\n\n def __init__(self, input_shape, num_classes, num_domains, hparams):\n super(ERM, self).__init__(input_shape, num_classes, num_domains, hparams)\n self.featurizer = networks.Featurizer(input_shape, self.hparams)\n self.classifier = nn.Linear(self.featurizer.n_outputs, num_classes)\n self.network = nn.Sequential(self.featurizer, self.classifier)\n self.optimizer = get_optimizer(\n hparams[\"optimizer\"],\n self.network.parameters(),\n lr=self.hparams[\"lr\"],\n weight_decay=self.hparams[\"weight_decay\"],\n )\n\n def update(self, x, y, **kwargs):\n all_x = torch.cat(x)\n all_y = torch.cat(y)\n all_preds = self.predict(all_x)\n while len(all_preds) > len(all_y):\n all_y = torch.cat([all_y, all_y], dim=0)\n loss = F.cross_entropy(all_preds, all_y)\n\n self.optimizer.zero_grad()\n loss.backward()\n self.optimizer.step()\n\n return {\"loss\": loss.item()}\n\n def predict(self, x):\n return self.network(x)\n\n\nclass Mixstyle(Algorithm):\n \"\"\"MixStyle w/o domain label (random shuffle)\"\"\"\n\n def __init__(self, input_shape, num_classes, num_domains, hparams):\n assert input_shape[1:3] == (224, 224), \"Mixstyle support R18 and R50 only\"\n super().__init__(input_shape, num_classes, num_domains, hparams)\n if hparams[\"resnet18\"]:\n network = resnet18_mixstyle_L234_p0d5_a0d1()\n else:\n network = resnet50_mixstyle_L234_p0d5_a0d1()\n self.featurizer = networks.ResNet(input_shape, self.hparams, network)\n\n self.classifier = nn.Linear(self.featurizer.n_outputs, num_classes)\n self.network = nn.Sequential(self.featurizer, self.classifier)\n self.optimizer = self.new_optimizer(self.network.parameters())\n\n def update(self, x, y, **kwargs):\n all_x = torch.cat(x)\n all_y = torch.cat(y)\n all_preds = self.predict(all_x)\n while len(all_preds) > len(all_y):\n all_y = torch.cat([all_y, all_y], dim=0)\n loss = F.cross_entropy(all_preds, all_y)\n\n self.optimizer.zero_grad()\n loss.backward()\n self.optimizer.step()\n\n return {\"loss\": loss.item()}\n\n def predict(self, x):\n return self.network(x)\n\n\nclass Mixstyle2(Algorithm):\n \"\"\"MixStyle w/ domain label\"\"\"\n\n def __init__(self, input_shape, num_classes, num_domains, hparams):\n assert input_shape[1:3] == (224, 224), \"Mixstyle support R18 and R50 only\"\n super().__init__(input_shape, num_classes, num_domains, hparams)\n if hparams[\"resnet18\"]:\n network = resnet18_mixstyle2_L234_p0d5_a0d1()\n else:\n network = resnet50_mixstyle2_L234_p0d5_a0d1()\n self.featurizer = networks.ResNet(input_shape, self.hparams, network)\n\n self.classifier = nn.Linear(self.featurizer.n_outputs, num_classes)\n self.network = nn.Sequential(self.featurizer, self.classifier)\n self.optimizer = self.new_optimizer(self.network.parameters())\n\n def pair_batches(self, xs, ys):\n xs = [x.chunk(2) for x in xs]\n ys = [y.chunk(2) for y in ys]\n N = len(xs)\n pairs = []\n for i in range(N):\n j = i + 1 if i < (N - 1) else 0\n xi, yi = xs[i][0], ys[i][0]\n xj, yj = xs[j][1], ys[j][1]\n\n pairs.append(((xi, yi), (xj, yj)))\n\n return pairs\n\n def update(self, x, y, **kwargs):\n pairs = self.pair_batches(x, y)\n loss = 0.0\n\n for (xi, yi), (xj, yj) in pairs:\n # Mixstyle2:\n # For the input x, the first half comes from one domain,\n # while the second half comes from the other domain.\n x2 = torch.cat([xi, xj])\n y2 = torch.cat([yi, yj])\n preds = self.predict(x2)\n while len(preds)> len(y2):\n y2 = torch.cat([y2, y2], dim=0)\n loss += F.cross_entropy(preds, y2)\n\n loss /= len(pairs)\n\n self.optimizer.zero_grad()\n loss.backward()\n self.optimizer.step()\n\n return {\"loss\": loss.item()}\n\n def predict(self, x):\n return self.network(x)\n\n\nclass ARM(ERM):\n \"\"\"Adaptive Risk Minimization (ARM)\"\"\"\n\n def __init__(self, input_shape, num_classes, num_domains, hparams):\n original_input_shape = input_shape\n input_shape = (1 + original_input_shape[0],) + original_input_shape[1:]\n super(ARM, self).__init__(input_shape, num_classes, num_domains, hparams)\n self.context_net = networks.ContextNet(original_input_shape)\n self.support_size = hparams[\"batch_size\"]\n\n def predict(self, x):\n batch_size, c, h, w = x.shape\n if batch_size % self.support_size == 0:\n meta_batch_size = batch_size // self.support_size\n support_size = self.support_size\n else:\n meta_batch_size, support_size = 1, batch_size\n context = self.context_net(x)\n context = context.reshape((meta_batch_size, support_size, 1, h, w))\n context = context.mean(dim=1)\n context = torch.repeat_interleave(context, repeats=support_size, dim=0)\n x = torch.cat([x, context], dim=1)\n return self.network(x)\n\n\nclass SAM(ERM):\n \"\"\"Sharpness-Aware Minimization\n \"\"\"\n @staticmethod\n def norm(tensor_list: List[torch.tensor], p=2):\n \"\"\"Compute p-norm for tensor list\"\"\"\n return torch.cat([x.flatten() for x in tensor_list]).norm(p)\n\n def update(self, x, y, **kwargs):\n all_x = torch.cat([xi for xi in x])\n all_y = torch.cat([yi for yi in y])\n loss = F.cross_entropy(self.predict(all_x), all_y)\n\n # 1. eps(w) = rho * g(w) / g(w).norm(2)\n # = (rho / g(w).norm(2)) * g(w)\n grad_w = autograd.grad(loss, self.network.parameters())\n scale = self.hparams[\"rho\"] / self.norm(grad_w)\n eps = [g * scale for g in grad_w]\n\n # 2. w' = w + eps(w)\n with torch.no_grad():\n for p, v in zip(self.network.parameters(), eps):\n p.add_(v)\n\n # 3. w = w - lr * g(w')\n loss = F.cross_entropy(self.predict(all_x), all_y)\n\n self.optimizer.zero_grad()\n loss.backward()\n # restore original network params\n with torch.no_grad():\n for p, v in zip(self.network.parameters(), eps):\n p.sub_(v)\n self.optimizer.step()\n\n return {\"loss\": loss.item()}\n\n\nclass AbstractDANN(Algorithm):\n \"\"\"Domain-Adversarial Neural Networks (abstract class)\"\"\"\n\n def __init__(self, input_shape, num_classes, num_domains, hparams, conditional, class_balance):\n\n super(AbstractDANN, self).__init__(input_shape, num_classes, num_domains, hparams)\n\n self.register_buffer(\"update_count\", torch.tensor([0]))\n self.conditional = conditional\n self.class_balance = class_balance\n\n # Algorithms\n self.featurizer = networks.Featurizer(input_shape, self.hparams)\n self.classifier = nn.Linear(self.featurizer.n_outputs, num_classes)\n self.discriminator = networks.MLP(self.featurizer.n_outputs, num_domains, self.hparams)\n self.class_embeddings = nn.Embedding(num_classes, self.featurizer.n_outputs)\n\n # Optimizers\n self.disc_opt = get_optimizer(\n hparams[\"optimizer\"],\n (list(self.discriminator.parameters()) + list(self.class_embeddings.parameters())),\n lr=self.hparams[\"lr_d\"],\n weight_decay=self.hparams[\"weight_decay_d\"],\n betas=(self.hparams[\"beta1\"], 0.9),\n )\n\n self.gen_opt = get_optimizer(\n hparams[\"optimizer\"],\n (list(self.featurizer.parameters()) + list(self.classifier.parameters())),\n lr=self.hparams[\"lr_g\"],\n weight_decay=self.hparams[\"weight_decay_g\"],\n betas=(self.hparams[\"beta1\"], 0.9),\n )\n\n def update(self, x, y, **kwargs):\n self.update_count += 1\n all_x = torch.cat([xi for xi in x])\n all_y = torch.cat([yi for yi in y])\n minibatches = to_minibatch(x, y)\n all_z = self.featurizer(all_x)\n if self.conditional:\n disc_input = all_z + self.class_embeddings(all_y)\n else:\n disc_input = all_z\n disc_out = self.discriminator(disc_input)\n disc_labels = torch.cat(\n [\n torch.full((x.shape[0],), i, dtype=torch.int64, device=\"cuda\")\n for i, (x, y) in enumerate(minibatches)\n ]\n )\n if len(disc_out) > len(disc_labels):\n disc_labels = torch.cat([disc_labels, disc_labels], dim=0)\n\n if self.class_balance:\n y_counts = F.one_hot(all_y).sum(dim=0)\n weights = 1.0 / (y_counts[all_y] * y_counts.shape[0]).float()\n disc_loss = F.cross_entropy(disc_out, disc_labels, reduction=\"none\")\n disc_loss = (weights * disc_loss).sum()\n else:\n disc_loss = F.cross_entropy(disc_out, disc_labels)\n\n disc_softmax = F.softmax(disc_out, dim=1)\n input_grad = autograd.grad(\n disc_softmax[:, disc_labels].sum(), [disc_input], create_graph=True\n )[0]\n grad_penalty = (input_grad ** 2).sum(dim=1).mean(dim=0)\n disc_loss += self.hparams[\"grad_penalty\"] * grad_penalty\n\n d_steps_per_g = self.hparams[\"d_steps_per_g_step\"]\n if self.update_count.item() % (1 + d_steps_per_g) < d_steps_per_g:\n\n self.disc_opt.zero_grad()\n disc_loss.backward()\n self.disc_opt.step()\n return {\"disc_loss\": disc_loss.item()}\n else:\n all_preds = self.classifier(all_z)\n if len(all_preds) > len(all_y):\n all_y = torch.cat([all_y, all_y], dim=0)\n classifier_loss = F.cross_entropy(all_preds, all_y)\n gen_loss = classifier_loss + (self.hparams[\"lambda\"] * -disc_loss)\n self.disc_opt.zero_grad()\n self.gen_opt.zero_grad()\n gen_loss.backward()\n self.gen_opt.step()\n return {\"gen_loss\": gen_loss.item()}\n\n def predict(self, x):\n return self.classifier(self.featurizer(x))\n\n\nclass DANN(AbstractDANN):\n \"\"\"Unconditional DANN\"\"\"\n\n def __init__(self, input_shape, num_classes, num_domains, hparams):\n super(DANN, self).__init__(\n input_shape,\n num_classes,\n num_domains,\n hparams,\n conditional=False,\n class_balance=False,\n )\n\n\nclass CDANN(AbstractDANN):\n \"\"\"Conditional DANN\"\"\"\n\n def __init__(self, input_shape, num_classes, num_domains, hparams):\n super(CDANN, self).__init__(\n input_shape,\n num_classes,\n num_domains,\n hparams,\n conditional=True,\n class_balance=True,\n )\n\n\nclass IRM(ERM):\n \"\"\"Invariant Risk Minimization\"\"\"\n\n def __init__(self, input_shape, num_classes, num_domains, hparams):\n super(IRM, self).__init__(input_shape, num_classes, num_domains, hparams)\n self.register_buffer(\"update_count\", torch.tensor([0]))\n\n @staticmethod\n def _irm_penalty(logits, y):\n scale = torch.tensor(1.0).cuda().requires_grad_()\n loss_1 = F.cross_entropy(logits[::2] * scale, y[::2])\n loss_2 = F.cross_entropy(logits[1::2] * scale, y[1::2])\n grad_1 = autograd.grad(loss_1, [scale], create_graph=True)[0]\n grad_2 = autograd.grad(loss_2, [scale], create_graph=True)[0]\n result = torch.sum(grad_1 * grad_2)\n return result\n\n def update(self, x, y, **kwargs):\n minibatches = to_minibatch(x, y)\n penalty_weight = (\n self.hparams[\"irm_lambda\"]\n if self.update_count >= self.hparams[\"irm_penalty_anneal_iters\"]\n else 1.0\n )\n nll = 0.0\n penalty = 0.0\n\n all_x = torch.cat([x for x, y in minibatches])\n all_logits = self.network(all_x)\n all_logits_idx = 0\n for i, (x, y) in enumerate(minibatches):\n logits = all_logits[all_logits_idx : all_logits_idx + x.shape[0]]\n all_logits_idx += x.shape[0]\n nll += F.cross_entropy(logits, y)\n penalty += self._irm_penalty(logits, y)\n nll /= len(minibatches)\n penalty /= len(minibatches)\n loss = nll + (penalty_weight * penalty)\n\n if self.update_count == self.hparams[\"irm_penalty_anneal_iters\"]:\n # Reset Adam, because it doesn't like the sharp jump in gradient\n # magnitudes that happens at this step.\n self.optimizer = get_optimizer(\n self.hparams[\"optimizer\"],\n self.network.parameters(),\n lr=self.hparams[\"lr\"],\n weight_decay=self.hparams[\"weight_decay\"],\n )\n\n self.optimizer.zero_grad()\n loss.backward()\n self.optimizer.step()\n\n self.update_count += 1\n return {\"loss\": loss.item(), \"nll\": nll.item(), \"penalty\": penalty.item()}\n\n\nclass VREx(ERM):\n \"\"\"V-REx algorithm from http://arxiv.org/abs/2003.00688\"\"\"\n\n def __init__(self, input_shape, num_classes, num_domains, hparams):\n super(VREx, self).__init__(input_shape, num_classes, num_domains, hparams)\n self.register_buffer(\"update_count\", torch.tensor([0]))\n\n def update(self, x, y, **kwargs):\n minibatches = to_minibatch(x, y)\n if self.update_count >= self.hparams[\"vrex_penalty_anneal_iters\"]:\n penalty_weight = self.hparams[\"vrex_lambda\"]\n else:\n penalty_weight = 1.0\n\n nll = 0.0\n\n all_x = torch.cat([x for x, y in minibatches])\n all_logits = self.network(all_x)\n all_logits_idx = 0\n losses = torch.zeros(len(minibatches))\n for i, (x, y) in enumerate(minibatches):\n logits = all_logits[all_logits_idx : all_logits_idx + x.shape[0]]\n all_logits_idx += x.shape[0]\n nll = F.cross_entropy(logits, y)\n losses[i] = nll\n\n mean = losses.mean()\n penalty = ((losses - mean) ** 2).mean()\n loss = mean + penalty_weight * penalty\n\n if self.update_count == self.hparams[\"vrex_penalty_anneal_iters\"]:\n # Reset Adam (like IRM), because it doesn't like the sharp jump in\n # gradient magnitudes that happens at this step.\n self.optimizer = get_optimizer(\n self.hparams[\"optimizer\"],\n self.network.parameters(),\n lr=self.hparams[\"lr\"],\n weight_decay=self.hparams[\"weight_decay\"],\n )\n\n self.optimizer.zero_grad()\n loss.backward()\n self.optimizer.step()\n\n self.update_count += 1\n return {\"loss\": loss.item(), \"nll\": nll.item(), \"penalty\": penalty.item()}\n\n\nclass Mixup(ERM):\n \"\"\"\n Mixup of minibatches from different domains\n https://arxiv.org/pdf/2001.00677.pdf\n https://arxiv.org/pdf/1912.01805.pdf\n \"\"\"\n\n def __init__(self, input_shape, num_classes, num_domains, hparams):\n super(Mixup, self).__init__(input_shape, num_classes, num_domains, hparams)\n\n def update(self, x, y, **kwargs):\n minibatches = to_minibatch(x, y)\n objective = 0\n\n for (xi, yi), (xj, yj) in random_pairs_of_minibatches(minibatches):\n lam = np.random.beta(self.hparams[\"mixup_alpha\"], self.hparams[\"mixup_alpha\"])\n\n x = lam * xi + (1 - lam) * xj\n predictions = self.predict(x)\n\n objective += lam * F.cross_entropy(predictions, yi)\n objective += (1 - lam) * F.cross_entropy(predictions, yj)\n\n objective /= len(minibatches)\n\n self.optimizer.zero_grad()\n objective.backward()\n self.optimizer.step()\n\n return {\"loss\": objective.item()}\n\n\nclass OrgMixup(ERM):\n \"\"\"\n Original Mixup independent with domains\n \"\"\"\n\n def update(self, x, y, **kwargs):\n x = torch.cat(x)\n y = torch.cat(y)\n\n indices = torch.randperm(x.size(0))\n x2 = x[indices]\n y2 = y[indices]\n\n lam = np.random.beta(self.hparams[\"mixup_alpha\"], self.hparams[\"mixup_alpha\"])\n\n x = lam * x + (1 - lam) * x2\n predictions = self.predict(x)\n\n objective = lam * F.cross_entropy(predictions, y)\n objective += (1 - lam) * F.cross_entropy(predictions, y2)\n\n self.optimizer.zero_grad()\n objective.backward()\n self.optimizer.step()\n\n return {\"loss\": objective.item()}\n\n\nclass CutMix(ERM):\n @staticmethod\n def rand_bbox(size, lam):\n W = size[2]\n H = size[3]\n cut_rat = np.sqrt(1.0 - lam)\n cut_w = np.int(W * cut_rat)\n cut_h = np.int(H * cut_rat)\n\n # uniform\n cx = np.random.randint(W)\n cy = np.random.randint(H)\n\n bbx1 = np.clip(cx - cut_w // 2, 0, W)\n bby1 = np.clip(cy - cut_h // 2, 0, H)\n bbx2 = np.clip(cx + cut_w // 2, 0, W)\n bby2 = np.clip(cy + cut_h // 2, 0, H)\n\n return bbx1, bby1, bbx2, bby2\n\n def update(self, x, y, **kwargs):\n # cutmix_prob is set to 1.0 for ImageNet and 0.5 for CIFAR100 in the original paper.\n x = torch.cat(x)\n y = torch.cat(y)\n\n r = np.random.rand(1)\n if self.hparams[\"beta\"] > 0 and r < self.hparams[\"cutmix_prob\"]:\n # generate mixed sample\n beta = self.hparams[\"beta\"]\n lam = np.random.beta(beta, beta)\n rand_index = torch.randperm(x.size()[0]).cuda()\n target_a = y\n target_b = y[rand_index]\n bbx1, bby1, bbx2, bby2 = self.rand_bbox(x.size(), lam)\n x[:, :, bbx1:bbx2, bby1:bby2] = x[rand_index, :, bbx1:bbx2, bby1:bby2]\n # adjust lambda to exactly match pixel ratio\n lam = 1 - ((bbx2 - bbx1) * (bby2 - bby1) / (x.size()[-1] * x.size()[-2]))\n # compute output\n output = self.predict(x)\n objective = F.cross_entropy(output, target_a) * lam + F.cross_entropy(\n output, target_b\n ) * (1.0 - lam)\n else:\n output = self.predict(x)\n objective = F.cross_entropy(output, y)\n\n self.optimizer.zero_grad()\n objective.backward()\n self.optimizer.step()\n\n return {\"loss\": objective.item()}\n\n\nclass GroupDRO(ERM):\n \"\"\"\n Robust ERM minimizes the error at the worst minibatch\n Algorithm 1 from [https://arxiv.org/pdf/1911.08731.pdf]\n \"\"\"\n\n def __init__(self, input_shape, num_classes, num_domains, hparams):\n super(GroupDRO, self).__init__(input_shape, num_classes, num_domains, hparams)\n self.register_buffer(\"q\", torch.Tensor())\n\n def update(self, x, y, **kwargs):\n minibatches = to_minibatch(x, y)\n device = \"cuda\" if minibatches[0][0].is_cuda else \"cpu\"\n\n if not len(self.q):\n self.q = torch.ones(len(minibatches)).to(device)\n\n losses = torch.zeros(len(minibatches)).to(device)\n\n for m in range(len(minibatches)):\n x, y = minibatches[m]\n losses[m] = F.cross_entropy(self.predict(x), y)\n self.q[m] *= (self.hparams[\"groupdro_eta\"] * losses[m].data).exp()\n\n self.q /= self.q.sum()\n\n loss = torch.dot(losses, self.q) / len(minibatches)\n\n self.optimizer.zero_grad()\n loss.backward()\n self.optimizer.step()\n\n return {\"loss\": loss.item()}\n\n\nclass MLDG(ERM):\n \"\"\"\n Model-Agnostic Meta-Learning\n Algorithm 1 / Equation (3) from: https://arxiv.org/pdf/1710.03463.pdf\n Related: https://arxiv.org/pdf/1703.03400.pdf\n Related: https://arxiv.org/pdf/1910.13580.pdf\n \"\"\"\n\n def __init__(self, input_shape, num_classes, num_domains, hparams):\n super(MLDG, self).__init__(input_shape, num_classes, num_domains, hparams)\n\n def update(self, x, y, **kwargs):\n \"\"\"\n Terms being computed:\n * Li = Loss(xi, yi, params)\n * Gi = Grad(Li, params)\n\n * Lj = Loss(xj, yj, Optimizer(params, grad(Li, params)))\n * Gj = Grad(Lj, params)\n\n * params = Optimizer(params, Grad(Li + beta * Lj, params))\n * = Optimizer(params, Gi + beta * Gj)\n\n That is, when calling .step(), we want grads to be Gi + beta * Gj\n\n For computational efficiency, we do not compute second derivatives.\n \"\"\"\n minibatches = to_minibatch(x, y)\n num_mb = len(minibatches)\n objective = 0\n\n self.optimizer.zero_grad()\n for p in self.network.parameters():\n if p.grad is None:\n p.grad = torch.zeros_like(p)\n\n for (xi, yi), (xj, yj) in random_pairs_of_minibatches(minibatches):\n # fine tune clone-network on task \"i\"\n inner_net = copy.deepcopy(self.network)\n\n inner_opt = get_optimizer(\n self.hparams[\"optimizer\"],\n # \"SGD\",\n inner_net.parameters(),\n lr=self.hparams[\"lr\"],\n weight_decay=self.hparams[\"weight_decay\"],\n )\n\n inner_obj = F.cross_entropy(inner_net(xi), yi)\n\n inner_opt.zero_grad()\n inner_obj.backward()\n inner_opt.step()\n\n # 1. Compute supervised loss for meta-train set\n # The network has now accumulated gradients Gi\n # The clone-network has now parameters P - lr * Gi\n for p_tgt, p_src in zip(self.network.parameters(), inner_net.parameters()):\n if p_src.grad is not None:\n p_tgt.grad.data.add_(p_src.grad.data / num_mb)\n\n # `objective` is populated for reporting purposes\n objective += inner_obj.item()\n\n # 2. Compute meta loss for meta-val set\n # this computes Gj on the clone-network\n loss_inner_j = F.cross_entropy(inner_net(xj), yj)\n grad_inner_j = autograd.grad(loss_inner_j, inner_net.parameters(), allow_unused=True)\n\n # `objective` is populated for reporting purposes\n objective += (self.hparams[\"mldg_beta\"] * loss_inner_j).item()\n\n for p, g_j in zip(self.network.parameters(), grad_inner_j):\n if g_j is not None:\n p.grad.data.add_(self.hparams[\"mldg_beta\"] * g_j.data / num_mb)\n\n # The network has now accumulated gradients Gi + beta * Gj\n # Repeat for all train-test splits, do .step()\n\n objective /= len(minibatches)\n\n self.optimizer.step()\n\n return {\"loss\": objective}\n\n\n# class SOMLDG(MLDG):\n# \"\"\"Second-order MLDG\"\"\"\n# # This commented \"update\" method back-propagates through the gradients of\n# # the inner update, as suggested in the original MAML paper. However, this\n# # is twice as expensive as the uncommented \"update\" method, which does not\n# # compute second-order derivatives, implementing the First-Order MAML\n# # method (FOMAML) described in the original MAML paper.\n\n# def update(self, x, y, **kwargs):\n# minibatches = to_minibatch(x, y)\n# objective = 0\n# beta = self.hparams[\"mldg_beta\"]\n# inner_iterations = self.hparams.get(\"inner_iterations\", 1)\n\n# self.optimizer.zero_grad()\n\n# with higher.innerloop_ctx(\n# self.network, self.optimizer, copy_initial_weights=False\n# ) as (inner_network, inner_optimizer):\n# for (xi, yi), (xj, yj) in random_pairs_of_minibatches(minibatches):\n# for inner_iteration in range(inner_iterations):\n# li = F.cross_entropy(inner_network(xi), yi)\n# inner_optimizer.step(li)\n\n# objective += F.cross_entropy(self.network(xi), yi)\n# objective += beta * F.cross_entropy(inner_network(xj), yj)\n\n# objective /= len(minibatches)\n# objective.backward()\n\n# self.optimizer.step()\n\n# return {\"loss\": objective.item()}\n\n\nclass AbstractMMD(ERM):\n \"\"\"\n Perform ERM while matching the pair-wise domain feature distributions\n using MMD (abstract class)\n \"\"\"\n\n def __init__(self, input_shape, num_classes, num_domains, hparams, gaussian):\n super(AbstractMMD, self).__init__(input_shape, num_classes, num_domains, hparams)\n if gaussian:\n self.kernel_type = \"gaussian\"\n else:\n self.kernel_type = \"mean_cov\"\n\n def my_cdist(self, x1, x2):\n x1_norm = x1.pow(2).sum(dim=-1, keepdim=True)\n x2_norm = x2.pow(2).sum(dim=-1, keepdim=True)\n res = torch.addmm(x2_norm.transpose(-2, -1), x1, x2.transpose(-2, -1), alpha=-2).add_(\n x1_norm\n )\n return res.clamp_min_(1e-30)\n\n def gaussian_kernel(self, x, y, gamma=(0.001, 0.01, 0.1, 1, 10, 100, 1000)):\n D = self.my_cdist(x, y)\n K = torch.zeros_like(D)\n\n for g in gamma:\n K.add_(torch.exp(D.mul(-g)))\n\n return K\n\n def mmd(self, x, y):\n if self.kernel_type == \"gaussian\":\n Kxx = self.gaussian_kernel(x, x).mean()\n Kyy = self.gaussian_kernel(y, y).mean()\n Kxy = self.gaussian_kernel(x, y).mean()\n return Kxx + Kyy - 2 * Kxy\n else:\n mean_x = x.mean(0, keepdim=True)\n mean_y = y.mean(0, keepdim=True)\n cent_x = x - mean_x\n cent_y = y - mean_y\n cova_x = (cent_x.t() @ cent_x) / (len(x) - 1)\n cova_y = (cent_y.t() @ cent_y) / (len(y) - 1)\n\n mean_diff = (mean_x - mean_y).pow(2).mean()\n cova_diff = (cova_x - cova_y).pow(2).mean()\n\n return mean_diff + cova_diff\n\n def update(self, x, y, **kwargs):\n minibatches = to_minibatch(x, y)\n objective = 0\n penalty = 0\n nmb = len(minibatches)\n\n features = [self.featurizer(xi) for xi, _ in minibatches]\n classifs = [self.classifier(fi) for fi in features]\n targets = [yi for _, yi in minibatches]\n\n for i in range(nmb):\n objective += F.cross_entropy(classifs[i], targets[i])\n for j in range(i + 1, nmb):\n penalty += self.mmd(features[i], features[j])\n\n objective /= nmb\n if nmb > 1:\n penalty /= nmb * (nmb - 1) / 2\n\n self.optimizer.zero_grad()\n (objective + (self.hparams[\"mmd_gamma\"] * penalty)).backward()\n self.optimizer.step()\n\n if torch.is_tensor(penalty):\n penalty = penalty.item()\n\n return {\"loss\": objective.item(), \"penalty\": penalty}\n\n\nclass MMD(AbstractMMD):\n \"\"\"\n MMD using Gaussian kernel\n \"\"\"\n\n def __init__(self, input_shape, num_classes, num_domains, hparams):\n super(MMD, self).__init__(input_shape, num_classes, num_domains, hparams, gaussian=True)\n\n\nclass CORAL(AbstractMMD):\n \"\"\"\n MMD using mean and covariance difference\n \"\"\"\n\n def __init__(self, input_shape, num_classes, num_domains, hparams):\n super(CORAL, self).__init__(input_shape, num_classes, num_domains, hparams, gaussian=False)\n\n\nclass MTL(Algorithm):\n \"\"\"\n A neural network version of\n Domain Generalization by Marginal Transfer Learning\n (https://arxiv.org/abs/1711.07910)\n \"\"\"\n\n def __init__(self, input_shape, num_classes, num_domains, hparams):\n super(MTL, self).__init__(input_shape, num_classes, num_domains, hparams)\n self.featurizer = networks.Featurizer(input_shape, self.hparams)\n self.classifier = nn.Linear(self.featurizer.n_outputs * 2, num_classes)\n self.optimizer = get_optimizer(\n hparams[\"optimizer\"],\n list(self.featurizer.parameters()) + list(self.classifier.parameters()),\n lr=self.hparams[\"lr\"],\n weight_decay=self.hparams[\"weight_decay\"],\n )\n\n self.register_buffer(\"embeddings\", torch.zeros(num_domains, self.featurizer.n_outputs))\n\n self.ema = self.hparams[\"mtl_ema\"]\n\n def update(self, x, y, **kwargs):\n minibatches = to_minibatch(x, y)\n loss = 0\n for env, (x, y) in enumerate(minibatches):\n loss += F.cross_entropy(self.predict(x, env), y)\n\n self.optimizer.zero_grad()\n loss.backward()\n self.optimizer.step()\n\n return {\"loss\": loss.item()}\n\n def update_embeddings_(self, features, env=None):\n return_embedding = features.mean(0)\n\n if env is not None:\n return_embedding = self.ema * return_embedding + (1 - self.ema) * self.embeddings[env]\n\n self.embeddings[env] = return_embedding.clone().detach()\n\n return return_embedding.view(1, -1).repeat(len(features), 1)\n\n def predict(self, x, env=None):\n features = self.featurizer(x)\n embedding = self.update_embeddings_(features, env).normal_()\n return self.classifier(torch.cat((features, embedding), 1))\n\n\nclass SagNet(Algorithm):\n \"\"\"\n Style Agnostic Network\n Algorithm 1 from: https://arxiv.org/abs/1910.11645\n \"\"\"\n\n def __init__(self, input_shape, num_classes, num_domains, hparams):\n super(SagNet, self).__init__(input_shape, num_classes, num_domains, hparams)\n # featurizer network\n self.network_f = networks.Featurizer(input_shape, self.hparams)\n # content network\n self.network_c = nn.Linear(self.network_f.n_outputs, num_classes)\n # style network\n self.network_s = nn.Linear(self.network_f.n_outputs, num_classes)\n\n # # This commented block of code implements something closer to the\n # # original paper, but is specific to ResNet and puts in disadvantage\n # # the other algorithms.\n # resnet_c = networks.Featurizer(input_shape, self.hparams)\n # resnet_s = networks.Featurizer(input_shape, self.hparams)\n # # featurizer network\n # self.network_f = torch.nn.Sequential(\n # resnet_c.network.conv1,\n # resnet_c.network.bn1,\n # resnet_c.network.relu,\n # resnet_c.network.maxpool,\n # resnet_c.network.layer1,\n # resnet_c.network.layer2,\n # resnet_c.network.layer3)\n # # content network\n # self.network_c = torch.nn.Sequential(\n # resnet_c.network.layer4,\n # resnet_c.network.avgpool,\n # networks.Flatten(),\n # resnet_c.network.fc)\n # # style network\n # self.network_s = torch.nn.Sequential(\n # resnet_s.network.layer4,\n # resnet_s.network.avgpool,\n # networks.Flatten(),\n # resnet_s.network.fc)\n\n def opt(p):\n return get_optimizer(\n hparams[\"optimizer\"], p, lr=hparams[\"lr\"], weight_decay=hparams[\"weight_decay\"]\n )\n\n self.optimizer_f = opt(self.network_f.parameters())\n self.optimizer_c = opt(self.network_c.parameters())\n self.optimizer_s = opt(self.network_s.parameters())\n self.weight_adv = hparams[\"sag_w_adv\"]\n\n def forward_c(self, x):\n # learning content network on randomized style\n return self.network_c(self.randomize(self.network_f(x), \"style\"))\n\n def forward_s(self, x):\n # learning style network on randomized content\n return self.network_s(self.randomize(self.network_f(x), \"content\"))\n\n def randomize(self, x, what=\"style\", eps=1e-5):\n sizes = x.size()\n alpha = torch.rand(sizes[0], 1).cuda()\n\n if len(sizes) == 4:\n x = x.view(sizes[0], sizes[1], -1)\n alpha = alpha.unsqueeze(-1)\n\n mean = x.mean(-1, keepdim=True)\n var = x.var(-1, keepdim=True)\n\n x = (x - mean) / (var + eps).sqrt()\n\n idx_swap = torch.randperm(sizes[0])\n if what == \"style\":\n mean = alpha * mean + (1 - alpha) * mean[idx_swap]\n var = alpha * var + (1 - alpha) * var[idx_swap]\n else:\n x = x[idx_swap].detach()\n\n x = x * (var + eps).sqrt() + mean\n return x.view(*sizes)\n\n def update(self, x, y, **kwargs):\n all_x = torch.cat([xi for xi in x])\n all_y = torch.cat([yi for yi in y])\n\n # learn content\n self.optimizer_f.zero_grad()\n self.optimizer_c.zero_grad()\n loss_c = F.cross_entropy(self.forward_c(all_x), all_y)\n loss_c.backward()\n self.optimizer_f.step()\n self.optimizer_c.step()\n\n # learn style\n self.optimizer_s.zero_grad()\n loss_s = F.cross_entropy(self.forward_s(all_x), all_y)\n loss_s.backward()\n self.optimizer_s.step()\n\n # learn adversary\n self.optimizer_f.zero_grad()\n loss_adv = -F.log_softmax(self.forward_s(all_x), dim=1).mean(1).mean()\n loss_adv = loss_adv * self.weight_adv\n loss_adv.backward()\n self.optimizer_f.step()\n\n return {\n \"loss_c\": loss_c.item(),\n \"loss_s\": loss_s.item(),\n \"loss_adv\": loss_adv.item(),\n }\n\n def predict(self, x):\n return self.network_c(self.network_f(x))\n\n\nclass RSC(ERM):\n def __init__(self, input_shape, num_classes, num_domains, hparams):\n super(RSC, self).__init__(input_shape, num_classes, num_domains, hparams)\n self.drop_f = (1 - hparams[\"rsc_f_drop_factor\"]) * 100\n self.drop_b = (1 - hparams[\"rsc_b_drop_factor\"]) * 100\n self.num_classes = num_classes\n\n def update(self, x, y, **kwargs):\n # inputs\n all_x = torch.cat([xi for xi in x])\n # labels\n all_y = torch.cat([yi for yi in y])\n # one-hot labels\n all_o = torch.nn.functional.one_hot(all_y, self.num_classes)\n # features\n all_f = self.featurizer(all_x)\n # predictions\n all_p = self.classifier(all_f)\n\n # Equation (1): compute gradients with respect to representation\n all_g = autograd.grad((all_p * all_o).sum(), all_f)[0]\n\n # Equation (2): compute top-gradient-percentile mask\n percentiles = np.percentile(all_g.cpu(), self.drop_f, axis=1)\n percentiles = torch.Tensor(percentiles)\n percentiles = percentiles.unsqueeze(1).repeat(1, all_g.size(1))\n mask_f = all_g.lt(percentiles.cuda()).float()\n\n # Equation (3): mute top-gradient-percentile activations\n all_f_muted = all_f * mask_f\n\n # Equation (4): compute muted predictions\n all_p_muted = self.classifier(all_f_muted)\n\n # Section 3.3: Batch Percentage\n all_s = F.softmax(all_p, dim=1)\n all_s_muted = F.softmax(all_p_muted, dim=1)\n changes = (all_s * all_o).sum(1) - (all_s_muted * all_o).sum(1)\n percentile = np.percentile(changes.detach().cpu(), self.drop_b)\n mask_b = changes.lt(percentile).float().view(-1, 1)\n mask = torch.logical_or(mask_f, mask_b).float()\n\n # Equations (3) and (4) again, this time mutting over examples\n all_p_muted_again = self.classifier(all_f * mask)\n\n # Equation (5): update\n loss = F.cross_entropy(all_p_muted_again, all_y)\n self.optimizer.zero_grad()\n loss.backward()\n self.optimizer.step()\n\n return {\"loss\": loss.item()}\n"
},
{
"path": "domain_generalization/swad/domainbed/datasets/__init__.py",
"content": "import torch\nimport numpy as np\n\nfrom domainbed.datasets import datasets\nfrom domainbed.lib import misc\nfrom domainbed.datasets import transforms as DBT\n\n\ndef set_transfroms(dset, data_type, hparams, algorithm_class=None):\n \"\"\"\n Args:\n data_type: ['train', 'valid', 'test', 'mnist']\n \"\"\"\n assert hparams[\"data_augmentation\"]\n\n additional_data = False\n if data_type == \"train\":\n dset.transforms = {\"x\": DBT.aug}\n additional_data = True\n elif data_type == \"valid\":\n if hparams[\"val_augment\"] is False:\n dset.transforms = {\"x\": DBT.basic}\n else:\n # Originally, DomainBed use same training augmentation policy to validation.\n # We turn off the augmentation for validation as default,\n # but left the option to reproducibility.\n dset.transforms = {\"x\": DBT.aug}\n elif data_type == \"test\":\n dset.transforms = {\"x\": DBT.basic}\n elif data_type == \"mnist\":\n # No augmentation for mnist\n dset.transforms = {\"x\": lambda x: x}\n else:\n raise ValueError(data_type)\n\n if additional_data and algorithm_class is not None:\n for key, transform in algorithm_class.transforms.items():\n dset.transforms[key] = transform\n\n\ndef get_dataset(test_envs, args, hparams, algorithm_class=None):\n \"\"\"Get dataset and split.\"\"\"\n is_mnist = \"MNIST\" in args.dataset\n dataset = vars(datasets)[args.dataset](args.data_dir)\n # if not isinstance(dataset, MultipleEnvironmentImageFolder):\n # raise ValueError(\"SMALL image datasets are not implemented (corrupted), for transform.\")\n\n in_splits = []\n out_splits = []\n for env_i, env in enumerate(dataset):\n # The split only depends on seed_hash (= trial_seed).\n # It means that the split is always identical only if use same trial_seed,\n # independent to run the code where, when, or how many times.\n out, in_ = split_dataset(\n env,\n int(len(env) * args.holdout_fraction),\n misc.seed_hash(args.trial_seed, env_i),\n )\n if env_i in test_envs:\n in_type = \"test\"\n out_type = \"test\"\n else:\n in_type = \"train\"\n out_type = \"valid\"\n\n if is_mnist:\n in_type = \"mnist\"\n out_type = \"mnist\"\n\n set_transfroms(in_, in_type, hparams, algorithm_class)\n set_transfroms(out, out_type, hparams, algorithm_class)\n\n if hparams[\"class_balanced\"]:\n in_weights = misc.make_weights_for_balanced_classes(in_)\n out_weights = misc.make_weights_for_balanced_classes(out)\n else:\n in_weights, out_weights = None, None\n in_splits.append((in_, in_weights))\n out_splits.append((out, out_weights))\n\n return dataset, in_splits, out_splits\n\n\nclass _SplitDataset(torch.utils.data.Dataset):\n \"\"\"Used by split_dataset\"\"\"\n\n def __init__(self, underlying_dataset, keys):\n super(_SplitDataset, self).__init__()\n self.underlying_dataset = underlying_dataset\n self.keys = keys\n self.transforms = {}\n\n self.direct_return = isinstance(underlying_dataset, _SplitDataset)\n\n def __getitem__(self, key):\n if self.direct_return:\n return self.underlying_dataset[self.keys[key]]\n\n x, y = self.underlying_dataset[self.keys[key]]\n ret = {\"y\": y}\n\n for key, transform in self.transforms.items():\n ret[key] = transform(x)\n\n return ret\n\n def __len__(self):\n return len(self.keys)\n\n\ndef split_dataset(dataset, n, seed=0):\n \"\"\"\n Return a pair of datasets corresponding to a random split of the given\n dataset, with n datapoints in the first dataset and the rest in the last,\n using the given random seed\n \"\"\"\n assert n <= len(dataset)\n keys = list(range(len(dataset)))\n np.random.RandomState(seed).shuffle(keys)\n keys_1 = keys[:n]\n keys_2 = keys[n:]\n return _SplitDataset(dataset, keys_1), _SplitDataset(dataset, keys_2)\n"
},
{
"path": "domain_generalization/swad/domainbed/datasets/datasets.py",
"content": "# Copyright (c) Facebook, Inc. and its affiliates. All Rights Reserved\n\nimport os\nimport torch\nfrom PIL import Image, ImageFile\nfrom torchvision import transforms as T\nfrom torch.utils.data import TensorDataset\nfrom torchvision.datasets import MNIST, ImageFolder\nfrom torchvision.transforms.functional import rotate\n\nImageFile.LOAD_TRUNCATED_IMAGES = True\n\nDATASETS = [\n # Debug\n \"Debug28\",\n \"Debug224\",\n # Small images\n \"ColoredMNIST\",\n \"RotatedMNIST\",\n # Big images\n \"VLCS\",\n \"PACS\",\n \"OfficeHome\",\n \"TerraIncognita\",\n \"DomainNet\",\n]\n\n\ndef get_dataset_class(dataset_name):\n \"\"\"Return the dataset class with the given name.\"\"\"\n if dataset_name not in globals():\n raise NotImplementedError(\"Dataset not found: {}\".format(dataset_name))\n return globals()[dataset_name]\n\n\ndef num_environments(dataset_name):\n return len(get_dataset_class(dataset_name).ENVIRONMENTS)\n\n\nclass MultipleDomainDataset:\n N_STEPS = 5001 # Default, subclasses may override\n CHECKPOINT_FREQ = 100 # Default, subclasses may override\n N_WORKERS = 4 # Default, subclasses may override\n ENVIRONMENTS = None # Subclasses should override\n INPUT_SHAPE = None # Subclasses should override\n\n def __getitem__(self, index):\n \"\"\"\n Return: sub-dataset for specific domain\n \"\"\"\n return self.datasets[index]\n\n def __len__(self):\n \"\"\"\n Return: # of sub-datasets\n \"\"\"\n return len(self.datasets)\n\n\nclass Debug(MultipleDomainDataset):\n def __init__(self, root):\n super().__init__()\n self.input_shape = self.INPUT_SHAPE\n self.num_classes = 2\n self.datasets = []\n for _ in [0, 1, 2]:\n self.datasets.append(\n TensorDataset(\n torch.randn(16, *self.INPUT_SHAPE),\n torch.randint(0, self.num_classes, (16,)),\n )\n )\n\n\nclass Debug28(Debug):\n INPUT_SHAPE = (3, 28, 28)\n ENVIRONMENTS = [\"0\", \"1\", \"2\"]\n\n\nclass Debug224(Debug):\n INPUT_SHAPE = (3, 224, 224)\n ENVIRONMENTS = [\"0\", \"1\", \"2\"]\n\n\nclass MultipleEnvironmentMNIST(MultipleDomainDataset):\n def __init__(self, root, environments, dataset_transform, input_shape, num_classes):\n \"\"\"\n Args:\n root: root dir for saving MNIST dataset\n environments: env properties for each dataset\n dataset_transform: dataset generator function\n \"\"\"\n super().__init__()\n if root is None:\n raise ValueError(\"Data directory not specified!\")\n\n original_dataset_tr = MNIST(root, train=True, download=True)\n original_dataset_te = MNIST(root, train=False, download=True)\n\n original_images = torch.cat((original_dataset_tr.data, original_dataset_te.data))\n\n original_labels = torch.cat((original_dataset_tr.targets, original_dataset_te.targets))\n\n shuffle = torch.randperm(len(original_images))\n\n original_images = original_images[shuffle]\n original_labels = original_labels[shuffle]\n\n self.datasets = []\n self.environments = environments\n\n for i in range(len(environments)):\n images = original_images[i :: len(environments)]\n labels = original_labels[i :: len(environments)]\n self.datasets.append(dataset_transform(images, labels, environments[i]))\n\n self.input_shape = input_shape\n self.num_classes = num_classes\n\n\nclass ColoredMNIST(MultipleEnvironmentMNIST):\n ENVIRONMENTS = [\"+90%\", \"+80%\", \"-90%\"]\n\n def __init__(self, root):\n super(ColoredMNIST, self).__init__(\n root,\n [0.1, 0.2, 0.9],\n self.color_dataset,\n (2, 28, 28),\n 2,\n )\n\n def color_dataset(self, images, labels, environment):\n # # Subsample 2x for computational convenience\n # images = images.reshape((-1, 28, 28))[:, ::2, ::2]\n # Assign a binary label based on the digit\n labels = (labels < 5).float()\n # Flip label with probability 0.25\n labels = self.torch_xor_(labels, self.torch_bernoulli_(0.25, len(labels)))\n\n # Assign a color based on the label; flip the color with probability e\n colors = self.torch_xor_(labels, self.torch_bernoulli_(environment, len(labels)))\n images = torch.stack([images, images], dim=1)\n # Apply the color to the image by zeroing out the other color channel\n images[torch.tensor(range(len(images))), (1 - colors).long(), :, :] *= 0\n\n x = images.float().div_(255.0)\n y = labels.view(-1).long()\n\n return TensorDataset(x, y)\n\n def torch_bernoulli_(self, p, size):\n return (torch.rand(size) < p).float()\n\n def torch_xor_(self, a, b):\n return (a - b).abs()\n\n\nclass RotatedMNIST(MultipleEnvironmentMNIST):\n ENVIRONMENTS = [\"0\", \"15\", \"30\", \"45\", \"60\", \"75\"]\n\n def __init__(self, root):\n super(RotatedMNIST, self).__init__(\n root,\n [0, 15, 30, 45, 60, 75],\n self.rotate_dataset,\n (1, 28, 28),\n 10,\n )\n\n def rotate_dataset(self, images, labels, angle):\n rotation = T.Compose(\n [\n T.ToPILImage(),\n T.Lambda(lambda x: rotate(x, angle, fill=(0,), resample=Image.BICUBIC)),\n T.ToTensor(),\n ]\n )\n\n x = torch.zeros(len(images), 1, 28, 28)\n for i in range(len(images)):\n x[i] = rotation(images[i])\n\n y = labels.view(-1)\n\n return TensorDataset(x, y)\n\n\nclass MultipleEnvironmentImageFolder(MultipleDomainDataset):\n def __init__(self, root):\n super().__init__()\n environments = [f.name for f in os.scandir(root) if f.is_dir()]\n environments = sorted(environments)\n self.environments = environments\n\n self.datasets = []\n for environment in environments:\n path = os.path.join(root, environment)\n env_dataset = ImageFolder(path)\n\n self.datasets.append(env_dataset)\n\n self.input_shape = (3, 224, 224)\n self.num_classes = len(self.datasets[-1].classes)\n\n\nclass VLCS(MultipleEnvironmentImageFolder):\n CHECKPOINT_FREQ = 200\n ENVIRONMENTS = [\"C\", \"L\", \"S\", \"V\"]\n\n def __init__(self, root):\n self.dir = os.path.join(root, \"VLCS/\")\n super().__init__(self.dir)\n\n\nclass PACS(MultipleEnvironmentImageFolder):\n CHECKPOINT_FREQ = 200\n ENVIRONMENTS = [\"A\", \"C\", \"P\", \"S\"]\n\n def __init__(self, root):\n self.dir = os.path.join(root, \"PACS/\")\n super().__init__(self.dir)\n\n\nclass DomainNet(MultipleEnvironmentImageFolder):\n CHECKPOINT_FREQ = 1000\n N_STEPS = 15001\n ENVIRONMENTS = [\"clip\", \"info\", \"paint\", \"quick\", \"real\", \"sketch\"]\n\n def __init__(self, root):\n self.dir = os.path.join(root, \"domain_net/\")\n super().__init__(self.dir)\n\n\nclass OfficeHome(MultipleEnvironmentImageFolder):\n CHECKPOINT_FREQ = 200\n ENVIRONMENTS = [\"A\", \"C\", \"P\", \"R\"]\n\n def __init__(self, root):\n self.dir = os.path.join(root, \"office_home/\")\n super().__init__(self.dir)\n\n\nclass TerraIncognita(MultipleEnvironmentImageFolder):\n CHECKPOINT_FREQ = 200\n ENVIRONMENTS = [\"L100\", \"L38\", \"L43\", \"L46\"]\n\n def __init__(self, root):\n self.dir = os.path.join(root, \"terra_incognita/\")\n super().__init__(self.dir)\n"
},
{
"path": "domain_generalization/swad/domainbed/datasets/transforms.py",
"content": "from torchvision import transforms as T\n\n\nbasic = T.Compose(\n [\n T.Resize((224, 224)),\n T.ToTensor(),\n T.Normalize(mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225]),\n ]\n)\naug = T.Compose(\n [\n T.RandomResizedCrop(224, scale=(0.7, 1.0)),\n T.RandomHorizontalFlip(),\n T.ColorJitter(0.3, 0.3, 0.3, 0.3),\n T.RandomGrayscale(p=0.1),\n T.ToTensor(),\n T.Normalize(mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225]),\n ]\n)\n"
},
{
"path": "domain_generalization/swad/domainbed/evaluator.py",
"content": "import collections\nimport torch\nimport torch.nn as nn\nimport torch.nn.functional as F\nfrom domainbed.lib.fast_data_loader import FastDataLoader\n\nif torch.cuda.is_available():\n device = \"cuda\"\nelse:\n device = \"cpu\"\n\n\ndef accuracy_from_loader(algorithm, loader, weights, debug=False):\n correct = 0\n total = 0\n losssum = 0.0\n weights_offset = 0\n\n algorithm.eval()\n\n for i, batch in enumerate(loader):\n x = batch[\"x\"].to(device)\n y = batch[\"y\"].to(device)\n\n with torch.no_grad():\n logits = algorithm.predict(x)\n loss = F.cross_entropy(logits, y).item()\n\n B = len(x)\n losssum += loss * B\n\n if weights is None:\n batch_weights = torch.ones(len(x))\n else:\n batch_weights = weights[weights_offset : weights_offset + len(x)]\n weights_offset += len(x)\n batch_weights = batch_weights.to(device)\n if logits.size(1) == 1:\n correct += (logits.gt(0).eq(y).float() * batch_weights).sum().item()\n else:\n correct += (logits.argmax(1).eq(y).float() * batch_weights).sum().item()\n total += batch_weights.sum().item()\n\n if debug:\n break\n\n algorithm.train()\n\n acc = correct / total\n loss = losssum / total\n return acc, loss\n\n\ndef accuracy(algorithm, loader_kwargs, weights, **kwargs):\n if isinstance(loader_kwargs, dict):\n loader = FastDataLoader(**loader_kwargs)\n elif isinstance(loader_kwargs, FastDataLoader):\n loader = loader_kwargs\n else:\n raise ValueError(loader_kwargs)\n return accuracy_from_loader(algorithm, loader, weights, **kwargs)\n\n\nclass Evaluator:\n def __init__(\n self, test_envs, eval_meta, n_envs, logger, evalmode=\"fast\", debug=False, target_env=None\n ):\n all_envs = list(range(n_envs))\n train_envs = sorted(set(all_envs) - set(test_envs))\n self.test_envs = test_envs\n self.train_envs = train_envs\n self.eval_meta = eval_meta\n self.n_envs = n_envs\n self.logger = logger\n self.evalmode = evalmode\n self.debug = debug\n\n if target_env is not None:\n self.set_target_env(target_env)\n\n def set_target_env(self, target_env):\n \"\"\"When len(test_envs) == 2, you can specify target env for computing exact test acc.\"\"\"\n self.test_envs = [target_env]\n\n def evaluate(self, algorithm, ret_losses=False):\n n_train_envs = len(self.train_envs)\n n_test_envs = len(self.test_envs)\n assert n_test_envs == 1\n summaries = collections.defaultdict(float)\n # for key order\n summaries[\"test_in\"] = 0.0\n summaries[\"test_out\"] = 0.0\n summaries[\"train_in\"] = 0.0\n summaries[\"train_out\"] = 0.0\n accuracies = {}\n losses = {}\n\n # order: in_splits + out_splits.\n for name, loader_kwargs, weights in self.eval_meta:\n # env\\d_[in|out]\n env_name, inout = name.split(\"_\")\n env_num = int(env_name[3:])\n\n skip_eval = self.evalmode == \"fast\" and inout == \"in\" and env_num not in self.test_envs\n if skip_eval:\n continue\n\n is_test = env_num in self.test_envs\n acc, loss = accuracy(algorithm, loader_kwargs, weights, debug=self.debug)\n accuracies[name] = acc\n losses[name] = loss\n\n if env_num in self.train_envs:\n summaries[\"train_\" + inout] += acc / n_train_envs\n if inout == \"out\":\n summaries[\"tr_\" + inout + \"loss\"] += loss / n_train_envs\n elif is_test:\n summaries[\"test_\" + inout] += acc / n_test_envs\n\n if ret_losses:\n return accuracies, summaries, losses\n else:\n return accuracies, summaries\n"
},
{
"path": "domain_generalization/swad/domainbed/hparams_registry.py",
"content": "# Copyright (c) Facebook, Inc. and its affiliates. All Rights Reserved\n\nimport numpy as np\n\n\ndef _hparams(algorithm, dataset, random_state):\n \"\"\"\n Global registry of hyperparams. Each entry is a (default, random) tuple.\n New algorithms / networks / etc. should add entries here.\n \"\"\"\n SMALL_IMAGES = [\"Debug28\", \"RotatedMNIST\", \"ColoredMNIST\"]\n\n hparams = {}\n\n hparams[\"data_augmentation\"] = (True, True)\n hparams[\"val_augment\"] = (False, False) # augmentation for in-domain validation set\n hparams[\"resnet18\"] = (False, False)\n hparams[\"resnet_dropout\"] = (0.0, random_state.choice([0.0, 0.1, 0.5]))\n hparams[\"class_balanced\"] = (False, False)\n hparams[\"scheduler\"] = (\"const\", \"const\")\n hparams[\"optimizer\"] = (\"adam\", \"adam\")\n\n hparams[\"freeze_bn\"] = (True, True)\n hparams[\"pretrained\"] = (True, True) # only for ResNet\n\n if dataset not in SMALL_IMAGES:\n hparams[\"lr\"] = (5e-5, 10 ** random_state.uniform(-5, -3.5))\n if dataset == \"DomainNet\":\n hparams[\"batch_size\"] = (32, int(2 ** random_state.uniform(3, 5)))\n else:\n hparams[\"batch_size\"] = (32, int(2 ** random_state.uniform(3, 5.5)))\n if algorithm == \"ARM\":\n hparams[\"batch_size\"] = (8, 8)\n else:\n hparams[\"lr\"] = (1e-3, 10 ** random_state.uniform(-4.5, -2.5))\n hparams[\"batch_size\"] = (64, int(2 ** random_state.uniform(3, 9)))\n\n if dataset in SMALL_IMAGES:\n hparams[\"weight_decay\"] = (0.0, 0.0)\n else:\n hparams[\"weight_decay\"] = (0.0, 10 ** random_state.uniform(-6, -2))\n\n if algorithm in [\"DANN\", \"CDANN\"]:\n if dataset not in SMALL_IMAGES:\n hparams[\"lr_g\"] = (5e-5, 10 ** random_state.uniform(-5, -3.5))\n hparams[\"lr_d\"] = (5e-5, 10 ** random_state.uniform(-5, -3.5))\n else:\n hparams[\"lr_g\"] = (1e-3, 10 ** random_state.uniform(-4.5, -2.5))\n hparams[\"lr_d\"] = (1e-3, 10 ** random_state.uniform(-4.5, -2.5))\n\n if dataset in SMALL_IMAGES:\n hparams[\"weight_decay_g\"] = (0.0, 0.0)\n else:\n hparams[\"weight_decay_g\"] = (0.0, 10 ** random_state.uniform(-6, -2))\n\n hparams[\"lambda\"] = (1.0, 10 ** random_state.uniform(-2, 2))\n hparams[\"weight_decay_d\"] = (0.0, 10 ** random_state.uniform(-6, -2))\n hparams[\"d_steps_per_g_step\"] = (1, int(2 ** random_state.uniform(0, 3)))\n hparams[\"grad_penalty\"] = (0.0, 10 ** random_state.uniform(-2, 1))\n hparams[\"beta1\"] = (0.5, random_state.choice([0.0, 0.5]))\n hparams[\"mlp_width\"] = (256, int(2 ** random_state.uniform(6, 10)))\n hparams[\"mlp_depth\"] = (3, int(random_state.choice([3, 4, 5])))\n hparams[\"mlp_dropout\"] = (0.0, random_state.choice([0.0, 0.1, 0.5]))\n elif algorithm == \"RSC\":\n hparams[\"rsc_f_drop_factor\"] = (1 / 3, random_state.uniform(0, 0.5))\n hparams[\"rsc_b_drop_factor\"] = (1 / 3, random_state.uniform(0, 0.5))\n elif algorithm == \"SagNet\":\n hparams[\"sag_w_adv\"] = (0.1, 10 ** random_state.uniform(-2, 1))\n elif algorithm == \"IRM\":\n hparams[\"irm_lambda\"] = (1e2, 10 ** random_state.uniform(-1, 5))\n hparams[\"irm_penalty_anneal_iters\"] = (\n 500,\n int(10 ** random_state.uniform(0, 4)),\n )\n elif algorithm in [\"Mixup\", \"OrgMixup\"]:\n hparams[\"mixup_alpha\"] = (0.2, 10 ** random_state.uniform(-1, -1))\n elif algorithm == \"GroupDRO\":\n hparams[\"groupdro_eta\"] = (1e-2, 10 ** random_state.uniform(-3, -1))\n elif algorithm in (\"MMD\", \"CORAL\"):\n hparams[\"mmd_gamma\"] = (1.0, 10 ** random_state.uniform(-1, 1))\n elif algorithm in (\"MLDG\", \"SOMLDG\"):\n hparams[\"mldg_beta\"] = (1.0, 10 ** random_state.uniform(-1, 1))\n elif algorithm == \"MTL\":\n hparams[\"mtl_ema\"] = (0.99, random_state.choice([0.5, 0.9, 0.99, 1.0]))\n elif algorithm == \"VREx\":\n hparams[\"vrex_lambda\"] = (1e1, 10 ** random_state.uniform(-1, 5))\n hparams[\"vrex_penalty_anneal_iters\"] = (\n 500,\n int(10 ** random_state.uniform(0, 4)),\n )\n elif algorithm == \"SAM\":\n hparams[\"rho\"] = (0.05, random_state.choice([0.01, 0.02, 0.05, 0.1]))\n elif algorithm == \"CutMix\":\n hparams[\"beta\"] = (1.0, 1.0)\n # cutmix_prob is set to 1.0 for ImageNet and 0.5 for CIFAR100 in the original paper.\n hparams[\"cutmix_prob\"] = (1.0, 1.0)\n\n return hparams\n\n\ndef default_hparams(algorithm, dataset):\n dummy_random_state = np.random.RandomState(0)\n return {a: b for a, (b, c) in _hparams(algorithm, dataset, dummy_random_state).items()}\n\n\ndef random_hparams(algorithm, dataset, seed):\n random_state = np.random.RandomState(seed)\n return {a: c for a, (b, c) in _hparams(algorithm, dataset, random_state).items()}\n"
},
{
"path": "domain_generalization/swad/domainbed/lib/fast_data_loader.py",
"content": "# Copyright (c) Facebook, Inc. and its affiliates. All Rights Reserved\n\nimport torch\n\n\nclass _InfiniteSampler(torch.utils.data.Sampler):\n \"\"\"Wraps another Sampler to yield an infinite stream.\"\"\"\n\n def __init__(self, sampler):\n self.sampler = sampler\n\n def __iter__(self):\n while True:\n for batch in self.sampler:\n yield batch\n\n\nclass InfiniteDataLoader:\n def __init__(self, dataset, weights, batch_size, num_workers):\n super().__init__()\n\n if weights:\n sampler = torch.utils.data.WeightedRandomSampler(\n weights, replacement=True, num_samples=batch_size\n )\n else:\n sampler = torch.utils.data.RandomSampler(dataset, replacement=True)\n\n batch_sampler = torch.utils.data.BatchSampler(\n sampler, batch_size=batch_size, drop_last=True\n )\n\n self._infinite_iterator = iter(\n torch.utils.data.DataLoader(\n dataset,\n num_workers=num_workers,\n batch_sampler=_InfiniteSampler(batch_sampler),\n )\n )\n\n def __iter__(self):\n while True:\n yield next(self._infinite_iterator)\n\n def __len__(self):\n raise ValueError\n\n\nclass FastDataLoader:\n \"\"\"\n DataLoader wrapper with slightly improved speed by not respawning worker\n processes at every epoch.\n \"\"\"\n\n def __init__(self, dataset, batch_size, num_workers, shuffle=False):\n super().__init__()\n\n if shuffle:\n sampler = torch.utils.data.RandomSampler(dataset, replacement=False)\n else:\n sampler = torch.utils.data.SequentialSampler(dataset)\n\n batch_sampler = torch.utils.data.BatchSampler(\n sampler,\n batch_size=batch_size,\n drop_last=False,\n )\n\n self._infinite_iterator = iter(\n torch.utils.data.DataLoader(\n dataset,\n num_workers=num_workers,\n batch_sampler=_InfiniteSampler(batch_sampler),\n )\n )\n\n self._length = len(batch_sampler)\n\n def __iter__(self):\n for _ in range(len(self)):\n yield next(self._infinite_iterator)\n\n def __len__(self):\n return self._length\n"
},
{
"path": "domain_generalization/swad/domainbed/lib/logger.py",
"content": "\"\"\" Singleton Logger \"\"\"\nimport sys\nimport logging\n\n\ndef levelize(levelname):\n \"\"\"Convert levelname to level only if it is levelname\"\"\"\n if isinstance(levelname, str):\n return logging.getLevelName(levelname)\n else:\n return levelname # already level\n\n\nclass ColorFormatter(logging.Formatter):\n color_dic = {\n \"DEBUG\": 37, # white\n \"INFO\": 36, # cyan\n \"WARNING\": 33, # yellow\n \"ERROR\": 31, # red\n \"CRITICAL\": 41, # white on red bg\n }\n\n def format(self, record):\n color = self.color_dic.get(record.levelname, 37) # default white\n record.levelname = \"\\033[{}m{}\\033[0m\".format(color, record.levelname)\n return logging.Formatter.format(self, record)\n\n\nclass Logger(logging.Logger):\n NAME = \"SingletonLogger\"\n\n @classmethod\n def get(cls, file_path=None, level=\"INFO\", colorize=True, track_code=False):\n logging.setLoggerClass(cls)\n logger = logging.getLogger(cls.NAME)\n logging.setLoggerClass(logging.Logger) # restore\n logger.setLevel(level)\n\n if logger.hasHandlers():\n # If logger already got all handlers (# handlers == 2), use the logger.\n # else, re-set handlers.\n if len(logger.handlers) == 2:\n return logger\n\n logger.handlers.clear()\n\n log_format = \"%(levelname)s %(asctime)s | %(message)s\"\n # log_format = '%(asctime)s | %(message)s'\n if track_code:\n log_format = (\n \"%(levelname)s::%(asctime)s | [%(filename)s] [%(funcName)s:%(lineno)d] \"\n \"%(message)s\"\n )\n date_format = \"%m/%d %H:%M:%S\"\n if colorize:\n formatter = ColorFormatter(log_format, date_format)\n else:\n formatter = logging.Formatter(log_format, date_format)\n\n # standard output handler\n # NOTE as default, StreamHandler use stderr stream instead of stdout stream.\n # Use StreamHandler(sys.stdout) for stdout stream.\n stream_handler = logging.StreamHandler(sys.stdout)\n stream_handler.setFormatter(formatter)\n logger.addHandler(stream_handler)\n\n if file_path:\n # file output handler\n file_handler = logging.FileHandler(file_path)\n file_handler.setFormatter(formatter)\n logger.addHandler(file_handler)\n\n logger.propagate = False\n\n return logger\n\n def nofmt(self, msg, *args, level=\"INFO\", **kwargs):\n level = levelize(level)\n formatters = self.remove_formats()\n super().log(level, msg, *args, **kwargs)\n self.set_formats(formatters)\n\n def remove_formats(self):\n \"\"\"Remove all formats from logger\"\"\"\n formatters = []\n for handler in self.handlers:\n formatters.append(handler.formatter)\n handler.setFormatter(logging.Formatter(\"%(message)s\"))\n\n return formatters\n\n def set_formats(self, formatters):\n \"\"\"Set formats to every handler of logger\"\"\"\n for handler, formatter in zip(self.handlers, formatters):\n handler.setFormatter(formatter)\n\n def set_file_handler(self, file_path):\n file_handler = logging.FileHandler(file_path)\n formatter = self.handlers[0].formatter\n file_handler.setFormatter(formatter)\n self.addHandler(file_handler)\n"
},
{
"path": "domain_generalization/swad/domainbed/lib/misc.py",
"content": "# Copyright (c) Facebook, Inc. and its affiliates. All Rights Reserved\n\n\"\"\"\nThings that don't belong anywhere else\n\"\"\"\n\nimport hashlib\nimport os\nimport shutil\nimport errno\nfrom datetime import datetime\nfrom collections import Counter\n\nimport numpy as np\nimport torch\nimport torch.nn as nn\n\n\ndef make_weights_for_balanced_classes(dataset):\n counts = Counter()\n classes = []\n for _, y in dataset:\n y = int(y)\n counts[y] += 1\n classes.append(y)\n\n n_classes = len(counts)\n\n weight_per_class = {}\n for y in counts:\n weight_per_class[y] = 1 / (counts[y] * n_classes)\n\n weights = torch.zeros(len(dataset))\n for i, y in enumerate(classes):\n weights[i] = weight_per_class[int(y)]\n\n return weights\n\n\ndef seed_hash(*args):\n \"\"\"\n Derive an integer hash from all args, for use as a random seed.\n \"\"\"\n args_str = str(args)\n return int(hashlib.md5(args_str.encode(\"utf-8\")).hexdigest(), 16) % (2 ** 31)\n\n\ndef to_row(row, colwidth=10, latex=False):\n \"\"\"Convert value list to row string\"\"\"\n if latex:\n sep = \" & \"\n end_ = \"\\\\\\\\\"\n else:\n sep = \" \"\n end_ = \"\"\n\n def format_val(x):\n if np.issubdtype(type(x), np.floating):\n x = \"{:.6f}\".format(x)\n return str(x).ljust(colwidth)[:colwidth]\n\n return sep.join([format_val(x) for x in row]) + \" \" + end_\n\n\ndef random_pairs_of_minibatches(minibatches):\n # n_tr_envs = len(minibatches)\n perm = torch.randperm(len(minibatches)).tolist()\n pairs = []\n\n for i in range(len(minibatches)):\n # j = cyclic(i + 1)\n j = i + 1 if i < (len(minibatches) - 1) else 0\n\n xi, yi = minibatches[perm[i]][0], minibatches[perm[i]][1]\n xj, yj = minibatches[perm[j]][0], minibatches[perm[j]][1]\n\n min_n = min(len(xi), len(xj))\n\n pairs.append(((xi[:min_n], yi[:min_n]), (xj[:min_n], yj[:min_n])))\n\n return pairs\n\n\n###########################################################\n# Custom utils\n###########################################################\n\n\ndef index_conditional_iterate(skip_condition, iterable, index):\n for i, x in enumerate(iterable):\n if skip_condition(i):\n continue\n\n if index:\n yield i, x\n else:\n yield x\n\n\nclass SplitIterator:\n def __init__(self, test_envs):\n self.test_envs = test_envs\n\n def train(self, iterable, index=False):\n return index_conditional_iterate(lambda idx: idx in self.test_envs, iterable, index)\n\n def test(self, iterable, index=False):\n return index_conditional_iterate(lambda idx: idx not in self.test_envs, iterable, index)\n\n\nclass AverageMeter:\n \"\"\"Computes and stores the average and current value\"\"\"\n\n def __init__(self):\n self.reset()\n\n def reset(self):\n \"\"\"Reset all statistics\"\"\"\n self.val = 0\n self.avg = 0\n self.sum = 0\n self.count = 0\n\n def update(self, val, n=1):\n \"\"\"Update statistics\"\"\"\n self.val = val\n self.sum += val * n\n self.count += n\n self.avg = self.sum / self.count\n\n def __repr__(self):\n return \"{:.3f} (val={:.3f}, count={})\".format(self.avg, self.val, self.count)\n\n\nclass AverageMeters:\n def __init__(self, *keys):\n self.keys = keys\n for k in keys:\n setattr(self, k, AverageMeter())\n\n def resets(self):\n for k in self.keys:\n getattr(self, k).reset()\n\n def updates(self, dic, n=1):\n for k, v in dic.items():\n getattr(self, k).update(v, n)\n\n def __repr__(self):\n return \" \".join([\"{}: {}\".format(k, str(getattr(self, k))) for k in self.keys])\n\n def get_averages(self):\n dic = {k: getattr(self, k).avg for k in self.keys}\n return dic\n\n\ndef timestamp(fmt=\"%y%m%d_%H-%M-%S\"):\n return datetime.now().strftime(fmt)\n\n\ndef makedirs(path):\n if not os.path.exists(path):\n try:\n os.makedirs(path)\n except OSError as exc:\n if exc.errno != errno.EEXIST:\n raise\n\n\ndef rm(path):\n \"\"\"remove dir recursively\"\"\"\n if os.path.isdir(path):\n shutil.rmtree(path, ignore_errors=True)\n elif os.path.exists(path):\n os.remove(path)\n\n\ndef cp(src, dst):\n shutil.copy2(src, dst)\n\n\ndef get_lr(optimizer):\n \"\"\"Assume that the optimizer has single lr\"\"\"\n lr = optimizer.param_groups[0][\"lr\"]\n\n return lr\n\n\n@torch.no_grad()\ndef hash_bn(module):\n summary = []\n for m in module.modules():\n if isinstance(m, (nn.BatchNorm1d, nn.BatchNorm2d, nn.BatchNorm3d)):\n w = m.weight.detach().mean().item()\n b = m.bias.detach().mean().item()\n rm = m.running_mean.detach().mean().item()\n rv = m.running_var.detach().mean().item()\n summary.append((w, b, rm, rv))\n w, b, rm, rv = [np.mean(col) for col in zip(*summary)]\n\n return w, b, rm, rv\n\n\ndef merge_dictlist(dictlist):\n \"\"\"Merge list of dicts into dict of lists, by grouping same key.\"\"\"\n ret = {k: [] for k in dictlist[0].keys()}\n for dic in dictlist:\n for data_key, v in dic.items():\n ret[data_key].append(v)\n return ret\n"
},
{
"path": "domain_generalization/swad/domainbed/lib/query.py",
"content": "# Copyright (c) Facebook, Inc. and its affiliates. All Rights Reserved\n\n\"\"\"Small query library.\"\"\"\n\nimport inspect\nimport json\nimport types\nimport warnings\n\nimport numpy as np\n\n\ndef make_selector_fn(selector):\n \"\"\"\n If selector is a function, return selector.\n Otherwise, return a function corresponding to the selector string. Examples\n of valid selector strings and the corresponding functions:\n x lambda obj: obj['x']\n x.y lambda obj: obj['x']['y']\n x,y lambda obj: (obj['x'], obj['y'])\n \"\"\"\n if isinstance(selector, str):\n if \",\" in selector:\n parts = selector.split(\",\")\n part_selectors = [make_selector_fn(part) for part in parts]\n return lambda obj: tuple(sel(obj) for sel in part_selectors)\n elif \".\" in selector:\n parts = selector.split(\".\")\n part_selectors = [make_selector_fn(part) for part in parts]\n\n def f(obj):\n for sel in part_selectors:\n obj = sel(obj)\n return obj\n\n return f\n else:\n key = selector.strip()\n return lambda obj: obj[key]\n elif isinstance(selector, types.FunctionType):\n return selector\n else:\n raise TypeError\n\n\ndef hashable(obj):\n try:\n hash(obj)\n return obj\n except TypeError:\n return json.dumps({\"_\": obj}, sort_keys=True)\n\n\nclass Q(object):\n def __init__(self, list_):\n super(Q, self).__init__()\n self._list = list_\n\n def __len__(self):\n return len(self._list)\n\n def __getitem__(self, key):\n return self._list[key]\n\n def __eq__(self, other):\n if isinstance(other, self.__class__):\n return self._list == other._list\n else:\n return self._list == other\n\n def __str__(self):\n return str(self._list)\n\n def __repr__(self):\n return repr(self._list)\n\n def _append(self, item):\n \"\"\"Unsafe, be careful you know what you're doing.\"\"\"\n self._list.append(item)\n\n def group(self, selector):\n \"\"\"\n Group elements by selector and return a list of (group, group_records)\n tuples.\n \"\"\"\n selector = make_selector_fn(selector)\n groups = {}\n for x in self._list:\n group = selector(x)\n group_key = hashable(group)\n if group_key not in groups:\n groups[group_key] = (group, Q([]))\n groups[group_key][1]._append(x)\n results = [groups[key] for key in sorted(groups.keys())]\n return Q(results)\n\n def group_map(self, selector, fn):\n \"\"\"\n Group elements by selector, apply fn to each group, and return a list\n of the results.\n \"\"\"\n return self.group(selector).map(fn)\n\n def map(self, fn):\n \"\"\"\n map self onto fn. If fn takes multiple args, tuple-unpacking\n is applied.\n \"\"\"\n if len(inspect.signature(fn).parameters) > 1:\n return Q([fn(*x) for x in self._list])\n else:\n return Q([fn(x) for x in self._list])\n\n def select(self, selector):\n selector = make_selector_fn(selector)\n return Q([selector(x) for x in self._list])\n\n def min(self):\n return min(self._list)\n\n def max(self):\n return max(self._list)\n\n def sum(self):\n return sum(self._list)\n\n def len(self):\n return len(self._list)\n\n def mean(self):\n with warnings.catch_warnings():\n warnings.simplefilter(\"ignore\")\n return float(np.mean(self._list))\n\n def std(self):\n with warnings.catch_warnings():\n warnings.simplefilter(\"ignore\")\n return float(np.std(self._list))\n\n def mean_std(self):\n return (self.mean(), self.std())\n\n def argmax(self, selector):\n selector = make_selector_fn(selector)\n return max(self._list, key=selector)\n\n def filter(self, fn):\n return Q([x for x in self._list if fn(x)])\n\n def filter_equals(self, selector, value):\n \"\"\"like [x for x in y if x.selector == value]\"\"\"\n selector = make_selector_fn(selector)\n return self.filter(lambda r: selector(r) == value)\n\n def filter_not_none(self):\n return self.filter(lambda r: r is not None)\n\n def filter_not_nan(self):\n return self.filter(lambda r: not np.isnan(r))\n\n def flatten(self):\n return Q([y for x in self._list for y in x])\n\n def unique(self):\n result = []\n result_set = set()\n for x in self._list:\n hashable_x = hashable(x)\n if hashable_x not in result_set:\n result_set.add(hashable_x)\n result.append(x)\n return Q(result)\n\n def sorted(self, key=None, reverse=False):\n if key is None:\n key = lambda x: x\n\n def key2(x):\n x = key(x)\n if isinstance(x, (np.floating, float)) and np.isnan(x):\n return float(\"-inf\")\n else:\n return x\n\n return Q(sorted(self._list, key=key2, reverse=reverse))\n"
},
{
"path": "domain_generalization/swad/domainbed/lib/swa_utils.py",
"content": "# Burrowed from https://github.com/pytorch/pytorch/blob/master/torch/optim/swa_utils.py\n# modified for the DomainBed.\nimport copy\nimport torch\nfrom torch.nn import Module\nfrom copy import deepcopy\n\n\nclass AveragedModel(Module):\n def __init__(self, model, device=None, avg_fn=None, rm_optimizer=False):\n super(AveragedModel, self).__init__()\n self.start_step = -1\n self.end_step = -1\n if isinstance(model, AveragedModel):\n # prevent nested averagedmodel\n model = model.module\n self.module = deepcopy(model)\n if rm_optimizer:\n for k, v in vars(self.module).items():\n if isinstance(v, torch.optim.Optimizer):\n setattr(self.module, k, None)\n\n if device is not None:\n self.module = self.module.to(device)\n\n self.register_buffer(\"n_averaged\", torch.tensor(0, dtype=torch.long, device=device))\n\n if avg_fn is None:\n def avg_fn(averaged_model_parameter, model_parameter, num_averaged):\n return averaged_model_parameter + (model_parameter - averaged_model_parameter) / (\n num_averaged + 1\n )\n\n self.avg_fn = avg_fn\n\n def forward(self, *args, **kwargs):\n # return self.predict(*args, **kwargs)\n return self.module(*args, **kwargs)\n\n def predict(self, *args, **kwargs):\n return self.module(*args, **kwargs)\n\n @property\n def network(self):\n return self.module.network\n\n def update_parameters(self, model, step=None, start_step=None, end_step=None):\n \"\"\"Update averaged model parameters\n\n Args:\n model: current model to update params\n step: current step. step is saved for log the averaged range\n start_step: set start_step only for first update\n end_step: set end_step\n \"\"\"\n if isinstance(model, AveragedModel):\n model = model.module\n for p_swa, p_model in zip(self.parameters(), model.parameters()):\n device = p_swa.device\n p_model_ = p_model.detach().to(device)\n if self.n_averaged == 0:\n p_swa.detach().copy_(p_model_)\n else:\n p_swa.detach().copy_(\n self.avg_fn(p_swa.detach(), p_model_, self.n_averaged.to(device))\n )\n self.n_averaged += 1\n\n if step is not None:\n if start_step is None:\n start_step = step\n if end_step is None:\n end_step = step\n\n if start_step is not None:\n if self.n_averaged == 1:\n self.start_step = start_step\n\n if end_step is not None:\n self.end_step = end_step\n\n def clone(self):\n clone = copy.deepcopy(self.module)\n clone.optimizer = clone.new_optimizer(clone.network.parameters())\n return clone\n\n\ndef cvt_dbiterator_to_loader(dbiterator, n_iter):\n \"\"\"Convert DB iterator to the loader\"\"\"\n for _ in range(n_iter):\n minibatches = [(x, y) for x, y in next(dbiterator)]\n all_x = torch.cat([x for x, y in minibatches])\n all_y = torch.cat([y for x, y in minibatches])\n\n yield all_x, all_y\n\n\n@torch.no_grad()\ndef update_bn(iterator, model, n_steps, device=\"cuda\"):\n momenta = {}\n for module in model.modules():\n if isinstance(module, torch.nn.modules.batchnorm._BatchNorm):\n module.running_mean = torch.zeros_like(module.running_mean)\n module.running_var = torch.ones_like(module.running_var)\n momenta[module] = module.momentum\n\n if not momenta:\n return\n\n was_training = model.training\n model.train()\n for module in momenta.keys():\n module.momentum = None\n module.num_batches_tracked *= 0\n\n for i in range(n_steps):\n # batches_dictlist: [{env0_data_key: tensor, env0_...}, env1_..., ...]\n batches_dictlist = next(iterator)\n x = torch.cat([dic[\"x\"] for dic in batches_dictlist])\n x = x.to(device)\n\n model(x)\n\n for bn_module in momenta.keys():\n bn_module.momentum = momenta[bn_module]\n model.train(was_training)\n"
},
{
"path": "domain_generalization/swad/domainbed/lib/wide_resnet.py",
"content": "# Copyright (c) Facebook, Inc. and its affiliates. All Rights Reserved\n\n\"\"\"\nFrom https://github.com/meliketoy/wide-resnet.pytorch\n\"\"\"\n\nimport sys\n\nimport numpy as np\nimport torch\nimport torch.nn as nn\nimport torch.nn.functional as F\nimport torch.nn.init as init\nfrom torch.autograd import Variable\n\n\ndef conv3x3(in_planes, out_planes, stride=1):\n return nn.Conv2d(in_planes, out_planes, kernel_size=3, stride=stride, padding=1, bias=True)\n\n\ndef conv_init(m):\n classname = m.__class__.__name__\n if classname.find(\"Conv\") != -1:\n init.xavier_uniform_(m.weight, gain=np.sqrt(2))\n init.constant_(m.bias, 0)\n elif classname.find(\"BatchNorm\") != -1:\n init.constant_(m.weight, 1)\n init.constant_(m.bias, 0)\n\n\nclass wide_basic(nn.Module):\n def __init__(self, in_planes, planes, dropout_rate, stride=1):\n super(wide_basic, self).__init__()\n self.bn1 = nn.BatchNorm2d(in_planes)\n self.conv1 = nn.Conv2d(in_planes, planes, kernel_size=3, padding=1, bias=True)\n self.dropout = nn.Dropout(p=dropout_rate)\n self.bn2 = nn.BatchNorm2d(planes)\n self.conv2 = nn.Conv2d(planes, planes, kernel_size=3, stride=stride, padding=1, bias=True)\n\n self.shortcut = nn.Sequential()\n if stride != 1 or in_planes != planes:\n self.shortcut = nn.Sequential(\n nn.Conv2d(in_planes, planes, kernel_size=1, stride=stride, bias=True),\n )\n\n def forward(self, x):\n out = self.dropout(self.conv1(F.relu(self.bn1(x))))\n out = self.conv2(F.relu(self.bn2(out)))\n out += self.shortcut(x)\n\n return out\n\n\nclass Wide_ResNet(nn.Module):\n \"\"\"Wide Resnet with the softmax layer chopped off\"\"\"\n\n def __init__(self, input_shape, depth, widen_factor, dropout_rate):\n super(Wide_ResNet, self).__init__()\n self.in_planes = 16\n\n assert (depth - 4) % 6 == 0, \"Wide-resnet depth should be 6n+4\"\n n = (depth - 4) / 6\n k = widen_factor\n\n # print('| Wide-Resnet %dx%d' % (depth, k))\n nStages = [16, 16 * k, 32 * k, 64 * k]\n\n self.conv1 = conv3x3(input_shape[0], nStages[0])\n self.layer1 = self._wide_layer(wide_basic, nStages[1], n, dropout_rate, stride=1)\n self.layer2 = self._wide_layer(wide_basic, nStages[2], n, dropout_rate, stride=2)\n self.layer3 = self._wide_layer(wide_basic, nStages[3], n, dropout_rate, stride=2)\n self.bn1 = nn.BatchNorm2d(nStages[3], momentum=0.9)\n\n self.n_outputs = nStages[3]\n\n def _wide_layer(self, block, planes, num_blocks, dropout_rate, stride):\n strides = [stride] + [1] * (int(num_blocks) - 1)\n layers = []\n\n for stride in strides:\n layers.append(block(self.in_planes, planes, dropout_rate, stride))\n self.in_planes = planes\n\n return nn.Sequential(*layers)\n\n def forward(self, x):\n out = self.conv1(x)\n out = self.layer1(out)\n out = self.layer2(out)\n out = self.layer3(out)\n out = F.relu(self.bn1(out))\n out = F.avg_pool2d(out, 8)\n return out[:, :, 0, 0]\n"
},
{
"path": "domain_generalization/swad/domainbed/lib/writers.py",
"content": "class Writer:\n def add_scalars(self, tag_scalar_dic, global_step):\n raise NotImplementedError()\n\n def add_scalars_with_prefix(self, tag_scalar_dic, global_step, prefix):\n tag_scalar_dic = {prefix + k: v for k, v in tag_scalar_dic.items()}\n self.add_scalars(tag_scalar_dic, global_step)\n\n\nclass TBWriter(Writer):\n def __init__(self, dir_path):\n from tensorboardX import SummaryWriter\n\n self.writer = SummaryWriter(dir_path, flush_secs=30)\n\n def add_scalars(self, tag_scalar_dic, global_step):\n for tag, scalar in tag_scalar_dic.items():\n self.writer.add_scalar(tag, scalar, global_step)\n\n\ndef get_writer(dir_path):\n \"\"\"\n Args:\n dir_path: tb dir\n \"\"\"\n writer = TBWriter(dir_path)\n\n return writer\n"
},
{
"path": "domain_generalization/swad/domainbed/lr_scheduler/__init__.py",
"content": "from .lr_scheduler import get_scheduler\n"
},
{
"path": "domain_generalization/swad/domainbed/lr_scheduler/lr_scheduler.py",
"content": "from torch.optim import lr_scheduler\n\n\ndef get_scheduler(name, optimizer, lr, total_steps, final_div_factor=1e4):\n name = name.lower()\n if name == \"onecycle\":\n scheduler = lr_scheduler.OneCycleLR(\n optimizer, lr, total_steps=total_steps, pct_start=0.1, final_div_factor=final_div_factor\n )\n elif name in [\"cos\", \"cosine\"]:\n eta_min = lr / final_div_factor\n scheduler = lr_scheduler.CosineAnnealingLR(optimizer, T_max=total_steps, eta_min=eta_min)\n elif name == \"step\":\n decay_pct = 0.6\n decay_step = int(total_steps * decay_pct)\n scheduler = lr_scheduler.StepLR(optimizer, step_size=decay_step, gamma=0.1)\n elif name in [\"const\", \"swa\"]:\n scheduler = lr_scheduler.LambdaLR(optimizer, lr_lambda=lambda step: 1.0)\n else:\n raise ValueError(name)\n\n return scheduler\n"
},
{
"path": "domain_generalization/swad/domainbed/misc/domain_net_duplicates.txt",
"content": "clipart/animal_migration/clipart_006_000227.jpg\nclipart/baseball_bat/clipart_018_000011.jpg\nclipart/baseball_bat/clipart_018_000021.jpg\nclipart/baseball_bat/clipart_018_000051.jpg\nclipart/baseball_bat/clipart_018_000072.jpg\nclipart/binoculars/clipart_031_000213.jpg\nclipart/bird/clipart_032_000255.jpg\nclipart/book/clipart_036_000137.jpg\nclipart/bowtie/clipart_039_000059.jpg\nclipart/bowtie/clipart_039_000068.jpg\nclipart/bowtie/clipart_039_000091.jpg\nclipart/bowtie/clipart_039_000115.jpg\nclipart/bowtie/clipart_039_000118.jpg\nclipart/bowtie/clipart_039_000121.jpg\nclipart/bowtie/clipart_039_000132.jpg\nclipart/bowtie/clipart_039_000134.jpg\nclipart/bowtie/clipart_039_000136.jpg\nclipart/bowtie/clipart_039_000138.jpg\nclipart/bracelet/clipart_040_000139.jpg\nclipart/bracelet/clipart_040_000248.jpg\nclipart/bread/clipart_042_000006.jpg\nclipart/bread/clipart_042_000081.jpg\nclipart/broom/clipart_045_000026.jpg\nclipart/cake/clipart_052_000004.jpg\nclipart/cake/clipart_052_000005.jpg\nclipart/cake/clipart_052_000007.jpg\nclipart/cake/clipart_052_000012.jpg\nclipart/cake/clipart_052_000013.jpg\nclipart/cake/clipart_052_000014.jpg\nclipart/cake/clipart_052_000015.jpg\nclipart/cake/clipart_052_000017.jpg\nclipart/cake/clipart_052_000029.jpg\nclipart/cake/clipart_052_000042.jpg\nclipart/cake/clipart_052_000043.jpg\nclipart/cake/clipart_052_000046.jpg\nclipart/cake/clipart_052_000048.jpg\nclipart/cake/clipart_052_000051.jpg\nclipart/cake/clipart_052_000055.jpg\nclipart/cake/clipart_052_000058.jpg\nclipart/cake/clipart_052_000061.jpg\nclipart/cake/clipart_052_000069.jpg\nclipart/cake/clipart_052_000070.jpg\nclipart/cake/clipart_052_000073.jpg\nclipart/cake/clipart_052_000074.jpg\nclipart/cake/clipart_052_000075.jpg\nclipart/cake/clipart_052_000078.jpg\nclipart/cake/clipart_052_000087.jpg\nclipart/cake/clipart_052_000088.jpg\nclipart/cake/clipart_052_000098.jpg\nclipart/candle/clipart_059_000004.jpg\nclipart/candle/clipart_059_000017.jpg\nclipart/candle/clipart_059_000043.jpg\nclipart/candle/clipart_059_000046.jpg\nclipart/candle/clipart_059_000083.jpg\nclipart/candle/clipart_059_000091.jpg\nclipart/candle/clipart_059_000098.jpg\nclipart/candle/clipart_059_000104.jpg\nclipart/candle/clipart_059_000105.jpg\nclipart/circle/clipart_072_000176.jpg\nclipart/circle/clipart_072_000180.jpg\nclipart/circle/clipart_072_000199.jpg\nclipart/clock/clipart_074_000056.jpg\nclipart/clock/clipart_074_000059.jpg\nclipart/coffee_cup/clipart_076_000178.jpg\nclipart/compass/clipart_077_000026.jpg\nclipart/compass/clipart_077_000079.jpg\nclipart/couch/clipart_081_000001.jpg\nclipart/couch/clipart_081_000002.jpg\nclipart/couch/clipart_081_000003.jpg\nclipart/couch/clipart_081_000004.jpg\nclipart/couch/clipart_081_000005.jpg\nclipart/couch/clipart_081_000006.jpg\nclipart/couch/clipart_081_000007.jpg\nclipart/couch/clipart_081_000008.jpg\nclipart/couch/clipart_081_000009.jpg\nclipart/couch/clipart_081_000010.jpg\nclipart/couch/clipart_081_000011.jpg\nclipart/couch/clipart_081_000012.jpg\nclipart/couch/clipart_081_000013.jpg\nclipart/couch/clipart_081_000014.jpg\nclipart/couch/clipart_081_000015.jpg\nclipart/couch/clipart_081_000016.jpg\nclipart/couch/clipart_081_000017.jpg\nclipart/couch/clipart_081_000018.jpg\nclipart/couch/clipart_081_000019.jpg\nclipart/couch/clipart_081_000020.jpg\nclipart/couch/clipart_081_000021.jpg\nclipart/couch/clipart_081_000022.jpg\nclipart/couch/clipart_081_000023.jpg\nclipart/couch/clipart_081_000024.jpg\nclipart/couch/clipart_081_000025.jpg\nclipart/couch/clipart_081_000026.jpg\nclipart/couch/clipart_081_000027.jpg\nclipart/couch/clipart_081_000028.jpg\nclipart/couch/clipart_081_000029.jpg\nclipart/couch/clipart_081_000030.jpg\nclipart/couch/clipart_081_000031.jpg\nclipart/couch/clipart_081_000032.jpg\nclipart/couch/clipart_081_000033.jpg\nclipart/couch/clipart_081_000034.jpg\nclipart/couch/clipart_081_000035.jpg\nclipart/couch/clipart_081_000036.jpg\nclipart/couch/clipart_081_000037.jpg\nclipart/couch/clipart_081_000038.jpg\nclipart/couch/clipart_081_000039.jpg\nclipart/couch/clipart_081_000040.jpg\nclipart/couch/clipart_081_000041.jpg\nclipart/couch/clipart_081_000042.jpg\nclipart/couch/clipart_081_000043.jpg\nclipart/couch/clipart_081_000044.jpg\nclipart/couch/clipart_081_000045.jpg\nclipart/couch/clipart_081_000046.jpg\nclipart/couch/clipart_081_000047.jpg\nclipart/couch/clipart_081_000048.jpg\nclipart/couch/clipart_081_000049.jpg\nclipart/couch/clipart_081_000050.jpg\nclipart/couch/clipart_081_000051.jpg\nclipart/couch/clipart_081_000052.jpg\nclipart/couch/clipart_081_000053.jpg\nclipart/couch/clipart_081_000054.jpg\nclipart/couch/clipart_081_000055.jpg\nclipart/couch/clipart_081_000056.jpg\nclipart/couch/clipart_081_000057.jpg\nclipart/couch/clipart_081_000058.jpg\nclipart/couch/clipart_081_000059.jpg\nclipart/couch/clipart_081_000060.jpg\nclipart/couch/clipart_081_000061.jpg\nclipart/couch/clipart_081_000062.jpg\nclipart/couch/clipart_081_000063.jpg\nclipart/couch/clipart_081_000064.jpg\nclipart/couch/clipart_081_000065.jpg\nclipart/couch/clipart_081_000066.jpg\nclipart/couch/clipart_081_000067.jpg\nclipart/couch/clipart_081_000068.jpg\nclipart/couch/clipart_081_000069.jpg\nclipart/couch/clipart_081_000070.jpg\nclipart/couch/clipart_081_000071.jpg\nclipart/couch/clipart_081_000072.jpg\nclipart/couch/clipart_081_000073.jpg\nclipart/couch/clipart_081_000074.jpg\nclipart/couch/clipart_081_000075.jpg\nclipart/couch/clipart_081_000076.jpg\nclipart/couch/clipart_081_000077.jpg\nclipart/couch/clipart_081_000078.jpg\nclipart/couch/clipart_081_000079.jpg\nclipart/couch/clipart_081_000080.jpg\nclipart/couch/clipart_081_000081.jpg\nclipart/couch/clipart_081_000082.jpg\nclipart/couch/clipart_081_000083.jpg\nclipart/couch/clipart_081_000084.jpg\nclipart/couch/clipart_081_000085.jpg\nclipart/couch/clipart_081_000086.jpg\nclipart/couch/clipart_081_000087.jpg\nclipart/couch/clipart_081_000088.jpg\nclipart/couch/clipart_081_000089.jpg\nclipart/couch/clipart_081_000090.jpg\nclipart/couch/clipart_081_000091.jpg\nclipart/couch/clipart_081_000092.jpg\nclipart/couch/clipart_081_000093.jpg\nclipart/couch/clipart_081_000094.jpg\nclipart/couch/clipart_081_000095.jpg\nclipart/couch/clipart_081_000096.jpg\nclipart/couch/clipart_081_000097.jpg\nclipart/couch/clipart_081_000098.jpg\nclipart/couch/clipart_081_000099.jpg\nclipart/couch/clipart_081_000100.jpg\nclipart/couch/clipart_081_000101.jpg\nclipart/couch/clipart_081_000103.jpg\nclipart/couch/clipart_081_000104.jpg\nclipart/couch/clipart_081_000105.jpg\nclipart/couch/clipart_081_000106.jpg\nclipart/couch/clipart_081_000108.jpg\nclipart/couch/clipart_081_000109.jpg\nclipart/couch/clipart_081_000110.jpg\nclipart/couch/clipart_081_000112.jpg\nclipart/couch/clipart_081_000113.jpg\nclipart/couch/clipart_081_000114.jpg\nclipart/couch/clipart_081_000115.jpg\nclipart/couch/clipart_081_000116.jpg\nclipart/couch/clipart_081_000118.jpg\nclipart/couch/clipart_081_000120.jpg\nclipart/couch/clipart_081_000122.jpg\nclipart/couch/clipart_081_000124.jpg\nclipart/couch/clipart_081_000125.jpg\nclipart/couch/clipart_081_000126.jpg\nclipart/couch/clipart_081_000127.jpg\nclipart/couch/clipart_081_000128.jpg\nclipart/couch/clipart_081_000129.jpg\nclipart/couch/clipart_081_000130.jpg\nclipart/couch/clipart_081_000131.jpg\nclipart/couch/clipart_081_000132.jpg\nclipart/couch/clipart_081_000133.jpg\nclipart/couch/clipart_081_000134.jpg\nclipart/couch/clipart_081_000135.jpg\nclipart/couch/clipart_081_000136.jpg\nclipart/couch/clipart_081_000137.jpg\nclipart/couch/clipart_081_000138.jpg\nclipart/couch/clipart_081_000139.jpg\nclipart/couch/clipart_081_000140.jpg\nclipart/couch/clipart_081_000141.jpg\nclipart/couch/clipart_081_000143.jpg\nclipart/couch/clipart_081_000144.jpg\nclipart/couch/clipart_081_000145.jpg\nclipart/couch/clipart_081_000146.jpg\nclipart/couch/clipart_081_000147.jpg\nclipart/couch/clipart_081_000148.jpg\nclipart/couch/clipart_081_000149.jpg\nclipart/couch/clipart_081_000150.jpg\nclipart/couch/clipart_081_000151.jpg\nclipart/couch/clipart_081_000152.jpg\nclipart/couch/clipart_081_000153.jpg\nclipart/couch/clipart_081_000154.jpg\nclipart/couch/clipart_081_000155.jpg\nclipart/couch/clipart_081_000156.jpg\nclipart/couch/clipart_081_000157.jpg\nclipart/couch/clipart_081_000158.jpg\nclipart/couch/clipart_081_000159.jpg\nclipart/couch/clipart_081_000160.jpg\nclipart/couch/clipart_081_000161.jpg\nclipart/couch/clipart_081_000162.jpg\nclipart/couch/clipart_081_000163.jpg\nclipart/couch/clipart_081_000164.jpg\nclipart/couch/clipart_081_000165.jpg\nclipart/couch/clipart_081_000166.jpg\nclipart/couch/clipart_081_000167.jpg\nclipart/couch/clipart_081_000168.jpg\nclipart/couch/clipart_081_000169.jpg\nclipart/couch/clipart_081_000171.jpg\nclipart/couch/clipart_081_000173.jpg\nclipart/couch/clipart_081_000174.jpg\nclipart/couch/clipart_081_000175.jpg\nclipart/couch/clipart_081_000176.jpg\nclipart/couch/clipart_081_000178.jpg\nclipart/couch/clipart_081_000180.jpg\nclipart/couch/clipart_081_000181.jpg\nclipart/couch/clipart_081_000182.jpg\nclipart/couch/clipart_081_000183.jpg\nclipart/couch/clipart_081_000184.jpg\nclipart/couch/clipart_081_000185.jpg\nclipart/couch/clipart_081_000186.jpg\nclipart/couch/clipart_081_000187.jpg\nclipart/couch/clipart_081_000189.jpg\nclipart/couch/clipart_081_000190.jpg\nclipart/couch/clipart_081_000192.jpg\nclipart/couch/clipart_081_000193.jpg\nclipart/couch/clipart_081_000194.jpg\nclipart/couch/clipart_081_000195.jpg\nclipart/couch/clipart_081_000196.jpg\nclipart/couch/clipart_081_000197.jpg\nclipart/couch/clipart_081_000198.jpg\nclipart/couch/clipart_081_000199.jpg\nclipart/couch/clipart_081_000200.jpg\nclipart/couch/clipart_081_000201.jpg\nclipart/couch/clipart_081_000202.jpg\nclipart/couch/clipart_081_000203.jpg\nclipart/couch/clipart_081_000204.jpg\nclipart/couch/clipart_081_000205.jpg\nclipart/couch/clipart_081_000206.jpg\nclipart/couch/clipart_081_000207.jpg\nclipart/couch/clipart_081_000208.jpg\nclipart/couch/clipart_081_000209.jpg\nclipart/couch/clipart_081_000210.jpg\nclipart/couch/clipart_081_000212.jpg\nclipart/couch/clipart_081_000213.jpg\nclipart/couch/clipart_081_000214.jpg\nclipart/couch/clipart_081_000215.jpg\nclipart/couch/clipart_081_000216.jpg\nclipart/couch/clipart_081_000217.jpg\nclipart/couch/clipart_081_000218.jpg\nclipart/couch/clipart_081_000219.jpg\nclipart/couch/clipart_081_000220.jpg\nclipart/couch/clipart_081_000221.jpg\nclipart/couch/clipart_081_000222.jpg\nclipart/couch/clipart_081_000224.jpg\nclipart/couch/clipart_081_000225.jpg\nclipart/couch/clipart_081_000228.jpg\nclipart/couch/clipart_081_000229.jpg\nclipart/couch/clipart_081_000230.jpg\nclipart/couch/clipart_081_000231.jpg\nclipart/couch/clipart_081_000232.jpg\nclipart/crown/clipart_086_000085.jpg\nclipart/cruise_ship/clipart_087_000086.jpg\nclipart/cruise_ship/clipart_087_000174.jpg\nclipart/cup/clipart_088_000001.jpg\nclipart/cup/clipart_088_000004.jpg\nclipart/cup/clipart_088_000008.jpg\nclipart/cup/clipart_088_000009.jpg\nclipart/cup/clipart_088_000011.jpg\nclipart/cup/clipart_088_000012.jpg\nclipart/cup/clipart_088_000030.jpg\nclipart/cup/clipart_088_000032.jpg\nclipart/cup/clipart_088_000039.jpg\nclipart/cup/clipart_088_000042.jpg\nclipart/cup/clipart_088_000069.jpg\nclipart/cup/clipart_088_000072.jpg\nclipart/cup/clipart_088_000077.jpg\nclipart/cup/clipart_088_000079.jpg\nclipart/cup/clipart_088_000081.jpg\nclipart/cup/clipart_088_000082.jpg\nclipart/cup/clipart_088_000087.jpg\nclipart/cup/clipart_088_000091.jpg\nclipart/cup/clipart_088_000093.jpg\nclipart/cup/clipart_088_000094.jpg\nclipart/cup/clipart_088_000096.jpg\nclipart/cup/clipart_088_000108.jpg\nclipart/cup/clipart_088_000111.jpg\nclipart/cup/clipart_088_000113.jpg\nclipart/cup/clipart_088_000117.jpg\nclipart/drill/clipart_098_000095.jpg\nclipart/dumbbell/clipart_101_000079.jpg\nclipart/eraser/clipart_106_000069.jpg\nclipart/eyeglasses/clipart_108_000102.jpg\nclipart/eyeglasses/clipart_108_000127.jpg\nclipart/face/clipart_109_000035.jpg\nclipart/fan/clipart_110_000055.jpg\nclipart/fan/clipart_110_000057.jpg\nclipart/fan/clipart_110_000069.jpg\nclipart/fan/clipart_110_000076.jpg\nclipart/fan/clipart_110_000085.jpg\nclipart/fan/clipart_110_000115.jpg\nclipart/fan/clipart_110_000120.jpg\nclipart/fence/clipart_112_000163.jpg\nclipart/firetruck/clipart_116_000028.jpg\nclipart/flashlight/clipart_119_000152.jpg\nclipart/flower/clipart_122_000069.jpg\nclipart/foot/clipart_124_000023.jpg\nclipart/fork/clipart_125_000106.jpg\nclipart/fork/clipart_125_000184.jpg\nclipart/garden/clipart_128_000049.jpg\nclipart/garden/clipart_128_000056.jpg\nclipart/garden/clipart_128_000058.jpg\nclipart/garden/clipart_128_000059.jpg\nclipart/garden/clipart_128_000061.jpg\nclipart/garden/clipart_128_000063.jpg\nclipart/giraffe/clipart_130_000004.jpg\nclipart/goatee/clipart_131_000002.jpg\nclipart/goatee/clipart_131_000005.jpg\nclipart/goatee/clipart_131_000015.jpg\nclipart/goatee/clipart_131_000017.jpg\nclipart/goatee/clipart_131_000037.jpg\nclipart/goatee/clipart_131_000043.jpg\nclipart/goatee/clipart_131_000057.jpg\nclipart/goatee/clipart_131_000067.jpg\nclipart/goatee/clipart_131_000068.jpg\nclipart/goatee/clipart_131_000092.jpg\nclipart/goatee/clipart_131_000101.jpg\nclipart/goatee/clipart_131_000104.jpg\nclipart/goatee/clipart_131_000118.jpg\nclipart/goatee/clipart_131_000123.jpg\nclipart/goatee/clipart_131_000124.jpg\nclipart/goatee/clipart_131_000130.jpg\nclipart/goatee/clipart_131_000132.jpg\nclipart/goatee/clipart_131_000152.jpg\nclipart/goatee/clipart_131_000155.jpg\nclipart/goatee/clipart_131_000166.jpg\nclipart/goatee/clipart_131_000179.jpg\nclipart/goatee/clipart_131_000182.jpg\nclipart/goatee/clipart_131_000184.jpg\nclipart/goatee/clipart_131_000195.jpg\nclipart/goatee/clipart_131_000196.jpg\nclipart/goatee/clipart_131_000202.jpg\nclipart/goatee/clipart_131_000203.jpg\nclipart/goatee/clipart_131_000214.jpg\nclipart/goatee/clipart_131_000227.jpg\nclipart/goatee/clipart_131_000237.jpg\nclipart/goatee/clipart_131_000244.jpg\nclipart/goatee/clipart_131_000254.jpg\nclipart/golf_club/clipart_132_000088.jpg\nclipart/golf_club/clipart_132_000147.jpg\nclipart/grass/clipart_134_000051.jpg\nclipart/grass/clipart_134_000110.jpg\nclipart/grass/clipart_134_000123.jpg\nclipart/guitar/clipart_135_000023.jpg\nclipart/guitar/clipart_135_000044.jpg\nclipart/guitar/clipart_135_000060.jpg\nclipart/hamburger/clipart_136_000079.jpg\nclipart/hamburger/clipart_136_000080.jpg\nclipart/hamburger/clipart_136_000085.jpg\nclipart/hamburger/clipart_136_000154.jpg\nclipart/hamburger/clipart_136_000159.jpg\nclipart/hamburger/clipart_136_000160.jpg\nclipart/hamburger/clipart_136_000162.jpg\nclipart/hamburger/clipart_136_000167.jpg\nclipart/hamburger/clipart_136_000169.jpg\nclipart/hamburger/clipart_136_000170.jpg\nclipart/hamburger/clipart_136_000171.jpg\nclipart/hamburger/clipart_136_000173.jpg\nclipart/hamburger/clipart_136_000175.jpg\nclipart/hamburger/clipart_136_000177.jpg\nclipart/hamburger/clipart_136_000179.jpg\nclipart/hamburger/clipart_136_000181.jpg\nclipart/hamburger/clipart_136_000187.jpg\nclipart/hand/clipart_138_000038.jpg\nclipart/hand/clipart_138_000044.jpg\nclipart/hand/clipart_138_000060.jpg\nclipart/hand/clipart_138_000087.jpg\nclipart/hand/clipart_138_000095.jpg\nclipart/harp/clipart_139_000056.jpg\nclipart/harp/clipart_139_000200.jpg\nclipart/headphones/clipart_141_000001.jpg\nclipart/headphones/clipart_141_000282.jpg\nclipart/helicopter/clipart_143_000115.jpg\nclipart/helicopter/clipart_143_000127.jpg\nclipart/hexagon/clipart_145_000112.jpg\nclipart/hexagon/clipart_145_000192.jpg\nclipart/hockey_stick/clipart_147_000036.jpg\nclipart/hockey_stick/clipart_147_000135.jpg\nclipart/horse/clipart_148_000062.jpg\nclipart/hot_air_balloon/clipart_150_000181.jpg\nclipart/hot_dog/clipart_151_000009.jpg\nclipart/hot_tub/clipart_152_000106.jpg\nclipart/house/clipart_154_000019.jpg\nclipart/hurricane/clipart_156_000004.jpg\nclipart/jail/clipart_159_000014.jpg\nclipart/kangaroo/clipart_160_000033.jpg\nclipart/key/clipart_161_000036.jpg\nclipart/keyboard/clipart_162_000010.jpg\nclipart/laptop/clipart_167_000013.jpg\nclipart/laptop/clipart_167_000014.jpg\nclipart/leg/clipart_169_000052.jpg\nclipart/lightning/clipart_173_000055.jpg\nclipart/lightning/clipart_173_000070.jpg\nclipart/lightning/clipart_173_000121.jpg\nclipart/lightning/clipart_173_000137.jpg\nclipart/matches/clipart_182_000024.jpg\nclipart/microphone/clipart_185_000070.jpg\nclipart/microphone/clipart_185_000110.jpg\nclipart/microphone/clipart_185_000132.jpg\nclipart/monkey/clipart_187_000025.jpg\nclipart/monkey/clipart_187_000076.jpg\nclipart/monkey/clipart_187_000098.jpg\nclipart/monkey/clipart_187_000120.jpg\nclipart/moustache/clipart_193_000021.jpg\nclipart/moustache/clipart_193_000040.jpg\nclipart/moustache/clipart_193_000064.jpg\nclipart/moustache/clipart_193_000081.jpg\nclipart/moustache/clipart_193_000102.jpg\nclipart/moustache/clipart_193_000122.jpg\nclipart/moustache/clipart_193_000140.jpg\nclipart/mug/clipart_195_000010.jpg\nclipart/mug/clipart_195_000028.jpg\nclipart/mug/clipart_195_000061.jpg\nclipart/mug/clipart_195_000076.jpg\nclipart/mug/clipart_195_000083.jpg\nclipart/mug/clipart_195_000094.jpg\nclipart/mug/clipart_195_000100.jpg\nclipart/mug/clipart_195_000103.jpg\nclipart/mug/clipart_195_000123.jpg\nclipart/mug/clipart_195_000126.jpg\nclipart/mug/clipart_195_000128.jpg\nclipart/mug/clipart_195_000131.jpg\nclipart/mug/clipart_195_000134.jpg\nclipart/mug/clipart_195_000141.jpg\nclipart/mug/clipart_195_000153.jpg\nclipart/mug/clipart_195_000161.jpg\nclipart/mushroom/clipart_196_000132.jpg\nclipart/nail/clipart_197_000034.jpg\nclipart/necklace/clipart_198_000008.jpg\nclipart/necklace/clipart_198_000038.jpg\nclipart/necklace/clipart_198_000043.jpg\nclipart/necklace/clipart_198_000057.jpg\nclipart/ocean/clipart_200_000052.jpg\nclipart/octagon/clipart_201_000016.jpg\nclipart/octagon/clipart_201_000024.jpg\nclipart/octagon/clipart_201_000026.jpg\nclipart/owl/clipart_205_000089.jpg\nclipart/owl/clipart_205_000121.jpg\nclipart/paint_can/clipart_207_000006.jpg\nclipart/paper_clip/clipart_211_000015.jpg\nclipart/paper_clip/clipart_211_000045.jpg\nclipart/paper_clip/clipart_211_000051.jpg\nclipart/paper_clip/clipart_211_000058.jpg\nclipart/paper_clip/clipart_211_000092.jpg\nclipart/paper_clip/clipart_211_000097.jpg\nclipart/paper_clip/clipart_211_000112.jpg\nclipart/parrot/clipart_213_000016.jpg\nclipart/parrot/clipart_213_000027.jpg\nclipart/parrot/clipart_213_000068.jpg\nclipart/peanut/clipart_215_000035.jpg\nclipart/picture_frame/clipart_222_000074.jpg\nclipart/picture_frame/clipart_222_000087.jpg\nclipart/pig/clipart_223_000029.jpg\nclipart/pillow/clipart_224_000026.jpg\nclipart/police_car/clipart_228_000028.jpg\nclipart/pond/clipart_229_000051.jpg\nclipart/pond/clipart_229_000092.jpg\nclipart/pool/clipart_230_000001.jpg\nclipart/popsicle/clipart_231_000005.jpg\nclipart/popsicle/clipart_231_000072.jpg\nclipart/popsicle/clipart_231_000093.jpg\nclipart/popsicle/clipart_231_000179.jpg\nclipart/purse/clipart_235_000010.jpg\nclipart/purse/clipart_235_000041.jpg\nclipart/rabbit/clipart_236_000057.jpg\nclipart/rabbit/clipart_236_000063.jpg\nclipart/raccoon/clipart_237_000046.jpg\nclipart/rain/clipart_239_000009.jpg\nclipart/rain/clipart_239_000013.jpg\nclipart/rain/clipart_239_000018.jpg\nclipart/rain/clipart_239_000022.jpg\nclipart/rain/clipart_239_000033.jpg\nclipart/rain/clipart_239_000035.jpg\nclipart/rain/clipart_239_000042.jpg\nclipart/rain/clipart_239_000047.jpg\nclipart/rainbow/clipart_240_000034.jpg\nclipart/remote_control/clipart_242_000013.jpg\nclipart/river/clipart_245_000076.jpg\nclipart/river/clipart_245_000082.jpg\nclipart/rollerskates/clipart_247_000022.jpg\nclipart/rollerskates/clipart_247_000068.jpg\nclipart/rollerskates/clipart_247_000069.jpg\nclipart/sandwich/clipart_249_000045.jpg\nclipart/sandwich/clipart_249_000064.jpg\nclipart/sandwich/clipart_249_000088.jpg\nclipart/sandwich/clipart_249_000109.jpg\nclipart/saxophone/clipart_251_000091.jpg\nclipart/school_bus/clipart_252_000001.jpg\nclipart/school_bus/clipart_252_000023.jpg\nclipart/school_bus/clipart_252_000026.jpg\nclipart/school_bus/clipart_252_000029.jpg\nclipart/school_bus/clipart_252_000043.jpg\nclipart/school_bus/clipart_252_000044.jpg\nclipart/school_bus/clipart_252_000048.jpg\nclipart/school_bus/clipart_252_000063.jpg\nclipart/school_bus/clipart_252_000069.jpg\nclipart/school_bus/clipart_252_000084.jpg\nclipart/school_bus/clipart_252_000088.jpg\nclipart/school_bus/clipart_252_000089.jpg\nclipart/school_bus/clipart_252_000094.jpg\nclipart/school_bus/clipart_252_000096.jpg\nclipart/school_bus/clipart_252_000135.jpg\nclipart/school_bus/clipart_252_000142.jpg\nclipart/school_bus/clipart_252_000147.jpg\nclipart/school_bus/clipart_252_000159.jpg\nclipart/school_bus/clipart_252_000167.jpg\nclipart/school_bus/clipart_252_000169.jpg\nclipart/school_bus/clipart_252_000177.jpg\nclipart/school_bus/clipart_252_000178.jpg\nclipart/school_bus/clipart_252_000183.jpg\nclipart/school_bus/clipart_252_000186.jpg\nclipart/school_bus/clipart_252_000188.jpg\nclipart/school_bus/clipart_252_000189.jpg\nclipart/school_bus/clipart_252_000198.jpg\nclipart/school_bus/clipart_252_000221.jpg\nclipart/school_bus/clipart_252_000229.jpg\nclipart/scissors/clipart_253_000088.jpg\nclipart/sea_turtle/clipart_256_000177.jpg\nclipart/sheep/clipart_259_000030.jpg\nclipart/shoe/clipart_260_000020.jpg\nclipart/shorts/clipart_261_000097.jpg\nclipart/shorts/clipart_261_000108.jpg\nclipart/shorts/clipart_261_000113.jpg\nclipart/shovel/clipart_262_000153.jpg\nclipart/shovel/clipart_262_000188.jpg\nclipart/skateboard/clipart_264_000191.jpg\nclipart/skateboard/clipart_264_000220.jpg\nclipart/smiley_face/clipart_268_000006.jpg\nclipart/snail/clipart_269_000070.jpg\nclipart/soccer_ball/clipart_274_000063.jpg\nclipart/sock/clipart_275_000034.jpg\nclipart/speedboat/clipart_276_000011.jpg\nclipart/speedboat/clipart_276_000109.jpg\nclipart/speedboat/clipart_276_000111.jpg\nclipart/spoon/clipart_278_000162.jpg\nclipart/spreadsheet/clipart_279_000150.jpg\nclipart/square/clipart_280_000034.jpg\nclipart/square/clipart_280_000066.jpg\nclipart/square/clipart_280_000123.jpg\nclipart/square/clipart_280_000153.jpg\nclipart/squiggle/clipart_281_000021.jpg\nclipart/star/clipart_284_000105.jpg\nclipart/star/clipart_284_000108.jpg\nclipart/stereo/clipart_286_000168.jpg\nclipart/stitches/clipart_288_000163.jpg\nclipart/stop_sign/clipart_289_000125.jpg\nclipart/stove/clipart_290_000161.jpg\nclipart/stove/clipart_290_000196.jpg\nclipart/strawberry/clipart_291_000003.jpg\nclipart/suitcase/clipart_295_000052.jpg\nclipart/suitcase/clipart_295_000250.jpg\nclipart/sun/clipart_296_000112.jpg\nclipart/sun/clipart_296_000139.jpg\nclipart/sun/clipart_296_000178.jpg\nclipart/sun/clipart_296_000210.jpg\nclipart/swan/clipart_297_000104.jpg\nclipart/swan/clipart_297_000313.jpg\nclipart/swing_set/clipart_299_000105.jpg\nclipart/swing_set/clipart_299_000123.jpg\nclipart/swing_set/clipart_299_000128.jpg\nclipart/sword/clipart_300_000002.jpg\nclipart/sword/clipart_300_000044.jpg\nclipart/sword/clipart_300_000131.jpg\nclipart/syringe/clipart_301_000010.jpg\nclipart/syringe/clipart_301_000024.jpg\nclipart/syringe/clipart_301_000084.jpg\nclipart/table/clipart_302_000115.jpg\nclipart/teapot/clipart_303_000181.jpg\nclipart/teddy-bear/clipart_304_000003.jpg\nclipart/teddy-bear/clipart_304_000004.jpg\nclipart/teddy-bear/clipart_304_000013.jpg\nclipart/teddy-bear/clipart_304_000014.jpg\nclipart/teddy-bear/clipart_304_000022.jpg\nclipart/teddy-bear/clipart_304_000024.jpg\nclipart/teddy-bear/clipart_304_000073.jpg\nclipart/teddy-bear/clipart_304_000077.jpg\nclipart/teddy-bear/clipart_304_000078.jpg\nclipart/teddy-bear/clipart_304_000087.jpg\nclipart/teddy-bear/clipart_304_000089.jpg\nclipart/teddy-bear/clipart_304_000092.jpg\nclipart/teddy-bear/clipart_304_000098.jpg\nclipart/teddy-bear/clipart_304_000102.jpg\nclipart/telephone/clipart_305_000130.jpg\nclipart/television/clipart_306_000079.jpg\nclipart/tennis_racquet/clipart_307_000052.jpg\nclipart/The_Eiffel_Tower/clipart_309_000077.jpg\nclipart/The_Great_Wall_of_China/clipart_310_000014.jpg\nclipart/tiger/clipart_312_000191.jpg\nclipart/toe/clipart_314_000029.jpg\nclipart/toe/clipart_314_000050.jpg\nclipart/toe/clipart_314_000072.jpg\nclipart/toe/clipart_314_000083.jpg\nclipart/toilet/clipart_315_000040.jpg\nclipart/toilet/clipart_315_000076.jpg\nclipart/toilet/clipart_315_000101.jpg\nclipart/tooth/clipart_316_000052.jpg\nclipart/tooth/clipart_316_000081.jpg\nclipart/toothbrush/clipart_317_000144.jpg\nclipart/toothpaste/clipart_318_000004.jpg\nclipart/toothpaste/clipart_318_000011.jpg\nclipart/toothpaste/clipart_318_000014.jpg\nclipart/toothpaste/clipart_318_000031.jpg\nclipart/toothpaste/clipart_318_000051.jpg\nclipart/toothpaste/clipart_318_000053.jpg\nclipart/toothpaste/clipart_318_000089.jpg\nclipart/toothpaste/clipart_318_000090.jpg\nclipart/tornado/clipart_319_000004.jpg\nclipart/tornado/clipart_319_000041.jpg\nclipart/tornado/clipart_319_000111.jpg\nclipart/tornado/clipart_319_000129.jpg\nclipart/tornado/clipart_319_000132.jpg\nclipart/tractor/clipart_320_000016.jpg\nclipart/traffic_light/clipart_321_000038.jpg\nclipart/traffic_light/clipart_321_000065.jpg\nclipart/traffic_light/clipart_321_000198.jpg\nclipart/train/clipart_322_000090.jpg\nclipart/tree/clipart_323_000023.jpg\nclipart/tree/clipart_323_000112.jpg\nclipart/triangle/clipart_324_000013.jpg\nclipart/triangle/clipart_324_000040.jpg\nclipart/triangle/clipart_324_000041.jpg\nclipart/triangle/clipart_324_000066.jpg\nclipart/triangle/clipart_324_000075.jpg\nclipart/triangle/clipart_324_000138.jpg\nclipart/triangle/clipart_324_000166.jpg\nclipart/trombone/clipart_325_000038.jpg\nclipart/trombone/clipart_325_000178.jpg\nclipart/truck/clipart_326_000005.jpg\nclipart/truck/clipart_326_000009.jpg\nclipart/truck/clipart_326_000019.jpg\nclipart/truck/clipart_326_000028.jpg\nclipart/truck/clipart_326_000032.jpg\nclipart/truck/clipart_326_000041.jpg\nclipart/truck/clipart_326_000042.jpg\nclipart/truck/clipart_326_000079.jpg\nclipart/truck/clipart_326_000099.jpg\nclipart/truck/clipart_326_000110.jpg\nclipart/trumpet/clipart_327_000082.jpg\nclipart/trumpet/clipart_327_000107.jpg\nclipart/t-shirt/clipart_328_000019.jpg\nclipart/umbrella/clipart_329_000064.jpg\nclipart/umbrella/clipart_329_000072.jpg\nclipart/umbrella/clipart_329_000090.jpg\nclipart/umbrella/clipart_329_000091.jpg\nclipart/umbrella/clipart_329_000131.jpg\nclipart/underwear/clipart_330_000081.jpg\nclipart/underwear/clipart_330_000084.jpg\nclipart/van/clipart_331_000048.jpg\nclipart/van/clipart_331_000049.jpg\nclipart/van/clipart_331_000120.jpg\nclipart/van/clipart_331_000152.jpg\nclipart/van/clipart_331_000172.jpg\nclipart/vase/clipart_332_000018.jpg\nclipart/violin/clipart_333_000028.jpg\nclipart/violin/clipart_333_000048.jpg\nclipart/violin/clipart_333_000095.jpg\nclipart/washing_machine/clipart_334_000151.jpg\nclipart/watermelon/clipart_335_000011.jpg\nclipart/watermelon/clipart_335_000134.jpg\nclipart/whale/clipart_337_000282.jpg\nclipart/wheel/clipart_338_000042.jpg\nclipart/wine_bottle/clipart_340_000182.jpg\nclipart/wine_bottle/clipart_340_000212.jpg\nclipart/wine_glass/clipart_341_000087.jpg\nclipart/wine_glass/clipart_341_000112.jpg\nclipart/wine_glass/clipart_341_000172.jpg\nclipart/wine_glass/clipart_341_000206.jpg\nclipart/yoga/clipart_343_000049.jpg\nclipart/zebra/clipart_344_000105.jpg\nclipart/zigzag/clipart_345_000011.jpg\nclipart/zigzag/clipart_345_000071.jpg\nclipart/zigzag/clipart_345_000097.jpg\nclipart/zigzag/clipart_345_000137.jpg\nclipart/zigzag/clipart_345_000188.jpg\nclipart/zigzag/clipart_345_000271.jpg\ninfograph/axe/infograph_012_000052.jpg\ninfograph/axe/infograph_012_000070.jpg\ninfograph/axe/infograph_012_000071.jpg\ninfograph/axe/infograph_012_000078.jpg\ninfograph/backpack/infograph_013_000100.jpg\ninfograph/banana/infograph_014_000063.jpg\ninfograph/banana/infograph_014_000272.jpg\ninfograph/bandage/infograph_015_000116.jpg\ninfograph/bandage/infograph_015_000182.jpg\ninfograph/barn/infograph_016_000105.jpg\ninfograph/barn/infograph_016_000110.jpg\ninfograph/barn/infograph_016_000129.jpg\ninfograph/baseball/infograph_017_000102.jpg\ninfograph/baseball_bat/infograph_018_000028.jpg\ninfograph/baseball_bat/infograph_018_000029.jpg\ninfograph/baseball_bat/infograph_018_000058.jpg\ninfograph/baseball_bat/infograph_018_000061.jpg\ninfograph/baseball_bat/infograph_018_000075.jpg\ninfograph/baseball_bat/infograph_018_000092.jpg\ninfograph/baseball_bat/infograph_018_000113.jpg\ninfograph/baseball_bat/infograph_018_000126.jpg\ninfograph/baseball_bat/infograph_018_000130.jpg\ninfograph/baseball_bat/infograph_018_000149.jpg\ninfograph/baseball_bat/infograph_018_000159.jpg\ninfograph/baseball_bat/infograph_018_000177.jpg\ninfograph/baseball_bat/infograph_018_000181.jpg\ninfograph/baseball_bat/infograph_018_000233.jpg\ninfograph/baseball_bat/infograph_018_000236.jpg\ninfograph/baseball_bat/infograph_018_000237.jpg\ninfograph/baseball_bat/infograph_018_000266.jpg\ninfograph/baseball_bat/infograph_018_000347.jpg\ninfograph/basketball/infograph_020_000014.jpg\ninfograph/basketball/infograph_020_000064.jpg\ninfograph/basketball/infograph_020_000110.jpg\ninfograph/basketball/infograph_020_000140.jpg\ninfograph/basketball/infograph_020_000143.jpg\ninfograph/basketball/infograph_020_000157.jpg\ninfograph/bat/infograph_021_000057.jpg\ninfograph/bat/infograph_021_000083.jpg\ninfograph/bear/infograph_024_000080.jpg\ninfograph/bee/infograph_027_000016.jpg\ninfograph/bench/infograph_029_000037.jpg\ninfograph/bird/infograph_032_000001.jpg\ninfograph/bird/infograph_032_000068.jpg\ninfograph/bird/infograph_032_000074.jpg\ninfograph/bird/infograph_032_000127.jpg\ninfograph/bird/infograph_032_000158.jpg\ninfograph/bird/infograph_032_000199.jpg\ninfograph/bread/infograph_042_000060.jpg\ninfograph/bread/infograph_042_000087.jpg\ninfograph/bread/infograph_042_000214.jpg\ninfograph/broccoli/infograph_044_000033.jpg\ninfograph/broccoli/infograph_044_000085.jpg\ninfograph/broccoli/infograph_044_000129.jpg\ninfograph/broccoli/infograph_044_000159.jpg\ninfograph/broccoli/infograph_044_000194.jpg\ninfograph/broccoli/infograph_044_000228.jpg\ninfograph/broccoli/infograph_044_000229.jpg\ninfograph/butterfly/infograph_050_000071.jpg\ninfograph/butterfly/infograph_050_000135.jpg\ninfograph/cake/infograph_052_000003.jpg\ninfograph/cake/infograph_052_000005.jpg\ninfograph/cake/infograph_052_000013.jpg\ninfograph/cake/infograph_052_000034.jpg\ninfograph/cake/infograph_052_000061.jpg\ninfograph/cake/infograph_052_000067.jpg\ninfograph/cake/infograph_052_000080.jpg\ninfograph/cake/infograph_052_000087.jpg\ninfograph/cake/infograph_052_000098.jpg\ninfograph/cake/infograph_052_000116.jpg\ninfograph/cake/infograph_052_000118.jpg\ninfograph/cake/infograph_052_000134.jpg\ninfograph/cake/infograph_052_000140.jpg\ninfograph/camera/infograph_056_000066.jpg\ninfograph/candle/infograph_059_000018.jpg\ninfograph/candle/infograph_059_000021.jpg\ninfograph/candle/infograph_059_000033.jpg\ninfograph/candle/infograph_059_000048.jpg\ninfograph/canoe/infograph_061_000010.jpg\ninfograph/carrot/infograph_063_000036.jpg\ninfograph/carrot/infograph_063_000038.jpg\ninfograph/carrot/infograph_063_000048.jpg\ninfograph/carrot/infograph_063_000056.jpg\ninfograph/carrot/infograph_063_000091.jpg\ninfograph/carrot/infograph_063_000105.jpg\ninfograph/carrot/infograph_063_000108.jpg\ninfograph/carrot/infograph_063_000173.jpg\ninfograph/carrot/infograph_063_000196.jpg\ninfograph/carrot/infograph_063_000197.jpg\ninfograph/carrot/infograph_063_000220.jpg\ninfograph/carrot/infograph_063_000229.jpg\ninfograph/carrot/infograph_063_000238.jpg\ninfograph/cell_phone/infograph_068_000009.jpg\ninfograph/cell_phone/infograph_068_000015.jpg\ninfograph/cell_phone/infograph_068_000139.jpg\ninfograph/chair/infograph_069_000017.jpg\ninfograph/chair/infograph_069_000092.jpg\ninfograph/clock/infograph_074_000017.jpg\ninfograph/computer/infograph_078_000044.jpg\ninfograph/cooler/infograph_080_000004.jpg\ninfograph/couch/infograph_081_000004.jpg\ninfograph/couch/infograph_081_000033.jpg\ninfograph/couch/infograph_081_000055.jpg\ninfograph/cow/infograph_082_000009.jpg\ninfograph/crocodile/infograph_085_000049.jpg\ninfograph/cup/infograph_088_000002.jpg\ninfograph/cup/infograph_088_000004.jpg\ninfograph/cup/infograph_088_000005.jpg\ninfograph/cup/infograph_088_000007.jpg\ninfograph/cup/infograph_088_000021.jpg\ninfograph/cup/infograph_088_000029.jpg\ninfograph/cup/infograph_088_000033.jpg\ninfograph/cup/infograph_088_000036.jpg\ninfograph/cup/infograph_088_000041.jpg\ninfograph/cup/infograph_088_000051.jpg\ninfograph/cup/infograph_088_000052.jpg\ninfograph/diving_board/infograph_091_000011.jpg\ninfograph/dog/infograph_092_000005.jpg\ninfograph/dog/infograph_092_000007.jpg\ninfograph/dog/infograph_092_000090.jpg\ninfograph/dog/infograph_092_000197.jpg\ninfograph/dog/infograph_092_000215.jpg\ninfograph/dolphin/infograph_093_000028.jpg\ninfograph/dolphin/infograph_093_000040.jpg\ninfograph/dolphin/infograph_093_000049.jpg\ninfograph/dolphin/infograph_093_000052.jpg\ninfograph/dolphin/infograph_093_000080.jpg\ninfograph/dolphin/infograph_093_000100.jpg\ninfograph/donut/infograph_094_000020.jpg\ninfograph/drill/infograph_098_000021.jpg\ninfograph/duck/infograph_100_000043.jpg\ninfograph/dumbbell/infograph_101_000021.jpg\ninfograph/dumbbell/infograph_101_000022.jpg\ninfograph/dumbbell/infograph_101_000047.jpg\ninfograph/elbow/infograph_103_000003.jpg\ninfograph/elbow/infograph_103_000010.jpg\ninfograph/elbow/infograph_103_000033.jpg\ninfograph/elbow/infograph_103_000052.jpg\ninfograph/elephant/infograph_104_000101.jpg\ninfograph/eye/infograph_107_000097.jpg\ninfograph/eyeglasses/infograph_108_000097.jpg\ninfograph/eyeglasses/infograph_108_000105.jpg\ninfograph/face/infograph_109_000005.jpg\ninfograph/face/infograph_109_000010.jpg\ninfograph/face/infograph_109_000033.jpg\ninfograph/face/infograph_109_000081.jpg\ninfograph/fan/infograph_110_000003.jpg\ninfograph/fan/infograph_110_000009.jpg\ninfograph/fan/infograph_110_000013.jpg\ninfograph/fan/infograph_110_000014.jpg\ninfograph/fan/infograph_110_000017.jpg\ninfograph/fan/infograph_110_000022.jpg\ninfograph/fan/infograph_110_000028.jpg\ninfograph/fan/infograph_110_000032.jpg\ninfograph/fan/infograph_110_000033.jpg\ninfograph/fan/infograph_110_000035.jpg\ninfograph/fan/infograph_110_000042.jpg\ninfograph/fan/infograph_110_000043.jpg\ninfograph/fan/infograph_110_000046.jpg\ninfograph/feather/infograph_111_000033.jpg\ninfograph/feather/infograph_111_000044.jpg\ninfograph/feather/infograph_111_000063.jpg\ninfograph/feather/infograph_111_000084.jpg\ninfograph/feather/infograph_111_000162.jpg\ninfograph/feather/infograph_111_000184.jpg\ninfograph/feather/infograph_111_000189.jpg\ninfograph/feather/infograph_111_000201.jpg\ninfograph/fireplace/infograph_115_000046.jpg\ninfograph/firetruck/infograph_116_000031.jpg\ninfograph/fish/infograph_117_000135.jpg\ninfograph/fish/infograph_117_000176.jpg\ninfograph/flashlight/infograph_119_000011.jpg\ninfograph/flashlight/infograph_119_000026.jpg\ninfograph/flashlight/infograph_119_000039.jpg\ninfograph/floor_lamp/infograph_121_000086.jpg\ninfograph/foot/infograph_124_000089.jpg\ninfograph/foot/infograph_124_000104.jpg\ninfograph/fork/infograph_125_000007.jpg\ninfograph/frog/infograph_126_000010.jpg\ninfograph/frying_pan/infograph_127_000004.jpg\ninfograph/frying_pan/infograph_127_000055.jpg\ninfograph/garden/infograph_128_000034.jpg\ninfograph/garden/infograph_128_000039.jpg\ninfograph/garden/infograph_128_000060.jpg\ninfograph/garden/infograph_128_000076.jpg\ninfograph/garden/infograph_128_000113.jpg\ninfograph/garden/infograph_128_000118.jpg\ninfograph/garden/infograph_128_000151.jpg\ninfograph/garden/infograph_128_000172.jpg\ninfograph/garden/infograph_128_000186.jpg\ninfograph/garden/infograph_128_000242.jpg\ninfograph/garden/infograph_128_000283.jpg\ninfograph/garden_hose/infograph_129_000002.jpg\ninfograph/garden_hose/infograph_129_000015.jpg\ninfograph/garden_hose/infograph_129_000031.jpg\ninfograph/giraffe/infograph_130_000003.jpg\ninfograph/giraffe/infograph_130_000087.jpg\ninfograph/giraffe/infograph_130_000106.jpg\ninfograph/giraffe/infograph_130_000149.jpg\ninfograph/goatee/infograph_131_000001.jpg\ninfograph/goatee/infograph_131_000005.jpg\ninfograph/goatee/infograph_131_000009.jpg\ninfograph/goatee/infograph_131_000016.jpg\ninfograph/goatee/infograph_131_000024.jpg\ninfograph/goatee/infograph_131_000054.jpg\ninfograph/goatee/infograph_131_000056.jpg\ninfograph/goatee/infograph_131_000059.jpg\ninfograph/goatee/infograph_131_000060.jpg\ninfograph/goatee/infograph_131_000067.jpg\ninfograph/goatee/infograph_131_000068.jpg\ninfograph/goatee/infograph_131_000069.jpg\ninfograph/goatee/infograph_131_000076.jpg\ninfograph/goatee/infograph_131_000082.jpg\ninfograph/goatee/infograph_131_000094.jpg\ninfograph/goatee/infograph_131_000109.jpg\ninfograph/goatee/infograph_131_000113.jpg\ninfograph/goatee/infograph_131_000118.jpg\ninfograph/goatee/infograph_131_000135.jpg\ninfograph/goatee/infograph_131_000147.jpg\ninfograph/goatee/infograph_131_000156.jpg\ninfograph/goatee/infograph_131_000166.jpg\ninfograph/goatee/infograph_131_000170.jpg\ninfograph/goatee/infograph_131_000198.jpg\ninfograph/goatee/infograph_131_000206.jpg\ninfograph/goatee/infograph_131_000208.jpg\ninfograph/goatee/infograph_131_000217.jpg\ninfograph/goatee/infograph_131_000227.jpg\ninfograph/goatee/infograph_131_000232.jpg\ninfograph/goatee/infograph_131_000236.jpg\ninfograph/grapes/infograph_133_000019.jpg\ninfograph/grapes/infograph_133_000067.jpg\ninfograph/grapes/infograph_133_000117.jpg\ninfograph/grass/infograph_134_000036.jpg\ninfograph/grass/infograph_134_000042.jpg\ninfograph/grass/infograph_134_000054.jpg\ninfograph/grass/infograph_134_000065.jpg\ninfograph/grass/infograph_134_000069.jpg\ninfograph/grass/infograph_134_000075.jpg\ninfograph/grass/infograph_134_000128.jpg\ninfograph/grass/infograph_134_000130.jpg\ninfograph/grass/infograph_134_000132.jpg\ninfograph/grass/infograph_134_000148.jpg\ninfograph/grass/infograph_134_000172.jpg\ninfograph/grass/infograph_134_000180.jpg\ninfograph/grass/infograph_134_000218.jpg\ninfograph/grass/infograph_134_000226.jpg\ninfograph/grass/infograph_134_000244.jpg\ninfograph/grass/infograph_134_000247.jpg\ninfograph/grass/infograph_134_000253.jpg\ninfograph/guitar/infograph_135_000089.jpg\ninfograph/guitar/infograph_135_000103.jpg\ninfograph/guitar/infograph_135_000159.jpg\ninfograph/hamburger/infograph_136_000040.jpg\ninfograph/hamburger/infograph_136_000114.jpg\ninfograph/hamburger/infograph_136_000117.jpg\ninfograph/hamburger/infograph_136_000175.jpg\ninfograph/hammer/infograph_137_000009.jpg\ninfograph/hammer/infograph_137_000041.jpg\ninfograph/hammer/infograph_137_000047.jpg\ninfograph/hand/infograph_138_000048.jpg\ninfograph/hand/infograph_138_000049.jpg\ninfograph/hand/infograph_138_000069.jpg\ninfograph/hand/infograph_138_000101.jpg\ninfograph/hand/infograph_138_000122.jpg\ninfograph/hand/infograph_138_000154.jpg\ninfograph/hand/infograph_138_000167.jpg\ninfograph/hand/infograph_138_000172.jpg\ninfograph/hand/infograph_138_000173.jpg\ninfograph/hand/infograph_138_000185.jpg\ninfograph/hand/infograph_138_000187.jpg\ninfograph/hand/infograph_138_000190.jpg\ninfograph/hand/infograph_138_000206.jpg\ninfograph/hand/infograph_138_000252.jpg\ninfograph/harp/infograph_139_000037.jpg\ninfograph/hat/infograph_140_000040.jpg\ninfograph/hat/infograph_140_000083.jpg\ninfograph/hat/infograph_140_000183.jpg\ninfograph/headphones/infograph_141_000158.jpg\ninfograph/helicopter/infograph_143_000015.jpg\ninfograph/helicopter/infograph_143_000208.jpg\ninfograph/helmet/infograph_144_000026.jpg\ninfograph/helmet/infograph_144_000041.jpg\ninfograph/helmet/infograph_144_000060.jpg\ninfograph/helmet/infograph_144_000078.jpg\ninfograph/helmet/infograph_144_000098.jpg\ninfograph/helmet/infograph_144_000099.jpg\ninfograph/helmet/infograph_144_000102.jpg\ninfograph/helmet/infograph_144_000107.jpg\ninfograph/helmet/infograph_144_000137.jpg\ninfograph/helmet/infograph_144_000140.jpg\ninfograph/helmet/infograph_144_000177.jpg\ninfograph/helmet/infograph_144_000180.jpg\ninfograph/helmet/infograph_144_000201.jpg\ninfograph/helmet/infograph_144_000227.jpg\ninfograph/hockey_stick/infograph_147_000009.jpg\ninfograph/hockey_stick/infograph_147_000011.jpg\ninfograph/hockey_stick/infograph_147_000017.jpg\ninfograph/hockey_stick/infograph_147_000018.jpg\ninfograph/hockey_stick/infograph_147_000027.jpg\ninfograph/hockey_stick/infograph_147_000030.jpg\ninfograph/hockey_stick/infograph_147_000052.jpg\ninfograph/hockey_stick/infograph_147_000058.jpg\ninfograph/hockey_stick/infograph_147_000061.jpg\ninfograph/hockey_stick/infograph_147_000064.jpg\ninfograph/hockey_stick/infograph_147_000069.jpg\ninfograph/hockey_stick/infograph_147_000077.jpg\ninfograph/hockey_stick/infograph_147_000100.jpg\ninfograph/hockey_stick/infograph_147_000125.jpg\ninfograph/hockey_stick/infograph_147_000130.jpg\ninfograph/hockey_stick/infograph_147_000178.jpg\ninfograph/hockey_stick/infograph_147_000197.jpg\ninfograph/horse/infograph_148_000002.jpg\ninfograph/horse/infograph_148_000040.jpg\ninfograph/horse/infograph_148_000061.jpg\ninfograph/horse/infograph_148_000081.jpg\ninfograph/horse/infograph_148_000103.jpg\ninfograph/horse/infograph_148_000146.jpg\ninfograph/hot_dog/infograph_151_000028.jpg\ninfograph/hot_dog/infograph_151_000034.jpg\ninfograph/hot_dog/infograph_151_000074.jpg\ninfograph/hot_dog/infograph_151_000075.jpg\ninfograph/hot_dog/infograph_151_000082.jpg\ninfograph/hot_dog/infograph_151_000105.jpg\ninfograph/hot_dog/infograph_151_000112.jpg\ninfograph/hot_dog/infograph_151_000113.jpg\ninfograph/hot_dog/infograph_151_000117.jpg\ninfograph/hot_dog/infograph_151_000133.jpg\ninfograph/hot_tub/infograph_152_000029.jpg\ninfograph/hot_tub/infograph_152_000050.jpg\ninfograph/hot_tub/infograph_152_000078.jpg\ninfograph/hot_tub/infograph_152_000085.jpg\ninfograph/house/infograph_154_000306.jpg\ninfograph/house_plant/infograph_155_000095.jpg\ninfograph/house_plant/infograph_155_000101.jpg\ninfograph/house_plant/infograph_155_000110.jpg\ninfograph/house_plant/infograph_155_000113.jpg\ninfograph/house_plant/infograph_155_000138.jpg\ninfograph/house_plant/infograph_155_000193.jpg\ninfograph/house_plant/infograph_155_000268.jpg\ninfograph/ice_cream/infograph_157_000038.jpg\ninfograph/ice_cream/infograph_157_000151.jpg\ninfograph/ice_cream/infograph_157_000178.jpg\ninfograph/ice_cream/infograph_157_000180.jpg\ninfograph/jacket/infograph_158_000023.jpg\ninfograph/jacket/infograph_158_000029.jpg\ninfograph/jacket/infograph_158_000078.jpg\ninfograph/keyboard/infograph_162_000053.jpg\ninfograph/knee/infograph_163_000018.jpg\ninfograph/lantern/infograph_166_000014.jpg\ninfograph/laptop/infograph_167_000008.jpg\ninfograph/laptop/infograph_167_000082.jpg\ninfograph/laptop/infograph_167_000112.jpg\ninfograph/leg/infograph_169_000018.jpg\ninfograph/leg/infograph_169_000032.jpg\ninfograph/leg/infograph_169_000063.jpg\ninfograph/leg/infograph_169_000082.jpg\ninfograph/leg/infograph_169_000085.jpg\ninfograph/leg/infograph_169_000087.jpg\ninfograph/leg/infograph_169_000116.jpg\ninfograph/leg/infograph_169_000118.jpg\ninfograph/leg/infograph_169_000158.jpg\ninfograph/light_bulb/infograph_170_000014.jpg\ninfograph/light_bulb/infograph_170_000086.jpg\ninfograph/light_bulb/infograph_170_000098.jpg\ninfograph/light_bulb/infograph_170_000174.jpg\ninfograph/lighter/infograph_171_000006.jpg\ninfograph/lighter/infograph_171_000008.jpg\ninfograph/line/infograph_174_000027.jpg\ninfograph/line/infograph_174_000098.jpg\ninfograph/line/infograph_174_000117.jpg\ninfograph/lobster/infograph_177_000009.jpg\ninfograph/lobster/infograph_177_000014.jpg\ninfograph/lobster/infograph_177_000017.jpg\ninfograph/lobster/infograph_177_000029.jpg\ninfograph/lobster/infograph_177_000033.jpg\ninfograph/lobster/infograph_177_000038.jpg\ninfograph/lobster/infograph_177_000043.jpg\ninfograph/map/infograph_180_000043.jpg\ninfograph/map/infograph_180_000044.jpg\ninfograph/matches/infograph_182_000005.jpg\ninfograph/microwave/infograph_186_000027.jpg\ninfograph/microwave/infograph_186_000083.jpg\ninfograph/microwave/infograph_186_000102.jpg\ninfograph/mosquito/infograph_189_000195.jpg\ninfograph/motorbike/infograph_190_000010.jpg\ninfograph/motorbike/infograph_190_000024.jpg\ninfograph/motorbike/infograph_190_000131.jpg\ninfograph/mouse/infograph_192_000019.jpg\ninfograph/mouse/infograph_192_000036.jpg\ninfograph/mouse/infograph_192_000043.jpg\ninfograph/moustache/infograph_193_000017.jpg\ninfograph/moustache/infograph_193_000023.jpg\ninfograph/moustache/infograph_193_000027.jpg\ninfograph/mug/infograph_195_000008.jpg\ninfograph/mug/infograph_195_000014.jpg\ninfograph/mug/infograph_195_000016.jpg\ninfograph/mug/infograph_195_000041.jpg\ninfograph/mushroom/infograph_196_000016.jpg\ninfograph/mushroom/infograph_196_000053.jpg\ninfograph/mushroom/infograph_196_000069.jpg\ninfograph/mushroom/infograph_196_000071.jpg\ninfograph/mushroom/infograph_196_000081.jpg\ninfograph/mushroom/infograph_196_000095.jpg\ninfograph/mushroom/infograph_196_000101.jpg\ninfograph/mushroom/infograph_196_000125.jpg\ninfograph/mushroom/infograph_196_000131.jpg\ninfograph/mushroom/infograph_196_000145.jpg\ninfograph/mushroom/infograph_196_000157.jpg\ninfograph/mushroom/infograph_196_000193.jpg\ninfograph/mushroom/infograph_196_000218.jpg\ninfograph/mushroom/infograph_196_000228.jpg\ninfograph/mushroom/infograph_196_000232.jpg\ninfograph/mushroom/infograph_196_000252.jpg\ninfograph/mushroom/infograph_196_000260.jpg\ninfograph/mushroom/infograph_196_000289.jpg\ninfograph/nail/infograph_197_000024.jpg\ninfograph/nail/infograph_197_000044.jpg\ninfograph/nail/infograph_197_000045.jpg\ninfograph/nail/infograph_197_000050.jpg\ninfograph/nail/infograph_197_000078.jpg\ninfograph/nail/infograph_197_000096.jpg\ninfograph/nail/infograph_197_000107.jpg\ninfograph/nail/infograph_197_000146.jpg\ninfograph/nail/infograph_197_000176.jpg\ninfograph/necklace/infograph_198_000013.jpg\ninfograph/necklace/infograph_198_000023.jpg\ninfograph/necklace/infograph_198_000060.jpg\ninfograph/necklace/infograph_198_000077.jpg\ninfograph/nose/infograph_199_000013.jpg\ninfograph/nose/infograph_199_000037.jpg\ninfograph/nose/infograph_199_000095.jpg\ninfograph/nose/infograph_199_000097.jpg\ninfograph/nose/infograph_199_000105.jpg\ninfograph/nose/infograph_199_000114.jpg\ninfograph/nose/infograph_199_000165.jpg\ninfograph/nose/infograph_199_000179.jpg\ninfograph/nose/infograph_199_000186.jpg\ninfograph/nose/infograph_199_000204.jpg\ninfograph/nose/infograph_199_000214.jpg\ninfograph/nose/infograph_199_000219.jpg\ninfograph/octopus/infograph_202_000001.jpg\ninfograph/octopus/infograph_202_000022.jpg\ninfograph/octopus/infograph_202_000028.jpg\ninfograph/onion/infograph_203_000028.jpg\ninfograph/onion/infograph_203_000039.jpg\ninfograph/oven/infograph_204_000006.jpg\ninfograph/oven/infograph_204_000012.jpg\ninfograph/oven/infograph_204_000013.jpg\ninfograph/oven/infograph_204_000023.jpg\ninfograph/oven/infograph_204_000027.jpg\ninfograph/oven/infograph_204_000031.jpg\ninfograph/oven/infograph_204_000032.jpg\ninfograph/oven/infograph_204_000037.jpg\ninfograph/oven/infograph_204_000040.jpg\ninfograph/oven/infograph_204_000051.jpg\ninfograph/owl/infograph_205_000004.jpg\ninfograph/owl/infograph_205_000039.jpg\ninfograph/owl/infograph_205_000048.jpg\ninfograph/owl/infograph_205_000086.jpg\ninfograph/owl/infograph_205_000092.jpg\ninfograph/owl/infograph_205_000100.jpg\ninfograph/owl/infograph_205_000111.jpg\ninfograph/paintbrush/infograph_206_000003.jpg\ninfograph/paint_can/infograph_207_000026.jpg\ninfograph/palm_tree/infograph_208_000001.jpg\ninfograph/palm_tree/infograph_208_000004.jpg\ninfograph/palm_tree/infograph_208_000024.jpg\ninfograph/palm_tree/infograph_208_000041.jpg\ninfograph/palm_tree/infograph_208_000079.jpg\ninfograph/palm_tree/infograph_208_000086.jpg\ninfograph/palm_tree/infograph_208_000109.jpg\ninfograph/palm_tree/infograph_208_000117.jpg\ninfograph/palm_tree/infograph_208_000146.jpg\ninfograph/palm_tree/infograph_208_000253.jpg\ninfograph/palm_tree/infograph_208_000268.jpg\ninfograph/palm_tree/infograph_208_000276.jpg\ninfograph/pants/infograph_210_000009.jpg\ninfograph/pants/infograph_210_000037.jpg\ninfograph/pants/infograph_210_000049.jpg\ninfograph/pants/infograph_210_000050.jpg\ninfograph/pants/infograph_210_000053.jpg\ninfograph/pants/infograph_210_000059.jpg\ninfograph/pants/infograph_210_000120.jpg\ninfograph/parrot/infograph_213_000052.jpg\ninfograph/peanut/infograph_215_000014.jpg\ninfograph/pear/infograph_216_000005.jpg\ninfograph/pear/infograph_216_000017.jpg\ninfograph/pear/infograph_216_000040.jpg\ninfograph/pear/infograph_216_000043.jpg\ninfograph/pear/infograph_216_000059.jpg\ninfograph/pear/infograph_216_000065.jpg\ninfograph/pear/infograph_216_000076.jpg\ninfograph/peas/infograph_217_000008.jpg\ninfograph/peas/infograph_217_000011.jpg\ninfograph/peas/infograph_217_000026.jpg\ninfograph/peas/infograph_217_000044.jpg\ninfograph/peas/infograph_217_000045.jpg\ninfograph/peas/infograph_217_000050.jpg\ninfograph/peas/infograph_217_000052.jpg\ninfograph/peas/infograph_217_000072.jpg\ninfograph/peas/infograph_217_000076.jpg\ninfograph/peas/infograph_217_000077.jpg\ninfograph/peas/infograph_217_000080.jpg\ninfograph/peas/infograph_217_000098.jpg\ninfograph/peas/infograph_217_000119.jpg\ninfograph/pencil/infograph_218_000033.jpg\ninfograph/pencil/infograph_218_000310.jpg\ninfograph/pencil/infograph_218_000361.jpg\ninfograph/penguin/infograph_219_000034.jpg\ninfograph/penguin/infograph_219_000092.jpg\ninfograph/penguin/infograph_219_000122.jpg\ninfograph/penguin/infograph_219_000149.jpg\ninfograph/piano/infograph_220_000006.jpg\ninfograph/piano/infograph_220_000013.jpg\ninfograph/piano/infograph_220_000019.jpg\ninfograph/pig/infograph_223_000121.jpg\ninfograph/pig/infograph_223_000187.jpg\ninfograph/pillow/infograph_224_000001.jpg\ninfograph/pillow/infograph_224_000003.jpg\ninfograph/pillow/infograph_224_000011.jpg\ninfograph/pillow/infograph_224_000047.jpg\ninfograph/pillow/infograph_224_000050.jpg\ninfograph/pillow/infograph_224_000087.jpg\ninfograph/pillow/infograph_224_000093.jpg\ninfograph/pillow/infograph_224_000097.jpg\ninfograph/pillow/infograph_224_000142.jpg\ninfograph/pillow/infograph_224_000154.jpg\ninfograph/pineapple/infograph_225_000005.jpg\ninfograph/pineapple/infograph_225_000045.jpg\ninfograph/pineapple/infograph_225_000048.jpg\ninfograph/pineapple/infograph_225_000065.jpg\ninfograph/pineapple/infograph_225_000067.jpg\ninfograph/pineapple/infograph_225_000086.jpg\ninfograph/pizza/infograph_226_000044.jpg\ninfograph/pizza/infograph_226_000090.jpg\ninfograph/pizza/infograph_226_000140.jpg\ninfograph/pizza/infograph_226_000146.jpg\ninfograph/pliers/infograph_227_000063.jpg\ninfograph/police_car/infograph_228_000035.jpg\ninfograph/pond/infograph_229_000012.jpg\ninfograph/pond/infograph_229_000015.jpg\ninfograph/pond/infograph_229_000047.jpg\ninfograph/pond/infograph_229_000062.jpg\ninfograph/pool/infograph_230_000011.jpg\ninfograph/pool/infograph_230_000076.jpg\ninfograph/pool/infograph_230_000091.jpg\ninfograph/pool/infograph_230_000106.jpg\ninfograph/popsicle/infograph_231_000047.jpg\ninfograph/popsicle/infograph_231_000057.jpg\ninfograph/potato/infograph_233_000010.jpg\ninfograph/potato/infograph_233_000026.jpg\ninfograph/potato/infograph_233_000036.jpg\ninfograph/potato/infograph_233_000055.jpg\ninfograph/power_outlet/infograph_234_000013.jpg\ninfograph/power_outlet/infograph_234_000022.jpg\ninfograph/power_outlet/infograph_234_000040.jpg\ninfograph/purse/infograph_235_000009.jpg\ninfograph/purse/infograph_235_000029.jpg\ninfograph/purse/infograph_235_000060.jpg\ninfograph/purse/infograph_235_000076.jpg\ninfograph/rabbit/infograph_236_000006.jpg\ninfograph/rabbit/infograph_236_000044.jpg\ninfograph/raccoon/infograph_237_000016.jpg\ninfograph/radio/infograph_238_000057.jpg\ninfograph/radio/infograph_238_000085.jpg\ninfograph/rain/infograph_239_000046.jpg\ninfograph/rain/infograph_239_000077.jpg\ninfograph/rake/infograph_241_000023.jpg\ninfograph/rake/infograph_241_000029.jpg\ninfograph/rake/infograph_241_000063.jpg\ninfograph/remote_control/infograph_242_000066.jpg\ninfograph/rhinoceros/infograph_243_000015.jpg\ninfograph/rhinoceros/infograph_243_000039.jpg\ninfograph/rhinoceros/infograph_243_000065.jpg\ninfograph/rifle/infograph_244_000064.jpg\ninfograph/river/infograph_245_000026.jpg\ninfograph/river/infograph_245_000038.jpg\ninfograph/river/infograph_245_000047.jpg\ninfograph/river/infograph_245_000050.jpg\ninfograph/river/infograph_245_000118.jpg\ninfograph/river/infograph_245_000144.jpg\ninfograph/sailboat/infograph_248_000054.jpg\ninfograph/sailboat/infograph_248_000067.jpg\ninfograph/sandwich/infograph_249_000006.jpg\ninfograph/sandwich/infograph_249_000009.jpg\ninfograph/sandwich/infograph_249_000018.jpg\ninfograph/sandwich/infograph_249_000023.jpg\ninfograph/sandwich/infograph_249_000028.jpg\ninfograph/sandwich/infograph_249_000039.jpg\ninfograph/sandwich/infograph_249_000048.jpg\ninfograph/sandwich/infograph_249_000066.jpg\ninfograph/sandwich/infograph_249_000071.jpg\ninfograph/sandwich/infograph_249_000074.jpg\ninfograph/sandwich/infograph_249_000082.jpg\ninfograph/sandwich/infograph_249_000085.jpg\ninfograph/sandwich/infograph_249_000092.jpg\ninfograph/sandwich/infograph_249_000094.jpg\ninfograph/sandwich/infograph_249_000107.jpg\ninfograph/saw/infograph_250_000006.jpg\ninfograph/saw/infograph_250_000016.jpg\ninfograph/saw/infograph_250_000020.jpg\ninfograph/saxophone/infograph_251_000007.jpg\ninfograph/saxophone/infograph_251_000009.jpg\ninfograph/saxophone/infograph_251_000067.jpg\ninfograph/saxophone/infograph_251_000069.jpg\ninfograph/school_bus/infograph_252_000003.jpg\ninfograph/school_bus/infograph_252_000006.jpg\ninfograph/school_bus/infograph_252_000010.jpg\ninfograph/school_bus/infograph_252_000012.jpg\ninfograph/school_bus/infograph_252_000014.jpg\ninfograph/school_bus/infograph_252_000018.jpg\ninfograph/school_bus/infograph_252_000019.jpg\ninfograph/school_bus/infograph_252_000028.jpg\ninfograph/school_bus/infograph_252_000035.jpg\ninfograph/school_bus/infograph_252_000036.jpg\ninfograph/school_bus/infograph_252_000044.jpg\ninfograph/school_bus/infograph_252_000048.jpg\ninfograph/school_bus/infograph_252_000049.jpg\ninfograph/school_bus/infograph_252_000051.jpg\ninfograph/school_bus/infograph_252_000054.jpg\ninfograph/school_bus/infograph_252_000063.jpg\ninfograph/school_bus/infograph_252_000066.jpg\ninfograph/school_bus/infograph_252_000071.jpg\ninfograph/school_bus/infograph_252_000074.jpg\ninfograph/school_bus/infograph_252_000075.jpg\ninfograph/school_bus/infograph_252_000085.jpg\ninfograph/school_bus/infograph_252_000086.jpg\ninfograph/school_bus/infograph_252_000091.jpg\ninfograph/school_bus/infograph_252_000093.jpg\ninfograph/school_bus/infograph_252_000100.jpg\ninfograph/school_bus/infograph_252_000105.jpg\ninfograph/school_bus/infograph_252_000112.jpg\ninfograph/school_bus/infograph_252_000119.jpg\ninfograph/school_bus/infograph_252_000121.jpg\ninfograph/school_bus/infograph_252_000125.jpg\ninfograph/school_bus/infograph_252_000126.jpg\ninfograph/school_bus/infograph_252_000127.jpg\ninfograph/school_bus/infograph_252_000129.jpg\ninfograph/school_bus/infograph_252_000131.jpg\ninfograph/school_bus/infograph_252_000132.jpg\ninfograph/school_bus/infograph_252_000133.jpg\ninfograph/school_bus/infograph_252_000134.jpg\ninfograph/school_bus/infograph_252_000136.jpg\ninfograph/school_bus/infograph_252_000138.jpg\ninfograph/school_bus/infograph_252_000139.jpg\ninfograph/scissors/infograph_253_000008.jpg\ninfograph/scissors/infograph_253_000015.jpg\ninfograph/scissors/infograph_253_000024.jpg\ninfograph/scissors/infograph_253_000060.jpg\ninfograph/scissors/infograph_253_000075.jpg\ninfograph/scorpion/infograph_254_000003.jpg\ninfograph/scorpion/infograph_254_000007.jpg\ninfograph/scorpion/infograph_254_000042.jpg\ninfograph/scorpion/infograph_254_000046.jpg\ninfograph/scorpion/infograph_254_000049.jpg\ninfograph/screwdriver/infograph_255_000004.jpg\ninfograph/screwdriver/infograph_255_000008.jpg\ninfograph/screwdriver/infograph_255_000020.jpg\ninfograph/screwdriver/infograph_255_000027.jpg\ninfograph/screwdriver/infograph_255_000030.jpg\ninfograph/sea_turtle/infograph_256_000020.jpg\ninfograph/sea_turtle/infograph_256_000033.jpg\ninfograph/sea_turtle/infograph_256_000055.jpg\ninfograph/sea_turtle/infograph_256_000058.jpg\ninfograph/sea_turtle/infograph_256_000069.jpg\ninfograph/sea_turtle/infograph_256_000086.jpg\ninfograph/sea_turtle/infograph_256_000087.jpg\ninfograph/sea_turtle/infograph_256_000112.jpg\ninfograph/shark/infograph_258_000011.jpg\ninfograph/shark/infograph_258_000073.jpg\ninfograph/shark/infograph_258_000109.jpg\ninfograph/shark/infograph_258_000231.jpg\ninfograph/shark/infograph_258_000278.jpg\ninfograph/sheep/infograph_259_000043.jpg\ninfograph/shoe/infograph_260_000037.jpg\ninfograph/shoe/infograph_260_000056.jpg\ninfograph/shoe/infograph_260_000142.jpg\ninfograph/shoe/infograph_260_000175.jpg\ninfograph/shoe/infograph_260_000183.jpg\ninfograph/shorts/infograph_261_000022.jpg\ninfograph/shorts/infograph_261_000029.jpg\ninfograph/skateboard/infograph_264_000046.jpg\ninfograph/skyscraper/infograph_266_000003.jpg\ninfograph/skyscraper/infograph_266_000125.jpg\ninfograph/snail/infograph_269_000014.jpg\ninfograph/snake/infograph_270_000010.jpg\ninfograph/snake/infograph_270_000012.jpg\ninfograph/snake/infograph_270_000021.jpg\ninfograph/snake/infograph_270_000056.jpg\ninfograph/snowman/infograph_273_000100.jpg\ninfograph/sock/infograph_275_000050.jpg\ninfograph/speedboat/infograph_276_000038.jpg\ninfograph/speedboat/infograph_276_000062.jpg\ninfograph/spider/infograph_277_000036.jpg\ninfograph/spreadsheet/infograph_279_000001.jpg\ninfograph/spreadsheet/infograph_279_000067.jpg\ninfograph/spreadsheet/infograph_279_000122.jpg\ninfograph/square/infograph_280_000074.jpg\ninfograph/square/infograph_280_000085.jpg\ninfograph/squiggle/infograph_281_000042.jpg\ninfograph/squirrel/infograph_282_000158.jpg\ninfograph/stairs/infograph_283_000013.jpg\ninfograph/steak/infograph_285_000016.jpg\ninfograph/steak/infograph_285_000102.jpg\ninfograph/steak/infograph_285_000162.jpg\ninfograph/steak/infograph_285_000168.jpg\ninfograph/steak/infograph_285_000198.jpg\ninfograph/steak/infograph_285_000206.jpg\ninfograph/steak/infograph_285_000211.jpg\ninfograph/steak/infograph_285_000349.jpg\ninfograph/stereo/infograph_286_000011.jpg\ninfograph/stereo/infograph_286_000283.jpg\ninfograph/stove/infograph_290_000001.jpg\ninfograph/stove/infograph_290_000047.jpg\ninfograph/stove/infograph_290_000084.jpg\ninfograph/stove/infograph_290_000124.jpg\ninfograph/strawberry/infograph_291_000274.jpg\ninfograph/strawberry/infograph_291_000279.jpg\ninfograph/string_bean/infograph_293_000009.jpg\ninfograph/string_bean/infograph_293_000020.jpg\ninfograph/string_bean/infograph_293_000086.jpg\ninfograph/submarine/infograph_294_000033.jpg\ninfograph/submarine/infograph_294_000086.jpg\ninfograph/suitcase/infograph_295_000060.jpg\ninfograph/suitcase/infograph_295_000143.jpg\ninfograph/suitcase/infograph_295_000159.jpg\ninfograph/suitcase/infograph_295_000183.jpg\ninfograph/sun/infograph_296_000006.jpg\ninfograph/sun/infograph_296_000147.jpg\ninfograph/sweater/infograph_298_000066.jpg\ninfograph/sweater/infograph_298_000074.jpg\ninfograph/sword/infograph_300_000019.jpg\ninfograph/syringe/infograph_301_000126.jpg\ninfograph/table/infograph_302_000011.jpg\ninfograph/table/infograph_302_000030.jpg\ninfograph/table/infograph_302_000069.jpg\ninfograph/table/infograph_302_000184.jpg\ninfograph/table/infograph_302_000240.jpg\ninfograph/table/infograph_302_000241.jpg\ninfograph/table/infograph_302_000245.jpg\ninfograph/table/infograph_302_000306.jpg\ninfograph/table/infograph_302_000316.jpg\ninfograph/table/infograph_302_000351.jpg\ninfograph/table/infograph_302_000387.jpg\ninfograph/table/infograph_302_000491.jpg\ninfograph/table/infograph_302_000566.jpg\ninfograph/table/infograph_302_000606.jpg\ninfograph/table/infograph_302_000675.jpg\ninfograph/table/infograph_302_000678.jpg\ninfograph/teddy-bear/infograph_304_000102.jpg\ninfograph/telephone/infograph_305_000227.jpg\ninfograph/telephone/infograph_305_000263.jpg\ninfograph/television/infograph_306_000011.jpg\ninfograph/television/infograph_306_000094.jpg\ninfograph/television/infograph_306_000138.jpg\ninfograph/television/infograph_306_000217.jpg\ninfograph/television/infograph_306_000366.jpg\ninfograph/television/infograph_306_000469.jpg\ninfograph/tennis_racquet/infograph_307_000046.jpg\ninfograph/tent/infograph_308_000150.jpg\ninfograph/tent/infograph_308_000207.jpg\ninfograph/The_Eiffel_Tower/infograph_309_000030.jpg\ninfograph/tiger/infograph_312_000008.jpg\ninfograph/tiger/infograph_312_000039.jpg\ninfograph/tiger/infograph_312_000049.jpg\ninfograph/tiger/infograph_312_000056.jpg\ninfograph/tiger/infograph_312_000062.jpg\ninfograph/tiger/infograph_312_000082.jpg\ninfograph/tiger/infograph_312_000091.jpg\ninfograph/tiger/infograph_312_000100.jpg\ninfograph/tiger/infograph_312_000136.jpg\ninfograph/tiger/infograph_312_000166.jpg\ninfograph/tiger/infograph_312_000197.jpg\ninfograph/tiger/infograph_312_000212.jpg\ninfograph/tiger/infograph_312_000224.jpg\ninfograph/tiger/infograph_312_000240.jpg\ninfograph/tiger/infograph_312_000253.jpg\ninfograph/tiger/infograph_312_000284.jpg\ninfograph/toaster/infograph_313_000007.jpg\ninfograph/toaster/infograph_313_000011.jpg\ninfograph/toaster/infograph_313_000030.jpg\ninfograph/toaster/infograph_313_000040.jpg\ninfograph/toaster/infograph_313_000060.jpg\ninfograph/toaster/infograph_313_000088.jpg\ninfograph/toaster/infograph_313_000093.jpg\ninfograph/toaster/infograph_313_000105.jpg\ninfograph/toaster/infograph_313_000145.jpg\ninfograph/toaster/infograph_313_000147.jpg\ninfograph/toaster/infograph_313_000180.jpg\ninfograph/toaster/infograph_313_000201.jpg\ninfograph/toaster/infograph_313_000252.jpg\ninfograph/toaster/infograph_313_000260.jpg\ninfograph/toaster/infograph_313_000265.jpg\ninfograph/toaster/infograph_313_000277.jpg\ninfograph/toaster/infograph_313_000286.jpg\ninfograph/toaster/infograph_313_000300.jpg\ninfograph/toaster/infograph_313_000315.jpg\ninfograph/toaster/infograph_313_000333.jpg\ninfograph/toe/infograph_314_000001.jpg\ninfograph/toe/infograph_314_000021.jpg\ninfograph/toe/infograph_314_000030.jpg\ninfograph/toe/infograph_314_000035.jpg\ninfograph/toe/infograph_314_000043.jpg\ninfograph/toe/infograph_314_000044.jpg\ninfograph/toe/infograph_314_000047.jpg\ninfograph/toe/infograph_314_000053.jpg\ninfograph/toe/infograph_314_000068.jpg\ninfograph/toe/infograph_314_000078.jpg\ninfograph/toe/infograph_314_000081.jpg\ninfograph/toe/infograph_314_000095.jpg\ninfograph/toe/infograph_314_000097.jpg\ninfograph/toe/infograph_314_000098.jpg\ninfograph/toe/infograph_314_000100.jpg\ninfograph/toe/infograph_314_000101.jpg\ninfograph/toe/infograph_314_000107.jpg\ninfograph/toe/infograph_314_000114.jpg\ninfograph/toe/infograph_314_000122.jpg\ninfograph/toe/infograph_314_000125.jpg\ninfograph/toe/infograph_314_000127.jpg\ninfograph/toe/infograph_314_000148.jpg\ninfograph/toe/infograph_314_000149.jpg\ninfograph/toe/infograph_314_000151.jpg\ninfograph/toe/infograph_314_000152.jpg\ninfograph/toe/infograph_314_000162.jpg\ninfograph/toe/infograph_314_000163.jpg\ninfograph/toe/infograph_314_000168.jpg\ninfograph/toe/infograph_314_000169.jpg\ninfograph/toe/infograph_314_000173.jpg\ninfograph/toe/infograph_314_000174.jpg\ninfograph/toe/infograph_314_000179.jpg\ninfograph/toe/infograph_314_000186.jpg\ninfograph/toe/infograph_314_000221.jpg\ninfograph/toe/infograph_314_000223.jpg\ninfograph/toe/infograph_314_000230.jpg\ninfograph/toe/infograph_314_000232.jpg\ninfograph/toe/infograph_314_000238.jpg\ninfograph/toe/infograph_314_000249.jpg\ninfograph/toe/infograph_314_000251.jpg\ninfograph/toe/infograph_314_000253.jpg\ninfograph/toe/infograph_314_000254.jpg\ninfograph/toe/infograph_314_000280.jpg\ninfograph/toe/infograph_314_000282.jpg\ninfograph/toe/infograph_314_000293.jpg\ninfograph/toe/infograph_314_000297.jpg\ninfograph/toe/infograph_314_000301.jpg\ninfograph/toe/infograph_314_000304.jpg\ninfograph/toe/infograph_314_000307.jpg\ninfograph/toe/infograph_314_000311.jpg\ninfograph/toe/infograph_314_000320.jpg\ninfograph/toe/infograph_314_000325.jpg\ninfograph/toe/infograph_314_000326.jpg\ninfograph/toe/infograph_314_000333.jpg\ninfograph/toe/infograph_314_000336.jpg\ninfograph/toe/infograph_314_000337.jpg\ninfograph/toe/infograph_314_000339.jpg\ninfograph/toe/infograph_314_000349.jpg\ninfograph/toe/infograph_314_000352.jpg\ninfograph/toe/infograph_314_000359.jpg\ninfograph/toe/infograph_314_000361.jpg\ninfograph/toe/infograph_314_000362.jpg\ninfograph/toe/infograph_314_000366.jpg\ninfograph/toe/infograph_314_000382.jpg\ninfograph/toe/infograph_314_000384.jpg\ninfograph/toe/infograph_314_000385.jpg\ninfograph/toe/infograph_314_000386.jpg\ninfograph/toe/infograph_314_000388.jpg\ninfograph/toe/infograph_314_000401.jpg\ninfograph/toe/infograph_314_000403.jpg\ninfograph/toe/infograph_314_000405.jpg\ninfograph/toilet/infograph_315_000008.jpg\ninfograph/toilet/infograph_315_000054.jpg\ninfograph/toilet/infograph_315_000060.jpg\ninfograph/toilet/infograph_315_000071.jpg\ninfograph/toilet/infograph_315_000126.jpg\ninfograph/toilet/infograph_315_000127.jpg\ninfograph/toilet/infograph_315_000154.jpg\ninfograph/toilet/infograph_315_000155.jpg\ninfograph/toilet/infograph_315_000172.jpg\ninfograph/toilet/infograph_315_000224.jpg\ninfograph/toilet/infograph_315_000280.jpg\ninfograph/toilet/infograph_315_000391.jpg\ninfograph/toilet/infograph_315_000418.jpg\ninfograph/toilet/infograph_315_000433.jpg\ninfograph/toilet/infograph_315_000483.jpg\ninfograph/toilet/infograph_315_000489.jpg\ninfograph/tooth/infograph_316_000027.jpg\ninfograph/tooth/infograph_316_000063.jpg\ninfograph/tooth/infograph_316_000106.jpg\ninfograph/tooth/infograph_316_000131.jpg\ninfograph/tooth/infograph_316_000140.jpg\ninfograph/tooth/infograph_316_000158.jpg\ninfograph/tooth/infograph_316_000218.jpg\ninfograph/tooth/infograph_316_000232.jpg\ninfograph/tooth/infograph_316_000320.jpg\ninfograph/tooth/infograph_316_000353.jpg\ninfograph/tooth/infograph_316_000406.jpg\ninfograph/toothbrush/infograph_317_000004.jpg\ninfograph/toothbrush/infograph_317_000014.jpg\ninfograph/toothbrush/infograph_317_000022.jpg\ninfograph/toothbrush/infograph_317_000046.jpg\ninfograph/toothbrush/infograph_317_000070.jpg\ninfograph/toothbrush/infograph_317_000101.jpg\ninfograph/toothbrush/infograph_317_000115.jpg\ninfograph/toothbrush/infograph_317_000116.jpg\ninfograph/toothbrush/infograph_317_000124.jpg\ninfograph/toothbrush/infograph_317_000147.jpg\ninfograph/toothbrush/infograph_317_000148.jpg\ninfograph/toothbrush/infograph_317_000151.jpg\ninfograph/toothbrush/infograph_317_000156.jpg\ninfograph/toothbrush/infograph_317_000161.jpg\ninfograph/toothbrush/infograph_317_000163.jpg\ninfograph/toothbrush/infograph_317_000182.jpg\ninfograph/toothbrush/infograph_317_000186.jpg\ninfograph/toothbrush/infograph_317_000209.jpg\ninfograph/toothbrush/infograph_317_000212.jpg\ninfograph/toothbrush/infograph_317_000231.jpg\ninfograph/toothbrush/infograph_317_000240.jpg\ninfograph/toothbrush/infograph_317_000250.jpg\ninfograph/toothbrush/infograph_317_000265.jpg\ninfograph/toothbrush/infograph_317_000273.jpg\ninfograph/toothbrush/infograph_317_000279.jpg\ninfograph/toothbrush/infograph_317_000284.jpg\ninfograph/toothbrush/infograph_317_000290.jpg\ninfograph/toothbrush/infograph_317_000291.jpg\ninfograph/toothbrush/infograph_317_000295.jpg\ninfograph/toothbrush/infograph_317_000300.jpg\ninfograph/toothbrush/infograph_317_000303.jpg\ninfograph/toothbrush/infograph_317_000314.jpg\ninfograph/toothbrush/infograph_317_000328.jpg\ninfograph/toothbrush/infograph_317_000331.jpg\ninfograph/toothbrush/infograph_317_000340.jpg\ninfograph/toothbrush/infograph_317_000356.jpg\ninfograph/toothbrush/infograph_317_000368.jpg\ninfograph/toothbrush/infograph_317_000391.jpg\ninfograph/toothbrush/infograph_317_000392.jpg\ninfograph/toothbrush/infograph_317_000400.jpg\ninfograph/toothbrush/infograph_317_000415.jpg\ninfograph/toothbrush/infograph_317_000419.jpg\ninfograph/toothbrush/infograph_317_000420.jpg\ninfograph/toothbrush/infograph_317_000421.jpg\ninfograph/toothbrush/infograph_317_000430.jpg\ninfograph/toothbrush/infograph_317_000437.jpg\ninfograph/toothbrush/infograph_317_000455.jpg\ninfograph/toothbrush/infograph_317_000456.jpg\ninfograph/toothbrush/infograph_317_000467.jpg\ninfograph/toothbrush/infograph_317_000478.jpg\ninfograph/toothbrush/infograph_317_000482.jpg\ninfograph/toothbrush/infograph_317_000494.jpg\ninfograph/toothbrush/infograph_317_000502.jpg\ninfograph/toothbrush/infograph_317_000515.jpg\ninfograph/toothbrush/infograph_317_000523.jpg\ninfograph/toothbrush/infograph_317_000525.jpg\ninfograph/toothbrush/infograph_317_000526.jpg\ninfograph/toothbrush/infograph_317_000531.jpg\ninfograph/toothbrush/infograph_317_000536.jpg\ninfograph/toothbrush/infograph_317_000556.jpg\ninfograph/toothpaste/infograph_318_000003.jpg\ninfograph/toothpaste/infograph_318_000009.jpg\ninfograph/toothpaste/infograph_318_000012.jpg\ninfograph/toothpaste/infograph_318_000016.jpg\ninfograph/toothpaste/infograph_318_000019.jpg\ninfograph/toothpaste/infograph_318_000020.jpg\ninfograph/toothpaste/infograph_318_000023.jpg\ninfograph/toothpaste/infograph_318_000024.jpg\ninfograph/toothpaste/infograph_318_000028.jpg\ninfograph/toothpaste/infograph_318_000029.jpg\ninfograph/toothpaste/infograph_318_000032.jpg\ninfograph/toothpaste/infograph_318_000033.jpg\ninfograph/toothpaste/infograph_318_000034.jpg\ninfograph/toothpaste/infograph_318_000044.jpg\ninfograph/toothpaste/infograph_318_000046.jpg\ninfograph/toothpaste/infograph_318_000047.jpg\ninfograph/toothpaste/infograph_318_000049.jpg\ninfograph/toothpaste/infograph_318_000053.jpg\ninfograph/toothpaste/infograph_318_000054.jpg\ninfograph/toothpaste/infograph_318_000062.jpg\ninfograph/toothpaste/infograph_318_000080.jpg\ninfograph/toothpaste/infograph_318_000081.jpg\ninfograph/toothpaste/infograph_318_000085.jpg\ninfograph/toothpaste/infograph_318_000086.jpg\ninfograph/toothpaste/infograph_318_000091.jpg\ninfograph/toothpaste/infograph_318_000096.jpg\ninfograph/toothpaste/infograph_318_000104.jpg\ninfograph/toothpaste/infograph_318_000106.jpg\ninfograph/toothpaste/infograph_318_000111.jpg\ninfograph/toothpaste/infograph_318_000114.jpg\ninfograph/toothpaste/infograph_318_000118.jpg\ninfograph/toothpaste/infograph_318_000120.jpg\ninfograph/toothpaste/infograph_318_000122.jpg\ninfograph/toothpaste/infograph_318_000132.jpg\ninfograph/toothpaste/infograph_318_000135.jpg\ninfograph/toothpaste/infograph_318_000136.jpg\ninfograph/toothpaste/infograph_318_000148.jpg\ninfograph/toothpaste/infograph_318_000153.jpg\ninfograph/toothpaste/infograph_318_000157.jpg\ninfograph/toothpaste/infograph_318_000159.jpg\ninfograph/toothpaste/infograph_318_000174.jpg\ninfograph/toothpaste/infograph_318_000176.jpg\ninfograph/toothpaste/infograph_318_000177.jpg\ninfograph/toothpaste/infograph_318_000178.jpg\ninfograph/toothpaste/infograph_318_000186.jpg\ninfograph/toothpaste/infograph_318_000188.jpg\ninfograph/toothpaste/infograph_318_000193.jpg\ninfograph/toothpaste/infograph_318_000194.jpg\ninfograph/toothpaste/infograph_318_000196.jpg\ninfograph/toothpaste/infograph_318_000200.jpg\ninfograph/toothpaste/infograph_318_000208.jpg\ninfograph/toothpaste/infograph_318_000214.jpg\ninfograph/toothpaste/infograph_318_000215.jpg\ninfograph/toothpaste/infograph_318_000219.jpg\ninfograph/toothpaste/infograph_318_000222.jpg\ninfograph/toothpaste/infograph_318_000224.jpg\ninfograph/toothpaste/infograph_318_000231.jpg\ninfograph/toothpaste/infograph_318_000235.jpg\ninfograph/toothpaste/infograph_318_000243.jpg\ninfograph/toothpaste/infograph_318_000245.jpg\ninfograph/toothpaste/infograph_318_000246.jpg\ninfograph/toothpaste/infograph_318_000247.jpg\ninfograph/toothpaste/infograph_318_000251.jpg\ninfograph/toothpaste/infograph_318_000264.jpg\ninfograph/toothpaste/infograph_318_000266.jpg\ninfograph/toothpaste/infograph_318_000269.jpg\ninfograph/toothpaste/infograph_318_000270.jpg\ninfograph/toothpaste/infograph_318_000273.jpg\ninfograph/toothpaste/infograph_318_000276.jpg\ninfograph/toothpaste/infograph_318_000287.jpg\ninfograph/toothpaste/infograph_318_000288.jpg\ninfograph/toothpaste/infograph_318_000297.jpg\ninfograph/toothpaste/infograph_318_000298.jpg\ninfograph/toothpaste/infograph_318_000305.jpg\ninfograph/toothpaste/infograph_318_000311.jpg\ninfograph/toothpaste/infograph_318_000317.jpg\ninfograph/toothpaste/infograph_318_000318.jpg\ninfograph/toothpaste/infograph_318_000324.jpg\ninfograph/toothpaste/infograph_318_000325.jpg\ninfograph/toothpaste/infograph_318_000326.jpg\ninfograph/toothpaste/infograph_318_000328.jpg\ninfograph/toothpaste/infograph_318_000329.jpg\ninfograph/toothpaste/infograph_318_000338.jpg\ninfograph/toothpaste/infograph_318_000341.jpg\ninfograph/toothpaste/infograph_318_000343.jpg\ninfograph/toothpaste/infograph_318_000346.jpg\ninfograph/toothpaste/infograph_318_000355.jpg\ninfograph/toothpaste/infograph_318_000357.jpg\ninfograph/toothpaste/infograph_318_000358.jpg\ninfograph/toothpaste/infograph_318_000360.jpg\ninfograph/toothpaste/infograph_318_000375.jpg\ninfograph/toothpaste/infograph_318_000381.jpg\ninfograph/toothpaste/infograph_318_000382.jpg\ninfograph/toothpaste/infograph_318_000384.jpg\ninfograph/toothpaste/infograph_318_000386.jpg\ninfograph/toothpaste/infograph_318_000387.jpg\ninfograph/toothpaste/infograph_318_000391.jpg\ninfograph/toothpaste/infograph_318_000395.jpg\ninfograph/toothpaste/infograph_318_000397.jpg\ninfograph/toothpaste/infograph_318_000411.jpg\ninfograph/toothpaste/infograph_318_000421.jpg\ninfograph/toothpaste/infograph_318_000423.jpg\ninfograph/toothpaste/infograph_318_000425.jpg\ninfograph/toothpaste/infograph_318_000431.jpg\ninfograph/toothpaste/infograph_318_000432.jpg\ninfograph/toothpaste/infograph_318_000435.jpg\ninfograph/toothpaste/infograph_318_000437.jpg\ninfograph/toothpaste/infograph_318_000452.jpg\ninfograph/toothpaste/infograph_318_000454.jpg\ninfograph/toothpaste/infograph_318_000455.jpg\ninfograph/toothpaste/infograph_318_000457.jpg\ninfograph/toothpaste/infograph_318_000460.jpg\ninfograph/toothpaste/infograph_318_000466.jpg\ninfograph/toothpaste/infograph_318_000467.jpg\ninfograph/toothpaste/infograph_318_000468.jpg\ninfograph/tornado/infograph_319_000048.jpg\ninfograph/tornado/infograph_319_000083.jpg\ninfograph/tornado/infograph_319_000089.jpg\ninfograph/tornado/infograph_319_000170.jpg\ninfograph/tornado/infograph_319_000260.jpg\ninfograph/tornado/infograph_319_000311.jpg\ninfograph/tractor/infograph_320_000006.jpg\ninfograph/tractor/infograph_320_000046.jpg\ninfograph/tractor/infograph_320_000060.jpg\ninfograph/tractor/infograph_320_000071.jpg\ninfograph/tractor/infograph_320_000085.jpg\ninfograph/tractor/infograph_320_000094.jpg\ninfograph/tractor/infograph_320_000107.jpg\ninfograph/tractor/infograph_320_000160.jpg\ninfograph/tractor/infograph_320_000186.jpg\ninfograph/tractor/infograph_320_000227.jpg\ninfograph/tractor/infograph_320_000266.jpg\ninfograph/tractor/infograph_320_000267.jpg\ninfograph/tractor/infograph_320_000274.jpg\ninfograph/tractor/infograph_320_000291.jpg\ninfograph/traffic_light/infograph_321_000040.jpg\ninfograph/traffic_light/infograph_321_000047.jpg\ninfograph/traffic_light/infograph_321_000088.jpg\ninfograph/traffic_light/infograph_321_000134.jpg\ninfograph/traffic_light/infograph_321_000180.jpg\ninfograph/traffic_light/infograph_321_000247.jpg\ninfograph/train/infograph_322_000016.jpg\ninfograph/train/infograph_322_000033.jpg\ninfograph/train/infograph_322_000098.jpg\ninfograph/train/infograph_322_000104.jpg\ninfograph/train/infograph_322_000123.jpg\ninfograph/train/infograph_322_000147.jpg\ninfograph/tree/infograph_323_000092.jpg\ninfograph/tree/infograph_323_000143.jpg\ninfograph/tree/infograph_323_000148.jpg\ninfograph/tree/infograph_323_000181.jpg\ninfograph/tree/infograph_323_000236.jpg\ninfograph/tree/infograph_323_000262.jpg\ninfograph/tree/infograph_323_000365.jpg\ninfograph/tree/infograph_323_000376.jpg\ninfograph/tree/infograph_323_000415.jpg\ninfograph/tree/infograph_323_000488.jpg\ninfograph/tree/infograph_323_000505.jpg\ninfograph/triangle/infograph_324_000232.jpg\ninfograph/trombone/infograph_325_000011.jpg\ninfograph/trombone/infograph_325_000033.jpg\ninfograph/trombone/infograph_325_000035.jpg\ninfograph/trombone/infograph_325_000113.jpg\ninfograph/trombone/infograph_325_000114.jpg\ninfograph/trombone/infograph_325_000129.jpg\ninfograph/trombone/infograph_325_000136.jpg\ninfograph/trombone/infograph_325_000143.jpg\ninfograph/trombone/infograph_325_000156.jpg\ninfograph/trombone/infograph_325_000157.jpg\ninfograph/trombone/infograph_325_000162.jpg\ninfograph/truck/infograph_326_000008.jpg\ninfograph/truck/infograph_326_000057.jpg\ninfograph/truck/infograph_326_000063.jpg\ninfograph/truck/infograph_326_000077.jpg\ninfograph/truck/infograph_326_000101.jpg\ninfograph/truck/infograph_326_000103.jpg\ninfograph/truck/infograph_326_000149.jpg\ninfograph/truck/infograph_326_000165.jpg\ninfograph/truck/infograph_326_000174.jpg\ninfograph/truck/infograph_326_000177.jpg\ninfograph/truck/infograph_326_000210.jpg\ninfograph/truck/infograph_326_000226.jpg\ninfograph/truck/infograph_326_000243.jpg\ninfograph/truck/infograph_326_000260.jpg\ninfograph/truck/infograph_326_000264.jpg\ninfograph/truck/infograph_326_000283.jpg\ninfograph/truck/infograph_326_000299.jpg\ninfograph/truck/infograph_326_000316.jpg\ninfograph/truck/infograph_326_000324.jpg\ninfograph/truck/infograph_326_000326.jpg\ninfograph/truck/infograph_326_000344.jpg\ninfograph/truck/infograph_326_000391.jpg\ninfograph/truck/infograph_326_000393.jpg\ninfograph/truck/infograph_326_000404.jpg\ninfograph/truck/infograph_326_000489.jpg\ninfograph/truck/infograph_326_000513.jpg\ninfograph/truck/infograph_326_000540.jpg\ninfograph/truck/infograph_326_000547.jpg\ninfograph/truck/infograph_326_000553.jpg\ninfograph/truck/infograph_326_000568.jpg\ninfograph/truck/infograph_326_000571.jpg\ninfograph/truck/infograph_326_000581.jpg\ninfograph/truck/infograph_326_000614.jpg\ninfograph/truck/infograph_326_000648.jpg\ninfograph/truck/infograph_326_000649.jpg\ninfograph/truck/infograph_326_000658.jpg\ninfograph/truck/infograph_326_000671.jpg\ninfograph/trumpet/infograph_327_000006.jpg\ninfograph/trumpet/infograph_327_000009.jpg\ninfograph/trumpet/infograph_327_000017.jpg\ninfograph/trumpet/infograph_327_000026.jpg\ninfograph/trumpet/infograph_327_000030.jpg\ninfograph/trumpet/infograph_327_000035.jpg\ninfograph/trumpet/infograph_327_000036.jpg\ninfograph/trumpet/infograph_327_000048.jpg\ninfograph/trumpet/infograph_327_000070.jpg\ninfograph/trumpet/infograph_327_000073.jpg\ninfograph/trumpet/infograph_327_000074.jpg\ninfograph/trumpet/infograph_327_000078.jpg\ninfograph/trumpet/infograph_327_000100.jpg\ninfograph/trumpet/infograph_327_000107.jpg\ninfograph/trumpet/infograph_327_000113.jpg\ninfograph/trumpet/infograph_327_000120.jpg\ninfograph/trumpet/infograph_327_000134.jpg\ninfograph/trumpet/infograph_327_000146.jpg\ninfograph/trumpet/infograph_327_000148.jpg\ninfograph/trumpet/infograph_327_000157.jpg\ninfograph/trumpet/infograph_327_000177.jpg\ninfograph/trumpet/infograph_327_000178.jpg\ninfograph/trumpet/infograph_327_000198.jpg\ninfograph/trumpet/infograph_327_000201.jpg\ninfograph/trumpet/infograph_327_000214.jpg\ninfograph/trumpet/infograph_327_000220.jpg\ninfograph/trumpet/infograph_327_000245.jpg\ninfograph/t-shirt/infograph_328_000054.jpg\ninfograph/t-shirt/infograph_328_000071.jpg\ninfograph/t-shirt/infograph_328_000074.jpg\ninfograph/t-shirt/infograph_328_000112.jpg\ninfograph/t-shirt/infograph_328_000128.jpg\ninfograph/t-shirt/infograph_328_000138.jpg\ninfograph/t-shirt/infograph_328_000154.jpg\ninfograph/t-shirt/infograph_328_000176.jpg\ninfograph/t-shirt/infograph_328_000246.jpg\ninfograph/umbrella/infograph_329_000041.jpg\ninfograph/umbrella/infograph_329_000064.jpg\ninfograph/umbrella/infograph_329_000114.jpg\ninfograph/umbrella/infograph_329_000123.jpg\ninfograph/umbrella/infograph_329_000187.jpg\ninfograph/umbrella/infograph_329_000191.jpg\ninfograph/umbrella/infograph_329_000225.jpg\ninfograph/umbrella/infograph_329_000229.jpg\ninfograph/umbrella/infograph_329_000235.jpg\ninfograph/umbrella/infograph_329_000305.jpg\ninfograph/umbrella/infograph_329_000316.jpg\ninfograph/umbrella/infograph_329_000320.jpg\ninfograph/umbrella/infograph_329_000334.jpg\ninfograph/umbrella/infograph_329_000341.jpg\ninfograph/umbrella/infograph_329_000409.jpg\ninfograph/umbrella/infograph_329_000442.jpg\ninfograph/umbrella/infograph_329_000449.jpg\ninfograph/umbrella/infograph_329_000478.jpg\ninfograph/underwear/infograph_330_000003.jpg\ninfograph/underwear/infograph_330_000063.jpg\ninfograph/underwear/infograph_330_000075.jpg\ninfograph/underwear/infograph_330_000089.jpg\ninfograph/underwear/infograph_330_000156.jpg\ninfograph/underwear/infograph_330_000158.jpg\ninfograph/underwear/infograph_330_000164.jpg\ninfograph/underwear/infograph_330_000172.jpg\ninfograph/underwear/infograph_330_000182.jpg\ninfograph/underwear/infograph_330_000195.jpg\ninfograph/underwear/infograph_330_000247.jpg\ninfograph/underwear/infograph_330_000292.jpg\ninfograph/underwear/infograph_330_000303.jpg\ninfograph/underwear/infograph_330_000329.jpg\ninfograph/underwear/infograph_330_000345.jpg\ninfograph/van/infograph_331_000046.jpg\ninfograph/van/infograph_331_000097.jpg\ninfograph/van/infograph_331_000102.jpg\ninfograph/van/infograph_331_000127.jpg\ninfograph/van/infograph_331_000144.jpg\ninfograph/van/infograph_331_000176.jpg\ninfograph/van/infograph_331_000179.jpg\ninfograph/van/infograph_331_000200.jpg\ninfograph/van/infograph_331_000202.jpg\ninfograph/van/infograph_331_000208.jpg\ninfograph/van/infograph_331_000234.jpg\ninfograph/van/infograph_331_000262.jpg\ninfograph/van/infograph_331_000297.jpg\ninfograph/van/infograph_331_000325.jpg\ninfograph/van/infograph_331_000335.jpg\ninfograph/van/infograph_331_000341.jpg\ninfograph/van/infograph_331_000349.jpg\ninfograph/van/infograph_331_000398.jpg\ninfograph/van/infograph_331_000428.jpg\ninfograph/van/infograph_331_000436.jpg\ninfograph/van/infograph_331_000464.jpg\ninfograph/van/infograph_331_000549.jpg\ninfograph/van/infograph_331_000550.jpg\ninfograph/van/infograph_331_000572.jpg\ninfograph/van/infograph_331_000575.jpg\ninfograph/van/infograph_331_000666.jpg\ninfograph/van/infograph_331_000702.jpg\ninfograph/van/infograph_331_000720.jpg\ninfograph/van/infograph_331_000749.jpg\ninfograph/van/infograph_331_000759.jpg\ninfograph/van/infograph_331_000762.jpg\ninfograph/van/infograph_331_000763.jpg\ninfograph/van/infograph_331_000773.jpg\ninfograph/van/infograph_331_000779.jpg\ninfograph/van/infograph_331_000790.jpg\ninfograph/van/infograph_331_000799.jpg\ninfograph/vase/infograph_332_000002.jpg\ninfograph/vase/infograph_332_000015.jpg\ninfograph/vase/infograph_332_000029.jpg\ninfograph/vase/infograph_332_000065.jpg\ninfograph/vase/infograph_332_000175.jpg\ninfograph/vase/infograph_332_000176.jpg\ninfograph/vase/infograph_332_000177.jpg\ninfograph/vase/infograph_332_000182.jpg\ninfograph/vase/infograph_332_000236.jpg\ninfograph/vase/infograph_332_000253.jpg\ninfograph/vase/infograph_332_000279.jpg\ninfograph/vase/infograph_332_000296.jpg\ninfograph/vase/infograph_332_000309.jpg\ninfograph/vase/infograph_332_000311.jpg\ninfograph/violin/infograph_333_000001.jpg\ninfograph/violin/infograph_333_000025.jpg\ninfograph/violin/infograph_333_000031.jpg\ninfograph/violin/infograph_333_000060.jpg\ninfograph/violin/infograph_333_000069.jpg\ninfograph/violin/infograph_333_000071.jpg\ninfograph/violin/infograph_333_000081.jpg\ninfograph/violin/infograph_333_000090.jpg\ninfograph/violin/infograph_333_000098.jpg\ninfograph/violin/infograph_333_000101.jpg\ninfograph/violin/infograph_333_000114.jpg\ninfograph/violin/infograph_333_000124.jpg\ninfograph/violin/infograph_333_000130.jpg\ninfograph/violin/infograph_333_000131.jpg\ninfograph/violin/infograph_333_000138.jpg\ninfograph/violin/infograph_333_000156.jpg\ninfograph/violin/infograph_333_000158.jpg\ninfograph/violin/infograph_333_000171.jpg\ninfograph/violin/infograph_333_000177.jpg\ninfograph/violin/infograph_333_000184.jpg\ninfograph/violin/infograph_333_000186.jpg\ninfograph/violin/infograph_333_000192.jpg\ninfograph/violin/infograph_333_000193.jpg\ninfograph/violin/infograph_333_000204.jpg\ninfograph/violin/infograph_333_000227.jpg\ninfograph/violin/infograph_333_000248.jpg\ninfograph/washing_machine/infograph_334_000018.jpg\ninfograph/washing_machine/infograph_334_000043.jpg\ninfograph/washing_machine/infograph_334_000054.jpg\ninfograph/washing_machine/infograph_334_000102.jpg\ninfograph/washing_machine/infograph_334_000176.jpg\ninfograph/washing_machine/infograph_334_000183.jpg\ninfograph/washing_machine/infograph_334_000185.jpg\ninfograph/washing_machine/infograph_334_000222.jpg\ninfograph/washing_machine/infograph_334_000226.jpg\ninfograph/washing_machine/infograph_334_000273.jpg\ninfograph/washing_machine/infograph_334_000305.jpg\ninfograph/washing_machine/infograph_334_000308.jpg\ninfograph/washing_machine/infograph_334_000345.jpg\ninfograph/washing_machine/infograph_334_000347.jpg\ninfograph/washing_machine/infograph_334_000376.jpg\ninfograph/washing_machine/infograph_334_000381.jpg\ninfograph/washing_machine/infograph_334_000393.jpg\ninfograph/washing_machine/infograph_334_000460.jpg\ninfograph/washing_machine/infograph_334_000466.jpg\ninfograph/watermelon/infograph_335_000006.jpg\ninfograph/watermelon/infograph_335_000010.jpg\ninfograph/watermelon/infograph_335_000020.jpg\ninfograph/watermelon/infograph_335_000033.jpg\ninfograph/watermelon/infograph_335_000034.jpg\ninfograph/watermelon/infograph_335_000045.jpg\ninfograph/watermelon/infograph_335_000046.jpg\ninfograph/watermelon/infograph_335_000072.jpg\ninfograph/watermelon/infograph_335_000080.jpg\ninfograph/watermelon/infograph_335_000082.jpg\ninfograph/watermelon/infograph_335_000087.jpg\ninfograph/watermelon/infograph_335_000094.jpg\ninfograph/watermelon/infograph_335_000105.jpg\ninfograph/watermelon/infograph_335_000107.jpg\ninfograph/watermelon/infograph_335_000117.jpg\ninfograph/watermelon/infograph_335_000144.jpg\ninfograph/watermelon/infograph_335_000157.jpg\ninfograph/watermelon/infograph_335_000170.jpg\ninfograph/watermelon/infograph_335_000175.jpg\ninfograph/watermelon/infograph_335_000178.jpg\ninfograph/watermelon/infograph_335_000181.jpg\ninfograph/watermelon/infograph_335_000189.jpg\ninfograph/watermelon/infograph_335_000196.jpg\ninfograph/watermelon/infograph_335_000237.jpg\ninfograph/watermelon/infograph_335_000275.jpg\ninfograph/watermelon/infograph_335_000277.jpg\ninfograph/watermelon/infograph_335_000285.jpg\ninfograph/watermelon/infograph_335_000295.jpg\ninfograph/watermelon/infograph_335_000317.jpg\ninfograph/watermelon/infograph_335_000358.jpg\ninfograph/watermelon/infograph_335_000359.jpg\ninfograph/watermelon/infograph_335_000376.jpg\ninfograph/watermelon/infograph_335_000384.jpg\ninfograph/watermelon/infograph_335_000393.jpg\ninfograph/waterslide/infograph_336_000004.jpg\ninfograph/waterslide/infograph_336_000018.jpg\ninfograph/waterslide/infograph_336_000052.jpg\ninfograph/waterslide/infograph_336_000080.jpg\ninfograph/waterslide/infograph_336_000094.jpg\ninfograph/waterslide/infograph_336_000139.jpg\ninfograph/waterslide/infograph_336_000168.jpg\ninfograph/waterslide/infograph_336_000207.jpg\ninfograph/waterslide/infograph_336_000272.jpg\ninfograph/waterslide/infograph_336_000279.jpg\ninfograph/waterslide/infograph_336_000308.jpg\ninfograph/whale/infograph_337_000003.jpg\ninfograph/whale/infograph_337_000006.jpg\ninfograph/whale/infograph_337_000010.jpg\ninfograph/whale/infograph_337_000011.jpg\ninfograph/whale/infograph_337_000017.jpg\ninfograph/whale/infograph_337_000025.jpg\ninfograph/whale/infograph_337_000038.jpg\ninfograph/whale/infograph_337_000070.jpg\ninfograph/whale/infograph_337_000073.jpg\ninfograph/whale/infograph_337_000080.jpg\ninfograph/whale/infograph_337_000089.jpg\ninfograph/whale/infograph_337_000090.jpg\ninfograph/whale/infograph_337_000096.jpg\ninfograph/whale/infograph_337_000103.jpg\ninfograph/whale/infograph_337_000104.jpg\ninfograph/whale/infograph_337_000107.jpg\ninfograph/whale/infograph_337_000114.jpg\ninfograph/whale/infograph_337_000124.jpg\ninfograph/whale/infograph_337_000128.jpg\ninfograph/whale/infograph_337_000138.jpg\ninfograph/whale/infograph_337_000139.jpg\ninfograph/whale/infograph_337_000146.jpg\ninfograph/whale/infograph_337_000162.jpg\ninfograph/whale/infograph_337_000169.jpg\ninfograph/whale/infograph_337_000183.jpg\ninfograph/whale/infograph_337_000186.jpg\ninfograph/whale/infograph_337_000196.jpg\ninfograph/whale/infograph_337_000198.jpg\ninfograph/whale/infograph_337_000215.jpg\ninfograph/whale/infograph_337_000226.jpg\ninfograph/whale/infograph_337_000230.jpg\ninfograph/whale/infograph_337_000238.jpg\ninfograph/whale/infograph_337_000240.jpg\ninfograph/whale/infograph_337_000247.jpg\ninfograph/whale/infograph_337_000287.jpg\ninfograph/whale/infograph_337_000289.jpg\ninfograph/whale/infograph_337_000294.jpg\ninfograph/whale/infograph_337_000300.jpg\ninfograph/whale/infograph_337_000302.jpg\ninfograph/whale/infograph_337_000304.jpg\ninfograph/whale/infograph_337_000308.jpg\ninfograph/whale/infograph_337_000315.jpg\ninfograph/whale/infograph_337_000317.jpg\ninfograph/whale/infograph_337_000318.jpg\ninfograph/whale/infograph_337_000321.jpg\ninfograph/whale/infograph_337_000325.jpg\ninfograph/whale/infograph_337_000327.jpg\ninfograph/whale/infograph_337_000346.jpg\ninfograph/whale/infograph_337_000347.jpg\ninfograph/whale/infograph_337_000350.jpg\ninfograph/whale/infograph_337_000360.jpg\ninfograph/whale/infograph_337_000376.jpg\ninfograph/whale/infograph_337_000381.jpg\ninfograph/whale/infograph_337_000384.jpg\ninfograph/whale/infograph_337_000388.jpg\ninfograph/whale/infograph_337_000400.jpg\ninfograph/whale/infograph_337_000409.jpg\ninfograph/whale/infograph_337_000410.jpg\ninfograph/whale/infograph_337_000418.jpg\ninfograph/whale/infograph_337_000419.jpg\ninfograph/whale/infograph_337_000426.jpg\ninfograph/wheel/infograph_338_000085.jpg\ninfograph/wheel/infograph_338_000147.jpg\ninfograph/wheel/infograph_338_000153.jpg\ninfograph/wheel/infograph_338_000158.jpg\ninfograph/wheel/infograph_338_000172.jpg\ninfograph/wheel/infograph_338_000239.jpg\ninfograph/wheel/infograph_338_000292.jpg\ninfograph/wheel/infograph_338_000338.jpg\ninfograph/wheel/infograph_338_000363.jpg\ninfograph/wheel/infograph_338_000367.jpg\ninfograph/wheel/infograph_338_000385.jpg\ninfograph/windmill/infograph_339_000001.jpg\ninfograph/windmill/infograph_339_000011.jpg\ninfograph/windmill/infograph_339_000026.jpg\ninfograph/windmill/infograph_339_000092.jpg\ninfograph/windmill/infograph_339_000105.jpg\ninfograph/windmill/infograph_339_000118.jpg\ninfograph/windmill/infograph_339_000123.jpg\ninfograph/windmill/infograph_339_000127.jpg\ninfograph/windmill/infograph_339_000128.jpg\ninfograph/windmill/infograph_339_000154.jpg\ninfograph/windmill/infograph_339_000159.jpg\ninfograph/windmill/infograph_339_000201.jpg\ninfograph/windmill/infograph_339_000217.jpg\ninfograph/windmill/infograph_339_000238.jpg\ninfograph/windmill/infograph_339_000244.jpg\ninfograph/windmill/infograph_339_000246.jpg\ninfograph/windmill/infograph_339_000264.jpg\ninfograph/windmill/infograph_339_000265.jpg\ninfograph/windmill/infograph_339_000283.jpg\ninfograph/windmill/infograph_339_000291.jpg\ninfograph/windmill/infograph_339_000334.jpg\ninfograph/windmill/infograph_339_000350.jpg\ninfograph/wine_bottle/infograph_340_000030.jpg\ninfograph/wine_bottle/infograph_340_000084.jpg\ninfograph/wine_bottle/infograph_340_000090.jpg\ninfograph/wine_bottle/infograph_340_000097.jpg\ninfograph/wine_bottle/infograph_340_000322.jpg\ninfograph/wine_bottle/infograph_340_000415.jpg\ninfograph/wine_bottle/infograph_340_000429.jpg\ninfograph/wine_glass/infograph_341_000009.jpg\ninfograph/wine_glass/infograph_341_000019.jpg\ninfograph/wine_glass/infograph_341_000022.jpg\ninfograph/wine_glass/infograph_341_000044.jpg\ninfograph/wine_glass/infograph_341_000064.jpg\ninfograph/wine_glass/infograph_341_000065.jpg\ninfograph/wine_glass/infograph_341_000075.jpg\ninfograph/wine_glass/infograph_341_000076.jpg\ninfograph/wine_glass/infograph_341_000080.jpg\ninfograph/wine_glass/infograph_341_000085.jpg\ninfograph/wine_glass/infograph_341_000087.jpg\ninfograph/wine_glass/infograph_341_000090.jpg\ninfograph/wine_glass/infograph_341_000101.jpg\ninfograph/wine_glass/infograph_341_000102.jpg\ninfograph/wine_glass/infograph_341_000104.jpg\ninfograph/wine_glass/infograph_341_000113.jpg\ninfograph/wine_glass/infograph_341_000115.jpg\ninfograph/wine_glass/infograph_341_000116.jpg\ninfograph/wine_glass/infograph_341_000123.jpg\ninfograph/wine_glass/infograph_341_000124.jpg\ninfograph/wine_glass/infograph_341_000127.jpg\ninfograph/wine_glass/infograph_341_000132.jpg\ninfograph/wine_glass/infograph_341_000133.jpg\ninfograph/wine_glass/infograph_341_000135.jpg\ninfograph/wine_glass/infograph_341_000141.jpg\ninfograph/wine_glass/infograph_341_000150.jpg\ninfograph/wine_glass/infograph_341_000157.jpg\ninfograph/wine_glass/infograph_341_000175.jpg\ninfograph/wine_glass/infograph_341_000180.jpg\ninfograph/wine_glass/infograph_341_000198.jpg\ninfograph/wine_glass/infograph_341_000207.jpg\ninfograph/wine_glass/infograph_341_000216.jpg\ninfograph/wine_glass/infograph_341_000239.jpg\ninfograph/wine_glass/infograph_341_000245.jpg\ninfograph/wine_glass/infograph_341_000247.jpg\ninfograph/wine_glass/infograph_341_000249.jpg\ninfograph/wine_glass/infograph_341_000257.jpg\ninfograph/wine_glass/infograph_341_000263.jpg\ninfograph/wine_glass/infograph_341_000270.jpg\ninfograph/wine_glass/infograph_341_000272.jpg\ninfograph/wine_glass/infograph_341_000278.jpg\ninfograph/wine_glass/infograph_341_000283.jpg\ninfograph/wine_glass/infograph_341_000284.jpg\ninfograph/wine_glass/infograph_341_000288.jpg\ninfograph/wine_glass/infograph_341_000303.jpg\ninfograph/wine_glass/infograph_341_000314.jpg\ninfograph/wine_glass/infograph_341_000317.jpg\ninfograph/wine_glass/infograph_341_000318.jpg\ninfograph/wine_glass/infograph_341_000321.jpg\ninfograph/wine_glass/infograph_341_000343.jpg\ninfograph/wine_glass/infograph_341_000345.jpg\ninfograph/wine_glass/infograph_341_000368.jpg\ninfograph/wine_glass/infograph_341_000378.jpg\ninfograph/wine_glass/infograph_341_000380.jpg\ninfograph/wine_glass/infograph_341_000394.jpg\ninfograph/wine_glass/infograph_341_000395.jpg\ninfograph/wine_glass/infograph_341_000399.jpg\ninfograph/wine_glass/infograph_341_000402.jpg\ninfograph/wine_glass/infograph_341_000408.jpg\ninfograph/wine_glass/infograph_341_000412.jpg\ninfograph/wine_glass/infograph_341_000423.jpg\ninfograph/wine_glass/infograph_341_000428.jpg\ninfograph/wine_glass/infograph_341_000435.jpg\ninfograph/wine_glass/infograph_341_000461.jpg\ninfograph/wine_glass/infograph_341_000464.jpg\ninfograph/wine_glass/infograph_341_000475.jpg\ninfograph/wine_glass/infograph_341_000490.jpg\ninfograph/wine_glass/infograph_341_000495.jpg\ninfograph/wine_glass/infograph_341_000499.jpg\ninfograph/wine_glass/infograph_341_000507.jpg\ninfograph/wine_glass/infograph_341_000511.jpg\ninfograph/wine_glass/infograph_341_000516.jpg\ninfograph/wine_glass/infograph_341_000518.jpg\ninfograph/wine_glass/infograph_341_000524.jpg\ninfograph/wine_glass/infograph_341_000530.jpg\ninfograph/wine_glass/infograph_341_000531.jpg\ninfograph/wine_glass/infograph_341_000550.jpg\ninfograph/wine_glass/infograph_341_000558.jpg\ninfograph/wine_glass/infograph_341_000579.jpg\ninfograph/wine_glass/infograph_341_000586.jpg\ninfograph/wine_glass/infograph_341_000597.jpg\ninfograph/wine_glass/infograph_341_000606.jpg\ninfograph/wine_glass/infograph_341_000607.jpg\ninfograph/wine_glass/infograph_341_000608.jpg\ninfograph/wine_glass/infograph_341_000613.jpg\ninfograph/wristwatch/infograph_342_000135.jpg\ninfograph/wristwatch/infograph_342_000233.jpg\ninfograph/wristwatch/infograph_342_000300.jpg\ninfograph/wristwatch/infograph_342_000334.jpg\ninfograph/yoga/infograph_343_000035.jpg\ninfograph/yoga/infograph_343_000124.jpg\ninfograph/yoga/infograph_343_000134.jpg\ninfograph/yoga/infograph_343_000163.jpg\ninfograph/yoga/infograph_343_000164.jpg\ninfograph/yoga/infograph_343_000234.jpg\ninfograph/yoga/infograph_343_000263.jpg\ninfograph/yoga/infograph_343_000287.jpg\ninfograph/yoga/infograph_343_000328.jpg\ninfograph/yoga/infograph_343_000422.jpg\ninfograph/zebra/infograph_344_000084.jpg\ninfograph/zebra/infograph_344_000103.jpg\ninfograph/zebra/infograph_344_000127.jpg\ninfograph/zebra/infograph_344_000136.jpg\ninfograph/zebra/infograph_344_000184.jpg\ninfograph/zebra/infograph_344_000188.jpg\ninfograph/zebra/infograph_344_000267.jpg\ninfograph/zebra/infograph_344_000284.jpg\ninfograph/zebra/infograph_344_000292.jpg\ninfograph/zebra/infograph_344_000297.jpg\ninfograph/zigzag/infograph_345_000027.jpg\ninfograph/zigzag/infograph_345_000050.jpg\ninfograph/zigzag/infograph_345_000102.jpg\ninfograph/zigzag/infograph_345_000143.jpg\ninfograph/zigzag/infograph_345_000163.jpg\ninfograph/zigzag/infograph_345_000173.jpg\ninfograph/zigzag/infograph_345_000209.jpg\ninfograph/zigzag/infograph_345_000220.jpg\ninfograph/zigzag/infograph_345_000221.jpg\ninfograph/zigzag/infograph_345_000230.jpg\ninfograph/zigzag/infograph_345_000276.jpg\ninfograph/zigzag/infograph_345_000287.jpg\ninfograph/zigzag/infograph_345_000333.jpg\ninfograph/zigzag/infograph_345_000347.jpg\npainting/airplane/painting_002_000134.jpg\npainting/airplane/painting_002_000169.jpg\npainting/airplane/painting_002_000171.jpg\npainting/airplane/painting_002_000197.jpg\npainting/airplane/painting_002_000203.jpg\npainting/airplane/painting_002_000208.jpg\npainting/animal_migration/painting_006_000103.jpg\npainting/animal_migration/painting_006_000209.jpg\npainting/animal_migration/painting_006_000292.jpg\npainting/animal_migration/painting_006_000340.jpg\npainting/animal_migration/painting_006_000424.jpg\npainting/animal_migration/painting_006_000576.jpg\npainting/animal_migration/painting_006_000579.jpg\npainting/anvil/painting_008_000051.jpg\npainting/asparagus/painting_011_000310.jpg\npainting/asparagus/painting_011_000317.jpg\npainting/asparagus/painting_011_000343.jpg\npainting/banana/painting_014_000064.jpg\npainting/banana/painting_014_000269.jpg\npainting/bandage/painting_015_000024.jpg\npainting/bandage/painting_015_000142.jpg\npainting/bandage/painting_015_000163.jpg\npainting/baseball/painting_017_000069.jpg\npainting/baseball_bat/painting_018_000001.jpg\npainting/baseball_bat/painting_018_000008.jpg\npainting/baseball_bat/painting_018_000064.jpg\npainting/baseball_bat/painting_018_000109.jpg\npainting/baseball_bat/painting_018_000115.jpg\npainting/baseball_bat/painting_018_000134.jpg\npainting/baseball_bat/painting_018_000135.jpg\npainting/baseball_bat/painting_018_000136.jpg\npainting/baseball_bat/painting_018_000143.jpg\npainting/baseball_bat/painting_018_000144.jpg\npainting/basket/painting_019_000112.jpg\npainting/basket/painting_019_000186.jpg\npainting/basket/painting_019_000206.jpg\npainting/basketball/painting_020_000118.jpg\npainting/basketball/painting_020_000140.jpg\npainting/bat/painting_021_000087.jpg\npainting/bat/painting_021_000271.jpg\npainting/bat/painting_021_000277.jpg\npainting/bathtub/painting_022_000036.jpg\npainting/beard/painting_025_000029.jpg\npainting/beard/painting_025_000336.jpg\npainting/bicycle/painting_030_000123.jpg\npainting/bicycle/painting_030_000177.jpg\npainting/bird/painting_032_000033.jpg\npainting/bird/painting_032_000102.jpg\npainting/blueberry/painting_035_000055.jpg\npainting/blueberry/painting_035_000160.jpg\npainting/book/painting_036_000019.jpg\npainting/bottlecap/painting_038_000294.jpg\npainting/bottlecap/painting_038_000309.jpg\npainting/bottlecap/painting_038_000323.jpg\npainting/bottlecap/painting_038_000324.jpg\npainting/bowtie/painting_039_000003.jpg\npainting/bowtie/painting_039_000122.jpg\npainting/bowtie/painting_039_000171.jpg\npainting/bowtie/painting_039_000172.jpg\npainting/bowtie/painting_039_000175.jpg\npainting/bowtie/painting_039_000211.jpg\npainting/bowtie/painting_039_000219.jpg\npainting/bowtie/painting_039_000229.jpg\npainting/bowtie/painting_039_000259.jpg\npainting/bowtie/painting_039_000260.jpg\npainting/bowtie/painting_039_000284.jpg\npainting/bowtie/painting_039_000285.jpg\npainting/bread/painting_042_000288.jpg\npainting/cake/painting_052_000009.jpg\npainting/cake/painting_052_000022.jpg\npainting/cake/painting_052_000039.jpg\npainting/cake/painting_052_000041.jpg\npainting/cake/painting_052_000047.jpg\npainting/cake/painting_052_000051.jpg\npainting/cake/painting_052_000052.jpg\npainting/cake/painting_052_000069.jpg\npainting/cake/painting_052_000072.jpg\npainting/cake/painting_052_000074.jpg\npainting/cake/painting_052_000084.jpg\npainting/cake/painting_052_000095.jpg\npainting/cake/painting_052_000102.jpg\npainting/cake/painting_052_000130.jpg\npainting/cake/painting_052_000132.jpg\npainting/cake/painting_052_000141.jpg\npainting/cake/painting_052_000144.jpg\npainting/cake/painting_052_000148.jpg\npainting/cake/painting_052_000156.jpg\npainting/cake/painting_052_000172.jpg\npainting/candle/painting_059_000031.jpg\npainting/candle/painting_059_000075.jpg\npainting/canoe/painting_061_000060.jpg\npainting/canoe/painting_061_000061.jpg\npainting/canoe/painting_061_000062.jpg\npainting/canoe/painting_061_000066.jpg\npainting/canoe/painting_061_000070.jpg\npainting/canoe/painting_061_000071.jpg\npainting/canoe/painting_061_000075.jpg\npainting/canoe/painting_061_000078.jpg\npainting/canoe/painting_061_000080.jpg\npainting/canoe/painting_061_000082.jpg\npainting/canoe/painting_061_000104.jpg\npainting/canoe/painting_061_000190.jpg\npainting/canoe/painting_061_000239.jpg\npainting/canoe/painting_061_000242.jpg\npainting/canoe/painting_061_000245.jpg\npainting/canoe/painting_061_000247.jpg\npainting/canoe/painting_061_000249.jpg\npainting/canoe/painting_061_000303.jpg\npainting/canoe/painting_061_000304.jpg\npainting/canoe/painting_061_000305.jpg\npainting/canoe/painting_061_000375.jpg\npainting/canoe/painting_061_000378.jpg\npainting/carrot/painting_063_000134.jpg\npainting/cell_phone/painting_068_000053.jpg\npainting/cell_phone/painting_068_000079.jpg\npainting/cell_phone/painting_068_000106.jpg\npainting/chair/painting_069_000048.jpg\npainting/clock/painting_074_000002.jpg\npainting/clock/painting_074_000085.jpg\npainting/clock/painting_074_000150.jpg\npainting/clock/painting_074_000174.jpg\npainting/clock/painting_074_000248.jpg\npainting/clock/painting_074_000261.jpg\npainting/cloud/painting_075_000148.jpg\npainting/cloud/painting_075_000190.jpg\npainting/coffee_cup/painting_076_000003.jpg\npainting/computer/painting_078_000008.jpg\npainting/computer/painting_078_000017.jpg\npainting/crayon/painting_084_000010.jpg\npainting/crown/painting_086_000066.jpg\npainting/cruise_ship/painting_087_000015.jpg\npainting/cruise_ship/painting_087_000142.jpg\npainting/cruise_ship/painting_087_000213.jpg\npainting/cup/painting_088_000001.jpg\npainting/cup/painting_088_000002.jpg\npainting/cup/painting_088_000006.jpg\npainting/cup/painting_088_000013.jpg\npainting/cup/painting_088_000016.jpg\npainting/cup/painting_088_000035.jpg\npainting/cup/painting_088_000036.jpg\npainting/cup/painting_088_000043.jpg\npainting/cup/painting_088_000048.jpg\npainting/cup/painting_088_000049.jpg\npainting/cup/painting_088_000054.jpg\npainting/cup/painting_088_000064.jpg\npainting/cup/painting_088_000077.jpg\npainting/cup/painting_088_000078.jpg\npainting/cup/painting_088_000088.jpg\npainting/cup/painting_088_000092.jpg\npainting/cup/painting_088_000094.jpg\npainting/cup/painting_088_000123.jpg\npainting/cup/painting_088_000128.jpg\npainting/cup/painting_088_000129.jpg\npainting/cup/painting_088_000135.jpg\npainting/cup/painting_088_000137.jpg\npainting/cup/painting_088_000149.jpg\npainting/cup/painting_088_000154.jpg\npainting/cup/painting_088_000156.jpg\npainting/cup/painting_088_000162.jpg\npainting/cup/painting_088_000170.jpg\npainting/cup/painting_088_000190.jpg\npainting/cup/painting_088_000192.jpg\npainting/cup/painting_088_000209.jpg\npainting/cup/painting_088_000210.jpg\npainting/cup/painting_088_000232.jpg\npainting/cup/painting_088_000233.jpg\npainting/cup/painting_088_000258.jpg\npainting/cup/painting_088_000278.jpg\npainting/cup/painting_088_000283.jpg\npainting/cup/painting_088_000293.jpg\npainting/cup/painting_088_000295.jpg\npainting/cup/painting_088_000297.jpg\npainting/cup/painting_088_000307.jpg\npainting/cup/painting_088_000308.jpg\npainting/cup/painting_088_000318.jpg\npainting/cup/painting_088_000333.jpg\npainting/cup/painting_088_000342.jpg\npainting/cup/painting_088_000346.jpg\npainting/cup/painting_088_000358.jpg\npainting/cup/painting_088_000363.jpg\npainting/cup/painting_088_000401.jpg\npainting/cup/painting_088_000413.jpg\npainting/cup/painting_088_000466.jpg\npainting/cup/painting_088_000467.jpg\npainting/cup/painting_088_000475.jpg\npainting/cup/painting_088_000481.jpg\npainting/cup/painting_088_000492.jpg\npainting/cup/painting_088_000500.jpg\npainting/cup/painting_088_000528.jpg\npainting/cup/painting_088_000532.jpg\npainting/cup/painting_088_000549.jpg\npainting/cup/painting_088_000559.jpg\npainting/cup/painting_088_000560.jpg\npainting/cup/painting_088_000561.jpg\npainting/cup/painting_088_000562.jpg\npainting/cup/painting_088_000563.jpg\npainting/cup/painting_088_000576.jpg\npainting/cup/painting_088_000577.jpg\npainting/cup/painting_088_000580.jpg\npainting/diving_board/painting_091_000033.jpg\npainting/diving_board/painting_091_000064.jpg\npainting/diving_board/painting_091_000102.jpg\npainting/diving_board/painting_091_000107.jpg\npainting/dog/painting_092_000179.jpg\npainting/dog/painting_092_000407.jpg\npainting/dog/painting_092_000522.jpg\npainting/drums/painting_099_000030.jpg\npainting/elbow/painting_103_000046.jpg\npainting/elbow/painting_103_000047.jpg\npainting/elbow/painting_103_000053.jpg\npainting/elbow/painting_103_000056.jpg\npainting/elbow/painting_103_000057.jpg\npainting/elbow/painting_103_000060.jpg\npainting/elbow/painting_103_000064.jpg\npainting/elbow/painting_103_000067.jpg\npainting/elbow/painting_103_000072.jpg\npainting/elbow/painting_103_000073.jpg\npainting/elbow/painting_103_000094.jpg\npainting/elbow/painting_103_000096.jpg\npainting/elephant/painting_104_000204.jpg\npainting/eye/painting_107_000049.jpg\npainting/eye/painting_107_000080.jpg\npainting/eye/painting_107_000098.jpg\npainting/eye/painting_107_000101.jpg\npainting/feather/painting_111_000097.jpg\npainting/firetruck/painting_116_000002.jpg\npainting/firetruck/painting_116_000004.jpg\npainting/firetruck/painting_116_000006.jpg\npainting/firetruck/painting_116_000008.jpg\npainting/firetruck/painting_116_000013.jpg\npainting/firetruck/painting_116_000014.jpg\npainting/firetruck/painting_116_000015.jpg\npainting/firetruck/painting_116_000019.jpg\npainting/firetruck/painting_116_000024.jpg\npainting/firetruck/painting_116_000026.jpg\npainting/firetruck/painting_116_000036.jpg\npainting/firetruck/painting_116_000039.jpg\npainting/firetruck/painting_116_000053.jpg\npainting/firetruck/painting_116_000156.jpg\npainting/firetruck/painting_116_000193.jpg\npainting/firetruck/painting_116_000304.jpg\npainting/fish/painting_117_000059.jpg\npainting/fish/painting_117_000060.jpg\npainting/fish/painting_117_000062.jpg\npainting/fish/painting_117_000067.jpg\npainting/fish/painting_117_000077.jpg\npainting/fish/painting_117_000078.jpg\npainting/fish/painting_117_000080.jpg\npainting/fish/painting_117_000082.jpg\npainting/fish/painting_117_000084.jpg\npainting/fish/painting_117_000085.jpg\npainting/fish/painting_117_000140.jpg\npainting/fish/painting_117_000143.jpg\npainting/fish/painting_117_000144.jpg\npainting/fish/painting_117_000145.jpg\npainting/fish/painting_117_000155.jpg\npainting/fish/painting_117_000157.jpg\npainting/fish/painting_117_000159.jpg\npainting/fish/painting_117_000170.jpg\npainting/fish/painting_117_000171.jpg\npainting/fish/painting_117_000176.jpg\npainting/fish/painting_117_000181.jpg\npainting/fish/painting_117_000189.jpg\npainting/fish/painting_117_000222.jpg\npainting/fish/painting_117_000223.jpg\npainting/fish/painting_117_000230.jpg\npainting/fish/painting_117_000235.jpg\npainting/fish/painting_117_000245.jpg\npainting/fish/painting_117_000249.jpg\npainting/fish/painting_117_000261.jpg\npainting/fish/painting_117_000263.jpg\npainting/fish/painting_117_000271.jpg\npainting/fish/painting_117_000283.jpg\npainting/fish/painting_117_000296.jpg\npainting/fish/painting_117_000308.jpg\npainting/fish/painting_117_000310.jpg\npainting/fish/painting_117_000321.jpg\npainting/fish/painting_117_000344.jpg\npainting/fish/painting_117_000345.jpg\npainting/fish/painting_117_000348.jpg\npainting/fish/painting_117_000352.jpg\npainting/fish/painting_117_000359.jpg\npainting/fish/painting_117_000360.jpg\npainting/fish/painting_117_000361.jpg\npainting/fish/painting_117_000369.jpg\npainting/fish/painting_117_000370.jpg\npainting/fish/painting_117_000377.jpg\npainting/fish/painting_117_000379.jpg\npainting/fish/painting_117_000380.jpg\npainting/fish/painting_117_000424.jpg\npainting/flamingo/painting_118_000057.jpg\npainting/flamingo/painting_118_000060.jpg\npainting/foot/painting_124_000045.jpg\npainting/garden/painting_128_000054.jpg\npainting/garden/painting_128_000174.jpg\npainting/goatee/painting_131_000004.jpg\npainting/goatee/painting_131_000019.jpg\npainting/goatee/painting_131_000024.jpg\npainting/goatee/painting_131_000027.jpg\npainting/goatee/painting_131_000030.jpg\npainting/goatee/painting_131_000036.jpg\npainting/goatee/painting_131_000046.jpg\npainting/goatee/painting_131_000052.jpg\npainting/goatee/painting_131_000053.jpg\npainting/goatee/painting_131_000061.jpg\npainting/goatee/painting_131_000074.jpg\npainting/goatee/painting_131_000076.jpg\npainting/goatee/painting_131_000080.jpg\npainting/goatee/painting_131_000086.jpg\npainting/goatee/painting_131_000088.jpg\npainting/goatee/painting_131_000091.jpg\npainting/goatee/painting_131_000096.jpg\npainting/goatee/painting_131_000111.jpg\npainting/goatee/painting_131_000114.jpg\npainting/goatee/painting_131_000122.jpg\npainting/golf_club/painting_132_000115.jpg\npainting/golf_club/painting_132_000123.jpg\npainting/golf_club/painting_132_000164.jpg\npainting/golf_club/painting_132_000210.jpg\npainting/golf_club/painting_132_000259.jpg\npainting/golf_club/painting_132_000317.jpg\npainting/golf_club/painting_132_000363.jpg\npainting/golf_club/painting_132_000417.jpg\npainting/golf_club/painting_132_000426.jpg\npainting/golf_club/painting_132_000434.jpg\npainting/golf_club/painting_132_000435.jpg\npainting/golf_club/painting_132_000437.jpg\npainting/golf_club/painting_132_000462.jpg\npainting/golf_club/painting_132_000556.jpg\npainting/grapes/painting_133_000048.jpg\npainting/grapes/painting_133_000107.jpg\npainting/grapes/painting_133_000109.jpg\npainting/grapes/painting_133_000114.jpg\npainting/grapes/painting_133_000279.jpg\npainting/grapes/painting_133_000312.jpg\npainting/grass/painting_134_000030.jpg\npainting/grass/painting_134_000086.jpg\npainting/grass/painting_134_000208.jpg\npainting/grass/painting_134_000256.jpg\npainting/grass/painting_134_000279.jpg\npainting/guitar/painting_135_000007.jpg\npainting/hamburger/painting_136_000110.jpg\npainting/hamburger/painting_136_000124.jpg\npainting/hammer/painting_137_000028.jpg\npainting/hand/painting_138_000007.jpg\npainting/hand/painting_138_000008.jpg\npainting/hand/painting_138_000034.jpg\npainting/hand/painting_138_000057.jpg\npainting/hand/painting_138_000102.jpg\npainting/hand/painting_138_000195.jpg\npainting/hat/painting_140_000036.jpg\npainting/hat/painting_140_000053.jpg\npainting/hat/painting_140_000070.jpg\npainting/hat/painting_140_000151.jpg\npainting/hat/painting_140_000198.jpg\npainting/hat/painting_140_000365.jpg\npainting/hedgehog/painting_142_000069.jpg\npainting/hockey_puck/painting_146_000002.jpg\npainting/hockey_puck/painting_146_000024.jpg\npainting/hockey_puck/painting_146_000068.jpg\npainting/hockey_puck/painting_146_000069.jpg\npainting/hockey_puck/painting_146_000092.jpg\npainting/hockey_puck/painting_146_000097.jpg\npainting/hockey_puck/painting_146_000098.jpg\npainting/hockey_puck/painting_146_000101.jpg\npainting/hockey_puck/painting_146_000117.jpg\npainting/hockey_puck/painting_146_000183.jpg\npainting/hockey_puck/painting_146_000193.jpg\npainting/hockey_puck/painting_146_000208.jpg\npainting/hockey_puck/painting_146_000220.jpg\npainting/hockey_stick/painting_147_000001.jpg\npainting/hockey_stick/painting_147_000005.jpg\npainting/hockey_stick/painting_147_000031.jpg\npainting/hockey_stick/painting_147_000057.jpg\npainting/hockey_stick/painting_147_000075.jpg\npainting/hockey_stick/painting_147_000146.jpg\npainting/hot_dog/painting_151_000128.jpg\npainting/hot_tub/painting_152_000051.jpg\npainting/hot_tub/painting_152_000054.jpg\npainting/hourglass/painting_153_000021.jpg\npainting/hourglass/painting_153_000079.jpg\npainting/hourglass/painting_153_000107.jpg\npainting/hourglass/painting_153_000110.jpg\npainting/hourglass/painting_153_000139.jpg\npainting/hourglass/painting_153_000169.jpg\npainting/house_plant/painting_155_000057.jpg\npainting/house_plant/painting_155_000168.jpg\npainting/house_plant/painting_155_000170.jpg\npainting/house_plant/painting_155_000171.jpg\npainting/house_plant/painting_155_000318.jpg\npainting/kangaroo/painting_160_000057.jpg\npainting/kangaroo/painting_160_000110.jpg\npainting/kangaroo/painting_160_000142.jpg\npainting/key/painting_161_000083.jpg\npainting/knee/painting_163_000059.jpg\npainting/knee/painting_163_000066.jpg\npainting/knee/painting_163_000121.jpg\npainting/ladder/painting_165_000102.jpg\npainting/ladder/painting_165_000306.jpg\npainting/laptop/painting_167_000005.jpg\npainting/laptop/painting_167_000019.jpg\npainting/laptop/painting_167_000047.jpg\npainting/laptop/painting_167_000071.jpg\npainting/laptop/painting_167_000080.jpg\npainting/laptop/painting_167_000149.jpg\npainting/leaf/painting_168_000029.jpg\npainting/leaf/painting_168_000198.jpg\npainting/light_bulb/painting_170_000335.jpg\npainting/lighthouse/painting_172_000289.jpg\npainting/lighthouse/painting_172_000336.jpg\npainting/lighthouse/painting_172_000341.jpg\npainting/lighthouse/painting_172_000358.jpg\npainting/lighthouse/painting_172_000360.jpg\npainting/lighthouse/painting_172_000361.jpg\npainting/lighthouse/painting_172_000392.jpg\npainting/lightning/painting_173_000066.jpg\npainting/lightning/painting_173_000144.jpg\npainting/line/painting_174_000266.jpg\npainting/line/painting_174_000329.jpg\npainting/lion/painting_175_000431.jpg\npainting/lobster/painting_177_000092.jpg\npainting/lobster/painting_177_000179.jpg\npainting/lollipop/painting_178_000026.jpg\npainting/lollipop/painting_178_000095.jpg\npainting/lollipop/painting_178_000143.jpg\npainting/lollipop/painting_178_000220.jpg\npainting/megaphone/painting_183_000015.jpg\npainting/megaphone/painting_183_000017.jpg\npainting/megaphone/painting_183_000078.jpg\npainting/megaphone/painting_183_000097.jpg\npainting/megaphone/painting_183_000154.jpg\npainting/megaphone/painting_183_000157.jpg\npainting/microphone/painting_185_000045.jpg\npainting/microphone/painting_185_000052.jpg\npainting/microphone/painting_185_000088.jpg\npainting/microphone/painting_185_000111.jpg\npainting/microphone/painting_185_000143.jpg\npainting/monkey/painting_187_000036.jpg\npainting/monkey/painting_187_000099.jpg\npainting/monkey/painting_187_000152.jpg\npainting/monkey/painting_187_000181.jpg\npainting/monkey/painting_187_000400.jpg\npainting/moon/painting_188_000037.jpg\npainting/moon/painting_188_000212.jpg\npainting/mouse/painting_192_000007.jpg\npainting/mouse/painting_192_000428.jpg\npainting/mouse/painting_192_000434.jpg\npainting/mouse/painting_192_000442.jpg\npainting/mouse/painting_192_000444.jpg\npainting/mouse/painting_192_000445.jpg\npainting/moustache/painting_193_000025.jpg\npainting/moustache/painting_193_000035.jpg\npainting/moustache/painting_193_000074.jpg\npainting/moustache/painting_193_000084.jpg\npainting/moustache/painting_193_000085.jpg\npainting/moustache/painting_193_000098.jpg\npainting/moustache/painting_193_000101.jpg\npainting/moustache/painting_193_000143.jpg\npainting/moustache/painting_193_000149.jpg\npainting/moustache/painting_193_000177.jpg\npainting/moustache/painting_193_000186.jpg\npainting/moustache/painting_193_000217.jpg\npainting/moustache/painting_193_000230.jpg\npainting/moustache/painting_193_000279.jpg\npainting/moustache/painting_193_000332.jpg\npainting/moustache/painting_193_000345.jpg\npainting/moustache/painting_193_000371.jpg\npainting/moustache/painting_193_000424.jpg\npainting/mouth/painting_194_000041.jpg\npainting/mug/painting_195_000003.jpg\npainting/mug/painting_195_000006.jpg\npainting/mug/painting_195_000009.jpg\npainting/mug/painting_195_000016.jpg\npainting/mug/painting_195_000018.jpg\npainting/mug/painting_195_000019.jpg\npainting/mug/painting_195_000022.jpg\npainting/mug/painting_195_000023.jpg\npainting/mug/painting_195_000025.jpg\npainting/mug/painting_195_000033.jpg\npainting/mug/painting_195_000034.jpg\npainting/mug/painting_195_000037.jpg\npainting/mug/painting_195_000038.jpg\npainting/mug/painting_195_000041.jpg\npainting/mug/painting_195_000043.jpg\npainting/mug/painting_195_000047.jpg\npainting/mug/painting_195_000049.jpg\npainting/mug/painting_195_000051.jpg\npainting/mug/painting_195_000060.jpg\npainting/mug/painting_195_000062.jpg\npainting/mug/painting_195_000063.jpg\npainting/mug/painting_195_000064.jpg\npainting/mug/painting_195_000070.jpg\npainting/mug/painting_195_000073.jpg\npainting/mug/painting_195_000075.jpg\npainting/mug/painting_195_000078.jpg\npainting/mug/painting_195_000080.jpg\npainting/mug/painting_195_000082.jpg\npainting/mug/painting_195_000083.jpg\npainting/mug/painting_195_000086.jpg\npainting/mug/painting_195_000091.jpg\npainting/mug/painting_195_000093.jpg\npainting/mug/painting_195_000094.jpg\npainting/mug/painting_195_000096.jpg\npainting/mug/painting_195_000097.jpg\npainting/mug/painting_195_000098.jpg\npainting/mug/painting_195_000101.jpg\npainting/mug/painting_195_000102.jpg\npainting/mug/painting_195_000105.jpg\npainting/mug/painting_195_000106.jpg\npainting/mug/painting_195_000108.jpg\npainting/mug/painting_195_000112.jpg\npainting/mug/painting_195_000113.jpg\npainting/mug/painting_195_000114.jpg\npainting/mug/painting_195_000116.jpg\npainting/mug/painting_195_000121.jpg\npainting/mug/painting_195_000125.jpg\npainting/mug/painting_195_000127.jpg\npainting/mug/painting_195_000137.jpg\npainting/mug/painting_195_000140.jpg\npainting/mug/painting_195_000144.jpg\npainting/mug/painting_195_000146.jpg\npainting/mug/painting_195_000148.jpg\npainting/mug/painting_195_000149.jpg\npainting/mug/painting_195_000151.jpg\npainting/mug/painting_195_000153.jpg\npainting/mug/painting_195_000154.jpg\npainting/mug/painting_195_000155.jpg\npainting/mug/painting_195_000157.jpg\npainting/mug/painting_195_000162.jpg\npainting/mug/painting_195_000170.jpg\npainting/mug/painting_195_000171.jpg\npainting/mug/painting_195_000173.jpg\npainting/mug/painting_195_000174.jpg\npainting/mug/painting_195_000175.jpg\npainting/mug/painting_195_000176.jpg\npainting/mug/painting_195_000183.jpg\npainting/mug/painting_195_000186.jpg\npainting/mug/painting_195_000188.jpg\npainting/mug/painting_195_000189.jpg\npainting/mug/painting_195_000200.jpg\npainting/mug/painting_195_000202.jpg\npainting/mug/painting_195_000203.jpg\npainting/mug/painting_195_000207.jpg\npainting/mug/painting_195_000212.jpg\npainting/mug/painting_195_000218.jpg\npainting/mug/painting_195_000220.jpg\npainting/mug/painting_195_000233.jpg\npainting/mug/painting_195_000236.jpg\npainting/mug/painting_195_000238.jpg\npainting/mug/painting_195_000241.jpg\npainting/mug/painting_195_000256.jpg\npainting/mug/painting_195_000258.jpg\npainting/mug/painting_195_000260.jpg\npainting/mug/painting_195_000264.jpg\npainting/mug/painting_195_000269.jpg\npainting/mug/painting_195_000281.jpg\npainting/mug/painting_195_000305.jpg\npainting/mug/painting_195_000308.jpg\npainting/mug/painting_195_000309.jpg\npainting/mug/painting_195_000310.jpg\npainting/mug/painting_195_000317.jpg\npainting/mug/painting_195_000322.jpg\npainting/mug/painting_195_000339.jpg\npainting/mug/painting_195_000345.jpg\npainting/mug/painting_195_000348.jpg\npainting/mug/painting_195_000351.jpg\npainting/mug/painting_195_000358.jpg\npainting/mug/painting_195_000361.jpg\npainting/mug/painting_195_000365.jpg\npainting/mug/painting_195_000371.jpg\npainting/mug/painting_195_000377.jpg\npainting/mug/painting_195_000387.jpg\npainting/mug/painting_195_000392.jpg\npainting/mug/painting_195_000398.jpg\npainting/mug/painting_195_000401.jpg\npainting/mug/painting_195_000403.jpg\npainting/mug/painting_195_000408.jpg\npainting/mug/painting_195_000413.jpg\npainting/mug/painting_195_000438.jpg\npainting/mug/painting_195_000449.jpg\npainting/mug/painting_195_000453.jpg\npainting/mug/painting_195_000464.jpg\npainting/mug/painting_195_000467.jpg\npainting/mug/painting_195_000476.jpg\npainting/mug/painting_195_000477.jpg\npainting/mug/painting_195_000481.jpg\npainting/mug/painting_195_000483.jpg\npainting/mug/painting_195_000484.jpg\npainting/mug/painting_195_000485.jpg\npainting/mug/painting_195_000486.jpg\npainting/mug/painting_195_000488.jpg\npainting/mug/painting_195_000489.jpg\npainting/mug/painting_195_000491.jpg\npainting/mug/painting_195_000495.jpg\npainting/mug/painting_195_000497.jpg\npainting/mushroom/painting_196_000099.jpg\npainting/necklace/painting_198_000060.jpg\npainting/necklace/painting_198_000092.jpg\npainting/nose/painting_199_000314.jpg\npainting/nose/painting_199_000376.jpg\npainting/nose/painting_199_000439.jpg\npainting/nose/painting_199_000444.jpg\npainting/ocean/painting_200_000012.jpg\npainting/ocean/painting_200_000090.jpg\npainting/ocean/painting_200_000151.jpg\npainting/ocean/painting_200_000282.jpg\npainting/ocean/painting_200_000293.jpg\npainting/ocean/painting_200_000314.jpg\npainting/octopus/painting_202_000011.jpg\npainting/octopus/painting_202_000148.jpg\npainting/octopus/painting_202_000227.jpg\npainting/owl/painting_205_000285.jpg\npainting/paintbrush/painting_206_000101.jpg\npainting/paintbrush/painting_206_000135.jpg\npainting/paintbrush/painting_206_000170.jpg\npainting/paintbrush/painting_206_000172.jpg\npainting/paintbrush/painting_206_000200.jpg\npainting/paintbrush/painting_206_000284.jpg\npainting/paintbrush/painting_206_000295.jpg\npainting/paintbrush/painting_206_000318.jpg\npainting/paint_can/painting_207_000026.jpg\npainting/paint_can/painting_207_000037.jpg\npainting/paint_can/painting_207_000055.jpg\npainting/paint_can/painting_207_000111.jpg\npainting/paint_can/painting_207_000158.jpg\npainting/paint_can/painting_207_000170.jpg\npainting/paint_can/painting_207_000171.jpg\npainting/palm_tree/painting_208_000002.jpg\npainting/palm_tree/painting_208_000020.jpg\npainting/palm_tree/painting_208_000041.jpg\npainting/palm_tree/painting_208_000058.jpg\npainting/palm_tree/painting_208_000119.jpg\npainting/palm_tree/painting_208_000160.jpg\npainting/palm_tree/painting_208_000164.jpg\npainting/palm_tree/painting_208_000186.jpg\npainting/palm_tree/painting_208_000190.jpg\npainting/palm_tree/painting_208_000213.jpg\npainting/palm_tree/painting_208_000323.jpg\npainting/palm_tree/painting_208_000385.jpg\npainting/palm_tree/painting_208_000404.jpg\npainting/palm_tree/painting_208_000447.jpg\npainting/palm_tree/painting_208_000454.jpg\npainting/palm_tree/painting_208_000490.jpg\npainting/palm_tree/painting_208_000501.jpg\npainting/palm_tree/painting_208_000526.jpg\npainting/palm_tree/painting_208_000543.jpg\npainting/palm_tree/painting_208_000559.jpg\npainting/palm_tree/painting_208_000560.jpg\npainting/palm_tree/painting_208_000572.jpg\npainting/panda/painting_209_000253.jpg\npainting/pants/painting_210_000196.jpg\npainting/pants/painting_210_000373.jpg\npainting/parachute/painting_212_000012.jpg\npainting/parachute/painting_212_000025.jpg\npainting/parachute/painting_212_000038.jpg\npainting/parachute/painting_212_000074.jpg\npainting/parachute/painting_212_000089.jpg\npainting/parachute/painting_212_000116.jpg\npainting/parachute/painting_212_000119.jpg\npainting/parrot/painting_213_000111.jpg\npainting/parrot/painting_213_000254.jpg\npainting/passport/painting_214_000019.jpg\npainting/pear/painting_216_000076.jpg\npainting/pear/painting_216_000113.jpg\npainting/peas/painting_217_000090.jpg\npainting/pencil/painting_218_000018.jpg\npainting/pencil/painting_218_000023.jpg\npainting/pencil/painting_218_000030.jpg\npainting/pencil/painting_218_000055.jpg\npainting/penguin/painting_219_000151.jpg\npainting/penguin/painting_219_000238.jpg\npainting/penguin/painting_219_000425.jpg\npainting/penguin/painting_219_000440.jpg\npainting/piano/painting_220_000039.jpg\npainting/piano/painting_220_000075.jpg\npainting/piano/painting_220_000085.jpg\npainting/piano/painting_220_000091.jpg\npainting/piano/painting_220_000096.jpg\npainting/piano/painting_220_000150.jpg\npainting/piano/painting_220_000167.jpg\npainting/piano/painting_220_000192.jpg\npainting/piano/painting_220_000216.jpg\npainting/piano/painting_220_000249.jpg\npainting/piano/painting_220_000267.jpg\npainting/piano/painting_220_000282.jpg\npainting/pickup_truck/painting_221_000135.jpg\npainting/picture_frame/painting_222_000011.jpg\npainting/picture_frame/painting_222_000019.jpg\npainting/picture_frame/painting_222_000156.jpg\npainting/pig/painting_223_000163.jpg\npainting/pillow/painting_224_000035.jpg\npainting/pillow/painting_224_000053.jpg\npainting/pillow/painting_224_000071.jpg\npainting/pillow/painting_224_000090.jpg\npainting/pillow/painting_224_000101.jpg\npainting/pillow/painting_224_000105.jpg\npainting/pillow/painting_224_000120.jpg\npainting/pliers/painting_227_000003.jpg\npainting/pliers/painting_227_000027.jpg\npainting/pliers/painting_227_000179.jpg\npainting/pliers/painting_227_000277.jpg\npainting/pond/painting_229_000028.jpg\npainting/pond/painting_229_000036.jpg\npainting/pond/painting_229_000271.jpg\npainting/pond/painting_229_000274.jpg\npainting/pond/painting_229_000377.jpg\npainting/pond/painting_229_000456.jpg\npainting/pond/painting_229_000458.jpg\npainting/pond/painting_229_000468.jpg\npainting/pond/painting_229_000543.jpg\npainting/pond/painting_229_000597.jpg\npainting/popsicle/painting_231_000024.jpg\npainting/postcard/painting_232_000050.jpg\npainting/power_outlet/painting_234_000017.jpg\npainting/rabbit/painting_236_000121.jpg\npainting/raccoon/painting_237_000154.jpg\npainting/rain/painting_239_000081.jpg\npainting/remote_control/painting_242_000103.jpg\npainting/rhinoceros/painting_243_000031.jpg\npainting/river/painting_245_000089.jpg\npainting/river/painting_245_000187.jpg\npainting/river/painting_245_000259.jpg\npainting/river/painting_245_000460.jpg\npainting/rollerskates/painting_247_000037.jpg\npainting/rollerskates/painting_247_000052.jpg\npainting/rollerskates/painting_247_000059.jpg\npainting/rollerskates/painting_247_000064.jpg\npainting/rollerskates/painting_247_000096.jpg\npainting/rollerskates/painting_247_000126.jpg\npainting/rollerskates/painting_247_000127.jpg\npainting/rollerskates/painting_247_000128.jpg\npainting/rollerskates/painting_247_000132.jpg\npainting/rollerskates/painting_247_000158.jpg\npainting/rollerskates/painting_247_000160.jpg\npainting/rollerskates/painting_247_000216.jpg\npainting/rollerskates/painting_247_000218.jpg\npainting/rollerskates/painting_247_000257.jpg\npainting/rollerskates/painting_247_000268.jpg\npainting/rollerskates/painting_247_000273.jpg\npainting/rollerskates/painting_247_000275.jpg\npainting/rollerskates/painting_247_000281.jpg\npainting/rollerskates/painting_247_000284.jpg\npainting/sailboat/painting_248_000144.jpg\npainting/sandwich/painting_249_000019.jpg\npainting/sandwich/painting_249_000027.jpg\npainting/sandwich/painting_249_000041.jpg\npainting/sandwich/painting_249_000062.jpg\npainting/sandwich/painting_249_000063.jpg\npainting/sandwich/painting_249_000093.jpg\npainting/sandwich/painting_249_000096.jpg\npainting/sandwich/painting_249_000105.jpg\npainting/sandwich/painting_249_000125.jpg\npainting/sandwich/painting_249_000136.jpg\npainting/saw/painting_250_000116.jpg\npainting/saxophone/painting_251_000148.jpg\npainting/saxophone/painting_251_000290.jpg\npainting/saxophone/painting_251_000331.jpg\npainting/school_bus/painting_252_000001.jpg\npainting/school_bus/painting_252_000009.jpg\npainting/school_bus/painting_252_000016.jpg\npainting/school_bus/painting_252_000024.jpg\npainting/school_bus/painting_252_000048.jpg\npainting/school_bus/painting_252_000059.jpg\npainting/scissors/painting_253_000011.jpg\npainting/scissors/painting_253_000014.jpg\npainting/scissors/painting_253_000024.jpg\npainting/scorpion/painting_254_000023.jpg\npainting/scorpion/painting_254_000033.jpg\npainting/scorpion/painting_254_000059.jpg\npainting/scorpion/painting_254_000090.jpg\npainting/scorpion/painting_254_000112.jpg\npainting/screwdriver/painting_255_000009.jpg\npainting/screwdriver/painting_255_000030.jpg\npainting/screwdriver/painting_255_000044.jpg\npainting/screwdriver/painting_255_000046.jpg\npainting/screwdriver/painting_255_000058.jpg\npainting/screwdriver/painting_255_000067.jpg\npainting/sea_turtle/painting_256_000043.jpg\npainting/sea_turtle/painting_256_000138.jpg\npainting/sea_turtle/painting_256_000257.jpg\npainting/sea_turtle/painting_256_000285.jpg\npainting/sea_turtle/painting_256_000289.jpg\npainting/sea_turtle/painting_256_000369.jpg\npainting/sea_turtle/painting_256_000373.jpg\npainting/sea_turtle/painting_256_000400.jpg\npainting/see_saw/painting_257_000039.jpg\npainting/see_saw/painting_257_000041.jpg\npainting/see_saw/painting_257_000065.jpg\npainting/see_saw/painting_257_000164.jpg\npainting/sheep/painting_259_000092.jpg\npainting/shorts/painting_261_000077.jpg\npainting/shorts/painting_261_000078.jpg\npainting/shovel/painting_262_000043.jpg\npainting/shovel/painting_262_000101.jpg\npainting/skateboard/painting_264_000014.jpg\npainting/skateboard/painting_264_000148.jpg\npainting/skull/painting_265_000143.jpg\npainting/skull/painting_265_000182.jpg\npainting/skull/painting_265_000183.jpg\npainting/skyscraper/painting_266_000045.jpg\npainting/skyscraper/painting_266_000152.jpg\npainting/snail/painting_269_000010.jpg\npainting/snail/painting_269_000181.jpg\npainting/snake/painting_270_000139.jpg\npainting/snake/painting_270_000199.jpg\npainting/snorkel/painting_271_000030.jpg\npainting/snorkel/painting_271_000031.jpg\npainting/snorkel/painting_271_000048.jpg\npainting/snorkel/painting_271_000049.jpg\npainting/snorkel/painting_271_000057.jpg\npainting/snorkel/painting_271_000063.jpg\npainting/snorkel/painting_271_000065.jpg\npainting/snorkel/painting_271_000072.jpg\npainting/snorkel/painting_271_000074.jpg\npainting/snorkel/painting_271_000075.jpg\npainting/snorkel/painting_271_000076.jpg\npainting/snorkel/painting_271_000084.jpg\npainting/snorkel/painting_271_000085.jpg\npainting/snorkel/painting_271_000088.jpg\npainting/snorkel/painting_271_000089.jpg\npainting/snorkel/painting_271_000090.jpg\npainting/snorkel/painting_271_000091.jpg\npainting/snorkel/painting_271_000092.jpg\npainting/snorkel/painting_271_000093.jpg\npainting/snorkel/painting_271_000097.jpg\npainting/snorkel/painting_271_000098.jpg\npainting/snorkel/painting_271_000101.jpg\npainting/snorkel/painting_271_000104.jpg\npainting/snorkel/painting_271_000105.jpg\npainting/snorkel/painting_271_000108.jpg\npainting/snorkel/painting_271_000109.jpg\npainting/snorkel/painting_271_000111.jpg\npainting/snorkel/painting_271_000116.jpg\npainting/snorkel/painting_271_000117.jpg\npainting/snorkel/painting_271_000118.jpg\npainting/snorkel/painting_271_000120.jpg\npainting/snorkel/painting_271_000128.jpg\npainting/snorkel/painting_271_000130.jpg\npainting/snorkel/painting_271_000132.jpg\npainting/snorkel/painting_271_000133.jpg\npainting/snorkel/painting_271_000136.jpg\npainting/snorkel/painting_271_000137.jpg\npainting/snorkel/painting_271_000142.jpg\npainting/snorkel/painting_271_000144.jpg\npainting/snorkel/painting_271_000145.jpg\npainting/snorkel/painting_271_000147.jpg\npainting/snorkel/painting_271_000148.jpg\npainting/snorkel/painting_271_000150.jpg\npainting/snorkel/painting_271_000151.jpg\npainting/snorkel/painting_271_000152.jpg\npainting/snorkel/painting_271_000153.jpg\npainting/snorkel/painting_271_000154.jpg\npainting/snorkel/painting_271_000156.jpg\npainting/snorkel/painting_271_000157.jpg\npainting/snorkel/painting_271_000158.jpg\npainting/snorkel/painting_271_000161.jpg\npainting/snorkel/painting_271_000162.jpg\npainting/snorkel/painting_271_000163.jpg\npainting/snorkel/painting_271_000164.jpg\npainting/snorkel/painting_271_000165.jpg\npainting/snorkel/painting_271_000166.jpg\npainting/snorkel/painting_271_000168.jpg\npainting/snorkel/painting_271_000170.jpg\npainting/snorkel/painting_271_000171.jpg\npainting/snorkel/painting_271_000175.jpg\npainting/snorkel/painting_271_000176.jpg\npainting/snorkel/painting_271_000177.jpg\npainting/snorkel/painting_271_000178.jpg\npainting/snorkel/painting_271_000179.jpg\npainting/snowflake/painting_272_000033.jpg\npainting/snowflake/painting_272_000046.jpg\npainting/snowflake/painting_272_000047.jpg\npainting/snowflake/painting_272_000054.jpg\npainting/snowflake/painting_272_000057.jpg\npainting/snowflake/painting_272_000059.jpg\npainting/snowflake/painting_272_000067.jpg\npainting/snowflake/painting_272_000084.jpg\npainting/snowflake/painting_272_000090.jpg\npainting/snowflake/painting_272_000100.jpg\npainting/snowflake/painting_272_000104.jpg\npainting/snowflake/painting_272_000109.jpg\npainting/snowflake/painting_272_000114.jpg\npainting/snowflake/painting_272_000125.jpg\npainting/snowflake/painting_272_000131.jpg\npainting/snowflake/painting_272_000144.jpg\npainting/snowflake/painting_272_000145.jpg\npainting/snowflake/painting_272_000147.jpg\npainting/snowflake/painting_272_000152.jpg\npainting/snowflake/painting_272_000154.jpg\npainting/snowflake/painting_272_000155.jpg\npainting/snowflake/painting_272_000161.jpg\npainting/snowflake/painting_272_000162.jpg\npainting/snowflake/painting_272_000169.jpg\npainting/snowflake/painting_272_000172.jpg\npainting/snowflake/painting_272_000173.jpg\npainting/snowflake/painting_272_000179.jpg\npainting/snowflake/painting_272_000180.jpg\npainting/snowflake/painting_272_000181.jpg\npainting/snowflake/painting_272_000182.jpg\npainting/snowflake/painting_272_000184.jpg\npainting/snowflake/painting_272_000190.jpg\npainting/snowflake/painting_272_000197.jpg\npainting/snowflake/painting_272_000199.jpg\npainting/snowflake/painting_272_000200.jpg\npainting/snowflake/painting_272_000204.jpg\npainting/snowflake/painting_272_000206.jpg\npainting/snowflake/painting_272_000208.jpg\npainting/snowflake/painting_272_000212.jpg\npainting/snowflake/painting_272_000222.jpg\npainting/snowflake/painting_272_000224.jpg\npainting/snowflake/painting_272_000227.jpg\npainting/snowflake/painting_272_000231.jpg\npainting/snowflake/painting_272_000236.jpg\npainting/snowflake/painting_272_000239.jpg\npainting/snowflake/painting_272_000241.jpg\npainting/snowflake/painting_272_000242.jpg\npainting/snowflake/painting_272_000244.jpg\npainting/snowflake/painting_272_000249.jpg\npainting/snowflake/painting_272_000253.jpg\npainting/snowflake/painting_272_000256.jpg\npainting/snowflake/painting_272_000257.jpg\npainting/snowflake/painting_272_000258.jpg\npainting/snowflake/painting_272_000264.jpg\npainting/snowflake/painting_272_000265.jpg\npainting/snowflake/painting_272_000267.jpg\npainting/snowflake/painting_272_000268.jpg\npainting/snowflake/painting_272_000271.jpg\npainting/snowflake/painting_272_000272.jpg\npainting/snowflake/painting_272_000273.jpg\npainting/snowflake/painting_272_000281.jpg\npainting/snowflake/painting_272_000285.jpg\npainting/snowflake/painting_272_000287.jpg\npainting/snowflake/painting_272_000290.jpg\npainting/snowflake/painting_272_000292.jpg\npainting/snowflake/painting_272_000294.jpg\npainting/snowflake/painting_272_000295.jpg\npainting/snowflake/painting_272_000296.jpg\npainting/snowflake/painting_272_000297.jpg\npainting/snowflake/painting_272_000298.jpg\npainting/snowflake/painting_272_000299.jpg\npainting/snowflake/painting_272_000301.jpg\npainting/snowflake/painting_272_000304.jpg\npainting/snowflake/painting_272_000307.jpg\npainting/snowflake/painting_272_000308.jpg\npainting/snowflake/painting_272_000310.jpg\npainting/snowflake/painting_272_000311.jpg\npainting/snowflake/painting_272_000312.jpg\npainting/snowflake/painting_272_000315.jpg\npainting/snowflake/painting_272_000320.jpg\npainting/snowflake/painting_272_000324.jpg\npainting/snowflake/painting_272_000325.jpg\npainting/snowflake/painting_272_000327.jpg\npainting/snowflake/painting_272_000330.jpg\npainting/snowflake/painting_272_000332.jpg\npainting/snowflake/painting_272_000341.jpg\npainting/snowflake/painting_272_000342.jpg\npainting/snowflake/painting_272_000343.jpg\npainting/snowflake/painting_272_000344.jpg\npainting/snowflake/painting_272_000345.jpg\npainting/snowflake/painting_272_000346.jpg\npainting/snowflake/painting_272_000348.jpg\npainting/snowflake/painting_272_000349.jpg\npainting/snowflake/painting_272_000350.jpg\npainting/snowflake/painting_272_000351.jpg\npainting/snowflake/painting_272_000352.jpg\npainting/snowflake/painting_272_000354.jpg\npainting/snowflake/painting_272_000356.jpg\npainting/snowflake/painting_272_000357.jpg\npainting/snowflake/painting_272_000359.jpg\npainting/snowflake/painting_272_000360.jpg\npainting/snowflake/painting_272_000362.jpg\npainting/snowflake/painting_272_000363.jpg\npainting/snowflake/painting_272_000366.jpg\npainting/snowflake/painting_272_000369.jpg\npainting/snowflake/painting_272_000371.jpg\npainting/snowflake/painting_272_000372.jpg\npainting/snowflake/painting_272_000373.jpg\npainting/snowflake/painting_272_000374.jpg\npainting/snowflake/painting_272_000375.jpg\npainting/snowflake/painting_272_000377.jpg\npainting/snowflake/painting_272_000378.jpg\npainting/snowflake/painting_272_000379.jpg\npainting/snowflake/painting_272_000380.jpg\npainting/snowflake/painting_272_000381.jpg\npainting/snowflake/painting_272_000382.jpg\npainting/snowflake/painting_272_000383.jpg\npainting/snowflake/painting_272_000385.jpg\npainting/snowflake/painting_272_000387.jpg\npainting/snowflake/painting_272_000388.jpg\npainting/snowflake/painting_272_000389.jpg\npainting/snowflake/painting_272_000390.jpg\npainting/snowflake/painting_272_000391.jpg\npainting/snowflake/painting_272_000392.jpg\npainting/snowflake/painting_272_000393.jpg\npainting/snowflake/painting_272_000396.jpg\npainting/snowflake/painting_272_000397.jpg\npainting/snowflake/painting_272_000398.jpg\npainting/snowflake/painting_272_000399.jpg\npainting/snowflake/painting_272_000400.jpg\npainting/snowflake/painting_272_000401.jpg\npainting/snowflake/painting_272_000402.jpg\npainting/snowflake/painting_272_000403.jpg\npainting/snowflake/painting_272_000404.jpg\npainting/snowflake/painting_272_000405.jpg\npainting/snowman/painting_273_000023.jpg\npainting/snowman/painting_273_000073.jpg\npainting/snowman/painting_273_000090.jpg\npainting/snowman/painting_273_000112.jpg\npainting/snowman/painting_273_000117.jpg\npainting/snowman/painting_273_000131.jpg\npainting/snowman/painting_273_000135.jpg\npainting/snowman/painting_273_000146.jpg\npainting/snowman/painting_273_000147.jpg\npainting/snowman/painting_273_000153.jpg\npainting/snowman/painting_273_000172.jpg\npainting/snowman/painting_273_000178.jpg\npainting/snowman/painting_273_000193.jpg\npainting/snowman/painting_273_000194.jpg\npainting/snowman/painting_273_000195.jpg\npainting/snowman/painting_273_000205.jpg\npainting/snowman/painting_273_000215.jpg\npainting/snowman/painting_273_000223.jpg\npainting/snowman/painting_273_000238.jpg\npainting/snowman/painting_273_000239.jpg\npainting/snowman/painting_273_000241.jpg\npainting/snowman/painting_273_000243.jpg\npainting/snowman/painting_273_000244.jpg\npainting/snowman/painting_273_000256.jpg\npainting/snowman/painting_273_000261.jpg\npainting/snowman/painting_273_000268.jpg\npainting/snowman/painting_273_000275.jpg\npainting/snowman/painting_273_000276.jpg\npainting/snowman/painting_273_000282.jpg\npainting/snowman/painting_273_000285.jpg\npainting/snowman/painting_273_000289.jpg\npainting/snowman/painting_273_000293.jpg\npainting/snowman/painting_273_000294.jpg\npainting/snowman/painting_273_000304.jpg\npainting/snowman/painting_273_000306.jpg\npainting/snowman/painting_273_000313.jpg\npainting/snowman/painting_273_000315.jpg\npainting/snowman/painting_273_000321.jpg\npainting/snowman/painting_273_000335.jpg\npainting/snowman/painting_273_000342.jpg\npainting/snowman/painting_273_000345.jpg\npainting/snowman/painting_273_000346.jpg\npainting/snowman/painting_273_000359.jpg\npainting/snowman/painting_273_000362.jpg\npainting/snowman/painting_273_000368.jpg\npainting/snowman/painting_273_000370.jpg\npainting/snowman/painting_273_000378.jpg\npainting/snowman/painting_273_000382.jpg\npainting/snowman/painting_273_000383.jpg\npainting/snowman/painting_273_000384.jpg\npainting/snowman/painting_273_000388.jpg\npainting/snowman/painting_273_000389.jpg\npainting/snowman/painting_273_000390.jpg\npainting/snowman/painting_273_000391.jpg\npainting/snowman/painting_273_000396.jpg\npainting/snowman/painting_273_000399.jpg\npainting/snowman/painting_273_000400.jpg\npainting/snowman/painting_273_000403.jpg\npainting/snowman/painting_273_000414.jpg\npainting/snowman/painting_273_000420.jpg\npainting/snowman/painting_273_000421.jpg\npainting/snowman/painting_273_000429.jpg\npainting/snowman/painting_273_000430.jpg\npainting/snowman/painting_273_000432.jpg\npainting/snowman/painting_273_000434.jpg\npainting/snowman/painting_273_000439.jpg\npainting/snowman/painting_273_000444.jpg\npainting/snowman/painting_273_000445.jpg\npainting/snowman/painting_273_000447.jpg\npainting/snowman/painting_273_000448.jpg\npainting/snowman/painting_273_000450.jpg\npainting/snowman/painting_273_000452.jpg\npainting/snowman/painting_273_000453.jpg\npainting/snowman/painting_273_000455.jpg\npainting/snowman/painting_273_000456.jpg\npainting/snowman/painting_273_000467.jpg\npainting/snowman/painting_273_000469.jpg\npainting/snowman/painting_273_000476.jpg\npainting/snowman/painting_273_000478.jpg\npainting/snowman/painting_273_000479.jpg\npainting/snowman/painting_273_000480.jpg\npainting/snowman/painting_273_000483.jpg\npainting/snowman/painting_273_000484.jpg\npainting/snowman/painting_273_000485.jpg\npainting/snowman/painting_273_000486.jpg\npainting/snowman/painting_273_000494.jpg\npainting/snowman/painting_273_000495.jpg\npainting/snowman/painting_273_000496.jpg\npainting/snowman/painting_273_000499.jpg\npainting/snowman/painting_273_000501.jpg\npainting/snowman/painting_273_000502.jpg\npainting/snowman/painting_273_000506.jpg\npainting/snowman/painting_273_000510.jpg\npainting/snowman/painting_273_000513.jpg\npainting/snowman/painting_273_000519.jpg\npainting/snowman/painting_273_000520.jpg\npainting/snowman/painting_273_000521.jpg\npainting/snowman/painting_273_000523.jpg\npainting/snowman/painting_273_000527.jpg\npainting/snowman/painting_273_000529.jpg\npainting/snowman/painting_273_000531.jpg\npainting/snowman/painting_273_000533.jpg\npainting/snowman/painting_273_000536.jpg\npainting/snowman/painting_273_000537.jpg\npainting/snowman/painting_273_000541.jpg\npainting/snowman/painting_273_000546.jpg\npainting/snowman/painting_273_000550.jpg\npainting/snowman/painting_273_000557.jpg\npainting/snowman/painting_273_000559.jpg\npainting/snowman/painting_273_000563.jpg\npainting/snowman/painting_273_000564.jpg\npainting/snowman/painting_273_000570.jpg\npainting/snowman/painting_273_000571.jpg\npainting/snowman/painting_273_000572.jpg\npainting/snowman/painting_273_000574.jpg\npainting/snowman/painting_273_000578.jpg\npainting/snowman/painting_273_000579.jpg\npainting/snowman/painting_273_000580.jpg\npainting/snowman/painting_273_000581.jpg\npainting/snowman/painting_273_000585.jpg\npainting/snowman/painting_273_000586.jpg\npainting/snowman/painting_273_000587.jpg\npainting/snowman/painting_273_000598.jpg\npainting/snowman/painting_273_000603.jpg\npainting/snowman/painting_273_000605.jpg\npainting/snowman/painting_273_000606.jpg\npainting/snowman/painting_273_000608.jpg\npainting/snowman/painting_273_000609.jpg\npainting/snowman/painting_273_000610.jpg\npainting/snowman/painting_273_000611.jpg\npainting/snowman/painting_273_000612.jpg\npainting/snowman/painting_273_000615.jpg\npainting/snowman/painting_273_000618.jpg\npainting/snowman/painting_273_000620.jpg\npainting/snowman/painting_273_000624.jpg\npainting/snowman/painting_273_000626.jpg\npainting/snowman/painting_273_000628.jpg\npainting/snowman/painting_273_000629.jpg\npainting/snowman/painting_273_000631.jpg\npainting/snowman/painting_273_000633.jpg\npainting/snowman/painting_273_000634.jpg\npainting/snowman/painting_273_000636.jpg\npainting/snowman/painting_273_000639.jpg\npainting/snowman/painting_273_000643.jpg\npainting/snowman/painting_273_000646.jpg\npainting/snowman/painting_273_000647.jpg\npainting/snowman/painting_273_000650.jpg\npainting/snowman/painting_273_000651.jpg\npainting/snowman/painting_273_000654.jpg\npainting/snowman/painting_273_000656.jpg\npainting/snowman/painting_273_000658.jpg\npainting/snowman/painting_273_000659.jpg\npainting/snowman/painting_273_000660.jpg\npainting/snowman/painting_273_000662.jpg\npainting/snowman/painting_273_000663.jpg\npainting/snowman/painting_273_000664.jpg\npainting/snowman/painting_273_000665.jpg\npainting/snowman/painting_273_000667.jpg\npainting/snowman/painting_273_000668.jpg\npainting/snowman/painting_273_000669.jpg\npainting/snowman/painting_273_000670.jpg\npainting/snowman/painting_273_000671.jpg\npainting/snowman/painting_273_000674.jpg\npainting/snowman/painting_273_000677.jpg\npainting/snowman/painting_273_000679.jpg\npainting/snowman/painting_273_000680.jpg\npainting/snowman/painting_273_000682.jpg\npainting/snowman/painting_273_000686.jpg\npainting/snowman/painting_273_000687.jpg\npainting/snowman/painting_273_000689.jpg\npainting/snowman/painting_273_000690.jpg\npainting/snowman/painting_273_000691.jpg\npainting/snowman/painting_273_000693.jpg\npainting/snowman/painting_273_000695.jpg\npainting/snowman/painting_273_000698.jpg\npainting/snowman/painting_273_000699.jpg\npainting/snowman/painting_273_000700.jpg\npainting/snowman/painting_273_000705.jpg\npainting/snowman/painting_273_000706.jpg\npainting/snowman/painting_273_000707.jpg\npainting/snowman/painting_273_000708.jpg\npainting/snowman/painting_273_000710.jpg\npainting/snowman/painting_273_000711.jpg\npainting/snowman/painting_273_000713.jpg\npainting/snowman/painting_273_000715.jpg\npainting/snowman/painting_273_000716.jpg\npainting/snowman/painting_273_000717.jpg\npainting/snowman/painting_273_000718.jpg\npainting/snowman/painting_273_000720.jpg\npainting/snowman/painting_273_000723.jpg\npainting/snowman/painting_273_000724.jpg\npainting/snowman/painting_273_000725.jpg\npainting/snowman/painting_273_000726.jpg\npainting/snowman/painting_273_000728.jpg\npainting/snowman/painting_273_000733.jpg\npainting/snowman/painting_273_000734.jpg\npainting/snowman/painting_273_000737.jpg\npainting/snowman/painting_273_000739.jpg\npainting/snowman/painting_273_000740.jpg\npainting/snowman/painting_273_000741.jpg\npainting/snowman/painting_273_000742.jpg\npainting/snowman/painting_273_000745.jpg\npainting/snowman/painting_273_000749.jpg\npainting/snowman/painting_273_000753.jpg\npainting/snowman/painting_273_000756.jpg\npainting/snowman/painting_273_000757.jpg\npainting/snowman/painting_273_000762.jpg\npainting/snowman/painting_273_000767.jpg\npainting/snowman/painting_273_000768.jpg\npainting/snowman/painting_273_000770.jpg\npainting/snowman/painting_273_000771.jpg\npainting/snowman/painting_273_000773.jpg\npainting/snowman/painting_273_000775.jpg\npainting/snowman/painting_273_000776.jpg\npainting/snowman/painting_273_000780.jpg\npainting/snowman/painting_273_000782.jpg\npainting/snowman/painting_273_000783.jpg\npainting/snowman/painting_273_000785.jpg\npainting/snowman/painting_273_000787.jpg\npainting/snowman/painting_273_000789.jpg\npainting/snowman/painting_273_000790.jpg\npainting/snowman/painting_273_000791.jpg\npainting/snowman/painting_273_000792.jpg\npainting/snowman/painting_273_000793.jpg\npainting/snowman/painting_273_000795.jpg\npainting/snowman/painting_273_000796.jpg\npainting/snowman/painting_273_000797.jpg\npainting/snowman/painting_273_000799.jpg\npainting/snowman/painting_273_000801.jpg\npainting/snowman/painting_273_000802.jpg\npainting/snowman/painting_273_000807.jpg\npainting/snowman/painting_273_000809.jpg\npainting/snowman/painting_273_000810.jpg\npainting/snowman/painting_273_000813.jpg\npainting/snowman/painting_273_000814.jpg\npainting/snowman/painting_273_000815.jpg\npainting/snowman/painting_273_000818.jpg\npainting/snowman/painting_273_000819.jpg\npainting/snowman/painting_273_000821.jpg\npainting/snowman/painting_273_000822.jpg\npainting/snowman/painting_273_000823.jpg\npainting/snowman/painting_273_000824.jpg\npainting/snowman/painting_273_000826.jpg\npainting/snowman/painting_273_000827.jpg\npainting/snowman/painting_273_000828.jpg\npainting/snowman/painting_273_000830.jpg\npainting/snowman/painting_273_000833.jpg\npainting/snowman/painting_273_000834.jpg\npainting/snowman/painting_273_000835.jpg\npainting/snowman/painting_273_000836.jpg\npainting/snowman/painting_273_000837.jpg\npainting/snowman/painting_273_000838.jpg\npainting/snowman/painting_273_000840.jpg\npainting/snowman/painting_273_000841.jpg\npainting/snowman/painting_273_000842.jpg\npainting/snowman/painting_273_000843.jpg\npainting/snowman/painting_273_000844.jpg\npainting/snowman/painting_273_000845.jpg\npainting/snowman/painting_273_000846.jpg\npainting/snowman/painting_273_000847.jpg\npainting/snowman/painting_273_000848.jpg\npainting/snowman/painting_273_000850.jpg\npainting/snowman/painting_273_000851.jpg\npainting/snowman/painting_273_000852.jpg\npainting/snowman/painting_273_000855.jpg\npainting/snowman/painting_273_000856.jpg\npainting/snowman/painting_273_000860.jpg\npainting/snowman/painting_273_000863.jpg\npainting/snowman/painting_273_000864.jpg\npainting/snowman/painting_273_000865.jpg\npainting/snowman/painting_273_000868.jpg\npainting/snowman/painting_273_000869.jpg\npainting/snowman/painting_273_000871.jpg\npainting/snowman/painting_273_000873.jpg\npainting/snowman/painting_273_000874.jpg\npainting/snowman/painting_273_000875.jpg\npainting/snowman/painting_273_000876.jpg\npainting/snowman/painting_273_000878.jpg\npainting/snowman/painting_273_000879.jpg\npainting/snowman/painting_273_000880.jpg\npainting/snowman/painting_273_000881.jpg\npainting/snowman/painting_273_000882.jpg\npainting/snowman/painting_273_000883.jpg\npainting/snowman/painting_273_000885.jpg\npainting/snowman/painting_273_000886.jpg\npainting/snowman/painting_273_000888.jpg\npainting/snowman/painting_273_000889.jpg\npainting/snowman/painting_273_000890.jpg\npainting/snowman/painting_273_000896.jpg\npainting/snowman/painting_273_000899.jpg\npainting/snowman/painting_273_000900.jpg\npainting/snowman/painting_273_000901.jpg\npainting/soccer_ball/painting_274_000016.jpg\npainting/soccer_ball/painting_274_000031.jpg\npainting/soccer_ball/painting_274_000045.jpg\npainting/soccer_ball/painting_274_000047.jpg\npainting/soccer_ball/painting_274_000061.jpg\npainting/soccer_ball/painting_274_000074.jpg\npainting/soccer_ball/painting_274_000079.jpg\npainting/soccer_ball/painting_274_000087.jpg\npainting/soccer_ball/painting_274_000091.jpg\npainting/soccer_ball/painting_274_000095.jpg\npainting/soccer_ball/painting_274_000100.jpg\npainting/soccer_ball/painting_274_000104.jpg\npainting/soccer_ball/painting_274_000109.jpg\npainting/soccer_ball/painting_274_000123.jpg\npainting/soccer_ball/painting_274_000127.jpg\npainting/soccer_ball/painting_274_000129.jpg\npainting/soccer_ball/painting_274_000132.jpg\npainting/soccer_ball/painting_274_000133.jpg\npainting/soccer_ball/painting_274_000144.jpg\npainting/soccer_ball/painting_274_000155.jpg\npainting/soccer_ball/painting_274_000161.jpg\npainting/soccer_ball/painting_274_000163.jpg\npainting/soccer_ball/painting_274_000172.jpg\npainting/soccer_ball/painting_274_000175.jpg\npainting/soccer_ball/painting_274_000176.jpg\npainting/soccer_ball/painting_274_000186.jpg\npainting/soccer_ball/painting_274_000190.jpg\npainting/soccer_ball/painting_274_000193.jpg\npainting/soccer_ball/painting_274_000195.jpg\npainting/soccer_ball/painting_274_000197.jpg\npainting/soccer_ball/painting_274_000202.jpg\npainting/soccer_ball/painting_274_000204.jpg\npainting/soccer_ball/painting_274_000205.jpg\npainting/soccer_ball/painting_274_000209.jpg\npainting/soccer_ball/painting_274_000215.jpg\npainting/soccer_ball/painting_274_000216.jpg\npainting/soccer_ball/painting_274_000217.jpg\npainting/soccer_ball/painting_274_000218.jpg\npainting/soccer_ball/painting_274_000219.jpg\npainting/soccer_ball/painting_274_000221.jpg\npainting/soccer_ball/painting_274_000223.jpg\npainting/soccer_ball/painting_274_000224.jpg\npainting/soccer_ball/painting_274_000225.jpg\npainting/soccer_ball/painting_274_000227.jpg\npainting/soccer_ball/painting_274_000234.jpg\npainting/soccer_ball/painting_274_000235.jpg\npainting/soccer_ball/painting_274_000242.jpg\npainting/soccer_ball/painting_274_000245.jpg\npainting/soccer_ball/painting_274_000249.jpg\npainting/soccer_ball/painting_274_000250.jpg\npainting/soccer_ball/painting_274_000257.jpg\npainting/soccer_ball/painting_274_000258.jpg\npainting/soccer_ball/painting_274_000259.jpg\npainting/soccer_ball/painting_274_000262.jpg\npainting/soccer_ball/painting_274_000265.jpg\npainting/soccer_ball/painting_274_000267.jpg\npainting/soccer_ball/painting_274_000268.jpg\npainting/sock/painting_275_000028.jpg\npainting/sock/painting_275_000029.jpg\npainting/speedboat/painting_276_000025.jpg\npainting/speedboat/painting_276_000026.jpg\npainting/speedboat/painting_276_000038.jpg\npainting/speedboat/painting_276_000039.jpg\npainting/speedboat/painting_276_000044.jpg\npainting/speedboat/painting_276_000045.jpg\npainting/speedboat/painting_276_000048.jpg\npainting/speedboat/painting_276_000052.jpg\npainting/speedboat/painting_276_000055.jpg\npainting/speedboat/painting_276_000056.jpg\npainting/speedboat/painting_276_000060.jpg\npainting/speedboat/painting_276_000069.jpg\npainting/speedboat/painting_276_000076.jpg\npainting/speedboat/painting_276_000078.jpg\npainting/speedboat/painting_276_000084.jpg\npainting/speedboat/painting_276_000088.jpg\npainting/speedboat/painting_276_000091.jpg\npainting/speedboat/painting_276_000094.jpg\npainting/speedboat/painting_276_000099.jpg\npainting/speedboat/painting_276_000102.jpg\npainting/speedboat/painting_276_000103.jpg\npainting/speedboat/painting_276_000105.jpg\npainting/speedboat/painting_276_000106.jpg\npainting/speedboat/painting_276_000109.jpg\npainting/speedboat/painting_276_000110.jpg\npainting/speedboat/painting_276_000111.jpg\npainting/speedboat/painting_276_000114.jpg\npainting/speedboat/painting_276_000115.jpg\npainting/speedboat/painting_276_000116.jpg\npainting/speedboat/painting_276_000118.jpg\npainting/speedboat/painting_276_000120.jpg\npainting/speedboat/painting_276_000125.jpg\npainting/speedboat/painting_276_000127.jpg\npainting/speedboat/painting_276_000130.jpg\npainting/speedboat/painting_276_000132.jpg\npainting/speedboat/painting_276_000133.jpg\npainting/speedboat/painting_276_000134.jpg\npainting/speedboat/painting_276_000135.jpg\npainting/speedboat/painting_276_000136.jpg\npainting/speedboat/painting_276_000137.jpg\npainting/speedboat/painting_276_000139.jpg\npainting/speedboat/painting_276_000140.jpg\npainting/spider/painting_277_000020.jpg\npainting/spider/painting_277_000031.jpg\npainting/spider/painting_277_000045.jpg\npainting/spider/painting_277_000049.jpg\npainting/spider/painting_277_000065.jpg\npainting/spider/painting_277_000069.jpg\npainting/spider/painting_277_000075.jpg\npainting/spider/painting_277_000091.jpg\npainting/spider/painting_277_000102.jpg\npainting/spider/painting_277_000105.jpg\npainting/spider/painting_277_000107.jpg\npainting/spider/painting_277_000113.jpg\npainting/spider/painting_277_000115.jpg\npainting/spider/painting_277_000116.jpg\npainting/spider/painting_277_000117.jpg\npainting/spider/painting_277_000119.jpg\npainting/spider/painting_277_000120.jpg\npainting/spider/painting_277_000128.jpg\npainting/spider/painting_277_000130.jpg\npainting/spider/painting_277_000134.jpg\npainting/spider/painting_277_000135.jpg\npainting/spider/painting_277_000137.jpg\npainting/spider/painting_277_000140.jpg\npainting/spider/painting_277_000143.jpg\npainting/spider/painting_277_000146.jpg\npainting/spider/painting_277_000155.jpg\npainting/spider/painting_277_000156.jpg\npainting/spider/painting_277_000163.jpg\npainting/spider/painting_277_000164.jpg\npainting/spider/painting_277_000166.jpg\npainting/spider/painting_277_000172.jpg\npainting/spider/painting_277_000174.jpg\npainting/spider/painting_277_000177.jpg\npainting/spider/painting_277_000179.jpg\npainting/spider/painting_277_000180.jpg\npainting/spider/painting_277_000185.jpg\npainting/spider/painting_277_000186.jpg\npainting/spider/painting_277_000190.jpg\npainting/spider/painting_277_000191.jpg\npainting/spider/painting_277_000192.jpg\npainting/spider/painting_277_000193.jpg\npainting/spider/painting_277_000194.jpg\npainting/spider/painting_277_000197.jpg\npainting/spider/painting_277_000198.jpg\npainting/spider/painting_277_000199.jpg\npainting/spider/painting_277_000202.jpg\npainting/spider/painting_277_000203.jpg\npainting/spider/painting_277_000205.jpg\npainting/spider/painting_277_000206.jpg\npainting/spider/painting_277_000207.jpg\npainting/spider/painting_277_000208.jpg\npainting/spider/painting_277_000210.jpg\npainting/spider/painting_277_000211.jpg\npainting/spider/painting_277_000212.jpg\npainting/spider/painting_277_000213.jpg\npainting/spider/painting_277_000215.jpg\npainting/spider/painting_277_000218.jpg\npainting/spider/painting_277_000219.jpg\npainting/spider/painting_277_000221.jpg\npainting/spider/painting_277_000223.jpg\npainting/spider/painting_277_000230.jpg\npainting/spider/painting_277_000232.jpg\npainting/spider/painting_277_000234.jpg\npainting/spider/painting_277_000235.jpg\npainting/spider/painting_277_000236.jpg\npainting/spider/painting_277_000237.jpg\npainting/spider/painting_277_000239.jpg\npainting/spider/painting_277_000240.jpg\npainting/spider/painting_277_000241.jpg\npainting/spider/painting_277_000242.jpg\npainting/spider/painting_277_000245.jpg\npainting/spider/painting_277_000249.jpg\npainting/spider/painting_277_000250.jpg\npainting/spider/painting_277_000251.jpg\npainting/spider/painting_277_000261.jpg\npainting/spider/painting_277_000263.jpg\npainting/spider/painting_277_000265.jpg\npainting/spider/painting_277_000267.jpg\npainting/spider/painting_277_000272.jpg\npainting/spider/painting_277_000273.jpg\npainting/spider/painting_277_000275.jpg\npainting/spider/painting_277_000277.jpg\npainting/spider/painting_277_000278.jpg\npainting/spider/painting_277_000279.jpg\npainting/spider/painting_277_000282.jpg\npainting/spider/painting_277_000283.jpg\npainting/spider/painting_277_000284.jpg\npainting/spider/painting_277_000286.jpg\npainting/spider/painting_277_000287.jpg\npainting/spider/painting_277_000288.jpg\npainting/spider/painting_277_000290.jpg\npainting/spider/painting_277_000292.jpg\npainting/spider/painting_277_000293.jpg\npainting/spider/painting_277_000294.jpg\npainting/spider/painting_277_000295.jpg\npainting/spider/painting_277_000296.jpg\npainting/spider/painting_277_000297.jpg\npainting/spider/painting_277_000298.jpg\npainting/spider/painting_277_000299.jpg\npainting/spider/painting_277_000300.jpg\npainting/spider/painting_277_000302.jpg\npainting/spider/painting_277_000303.jpg\npainting/spider/painting_277_000304.jpg\npainting/spider/painting_277_000305.jpg\npainting/spider/painting_277_000306.jpg\npainting/spider/painting_277_000307.jpg\npainting/spoon/painting_278_000001.jpg\npainting/spoon/painting_278_000016.jpg\npainting/spoon/painting_278_000024.jpg\npainting/spoon/painting_278_000026.jpg\npainting/spoon/painting_278_000040.jpg\npainting/spoon/painting_278_000045.jpg\npainting/spoon/painting_278_000046.jpg\npainting/spoon/painting_278_000050.jpg\npainting/spoon/painting_278_000052.jpg\npainting/spoon/painting_278_000055.jpg\npainting/spoon/painting_278_000057.jpg\npainting/spoon/painting_278_000064.jpg\npainting/spoon/painting_278_000074.jpg\npainting/spoon/painting_278_000076.jpg\npainting/spoon/painting_278_000081.jpg\npainting/spoon/painting_278_000083.jpg\npainting/spoon/painting_278_000085.jpg\npainting/spoon/painting_278_000089.jpg\npainting/spoon/painting_278_000091.jpg\npainting/spoon/painting_278_000093.jpg\npainting/spoon/painting_278_000095.jpg\npainting/spoon/painting_278_000096.jpg\npainting/spoon/painting_278_000099.jpg\npainting/spoon/painting_278_000101.jpg\npainting/spoon/painting_278_000106.jpg\npainting/spoon/painting_278_000107.jpg\npainting/spoon/painting_278_000109.jpg\npainting/spoon/painting_278_000110.jpg\npainting/spoon/painting_278_000114.jpg\npainting/spoon/painting_278_000115.jpg\npainting/spoon/painting_278_000120.jpg\npainting/spoon/painting_278_000121.jpg\npainting/spoon/painting_278_000123.jpg\npainting/spoon/painting_278_000125.jpg\npainting/spoon/painting_278_000126.jpg\npainting/spoon/painting_278_000127.jpg\npainting/spoon/painting_278_000128.jpg\npainting/spoon/painting_278_000130.jpg\npainting/spoon/painting_278_000138.jpg\npainting/spoon/painting_278_000139.jpg\npainting/spoon/painting_278_000142.jpg\npainting/spoon/painting_278_000143.jpg\npainting/spoon/painting_278_000144.jpg\npainting/spoon/painting_278_000145.jpg\npainting/spoon/painting_278_000146.jpg\npainting/spoon/painting_278_000147.jpg\npainting/spoon/painting_278_000149.jpg\npainting/spoon/painting_278_000151.jpg\npainting/spoon/painting_278_000152.jpg\npainting/spoon/painting_278_000153.jpg\npainting/spoon/painting_278_000154.jpg\npainting/spoon/painting_278_000157.jpg\npainting/spreadsheet/painting_279_000003.jpg\npainting/square/painting_280_000026.jpg\npainting/square/painting_280_000034.jpg\npainting/square/painting_280_000046.jpg\npainting/square/painting_280_000053.jpg\npainting/square/painting_280_000055.jpg\npainting/square/painting_280_000056.jpg\npainting/square/painting_280_000070.jpg\npainting/square/painting_280_000071.jpg\npainting/square/painting_280_000072.jpg\npainting/square/painting_280_000073.jpg\npainting/square/painting_280_000074.jpg\npainting/square/painting_280_000076.jpg\npainting/square/painting_280_000077.jpg\npainting/square/painting_280_000078.jpg\npainting/square/painting_280_000080.jpg\npainting/square/painting_280_000082.jpg\npainting/square/painting_280_000086.jpg\npainting/square/painting_280_000091.jpg\npainting/square/painting_280_000096.jpg\npainting/square/painting_280_000101.jpg\npainting/square/painting_280_000104.jpg\npainting/square/painting_280_000108.jpg\npainting/square/painting_280_000112.jpg\npainting/square/painting_280_000115.jpg\npainting/square/painting_280_000116.jpg\npainting/square/painting_280_000120.jpg\npainting/square/painting_280_000121.jpg\npainting/square/painting_280_000124.jpg\npainting/square/painting_280_000128.jpg\npainting/square/painting_280_000135.jpg\npainting/square/painting_280_000138.jpg\npainting/square/painting_280_000139.jpg\npainting/square/painting_280_000142.jpg\npainting/square/painting_280_000143.jpg\npainting/square/painting_280_000144.jpg\npainting/squiggle/painting_281_000021.jpg\npainting/squiggle/painting_281_000024.jpg\npainting/squiggle/painting_281_000027.jpg\npainting/squiggle/painting_281_000049.jpg\npainting/squiggle/painting_281_000079.jpg\npainting/squiggle/painting_281_000084.jpg\npainting/squiggle/painting_281_000088.jpg\npainting/squiggle/painting_281_000097.jpg\npainting/squiggle/painting_281_000104.jpg\npainting/squiggle/painting_281_000106.jpg\npainting/squiggle/painting_281_000108.jpg\npainting/squiggle/painting_281_000116.jpg\npainting/squiggle/painting_281_000121.jpg\npainting/squiggle/painting_281_000123.jpg\npainting/squiggle/painting_281_000131.jpg\npainting/squiggle/painting_281_000144.jpg\npainting/squiggle/painting_281_000150.jpg\npainting/squiggle/painting_281_000152.jpg\npainting/squiggle/painting_281_000153.jpg\npainting/squiggle/painting_281_000166.jpg\npainting/squiggle/painting_281_000168.jpg\npainting/squiggle/painting_281_000173.jpg\npainting/squiggle/painting_281_000175.jpg\npainting/squiggle/painting_281_000178.jpg\npainting/squiggle/painting_281_000181.jpg\npainting/squiggle/painting_281_000182.jpg\npainting/squiggle/painting_281_000184.jpg\npainting/squiggle/painting_281_000191.jpg\npainting/squiggle/painting_281_000200.jpg\npainting/squiggle/painting_281_000202.jpg\npainting/squiggle/painting_281_000206.jpg\npainting/squiggle/painting_281_000219.jpg\npainting/squiggle/painting_281_000221.jpg\npainting/squiggle/painting_281_000223.jpg\npainting/squiggle/painting_281_000227.jpg\npainting/squiggle/painting_281_000237.jpg\npainting/squiggle/painting_281_000246.jpg\npainting/squiggle/painting_281_000249.jpg\npainting/squiggle/painting_281_000250.jpg\npainting/squiggle/painting_281_000253.jpg\npainting/squiggle/painting_281_000260.jpg\npainting/squiggle/painting_281_000262.jpg\npainting/squiggle/painting_281_000263.jpg\npainting/squiggle/painting_281_000265.jpg\npainting/squiggle/painting_281_000268.jpg\npainting/squiggle/painting_281_000269.jpg\npainting/squiggle/painting_281_000270.jpg\npainting/squiggle/painting_281_000271.jpg\npainting/squiggle/painting_281_000273.jpg\npainting/squiggle/painting_281_000275.jpg\npainting/squiggle/painting_281_000277.jpg\npainting/squiggle/painting_281_000278.jpg\npainting/squiggle/painting_281_000284.jpg\npainting/squiggle/painting_281_000287.jpg\npainting/squiggle/painting_281_000295.jpg\npainting/squiggle/painting_281_000300.jpg\npainting/squiggle/painting_281_000305.jpg\npainting/squiggle/painting_281_000306.jpg\npainting/squiggle/painting_281_000308.jpg\npainting/squiggle/painting_281_000309.jpg\npainting/squiggle/painting_281_000311.jpg\npainting/squiggle/painting_281_000313.jpg\npainting/squiggle/painting_281_000315.jpg\npainting/squiggle/painting_281_000316.jpg\npainting/squiggle/painting_281_000317.jpg\npainting/squiggle/painting_281_000324.jpg\npainting/squiggle/painting_281_000328.jpg\npainting/squiggle/painting_281_000329.jpg\npainting/squiggle/painting_281_000338.jpg\npainting/squiggle/painting_281_000341.jpg\npainting/squiggle/painting_281_000345.jpg\npainting/squiggle/painting_281_000346.jpg\npainting/squiggle/painting_281_000348.jpg\npainting/squiggle/painting_281_000351.jpg\npainting/squiggle/painting_281_000360.jpg\npainting/squiggle/painting_281_000361.jpg\npainting/squiggle/painting_281_000362.jpg\npainting/squiggle/painting_281_000368.jpg\npainting/squiggle/painting_281_000369.jpg\npainting/squiggle/painting_281_000370.jpg\npainting/squiggle/painting_281_000371.jpg\npainting/squiggle/painting_281_000372.jpg\npainting/squiggle/painting_281_000375.jpg\npainting/squiggle/painting_281_000379.jpg\npainting/squiggle/painting_281_000382.jpg\npainting/squiggle/painting_281_000384.jpg\npainting/squiggle/painting_281_000386.jpg\npainting/squiggle/painting_281_000388.jpg\npainting/squiggle/painting_281_000390.jpg\npainting/squiggle/painting_281_000391.jpg\npainting/squiggle/painting_281_000392.jpg\npainting/squiggle/painting_281_000393.jpg\npainting/squiggle/painting_281_000395.jpg\npainting/squiggle/painting_281_000403.jpg\npainting/squiggle/painting_281_000414.jpg\npainting/squiggle/painting_281_000416.jpg\npainting/squiggle/painting_281_000417.jpg\npainting/squiggle/painting_281_000419.jpg\npainting/squiggle/painting_281_000422.jpg\npainting/squiggle/painting_281_000426.jpg\npainting/squiggle/painting_281_000430.jpg\npainting/squiggle/painting_281_000431.jpg\npainting/squiggle/painting_281_000434.jpg\npainting/squiggle/painting_281_000435.jpg\npainting/squiggle/painting_281_000439.jpg\npainting/squiggle/painting_281_000443.jpg\npainting/squiggle/painting_281_000444.jpg\npainting/squiggle/painting_281_000447.jpg\npainting/squiggle/painting_281_000448.jpg\npainting/squiggle/painting_281_000460.jpg\npainting/squiggle/painting_281_000462.jpg\npainting/squiggle/painting_281_000463.jpg\npainting/squiggle/painting_281_000468.jpg\npainting/squiggle/painting_281_000469.jpg\npainting/squiggle/painting_281_000470.jpg\npainting/squiggle/painting_281_000472.jpg\npainting/squiggle/painting_281_000474.jpg\npainting/squiggle/painting_281_000480.jpg\npainting/squiggle/painting_281_000481.jpg\npainting/squiggle/painting_281_000484.jpg\npainting/squiggle/painting_281_000485.jpg\npainting/squiggle/painting_281_000488.jpg\npainting/squiggle/painting_281_000491.jpg\npainting/squiggle/painting_281_000494.jpg\npainting/squiggle/painting_281_000495.jpg\npainting/squiggle/painting_281_000497.jpg\npainting/squiggle/painting_281_000498.jpg\npainting/squiggle/painting_281_000500.jpg\npainting/squiggle/painting_281_000501.jpg\npainting/squiggle/painting_281_000502.jpg\npainting/squiggle/painting_281_000505.jpg\npainting/squiggle/painting_281_000509.jpg\npainting/squiggle/painting_281_000510.jpg\npainting/squiggle/painting_281_000511.jpg\npainting/squiggle/painting_281_000512.jpg\npainting/squiggle/painting_281_000514.jpg\npainting/squiggle/painting_281_000516.jpg\npainting/squiggle/painting_281_000518.jpg\npainting/squiggle/painting_281_000522.jpg\npainting/squiggle/painting_281_000525.jpg\npainting/squiggle/painting_281_000526.jpg\npainting/squiggle/painting_281_000527.jpg\npainting/squiggle/painting_281_000529.jpg\npainting/squiggle/painting_281_000530.jpg\npainting/squiggle/painting_281_000531.jpg\npainting/squiggle/painting_281_000533.jpg\npainting/squiggle/painting_281_000534.jpg\npainting/squiggle/painting_281_000536.jpg\npainting/squiggle/painting_281_000538.jpg\npainting/squiggle/painting_281_000541.jpg\npainting/squiggle/painting_281_000542.jpg\npainting/squiggle/painting_281_000543.jpg\npainting/squiggle/painting_281_000549.jpg\npainting/squiggle/painting_281_000551.jpg\npainting/squiggle/painting_281_000552.jpg\npainting/squiggle/painting_281_000555.jpg\npainting/squiggle/painting_281_000558.jpg\npainting/squiggle/painting_281_000559.jpg\npainting/squiggle/painting_281_000561.jpg\npainting/squiggle/painting_281_000562.jpg\npainting/squiggle/painting_281_000563.jpg\npainting/squiggle/painting_281_000566.jpg\npainting/squiggle/painting_281_000569.jpg\npainting/squiggle/painting_281_000571.jpg\npainting/squiggle/painting_281_000572.jpg\npainting/squiggle/painting_281_000573.jpg\npainting/squiggle/painting_281_000576.jpg\npainting/squiggle/painting_281_000578.jpg\npainting/squiggle/painting_281_000581.jpg\npainting/squiggle/painting_281_000582.jpg\npainting/squiggle/painting_281_000583.jpg\npainting/squiggle/painting_281_000584.jpg\npainting/squiggle/painting_281_000585.jpg\npainting/squiggle/painting_281_000586.jpg\npainting/squiggle/painting_281_000590.jpg\npainting/squiggle/painting_281_000591.jpg\npainting/squiggle/painting_281_000594.jpg\npainting/squiggle/painting_281_000596.jpg\npainting/squiggle/painting_281_000598.jpg\npainting/squiggle/painting_281_000601.jpg\npainting/squiggle/painting_281_000603.jpg\npainting/squiggle/painting_281_000604.jpg\npainting/squiggle/painting_281_000606.jpg\npainting/squiggle/painting_281_000608.jpg\npainting/squiggle/painting_281_000611.jpg\npainting/squiggle/painting_281_000612.jpg\npainting/squiggle/painting_281_000613.jpg\npainting/squiggle/painting_281_000614.jpg\npainting/squiggle/painting_281_000616.jpg\npainting/squiggle/painting_281_000617.jpg\npainting/squiggle/painting_281_000618.jpg\npainting/squiggle/painting_281_000619.jpg\npainting/squiggle/painting_281_000625.jpg\npainting/squiggle/painting_281_000630.jpg\npainting/squiggle/painting_281_000631.jpg\npainting/squiggle/painting_281_000632.jpg\npainting/squiggle/painting_281_000633.jpg\npainting/squiggle/painting_281_000635.jpg\npainting/squiggle/painting_281_000636.jpg\npainting/squiggle/painting_281_000637.jpg\npainting/squiggle/painting_281_000638.jpg\npainting/squiggle/painting_281_000640.jpg\npainting/squiggle/painting_281_000644.jpg\npainting/squiggle/painting_281_000645.jpg\npainting/squiggle/painting_281_000648.jpg\npainting/squiggle/painting_281_000649.jpg\npainting/squiggle/painting_281_000650.jpg\npainting/squiggle/painting_281_000651.jpg\npainting/squiggle/painting_281_000654.jpg\npainting/squiggle/painting_281_000655.jpg\npainting/squiggle/painting_281_000657.jpg\npainting/squiggle/painting_281_000658.jpg\npainting/squiggle/painting_281_000659.jpg\npainting/squiggle/painting_281_000660.jpg\npainting/squiggle/painting_281_000661.jpg\npainting/squiggle/painting_281_000663.jpg\npainting/squiggle/painting_281_000665.jpg\npainting/squiggle/painting_281_000666.jpg\npainting/squiggle/painting_281_000668.jpg\npainting/squiggle/painting_281_000670.jpg\npainting/squiggle/painting_281_000671.jpg\npainting/squiggle/painting_281_000673.jpg\npainting/squiggle/painting_281_000674.jpg\npainting/squirrel/painting_282_000026.jpg\npainting/squirrel/painting_282_000059.jpg\npainting/squirrel/painting_282_000079.jpg\npainting/squirrel/painting_282_000082.jpg\npainting/squirrel/painting_282_000090.jpg\npainting/squirrel/painting_282_000097.jpg\npainting/squirrel/painting_282_000108.jpg\npainting/squirrel/painting_282_000122.jpg\npainting/squirrel/painting_282_000123.jpg\npainting/squirrel/painting_282_000127.jpg\npainting/squirrel/painting_282_000131.jpg\npainting/squirrel/painting_282_000136.jpg\npainting/squirrel/painting_282_000143.jpg\npainting/squirrel/painting_282_000146.jpg\npainting/squirrel/painting_282_000163.jpg\npainting/squirrel/painting_282_000176.jpg\npainting/squirrel/painting_282_000180.jpg\npainting/squirrel/painting_282_000183.jpg\npainting/squirrel/painting_282_000185.jpg\npainting/squirrel/painting_282_000188.jpg\npainting/squirrel/painting_282_000214.jpg\npainting/squirrel/painting_282_000218.jpg\npainting/squirrel/painting_282_000223.jpg\npainting/squirrel/painting_282_000231.jpg\npainting/squirrel/painting_282_000232.jpg\npainting/squirrel/painting_282_000241.jpg\npainting/squirrel/painting_282_000248.jpg\npainting/squirrel/painting_282_000259.jpg\npainting/squirrel/painting_282_000260.jpg\npainting/squirrel/painting_282_000262.jpg\npainting/squirrel/painting_282_000264.jpg\npainting/squirrel/painting_282_000266.jpg\npainting/squirrel/painting_282_000270.jpg\npainting/squirrel/painting_282_000280.jpg\npainting/squirrel/painting_282_000281.jpg\npainting/squirrel/painting_282_000294.jpg\npainting/squirrel/painting_282_000295.jpg\npainting/squirrel/painting_282_000296.jpg\npainting/squirrel/painting_282_000298.jpg\npainting/squirrel/painting_282_000300.jpg\npainting/squirrel/painting_282_000304.jpg\npainting/squirrel/painting_282_000306.jpg\npainting/squirrel/painting_282_000308.jpg\npainting/squirrel/painting_282_000310.jpg\npainting/squirrel/painting_282_000316.jpg\npainting/squirrel/painting_282_000317.jpg\npainting/squirrel/painting_282_000318.jpg\npainting/squirrel/painting_282_000323.jpg\npainting/squirrel/painting_282_000328.jpg\npainting/squirrel/painting_282_000335.jpg\npainting/squirrel/painting_282_000337.jpg\npainting/squirrel/painting_282_000339.jpg\npainting/squirrel/painting_282_000340.jpg\npainting/squirrel/painting_282_000345.jpg\npainting/squirrel/painting_282_000351.jpg\npainting/squirrel/painting_282_000352.jpg\npainting/squirrel/painting_282_000353.jpg\npainting/squirrel/painting_282_000354.jpg\npainting/squirrel/painting_282_000355.jpg\npainting/squirrel/painting_282_000358.jpg\npainting/squirrel/painting_282_000363.jpg\npainting/squirrel/painting_282_000374.jpg\npainting/squirrel/painting_282_000377.jpg\npainting/squirrel/painting_282_000383.jpg\npainting/squirrel/painting_282_000388.jpg\npainting/squirrel/painting_282_000389.jpg\npainting/squirrel/painting_282_000395.jpg\npainting/squirrel/painting_282_000396.jpg\npainting/squirrel/painting_282_000397.jpg\npainting/squirrel/painting_282_000399.jpg\npainting/squirrel/painting_282_000404.jpg\npainting/squirrel/painting_282_000410.jpg\npainting/squirrel/painting_282_000411.jpg\npainting/squirrel/painting_282_000414.jpg\npainting/squirrel/painting_282_000421.jpg\npainting/squirrel/painting_282_000422.jpg\npainting/squirrel/painting_282_000423.jpg\npainting/squirrel/painting_282_000426.jpg\npainting/squirrel/painting_282_000427.jpg\npainting/squirrel/painting_282_000429.jpg\npainting/squirrel/painting_282_000431.jpg\npainting/squirrel/painting_282_000432.jpg\npainting/squirrel/painting_282_000434.jpg\npainting/squirrel/painting_282_000435.jpg\npainting/squirrel/painting_282_000436.jpg\npainting/squirrel/painting_282_000443.jpg\npainting/squirrel/painting_282_000444.jpg\npainting/squirrel/painting_282_000446.jpg\npainting/squirrel/painting_282_000447.jpg\npainting/squirrel/painting_282_000456.jpg\npainting/squirrel/painting_282_000458.jpg\npainting/squirrel/painting_282_000462.jpg\npainting/squirrel/painting_282_000463.jpg\npainting/squirrel/painting_282_000466.jpg\npainting/squirrel/painting_282_000478.jpg\npainting/squirrel/painting_282_000480.jpg\npainting/squirrel/painting_282_000482.jpg\npainting/squirrel/painting_282_000483.jpg\npainting/squirrel/painting_282_000486.jpg\npainting/squirrel/painting_282_000488.jpg\npainting/squirrel/painting_282_000494.jpg\npainting/squirrel/painting_282_000495.jpg\npainting/squirrel/painting_282_000496.jpg\npainting/squirrel/painting_282_000497.jpg\npainting/squirrel/painting_282_000502.jpg\npainting/squirrel/painting_282_000503.jpg\npainting/squirrel/painting_282_000506.jpg\npainting/squirrel/painting_282_000507.jpg\npainting/squirrel/painting_282_000508.jpg\npainting/squirrel/painting_282_000510.jpg\npainting/squirrel/painting_282_000511.jpg\npainting/squirrel/painting_282_000513.jpg\npainting/squirrel/painting_282_000514.jpg\npainting/squirrel/painting_282_000517.jpg\npainting/squirrel/painting_282_000519.jpg\npainting/squirrel/painting_282_000524.jpg\npainting/squirrel/painting_282_000525.jpg\npainting/squirrel/painting_282_000526.jpg\npainting/squirrel/painting_282_000529.jpg\npainting/squirrel/painting_282_000531.jpg\npainting/squirrel/painting_282_000534.jpg\npainting/squirrel/painting_282_000536.jpg\npainting/squirrel/painting_282_000538.jpg\npainting/squirrel/painting_282_000539.jpg\npainting/squirrel/painting_282_000542.jpg\npainting/squirrel/painting_282_000543.jpg\npainting/squirrel/painting_282_000544.jpg\npainting/squirrel/painting_282_000548.jpg\npainting/squirrel/painting_282_000550.jpg\npainting/squirrel/painting_282_000552.jpg\npainting/squirrel/painting_282_000555.jpg\npainting/squirrel/painting_282_000557.jpg\npainting/squirrel/painting_282_000558.jpg\npainting/squirrel/painting_282_000561.jpg\npainting/squirrel/painting_282_000564.jpg\npainting/squirrel/painting_282_000566.jpg\npainting/squirrel/painting_282_000567.jpg\npainting/squirrel/painting_282_000568.jpg\npainting/squirrel/painting_282_000569.jpg\npainting/squirrel/painting_282_000572.jpg\npainting/squirrel/painting_282_000573.jpg\npainting/squirrel/painting_282_000577.jpg\npainting/squirrel/painting_282_000581.jpg\npainting/squirrel/painting_282_000582.jpg\npainting/squirrel/painting_282_000583.jpg\npainting/squirrel/painting_282_000584.jpg\npainting/squirrel/painting_282_000586.jpg\npainting/squirrel/painting_282_000587.jpg\npainting/squirrel/painting_282_000588.jpg\npainting/squirrel/painting_282_000589.jpg\npainting/squirrel/painting_282_000590.jpg\npainting/squirrel/painting_282_000591.jpg\npainting/squirrel/painting_282_000592.jpg\npainting/squirrel/painting_282_000597.jpg\npainting/squirrel/painting_282_000598.jpg\npainting/squirrel/painting_282_000599.jpg\npainting/squirrel/painting_282_000607.jpg\npainting/squirrel/painting_282_000609.jpg\npainting/squirrel/painting_282_000615.jpg\npainting/squirrel/painting_282_000618.jpg\npainting/squirrel/painting_282_000622.jpg\npainting/squirrel/painting_282_000625.jpg\npainting/squirrel/painting_282_000626.jpg\npainting/squirrel/painting_282_000627.jpg\npainting/squirrel/painting_282_000629.jpg\npainting/squirrel/painting_282_000631.jpg\npainting/squirrel/painting_282_000632.jpg\npainting/squirrel/painting_282_000634.jpg\npainting/squirrel/painting_282_000636.jpg\npainting/squirrel/painting_282_000639.jpg\npainting/squirrel/painting_282_000640.jpg\npainting/squirrel/painting_282_000643.jpg\npainting/squirrel/painting_282_000644.jpg\npainting/squirrel/painting_282_000646.jpg\npainting/squirrel/painting_282_000647.jpg\npainting/squirrel/painting_282_000648.jpg\npainting/squirrel/painting_282_000649.jpg\npainting/squirrel/painting_282_000650.jpg\npainting/squirrel/painting_282_000651.jpg\npainting/squirrel/painting_282_000653.jpg\npainting/squirrel/painting_282_000656.jpg\npainting/squirrel/painting_282_000657.jpg\npainting/squirrel/painting_282_000658.jpg\npainting/squirrel/painting_282_000659.jpg\npainting/squirrel/painting_282_000660.jpg\npainting/squirrel/painting_282_000664.jpg\npainting/squirrel/painting_282_000665.jpg\npainting/squirrel/painting_282_000666.jpg\npainting/squirrel/painting_282_000668.jpg\npainting/squirrel/painting_282_000674.jpg\npainting/squirrel/painting_282_000676.jpg\npainting/squirrel/painting_282_000677.jpg\npainting/squirrel/painting_282_000679.jpg\npainting/squirrel/painting_282_000681.jpg\npainting/squirrel/painting_282_000682.jpg\npainting/squirrel/painting_282_000684.jpg\npainting/squirrel/painting_282_000685.jpg\npainting/squirrel/painting_282_000686.jpg\npainting/squirrel/painting_282_000691.jpg\npainting/squirrel/painting_282_000693.jpg\npainting/squirrel/painting_282_000694.jpg\npainting/squirrel/painting_282_000695.jpg\npainting/squirrel/painting_282_000696.jpg\npainting/squirrel/painting_282_000698.jpg\npainting/squirrel/painting_282_000701.jpg\npainting/squirrel/painting_282_000703.jpg\npainting/squirrel/painting_282_000704.jpg\npainting/squirrel/painting_282_000705.jpg\npainting/squirrel/painting_282_000706.jpg\npainting/squirrel/painting_282_000707.jpg\npainting/squirrel/painting_282_000708.jpg\npainting/squirrel/painting_282_000709.jpg\npainting/squirrel/painting_282_000715.jpg\npainting/squirrel/painting_282_000716.jpg\npainting/squirrel/painting_282_000718.jpg\npainting/squirrel/painting_282_000720.jpg\npainting/squirrel/painting_282_000724.jpg\npainting/squirrel/painting_282_000726.jpg\npainting/squirrel/painting_282_000727.jpg\npainting/squirrel/painting_282_000729.jpg\npainting/squirrel/painting_282_000730.jpg\npainting/squirrel/painting_282_000731.jpg\npainting/squirrel/painting_282_000733.jpg\npainting/squirrel/painting_282_000734.jpg\npainting/squirrel/painting_282_000745.jpg\npainting/squirrel/painting_282_000746.jpg\npainting/squirrel/painting_282_000753.jpg\npainting/squirrel/painting_282_000755.jpg\npainting/squirrel/painting_282_000756.jpg\npainting/squirrel/painting_282_000757.jpg\npainting/squirrel/painting_282_000761.jpg\npainting/squirrel/painting_282_000762.jpg\npainting/squirrel/painting_282_000763.jpg\npainting/squirrel/painting_282_000764.jpg\npainting/squirrel/painting_282_000765.jpg\npainting/squirrel/painting_282_000766.jpg\npainting/squirrel/painting_282_000767.jpg\npainting/squirrel/painting_282_000770.jpg\npainting/squirrel/painting_282_000771.jpg\npainting/squirrel/painting_282_000773.jpg\npainting/squirrel/painting_282_000774.jpg\npainting/squirrel/painting_282_000775.jpg\npainting/squirrel/painting_282_000777.jpg\npainting/squirrel/painting_282_000778.jpg\npainting/squirrel/painting_282_000779.jpg\npainting/stairs/painting_283_000003.jpg\npainting/stairs/painting_283_000012.jpg\npainting/stairs/painting_283_000017.jpg\npainting/stairs/painting_283_000019.jpg\npainting/stairs/painting_283_000022.jpg\npainting/stairs/painting_283_000023.jpg\npainting/stairs/painting_283_000024.jpg\npainting/stairs/painting_283_000026.jpg\npainting/stairs/painting_283_000027.jpg\npainting/star/painting_284_000013.jpg\npainting/star/painting_284_000014.jpg\npainting/star/painting_284_000017.jpg\npainting/star/painting_284_000030.jpg\npainting/star/painting_284_000037.jpg\npainting/star/painting_284_000041.jpg\npainting/star/painting_284_000042.jpg\npainting/star/painting_284_000043.jpg\npainting/star/painting_284_000046.jpg\npainting/star/painting_284_000048.jpg\npainting/star/painting_284_000052.jpg\npainting/star/painting_284_000054.jpg\npainting/star/painting_284_000057.jpg\npainting/star/painting_284_000059.jpg\npainting/star/painting_284_000060.jpg\npainting/star/painting_284_000066.jpg\npainting/star/painting_284_000067.jpg\npainting/star/painting_284_000072.jpg\npainting/star/painting_284_000074.jpg\npainting/star/painting_284_000078.jpg\npainting/star/painting_284_000085.jpg\npainting/star/painting_284_000088.jpg\npainting/star/painting_284_000089.jpg\npainting/star/painting_284_000092.jpg\npainting/star/painting_284_000095.jpg\npainting/star/painting_284_000096.jpg\npainting/steak/painting_285_000010.jpg\npainting/steak/painting_285_000013.jpg\npainting/steak/painting_285_000017.jpg\npainting/steak/painting_285_000023.jpg\npainting/steak/painting_285_000024.jpg\npainting/steak/painting_285_000029.jpg\npainting/steak/painting_285_000039.jpg\npainting/steak/painting_285_000042.jpg\npainting/steak/painting_285_000043.jpg\npainting/steak/painting_285_000044.jpg\npainting/steak/painting_285_000045.jpg\npainting/steak/painting_285_000047.jpg\npainting/stereo/painting_286_000004.jpg\npainting/stereo/painting_286_000005.jpg\npainting/stereo/painting_286_000008.jpg\npainting/stereo/painting_286_000012.jpg\npainting/stethoscope/painting_287_000049.jpg\npainting/stethoscope/painting_287_000078.jpg\npainting/stethoscope/painting_287_000086.jpg\npainting/stethoscope/painting_287_000091.jpg\npainting/stethoscope/painting_287_000113.jpg\npainting/stethoscope/painting_287_000116.jpg\npainting/stethoscope/painting_287_000118.jpg\npainting/stethoscope/painting_287_000121.jpg\npainting/stethoscope/painting_287_000124.jpg\npainting/stethoscope/painting_287_000125.jpg\npainting/stethoscope/painting_287_000130.jpg\npainting/stethoscope/painting_287_000131.jpg\npainting/stethoscope/painting_287_000133.jpg\npainting/stethoscope/painting_287_000140.jpg\npainting/stethoscope/painting_287_000144.jpg\npainting/stethoscope/painting_287_000150.jpg\npainting/stethoscope/painting_287_000152.jpg\npainting/stethoscope/painting_287_000156.jpg\npainting/stethoscope/painting_287_000158.jpg\npainting/stethoscope/painting_287_000160.jpg\npainting/stethoscope/painting_287_000161.jpg\npainting/stethoscope/painting_287_000167.jpg\npainting/stethoscope/painting_287_000168.jpg\npainting/stethoscope/painting_287_000169.jpg\npainting/stethoscope/painting_287_000185.jpg\npainting/stethoscope/painting_287_000188.jpg\npainting/stethoscope/painting_287_000200.jpg\npainting/stethoscope/painting_287_000207.jpg\npainting/stethoscope/painting_287_000208.jpg\npainting/stethoscope/painting_287_000211.jpg\npainting/stethoscope/painting_287_000215.jpg\npainting/stethoscope/painting_287_000218.jpg\npainting/stethoscope/painting_287_000220.jpg\npainting/stethoscope/painting_287_000222.jpg\npainting/stethoscope/painting_287_000231.jpg\npainting/stethoscope/painting_287_000239.jpg\npainting/stethoscope/painting_287_000246.jpg\npainting/stethoscope/painting_287_000248.jpg\npainting/stethoscope/painting_287_000250.jpg\npainting/stethoscope/painting_287_000254.jpg\npainting/stethoscope/painting_287_000256.jpg\npainting/stethoscope/painting_287_000264.jpg\npainting/stethoscope/painting_287_000273.jpg\npainting/stethoscope/painting_287_000274.jpg\npainting/stethoscope/painting_287_000284.jpg\npainting/stethoscope/painting_287_000286.jpg\npainting/stethoscope/painting_287_000287.jpg\npainting/stethoscope/painting_287_000289.jpg\npainting/stethoscope/painting_287_000290.jpg\npainting/stethoscope/painting_287_000293.jpg\npainting/stethoscope/painting_287_000294.jpg\npainting/stethoscope/painting_287_000298.jpg\npainting/stethoscope/painting_287_000301.jpg\npainting/stethoscope/painting_287_000302.jpg\npainting/stethoscope/painting_287_000304.jpg\npainting/stethoscope/painting_287_000306.jpg\npainting/stethoscope/painting_287_000310.jpg\npainting/stethoscope/painting_287_000315.jpg\npainting/stethoscope/painting_287_000316.jpg\npainting/stethoscope/painting_287_000319.jpg\npainting/stethoscope/painting_287_000323.jpg\npainting/stethoscope/painting_287_000328.jpg\npainting/stethoscope/painting_287_000330.jpg\npainting/stethoscope/painting_287_000334.jpg\npainting/stethoscope/painting_287_000335.jpg\npainting/stethoscope/painting_287_000338.jpg\npainting/stethoscope/painting_287_000341.jpg\npainting/stethoscope/painting_287_000342.jpg\npainting/stitches/painting_288_000007.jpg\npainting/stitches/painting_288_000012.jpg\npainting/stitches/painting_288_000013.jpg\npainting/stop_sign/painting_289_000015.jpg\npainting/stop_sign/painting_289_000031.jpg\npainting/stop_sign/painting_289_000043.jpg\npainting/stop_sign/painting_289_000046.jpg\npainting/stop_sign/painting_289_000050.jpg\npainting/stop_sign/painting_289_000058.jpg\npainting/stop_sign/painting_289_000061.jpg\npainting/stop_sign/painting_289_000062.jpg\npainting/stop_sign/painting_289_000065.jpg\npainting/stop_sign/painting_289_000081.jpg\npainting/stove/painting_290_000009.jpg\npainting/stove/painting_290_000016.jpg\npainting/strawberry/painting_291_000073.jpg\npainting/strawberry/painting_291_000109.jpg\npainting/strawberry/painting_291_000110.jpg\npainting/strawberry/painting_291_000124.jpg\npainting/strawberry/painting_291_000132.jpg\npainting/strawberry/painting_291_000133.jpg\npainting/strawberry/painting_291_000134.jpg\npainting/strawberry/painting_291_000137.jpg\npainting/strawberry/painting_291_000141.jpg\npainting/strawberry/painting_291_000144.jpg\npainting/strawberry/painting_291_000153.jpg\npainting/strawberry/painting_291_000157.jpg\npainting/strawberry/painting_291_000175.jpg\npainting/strawberry/painting_291_000177.jpg\npainting/strawberry/painting_291_000178.jpg\npainting/strawberry/painting_291_000186.jpg\npainting/strawberry/painting_291_000192.jpg\npainting/strawberry/painting_291_000193.jpg\npainting/strawberry/painting_291_000198.jpg\npainting/strawberry/painting_291_000207.jpg\npainting/strawberry/painting_291_000209.jpg\npainting/strawberry/painting_291_000210.jpg\npainting/strawberry/painting_291_000212.jpg\npainting/strawberry/painting_291_000216.jpg\npainting/strawberry/painting_291_000222.jpg\npainting/strawberry/painting_291_000223.jpg\npainting/strawberry/painting_291_000225.jpg\npainting/strawberry/painting_291_000226.jpg\npainting/strawberry/painting_291_000227.jpg\npainting/strawberry/painting_291_000234.jpg\npainting/strawberry/painting_291_000237.jpg\npainting/strawberry/painting_291_000240.jpg\npainting/strawberry/painting_291_000243.jpg\npainting/strawberry/painting_291_000246.jpg\npainting/strawberry/painting_291_000254.jpg\npainting/strawberry/painting_291_000255.jpg\npainting/strawberry/painting_291_000256.jpg\npainting/strawberry/painting_291_000264.jpg\npainting/strawberry/painting_291_000265.jpg\npainting/strawberry/painting_291_000273.jpg\npainting/strawberry/painting_291_000275.jpg\npainting/strawberry/painting_291_000278.jpg\npainting/strawberry/painting_291_000279.jpg\npainting/strawberry/painting_291_000283.jpg\npainting/strawberry/painting_291_000284.jpg\npainting/strawberry/painting_291_000293.jpg\npainting/strawberry/painting_291_000297.jpg\npainting/strawberry/painting_291_000298.jpg\npainting/strawberry/painting_291_000300.jpg\npainting/strawberry/painting_291_000304.jpg\npainting/strawberry/painting_291_000310.jpg\npainting/strawberry/painting_291_000311.jpg\npainting/strawberry/painting_291_000312.jpg\npainting/strawberry/painting_291_000314.jpg\npainting/strawberry/painting_291_000315.jpg\npainting/strawberry/painting_291_000316.jpg\npainting/strawberry/painting_291_000317.jpg\npainting/strawberry/painting_291_000320.jpg\npainting/strawberry/painting_291_000322.jpg\npainting/strawberry/painting_291_000323.jpg\npainting/strawberry/painting_291_000325.jpg\npainting/strawberry/painting_291_000330.jpg\npainting/strawberry/painting_291_000334.jpg\npainting/strawberry/painting_291_000337.jpg\npainting/strawberry/painting_291_000338.jpg\npainting/strawberry/painting_291_000340.jpg\npainting/strawberry/painting_291_000341.jpg\npainting/strawberry/painting_291_000342.jpg\npainting/strawberry/painting_291_000345.jpg\npainting/strawberry/painting_291_000348.jpg\npainting/strawberry/painting_291_000354.jpg\npainting/strawberry/painting_291_000355.jpg\npainting/strawberry/painting_291_000356.jpg\npainting/strawberry/painting_291_000361.jpg\npainting/strawberry/painting_291_000362.jpg\npainting/strawberry/painting_291_000363.jpg\npainting/strawberry/painting_291_000364.jpg\npainting/strawberry/painting_291_000368.jpg\npainting/strawberry/painting_291_000371.jpg\npainting/strawberry/painting_291_000373.jpg\npainting/strawberry/painting_291_000376.jpg\npainting/strawberry/painting_291_000377.jpg\npainting/strawberry/painting_291_000378.jpg\npainting/strawberry/painting_291_000379.jpg\npainting/strawberry/painting_291_000380.jpg\npainting/strawberry/painting_291_000383.jpg\npainting/strawberry/painting_291_000384.jpg\npainting/strawberry/painting_291_000385.jpg\npainting/strawberry/painting_291_000387.jpg\npainting/strawberry/painting_291_000388.jpg\npainting/strawberry/painting_291_000392.jpg\npainting/strawberry/painting_291_000394.jpg\npainting/strawberry/painting_291_000395.jpg\npainting/strawberry/painting_291_000397.jpg\npainting/strawberry/painting_291_000401.jpg\npainting/strawberry/painting_291_000402.jpg\npainting/strawberry/painting_291_000405.jpg\npainting/strawberry/painting_291_000406.jpg\npainting/strawberry/painting_291_000407.jpg\npainting/strawberry/painting_291_000409.jpg\npainting/strawberry/painting_291_000410.jpg\npainting/strawberry/painting_291_000411.jpg\npainting/strawberry/painting_291_000412.jpg\npainting/strawberry/painting_291_000413.jpg\npainting/strawberry/painting_291_000418.jpg\npainting/strawberry/painting_291_000419.jpg\npainting/strawberry/painting_291_000423.jpg\npainting/strawberry/painting_291_000427.jpg\npainting/strawberry/painting_291_000433.jpg\npainting/strawberry/painting_291_000435.jpg\npainting/strawberry/painting_291_000436.jpg\npainting/strawberry/painting_291_000439.jpg\npainting/strawberry/painting_291_000444.jpg\npainting/strawberry/painting_291_000445.jpg\npainting/strawberry/painting_291_000448.jpg\npainting/strawberry/painting_291_000449.jpg\npainting/strawberry/painting_291_000450.jpg\npainting/strawberry/painting_291_000453.jpg\npainting/strawberry/painting_291_000454.jpg\npainting/strawberry/painting_291_000458.jpg\npainting/strawberry/painting_291_000459.jpg\npainting/strawberry/painting_291_000460.jpg\npainting/strawberry/painting_291_000461.jpg\npainting/strawberry/painting_291_000462.jpg\npainting/strawberry/painting_291_000465.jpg\npainting/strawberry/painting_291_000466.jpg\npainting/strawberry/painting_291_000469.jpg\npainting/strawberry/painting_291_000470.jpg\npainting/strawberry/painting_291_000471.jpg\npainting/strawberry/painting_291_000472.jpg\npainting/strawberry/painting_291_000474.jpg\npainting/strawberry/painting_291_000476.jpg\npainting/strawberry/painting_291_000477.jpg\npainting/strawberry/painting_291_000478.jpg\npainting/strawberry/painting_291_000482.jpg\npainting/strawberry/painting_291_000485.jpg\npainting/strawberry/painting_291_000487.jpg\npainting/strawberry/painting_291_000495.jpg\npainting/strawberry/painting_291_000497.jpg\npainting/strawberry/painting_291_000498.jpg\npainting/strawberry/painting_291_000500.jpg\npainting/strawberry/painting_291_000501.jpg\npainting/strawberry/painting_291_000502.jpg\npainting/strawberry/painting_291_000505.jpg\npainting/strawberry/painting_291_000507.jpg\npainting/strawberry/painting_291_000508.jpg\npainting/strawberry/painting_291_000509.jpg\npainting/strawberry/painting_291_000510.jpg\npainting/strawberry/painting_291_000511.jpg\npainting/strawberry/painting_291_000514.jpg\npainting/strawberry/painting_291_000516.jpg\npainting/strawberry/painting_291_000524.jpg\npainting/strawberry/painting_291_000525.jpg\npainting/streetlight/painting_292_000036.jpg\npainting/streetlight/painting_292_000096.jpg\npainting/streetlight/painting_292_000099.jpg\npainting/streetlight/painting_292_000126.jpg\npainting/streetlight/painting_292_000137.jpg\npainting/streetlight/painting_292_000149.jpg\npainting/streetlight/painting_292_000159.jpg\npainting/streetlight/painting_292_000161.jpg\npainting/streetlight/painting_292_000165.jpg\npainting/streetlight/painting_292_000169.jpg\npainting/streetlight/painting_292_000178.jpg\npainting/streetlight/painting_292_000183.jpg\npainting/streetlight/painting_292_000185.jpg\npainting/streetlight/painting_292_000191.jpg\npainting/streetlight/painting_292_000205.jpg\npainting/streetlight/painting_292_000213.jpg\npainting/streetlight/painting_292_000216.jpg\npainting/streetlight/painting_292_000219.jpg\npainting/streetlight/painting_292_000231.jpg\npainting/streetlight/painting_292_000235.jpg\npainting/streetlight/painting_292_000242.jpg\npainting/streetlight/painting_292_000256.jpg\npainting/streetlight/painting_292_000261.jpg\npainting/streetlight/painting_292_000265.jpg\npainting/streetlight/painting_292_000266.jpg\npainting/streetlight/painting_292_000268.jpg\npainting/streetlight/painting_292_000270.jpg\npainting/streetlight/painting_292_000273.jpg\npainting/streetlight/painting_292_000274.jpg\npainting/streetlight/painting_292_000282.jpg\npainting/streetlight/painting_292_000292.jpg\npainting/streetlight/painting_292_000305.jpg\npainting/streetlight/painting_292_000309.jpg\npainting/streetlight/painting_292_000314.jpg\npainting/streetlight/painting_292_000318.jpg\npainting/streetlight/painting_292_000319.jpg\npainting/streetlight/painting_292_000324.jpg\npainting/streetlight/painting_292_000326.jpg\npainting/streetlight/painting_292_000331.jpg\npainting/streetlight/painting_292_000337.jpg\npainting/streetlight/painting_292_000344.jpg\npainting/streetlight/painting_292_000346.jpg\npainting/streetlight/painting_292_000347.jpg\npainting/streetlight/painting_292_000348.jpg\npainting/streetlight/painting_292_000352.jpg\npainting/streetlight/painting_292_000354.jpg\npainting/streetlight/painting_292_000358.jpg\npainting/streetlight/painting_292_000362.jpg\npainting/streetlight/painting_292_000364.jpg\npainting/streetlight/painting_292_000371.jpg\npainting/streetlight/painting_292_000375.jpg\npainting/streetlight/painting_292_000377.jpg\npainting/streetlight/painting_292_000384.jpg\npainting/streetlight/painting_292_000387.jpg\npainting/streetlight/painting_292_000388.jpg\npainting/streetlight/painting_292_000391.jpg\npainting/streetlight/painting_292_000398.jpg\npainting/streetlight/painting_292_000402.jpg\npainting/streetlight/painting_292_000403.jpg\npainting/streetlight/painting_292_000404.jpg\npainting/streetlight/painting_292_000407.jpg\npainting/streetlight/painting_292_000408.jpg\npainting/streetlight/painting_292_000410.jpg\npainting/streetlight/painting_292_000414.jpg\npainting/streetlight/painting_292_000417.jpg\npainting/streetlight/painting_292_000425.jpg\npainting/streetlight/painting_292_000429.jpg\npainting/streetlight/painting_292_000432.jpg\npainting/streetlight/painting_292_000438.jpg\npainting/streetlight/painting_292_000440.jpg\npainting/streetlight/painting_292_000447.jpg\npainting/streetlight/painting_292_000449.jpg\npainting/streetlight/painting_292_000450.jpg\npainting/streetlight/painting_292_000452.jpg\npainting/streetlight/painting_292_000460.jpg\npainting/streetlight/painting_292_000464.jpg\npainting/streetlight/painting_292_000465.jpg\npainting/streetlight/painting_292_000468.jpg\npainting/streetlight/painting_292_000471.jpg\npainting/streetlight/painting_292_000473.jpg\npainting/streetlight/painting_292_000475.jpg\npainting/streetlight/painting_292_000481.jpg\npainting/streetlight/painting_292_000482.jpg\npainting/streetlight/painting_292_000483.jpg\npainting/streetlight/painting_292_000485.jpg\npainting/streetlight/painting_292_000489.jpg\npainting/streetlight/painting_292_000490.jpg\npainting/streetlight/painting_292_000492.jpg\npainting/streetlight/painting_292_000496.jpg\npainting/streetlight/painting_292_000497.jpg\npainting/streetlight/painting_292_000502.jpg\npainting/streetlight/painting_292_000503.jpg\npainting/streetlight/painting_292_000505.jpg\npainting/streetlight/painting_292_000508.jpg\npainting/streetlight/painting_292_000509.jpg\npainting/streetlight/painting_292_000510.jpg\npainting/streetlight/painting_292_000511.jpg\npainting/streetlight/painting_292_000512.jpg\npainting/streetlight/painting_292_000520.jpg\npainting/streetlight/painting_292_000521.jpg\npainting/streetlight/painting_292_000524.jpg\npainting/streetlight/painting_292_000526.jpg\npainting/streetlight/painting_292_000533.jpg\npainting/streetlight/painting_292_000534.jpg\npainting/streetlight/painting_292_000537.jpg\npainting/string_bean/painting_293_000008.jpg\npainting/string_bean/painting_293_000026.jpg\npainting/string_bean/painting_293_000028.jpg\npainting/string_bean/painting_293_000038.jpg\npainting/string_bean/painting_293_000044.jpg\npainting/string_bean/painting_293_000045.jpg\npainting/string_bean/painting_293_000049.jpg\npainting/string_bean/painting_293_000050.jpg\npainting/string_bean/painting_293_000056.jpg\npainting/string_bean/painting_293_000057.jpg\npainting/string_bean/painting_293_000059.jpg\npainting/string_bean/painting_293_000061.jpg\npainting/submarine/painting_294_000055.jpg\npainting/submarine/painting_294_000056.jpg\npainting/submarine/painting_294_000067.jpg\npainting/submarine/painting_294_000071.jpg\npainting/submarine/painting_294_000095.jpg\npainting/submarine/painting_294_000102.jpg\npainting/submarine/painting_294_000133.jpg\npainting/submarine/painting_294_000140.jpg\npainting/submarine/painting_294_000157.jpg\npainting/submarine/painting_294_000178.jpg\npainting/submarine/painting_294_000181.jpg\npainting/submarine/painting_294_000185.jpg\npainting/submarine/painting_294_000186.jpg\npainting/submarine/painting_294_000189.jpg\npainting/submarine/painting_294_000190.jpg\npainting/submarine/painting_294_000196.jpg\npainting/submarine/painting_294_000202.jpg\npainting/submarine/painting_294_000206.jpg\npainting/submarine/painting_294_000209.jpg\npainting/submarine/painting_294_000212.jpg\npainting/submarine/painting_294_000231.jpg\npainting/submarine/painting_294_000232.jpg\npainting/submarine/painting_294_000239.jpg\npainting/submarine/painting_294_000242.jpg\npainting/submarine/painting_294_000247.jpg\npainting/submarine/painting_294_000253.jpg\npainting/submarine/painting_294_000258.jpg\npainting/submarine/painting_294_000259.jpg\npainting/submarine/painting_294_000262.jpg\npainting/submarine/painting_294_000264.jpg\npainting/submarine/painting_294_000271.jpg\npainting/submarine/painting_294_000273.jpg\npainting/submarine/painting_294_000275.jpg\npainting/submarine/painting_294_000277.jpg\npainting/submarine/painting_294_000287.jpg\npainting/submarine/painting_294_000289.jpg\npainting/submarine/painting_294_000291.jpg\npainting/submarine/painting_294_000293.jpg\npainting/submarine/painting_294_000306.jpg\npainting/submarine/painting_294_000313.jpg\npainting/submarine/painting_294_000314.jpg\npainting/submarine/painting_294_000317.jpg\npainting/submarine/painting_294_000324.jpg\npainting/submarine/painting_294_000326.jpg\npainting/submarine/painting_294_000327.jpg\npainting/submarine/painting_294_000330.jpg\npainting/submarine/painting_294_000331.jpg\npainting/submarine/painting_294_000332.jpg\npainting/submarine/painting_294_000337.jpg\npainting/submarine/painting_294_000345.jpg\npainting/submarine/painting_294_000346.jpg\npainting/submarine/painting_294_000353.jpg\npainting/submarine/painting_294_000358.jpg\npainting/submarine/painting_294_000360.jpg\npainting/submarine/painting_294_000362.jpg\npainting/submarine/painting_294_000363.jpg\npainting/submarine/painting_294_000366.jpg\npainting/submarine/painting_294_000377.jpg\npainting/submarine/painting_294_000378.jpg\npainting/submarine/painting_294_000380.jpg\npainting/submarine/painting_294_000382.jpg\npainting/submarine/painting_294_000383.jpg\npainting/submarine/painting_294_000384.jpg\npainting/submarine/painting_294_000388.jpg\npainting/submarine/painting_294_000389.jpg\npainting/submarine/painting_294_000390.jpg\npainting/submarine/painting_294_000392.jpg\npainting/submarine/painting_294_000394.jpg\npainting/submarine/painting_294_000396.jpg\npainting/submarine/painting_294_000397.jpg\npainting/submarine/painting_294_000399.jpg\npainting/submarine/painting_294_000401.jpg\npainting/submarine/painting_294_000408.jpg\npainting/submarine/painting_294_000409.jpg\npainting/submarine/painting_294_000411.jpg\npainting/submarine/painting_294_000413.jpg\npainting/submarine/painting_294_000414.jpg\npainting/submarine/painting_294_000415.jpg\npainting/submarine/painting_294_000416.jpg\npainting/submarine/painting_294_000417.jpg\npainting/submarine/painting_294_000418.jpg\npainting/submarine/painting_294_000420.jpg\npainting/submarine/painting_294_000421.jpg\npainting/submarine/painting_294_000429.jpg\npainting/submarine/painting_294_000431.jpg\npainting/submarine/painting_294_000432.jpg\npainting/submarine/painting_294_000434.jpg\npainting/submarine/painting_294_000435.jpg\npainting/submarine/painting_294_000438.jpg\npainting/submarine/painting_294_000444.jpg\npainting/submarine/painting_294_000445.jpg\npainting/submarine/painting_294_000448.jpg\npainting/submarine/painting_294_000449.jpg\npainting/submarine/painting_294_000452.jpg\npainting/submarine/painting_294_000453.jpg\npainting/submarine/painting_294_000455.jpg\npainting/submarine/painting_294_000456.jpg\npainting/submarine/painting_294_000458.jpg\npainting/submarine/painting_294_000460.jpg\npainting/submarine/painting_294_000464.jpg\npainting/submarine/painting_294_000467.jpg\npainting/submarine/painting_294_000469.jpg\npainting/submarine/painting_294_000472.jpg\npainting/submarine/painting_294_000477.jpg\npainting/submarine/painting_294_000478.jpg\npainting/submarine/painting_294_000484.jpg\npainting/submarine/painting_294_000485.jpg\npainting/submarine/painting_294_000486.jpg\npainting/submarine/painting_294_000487.jpg\npainting/submarine/painting_294_000488.jpg\npainting/submarine/painting_294_000490.jpg\npainting/submarine/painting_294_000491.jpg\npainting/submarine/painting_294_000494.jpg\npainting/submarine/painting_294_000498.jpg\npainting/submarine/painting_294_000500.jpg\npainting/submarine/painting_294_000503.jpg\npainting/submarine/painting_294_000504.jpg\npainting/submarine/painting_294_000505.jpg\npainting/submarine/painting_294_000507.jpg\npainting/submarine/painting_294_000511.jpg\npainting/submarine/painting_294_000513.jpg\npainting/submarine/painting_294_000516.jpg\npainting/submarine/painting_294_000517.jpg\npainting/submarine/painting_294_000518.jpg\npainting/submarine/painting_294_000519.jpg\npainting/submarine/painting_294_000523.jpg\npainting/submarine/painting_294_000524.jpg\npainting/submarine/painting_294_000527.jpg\npainting/submarine/painting_294_000528.jpg\npainting/submarine/painting_294_000530.jpg\npainting/submarine/painting_294_000531.jpg\npainting/submarine/painting_294_000533.jpg\npainting/submarine/painting_294_000534.jpg\npainting/submarine/painting_294_000536.jpg\npainting/submarine/painting_294_000537.jpg\npainting/submarine/painting_294_000539.jpg\npainting/submarine/painting_294_000541.jpg\npainting/submarine/painting_294_000544.jpg\npainting/submarine/painting_294_000545.jpg\npainting/submarine/painting_294_000547.jpg\npainting/submarine/painting_294_000548.jpg\npainting/submarine/painting_294_000549.jpg\npainting/submarine/painting_294_000550.jpg\npainting/suitcase/painting_295_000026.jpg\npainting/suitcase/painting_295_000032.jpg\npainting/suitcase/painting_295_000039.jpg\npainting/suitcase/painting_295_000061.jpg\npainting/suitcase/painting_295_000065.jpg\npainting/suitcase/painting_295_000072.jpg\npainting/suitcase/painting_295_000074.jpg\npainting/suitcase/painting_295_000079.jpg\npainting/suitcase/painting_295_000080.jpg\npainting/suitcase/painting_295_000086.jpg\npainting/suitcase/painting_295_000097.jpg\npainting/suitcase/painting_295_000100.jpg\npainting/suitcase/painting_295_000109.jpg\npainting/suitcase/painting_295_000111.jpg\npainting/suitcase/painting_295_000114.jpg\npainting/suitcase/painting_295_000116.jpg\npainting/suitcase/painting_295_000117.jpg\npainting/suitcase/painting_295_000118.jpg\npainting/suitcase/painting_295_000124.jpg\npainting/suitcase/painting_295_000128.jpg\npainting/suitcase/painting_295_000130.jpg\npainting/suitcase/painting_295_000137.jpg\npainting/suitcase/painting_295_000144.jpg\npainting/suitcase/painting_295_000146.jpg\npainting/suitcase/painting_295_000148.jpg\npainting/suitcase/painting_295_000151.jpg\npainting/suitcase/painting_295_000156.jpg\npainting/suitcase/painting_295_000159.jpg\npainting/suitcase/painting_295_000161.jpg\npainting/suitcase/painting_295_000162.jpg\npainting/suitcase/painting_295_000163.jpg\npainting/suitcase/painting_295_000165.jpg\npainting/suitcase/painting_295_000166.jpg\npainting/suitcase/painting_295_000167.jpg\npainting/suitcase/painting_295_000169.jpg\npainting/suitcase/painting_295_000170.jpg\npainting/suitcase/painting_295_000174.jpg\npainting/suitcase/painting_295_000175.jpg\npainting/suitcase/painting_295_000178.jpg\npainting/suitcase/painting_295_000182.jpg\npainting/suitcase/painting_295_000183.jpg\npainting/suitcase/painting_295_000185.jpg\npainting/suitcase/painting_295_000187.jpg\npainting/sun/painting_296_000010.jpg\npainting/sun/painting_296_000017.jpg\npainting/sun/painting_296_000030.jpg\npainting/sun/painting_296_000050.jpg\npainting/sun/painting_296_000067.jpg\npainting/sun/painting_296_000072.jpg\npainting/sun/painting_296_000090.jpg\npainting/sun/painting_296_000091.jpg\npainting/sun/painting_296_000095.jpg\npainting/sun/painting_296_000102.jpg\npainting/sun/painting_296_000110.jpg\npainting/sun/painting_296_000113.jpg\npainting/sun/painting_296_000118.jpg\npainting/sun/painting_296_000122.jpg\npainting/sun/painting_296_000133.jpg\npainting/sun/painting_296_000139.jpg\npainting/sun/painting_296_000140.jpg\npainting/sun/painting_296_000150.jpg\npainting/sun/painting_296_000155.jpg\npainting/sun/painting_296_000158.jpg\npainting/sun/painting_296_000163.jpg\npainting/sun/painting_296_000168.jpg\npainting/sun/painting_296_000169.jpg\npainting/sun/painting_296_000171.jpg\npainting/sun/painting_296_000174.jpg\npainting/sun/painting_296_000175.jpg\npainting/sun/painting_296_000178.jpg\npainting/sun/painting_296_000186.jpg\npainting/sun/painting_296_000190.jpg\npainting/sun/painting_296_000191.jpg\npainting/sun/painting_296_000192.jpg\npainting/sun/painting_296_000195.jpg\npainting/sun/painting_296_000198.jpg\npainting/sun/painting_296_000200.jpg\npainting/sun/painting_296_000202.jpg\npainting/sun/painting_296_000206.jpg\npainting/sun/painting_296_000208.jpg\npainting/sun/painting_296_000209.jpg\npainting/sun/painting_296_000211.jpg\npainting/sun/painting_296_000212.jpg\npainting/sun/painting_296_000213.jpg\npainting/sun/painting_296_000217.jpg\npainting/sun/painting_296_000218.jpg\npainting/sun/painting_296_000221.jpg\npainting/sun/painting_296_000224.jpg\npainting/sun/painting_296_000226.jpg\npainting/sun/painting_296_000237.jpg\npainting/sun/painting_296_000243.jpg\npainting/sun/painting_296_000245.jpg\npainting/sun/painting_296_000247.jpg\npainting/sun/painting_296_000251.jpg\npainting/sun/painting_296_000252.jpg\npainting/sun/painting_296_000253.jpg\npainting/sun/painting_296_000255.jpg\npainting/sun/painting_296_000258.jpg\npainting/sun/painting_296_000267.jpg\npainting/sun/painting_296_000269.jpg\npainting/sun/painting_296_000271.jpg\npainting/sun/painting_296_000272.jpg\npainting/sun/painting_296_000275.jpg\npainting/sun/painting_296_000280.jpg\npainting/sun/painting_296_000281.jpg\npainting/sun/painting_296_000283.jpg\npainting/sun/painting_296_000285.jpg\npainting/sun/painting_296_000287.jpg\npainting/sun/painting_296_000288.jpg\npainting/sun/painting_296_000289.jpg\npainting/sun/painting_296_000290.jpg\npainting/sun/painting_296_000294.jpg\npainting/sun/painting_296_000295.jpg\npainting/sun/painting_296_000297.jpg\npainting/sun/painting_296_000299.jpg\npainting/sun/painting_296_000300.jpg\npainting/sun/painting_296_000301.jpg\npainting/sun/painting_296_000302.jpg\npainting/sun/painting_296_000304.jpg\npainting/sun/painting_296_000305.jpg\npainting/sun/painting_296_000310.jpg\npainting/sun/painting_296_000313.jpg\npainting/sun/painting_296_000316.jpg\npainting/sun/painting_296_000317.jpg\npainting/sun/painting_296_000319.jpg\npainting/sun/painting_296_000320.jpg\npainting/sun/painting_296_000323.jpg\npainting/sun/painting_296_000324.jpg\npainting/sun/painting_296_000326.jpg\npainting/sun/painting_296_000327.jpg\npainting/sun/painting_296_000328.jpg\npainting/sun/painting_296_000330.jpg\npainting/sun/painting_296_000331.jpg\npainting/sun/painting_296_000332.jpg\npainting/sun/painting_296_000336.jpg\npainting/sun/painting_296_000337.jpg\npainting/sun/painting_296_000340.jpg\npainting/sun/painting_296_000342.jpg\npainting/sun/painting_296_000343.jpg\npainting/sun/painting_296_000345.jpg\npainting/sun/painting_296_000347.jpg\npainting/sun/painting_296_000350.jpg\npainting/sun/painting_296_000353.jpg\npainting/sun/painting_296_000354.jpg\npainting/sun/painting_296_000356.jpg\npainting/sun/painting_296_000358.jpg\npainting/sun/painting_296_000359.jpg\npainting/sun/painting_296_000361.jpg\npainting/sun/painting_296_000362.jpg\npainting/sun/painting_296_000363.jpg\npainting/sun/painting_296_000365.jpg\npainting/sun/painting_296_000367.jpg\npainting/sun/painting_296_000369.jpg\npainting/sun/painting_296_000373.jpg\npainting/sun/painting_296_000374.jpg\npainting/sun/painting_296_000375.jpg\npainting/sun/painting_296_000376.jpg\npainting/sun/painting_296_000377.jpg\npainting/sun/painting_296_000378.jpg\npainting/sun/painting_296_000379.jpg\npainting/sun/painting_296_000380.jpg\npainting/sun/painting_296_000381.jpg\npainting/sun/painting_296_000382.jpg\npainting/sun/painting_296_000388.jpg\npainting/sun/painting_296_000390.jpg\npainting/sun/painting_296_000392.jpg\npainting/sun/painting_296_000393.jpg\npainting/sun/painting_296_000396.jpg\npainting/sun/painting_296_000397.jpg\npainting/sun/painting_296_000398.jpg\npainting/sun/painting_296_000399.jpg\npainting/sun/painting_296_000402.jpg\npainting/sun/painting_296_000403.jpg\npainting/sun/painting_296_000404.jpg\npainting/sun/painting_296_000406.jpg\npainting/sun/painting_296_000408.jpg\npainting/sun/painting_296_000409.jpg\npainting/sun/painting_296_000410.jpg\npainting/sun/painting_296_000413.jpg\npainting/sun/painting_296_000416.jpg\npainting/sun/painting_296_000419.jpg\npainting/sun/painting_296_000421.jpg\npainting/sun/painting_296_000422.jpg\npainting/sun/painting_296_000423.jpg\npainting/sun/painting_296_000424.jpg\npainting/sun/painting_296_000425.jpg\npainting/sun/painting_296_000427.jpg\npainting/sun/painting_296_000428.jpg\npainting/sun/painting_296_000430.jpg\npainting/sun/painting_296_000432.jpg\npainting/sun/painting_296_000434.jpg\npainting/sun/painting_296_000436.jpg\npainting/sun/painting_296_000439.jpg\npainting/sun/painting_296_000440.jpg\npainting/sun/painting_296_000442.jpg\npainting/sun/painting_296_000445.jpg\npainting/sun/painting_296_000447.jpg\npainting/sun/painting_296_000448.jpg\npainting/sun/painting_296_000449.jpg\npainting/sun/painting_296_000450.jpg\npainting/sun/painting_296_000451.jpg\npainting/sun/painting_296_000453.jpg\npainting/sun/painting_296_000456.jpg\npainting/sun/painting_296_000458.jpg\npainting/sun/painting_296_000460.jpg\npainting/sun/painting_296_000461.jpg\npainting/sun/painting_296_000462.jpg\npainting/sun/painting_296_000463.jpg\npainting/sun/painting_296_000464.jpg\npainting/sun/painting_296_000465.jpg\npainting/sun/painting_296_000468.jpg\npainting/sun/painting_296_000470.jpg\npainting/sun/painting_296_000471.jpg\npainting/sun/painting_296_000472.jpg\npainting/sun/painting_296_000473.jpg\npainting/sun/painting_296_000476.jpg\npainting/sun/painting_296_000477.jpg\npainting/sun/painting_296_000480.jpg\npainting/sun/painting_296_000482.jpg\npainting/sun/painting_296_000483.jpg\npainting/sun/painting_296_000484.jpg\npainting/sun/painting_296_000486.jpg\npainting/sun/painting_296_000487.jpg\npainting/sun/painting_296_000489.jpg\npainting/sun/painting_296_000490.jpg\npainting/sun/painting_296_000493.jpg\npainting/sun/painting_296_000497.jpg\npainting/sun/painting_296_000498.jpg\npainting/sun/painting_296_000499.jpg\npainting/sun/painting_296_000500.jpg\npainting/sun/painting_296_000502.jpg\npainting/sun/painting_296_000505.jpg\npainting/sun/painting_296_000509.jpg\npainting/sun/painting_296_000511.jpg\npainting/sun/painting_296_000512.jpg\npainting/sun/painting_296_000513.jpg\npainting/sun/painting_296_000515.jpg\npainting/sun/painting_296_000517.jpg\npainting/sun/painting_296_000519.jpg\npainting/sun/painting_296_000520.jpg\npainting/sun/painting_296_000523.jpg\npainting/sun/painting_296_000524.jpg\npainting/sun/painting_296_000525.jpg\npainting/sun/painting_296_000526.jpg\npainting/sun/painting_296_000527.jpg\npainting/sun/painting_296_000528.jpg\npainting/sun/painting_296_000529.jpg\npainting/sun/painting_296_000530.jpg\npainting/sun/painting_296_000531.jpg\npainting/sun/painting_296_000533.jpg\npainting/sun/painting_296_000534.jpg\npainting/sun/painting_296_000535.jpg\npainting/sun/painting_296_000537.jpg\npainting/sun/painting_296_000538.jpg\npainting/sun/painting_296_000539.jpg\npainting/sun/painting_296_000540.jpg\npainting/sun/painting_296_000541.jpg\npainting/sun/painting_296_000542.jpg\npainting/sun/painting_296_000543.jpg\npainting/sun/painting_296_000544.jpg\npainting/sun/painting_296_000545.jpg\npainting/sun/painting_296_000546.jpg\npainting/sun/painting_296_000548.jpg\npainting/sun/painting_296_000550.jpg\npainting/sun/painting_296_000551.jpg\npainting/sun/painting_296_000552.jpg\npainting/sun/painting_296_000555.jpg\npainting/sun/painting_296_000556.jpg\npainting/sun/painting_296_000557.jpg\npainting/sun/painting_296_000558.jpg\npainting/sun/painting_296_000559.jpg\npainting/sun/painting_296_000560.jpg\npainting/sun/painting_296_000561.jpg\npainting/sun/painting_296_000562.jpg\npainting/sun/painting_296_000563.jpg\npainting/sun/painting_296_000565.jpg\npainting/sun/painting_296_000566.jpg\npainting/sun/painting_296_000567.jpg\npainting/sun/painting_296_000568.jpg\npainting/sun/painting_296_000569.jpg\npainting/sun/painting_296_000570.jpg\npainting/sun/painting_296_000572.jpg\npainting/swan/painting_297_000009.jpg\npainting/swan/painting_297_000034.jpg\npainting/swan/painting_297_000049.jpg\npainting/swan/painting_297_000052.jpg\npainting/swan/painting_297_000075.jpg\npainting/swan/painting_297_000083.jpg\npainting/swan/painting_297_000087.jpg\npainting/swan/painting_297_000100.jpg\npainting/swan/painting_297_000101.jpg\npainting/swan/painting_297_000109.jpg\npainting/swan/painting_297_000111.jpg\npainting/swan/painting_297_000121.jpg\npainting/swan/painting_297_000123.jpg\npainting/swan/painting_297_000133.jpg\npainting/swan/painting_297_000136.jpg\npainting/swan/painting_297_000139.jpg\npainting/swan/painting_297_000141.jpg\npainting/swan/painting_297_000153.jpg\npainting/swan/painting_297_000155.jpg\npainting/swan/painting_297_000157.jpg\npainting/swan/painting_297_000158.jpg\npainting/swan/painting_297_000163.jpg\npainting/swan/painting_297_000166.jpg\npainting/swan/painting_297_000172.jpg\npainting/swan/painting_297_000177.jpg\npainting/swan/painting_297_000178.jpg\npainting/swan/painting_297_000180.jpg\npainting/swan/painting_297_000191.jpg\npainting/swan/painting_297_000196.jpg\npainting/swan/painting_297_000197.jpg\npainting/swan/painting_297_000202.jpg\npainting/swan/painting_297_000204.jpg\npainting/swan/painting_297_000215.jpg\npainting/swan/painting_297_000217.jpg\npainting/swan/painting_297_000219.jpg\npainting/swan/painting_297_000220.jpg\npainting/swan/painting_297_000224.jpg\npainting/swan/painting_297_000226.jpg\npainting/swan/painting_297_000230.jpg\npainting/swan/painting_297_000232.jpg\npainting/swan/painting_297_000234.jpg\npainting/swan/painting_297_000235.jpg\npainting/swan/painting_297_000236.jpg\npainting/swan/painting_297_000239.jpg\npainting/swan/painting_297_000250.jpg\npainting/swan/painting_297_000253.jpg\npainting/swan/painting_297_000257.jpg\npainting/swan/painting_297_000259.jpg\npainting/swan/painting_297_000265.jpg\npainting/swan/painting_297_000271.jpg\npainting/swan/painting_297_000272.jpg\npainting/swan/painting_297_000273.jpg\npainting/swan/painting_297_000278.jpg\npainting/swan/painting_297_000280.jpg\npainting/swan/painting_297_000282.jpg\npainting/swan/painting_297_000287.jpg\npainting/swan/painting_297_000288.jpg\npainting/swan/painting_297_000291.jpg\npainting/swan/painting_297_000294.jpg\npainting/swan/painting_297_000295.jpg\npainting/swan/painting_297_000297.jpg\npainting/swan/painting_297_000298.jpg\npainting/swan/painting_297_000299.jpg\npainting/swan/painting_297_000302.jpg\npainting/swan/painting_297_000308.jpg\npainting/swan/painting_297_000312.jpg\npainting/swan/painting_297_000316.jpg\npainting/swan/painting_297_000317.jpg\npainting/swan/painting_297_000320.jpg\npainting/swan/painting_297_000322.jpg\npainting/swan/painting_297_000325.jpg\npainting/swan/painting_297_000326.jpg\npainting/swan/painting_297_000329.jpg\npainting/swan/painting_297_000330.jpg\npainting/swan/painting_297_000331.jpg\npainting/swan/painting_297_000332.jpg\npainting/swan/painting_297_000333.jpg\npainting/swan/painting_297_000338.jpg\npainting/swan/painting_297_000339.jpg\npainting/swan/painting_297_000340.jpg\npainting/swan/painting_297_000342.jpg\npainting/swan/painting_297_000346.jpg\npainting/swan/painting_297_000347.jpg\npainting/swan/painting_297_000348.jpg\npainting/swan/painting_297_000349.jpg\npainting/swan/painting_297_000350.jpg\npainting/swan/painting_297_000351.jpg\npainting/swan/painting_297_000356.jpg\npainting/swan/painting_297_000357.jpg\npainting/swan/painting_297_000358.jpg\npainting/swan/painting_297_000359.jpg\npainting/swan/painting_297_000360.jpg\npainting/swan/painting_297_000363.jpg\npainting/swan/painting_297_000364.jpg\npainting/swan/painting_297_000365.jpg\npainting/swan/painting_297_000366.jpg\npainting/swan/painting_297_000369.jpg\npainting/swan/painting_297_000371.jpg\npainting/swan/painting_297_000372.jpg\npainting/swan/painting_297_000373.jpg\npainting/swan/painting_297_000374.jpg\npainting/swan/painting_297_000375.jpg\npainting/swan/painting_297_000377.jpg\npainting/swan/painting_297_000379.jpg\npainting/swan/painting_297_000381.jpg\npainting/swan/painting_297_000384.jpg\npainting/swan/painting_297_000385.jpg\npainting/swan/painting_297_000388.jpg\npainting/swan/painting_297_000389.jpg\npainting/swan/painting_297_000391.jpg\npainting/swan/painting_297_000392.jpg\npainting/swan/painting_297_000395.jpg\npainting/swan/painting_297_000396.jpg\npainting/swan/painting_297_000402.jpg\npainting/swan/painting_297_000403.jpg\npainting/swan/painting_297_000404.jpg\npainting/swan/painting_297_000406.jpg\npainting/swan/painting_297_000407.jpg\npainting/swan/painting_297_000409.jpg\npainting/swan/painting_297_000411.jpg\npainting/swan/painting_297_000413.jpg\npainting/swan/painting_297_000414.jpg\npainting/swan/painting_297_000415.jpg\npainting/swan/painting_297_000417.jpg\npainting/swan/painting_297_000420.jpg\npainting/swan/painting_297_000422.jpg\npainting/swan/painting_297_000423.jpg\npainting/swan/painting_297_000427.jpg\npainting/swan/painting_297_000429.jpg\npainting/swan/painting_297_000430.jpg\npainting/swan/painting_297_000431.jpg\npainting/swan/painting_297_000432.jpg\npainting/swan/painting_297_000433.jpg\npainting/swan/painting_297_000434.jpg\npainting/swan/painting_297_000438.jpg\npainting/swan/painting_297_000440.jpg\npainting/swan/painting_297_000441.jpg\npainting/swan/painting_297_000442.jpg\npainting/swan/painting_297_000445.jpg\npainting/swan/painting_297_000453.jpg\npainting/swan/painting_297_000454.jpg\npainting/swan/painting_297_000455.jpg\npainting/swan/painting_297_000457.jpg\npainting/swan/painting_297_000459.jpg\npainting/swan/painting_297_000460.jpg\npainting/swan/painting_297_000461.jpg\npainting/swan/painting_297_000463.jpg\npainting/swan/painting_297_000465.jpg\npainting/swan/painting_297_000466.jpg\npainting/swan/painting_297_000467.jpg\npainting/swan/painting_297_000470.jpg\npainting/swan/painting_297_000471.jpg\npainting/swan/painting_297_000473.jpg\npainting/swan/painting_297_000475.jpg\npainting/swan/painting_297_000476.jpg\npainting/swan/painting_297_000478.jpg\npainting/swan/painting_297_000479.jpg\npainting/swan/painting_297_000483.jpg\npainting/swan/painting_297_000484.jpg\npainting/swan/painting_297_000486.jpg\npainting/swan/painting_297_000487.jpg\npainting/swan/painting_297_000489.jpg\npainting/swan/painting_297_000490.jpg\npainting/swan/painting_297_000491.jpg\npainting/swan/painting_297_000493.jpg\npainting/swan/painting_297_000494.jpg\npainting/swan/painting_297_000496.jpg\npainting/swan/painting_297_000498.jpg\npainting/swan/painting_297_000499.jpg\npainting/swan/painting_297_000501.jpg\npainting/swan/painting_297_000502.jpg\npainting/swan/painting_297_000503.jpg\npainting/swan/painting_297_000504.jpg\npainting/swan/painting_297_000505.jpg\npainting/swan/painting_297_000507.jpg\npainting/sweater/painting_298_000013.jpg\npainting/sweater/painting_298_000014.jpg\npainting/sweater/painting_298_000032.jpg\npainting/sweater/painting_298_000034.jpg\npainting/sweater/painting_298_000039.jpg\npainting/sweater/painting_298_000040.jpg\npainting/sweater/painting_298_000046.jpg\npainting/sweater/painting_298_000054.jpg\npainting/sweater/painting_298_000063.jpg\npainting/sweater/painting_298_000069.jpg\npainting/sweater/painting_298_000073.jpg\npainting/sweater/painting_298_000077.jpg\npainting/sweater/painting_298_000079.jpg\npainting/sweater/painting_298_000088.jpg\npainting/sweater/painting_298_000091.jpg\npainting/sweater/painting_298_000095.jpg\npainting/sweater/painting_298_000098.jpg\npainting/sweater/painting_298_000100.jpg\npainting/sweater/painting_298_000101.jpg\npainting/sweater/painting_298_000102.jpg\npainting/sweater/painting_298_000103.jpg\npainting/sweater/painting_298_000106.jpg\npainting/sweater/painting_298_000107.jpg\npainting/sweater/painting_298_000111.jpg\npainting/sweater/painting_298_000113.jpg\npainting/sweater/painting_298_000114.jpg\npainting/sweater/painting_298_000116.jpg\npainting/sweater/painting_298_000118.jpg\npainting/sweater/painting_298_000122.jpg\npainting/sweater/painting_298_000124.jpg\npainting/sweater/painting_298_000126.jpg\npainting/sweater/painting_298_000128.jpg\npainting/sweater/painting_298_000129.jpg\npainting/sweater/painting_298_000131.jpg\npainting/sweater/painting_298_000133.jpg\npainting/sweater/painting_298_000134.jpg\npainting/sweater/painting_298_000136.jpg\npainting/sweater/painting_298_000138.jpg\npainting/sweater/painting_298_000139.jpg\npainting/sweater/painting_298_000140.jpg\npainting/sweater/painting_298_000142.jpg\npainting/sweater/painting_298_000143.jpg\npainting/sweater/painting_298_000145.jpg\npainting/sweater/painting_298_000147.jpg\npainting/sweater/painting_298_000149.jpg\npainting/sweater/painting_298_000150.jpg\npainting/sweater/painting_298_000151.jpg\npainting/sweater/painting_298_000153.jpg\npainting/swing_set/painting_299_000020.jpg\npainting/swing_set/painting_299_000036.jpg\npainting/swing_set/painting_299_000052.jpg\npainting/swing_set/painting_299_000057.jpg\npainting/swing_set/painting_299_000058.jpg\npainting/swing_set/painting_299_000059.jpg\npainting/swing_set/painting_299_000064.jpg\npainting/swing_set/painting_299_000065.jpg\npainting/swing_set/painting_299_000069.jpg\npainting/swing_set/painting_299_000070.jpg\npainting/swing_set/painting_299_000073.jpg\npainting/swing_set/painting_299_000076.jpg\npainting/swing_set/painting_299_000081.jpg\npainting/swing_set/painting_299_000082.jpg\npainting/swing_set/painting_299_000085.jpg\npainting/swing_set/painting_299_000087.jpg\npainting/swing_set/painting_299_000091.jpg\npainting/swing_set/painting_299_000099.jpg\npainting/swing_set/painting_299_000100.jpg\npainting/swing_set/painting_299_000101.jpg\npainting/swing_set/painting_299_000110.jpg\npainting/swing_set/painting_299_000111.jpg\npainting/swing_set/painting_299_000113.jpg\npainting/swing_set/painting_299_000116.jpg\npainting/swing_set/painting_299_000117.jpg\npainting/swing_set/painting_299_000122.jpg\npainting/swing_set/painting_299_000127.jpg\npainting/swing_set/painting_299_000129.jpg\npainting/sword/painting_300_000041.jpg\npainting/sword/painting_300_000044.jpg\npainting/sword/painting_300_000045.jpg\npainting/sword/painting_300_000050.jpg\npainting/sword/painting_300_000058.jpg\npainting/sword/painting_300_000067.jpg\npainting/sword/painting_300_000078.jpg\npainting/sword/painting_300_000088.jpg\npainting/sword/painting_300_000089.jpg\npainting/sword/painting_300_000107.jpg\npainting/sword/painting_300_000119.jpg\npainting/sword/painting_300_000131.jpg\npainting/sword/painting_300_000133.jpg\npainting/sword/painting_300_000137.jpg\npainting/sword/painting_300_000138.jpg\npainting/sword/painting_300_000146.jpg\npainting/sword/painting_300_000147.jpg\npainting/sword/painting_300_000150.jpg\npainting/sword/painting_300_000155.jpg\npainting/sword/painting_300_000156.jpg\npainting/sword/painting_300_000157.jpg\npainting/sword/painting_300_000161.jpg\npainting/sword/painting_300_000171.jpg\npainting/sword/painting_300_000174.jpg\npainting/sword/painting_300_000182.jpg\npainting/sword/painting_300_000183.jpg\npainting/sword/painting_300_000186.jpg\npainting/sword/painting_300_000187.jpg\npainting/sword/painting_300_000190.jpg\npainting/sword/painting_300_000195.jpg\npainting/sword/painting_300_000196.jpg\npainting/sword/painting_300_000197.jpg\npainting/sword/painting_300_000201.jpg\npainting/sword/painting_300_000207.jpg\npainting/sword/painting_300_000222.jpg\npainting/sword/painting_300_000226.jpg\npainting/sword/painting_300_000229.jpg\npainting/sword/painting_300_000232.jpg\npainting/sword/painting_300_000236.jpg\npainting/sword/painting_300_000240.jpg\npainting/sword/painting_300_000241.jpg\npainting/sword/painting_300_000244.jpg\npainting/sword/painting_300_000246.jpg\npainting/sword/painting_300_000250.jpg\npainting/sword/painting_300_000251.jpg\npainting/sword/painting_300_000254.jpg\npainting/sword/painting_300_000257.jpg\npainting/sword/painting_300_000262.jpg\npainting/sword/painting_300_000263.jpg\npainting/sword/painting_300_000266.jpg\npainting/sword/painting_300_000268.jpg\npainting/sword/painting_300_000269.jpg\npainting/sword/painting_300_000272.jpg\npainting/sword/painting_300_000273.jpg\npainting/sword/painting_300_000277.jpg\npainting/sword/painting_300_000280.jpg\npainting/sword/painting_300_000282.jpg\npainting/sword/painting_300_000286.jpg\npainting/sword/painting_300_000287.jpg\npainting/sword/painting_300_000293.jpg\npainting/sword/painting_300_000294.jpg\npainting/sword/painting_300_000295.jpg\npainting/sword/painting_300_000297.jpg\npainting/sword/painting_300_000298.jpg\npainting/sword/painting_300_000299.jpg\npainting/sword/painting_300_000305.jpg\npainting/sword/painting_300_000311.jpg\npainting/sword/painting_300_000313.jpg\npainting/sword/painting_300_000314.jpg\npainting/sword/painting_300_000315.jpg\npainting/sword/painting_300_000318.jpg\npainting/sword/painting_300_000320.jpg\npainting/sword/painting_300_000321.jpg\npainting/sword/painting_300_000325.jpg\npainting/sword/painting_300_000326.jpg\npainting/sword/painting_300_000327.jpg\npainting/sword/painting_300_000330.jpg\npainting/sword/painting_300_000332.jpg\npainting/sword/painting_300_000334.jpg\npainting/sword/painting_300_000335.jpg\npainting/sword/painting_300_000338.jpg\npainting/sword/painting_300_000339.jpg\npainting/sword/painting_300_000341.jpg\npainting/sword/painting_300_000343.jpg\npainting/sword/painting_300_000344.jpg\npainting/sword/painting_300_000346.jpg\npainting/sword/painting_300_000347.jpg\npainting/sword/painting_300_000349.jpg\npainting/sword/painting_300_000350.jpg\npainting/sword/painting_300_000352.jpg\npainting/sword/painting_300_000354.jpg\npainting/sword/painting_300_000356.jpg\npainting/sword/painting_300_000359.jpg\npainting/sword/painting_300_000361.jpg\npainting/sword/painting_300_000362.jpg\npainting/sword/painting_300_000363.jpg\npainting/sword/painting_300_000364.jpg\npainting/sword/painting_300_000365.jpg\npainting/sword/painting_300_000366.jpg\npainting/sword/painting_300_000367.jpg\npainting/sword/painting_300_000370.jpg\npainting/sword/painting_300_000371.jpg\npainting/sword/painting_300_000374.jpg\npainting/sword/painting_300_000375.jpg\npainting/sword/painting_300_000376.jpg\npainting/sword/painting_300_000378.jpg\npainting/sword/painting_300_000381.jpg\npainting/sword/painting_300_000383.jpg\npainting/sword/painting_300_000387.jpg\npainting/sword/painting_300_000388.jpg\npainting/sword/painting_300_000390.jpg\npainting/sword/painting_300_000394.jpg\npainting/sword/painting_300_000395.jpg\npainting/sword/painting_300_000397.jpg\npainting/sword/painting_300_000398.jpg\npainting/sword/painting_300_000400.jpg\npainting/sword/painting_300_000401.jpg\npainting/sword/painting_300_000402.jpg\npainting/sword/painting_300_000403.jpg\npainting/sword/painting_300_000404.jpg\npainting/sword/painting_300_000408.jpg\npainting/sword/painting_300_000410.jpg\npainting/sword/painting_300_000411.jpg\npainting/sword/painting_300_000412.jpg\npainting/sword/painting_300_000413.jpg\npainting/sword/painting_300_000415.jpg\npainting/sword/painting_300_000417.jpg\npainting/sword/painting_300_000418.jpg\npainting/sword/painting_300_000419.jpg\npainting/sword/painting_300_000421.jpg\npainting/sword/painting_300_000423.jpg\npainting/sword/painting_300_000424.jpg\npainting/sword/painting_300_000426.jpg\npainting/sword/painting_300_000430.jpg\npainting/sword/painting_300_000431.jpg\npainting/sword/painting_300_000433.jpg\npainting/sword/painting_300_000434.jpg\npainting/sword/painting_300_000435.jpg\npainting/sword/painting_300_000437.jpg\npainting/sword/painting_300_000438.jpg\npainting/sword/painting_300_000439.jpg\npainting/sword/painting_300_000440.jpg\npainting/sword/painting_300_000441.jpg\npainting/sword/painting_300_000442.jpg\npainting/sword/painting_300_000444.jpg\npainting/sword/painting_300_000445.jpg\npainting/sword/painting_300_000447.jpg\npainting/sword/painting_300_000449.jpg\npainting/sword/painting_300_000450.jpg\npainting/sword/painting_300_000451.jpg\npainting/sword/painting_300_000453.jpg\npainting/sword/painting_300_000455.jpg\npainting/sword/painting_300_000456.jpg\npainting/sword/painting_300_000457.jpg\npainting/sword/painting_300_000458.jpg\npainting/sword/painting_300_000459.jpg\npainting/sword/painting_300_000460.jpg\npainting/sword/painting_300_000462.jpg\npainting/sword/painting_300_000464.jpg\npainting/sword/painting_300_000465.jpg\npainting/sword/painting_300_000466.jpg\npainting/sword/painting_300_000467.jpg\npainting/sword/painting_300_000470.jpg\npainting/syringe/painting_301_000006.jpg\npainting/syringe/painting_301_000007.jpg\npainting/syringe/painting_301_000008.jpg\npainting/syringe/painting_301_000009.jpg\npainting/syringe/painting_301_000010.jpg\npainting/table/painting_302_000006.jpg\npainting/table/painting_302_000016.jpg\npainting/table/painting_302_000054.jpg\npainting/teapot/painting_303_000058.jpg\npainting/teapot/painting_303_000115.jpg\npainting/teapot/painting_303_000133.jpg\npainting/teapot/painting_303_000224.jpg\npainting/teapot/painting_303_000353.jpg\npainting/teapot/painting_303_000389.jpg\npainting/teddy-bear/painting_304_000001.jpg\npainting/teddy-bear/painting_304_000020.jpg\npainting/teddy-bear/painting_304_000021.jpg\npainting/teddy-bear/painting_304_000024.jpg\npainting/teddy-bear/painting_304_000025.jpg\npainting/teddy-bear/painting_304_000026.jpg\npainting/teddy-bear/painting_304_000029.jpg\npainting/teddy-bear/painting_304_000033.jpg\npainting/teddy-bear/painting_304_000036.jpg\npainting/teddy-bear/painting_304_000043.jpg\npainting/teddy-bear/painting_304_000048.jpg\npainting/teddy-bear/painting_304_000049.jpg\npainting/teddy-bear/painting_304_000056.jpg\npainting/teddy-bear/painting_304_000061.jpg\npainting/teddy-bear/painting_304_000065.jpg\npainting/teddy-bear/painting_304_000072.jpg\npainting/teddy-bear/painting_304_000080.jpg\npainting/teddy-bear/painting_304_000081.jpg\npainting/teddy-bear/painting_304_000087.jpg\npainting/teddy-bear/painting_304_000098.jpg\npainting/teddy-bear/painting_304_000109.jpg\npainting/teddy-bear/painting_304_000112.jpg\npainting/teddy-bear/painting_304_000118.jpg\npainting/teddy-bear/painting_304_000148.jpg\npainting/teddy-bear/painting_304_000151.jpg\npainting/teddy-bear/painting_304_000161.jpg\npainting/telephone/painting_305_000076.jpg\npainting/television/painting_306_000035.jpg\npainting/tent/painting_308_000095.jpg\npainting/The_Eiffel_Tower/painting_309_000019.jpg\npainting/The_Eiffel_Tower/painting_309_000063.jpg\npainting/The_Eiffel_Tower/painting_309_000125.jpg\npainting/The_Eiffel_Tower/painting_309_000127.jpg\npainting/The_Great_Wall_of_China/painting_310_000017.jpg\npainting/The_Great_Wall_of_China/painting_310_000042.jpg\npainting/The_Great_Wall_of_China/painting_310_000049.jpg\npainting/The_Great_Wall_of_China/painting_310_000054.jpg\npainting/The_Mona_Lisa/painting_311_000009.jpg\npainting/The_Mona_Lisa/painting_311_000028.jpg\npainting/The_Mona_Lisa/painting_311_000036.jpg\npainting/The_Mona_Lisa/painting_311_000039.jpg\npainting/The_Mona_Lisa/painting_311_000053.jpg\npainting/The_Mona_Lisa/painting_311_000081.jpg\npainting/tiger/painting_312_000256.jpg\npainting/tiger/painting_312_000395.jpg\npainting/toaster/painting_313_000060.jpg\npainting/toaster/painting_313_000061.jpg\npainting/toaster/painting_313_000094.jpg\npainting/toe/painting_314_000003.jpg\npainting/toe/painting_314_000004.jpg\npainting/toe/painting_314_000005.jpg\npainting/toe/painting_314_000006.jpg\npainting/toe/painting_314_000007.jpg\npainting/toe/painting_314_000008.jpg\npainting/toe/painting_314_000009.jpg\npainting/toe/painting_314_000010.jpg\npainting/toe/painting_314_000011.jpg\npainting/toe/painting_314_000012.jpg\npainting/toothbrush/painting_317_000001.jpg\npainting/toothpaste/painting_318_000011.jpg\npainting/toothpaste/painting_318_000028.jpg\npainting/tornado/painting_319_000115.jpg\npainting/tornado/painting_319_000124.jpg\npainting/tornado/painting_319_000310.jpg\npainting/tornado/painting_319_000326.jpg\npainting/tractor/painting_320_000146.jpg\npainting/traffic_light/painting_321_000020.jpg\npainting/traffic_light/painting_321_000031.jpg\npainting/traffic_light/painting_321_000050.jpg\npainting/train/painting_322_000210.jpg\npainting/train/painting_322_000325.jpg\npainting/tree/painting_323_000056.jpg\npainting/tree/painting_323_000070.jpg\npainting/tree/painting_323_000092.jpg\npainting/tree/painting_323_000113.jpg\npainting/tree/painting_323_000125.jpg\npainting/tree/painting_323_000140.jpg\npainting/tree/painting_323_000171.jpg\npainting/tree/painting_323_000182.jpg\npainting/tree/painting_323_000247.jpg\npainting/tree/painting_323_000270.jpg\npainting/tree/painting_323_000284.jpg\npainting/tree/painting_323_000301.jpg\npainting/tree/painting_323_000316.jpg\npainting/tree/painting_323_000324.jpg\npainting/tree/painting_323_000417.jpg\npainting/tree/painting_323_000421.jpg\npainting/tree/painting_323_000464.jpg\npainting/tree/painting_323_000469.jpg\npainting/tree/painting_323_000482.jpg\npainting/tree/painting_323_000566.jpg\npainting/triangle/painting_324_000230.jpg\npainting/truck/painting_326_000001.jpg\npainting/truck/painting_326_000002.jpg\npainting/truck/painting_326_000007.jpg\npainting/truck/painting_326_000010.jpg\npainting/truck/painting_326_000012.jpg\npainting/truck/painting_326_000017.jpg\npainting/truck/painting_326_000019.jpg\npainting/truck/painting_326_000020.jpg\npainting/truck/painting_326_000021.jpg\npainting/truck/painting_326_000022.jpg\npainting/truck/painting_326_000025.jpg\npainting/truck/painting_326_000031.jpg\npainting/truck/painting_326_000032.jpg\npainting/truck/painting_326_000040.jpg\npainting/truck/painting_326_000045.jpg\npainting/truck/painting_326_000046.jpg\npainting/truck/painting_326_000050.jpg\npainting/truck/painting_326_000055.jpg\npainting/truck/painting_326_000065.jpg\npainting/truck/painting_326_000074.jpg\npainting/truck/painting_326_000075.jpg\npainting/truck/painting_326_000076.jpg\npainting/truck/painting_326_000084.jpg\npainting/truck/painting_326_000091.jpg\npainting/truck/painting_326_000093.jpg\npainting/truck/painting_326_000094.jpg\npainting/truck/painting_326_000099.jpg\npainting/truck/painting_326_000102.jpg\npainting/truck/painting_326_000109.jpg\npainting/truck/painting_326_000110.jpg\npainting/truck/painting_326_000111.jpg\npainting/truck/painting_326_000113.jpg\npainting/truck/painting_326_000115.jpg\npainting/truck/painting_326_000117.jpg\npainting/truck/painting_326_000118.jpg\npainting/trumpet/painting_327_000083.jpg\npainting/t-shirt/painting_328_000002.jpg\npainting/t-shirt/painting_328_000004.jpg\npainting/t-shirt/painting_328_000005.jpg\npainting/t-shirt/painting_328_000006.jpg\npainting/t-shirt/painting_328_000007.jpg\npainting/t-shirt/painting_328_000008.jpg\npainting/t-shirt/painting_328_000009.jpg\npainting/t-shirt/painting_328_000010.jpg\npainting/t-shirt/painting_328_000011.jpg\npainting/t-shirt/painting_328_000012.jpg\npainting/umbrella/painting_329_000013.jpg\npainting/umbrella/painting_329_000019.jpg\npainting/umbrella/painting_329_000039.jpg\npainting/umbrella/painting_329_000052.jpg\npainting/umbrella/painting_329_000071.jpg\npainting/umbrella/painting_329_000078.jpg\npainting/umbrella/painting_329_000084.jpg\npainting/umbrella/painting_329_000098.jpg\npainting/umbrella/painting_329_000106.jpg\npainting/umbrella/painting_329_000205.jpg\npainting/umbrella/painting_329_000207.jpg\npainting/umbrella/painting_329_000218.jpg\npainting/underwear/painting_330_000010.jpg\npainting/underwear/painting_330_000011.jpg\npainting/underwear/painting_330_000012.jpg\npainting/vase/painting_332_000093.jpg\npainting/vase/painting_332_000182.jpg\npainting/vase/painting_332_000230.jpg\npainting/violin/painting_333_000011.jpg\npainting/violin/painting_333_000058.jpg\npainting/violin/painting_333_000158.jpg\npainting/violin/painting_333_000197.jpg\npainting/watermelon/painting_335_000208.jpg\npainting/waterslide/painting_336_000009.jpg\npainting/waterslide/painting_336_000010.jpg\npainting/waterslide/painting_336_000011.jpg\npainting/waterslide/painting_336_000012.jpg\npainting/whale/painting_337_000172.jpg\npainting/whale/painting_337_000283.jpg\npainting/whale/painting_337_000323.jpg\npainting/windmill/painting_339_000073.jpg\npainting/wine_glass/painting_341_000041.jpg\npainting/wine_glass/painting_341_000049.jpg\npainting/wine_glass/painting_341_000102.jpg\npainting/wine_glass/painting_341_000125.jpg\npainting/wine_glass/painting_341_000126.jpg\npainting/wine_glass/painting_341_000151.jpg\npainting/wristwatch/painting_342_000018.jpg\npainting/zebra/painting_344_000149.jpg\nreal/asparagus/real_011_000194.jpg\nreal/asparagus/real_011_000593.jpg\nreal/asparagus/real_011_000615.jpg\nreal/banana/real_014_000223.jpg\nreal/bandage/real_015_000046.jpg\nreal/bandage/real_015_000077.jpg\nreal/bandage/real_015_000153.jpg\nreal/bandage/real_015_000155.jpg\nreal/bandage/real_015_000198.jpg\nreal/bandage/real_015_000208.jpg\nreal/barn/real_016_000003.jpg\nreal/baseball_bat/real_018_000023.jpg\nreal/baseball_bat/real_018_000064.jpg\nreal/baseball_bat/real_018_000097.jpg\nreal/basket/real_019_000426.jpg\nreal/bat/real_021_000057.jpg\nreal/bee/real_027_000022.jpg\nreal/bee/real_027_000095.jpg\nreal/bee/real_027_000100.jpg\nreal/bee/real_027_000160.jpg\nreal/bee/real_027_000225.jpg\nreal/bee/real_027_000226.jpg\nreal/bee/real_027_000251.jpg\nreal/bee/real_027_000319.jpg\nreal/bee/real_027_000330.jpg\nreal/bee/real_027_000335.jpg\nreal/bee/real_027_000367.jpg\nreal/bee/real_027_000371.jpg\nreal/bee/real_027_000378.jpg\nreal/bee/real_027_000433.jpg\nreal/belt/real_028_000613.jpg\nreal/bench/real_029_000152.jpg\nreal/bench/real_029_000200.jpg\nreal/bicycle/real_030_000229.jpg\nreal/binoculars/real_031_000040.jpg\nreal/binoculars/real_031_000057.jpg\nreal/binoculars/real_031_000076.jpg\nreal/binoculars/real_031_000259.jpg\nreal/bird/real_032_000313.jpg\nreal/bird/real_032_000411.jpg\nreal/bird/real_032_000495.jpg\nreal/bird/real_032_000517.jpg\nreal/birthday_cake/real_033_000106.jpg\nreal/birthday_cake/real_033_000121.jpg\nreal/birthday_cake/real_033_000126.jpg\nreal/birthday_cake/real_033_000129.jpg\nreal/birthday_cake/real_033_000132.jpg\nreal/birthday_cake/real_033_000167.jpg\nreal/birthday_cake/real_033_000174.jpg\nreal/birthday_cake/real_033_000175.jpg\nreal/birthday_cake/real_033_000186.jpg\nreal/birthday_cake/real_033_000190.jpg\nreal/birthday_cake/real_033_000201.jpg\nreal/birthday_cake/real_033_000230.jpg\nreal/blueberry/real_035_000272.jpg\nreal/book/real_036_000009.jpg\nreal/book/real_036_000053.jpg\nreal/book/real_036_000082.jpg\nreal/book/real_036_000089.jpg\nreal/book/real_036_000092.jpg\nreal/book/real_036_000094.jpg\nreal/book/real_036_000135.jpg\nreal/book/real_036_000152.jpg\nreal/book/real_036_000202.jpg\nreal/book/real_036_000258.jpg\nreal/book/real_036_000285.jpg\nreal/book/real_036_000296.jpg\nreal/book/real_036_000306.jpg\nreal/book/real_036_000324.jpg\nreal/book/real_036_000360.jpg\nreal/book/real_036_000385.jpg\nreal/book/real_036_000530.jpg\nreal/book/real_036_000588.jpg\nreal/book/real_036_000670.jpg\nreal/book/real_036_000671.jpg\nreal/boomerang/real_037_000012.jpg\nreal/boomerang/real_037_000124.jpg\nreal/boomerang/real_037_000204.jpg\nreal/bottlecap/real_038_000030.jpg\nreal/bottlecap/real_038_000132.jpg\nreal/bottlecap/real_038_000199.jpg\nreal/bottlecap/real_038_000208.jpg\nreal/bottlecap/real_038_000277.jpg\nreal/bottlecap/real_038_000289.jpg\nreal/bottlecap/real_038_000318.jpg\nreal/bottlecap/real_038_000331.jpg\nreal/bottlecap/real_038_000363.jpg\nreal/bottlecap/real_038_000365.jpg\nreal/bottlecap/real_038_000389.jpg\nreal/bottlecap/real_038_000428.jpg\nreal/bottlecap/real_038_000446.jpg\nreal/bottlecap/real_038_000473.jpg\nreal/bottlecap/real_038_000484.jpg\nreal/bottlecap/real_038_000487.jpg\nreal/bottlecap/real_038_000523.jpg\nreal/bottlecap/real_038_000531.jpg\nreal/bottlecap/real_038_000567.jpg\nreal/bottlecap/real_038_000577.jpg\nreal/bottlecap/real_038_000590.jpg\nreal/bottlecap/real_038_000593.jpg\nreal/bowtie/real_039_000045.jpg\nreal/bowtie/real_039_000056.jpg\nreal/bowtie/real_039_000082.jpg\nreal/bowtie/real_039_000089.jpg\nreal/bowtie/real_039_000150.jpg\nreal/bowtie/real_039_000214.jpg\nreal/bowtie/real_039_000224.jpg\nreal/bowtie/real_039_000266.jpg\nreal/bowtie/real_039_000278.jpg\nreal/bowtie/real_039_000319.jpg\nreal/bowtie/real_039_000352.jpg\nreal/bowtie/real_039_000358.jpg\nreal/bowtie/real_039_000394.jpg\nreal/bowtie/real_039_000396.jpg\nreal/bowtie/real_039_000474.jpg\nreal/bowtie/real_039_000489.jpg\nreal/bowtie/real_039_000506.jpg\nreal/bracelet/real_040_000476.jpg\nreal/bracelet/real_040_000712.jpg\nreal/brain/real_041_000024.jpg\nreal/brain/real_041_000058.jpg\nreal/brain/real_041_000078.jpg\nreal/brain/real_041_000192.jpg\nreal/brain/real_041_000317.jpg\nreal/brain/real_041_000388.jpg\nreal/brain/real_041_000507.jpg\nreal/brain/real_041_000590.jpg\nreal/brain/real_041_000711.jpg\nreal/bread/real_042_000045.jpg\nreal/bread/real_042_000690.jpg\nreal/bridge/real_043_000127.jpg\nreal/bridge/real_043_000706.jpg\nreal/broccoli/real_044_000140.jpg\nreal/broccoli/real_044_000497.jpg\nreal/broom/real_045_000022.jpg\nreal/broom/real_045_000242.jpg\nreal/broom/real_045_000496.jpg\nreal/broom/real_045_000521.jpg\nreal/broom/real_045_000601.jpg\nreal/broom/real_045_000608.jpg\nreal/broom/real_045_000619.jpg\nreal/bucket/real_046_000039.jpg\nreal/bucket/real_046_000075.jpg\nreal/bucket/real_046_000147.jpg\nreal/bucket/real_046_000253.jpg\nreal/bucket/real_046_000304.jpg\nreal/bucket/real_046_000305.jpg\nreal/bucket/real_046_000324.jpg\nreal/bulldozer/real_047_000027.jpg\nreal/bulldozer/real_047_000055.jpg\nreal/bulldozer/real_047_000466.jpg\nreal/bulldozer/real_047_000555.jpg\nreal/bulldozer/real_047_000573.jpg\nreal/bulldozer/real_047_000580.jpg\nreal/bus/real_048_000417.jpg\nreal/butterfly/real_050_000072.jpg\nreal/butterfly/real_050_000114.jpg\nreal/butterfly/real_050_000158.jpg\nreal/butterfly/real_050_000198.jpg\nreal/butterfly/real_050_000269.jpg\nreal/butterfly/real_050_000312.jpg\nreal/butterfly/real_050_000326.jpg\nreal/butterfly/real_050_000495.jpg\nreal/butterfly/real_050_000496.jpg\nreal/butterfly/real_050_000498.jpg\nreal/butterfly/real_050_000506.jpg\nreal/butterfly/real_050_000540.jpg\nreal/butterfly/real_050_000542.jpg\nreal/butterfly/real_050_000548.jpg\nreal/butterfly/real_050_000573.jpg\nreal/butterfly/real_050_000593.jpg\nreal/butterfly/real_050_000594.jpg\nreal/butterfly/real_050_000595.jpg\nreal/butterfly/real_050_000631.jpg\nreal/cactus/real_051_000427.jpg\nreal/cake/real_052_000006.jpg\nreal/cake/real_052_000073.jpg\nreal/cake/real_052_000285.jpg\nreal/cake/real_052_000340.jpg\nreal/cake/real_052_000343.jpg\nreal/cake/real_052_000435.jpg\nreal/cake/real_052_000444.jpg\nreal/cake/real_052_000488.jpg\nreal/cake/real_052_000526.jpg\nreal/cake/real_052_000538.jpg\nreal/cake/real_052_000604.jpg\nreal/cake/real_052_000672.jpg\nreal/camel/real_055_000006.jpg\nreal/camel/real_055_000066.jpg\nreal/camel/real_055_000147.jpg\nreal/camel/real_055_000171.jpg\nreal/camel/real_055_000270.jpg\nreal/camel/real_055_000301.jpg\nreal/camera/real_056_000070.jpg\nreal/camera/real_056_000352.jpg\nreal/camouflage/real_057_000119.jpg\nreal/campfire/real_058_000100.jpg\nreal/campfire/real_058_000174.jpg\nreal/campfire/real_058_000193.jpg\nreal/campfire/real_058_000206.jpg\nreal/campfire/real_058_000249.jpg\nreal/campfire/real_058_000264.jpg\nreal/campfire/real_058_000273.jpg\nreal/campfire/real_058_000279.jpg\nreal/campfire/real_058_000341.jpg\nreal/campfire/real_058_000361.jpg\nreal/campfire/real_058_000408.jpg\nreal/campfire/real_058_000439.jpg\nreal/candle/real_059_000429.jpg\nreal/candle/real_059_000465.jpg\nreal/cannon/real_060_000006.jpg\nreal/cannon/real_060_000079.jpg\nreal/cannon/real_060_000126.jpg\nreal/cannon/real_060_000244.jpg\nreal/cannon/real_060_000258.jpg\nreal/canoe/real_061_000274.jpg\nreal/canoe/real_061_000555.jpg\nreal/car/real_062_000372.jpg\nreal/carrot/real_063_000126.jpg\nreal/carrot/real_063_000179.jpg\nreal/cat/real_065_000117.jpg\nreal/cello/real_067_000108.jpg\nreal/cello/real_067_000368.jpg\nreal/circle/real_072_000014.jpg\nreal/circle/real_072_000065.jpg\nreal/circle/real_072_000067.jpg\nreal/circle/real_072_000079.jpg\nreal/circle/real_072_000110.jpg\nreal/circle/real_072_000112.jpg\nreal/circle/real_072_000123.jpg\nreal/circle/real_072_000136.jpg\nreal/circle/real_072_000139.jpg\nreal/circle/real_072_000146.jpg\nreal/circle/real_072_000152.jpg\nreal/circle/real_072_000157.jpg\nreal/circle/real_072_000164.jpg\nreal/circle/real_072_000173.jpg\nreal/circle/real_072_000185.jpg\nreal/circle/real_072_000211.jpg\nreal/circle/real_072_000242.jpg\nreal/clarinet/real_073_000063.jpg\nreal/clarinet/real_073_000084.jpg\nreal/clarinet/real_073_000160.jpg\nreal/clarinet/real_073_000366.jpg\nreal/clock/real_074_000045.jpg\nreal/clock/real_074_000063.jpg\nreal/clock/real_074_000064.jpg\nreal/clock/real_074_000087.jpg\nreal/clock/real_074_000089.jpg\nreal/clock/real_074_000132.jpg\nreal/clock/real_074_000164.jpg\nreal/clock/real_074_000224.jpg\nreal/clock/real_074_000227.jpg\nreal/clock/real_074_000249.jpg\nreal/clock/real_074_000270.jpg\nreal/clock/real_074_000271.jpg\nreal/clock/real_074_000322.jpg\nreal/clock/real_074_000354.jpg\nreal/clock/real_074_000375.jpg\nreal/clock/real_074_000410.jpg\nreal/clock/real_074_000418.jpg\nreal/clock/real_074_000424.jpg\nreal/clock/real_074_000429.jpg\nreal/clock/real_074_000507.jpg\nreal/clock/real_074_000508.jpg\nreal/clock/real_074_000522.jpg\nreal/clock/real_074_000523.jpg\nreal/clock/real_074_000536.jpg\nreal/clock/real_074_000543.jpg\nreal/clock/real_074_000551.jpg\nreal/clock/real_074_000577.jpg\nreal/clock/real_074_000595.jpg\nreal/clock/real_074_000605.jpg\nreal/coffee_cup/real_076_000041.jpg\nreal/coffee_cup/real_076_000089.jpg\nreal/coffee_cup/real_076_000095.jpg\nreal/coffee_cup/real_076_000102.jpg\nreal/coffee_cup/real_076_000128.jpg\nreal/coffee_cup/real_076_000159.jpg\nreal/coffee_cup/real_076_000182.jpg\nreal/coffee_cup/real_076_000202.jpg\nreal/coffee_cup/real_076_000234.jpg\nreal/coffee_cup/real_076_000244.jpg\nreal/coffee_cup/real_076_000253.jpg\nreal/coffee_cup/real_076_000289.jpg\nreal/coffee_cup/real_076_000322.jpg\nreal/coffee_cup/real_076_000334.jpg\nreal/coffee_cup/real_076_000345.jpg\nreal/coffee_cup/real_076_000358.jpg\nreal/coffee_cup/real_076_000398.jpg\nreal/coffee_cup/real_076_000402.jpg\nreal/coffee_cup/real_076_000430.jpg\nreal/coffee_cup/real_076_000499.jpg\nreal/coffee_cup/real_076_000502.jpg\nreal/coffee_cup/real_076_000547.jpg\nreal/coffee_cup/real_076_000567.jpg\nreal/coffee_cup/real_076_000581.jpg\nreal/coffee_cup/real_076_000594.jpg\nreal/compass/real_077_000051.jpg\nreal/compass/real_077_000062.jpg\nreal/compass/real_077_000072.jpg\nreal/compass/real_077_000073.jpg\nreal/compass/real_077_000077.jpg\nreal/compass/real_077_000081.jpg\nreal/compass/real_077_000103.jpg\nreal/compass/real_077_000106.jpg\nreal/compass/real_077_000134.jpg\nreal/compass/real_077_000137.jpg\nreal/compass/real_077_000139.jpg\nreal/compass/real_077_000157.jpg\nreal/compass/real_077_000162.jpg\nreal/compass/real_077_000190.jpg\nreal/compass/real_077_000194.jpg\nreal/compass/real_077_000199.jpg\nreal/compass/real_077_000211.jpg\nreal/compass/real_077_000220.jpg\nreal/compass/real_077_000229.jpg\nreal/compass/real_077_000256.jpg\nreal/compass/real_077_000261.jpg\nreal/compass/real_077_000271.jpg\nreal/computer/real_078_000012.jpg\nreal/computer/real_078_000117.jpg\nreal/computer/real_078_000165.jpg\nreal/computer/real_078_000226.jpg\nreal/computer/real_078_000227.jpg\nreal/computer/real_078_000271.jpg\nreal/computer/real_078_000351.jpg\nreal/cookie/real_079_000112.jpg\nreal/cookie/real_079_000440.jpg\nreal/cooler/real_080_000095.jpg\nreal/cooler/real_080_000156.jpg\nreal/cooler/real_080_000333.jpg\nreal/cooler/real_080_000414.jpg\nreal/cooler/real_080_000484.jpg\nreal/couch/real_081_000554.jpg\nreal/cow/real_082_000145.jpg\nreal/cow/real_082_000335.jpg\nreal/cow/real_082_000427.jpg\nreal/cow/real_082_000443.jpg\nreal/crab/real_083_000062.jpg\nreal/crab/real_083_000219.jpg\nreal/crab/real_083_000475.jpg\nreal/crab/real_083_000528.jpg\nreal/crab/real_083_000572.jpg\nreal/crab/real_083_000579.jpg\nreal/crab/real_083_000607.jpg\nreal/crab/real_083_000698.jpg\nreal/crab/real_083_000705.jpg\nreal/crayon/real_084_000081.jpg\nreal/crayon/real_084_000135.jpg\nreal/crayon/real_084_000199.jpg\nreal/crayon/real_084_000207.jpg\nreal/crayon/real_084_000218.jpg\nreal/crayon/real_084_000275.jpg\nreal/crayon/real_084_000484.jpg\nreal/crayon/real_084_000486.jpg\nreal/crayon/real_084_000487.jpg\nreal/crocodile/real_085_000631.jpg\nreal/crown/real_086_000009.jpg\nreal/crown/real_086_000014.jpg\nreal/crown/real_086_000025.jpg\nreal/crown/real_086_000030.jpg\nreal/crown/real_086_000034.jpg\nreal/crown/real_086_000050.jpg\nreal/crown/real_086_000062.jpg\nreal/crown/real_086_000074.jpg\nreal/crown/real_086_000077.jpg\nreal/crown/real_086_000078.jpg\nreal/crown/real_086_000086.jpg\nreal/crown/real_086_000095.jpg\nreal/crown/real_086_000105.jpg\nreal/crown/real_086_000145.jpg\nreal/crown/real_086_000156.jpg\nreal/crown/real_086_000160.jpg\nreal/cruise_ship/real_087_000224.jpg\nreal/cruise_ship/real_087_000235.jpg\nreal/cruise_ship/real_087_000276.jpg\nreal/cruise_ship/real_087_000488.jpg\nreal/cruise_ship/real_087_000580.jpg\nreal/cup/real_088_000006.jpg\nreal/cup/real_088_000009.jpg\nreal/cup/real_088_000042.jpg\nreal/cup/real_088_000046.jpg\nreal/cup/real_088_000047.jpg\nreal/cup/real_088_000064.jpg\nreal/cup/real_088_000097.jpg\nreal/cup/real_088_000105.jpg\nreal/cup/real_088_000106.jpg\nreal/cup/real_088_000113.jpg\nreal/cup/real_088_000120.jpg\nreal/cup/real_088_000142.jpg\nreal/cup/real_088_000150.jpg\nreal/cup/real_088_000160.jpg\nreal/cup/real_088_000162.jpg\nreal/cup/real_088_000171.jpg\nreal/cup/real_088_000176.jpg\nreal/cup/real_088_000193.jpg\nreal/cup/real_088_000207.jpg\nreal/cup/real_088_000232.jpg\nreal/cup/real_088_000237.jpg\nreal/cup/real_088_000240.jpg\nreal/cup/real_088_000249.jpg\nreal/cup/real_088_000260.jpg\nreal/cup/real_088_000287.jpg\nreal/cup/real_088_000296.jpg\nreal/cup/real_088_000297.jpg\nreal/cup/real_088_000308.jpg\nreal/cup/real_088_000328.jpg\nreal/cup/real_088_000331.jpg\nreal/cup/real_088_000341.jpg\nreal/cup/real_088_000373.jpg\nreal/cup/real_088_000376.jpg\nreal/cup/real_088_000379.jpg\nreal/diamond/real_089_000071.jpg\nreal/diamond/real_089_000083.jpg\nreal/diamond/real_089_000116.jpg\nreal/diamond/real_089_000143.jpg\nreal/diamond/real_089_000151.jpg\nreal/diamond/real_089_000195.jpg\nreal/diamond/real_089_000204.jpg\nreal/diamond/real_089_000268.jpg\nreal/diamond/real_089_000285.jpg\nreal/diamond/real_089_000353.jpg\nreal/diamond/real_089_000379.jpg\nreal/diamond/real_089_000496.jpg\nreal/diamond/real_089_000498.jpg\nreal/diamond/real_089_000499.jpg\nreal/diamond/real_089_000527.jpg\nreal/diamond/real_089_000529.jpg\nreal/diamond/real_089_000533.jpg\nreal/diamond/real_089_000535.jpg\nreal/diamond/real_089_000548.jpg\nreal/diamond/real_089_000553.jpg\nreal/diamond/real_089_000572.jpg\nreal/diamond/real_089_000577.jpg\nreal/dishwasher/real_090_000320.jpg\nreal/dishwasher/real_090_000435.jpg\nreal/diving_board/real_091_000050.jpg\nreal/diving_board/real_091_000064.jpg\nreal/diving_board/real_091_000075.jpg\nreal/diving_board/real_091_000076.jpg\nreal/diving_board/real_091_000095.jpg\nreal/diving_board/real_091_000096.jpg\nreal/diving_board/real_091_000130.jpg\nreal/diving_board/real_091_000131.jpg\nreal/diving_board/real_091_000140.jpg\nreal/diving_board/real_091_000161.jpg\nreal/diving_board/real_091_000189.jpg\nreal/diving_board/real_091_000216.jpg\nreal/diving_board/real_091_000252.jpg\nreal/diving_board/real_091_000273.jpg\nreal/diving_board/real_091_000281.jpg\nreal/diving_board/real_091_000291.jpg\nreal/diving_board/real_091_000325.jpg\nreal/diving_board/real_091_000372.jpg\nreal/diving_board/real_091_000387.jpg\nreal/diving_board/real_091_000444.jpg\nreal/diving_board/real_091_000447.jpg\nreal/diving_board/real_091_000462.jpg\nreal/diving_board/real_091_000496.jpg\nreal/diving_board/real_091_000504.jpg\nreal/diving_board/real_091_000548.jpg\nreal/diving_board/real_091_000586.jpg\nreal/dog/real_092_000099.jpg\nreal/dog/real_092_000523.jpg\nreal/dog/real_092_000676.jpg\nreal/dolphin/real_093_000025.jpg\nreal/dolphin/real_093_000326.jpg\nreal/donut/real_094_000276.jpg\nreal/donut/real_094_000279.jpg\nreal/donut/real_094_000451.jpg\nreal/door/real_095_000083.jpg\nreal/door/real_095_000085.jpg\nreal/door/real_095_000114.jpg\nreal/door/real_095_000145.jpg\nreal/door/real_095_000153.jpg\nreal/door/real_095_000296.jpg\nreal/door/real_095_000324.jpg\nreal/door/real_095_000337.jpg\nreal/dragon/real_096_000059.jpg\nreal/dragon/real_096_000090.jpg\nreal/dragon/real_096_000233.jpg\nreal/dragon/real_096_000304.jpg\nreal/dragon/real_096_000337.jpg\nreal/dragon/real_096_000353.jpg\nreal/dresser/real_097_000141.jpg\nreal/drill/real_098_000046.jpg\nreal/drill/real_098_000200.jpg\nreal/drill/real_098_000286.jpg\nreal/drill/real_098_000389.jpg\nreal/drill/real_098_000396.jpg\nreal/drill/real_098_000429.jpg\nreal/drill/real_098_000500.jpg\nreal/drill/real_098_000501.jpg\nreal/duck/real_100_000085.jpg\nreal/duck/real_100_000182.jpg\nreal/duck/real_100_000341.jpg\nreal/dumbbell/real_101_000124.jpg\nreal/dumbbell/real_101_000127.jpg\nreal/dumbbell/real_101_000135.jpg\nreal/dumbbell/real_101_000169.jpg\nreal/dumbbell/real_101_000276.jpg\nreal/dumbbell/real_101_000380.jpg\nreal/dumbbell/real_101_000396.jpg\nreal/ear/real_102_000018.jpg\nreal/ear/real_102_000023.jpg\nreal/ear/real_102_000052.jpg\nreal/ear/real_102_000107.jpg\nreal/ear/real_102_000176.jpg\nreal/ear/real_102_000183.jpg\nreal/ear/real_102_000330.jpg\nreal/elbow/real_103_000015.jpg\nreal/elbow/real_103_000075.jpg\nreal/elbow/real_103_000152.jpg\nreal/elbow/real_103_000226.jpg\nreal/elbow/real_103_000368.jpg\nreal/elephant/real_104_000222.jpg\nreal/elephant/real_104_000241.jpg\nreal/elephant/real_104_000347.jpg\nreal/elephant/real_104_000370.jpg\nreal/elephant/real_104_000678.jpg\nreal/elephant/real_104_000728.jpg\nreal/envelope/real_105_000025.jpg\nreal/envelope/real_105_000093.jpg\nreal/envelope/real_105_000102.jpg\nreal/envelope/real_105_000251.jpg\nreal/envelope/real_105_000343.jpg\nreal/envelope/real_105_000411.jpg\nreal/envelope/real_105_000515.jpg\nreal/envelope/real_105_000580.jpg\nreal/envelope/real_105_000595.jpg\nreal/envelope/real_105_000635.jpg\nreal/eraser/real_106_000008.jpg\nreal/eraser/real_106_000136.jpg\nreal/eraser/real_106_000148.jpg\nreal/eraser/real_106_000194.jpg\nreal/eye/real_107_000014.jpg\nreal/eye/real_107_000047.jpg\nreal/eye/real_107_000061.jpg\nreal/eye/real_107_000098.jpg\nreal/eye/real_107_000151.jpg\nreal/eye/real_107_000182.jpg\nreal/eye/real_107_000253.jpg\nreal/eye/real_107_000255.jpg\nreal/eye/real_107_000281.jpg\nreal/eye/real_107_000301.jpg\nreal/eye/real_107_000328.jpg\nreal/eye/real_107_000372.jpg\nreal/eye/real_107_000381.jpg\nreal/eye/real_107_000463.jpg\nreal/eye/real_107_000468.jpg\nreal/eye/real_107_000687.jpg\nreal/eyeglasses/real_108_000076.jpg\nreal/eyeglasses/real_108_000079.jpg\nreal/eyeglasses/real_108_000194.jpg\nreal/eyeglasses/real_108_000220.jpg\nreal/eyeglasses/real_108_000278.jpg\nreal/eyeglasses/real_108_000314.jpg\nreal/eyeglasses/real_108_000382.jpg\nreal/eyeglasses/real_108_000408.jpg\nreal/eyeglasses/real_108_000445.jpg\nreal/eyeglasses/real_108_000462.jpg\nreal/eyeglasses/real_108_000521.jpg\nreal/eyeglasses/real_108_000571.jpg\nreal/face/real_109_000124.jpg\nreal/face/real_109_000283.jpg\nreal/face/real_109_000661.jpg\nreal/fan/real_110_000005.jpg\nreal/fan/real_110_000008.jpg\nreal/fan/real_110_000096.jpg\nreal/fan/real_110_000100.jpg\nreal/fan/real_110_000119.jpg\nreal/fan/real_110_000153.jpg\nreal/fan/real_110_000200.jpg\nreal/fan/real_110_000237.jpg\nreal/fan/real_110_000249.jpg\nreal/fan/real_110_000263.jpg\nreal/fan/real_110_000271.jpg\nreal/fan/real_110_000279.jpg\nreal/fan/real_110_000287.jpg\nreal/fan/real_110_000291.jpg\nreal/fan/real_110_000395.jpg\nreal/fan/real_110_000422.jpg\nreal/fan/real_110_000434.jpg\nreal/fan/real_110_000436.jpg\nreal/fan/real_110_000444.jpg\nreal/fan/real_110_000450.jpg\nreal/feather/real_111_000005.jpg\nreal/feather/real_111_000006.jpg\nreal/feather/real_111_000011.jpg\nreal/feather/real_111_000019.jpg\nreal/feather/real_111_000025.jpg\nreal/feather/real_111_000031.jpg\nreal/feather/real_111_000035.jpg\nreal/feather/real_111_000051.jpg\nreal/feather/real_111_000057.jpg\nreal/feather/real_111_000084.jpg\nreal/feather/real_111_000131.jpg\nreal/feather/real_111_000156.jpg\nreal/feather/real_111_000172.jpg\nreal/feather/real_111_000177.jpg\nreal/feather/real_111_000197.jpg\nreal/feather/real_111_000215.jpg\nreal/feather/real_111_000220.jpg\nreal/feather/real_111_000229.jpg\nreal/feather/real_111_000285.jpg\nreal/feather/real_111_000416.jpg\nreal/feather/real_111_000466.jpg\nreal/feather/real_111_000479.jpg\nreal/feather/real_111_000484.jpg\nreal/feather/real_111_000490.jpg\nreal/fence/real_112_000044.jpg\nreal/fence/real_112_000084.jpg\nreal/fence/real_112_000146.jpg\nreal/fence/real_112_000514.jpg\nreal/finger/real_113_000008.jpg\nreal/finger/real_113_000037.jpg\nreal/finger/real_113_000062.jpg\nreal/finger/real_113_000076.jpg\nreal/finger/real_113_000117.jpg\nreal/finger/real_113_000138.jpg\nreal/finger/real_113_000142.jpg\nreal/finger/real_113_000188.jpg\nreal/finger/real_113_000623.jpg\nreal/fire_hydrant/real_114_000049.jpg\nreal/fire_hydrant/real_114_000104.jpg\nreal/fire_hydrant/real_114_000132.jpg\nreal/fire_hydrant/real_114_000174.jpg\nreal/fire_hydrant/real_114_000328.jpg\nreal/fire_hydrant/real_114_000371.jpg\nreal/fire_hydrant/real_114_000457.jpg\nreal/fire_hydrant/real_114_000472.jpg\nreal/fire_hydrant/real_114_000539.jpg\nreal/fire_hydrant/real_114_000552.jpg\nreal/firetruck/real_116_000061.jpg\nreal/firetruck/real_116_000091.jpg\nreal/firetruck/real_116_000092.jpg\nreal/firetruck/real_116_000102.jpg\nreal/firetruck/real_116_000106.jpg\nreal/firetruck/real_116_000116.jpg\nreal/firetruck/real_116_000138.jpg\nreal/firetruck/real_116_000158.jpg\nreal/firetruck/real_116_000177.jpg\nreal/firetruck/real_116_000214.jpg\nreal/firetruck/real_116_000220.jpg\nreal/firetruck/real_116_000223.jpg\nreal/firetruck/real_116_000225.jpg\nreal/firetruck/real_116_000234.jpg\nreal/firetruck/real_116_000236.jpg\nreal/firetruck/real_116_000248.jpg\nreal/firetruck/real_116_000263.jpg\nreal/firetruck/real_116_000271.jpg\nreal/firetruck/real_116_000310.jpg\nreal/firetruck/real_116_000321.jpg\nreal/firetruck/real_116_000326.jpg\nreal/firetruck/real_116_000351.jpg\nreal/firetruck/real_116_000368.jpg\nreal/firetruck/real_116_000374.jpg\nreal/firetruck/real_116_000401.jpg\nreal/firetruck/real_116_000407.jpg\nreal/firetruck/real_116_000434.jpg\nreal/firetruck/real_116_000454.jpg\nreal/firetruck/real_116_000473.jpg\nreal/firetruck/real_116_000475.jpg\nreal/firetruck/real_116_000479.jpg\nreal/firetruck/real_116_000481.jpg\nreal/firetruck/real_116_000490.jpg\nreal/firetruck/real_116_000520.jpg\nreal/firetruck/real_116_000522.jpg\nreal/firetruck/real_116_000523.jpg\nreal/firetruck/real_116_000537.jpg\nreal/fish/real_117_000172.jpg\nreal/fish/real_117_000326.jpg\nreal/fish/real_117_000395.jpg\nreal/fish/real_117_000412.jpg\nreal/flamingo/real_118_000058.jpg\nreal/flamingo/real_118_000099.jpg\nreal/flamingo/real_118_000110.jpg\nreal/flamingo/real_118_000247.jpg\nreal/flamingo/real_118_000272.jpg\nreal/flamingo/real_118_000301.jpg\nreal/flamingo/real_118_000316.jpg\nreal/flamingo/real_118_000321.jpg\nreal/flamingo/real_118_000327.jpg\nreal/flamingo/real_118_000354.jpg\nreal/flamingo/real_118_000400.jpg\nreal/flamingo/real_118_000422.jpg\nreal/flamingo/real_118_000426.jpg\nreal/flamingo/real_118_000448.jpg\nreal/flamingo/real_118_000516.jpg\nreal/flamingo/real_118_000520.jpg\nreal/flamingo/real_118_000542.jpg\nreal/flashlight/real_119_000263.jpg\nreal/flashlight/real_119_000337.jpg\nreal/flip_flops/real_120_000048.jpg\nreal/flip_flops/real_120_000072.jpg\nreal/flip_flops/real_120_000105.jpg\nreal/flip_flops/real_120_000166.jpg\nreal/flip_flops/real_120_000190.jpg\nreal/flip_flops/real_120_000268.jpg\nreal/flip_flops/real_120_000288.jpg\nreal/flip_flops/real_120_000421.jpg\nreal/flip_flops/real_120_000443.jpg\nreal/flower/real_122_000161.jpg\nreal/flower/real_122_000319.jpg\nreal/flying_saucer/real_123_000008.jpg\nreal/flying_saucer/real_123_000052.jpg\nreal/flying_saucer/real_123_000062.jpg\nreal/flying_saucer/real_123_000074.jpg\nreal/flying_saucer/real_123_000085.jpg\nreal/flying_saucer/real_123_000088.jpg\nreal/flying_saucer/real_123_000092.jpg\nreal/flying_saucer/real_123_000095.jpg\nreal/flying_saucer/real_123_000099.jpg\nreal/flying_saucer/real_123_000103.jpg\nreal/flying_saucer/real_123_000117.jpg\nreal/flying_saucer/real_123_000167.jpg\nreal/flying_saucer/real_123_000172.jpg\nreal/flying_saucer/real_123_000200.jpg\nreal/flying_saucer/real_123_000211.jpg\nreal/flying_saucer/real_123_000214.jpg\nreal/flying_saucer/real_123_000254.jpg\nreal/flying_saucer/real_123_000288.jpg\nreal/flying_saucer/real_123_000299.jpg\nreal/flying_saucer/real_123_000349.jpg\nreal/flying_saucer/real_123_000350.jpg\nreal/flying_saucer/real_123_000364.jpg\nreal/flying_saucer/real_123_000392.jpg\nreal/foot/real_124_000033.jpg\nreal/foot/real_124_000077.jpg\nreal/foot/real_124_000520.jpg\nreal/foot/real_124_000543.jpg\nreal/fork/real_125_000029.jpg\nreal/fork/real_125_000143.jpg\nreal/fork/real_125_000157.jpg\nreal/fork/real_125_000162.jpg\nreal/fork/real_125_000170.jpg\nreal/fork/real_125_000220.jpg\nreal/fork/real_125_000230.jpg\nreal/fork/real_125_000305.jpg\nreal/garden_hose/real_129_000012.jpg\nreal/garden_hose/real_129_000089.jpg\nreal/garden_hose/real_129_000097.jpg\nreal/garden_hose/real_129_000180.jpg\nreal/garden_hose/real_129_000201.jpg\nreal/garden_hose/real_129_000247.jpg\nreal/garden_hose/real_129_000327.jpg\nreal/garden_hose/real_129_000355.jpg\nreal/garden_hose/real_129_000360.jpg\nreal/garden_hose/real_129_000380.jpg\nreal/giraffe/real_130_000230.jpg\nreal/giraffe/real_130_000408.jpg\nreal/giraffe/real_130_000496.jpg\nreal/goatee/real_131_000152.jpg\nreal/goatee/real_131_000224.jpg\nreal/goatee/real_131_000247.jpg\nreal/goatee/real_131_000333.jpg\nreal/goatee/real_131_000341.jpg\nreal/goatee/real_131_000367.jpg\nreal/goatee/real_131_000442.jpg\nreal/goatee/real_131_000546.jpg\nreal/golf_club/real_132_000097.jpg\nreal/golf_club/real_132_000110.jpg\nreal/golf_club/real_132_000178.jpg\nreal/golf_club/real_132_000181.jpg\nreal/golf_club/real_132_000187.jpg\nreal/golf_club/real_132_000190.jpg\nreal/golf_club/real_132_000191.jpg\nreal/golf_club/real_132_000194.jpg\nreal/golf_club/real_132_000195.jpg\nreal/golf_club/real_132_000198.jpg\nreal/golf_club/real_132_000224.jpg\nreal/golf_club/real_132_000229.jpg\nreal/golf_club/real_132_000268.jpg\nreal/golf_club/real_132_000276.jpg\nreal/golf_club/real_132_000280.jpg\nreal/golf_club/real_132_000282.jpg\nreal/golf_club/real_132_000285.jpg\nreal/golf_club/real_132_000289.jpg\nreal/golf_club/real_132_000291.jpg\nreal/golf_club/real_132_000293.jpg\nreal/golf_club/real_132_000296.jpg\nreal/golf_club/real_132_000299.jpg\nreal/golf_club/real_132_000303.jpg\nreal/golf_club/real_132_000342.jpg\nreal/golf_club/real_132_000381.jpg\nreal/golf_club/real_132_000443.jpg\nreal/golf_club/real_132_000446.jpg\nreal/golf_club/real_132_000465.jpg\nreal/golf_club/real_132_000482.jpg\nreal/golf_club/real_132_000490.jpg\nreal/golf_club/real_132_000510.jpg\nreal/golf_club/real_132_000536.jpg\nreal/golf_club/real_132_000541.jpg\nreal/grapes/real_133_000405.jpg\nreal/grapes/real_133_000463.jpg\nreal/grapes/real_133_000490.jpg\nreal/grapes/real_133_000519.jpg\nreal/grapes/real_133_000554.jpg\nreal/grapes/real_133_000637.jpg\nreal/grass/real_134_000050.jpg\nreal/grass/real_134_000087.jpg\nreal/grass/real_134_000111.jpg\nreal/grass/real_134_000129.jpg\nreal/grass/real_134_000167.jpg\nreal/grass/real_134_000186.jpg\nreal/grass/real_134_000236.jpg\nreal/grass/real_134_000295.jpg\nreal/grass/real_134_000342.jpg\nreal/guitar/real_135_000043.jpg\nreal/guitar/real_135_000279.jpg\nreal/guitar/real_135_000496.jpg\nreal/guitar/real_135_000527.jpg\nreal/guitar/real_135_000562.jpg\nreal/guitar/real_135_000582.jpg\nreal/hamburger/real_136_000163.jpg\nreal/hammer/real_137_000079.jpg\nreal/hammer/real_137_000151.jpg\nreal/hammer/real_137_000170.jpg\nreal/hammer/real_137_000204.jpg\nreal/hammer/real_137_000253.jpg\nreal/hammer/real_137_000271.jpg\nreal/hand/real_138_000006.jpg\nreal/hand/real_138_000011.jpg\nreal/hand/real_138_000034.jpg\nreal/hand/real_138_000038.jpg\nreal/hand/real_138_000041.jpg\nreal/hand/real_138_000078.jpg\nreal/hand/real_138_000094.jpg\nreal/hand/real_138_000113.jpg\nreal/hand/real_138_000116.jpg\nreal/hand/real_138_000157.jpg\nreal/hand/real_138_000205.jpg\nreal/hand/real_138_000210.jpg\nreal/hand/real_138_000227.jpg\nreal/hand/real_138_000228.jpg\nreal/hand/real_138_000246.jpg\nreal/hand/real_138_000247.jpg\nreal/hand/real_138_000251.jpg\nreal/hand/real_138_000252.jpg\nreal/hand/real_138_000282.jpg\nreal/hand/real_138_000286.jpg\nreal/hand/real_138_000292.jpg\nreal/hand/real_138_000297.jpg\nreal/hand/real_138_000299.jpg\nreal/hand/real_138_000301.jpg\nreal/hand/real_138_000309.jpg\nreal/hand/real_138_000310.jpg\nreal/hand/real_138_000357.jpg\nreal/hand/real_138_000371.jpg\nreal/hand/real_138_000394.jpg\nreal/hand/real_138_000421.jpg\nreal/hand/real_138_000422.jpg\nreal/hand/real_138_000459.jpg\nreal/hand/real_138_000474.jpg\nreal/hand/real_138_000485.jpg\nreal/hand/real_138_000490.jpg\nreal/hand/real_138_000520.jpg\nreal/hand/real_138_000535.jpg\nreal/hand/real_138_000538.jpg\nreal/hand/real_138_000540.jpg\nreal/hand/real_138_000543.jpg\nreal/hand/real_138_000557.jpg\nreal/hand/real_138_000560.jpg\nreal/harp/real_139_000038.jpg\nreal/harp/real_139_000051.jpg\nreal/harp/real_139_000062.jpg\nreal/harp/real_139_000087.jpg\nreal/harp/real_139_000290.jpg\nreal/harp/real_139_000320.jpg\nreal/harp/real_139_000479.jpg\nreal/harp/real_139_000623.jpg\nreal/hat/real_140_000178.jpg\nreal/hat/real_140_000242.jpg\nreal/hat/real_140_000301.jpg\nreal/hat/real_140_000311.jpg\nreal/hat/real_140_000405.jpg\nreal/headphones/real_141_000320.jpg\nreal/headphones/real_141_000326.jpg\nreal/headphones/real_141_000410.jpg\nreal/headphones/real_141_000430.jpg\nreal/hedgehog/real_142_000044.jpg\nreal/hedgehog/real_142_000095.jpg\nreal/helicopter/real_143_000015.jpg\nreal/helicopter/real_143_000689.jpg\nreal/helmet/real_144_000322.jpg\nreal/helmet/real_144_000421.jpg\nreal/helmet/real_144_000476.jpg\nreal/hexagon/real_145_000043.jpg\nreal/hexagon/real_145_000064.jpg\nreal/hexagon/real_145_000078.jpg\nreal/hexagon/real_145_000096.jpg\nreal/hexagon/real_145_000121.jpg\nreal/hexagon/real_145_000188.jpg\nreal/hexagon/real_145_000190.jpg\nreal/hexagon/real_145_000255.jpg\nreal/hexagon/real_145_000261.jpg\nreal/hexagon/real_145_000321.jpg\nreal/hexagon/real_145_000384.jpg\nreal/hexagon/real_145_000421.jpg\nreal/hexagon/real_145_000422.jpg\nreal/hexagon/real_145_000437.jpg\nreal/hexagon/real_145_000464.jpg\nreal/hexagon/real_145_000501.jpg\nreal/hexagon/real_145_000526.jpg\nreal/hexagon/real_145_000540.jpg\nreal/hockey_puck/real_146_000030.jpg\nreal/hockey_puck/real_146_000057.jpg\nreal/hockey_puck/real_146_000093.jpg\nreal/hockey_puck/real_146_000118.jpg\nreal/hockey_puck/real_146_000119.jpg\nreal/hockey_puck/real_146_000122.jpg\nreal/hockey_puck/real_146_000130.jpg\nreal/hockey_puck/real_146_000132.jpg\nreal/hockey_puck/real_146_000144.jpg\nreal/hockey_puck/real_146_000178.jpg\nreal/hockey_puck/real_146_000182.jpg\nreal/hockey_puck/real_146_000204.jpg\nreal/hockey_puck/real_146_000232.jpg\nreal/hockey_puck/real_146_000234.jpg\nreal/hockey_puck/real_146_000239.jpg\nreal/hockey_puck/real_146_000245.jpg\nreal/hockey_puck/real_146_000247.jpg\nreal/hockey_puck/real_146_000248.jpg\nreal/hockey_puck/real_146_000259.jpg\nreal/hockey_puck/real_146_000260.jpg\nreal/hockey_puck/real_146_000266.jpg\nreal/hockey_puck/real_146_000279.jpg\nreal/hockey_puck/real_146_000311.jpg\nreal/hockey_puck/real_146_000319.jpg\nreal/hockey_puck/real_146_000329.jpg\nreal/hockey_puck/real_146_000348.jpg\nreal/hockey_puck/real_146_000377.jpg\nreal/hockey_puck/real_146_000382.jpg\nreal/hockey_puck/real_146_000384.jpg\nreal/hockey_stick/real_147_000023.jpg\nreal/hockey_stick/real_147_000024.jpg\nreal/hockey_stick/real_147_000032.jpg\nreal/hockey_stick/real_147_000042.jpg\nreal/hockey_stick/real_147_000049.jpg\nreal/hockey_stick/real_147_000051.jpg\nreal/hockey_stick/real_147_000075.jpg\nreal/hockey_stick/real_147_000080.jpg\nreal/hockey_stick/real_147_000088.jpg\nreal/hockey_stick/real_147_000103.jpg\nreal/hockey_stick/real_147_000114.jpg\nreal/hockey_stick/real_147_000126.jpg\nreal/hockey_stick/real_147_000129.jpg\nreal/hockey_stick/real_147_000141.jpg\nreal/hockey_stick/real_147_000148.jpg\nreal/hockey_stick/real_147_000155.jpg\nreal/hockey_stick/real_147_000181.jpg\nreal/hockey_stick/real_147_000188.jpg\nreal/hockey_stick/real_147_000193.jpg\nreal/hockey_stick/real_147_000199.jpg\nreal/hockey_stick/real_147_000213.jpg\nreal/hockey_stick/real_147_000221.jpg\nreal/hockey_stick/real_147_000241.jpg\nreal/hockey_stick/real_147_000243.jpg\nreal/hockey_stick/real_147_000287.jpg\nreal/hockey_stick/real_147_000296.jpg\nreal/hockey_stick/real_147_000308.jpg\nreal/hockey_stick/real_147_000321.jpg\nreal/hockey_stick/real_147_000322.jpg\nreal/hockey_stick/real_147_000352.jpg\nreal/hockey_stick/real_147_000355.jpg\nreal/hockey_stick/real_147_000359.jpg\nreal/hockey_stick/real_147_000362.jpg\nreal/hockey_stick/real_147_000366.jpg\nreal/hockey_stick/real_147_000379.jpg\nreal/hockey_stick/real_147_000388.jpg\nreal/hockey_stick/real_147_000392.jpg\nreal/horse/real_148_000081.jpg\nreal/horse/real_148_000092.jpg\nreal/horse/real_148_000110.jpg\nreal/horse/real_148_000243.jpg\nreal/horse/real_148_000247.jpg\nreal/horse/real_148_000280.jpg\nreal/horse/real_148_000338.jpg\nreal/horse/real_148_000341.jpg\nreal/horse/real_148_000344.jpg\nreal/horse/real_148_000365.jpg\nreal/horse/real_148_000366.jpg\nreal/horse/real_148_000438.jpg\nreal/horse/real_148_000439.jpg\nreal/horse/real_148_000441.jpg\nreal/horse/real_148_000442.jpg\nreal/horse/real_148_000443.jpg\nreal/horse/real_148_000447.jpg\nreal/horse/real_148_000448.jpg\nreal/horse/real_148_000452.jpg\nreal/horse/real_148_000453.jpg\nreal/horse/real_148_000455.jpg\nreal/horse/real_148_000457.jpg\nreal/horse/real_148_000459.jpg\nreal/horse/real_148_000460.jpg\nreal/horse/real_148_000461.jpg\nreal/horse/real_148_000462.jpg\nreal/horse/real_148_000463.jpg\nreal/horse/real_148_000464.jpg\nreal/horse/real_148_000465.jpg\nreal/horse/real_148_000468.jpg\nreal/horse/real_148_000469.jpg\nreal/horse/real_148_000470.jpg\nreal/horse/real_148_000471.jpg\nreal/horse/real_148_000473.jpg\nreal/horse/real_148_000477.jpg\nreal/horse/real_148_000480.jpg\nreal/horse/real_148_000487.jpg\nreal/horse/real_148_000488.jpg\nreal/horse/real_148_000491.jpg\nreal/horse/real_148_000493.jpg\nreal/horse/real_148_000499.jpg\nreal/horse/real_148_000526.jpg\nreal/horse/real_148_000567.jpg\nreal/horse/real_148_000568.jpg\nreal/horse/real_148_000569.jpg\nreal/horse/real_148_000570.jpg\nreal/horse/real_148_000608.jpg\nreal/horse/real_148_000624.jpg\nreal/horse/real_148_000625.jpg\nreal/horse/real_148_000629.jpg\nreal/hospital/real_149_000197.jpg\nreal/hospital/real_149_000200.jpg\nreal/hospital/real_149_000224.jpg\nreal/hospital/real_149_000318.jpg\nreal/hospital/real_149_000390.jpg\nreal/hospital/real_149_000517.jpg\nreal/hospital/real_149_000668.jpg\nreal/hot_air_balloon/real_150_000094.jpg\nreal/hot_air_balloon/real_150_000130.jpg\nreal/hot_air_balloon/real_150_000141.jpg\nreal/hot_air_balloon/real_150_000237.jpg\nreal/hot_air_balloon/real_150_000261.jpg\nreal/hot_air_balloon/real_150_000357.jpg\nreal/hot_air_balloon/real_150_000378.jpg\nreal/hot_air_balloon/real_150_000485.jpg\nreal/hot_air_balloon/real_150_000569.jpg\nreal/hot_air_balloon/real_150_000649.jpg\nreal/hot_air_balloon/real_150_000654.jpg\nreal/hot_air_balloon/real_150_000720.jpg\nreal/hot_dog/real_151_000032.jpg\nreal/hot_dog/real_151_000045.jpg\nreal/hot_dog/real_151_000208.jpg\nreal/hot_dog/real_151_000248.jpg\nreal/hot_dog/real_151_000509.jpg\nreal/hot_dog/real_151_000563.jpg\nreal/hot_tub/real_152_000244.jpg\nreal/hot_tub/real_152_000318.jpg\nreal/hot_tub/real_152_000460.jpg\nreal/hot_tub/real_152_000719.jpg\nreal/hot_tub/real_152_000743.jpg\nreal/hourglass/real_153_000010.jpg\nreal/hourglass/real_153_000064.jpg\nreal/hourglass/real_153_000115.jpg\nreal/hourglass/real_153_000118.jpg\nreal/hourglass/real_153_000122.jpg\nreal/hourglass/real_153_000160.jpg\nreal/hourglass/real_153_000201.jpg\nreal/hourglass/real_153_000211.jpg\nreal/hourglass/real_153_000212.jpg\nreal/hourglass/real_153_000226.jpg\nreal/hourglass/real_153_000232.jpg\nreal/house_plant/real_155_000192.jpg\nreal/ice_cream/real_157_000075.jpg\nreal/ice_cream/real_157_000392.jpg\nreal/ice_cream/real_157_000461.jpg\nreal/ice_cream/real_157_000648.jpg\nreal/jacket/real_158_000162.jpg\nreal/jail/real_159_000224.jpg\nreal/kangaroo/real_160_000420.jpg\nreal/kangaroo/real_160_000426.jpg\nreal/kangaroo/real_160_000581.jpg\nreal/key/real_161_000018.jpg\nreal/key/real_161_000024.jpg\nreal/key/real_161_000026.jpg\nreal/key/real_161_000061.jpg\nreal/key/real_161_000063.jpg\nreal/key/real_161_000068.jpg\nreal/key/real_161_000080.jpg\nreal/key/real_161_000118.jpg\nreal/key/real_161_000131.jpg\nreal/key/real_161_000142.jpg\nreal/key/real_161_000148.jpg\nreal/key/real_161_000167.jpg\nreal/key/real_161_000197.jpg\nreal/key/real_161_000216.jpg\nreal/keyboard/real_162_000120.jpg\nreal/keyboard/real_162_000495.jpg\nreal/knee/real_163_000083.jpg\nreal/knee/real_163_000120.jpg\nreal/knee/real_163_000256.jpg\nreal/knee/real_163_000293.jpg\nreal/knee/real_163_000479.jpg\nreal/knife/real_164_000035.jpg\nreal/knife/real_164_000140.jpg\nreal/knife/real_164_000455.jpg\nreal/ladder/real_165_000137.jpg\nreal/ladder/real_165_000171.jpg\nreal/ladder/real_165_000221.jpg\nreal/ladder/real_165_000241.jpg\nreal/ladder/real_165_000274.jpg\nreal/ladder/real_165_000355.jpg\nreal/ladder/real_165_000415.jpg\nreal/lantern/real_166_000023.jpg\nreal/lantern/real_166_000068.jpg\nreal/lantern/real_166_000097.jpg\nreal/lantern/real_166_000210.jpg\nreal/lantern/real_166_000254.jpg\nreal/lantern/real_166_000309.jpg\nreal/lantern/real_166_000312.jpg\nreal/lantern/real_166_000417.jpg\nreal/laptop/real_167_000001.jpg\nreal/laptop/real_167_000025.jpg\nreal/laptop/real_167_000035.jpg\nreal/laptop/real_167_000099.jpg\nreal/laptop/real_167_000136.jpg\nreal/laptop/real_167_000211.jpg\nreal/laptop/real_167_000286.jpg\nreal/laptop/real_167_000341.jpg\nreal/laptop/real_167_000353.jpg\nreal/laptop/real_167_000367.jpg\nreal/laptop/real_167_000375.jpg\nreal/leaf/real_168_000072.jpg\nreal/leaf/real_168_000162.jpg\nreal/leaf/real_168_000229.jpg\nreal/leaf/real_168_000268.jpg\nreal/leaf/real_168_000275.jpg\nreal/leaf/real_168_000276.jpg\nreal/leaf/real_168_000349.jpg\nreal/leg/real_169_000035.jpg\nreal/leg/real_169_000055.jpg\nreal/leg/real_169_000200.jpg\nreal/leg/real_169_000210.jpg\nreal/leg/real_169_000269.jpg\nreal/leg/real_169_000304.jpg\nreal/leg/real_169_000327.jpg\nreal/leg/real_169_000350.jpg\nreal/leg/real_169_000352.jpg\nreal/leg/real_169_000401.jpg\nreal/leg/real_169_000457.jpg\nreal/light_bulb/real_170_000004.jpg\nreal/light_bulb/real_170_000010.jpg\nreal/light_bulb/real_170_000129.jpg\nreal/light_bulb/real_170_000153.jpg\nreal/light_bulb/real_170_000242.jpg\nreal/lighter/real_171_000514.jpg\nreal/lighthouse/real_172_000189.jpg\nreal/lighthouse/real_172_000250.jpg\nreal/lighthouse/real_172_000392.jpg\nreal/lighthouse/real_172_000401.jpg\nreal/lighthouse/real_172_000471.jpg\nreal/lighthouse/real_172_000590.jpg\nreal/lighthouse/real_172_000627.jpg\nreal/lighthouse/real_172_000628.jpg\nreal/lightning/real_173_000337.jpg\nreal/line/real_174_000001.jpg\nreal/line/real_174_000006.jpg\nreal/line/real_174_000043.jpg\nreal/line/real_174_000046.jpg\nreal/line/real_174_000073.jpg\nreal/lobster/real_177_000459.jpg\nreal/lollipop/real_178_000153.jpg\nreal/lollipop/real_178_000192.jpg\nreal/lollipop/real_178_000253.jpg\nreal/lollipop/real_178_000379.jpg\nreal/lollipop/real_178_000482.jpg\nreal/mailbox/real_179_000005.jpg\nreal/map/real_180_000141.jpg\nreal/map/real_180_000488.jpg\nreal/map/real_180_000490.jpg\nreal/marker/real_181_000025.jpg\nreal/marker/real_181_000058.jpg\nreal/marker/real_181_000084.jpg\nreal/marker/real_181_000113.jpg\nreal/marker/real_181_000127.jpg\nreal/marker/real_181_000156.jpg\nreal/marker/real_181_000184.jpg\nreal/marker/real_181_000243.jpg\nreal/marker/real_181_000260.jpg\nreal/marker/real_181_000330.jpg\nreal/matches/real_182_000045.jpg\nreal/megaphone/real_183_000014.jpg\nreal/megaphone/real_183_000046.jpg\nreal/megaphone/real_183_000058.jpg\nreal/megaphone/real_183_000069.jpg\nreal/megaphone/real_183_000133.jpg\nreal/megaphone/real_183_000156.jpg\nreal/megaphone/real_183_000169.jpg\nreal/megaphone/real_183_000173.jpg\nreal/megaphone/real_183_000191.jpg\nreal/megaphone/real_183_000195.jpg\nreal/megaphone/real_183_000237.jpg\nreal/megaphone/real_183_000249.jpg\nreal/megaphone/real_183_000255.jpg\nreal/megaphone/real_183_000262.jpg\nreal/megaphone/real_183_000263.jpg\nreal/megaphone/real_183_000272.jpg\nreal/megaphone/real_183_000286.jpg\nreal/megaphone/real_183_000317.jpg\nreal/megaphone/real_183_000319.jpg\nreal/megaphone/real_183_000325.jpg\nreal/megaphone/real_183_000351.jpg\nreal/megaphone/real_183_000358.jpg\nreal/megaphone/real_183_000374.jpg\nreal/megaphone/real_183_000406.jpg\nreal/megaphone/real_183_000413.jpg\nreal/megaphone/real_183_000420.jpg\nreal/megaphone/real_183_000432.jpg\nreal/megaphone/real_183_000442.jpg\nreal/megaphone/real_183_000443.jpg\nreal/megaphone/real_183_000454.jpg\nreal/megaphone/real_183_000461.jpg\nreal/megaphone/real_183_000464.jpg\nreal/megaphone/real_183_000469.jpg\nreal/megaphone/real_183_000479.jpg\nreal/megaphone/real_183_000491.jpg\nreal/megaphone/real_183_000527.jpg\nreal/megaphone/real_183_000528.jpg\nreal/megaphone/real_183_000539.jpg\nreal/megaphone/real_183_000544.jpg\nreal/mermaid/real_184_000342.jpg\nreal/mermaid/real_184_000371.jpg\nreal/mermaid/real_184_000378.jpg\nreal/microphone/real_185_000104.jpg\nreal/microphone/real_185_000211.jpg\nreal/microphone/real_185_000339.jpg\nreal/microphone/real_185_000359.jpg\nreal/microphone/real_185_000362.jpg\nreal/microphone/real_185_000376.jpg\nreal/microphone/real_185_000383.jpg\nreal/microphone/real_185_000451.jpg\nreal/microphone/real_185_000469.jpg\nreal/microphone/real_185_000473.jpg\nreal/moon/real_188_000312.jpg\nreal/moon/real_188_000367.jpg\nreal/mosquito/real_189_000008.jpg\nreal/mosquito/real_189_000451.jpg\nreal/mosquito/real_189_000514.jpg\nreal/mountain/real_191_000462.jpg\nreal/mountain/real_191_000476.jpg\nreal/mountain/real_191_000696.jpg\nreal/mouse/real_192_000007.jpg\nreal/mouse/real_192_000061.jpg\nreal/mouse/real_192_000089.jpg\nreal/mouse/real_192_000090.jpg\nreal/mouse/real_192_000091.jpg\nreal/mouse/real_192_000092.jpg\nreal/mouse/real_192_000093.jpg\nreal/mouse/real_192_000094.jpg\nreal/mouse/real_192_000097.jpg\nreal/mouse/real_192_000101.jpg\nreal/mouse/real_192_000107.jpg\nreal/mouse/real_192_000117.jpg\nreal/mouse/real_192_000139.jpg\nreal/moustache/real_193_000009.jpg\nreal/moustache/real_193_000015.jpg\nreal/moustache/real_193_000045.jpg\nreal/moustache/real_193_000074.jpg\nreal/moustache/real_193_000119.jpg\nreal/moustache/real_193_000148.jpg\nreal/moustache/real_193_000172.jpg\nreal/moustache/real_193_000185.jpg\nreal/moustache/real_193_000198.jpg\nreal/moustache/real_193_000215.jpg\nreal/moustache/real_193_000221.jpg\nreal/moustache/real_193_000252.jpg\nreal/moustache/real_193_000279.jpg\nreal/moustache/real_193_000301.jpg\nreal/moustache/real_193_000318.jpg\nreal/moustache/real_193_000321.jpg\nreal/moustache/real_193_000368.jpg\nreal/moustache/real_193_000406.jpg\nreal/moustache/real_193_000410.jpg\nreal/moustache/real_193_000415.jpg\nreal/mouth/real_194_000041.jpg\nreal/mouth/real_194_000048.jpg\nreal/mouth/real_194_000058.jpg\nreal/mouth/real_194_000076.jpg\nreal/mouth/real_194_000112.jpg\nreal/mouth/real_194_000114.jpg\nreal/mouth/real_194_000131.jpg\nreal/mouth/real_194_000139.jpg\nreal/mouth/real_194_000147.jpg\nreal/mouth/real_194_000152.jpg\nreal/mouth/real_194_000156.jpg\nreal/mouth/real_194_000170.jpg\nreal/mouth/real_194_000187.jpg\nreal/mouth/real_194_000203.jpg\nreal/mouth/real_194_000207.jpg\nreal/mouth/real_194_000237.jpg\nreal/mouth/real_194_000256.jpg\nreal/mouth/real_194_000285.jpg\nreal/mouth/real_194_000326.jpg\nreal/mouth/real_194_000368.jpg\nreal/mug/real_195_000014.jpg\nreal/mug/real_195_000080.jpg\nreal/mug/real_195_000082.jpg\nreal/mug/real_195_000098.jpg\nreal/mug/real_195_000108.jpg\nreal/mug/real_195_000109.jpg\nreal/mug/real_195_000113.jpg\nreal/mug/real_195_000137.jpg\nreal/mug/real_195_000138.jpg\nreal/mug/real_195_000146.jpg\nreal/mug/real_195_000154.jpg\nreal/mug/real_195_000183.jpg\nreal/mug/real_195_000186.jpg\nreal/mug/real_195_000203.jpg\nreal/mug/real_195_000205.jpg\nreal/mug/real_195_000213.jpg\nreal/mug/real_195_000242.jpg\nreal/mug/real_195_000251.jpg\nreal/mug/real_195_000272.jpg\nreal/mug/real_195_000283.jpg\nreal/mug/real_195_000300.jpg\nreal/mug/real_195_000306.jpg\nreal/mug/real_195_000320.jpg\nreal/mug/real_195_000345.jpg\nreal/mug/real_195_000373.jpg\nreal/mug/real_195_000376.jpg\nreal/mug/real_195_000379.jpg\nreal/mug/real_195_000384.jpg\nreal/mug/real_195_000420.jpg\nreal/mug/real_195_000444.jpg\nreal/mug/real_195_000462.jpg\nreal/mug/real_195_000463.jpg\nreal/mug/real_195_000474.jpg\nreal/mug/real_195_000496.jpg\nreal/mug/real_195_000501.jpg\nreal/mug/real_195_000503.jpg\nreal/mug/real_195_000564.jpg\nreal/mug/real_195_000592.jpg\nreal/mug/real_195_000595.jpg\nreal/mug/real_195_000596.jpg\nreal/mug/real_195_000598.jpg\nreal/mushroom/real_196_000104.jpg\nreal/mushroom/real_196_000591.jpg\nreal/mushroom/real_196_000628.jpg\nreal/nail/real_197_000072.jpg\nreal/nail/real_197_000091.jpg\nreal/nail/real_197_000101.jpg\nreal/nail/real_197_000110.jpg\nreal/nail/real_197_000179.jpg\nreal/nail/real_197_000183.jpg\nreal/nail/real_197_000187.jpg\nreal/nail/real_197_000188.jpg\nreal/nail/real_197_000201.jpg\nreal/nail/real_197_000210.jpg\nreal/nail/real_197_000288.jpg\nreal/nail/real_197_000338.jpg\nreal/nail/real_197_000354.jpg\nreal/nail/real_197_000357.jpg\nreal/nail/real_197_000450.jpg\nreal/nail/real_197_000596.jpg\nreal/nail/real_197_000606.jpg\nreal/nail/real_197_000623.jpg\nreal/nail/real_197_000643.jpg\nreal/nail/real_197_000649.jpg\nreal/nail/real_197_000659.jpg\nreal/nail/real_197_000660.jpg\nreal/necklace/real_198_000057.jpg\nreal/necklace/real_198_000063.jpg\nreal/necklace/real_198_000158.jpg\nreal/necklace/real_198_000200.jpg\nreal/necklace/real_198_000408.jpg\nreal/necklace/real_198_000644.jpg\nreal/nose/real_199_000018.jpg\nreal/nose/real_199_000066.jpg\nreal/nose/real_199_000096.jpg\nreal/nose/real_199_000133.jpg\nreal/nose/real_199_000188.jpg\nreal/nose/real_199_000237.jpg\nreal/ocean/real_200_000159.jpg\nreal/ocean/real_200_000224.jpg\nreal/ocean/real_200_000379.jpg\nreal/octagon/real_201_000001.jpg\nreal/octagon/real_201_000006.jpg\nreal/octagon/real_201_000012.jpg\nreal/octagon/real_201_000041.jpg\nreal/octagon/real_201_000045.jpg\nreal/octagon/real_201_000060.jpg\nreal/octagon/real_201_000082.jpg\nreal/octagon/real_201_000205.jpg\nreal/octagon/real_201_000217.jpg\nreal/octagon/real_201_000280.jpg\nreal/octagon/real_201_000301.jpg\nreal/octagon/real_201_000340.jpg\nreal/octagon/real_201_000349.jpg\nreal/octagon/real_201_000464.jpg\nreal/octopus/real_202_000010.jpg\nreal/octopus/real_202_000018.jpg\nreal/octopus/real_202_000086.jpg\nreal/octopus/real_202_000144.jpg\nreal/octopus/real_202_000352.jpg\nreal/octopus/real_202_000457.jpg\nreal/octopus/real_202_000533.jpg\nreal/octopus/real_202_000606.jpg\nreal/octopus/real_202_000622.jpg\nreal/onion/real_203_000025.jpg\nreal/onion/real_203_000034.jpg\nreal/onion/real_203_000284.jpg\nreal/onion/real_203_000345.jpg\nreal/oven/real_204_000019.jpg\nreal/oven/real_204_000118.jpg\nreal/oven/real_204_000158.jpg\nreal/oven/real_204_000217.jpg\nreal/oven/real_204_000287.jpg\nreal/oven/real_204_000296.jpg\nreal/oven/real_204_000330.jpg\nreal/owl/real_205_000127.jpg\nreal/owl/real_205_000261.jpg\nreal/owl/real_205_000291.jpg\nreal/owl/real_205_000408.jpg\nreal/owl/real_205_000446.jpg\nreal/owl/real_205_000519.jpg\nreal/owl/real_205_000532.jpg\nreal/owl/real_205_000650.jpg\nreal/owl/real_205_000757.jpg\nreal/paintbrush/real_206_000001.jpg\nreal/paintbrush/real_206_000012.jpg\nreal/paintbrush/real_206_000020.jpg\nreal/paintbrush/real_206_000025.jpg\nreal/paintbrush/real_206_000028.jpg\nreal/paintbrush/real_206_000041.jpg\nreal/paintbrush/real_206_000042.jpg\nreal/paintbrush/real_206_000043.jpg\nreal/paintbrush/real_206_000046.jpg\nreal/paintbrush/real_206_000048.jpg\nreal/paintbrush/real_206_000052.jpg\nreal/paintbrush/real_206_000057.jpg\nreal/paintbrush/real_206_000059.jpg\nreal/paintbrush/real_206_000064.jpg\nreal/paintbrush/real_206_000085.jpg\nreal/paintbrush/real_206_000105.jpg\nreal/paintbrush/real_206_000109.jpg\nreal/paintbrush/real_206_000112.jpg\nreal/paintbrush/real_206_000113.jpg\nreal/paintbrush/real_206_000116.jpg\nreal/paintbrush/real_206_000120.jpg\nreal/paintbrush/real_206_000123.jpg\nreal/paintbrush/real_206_000125.jpg\nreal/paintbrush/real_206_000132.jpg\nreal/paintbrush/real_206_000135.jpg\nreal/paintbrush/real_206_000153.jpg\nreal/paintbrush/real_206_000154.jpg\nreal/paintbrush/real_206_000160.jpg\nreal/paintbrush/real_206_000171.jpg\nreal/paintbrush/real_206_000180.jpg\nreal/paintbrush/real_206_000189.jpg\nreal/paintbrush/real_206_000250.jpg\nreal/paintbrush/real_206_000258.jpg\nreal/paintbrush/real_206_000260.jpg\nreal/paintbrush/real_206_000268.jpg\nreal/paintbrush/real_206_000298.jpg\nreal/paintbrush/real_206_000315.jpg\nreal/paintbrush/real_206_000320.jpg\nreal/paintbrush/real_206_000321.jpg\nreal/paintbrush/real_206_000359.jpg\nreal/paintbrush/real_206_000369.jpg\nreal/paintbrush/real_206_000384.jpg\nreal/paintbrush/real_206_000385.jpg\nreal/paintbrush/real_206_000388.jpg\nreal/paintbrush/real_206_000390.jpg\nreal/paintbrush/real_206_000401.jpg\nreal/paint_can/real_207_000034.jpg\nreal/paint_can/real_207_000036.jpg\nreal/paint_can/real_207_000065.jpg\nreal/paint_can/real_207_000108.jpg\nreal/paint_can/real_207_000157.jpg\nreal/paint_can/real_207_000175.jpg\nreal/paint_can/real_207_000183.jpg\nreal/paint_can/real_207_000193.jpg\nreal/paint_can/real_207_000201.jpg\nreal/paint_can/real_207_000208.jpg\nreal/paint_can/real_207_000225.jpg\nreal/paint_can/real_207_000257.jpg\nreal/paint_can/real_207_000266.jpg\nreal/paint_can/real_207_000278.jpg\nreal/paint_can/real_207_000279.jpg\nreal/paint_can/real_207_000313.jpg\nreal/paint_can/real_207_000330.jpg\nreal/paint_can/real_207_000347.jpg\nreal/paint_can/real_207_000368.jpg\nreal/paint_can/real_207_000384.jpg\nreal/paint_can/real_207_000408.jpg\nreal/paint_can/real_207_000437.jpg\nreal/paint_can/real_207_000467.jpg\nreal/paint_can/real_207_000480.jpg\nreal/paint_can/real_207_000481.jpg\nreal/paint_can/real_207_000482.jpg\nreal/paint_can/real_207_000543.jpg\nreal/paint_can/real_207_000551.jpg\nreal/palm_tree/real_208_000044.jpg\nreal/palm_tree/real_208_000104.jpg\nreal/palm_tree/real_208_000113.jpg\nreal/palm_tree/real_208_000119.jpg\nreal/palm_tree/real_208_000133.jpg\nreal/palm_tree/real_208_000138.jpg\nreal/palm_tree/real_208_000152.jpg\nreal/palm_tree/real_208_000189.jpg\nreal/palm_tree/real_208_000213.jpg\nreal/palm_tree/real_208_000218.jpg\nreal/palm_tree/real_208_000222.jpg\nreal/panda/real_209_000112.jpg\nreal/panda/real_209_000412.jpg\nreal/panda/real_209_000453.jpg\nreal/panda/real_209_000457.jpg\nreal/pants/real_210_000090.jpg\nreal/pants/real_210_000131.jpg\nreal/pants/real_210_000254.jpg\nreal/pants/real_210_000337.jpg\nreal/paper_clip/real_211_000017.jpg\nreal/paper_clip/real_211_000026.jpg\nreal/paper_clip/real_211_000027.jpg\nreal/paper_clip/real_211_000030.jpg\nreal/paper_clip/real_211_000038.jpg\nreal/paper_clip/real_211_000043.jpg\nreal/paper_clip/real_211_000048.jpg\nreal/paper_clip/real_211_000049.jpg\nreal/paper_clip/real_211_000051.jpg\nreal/paper_clip/real_211_000054.jpg\nreal/paper_clip/real_211_000081.jpg\nreal/paper_clip/real_211_000082.jpg\nreal/paper_clip/real_211_000109.jpg\nreal/paper_clip/real_211_000110.jpg\nreal/paper_clip/real_211_000114.jpg\nreal/paper_clip/real_211_000118.jpg\nreal/paper_clip/real_211_000119.jpg\nreal/paper_clip/real_211_000155.jpg\nreal/paper_clip/real_211_000175.jpg\nreal/paper_clip/real_211_000185.jpg\nreal/paper_clip/real_211_000213.jpg\nreal/paper_clip/real_211_000228.jpg\nreal/paper_clip/real_211_000269.jpg\nreal/paper_clip/real_211_000271.jpg\nreal/paper_clip/real_211_000273.jpg\nreal/paper_clip/real_211_000274.jpg\nreal/paper_clip/real_211_000308.jpg\nreal/paper_clip/real_211_000320.jpg\nreal/paper_clip/real_211_000358.jpg\nreal/paper_clip/real_211_000393.jpg\nreal/paper_clip/real_211_000396.jpg\nreal/paper_clip/real_211_000422.jpg\nreal/paper_clip/real_211_000423.jpg\nreal/paper_clip/real_211_000446.jpg\nreal/paper_clip/real_211_000447.jpg\nreal/paper_clip/real_211_000450.jpg\nreal/paper_clip/real_211_000453.jpg\nreal/paper_clip/real_211_000479.jpg\nreal/paper_clip/real_211_000480.jpg\nreal/paper_clip/real_211_000528.jpg\nreal/paper_clip/real_211_000533.jpg\nreal/paper_clip/real_211_000544.jpg\nreal/parachute/real_212_000108.jpg\nreal/parachute/real_212_000194.jpg\nreal/parachute/real_212_000214.jpg\nreal/parachute/real_212_000225.jpg\nreal/parachute/real_212_000348.jpg\nreal/parachute/real_212_000427.jpg\nreal/parachute/real_212_000445.jpg\nreal/parachute/real_212_000493.jpg\nreal/parachute/real_212_000496.jpg\nreal/parachute/real_212_000544.jpg\nreal/parachute/real_212_000546.jpg\nreal/parachute/real_212_000590.jpg\nreal/parachute/real_212_000615.jpg\nreal/parrot/real_213_000016.jpg\nreal/parrot/real_213_000022.jpg\nreal/parrot/real_213_000052.jpg\nreal/parrot/real_213_000148.jpg\nreal/parrot/real_213_000351.jpg\nreal/parrot/real_213_000545.jpg\nreal/parrot/real_213_000748.jpg\nreal/passport/real_214_000294.jpg\nreal/peanut/real_215_000050.jpg\nreal/peanut/real_215_000101.jpg\nreal/peanut/real_215_000148.jpg\nreal/peanut/real_215_000182.jpg\nreal/peanut/real_215_000203.jpg\nreal/peanut/real_215_000273.jpg\nreal/pear/real_216_000117.jpg\nreal/pear/real_216_000279.jpg\nreal/pear/real_216_000295.jpg\nreal/pear/real_216_000326.jpg\nreal/peas/real_217_000239.jpg\nreal/pencil/real_218_000034.jpg\nreal/pencil/real_218_000244.jpg\nreal/pencil/real_218_000260.jpg\nreal/pencil/real_218_000432.jpg\nreal/penguin/real_219_000028.jpg\nreal/penguin/real_219_000275.jpg\nreal/penguin/real_219_000339.jpg\nreal/penguin/real_219_000478.jpg\nreal/piano/real_220_000346.jpg\nreal/piano/real_220_000564.jpg\nreal/pickup_truck/real_221_000112.jpg\nreal/picture_frame/real_222_000010.jpg\nreal/picture_frame/real_222_000024.jpg\nreal/picture_frame/real_222_000110.jpg\nreal/picture_frame/real_222_000115.jpg\nreal/picture_frame/real_222_000182.jpg\nreal/pig/real_223_000366.jpg\nreal/pig/real_223_000491.jpg\nreal/pig/real_223_000497.jpg\nreal/pig/real_223_000511.jpg\nreal/pillow/real_224_000095.jpg\nreal/pillow/real_224_000097.jpg\nreal/pillow/real_224_000103.jpg\nreal/pillow/real_224_000318.jpg\nreal/pillow/real_224_000354.jpg\nreal/pillow/real_224_000377.jpg\nreal/pillow/real_224_000389.jpg\nreal/pillow/real_224_000456.jpg\nreal/pillow/real_224_000505.jpg\nreal/pineapple/real_225_000005.jpg\nreal/pineapple/real_225_000071.jpg\nreal/pineapple/real_225_000074.jpg\nreal/pineapple/real_225_000205.jpg\nreal/pineapple/real_225_000286.jpg\nreal/pineapple/real_225_000444.jpg\nreal/pineapple/real_225_000462.jpg\nreal/pineapple/real_225_000500.jpg\nreal/pineapple/real_225_000531.jpg\nreal/pineapple/real_225_000648.jpg\nreal/pizza/real_226_000362.jpg\nreal/pliers/real_227_000304.jpg\nreal/police_car/real_228_000067.jpg\nreal/police_car/real_228_000101.jpg\nreal/police_car/real_228_000123.jpg\nreal/police_car/real_228_000198.jpg\nreal/police_car/real_228_000262.jpg\nreal/police_car/real_228_000530.jpg\nreal/police_car/real_228_000659.jpg\nreal/popsicle/real_231_000078.jpg\nreal/popsicle/real_231_000081.jpg\nreal/popsicle/real_231_000110.jpg\nreal/popsicle/real_231_000122.jpg\nreal/popsicle/real_231_000128.jpg\nreal/popsicle/real_231_000176.jpg\nreal/popsicle/real_231_000190.jpg\nreal/popsicle/real_231_000226.jpg\nreal/popsicle/real_231_000267.jpg\nreal/popsicle/real_231_000311.jpg\nreal/popsicle/real_231_000317.jpg\nreal/popsicle/real_231_000333.jpg\nreal/popsicle/real_231_000432.jpg\nreal/popsicle/real_231_000532.jpg\nreal/popsicle/real_231_000558.jpg\nreal/popsicle/real_231_000575.jpg\nreal/popsicle/real_231_000587.jpg\nreal/postcard/real_232_000038.jpg\nreal/postcard/real_232_000095.jpg\nreal/postcard/real_232_000098.jpg\nreal/postcard/real_232_000219.jpg\nreal/postcard/real_232_000506.jpg\nreal/power_outlet/real_234_000005.jpg\nreal/power_outlet/real_234_000171.jpg\nreal/power_outlet/real_234_000385.jpg\nreal/power_outlet/real_234_000402.jpg\nreal/power_outlet/real_234_000455.jpg\nreal/rabbit/real_236_000087.jpg\nreal/rabbit/real_236_000115.jpg\nreal/rabbit/real_236_000314.jpg\nreal/rabbit/real_236_000574.jpg\nreal/rabbit/real_236_000579.jpg\nreal/raccoon/real_237_000456.jpg\nreal/radio/real_238_000173.jpg\nreal/rain/real_239_000022.jpg\nreal/rain/real_239_000035.jpg\nreal/rain/real_239_000068.jpg\nreal/rain/real_239_000097.jpg\nreal/rain/real_239_000098.jpg\nreal/rain/real_239_000101.jpg\nreal/rain/real_239_000102.jpg\nreal/rain/real_239_000103.jpg\nreal/rain/real_239_000104.jpg\nreal/rain/real_239_000105.jpg\nreal/rain/real_239_000106.jpg\nreal/rain/real_239_000110.jpg\nreal/rain/real_239_000111.jpg\nreal/rain/real_239_000112.jpg\nreal/rain/real_239_000203.jpg\nreal/rain/real_239_000262.jpg\nreal/rain/real_239_000264.jpg\nreal/rain/real_239_000266.jpg\nreal/rain/real_239_000267.jpg\nreal/rain/real_239_000268.jpg\nreal/rain/real_239_000270.jpg\nreal/rain/real_239_000271.jpg\nreal/rain/real_239_000273.jpg\nreal/rainbow/real_240_000026.jpg\nreal/rainbow/real_240_000032.jpg\nreal/rainbow/real_240_000038.jpg\nreal/rainbow/real_240_000094.jpg\nreal/rainbow/real_240_000126.jpg\nreal/rainbow/real_240_000143.jpg\nreal/rainbow/real_240_000162.jpg\nreal/rainbow/real_240_000164.jpg\nreal/rainbow/real_240_000165.jpg\nreal/rainbow/real_240_000190.jpg\nreal/rainbow/real_240_000222.jpg\nreal/rake/real_241_000309.jpg\nreal/rake/real_241_000378.jpg\nreal/rake/real_241_000580.jpg\nreal/remote_control/real_242_000049.jpg\nreal/remote_control/real_242_000087.jpg\nreal/remote_control/real_242_000128.jpg\nreal/remote_control/real_242_000448.jpg\nreal/rhinoceros/real_243_000230.jpg\nreal/rhinoceros/real_243_000667.jpg\nreal/rifle/real_244_000239.jpg\nreal/river/real_245_000387.jpg\nreal/river/real_245_000414.jpg\nreal/roller_coaster/real_246_000256.jpg\nreal/rollerskates/real_247_000186.jpg\nreal/rollerskates/real_247_000270.jpg\nreal/rollerskates/real_247_000339.jpg\nreal/rollerskates/real_247_000401.jpg\nreal/rollerskates/real_247_000474.jpg\nreal/rollerskates/real_247_000482.jpg\nreal/sailboat/real_248_000180.jpg\nreal/sailboat/real_248_000183.jpg\nreal/sailboat/real_248_000314.jpg\nreal/sailboat/real_248_000358.jpg\nreal/sailboat/real_248_000359.jpg\nreal/sandwich/real_249_000022.jpg\nreal/sandwich/real_249_000133.jpg\nreal/sandwich/real_249_000144.jpg\nreal/sandwich/real_249_000178.jpg\nreal/sandwich/real_249_000328.jpg\nreal/sandwich/real_249_000431.jpg\nreal/sandwich/real_249_000444.jpg\nreal/saxophone/real_251_000056.jpg\nreal/saxophone/real_251_000085.jpg\nreal/saxophone/real_251_000109.jpg\nreal/saxophone/real_251_000114.jpg\nreal/saxophone/real_251_000197.jpg\nreal/saxophone/real_251_000292.jpg\nreal/saxophone/real_251_000359.jpg\nreal/saxophone/real_251_000363.jpg\nreal/saxophone/real_251_000414.jpg\nreal/saxophone/real_251_000424.jpg\nreal/school_bus/real_252_000013.jpg\nreal/school_bus/real_252_000023.jpg\nreal/school_bus/real_252_000024.jpg\nreal/school_bus/real_252_000037.jpg\nreal/school_bus/real_252_000043.jpg\nreal/school_bus/real_252_000052.jpg\nreal/school_bus/real_252_000063.jpg\nreal/school_bus/real_252_000066.jpg\nreal/school_bus/real_252_000067.jpg\nreal/school_bus/real_252_000068.jpg\nreal/school_bus/real_252_000074.jpg\nreal/school_bus/real_252_000087.jpg\nreal/school_bus/real_252_000114.jpg\nreal/school_bus/real_252_000116.jpg\nreal/school_bus/real_252_000137.jpg\nreal/school_bus/real_252_000164.jpg\nreal/school_bus/real_252_000178.jpg\nreal/school_bus/real_252_000183.jpg\nreal/school_bus/real_252_000197.jpg\nreal/school_bus/real_252_000206.jpg\nreal/school_bus/real_252_000218.jpg\nreal/school_bus/real_252_000254.jpg\nreal/school_bus/real_252_000275.jpg\nreal/school_bus/real_252_000292.jpg\nreal/school_bus/real_252_000302.jpg\nreal/school_bus/real_252_000312.jpg\nreal/school_bus/real_252_000315.jpg\nreal/school_bus/real_252_000333.jpg\nreal/school_bus/real_252_000335.jpg\nreal/school_bus/real_252_000356.jpg\nreal/school_bus/real_252_000381.jpg\nreal/school_bus/real_252_000459.jpg\nreal/school_bus/real_252_000464.jpg\nreal/scissors/real_253_000009.jpg\nreal/scissors/real_253_000337.jpg\nreal/scissors/real_253_000371.jpg\nreal/scissors/real_253_000412.jpg\nreal/scorpion/real_254_000113.jpg\nreal/scorpion/real_254_000427.jpg\nreal/screwdriver/real_255_000295.jpg\nreal/sea_turtle/real_256_000032.jpg\nreal/sea_turtle/real_256_000037.jpg\nreal/sea_turtle/real_256_000226.jpg\nreal/sea_turtle/real_256_000377.jpg\nreal/sea_turtle/real_256_000476.jpg\nreal/sea_turtle/real_256_000531.jpg\nreal/sea_turtle/real_256_000619.jpg\nreal/see_saw/real_257_000017.jpg\nreal/see_saw/real_257_000049.jpg\nreal/see_saw/real_257_000116.jpg\nreal/see_saw/real_257_000190.jpg\nreal/see_saw/real_257_000199.jpg\nreal/see_saw/real_257_000209.jpg\nreal/see_saw/real_257_000224.jpg\nreal/sheep/real_259_000212.jpg\nreal/sheep/real_259_000393.jpg\nreal/shovel/real_262_000140.jpg\nreal/shovel/real_262_000347.jpg\nreal/shovel/real_262_000430.jpg\nreal/sink/real_263_000060.jpg\nreal/skateboard/real_264_000114.jpg\nreal/skateboard/real_264_000251.jpg\nreal/skull/real_265_000028.jpg\nreal/skull/real_265_000080.jpg\nreal/skull/real_265_000123.jpg\nreal/skull/real_265_000187.jpg\nreal/skull/real_265_000213.jpg\nreal/skull/real_265_000217.jpg\nreal/skull/real_265_000239.jpg\nreal/skull/real_265_000254.jpg\nreal/skull/real_265_000262.jpg\nreal/skull/real_265_000296.jpg\nreal/skull/real_265_000304.jpg\nreal/skull/real_265_000305.jpg\nreal/sleeping_bag/real_267_000153.jpg\nreal/smiley_face/real_268_000003.jpg\nreal/smiley_face/real_268_000004.jpg\nreal/smiley_face/real_268_000019.jpg\nreal/smiley_face/real_268_000029.jpg\nreal/smiley_face/real_268_000034.jpg\nreal/smiley_face/real_268_000039.jpg\nreal/smiley_face/real_268_000043.jpg\nreal/smiley_face/real_268_000054.jpg\nreal/smiley_face/real_268_000058.jpg\nreal/smiley_face/real_268_000060.jpg\nreal/smiley_face/real_268_000069.jpg\nreal/smiley_face/real_268_000084.jpg\nreal/smiley_face/real_268_000120.jpg\nreal/smiley_face/real_268_000129.jpg\nreal/smiley_face/real_268_000133.jpg\nreal/smiley_face/real_268_000142.jpg\nreal/smiley_face/real_268_000143.jpg\nreal/smiley_face/real_268_000144.jpg\nreal/smiley_face/real_268_000152.jpg\nreal/smiley_face/real_268_000155.jpg\nreal/smiley_face/real_268_000193.jpg\nreal/snake/real_270_000006.jpg\nreal/soccer_ball/real_274_000005.jpg\nreal/soccer_ball/real_274_000037.jpg\nreal/soccer_ball/real_274_000065.jpg\nreal/soccer_ball/real_274_000072.jpg\nreal/soccer_ball/real_274_000073.jpg\nreal/soccer_ball/real_274_000120.jpg\nreal/soccer_ball/real_274_000128.jpg\nreal/soccer_ball/real_274_000130.jpg\nreal/soccer_ball/real_274_000143.jpg\nreal/soccer_ball/real_274_000153.jpg\nreal/soccer_ball/real_274_000170.jpg\nreal/soccer_ball/real_274_000196.jpg\nreal/sock/real_275_000014.jpg\nreal/sock/real_275_000370.jpg\nreal/sock/real_275_000459.jpg\nreal/speedboat/real_276_000568.jpg\nreal/spider/real_277_000016.jpg\nreal/spoon/real_278_000061.jpg\nreal/spoon/real_278_000063.jpg\nreal/spoon/real_278_000179.jpg\nreal/spoon/real_278_000330.jpg\nreal/spoon/real_278_000335.jpg\nreal/spoon/real_278_000383.jpg\nreal/spoon/real_278_000392.jpg\nreal/spoon/real_278_000407.jpg\nreal/spoon/real_278_000420.jpg\nreal/spoon/real_278_000481.jpg\nreal/spoon/real_278_000493.jpg\nreal/spoon/real_278_000519.jpg\nreal/spreadsheet/real_279_000042.jpg\nreal/square/real_280_000011.jpg\nreal/square/real_280_000015.jpg\nreal/square/real_280_000017.jpg\nreal/square/real_280_000028.jpg\nreal/square/real_280_000034.jpg\nreal/square/real_280_000035.jpg\nreal/square/real_280_000055.jpg\nreal/square/real_280_000083.jpg\nreal/squiggle/real_281_000012.jpg\nreal/squiggle/real_281_000018.jpg\nreal/squiggle/real_281_000045.jpg\nreal/squiggle/real_281_000071.jpg\nreal/stairs/real_283_000134.jpg\nreal/stairs/real_283_000239.jpg\nreal/stethoscope/real_287_000004.jpg\nreal/stethoscope/real_287_000006.jpg\nreal/stethoscope/real_287_000016.jpg\nreal/stethoscope/real_287_000035.jpg\nreal/stethoscope/real_287_000044.jpg\nreal/stethoscope/real_287_000122.jpg\nreal/stethoscope/real_287_000155.jpg\nreal/stethoscope/real_287_000173.jpg\nreal/stethoscope/real_287_000238.jpg\nreal/stethoscope/real_287_000239.jpg\nreal/stethoscope/real_287_000322.jpg\nreal/stethoscope/real_287_000343.jpg\nreal/stethoscope/real_287_000354.jpg\nreal/stethoscope/real_287_000369.jpg\nreal/stethoscope/real_287_000387.jpg\nreal/stethoscope/real_287_000402.jpg\nreal/stethoscope/real_287_000428.jpg\nreal/stethoscope/real_287_000435.jpg\nreal/stethoscope/real_287_000443.jpg\nreal/stethoscope/real_287_000449.jpg\nreal/stethoscope/real_287_000459.jpg\nreal/stethoscope/real_287_000461.jpg\nreal/stethoscope/real_287_000471.jpg\nreal/stop_sign/real_289_000138.jpg\nreal/stop_sign/real_289_000140.jpg\nreal/stove/real_290_000123.jpg\nreal/stove/real_290_000350.jpg\nreal/strawberry/real_291_000381.jpg\nreal/strawberry/real_291_000448.jpg\nreal/strawberry/real_291_000453.jpg\nreal/streetlight/real_292_000012.jpg\nreal/streetlight/real_292_000114.jpg\nreal/streetlight/real_292_000160.jpg\nreal/streetlight/real_292_000300.jpg\nreal/streetlight/real_292_000349.jpg\nreal/string_bean/real_293_000182.jpg\nreal/string_bean/real_293_000230.jpg\nreal/string_bean/real_293_000484.jpg\nreal/submarine/real_294_000195.jpg\nreal/suitcase/real_295_000092.jpg\nreal/suitcase/real_295_000139.jpg\nreal/swing_set/real_299_000300.jpg\nreal/sword/real_300_000266.jpg\nreal/sword/real_300_000276.jpg\nreal/sword/real_300_000389.jpg\nreal/sword/real_300_000400.jpg\nreal/sword/real_300_000419.jpg\nreal/sword/real_300_000431.jpg\nreal/sword/real_300_000440.jpg\nreal/syringe/real_301_000124.jpg\nreal/syringe/real_301_000278.jpg\nreal/syringe/real_301_000315.jpg\nreal/syringe/real_301_000337.jpg\nreal/syringe/real_301_000387.jpg\nreal/syringe/real_301_000547.jpg\nreal/table/real_302_000151.jpg\nreal/table/real_302_000233.jpg\nreal/table/real_302_000274.jpg\nreal/table/real_302_000433.jpg\nreal/teapot/real_303_000141.jpg\nreal/teapot/real_303_000269.jpg\nreal/teapot/real_303_000298.jpg\nreal/teapot/real_303_000303.jpg\nreal/teapot/real_303_000569.jpg\nreal/teapot/real_303_000573.jpg\nreal/teapot/real_303_000582.jpg\nreal/teddy-bear/real_304_000005.jpg\nreal/teddy-bear/real_304_000097.jpg\nreal/teddy-bear/real_304_000142.jpg\nreal/teddy-bear/real_304_000452.jpg\nreal/telephone/real_305_000084.jpg\nreal/telephone/real_305_000390.jpg\nreal/television/real_306_000106.jpg\nreal/television/real_306_000168.jpg\nreal/television/real_306_000201.jpg\nreal/television/real_306_000202.jpg\nreal/television/real_306_000229.jpg\nreal/television/real_306_000256.jpg\nreal/television/real_306_000317.jpg\nreal/television/real_306_000373.jpg\nreal/tennis_racquet/real_307_000001.jpg\nreal/tennis_racquet/real_307_000003.jpg\nreal/tennis_racquet/real_307_000043.jpg\nreal/tennis_racquet/real_307_000088.jpg\nreal/tennis_racquet/real_307_000090.jpg\nreal/tennis_racquet/real_307_000101.jpg\nreal/tennis_racquet/real_307_000117.jpg\nreal/tennis_racquet/real_307_000128.jpg\nreal/tennis_racquet/real_307_000137.jpg\nreal/tennis_racquet/real_307_000164.jpg\nreal/tennis_racquet/real_307_000209.jpg\nreal/tent/real_308_000433.jpg\nreal/tent/real_308_000468.jpg\nreal/tent/real_308_000510.jpg\nreal/tent/real_308_000521.jpg\nreal/The_Eiffel_Tower/real_309_000007.jpg\nreal/The_Eiffel_Tower/real_309_000021.jpg\nreal/The_Eiffel_Tower/real_309_000062.jpg\nreal/The_Eiffel_Tower/real_309_000098.jpg\nreal/The_Eiffel_Tower/real_309_000141.jpg\nreal/The_Eiffel_Tower/real_309_000145.jpg\nreal/The_Eiffel_Tower/real_309_000183.jpg\nreal/The_Eiffel_Tower/real_309_000189.jpg\nreal/The_Eiffel_Tower/real_309_000190.jpg\nreal/The_Eiffel_Tower/real_309_000192.jpg\nreal/The_Eiffel_Tower/real_309_000196.jpg\nreal/The_Eiffel_Tower/real_309_000199.jpg\nreal/The_Eiffel_Tower/real_309_000202.jpg\nreal/The_Eiffel_Tower/real_309_000203.jpg\nreal/The_Eiffel_Tower/real_309_000205.jpg\nreal/The_Eiffel_Tower/real_309_000210.jpg\nreal/The_Eiffel_Tower/real_309_000211.jpg\nreal/The_Eiffel_Tower/real_309_000216.jpg\nreal/The_Eiffel_Tower/real_309_000217.jpg\nreal/The_Eiffel_Tower/real_309_000222.jpg\nreal/The_Eiffel_Tower/real_309_000239.jpg\nreal/The_Eiffel_Tower/real_309_000240.jpg\nreal/The_Eiffel_Tower/real_309_000242.jpg\nreal/The_Eiffel_Tower/real_309_000250.jpg\nreal/The_Eiffel_Tower/real_309_000287.jpg\nreal/The_Eiffel_Tower/real_309_000309.jpg\nreal/The_Eiffel_Tower/real_309_000345.jpg\nreal/The_Eiffel_Tower/real_309_000379.jpg\nreal/The_Eiffel_Tower/real_309_000436.jpg\nreal/The_Eiffel_Tower/real_309_000438.jpg\nreal/The_Eiffel_Tower/real_309_000451.jpg\nreal/The_Eiffel_Tower/real_309_000521.jpg\nreal/The_Eiffel_Tower/real_309_000527.jpg\nreal/The_Eiffel_Tower/real_309_000531.jpg\nreal/The_Eiffel_Tower/real_309_000532.jpg\nreal/The_Eiffel_Tower/real_309_000534.jpg\nreal/The_Eiffel_Tower/real_309_000536.jpg\nreal/The_Eiffel_Tower/real_309_000544.jpg\nreal/The_Eiffel_Tower/real_309_000549.jpg\nreal/The_Eiffel_Tower/real_309_000552.jpg\nreal/The_Great_Wall_of_China/real_310_000044.jpg\nreal/The_Great_Wall_of_China/real_310_000084.jpg\nreal/The_Great_Wall_of_China/real_310_000147.jpg\nreal/The_Great_Wall_of_China/real_310_000158.jpg\nreal/The_Great_Wall_of_China/real_310_000238.jpg\nreal/The_Great_Wall_of_China/real_310_000296.jpg\nreal/The_Mona_Lisa/real_311_000001.jpg\nreal/The_Mona_Lisa/real_311_000002.jpg\nreal/The_Mona_Lisa/real_311_000004.jpg\nreal/The_Mona_Lisa/real_311_000006.jpg\nreal/The_Mona_Lisa/real_311_000007.jpg\nreal/The_Mona_Lisa/real_311_000008.jpg\nreal/The_Mona_Lisa/real_311_000010.jpg\nreal/The_Mona_Lisa/real_311_000011.jpg\nreal/The_Mona_Lisa/real_311_000016.jpg\nreal/The_Mona_Lisa/real_311_000017.jpg\nreal/The_Mona_Lisa/real_311_000018.jpg\nreal/The_Mona_Lisa/real_311_000019.jpg\nreal/The_Mona_Lisa/real_311_000020.jpg\nreal/The_Mona_Lisa/real_311_000021.jpg\nreal/The_Mona_Lisa/real_311_000023.jpg\nreal/The_Mona_Lisa/real_311_000027.jpg\nreal/The_Mona_Lisa/real_311_000028.jpg\nreal/The_Mona_Lisa/real_311_000031.jpg\nreal/The_Mona_Lisa/real_311_000035.jpg\nreal/The_Mona_Lisa/real_311_000037.jpg\nreal/The_Mona_Lisa/real_311_000044.jpg\nreal/The_Mona_Lisa/real_311_000045.jpg\nreal/The_Mona_Lisa/real_311_000047.jpg\nreal/The_Mona_Lisa/real_311_000051.jpg\nreal/The_Mona_Lisa/real_311_000052.jpg\nreal/The_Mona_Lisa/real_311_000054.jpg\nreal/The_Mona_Lisa/real_311_000060.jpg\nreal/The_Mona_Lisa/real_311_000062.jpg\nreal/The_Mona_Lisa/real_311_000067.jpg\nreal/The_Mona_Lisa/real_311_000069.jpg\nreal/The_Mona_Lisa/real_311_000070.jpg\nreal/The_Mona_Lisa/real_311_000072.jpg\nreal/The_Mona_Lisa/real_311_000076.jpg\nreal/The_Mona_Lisa/real_311_000077.jpg\nreal/The_Mona_Lisa/real_311_000084.jpg\nreal/The_Mona_Lisa/real_311_000086.jpg\nreal/The_Mona_Lisa/real_311_000090.jpg\nreal/The_Mona_Lisa/real_311_000095.jpg\nreal/The_Mona_Lisa/real_311_000097.jpg\nreal/The_Mona_Lisa/real_311_000107.jpg\nreal/The_Mona_Lisa/real_311_000109.jpg\nreal/The_Mona_Lisa/real_311_000113.jpg\nreal/The_Mona_Lisa/real_311_000115.jpg\nreal/The_Mona_Lisa/real_311_000118.jpg\nreal/The_Mona_Lisa/real_311_000120.jpg\nreal/The_Mona_Lisa/real_311_000123.jpg\nreal/The_Mona_Lisa/real_311_000125.jpg\nreal/The_Mona_Lisa/real_311_000127.jpg\nreal/The_Mona_Lisa/real_311_000140.jpg\nreal/The_Mona_Lisa/real_311_000143.jpg\nreal/The_Mona_Lisa/real_311_000148.jpg\nreal/The_Mona_Lisa/real_311_000158.jpg\nreal/The_Mona_Lisa/real_311_000160.jpg\nreal/The_Mona_Lisa/real_311_000161.jpg\nreal/The_Mona_Lisa/real_311_000165.jpg\nreal/The_Mona_Lisa/real_311_000166.jpg\nreal/The_Mona_Lisa/real_311_000170.jpg\nreal/The_Mona_Lisa/real_311_000171.jpg\nreal/The_Mona_Lisa/real_311_000172.jpg\nreal/The_Mona_Lisa/real_311_000186.jpg\nreal/The_Mona_Lisa/real_311_000211.jpg\nreal/The_Mona_Lisa/real_311_000220.jpg\nreal/The_Mona_Lisa/real_311_000222.jpg\nreal/The_Mona_Lisa/real_311_000230.jpg\nreal/The_Mona_Lisa/real_311_000243.jpg\nreal/The_Mona_Lisa/real_311_000244.jpg\nreal/The_Mona_Lisa/real_311_000251.jpg\nreal/The_Mona_Lisa/real_311_000271.jpg\nreal/The_Mona_Lisa/real_311_000273.jpg\nreal/tiger/real_312_000063.jpg\nreal/tiger/real_312_000120.jpg\nreal/tiger/real_312_000129.jpg\nreal/tiger/real_312_000198.jpg\nreal/tiger/real_312_000233.jpg\nreal/tiger/real_312_000246.jpg\nreal/tiger/real_312_000259.jpg\nreal/tiger/real_312_000310.jpg\nreal/tiger/real_312_000313.jpg\nreal/tiger/real_312_000320.jpg\nreal/tiger/real_312_000324.jpg\nreal/tiger/real_312_000358.jpg\nreal/tiger/real_312_000372.jpg\nreal/tiger/real_312_000599.jpg\nreal/toaster/real_313_000069.jpg\nreal/toaster/real_313_000089.jpg\nreal/toaster/real_313_000163.jpg\nreal/toaster/real_313_000236.jpg\nreal/toaster/real_313_000344.jpg\nreal/toaster/real_313_000346.jpg\nreal/toaster/real_313_000387.jpg\nreal/toaster/real_313_000401.jpg\nreal/toaster/real_313_000402.jpg\nreal/toaster/real_313_000450.jpg\nreal/toaster/real_313_000497.jpg\nreal/toe/real_314_000033.jpg\nreal/toe/real_314_000076.jpg\nreal/toe/real_314_000115.jpg\nreal/toe/real_314_000179.jpg\nreal/toe/real_314_000196.jpg\nreal/toe/real_314_000221.jpg\nreal/toe/real_314_000249.jpg\nreal/toe/real_314_000261.jpg\nreal/toe/real_314_000271.jpg\nreal/toe/real_314_000278.jpg\nreal/toe/real_314_000301.jpg\nreal/toe/real_314_000302.jpg\nreal/toe/real_314_000339.jpg\nreal/toilet/real_315_000443.jpg\nreal/toilet/real_315_000468.jpg\nreal/tooth/real_316_000006.jpg\nreal/tooth/real_316_000084.jpg\nreal/toothbrush/real_317_000079.jpg\nreal/toothbrush/real_317_000122.jpg\nreal/toothbrush/real_317_000195.jpg\nreal/toothbrush/real_317_000409.jpg\nreal/toothpaste/real_318_000342.jpg\nreal/tornado/real_319_000002.jpg\nreal/tornado/real_319_000004.jpg\nreal/tornado/real_319_000025.jpg\nreal/tornado/real_319_000093.jpg\nreal/tornado/real_319_000094.jpg\nreal/tornado/real_319_000095.jpg\nreal/tornado/real_319_000096.jpg\nreal/tornado/real_319_000098.jpg\nreal/tornado/real_319_000099.jpg\nreal/tornado/real_319_000100.jpg\nreal/tornado/real_319_000101.jpg\nreal/tornado/real_319_000102.jpg\nreal/tornado/real_319_000103.jpg\nreal/tornado/real_319_000104.jpg\nreal/tornado/real_319_000106.jpg\nreal/tornado/real_319_000107.jpg\nreal/tornado/real_319_000108.jpg\nreal/tornado/real_319_000110.jpg\nreal/tornado/real_319_000111.jpg\nreal/tornado/real_319_000137.jpg\nreal/tornado/real_319_000173.jpg\nreal/tornado/real_319_000275.jpg\nreal/tornado/real_319_000279.jpg\nreal/tornado/real_319_000295.jpg\nreal/tornado/real_319_000351.jpg\nreal/tornado/real_319_000497.jpg\nreal/traffic_light/real_321_000020.jpg\nreal/traffic_light/real_321_000056.jpg\nreal/traffic_light/real_321_000083.jpg\nreal/traffic_light/real_321_000132.jpg\nreal/traffic_light/real_321_000175.jpg\nreal/traffic_light/real_321_000186.jpg\nreal/traffic_light/real_321_000219.jpg\nreal/traffic_light/real_321_000285.jpg\nreal/train/real_322_000301.jpg\nreal/tree/real_323_000089.jpg\nreal/tree/real_323_000104.jpg\nreal/tree/real_323_000242.jpg\nreal/tree/real_323_000515.jpg\nreal/tree/real_323_000522.jpg\nreal/tree/real_323_000528.jpg\nreal/triangle/real_324_000007.jpg\nreal/triangle/real_324_000010.jpg\nreal/triangle/real_324_000016.jpg\nreal/triangle/real_324_000020.jpg\nreal/triangle/real_324_000050.jpg\nreal/triangle/real_324_000075.jpg\nreal/triangle/real_324_000093.jpg\nreal/triangle/real_324_000105.jpg\nreal/triangle/real_324_000178.jpg\nreal/triangle/real_324_000243.jpg\nreal/triangle/real_324_000259.jpg\nreal/triangle/real_324_000279.jpg\nreal/triangle/real_324_000293.jpg\nreal/triangle/real_324_000311.jpg\nreal/triangle/real_324_000373.jpg\nreal/triangle/real_324_000375.jpg\nreal/triangle/real_324_000376.jpg\nreal/trombone/real_325_000041.jpg\nreal/trombone/real_325_000075.jpg\nreal/trombone/real_325_000185.jpg\nreal/trombone/real_325_000291.jpg\nreal/truck/real_326_000121.jpg\nreal/truck/real_326_000135.jpg\nreal/truck/real_326_000198.jpg\nreal/truck/real_326_000233.jpg\nreal/truck/real_326_000239.jpg\nreal/truck/real_326_000259.jpg\nreal/truck/real_326_000264.jpg\nreal/truck/real_326_000334.jpg\nreal/truck/real_326_000340.jpg\nreal/truck/real_326_000366.jpg\nreal/truck/real_326_000395.jpg\nreal/truck/real_326_000428.jpg\nreal/truck/real_326_000460.jpg\nreal/truck/real_326_000497.jpg\nreal/truck/real_326_000503.jpg\nreal/truck/real_326_000540.jpg\nreal/truck/real_326_000542.jpg\nreal/truck/real_326_000594.jpg\nreal/truck/real_326_000617.jpg\nreal/trumpet/real_327_000046.jpg\nreal/trumpet/real_327_000108.jpg\nreal/trumpet/real_327_000147.jpg\nreal/trumpet/real_327_000275.jpg\nreal/trumpet/real_327_000295.jpg\nreal/trumpet/real_327_000309.jpg\nreal/trumpet/real_327_000326.jpg\nreal/trumpet/real_327_000384.jpg\nreal/umbrella/real_329_000025.jpg\nreal/umbrella/real_329_000077.jpg\nreal/umbrella/real_329_000130.jpg\nreal/umbrella/real_329_000160.jpg\nreal/umbrella/real_329_000204.jpg\nreal/umbrella/real_329_000218.jpg\nreal/umbrella/real_329_000232.jpg\nreal/umbrella/real_329_000303.jpg\nreal/umbrella/real_329_000308.jpg\nreal/umbrella/real_329_000328.jpg\nreal/umbrella/real_329_000338.jpg\nreal/umbrella/real_329_000347.jpg\nreal/van/real_331_000027.jpg\nreal/van/real_331_000048.jpg\nreal/van/real_331_000370.jpg\nreal/van/real_331_000382.jpg\nreal/vase/real_332_000034.jpg\nreal/vase/real_332_000382.jpg\nreal/vase/real_332_000423.jpg\nreal/vase/real_332_000429.jpg\nreal/vase/real_332_000514.jpg\nreal/vase/real_332_000562.jpg\nreal/violin/real_333_000199.jpg\nreal/violin/real_333_000265.jpg\nreal/violin/real_333_000324.jpg\nreal/violin/real_333_000349.jpg\nreal/violin/real_333_000353.jpg\nreal/violin/real_333_000354.jpg\nreal/washing_machine/real_334_000126.jpg\nreal/washing_machine/real_334_000371.jpg\nreal/washing_machine/real_334_000426.jpg\nreal/watermelon/real_335_000067.jpg\nreal/watermelon/real_335_000097.jpg\nreal/watermelon/real_335_000121.jpg\nreal/watermelon/real_335_000196.jpg\nreal/watermelon/real_335_000242.jpg\nreal/watermelon/real_335_000244.jpg\nreal/watermelon/real_335_000261.jpg\nreal/watermelon/real_335_000293.jpg\nreal/watermelon/real_335_000472.jpg\nreal/watermelon/real_335_000534.jpg\nreal/waterslide/real_336_000069.jpg\nreal/waterslide/real_336_000584.jpg\nreal/whale/real_337_000086.jpg\nreal/whale/real_337_000299.jpg\nreal/whale/real_337_000344.jpg\nreal/whale/real_337_000406.jpg\nreal/whale/real_337_000486.jpg\nreal/whale/real_337_000494.jpg\nreal/whale/real_337_000583.jpg\nreal/whale/real_337_000588.jpg\nreal/wheel/real_338_000006.jpg\nreal/wheel/real_338_000086.jpg\nreal/wheel/real_338_000094.jpg\nreal/wheel/real_338_000101.jpg\nreal/wheel/real_338_000112.jpg\nreal/wheel/real_338_000174.jpg\nreal/wheel/real_338_000187.jpg\nreal/wheel/real_338_000302.jpg\nreal/wheel/real_338_000337.jpg\nreal/wheel/real_338_000387.jpg\nreal/windmill/real_339_000082.jpg\nreal/windmill/real_339_000128.jpg\nreal/windmill/real_339_000202.jpg\nreal/windmill/real_339_000499.jpg\nreal/windmill/real_339_000552.jpg\nreal/windmill/real_339_000589.jpg\nreal/wine_bottle/real_340_000030.jpg\nreal/wine_bottle/real_340_000034.jpg\nreal/wine_bottle/real_340_000088.jpg\nreal/wine_bottle/real_340_000138.jpg\nreal/wine_bottle/real_340_000152.jpg\nreal/wine_bottle/real_340_000183.jpg\nreal/wine_bottle/real_340_000215.jpg\nreal/wine_bottle/real_340_000281.jpg\nreal/wine_glass/real_341_000030.jpg\nreal/wine_glass/real_341_000053.jpg\nreal/wine_glass/real_341_000090.jpg\nreal/wine_glass/real_341_000094.jpg\nreal/wine_glass/real_341_000179.jpg\nreal/wine_glass/real_341_000180.jpg\nreal/wine_glass/real_341_000181.jpg\nreal/wine_glass/real_341_000236.jpg\nreal/wine_glass/real_341_000237.jpg\nreal/wine_glass/real_341_000258.jpg\nreal/wine_glass/real_341_000275.jpg\nreal/wristwatch/real_342_000229.jpg\nreal/wristwatch/real_342_000300.jpg\nreal/wristwatch/real_342_000338.jpg\nreal/wristwatch/real_342_000388.jpg\nreal/wristwatch/real_342_000544.jpg\nreal/wristwatch/real_342_000548.jpg\nreal/yoga/real_343_000143.jpg\nreal/yoga/real_343_000198.jpg\nreal/yoga/real_343_000233.jpg\nreal/yoga/real_343_000284.jpg\nreal/yoga/real_343_000292.jpg\nreal/yoga/real_343_000293.jpg\nreal/yoga/real_343_000296.jpg\nreal/yoga/real_343_000297.jpg\nreal/yoga/real_343_000313.jpg\nreal/zebra/real_344_000095.jpg\nreal/zebra/real_344_000121.jpg\nreal/zebra/real_344_000138.jpg\nreal/zebra/real_344_000145.jpg\nreal/zebra/real_344_000188.jpg\nreal/zebra/real_344_000258.jpg\nreal/zebra/real_344_000266.jpg\nreal/zebra/real_344_000350.jpg\nreal/zebra/real_344_000383.jpg\nreal/zebra/real_344_000531.jpg\nreal/zebra/real_344_000551.jpg\nreal/zebra/real_344_000559.jpg\nreal/zebra/real_344_000614.jpg\nreal/zebra/real_344_000620.jpg\nreal/zigzag/real_345_000004.jpg\nreal/zigzag/real_345_000006.jpg\nreal/zigzag/real_345_000009.jpg\nreal/zigzag/real_345_000012.jpg\nreal/zigzag/real_345_000016.jpg\nreal/zigzag/real_345_000017.jpg\nreal/zigzag/real_345_000026.jpg\nreal/zigzag/real_345_000032.jpg\nreal/zigzag/real_345_000096.jpg\nreal/zigzag/real_345_000116.jpg\nreal/zigzag/real_345_000125.jpg\nreal/zigzag/real_345_000127.jpg\nreal/zigzag/real_345_000139.jpg\nreal/zigzag/real_345_000140.jpg\nreal/zigzag/real_345_000160.jpg\nreal/zigzag/real_345_000164.jpg\nreal/zigzag/real_345_000166.jpg\nreal/zigzag/real_345_000176.jpg\nreal/zigzag/real_345_000178.jpg\nreal/zigzag/real_345_000241.jpg\nreal/zigzag/real_345_000247.jpg\nreal/zigzag/real_345_000248.jpg\nreal/zigzag/real_345_000252.jpg\nreal/zigzag/real_345_000275.jpg\nreal/zigzag/real_345_000340.jpg\nreal/zigzag/real_345_000354.jpg\nreal/zigzag/real_345_000384.jpg\nreal/zigzag/real_345_000423.jpg\nreal/zigzag/real_345_000443.jpg\nreal/zigzag/real_345_000460.jpg\nreal/zigzag/real_345_000470.jpg\nreal/zigzag/real_345_000498.jpg\nreal/zigzag/real_345_000510.jpg\nreal/zigzag/real_345_000511.jpg\nreal/zigzag/real_345_000513.jpg\nsketch/angel/sketch_005_000012.jpg\nsketch/animal_migration/sketch_006_000076.jpg\nsketch/anvil/sketch_008_000050.jpg\nsketch/anvil/sketch_008_000091.jpg\nsketch/asparagus/sketch_011_000041.jpg\nsketch/asparagus/sketch_011_000099.jpg\nsketch/axe/sketch_012_000030.jpg\nsketch/axe/sketch_012_000106.jpg\nsketch/axe/sketch_012_000155.jpg\nsketch/axe/sketch_012_000157.jpg\nsketch/axe/sketch_012_000163.jpg\nsketch/axe/sketch_012_000165.jpg\nsketch/axe/sketch_012_000167.jpg\nsketch/axe/sketch_012_000168.jpg\nsketch/axe/sketch_012_000169.jpg\nsketch/axe/sketch_012_000170.jpg\nsketch/axe/sketch_012_000172.jpg\nsketch/axe/sketch_012_000177.jpg\nsketch/axe/sketch_012_000180.jpg\nsketch/axe/sketch_012_000199.jpg\nsketch/axe/sketch_012_000202.jpg\nsketch/backpack/sketch_013_000031.jpg\nsketch/banana/sketch_014_000161.jpg\nsketch/baseball_bat/sketch_018_000061.jpg\nsketch/baseball_bat/sketch_018_000082.jpg\nsketch/baseball_bat/sketch_018_000131.jpg\nsketch/baseball_bat/sketch_018_000189.jpg\nsketch/basket/sketch_019_000124.jpg\nsketch/basket/sketch_019_000150.jpg\nsketch/basket/sketch_019_000175.jpg\nsketch/basketball/sketch_020_000096.jpg\nsketch/bathtub/sketch_022_000001.jpg\nsketch/beard/sketch_025_000221.jpg\nsketch/bench/sketch_029_000067.jpg\nsketch/bench/sketch_029_000069.jpg\nsketch/bench/sketch_029_000074.jpg\nsketch/bench/sketch_029_000079.jpg\nsketch/bench/sketch_029_000178.jpg\nsketch/bench/sketch_029_000179.jpg\nsketch/bench/sketch_029_000183.jpg\nsketch/bench/sketch_029_000184.jpg\nsketch/bench/sketch_029_000188.jpg\nsketch/bench/sketch_029_000189.jpg\nsketch/bench/sketch_029_000190.jpg\nsketch/bench/sketch_029_000191.jpg\nsketch/bench/sketch_029_000195.jpg\nsketch/bench/sketch_029_000199.jpg\nsketch/bench/sketch_029_000200.jpg\nsketch/bench/sketch_029_000201.jpg\nsketch/bench/sketch_029_000202.jpg\nsketch/bench/sketch_029_000203.jpg\nsketch/bench/sketch_029_000214.jpg\nsketch/binoculars/sketch_031_000015.jpg\nsketch/binoculars/sketch_031_000053.jpg\nsketch/binoculars/sketch_031_000096.jpg\nsketch/binoculars/sketch_031_000176.jpg\nsketch/binoculars/sketch_031_000197.jpg\nsketch/blueberry/sketch_035_000072.jpg\nsketch/boomerang/sketch_037_000047.jpg\nsketch/boomerang/sketch_037_000058.jpg\nsketch/bowtie/sketch_039_000231.jpg\nsketch/bowtie/sketch_039_000291.jpg\nsketch/bowtie/sketch_039_000320.jpg\nsketch/bracelet/sketch_040_000035.jpg\nsketch/bread/sketch_042_000029.jpg\nsketch/bridge/sketch_043_000167.jpg\nsketch/broccoli/sketch_044_000063.jpg\nsketch/broom/sketch_045_000219.jpg\nsketch/bus/sketch_048_000160.jpg\nsketch/bus/sketch_048_000164.jpg\nsketch/bus/sketch_048_000168.jpg\nsketch/bus/sketch_048_000169.jpg\nsketch/bus/sketch_048_000171.jpg\nsketch/bus/sketch_048_000172.jpg\nsketch/bus/sketch_048_000173.jpg\nsketch/bus/sketch_048_000175.jpg\nsketch/bus/sketch_048_000177.jpg\nsketch/bus/sketch_048_000180.jpg\nsketch/bus/sketch_048_000183.jpg\nsketch/bush/sketch_049_000029.jpg\nsketch/bush/sketch_049_000039.jpg\nsketch/bush/sketch_049_000047.jpg\nsketch/bush/sketch_049_000102.jpg\nsketch/bush/sketch_049_000171.jpg\nsketch/bush/sketch_049_000500.jpg\nsketch/bush/sketch_049_000599.jpg\nsketch/cake/sketch_052_000001.jpg\nsketch/cake/sketch_052_000002.jpg\nsketch/cake/sketch_052_000003.jpg\nsketch/cake/sketch_052_000007.jpg\nsketch/cake/sketch_052_000010.jpg\nsketch/cake/sketch_052_000022.jpg\nsketch/cake/sketch_052_000049.jpg\nsketch/cake/sketch_052_000050.jpg\nsketch/cake/sketch_052_000059.jpg\nsketch/cake/sketch_052_000076.jpg\nsketch/campfire/sketch_058_000050.jpg\nsketch/campfire/sketch_058_000072.jpg\nsketch/clarinet/sketch_073_000009.jpg\nsketch/clarinet/sketch_073_000015.jpg\nsketch/clarinet/sketch_073_000018.jpg\nsketch/compass/sketch_077_000013.jpg\nsketch/computer/sketch_078_000011.jpg\nsketch/computer/sketch_078_000025.jpg\nsketch/cooler/sketch_080_000021.jpg\nsketch/cooler/sketch_080_000035.jpg\nsketch/cooler/sketch_080_000078.jpg\nsketch/cooler/sketch_080_000086.jpg\nsketch/crown/sketch_086_000084.jpg\nsketch/crown/sketch_086_000107.jpg\nsketch/crown/sketch_086_000142.jpg\nsketch/cruise_ship/sketch_087_000130.jpg\nsketch/cruise_ship/sketch_087_000145.jpg\nsketch/cruise_ship/sketch_087_000147.jpg\nsketch/cup/sketch_088_000008.jpg\nsketch/cup/sketch_088_000035.jpg\nsketch/cup/sketch_088_000137.jpg\nsketch/cup/sketch_088_000229.jpg\nsketch/cup/sketch_088_000231.jpg\nsketch/cup/sketch_088_000260.jpg\nsketch/cup/sketch_088_000357.jpg\nsketch/cup/sketch_088_000366.jpg\nsketch/diamond/sketch_089_000078.jpg\nsketch/diamond/sketch_089_000102.jpg\nsketch/dishwasher/sketch_090_000019.jpg\nsketch/diving_board/sketch_091_000001.jpg\nsketch/diving_board/sketch_091_000016.jpg\nsketch/diving_board/sketch_091_000021.jpg\nsketch/diving_board/sketch_091_000030.jpg\nsketch/diving_board/sketch_091_000033.jpg\nsketch/diving_board/sketch_091_000038.jpg\nsketch/dog/sketch_092_000300.jpg\nsketch/door/sketch_095_000095.jpg\nsketch/dragon/sketch_096_000051.jpg\nsketch/duck/sketch_100_000274.jpg\nsketch/dumbbell/sketch_101_000054.jpg\nsketch/dumbbell/sketch_101_000125.jpg\nsketch/dumbbell/sketch_101_000143.jpg\nsketch/dumbbell/sketch_101_000161.jpg\nsketch/dumbbell/sketch_101_000173.jpg\nsketch/dumbbell/sketch_101_000189.jpg\nsketch/elbow/sketch_103_000029.jpg\nsketch/elbow/sketch_103_000086.jpg\nsketch/elbow/sketch_103_000115.jpg\nsketch/elbow/sketch_103_000152.jpg\nsketch/envelope/sketch_105_000004.jpg\nsketch/envelope/sketch_105_000065.jpg\nsketch/envelope/sketch_105_000156.jpg\nsketch/eraser/sketch_106_000011.jpg\nsketch/eraser/sketch_106_000045.jpg\nsketch/eraser/sketch_106_000048.jpg\nsketch/eyeglasses/sketch_108_000032.jpg\nsketch/eyeglasses/sketch_108_000129.jpg\nsketch/eyeglasses/sketch_108_000207.jpg\nsketch/face/sketch_109_000120.jpg\nsketch/face/sketch_109_000229.jpg\nsketch/face/sketch_109_000304.jpg\nsketch/face/sketch_109_000379.jpg\nsketch/fan/sketch_110_000007.jpg\nsketch/fan/sketch_110_000014.jpg\nsketch/fan/sketch_110_000031.jpg\nsketch/fan/sketch_110_000034.jpg\nsketch/fan/sketch_110_000038.jpg\nsketch/fan/sketch_110_000049.jpg\nsketch/fan/sketch_110_000051.jpg\nsketch/fan/sketch_110_000053.jpg\nsketch/feather/sketch_111_000072.jpg\nsketch/feather/sketch_111_000088.jpg\nsketch/feather/sketch_111_000092.jpg\nsketch/feather/sketch_111_000094.jpg\nsketch/feather/sketch_111_000095.jpg\nsketch/feather/sketch_111_000097.jpg\nsketch/feather/sketch_111_000098.jpg\nsketch/feather/sketch_111_000099.jpg\nsketch/feather/sketch_111_000102.jpg\nsketch/feather/sketch_111_000104.jpg\nsketch/feather/sketch_111_000105.jpg\nsketch/feather/sketch_111_000108.jpg\nsketch/feather/sketch_111_000118.jpg\nsketch/feather/sketch_111_000123.jpg\nsketch/feather/sketch_111_000126.jpg\nsketch/feather/sketch_111_000147.jpg\nsketch/feather/sketch_111_000165.jpg\nsketch/feather/sketch_111_000169.jpg\nsketch/feather/sketch_111_000176.jpg\nsketch/feather/sketch_111_000180.jpg\nsketch/feather/sketch_111_000188.jpg\nsketch/feather/sketch_111_000213.jpg\nsketch/feather/sketch_111_000269.jpg\nsketch/feather/sketch_111_000282.jpg\nsketch/feather/sketch_111_000293.jpg\nsketch/feather/sketch_111_000312.jpg\nsketch/feather/sketch_111_000319.jpg\nsketch/feather/sketch_111_000322.jpg\nsketch/feather/sketch_111_000329.jpg\nsketch/feather/sketch_111_000333.jpg\nsketch/feather/sketch_111_000335.jpg\nsketch/fire_hydrant/sketch_114_000027.jpg\nsketch/fire_hydrant/sketch_114_000067.jpg\nsketch/fire_hydrant/sketch_114_000103.jpg\nsketch/fire_hydrant/sketch_114_000104.jpg\nsketch/fire_hydrant/sketch_114_000106.jpg\nsketch/fire_hydrant/sketch_114_000133.jpg\nsketch/fireplace/sketch_115_000010.jpg\nsketch/fireplace/sketch_115_000081.jpg\nsketch/fish/sketch_117_000142.jpg\nsketch/fish/sketch_117_000369.jpg\nsketch/flashlight/sketch_119_000026.jpg\nsketch/flashlight/sketch_119_000061.jpg\nsketch/flashlight/sketch_119_000087.jpg\nsketch/flip_flops/sketch_120_000083.jpg\nsketch/floor_lamp/sketch_121_000026.jpg\nsketch/floor_lamp/sketch_121_000072.jpg\nsketch/floor_lamp/sketch_121_000124.jpg\nsketch/floor_lamp/sketch_121_000267.jpg\nsketch/flying_saucer/sketch_123_000020.jpg\nsketch/flying_saucer/sketch_123_000030.jpg\nsketch/flying_saucer/sketch_123_000043.jpg\nsketch/flying_saucer/sketch_123_000050.jpg\nsketch/foot/sketch_124_000006.jpg\nsketch/foot/sketch_124_000248.jpg\nsketch/foot/sketch_124_000252.jpg\nsketch/foot/sketch_124_000256.jpg\nsketch/foot/sketch_124_000258.jpg\nsketch/foot/sketch_124_000260.jpg\nsketch/frying_pan/sketch_127_000120.jpg\nsketch/garden/sketch_128_000077.jpg\nsketch/goatee/sketch_131_000020.jpg\nsketch/goatee/sketch_131_000029.jpg\nsketch/goatee/sketch_131_000037.jpg\nsketch/goatee/sketch_131_000038.jpg\nsketch/goatee/sketch_131_000042.jpg\nsketch/goatee/sketch_131_000044.jpg\nsketch/goatee/sketch_131_000049.jpg\nsketch/goatee/sketch_131_000052.jpg\nsketch/goatee/sketch_131_000057.jpg\nsketch/goatee/sketch_131_000058.jpg\nsketch/goatee/sketch_131_000067.jpg\nsketch/goatee/sketch_131_000072.jpg\nsketch/goatee/sketch_131_000074.jpg\nsketch/goatee/sketch_131_000076.jpg\nsketch/goatee/sketch_131_000083.jpg\nsketch/goatee/sketch_131_000085.jpg\nsketch/goatee/sketch_131_000088.jpg\nsketch/goatee/sketch_131_000090.jpg\nsketch/goatee/sketch_131_000097.jpg\nsketch/goatee/sketch_131_000099.jpg\nsketch/goatee/sketch_131_000105.jpg\nsketch/goatee/sketch_131_000117.jpg\nsketch/goatee/sketch_131_000118.jpg\nsketch/goatee/sketch_131_000123.jpg\nsketch/goatee/sketch_131_000128.jpg\nsketch/goatee/sketch_131_000136.jpg\nsketch/goatee/sketch_131_000137.jpg\nsketch/goatee/sketch_131_000140.jpg\nsketch/goatee/sketch_131_000148.jpg\nsketch/goatee/sketch_131_000160.jpg\nsketch/goatee/sketch_131_000180.jpg\nsketch/goatee/sketch_131_000187.jpg\nsketch/goatee/sketch_131_000189.jpg\nsketch/goatee/sketch_131_000191.jpg\nsketch/goatee/sketch_131_000192.jpg\nsketch/goatee/sketch_131_000197.jpg\nsketch/goatee/sketch_131_000200.jpg\nsketch/goatee/sketch_131_000206.jpg\nsketch/golf_club/sketch_132_000012.jpg\nsketch/golf_club/sketch_132_000064.jpg\nsketch/golf_club/sketch_132_000307.jpg\nsketch/golf_club/sketch_132_000308.jpg\nsketch/golf_club/sketch_132_000484.jpg\nsketch/grass/sketch_134_000072.jpg\nsketch/guitar/sketch_135_000100.jpg\nsketch/guitar/sketch_135_000131.jpg\nsketch/hamburger/sketch_136_000068.jpg\nsketch/hamburger/sketch_136_000155.jpg\nsketch/hand/sketch_138_000021.jpg\nsketch/hand/sketch_138_000022.jpg\nsketch/hand/sketch_138_000028.jpg\nsketch/hand/sketch_138_000029.jpg\nsketch/hand/sketch_138_000033.jpg\nsketch/hand/sketch_138_000034.jpg\nsketch/hand/sketch_138_000063.jpg\nsketch/hand/sketch_138_000069.jpg\nsketch/hand/sketch_138_000080.jpg\nsketch/hand/sketch_138_000086.jpg\nsketch/hand/sketch_138_000091.jpg\nsketch/hand/sketch_138_000109.jpg\nsketch/hand/sketch_138_000111.jpg\nsketch/hand/sketch_138_000134.jpg\nsketch/hand/sketch_138_000167.jpg\nsketch/hand/sketch_138_000173.jpg\nsketch/hand/sketch_138_000179.jpg\nsketch/hand/sketch_138_000183.jpg\nsketch/hand/sketch_138_000186.jpg\nsketch/hand/sketch_138_000188.jpg\nsketch/hand/sketch_138_000209.jpg\nsketch/hand/sketch_138_000215.jpg\nsketch/hand/sketch_138_000216.jpg\nsketch/hand/sketch_138_000229.jpg\nsketch/hand/sketch_138_000243.jpg\nsketch/hand/sketch_138_000260.jpg\nsketch/hand/sketch_138_000262.jpg\nsketch/hand/sketch_138_000264.jpg\nsketch/headphones/sketch_141_000039.jpg\nsketch/helicopter/sketch_143_000025.jpg\nsketch/helicopter/sketch_143_000096.jpg\nsketch/helicopter/sketch_143_000176.jpg\nsketch/helmet/sketch_144_000146.jpg\nsketch/hexagon/sketch_145_000029.jpg\nsketch/hockey_puck/sketch_146_000011.jpg\nsketch/hockey_puck/sketch_146_000024.jpg\nsketch/hockey_puck/sketch_146_000047.jpg\nsketch/hockey_puck/sketch_146_000059.jpg\nsketch/hockey_puck/sketch_146_000086.jpg\nsketch/hockey_puck/sketch_146_000090.jpg\nsketch/hockey_stick/sketch_147_000009.jpg\nsketch/hockey_stick/sketch_147_000019.jpg\nsketch/hockey_stick/sketch_147_000022.jpg\nsketch/hockey_stick/sketch_147_000030.jpg\nsketch/hockey_stick/sketch_147_000042.jpg\nsketch/hockey_stick/sketch_147_000048.jpg\nsketch/hockey_stick/sketch_147_000058.jpg\nsketch/hockey_stick/sketch_147_000071.jpg\nsketch/hockey_stick/sketch_147_000073.jpg\nsketch/hockey_stick/sketch_147_000080.jpg\nsketch/hockey_stick/sketch_147_000097.jpg\nsketch/hockey_stick/sketch_147_000103.jpg\nsketch/hockey_stick/sketch_147_000106.jpg\nsketch/hockey_stick/sketch_147_000109.jpg\nsketch/hockey_stick/sketch_147_000110.jpg\nsketch/hockey_stick/sketch_147_000111.jpg\nsketch/hockey_stick/sketch_147_000114.jpg\nsketch/hockey_stick/sketch_147_000116.jpg\nsketch/horse/sketch_148_000054.jpg\nsketch/horse/sketch_148_000093.jpg\nsketch/hot_dog/sketch_151_000040.jpg\nsketch/hot_dog/sketch_151_000054.jpg\nsketch/hot_dog/sketch_151_000081.jpg\nsketch/hot_dog/sketch_151_000084.jpg\nsketch/hot_tub/sketch_152_000032.jpg\nsketch/hourglass/sketch_153_000115.jpg\nsketch/hurricane/sketch_156_000040.jpg\nsketch/hurricane/sketch_156_000079.jpg\nsketch/hurricane/sketch_156_000088.jpg\nsketch/ice_cream/sketch_157_000037.jpg\nsketch/jacket/sketch_158_000007.jpg\nsketch/jacket/sketch_158_000015.jpg\nsketch/jacket/sketch_158_000044.jpg\nsketch/jacket/sketch_158_000083.jpg\nsketch/jacket/sketch_158_000084.jpg\nsketch/kangaroo/sketch_160_000060.jpg\nsketch/knee/sketch_163_000073.jpg\nsketch/knife/sketch_164_000057.jpg\nsketch/knife/sketch_164_000071.jpg\nsketch/knife/sketch_164_000072.jpg\nsketch/knife/sketch_164_000076.jpg\nsketch/knife/sketch_164_000112.jpg\nsketch/ladder/sketch_165_000124.jpg\nsketch/ladder/sketch_165_000185.jpg\nsketch/lantern/sketch_166_000037.jpg\nsketch/lantern/sketch_166_000039.jpg\nsketch/laptop/sketch_167_000067.jpg\nsketch/laptop/sketch_167_000106.jpg\nsketch/laptop/sketch_167_000108.jpg\nsketch/laptop/sketch_167_000109.jpg\nsketch/laptop/sketch_167_000147.jpg\nsketch/laptop/sketch_167_000209.jpg\nsketch/laptop/sketch_167_000295.jpg\nsketch/leaf/sketch_168_000079.jpg\nsketch/leaf/sketch_168_000204.jpg\nsketch/leg/sketch_169_000002.jpg\nsketch/leg/sketch_169_000003.jpg\nsketch/leg/sketch_169_000004.jpg\nsketch/leg/sketch_169_000007.jpg\nsketch/leg/sketch_169_000024.jpg\nsketch/leg/sketch_169_000029.jpg\nsketch/leg/sketch_169_000078.jpg\nsketch/leg/sketch_169_000123.jpg\nsketch/leg/sketch_169_000128.jpg\nsketch/leg/sketch_169_000142.jpg\nsketch/lighthouse/sketch_172_000100.jpg\nsketch/lighthouse/sketch_172_000216.jpg\nsketch/lightning/sketch_173_000001.jpg\nsketch/lightning/sketch_173_000012.jpg\nsketch/lightning/sketch_173_000013.jpg\nsketch/lightning/sketch_173_000083.jpg\nsketch/line/sketch_174_000020.jpg\nsketch/lion/sketch_175_000021.jpg\nsketch/lobster/sketch_177_000040.jpg\nsketch/lobster/sketch_177_000119.jpg\nsketch/lollipop/sketch_178_000101.jpg\nsketch/mailbox/sketch_179_000097.jpg\nsketch/matches/sketch_182_000032.jpg\nsketch/matches/sketch_182_000046.jpg\nsketch/microphone/sketch_185_000015.jpg\nsketch/microphone/sketch_185_000077.jpg\nsketch/microphone/sketch_185_000121.jpg\nsketch/mosquito/sketch_189_000043.jpg\nsketch/mountain/sketch_191_000054.jpg\nsketch/moustache/sketch_193_000004.jpg\nsketch/moustache/sketch_193_000005.jpg\nsketch/moustache/sketch_193_000011.jpg\nsketch/moustache/sketch_193_000013.jpg\nsketch/moustache/sketch_193_000019.jpg\nsketch/moustache/sketch_193_000021.jpg\nsketch/moustache/sketch_193_000034.jpg\nsketch/moustache/sketch_193_000047.jpg\nsketch/moustache/sketch_193_000056.jpg\nsketch/moustache/sketch_193_000065.jpg\nsketch/moustache/sketch_193_000072.jpg\nsketch/moustache/sketch_193_000079.jpg\nsketch/moustache/sketch_193_000085.jpg\nsketch/moustache/sketch_193_000087.jpg\nsketch/mouth/sketch_194_000049.jpg\nsketch/mug/sketch_195_000004.jpg\nsketch/mug/sketch_195_000007.jpg\nsketch/mug/sketch_195_000017.jpg\nsketch/mug/sketch_195_000024.jpg\nsketch/mug/sketch_195_000030.jpg\nsketch/mug/sketch_195_000033.jpg\nsketch/mug/sketch_195_000042.jpg\nsketch/mug/sketch_195_000055.jpg\nsketch/mug/sketch_195_000061.jpg\nsketch/mug/sketch_195_000078.jpg\nsketch/mug/sketch_195_000105.jpg\nsketch/mug/sketch_195_000150.jpg\nsketch/mug/sketch_195_000161.jpg\nsketch/mug/sketch_195_000167.jpg\nsketch/mug/sketch_195_000178.jpg\nsketch/nail/sketch_197_000019.jpg\nsketch/nose/sketch_199_000018.jpg\nsketch/nose/sketch_199_000032.jpg\nsketch/nose/sketch_199_000042.jpg\nsketch/nose/sketch_199_000081.jpg\nsketch/ocean/sketch_200_000027.jpg\nsketch/ocean/sketch_200_000039.jpg\nsketch/ocean/sketch_200_000043.jpg\nsketch/ocean/sketch_200_000060.jpg\nsketch/ocean/sketch_200_000071.jpg\nsketch/octagon/sketch_201_000027.jpg\nsketch/oven/sketch_204_000001.jpg\nsketch/oven/sketch_204_000003.jpg\nsketch/oven/sketch_204_000013.jpg\nsketch/oven/sketch_204_000023.jpg\nsketch/oven/sketch_204_000024.jpg\nsketch/oven/sketch_204_000034.jpg\nsketch/oven/sketch_204_000039.jpg\nsketch/oven/sketch_204_000046.jpg\nsketch/oven/sketch_204_000048.jpg\nsketch/oven/sketch_204_000078.jpg\nsketch/oven/sketch_204_000099.jpg\nsketch/oven/sketch_204_000103.jpg\nsketch/oven/sketch_204_000104.jpg\nsketch/oven/sketch_204_000111.jpg\nsketch/oven/sketch_204_000113.jpg\nsketch/oven/sketch_204_000116.jpg\nsketch/oven/sketch_204_000129.jpg\nsketch/oven/sketch_204_000130.jpg\nsketch/oven/sketch_204_000141.jpg\nsketch/oven/sketch_204_000158.jpg\nsketch/oven/sketch_204_000165.jpg\nsketch/oven/sketch_204_000168.jpg\nsketch/oven/sketch_204_000169.jpg\nsketch/oven/sketch_204_000173.jpg\nsketch/oven/sketch_204_000174.jpg\nsketch/oven/sketch_204_000175.jpg\nsketch/owl/sketch_205_000114.jpg\nsketch/paint_can/sketch_207_000002.jpg\nsketch/paint_can/sketch_207_000011.jpg\nsketch/paint_can/sketch_207_000012.jpg\nsketch/palm_tree/sketch_208_000101.jpg\nsketch/palm_tree/sketch_208_000160.jpg\nsketch/palm_tree/sketch_208_000163.jpg\nsketch/panda/sketch_209_000035.jpg\nsketch/pants/sketch_210_000089.jpg\nsketch/paper_clip/sketch_211_000001.jpg\nsketch/paper_clip/sketch_211_000053.jpg\nsketch/parachute/sketch_212_000087.jpg\nsketch/parrot/sketch_213_000083.jpg\nsketch/parrot/sketch_213_000144.jpg\nsketch/parrot/sketch_213_000161.jpg\nsketch/parrot/sketch_213_000185.jpg\nsketch/parrot/sketch_213_000209.jpg\nsketch/passport/sketch_214_000043.jpg\nsketch/passport/sketch_214_000073.jpg\nsketch/passport/sketch_214_000086.jpg\nsketch/pear/sketch_216_000157.jpg\nsketch/peas/sketch_217_000035.jpg\nsketch/pickup_truck/sketch_221_000098.jpg\nsketch/picture_frame/sketch_222_000038.jpg\nsketch/pig/sketch_223_000217.jpg\nsketch/pizza/sketch_226_000009.jpg\nsketch/police_car/sketch_228_000054.jpg\nsketch/pond/sketch_229_000040.jpg\nsketch/postcard/sketch_232_000019.jpg\nsketch/potato/sketch_233_000044.jpg\nsketch/potato/sketch_233_000060.jpg\nsketch/power_outlet/sketch_234_000017.jpg\nsketch/power_outlet/sketch_234_000090.jpg\nsketch/purse/sketch_235_000058.jpg\nsketch/purse/sketch_235_000068.jpg\nsketch/rabbit/sketch_236_000034.jpg\nsketch/rabbit/sketch_236_000045.jpg\nsketch/rabbit/sketch_236_000081.jpg\nsketch/raccoon/sketch_237_000106.jpg\nsketch/raccoon/sketch_237_000181.jpg\nsketch/rain/sketch_239_000057.jpg\nsketch/rain/sketch_239_000126.jpg\nsketch/rain/sketch_239_000222.jpg\nsketch/rainbow/sketch_240_000006.jpg\nsketch/rake/sketch_241_000007.jpg\nsketch/remote_control/sketch_242_000027.jpg\nsketch/river/sketch_245_000009.jpg\nsketch/river/sketch_245_000011.jpg\nsketch/river/sketch_245_000021.jpg\nsketch/river/sketch_245_000036.jpg\nsketch/river/sketch_245_000038.jpg\nsketch/river/sketch_245_000045.jpg\nsketch/river/sketch_245_000057.jpg\nsketch/river/sketch_245_000098.jpg\nsketch/river/sketch_245_000106.jpg\nsketch/river/sketch_245_000108.jpg\nsketch/rollerskates/sketch_247_000037.jpg\nsketch/rollerskates/sketch_247_000041.jpg\nsketch/rollerskates/sketch_247_000047.jpg\nsketch/rollerskates/sketch_247_000050.jpg\nsketch/rollerskates/sketch_247_000101.jpg\nsketch/rollerskates/sketch_247_000106.jpg\nsketch/rollerskates/sketch_247_000110.jpg\nsketch/rollerskates/sketch_247_000132.jpg\nsketch/rollerskates/sketch_247_000138.jpg\nsketch/sailboat/sketch_248_000323.jpg\nsketch/sandwich/sketch_249_000077.jpg\nsketch/sandwich/sketch_249_000107.jpg\nsketch/saw/sketch_250_000059.jpg\nsketch/saw/sketch_250_000061.jpg\nsketch/saw/sketch_250_000097.jpg\nsketch/saw/sketch_250_000105.jpg\nsketch/saxophone/sketch_251_000191.jpg\nsketch/saxophone/sketch_251_000249.jpg\nsketch/school_bus/sketch_252_000001.jpg\nsketch/school_bus/sketch_252_000005.jpg\nsketch/school_bus/sketch_252_000020.jpg\nsketch/school_bus/sketch_252_000028.jpg\nsketch/school_bus/sketch_252_000031.jpg\nsketch/school_bus/sketch_252_000032.jpg\nsketch/school_bus/sketch_252_000033.jpg\nsketch/school_bus/sketch_252_000037.jpg\nsketch/school_bus/sketch_252_000044.jpg\nsketch/school_bus/sketch_252_000047.jpg\nsketch/school_bus/sketch_252_000053.jpg\nsketch/school_bus/sketch_252_000054.jpg\nsketch/school_bus/sketch_252_000059.jpg\nsketch/school_bus/sketch_252_000062.jpg\nsketch/school_bus/sketch_252_000076.jpg\nsketch/school_bus/sketch_252_000077.jpg\nsketch/school_bus/sketch_252_000081.jpg\nsketch/school_bus/sketch_252_000097.jpg\nsketch/school_bus/sketch_252_000098.jpg\nsketch/school_bus/sketch_252_000108.jpg\nsketch/school_bus/sketch_252_000109.jpg\nsketch/school_bus/sketch_252_000118.jpg\nsketch/school_bus/sketch_252_000121.jpg\nsketch/school_bus/sketch_252_000129.jpg\nsketch/school_bus/sketch_252_000130.jpg\nsketch/school_bus/sketch_252_000133.jpg\nsketch/school_bus/sketch_252_000158.jpg\nsketch/school_bus/sketch_252_000161.jpg\nsketch/school_bus/sketch_252_000183.jpg\nsketch/school_bus/sketch_252_000194.jpg\nsketch/school_bus/sketch_252_000211.jpg\nsketch/school_bus/sketch_252_000223.jpg\nsketch/school_bus/sketch_252_000256.jpg\nsketch/school_bus/sketch_252_000263.jpg\nsketch/school_bus/sketch_252_000284.jpg\nsketch/school_bus/sketch_252_000332.jpg\nsketch/school_bus/sketch_252_000334.jpg\nsketch/school_bus/sketch_252_000352.jpg\nsketch/school_bus/sketch_252_000353.jpg\nsketch/school_bus/sketch_252_000355.jpg\nsketch/school_bus/sketch_252_000357.jpg\nsketch/school_bus/sketch_252_000358.jpg\nsketch/school_bus/sketch_252_000359.jpg\nsketch/school_bus/sketch_252_000363.jpg\nsketch/school_bus/sketch_252_000365.jpg\nsketch/school_bus/sketch_252_000373.jpg\nsketch/school_bus/sketch_252_000380.jpg\nsketch/school_bus/sketch_252_000384.jpg\nsketch/school_bus/sketch_252_000386.jpg\nsketch/school_bus/sketch_252_000387.jpg\nsketch/school_bus/sketch_252_000389.jpg\nsketch/school_bus/sketch_252_000395.jpg\nsketch/school_bus/sketch_252_000398.jpg\nsketch/school_bus/sketch_252_000401.jpg\nsketch/scissors/sketch_253_000154.jpg\nsketch/scissors/sketch_253_000201.jpg\nsketch/scissors/sketch_253_000224.jpg\nsketch/scissors/sketch_253_000241.jpg\nsketch/scissors/sketch_253_000294.jpg\nsketch/scorpion/sketch_254_000141.jpg\nsketch/screwdriver/sketch_255_000029.jpg\nsketch/screwdriver/sketch_255_000039.jpg\nsketch/screwdriver/sketch_255_000121.jpg\nsketch/screwdriver/sketch_255_000149.jpg\nsketch/screwdriver/sketch_255_000175.jpg\nsketch/screwdriver/sketch_255_000184.jpg\nsketch/screwdriver/sketch_255_000188.jpg\nsketch/screwdriver/sketch_255_000193.jpg\nsketch/screwdriver/sketch_255_000217.jpg\nsketch/screwdriver/sketch_255_000241.jpg\nsketch/screwdriver/sketch_255_000294.jpg\nsketch/screwdriver/sketch_255_000302.jpg\nsketch/screwdriver/sketch_255_000303.jpg\nsketch/screwdriver/sketch_255_000362.jpg\nsketch/screwdriver/sketch_255_000363.jpg\nsketch/see_saw/sketch_257_000008.jpg\nsketch/see_saw/sketch_257_000174.jpg\nsketch/see_saw/sketch_257_000301.jpg\nsketch/shark/sketch_258_000095.jpg\nsketch/shark/sketch_258_000096.jpg\nsketch/shark/sketch_258_000113.jpg\nsketch/shark/sketch_258_000343.jpg\nsketch/shark/sketch_258_000402.jpg\nsketch/shark/sketch_258_000404.jpg\nsketch/shark/sketch_258_000405.jpg\nsketch/shark/sketch_258_000407.jpg\nsketch/shark/sketch_258_000409.jpg\nsketch/shark/sketch_258_000410.jpg\nsketch/shark/sketch_258_000411.jpg\nsketch/shark/sketch_258_000412.jpg\nsketch/shark/sketch_258_000413.jpg\nsketch/shark/sketch_258_000414.jpg\nsketch/shark/sketch_258_000415.jpg\nsketch/shark/sketch_258_000417.jpg\nsketch/shark/sketch_258_000418.jpg\nsketch/shark/sketch_258_000419.jpg\nsketch/shark/sketch_258_000421.jpg\nsketch/shark/sketch_258_000422.jpg\nsketch/shark/sketch_258_000423.jpg\nsketch/shark/sketch_258_000425.jpg\nsketch/shark/sketch_258_000426.jpg\nsketch/shark/sketch_258_000429.jpg\nsketch/shark/sketch_258_000430.jpg\nsketch/shark/sketch_258_000432.jpg\nsketch/shark/sketch_258_000434.jpg\nsketch/shark/sketch_258_000436.jpg\nsketch/shark/sketch_258_000438.jpg\nsketch/shark/sketch_258_000439.jpg\nsketch/shark/sketch_258_000441.jpg\nsketch/shark/sketch_258_000443.jpg\nsketch/shark/sketch_258_000445.jpg\nsketch/shark/sketch_258_000447.jpg\nsketch/shark/sketch_258_000450.jpg\nsketch/shark/sketch_258_000452.jpg\nsketch/shark/sketch_258_000454.jpg\nsketch/shark/sketch_258_000456.jpg\nsketch/shark/sketch_258_000458.jpg\nsketch/shark/sketch_258_000528.jpg\nsketch/sheep/sketch_259_000110.jpg\nsketch/sheep/sketch_259_000112.jpg\nsketch/sheep/sketch_259_000114.jpg\nsketch/sheep/sketch_259_000116.jpg\nsketch/sheep/sketch_259_000125.jpg\nsketch/sheep/sketch_259_000137.jpg\nsketch/sheep/sketch_259_000143.jpg\nsketch/sheep/sketch_259_000171.jpg\nsketch/sheep/sketch_259_000182.jpg\nsketch/sheep/sketch_259_000210.jpg\nsketch/sheep/sketch_259_000315.jpg\nsketch/sheep/sketch_259_000317.jpg\nsketch/sheep/sketch_259_000326.jpg\nsketch/sheep/sketch_259_000339.jpg\nsketch/sheep/sketch_259_000340.jpg\nsketch/sheep/sketch_259_000351.jpg\nsketch/sheep/sketch_259_000358.jpg\nsketch/sheep/sketch_259_000379.jpg\nsketch/shoe/sketch_260_000208.jpg\nsketch/shoe/sketch_260_000444.jpg\nsketch/shoe/sketch_260_000568.jpg\nsketch/shorts/sketch_261_000030.jpg\nsketch/shorts/sketch_261_000060.jpg\nsketch/shorts/sketch_261_000316.jpg\nsketch/shorts/sketch_261_000321.jpg\nsketch/shorts/sketch_261_000326.jpg\nsketch/shorts/sketch_261_000334.jpg\nsketch/shorts/sketch_261_000335.jpg\nsketch/shorts/sketch_261_000341.jpg\nsketch/shorts/sketch_261_000348.jpg\nsketch/shorts/sketch_261_000352.jpg\nsketch/shorts/sketch_261_000360.jpg\nsketch/shorts/sketch_261_000401.jpg\nsketch/shovel/sketch_262_000005.jpg\nsketch/shovel/sketch_262_000006.jpg\nsketch/shovel/sketch_262_000054.jpg\nsketch/shovel/sketch_262_000056.jpg\nsketch/shovel/sketch_262_000065.jpg\nsketch/shovel/sketch_262_000085.jpg\nsketch/shovel/sketch_262_000092.jpg\nsketch/shovel/sketch_262_000146.jpg\nsketch/shovel/sketch_262_000152.jpg\nsketch/shovel/sketch_262_000169.jpg\nsketch/shovel/sketch_262_000176.jpg\nsketch/shovel/sketch_262_000205.jpg\nsketch/shovel/sketch_262_000215.jpg\nsketch/shovel/sketch_262_000221.jpg\nsketch/shovel/sketch_262_000277.jpg\nsketch/shovel/sketch_262_000304.jpg\nsketch/shovel/sketch_262_000310.jpg\nsketch/shovel/sketch_262_000361.jpg\nsketch/shovel/sketch_262_000381.jpg\nsketch/shovel/sketch_262_000413.jpg\nsketch/shovel/sketch_262_000428.jpg\nsketch/shovel/sketch_262_000487.jpg\nsketch/shovel/sketch_262_000488.jpg\nsketch/shovel/sketch_262_000523.jpg\nsketch/shovel/sketch_262_000566.jpg\nsketch/shovel/sketch_262_000574.jpg\nsketch/shovel/sketch_262_000597.jpg\nsketch/shovel/sketch_262_000604.jpg\nsketch/shovel/sketch_262_000613.jpg\nsketch/sink/sketch_263_000199.jpg\nsketch/sink/sketch_263_000223.jpg\nsketch/skateboard/sketch_264_000002.jpg\nsketch/skateboard/sketch_264_000343.jpg\nsketch/skateboard/sketch_264_000349.jpg\nsketch/skateboard/sketch_264_000352.jpg\nsketch/skull/sketch_265_000029.jpg\nsketch/skull/sketch_265_000065.jpg\nsketch/skull/sketch_265_000099.jpg\nsketch/skull/sketch_265_000138.jpg\nsketch/skull/sketch_265_000140.jpg\nsketch/skull/sketch_265_000178.jpg\nsketch/skull/sketch_265_000437.jpg\nsketch/skull/sketch_265_000470.jpg\nsketch/skull/sketch_265_000558.jpg\nsketch/skull/sketch_265_000587.jpg\nsketch/skull/sketch_265_000591.jpg\nsketch/skull/sketch_265_000600.jpg\nsketch/skyscraper/sketch_266_000384.jpg\nsketch/sleeping_bag/sketch_267_000001.jpg\nsketch/sleeping_bag/sketch_267_000569.jpg\nsketch/smiley_face/sketch_268_000030.jpg\nsketch/smiley_face/sketch_268_000263.jpg\nsketch/smiley_face/sketch_268_000289.jpg\nsketch/smiley_face/sketch_268_000332.jpg\nsketch/snail/sketch_269_000132.jpg\nsketch/snail/sketch_269_000152.jpg\nsketch/snail/sketch_269_000170.jpg\nsketch/snail/sketch_269_000312.jpg\nsketch/snake/sketch_270_000058.jpg\nsketch/snake/sketch_270_000197.jpg\nsketch/snake/sketch_270_000364.jpg\nsketch/snake/sketch_270_000372.jpg\nsketch/snake/sketch_270_000381.jpg\nsketch/snorkel/sketch_271_000070.jpg\nsketch/snorkel/sketch_271_000124.jpg\nsketch/snorkel/sketch_271_000131.jpg\nsketch/snorkel/sketch_271_000241.jpg\nsketch/snorkel/sketch_271_000337.jpg\nsketch/snorkel/sketch_271_000351.jpg\nsketch/snorkel/sketch_271_000363.jpg\nsketch/snowflake/sketch_272_000125.jpg\nsketch/snowman/sketch_273_000281.jpg\nsketch/snowman/sketch_273_000456.jpg\nsketch/soccer_ball/sketch_274_000042.jpg\nsketch/soccer_ball/sketch_274_000175.jpg\nsketch/soccer_ball/sketch_274_000334.jpg\nsketch/soccer_ball/sketch_274_000343.jpg\nsketch/sock/sketch_275_000047.jpg\nsketch/sock/sketch_275_000086.jpg\nsketch/sock/sketch_275_000121.jpg\nsketch/sock/sketch_275_000175.jpg\nsketch/sock/sketch_275_000221.jpg\nsketch/sock/sketch_275_000305.jpg\nsketch/sock/sketch_275_000321.jpg\nsketch/sock/sketch_275_000364.jpg\nsketch/sock/sketch_275_000397.jpg\nsketch/speedboat/sketch_276_000002.jpg\nsketch/speedboat/sketch_276_000058.jpg\nsketch/speedboat/sketch_276_000082.jpg\nsketch/speedboat/sketch_276_000088.jpg\nsketch/speedboat/sketch_276_000104.jpg\nsketch/speedboat/sketch_276_000125.jpg\nsketch/speedboat/sketch_276_000126.jpg\nsketch/speedboat/sketch_276_000204.jpg\nsketch/speedboat/sketch_276_000207.jpg\nsketch/speedboat/sketch_276_000218.jpg\nsketch/speedboat/sketch_276_000263.jpg\nsketch/speedboat/sketch_276_000334.jpg\nsketch/speedboat/sketch_276_000365.jpg\nsketch/speedboat/sketch_276_000409.jpg\nsketch/speedboat/sketch_276_000440.jpg\nsketch/spider/sketch_277_000237.jpg\nsketch/spider/sketch_277_000311.jpg\nsketch/spoon/sketch_278_000015.jpg\nsketch/spoon/sketch_278_000076.jpg\nsketch/spoon/sketch_278_000167.jpg\nsketch/spoon/sketch_278_000263.jpg\nsketch/spoon/sketch_278_000275.jpg\nsketch/spoon/sketch_278_000299.jpg\nsketch/spreadsheet/sketch_279_000012.jpg\nsketch/spreadsheet/sketch_279_000174.jpg\nsketch/spreadsheet/sketch_279_000384.jpg\nsketch/spreadsheet/sketch_279_000450.jpg\nsketch/square/sketch_280_000116.jpg\nsketch/square/sketch_280_000134.jpg\nsketch/square/sketch_280_000149.jpg\nsketch/square/sketch_280_000262.jpg\nsketch/square/sketch_280_000277.jpg\nsketch/square/sketch_280_000321.jpg\nsketch/square/sketch_280_000352.jpg\nsketch/square/sketch_280_000469.jpg\nsketch/square/sketch_280_000499.jpg\nsketch/square/sketch_280_000550.jpg\nsketch/square/sketch_280_000564.jpg\nsketch/square/sketch_280_000598.jpg\nsketch/square/sketch_280_000695.jpg\nsketch/squiggle/sketch_281_000011.jpg\nsketch/squiggle/sketch_281_000041.jpg\nsketch/squiggle/sketch_281_000164.jpg\nsketch/squiggle/sketch_281_000230.jpg\nsketch/squiggle/sketch_281_000373.jpg\nsketch/squirrel/sketch_282_000344.jpg\nsketch/squirrel/sketch_282_000368.jpg\nsketch/stairs/sketch_283_000203.jpg\nsketch/stairs/sketch_283_000415.jpg\nsketch/stairs/sketch_283_000446.jpg\nsketch/stairs/sketch_283_000516.jpg\nsketch/star/sketch_284_000037.jpg\nsketch/steak/sketch_285_000067.jpg\nsketch/steak/sketch_285_000099.jpg\nsketch/stereo/sketch_286_000004.jpg\nsketch/stereo/sketch_286_000015.jpg\nsketch/stereo/sketch_286_000085.jpg\nsketch/stethoscope/sketch_287_000011.jpg\nsketch/stethoscope/sketch_287_000015.jpg\nsketch/stethoscope/sketch_287_000052.jpg\nsketch/stethoscope/sketch_287_000081.jpg\nsketch/stethoscope/sketch_287_000190.jpg\nsketch/stethoscope/sketch_287_000191.jpg\nsketch/stethoscope/sketch_287_000206.jpg\nsketch/stethoscope/sketch_287_000211.jpg\nsketch/stethoscope/sketch_287_000212.jpg\nsketch/stitches/sketch_288_000005.jpg\nsketch/stop_sign/sketch_289_000091.jpg\nsketch/stove/sketch_290_000005.jpg\nsketch/stove/sketch_290_000051.jpg\nsketch/stove/sketch_290_000058.jpg\nsketch/stove/sketch_290_000082.jpg\nsketch/stove/sketch_290_000091.jpg\nsketch/stove/sketch_290_000172.jpg\nsketch/stove/sketch_290_000229.jpg\nsketch/strawberry/sketch_291_000047.jpg\nsketch/strawberry/sketch_291_000181.jpg\nsketch/streetlight/sketch_292_000061.jpg\nsketch/streetlight/sketch_292_000062.jpg\nsketch/streetlight/sketch_292_000063.jpg\nsketch/streetlight/sketch_292_000143.jpg\nsketch/streetlight/sketch_292_000195.jpg\nsketch/streetlight/sketch_292_000200.jpg\nsketch/streetlight/sketch_292_000230.jpg\nsketch/string_bean/sketch_293_000024.jpg\nsketch/string_bean/sketch_293_000049.jpg\nsketch/string_bean/sketch_293_000054.jpg\nsketch/string_bean/sketch_293_000056.jpg\nsketch/string_bean/sketch_293_000063.jpg\nsketch/submarine/sketch_294_000011.jpg\nsketch/submarine/sketch_294_000049.jpg\nsketch/submarine/sketch_294_000117.jpg\nsketch/submarine/sketch_294_000183.jpg\nsketch/sun/sketch_296_000134.jpg\nsketch/sun/sketch_296_000174.jpg\nsketch/swan/sketch_297_000062.jpg\nsketch/swan/sketch_297_000082.jpg\nsketch/swan/sketch_297_000112.jpg\nsketch/swing_set/sketch_299_000015.jpg\nsketch/swing_set/sketch_299_000041.jpg\nsketch/swing_set/sketch_299_000042.jpg\nsketch/sword/sketch_300_000062.jpg\nsketch/sword/sketch_300_000084.jpg\nsketch/sword/sketch_300_000089.jpg\nsketch/sword/sketch_300_000139.jpg\nsketch/sword/sketch_300_000146.jpg\nsketch/sword/sketch_300_000152.jpg\nsketch/sword/sketch_300_000156.jpg\nsketch/sword/sketch_300_000188.jpg\nsketch/sword/sketch_300_000243.jpg\nsketch/sword/sketch_300_000292.jpg\nsketch/sword/sketch_300_000378.jpg\nsketch/syringe/sketch_301_000048.jpg\nsketch/syringe/sketch_301_000103.jpg\nsketch/syringe/sketch_301_000148.jpg\nsketch/syringe/sketch_301_000149.jpg\nsketch/syringe/sketch_301_000150.jpg\nsketch/syringe/sketch_301_000173.jpg\nsketch/syringe/sketch_301_000176.jpg\nsketch/syringe/sketch_301_000195.jpg\nsketch/syringe/sketch_301_000221.jpg\nsketch/table/sketch_302_000004.jpg\nsketch/table/sketch_302_000032.jpg\nsketch/table/sketch_302_000048.jpg\nsketch/table/sketch_302_000050.jpg\nsketch/table/sketch_302_000065.jpg\nsketch/table/sketch_302_000108.jpg\nsketch/table/sketch_302_000126.jpg\nsketch/table/sketch_302_000156.jpg\nsketch/table/sketch_302_000172.jpg\nsketch/table/sketch_302_000190.jpg\nsketch/table/sketch_302_000202.jpg\nsketch/table/sketch_302_000285.jpg\nsketch/table/sketch_302_000299.jpg\nsketch/teapot/sketch_303_000003.jpg\nsketch/teapot/sketch_303_000007.jpg\nsketch/teapot/sketch_303_000020.jpg\nsketch/teapot/sketch_303_000043.jpg\nsketch/teapot/sketch_303_000117.jpg\nsketch/teapot/sketch_303_000164.jpg\nsketch/teapot/sketch_303_000201.jpg\nsketch/teapot/sketch_303_000215.jpg\nsketch/teapot/sketch_303_000242.jpg\nsketch/teapot/sketch_303_000252.jpg\nsketch/teapot/sketch_303_000278.jpg\nsketch/teapot/sketch_303_000320.jpg\nsketch/teddy-bear/sketch_304_000001.jpg\nsketch/teddy-bear/sketch_304_000002.jpg\nsketch/teddy-bear/sketch_304_000004.jpg\nsketch/teddy-bear/sketch_304_000005.jpg\nsketch/teddy-bear/sketch_304_000006.jpg\nsketch/teddy-bear/sketch_304_000007.jpg\nsketch/teddy-bear/sketch_304_000009.jpg\nsketch/teddy-bear/sketch_304_000010.jpg\nsketch/teddy-bear/sketch_304_000012.jpg\nsketch/teddy-bear/sketch_304_000017.jpg\nsketch/teddy-bear/sketch_304_000018.jpg\nsketch/teddy-bear/sketch_304_000019.jpg\nsketch/teddy-bear/sketch_304_000020.jpg\nsketch/teddy-bear/sketch_304_000022.jpg\nsketch/teddy-bear/sketch_304_000023.jpg\nsketch/teddy-bear/sketch_304_000029.jpg\nsketch/teddy-bear/sketch_304_000031.jpg\nsketch/teddy-bear/sketch_304_000032.jpg\nsketch/teddy-bear/sketch_304_000033.jpg\nsketch/teddy-bear/sketch_304_000035.jpg\nsketch/teddy-bear/sketch_304_000036.jpg\nsketch/teddy-bear/sketch_304_000044.jpg\nsketch/teddy-bear/sketch_304_000047.jpg\nsketch/teddy-bear/sketch_304_000049.jpg\nsketch/teddy-bear/sketch_304_000053.jpg\nsketch/teddy-bear/sketch_304_000059.jpg\nsketch/teddy-bear/sketch_304_000062.jpg\nsketch/teddy-bear/sketch_304_000072.jpg\nsketch/teddy-bear/sketch_304_000080.jpg\nsketch/teddy-bear/sketch_304_000081.jpg\nsketch/teddy-bear/sketch_304_000084.jpg\nsketch/teddy-bear/sketch_304_000144.jpg\nsketch/teddy-bear/sketch_304_000156.jpg\nsketch/teddy-bear/sketch_304_000157.jpg\nsketch/teddy-bear/sketch_304_000160.jpg\nsketch/teddy-bear/sketch_304_000172.jpg\nsketch/teddy-bear/sketch_304_000216.jpg\nsketch/telephone/sketch_305_000019.jpg\nsketch/telephone/sketch_305_000144.jpg\nsketch/telephone/sketch_305_000195.jpg\nsketch/television/sketch_306_000045.jpg\nsketch/television/sketch_306_000079.jpg\nsketch/television/sketch_306_000114.jpg\nsketch/tent/sketch_308_000015.jpg\nsketch/tent/sketch_308_000219.jpg\nsketch/tent/sketch_308_000229.jpg\nsketch/tent/sketch_308_000243.jpg\nsketch/The_Eiffel_Tower/sketch_309_000084.jpg\nsketch/The_Eiffel_Tower/sketch_309_000228.jpg\nsketch/The_Eiffel_Tower/sketch_309_000264.jpg\nsketch/The_Great_Wall_of_China/sketch_310_000013.jpg\nsketch/The_Great_Wall_of_China/sketch_310_000020.jpg\nsketch/The_Great_Wall_of_China/sketch_310_000023.jpg\nsketch/The_Great_Wall_of_China/sketch_310_000029.jpg\nsketch/The_Great_Wall_of_China/sketch_310_000038.jpg\nsketch/The_Great_Wall_of_China/sketch_310_000048.jpg\nsketch/The_Great_Wall_of_China/sketch_310_000063.jpg\nsketch/The_Great_Wall_of_China/sketch_310_000099.jpg\nsketch/The_Great_Wall_of_China/sketch_310_000113.jpg\nsketch/The_Great_Wall_of_China/sketch_310_000116.jpg\nsketch/The_Great_Wall_of_China/sketch_310_000135.jpg\nsketch/The_Great_Wall_of_China/sketch_310_000147.jpg\nsketch/The_Mona_Lisa/sketch_311_000049.jpg\nsketch/The_Mona_Lisa/sketch_311_000113.jpg\nsketch/The_Mona_Lisa/sketch_311_000143.jpg\nsketch/tiger/sketch_312_000005.jpg\nsketch/tiger/sketch_312_000017.jpg\nsketch/tiger/sketch_312_000029.jpg\nsketch/tiger/sketch_312_000061.jpg\nsketch/tiger/sketch_312_000213.jpg\nsketch/tiger/sketch_312_000279.jpg\nsketch/tiger/sketch_312_000336.jpg\nsketch/toaster/sketch_313_000021.jpg\nsketch/toaster/sketch_313_000025.jpg\nsketch/toaster/sketch_313_000035.jpg\nsketch/toaster/sketch_313_000049.jpg\nsketch/toaster/sketch_313_000079.jpg\nsketch/toaster/sketch_313_000181.jpg\nsketch/toe/sketch_314_000005.jpg\nsketch/toe/sketch_314_000006.jpg\nsketch/toe/sketch_314_000008.jpg\nsketch/toe/sketch_314_000010.jpg\nsketch/toe/sketch_314_000012.jpg\nsketch/toe/sketch_314_000013.jpg\nsketch/toe/sketch_314_000015.jpg\nsketch/toe/sketch_314_000020.jpg\nsketch/toe/sketch_314_000024.jpg\nsketch/toe/sketch_314_000027.jpg\nsketch/toe/sketch_314_000036.jpg\nsketch/toe/sketch_314_000043.jpg\nsketch/toilet/sketch_315_000017.jpg\nsketch/toilet/sketch_315_000044.jpg\nsketch/toilet/sketch_315_000047.jpg\nsketch/toilet/sketch_315_000055.jpg\nsketch/tooth/sketch_316_000022.jpg\nsketch/tooth/sketch_316_000091.jpg\nsketch/tooth/sketch_316_000106.jpg\nsketch/toothbrush/sketch_317_000107.jpg\nsketch/toothbrush/sketch_317_000173.jpg\nsketch/toothbrush/sketch_317_000174.jpg\nsketch/toothpaste/sketch_318_000015.jpg\nsketch/toothpaste/sketch_318_000025.jpg\nsketch/toothpaste/sketch_318_000030.jpg\nsketch/toothpaste/sketch_318_000036.jpg\nsketch/toothpaste/sketch_318_000038.jpg\nsketch/toothpaste/sketch_318_000042.jpg\nsketch/toothpaste/sketch_318_000049.jpg\nsketch/toothpaste/sketch_318_000054.jpg\nsketch/toothpaste/sketch_318_000059.jpg\nsketch/toothpaste/sketch_318_000061.jpg\nsketch/toothpaste/sketch_318_000065.jpg\nsketch/toothpaste/sketch_318_000067.jpg\nsketch/toothpaste/sketch_318_000071.jpg\nsketch/toothpaste/sketch_318_000073.jpg\nsketch/toothpaste/sketch_318_000074.jpg\nsketch/toothpaste/sketch_318_000080.jpg\nsketch/toothpaste/sketch_318_000082.jpg\nsketch/toothpaste/sketch_318_000083.jpg\nsketch/toothpaste/sketch_318_000084.jpg\nsketch/toothpaste/sketch_318_000087.jpg\nsketch/toothpaste/sketch_318_000088.jpg\nsketch/toothpaste/sketch_318_000092.jpg\nsketch/toothpaste/sketch_318_000094.jpg\nsketch/toothpaste/sketch_318_000096.jpg\nsketch/toothpaste/sketch_318_000103.jpg\nsketch/toothpaste/sketch_318_000110.jpg\nsketch/toothpaste/sketch_318_000112.jpg\nsketch/toothpaste/sketch_318_000114.jpg\nsketch/toothpaste/sketch_318_000117.jpg\nsketch/toothpaste/sketch_318_000120.jpg\nsketch/toothpaste/sketch_318_000132.jpg\nsketch/toothpaste/sketch_318_000139.jpg\nsketch/toothpaste/sketch_318_000149.jpg\nsketch/toothpaste/sketch_318_000150.jpg\nsketch/toothpaste/sketch_318_000156.jpg\nsketch/toothpaste/sketch_318_000161.jpg\nsketch/toothpaste/sketch_318_000165.jpg\nsketch/toothpaste/sketch_318_000166.jpg\nsketch/toothpaste/sketch_318_000169.jpg\nsketch/toothpaste/sketch_318_000176.jpg\nsketch/toothpaste/sketch_318_000179.jpg\nsketch/toothpaste/sketch_318_000181.jpg\nsketch/toothpaste/sketch_318_000185.jpg\nsketch/toothpaste/sketch_318_000186.jpg\nsketch/toothpaste/sketch_318_000188.jpg\nsketch/toothpaste/sketch_318_000189.jpg\nsketch/toothpaste/sketch_318_000191.jpg\nsketch/tornado/sketch_319_000015.jpg\nsketch/tornado/sketch_319_000021.jpg\nsketch/tornado/sketch_319_000037.jpg\nsketch/tornado/sketch_319_000050.jpg\nsketch/tornado/sketch_319_000053.jpg\nsketch/tornado/sketch_319_000076.jpg\nsketch/tornado/sketch_319_000099.jpg\nsketch/tornado/sketch_319_000115.jpg\nsketch/tornado/sketch_319_000128.jpg\nsketch/tornado/sketch_319_000154.jpg\nsketch/tornado/sketch_319_000156.jpg\nsketch/tornado/sketch_319_000190.jpg\nsketch/tornado/sketch_319_000193.jpg\nsketch/tornado/sketch_319_000195.jpg\nsketch/tractor/sketch_320_000023.jpg\nsketch/tractor/sketch_320_000228.jpg\nsketch/traffic_light/sketch_321_000010.jpg\nsketch/traffic_light/sketch_321_000020.jpg\nsketch/traffic_light/sketch_321_000065.jpg\nsketch/traffic_light/sketch_321_000101.jpg\nsketch/traffic_light/sketch_321_000120.jpg\nsketch/tree/sketch_323_000015.jpg\nsketch/tree/sketch_323_000033.jpg\nsketch/tree/sketch_323_000111.jpg\nsketch/tree/sketch_323_000122.jpg\nsketch/tree/sketch_323_000144.jpg\nsketch/tree/sketch_323_000201.jpg\nsketch/tree/sketch_323_000261.jpg\nsketch/tree/sketch_323_000357.jpg\nsketch/tree/sketch_323_000391.jpg\nsketch/tree/sketch_323_000396.jpg\nsketch/tree/sketch_323_000410.jpg\nsketch/tree/sketch_323_000417.jpg\nsketch/tree/sketch_323_000422.jpg\nsketch/tree/sketch_323_000469.jpg\nsketch/triangle/sketch_324_000058.jpg\nsketch/triangle/sketch_324_000127.jpg\nsketch/triangle/sketch_324_000133.jpg\nsketch/triangle/sketch_324_000149.jpg\nsketch/triangle/sketch_324_000175.jpg\nsketch/triangle/sketch_324_000184.jpg\nsketch/triangle/sketch_324_000204.jpg\nsketch/triangle/sketch_324_000265.jpg\nsketch/triangle/sketch_324_000273.jpg\nsketch/trombone/sketch_325_000011.jpg\nsketch/trombone/sketch_325_000024.jpg\nsketch/trombone/sketch_325_000027.jpg\nsketch/trombone/sketch_325_000039.jpg\nsketch/trombone/sketch_325_000068.jpg\nsketch/truck/sketch_326_000018.jpg\nsketch/truck/sketch_326_000020.jpg\nsketch/truck/sketch_326_000027.jpg\nsketch/truck/sketch_326_000043.jpg\nsketch/truck/sketch_326_000047.jpg\nsketch/truck/sketch_326_000060.jpg\nsketch/truck/sketch_326_000081.jpg\nsketch/truck/sketch_326_000082.jpg\nsketch/truck/sketch_326_000099.jpg\nsketch/truck/sketch_326_000106.jpg\nsketch/truck/sketch_326_000111.jpg\nsketch/truck/sketch_326_000130.jpg\nsketch/truck/sketch_326_000133.jpg\nsketch/truck/sketch_326_000157.jpg\nsketch/truck/sketch_326_000161.jpg\nsketch/truck/sketch_326_000166.jpg\nsketch/truck/sketch_326_000167.jpg\nsketch/truck/sketch_326_000175.jpg\nsketch/truck/sketch_326_000177.jpg\nsketch/truck/sketch_326_000178.jpg\nsketch/truck/sketch_326_000184.jpg\nsketch/truck/sketch_326_000209.jpg\nsketch/truck/sketch_326_000237.jpg\nsketch/truck/sketch_326_000242.jpg\nsketch/truck/sketch_326_000249.jpg\nsketch/truck/sketch_326_000255.jpg\nsketch/trumpet/sketch_327_000001.jpg\nsketch/trumpet/sketch_327_000039.jpg\nsketch/trumpet/sketch_327_000040.jpg\nsketch/trumpet/sketch_327_000100.jpg\nsketch/trumpet/sketch_327_000129.jpg\nsketch/trumpet/sketch_327_000136.jpg\nsketch/trumpet/sketch_327_000154.jpg\nsketch/trumpet/sketch_327_000180.jpg\nsketch/t-shirt/sketch_328_000035.jpg\nsketch/t-shirt/sketch_328_000060.jpg\nsketch/t-shirt/sketch_328_000089.jpg\nsketch/t-shirt/sketch_328_000093.jpg\nsketch/t-shirt/sketch_328_000113.jpg\nsketch/t-shirt/sketch_328_000130.jpg\nsketch/t-shirt/sketch_328_000155.jpg\nsketch/umbrella/sketch_329_000041.jpg\nsketch/umbrella/sketch_329_000053.jpg\nsketch/umbrella/sketch_329_000060.jpg\nsketch/umbrella/sketch_329_000105.jpg\nsketch/umbrella/sketch_329_000169.jpg\nsketch/umbrella/sketch_329_000186.jpg\nsketch/umbrella/sketch_329_000233.jpg\nsketch/umbrella/sketch_329_000267.jpg\nsketch/umbrella/sketch_329_000271.jpg\nsketch/underwear/sketch_330_000041.jpg\nsketch/underwear/sketch_330_000053.jpg\nsketch/underwear/sketch_330_000090.jpg\nsketch/underwear/sketch_330_000091.jpg\nsketch/underwear/sketch_330_000094.jpg\nsketch/underwear/sketch_330_000096.jpg\nsketch/underwear/sketch_330_000106.jpg\nsketch/underwear/sketch_330_000131.jpg\nsketch/van/sketch_331_000001.jpg\nsketch/van/sketch_331_000023.jpg\nsketch/van/sketch_331_000066.jpg\nsketch/van/sketch_331_000098.jpg\nsketch/van/sketch_331_000137.jpg\nsketch/vase/sketch_332_000016.jpg\nsketch/vase/sketch_332_000097.jpg\nsketch/vase/sketch_332_000112.jpg\nsketch/vase/sketch_332_000166.jpg\nsketch/vase/sketch_332_000183.jpg\nsketch/violin/sketch_333_000103.jpg\nsketch/washing_machine/sketch_334_000054.jpg\nsketch/washing_machine/sketch_334_000068.jpg\nsketch/washing_machine/sketch_334_000081.jpg\nsketch/washing_machine/sketch_334_000084.jpg\nsketch/watermelon/sketch_335_000108.jpg\nsketch/waterslide/sketch_336_000032.jpg\nsketch/waterslide/sketch_336_000038.jpg\nsketch/waterslide/sketch_336_000046.jpg\nsketch/waterslide/sketch_336_000051.jpg\nsketch/waterslide/sketch_336_000056.jpg\nsketch/waterslide/sketch_336_000067.jpg\nsketch/waterslide/sketch_336_000088.jpg\nsketch/waterslide/sketch_336_000103.jpg\nsketch/whale/sketch_337_000012.jpg\nsketch/whale/sketch_337_000058.jpg\nsketch/whale/sketch_337_000106.jpg\nsketch/wheel/sketch_338_000154.jpg\nsketch/windmill/sketch_339_000021.jpg\nsketch/windmill/sketch_339_000106.jpg\nsketch/windmill/sketch_339_000151.jpg\nsketch/wine_bottle/sketch_340_000021.jpg\nsketch/wine_bottle/sketch_340_000059.jpg\nsketch/wine_bottle/sketch_340_000078.jpg\nsketch/wine_bottle/sketch_340_000090.jpg\nsketch/wine_bottle/sketch_340_000091.jpg\nsketch/wine_bottle/sketch_340_000095.jpg\nsketch/wine_bottle/sketch_340_000119.jpg\nsketch/wine_bottle/sketch_340_000159.jpg\nsketch/wine_bottle/sketch_340_000222.jpg\nsketch/wine_glass/sketch_341_000004.jpg\nsketch/wine_glass/sketch_341_000006.jpg\nsketch/wine_glass/sketch_341_000020.jpg\nsketch/wine_glass/sketch_341_000028.jpg\nsketch/wine_glass/sketch_341_000096.jpg\nsketch/wine_glass/sketch_341_000105.jpg\nsketch/wine_glass/sketch_341_000107.jpg\nsketch/wine_glass/sketch_341_000113.jpg\nsketch/wine_glass/sketch_341_000125.jpg\nsketch/wine_glass/sketch_341_000127.jpg\nsketch/wine_glass/sketch_341_000148.jpg\nsketch/wine_glass/sketch_341_000162.jpg\nsketch/wine_glass/sketch_341_000163.jpg\nsketch/wine_glass/sketch_341_000167.jpg\nsketch/wine_glass/sketch_341_000180.jpg\nsketch/wine_glass/sketch_341_000181.jpg\nsketch/wine_glass/sketch_341_000186.jpg\nsketch/wine_glass/sketch_341_000209.jpg\nsketch/wine_glass/sketch_341_000211.jpg\nsketch/wine_glass/sketch_341_000217.jpg\nsketch/wine_glass/sketch_341_000225.jpg\nsketch/wine_glass/sketch_341_000239.jpg\nsketch/wristwatch/sketch_342_000006.jpg\nsketch/wristwatch/sketch_342_000013.jpg\nsketch/wristwatch/sketch_342_000024.jpg\nsketch/wristwatch/sketch_342_000077.jpg\nsketch/wristwatch/sketch_342_000085.jpg\nsketch/wristwatch/sketch_342_000086.jpg\nsketch/wristwatch/sketch_342_000092.jpg\nsketch/wristwatch/sketch_342_000094.jpg\nsketch/wristwatch/sketch_342_000101.jpg\nsketch/wristwatch/sketch_342_000103.jpg\nsketch/wristwatch/sketch_342_000115.jpg\nsketch/wristwatch/sketch_342_000143.jpg\nsketch/wristwatch/sketch_342_000172.jpg\nsketch/wristwatch/sketch_342_000208.jpg\nsketch/wristwatch/sketch_342_000209.jpg\nsketch/wristwatch/sketch_342_000217.jpg\nsketch/zebra/sketch_344_000055.jpg\nsketch/zebra/sketch_344_000121.jpg\nsketch/zebra/sketch_344_000133.jpg\n"
},
{
"path": "domain_generalization/swad/domainbed/models/mixstyle.py",
"content": "\"\"\"\nhttps://github.com/KaiyangZhou/mixstyle-release/blob/master/imcls/models/mixstyle.py\n\"\"\"\nimport random\nimport torch\nimport torch.nn as nn\n\n\nclass MixStyle(nn.Module):\n \"\"\"MixStyle.\n Reference:\n Zhou et al. Domain Generalization with MixStyle. ICLR 2021.\n \"\"\"\n\n def __init__(self, p=0.5, alpha=0.3, eps=1e-6):\n \"\"\"\n Args:\n p (float): probability of using MixStyle.\n alpha (float): parameter of the Beta distribution.\n eps (float): scaling parameter to avoid numerical issues.\n \"\"\"\n super().__init__()\n self.p = p\n self.beta = torch.distributions.Beta(alpha, alpha)\n self.eps = eps\n self.alpha = alpha\n\n print(\"* MixStyle params\")\n print(f\"- p: {p}\")\n print(f\"- alpha: {alpha}\")\n\n def __repr__(self):\n return f\"MixStyle(p={self.p}, alpha={self.alpha}, eps={self.eps})\"\n\n def forward(self, x):\n if not self.training:\n return x\n\n if random.random() > self.p:\n return x\n\n B = x.size(0)\n\n mu = x.mean(dim=[2, 3], keepdim=True)\n var = x.var(dim=[2, 3], keepdim=True)\n sig = (var + self.eps).sqrt()\n mu, sig = mu.detach(), sig.detach()\n x_normed = (x - mu) / sig\n\n lmda = self.beta.sample((B, 1, 1, 1))\n lmda = lmda.to(x.device)\n\n perm = torch.randperm(B)\n mu2, sig2 = mu[perm], sig[perm]\n mu_mix = mu * lmda + mu2 * (1 - lmda)\n sig_mix = sig * lmda + sig2 * (1 - lmda)\n\n return x_normed * sig_mix + mu_mix\n\n\nclass MixStyle2(nn.Module):\n \"\"\"MixStyle (w/ domain prior).\n The input should contain two equal-sized mini-batches from two distinct domains.\n Reference:\n Zhou et al. Domain Generalization with MixStyle. ICLR 2021.\n \"\"\"\n\n def __init__(self, p=0.5, alpha=0.3, eps=1e-6):\n \"\"\"\n Args:\n p (float): probability of using MixStyle.\n alpha (float): parameter of the Beta distribution.\n eps (float): scaling parameter to avoid numerical issues.\n \"\"\"\n super().__init__()\n self.p = p\n self.beta = torch.distributions.Beta(alpha, alpha)\n self.eps = eps\n self.alpha = alpha\n\n print(\"* MixStyle params\")\n print(f\"- p: {p}\")\n print(f\"- alpha: {alpha}\")\n\n def __repr__(self):\n return f\"MixStyle(p={self.p}, alpha={self.alpha}, eps={self.eps})\"\n\n def forward(self, x):\n \"\"\"\n For the input x, the first half comes from one domain,\n while the second half comes from the other domain.\n \"\"\"\n if not self.training:\n return x\n\n if random.random() > self.p:\n return x\n\n B = x.size(0)\n\n mu = x.mean(dim=[2, 3], keepdim=True)\n var = x.var(dim=[2, 3], keepdim=True)\n sig = (var + self.eps).sqrt()\n mu, sig = mu.detach(), sig.detach()\n x_normed = (x - mu) / sig\n\n lmda = self.beta.sample((B, 1, 1, 1))\n lmda = lmda.to(x.device)\n\n perm = torch.arange(B - 1, -1, -1) # inverse index\n perm_b, perm_a = perm.chunk(2)\n perm_b = perm_b[torch.randperm(B // 2)]\n perm_a = perm_a[torch.randperm(B // 2)]\n perm = torch.cat([perm_b, perm_a], 0)\n\n mu2, sig2 = mu[perm], sig[perm]\n mu_mix = mu * lmda + mu2 * (1 - lmda)\n sig_mix = sig * lmda + sig2 * (1 - lmda)\n\n return x_normed * sig_mix + mu_mix\n"
},
{
"path": "domain_generalization/swad/domainbed/models/resnet_mixstyle.py",
"content": "\"\"\"MixStyle w/ random shuffle\nhttps://github.com/KaiyangZhou/mixstyle-release/blob/master/imcls/models/resnet_mixstyle.py\n\"\"\"\nimport torch\nimport torch.nn as nn\nimport torch.utils.model_zoo as model_zoo\n\nfrom .mixstyle import MixStyle\n\nmodel_urls = {\n \"resnet18\": \"https://download.pytorch.org/models/resnet18-5c106cde.pth\",\n \"resnet34\": \"https://download.pytorch.org/models/resnet34-333f7ec4.pth\",\n \"resnet50\": \"https://download.pytorch.org/models/resnet50-19c8e357.pth\",\n \"resnet101\": \"https://download.pytorch.org/models/resnet101-5d3b4d8f.pth\",\n \"resnet152\": \"https://download.pytorch.org/models/resnet152-b121ed2d.pth\",\n}\n\n\ndef conv3x3(in_planes, out_planes, stride=1):\n \"\"\"3x3 convolution with padding\"\"\"\n return nn.Conv2d(in_planes, out_planes, kernel_size=3, stride=stride, padding=1, bias=False)\n\n\nclass BasicBlock(nn.Module):\n expansion = 1\n\n def __init__(self, inplanes, planes, stride=1, downsample=None):\n super().__init__()\n self.conv1 = conv3x3(inplanes, planes, stride)\n self.bn1 = nn.BatchNorm2d(planes)\n self.relu = nn.ReLU(inplace=True)\n self.conv2 = conv3x3(planes, planes)\n self.bn2 = nn.BatchNorm2d(planes)\n self.downsample = downsample\n self.stride = stride\n\n def forward(self, x):\n residual = x\n\n out = self.conv1(x)\n out = self.bn1(out)\n out = self.relu(out)\n\n out = self.conv2(out)\n out = self.bn2(out)\n\n if self.downsample is not None:\n residual = self.downsample(x)\n\n out += residual\n out = self.relu(out)\n\n return out\n\n\nclass Bottleneck(nn.Module):\n expansion = 4\n\n def __init__(self, inplanes, planes, stride=1, downsample=None):\n super().__init__()\n self.conv1 = nn.Conv2d(inplanes, planes, kernel_size=1, bias=False)\n self.bn1 = nn.BatchNorm2d(planes)\n self.conv2 = nn.Conv2d(planes, planes, kernel_size=3, stride=stride, padding=1, bias=False)\n self.bn2 = nn.BatchNorm2d(planes)\n self.conv3 = nn.Conv2d(planes, planes * self.expansion, kernel_size=1, bias=False)\n self.bn3 = nn.BatchNorm2d(planes * self.expansion)\n self.relu = nn.ReLU(inplace=True)\n self.downsample = downsample\n self.stride = stride\n\n def forward(self, x):\n residual = x\n\n out = self.conv1(x)\n out = self.bn1(out)\n out = self.relu(out)\n\n out = self.conv2(out)\n out = self.bn2(out)\n out = self.relu(out)\n\n out = self.conv3(out)\n out = self.bn3(out)\n\n if self.downsample is not None:\n residual = self.downsample(x)\n\n out += residual\n out = self.relu(out)\n\n return out\n\n\nclass ResNet(nn.Module):\n def __init__(\n self, block, layers, mixstyle_layers=[], mixstyle_p=0.5, mixstyle_alpha=0.3, **kwargs\n ):\n self.inplanes = 64\n super().__init__()\n\n # backbone network\n self.conv1 = nn.Conv2d(3, 64, kernel_size=7, stride=2, padding=3, bias=False)\n self.bn1 = nn.BatchNorm2d(64)\n self.relu = nn.ReLU(inplace=True)\n self.maxpool = nn.MaxPool2d(kernel_size=3, stride=2, padding=1)\n self.layer1 = self._make_layer(block, 64, layers[0])\n self.layer2 = self._make_layer(block, 128, layers[1], stride=2)\n self.layer3 = self._make_layer(block, 256, layers[2], stride=2)\n self.layer4 = self._make_layer(block, 512, layers[3], stride=2)\n self.global_avgpool = nn.AdaptiveAvgPool2d(1)\n\n self.mixstyle = None\n if mixstyle_layers:\n self.mixstyle = MixStyle(p=mixstyle_p, alpha=mixstyle_alpha)\n for layer_name in mixstyle_layers:\n assert layer_name in [\"conv1\", \"conv2_x\", \"conv3_x\", \"conv4_x\", \"conv5_x\"]\n print(\"Insert MixStyle after the following layers: {}\".format(mixstyle_layers))\n self.mixstyle_layers = mixstyle_layers\n\n self._out_features = 512 * block.expansion\n self.fc = nn.Identity() # for DomainBed compatibility\n\n self._init_params()\n\n def _make_layer(self, block, planes, blocks, stride=1):\n downsample = None\n if stride != 1 or self.inplanes != planes * block.expansion:\n downsample = nn.Sequential(\n nn.Conv2d(\n self.inplanes,\n planes * block.expansion,\n kernel_size=1,\n stride=stride,\n bias=False,\n ),\n nn.BatchNorm2d(planes * block.expansion),\n )\n\n layers = []\n layers.append(block(self.inplanes, planes, stride, downsample))\n self.inplanes = planes * block.expansion\n for i in range(1, blocks):\n layers.append(block(self.inplanes, planes))\n\n return nn.Sequential(*layers)\n\n def _init_params(self):\n for m in self.modules():\n if isinstance(m, nn.Conv2d):\n nn.init.kaiming_normal_(m.weight, mode=\"fan_out\", nonlinearity=\"relu\")\n if m.bias is not None:\n nn.init.constant_(m.bias, 0)\n elif isinstance(m, nn.BatchNorm2d):\n nn.init.constant_(m.weight, 1)\n nn.init.constant_(m.bias, 0)\n elif isinstance(m, nn.BatchNorm1d):\n nn.init.constant_(m.weight, 1)\n nn.init.constant_(m.bias, 0)\n elif isinstance(m, nn.Linear):\n nn.init.normal_(m.weight, 0, 0.01)\n if m.bias is not None:\n nn.init.constant_(m.bias, 0)\n\n def compute_style(self, x):\n mu = x.mean(dim=[2, 3])\n sig = x.std(dim=[2, 3])\n return torch.cat([mu, sig], 1)\n\n def featuremaps(self, x):\n x = self.conv1(x)\n x = self.bn1(x)\n x = self.relu(x)\n x = self.maxpool(x)\n\n x = self.layer1(x)\n if \"conv2_x\" in self.mixstyle_layers:\n x = self.mixstyle(x)\n\n x = self.layer2(x)\n if \"conv3_x\" in self.mixstyle_layers:\n x = self.mixstyle(x)\n\n x = self.layer3(x)\n if \"conv4_x\" in self.mixstyle_layers:\n x = self.mixstyle(x)\n\n x = self.layer4(x)\n if \"conv5_x\" in self.mixstyle_layers:\n x = self.mixstyle(x)\n\n return x\n\n def forward(self, x):\n f = self.featuremaps(x)\n v = self.global_avgpool(f)\n return v.view(v.size(0), -1)\n\n\ndef init_pretrained_weights(model, model_url):\n pretrain_dict = model_zoo.load_url(model_url)\n model.load_state_dict(pretrain_dict, strict=False)\n\n\ndef resnet18_mixstyle_L234_p0d5_a0d1(pretrained=True, **kwargs):\n model = ResNet(\n block=BasicBlock,\n layers=[2, 2, 2, 2],\n mixstyle_layers=[\"conv2_x\", \"conv3_x\", \"conv4_x\"],\n mixstyle_p=0.5,\n mixstyle_alpha=0.1,\n )\n\n if pretrained:\n init_pretrained_weights(model, model_urls[\"resnet18\"])\n\n return model\n\n\ndef resnet50_mixstyle_L234_p0d5_a0d1(pretrained=True, **kwargs):\n model = ResNet(\n block=Bottleneck,\n layers=[3, 4, 6, 3],\n mixstyle_layers=[\"conv2_x\", \"conv3_x\", \"conv4_x\"],\n mixstyle_p=0.5,\n mixstyle_alpha=0.1,\n )\n\n if pretrained:\n init_pretrained_weights(model, model_urls[\"resnet50\"])\n\n return model\n"
},
{
"path": "domain_generalization/swad/domainbed/models/resnet_mixstyle2.py",
"content": "\"\"\"MixStyle w/ domain label\nhttps://github.com/KaiyangZhou/mixstyle-release/blob/master/imcls/models/resnet_mixstyle2.py\n\"\"\"\nimport random\nimport torch\nimport torch.nn as nn\nimport torch.utils.model_zoo as model_zoo\n\nfrom .mixstyle import MixStyle2 as MixStyle\n\nmodel_urls = {\n \"resnet18\": \"https://download.pytorch.org/models/resnet18-5c106cde.pth\",\n \"resnet34\": \"https://download.pytorch.org/models/resnet34-333f7ec4.pth\",\n \"resnet50\": \"https://download.pytorch.org/models/resnet50-19c8e357.pth\",\n \"resnet101\": \"https://download.pytorch.org/models/resnet101-5d3b4d8f.pth\",\n \"resnet152\": \"https://download.pytorch.org/models/resnet152-b121ed2d.pth\",\n}\n\n\ndef conv3x3(in_planes, out_planes, stride=1):\n \"\"\"3x3 convolution with padding\"\"\"\n return nn.Conv2d(in_planes, out_planes, kernel_size=3, stride=stride, padding=1, bias=False)\n\n\nclass BasicBlock(nn.Module):\n expansion = 1\n\n def __init__(self, inplanes, planes, stride=1, downsample=None):\n super().__init__()\n self.conv1 = conv3x3(inplanes, planes, stride)\n self.bn1 = nn.BatchNorm2d(planes)\n self.relu = nn.ReLU(inplace=True)\n self.conv2 = conv3x3(planes, planes)\n self.bn2 = nn.BatchNorm2d(planes)\n self.downsample = downsample\n self.stride = stride\n\n def forward(self, x):\n residual = x\n\n out = self.conv1(x)\n out = self.bn1(out)\n out = self.relu(out)\n\n out = self.conv2(out)\n out = self.bn2(out)\n\n if self.downsample is not None:\n residual = self.downsample(x)\n\n out += residual\n out = self.relu(out)\n\n return out\n\n\nclass Bottleneck(nn.Module):\n expansion = 4\n\n def __init__(self, inplanes, planes, stride=1, downsample=None):\n super().__init__()\n self.conv1 = nn.Conv2d(inplanes, planes, kernel_size=1, bias=False)\n self.bn1 = nn.BatchNorm2d(planes)\n self.conv2 = nn.Conv2d(planes, planes, kernel_size=3, stride=stride, padding=1, bias=False)\n self.bn2 = nn.BatchNorm2d(planes)\n self.conv3 = nn.Conv2d(planes, planes * self.expansion, kernel_size=1, bias=False)\n self.bn3 = nn.BatchNorm2d(planes * self.expansion)\n self.relu = nn.ReLU(inplace=True)\n self.downsample = downsample\n self.stride = stride\n\n def forward(self, x):\n residual = x\n\n out = self.conv1(x)\n out = self.bn1(out)\n out = self.relu(out)\n\n out = self.conv2(out)\n out = self.bn2(out)\n out = self.relu(out)\n\n out = self.conv3(out)\n out = self.bn3(out)\n\n if self.downsample is not None:\n residual = self.downsample(x)\n\n out += residual\n out = self.relu(out)\n\n return out\n\n\nclass ResNet(nn.Module):\n def __init__(\n self, block, layers, mixstyle_layers=[], mixstyle_p=0.5, mixstyle_alpha=0.3, **kwargs\n ):\n self.inplanes = 64\n super().__init__()\n\n # backbone network\n self.conv1 = nn.Conv2d(3, 64, kernel_size=7, stride=2, padding=3, bias=False)\n self.bn1 = nn.BatchNorm2d(64)\n self.relu = nn.ReLU(inplace=True)\n self.maxpool = nn.MaxPool2d(kernel_size=3, stride=2, padding=1)\n self.layer1 = self._make_layer(block, 64, layers[0])\n self.layer2 = self._make_layer(block, 128, layers[1], stride=2)\n self.layer3 = self._make_layer(block, 256, layers[2], stride=2)\n self.layer4 = self._make_layer(block, 512, layers[3], stride=2)\n self.global_avgpool = nn.AdaptiveAvgPool2d(1)\n\n self.mixstyle = None\n if mixstyle_layers:\n self.mixstyle = MixStyle(p=mixstyle_p, alpha=mixstyle_alpha)\n for layer_name in mixstyle_layers:\n assert layer_name in [\"conv1\", \"conv2_x\", \"conv3_x\", \"conv4_x\", \"conv5_x\"]\n print(\"Insert MixStyle after the following layers: {}\".format(mixstyle_layers))\n self.mixstyle_layers = mixstyle_layers\n\n self._out_features = 512 * block.expansion\n self.fc = nn.Identity() # for DomainBed compatibility\n\n self._init_params()\n\n def _make_layer(self, block, planes, blocks, stride=1):\n downsample = None\n if stride != 1 or self.inplanes != planes * block.expansion:\n downsample = nn.Sequential(\n nn.Conv2d(\n self.inplanes,\n planes * block.expansion,\n kernel_size=1,\n stride=stride,\n bias=False,\n ),\n nn.BatchNorm2d(planes * block.expansion),\n )\n\n layers = []\n layers.append(block(self.inplanes, planes, stride, downsample))\n self.inplanes = planes * block.expansion\n for i in range(1, blocks):\n layers.append(block(self.inplanes, planes))\n\n return nn.Sequential(*layers)\n\n def _init_params(self):\n for m in self.modules():\n if isinstance(m, nn.Conv2d):\n nn.init.kaiming_normal_(m.weight, mode=\"fan_out\", nonlinearity=\"relu\")\n if m.bias is not None:\n nn.init.constant_(m.bias, 0)\n elif isinstance(m, nn.BatchNorm2d):\n nn.init.constant_(m.weight, 1)\n nn.init.constant_(m.bias, 0)\n elif isinstance(m, nn.BatchNorm1d):\n nn.init.constant_(m.weight, 1)\n nn.init.constant_(m.bias, 0)\n elif isinstance(m, nn.Linear):\n nn.init.normal_(m.weight, 0, 0.01)\n if m.bias is not None:\n nn.init.constant_(m.bias, 0)\n\n def compute_style(self, x):\n mu = x.mean(dim=[2, 3])\n sig = x.std(dim=[2, 3])\n return torch.cat([mu, sig], 1)\n\n def featuremaps(self, x):\n x = self.conv1(x)\n x = self.bn1(x)\n x = self.relu(x)\n x = self.maxpool(x)\n\n x = self.layer1(x)\n if \"conv2_x\" in self.mixstyle_layers:\n x = self.mixstyle(x)\n\n x = self.layer2(x)\n if \"conv3_x\" in self.mixstyle_layers:\n x = self.mixstyle(x)\n\n x = self.layer3(x)\n if \"conv4_x\" in self.mixstyle_layers:\n x = self.mixstyle(x)\n\n x = self.layer4(x)\n if \"conv5_x\" in self.mixstyle_layers:\n x = self.mixstyle(x)\n\n return x\n\n def forward(self, x):\n f = self.featuremaps(x)\n v = self.global_avgpool(f)\n return v.view(v.size(0), -1)\n\n\ndef init_pretrained_weights(model, model_url):\n pretrain_dict = model_zoo.load_url(model_url)\n model.load_state_dict(pretrain_dict, strict=False)\n\n\n\"\"\"\nResidual network configurations:\n--\nresnet18: block=BasicBlock, layers=[2, 2, 2, 2]\nresnet34: block=BasicBlock, layers=[3, 4, 6, 3]\nresnet50: block=Bottleneck, layers=[3, 4, 6, 3]\nresnet101: block=Bottleneck, layers=[3, 4, 23, 3]\nresnet152: block=Bottleneck, layers=[3, 8, 36, 3]\n\"\"\"\n\n\ndef resnet18_mixstyle2_L234_p0d5_a0d1(pretrained=True, **kwargs):\n model = ResNet(\n block=BasicBlock,\n layers=[2, 2, 2, 2],\n mixstyle_layers=[\"conv2_x\", \"conv3_x\", \"conv4_x\"],\n mixstyle_p=0.5,\n mixstyle_alpha=0.1,\n )\n\n if pretrained:\n init_pretrained_weights(model, model_urls[\"resnet18\"])\n\n return model\n\n\ndef resnet50_mixstyle2_L234_p0d5_a0d1(pretrained=True, **kwargs):\n model = ResNet(\n block=Bottleneck,\n layers=[3, 4, 6, 3],\n mixstyle_layers=[\"conv2_x\", \"conv3_x\", \"conv4_x\"],\n mixstyle_p=0.5,\n mixstyle_alpha=0.1,\n )\n\n if pretrained:\n init_pretrained_weights(model, model_urls[\"resnet50\"])\n\n return model\n"
},
{
"path": "domain_generalization/swad/domainbed/networks.py",
"content": "# Copyright (c) Facebook, Inc. and its affiliates. All Rights Reserved\n\nimport torch\nimport torch.nn as nn\nimport torch.nn.functional as F\nimport torchvision.models\n\nfrom domainbed.lib import wide_resnet\n\n\nclass Identity(nn.Module):\n \"\"\"An identity layer\"\"\"\n\n def __init__(self):\n super(Identity, self).__init__()\n\n def forward(self, x):\n return x\n\n\nclass SqueezeLastTwo(nn.Module):\n \"\"\"\n A module which squeezes the last two dimensions,\n ordinary squeeze can be a problem for batch size 1\n \"\"\"\n\n def __init__(self):\n super(SqueezeLastTwo, self).__init__()\n\n def forward(self, x):\n return x.view(x.shape[0], x.shape[1])\n\n\nclass MLP(nn.Module):\n \"\"\"Just an MLP\"\"\"\n\n def __init__(self, n_inputs, n_outputs, hparams):\n super(MLP, self).__init__()\n self.input = nn.Linear(n_inputs, hparams[\"mlp_width\"])\n self.dropout = nn.Dropout(hparams[\"mlp_dropout\"])\n self.hiddens = nn.ModuleList(\n [\n nn.Linear(hparams[\"mlp_width\"], hparams[\"mlp_width\"])\n for _ in range(hparams[\"mlp_depth\"] - 2)\n ]\n )\n self.output = nn.Linear(hparams[\"mlp_width\"], n_outputs)\n self.n_outputs = n_outputs\n\n def forward(self, x):\n x = self.input(x)\n x = self.dropout(x)\n x = F.relu(x)\n for hidden in self.hiddens:\n x = hidden(x)\n x = self.dropout(x)\n x = F.relu(x)\n x = self.output(x)\n return x\n\n\nclass ResNet(torch.nn.Module):\n \"\"\"ResNet with the softmax chopped off and the batchnorm frozen\"\"\"\n\n def __init__(self, input_shape, hparams, network=None):\n super(ResNet, self).__init__()\n if hparams[\"resnet18\"]:\n if network is None:\n network = torchvision.models.resnet18(pretrained=hparams[\"pretrained\"])\n self.network = network\n self.n_outputs = 512\n else:\n if network is None:\n network = torchvision.models.resnet50(pretrained=hparams[\"pretrained\"])\n self.network = network\n self.n_outputs = 2048\n\n # adapt number of channels\n nc = input_shape[0]\n if nc != 3:\n tmp = self.network.conv1.weight.data.clone()\n\n self.network.conv1 = nn.Conv2d(\n nc, 64, kernel_size=(7, 7), stride=(2, 2), padding=(3, 3), bias=False\n )\n\n for i in range(nc):\n self.network.conv1.weight.data[:, i, :, :] = tmp[:, i % 3, :, :]\n\n # save memory\n del self.network.fc\n self.network.fc = Identity()\n\n self.hparams = hparams\n self.dropout = nn.Dropout(hparams[\"resnet_dropout\"])\n self.freeze_bn()\n\n def forward(self, x):\n \"\"\"Encode x into a feature vector of size n_outputs.\"\"\"\n return self.dropout(self.network(x))\n\n def train(self, mode=True):\n \"\"\"\n Override the default train() to freeze the BN parameters\n \"\"\"\n super().train(mode)\n self.freeze_bn()\n\n def freeze_bn(self):\n if self.hparams[\"freeze_bn\"] is False:\n return\n\n for m in self.network.modules():\n if isinstance(m, nn.BatchNorm2d):\n m.eval()\n\n\nclass MNIST_CNN(nn.Module):\n \"\"\"\n Hand-tuned architecture for MNIST.\n Weirdness I've noticed so far with this architecture:\n - adding a linear layer after the mean-pool in features hurts\n RotatedMNIST-100 generalization severely.\n \"\"\"\n\n n_outputs = 128\n\n def __init__(self, input_shape):\n super(MNIST_CNN, self).__init__()\n self.conv1 = nn.Conv2d(input_shape[0], 64, 3, 1, padding=1)\n self.conv2 = nn.Conv2d(64, 128, 3, stride=2, padding=1)\n self.conv3 = nn.Conv2d(128, 128, 3, 1, padding=1)\n self.conv4 = nn.Conv2d(128, 128, 3, 1, padding=1)\n\n self.bn0 = nn.GroupNorm(8, 64)\n self.bn1 = nn.GroupNorm(8, 128)\n self.bn2 = nn.GroupNorm(8, 128)\n self.bn3 = nn.GroupNorm(8, 128)\n\n self.avgpool = nn.AdaptiveAvgPool2d((1, 1))\n self.squeezeLastTwo = SqueezeLastTwo()\n\n def forward(self, x):\n x = self.conv1(x)\n x = F.relu(x)\n x = self.bn0(x)\n\n x = self.conv2(x)\n x = F.relu(x)\n x = self.bn1(x)\n\n x = self.conv3(x)\n x = F.relu(x)\n x = self.bn2(x)\n\n x = self.conv4(x)\n x = F.relu(x)\n x = self.bn3(x)\n\n x = self.avgpool(x)\n x = self.squeezeLastTwo(x)\n return x\n\n\nclass ContextNet(nn.Module):\n def __init__(self, input_shape):\n super(ContextNet, self).__init__()\n\n # Keep same dimensions\n padding = (5 - 1) // 2\n self.context_net = nn.Sequential(\n nn.Conv2d(input_shape[0], 64, 5, padding=padding),\n nn.BatchNorm2d(64),\n nn.ReLU(),\n nn.Conv2d(64, 64, 5, padding=padding),\n nn.BatchNorm2d(64),\n nn.ReLU(),\n nn.Conv2d(64, 1, 5, padding=padding),\n )\n\n def forward(self, x):\n return self.context_net(x)\n\n\ndef Featurizer(input_shape, hparams):\n \"\"\"Auto-select an appropriate featurizer for the given input shape.\"\"\"\n if len(input_shape) == 1:\n return MLP(input_shape[0], 128, hparams)\n elif input_shape[1:3] == (28, 28):\n return MNIST_CNN(input_shape)\n elif input_shape[1:3] == (32, 32):\n return wide_resnet.Wide_ResNet(input_shape, 16, 2, 0.0)\n elif input_shape[1:3] == (224, 224):\n return ResNet(input_shape, hparams)\n else:\n raise NotImplementedError(f\"Input shape {input_shape} is not supported\")\n"
},
{
"path": "domain_generalization/swad/domainbed/optimizers.py",
"content": "import torch\n\n\ndef get_optimizer(name, params, **kwargs):\n name = name.lower()\n optimizers = {\"adam\": torch.optim.Adam, \"sgd\": torch.optim.SGD, \"adamw\": torch.optim.AdamW}\n optim_cls = optimizers[name]\n\n return optim_cls(params, **kwargs)\n"
},
{
"path": "domain_generalization/swad/domainbed/scripts/download.py",
"content": "# Copyright (c) Facebook, Inc. and its affiliates. All Rights Reserved\n\nfrom torchvision.datasets import MNIST\nimport xml.etree.ElementTree as ET\nfrom zipfile import ZipFile\nimport argparse\nimport tarfile\nimport shutil\nimport gdown\nimport uuid\nimport json\nimport os\n\n\n# utils #######################################################################\n\n\ndef stage_path(data_dir, name):\n full_path = os.path.join(data_dir, name)\n\n if not os.path.exists(full_path):\n os.makedirs(full_path)\n\n return full_path\n\n\ndef download_and_extract(url, dst, remove=True):\n gdown.download(url, dst, quiet=False)\n\n if dst.endswith(\".tar.gz\"):\n tar = tarfile.open(dst, \"r:gz\")\n tar.extractall(os.path.dirname(dst))\n tar.close()\n\n if dst.endswith(\".tar\"):\n tar = tarfile.open(dst, \"r:\")\n tar.extractall(os.path.dirname(dst))\n tar.close()\n\n if dst.endswith(\".zip\"):\n zf = ZipFile(dst, \"r\")\n zf.extractall(os.path.dirname(dst))\n zf.close()\n\n if remove:\n os.remove(dst)\n\n\n# VLCS ########################################################################\n\n# Slower, but builds dataset from the original sources\n#\n# def download_vlcs(data_dir):\n# full_path = stage_path(data_dir, \"VLCS\")\n#\n# tmp_path = os.path.join(full_path, \"tmp/\")\n# if not os.path.exists(tmp_path):\n# os.makedirs(tmp_path)\n#\n# with open(\"domainbed/misc/vlcs_files.txt\", \"r\") as f:\n# lines = f.readlines()\n# files = [line.strip().split() for line in lines]\n#\n# download_and_extract(\"http://pjreddie.com/media/files/VOCtrainval_06-Nov-2007.tar\",\n# os.path.join(tmp_path, \"voc2007_trainval.tar\"))\n#\n# download_and_extract(\"https://drive.google.com/uc?id=1I8ydxaAQunz9R_qFFdBFtw6rFTUW9goz\",\n# os.path.join(tmp_path, \"caltech101.tar.gz\"))\n#\n# download_and_extract(\"http://groups.csail.mit.edu/vision/Hcontext/data/sun09_hcontext.tar\",\n# os.path.join(tmp_path, \"sun09_hcontext.tar\"))\n#\n# tar = tarfile.open(os.path.join(tmp_path, \"sun09.tar\"), \"r:\")\n# tar.extractall(tmp_path)\n# tar.close()\n#\n# for src, dst in files:\n# class_folder = os.path.join(data_dir, dst)\n#\n# if not os.path.exists(class_folder):\n# os.makedirs(class_folder)\n#\n# dst = os.path.join(class_folder, uuid.uuid4().hex + \".jpg\")\n#\n# if \"labelme\" in src:\n# # download labelme from the web\n# gdown.download(src, dst, quiet=False)\n# else:\n# src = os.path.join(tmp_path, src)\n# shutil.copyfile(src, dst)\n#\n# shutil.rmtree(tmp_path)\n\n\ndef download_vlcs(data_dir):\n # Original URL: http://www.eecs.qmul.ac.uk/~dl307/project_iccv2017\n full_path = stage_path(data_dir, \"VLCS\")\n\n download_and_extract(\n \"https://drive.google.com/uc?id=1skwblH1_okBwxWxmRsp9_qi15hyPpxg8\",\n os.path.join(data_dir, \"VLCS.tar.gz\"),\n )\n\n\n# MNIST #######################################################################\n\n\ndef download_mnist(data_dir):\n # Original URL: http://yann.lecun.com/exdb/mnist/\n full_path = stage_path(data_dir, \"MNIST\")\n MNIST(full_path, download=True)\n\n\n# PACS ########################################################################\n\n\ndef download_pacs(data_dir):\n # Original URL: http://www.eecs.qmul.ac.uk/~dl307/project_iccv2017\n full_path = stage_path(data_dir, \"PACS\")\n\n download_and_extract(\n \"https://drive.google.com/uc?id=1JFr8f805nMUelQWWmfnJR3y4_SYoN5Pd\",\n os.path.join(data_dir, \"PACS.zip\"),\n )\n\n os.rename(os.path.join(data_dir, \"kfold\"), full_path)\n\n\n# Office-Home #################################################################\n\n\ndef download_office_home(data_dir):\n # Original URL: http://hemanthdv.org/OfficeHome-Dataset/\n full_path = stage_path(data_dir, \"office_home\")\n\n download_and_extract(\n \"https://drive.google.com/uc?id=0B81rNlvomiwed0V1YUxQdC1uOTg\",\n os.path.join(data_dir, \"office_home.zip\"),\n )\n\n os.rename(os.path.join(data_dir, \"OfficeHomeDataset_10072016\"), full_path)\n\n\n# DomainNET ###################################################################\n\n\ndef download_domain_net(data_dir):\n # Original URL: http://ai.bu.edu/M3SDA/\n full_path = stage_path(data_dir, \"domain_net\")\n\n urls = [\n \"http://csr.bu.edu/ftp/visda/2019/multi-source/groundtruth/clipart.zip\",\n \"http://csr.bu.edu/ftp/visda/2019/multi-source/infograph.zip\",\n \"http://csr.bu.edu/ftp/visda/2019/multi-source/groundtruth/painting.zip\",\n \"http://csr.bu.edu/ftp/visda/2019/multi-source/quickdraw.zip\",\n \"http://csr.bu.edu/ftp/visda/2019/multi-source/real.zip\",\n \"http://csr.bu.edu/ftp/visda/2019/multi-source/sketch.zip\",\n ]\n\n for url in urls:\n download_and_extract(url, os.path.join(full_path, url.split(\"/\")[-1]))\n\n with open(\"domainbed/misc/domain_net_duplicates.txt\", \"r\") as f:\n for line in f.readlines():\n try:\n os.remove(os.path.join(full_path, line.strip()))\n except OSError:\n pass\n\n\n# TerraIncognita ##############################################################\n\n\ndef download_terra_incognita(data_dir):\n # Original URL: https://beerys.github.io/CaltechCameraTraps/\n # New URL: http://lila.science/datasets/caltech-camera-traps\n\n full_path = stage_path(data_dir, \"terra_incognita\")\n\n download_and_extract(\n \"https://lilablobssc.blob.core.windows.net/caltechcameratraps/eccv_18_all_images_sm.tar.gz\",\n os.path.join(full_path, \"terra_incognita_images.tar.gz\"),\n )\n\n download_and_extract(\n \"https://lilablobssc.blob.core.windows.net/caltechcameratraps/labels/caltech_camera_traps.json.zip\",\n os.path.join(full_path, \"caltech_camera_traps.json.zip\"),\n )\n\n include_locations = [\"38\", \"46\", \"100\", \"43\"]\n\n include_categories = [\n \"bird\",\n \"bobcat\",\n \"cat\",\n \"coyote\",\n \"dog\",\n \"empty\",\n \"opossum\",\n \"rabbit\",\n \"raccoon\",\n \"squirrel\",\n ]\n\n images_folder = os.path.join(full_path, \"eccv_18_all_images_sm/\")\n annotations_file = os.path.join(full_path, \"caltech_images_20210113.json\")\n destination_folder = full_path\n\n stats = {}\n\n if not os.path.exists(destination_folder):\n os.mkdir(destination_folder)\n\n with open(annotations_file, \"r\") as f:\n data = json.load(f)\n\n category_dict = {}\n for item in data[\"categories\"]:\n category_dict[item[\"id\"]] = item[\"name\"]\n\n for image in data[\"images\"]:\n image_location = image[\"location\"]\n\n if image_location not in include_locations:\n continue\n\n loc_folder = os.path.join(destination_folder, \"location_\" + str(image_location) + \"/\")\n\n if not os.path.exists(loc_folder):\n os.mkdir(loc_folder)\n\n image_id = image[\"id\"]\n image_fname = image[\"file_name\"]\n\n for annotation in data[\"annotations\"]:\n if annotation[\"image_id\"] == image_id:\n if image_location not in stats:\n stats[image_location] = {}\n\n category = category_dict[annotation[\"category_id\"]]\n\n if category not in include_categories:\n continue\n\n if category not in stats[image_location]:\n stats[image_location][category] = 0\n else:\n stats[image_location][category] += 1\n\n loc_cat_folder = os.path.join(loc_folder, category + \"/\")\n\n if not os.path.exists(loc_cat_folder):\n os.mkdir(loc_cat_folder)\n\n dst_path = os.path.join(loc_cat_folder, image_fname)\n src_path = os.path.join(images_folder, image_fname)\n\n shutil.copyfile(src_path, dst_path)\n\n shutil.rmtree(images_folder)\n os.remove(annotations_file)\n\n\nif __name__ == \"__main__\":\n parser = argparse.ArgumentParser(description=\"Download datasets\")\n parser.add_argument(\"--data_dir\", type=str, required=True)\n args = parser.parse_args()\n\n download_mnist(args.data_dir)\n download_pacs(args.data_dir)\n download_vlcs(args.data_dir)\n download_domain_net(args.data_dir)\n download_office_home(args.data_dir)\n download_terra_incognita(args.data_dir)\n"
},
{
"path": "domain_generalization/swad/domainbed/swad.py",
"content": "import copy\nfrom collections import deque\nimport numpy as np\nfrom domainbed.lib import swa_utils\n\n\nclass SWADBase:\n def update_and_evaluate(self, segment_swa, val_acc, val_loss, prt_fn):\n raise NotImplementedError()\n\n def get_final_model(self):\n raise NotImplementedError()\n\n\nclass IIDMax(SWADBase):\n \"\"\"SWAD start from iid max acc and select last by iid max swa acc\"\"\"\n\n def __init__(self, evaluator, **kwargs):\n self.iid_max_acc = 0.0\n self.swa_max_acc = 0.0\n self.avgmodel = None\n self.final_model = None\n self.evaluator = evaluator\n\n def update_and_evaluate(self, segment_swa, val_acc, val_loss, prt_fn):\n if self.iid_max_acc < val_acc:\n self.iid_max_acc = val_acc\n self.avgmodel = swa_utils.AveragedModel(segment_swa.module, rm_optimizer=True)\n self.avgmodel.start_step = segment_swa.start_step\n\n self.avgmodel.update_parameters(segment_swa.module)\n self.avgmodel.end_step = segment_swa.end_step\n\n # evaluate\n accuracies, summaries = self.evaluator.evaluate(self.avgmodel)\n results = {**summaries, **accuracies}\n prt_fn(results, self.avgmodel)\n\n swa_val_acc = results[\"train_out\"]\n if swa_val_acc > self.swa_max_acc:\n self.swa_max_acc = swa_val_acc\n self.final_model = copy.deepcopy(self.avgmodel)\n\n def get_final_model(self):\n return self.final_model\n\n\nclass LossValley(SWADBase):\n \"\"\"IIDMax has a potential problem that bias to validation dataset.\n LossValley choose SWAD range by detecting loss valley.\n \"\"\"\n\n def __init__(self, evaluator, n_converge, n_tolerance, tolerance_ratio, **kwargs):\n \"\"\"\n Args:\n evaluator\n n_converge: converge detector window size.\n n_tolerance: loss min smoothing window size\n tolerance_ratio: decision ratio for dead loss valley\n \"\"\"\n self.evaluator = evaluator\n self.n_converge = n_converge\n self.n_tolerance = n_tolerance\n self.tolerance_ratio = tolerance_ratio\n\n self.converge_Q = deque(maxlen=n_converge)\n self.smooth_Q = deque(maxlen=n_tolerance)\n\n self.final_model = None\n\n self.converge_step = None\n self.dead_valley = False\n self.threshold = None\n\n def get_smooth_loss(self, idx):\n smooth_loss = min([model.end_loss for model in list(self.smooth_Q)[idx:]])\n return smooth_loss\n\n @property\n def is_converged(self):\n return self.converge_step is not None\n\n def update_and_evaluate(self, segment_swa, val_acc, val_loss, prt_fn):\n if self.dead_valley:\n return\n\n frozen = copy.deepcopy(segment_swa)\n frozen.end_loss = val_loss\n self.converge_Q.append(frozen)\n self.smooth_Q.append(frozen)\n\n if not self.is_converged:\n if len(self.converge_Q) < self.n_converge:\n return\n\n min_idx = np.argmin([model.end_loss for model in self.converge_Q])\n untilmin_segment_swa = self.converge_Q[min_idx] # until-min segment swa.\n if min_idx == 0:\n self.converge_step = self.converge_Q[0].end_step\n self.final_model = swa_utils.AveragedModel(untilmin_segment_swa)\n\n th_base = np.mean([model.end_loss for model in self.converge_Q])\n self.threshold = th_base * (1.0 + self.tolerance_ratio)\n\n if self.n_tolerance < self.n_converge:\n for i in range(self.n_converge - self.n_tolerance):\n model = self.converge_Q[1 + i]\n self.final_model.update_parameters(\n model, start_step=model.start_step, end_step=model.end_step\n )\n elif self.n_tolerance > self.n_converge:\n converge_idx = self.n_tolerance - self.n_converge\n Q = list(self.smooth_Q)[: converge_idx + 1]\n start_idx = 0\n for i in reversed(range(len(Q))):\n model = Q[i]\n if model.end_loss > self.threshold:\n start_idx = i + 1\n break\n for model in Q[start_idx + 1 :]:\n self.final_model.update_parameters(\n model, start_step=model.start_step, end_step=model.end_step\n )\n print(\n f\"Model converged at step {self.converge_step}, \"\n f\"Start step = {self.final_model.start_step}; \"\n f\"Threshold = {self.threshold:.6f}, \"\n )\n return\n\n if self.smooth_Q[0].end_step < self.converge_step:\n return\n\n # converged -> loss valley\n min_vloss = self.get_smooth_loss(0)\n if min_vloss > self.threshold:\n self.dead_valley = True\n print(f\"Valley is dead at step {self.final_model.end_step}\")\n return\n\n model = self.smooth_Q[0]\n self.final_model.update_parameters(\n model, start_step=model.start_step, end_step=model.end_step\n )\n\n def get_final_model(self):\n if not self.is_converged:\n self.evaluator.logger.error(\n \"Requested final model, but model is not yet converged; return last model instead\"\n )\n return self.converge_Q[-1]\n\n if not self.dead_valley:\n self.smooth_Q.popleft()\n while self.smooth_Q:\n smooth_loss = self.get_smooth_loss(0)\n if smooth_loss > self.threshold:\n break\n segment_swa = self.smooth_Q.popleft()\n self.final_model.update_parameters(segment_swa, step=segment_swa.end_step)\n\n return self.final_model\n"
},
{
"path": "domain_generalization/swad/domainbed/trainer.py",
"content": "import collections\nimport json\nimport time\nimport copy\nfrom pathlib import Path\n\nimport numpy as np\nimport torch\nimport torch.utils.data\n\nfrom domainbed.datasets import get_dataset, split_dataset\nfrom domainbed import algorithms\nfrom domainbed.evaluator import Evaluator\nfrom domainbed.lr_scheduler import get_scheduler\nfrom domainbed.lib import misc\nfrom domainbed.lib import swa_utils\nfrom domainbed.lib.query import Q\nfrom domainbed.lib.fast_data_loader import InfiniteDataLoader, FastDataLoader\nfrom domainbed import swad as swad_module\n\nif torch.cuda.is_available():\n device = \"cuda\"\nelse:\n device = \"cpu\"\n\n\ndef json_handler(v):\n if isinstance(v, (Path, range)):\n return str(v)\n raise TypeError(f\"`{type(v)}` is not JSON Serializable\")\n\n\nclass TransformerDecorator(torch.nn.Module):\n def __init__(self, classifier, add_bt = 0, eval_batch=0):\n super(TransformerDecorator, self).__init__()\n self.add_bt = add_bt\n self.eval_batch = eval_batch\n if add_bt:\n self.encoder_layers = torch.nn.TransformerEncoderLayer(classifier.in_features, 4, classifier.in_features, 0.5)\n # if add_bt == 12:\n # self.encoder_layers = torch.nn.TransformerEncoder(self.encoder_layers, 2)\n # if add_bt == 2:\n # self.encoder_layers = torch.nn.TransformerEncoderLayer(classifier.in_features, 4, 512, 0.5)\n self.fc = classifier\n\n def forward(self, x):\n if self.training and self.add_bt:\n old_x = x\n x = x.unsqueeze(1)\n x = self.encoder_layers(x)\n x = x.squeeze(1)\n if self.add_bt in [16, 2, 12]:\n x = torch.cat([old_x, x], dim=0)\n x = self.fc(x)\n return x\n\ndef train(test_envs, args, hparams, n_steps, checkpoint_freq, logger, writer, target_env=None):\n logger.info(\"\")\n\n #######################################################\n # setup dataset & loader\n #######################################################\n args.real_test_envs = test_envs # for log\n algorithm_class = algorithms.get_algorithm_class(args.algorithm)\n dataset, in_splits, out_splits = get_dataset(test_envs, args, hparams, algorithm_class)\n test_splits = []\n if hparams.indomain_test > 0.0:\n logger.info(\"!!! In-domain test mode On !!!\")\n assert hparams[\"val_augment\"] is False, (\n \"indomain_test split the val set into val/test sets. \"\n \"Therefore, the val set should be not augmented.\"\n )\n val_splits = []\n for env_i, (out_split, _weights) in enumerate(out_splits):\n n = len(out_split) // 2\n seed = misc.seed_hash(args.trial_seed, env_i)\n val_split, test_split = split_dataset(out_split, n, seed=seed)\n val_splits.append((val_split, None))\n test_splits.append((test_split, None))\n logger.info(\n \"env %d: out (#%d) -> val (#%d) / test (#%d)\"\n % (env_i, len(out_split), len(val_split), len(test_split))\n )\n out_splits = val_splits\n\n if target_env is not None:\n testenv_name = f\"te_{dataset.environments[target_env]}\"\n logger.info(f\"Target env = {target_env}\")\n else:\n testenv_properties = [str(dataset.environments[i]) for i in test_envs]\n testenv_name = \"te_\" + \"_\".join(testenv_properties)\n\n logger.info(\n \"Testenv name escaping {} -> {}\".format(testenv_name, testenv_name.replace(\".\", \"\"))\n )\n testenv_name = testenv_name.replace(\".\", \"\")\n logger.info(f\"Test envs = {test_envs}, name = {testenv_name}\")\n\n n_envs = len(dataset)\n train_envs = sorted(set(range(n_envs)) - set(test_envs))\n iterator = misc.SplitIterator(test_envs)\n batch_sizes = np.full([n_envs], hparams[\"batch_size\"], dtype=np.int)\n\n batch_sizes[test_envs] = 0\n batch_sizes = batch_sizes.tolist()\n\n logger.info(f\"Batch sizes for each domain: {batch_sizes} (total={sum(batch_sizes)})\")\n\n # calculate steps per epoch\n steps_per_epochs = [\n len(env) / batch_size\n for (env, _), batch_size in iterator.train(zip(in_splits, batch_sizes))\n ]\n steps_per_epoch = min(steps_per_epochs)\n # epoch is computed by steps_per_epoch\n prt_steps = \", \".join([f\"{step:.2f}\" for step in steps_per_epochs])\n logger.info(f\"steps-per-epoch for each domain: {prt_steps} -> min = {steps_per_epoch:.2f}\")\n\n # setup loaders\n train_loaders = [\n InfiniteDataLoader(\n dataset=env,\n weights=env_weights,\n batch_size=batch_size,\n num_workers=dataset.N_WORKERS,\n )\n for (env, env_weights), batch_size in iterator.train(zip(in_splits, batch_sizes))\n ]\n\n # setup eval loaders\n eval_loaders_kwargs = []\n for i, (env, _) in enumerate(in_splits + out_splits + test_splits):\n batchsize = hparams[\"test_batchsize\"]\n loader_kwargs = {\"dataset\": env, \"batch_size\": batchsize, \"num_workers\": dataset.N_WORKERS}\n if args.prebuild_loader:\n loader_kwargs = FastDataLoader(**loader_kwargs)\n eval_loaders_kwargs.append(loader_kwargs)\n\n eval_weights = [None for _, weights in (in_splits + out_splits + test_splits)]\n eval_loader_names = [\"env{}_in\".format(i) for i in range(len(in_splits))]\n eval_loader_names += [\"env{}_out\".format(i) for i in range(len(out_splits))]\n eval_loader_names += [\"env{}_inTE\".format(i) for i in range(len(test_splits))]\n eval_meta = list(zip(eval_loader_names, eval_loaders_kwargs, eval_weights))\n\n #######################################################\n # setup algorithm (model)\n #######################################################\n algorithm = algorithm_class(\n dataset.input_shape,\n dataset.num_classes,\n len(dataset) - len(test_envs),\n hparams,\n )\n algorithm.classifier = TransformerDecorator(algorithm.classifier, args.add_bt)\n algorithm.network = torch.nn.Sequential(algorithm.featurizer, algorithm.classifier)\n from domainbed.optimizers import get_optimizer\n algorithm.optimizer = get_optimizer(\n hparams[\"optimizer\"],\n [{'params': [p for n, p in algorithm.network.named_parameters() if n.__contains__('encoder')],\n 'lr': algorithm.hparams[\"lr\"]*args.lr_scale_bt, },\n {'params': [p for n, p in algorithm.network.named_parameters() if not n.__contains__('encoder')],\n 'lr': algorithm.hparams[\"lr\"]}],\n lr=algorithm.hparams[\"lr\"],\n weight_decay=algorithm.hparams[\"weight_decay\"],\n )\n print('begin', algorithm.hparams[\"lr\"])\n print(algorithm.optimizer)\n print(algorithm.classifier)\n\n algorithm.to(device)\n\n n_params = sum([p.numel() for p in algorithm.parameters()])\n logger.info(\"# of params = %d\" % n_params)\n\n # setup scheduler\n scheduler = get_scheduler(\n hparams[\"scheduler\"],\n algorithm.optimizer,\n hparams[\"lr\"],\n n_steps,\n )\n\n train_minibatches_iterator = zip(*train_loaders)\n checkpoint_vals = collections.defaultdict(lambda: [])\n\n #######################################################\n # start training loop\n #######################################################\n evaluator = Evaluator(\n test_envs,\n eval_meta,\n n_envs,\n logger,\n evalmode=args.evalmode,\n debug=args.debug,\n target_env=target_env,\n )\n\n swad = None\n if hparams[\"swad\"]:\n swad_algorithm = swa_utils.AveragedModel(algorithm)\n swad_cls = getattr(swad_module, hparams[\"swad\"])\n swad = swad_cls(evaluator, **hparams.swad_kwargs)\n\n last_results_keys = None\n records = []\n epochs_path = args.out_dir / \"results.jsonl\"\n\n for step in range(n_steps):\n step_start_time = time.time()\n # batches_dictlist: [{env0_data_key: tensor, env0_...}, env1_..., ...]\n batches_dictlist = next(train_minibatches_iterator)\n # batches: {data_key: [env0_tensor, ...], ...}\n batches = misc.merge_dictlist(batches_dictlist)\n # to device\n batches = {\n key: [tensor.to(device) for tensor in tensorlist] for key, tensorlist in batches.items()\n }\n\n inputs = {**batches, \"step\": step}\n step_vals = algorithm.update(**inputs)\n for key, val in step_vals.items():\n checkpoint_vals[key].append(val)\n checkpoint_vals[\"step_time\"].append(time.time() - step_start_time)\n\n if swad:\n # swad_algorithm is segment_swa for swad\n swad_algorithm.update_parameters(algorithm, step=step)\n\n scheduler.step()\n\n if step % checkpoint_freq == 0:\n results = {\n \"step\": step,\n \"epoch\": step / steps_per_epoch,\n }\n\n for key, val in checkpoint_vals.items():\n results[key] = np.mean(val)\n\n eval_start_time = time.time()\n accuracies, summaries = evaluator.evaluate(algorithm)\n results[\"eval_time\"] = time.time() - eval_start_time\n\n # results = (epochs, loss, step, step_time)\n results_keys = list(summaries.keys()) + sorted(accuracies.keys()) + list(results.keys())\n # merge results\n results.update(summaries)\n results.update(accuracies)\n\n # print\n if results_keys != last_results_keys:\n logger.info(misc.to_row(results_keys))\n last_results_keys = results_keys\n logger.info(misc.to_row([results[key] for key in results_keys]))\n records.append(copy.deepcopy(results))\n\n # update results to record\n results.update({\"hparams\": dict(hparams), \"args\": vars(args)})\n\n with open(epochs_path, \"a\") as f:\n f.write(json.dumps(results, sort_keys=True, default=json_handler) + \"\\n\")\n\n checkpoint_vals = collections.defaultdict(lambda: [])\n\n writer.add_scalars_with_prefix(summaries, step, f\"{testenv_name}/summary/\")\n writer.add_scalars_with_prefix(accuracies, step, f\"{testenv_name}/all/\")\n\n if args.model_save and step >= args.model_save:\n ckpt_dir = args.out_dir / \"checkpoints\"\n ckpt_dir.mkdir(exist_ok=True)\n\n test_env_str = \",\".join(map(str, test_envs))\n filename = \"TE{}_{}.pth\".format(test_env_str, step)\n if len(test_envs) > 1 and target_env is not None:\n train_env_str = \",\".join(map(str, train_envs))\n filename = f\"TE{target_env}_TR{train_env_str}_{step}.pth\"\n path = ckpt_dir / filename\n\n save_dict = {\n \"args\": vars(args),\n \"model_hparams\": dict(hparams),\n \"test_envs\": test_envs,\n \"model_dict\": algorithm.cpu().state_dict(),\n }\n algorithm.cuda()\n if not args.debug:\n torch.save(save_dict, path)\n else:\n logger.debug(\"DEBUG Mode -> no save (org path: %s)\" % path)\n\n # swad\n if swad:\n def prt_results_fn(results, avgmodel):\n step_str = f\" [{avgmodel.start_step}-{avgmodel.end_step}]\"\n row = misc.to_row([results[key] for key in results_keys if key in results])\n logger.info(row + step_str)\n\n swad.update_and_evaluate(\n swad_algorithm, results[\"train_out\"], results[\"tr_outloss\"], prt_results_fn\n )\n\n if hasattr(swad, \"dead_valley\") and swad.dead_valley:\n logger.info(\"SWAD valley is dead -> early stop !\")\n break\n\n swad_algorithm = swa_utils.AveragedModel(algorithm) # reset\n\n if step % args.tb_freq == 0:\n # add step values only for tb log\n step_vals[\"lr\"] = scheduler.get_last_lr()[0]\n writer.add_scalars_with_prefix(step_vals, step, f\"{testenv_name}/summary/\")\n\n # find best\n logger.info(\"---\")\n records = Q(records)\n oracle_best = records.argmax(\"test_out\")[\"test_in\"]\n iid_best = records.argmax(\"train_out\")[\"test_in\"]\n last = records[-1][\"test_in\"]\n\n if hparams.indomain_test:\n # if test set exist, use test set for indomain results\n in_key = \"train_inTE\"\n else:\n in_key = \"train_out\"\n\n iid_best_indomain = records.argmax(\"train_out\")[in_key]\n last_indomain = records[-1][in_key]\n\n ret = {\n \"oracle\": oracle_best,\n \"iid\": iid_best,\n \"last\": last,\n \"last (inD)\": last_indomain,\n \"iid (inD)\": iid_best_indomain,\n }\n\n # Evaluate SWAD\n if swad:\n swad_algorithm = swad.get_final_model()\n if hparams[\"freeze_bn\"] is False:\n n_steps = 500 if not args.debug else 10\n logger.warning(f\"Update SWAD BN statistics for {n_steps} steps ...\")\n swa_utils.update_bn(train_minibatches_iterator, swad_algorithm, n_steps)\n\n logger.warning(\"Evaluate SWAD ...\")\n accuracies, summaries = evaluator.evaluate(swad_algorithm)\n results = {**summaries, **accuracies}\n start = swad_algorithm.start_step\n end = swad_algorithm.end_step\n step_str = f\" [{start}-{end}] (N={swad_algorithm.n_averaged})\"\n row = misc.to_row([results[key] for key in results_keys if key in results]) + step_str\n logger.info(row)\n\n ret[\"SWAD\"] = results[\"test_in\"]\n ret[\"SWAD (inD)\"] = results[in_key]\n\n for k, acc in ret.items():\n logger.info(f\"{k} = {acc:.3%}\")\n\n return ret, records\n"
},
{
"path": "domain_generalization/swad/requirements.txt",
"content": "prettytable==2.0.0\nnumpy==1.19.4\ntorch==1.7.0+cu92\ngdown==3.12.2\nimageio==2.9.0\nsconf==0.2.3\ntorchvision==0.8.1+cu92\ntqdm==4.51.0\ntensorboard==2.3.0\nPillow==8.1.0\ntensorboardX==2.1\n"
},
{
"path": "domain_generalization/swad/train_all.py",
"content": "import argparse\nimport collections\nimport random\nimport sys\nfrom pathlib import Path\n\nimport numpy as np\nimport PIL\nimport torch\nimport torchvision\nfrom sconf import Config\nfrom prettytable import PrettyTable\n\nfrom domainbed.datasets import get_dataset\nfrom domainbed import hparams_registry\nfrom domainbed.lib import misc\nfrom domainbed.lib.writers import get_writer\nfrom domainbed.lib.logger import Logger\nfrom domainbed.trainer import train\n\n\ndef main():\n parser = argparse.ArgumentParser(description=\"Domain generalization\")\n parser.add_argument(\"name\", type=str)\n parser.add_argument(\"configs\", nargs=\"*\")\n parser.add_argument(\"--data_dir\", type=str, default=\"datadir/\")\n parser.add_argument(\"--dataset\", type=str, default=\"PACS\")\n parser.add_argument(\"--algorithm\", type=str, default=\"ERM\")\n parser.add_argument(\n \"--trial_seed\",\n type=int,\n default=0,\n help=\"Trial number (used for seeding split_dataset and random_hparams).\",\n )\n parser.add_argument(\"--seed\", type=int, default=0, help=\"Seed for everything else\")\n parser.add_argument(\n \"--steps\", type=int, default=None, help=\"Number of steps. Default is dataset-dependent.\"\n )\n parser.add_argument(\n \"--checkpoint_freq\",\n type=int,\n default=None,\n help=\"Checkpoint every N steps. Default is dataset-dependent.\",\n )\n parser.add_argument(\"--test_envs\", type=int, nargs=\"+\", default=None) # sketch in PACS\n parser.add_argument(\"--holdout_fraction\", type=float, default=0.2)\n parser.add_argument(\"--model_save\", default=None, type=int, help=\"Model save start step\")\n parser.add_argument(\"--deterministic\", action=\"store_true\")\n parser.add_argument(\"--tb_freq\", default=10)\n parser.add_argument(\"--add_bt\", default=0, type=int)\n parser.add_argument(\"--baseline\", action=\"store_true\", help=\"baseline\")\n parser.add_argument(\"--lr_scale_bt\", default=1., type=float, help='')\n parser.add_argument(\"--resnet18\", action=\"store_true\", help=\"baseline\")\n parser.add_argument(\"--debug\", action=\"store_true\", help=\"Run w/ debug mode\")\n parser.add_argument(\"--show\", action=\"store_true\", help=\"Show args and hparams w/o run\")\n parser.add_argument(\n \"--evalmode\",\n default=\"fast\",\n help=\"[fast, all]. if fast, ignore train_in datasets in evaluation time.\",\n )\n parser.add_argument(\"--prebuild_loader\", action=\"store_true\", help=\"Pre-build eval loaders\")\n args, left_argv = parser.parse_known_args()\n\n # setup hparams\n hparams = hparams_registry.default_hparams(args.algorithm, args.dataset)\n\n keys = [\"config.yaml\"] + args.configs\n keys = [open(key, encoding=\"utf8\") for key in keys]\n hparams = Config(*keys, default=hparams)\n hparams.argv_update(left_argv)\n if args.baseline:\n hparams['swad'] =None\n if args.resnet18:\n hparams['resnet18'] = (True, True)\n\n # setup debug\n if args.debug:\n args.checkpoint_freq = 5\n args.steps = 10\n args.name += \"_debug\"\n\n timestamp = misc.timestamp()\n args.unique_name = f\"{timestamp}_{args.name}\"\n\n # path setup\n args.work_dir = Path(\".\")\n args.data_dir = Path(args.data_dir)\n\n args.out_root = args.work_dir / Path(\"train_output\") / args.dataset\n args.out_dir = args.out_root / args.unique_name\n args.out_dir.mkdir(exist_ok=True, parents=True)\n\n writer = get_writer(args.out_root / \"runs\" / args.unique_name)\n logger = Logger.get(args.out_dir / \"log.txt\")\n if args.debug:\n logger.setLevel(\"DEBUG\")\n cmd = \" \".join(sys.argv)\n logger.info(f\"Command :: {cmd}\")\n\n logger.nofmt(\"Environment:\")\n logger.nofmt(\"\\tPython: {}\".format(sys.version.split(\" \")[0]))\n logger.nofmt(\"\\tPyTorch: {}\".format(torch.__version__))\n logger.nofmt(\"\\tTorchvision: {}\".format(torchvision.__version__))\n logger.nofmt(\"\\tCUDA: {}\".format(torch.version.cuda))\n logger.nofmt(\"\\tCUDNN: {}\".format(torch.backends.cudnn.version()))\n logger.nofmt(\"\\tNumPy: {}\".format(np.__version__))\n logger.nofmt(\"\\tPIL: {}\".format(PIL.__version__))\n\n # Different to DomainBed, we support CUDA only.\n assert torch.cuda.is_available(), \"CUDA is not available\"\n\n logger.nofmt(\"Args:\")\n for k, v in sorted(vars(args).items()):\n logger.nofmt(\"\\t{}: {}\".format(k, v))\n\n logger.nofmt(\"HParams:\")\n for line in hparams.dumps().split(\"\\n\"):\n logger.nofmt(\"\\t\" + line)\n\n if args.show:\n exit()\n\n # seed\n random.seed(args.seed)\n np.random.seed(args.seed)\n torch.manual_seed(args.seed)\n torch.backends.cudnn.deterministic = args.deterministic\n torch.backends.cudnn.benchmark = not args.deterministic\n\n # Dummy datasets for logging information.\n # Real dataset will be re-assigned in train function.\n # test_envs only decide transforms; simply set to zero.\n dataset, _in_splits, _out_splits = get_dataset([0], args, hparams)\n\n # print dataset information\n logger.nofmt(\"Dataset:\")\n logger.nofmt(f\"\\t[{args.dataset}] #envs={len(dataset)}, #classes={dataset.num_classes}\")\n for i, env_property in enumerate(dataset.environments):\n logger.nofmt(f\"\\tenv{i}: {env_property} (#{len(dataset[i])})\")\n logger.nofmt(\"\")\n\n n_steps = args.steps or dataset.N_STEPS\n checkpoint_freq = args.checkpoint_freq or dataset.CHECKPOINT_FREQ\n logger.info(f\"n_steps = {n_steps}\")\n logger.info(f\"checkpoint_freq = {checkpoint_freq}\")\n\n org_n_steps = n_steps\n n_steps = (n_steps // checkpoint_freq) * checkpoint_freq + 1\n logger.info(f\"n_steps is updated to {org_n_steps} => {n_steps} for checkpointing\")\n\n if not args.test_envs:\n args.test_envs = [[te] for te in range(len(dataset))]\n logger.info(f\"Target test envs = {args.test_envs}\")\n\n ###########################################################################\n # Run\n ###########################################################################\n all_records = []\n results = collections.defaultdict(list)\n\n for test_env in args.test_envs:\n res, records = train(\n test_env,\n args=args,\n hparams=hparams,\n n_steps=n_steps,\n checkpoint_freq=checkpoint_freq,\n logger=logger,\n writer=writer,\n )\n all_records.append(records)\n for k, v in res.items():\n results[k].append(v)\n\n # log summary table\n logger.info(\"=== Summary ===\")\n logger.info(f\"Command: {' '.join(sys.argv)}\")\n logger.info(\"Unique name: %s\" % args.unique_name)\n logger.info(\"Out path: %s\" % args.out_dir)\n logger.info(\"Algorithm: %s\" % args.algorithm)\n logger.info(\"Dataset: %s\" % args.dataset)\n\n table = PrettyTable([\"Selection\"] + dataset.environments + [\"Avg.\"])\n for key, row in results.items():\n row.append(np.mean(row))\n row = [f\"{acc:.3%}\" for acc in row]\n table.add_row([key] + row)\n logger.nofmt(table)\n\n\nif __name__ == \"__main__\":\n main()\n"
},
{
"path": "long-tailed_recognition/BalancedSoftmax/.gitignore",
"content": "logs\n*pkl\n*pyc\n.vscode\n*.sh\nruns\n.vscode\n.idea"
},
{
"path": "long-tailed_recognition/BalancedSoftmax/CODE_OF_CONDUCT.md",
"content": "# Code of Conduct\n\nFacebook has adopted a Code of Conduct that we expect project participants to adhere to.\nPlease read the [full text](https://code.fb.com/codeofconduct/)\nso that you can understand what actions will and will not be tolerated."
},
{
"path": "long-tailed_recognition/BalancedSoftmax/CONTRIBUTING.md",
"content": "# Contributing to Classifier-Balancing\nWe want to make contributing to this project as easy and transparent as\npossible.\n\n## Our Development Process\nMinor changes and improvements will be released on an ongoing basis. Larger changes (e.g., changesets implementing a new paper) will be released on a more periodic basis.\n\n## Pull Requests\nWe actively welcome your pull requests.\n\n1. Fork the repo and create your branch from `master`.\n2. If you've added code that should be tested, add tests.\n3. If you've changed APIs, update the documentation.\n4. Ensure the test suite passes.\n5. Make sure your code lints.\n6. If you haven't already, complete the Contributor License Agreement (\"CLA\").\n\n## Contributor License Agreement (\"CLA\")\nIn order to accept your pull request, we need you to submit a CLA. You only need\nto do this once to work on any of Facebook's open source projects.\n\nComplete your CLA here: \n\n## Issues\nWe use GitHub issues to track public bugs. Please ensure your description is\nclear and has sufficient instructions to be able to reproduce the issue.\n\nFacebook has a [bounty program](https://www.facebook.com/whitehat/) for the safe\ndisclosure of security bugs. In those cases, please go through the process\noutlined on that page and do not file a public issue.\n\n## Coding Style \n* 4 spaces for indentation rather than tabs\n\n\n## License\nBy contributing to Classifier-Balancing, you agree that your contributions will be licensed\nunder the LICENSE file in the root directory of this source tree.\n"
},
{
"path": "long-tailed_recognition/BalancedSoftmax/LICENSE",
"content": "BSD License\n\nFor Classifier-Balacing software\n\nCopyright (c) Facebook, Inc. and its affiliates. All rights reserved.\n\nRedistribution and use in source and binary forms, with or without modification,\nare permitted provided that the following conditions are met:\n\n * Redistributions of source code must retain the above copyright notice, this\n list of conditions and the following disclaimer.\n\n * Redistributions in binary form must reproduce the above copyright notice,\n this list of conditions and the following disclaimer in the documentation\n and/or other materials provided with the distribution.\n\n * Neither the name Facebook nor the names of its contributors may be used to\n endorse or promote products derived from this software without specific\n prior written permission.\n\nTHIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS \"AS IS\" AND\nANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED\nWARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE\nDISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR\nANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES\n(INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;\nLOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON\nANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT\n(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS\nSOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.\n\nPortions of the source code are from the OLTR project and are licensed under\nthe following terms:\n\nBSD 3-Clause License\n\nCopyright (c) 2019, Zhongqi Miao. All rights reserved.\n\nRedistribution and use in source and binary forms, with or without\nmodification, are permitted provided that the following conditions are met:\n\n1. Redistributions of source code must retain the above copyright notice, this\n list of conditions and the following disclaimer.\n\n2. Redistributions in binary form must reproduce the above copyright notice,\n this list of conditions and the following disclaimer in the documentation\n and/or other materials provided with the distribution.\n\n3. Neither the name of the copyright holder nor the names of its\n contributors may be used to endorse or promote products derived from\n this software without specific prior written permission.\n\nTHIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS \"AS IS\"\nAND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE\nIMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE\nDISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE\nFOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL\nDAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR\nSERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER\nCAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,\nOR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE\nOF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE."
},
{
"path": "long-tailed_recognition/BalancedSoftmax/README.md",
"content": "# BatchFormer for BalancedSoftmax\n\nCode for the paper BatchFormer on BalancedSoftmax. This repository includes the ablation study codes and thus might also include some redudanct code.\n\n## Requirements \n* Python 3\n* [PyTorch](https://pytorch.org/) (version == 1.4)\n* [yaml](https://pyyaml.org/wiki/PyYAMLDocumentation)\n* [higher](https://github.com/facebookresearch/higher)(version == 0.2.1)\n\n\n\n## Training\n### End-to-end Training\n\n- Balanced Softmax\n```bash\npython main.py --cfg ./config/ImageNet_LT/balanced_softmax_resnet10.yaml\n```\n\n- BatchFormer\n```bash\npython main.py --cfg ./config/ImageNet_LT/balanced_softmax_resnet10.yaml --add_bt 1\n```\n\n## Evaluation\n\nModel evaluation can be done using the following command:\n```bash\npython main.py --cfg ./config/CIFAR10_LT/balanced_softmax_imba200.yaml --test\n```\n\nWe use debug=4 to stat the gradients for other images in the mini-batch.\n\n## Citation\n```bibtex\n@inproceedings{hou2022batch,\n title={BatchFormer: Learning to Explore Sample Relationships for Robust Representation Learning},\n author={Hou, Zhi and Yu, Baosheng and Tao, Dacheng},\n booktitle={CVPR},\n year={2022}\n}\n```\n\n\n```bibtex\n@inproceedings{\n Ren2020balms,\n title={Balanced Meta-Softmax for Long-Tailed Visual Recognition},\n author={Jiawei Ren and Cunjun Yu and Shunan Sheng and Xiao Ma and Haiyu Zhao and Shuai Yi and Hongsheng Li},\n booktitle={Proceedings of Neural Information Processing Systems(NeurIPS)},\n month = {Dec},\n year={2020}\n}\n```\n\nCode are based on [Balanced Meta-Softmax](https://github.com/jiawei-ren/BalancedMetaSoftmax-Classification.git)\n"
},
{
"path": "long-tailed_recognition/BalancedSoftmax/cls_freq/CIFAR-10-LT_IMBA100.json",
"content": "[5000, 2997, 1796, 1077, 645, 387, 232, 139, 83, 50]"
},
{
"path": "long-tailed_recognition/BalancedSoftmax/cls_freq/CIFAR-10-LT_IMBA200.json",
"content": "[5000, 2775, 1540, 854, 474, 263, 146, 81, 45, 25]"
},
{
"path": "long-tailed_recognition/BalancedSoftmax/cls_freq/CIFAR-100-LT_IMBA1.json",
"content": "[500, 500, 500, 500, 500, 500, 500, 500, 500, 500, 500, 500, 500, 500, 500, 500, 500, 500, 500, 500, 500, 500, 500, 500, 500, 500, 500, 500, 500, 500, 500, 500, 500, 500, 500, 500, 500, 500, 500, 500, 500, 500, 500, 500, 500, 500, 500, 500, 500, 500, 500, 500, 500, 500, 500, 500, 500, 500, 500, 500, 500, 500, 500, 500, 500, 500, 500, 500, 500, 500, 500, 500, 500, 500, 500, 500, 500, 500, 500, 500, 500, 500, 500, 500, 500, 500, 500, 500, 500, 500, 500, 500, 500, 500, 500, 500, 500, 500, 500, 500]"
},
{
"path": "long-tailed_recognition/BalancedSoftmax/cls_freq/CIFAR-100-LT_IMBA100.json",
"content": "[500, 477, 455, 434, 415, 396, 378, 361, 344, 328, 314, 299, 286, 273, 260, 248, 237, 226, 216, 206, 197, 188, 179, 171, 163, 156, 149, 142, 135, 129, 123, 118, 112, 107, 102, 98, 93, 89, 85, 81, 77, 74, 70, 67, 64, 61, 58, 56, 53, 51, 48, 46, 44, 42, 40, 38, 36, 35, 33, 32, 30, 29, 27, 26, 25, 24, 23, 22, 21, 20, 19, 18, 17, 16, 15, 15, 14, 13, 13, 12, 12, 11, 11, 10, 10, 9, 9, 8, 8, 7, 7, 7, 6, 6, 6, 6, 5, 5, 5, 5]"
},
{
"path": "long-tailed_recognition/BalancedSoftmax/cls_freq/CIFAR-100-LT_IMBA200.json",
"content": "[500, 473, 449, 425, 403, 382, 362, 343, 325, 308, 292, 277, 263, 249, 236, 224, 212, 201, 190, 180, 171, 162, 154, 146, 138, 131, 124, 117, 111, 105, 100, 95, 90, 85, 81, 76, 72, 69, 65, 62, 58, 55, 52, 50, 47, 44, 42, 40, 38, 36, 34, 32, 30, 29, 27, 26, 24, 23, 22, 21, 20, 19, 18, 17, 16, 15, 14, 13, 13, 12, 11, 11, 10, 10, 9, 9, 8, 8, 7, 7, 6, 6, 6, 5, 5, 5, 5, 4, 4, 4, 4, 3, 3, 3, 3, 3, 2, 2, 2, 2]"
},
{
"path": "long-tailed_recognition/BalancedSoftmax/cls_freq/ImageNet_LT.json",
"content": "[50, 250, 516, 352, 143, 92, 47, 45, 150, 50, 34, 55, 25, 35, 121, 7, 232, 13, 392, 157, 87, 181, 177, 171, 114, 56, 134, 56, 279, 248, 287, 83, 65, 56, 118, 43, 68, 39, 14, 96, 80, 15, 236, 103, 64, 54, 346, 32, 201, 29, 376, 27, 189, 96, 69, 157, 16, 173, 5, 34, 34, 150, 57, 78, 31, 8, 44, 297, 349, 72, 35, 36, 428, 10, 137, 58, 80, 44, 30, 15, 126, 93, 34, 135, 226, 281, 1280, 19, 38, 502, 186, 81, 61, 222, 15, 28, 55, 819, 74, 24, 10, 93, 14, 111, 61, 32, 116, 133, 128, 66, 35, 50, 393, 47, 179, 124, 79, 604, 72, 240, 22, 148, 26, 64, 20, 23, 246, 11, 79, 161, 78, 370, 197, 155, 285, 59, 59, 178, 43, 75, 11, 23, 16, 74, 27, 87, 86, 105, 189, 37, 126, 53, 10, 30, 77, 5, 245, 30, 100, 37, 69, 119, 26, 47, 81, 100, 25, 35, 112, 35, 7, 35, 272, 469, 150, 100, 199, 137, 69, 42, 10, 20, 367, 49, 73, 38, 56, 75, 89, 24, 35, 244, 305, 23, 106, 77, 10, 59, 90, 33, 26, 214, 43, 50, 18, 125, 110, 119, 85, 43, 11, 11, 114, 171, 24, 54, 87, 39, 86, 496, 91, 52, 56, 11, 117, 48, 62, 100, 41, 138, 22, 7, 50, 51, 6, 281, 29, 13, 36, 75, 126, 127, 59, 56, 172, 119, 90, 163, 17, 53, 181, 81, 63, 167, 45, 37, 39, 282, 71, 32, 12, 137, 102, 97, 60, 14, 66, 14, 6, 132, 303, 81, 167, 28, 15, 72, 187, 20, 310, 137, 241, 123, 27, 19, 186, 12, 12, 42, 71, 96, 71, 53, 53, 19, 62, 43, 330, 41, 11, 42, 43, 22, 208, 18, 566, 46, 19, 10, 58, 306, 67, 119, 50, 51, 87, 143, 37, 16, 611, 290, 170, 114, 19, 79, 81, 128, 77, 332, 8, 446, 42, 122, 86, 1121, 48, 109, 133, 7, 174, 17, 283, 140, 27, 15, 30, 134, 167, 198, 149, 16, 122, 23, 194, 149, 23, 156, 5, 127, 11, 31, 49, 143, 145, 169, 20, 158, 87, 128, 96, 65, 49, 391, 349, 83, 123, 232, 12, 51, 137, 31, 73, 32, 41, 65, 6, 6, 85, 231, 171, 28, 108, 96, 101, 32, 122, 26, 165, 10, 372, 34, 31, 63, 679, 34, 257, 27, 83, 23, 275, 13, 22, 47, 118, 411, 159, 124, 45, 1053, 111, 184, 68, 52, 41, 150, 10, 47, 109, 32, 69, 34, 38, 10, 30, 82, 16, 157, 14, 141, 10, 58, 25, 152, 165, 112, 480, 91, 43, 113, 22, 236, 48, 137, 48, 147, 511, 17, 86, 33, 44, 36, 333, 19, 48, 15, 224, 108, 152, 16, 186, 114, 31, 82, 125, 155, 59, 186, 188, 134, 25, 121, 288, 39, 8, 32, 41, 346, 127, 232, 123, 81, 652, 6, 154, 203, 48, 66, 178, 45, 6, 22, 85, 102, 44, 105, 44, 54, 42, 20, 122, 220, 82, 44, 43, 75, 8, 142, 82, 38, 231, 619, 162, 62, 134, 8, 71, 41, 86, 55, 116, 35, 94, 100, 330, 54, 87, 143, 110, 191, 6, 48, 153, 84, 16, 55, 10, 422, 92, 25, 507, 168, 104, 83, 70, 47, 382, 148, 10, 164, 30, 310, 381, 174, 92, 87, 268, 202, 1173, 110, 8, 125, 51, 340, 37, 209, 71, 14, 14, 46, 252, 14, 239, 58, 22, 112, 22, 101, 130, 278, 18, 220, 21, 253, 39, 292, 317, 179, 36, 305, 32, 65, 169, 683, 98, 376, 125, 39, 8, 28, 298, 132, 43, 69, 80, 413, 19, 25, 354, 120, 72, 86, 50, 76, 72, 102, 310, 21, 26, 119, 140, 75, 63, 57, 10, 10, 31, 106, 54, 100, 90, 64, 88, 34, 90, 36, 85, 82, 6, 64, 86, 29, 6, 30, 204, 27, 209, 115, 24, 72, 253, 55, 10, 33, 104, 38, 46, 296, 26, 128, 170, 15, 66, 130, 138, 28, 1246, 107, 63, 75, 108, 167, 61, 26, 83, 77, 42, 113, 103, 264, 59, 16, 49, 29, 33, 33, 481, 29, 45, 24, 38, 165, 82, 36, 15, 60, 62, 72, 73, 25, 8, 182, 99, 283, 101, 297, 28, 194, 26, 141, 14, 78, 13, 67, 82, 63, 103, 454, 6, 22, 66, 93, 8, 11, 131, 137, 40, 66, 25, 472, 9, 183, 32, 14, 43, 299, 56, 40, 392, 5, 20, 31, 32, 83, 194, 81, 80, 787, 7, 137, 50, 9, 133, 34, 174, 315, 35, 290, 9, 263, 164, 86, 129, 20, 53, 175, 43, 775, 11, 54, 12, 137, 31, 87, 32, 229, 96, 161, 126, 271, 221, 27, 28, 36, 14, 85, 197, 82, 76, 15, 63, 130, 11, 85, 134, 211, 21, 328, 302, 205, 21, 53, 102, 93, 111, 19, 41, 17, 218, 128, 242, 188, 166, 105, 46, 7, 189, 20, 73, 247, 229, 35, 343, 95, 146, 57, 285, 43, 60, 84, 64, 28, 17, 44, 18, 39, 175, 25, 60, 134, 8, 54, 46, 31, 17, 10, 9, 143, 114, 101, 157, 70, 6, 75, 192, 64, 65, 5, 8, 9, 31, 15, 642, 31, 107, 11, 69, 459, 238, 19, 31, 134, 252, 25, 456, 15, 80, 102, 274, 6, 32, 50, 140, 104, 112, 133, 28, 182, 170, 76, 173, 272, 20, 102, 30, 138, 150, 116, 110, 216, 42, 39, 116, 16, 13, 215, 85, 155, 19, 7, 36, 66, 323, 22, 284, 172, 398, 110, 56, 25, 53, 534, 43, 38, 26, 418, 141, 7, 36, 55, 20, 63, 45, 18, 80, 67, 151, 94, 103, 284, 180, 23, 175, 472, 133, 99, 71, 67, 80, 32, 113, 17, 57, 172, 17, 19, 93, 124, 23, 55, 258, 108, 72, 33, 165, 95, 52, 45, 625, 73, 217, 98, 45, 34, 41, 39, 714, 143, 141, 233, 119, 63, 236, 150, 19, 15, 35, 167, 19, 25, 21, 21]"
},
{
"path": "long-tailed_recognition/BalancedSoftmax/cls_freq/Places_LT.json",
"content": "[219, 20, 115, 86, 229, 33, 25, 4980, 108, 2612, 33, 39, 2400, 87, 12, 225, 41, 98, 191, 1582, 37, 78, 48, 7, 1530, 56, 93, 84, 18, 183, 155, 72, 92, 78, 10, 120, 22, 6, 143, 98, 195, 36, 1487, 93, 73, 1406, 8, 40, 1380, 138, 11, 14, 1085, 43, 59, 20, 6, 129, 103, 98, 46, 41, 1036, 82, 14, 23, 1023, 841, 8, 91, 20, 30, 19, 34, 44, 11, 198, 129, 135, 132, 29, 830, 725, 11, 689, 88, 668, 48, 5, 38, 643, 605, 71, 6, 566, 72, 34, 547, 54, 72, 6, 33, 36, 97, 76, 78, 521, 97, 116, 157, 499, 8, 99, 74, 24, 62, 28, 128, 176, 72, 87, 482, 148, 86, 40, 147, 5, 108, 6, 8, 35, 24, 480, 32, 140, 7, 84, 6, 46, 22, 104, 85, 118, 59, 72, 36, 34, 80, 33, 100, 475, 470, 79, 164, 451, 7, 28, 171, 65, 15, 8, 80, 20, 208, 450, 28, 9, 60, 9, 10, 13, 431, 9, 22, 34, 428, 72, 17, 34, 42, 73, 98, 188, 422, 42, 20, 60, 21, 6, 39, 50, 23, 408, 167, 99, 5, 25, 97, 13, 105, 55, 13, 41, 392, 389, 383, 54, 12, 17, 207, 57, 18, 63, 81, 374, 371, 48, 367, 25, 16, 366, 98, 7, 104, 364, 28, 89, 68, 15, 17, 46, 113, 334, 56, 60, 12, 55, 51, 87, 58, 26, 15, 22, 331, 179, 318, 12, 18, 8, 94, 65, 71, 315, 29, 173, 9, 14, 315, 226, 124, 33, 9, 9, 51, 15, 52, 309, 34, 209, 61, 116, 298, 298, 149, 60, 70, 89, 218, 291, 290, 30, 36, 16, 186, 287, 36, 195, 80, 283, 39, 141, 14, 181, 152, 11, 6, 22, 39, 19, 280, 11, 173, 11, 43, 112, 177, 111, 279, 108, 153, 103, 19, 265, 264, 95, 72, 40, 263, 19, 229, 45, 54, 12, 260, 253, 32, 253, 27, 52, 34, 250, 10, 11, 49, 250, 17, 24, 242, 74, 14, 48, 216, 241, 16, 238, 232, 8, 33, 205, 93, 46, 7, 46, 82, 44, 230, 91, 218, 21, 69, 156, 206, 103, 14, 20]"
},
{
"path": "long-tailed_recognition/BalancedSoftmax/cls_freq/iNaturalist18.json",
"content": "[22, 34, 21, 27, 15, 12, 22, 23, 28, 15, 144, 28, 12, 23, 111, 554, 26, 22, 142, 23, 13, 27, 33, 10, 132, 26, 28, 19, 17, 23, 10, 29, 12, 65, 14, 17, 178, 10, 21, 17, 23, 13, 22, 19, 53, 16, 12, 24, 18, 25, 13, 28, 13, 18, 17, 31, 19, 200, 22, 34, 26, 11, 7, 11, 11, 15, 25, 9, 11, 9, 11, 15, 23, 20, 19, 22, 146, 17, 21, 11, 33, 10, 27, 13, 10, 20, 23, 126, 12, 32, 23, 16, 25, 81, 46, 18, 32, 23, 12, 16, 44, 11, 25, 13, 12, 21, 27, 19, 20, 24, 22, 19, 64, 20, 19, 15, 62, 26, 17, 28, 19, 25, 33, 21, 10, 59, 23, 15, 17, 19, 17, 17, 25, 118, 12, 11, 27, 19, 17, 22, 16, 57, 11, 13, 21, 25, 15, 25, 22, 24, 26, 17, 18, 27, 19, 26, 76, 24, 25, 16, 17, 12, 14, 16, 13, 23, 12, 38, 26, 10, 23, 272, 11, 19, 22, 17, 18, 23, 22, 25, 24, 24, 23, 20, 35, 13, 24, 18, 20, 21, 25, 23, 24, 24, 20, 20, 15, 21, 139, 12, 23, 21, 22, 26, 26, 15, 12, 41, 20, 25, 22, 25, 11, 197, 25, 14, 72, 14, 14, 20, 14, 18, 23, 16, 27, 23, 23, 22, 18, 22, 22, 16, 18, 25, 25, 17, 27, 15, 22, 696, 20, 11, 29, 26, 64, 12, 149, 24, 948, 17, 430, 25, 19, 27, 34, 13, 12, 24, 26, 25, 28, 19, 18, 18, 20, 19, 54, 25, 16, 10, 23, 34, 25, 16, 11, 13, 15, 17, 80, 27, 25, 12, 129, 31, 11, 17, 14, 29, 27, 25, 24, 34, 23, 38, 202, 11, 28, 38, 26, 27, 22, 12, 25, 8, 78, 16, 25, 28, 27, 13, 24, 14, 29, 28, 36, 18, 27, 24, 43, 29, 22, 7, 16, 11, 31, 10, 24, 17, 26, 23, 13, 16, 25, 26, 31, 8, 20, 21, 13, 22, 25, 27, 11, 14, 14, 24, 15, 10, 16, 17, 80, 16, 18, 17, 12, 13, 25, 32, 30, 19, 15, 26, 17, 25, 18, 20, 12, 19, 26, 124, 19, 26, 16, 14, 15, 14, 13, 24, 39, 21, 25, 37, 12, 33, 20, 26, 27, 25, 16, 28, 21, 25, 16, 12, 11, 12, 25, 15, 10, 59, 19, 24, 22, 21, 34, 52, 26, 27, 22, 23, 15, 16, 14, 34, 20, 11, 59, 12, 22, 119, 16, 20, 18, 384, 12, 19, 26, 14, 25, 19, 14, 25, 21, 11, 14, 20, 22, 44, 25, 12, 8, 23, 69, 19, 9, 23, 26, 19, 16, 12, 28, 7, 60, 15, 19, 12, 108, 29, 10, 23, 12, 15, 22, 15, 28, 24, 21, 23, 173, 19, 16, 23, 22, 22, 24, 16, 17, 36, 19, 36, 26, 56, 23, 24, 34, 21, 13, 21, 1000, 14, 10, 22, 25, 26, 30, 24, 358, 20, 9, 27, 14, 286, 10, 28, 15, 117, 92, 10, 81, 22, 8, 21, 18, 19, 25, 67, 8, 14, 280, 19, 28, 13, 22, 58, 16, 25, 20, 22, 25, 14, 23, 18, 9, 21, 16, 18, 24, 23, 12, 27, 13, 17, 16, 27, 17, 24, 21, 27, 14, 21, 10, 12, 10, 8, 21, 7, 15, 14, 15, 13, 25, 24, 28, 23, 25, 12, 21, 27, 18, 134, 49, 24, 15, 17, 102, 22, 26, 25, 14, 26, 9, 14, 26, 53, 20, 63, 8, 47, 21, 14, 49, 32, 18, 8, 21, 19, 26, 26, 267, 24, 17, 20, 23, 16, 12, 29, 23, 14, 9, 12, 19, 26, 19, 44, 19, 28, 16, 11, 25, 20, 10, 22, 23, 27, 11, 12, 33, 8, 21, 20, 27, 20, 19, 9, 21, 26, 18, 10, 13, 11, 20, 26, 17, 36, 18, 18, 30, 16, 21, 24, 15, 23, 16, 54, 23, 18, 15, 58, 20, 14, 23, 22, 18, 31, 16, 14, 25, 18, 25, 18, 15, 19, 8, 15, 16, 24, 25, 31, 25, 27, 23, 8, 8, 15, 13, 28, 11, 113, 12, 64, 18, 34, 38, 24, 22, 28, 9, 47, 18, 25, 15, 11, 10, 31, 24, 28, 13, 22, 14, 10, 24, 17, 23, 156, 27, 25, 26, 16, 13, 22, 14, 11, 43, 18, 23, 22, 25, 26, 16, 28, 19, 113, 28, 14, 12, 33, 27, 17, 11, 16, 10, 114, 19, 25, 22, 25, 27, 22, 26, 17, 33, 15, 22, 16, 133, 20, 14, 26, 24, 29, 23, 26, 25, 56, 49, 34, 13, 19, 31, 15, 21, 18, 24, 14, 20, 157, 92, 26, 26, 76, 15, 20, 59, 67, 28, 13, 9, 32, 25, 24, 26, 18, 11, 133, 56, 27, 73, 21, 22, 8, 24, 12, 12, 13, 27, 22, 11, 14, 27, 17, 20, 14, 18, 20, 10, 13, 33, 11, 35, 37, 8, 18, 29, 14, 12, 18, 26, 9, 14, 12, 24, 274, 21, 18, 19, 35, 26, 27, 14, 14, 16, 15, 24, 65, 23, 13, 23, 27, 28, 19, 8, 17, 29, 27, 22, 25, 31, 26, 24, 11, 27, 11, 12, 18, 23, 25, 17, 15, 18, 10, 28, 11, 19, 27, 38, 23, 128, 10, 31, 17, 25, 13, 17, 37, 13, 25, 25, 25, 20, 11, 23, 26, 46, 14, 9, 12, 26, 19, 21, 15, 26, 17, 8, 28, 29, 31, 26, 13, 20, 27, 15, 39, 16, 20, 19, 19, 12, 13, 23, 17, 14, 15, 31, 35, 39, 24, 28, 19, 21, 9, 18, 64, 27, 28, 19, 23, 25, 11, 14, 37, 13, 14, 21, 33, 21, 24, 15, 13, 37, 42, 17, 38, 9, 20, 17, 22, 13, 22, 21, 16, 16, 24, 13, 24, 25, 19, 16, 28, 35, 29, 15, 18, 9, 22, 28, 11, 19, 18, 24, 32, 12, 32, 30, 12, 27, 28, 18, 11, 14, 11, 16, 26, 16, 30, 22, 27, 23, 57, 35, 27, 16, 19, 14, 25, 80, 19, 25, 20, 25, 24, 25, 62, 28, 29, 37, 26, 18, 18, 11, 34, 15, 33, 21, 19, 13, 27, 10, 24, 22, 19, 30, 119, 13, 26, 23, 18, 33, 215, 27, 15, 14, 21, 49, 14, 11, 24, 29, 33, 20, 19, 442, 41, 18, 70, 93, 29, 29, 13, 63, 26, 96, 41, 12, 22, 20, 15, 20, 28, 783, 15, 139, 70, 27, 36, 146, 30, 33, 27, 25, 28, 20, 94, 261, 21, 86, 9, 127, 22, 29, 20, 218, 18, 26, 71, 18, 19, 24, 18, 13, 17, 14, 18, 16, 29, 73, 14, 32, 39, 76, 21, 12, 24, 23, 26, 11, 25, 18, 11, 198, 116, 23, 20, 24, 24, 15, 18, 27, 21, 14, 19, 19, 17, 8, 28, 53, 172, 36, 35, 13, 57, 23, 10, 16, 22, 81, 41, 19, 151, 75, 10, 20, 15, 472, 18, 15, 24, 148, 16, 25, 18, 18, 15, 19, 86, 17, 109, 19, 6, 8, 12, 13, 17, 30, 149, 11, 24, 25, 17, 21, 13, 26, 20, 25, 10, 29, 22, 25, 30, 131, 20, 14, 11, 14, 13, 20, 24, 20, 19, 22, 15, 25, 11, 10, 37, 19, 11, 28, 26, 45, 18, 15, 27, 28, 27, 18, 716, 26, 25, 22, 8, 22, 14, 21, 13, 26, 14, 27, 28, 17, 9, 24, 24, 9, 22, 22, 12, 21, 27, 23, 13, 10, 14, 12, 39, 23, 34, 18, 16, 16, 36, 19, 23, 21, 39, 18, 19, 15, 16, 21, 20, 9, 21, 15, 33, 17, 19, 13, 24, 19, 17, 14, 19, 19, 18, 19, 21, 20, 9, 13, 9, 31, 23, 8, 19, 67, 17, 29, 20, 28, 16, 8, 17, 31, 24, 20, 23, 40, 22, 9, 22, 29, 41, 22, 25, 28, 17, 26, 75, 18, 17, 28, 157, 30, 9, 21, 18, 24, 21, 17, 15, 26, 18, 14, 15, 11, 15, 20, 25, 9, 15, 18, 21, 34, 28, 34, 15, 16, 23, 19, 20, 17, 24, 8, 17, 54, 20, 17, 21, 9, 22, 14, 20, 20, 20, 10, 19, 19, 11, 20, 15, 17, 13, 13, 23, 22, 19, 26, 23, 14, 27, 18, 27, 18, 28, 13, 20, 14, 18, 11, 18, 12, 33, 24, 15, 19, 27, 27, 13, 22, 10, 18, 12, 22, 32, 10, 15, 234, 32, 14, 19, 113, 41, 82, 922, 52, 9, 49, 169, 75, 32, 20, 29, 26, 14, 15, 40, 22, 270, 127, 36, 34, 18, 22, 23, 26, 10, 14, 20, 26, 241, 17, 15, 19, 12, 10, 22, 150, 43, 50, 22, 54, 317, 13, 15, 12, 488, 27, 1000, 28, 42, 14, 11, 27, 28, 94, 12, 8, 14, 25, 15, 20, 14, 28, 15, 187, 46, 15, 22, 281, 42, 25, 16, 16, 9, 9, 23, 14, 25, 173, 24, 22, 28, 14, 25, 16, 18, 28, 18, 20, 12, 20, 29, 12, 1000, 8, 12, 12, 12, 15, 9, 25, 14, 26, 14, 61, 96, 30, 13, 24, 257, 327, 441, 16, 174, 25, 62, 51, 174, 30, 24, 11, 12, 100, 20, 22, 8, 15, 17, 21, 18, 31, 13, 447, 39, 9, 17, 23, 114, 18, 26, 11, 39, 189, 191, 83, 632, 49, 22, 216, 258, 45, 20, 16, 24, 28, 25, 10, 94, 10, 29, 16, 26, 256, 17, 22, 151, 1000, 615, 75, 90, 553, 26, 17, 18, 28, 23, 16, 20, 21, 14, 492, 65, 18, 14, 12, 18, 19, 25, 17, 51, 17, 6, 9, 122, 8, 846, 68, 12, 75, 36, 23, 506, 13, 6, 254, 12, 198, 25, 283, 15, 10, 22, 25, 24, 25, 27, 18, 106, 88, 27, 13, 69, 20, 26, 20, 25, 9, 19, 27, 30, 406, 110, 15, 15, 15, 23, 16, 11, 14, 13, 40, 31, 41, 17, 188, 50, 27, 220, 29, 19, 56, 45, 1000, 23, 19, 24, 18, 23, 245, 187, 16, 30, 63, 28, 42, 20, 18, 18, 29, 22, 25, 24, 20, 15, 26, 28, 19, 19, 19, 13, 28, 20, 25, 28, 19, 16, 32, 25, 13, 12, 39, 22, 21, 9, 24, 24, 12, 27, 18, 28, 30, 20, 30, 34, 19, 24, 10, 12, 24, 15, 13, 13, 24, 21, 249, 13, 189, 28, 21, 91, 16, 108, 85, 26, 11, 32, 14, 23, 14, 17, 15, 18, 31, 19, 59, 41, 17, 13, 10, 21, 12, 25, 103, 21, 80, 16, 20, 14, 295, 22, 40, 66, 21, 23, 32, 32, 21, 10, 19, 22, 25, 21, 23, 20, 21, 16, 8, 25, 23, 16, 21, 21, 25, 23, 9, 27, 21, 13, 19, 27, 19, 10, 8, 15, 18, 25, 17, 65, 16, 14, 28, 18, 16, 28, 37, 15, 22, 20, 18, 14, 28, 18, 10, 20, 18, 23, 8, 22, 24, 20, 23, 16, 21, 21, 16, 16, 11, 13, 24, 10, 13, 16, 17, 19, 19, 17, 18, 11, 22, 28, 27, 160, 33, 18, 18, 57, 8, 47, 25, 20, 20, 16, 9, 28, 28, 25, 14, 14, 19, 26, 24, 11, 23, 11, 60, 15, 27, 535, 37, 19, 27, 29, 27, 11, 150, 20, 23, 33, 27, 15, 360, 34, 30, 234, 38, 26, 25, 25, 12, 15, 23, 28, 157, 11, 11, 115, 16, 134, 41, 305, 49, 18, 28, 60, 18, 232, 38, 116, 247, 21, 29, 18, 18, 20, 30, 117, 19, 36, 237, 15, 15, 14, 43, 20, 13, 13, 92, 9, 11, 18, 18, 196, 22, 44, 25, 109, 17, 13, 34, 280, 262, 12, 256, 20, 28, 17, 124, 33, 21, 23, 12, 9, 10, 15, 15, 22, 24, 17, 20, 9, 19, 15, 37, 31, 46, 15, 47, 24, 10, 21, 9, 35, 16, 28, 25, 19, 28, 7, 20, 12, 12, 7, 18, 14, 11, 9, 19, 29, 20, 32, 10, 79, 12, 30, 75, 14, 18, 20, 17, 16, 12, 21, 22, 16, 34, 125, 11, 17, 31, 22, 20, 116, 311, 25, 30, 32, 15, 73, 44, 19, 135, 22, 61, 24, 31, 1000, 41, 15, 74, 9, 99, 40, 121, 24, 14, 107, 11, 19, 47, 155, 51, 12, 11, 81, 20, 220, 760, 90, 279, 105, 325, 40, 147, 31, 16, 13, 42, 7, 14, 22, 36, 279, 11, 17, 14, 13, 20, 12, 11, 21, 14, 24, 1000, 18, 34, 141, 58, 14, 38, 391, 983, 19, 24, 41, 19, 25, 38, 573, 139, 28, 62, 51, 18, 14, 73, 279, 17, 18, 224, 166, 11, 7, 17, 17, 20, 29, 30, 29, 10, 23, 30, 26, 19, 19, 16, 23, 27, 29, 8, 21, 25, 15, 28, 17, 14, 42, 129, 11, 9, 15, 23, 27, 19, 16, 22, 67, 22, 19, 19, 54, 21, 22, 28, 29, 40, 17, 18, 14, 12, 20, 24, 26, 14, 21, 14, 11, 24, 22, 42, 25, 26, 32, 26, 15, 27, 19, 24, 20, 16, 11, 10, 8, 12, 23, 15, 12, 9, 22, 24, 25, 15, 20, 17, 21, 11, 28, 119, 16, 30, 24, 33, 28, 19, 71, 13, 13, 210, 33, 16, 28, 28, 24, 13, 28, 20, 17, 19, 18, 25, 27, 21, 18, 26, 18, 17, 18, 40, 26, 27, 26, 28, 22, 33, 26, 31, 14, 24, 8, 243, 27, 23, 26, 20, 17, 28, 28, 10, 63, 20, 19, 28, 26, 16, 20, 17, 24, 14, 17, 16, 28, 25, 14, 21, 23, 25, 25, 20, 16, 25, 28, 111, 24, 20, 24, 12, 24, 18, 14, 56, 24, 24, 25, 23, 17, 20, 23, 14, 28, 19, 23, 21, 15, 17, 15, 15, 36, 23, 26, 21, 24, 11, 26, 16, 25, 13, 26, 16, 20, 27, 19, 28, 26, 21, 18, 17, 17, 17, 15, 11, 10, 35, 17, 74, 13, 26, 131, 27, 22, 43, 24, 18, 16, 22, 28, 18, 22, 44, 14, 18, 18, 13, 15, 27, 20, 19, 21, 15, 15, 27, 11, 19, 20, 24, 19, 25, 12, 24, 17, 22, 21, 16, 25, 17, 25, 24, 16, 10, 22, 28, 15, 26, 11, 19, 27, 19, 24, 16, 26, 28, 19, 25, 14, 18, 26, 13, 14, 10, 23, 14, 16, 24, 13, 20, 28, 17, 25, 12, 26, 27, 20, 22, 10, 20, 23, 13, 23, 18, 10, 27, 11, 15, 14, 27, 22, 18, 19, 25, 21, 23, 25, 13, 17, 18, 10, 15, 24, 18, 25, 13, 14, 21, 18, 14, 28, 25, 18, 21, 27, 20, 18, 20, 23, 12, 27, 15, 25, 24, 14, 12, 11, 21, 27, 16, 28, 28, 26, 28, 22, 20, 17, 24, 26, 28, 21, 22, 23, 16, 12, 19, 16, 12, 24, 11, 28, 24, 14, 28, 20, 19, 22, 20, 23, 22, 15, 21, 13, 16, 13, 13, 20, 14, 27, 12, 20, 23, 28, 20, 23, 28, 23, 20, 25, 26, 22, 13, 26, 18, 25, 20, 20, 14, 28, 20, 23, 17, 13, 14, 22, 25, 16, 19, 24, 13, 28, 16, 20, 21, 20, 13, 18, 21, 22, 30, 25, 18, 17, 15, 47, 13, 27, 19, 15, 62, 12, 34, 37, 25, 27, 13, 27, 26, 19, 20, 11, 23, 16, 25, 20, 18, 19, 24, 23, 12, 10, 13, 25, 13, 19, 28, 25, 23, 26, 25, 25, 27, 23, 28, 14, 15, 19, 16, 18, 15, 13, 19, 22, 15, 20, 20, 25, 19, 19, 18, 22, 13, 11, 23, 25, 27, 14, 27, 13, 16, 26, 19, 27, 17, 24, 12, 42, 11, 16, 13, 19, 24, 16, 23, 19, 25, 14, 13, 605, 238, 25, 29, 167, 173, 134, 82, 20, 145, 11, 15, 21, 22, 10, 24, 13, 622, 18, 26, 19, 6, 7, 24, 29, 28, 18, 20, 381, 20, 25, 82, 35, 285, 32, 22, 15, 21, 7, 22, 157, 28, 32, 85, 22, 178, 28, 24, 128, 87, 443, 25, 12, 27, 55, 25, 13, 14, 24, 24, 15, 24, 40, 41, 13, 25, 22, 20, 21, 27, 19, 19, 22, 15, 24, 28, 26, 184, 835, 585, 26, 28, 149, 264, 12, 185, 264, 21, 19, 14, 14, 40, 43, 11, 19, 17, 78, 12, 20, 10, 27, 25, 150, 14, 120, 43, 23, 51, 18, 28, 28, 20, 12, 82, 425, 24, 26, 14, 18, 168, 51, 21, 15, 26, 23, 26, 41, 312, 14, 28, 17, 17, 8, 460, 17, 72, 14, 162, 251, 12, 21, 19, 15, 25, 19, 14, 763, 107, 19, 19, 110, 26, 12, 23, 366, 28, 1000, 83, 776, 107, 98, 44, 16, 205, 56, 13, 12, 21, 20, 28, 31, 11, 5, 22, 9, 250, 49, 23, 23, 16, 16, 15, 23, 11, 966, 15, 15, 94, 25, 21, 14, 17, 61, 147, 11, 15, 94, 12, 16, 13, 14, 21, 49, 120, 12, 21, 15, 19, 1000, 671, 123, 20, 328, 11, 354, 28, 19, 31, 16, 1000, 27, 105, 43, 24, 10, 140, 180, 27, 6, 9, 25, 7, 26, 268, 186, 19, 485, 29, 78, 158, 22, 182, 134, 1000, 52, 659, 382, 42, 197, 35, 27, 72, 26, 90, 72, 45, 29, 28, 433, 268, 22, 71, 415, 4, 27, 6, 18, 4, 30, 266, 22, 24, 437, 280, 585, 16, 26, 69, 12, 2, 6, 5, 11, 15, 24, 110, 16, 22, 17, 25, 25, 15, 26, 13, 17, 25, 23, 33, 35, 38, 27, 21, 21, 95, 28, 19, 16, 30, 25, 16, 33, 47, 36, 17, 15, 93, 401, 876, 31, 18, 14, 27, 24, 16, 13, 20, 212, 7, 20, 11, 15, 17, 25, 21, 28, 28, 23, 34, 90, 129, 18, 27, 19, 1000, 585, 37, 16, 14, 92, 26, 11, 26, 172, 20, 24, 18, 28, 17, 13, 11, 16, 24, 62, 130, 7, 28, 253, 17, 1000, 30, 27, 21, 52, 353, 22, 34, 14, 10, 15, 46, 248, 22, 15, 129, 95, 103, 26, 14, 27, 62, 10, 15, 20, 17, 35, 179, 188, 119, 17, 17, 290, 22, 17, 350, 24, 255, 132, 15, 928, 149, 55, 124, 31, 328, 34, 157, 39, 673, 18, 21, 500, 54, 25, 23, 35, 192, 259, 116, 33, 33, 13, 78, 134, 255, 82, 43, 43, 623, 470, 13, 41, 643, 313, 162, 17, 359, 207, 131, 62, 17, 475, 26, 64, 24, 67, 130, 51, 356, 280, 26, 287, 80, 97, 17, 138, 58, 442, 679, 158, 16, 25, 15, 17, 27, 19, 27, 91, 10, 25, 13, 28, 227, 18, 11, 23, 15, 1000, 28, 130, 412, 83, 59, 22, 26, 41, 869, 46, 15, 16, 137, 34, 25, 14, 25, 12, 59, 673, 25, 9, 24, 12, 35, 26, 13, 16, 547, 32, 26, 1000, 49, 38, 76, 10, 12, 22, 15, 11, 18, 622, 13, 73, 23, 60, 24, 26, 17, 14, 12, 27, 15, 16, 23, 26, 29, 18, 22, 43, 20, 18, 14, 15, 88, 17, 209, 12, 400, 15, 62, 14, 26, 297, 27, 184, 17, 322, 23, 930, 233, 25, 10, 23, 27, 21, 19, 85, 7, 26, 376, 21, 78, 54, 72, 18, 29, 73, 18, 21, 57, 19, 15, 1000, 54, 13, 139, 23, 19, 454, 35, 88, 86, 28, 349, 40, 27, 28, 10, 15, 15, 1000, 27, 221, 28, 121, 399, 30, 29, 17, 6, 81, 88, 23, 20, 46, 22, 28, 22, 15, 26, 40, 15, 17, 24, 18, 37, 477, 17, 48, 178, 8, 25, 20, 857, 42, 21, 14, 27, 47, 21, 13, 1000, 79, 10, 76, 235, 236, 324, 15, 257, 247, 218, 127, 290, 93, 267, 18, 15, 55, 21, 12, 13, 24, 16, 582, 19, 40, 52, 910, 677, 119, 101, 11, 14, 38, 9, 66, 16, 768, 391, 15, 21, 18, 85, 30, 28, 18, 26, 22, 99, 15, 18, 75, 61, 118, 29, 165, 55, 13, 21, 20, 14, 84, 10, 23, 26, 15, 13, 20, 12, 14, 132, 23, 20, 24, 26, 13, 22, 46, 17, 14, 14, 17, 20, 62, 24, 117, 99, 24, 36, 32, 12, 24, 13, 235, 40, 1000, 164, 16, 13, 7, 67, 38, 31, 26, 19, 261, 761, 1000, 26, 8, 23, 24, 15, 16, 31, 30, 800, 11, 12, 196, 9, 138, 24, 16, 14, 31, 14, 160, 30, 19, 471, 108, 20, 1000, 34, 15, 72, 90, 46, 372, 32, 26, 131, 19, 254, 326, 78, 23, 19, 30, 18, 11, 75, 137, 157, 562, 9, 19, 37, 23, 14, 106, 989, 383, 23, 71, 232, 141, 51, 25, 7, 19, 15, 66, 14, 21, 388, 19, 19, 10, 10, 26, 25, 21, 25, 21, 9, 13, 115, 26, 18, 20, 21, 9, 204, 476, 24, 19, 30, 85, 22, 18, 1000, 11, 17, 373, 38, 136, 197, 18, 23, 24, 14, 25, 8, 20, 143, 21, 15, 26, 12, 24, 212, 24, 43, 27, 25, 22, 22, 23, 198, 28, 26, 24, 22, 21, 21, 50, 12, 23, 119, 30, 23, 16, 35, 12, 20, 18, 21, 13, 22, 16, 19, 13, 21, 13, 16, 27, 26, 63, 21, 67, 385, 152, 23, 12, 133, 14, 144, 13, 50, 733, 339, 105, 13, 18, 9, 242, 15, 27, 20, 29, 395, 17, 15, 30, 602, 26, 263, 22, 366, 15, 50, 151, 16, 25, 42, 53, 64, 128, 114, 64, 53, 1000, 43, 44, 60, 17, 26, 124, 82, 36, 184, 120, 496, 126, 10, 233, 42, 252, 149, 14, 63, 19, 12, 44, 133, 32, 17, 17, 18, 297, 1000, 7, 117, 243, 1000, 20, 506, 215, 18, 572, 242, 26, 21, 13, 12, 34, 161, 539, 86, 31, 78, 599, 122, 115, 639, 369, 46, 1000, 108, 26, 1000, 17, 23, 27, 104, 27, 12, 6, 24, 7, 13, 32, 9, 21, 27, 14, 20, 13, 26, 9, 22, 13, 19, 18, 471, 9, 26, 20, 60, 141, 38, 20, 12, 23, 10, 17, 13, 14, 638, 160, 80, 26, 10, 79, 25, 39, 228, 654, 78, 71, 23, 27, 25, 12, 118, 13, 20, 28, 22, 15, 21, 15, 22, 23, 1000, 32, 30, 14, 13, 32, 19, 28, 23, 27, 24, 48, 28, 17, 16, 19, 22, 37, 14, 35, 27, 18, 27, 19, 189, 22, 17, 27, 15, 15, 35, 67, 16, 16, 29, 35, 23, 38, 146, 23, 13, 10, 72, 109, 159, 530, 16, 13, 377, 284, 37, 20, 30, 366, 92, 50, 90, 12, 31, 141, 500, 695, 23, 14, 25, 18, 13, 124, 15, 10, 21, 28, 409, 1000, 83, 64, 12, 97, 26, 23, 47, 91, 59, 15, 13, 64, 17, 17, 12, 11, 19, 24, 9, 31, 181, 118, 28, 30, 11, 15, 169, 378, 683, 913, 690, 343, 12, 23, 24, 371, 133, 391, 332, 208, 28, 11, 16, 28, 18, 13, 140, 94, 62, 119, 25, 66, 67, 16, 1000, 306, 12, 1000, 13, 10, 15, 13, 30, 99, 21, 499, 44, 961, 441, 144, 115, 187, 1000, 344, 15, 25, 417, 21, 935, 22, 17, 10, 59, 36, 26, 754, 1000, 22, 14, 523, 19, 10, 245, 311, 101, 15, 26, 8, 7, 15, 18, 25, 20, 22, 32, 47, 24, 18, 21, 15, 27, 756, 17, 13, 13, 21, 109, 15, 21, 14, 28, 28, 52, 16, 415, 19, 327, 760, 95, 167, 495, 25, 13, 17, 10, 26, 52, 15, 13, 180, 1000, 203, 357, 25, 89, 346, 11, 19, 83, 244, 218, 122, 53, 21, 365, 793, 22, 27, 61, 15, 56, 17, 26, 34, 24, 22, 25, 27, 43, 7, 28, 17, 19, 39, 16, 23, 10, 14, 25, 23, 17, 35, 29, 13, 26, 15, 21, 12, 13, 45, 19, 18, 157, 25, 22, 22, 21, 15, 14, 26, 21, 52, 32, 195, 26, 20, 27, 18, 21, 19, 31, 18, 190, 24, 26, 14, 107, 411, 64, 28, 95, 17, 27, 28, 10, 18, 18, 234, 96, 263, 225, 9, 88, 18, 13, 13, 1000, 331, 26, 217, 115, 133, 9, 225, 34, 59, 50, 31, 24, 38, 30, 15, 23, 25, 24, 21, 9, 24, 14, 15, 19, 18, 27, 28, 24, 22, 26, 14, 16, 10, 24, 27, 16, 26, 24, 13, 25, 27, 23, 22, 167, 36, 8, 10, 93, 21, 14, 26, 37, 12, 18, 20, 11, 17, 18, 19, 17, 14, 43, 13, 10, 21, 22, 104, 19, 12, 21, 13, 37, 11, 10, 16, 19, 235, 153, 5, 9, 20, 950, 1000, 21, 36, 45, 30, 160, 86, 21, 9, 900, 26, 20, 22, 233, 8, 25, 23, 483, 25, 55, 16, 23, 18, 472, 42, 23, 39, 22, 18, 13, 20, 20, 25, 14, 31, 20, 274, 14, 14, 139, 181, 24, 20, 12, 14, 21, 11, 32, 20, 24, 24, 77, 20, 98, 20, 15, 38, 341, 11, 23, 86, 407, 31, 122, 18, 1000, 359, 39, 15, 18, 21, 20, 11, 16, 235, 14, 11, 19, 32, 20, 86, 35, 27, 18, 17, 22, 28, 80, 16, 21, 19, 27, 23, 17, 294, 20, 12, 380, 26, 24, 14, 15, 25, 12, 63, 20, 20, 125, 28, 21, 28, 16, 16, 29, 254, 64, 8, 75, 16, 223, 70, 26, 512, 16, 138, 24, 61, 8, 14, 19, 28, 28, 20, 10, 35, 21, 17, 22, 16, 19, 26, 84, 444, 11, 22, 133, 27, 15, 25, 20, 194, 78, 27, 23, 11, 82, 150, 21, 12, 149, 17, 95, 20, 11, 120, 206, 530, 206, 17, 15, 13, 82, 1000, 13, 11, 154, 1000, 81, 19, 13, 38, 23, 20, 21, 499, 11, 19, 68, 420, 15, 24, 12, 21, 35, 27, 490, 15, 61, 27, 14, 203, 10, 13, 12, 16, 19, 21, 21, 10, 39, 12, 537, 10, 14, 22, 7, 24, 25, 24, 18, 26, 17, 47, 25, 12, 17, 137, 255, 26, 53, 14, 13, 23, 20, 23, 26, 25, 231, 52, 14, 6, 19, 97, 34, 17, 60, 12, 16, 24, 28, 157, 8, 13, 51, 17, 20, 33, 6, 24, 15, 25, 13, 15, 25, 13, 51, 10, 216, 104, 162, 244, 24, 169, 113, 135, 284, 11, 13, 675, 23, 126, 47, 16, 24, 170, 43, 19, 59, 35, 131, 24, 182, 244, 698, 26, 22, 28, 23, 19, 10, 11, 73, 20, 105, 961, 8, 15, 10, 24, 379, 17, 18, 65, 49, 21, 20, 27, 24, 43, 11, 337, 9, 15, 12, 16, 17, 14, 21, 41, 142, 58, 245, 74, 13, 13, 154, 7, 25, 14, 25, 17, 33, 23, 50, 22, 101, 111, 14, 19, 15, 104, 272, 62, 12, 46, 12, 22, 25, 75, 35, 26, 106, 17, 19, 9, 26, 20, 1000, 393, 36, 64, 14, 29, 31, 63, 31, 34, 17, 16, 11, 118, 876, 24, 37, 24, 22, 22, 8, 8, 208, 14, 42, 30, 117, 192, 25, 11, 22, 15, 26, 9, 9, 20, 18, 32, 186, 47, 17, 25, 24, 57, 19, 229, 17, 25, 14, 12, 18, 15, 15, 15, 302, 368, 16, 12, 21, 649, 24, 33, 96, 43, 7, 352, 7, 28, 7, 26, 39, 66, 24, 63, 22, 12, 25, 12, 24, 117, 46, 662, 28, 84, 575, 27, 29, 10, 13, 9, 69, 540, 158, 1000, 12, 24, 26, 20, 28, 12, 24, 17, 41, 15, 29, 98, 15, 47, 9, 11, 32, 152, 28, 11, 44, 91, 150, 145, 24, 55, 14, 21, 22, 16, 22, 22, 25, 23, 20, 19, 18, 9, 20, 25, 17, 63, 20, 45, 12, 16, 24, 13, 9, 17, 33, 569, 38, 26, 12, 29, 21, 24, 173, 41, 151, 26, 23, 20, 21, 24, 12, 17, 30, 147, 15, 25, 32, 20, 26, 23, 15, 21, 24, 9, 22, 28, 24, 13, 26, 16, 141, 17, 12, 20, 14, 18, 20, 24, 15, 13, 37, 28, 27, 13, 27, 23, 28, 14, 14, 23, 20, 61, 11, 21, 18, 12, 20, 24, 20, 21, 28, 28, 15, 22, 14, 18, 15, 51, 27, 21, 20, 20, 19, 86, 28, 18, 19, 28, 27, 20, 22, 17, 19, 21, 22, 28, 27, 17, 21, 19, 19, 21, 27, 14, 22, 24, 18, 13, 13, 11, 22, 28, 27, 9, 16, 20, 18, 18, 20, 44, 45, 19, 13, 37, 27, 28, 28, 18, 10, 23, 14, 10, 24, 11, 107, 20, 14, 24, 18, 24, 28, 27, 20, 21, 22, 15, 16, 18, 23, 11, 18, 28, 21, 16, 9, 17, 19, 16, 14, 16, 15, 12, 17, 23, 18, 70, 21, 28, 11, 7, 15, 19, 25, 26, 28, 26, 27, 28, 23, 13, 11, 7, 11, 7, 9, 13, 8, 28, 7, 56, 160, 131, 7, 15, 22, 17, 9, 105, 163, 11, 26, 25, 28, 7, 28, 14, 23, 27, 307, 17, 23, 13, 26, 55, 11, 9, 21, 22, 23, 14, 14, 27, 13, 16, 21, 25, 189, 266, 9, 10, 7, 14, 15, 16, 11, 14, 15, 23, 24, 25, 36, 97, 26, 9, 27, 15, 16, 19, 18, 19, 24, 9, 13, 17, 20, 63, 706, 21, 35, 116, 18, 11, 26, 16, 10, 26, 120, 22, 13, 20, 22, 26, 21, 27, 24, 17, 10, 21, 27, 93, 28, 24, 13, 32, 22, 21, 15, 19, 19, 20, 77, 30, 23, 71, 66, 14, 28, 22, 16, 22, 25, 10, 32, 15, 12, 41, 9, 22, 28, 10, 28, 12, 47, 28, 25, 26, 19, 16, 27, 18, 57, 30, 37, 25, 9, 21, 11, 27, 16, 20, 14, 23, 14, 28, 24, 18, 17, 25, 17, 9, 53, 51, 16, 28, 12, 17, 28, 27, 11, 20, 23, 27, 23, 9, 14, 30, 9, 10, 22, 13, 23, 11, 12, 14, 10, 19, 34, 26, 9, 8, 127, 28, 27, 25, 18, 13, 25, 14, 9, 77, 16, 22, 17, 15, 23, 26, 20, 19, 21, 13, 15, 20, 17, 63, 9, 10, 28, 24, 37, 13, 9, 13, 21, 22, 10, 9, 13, 12, 20, 22, 17, 9, 20, 18, 20, 17, 28, 14, 23, 33, 25, 12, 21, 13, 13, 7, 22, 25, 9, 15, 119, 28, 20, 24, 11, 24, 15, 10, 14, 14, 12, 24, 9, 11, 12, 333, 13, 20, 13, 18, 18, 9, 14, 10, 27, 24, 16, 16, 19, 28, 27, 20, 16, 15, 9, 17, 10, 9, 18, 9, 24, 16, 25, 16, 32, 13, 10, 18, 24, 22, 14, 27, 7, 21, 19, 26, 17, 10, 18, 13, 19, 24, 10, 17, 12, 23, 18, 9, 18, 45, 16, 10, 21, 14, 14, 8, 20, 61, 10, 13, 19, 22, 10, 25, 16, 20, 8, 12, 23, 19, 14, 19, 10, 17, 7, 19, 13, 25, 26, 21, 8, 27, 13, 20, 22, 15, 23, 14, 25, 19, 22, 18, 16, 21, 10, 28, 14, 24, 26, 17, 28, 13, 17, 19, 11, 9, 28, 9, 17, 20, 16, 15, 12, 20, 22, 17, 12, 18, 15, 27, 12, 19, 24, 15, 19, 15, 19, 22, 21, 26, 14, 19, 15, 22, 14, 21, 10, 37, 25, 56, 9, 13, 16, 10, 11, 20, 20, 18, 23, 22, 24, 12, 22, 17, 10, 15, 14, 11, 79, 25, 20, 12, 18, 101, 20, 12, 23, 26, 227, 18, 24, 12, 13, 10, 22, 13, 12, 10, 26, 20, 16, 13, 30, 12, 11, 17, 17, 28, 17, 26, 15, 25, 12, 42, 22, 16, 28, 23, 21, 21, 26, 23, 12, 26, 11, 25, 11, 24, 15, 19, 22, 25, 24, 14, 70, 11, 54, 25, 36, 14, 26, 19, 30, 36, 8, 25, 56, 78, 29, 16, 11, 11, 30, 12, 14, 21, 53, 18, 38, 602, 58, 22, 20, 183, 18, 17, 44, 18, 13, 28, 15, 16, 21, 21, 18, 12, 23, 210, 49, 24, 31, 19, 16, 20, 28, 21, 19, 19, 45, 51, 14, 18, 18, 27, 16, 105, 23, 15, 9, 10, 59, 8, 23, 20, 12, 21, 11, 21, 12, 16, 14, 21, 15, 21, 14, 11, 27, 17, 20, 101, 103, 15, 11, 28, 10, 215, 15, 17, 17, 19, 8, 235, 12, 23, 16, 25, 22, 187, 29, 326, 70, 14, 25, 17, 27, 15, 16, 26, 36, 15, 22, 11, 23, 25, 29, 123, 94, 26, 16, 23, 16, 30, 24, 26, 24, 25, 27, 13, 58, 15, 16, 24, 52, 15, 28, 23, 18, 27, 59, 20, 28, 25, 123, 13, 78, 11, 21, 28, 20, 203, 15, 15, 11, 23, 16, 18, 13, 13, 15, 37, 20, 27, 14, 27, 11, 67, 23, 17, 24, 23, 23, 11, 16, 223, 20, 28, 32, 15, 18, 18, 12, 24, 17, 32, 20, 114, 19, 26, 43, 87, 18, 21, 26, 22, 14, 20, 22, 22, 24, 28, 14, 17, 13, 21, 16, 24, 24, 16, 19, 27, 16, 21, 25, 13, 17, 20, 23, 20, 27, 11, 41, 12, 23, 14, 28, 14, 10, 16, 15, 22, 17, 28, 20, 10, 9, 17, 15, 9, 27, 15, 42, 18, 65, 16, 18, 84, 45, 17, 31, 24, 17, 75, 128, 17, 30, 26, 22, 37, 40, 86, 25, 22, 11, 19, 17, 11, 30, 55, 28, 14, 40, 16, 13, 127, 45, 290, 17, 25, 73, 16, 17, 15, 24, 14, 20, 23, 29, 20, 121, 21, 17, 10, 57, 25, 15, 17, 18, 31, 270, 18, 168, 19, 96, 84, 48, 81, 39, 11, 70, 10, 24, 20, 19, 29, 14, 21, 28, 15, 27, 23, 19, 127, 18, 120, 20, 17, 23, 17, 16, 15, 25, 20, 14, 26, 18, 28, 18, 25, 24, 18, 26, 17, 31, 17, 24, 27, 21, 27, 21, 13, 27, 23, 15, 28, 13, 10, 11, 20, 17, 11, 36, 18, 18, 24, 15, 14, 15, 20, 20, 14, 22, 22, 24, 13, 23, 19, 28, 17, 23, 27, 55, 44, 18, 21, 18, 80, 22, 14, 25, 26, 47, 16, 12, 49, 11, 32, 12, 23, 20, 10, 19, 26, 71, 21, 15, 31, 71, 12, 21, 15, 16, 12, 22, 15, 24, 22, 20, 18, 19, 17, 23, 12, 22, 10, 12, 25, 20, 35, 20, 28, 17, 15, 17, 21, 17, 17, 27, 16, 17, 76, 26, 171, 16, 28, 51, 23, 17, 16, 25, 62, 54, 18, 9, 19, 21, 12, 20, 26, 9, 17, 232, 29, 25, 14, 23, 14, 18, 21, 16, 24, 45, 15, 16, 16, 8, 26, 10, 20, 19, 11, 21, 21, 138, 13, 22, 488, 19, 20, 23, 47, 13, 23, 52, 20, 270, 25, 11, 158, 12, 26, 24, 22, 18, 13, 14, 18, 161, 26, 34, 27, 43, 19, 39, 26, 23, 28, 10, 31, 15, 49, 18, 28, 48, 21, 16, 68, 32, 15, 353, 28, 20, 19, 33, 16, 35, 43, 9, 77, 19, 12, 28, 22, 18, 26, 48, 67, 14, 30, 30, 64, 22, 44, 25, 50, 25, 10, 8, 173, 47, 80, 17, 920, 17, 17, 24, 27, 65, 116, 13, 16, 89, 29, 17, 8, 45, 97, 12, 22, 137, 28, 15, 18, 17, 31, 36, 23, 21, 27, 22, 17, 12, 25, 287, 11, 134, 20, 23, 17, 19, 29, 30, 28, 21, 27, 336, 159, 15, 10, 24, 32, 9, 162, 12, 16, 19, 26, 37, 20, 23, 15, 26, 41, 19, 14, 13, 26, 85, 20, 63, 14, 28, 13, 18, 25, 13, 43, 55, 16, 21, 68, 28, 16, 18, 26, 19, 28, 11, 20, 325, 24, 20, 118, 24, 20, 19, 46, 24, 11, 21, 23, 29, 284, 84, 27, 13, 56, 11, 10, 37, 33, 36, 14, 21, 25, 23, 10, 11, 19, 22, 31, 27, 21, 18, 18, 97, 52, 69, 22, 15, 22, 11, 77, 25, 21, 68, 40, 18, 18, 62, 28, 51, 17, 99, 41, 37, 25, 51, 28, 21, 16, 250, 21, 26, 25, 22, 11, 28, 27, 19, 10, 72, 26, 17, 16, 130, 17, 13, 23, 24, 21, 20, 13, 102, 46, 17, 35, 11, 112, 18, 25, 16, 20, 57, 18, 22, 27, 38, 17, 17, 17, 19, 84, 27, 25, 17, 16, 58, 23, 21, 221, 14, 20, 16, 20, 9, 40, 20, 12, 22, 48, 22, 141, 30, 17, 12, 64, 10, 20, 32, 15, 14, 24, 20, 27, 23, 20, 18, 18, 17, 24, 36, 25, 9, 27, 25, 11, 8, 24, 8, 15, 14, 11, 21, 30, 10, 14, 17, 27, 23, 11, 29, 18, 22, 21, 28, 25, 20, 8, 16, 17, 21, 132, 255, 26, 19, 12, 15, 18, 21, 28, 22, 27, 12, 15, 27, 9, 20, 14, 16, 16, 11, 13, 26, 17, 83, 12, 10, 43, 155, 42, 49, 29, 18, 11, 28, 9, 21, 15, 21, 8, 19, 18, 23, 20, 13, 16, 11, 67, 85, 12, 13, 32, 15, 14, 10, 10, 162, 25, 23, 11, 8, 15, 19, 22, 27, 20, 88, 15, 854, 20, 47, 18, 13, 67, 31, 14, 15, 12, 211, 29, 22, 12, 35, 49, 42, 22, 17, 14, 14, 24, 22, 23, 12, 17, 112, 12, 17, 14, 14, 14, 16, 18, 19, 82, 16, 32, 21, 20, 10, 32, 192, 20, 17, 96, 14, 29, 20, 18, 15, 15, 23, 26, 20, 17, 85, 13, 15, 94, 11, 66, 20, 12, 13, 85, 23, 18, 17, 24, 59, 10, 36, 22, 24, 25, 83, 16, 18, 27, 28, 12, 20, 32, 19, 24, 20, 11, 14, 24, 21, 24, 11, 22, 26, 13, 21, 25, 22, 21, 22, 10, 20, 388, 12, 20, 18, 40, 18, 19, 9, 64, 123, 100, 141, 64, 21, 25, 15, 21, 19, 26, 21, 69, 16, 19, 27, 104, 89, 22, 13, 24, 21, 48, 45, 25, 108, 18, 11, 11, 13, 26, 55, 15, 17, 15, 21, 16, 104, 26, 25, 68, 23, 9, 16, 10, 16, 23, 25, 16, 23, 26, 24, 12, 25, 23, 34, 11, 22, 24, 15, 12, 24, 17, 16, 21, 16, 10, 20, 13, 28, 18, 27, 22, 60, 28, 25, 8, 28, 22, 19, 22, 9, 67, 84, 11, 21, 18, 31, 11, 22, 11, 21, 32, 10, 17, 13, 22, 13, 17, 13, 27, 14, 11, 23, 13, 27, 24, 10, 24, 12, 22, 14, 18, 12, 82, 10, 25, 18, 47, 19, 59, 91, 14, 13, 26, 12, 26, 22, 25, 21, 13, 25, 19, 17, 26, 13, 22, 66, 17, 17, 26, 10, 17, 26, 17, 26, 9, 9, 55, 29, 33, 60, 71, 25, 22, 23, 51, 17, 28, 10, 15, 21, 127, 15, 10, 20, 14, 8, 26, 97, 29, 20, 15, 87, 21, 28, 20, 260, 25, 30, 222, 26, 21, 27, 43, 19, 21, 34, 11, 14, 25, 443, 19, 17, 35, 23, 13, 23, 15, 24, 33, 20, 26, 10, 26, 13, 20, 18, 365, 21, 24, 56, 22, 32, 12, 11, 321, 14, 36, 29, 14, 18, 10, 8, 16, 92, 15, 35, 14, 88, 113, 13, 28, 26, 123, 20, 21, 59, 51, 35, 19, 31, 12, 26, 22, 11, 16, 22, 31, 201, 52, 129, 22, 12, 151, 26, 16, 13, 13, 28, 22, 25, 12, 10, 22, 17, 21, 23, 26, 16, 56, 15, 28, 15, 46, 53, 79, 16, 291, 118, 67, 20, 10, 131, 10, 7, 34, 14, 19, 57, 62, 16, 15, 12, 44, 19, 110, 72, 203, 34, 20, 32, 10, 14, 14, 20, 16, 83, 24, 12, 24, 28, 9, 25, 429, 85, 13, 20, 40, 20, 8, 28, 20, 253, 21, 13, 35, 122, 30, 28, 14, 20, 20, 36, 24, 19, 13, 18, 17, 26, 95, 106, 9, 138, 16, 36, 19, 13, 48, 363, 13, 70, 13, 30, 20, 27, 20, 24, 51, 28, 12, 22, 90, 10, 14, 17, 75, 30, 22, 13, 18, 23, 123, 18, 28, 28, 18, 27, 22, 64, 24, 13, 19, 26, 16, 17, 24, 19, 26, 66, 18, 16, 14, 14, 17, 34, 352, 15, 20, 6, 32, 7, 25, 20, 20, 30, 10, 37, 13, 20, 58, 13, 8, 15, 24, 34, 21, 14, 23, 25, 161, 22, 23, 14, 27, 14, 20, 17, 11, 12, 28, 24, 17, 24, 17, 121, 18, 16, 21, 45, 24, 32, 29, 12, 16, 21, 27, 17, 19, 15, 29, 24, 20, 20, 24, 20, 25, 28, 12, 20, 18, 12, 37, 18, 264, 28, 152, 24, 23, 20, 15, 67, 13, 13, 28, 27, 15, 23, 17, 51, 110, 22, 17, 15, 25, 14, 23, 9, 15, 45, 28, 28, 27, 14, 22, 21, 247, 16, 29, 104, 20, 25, 9, 33, 26, 16, 21, 18, 21, 15, 17, 140, 23, 16, 11, 27, 28, 90, 17, 45, 29, 140, 77, 16, 28, 13, 14, 16, 7, 20, 23, 9, 28, 20, 16, 16, 129, 14, 18, 10, 17, 151, 97, 17, 13, 23, 23, 30, 23, 29, 27, 15, 22, 15, 15, 38, 10, 28, 322, 492, 24, 15, 22, 78, 7, 41, 16, 23, 28, 102, 16, 80, 15, 30, 20, 22, 38, 49, 17, 8, 344, 17, 28, 302, 31, 14, 80, 58, 82, 11, 74, 23, 708, 80, 276, 312, 11, 18, 22, 17, 23, 16, 17, 24, 18, 20, 77, 21, 76, 89, 44, 54, 45, 31, 419, 15, 13, 77, 15, 12, 23, 33, 29, 40, 115, 17, 31, 67, 20, 69, 43, 8, 14, 13, 19, 24, 25, 17, 27, 40, 47, 26, 26, 26, 28, 273, 18, 82, 18, 19, 77, 23, 160, 44, 19, 23, 22, 118, 17, 61, 25, 12, 14, 15, 560, 17, 204, 56, 77, 254, 14, 27, 19, 20, 13, 8, 26, 23, 34, 29, 143, 94, 15, 22, 21, 24, 19, 28, 13, 22, 20, 23, 22, 24, 57, 21, 20, 18, 16, 11, 8, 25, 46, 207, 25, 22, 11, 12, 11, 15, 29, 25, 39, 25, 60, 39, 76, 20, 26, 10, 33, 25, 17, 233, 12, 8, 12, 27, 25, 27, 67, 16, 23, 29, 22, 233, 18, 14, 27, 21, 27, 64, 217, 15, 9, 21, 15, 151, 17, 27, 164, 15, 12, 25, 19, 19, 12, 13, 16, 20, 21, 24, 9, 130, 24, 35, 18, 21, 24, 33, 19, 24, 27, 12, 30, 13, 21, 28, 12, 32, 25, 13, 8, 20, 9, 156, 22, 22, 20, 139, 152, 35, 15, 13, 172, 25, 16, 16, 25, 31, 19, 73, 140, 46, 31, 9, 307, 12, 117, 27, 74, 16, 27, 24, 28, 35, 51, 193, 16, 13, 22, 13, 31, 14, 29, 19, 12, 28, 19, 15, 11, 17, 23, 23, 53, 24, 23, 27, 19, 31, 21, 18, 14, 13, 18, 28, 9, 18, 26, 12, 255, 29, 27, 15, 13, 24, 9, 41, 31, 16, 26, 26, 8, 20, 10, 20, 11, 8, 64, 106, 14, 21, 23, 13, 10, 18, 23, 21, 19, 28, 89, 27, 33, 26, 15, 24, 13, 28, 11, 21, 21, 24, 79, 15, 22, 26, 18, 23, 19, 24, 488, 27, 28, 36, 21, 258, 19, 10, 19, 40, 21, 27, 12, 22, 124, 63, 20, 22, 17, 20, 40, 144, 22, 22, 29, 27, 14, 182, 16, 25, 13, 16, 19, 21, 14, 24, 27, 13, 16, 23, 35, 14, 27, 23, 16, 20, 16, 13, 16, 14, 27, 251, 184, 12, 16, 23, 9, 27, 20, 17, 11, 27, 13, 107, 26, 12, 25, 11, 34, 19, 50, 23, 102, 25, 19, 51, 17, 434, 11, 167, 20, 186, 13, 19, 20, 164, 16, 27, 24, 123, 19, 16, 76, 96, 26, 21, 14, 62, 21, 21, 246, 249, 30, 28, 24, 123, 316, 16, 18, 34, 25, 18, 13, 20, 23, 16, 14, 18, 21, 15, 73, 24, 13, 8, 40, 78, 17, 22, 44, 22, 17, 36, 32, 20, 20, 19, 14, 9, 9, 28, 100, 13, 21, 23, 26, 14, 24, 7, 27, 18, 19, 17, 22, 23, 24, 13, 17, 274, 26, 19, 68, 25, 23, 24, 27, 106, 19, 25, 16, 17, 18, 94, 59, 20, 24, 12, 26, 18, 15, 31, 97, 42, 59, 15, 21, 170, 19, 26, 24, 13, 21, 11, 27, 21, 36, 19, 32, 25, 73, 16, 25, 24, 22, 16, 17, 20, 32, 35, 21, 30, 98, 21, 12, 16, 18, 27, 19, 15, 24, 11, 22, 59, 23, 32, 22, 13, 19, 32, 21, 19, 26, 27, 36, 11, 52, 12, 18, 27, 18, 11, 16, 13, 19, 19, 13, 59, 45, 10, 16, 27, 143, 18, 13, 18, 18, 24, 10, 23, 8, 20, 18, 13, 22, 27, 14, 268, 25, 24, 14, 20, 24, 11, 62, 14, 16, 34, 11, 20, 26, 26, 5, 11, 34, 7, 14, 39, 75, 111, 15, 27, 25, 15, 11, 49, 14, 19, 76, 27, 17, 9, 10, 28, 15, 198, 33, 13, 121, 34, 21, 37, 17, 11, 17, 13, 24, 12, 28, 82, 185, 21, 117, 49, 20, 25, 152, 21, 22, 22, 81, 32, 20, 21, 293, 187, 10, 14, 26, 24, 20, 26, 29, 42, 10, 26, 31, 25, 170, 74, 45, 13, 37, 391, 15, 25, 47, 20, 24, 81, 22, 27, 16, 20, 44, 161, 62, 27, 20, 758, 28, 12, 24, 21, 35, 25, 323, 19, 24, 124, 82, 85, 60, 42, 20, 12, 23, 22, 10, 54, 22, 168, 11, 9, 182, 17, 214, 16, 23, 20, 44, 18, 15, 57, 64, 22, 63, 22, 34, 29, 19, 24, 24, 20, 99, 19, 23, 34, 21, 15, 14, 37, 26, 18, 21, 26, 8, 26, 15, 13, 11, 26, 61, 13, 78, 18, 22, 21, 63, 48, 13, 17, 61, 22, 8, 16, 27, 18, 68, 15, 85, 13, 19, 19, 19, 27, 13, 17, 22, 14, 19, 51, 141, 20, 20, 14, 19, 15, 225, 11, 15, 9, 14, 16, 20, 176, 13, 14, 12, 18, 19, 16, 11, 30, 12, 15, 23, 18, 22, 21, 20, 68, 68, 12, 24, 37, 122, 59, 17, 38, 12, 33, 384, 98, 28, 17, 13, 23, 14, 41, 8, 12, 11, 154, 31, 13, 22, 9, 25, 24, 18, 15, 20, 31, 18, 223, 23, 70, 12, 23, 12, 19, 62, 14, 173, 25, 17, 96, 22, 19, 24, 10, 30, 20, 17, 66, 18, 22, 92, 172, 14, 33, 9, 14, 90, 41, 22, 135, 14, 25, 26, 27, 34, 20, 37, 30, 151, 45, 26, 12, 11, 25, 17, 42, 28, 12, 16, 196, 39, 22, 27, 18, 25, 20, 26, 26, 19, 233, 22, 17, 63, 41, 49, 66, 15, 66, 266, 56, 604, 719, 12, 13, 29, 30, 16, 14, 24, 28, 14, 18, 17, 33, 21, 13, 246, 310, 12, 216, 121, 30, 532, 112, 572, 76, 231, 18, 32, 16, 41, 18, 106, 275, 14, 10, 21, 30, 12, 34, 22, 16, 16, 25, 19, 21, 32, 21, 21, 17, 25, 33, 15, 14, 35, 23, 18, 11, 14, 18, 27, 51, 24, 28, 15, 127, 21, 25, 17, 16, 21, 28, 23, 16, 24, 13, 28, 34, 22, 21, 19, 20, 27, 21, 39, 105, 25, 11, 75, 15, 33, 26, 28, 16, 25, 118, 37, 18, 24, 36, 12, 25, 16, 16, 14, 21, 13, 18, 20, 84, 19, 14, 17, 21, 20, 25, 11, 12, 20, 23, 14, 17, 25, 20, 13, 18, 25, 29, 115, 17, 22, 21, 20, 17, 214, 27, 19, 20, 9, 13, 17, 18, 81, 33, 199, 156, 24, 62, 10, 30, 18, 9, 91, 21, 19, 29, 39, 6, 9, 17, 485, 15, 19, 15, 19, 52, 10, 10, 15, 14, 15, 15, 154, 58, 12, 18, 8, 26, 24, 52, 27, 42, 14, 29, 31, 47, 21, 14, 25, 20, 15, 210, 20, 24, 350, 109, 45, 172, 64, 9, 228, 25, 17, 15, 18, 129, 14, 160, 13, 22, 26, 14, 48, 10, 17, 16, 18, 27, 14, 25, 27, 24, 57, 19, 80, 42, 628, 13, 10, 13, 203, 422, 61, 38, 24, 28, 14, 13, 10, 23, 26, 66, 89, 9, 26, 15, 41, 12, 27, 10, 26, 12, 21, 16, 29, 19, 15, 18, 24, 23, 14, 26, 31, 98, 143, 138, 9, 34, 10, 55, 9, 11, 27, 25, 19, 63, 15, 96, 29, 34, 9, 47, 18, 96, 14, 20, 23, 22, 33, 15, 51, 32, 34, 17, 19, 21, 21, 61, 12, 44, 14, 49, 21, 37, 30, 11, 21, 29, 330, 46, 24, 26, 31, 11, 19, 23, 20, 32, 36, 14, 14, 26, 374, 21, 26, 16, 332, 10, 62, 19, 14, 19, 17, 17, 79, 401, 16, 107, 19, 12, 21, 32, 52, 54, 43, 77, 37, 9, 17, 55, 146, 20, 20, 13, 20, 16, 27, 59, 25, 22, 14, 9, 21, 76, 15, 11, 20, 26, 33, 47, 14, 14, 20, 17, 18, 23, 20, 75, 7, 17, 11, 13, 28, 26, 21, 20, 12, 14, 12, 17, 25, 14, 16, 21, 10, 15, 20, 66, 96, 22, 27]"
},
{
"path": "long-tailed_recognition/BalancedSoftmax/config/CIFAR100_LT/balanced_softmax_imba100.yaml",
"content": "coslr: false\ncoslrwarmup: true\ncriterions:\n PerformanceLoss:\n def_file: ./loss/BalancedSoftmaxLoss.py\n loss_params: {freq_path: ./cls_freq/CIFAR-100-LT_IMBA100.json}\n optim_params: null\n weight: 1.0\nendlr: 0.0\nwarmup_iterations: 800\nbase_lr: 0.05\nwarmup_lr: 0.1\nlast: false\nmemory: {centroids: false, init_centroids: false}\nmodel_dir: null\nnetworks:\n classifier:\n def_file: ./models/DotProductClassifier.py\n optim_params: {lr: 0.05, momentum: 0.9, weight_decay: 0.0005}\n params: {feat_dim: 64, num_classes: 100}\n feat_model:\n def_file: ./models/ResNet32Feature.py\n fix: false\n optim_params: {lr: 0.05, momentum: 0.9, weight_decay: 0.0005}\n params: {}\nshuffle: false\ntraining_opt:\n backbone: resnet32\n batch_size: 512\n dataset: CIFAR100_LT\n display_step: 10\n feature_dim: 64\n log_dir: ./logs/CIFAR100_LT/models/resnet32_balanced_softmax_imba100\n num_classes: 100\n num_iterations: 13000\n num_workers: 4\n open_threshold: 0.1\n sampler: null\n scheduler_params: {gamma: 0.1, step_size: 3}\n sub_dir: models\n cifar_imb_ratio: 100\n"
},
{
"path": "long-tailed_recognition/BalancedSoftmax/config/CIFAR100_LT/balanced_softmax_imba200.yaml",
"content": "coslr: false\ncoslrwarmup: true\ncriterions:\n PerformanceLoss:\n def_file: ./loss/BalancedSoftmaxLoss.py\n loss_params: {freq_path: ./cls_freq/CIFAR-100-LT_IMBA200.json}\n optim_params: null\n weight: 1.0\nendlr: 0.0\nwarmup_iterations: 800\nbase_lr: 0.05\nwarmup_lr: 0.1\nlast: false\nmemory: {centroids: false, init_centroids: false}\nmodel_dir: null\nnetworks:\n classifier:\n def_file: ./models/DotProductClassifier.py\n optim_params: {lr: 0.05, momentum: 0.9, weight_decay: 0.0005}\n params: {feat_dim: 64, num_classes: 100}\n feat_model:\n def_file: ./models/ResNet32Feature.py\n fix: false\n optim_params: {lr: 0.05, momentum: 0.9, weight_decay: 0.0005}\n params: {}\nshuffle: false\ntraining_opt:\n backbone: resnet32\n batch_size: 512\n dataset: CIFAR100_LT\n display_step: 10\n feature_dim: 64\n log_dir: ./logs/CIFAR100_LT/models/resnet32_balanced_softmax_imba200\n num_classes: 100\n num_iterations: 13000\n num_workers: 4\n open_threshold: 0.1\n sampler: null\n scheduler_params: {gamma: 0.1, step_size: 3}\n sub_dir: models\n cifar_imb_ratio: 200\n"
},
{
"path": "long-tailed_recognition/BalancedSoftmax/config/CIFAR100_LT/balms_imba100.yaml",
"content": "coslr: true\ncriterions:\n PerformanceLoss:\n def_file: ./loss/BalancedSoftmaxLoss.py\n loss_params: {freq_path: ./cls_freq/CIFAR-100-LT_IMBA100.json}\n optim_params: null\n weight: 1.0\nendlr: 0.0\nlast: false\nmemory: {centroids: false, init_centroids: false}\nmodel_dir: ./logs/CIFAR100_LT/models/resnet32_softmax_imba100\nnetworks:\n classifier:\n def_file: ./models/DotProductClassifier.py\n optim_params: {lr: 0.1, momentum: 0.9, weight_decay: 0.0005}\n params: {feat_dim: 64, num_classes: 100}\n feat_model:\n def_file: ./models/ResNet32Feature.py\n fix: true\n optim_params: {lr: 0.1, momentum: 0.9, weight_decay: 0.0005}\n params: {}\nshuffle: false\ntraining_opt:\n backbone: resnet32\n batch_size: 512\n dataset: CIFAR100_LT\n display_step: 10\n feature_dim: 64\n log_dir: ./logs/CIFAR100_LT/clslearn/resnet32_balms_imba100\n num_classes: 100\n num_iterations: 2000\n num_workers: 4\n open_threshold: 0.1\n sampler: {def_file: ./data/MetaSampler.py, type: MetaSampler}\n scheduler_params: {gamma: 0.1, step_size: 3}\n sub_dir: models\n cifar_imb_ratio: 100\n"
},
{
"path": "long-tailed_recognition/BalancedSoftmax/config/CIFAR100_LT/balms_imba200.yaml",
"content": "coslr: true\ncriterions:\n PerformanceLoss:\n def_file: ./loss/BalancedSoftmaxLoss.py\n loss_params: {freq_path: ./cls_freq/CIFAR-100-LT_IMBA200.json}\n optim_params: null\n weight: 1.0\nendlr: 0.0\nlast: false\nmemory: {centroids: false, init_centroids: false}\nmodel_dir: ./logs/CIFAR100_LT/models/resnet32_softmax_imba200\nnetworks:\n classifier:\n def_file: ./models/DotProductClassifier.py\n optim_params: {lr: 0.1, momentum: 0.9, weight_decay: 0.0005}\n params: {feat_dim: 64, num_classes: 100}\n feat_model:\n def_file: ./models/ResNet32Feature.py\n fix: true\n optim_params: {lr: 0.1, momentum: 0.9, weight_decay: 0.0005}\n params: {}\nshuffle: false\ntraining_opt:\n backbone: resnet32\n batch_size: 512\n dataset: CIFAR100_LT\n display_step: 10\n feature_dim: 64\n log_dir: ./logs/CIFAR100_LT/clslearn/resnet32_balms_imba200\n num_classes: 100\n num_iterations: 2000\n num_workers: 4\n open_threshold: 0.1\n sampler: {def_file: ./data/MetaSampler.py, type: MetaSampler}\n scheduler_params: {gamma: 0.1, step_size: 3}\n sub_dir: models\n cifar_imb_ratio: 200\n"
},
{
"path": "long-tailed_recognition/BalancedSoftmax/config/CIFAR100_LT/decouple_balanced_softmax_imba100.yaml",
"content": "coslr: true\ncriterions:\n PerformanceLoss:\n def_file: ./loss/BalancedSoftmaxLoss.py\n loss_params: {freq_path: ./cls_freq/CIFAR-100-LT_IMBA100.json}\n optim_params: null\n weight: 1.0\nendlr: 0.0\nlast: false\nmemory: {centroids: false, init_centroids: false}\nmodel_dir: ./logs/CIFAR100_LT/models/resnet32_softmax_imba100\nnetworks:\n classifier:\n def_file: ./models/DotProductClassifier.py\n optim_params: {lr: 0.1, momentum: 0.9, weight_decay: 0.0005}\n params: {feat_dim: 64, num_classes: 100}\n feat_model:\n def_file: ./models/ResNet32Feature.py\n fix: true\n optim_params: {lr: 0.1, momentum: 0.9, weight_decay: 0.0005}\n params: {}\nshuffle: false\ntraining_opt:\n backbone: resnet32\n batch_size: 512\n dataset: CIFAR100_LT\n display_step: 10\n feature_dim: 64\n log_dir: ./logs/CIFAR100_LT/clslearn/resnet32_decouple_balanced_softmax_imba100\n num_classes: 100\n num_iterations: 2000\n num_workers: 4\n open_threshold: 0.1\n sampler: null\n scheduler_params: {gamma: 0.1, step_size: 3}\n sub_dir: models\n cifar_imb_ratio: 100\n"
},
{
"path": "long-tailed_recognition/BalancedSoftmax/config/CIFAR100_LT/decouple_balanced_softmax_imba200.yaml",
"content": "coslr: true\ncriterions:\n PerformanceLoss:\n def_file: ./loss/BalancedSoftmaxLoss.py\n loss_params: {freq_path: ./cls_freq/CIFAR-100-LT_IMBA200.json}\n optim_params: null\n weight: 1.0\nendlr: 0.0\nlast: false\nmemory: {centroids: false, init_centroids: false}\nmodel_dir: ./logs/CIFAR100_LT/models/resnet32_softmax_imba200\nnetworks:\n classifier:\n def_file: ./models/DotProductClassifier.py\n optim_params: {lr: 0.1, momentum: 0.9, weight_decay: 0.0005}\n params: {feat_dim: 64, num_classes: 100}\n feat_model:\n def_file: ./models/ResNet32Feature.py\n fix: true\n optim_params: {lr: 0.1, momentum: 0.9, weight_decay: 0.0005}\n params: {}\nshuffle: false\ntraining_opt:\n backbone: resnet32\n batch_size: 512\n dataset: CIFAR100_LT\n display_step: 10\n feature_dim: 64\n log_dir: ./logs/CIFAR100_LT/clslearn/resnet32_decouple_balanced_softmax_imba200\n num_classes: 100\n num_iterations: 2000\n num_workers: 4\n open_threshold: 0.1\n sampler: null\n scheduler_params: {gamma: 0.1, step_size: 3}\n sub_dir: models\n cifar_imb_ratio: 200\n"
},
{
"path": "long-tailed_recognition/BalancedSoftmax/config/CIFAR100_LT/softmax_imba.yaml",
"content": "coslr: false\ncoslrwarmup: true\ncriterions:\n PerformanceLoss:\n def_file: ./loss/SoftmaxLoss.py\n loss_params: {}\n optim_params: null\n weight: 1.0\nendlr: 0.0\nwarmup_iterations: 800\nbase_lr: 0.05\nwarmup_lr: 0.1\nlast: false\nmemory: {centroids: false, init_centroids: false}\nmodel_dir: null\nnetworks:\n classifier:\n def_file: ./models/DotProductClassifier.py\n optim_params: {lr: 0.05, momentum: 0.9, weight_decay: 0.0005}\n params: {feat_dim: 64, num_classes: 100}\n feat_model:\n def_file: ./models/ResNet32Feature.py\n fix: false\n optim_params: {lr: 0.05, momentum: 0.9, weight_decay: 0.0005}\n params: {}\nshuffle: false\ntraining_opt:\n backbone: resnet32\n batch_size: 512\n dataset: CIFAR100_LT\n display_step: 10\n feature_dim: 64\n log_dir: ./logs/CIFAR100_LT/models/resnet32_softmax_imba100\n num_classes: 100\n num_iterations: 13000\n num_workers: 4\n open_threshold: 0.1\n sampler: null\n scheduler_params: {gamma: 0.1, step_size: 3}\n sub_dir: models\n cifar_imb_ratio: 1\n"
},
{
"path": "long-tailed_recognition/BalancedSoftmax/config/CIFAR100_LT/softmax_imba100.yaml",
"content": "coslr: false\ncoslrwarmup: true\ncriterions:\n PerformanceLoss:\n def_file: ./loss/SoftmaxLoss.py\n loss_params: {}\n optim_params: null\n weight: 1.0\nendlr: 0.0\nwarmup_iterations: 800\nbase_lr: 0.05\nwarmup_lr: 0.1\nlast: false\nmemory: {centroids: false, init_centroids: false}\nmodel_dir: null\nnetworks:\n classifier:\n def_file: ./models/DotProductClassifier.py\n optim_params: {lr: 0.05, momentum: 0.9, weight_decay: 0.0005}\n params: {feat_dim: 64, num_classes: 100}\n feat_model:\n def_file: ./models/ResNet32Feature.py\n fix: false\n optim_params: {lr: 0.05, momentum: 0.9, weight_decay: 0.0005}\n params: {}\nshuffle: false\ntraining_opt:\n backbone: resnet32\n batch_size: 512\n dataset: CIFAR100_LT\n display_step: 10\n feature_dim: 64\n log_dir: ./logs/CIFAR100_LT/models/resnet32_softmax_imba100\n num_classes: 100\n num_iterations: 13000\n num_workers: 4\n open_threshold: 0.1\n sampler: null\n scheduler_params: {gamma: 0.1, step_size: 3}\n sub_dir: models\n cifar_imb_ratio: 100\n"
},
{
"path": "long-tailed_recognition/BalancedSoftmax/config/CIFAR100_LT/softmax_imba200.yaml",
"content": "coslr: false\ncoslrwarmup: true\ncriterions:\n PerformanceLoss:\n def_file: ./loss/SoftmaxLoss.py\n loss_params: {}\n optim_params: null\n weight: 1.0\nendlr: 0.0\nwarmup_iterations: 800\nbase_lr: 0.05\nwarmup_lr: 0.1\nlast: false\nmemory: {centroids: false, init_centroids: false}\nmodel_dir: null\nnetworks:\n classifier:\n def_file: ./models/DotProductClassifier.py\n optim_params: {lr: 0.05, momentum: 0.9, weight_decay: 0.0005}\n params: {feat_dim: 64, num_classes: 100}\n feat_model:\n def_file: ./models/ResNet32Feature.py\n fix: false\n optim_params: {lr: 0.05, momentum: 0.9, weight_decay: 0.0005}\n params: {}\nshuffle: false\ntraining_opt:\n backbone: resnet32\n batch_size: 512\n dataset: CIFAR100_LT\n display_step: 10\n feature_dim: 64\n log_dir: ./logs/CIFAR100_LT/models/resnet32_softmax_imba200\n num_classes: 100\n num_iterations: 13000\n num_workers: 4\n open_threshold: 0.1\n sampler: null\n scheduler_params: {gamma: 0.1, step_size: 3}\n sub_dir: models\n cifar_imb_ratio: 200\n"
},
{
"path": "long-tailed_recognition/BalancedSoftmax/config/CIFAR10_LT/balanced_softmax_imba200.yaml",
"content": "coslr: false\ncoslrwarmup: true\ncriterions:\n PerformanceLoss:\n def_file: ./loss/BalancedSoftmaxLoss.py\n loss_params: {freq_path: ./cls_freq/CIFAR-10-LT_IMBA200.json}\n optim_params: null\n weight: 1.0\nendlr: 0.0\nwarmup_iterations: 800\nbase_lr: 0.05\nwarmup_lr: 0.1\nlast: false\nmemory: {centroids: false, init_centroids: false}\nmodel_dir: null\nnetworks:\n classifier:\n def_file: ./models/DotProductClassifier.py\n optim_params: {lr: 0.05, momentum: 0.9, weight_decay: 0.0005}\n params: {feat_dim: 64, num_classes: 10}\n feat_model:\n def_file: ./models/ResNet32Feature.py\n fix: false\n optim_params: {lr: 0.05, momentum: 0.9, weight_decay: 0.0005}\n params: {}\nshuffle: false\ntraining_opt:\n backbone: resnet32\n batch_size: 512\n dataset: CIFAR10_LT\n display_step: 10\n feature_dim: 64\n log_dir: ./logs/CIFAR10_LT/models/resnet32_balanced_softmax_imba200\n num_classes: 10\n num_iterations: 13000\n num_workers: 4\n open_threshold: 0.1\n sampler: null\n scheduler_params: {gamma: 0.1, step_size: 3}\n sub_dir: models\n cifar_imb_ratio: 200\n"
},
{
"path": "long-tailed_recognition/BalancedSoftmax/config/CIFAR10_LT/balms_imba200.yaml",
"content": "coslr: true\ncriterions:\n PerformanceLoss:\n def_file: ./loss/BalancedSoftmaxLoss.py\n loss_params: {freq_path: ./cls_freq/CIFAR-10-LT_IMBA200.json}\n optim_params: null\n weight: 1.0\nendlr: 0.0\nlast: false\nmemory: {centroids: false, init_centroids: false}\nmodel_dir: ./logs/CIFAR10_LT/models/resnet32_softmax_imba200\nnetworks:\n classifier:\n def_file: ./models/DotProductClassifier.py\n optim_params: {lr: 0.1, momentum: 0.9, weight_decay: 0.0005}\n params: {feat_dim: 64, num_classes: 10}\n feat_model:\n def_file: ./models/ResNet32Feature.py\n fix: true\n optim_params: {lr: 0.1, momentum: 0.9, weight_decay: 0.0005}\n params: {}\nshuffle: false\ntraining_opt:\n backbone: resnet32\n batch_size: 512\n dataset: CIFAR10_LT\n display_step: 10\n feature_dim: 64\n log_dir: ./logs/CIFAR10_LT/clslearn/resnet32_balms_imba200\n num_classes: 10\n num_iterations: 2000\n num_workers: 4\n open_threshold: 0.1\n sampler: {def_file: ./data/MetaSampler.py, type: MetaSampler}\n scheduler_params: {gamma: 0.1, step_size: 3}\n sub_dir: models\n cifar_imb_ratio: 200\n"
},
{
"path": "long-tailed_recognition/BalancedSoftmax/config/CIFAR10_LT/decouple_balanced_softmax_imba200.yaml",
"content": "coslr: true\ncriterions:\n PerformanceLoss:\n def_file: ./loss/BalancedSoftmaxLoss.py\n loss_params: {freq_path: ./cls_freq/CIFAR-10-LT_IMBA200.json}\n optim_params: null\n weight: 1.0\nendlr: 0.0\nlast: false\nmemory: {centroids: false, init_centroids: false}\nmodel_dir: ./logs/CIFAR10_LT/models/resnet32_softmax_imba200\nnetworks:\n classifier:\n def_file: ./models/DotProductClassifier.py\n optim_params: {lr: 0.1, momentum: 0.9, weight_decay: 0.0005}\n params: {feat_dim: 64, num_classes: 10}\n feat_model:\n def_file: ./models/ResNet32Feature.py\n fix: true\n optim_params: {lr: 0.1, momentum: 0.9, weight_decay: 0.0005}\n params: {}\nshuffle: false\ntraining_opt:\n backbone: resnet32\n batch_size: 512\n dataset: CIFAR10_LT\n display_step: 10\n feature_dim: 64\n log_dir: ./logs/CIFAR10_LT/clslearn/resnet32_decouple_balanced_softmax_imba200\n num_classes: 10\n num_iterations: 2000\n num_workers: 4\n open_threshold: 0.1\n sampler: null\n scheduler_params: {gamma: 0.1, step_size: 3}\n sub_dir: models\n cifar_imb_ratio: 200\n"
},
{
"path": "long-tailed_recognition/BalancedSoftmax/config/CIFAR10_LT/softmax_imba200.yaml",
"content": "coslr: false\ncoslrwarmup: true\ncriterions:\n PerformanceLoss:\n def_file: ./loss/SoftmaxLoss.py\n loss_params: {}\n optim_params: null\n weight: 1.0\nendlr: 0.0\nwarmup_iterations: 800\nbase_lr: 0.05\nwarmup_lr: 0.1\nlast: false\nmemory: {centroids: false, init_centroids: false}\nmodel_dir: null\nnetworks:\n classifier:\n def_file: ./models/DotProductClassifier.py\n optim_params: {lr: 0.05, momentum: 0.9, weight_decay: 0.0005}\n params: {feat_dim: 64, num_classes: 10}\n feat_model:\n def_file: ./models/ResNet32Feature.py\n fix: false\n optim_params: {lr: 0.05, momentum: 0.9, weight_decay: 0.0005}\n params: {}\nshuffle: false\ntraining_opt:\n backbone: resnet32\n batch_size: 512\n dataset: CIFAR10_LT\n display_step: 10\n feature_dim: 64\n log_dir: ./logs/CIFAR10_LT/models/resnet32_softmax_imba200\n num_classes: 10\n num_iterations: 13000\n num_workers: 4\n open_threshold: 0.1\n sampler: null\n scheduler_params: {gamma: 0.1, step_size: 3}\n sub_dir: models\n cifar_imb_ratio: 200\n"
},
{
"path": "long-tailed_recognition/BalancedSoftmax/config/ImageNet_LT/balanced_softmax_resnet10.yaml",
"content": "coslr: true\ncriterions:\n PerformanceLoss:\n def_file: ./loss/BalancedSoftmaxLoss.py\n loss_params: {freq_path: ./cls_freq/ImageNet_LT.json}\n optim_params: null\n weight: 1.0\nendlr: 0.0\nlast: false\nmemory: {centroids: false, init_centroids: false}\nmodel_dir: null\nnetworks:\n classifier:\n def_file: ./models/DotProductClassifier.py\n optim_params: {lr: 0.05, momentum: 0.9, weight_decay: 0.0005}\n params: {dataset: ImageNet_LT, feat_dim: 512, log_dir: ./logs/ImageNet_LT/models/balanced_softmax_resnet10,\n num_classes: 1000, stage1_weights: false}\n feat_model:\n def_file: ./models/ResNet10Feature.py\n fix: false\n optim_params: {lr: 0.05, momentum: 0.9, weight_decay: 0.0005}\n params: {dataset: ImageNet_LT, dropout: null, log_dir: ./logs/ImageNet_LT/models/balanced_softmax_resnet10,\n stage1_weights: false, use_fc: false, use_selfatt: false}\nshuffle: false\ntraining_opt:\n backbone: resnet10\n batch_size: 128\n dataset: ImageNet_LT\n display_step: 10\n feature_dim: 512\n log_dir: ./logs/ImageNet_LT/models/balanced_softmax_resnet10\n log_root: /logs/ImageNet_LT\n num_classes: 1000\n num_epochs: 90\n num_workers: 12\n open_threshold: 0.1\n sampler: null\n scheduler_params: {gamma: 0.1, step_size: 30}\n stage: balanced_softmax_resnet10\n sub_dir: models\n"
},
{
"path": "long-tailed_recognition/BalancedSoftmax/config/ImageNet_LT/balanced_softmax_resnet50.yaml",
"content": "coslr: true\ncriterions:\n PerformanceLoss:\n def_file: ./loss/BalancedSoftmaxLoss.py\n loss_params: {freq_path: ./cls_freq/ImageNet_LT.json}\n optim_params: null\n weight: 1.0\nendlr: 0.0\nlast: false\nmemory: {centroids: false, init_centroids: false}\nmodel_dir: null\nnetworks:\n classifier:\n def_file: ./models/DotProductClassifier.py\n optim_params: {lr: 0.05, momentum: 0.9, weight_decay: 0.0005}\n params: {dataset: ImageNet_LT, feat_dim: 2048, log_dir: ./logs/ImageNet_LT/models/balanced_softmax_resnet50,\n num_classes: 1000, stage1_weights: false}\n feat_model:\n def_file: ./models/ResNet50Feature.py\n fix: false\n optim_params: {lr: 0.05, momentum: 0.9, weight_decay: 0.0005}\n params: {dataset: ImageNet_LT, dropout: null, log_dir: ./logs/ImageNet_LT/models/balanced_softmax_resnet50,\n stage1_weights: false, use_fc: false, use_selfatt: false}\nshuffle: false\ntraining_opt:\n backbone: resnet50\n batch_size: 128\n dataset: ImageNet_LT\n display_step: 100\n feature_dim: 2048\n log_dir: ./logs/ImageNet_LT/models/balanced_softmax_resnet50\n log_root: /logs/ImageNet_LT\n num_classes: 1000\n num_epochs: 90\n num_workers: 12\n open_threshold: 0.1\n sampler: null\n scheduler_params: {gamma: 0.1, step_size: 30}\n stage: balanced_softmax_resnet50\n sub_dir: models\n"
},
{
"path": "long-tailed_recognition/BalancedSoftmax/config/ImageNet_LT/balms_resnet10.yaml",
"content": "coslr: true\ncriterions:\n PerformanceLoss:\n def_file: ./loss/BalancedSoftmaxLoss.py\n loss_params: {freq_path: ./cls_freq/ImageNet_LT.json}\n optim_params: null\n weight: 1.0\nendlr: 0.0\nlast: false\nmemory: {centroids: false, init_centroids: false}\nmodel_dir: ./logs/ImageNet_LT/models/resnet10_uniform_e90\nnetworks:\n classifier:\n def_file: ./models/DotProductClassifier.py\n optim_params: {lr: 0.125, momentum: 0.9, weight_decay: 0.0005}\n params: {dataset: ImageNet_LT, feat_dim: 512, log_dir: ./logs/ImageNet_LT/clslearn/balms_resnet10,\n num_classes: 1000, stage1_weights: false}\n feat_model:\n def_file: ./models/ResNet10Feature.py\n fix: true\n optim_params: {lr: 0.05, momentum: 0.9, weight_decay: 0.0005}\n params: {dataset: ImageNet_LT, dropout: null, log_dir: ./logs/ImageNet_LT/clslearn/balms_resnet10,\n stage1_weights: false, use_fc: false, use_selfatt: false}\nshuffle: false\ntraining_opt:\n backbone: resnet10\n batch_size: 128\n dataset: ImageNet_LT\n display_step: 10\n feature_dim: 512\n log_dir: ./logs/ImageNet_LT/clslearn/balms_resnet10\n log_root: ./logs/ImageNet_LT\n num_classes: 1000\n num_epochs: 20\n num_workers: 12\n open_threshold: 0.1\n sampler: {def_file: ./data/MetaSampler.py, type: MetaSampler, meta_batch_size: 1024, init_pow: 0.1, lr: 0.001}\n scheduler_params: {gamma: 0.1, step_size: 3}\n stage: balms_resnet10\n sub_dir: clslearn\n"
},
{
"path": "long-tailed_recognition/BalancedSoftmax/config/ImageNet_LT/balms_resnet50.yaml",
"content": "coslr: true\ncriterions:\n PerformanceLoss:\n def_file: ./loss/BalancedSoftmaxLoss.py\n loss_params: {freq_path: ./cls_freq/ImageNet_LT.json}\n optim_params: null\n weight: 1.0\nendlr: 0.0\nlast: false\nmemory: {centroids: false, init_centroids: false}\nmodel_dir: ./logs/ImageNet_LT/models/resnet10_uniform_e90\nnetworks:\n classifier:\n def_file: ./models/DotProductClassifier.py\n optim_params: {lr: 0.5, momentum: 0.9, weight_decay: 0.0005}\n params: {dataset: ImageNet_LT, feat_dim: 512, log_dir: ./logs/ImageNet_LT/clslearn/balms_resnet10,\n num_classes: 1000, stage1_weights: false}\n feat_model:\n def_file: ./models/ResNet10Feature.py\n fix: true\n optim_params: {lr: 0.2, momentum: 0.9, weight_decay: 0.0005}\n params: {dataset: ImageNet_LT, dropout: null, log_dir: ./logs/ImageNet_LT/clslearn/balms_resnet10,\n stage1_weights: false, use_fc: false, use_selfatt: false}\nshuffle: false\ntraining_opt:\n backbone: resnet10\n batch_size: 512\n dataset: ImageNet_LT\n display_step: 10\n feature_dim: 512\n log_dir: ./logs/ImageNet_LT/clslearn/balms_resnet10\n log_root: ./logs/ImageNet_LT\n num_classes: 1000\n num_epochs: 20\n num_workers: 12\n open_threshold: 0.1\n sampler: {def_file: ./data/MetaSampler.py, type: MetaSampler, meta_batch_size: 1024, init_pow: 0.1, lr: 0.001}\n scheduler_params: {gamma: 0.1, step_size: 3}\n stage: balms_resnet10\n sub_dir: clslearn\n"
},
{
"path": "long-tailed_recognition/BalancedSoftmax/config/ImageNet_LT/cls_crt.yaml",
"content": "coslr: true\ncriterions:\n PerformanceLoss:\n def_file: ./loss/SoftmaxLoss.py\n loss_params: {}\n optim_params: null\n weight: 1.0\nendlr: 0.0\nlast: false\nmemory: {centroids: false, init_centroids: false}\nmodel_dir: ./logs/ImageNet_LT/models/resnext50_uniform_e90\nnetworks:\n classifier:\n def_file: ./models/DotProductClassifier.py\n optim_params: {lr: 0.2, momentum: 0.9, weight_decay: 0.0005}\n params: {dataset: ImageNet_LT, feat_dim: 2048, log_dir: ./logs/ImageNet_LT/clslearn/retrain_uni2ban,\n num_classes: 1000, stage1_weights: false}\n feat_model:\n def_file: ./models/ResNext50Feature.py\n fix: true\n optim_params: {lr: 0.2, momentum: 0.9, weight_decay: 0.0005}\n params: {dataset: ImageNet_LT, dropout: null, log_dir: ./logs/ImageNet_LT/clslearn/retrain_uni2ban,\n stage1_weights: false, use_fc: false, use_selfatt: false}\nshuffle: true\ntraining_opt:\n backbone: resnext50\n batch_size: 512\n dataset: ImageNet_LT\n display_step: 10\n feature_dim: 2048\n log_dir: ./logs/ImageNet_LT/clslearn/retrain_uni2ban\n log_root: ./logs/ImageNet_LT\n num_classes: 1000\n num_epochs: 10\n num_workers: 12\n open_threshold: 0.1\n sampler: {def_file: ./data/ClassAwareSampler.py, num_samples_cls: 4, type: ClassAwareSampler}\n scheduler_params: {gamma: 0.1, step_size: 3}\n stage: retrain_uni2ban\n sub_dir: clslearn\n"
},
{
"path": "long-tailed_recognition/BalancedSoftmax/config/ImageNet_LT/cls_lws.yaml",
"content": "coslr: true\ncriterions:\n PerformanceLoss:\n def_file: ./loss/SoftmaxLoss.py\n loss_params: {}\n optim_params: null\n weight: 1.0\nendlr: 0.0\nlast: false\nmemory: {centroids: false, init_centroids: false}\nmodel_dir: ./logs/ImageNet_LT/models/resnext50_uniform_e90\nnetworks:\n classifier:\n def_file: ./models/TauNormClassifier.py\n optim_params: {lr: 0.2, momentum: 0.9, weight_decay: 0.0005}\n params: {dataset: ImageNet_LT, feat_dim: 2048, log_dir: ./logs/ImageNet_LT/clslearn/lws_uni2ban,\n num_classes: 1000, stage1_weights: false}\n feat_model:\n def_file: ./models/ResNext50Feature.py\n fix: true\n optim_params: {lr: 0.2, momentum: 0.9, weight_decay: 0.0005}\n params: {dataset: ImageNet_LT, dropout: null, log_dir: ./logs/ImageNet_LT/clslearn/lws_uni2ban,\n stage1_weights: false, use_fc: false, use_selfatt: false}\nshuffle: true\ntraining_opt:\n backbone: resnext50\n batch_size: 512\n dataset: ImageNet_LT\n display_step: 10\n feature_dim: 2048\n log_dir: ./logs/ImageNet_LT/clslearn/lws_uni2ban\n log_root: ./logs/ImageNet_LT\n num_classes: 1000\n num_epochs: 5\n num_workers: 12\n open_threshold: 0.1\n sampler: {def_file: ./data/ClassAwareSampler.py, num_samples_cls: 4, type: ClassAwareSampler}\n scheduler_params: {gamma: 0.1, step_size: 1}\n stage: lws_uni2ban\n sub_dir: clslearn\n"
},
{
"path": "long-tailed_recognition/BalancedSoftmax/config/ImageNet_LT/decouple_balanced_softmax_resnet10.yaml",
"content": "coslr: true\ncriterions:\n PerformanceLoss:\n def_file: ./loss/BalancedSoftmaxLoss.py\n loss_params: {freq_path: ./cls_freq/ImageNet_LT.json}\n optim_params: null\n weight: 1.0\nendlr: 0.0\nlast: false\nmemory: {centroids: false, init_centroids: false}\nmodel_dir: ./logs/ImageNet_LT/models/resnet10_uniform_e90\nnetworks:\n classifier:\n def_file: ./models/DotProductClassifier.py\n optim_params: {lr: 0.4, momentum: 0.9, weight_decay: 0.0005}\n params: {dataset: ImageNet_LT, feat_dim: 512, log_dir: ./logs/ImageNet_LT/clslearn/decouple_balanced_softmax_resnet10,\n num_classes: 1000, stage1_weights: false}\n feat_model:\n def_file: ./models/ResNet10Feature.py\n fix: true\n optim_params: {lr: 0.2, momentum: 0.9, weight_decay: 0.0005}\n params: {dataset: ImageNet_LT, dropout: null, log_dir: ./logs/ImageNet_LT/clslearn/decouple_balanced_softmax_resnet10,\n stage1_weights: false, use_fc: false, use_selfatt: false}\nshuffle: true\ntraining_opt:\n backbone: resnet10\n batch_size: 512\n dataset: ImageNet_LT\n display_step: 10\n feature_dim: 512\n log_dir: ./logs/ImageNet_LT/clslearn/decouple_balanced_softmax_resnet10\n log_root: ./logs/ImageNet_LT\n num_classes: 1000\n num_epochs: 10\n num_workers: 12\n open_threshold: 0.1\n sampler: null\n scheduler_params: {gamma: 0.1, step_size: 3}\n stage: decouple_balanced_softmax_resnet10\n sub_dir: clslearn\n"
},
{
"path": "long-tailed_recognition/BalancedSoftmax/config/ImageNet_LT/feat_balance.yaml",
"content": "coslr: true\ncriterions:\n PerformanceLoss:\n def_file: ./loss/SoftmaxLoss.py\n loss_params: {}\n optim_params: null\n weight: 1.0\nendlr: 0.0\nlast: false\nmemory: {centroids: false, init_centroids: false}\nmodel_dir: null\nnetworks:\n classifier:\n def_file: ./models/DotProductClassifier.py\n optim_params: {lr: 0.2, momentum: 0.9, weight_decay: 0.0005}\n params: {dataset: ImageNet_LT, feat_dim: 2048, log_dir: ./logs/ImageNet_LT/models/resnext50_banlance_e90,\n num_classes: 1000, stage1_weights: false}\n feat_model:\n def_file: ./models/ResNext50Feature.py\n fix: false\n optim_params: {lr: 0.2, momentum: 0.9, weight_decay: 0.0005}\n params: {dataset: ImageNet_LT, dropout: null, log_dir: ./logs/ImageNet_LT/models/resnext50_banlance_e90,\n stage1_weights: false, use_fc: false, use_selfatt: false}\nshuffle: true\ntraining_opt:\n backbone: resnext50\n batch_size: 128\n dataset: ImageNet_LT\n display_step: 10\n feature_dim: 2048\n log_dir: ./logs/ImageNet_LT/models/resnext50_banlance_e90\n log_root: ./logs/ImageNet_LT\n num_classes: 1000\n num_epochs: 90\n num_workers: 12\n open_threshold: 0.1\n sampler: {def_file: ./data/ClassAwareSampler.py, num_samples_cls: 4, type: ClassAwareSampler}\n scheduler_params: {gamma: 0.1, step_size: 30}\n stage: resnext50_banlance_e90\n sub_dir: models\n"
},
{
"path": "long-tailed_recognition/BalancedSoftmax/config/ImageNet_LT/feat_shift.yaml",
"content": "coslr: true\ncriterions:\n PerformanceLoss:\n def_file: ./loss/SoftmaxLoss.py\n loss_params: {}\n optim_params: null\n weight: 1.0\nendlr: 0.0\nlast: false\nmemory: {centroids: false, init_centroids: false}\nmodel_dir: null\nnetworks:\n classifier:\n def_file: ./models/DotProductClassifier.py\n optim_params: {lr: 0.2, momentum: 0.9, weight_decay: 0.0005}\n params: {dataset: ImageNet_LT, feat_dim: 2048, log_dir: ./logs/ImageNet_LT/models/resnext50_shift_e90,\n num_classes: 1000, stage1_weights: false}\n feat_model:\n def_file: ./models/ResNext50Feature.py\n fix: false\n optim_params: {lr: 0.2, momentum: 0.9, weight_decay: 0.0005}\n params: {dataset: ImageNet_LT, dropout: null, log_dir: ./logs/ImageNet_LT/models/resnext50_shift_e90,\n stage1_weights: false, use_fc: false, use_selfatt: false}\nshuffle: false\ntraining_opt:\n backbone: resnext50\n batch_size: 512\n dataset: ImageNet_LT\n display_step: 10\n feature_dim: 2048\n log_dir: ./logs/ImageNet_LT/models/resnext50_shift_e90\n log_root: ./logs/ImageNet_LT\n num_classes: 1000\n num_epochs: 90\n num_workers: 12\n open_threshold: 0.1\n sampler: {alpha: 1.0, cycle: 0, decay_gap: 30, def_file: ./data/ClassPrioritySampler.py,\n epochs: 90, fixed_scale: 1, lam: null, manual_only: true, momentum: 0.0, nroot: null,\n pri_mode: train, ptype: score, rescale: false, root_decay: null, type: ClassPrioritySampler}\n scheduler_params: {gamma: 0.1, step_size: 30}\n stage: resnext50_shift_e90\n sub_dir: models\n"
},
{
"path": "long-tailed_recognition/BalancedSoftmax/config/ImageNet_LT/feat_squareroot.yaml",
"content": "coslr: true\ncriterions:\n PerformanceLoss:\n def_file: ./loss/SoftmaxLoss.py\n loss_params: {}\n optim_params: null\n weight: 1.0\nendlr: 0.0\nlast: false\nmemory: {centroids: false, init_centroids: false}\nmodel_dir: null\nnetworks:\n classifier:\n def_file: ./models/DotProductClassifier.py\n optim_params: {lr: 0.2, momentum: 0.9, weight_decay: 0.0005}\n params: {dataset: ImageNet_LT, feat_dim: 2048, log_dir: ./logs/ImageNet_LT/models/resnext50_square_e90,\n num_classes: 1000, stage1_weights: false}\n feat_model:\n def_file: ./models/ResNext50Feature.py\n fix: false\n optim_params: {lr: 0.2, momentum: 0.9, weight_decay: 0.0005}\n params: {dataset: ImageNet_LT, dropout: null, log_dir: ./logs/ImageNet_LT/models/resnext50_square_e90,\n stage1_weights: false, use_fc: false, use_selfatt: false}\nshuffle: false\ntraining_opt:\n backbone: resnext50\n batch_size: 512\n dataset: ImageNet_LT\n display_step: 10\n feature_dim: 2048\n log_dir: ./logs/ImageNet_LT/models/resnext50_square_e90\n log_root: ./logs/ImageNet_LT\n num_classes: 1000\n num_epochs: 90\n num_workers: 12\n open_threshold: 0.1\n sampler: {alpha: 1.0, cycle: 0, decay_gap: 30, def_file: ./data/MixedPrioritizedSampler.py,\n epochs: 90, fixed_scale: 1, lam: 1.0, manual_only: true, nroot: 2.0, ptype: score,\n rescale: false, root_decay: null, type: MixedPrioritizedSampler}\n scheduler_params: {gamma: 0.1, step_size: 30}\n stage: resnext50_square_e90\n sub_dir: models\n"
},
{
"path": "long-tailed_recognition/BalancedSoftmax/config/ImageNet_LT/feat_uniform.yaml",
"content": "coslr: true\ncriterions:\n PerformanceLoss:\n def_file: ./loss/SoftmaxLoss.py\n loss_params: {}\n optim_params: null\n weight: 1.0\nendlr: 0.0\nlast: false\nmemory: {centroids: false, init_centroids: false}\nmodel_dir: null\nnetworks:\n classifier:\n def_file: ./models/DotProductClassifier.py\n optim_params: {lr: 0.2, momentum: 0.9, weight_decay: 0.0005}\n params: {dataset: ImageNet_LT, feat_dim: 2048, log_dir: ./logs/ImageNet_LT/models/resnext50_uniform_e90,\n num_classes: 1000, stage1_weights: false}\n feat_model:\n def_file: ./models/ResNext50Feature.py\n fix: false\n optim_params: {lr: 0.2, momentum: 0.9, weight_decay: 0.0005}\n params: {dataset: ImageNet_LT, dropout: null, log_dir: ./logs/ImageNet_LT/models/resnext50_uniform_e90,\n stage1_weights: false, use_fc: false, use_selfatt: false}\nshuffle: false\ntraining_opt:\n backbone: resnext50\n batch_size: 512\n dataset: ImageNet_LT\n display_step: 10\n feature_dim: 2048\n log_dir: ./logs/ImageNet_LT/models/resnext50_uniform_e90\n log_root: /logs/ImageNet_LT\n num_classes: 1000\n num_epochs: 90\n num_workers: 12\n open_threshold: 0.1\n sampler: null\n scheduler_params: {gamma: 0.1, step_size: 30}\n stage: resnext50_uniform_e90\n sub_dir: models\n"
},
{
"path": "long-tailed_recognition/BalancedSoftmax/config/ImageNet_LT/feat_uniform_resnet10.yaml",
"content": "coslr: true\ncriterions:\n PerformanceLoss:\n def_file: ./loss/SoftmaxLoss.py\n loss_params: {}\n optim_params: null\n weight: 1.0\nendlr: 0.0\nlast: false\nmemory: {centroids: false, init_centroids: false}\nmodel_dir: null\nnetworks:\n classifier:\n def_file: ./models/DotProductClassifier.py\n optim_params: {lr: 0.2, momentum: 0.9, weight_decay: 0.0005}\n params: {dataset: ImageNet_LT, feat_dim: 512, log_dir: ./logs/ImageNet_LT/models/resnet10_uniform_e90,\n num_classes: 1000, stage1_weights: false}\n feat_model:\n def_file: ./models/ResNet10Feature.py\n fix: false\n optim_params: {lr: 0.2, momentum: 0.9, weight_decay: 0.0005}\n params: {dataset: ImageNet_LT, dropout: null, log_dir: ./logs/ImageNet_LT/models/resnet10_uniform_e90,\n stage1_weights: false, use_fc: false, use_selfatt: false}\nshuffle: false\ntraining_opt:\n backbone: resnet10\n batch_size: 512\n dataset: ImageNet_LT\n display_step: 10\n feature_dim: 512\n log_dir: ./logs/ImageNet_LT/models/resnet10_uniform_e90\n log_root: /logs/ImageNet_LT\n num_classes: 1000\n num_epochs: 90\n num_workers: 12\n open_threshold: 0.1\n sampler: null\n scheduler_params: {gamma: 0.1, step_size: 30}\n stage: resnet10_uniform_e90\n sub_dir: models\n"
},
{
"path": "long-tailed_recognition/BalancedSoftmax/config/ImageNet_LT/feat_uniform_resnet50.yaml",
"content": "coslr: true\ncriterions:\n PerformanceLoss:\n def_file: ./loss/SoftmaxLoss.py\n loss_params: {}\n optim_params: null\n weight: 1.0\nendlr: 0.0\nlast: false\nmemory: {centroids: false, init_centroids: false}\nmodel_dir: null\nnetworks:\n classifier:\n def_file: ./models/DotProductClassifier.py\n optim_params: {lr: 0.05, momentum: 0.9, weight_decay: 0.0005}\n params: {dataset: ImageNet_LT, feat_dim: 2048, log_dir: ./logs/ImageNet_LT/models/l05_resnet50_uniform_e90,\n num_classes: 1000, stage1_weights: false}\n feat_model:\n def_file: ./models/ResNet50Feature.py\n fix: false\n optim_params: {lr: 0.05, momentum: 0.9, weight_decay: 0.0005}\n params: {dataset: ImageNet_LT, dropout: null, log_dir: ./logs/ImageNet_LT/models/l05_resnet50_uniform_e90,\n stage1_weights: false, use_fc: false, use_selfatt: false}\nshuffle: false\ntraining_opt:\n backbone: resnet50\n batch_size: 128\n dataset: ImageNet_LT\n display_step: 10\n feature_dim: 2048\n log_dir: ./logs/ImageNet_LT/models/l05_resnet50_uniform_e90\n log_root: /logs/ImageNet_LT\n num_classes: 1000\n num_epochs: 90\n num_workers: 12\n open_threshold: 0.1\n sampler: null\n scheduler_params: {gamma: 0.1, step_size: 30}\n stage: l05_resnet50_uniform_e90\n sub_dir: models\n"
},
{
"path": "long-tailed_recognition/BalancedSoftmax/config/ImageNet_LT/softmax_resnet10.yaml",
"content": "coslr: true\ncriterions:\n PerformanceLoss:\n def_file: ./loss/SoftmaxLoss.py\n loss_params: {}\n optim_params: null\n weight: 1.0\nendlr: 0.0\nlast: false\nmemory: {centroids: false, init_centroids: false}\nmodel_dir: ./logs/ImageNet_LT/models/resnet10_uniform_e90\nnetworks:\n classifier:\n def_file: ./models/DotProductClassifier.py\n optim_params: {lr: 0.2, momentum: 0.9, weight_decay: 0.0005}\n params: {dataset: ImageNet_LT, feat_dim: 512, log_dir: ./logs/ImageNet_LT/clslearn/softmax_resnet10_cbs,\n num_classes: 1000, stage1_weights: false}\n feat_model:\n def_file: ./models/ResNet10Feature.py\n fix: true\n optim_params: {lr: 0.2, momentum: 0.9, weight_decay: 0.0005}\n params: {dataset: ImageNet_LT, dropout: null, log_dir: ./logs/ImageNet_LT/clslearn/softmax_resnet10_cbs,\n stage1_weights: false, use_fc: false, use_selfatt: false}\nshuffle: false\ntraining_opt:\n backbone: resnet10\n batch_size: 512\n dataset: ImageNet_LT\n display_step: 10\n feature_dim: 512\n log_dir: ./logs/ImageNet_LT/clslearn/softmax_resnet10_cbs\n log_root: ./logs/ImageNet_LT\n num_classes: 1000\n num_epochs: 10\n num_workers: 12\n open_threshold: 0.1\n sampler: null\n scheduler_params: {gamma: 0.1, step_size: 3}\n stage: softmax_resnet10_cbs\n sub_dir: clslearn\n"
},
{
"path": "long-tailed_recognition/BalancedSoftmax/config/ImageNet_LT/softmax_resnet50_feat.yaml",
"content": "coslr: true\ncriterions:\n PerformanceLoss:\n def_file: ./loss/SoftmaxLoss.py\n loss_params: {}\n optim_params: null\n weight: 1.0\nendlr: 0.0\nlast: false\nmemory: {centroids: false, init_centroids: false}\nmodel_dir: null\nnetworks:\n classifier:\n def_file: ./models/DotProductClassifier.py\n optim_params: {lr: 0.2, momentum: 0.9, weight_decay: 0.0005}\n params: {dataset: ImageNet_LT, feat_dim: 512, log_dir: ./logs/ImageNet_LT/clslearn/softmax_resnet50_cbs,\n num_classes: 1000, stage1_weights: false}\n feat_model:\n def_file: ./models/ResNet50Feature.py\n fix: false\n optim_params: {lr: 0.2, momentum: 0.9, weight_decay: 0.0005}\n params: {dataset: ImageNet_LT, dropout: null, log_dir: ./logs/ImageNet_LT/models/softmax_resnet50_cbs,\n stage1_weights: false, use_fc: false, use_selfatt: false}\nshuffle: false\ntraining_opt:\n backbone: resnet10\n batch_size: 128\n dataset: ImageNet_LT\n display_step: 10\n feature_dim: 2048\n log_dir: ./logs/ImageNet_LT/models/softmax_resnet50_cbs\n log_root: ./logs/ImageNet_LT\n num_classes: 1000\n num_epochs: 90\n num_workers: 12\n open_threshold: 0.1\n sampler: null\n scheduler_params: {gamma: 0.1, step_size: 30}\n stage: softmax_resnet50_e90\n sub_dir: models\n"
},
{
"path": "long-tailed_recognition/BalancedSoftmax/config/Places_LT/balanced_softmax.yaml",
"content": "coslr: true\ncriterions:\n PerformanceLoss:\n def_file: ./loss/BalancedSoftmaxLoss.py\n loss_params: {freq_path: ./cls_freq/Places_LT.json}\n optim_params: null\n weight: 1.0\nendlr: 0.0\nlast: false\nmemory: {centroids: false, init_centroids: false}\nmodel_dir: ./logs/Places_LT/models/dir/resnet152_uniform.pth\nnetworks:\n classifier:\n def_file: ./models/DotProductClassifier.py\n optim_params: {lr: 0.02, momentum: 0.9, weight_decay: 0.0005}\n params: {dataset: Places_LT, feat_dim: 2048, num_classes: 365, stage1_weights: false}\n feat_model:\n def_file: ./models/ResNet152Feature.py\n fix: true\n optim_params: {lr: 0.01, momentum: 0.9, weight_decay: 0.0005}\n params: {dataset: Places_LT, dropout: null, stage1_weights: false,\n use_fc: false, use_selfatt: false}\nshuffle: true\ntraining_opt:\n batch_size: 128\n dataset: Places_LT\n display_step: 10\n feature_dim: 2048\n log_dir: ./logs/Places_LT/clslearn/balanced_softmax\n log_root: ./logs/Places_LT\n num_classes: 365\n num_epochs: 90\n num_workers: 12\n open_threshold: 0.1\n sampler: {def_file: ./data/ClassAwareSampler.py, num_samples_cls: 4, type: ClassAwareSampler}\n scheduler_params: {gamma: 0.1, step_size: 30}\n stage: balanced_softmax\n sub_dir: clslearn\n"
},
{
"path": "long-tailed_recognition/BalancedSoftmax/config/Places_LT/balms.yaml",
"content": "coslr: true\ncriterions:\n PerformanceLoss:\n def_file: ./loss/BalancedSoftmaxLoss.py\n loss_params: {freq_path: ./cls_freq/Places_LT.json}\n optim_params: null\n weight: 1.0\nendlr: 0.0\nlast: false\nmemory: {centroids: false, init_centroids: false}\nmodel_dir: ./logs/Places_LT/models/dir/resnet152_uniform.pth\nnetworks:\n classifier:\n def_file: ./models/DotProductClassifier.py\n optim_params: {lr: 0.02, momentum: 0.9, weight_decay: 0.0005}\n params: {dataset: Places_LT, feat_dim: 2048, num_classes: 365, stage1_weights: false}\n feat_model:\n def_file: ./models/ResNet152Feature.py\n fix: true\n optim_params: {lr: 0.01, momentum: 0.9, weight_decay: 0.0005}\n params: {dataset: Places_LT, dropout: null, stage1_weights: false,\n use_fc: false, use_selfatt: false}\nshuffle: false\ntraining_opt:\n batch_size: 256\n dataset: Places_LT\n display_step: 10\n feature_dim: 2048\n log_dir: ./logs/Places_LT/clslearn/balms\n log_root: ./logs/Places_LT\n num_classes: 365\n num_epochs: 10\n num_workers: 4\n open_threshold: 0.1\n sampler: {def_file: ./data/MetaSampler.py, type: MetaSampler, meta_batch_size: 512, init_pow: 0.3, lr: 0.001}\n scheduler_params: {gamma: 0.1, step_size: 30}\n stage: balms\n sub_dir: clslearn\n"
},
{
"path": "long-tailed_recognition/BalancedSoftmax/config/Places_LT/cls_crt.yaml",
"content": "coslr: true\ncriterions:\n PerformanceLoss:\n def_file: ./loss/SoftmaxLoss.py\n loss_params: {}\n optim_params: null\n weight: 1.0\nendlr: 0.0\nlast: false\nmemory: {centroids: false, init_centroids: false}\nmodel_dir: ./logs/Places_LT/models/dir\nnetworks:\n classifier:\n def_file: ./models/DotProductClassifier.py\n optim_params: {lr: 0.1, momentum: 0.9, weight_decay: 0.0005}\n params: {dataset: Places_LT, feat_dim: 2048, num_classes: 365, stage1_weights: false}\n feat_model:\n def_file: ./models/ResNet152Feature.py\n fix: true\n optim_params: {lr: 0.01, momentum: 0.9, weight_decay: 0.0005}\n params: {dataset: Places_LT, dropout: null, stage1_weights: false,\n use_fc: false, use_selfatt: false}\nshuffle: true\ntraining_opt:\n batch_size: 256\n dataset: Places_LT\n display_step: 10\n feature_dim: 2048\n log_dir: ./logs/Places_LT/clslearn/retrain_uni2ban\n log_root: ./logs/Places_LT\n num_classes: 365\n num_epochs: 10\n num_workers: 4\n open_threshold: 0.1\n sampler: {def_file: ./data/ClassAwareSampler.py, num_samples_cls: 4, type: ClassAwareSampler}\n scheduler_params: {gamma: 0.1, step_size: 30}\n stage: retrain_uni2ban\n sub_dir: clslearn\n"
},
{
"path": "long-tailed_recognition/BalancedSoftmax/config/Places_LT/cls_lws.yaml",
"content": "coslr: true\ncriterions:\n PerformanceLoss:\n def_file: ./loss/SoftmaxLoss.py\n loss_params: {}\n optim_params: null\n weight: 1.0\nendlr: 0.0\nlast: false\nmemory: {centroids: false, init_centroids: false}\nmodel_dir: ./logs/Places_LT/models/dir/resnet152_uniform.pth\nnetworks:\n classifier:\n def_file: ./models/TauNormClassifier.py\n optim_params: {lr: 0.1, momentum: 0.9, weight_decay: 0.0005}\n params: {dataset: Places_LT, feat_dim: 2048, num_classes: 365, stage1_weights: false}\n feat_model:\n def_file: ./models/ResNet152Feature.py\n fix: true\n optim_params: {lr: 0.01, momentum: 0.9, weight_decay: 0.0005}\n params: {dataset: Places_LT, dropout: null, stage1_weights: false,\n use_fc: false, use_selfatt: false}\nshuffle: true\ntraining_opt:\n batch_size: 256\n dataset: Places_LT\n display_step: 10\n feature_dim: 2048\n log_dir: ./logs/Places_LT/clslearn/lws_uni2ban\n log_root: ./logs/Places_LT\n num_classes: 365\n num_epochs: 5\n num_workers: 4\n open_threshold: 0.1\n sampler: {def_file: ./data/ClassAwareSampler.py, num_samples_cls: 4, type: ClassAwareSampler}\n scheduler_params: {gamma: 0.1, step_size: 30}\n stage: lws_uni2ban\n sub_dir: clslearn\n"
},
{
"path": "long-tailed_recognition/BalancedSoftmax/config/Places_LT/decouple_balanced_softmax.yaml",
"content": "coslr: true\ncriterions:\n PerformanceLoss:\n def_file: ./loss/BalancedSoftmaxLoss.py\n loss_params: {freq_path: ./cls_freq/Places_LT.json}\n optim_params: null\n weight: 1.0\nendlr: 0.0\nlast: false\nmemory: {centroids: false, init_centroids: false}\nmodel_dir: ./logs/Places_LT/models/dir/resnet152_uniform.pth\nnetworks:\n classifier:\n def_file: ./models/DotProductClassifier.py\n optim_params: {lr: 0.02, momentum: 0.9, weight_decay: 0.0005}\n params: {dataset: Places_LT, feat_dim: 2048, num_classes: 365, stage1_weights: false}\n feat_model:\n def_file: ./models/ResNet152Feature.py\n fix: true\n optim_params: {lr: 0.01, momentum: 0.9, weight_decay: 0.0005}\n params: {dataset: Places_LT, dropout: null, stage1_weights: false,\n use_fc: false, use_selfatt: false}\nshuffle: true\ntraining_opt:\n batch_size: 256\n dataset: Places_LT\n display_step: 10\n feature_dim: 2048\n log_dir: ./logs/Places_LT/clslearn/decouple_balanced_softmax\n log_root: ./logs/Places_LT\n num_classes: 365\n num_epochs: 10\n num_workers: 4\n open_threshold: 0.1\n sampler: {def_file: ./data/ClassAwareSampler.py, num_samples_cls: 4, type: ClassAwareSampler}\n scheduler_params: {gamma: 0.1, step_size: 30}\n stage: decouple_balanced_softmax\n sub_dir: clslearn\n"
},
{
"path": "long-tailed_recognition/BalancedSoftmax/config/Places_LT/feat_unifrom.yaml",
"content": "coslr: true\ncriterions:\n PerformanceLoss:\n def_file: ./loss/BalancedSoftmaxLoss.py\n loss_params: {freq_path: ./cls_freq/Places_LT.json}\n optim_params: null\n weight: 1.0\nendlr: 0.0\nlast: false\nmemory: {centroids: false, init_centroids: false}\nmodel_dir: null\nnetworks:\n classifier:\n def_file: ./models/DotProductClassifier.py\n optim_params: {lr: 0.2, momentum: 0.9, weight_decay: 0.0001}\n params: {dataset: iNaturalist18, feat_dim: 2048, log_dir: ./logs/iNaturalist18/models/resnet50_uniform_e90,\n num_classes: 8142, stage1_weights: false}\n feat_model:\n def_file: ./models/ResNet50Feature.py\n fix: false\n optim_params: {lr: 0.2, momentum: 0.9, weight_decay: 0.0001}\n params: {dataset: iNaturalist18, dropout: null, log_dir: ./logs/iNaturalist18/models/resnet50_uniform_e90,\n stage1_weights: false, use_fc: false, use_selfatt: false}\nshuffle: false\ntraining_opt:\n backbone: resnet50\n batch_size: 128\n dataset: iNaturalist18\n display_step: 10\n feature_dim: 2048\n log_dir: ./logs/iNaturalist18/models/resnet50_uniform_e90\n log_root: ./logs/iNaturalist18\n num_classes: 8142\n num_epochs: 90\n num_workers: 12\n open_threshold: 0.1\n sampler: null\n scheduler_params: {gamma: 0.1, step_size: 30}\n stage: resnet50_uniform_e90\n sub_dir: models\n"
},
{
"path": "long-tailed_recognition/BalancedSoftmax/config/iNaturalist18/balanced_softmax.yaml",
"content": "coslr: true\ncriterions:\n PerformanceLoss:\n def_file: ./loss/BalancedSoftmaxLoss.py\n loss_params: {freq_path: ./cls_freq/iNaturalist18.json}\n optim_params: null\n weight: 1.0\nendlr: 0.0\nlast: false\nmemory: {centroids: false, init_centroids: false}\nmodel_dir: null\nnetworks:\n classifier:\n def_file: ./models/DotProductClassifier.py\n optim_params: {lr: 0.05, momentum: 0.9, weight_decay: 0.0001}\n params: {dataset: iNaturalist18, feat_dim: 2048, log_dir: ./logs/iNaturalist18/models/balanced_softmax_resnet50,\n num_classes: 8142, stage1_weights: false}\n feat_model:\n def_file: ./models/ResNet50Feature.py\n fix: false\n optim_params: {lr: 0.05, momentum: 0.9, weight_decay: 0.0001}\n params: {dataset: iNaturalist18, dropout: null, log_dir: ./logs/iNaturalist18/models/balanced_softmax_resnet50,\n stage1_weights: false, use_fc: false, use_selfatt: false}\nshuffle: false\ntraining_opt:\n backbone: resnet50\n batch_size: 128\n dataset: iNaturalist18\n display_step: 10\n feature_dim: 2048\n log_dir: ./logs/iNaturalist18/models/balanced_softmax_resnet50\n log_root: ./logs/iNaturalist18\n num_classes: 8142\n num_epochs: 90\n num_workers: 12\n open_threshold: 0.1\n sampler: null\n scheduler_params: {gamma: 0.1, step_size: 30}\n stage: resnet50_uniform_e90\n sub_dir: models\n"
},
{
"path": "long-tailed_recognition/BalancedSoftmax/config/iNaturalist18/cls_crt.yaml",
"content": "coslr: true\ncriterions:\n PerformanceLoss:\n def_file: ./loss/SoftmaxLoss.py\n loss_params: {}\n optim_params: null\n weight: 1.0\nendlr: 0.0\nlast: false\nmemory: {centroids: false, init_centroids: false}\nmodel_dir: ./logs/iNaturalist18/models/resnet50_uniform_e90\nnetworks:\n classifier:\n def_file: ./models/DotProductClassifier.py\n optim_params: {lr: 0.2, momentum: 0.9, weight_decay: 0.0001}\n params: {dataset: iNaturalist18, feat_dim: 2048, log_dir: ./logs/iNaturalist18/clslearn/retrain_uni2ban,\n num_classes: 8142, stage1_weights: false}\n feat_model:\n def_file: ./models/ResNet50Feature.py\n fix: true\n optim_params: {lr: 0.2, momentum: 0.9, weight_decay: 0.0001}\n params: {dataset: iNaturalist18, dropout: null, log_dir: ./logs/iNaturalist18/clslearn/retrain_uni2ban,\n stage1_weights: false, use_fc: false, use_selfatt: false}\nshuffle: true\ntraining_opt:\n backbone: resnet50\n batch_size: 512\n dataset: iNaturalist18\n display_step: 10\n feature_dim: 2048\n log_dir: ./logs/iNaturalist18/clslearn/retrain_uni2ban\n log_root: ./logs/iNaturalist18\n num_classes: 8142\n num_epochs: 30\n num_workers: 12\n open_threshold: 0.1\n sampler: {def_file: ./data/ClassAwareSampler.py, num_samples_cls: 4, type: ClassAwareSampler}\n scheduler_params: {gamma: 0.1, step_size: 10}\n stage: retrain_uni2ban\n sub_dir: clslearn\n"
},
{
"path": "long-tailed_recognition/BalancedSoftmax/config/iNaturalist18/cls_lws.yaml",
"content": "coslr: true\ncriterions:\n PerformanceLoss:\n def_file: ./loss/SoftmaxLoss.py\n loss_params: {}\n optim_params: null\n weight: 1.0\nendlr: 0.0\nlast: false\nmemory: {centroids: false, init_centroids: false}\nmodel_dir: ./logs/iNaturalist18/models/resnet50_uniform_e90\nnetworks:\n classifier:\n def_file: ./models/TauNormClassifier.py\n optim_params: {lr: 0.2, momentum: 0.9, weight_decay: 0.0001}\n params: {dataset: iNaturalist18, feat_dim: 2048, log_dir: ./logs/iNaturalist18/clslearn/lws_uni2ban,\n num_classes: 8142, stage1_weights: false}\n feat_model:\n def_file: ./models/ResNet50Feature.py\n fix: true\n optim_params: {lr: 0.2, momentum: 0.9, weight_decay: 0.0001}\n params: {dataset: iNaturalist18, dropout: null, log_dir: ./logs/iNaturalist18/clslearn/lws_uni2ban,\n stage1_weights: false, use_fc: false, use_selfatt: false}\nshuffle: true\ntraining_opt:\n backbone: resnet50\n batch_size: 512\n dataset: iNaturalist18\n display_step: 10\n feature_dim: 2048\n log_dir: ./logs/iNaturalist18/clslearn/lws_uni2ban\n log_root: ./logs/iNaturalist18\n num_classes: 8142\n num_epochs: 5\n num_workers: 12\n open_threshold: 0.1\n sampler: {def_file: ./data/ClassAwareSampler.py, num_samples_cls: 4, type: ClassAwareSampler}\n scheduler_params: {gamma: 0.1, step_size: 1}\n stage: lws_uni2ban\n sub_dir: clslearn\n"
},
{
"path": "long-tailed_recognition/BalancedSoftmax/config/iNaturalist18/feat_unifrom.yaml",
"content": "coslr: true\ncriterions:\n PerformanceLoss:\n def_file: ./loss/SoftmaxLoss.py\n loss_params: {}\n optim_params: null\n weight: 1.0\nendlr: 0.0\nlast: false\nmemory: {centroids: false, init_centroids: false}\nmodel_dir: null\nnetworks:\n classifier:\n def_file: ./models/DotProductClassifier.py\n optim_params: {lr: 0.05, momentum: 0.9, weight_decay: 0.0001}\n params: {dataset: iNaturalist18, feat_dim: 2048, log_dir: ./logs/iNaturalist18/models/resnet50_uniform_e90,\n num_classes: 8142, stage1_weights: false}\n feat_model:\n def_file: ./models/ResNet50Feature.py\n fix: false\n optim_params: {lr: 0.05, momentum: 0.9, weight_decay: 0.0001}\n params: {dataset: iNaturalist18, dropout: null, log_dir: ./logs/iNaturalist18/models/resnet50_uniform_e90,\n stage1_weights: false, use_fc: false, use_selfatt: false}\nshuffle: false\ntraining_opt:\n backbone: resnet50\n batch_size: 128\n dataset: iNaturalist18\n display_step: 10\n feature_dim: 2048\n log_dir: ./logs/iNaturalist18/models/resnet50_uniform_e90\n log_root: ./logs/iNaturalist18\n num_classes: 8142\n num_epochs: 90\n num_workers: 12\n open_threshold: 0.1\n sampler: null\n scheduler_params: {gamma: 0.1, step_size: 30}\n stage: resnet50_uniform_e90\n sub_dir: models\n"
},
{
"path": "long-tailed_recognition/BalancedSoftmax/data/ClassAwareSampler.py",
"content": "\"\"\"Copyright (c) Facebook, Inc. and its affiliates.\nAll rights reserved.\n\nThis source code is licensed under the license found in the\nLICENSE file in the root directory of this source tree.\n\nPortions of the source code are from the OLTR project which\nnotice below and in LICENSE in the root directory of\nthis source tree.\n\nCopyright (c) 2019, Zhongqi Miao\nAll rights reserved.\n\"\"\"\n\nimport random\nimport numpy as np\nfrom torch.utils.data.sampler import Sampler\nimport pdb\n\n##################################\n## Class-aware sampling, partly implemented by frombeijingwithlove\n##################################\n\nclass RandomCycleIter:\n \n def __init__ (self, data, test_mode=False):\n self.data_list = list(data)\n self.length = len(self.data_list)\n self.i = self.length - 1\n self.test_mode = test_mode\n \n def __iter__ (self):\n return self\n \n def __next__ (self):\n self.i += 1\n \n if self.i == self.length:\n self.i = 0\n if not self.test_mode:\n random.shuffle(self.data_list)\n \n return self.data_list[self.i]\n \ndef class_aware_sample_generator (cls_iter, data_iter_list, n, num_samples_cls=1, is_infinite=False):\n\n i = 0\n j = 0\n while i < n or is_infinite:\n \n# yield next(data_iter_list[next(cls_iter)])\n \n if j >= num_samples_cls:\n j = 0\n \n if j == 0:\n temp_tuple = next(zip(*[data_iter_list[next(cls_iter)]]*num_samples_cls))\n yield temp_tuple[j]\n else:\n yield temp_tuple[j]\n \n i += 1\n j += 1\n\nclass ClassAwareSampler (Sampler):\n \n def __init__(self, data_source, num_samples_cls=1, is_infinite=False):\n num_classes = len(np.unique(data_source.labels))\n self.class_iter = RandomCycleIter(range(num_classes))\n cls_data_list = [list() for _ in range(num_classes)]\n for i, label in enumerate(data_source.labels):\n cls_data_list[label].append(i)\n self.data_iter_list = [RandomCycleIter(x) for x in cls_data_list]\n self.num_samples = max([len(x) for x in cls_data_list]) * len(cls_data_list)\n self.num_samples_cls = num_samples_cls\n\n self.is_infinite = is_infinite\n \n def __iter__ (self):\n return class_aware_sample_generator(self.class_iter, self.data_iter_list,\n self.num_samples, self.num_samples_cls, self.is_infinite)\n \n def __len__ (self):\n return self.num_samples\n \ndef get_sampler():\n return ClassAwareSampler\n\n##################################"
},
{
"path": "long-tailed_recognition/BalancedSoftmax/data/ClassPrioritySampler.py",
"content": "\"\"\"Copyright (c) Facebook, Inc. and its affiliates.\nAll rights reserved.\n\nThis source code is licensed under the license found in the\nLICENSE file in the root directory of this source tree.\n\"\"\"\n\n\nimport random\nimport numpy as np\nfrom torch.utils.data.sampler import Sampler\n\n\nclass RandomCycleIter:\n \n def __init__ (self, data, test_mode=False):\n self.data_list = list(data)\n self.length = len(self.data_list)\n self.i = self.length - 1\n self.test_mode = test_mode\n \n def __iter__ (self):\n return self\n \n def __next__ (self):\n self.i += 1\n \n if self.i == self.length:\n self.i = 0\n if not self.test_mode:\n random.shuffle(self.data_list)\n \n return self.data_list[self.i]\n\n\nclass PriorityTree(object):\n def __init__(self, capacity, init_weights, fixed_weights=None, fixed_scale=1.0,\n alpha=1.0):\n \"\"\"\n fixed_weights: weights that wont be updated by self.update()\n \"\"\"\n assert fixed_weights is None or len(fixed_weights) == capacity\n assert len(init_weights) == capacity\n self.alpha = alpha\n self._capacity = capacity\n self._tree_size = 2 * capacity - 1\n self.fixed_scale = fixed_scale\n self.fixed_weights = np.zeros(self._capacity) if fixed_weights is None \\\n else fixed_weights\n self.tree = np.zeros(self._tree_size)\n self._initialized = False\n self.initialize(init_weights)\n\n def initialize(self, init_weights):\n \"\"\"Initialize the tree.\"\"\"\n\n # Rescale the fixed_weights if it is not zero\n self.fixed_scale_init = self.fixed_scale\n if self.fixed_weights.sum() > 0 and init_weights.sum() > 0:\n self.fixed_scale_init *= init_weights.sum() / self.fixed_weights.sum()\n self.fixed_weights *= self.fixed_scale * init_weights.sum() \\\n / self.fixed_weights.sum()\n print('FixedWeights: {}'.format(self.fixed_weights.sum()))\n\n self.update_whole(init_weights + self.fixed_weights)\n self._initialized = True\n \n def reset_adaptive_weights(self, adaptive_weights):\n self.update_whole(self.fixed_weights + adaptive_weights)\n \n def reset_fixed_weights(self, fixed_weights, rescale=False):\n \"\"\" Reset the manually designed weights and \n update the whole tree accordingly.\n\n @rescale: rescale the fixed_weights such that \n fixed_weights.sum() = self.fixed_scale * adaptive_weights.sum()\n \"\"\"\n\n adaptive_weights = self.get_adaptive_weights()\n fixed_sum = fixed_weights.sum()\n if rescale and fixed_sum > 0:\n # Rescale fixedweight based on adaptive weights\n scale = self.fixed_scale * adaptive_weights.sum() / fixed_sum\n else:\n # Rescale fixedweight based on previous fixedweight\n scale = self.fixed_weights.sum() / fixed_sum\n self.fixed_weights = fixed_weights * scale\n self.update_whole(self.fixed_weights + adaptive_weights)\n \n def update_whole(self, total_weights):\n \"\"\" Update the whole tree based on per-example sampling weights \"\"\"\n if self.alpha != 1:\n total_weights = np.power(total_weights, self.alpha)\n lefti = self.pointer_to_treeidx(0)\n righti = self.pointer_to_treeidx(self.capacity-1)\n self.tree[lefti:righti+1] = total_weights\n\n # Iteratively find a parent layer\n while lefti != 0 and righti != 0:\n lefti = (lefti - 1) // 2 if lefti != 0 else 0\n righti = (righti - 1) // 2 if righti != 0 else 0\n \n # Assign paraent weights from right to left\n for i in range(righti, lefti-1, -1):\n self.tree[i] = self.tree[2*i+1] + self.tree[2*i+2]\n \n def get_adaptive_weights(self):\n \"\"\" Get the instance-aware weights, that are not mannually designed\"\"\"\n if self.alpha == 1:\n return self.get_total_weights() - self.fixed_weights\n else:\n return self.get_raw_total_weights() - self.fixed_weights\n \n def get_total_weights(self):\n \"\"\" Get the per-example sampling weights\n return shape: [capacity]\n \"\"\"\n lefti = self.pointer_to_treeidx(0)\n righti = self.pointer_to_treeidx(self.capacity-1)\n return self.tree[lefti:righti+1]\n\n def get_raw_total_weights(self):\n \"\"\" Get the per-example sampling weights\n return shape: [capacity]\n \"\"\"\n lefti = self.pointer_to_treeidx(0)\n righti = self.pointer_to_treeidx(self.capacity-1)\n return np.power(self.tree[lefti:righti+1], 1/self.alpha)\n\n @property\n def size(self):\n return self._tree_size\n\n @property\n def capacity(self):\n return self._capacity\n\n def __len__(self):\n return self.capacity\n\n def pointer_to_treeidx(self, pointer):\n assert pointer < self.capacity\n return int(pointer + self.capacity - 1)\n\n def update(self, pointer, priority):\n assert pointer < self.capacity\n tree_idx = self.pointer_to_treeidx(pointer)\n priority += self.fixed_weights[pointer]\n if self.alpha != 1:\n priority = np.power(priority, self.alpha)\n delta = priority - self.tree[tree_idx]\n self.tree[tree_idx] = priority\n while tree_idx != 0:\n tree_idx = (tree_idx - 1) // 2\n self.tree[tree_idx] += delta\n \n def update_delta(self, pointer, delta):\n assert pointer < self.capacity\n tree_idx = self.pointer_to_treeidx(pointer)\n ratio = 1- self.fixed_weights[pointer] / self.tree[tree_idx]\n # delta *= ratio\n if self.alpha != 1:\n # Update delta\n if self.tree[tree_idx] < 0 or \\\n np.power(self.tree[tree_idx], 1/self.alpha) + delta < 0:\n import pdb; pdb.set_trace()\n delta = np.power(np.power(self.tree[tree_idx], 1/self.alpha) + delta,\n self.alpha) \\\n - self.tree[tree_idx]\n self.tree[tree_idx] += delta\n while tree_idx != 0:\n tree_idx = (tree_idx - 1) // 2\n self.tree[tree_idx] += delta\n\n def get_leaf(self, value):\n assert self._initialized, 'PriorityTree not initialized!!!!'\n assert self.total > 0, 'No priority weights setted!!'\n parent = 0\n while True:\n left_child = 2 * parent + 1\n right_child = 2 * parent + 2\n if left_child >= len(self.tree):\n tgt_leaf = parent\n break\n if value < self.tree[left_child]:\n parent = left_child\n else:\n value -= self.tree[left_child]\n parent = right_child\n data_idx = tgt_leaf - self.capacity + 1\n return data_idx, self.tree[tgt_leaf] # data idx, priority\n\n @property\n def total(self):\n assert self._initialized, 'PriorityTree not initialized!!!!'\n return self.tree[0]\n\n @property\n def max(self):\n return np.max(self.tree[-self.capacity:])\n\n @property\n def min(self):\n assert self._initialized, 'PriorityTree not initialized!!!!'\n return np.min(self.tree[-self.capacity:])\n \n def get_weights(self):\n wdict = {'fixed_weights': self.fixed_weights, \n 'total_weights': self.get_total_weights()}\n if self.alpha != 1:\n wdict.update({'raw_total_weights': self.get_raw_total_weights(),\n 'alpha': self.alpha})\n\n return wdict\n\nclass ClassPrioritySampler(Sampler):\n \"\"\"\n A sampler combining manually designed sampling strategy and prioritized \n sampling strategy.\n\n Manually disigned strategy contains two parts:\n\n $$ manual_weights = lam * balanced_weights + (1-lam) uniform_weights\n \n Here we use a generalized version of balanced weights as follows,\n when n limits to infinity, balanced_weights = real_balanced_weights\n \n $$ balanced_weights = uniform_weights ^ (1/n)\n \n Then the balanced weights are scaled such that \n \n $$ balanced_weights.sum() = balance_scale * uniform_weights.sum()\n\n Note: above weights are per-class weights\n\n Overall sampling weights are given as \n $$ sampling_weights = manual_weights * fixed_scale + priority_weights\n\n Arguments:\n @dataset: A dataset \n @balance_scale: The scale of balanced_weights\n @lam: A weight to combine balanced weights and uniform weights\n - None for shifting sampling\n - 0 for uniform sampling\n - 1 for balanced sampling\n @fixed_scale: The scale of manually designed weights\n - fixed_scale < 0 means, the manually designed distribution will \n be used as the backend distribution of priorities. \n @cycle: shifting strategy\n - 0 for linear shifting: 3 -> 2 - > 1\n - 1 for periodic shifting: \n 3 -> 2 - > 1 -> 3 -> 2 - > 1 -> 3 -> 2 - > 1\n - 2 for cosine-like periodic shifting:\n 3 -> 2 - > 1 -> 1 -> 2 - > 3 -> 3 -> 2 - > 1\n @nroot:\n - None for truly balanced weights\n - >= 2 for pseudo-balanced weights \n @rescale: whether to rebalance the manual weights and priority weights\n every epoch\n @root_decay:\n - 'exp': for exponential decay \n - 'linear': for linear decay \n \"\"\"\n def __init__(self, dataset, balance_scale=1.0, fixed_scale=1.0,\n lam=None, epochs=90, cycle=0, nroot=None, manual_only=False,\n rescale=False, root_decay=None, decay_gap=30, ptype='score',\n pri_mode='train', momentum=0., alpha=1.0):\n \"\"\"\n \"\"\"\n self.dataset = dataset\n self.balance_scale = balance_scale\n self.fixed_scale = fixed_scale\n self.epochs = epochs\n self.lam = lam\n self.cycle = cycle\n self.nroot = nroot\n self.rescale = rescale\n self.manual_only = manual_only\n self.root_decay = root_decay\n self.decay_gap = decay_gap\n self.ptype = ptype\n self.pri_mode = pri_mode\n self.num_samples = len(dataset)\n self.manual_as_backend = False\n self.momentum = momentum\n self.alpha = alpha\n\n assert 0. <= self.momentum <= 1.0\n assert 0. <= self.alpha\n\n # Change the backend distribution of priority if needed\n if self.fixed_scale <
![](\"rela_illu.png\")
\n![](\"BT_illu_dim.png\")
\n![](\".github/cait.png\")
\n![](\".github/resmlp.png\")
\n![](\"https://github.com/thuml/Transfer-Learning-Library/blob/master/TransLearn.png\"/)
\n\n## Introduction\n*Transfer-Learn* is an open-source and well-documented library for Transfer Learning. It is based on pure PyTorch with high performance and friendly API. Our code is pythonic, and the design is consistent with torchvision. You can easily develop new algorithms, or readily apply existing algorithms.\n\nThe currently supported algorithms include:\n\n##### [Domain Adaptation for Classification](/examples/domain_adaptation/image_classification)\n- [Domain Adversarial Neural Network (DANN)](https://arxiv.org/abs/1505.07818)\n- [Deep Adaptation Network (DAN)](https://arxiv.org/pdf/1502.02791)\n- [Joint Adaptation Network (JAN)](https://arxiv.org/abs/1605.06636)\n- [Adversarial Discriminative Domain Adaptation (ADDA)](https://arxiv.org/pdf/1702.05464.pdf)\n- [Conditional Domain Adversarial Network (CDAN)](https://arxiv.org/abs/1705.10667)\n- [Maximum Classifier Discrepancy (MCD)](https://arxiv.org/abs/1712.02560)\n- [Adaptive Feature Norm (AFN)](https://arxiv.org/pdf/1811.07456v2.pdf)\n- [Margin Disparity Discrepancy (MDD)](https://arxiv.org/abs/1904.05801)\n- [Batch Spectral Penalization (BSP)](http://ise.thss.tsinghua.edu.cn/~mlong/doc/batch-spectral-penalization-icml19.pdf)\n- [Minimum Class Confusion (MCC)](https://arxiv.org/abs/1912.03699)\n\n##### [Partial Domain Adaptation](/examples/domain_adaptation/partial_domain_adaptation/)\n- [Domain Adversarial Neural Network (DANN)](https://arxiv.org/abs/1505.07818)\n- [Partial Adversarial Domain Adaptation (PADA)](https://arxiv.org/abs/1808.04205)\n- [Importance Weighted Adversarial Nets (IWAN)](https://arxiv.org/abs/1803.09210)\n- [Adaptive Feature Norm (AFN)](https://arxiv.org/pdf/1811.07456v2.pdf)\n\n##### [Open-set Domain Adaptation](/examples/domain_adaptation/openset_domain_adaptation/)\n- [Open Set Domain Adaptation (OSBP)](https://arxiv.org/abs/1804.10427)\n\n##### [Domain Adaptation for Semantic Segmentation](/examples/domain_adaptation/semantic_segmentation/)\n- [Cycle-Consistent Adversarial Networks (CycleGAN)](https://arxiv.org/pdf/1703.10593.pdf)\n- [CyCADA: Cycle-Consistent Adversarial Domain Adaptation](https://arxiv.org/abs/1711.03213)\n- [Adversarial Entropy Minimization (ADVENT)](https://arxiv.org/abs/1811.12833)\n- [Fourier Domain Adaptation (FDA)](https://arxiv.org/abs/2004.05498)\n\n##### [Domain Adaptation for Keypoint Detection](/examples/domain_adaptation/keypoint_detection)\n- [Regressive Domain Adaptation for Unsupervised Keypoint Detection (RegDA, CVPR 2021)](http://ise.thss.tsinghua.edu.cn/~mlong/doc/regressive-domain-adaptation-cvpr21.pdf)\n\n##### [Domain Adaptation for Person Re-identification](/examples/domain_adaptation/re_identification/)\n- [Two at Once: Enhancing Learning and Generalization Capacities via IBN-Net (IBN-Net, ECCV 2018)](https://openaccess.thecvf.com/content_ECCV_2018/papers/Xingang_Pan_Two_at_Once_ECCV_2018_paper.pdf)\n- [Mutual Mean-Teaching: Pseudo Label Refinery for Unsupervised Domain Adaptation on Person Re-identification (MMT, ICLR 2020)](https://arxiv.org/abs/2001.01526)\n- [Similarity Preserving Generative Adversarial Network (SPGAN, CVPR 2018)](https://arxiv.org/pdf/1811.10551.pdf)\n\n##### [Task Adaptation for Image Classification](/examples/task_adaptation/image_classification/)\n- [Explicit inductive bias for transfer learning with convolutional networks\n (L2-SP, ICML 2018)](https://arxiv.org/abs/1802.01483)\n- [Catastrophic Forgetting Meets Negative Transfer: Batch Spectral Shrinkage for Safe Transfer Learning (BSS, NIPS 2019)](https://proceedings.neurips.cc/paper/2019/file/c6bff625bdb0393992c9d4db0c6bbe45-Paper.pdf)\n- [DEep Learning Transfer using Fea- ture Map with Attention for convolutional networks (DELTA, ICLR 2019)](https://openreview.net/pdf?id=rkgbwsAcYm)\n- [Co-Tuning for Transfer Learning (Co-Tuning, NIPS 2020)](http://ise.thss.tsinghua.edu.cn/~mlong/doc/co-tuning-for-transfer-learning-nips20.pdf)\n- [Stochastic Normalization (StochNorm, NIPS 2020)](https://papers.nips.cc/paper/2020/file/bc573864331a9e42e4511de6f678aa83-Paper.pdf)\n- [Learning Without Forgetting (LWF, ECCV 2016)](https://arxiv.org/abs/1606.09282)\n- [Bi-tuning of Pre-trained Representations (Bi-Tuning)](https://arxiv.org/abs/2011.06182?utm_source=feedburner&utm_medium=feed&utm_campaign=Feed%3A+arxiv%2FQSXk+%28ExcitingAds%21+cs+updates+on+arXiv.org%29)\n\n##### [Domain Generalization for Classification](/examples/domain_generalization/image_classification/)\n- [Two at Once: Enhancing Learning and Generalization Capacities via IBN-Net (IBN-Net, ECCV 2018)](https://openaccess.thecvf.com/content_ECCV_2018/papers/Xingang_Pan_Two_at_Once_ECCV_2018_paper.pdf)\n- [Domain Generalization with MixStyle (MixStyle, ICLR 2021)](https://arxiv.org/abs/2104.02008)\n- [Learning to Generalize: Meta-Learning for Domain Generalization (MLDG, AAAI 2018)](https://arxiv.org/pdf/1710.03463.pdf)\n- [Invariant Risk Minimization (IRM)](https://arxiv.org/abs/1907.02893)\n- [Out-of-Distribution Generalization via Risk Extrapolation (VREx, ICML 2021)](https://arxiv.org/abs/2003.00688)\n- [Distributionally Robust Neural Networks for Group Shifts: On the Importance of Regularization for Worst-Case Generalization(GroupDRO)](https://arxiv.org/abs/1911.08731)\n- [Deep CORAL: Correlation Alignment for Deep Domain Adaptation (Deep Coral, ECCV 2016)](https://arxiv.org/abs/1607.01719)\n\n##### [Domain Generalization for Person Re-identification](/examples/domain_generalization/re_identification/)\n- [Two at Once: Enhancing Learning and Generalization Capacities via IBN-Net (IBN-Net, ECCV 2018)](https://openaccess.thecvf.com/content_ECCV_2018/papers/Xingang_Pan_Two_at_Once_ECCV_2018_paper.pdf)\n- [Domain Generalization with MixStyle (MixStyle, ICLR 2021)](https://arxiv.org/abs/2104.02008)\n\nWe are planning to add\n- DA for Object Detection\n- TA for text classification\n\n## Installation\n\nTo use ``dalib``, ``talib``, ``dglib``, and ``common`` in other places, you need to install Transfer-Learn,\n```shell\npython setup.py install\n```\nNote that we do not support *pip install* currently.\n\nFor flexible use and modification of Transfer-Learn, please git clone the library.\n\n\n## Documentation\nYou can find the tutorial and API documentation on the website: [Documentation (please open in Firefox or Safari)](http://170.106.108.162/index.html). Note that this link is only for temporary use. You can also build the doc by yourself following the instructions in http://170.106.108.162/get_started/faq.html.\n\nAlso, we have examples in the directory `examples`. A typical usage is \n```shell script\n# Train a DANN on Office-31 Amazon -> Webcam task using ResNet 50.\n# Assume you have put the datasets under the path `data/office-31`, \n# or you are glad to download the datasets automatically from the Internet to this path\npython dann.py data/office31 -d Office31 -s A -t W -a resnet50 --epochs 20\n```\n\nIn the directory `examples`, you can find all the necessary running scripts to reproduce the benchmarks with specified hyper-parameters.\n\n## Contributing\nWe appreciate all contributions. If you are planning to contribute back bug-fixes, please do so without any further discussion. If you plan to contribute new features, utility functions or extensions, please first open an issue and discuss the feature with us. \n\n## Disclaimer on Datasets\n\nThis is a utility library that downloads and prepares public datasets. We do not host or distribute these datasets, vouch for their quality or fairness, or claim that you have licenses to use the dataset. It is your responsibility to determine whether you have permission to use the dataset under the dataset's license.\n\nIf you're a dataset owner and wish to update any part of it (description, citation, etc.), or do not want your dataset to be included in this library, please get in touch through a GitHub issue. Thanks for your contribution to the ML community!\n\n\n## Contact\nIf you have any problem with our code or have some suggestions, including the future feature, feel free to contact \n- Junguang Jiang (JiangJunguang1123@outlook.com)\n- Baixu Chen (cbx_99_hasta@outlook.com)\n- Mingsheng Long (longmingsheng@gmail.com)\n\nor describe it in Issues.\n\nFor Q&A in Chinese, you can choose to ask questions here before sending an email. [迁移学习算法库答疑专区](https://zhuanlan.zhihu.com/p/248104070)\n\n## Citation\n\nIf you use this toolbox or benchmark in your research, please cite this project. \n\n```latex\n@misc{dalib,\n author = {Junguang Jiang, Baixu Chen, Bo Fu, Mingsheng Long},\n title = {Transfer-Learning-library},\n year = {2020},\n publisher = {GitHub},\n journal = {GitHub repository},\n howpublished = {\\url{https://github.com/thuml/Transfer-Learning-Library}},\n}\n```\n\n## Acknowledgment\n\nWe would like to thank School of Software, Tsinghua University and The National Engineering Laboratory for Big Data Software for providing such an excellent ML research platform.\n\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/common/__init__.py",
"content": "__all__ = ['modules', 'utils', 'vision']\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/common/loss/__init__.py",
"content": "import torch.nn as nn\nimport torch.nn.functional as F\n\n\nclass KnowledgeDistillationLoss(nn.Module):\n \"\"\"Knowledge Distillation Loss.\n\n Args:\n T (double): Temperature. Default: 1.\n reduction (str, optional): Specifies the reduction to apply to the output:\n ``'none'`` | ``'mean'`` | ``'sum'``. ``'none'``: no reduction will be applied,\n ``'mean'``: the sum of the output will be divided by the number of\n elements in the output, ``'sum'``: the output will be summed. Default: ``'batchmean'``\n\n Inputs:\n - y_student (tensor): logits output of the student\n - y_teacher (tensor): logits output of the teacher\n\n Shape:\n - y_student: (minibatch, `num_classes`)\n - y_teacher: (minibatch, `num_classes`)\n\n \"\"\"\n def __init__(self, T=1., reduction='batchmean'):\n super(KnowledgeDistillationLoss, self).__init__()\n self.T = T\n self.kl = nn.KLDivLoss(reduction=reduction)\n\n def forward(self, y_student, y_teacher):\n \"\"\"\"\"\"\n return self.kl(F.log_softmax(y_student / self.T, dim=-1), F.softmax(y_teacher / self.T, dim=-1))\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/common/modules/__init__.py",
"content": "from .classifier import *\nfrom .regressor import *\n\n__all__ = ['classifier', 'regressor']"
},
{
"path": "domain_generalization/Transfer-Learning-Library/common/modules/classifier.py",
"content": "\"\"\"\n@author: Junguang Jiang\n@contact: JiangJunguang1123@outlook.com\n\"\"\"\nfrom typing import Tuple, Optional, List, Dict\nimport torch.nn as nn\nimport torch\n\n__all__ = ['Classifier']\n\n\nclass Classifier(nn.Module):\n \"\"\"A generic Classifier class for domain adaptation.\n\n Args:\n backbone (torch.nn.Module): Any backbone to extract 2-d features from data\n num_classes (int): Number of classes\n bottleneck (torch.nn.Module, optional): Any bottleneck layer. Use no bottleneck by default\n bottleneck_dim (int, optional): Feature dimension of the bottleneck layer. Default: -1\n head (torch.nn.Module, optional): Any classifier head. Use :class:`torch.nn.Linear` by default\n finetune (bool): Whether finetune the classifier or train from scratch. Default: True\n\n .. note::\n Different classifiers are used in different domain adaptation algorithms to achieve better accuracy\n respectively, and we provide a suggested `Classifier` for different algorithms.\n Remember they are not the core of algorithms. You can implement your own `Classifier` and combine it with\n the domain adaptation algorithm in this algorithm library.\n\n .. note::\n The learning rate of this classifier is set 10 times to that of the feature extractor for better accuracy\n by default. If you have other optimization strategies, please over-ride :meth:`~Classifier.get_parameters`.\n\n Inputs:\n - x (tensor): input data fed to `backbone`\n\n Outputs:\n - predictions: classifier's predictions\n - features: features after `bottleneck` layer and before `head` layer\n\n Shape:\n - Inputs: (minibatch, *) where * means, any number of additional dimensions\n - predictions: (minibatch, `num_classes`)\n - features: (minibatch, `features_dim`)\n\n \"\"\"\n\n def __init__(self, backbone: nn.Module, num_classes: int, bottleneck: Optional[nn.Module] = None,\n bottleneck_dim: Optional[int] = -1, head: Optional[nn.Module] = None, finetune=True, pool_layer=None):\n super(Classifier, self).__init__()\n self.backbone = backbone\n self.num_classes = num_classes\n if pool_layer is None:\n self.pool_layer = nn.Sequential(\n nn.AdaptiveAvgPool2d(output_size=(1, 1)),\n nn.Flatten()\n )\n else:\n self.pool_layer = pool_layer\n if bottleneck is None:\n self.bottleneck = nn.Identity()\n self._features_dim = backbone.out_features\n else:\n self.bottleneck = bottleneck\n assert bottleneck_dim > 0\n self._features_dim = bottleneck_dim\n\n if head is None:\n self.head = nn.Linear(self._features_dim, num_classes)\n else:\n self.head = head\n self.finetune = finetune\n\n @property\n def features_dim(self) -> int:\n \"\"\"The dimension of features before the final `head` layer\"\"\"\n return self._features_dim\n\n def forward(self, x: torch.Tensor) -> Tuple[torch.Tensor, torch.Tensor]:\n \"\"\"\"\"\"\n f = self.pool_layer(self.backbone(x))\n f = self.bottleneck(f)\n predictions = self.head(f)\n if self.training:\n return predictions, f\n else:\n return predictions\n\n def get_parameters(self, base_lr=1.0) -> List[Dict]:\n \"\"\"A parameter list which decides optimization hyper-parameters,\n such as the relative learning rate of each layer\n \"\"\"\n params = [\n {\"params\": self.backbone.parameters(), \"lr\": 0.1 * base_lr if self.finetune else 1.0 * base_lr},\n {\"params\": self.bottleneck.parameters(), \"lr\": 1.0 * base_lr},\n {\"params\": self.head.parameters(), \"lr\": 1.0 * base_lr},\n ]\n\n return params\n\n\nclass ImageClassifier(Classifier):\n pass\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/common/modules/regressor.py",
"content": "\"\"\"\n@author: Junguang Jiang\n@contact: JiangJunguang1123@outlook.com\n\"\"\"\nfrom typing import Tuple, Optional, List, Dict\nimport torch.nn as nn\nimport torch\n\n__all__ = ['Regressor']\n\n\nclass Regressor(nn.Module):\n \"\"\"A generic Regressor class for domain adaptation.\n\n Args:\n backbone (torch.nn.Module): Any backbone to extract 2-d features from data\n num_factors (int): Number of factors\n bottleneck (torch.nn.Module, optional): Any bottleneck layer. Use no bottleneck by default\n bottleneck_dim (int, optional): Feature dimension of the bottleneck layer. Default: -1\n head (torch.nn.Module, optional): Any classifier head. Use `nn.Linear` by default\n finetune (bool): Whether finetune the classifier or train from scratch. Default: True\n\n .. note::\n The learning rate of this regressor is set 10 times to that of the feature extractor for better accuracy\n by default. If you have other optimization strategies, please over-ride :meth:`~Regressor.get_parameters`.\n\n Inputs:\n - x (tensor): input data fed to `backbone`\n\n Outputs:\n - predictions: regressor's predictions\n - features: features after `bottleneck` layer and before `head` layer\n\n Shape:\n - Inputs: (minibatch, *) where * means, any number of additional dimensions\n - predictions: (minibatch, `num_factors`)\n - features: (minibatch, `features_dim`)\n\n \"\"\"\n\n def __init__(self, backbone: nn.Module, num_factors: int, bottleneck: Optional[nn.Module] = None,\n bottleneck_dim=-1, head: Optional[nn.Module] = None, finetune=True):\n super(Regressor, self).__init__()\n self.backbone = backbone\n self.num_factors = num_factors\n if bottleneck is None:\n feature_dim = backbone.out_features\n self.bottleneck = nn.Sequential(\n nn.Conv2d(feature_dim, feature_dim, kernel_size=3, stride=1, padding=1),\n nn.BatchNorm2d(feature_dim, feature_dim),\n nn.ReLU(),\n nn.AdaptiveAvgPool2d(output_size=(1, 1)),\n nn.Flatten()\n )\n self._features_dim = feature_dim\n else:\n self.bottleneck = bottleneck\n assert bottleneck_dim > 0\n self._features_dim = bottleneck_dim\n\n if head is None:\n self.head = nn.Sequential(\n nn.Linear(self._features_dim, num_factors),\n nn.Sigmoid()\n )\n else:\n self.head = head\n self.finetune = finetune\n\n @property\n def features_dim(self) -> int:\n \"\"\"The dimension of features before the final `head` layer\"\"\"\n return self._features_dim\n\n def forward(self, x: torch.Tensor) -> Tuple[torch.Tensor, torch.Tensor]:\n \"\"\"\"\"\"\n f = self.backbone(x)\n f = self.bottleneck(f)\n predictions = self.head(f)\n if self.training:\n return predictions, f\n else:\n return predictions\n\n def get_parameters(self, base_lr=1.0) -> List[Dict]:\n \"\"\"A parameter list which decides optimization hyper-parameters,\n such as the relative learning rate of each layer\n \"\"\"\n params = [\n {\"params\": self.backbone.parameters(), \"lr\": 0.1 * base_lr if self.finetune else 1.0 * base_lr},\n {\"params\": self.bottleneck.parameters(), \"lr\": 1.0 * base_lr},\n {\"params\": self.head.parameters(), \"lr\": 1.0 * base_lr},\n ]\n\n return params\n\n\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/common/utils/__init__.py",
"content": "from .logger import CompleteLogger\nfrom .meter import *\nfrom .data import ForeverDataIterator\n\n__all__ = ['metric', 'analysis', 'meter', 'data', 'logger']"
},
{
"path": "domain_generalization/Transfer-Learning-Library/common/utils/analysis/__init__.py",
"content": "import torch\nfrom torch.utils.data import DataLoader\nimport torch.nn as nn\nimport tqdm\n\n\ndef collect_feature(data_loader: DataLoader, feature_extractor: nn.Module,\n device: torch.device, max_num_features=None) -> torch.Tensor:\n \"\"\"\n Fetch data from `data_loader`, and then use `feature_extractor` to collect features\n\n Args:\n data_loader (torch.utils.data.DataLoader): Data loader.\n feature_extractor (torch.nn.Module): A feature extractor.\n device (torch.device)\n max_num_features (int): The max number of features to return\n\n Returns:\n Features in shape (min(len(data_loader), max_num_features * mini-batch size), :math:`|\\mathcal{F}|`).\n \"\"\"\n feature_extractor.eval()\n all_features = []\n with torch.no_grad():\n for i, (images, target) in enumerate(tqdm.tqdm(data_loader)):\n if max_num_features is not None and i >= max_num_features:\n break\n images = images.to(device)\n feature = feature_extractor(images).cpu()\n all_features.append(feature)\n return torch.cat(all_features, dim=0)\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/common/utils/analysis/a_distance.py",
"content": "\"\"\"\n@author: Junguang Jiang\n@contact: JiangJunguang1123@outlook.com\n\"\"\"\nfrom torch.utils.data import TensorDataset\nimport torch\nimport torch.nn as nn\nimport torch.nn.functional as F\nfrom torch.utils.data import DataLoader\nfrom torch.optim import SGD\nfrom ..meter import AverageMeter\nfrom ..metric import binary_accuracy\n\n\nclass ANet(nn.Module):\n def __init__(self, in_feature):\n super(ANet, self).__init__()\n self.layer = nn.Linear(in_feature, 1)\n self.sigmoid = nn.Sigmoid()\n\n def forward(self, x):\n x = self.layer(x)\n x = self.sigmoid(x)\n return x\n\n\ndef calculate(source_feature: torch.Tensor, target_feature: torch.Tensor,\n device, progress=True, training_epochs=10):\n \"\"\"\n Calculate the :math:`\\mathcal{A}`-distance, which is a measure for distribution discrepancy.\n\n The definition is :math:`dist_\\mathcal{A} = 2 (1-2\\epsilon)`, where :math:`\\epsilon` is the\n test error of a classifier trained to discriminate the source from the target.\n\n Args:\n source_feature (tensor): features from source domain in shape :math:`(minibatch, F)`\n target_feature (tensor): features from target domain in shape :math:`(minibatch, F)`\n device (torch.device)\n progress (bool): if True, displays a the progress of training A-Net\n training_epochs (int): the number of epochs when training the classifier\n\n Returns:\n :math:`\\mathcal{A}`-distance\n \"\"\"\n source_label = torch.ones((source_feature.shape[0], 1))\n target_label = torch.zeros((target_feature.shape[0], 1))\n feature = torch.cat([source_feature, target_feature], dim=0)\n label = torch.cat([source_label, target_label], dim=0)\n\n dataset = TensorDataset(feature, label)\n length = len(dataset)\n train_size = int(0.8 * length)\n val_size = length - train_size\n train_set, val_set = torch.utils.data.random_split(dataset, [train_size, val_size])\n train_loader = DataLoader(train_set, batch_size=2, shuffle=True)\n val_loader = DataLoader(val_set, batch_size=8, shuffle=False)\n\n anet = ANet(feature.shape[1]).to(device)\n optimizer = SGD(anet.parameters(), lr=0.01)\n a_distance = 2.0\n for epoch in range(training_epochs):\n anet.train()\n for (x, label) in train_loader:\n x = x.to(device)\n label = label.to(device)\n anet.zero_grad()\n y = anet(x)\n loss = F.binary_cross_entropy(y, label)\n loss.backward()\n optimizer.step()\n\n anet.eval()\n meter = AverageMeter(\"accuracy\", \":4.2f\")\n with torch.no_grad():\n for (x, label) in val_loader:\n x = x.to(device)\n label = label.to(device)\n y = anet(x)\n acc = binary_accuracy(y, label)\n meter.update(acc, x.shape[0])\n error = 1 - meter.avg / 100\n a_distance = 2 * (1 - 2 * error)\n if progress:\n print(\"epoch {} accuracy: {} A-dist: {}\".format(epoch, meter.avg, a_distance))\n\n return a_distance\n\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/common/utils/analysis/tsne.py",
"content": "\"\"\"\n@author: Junguang Jiang\n@contact: JiangJunguang1123@outlook.com\n\"\"\"\nimport torch\nimport matplotlib\n\nmatplotlib.use('Agg')\nfrom sklearn.manifold import TSNE\nimport numpy as np\nimport matplotlib.pyplot as plt\nimport matplotlib.colors as col\n\n\ndef visualize(source_feature: torch.Tensor, target_feature: torch.Tensor,\n filename: str, source_color='r', target_color='b'):\n \"\"\"\n Visualize features from different domains using t-SNE.\n\n Args:\n source_feature (tensor): features from source domain in shape :math:`(minibatch, F)`\n target_feature (tensor): features from target domain in shape :math:`(minibatch, F)`\n filename (str): the file name to save t-SNE\n source_color (str): the color of the source features. Default: 'r'\n target_color (str): the color of the target features. Default: 'b'\n\n \"\"\"\n source_feature = source_feature.numpy()\n target_feature = target_feature.numpy()\n features = np.concatenate([source_feature, target_feature], axis=0)\n\n # map features to 2-d using TSNE\n X_tsne = TSNE(n_components=2, random_state=33).fit_transform(features)\n\n # domain labels, 1 represents source while 0 represents target\n domains = np.concatenate((np.ones(len(source_feature)), np.zeros(len(target_feature))))\n\n # visualize using matplotlib\n fig, ax = plt.subplots(figsize=(10, 10))\n ax.spines['top'].set_visible(False)\n ax.spines['right'].set_visible(False)\n ax.spines['bottom'].set_visible(False)\n ax.spines['left'].set_visible(False)\n plt.scatter(X_tsne[:, 0], X_tsne[:, 1], c=domains, cmap=col.ListedColormap([target_color, source_color]), s=20)\n plt.xticks([])\n plt.yticks([])\n plt.savefig(filename)\n"
},
{
"path": "domain_generalization/Transfer-Learning-Library/common/utils/data.py",
"content": "\"\"\"\n@author: Junguang Jiang\n@contact: JiangJunguang1123@outlook.com\n\"\"\"\nimport itertools\nimport random\nimport numpy as np\n\nimport torch\nfrom torch.utils.data import Sampler\nfrom torch.utils.data import DataLoader, Dataset\nfrom typing import TypeVar, Iterable, Dict, List\n\nT_co = TypeVar('T_co', covariant=True)\nT = TypeVar('T')\n\n\ndef send_to_device(tensor, device):\n \"\"\"\n Recursively sends the elements in a nested list/tuple/dictionary of tensors to a given device.\n\n Args:\n tensor (nested list/tuple/dictionary of :obj:`torch.Tensor`):\n The data to send to a given device.\n device (:obj:`torch.device`):\n The device to send the data to\n\n Returns:\n The same data structure as :obj:`tensor` with all tensors sent to the proper device.\n \"\"\"\n if isinstance(tensor, (list, tuple)):\n return type(tensor)(send_to_device(t, device) for t in tensor)\n elif isinstance(tensor, dict):\n return type(tensor)({k: send_to_device(v, device) for k, v in tensor.items()})\n elif not hasattr(tensor, \"to\"):\n return tensor\n return tensor.to(device)\n\n\nclass ForeverDataIterator:\n r\"\"\"A data iterator that will never stop producing data\"\"\"\n\n def __init__(self, data_loader: DataLoader, device=None):\n self.data_loader = data_loader\n self.iter = iter(self.data_loader)\n self.device = device\n\n def __next__(self):\n try:\n data = next(self.iter)\n if self.device is not None:\n data = send_to_device(data, self.device)\n except StopIteration:\n self.iter = iter(self.data_loader)\n data = next(self.iter)\n if self.device is not None:\n data = send_to_device(data, self.device)\n return data\n\n def __len__(self):\n return len(self.data_loader)\n\n\nclass RandomMultipleGallerySampler(Sampler):\n r\"\"\"Sampler from `In defense of the Triplet Loss for Person Re-Identification\n (ICCV 2017)