Here, we provide the pretrained MoCo-V3 model corresponding to [this strategy](https://github.com/zhihou7/BatchFormer/blob/cc7a060c6395cb1dc8a7c577df77f52a44deaf9d/moco-v3/moco/builder.py#L139).
#### Domain Generalization
##### ResNet-18
PACS
VLCS
OfficeHome
Terra
SWAD[5]
82.9
76.3
62.1
42.1
+BatchFormer
83.7
76.9
64.3
44.8
#### Compositional Zero-Shot Learning
MIT-States(AUC)
MIT-States(HM)
UT-Zap50K(AUC)
UT-Zap50K(HM)
C-GQA(AUC)
C-GQA(HM)
CGE*[6]
6.3
20.0
31.5
46.5
3.7
14.9
+BatchFormer
6.7
20.6
34.6
49.0
3.8
15.5
#### Few-Shot Learning
Experiments on CUB.
Unseen
Seen
Harmonic mean
CUB[7]*
67.5
65.1
66.3
+BatchFormer
68.2
65.8
67.0
#### Image Classification (V2)
Top-1
Top-5
DeiT-T
72.2
91.1
+BatchFormerV2
72.7
91.5
DeiT-S
79.8
95.0
+BatchFormerV2
80.4
95.2
DeiT-B
81.7
95.5
+BatchFormerV2
82.2
95.8
#### Reference
1. Long-tailed recognition by routing diverse distribution-aware experts. In ICLR, 2021
2. Parametric contrastive learning. In ICCV, 2021
3. Improved baselines with momentum contrastive learning.
4. An empirical study of training self-supervised vision transformers. In CVPR, 2021
5. Domain generalization by seeking flat minima. In NeurIPS, 2021.
6. Learning graph embeddings for compositional zero-shot learning. In CVPR, 2021
7. Contrastive learning based hybrid networks for long- tailed image classification. In CVPR, 2021
## PyTorch Code
The proposed BatchFormer can be implemented with a few lines as follows,
def BatchFormer(x, y, encoder, is_training):
# x: input features with the shape [N, C]
# encoder: TransformerEncoderLayer(C,4,C,0.5)
if not is_training:
return x, y
pre_x = x
x = encoder(x.unsqueeze(1)).squeeze(1)
x = torch.cat([pre_x, x], dim=0)
y = torch.cat([y, y], dim=0)
return x, y
## Citation
If you find this repository helpful, please consider cite:
@inproceedings{hou2022batch,
title={BatchFormer: Learning to Explore Sample Relationships for Robust Representation Learning},
author={Hou, Zhi and Yu, Baosheng and Tao, Dacheng},
booktitle={CVPR},
year={2022}
}
@article{hou2022batchformerv2,
title={BatchFormerV2: Exploring Sample Relationships for Dense Representation Learning},
author={Hou, Zhi and Yu, Baosheng and Wang, Chaoyue and Zhan, Yibing and Tao, Dacheng},
journal={arXiv preprint arXiv:2204.01254},
year={2022}
}
Feel free to contact "zhou9878 at uni dot sydney dot edu dot au" if you have any questions.
================================================
FILE: batchformer-v2/deformable-detr/LICENSE
================================================
Copyright (c) 2020 SenseTime. All Rights Reserved.
Apache License
Version 2.0, January 2004
http://www.apache.org/licenses/
TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
1. Definitions.
"License" shall mean the terms and conditions for use, reproduction,
and distribution as defined by Sections 1 through 9 of this document.
"Licensor" shall mean the copyright owner or entity authorized by
the copyright owner that is granting the License.
"Legal Entity" shall mean the union of the acting entity and all
other entities that control, are controlled by, or are under common
control with that entity. For the purposes of this definition,
"control" means (i) the power, direct or indirect, to cause the
direction or management of such entity, whether by contract or
otherwise, or (ii) ownership of fifty percent (50%) or more of the
outstanding shares, or (iii) beneficial ownership of such entity.
"You" (or "Your") shall mean an individual or Legal Entity
exercising permissions granted by this License.
"Source" form shall mean the preferred form for making modifications,
including but not limited to software source code, documentation
source, and configuration files.
"Object" form shall mean any form resulting from mechanical
transformation or translation of a Source form, including but
not limited to compiled object code, generated documentation,
and conversions to other media types.
"Work" shall mean the work of authorship, whether in Source or
Object form, made available under the License, as indicated by a
copyright notice that is included in or attached to the work
(an example is provided in the Appendix below).
"Derivative Works" shall mean any work, whether in Source or Object
form, that is based on (or derived from) the Work and for which the
editorial revisions, annotations, elaborations, or other modifications
represent, as a whole, an original work of authorship. For the purposes
of this License, Derivative Works shall not include works that remain
separable from, or merely link (or bind by name) to the interfaces of,
the Work and Derivative Works thereof.
"Contribution" shall mean any work of authorship, including
the original version of the Work and any modifications or additions
to that Work or Derivative Works thereof, that is intentionally
submitted to Licensor for inclusion in the Work by the copyright owner
or by an individual or Legal Entity authorized to submit on behalf of
the copyright owner. For the purposes of this definition, "submitted"
means any form of electronic, verbal, or written communication sent
to the Licensor or its representatives, including but not limited to
communication on electronic mailing lists, source code control systems,
and issue tracking systems that are managed by, or on behalf of, the
Licensor for the purpose of discussing and improving the Work, but
excluding communication that is conspicuously marked or otherwise
designated in writing by the copyright owner as "Not a Contribution."
"Contributor" shall mean Licensor and any individual or Legal Entity
on behalf of whom a Contribution has been received by Licensor and
subsequently incorporated within the Work.
2. Grant of Copyright License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
copyright license to reproduce, prepare Derivative Works of,
publicly display, publicly perform, sublicense, and distribute the
Work and such Derivative Works in Source or Object form.
3. Grant of Patent License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
(except as stated in this section) patent license to make, have made,
use, offer to sell, sell, import, and otherwise transfer the Work,
where such license applies only to those patent claims licensable
by such Contributor that are necessarily infringed by their
Contribution(s) alone or by combination of their Contribution(s)
with the Work to which such Contribution(s) was submitted. If You
institute patent litigation against any entity (including a
cross-claim or counterclaim in a lawsuit) alleging that the Work
or a Contribution incorporated within the Work constitutes direct
or contributory patent infringement, then any patent licenses
granted to You under this License for that Work shall terminate
as of the date such litigation is filed.
4. Redistribution. You may reproduce and distribute copies of the
Work or Derivative Works thereof in any medium, with or without
modifications, and in Source or Object form, provided that You
meet the following conditions:
(a) You must give any other recipients of the Work or
Derivative Works a copy of this License; and
(b) You must cause any modified files to carry prominent notices
stating that You changed the files; and
(c) You must retain, in the Source form of any Derivative Works
that You distribute, all copyright, patent, trademark, and
attribution notices from the Source form of the Work,
excluding those notices that do not pertain to any part of
the Derivative Works; and
(d) If the Work includes a "NOTICE" text file as part of its
distribution, then any Derivative Works that You distribute must
include a readable copy of the attribution notices contained
within such NOTICE file, excluding those notices that do not
pertain to any part of the Derivative Works, in at least one
of the following places: within a NOTICE text file distributed
as part of the Derivative Works; within the Source form or
documentation, if provided along with the Derivative Works; or,
within a display generated by the Derivative Works, if and
wherever such third-party notices normally appear. The contents
of the NOTICE file are for informational purposes only and
do not modify the License. You may add Your own attribution
notices within Derivative Works that You distribute, alongside
or as an addendum to the NOTICE text from the Work, provided
that such additional attribution notices cannot be construed
as modifying the License.
You may add Your own copyright statement to Your modifications and
may provide additional or different license terms and conditions
for use, reproduction, or distribution of Your modifications, or
for any such Derivative Works as a whole, provided Your use,
reproduction, and distribution of the Work otherwise complies with
the conditions stated in this License.
5. Submission of Contributions. Unless You explicitly state otherwise,
any Contribution intentionally submitted for inclusion in the Work
by You to the Licensor shall be under the terms and conditions of
this License, without any additional terms or conditions.
Notwithstanding the above, nothing herein shall supersede or modify
the terms of any separate license agreement you may have executed
with Licensor regarding such Contributions.
6. Trademarks. This License does not grant permission to use the trade
names, trademarks, service marks, or product names of the Licensor,
except as required for reasonable and customary use in describing the
origin of the Work and reproducing the content of the NOTICE file.
7. Disclaimer of Warranty. Unless required by applicable law or
agreed to in writing, Licensor provides the Work (and each
Contributor provides its Contributions) on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
implied, including, without limitation, any warranties or conditions
of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
PARTICULAR PURPOSE. You are solely responsible for determining the
appropriateness of using or redistributing the Work and assume any
risks associated with Your exercise of permissions under this License.
8. Limitation of Liability. In no event and under no legal theory,
whether in tort (including negligence), contract, or otherwise,
unless required by applicable law (such as deliberate and grossly
negligent acts) or agreed to in writing, shall any Contributor be
liable to You for damages, including any direct, indirect, special,
incidental, or consequential damages of any character arising as a
result of this License or out of the use or inability to use the
Work (including but not limited to damages for loss of goodwill,
work stoppage, computer failure or malfunction, or any and all
other commercial damages or losses), even if such Contributor
has been advised of the possibility of such damages.
9. Accepting Warranty or Additional Liability. While redistributing
the Work or Derivative Works thereof, You may choose to offer,
and charge a fee for, acceptance of support, warranty, indemnity,
or other liability obligations and/or rights consistent with this
License. However, in accepting such obligations, You may act only
on Your own behalf and on Your sole responsibility, not on behalf
of any other Contributor, and only if You agree to indemnify,
defend, and hold each Contributor harmless for any liability
incurred by, or claims asserted against, such Contributor by reason
of your accepting any such warranty or additional liability.
END OF TERMS AND CONDITIONS
APPENDIX: How to apply the Apache License to your work.
To apply the Apache License to your work, attach the following
boilerplate notice, with the fields enclosed by brackets "[]"
replaced with your own identifying information. (Don't include
the brackets!) The text should be enclosed in the appropriate
comment syntax for the file format. We also recommend that a
file or class name and description of purpose be included on the
same "printed page" as the copyright notice for easier
identification within third-party archives.
Copyright 2020 SenseTime
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
DETR
Copyright 2020 - present, Facebook, Inc
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
================================================
FILE: batchformer-v2/deformable-detr/README.md
================================================
# BatchFormerV2: Exploring Sample Relationships for Dense Representation Learning
This repository is based on Deformable DETR. One can follow the instructions in Deformable DETR to install Deformable DETR first.
This repository has changed a few lines of the core code based on Deformable DETR. Check the modifications by:
```
diff models/deformable_transformer.py <(curl https://raw.githubusercontent.com/fundamentalvision/Deformable-DETR/main/models/deformable_transformer.py)
```
I mainly changes the two parts of the code:
1. apply batchformer t get two streams, https://github.com/zhihou7/BatchFormer/blob/e22f5ff895b04d10f3aa4e745f9a226f3d9b7641/batchformer-v2/models/deformable_transformer.py#L278
2. set the labels for batchformerv2 stream. https://github.com/zhihou7/BatchFormer/blob/e22f5ff895b04d10f3aa4e745f9a226f3d9b7641/batchformer-v2/engine.py#L43
## Main Results
| Method | AP |
| ----------- | ----------- |
| Deformable DETR | 43.8 |
| + BatchFormerV2 | 45.5 |
## Installation
### Compiling CUDA operators
```bash
cd ./models/ops
sh ./make.sh
# unit test (should see all checking is True)
python test.py
```
## Usage
### Dataset preparation
Please download [COCO 2017 dataset](https://cocodataset.org/) and organize them as following:
```
code_root/
└── data/
└── coco/
├── train2017/
├── val2017/
└── annotations/
├── instances_train2017.json
└── instances_val2017.json
```
### Training
#### Training on single node
For example, the command for training Deformable DETR on 8 GPUs is as following:
```bash
GPUS_PER_NODE=8 ./tools/run_dist_launch.sh 8 ./configs/r50_deformable_detr.sh --batch-size 3 --share_bf 0 --bf 1 --insert_idx 0
```
"--share_bf 0 --bf 1 --insert_idx 0" is parameters for BatchFormerV2
### Evaluation
You can get the config file and pretrained model of Deformable DETR (the link is in "Main Results" session), then run following command to evaluate it on COCO 2017 validation set:
```bash
--resume --eval
```
If you find this repository is helpful, please consider to cite
@inproceedings{hou2022batch,
title={BatchFormer: Learning to Explore Sample Relationships for Robust Representation Learning},
author={Hou, Zhi and Yu, Baosheng and Tao, Dacheng},
booktitle={CVPR},
year={2022}
}
@article{hou2022batchformerv2,
title={BatchFormerV2: Exploring Sample Relationships for Dense Representation Learning},
author={Hou, Zhi and Yu, Baosheng and Wang, Chaoyue and Zhan, Yibing and Tao, Dacheng},
journal={arXiv preprint arXiv:2204.01254},
year={2022}
}
================================================
FILE: batchformer-v2/deformable-detr/benchmark.py
================================================
# ------------------------------------------------------------------------
# Deformable DETR
# Copyright (c) 2020 SenseTime. All Rights Reserved.
# Licensed under the Apache License, Version 2.0 [see LICENSE for details]
# ------------------------------------------------------------------------
"""
Benchmark inference speed of Deformable DETR.
"""
import os
import time
import argparse
import torch
from main import get_args_parser as get_main_args_parser
from models import build_model
from datasets import build_dataset
from util.misc import nested_tensor_from_tensor_list
def get_benckmark_arg_parser():
parser = argparse.ArgumentParser('Benchmark inference speed of Deformable DETR.')
parser.add_argument('--num_iters', type=int, default=300, help='total iters to benchmark speed')
parser.add_argument('--warm_iters', type=int, default=5, help='ignore first several iters that are very slow')
parser.add_argument('--batch_size', type=int, default=1, help='batch size in inference')
parser.add_argument('--resume', type=str, help='load the pre-trained checkpoint')
return parser
@torch.no_grad()
def measure_average_inference_time(model, inputs, num_iters=100, warm_iters=5):
ts = []
for iter_ in range(num_iters):
torch.cuda.synchronize()
t_ = time.perf_counter()
model(inputs)
torch.cuda.synchronize()
t = time.perf_counter() - t_
if iter_ >= warm_iters:
ts.append(t)
print(ts)
return sum(ts) / len(ts)
def benchmark():
args, _ = get_benckmark_arg_parser().parse_known_args()
main_args = get_main_args_parser().parse_args(_)
assert args.warm_iters < args.num_iters and args.num_iters > 0 and args.warm_iters >= 0
assert args.batch_size > 0
assert args.resume is None or os.path.exists(args.resume)
dataset = build_dataset('val', main_args)
model, _, _ = build_model(main_args)
model.cuda()
model.eval()
if args.resume is not None:
ckpt = torch.load(args.resume, map_location=lambda storage, loc: storage)
model.load_state_dict(ckpt['model'])
inputs = nested_tensor_from_tensor_list([dataset.__getitem__(0)[0].cuda() for _ in range(args.batch_size)])
t = measure_average_inference_time(model, inputs, args.num_iters, args.warm_iters)
return 1.0 / t * args.batch_size
if __name__ == '__main__':
fps = benchmark()
print(f'Inference Speed: {fps:.1f} FPS')
================================================
FILE: batchformer-v2/deformable-detr/configs/r101_deformable_detr.sh
================================================
#!/usr/bin/env bash
source /usr/local/anaconda/4.2.0/etc/profile.d/conda.sh
conda activate pyt
set -x
EXP_DIR=exps/r101_deformable_detr
PY_ARGS=${@:1}
python -u main.py \
--output_dir ${EXP_DIR} \
${PY_ARGS}
================================================
FILE: batchformer-v2/deformable-detr/configs/r50_deformable_detr.sh
================================================
#!/usr/bin/env bash
set -x
EXP_DIR=exps/r50_deformable_detr
PY_ARGS=${@:1}
python -u main.py \
--output_dir ${EXP_DIR} \
${PY_ARGS}
================================================
FILE: batchformer-v2/deformable-detr/configs/r50_deformable_detr_plus_iterative_bbox_refinement.sh
================================================
#!/usr/bin/env bash
set -x
EXP_DIR=exps/r50_deformable_detr_plus_iterative_bbox_refinement
PY_ARGS=${@:1}
python -u main.py \
--output_dir ${EXP_DIR} \
--with_box_refine \
${PY_ARGS}
================================================
FILE: batchformer-v2/deformable-detr/configs/r50_deformable_detr_plus_iterative_bbox_refinement_plus_plus_two_stage.sh
================================================
#!/usr/bin/env bash
set -x
EXP_DIR=exps/r50_deformable_detr_plus_iterative_bbox_refinement_plus_plus_two_stage
PY_ARGS=${@:1}
python -u main.py \
--output_dir ${EXP_DIR} \
--with_box_refine \
--two_stage \
${PY_ARGS}
================================================
FILE: batchformer-v2/deformable-detr/configs/r50_deformable_detr_single_scale.sh
================================================
#!/usr/bin/env bash
set -x
EXP_DIR=exps/r50_deformable_detr_single_scale
PY_ARGS=${@:1}
python -u main.py \
--num_feature_levels 1 \
--output_dir ${EXP_DIR} \
${PY_ARGS}
================================================
FILE: batchformer-v2/deformable-detr/configs/r50_deformable_detr_single_scale_dc5.sh
================================================
#!/usr/bin/env bash
set -x
EXP_DIR=exps/r50_deformable_detr_single_scale_dc5
PY_ARGS=${@:1}
python -u main.py \
--num_feature_levels 1 \
--dilation \
--output_dir ${EXP_DIR} \
${PY_ARGS}
================================================
FILE: batchformer-v2/deformable-detr/datasets/__init__.py
================================================
# ------------------------------------------------------------------------
# Deformable DETR
# Copyright (c) 2020 SenseTime. All Rights Reserved.
# Licensed under the Apache License, Version 2.0 [see LICENSE for details]
# ------------------------------------------------------------------------
# Modified from DETR (https://github.com/facebookresearch/detr)
# Copyright (c) Facebook, Inc. and its affiliates. All Rights Reserved
# ------------------------------------------------------------------------
import torch.utils.data
from .torchvision_datasets import CocoDetection
from .coco import build as build_coco
def get_coco_api_from_dataset(dataset):
for _ in range(10):
# if isinstance(dataset, torchvision.datasets.CocoDetection):
# break
if isinstance(dataset, torch.utils.data.Subset):
dataset = dataset.dataset
if isinstance(dataset, CocoDetection):
return dataset.coco
def build_dataset(image_set, args):
if args.dataset_file == 'coco':
return build_coco(image_set, args)
if args.dataset_file == 'coco_panoptic':
# to avoid making panopticapi required for coco
from .coco_panoptic import build as build_coco_panoptic
return build_coco_panoptic(image_set, args)
raise ValueError(f'dataset {args.dataset_file} not supported')
================================================
FILE: batchformer-v2/deformable-detr/datasets/coco.py
================================================
# ------------------------------------------------------------------------
# Deformable DETR
# Copyright (c) 2020 SenseTime. All Rights Reserved.
# Licensed under the Apache License, Version 2.0 [see LICENSE for details]
# ------------------------------------------------------------------------
# Modified from DETR (https://github.com/facebookresearch/detr)
# Copyright (c) Facebook, Inc. and its affiliates. All Rights Reserved
# ------------------------------------------------------------------------
"""
COCO dataset which returns image_id for evaluation.
Mostly copy-paste from https://github.com/pytorch/vision/blob/13b35ff/references/detection/coco_utils.py
"""
from pathlib import Path
import torch
import torch.utils.data
from pycocotools import mask as coco_mask
from .torchvision_datasets import CocoDetection as TvCocoDetection
from util.misc import get_local_rank, get_local_size
import datasets.transforms as T
class CocoDetection(TvCocoDetection):
def __init__(self, img_folder, ann_file, transforms, return_masks, cache_mode=False, local_rank=0, local_size=1):
super(CocoDetection, self).__init__(img_folder, ann_file,
cache_mode=cache_mode, local_rank=local_rank, local_size=local_size)
self._transforms = transforms
self.prepare = ConvertCocoPolysToMask(return_masks)
def __getitem__(self, idx):
img, target = super(CocoDetection, self).__getitem__(idx)
image_id = self.ids[idx]
target = {'image_id': image_id, 'annotations': target}
img, target = self.prepare(img, target)
if self._transforms is not None:
img, target = self._transforms(img, target)
return img, target
def convert_coco_poly_to_mask(segmentations, height, width):
masks = []
for polygons in segmentations:
rles = coco_mask.frPyObjects(polygons, height, width)
mask = coco_mask.decode(rles)
if len(mask.shape) < 3:
mask = mask[..., None]
mask = torch.as_tensor(mask, dtype=torch.uint8)
mask = mask.any(dim=2)
masks.append(mask)
if masks:
masks = torch.stack(masks, dim=0)
else:
masks = torch.zeros((0, height, width), dtype=torch.uint8)
return masks
class ConvertCocoPolysToMask(object):
def __init__(self, return_masks=False):
self.return_masks = return_masks
def __call__(self, image, target):
w, h = image.size
image_id = target["image_id"]
image_id = torch.tensor([image_id])
anno = target["annotations"]
anno = [obj for obj in anno if 'iscrowd' not in obj or obj['iscrowd'] == 0]
boxes = [obj["bbox"] for obj in anno]
# guard against no boxes via resizing
boxes = torch.as_tensor(boxes, dtype=torch.float32).reshape(-1, 4)
boxes[:, 2:] += boxes[:, :2]
boxes[:, 0::2].clamp_(min=0, max=w)
boxes[:, 1::2].clamp_(min=0, max=h)
classes = [obj["category_id"] for obj in anno]
classes = torch.tensor(classes, dtype=torch.int64)
if self.return_masks:
segmentations = [obj["segmentation"] for obj in anno]
masks = convert_coco_poly_to_mask(segmentations, h, w)
keypoints = None
if anno and "keypoints" in anno[0]:
keypoints = [obj["keypoints"] for obj in anno]
keypoints = torch.as_tensor(keypoints, dtype=torch.float32)
num_keypoints = keypoints.shape[0]
if num_keypoints:
keypoints = keypoints.view(num_keypoints, -1, 3)
keep = (boxes[:, 3] > boxes[:, 1]) & (boxes[:, 2] > boxes[:, 0])
boxes = boxes[keep]
classes = classes[keep]
if self.return_masks:
masks = masks[keep]
if keypoints is not None:
keypoints = keypoints[keep]
target = {}
target["boxes"] = boxes
target["labels"] = classes
if self.return_masks:
target["masks"] = masks
target["image_id"] = image_id
if keypoints is not None:
target["keypoints"] = keypoints
# for conversion to coco api
area = torch.tensor([obj["area"] for obj in anno])
iscrowd = torch.tensor([obj["iscrowd"] if "iscrowd" in obj else 0 for obj in anno])
target["area"] = area[keep]
target["iscrowd"] = iscrowd[keep]
target["orig_size"] = torch.as_tensor([int(h), int(w)])
target["size"] = torch.as_tensor([int(h), int(w)])
return image, target
def make_coco_transforms(image_set):
normalize = T.Compose([
T.ToTensor(),
T.Normalize([0.485, 0.456, 0.406], [0.229, 0.224, 0.225])
])
scales = [480, 512, 544, 576, 608, 640, 672, 704, 736, 768, 800]
if image_set == 'train':
return T.Compose([
T.RandomHorizontalFlip(),
T.RandomSelect(
T.RandomResize(scales, max_size=1333),
T.Compose([
T.RandomResize([400, 500, 600]),
T.RandomSizeCrop(384, 600),
T.RandomResize(scales, max_size=1333),
])
),
normalize,
])
if image_set == 'val':
return T.Compose([
T.RandomResize([800], max_size=1333),
normalize,
])
raise ValueError(f'unknown {image_set}')
def build(image_set, args):
root = Path(args.coco_path)
assert root.exists(), f'provided COCO path {root} does not exist'
mode = 'instances'
PATHS = {
"train": (root / "train2017", root / "annotations" / f'{mode}_train2017.json'),
"val": (root / "val2017", root / "annotations" / f'{mode}_val2017.json'),
}
img_folder, ann_file = PATHS[image_set]
dataset = CocoDetection(img_folder, ann_file, transforms=make_coco_transforms(image_set), return_masks=args.masks,
cache_mode=args.cache_mode, local_rank=get_local_rank(), local_size=get_local_size())
return dataset
================================================
FILE: batchformer-v2/deformable-detr/datasets/coco_eval.py
================================================
# ------------------------------------------------------------------------
# Deformable DETR
# Copyright (c) 2020 SenseTime. All Rights Reserved.
# Licensed under the Apache License, Version 2.0 [see LICENSE for details]
# ------------------------------------------------------------------------
# Modified from DETR (https://github.com/facebookresearch/detr)
# Copyright (c) Facebook, Inc. and its affiliates. All Rights Reserved
# ------------------------------------------------------------------------
"""
COCO evaluator that works in distributed mode.
Mostly copy-paste from https://github.com/pytorch/vision/blob/edfd5a7/references/detection/coco_eval.py
The difference is that there is less copy-pasting from pycocotools
in the end of the file, as python3 can suppress prints with contextlib
"""
import os
import contextlib
import copy
import numpy as np
import torch
from pycocotools.cocoeval import COCOeval
from pycocotools.coco import COCO
import pycocotools.mask as mask_util
from util.misc import all_gather
class CocoEvaluator(object):
def __init__(self, coco_gt, iou_types):
assert isinstance(iou_types, (list, tuple))
coco_gt = copy.deepcopy(coco_gt)
self.coco_gt = coco_gt
self.iou_types = iou_types
self.coco_eval = {}
for iou_type in iou_types:
self.coco_eval[iou_type] = COCOeval(coco_gt, iouType=iou_type)
self.img_ids = []
self.eval_imgs = {k: [] for k in iou_types}
def update(self, predictions):
img_ids = list(np.unique(list(predictions.keys())))
self.img_ids.extend(img_ids)
for iou_type in self.iou_types:
results = self.prepare(predictions, iou_type)
# suppress pycocotools prints
with open(os.devnull, 'w') as devnull:
with contextlib.redirect_stdout(devnull):
coco_dt = COCO.loadRes(self.coco_gt, results) if results else COCO()
coco_eval = self.coco_eval[iou_type]
coco_eval.cocoDt = coco_dt
coco_eval.params.imgIds = list(img_ids)
img_ids, eval_imgs = evaluate(coco_eval)
self.eval_imgs[iou_type].append(eval_imgs)
def synchronize_between_processes(self):
for iou_type in self.iou_types:
self.eval_imgs[iou_type] = np.concatenate(self.eval_imgs[iou_type], 2)
create_common_coco_eval(self.coco_eval[iou_type], self.img_ids, self.eval_imgs[iou_type])
def accumulate(self):
for coco_eval in self.coco_eval.values():
coco_eval.accumulate()
def summarize(self):
for iou_type, coco_eval in self.coco_eval.items():
print("IoU metric: {}".format(iou_type))
coco_eval.summarize()
def prepare(self, predictions, iou_type):
if iou_type == "bbox":
return self.prepare_for_coco_detection(predictions)
elif iou_type == "segm":
return self.prepare_for_coco_segmentation(predictions)
elif iou_type == "keypoints":
return self.prepare_for_coco_keypoint(predictions)
else:
raise ValueError("Unknown iou type {}".format(iou_type))
def prepare_for_coco_detection(self, predictions):
coco_results = []
for original_id, prediction in predictions.items():
if len(prediction) == 0:
continue
boxes = prediction["boxes"]
boxes = convert_to_xywh(boxes).tolist()
scores = prediction["scores"].tolist()
labels = prediction["labels"].tolist()
coco_results.extend(
[
{
"image_id": original_id,
"category_id": labels[k],
"bbox": box,
"score": scores[k],
}
for k, box in enumerate(boxes)
]
)
return coco_results
def prepare_for_coco_segmentation(self, predictions):
coco_results = []
for original_id, prediction in predictions.items():
if len(prediction) == 0:
continue
scores = prediction["scores"]
labels = prediction["labels"]
masks = prediction["masks"]
masks = masks > 0.5
scores = prediction["scores"].tolist()
labels = prediction["labels"].tolist()
rles = [
mask_util.encode(np.array(mask[0, :, :, np.newaxis], dtype=np.uint8, order="F"))[0]
for mask in masks
]
for rle in rles:
rle["counts"] = rle["counts"].decode("utf-8")
coco_results.extend(
[
{
"image_id": original_id,
"category_id": labels[k],
"segmentation": rle,
"score": scores[k],
}
for k, rle in enumerate(rles)
]
)
return coco_results
def prepare_for_coco_keypoint(self, predictions):
coco_results = []
for original_id, prediction in predictions.items():
if len(prediction) == 0:
continue
boxes = prediction["boxes"]
boxes = convert_to_xywh(boxes).tolist()
scores = prediction["scores"].tolist()
labels = prediction["labels"].tolist()
keypoints = prediction["keypoints"]
keypoints = keypoints.flatten(start_dim=1).tolist()
coco_results.extend(
[
{
"image_id": original_id,
"category_id": labels[k],
'keypoints': keypoint,
"score": scores[k],
}
for k, keypoint in enumerate(keypoints)
]
)
return coco_results
def convert_to_xywh(boxes):
xmin, ymin, xmax, ymax = boxes.unbind(1)
return torch.stack((xmin, ymin, xmax - xmin, ymax - ymin), dim=1)
def merge(img_ids, eval_imgs):
all_img_ids = all_gather(img_ids)
all_eval_imgs = all_gather(eval_imgs)
merged_img_ids = []
for p in all_img_ids:
merged_img_ids.extend(p)
merged_eval_imgs = []
for p in all_eval_imgs:
merged_eval_imgs.append(p)
merged_img_ids = np.array(merged_img_ids)
merged_eval_imgs = np.concatenate(merged_eval_imgs, 2)
# keep only unique (and in sorted order) images
merged_img_ids, idx = np.unique(merged_img_ids, return_index=True)
merged_eval_imgs = merged_eval_imgs[..., idx]
return merged_img_ids, merged_eval_imgs
def create_common_coco_eval(coco_eval, img_ids, eval_imgs):
img_ids, eval_imgs = merge(img_ids, eval_imgs)
img_ids = list(img_ids)
eval_imgs = list(eval_imgs.flatten())
coco_eval.evalImgs = eval_imgs
coco_eval.params.imgIds = img_ids
coco_eval._paramsEval = copy.deepcopy(coco_eval.params)
#################################################################
# From pycocotools, just removed the prints and fixed
# a Python3 bug about unicode not defined
#################################################################
def evaluate(self):
'''
Run per image evaluation on given images and store results (a list of dict) in self.evalImgs
:return: None
'''
# tic = time.time()
# print('Running per image evaluation...')
p = self.params
# add backward compatibility if useSegm is specified in params
if p.useSegm is not None:
p.iouType = 'segm' if p.useSegm == 1 else 'bbox'
print('useSegm (deprecated) is not None. Running {} evaluation'.format(p.iouType))
# print('Evaluate annotation type *{}*'.format(p.iouType))
p.imgIds = list(np.unique(p.imgIds))
if p.useCats:
p.catIds = list(np.unique(p.catIds))
p.maxDets = sorted(p.maxDets)
self.params = p
self._prepare()
# loop through images, area range, max detection number
catIds = p.catIds if p.useCats else [-1]
if p.iouType == 'segm' or p.iouType == 'bbox':
computeIoU = self.computeIoU
elif p.iouType == 'keypoints':
computeIoU = self.computeOks
self.ious = {
(imgId, catId): computeIoU(imgId, catId)
for imgId in p.imgIds
for catId in catIds}
evaluateImg = self.evaluateImg
maxDet = p.maxDets[-1]
evalImgs = [
evaluateImg(imgId, catId, areaRng, maxDet)
for catId in catIds
for areaRng in p.areaRng
for imgId in p.imgIds
]
# this is NOT in the pycocotools code, but could be done outside
evalImgs = np.asarray(evalImgs).reshape(len(catIds), len(p.areaRng), len(p.imgIds))
self._paramsEval = copy.deepcopy(self.params)
# toc = time.time()
# print('DONE (t={:0.2f}s).'.format(toc-tic))
return p.imgIds, evalImgs
#################################################################
# end of straight copy from pycocotools, just removing the prints
#################################################################
================================================
FILE: batchformer-v2/deformable-detr/datasets/coco_panoptic.py
================================================
# ------------------------------------------------------------------------
# Deformable DETR
# Copyright (c) 2020 SenseTime. All Rights Reserved.
# Licensed under the Apache License, Version 2.0 [see LICENSE for details]
# ------------------------------------------------------------------------
# Modified from DETR (https://github.com/facebookresearch/detr)
# Copyright (c) Facebook, Inc. and its affiliates. All Rights Reserved
# ------------------------------------------------------------------------
import json
from pathlib import Path
import numpy as np
import torch
from PIL import Image
from panopticapi.utils import rgb2id
from util.box_ops import masks_to_boxes
from .coco import make_coco_transforms
class CocoPanoptic:
def __init__(self, img_folder, ann_folder, ann_file, transforms=None, return_masks=True):
with open(ann_file, 'r') as f:
self.coco = json.load(f)
# sort 'images' field so that they are aligned with 'annotations'
# i.e., in alphabetical order
self.coco['images'] = sorted(self.coco['images'], key=lambda x: x['id'])
# sanity check
if "annotations" in self.coco:
for img, ann in zip(self.coco['images'], self.coco['annotations']):
assert img['file_name'][:-4] == ann['file_name'][:-4]
self.img_folder = img_folder
self.ann_folder = ann_folder
self.ann_file = ann_file
self.transforms = transforms
self.return_masks = return_masks
def __getitem__(self, idx):
ann_info = self.coco['annotations'][idx] if "annotations" in self.coco else self.coco['images'][idx]
img_path = Path(self.img_folder) / ann_info['file_name'].replace('.png', '.jpg')
ann_path = Path(self.ann_folder) / ann_info['file_name']
img = Image.open(img_path).convert('RGB')
w, h = img.size
if "segments_info" in ann_info:
masks = np.asarray(Image.open(ann_path), dtype=np.uint32)
masks = rgb2id(masks)
ids = np.array([ann['id'] for ann in ann_info['segments_info']])
masks = masks == ids[:, None, None]
masks = torch.as_tensor(masks, dtype=torch.uint8)
labels = torch.tensor([ann['category_id'] for ann in ann_info['segments_info']], dtype=torch.int64)
target = {}
target['image_id'] = torch.tensor([ann_info['image_id'] if "image_id" in ann_info else ann_info["id"]])
if self.return_masks:
target['masks'] = masks
target['labels'] = labels
target["boxes"] = masks_to_boxes(masks)
target['size'] = torch.as_tensor([int(h), int(w)])
target['orig_size'] = torch.as_tensor([int(h), int(w)])
if "segments_info" in ann_info:
for name in ['iscrowd', 'area']:
target[name] = torch.tensor([ann[name] for ann in ann_info['segments_info']])
if self.transforms is not None:
img, target = self.transforms(img, target)
return img, target
def __len__(self):
return len(self.coco['images'])
def get_height_and_width(self, idx):
img_info = self.coco['images'][idx]
height = img_info['height']
width = img_info['width']
return height, width
def build(image_set, args):
img_folder_root = Path(args.coco_path)
ann_folder_root = Path(args.coco_panoptic_path)
assert img_folder_root.exists(), f'provided COCO path {img_folder_root} does not exist'
assert ann_folder_root.exists(), f'provided COCO path {ann_folder_root} does not exist'
mode = 'panoptic'
PATHS = {
"train": ("train2017", Path("annotations") / f'{mode}_train2017.json'),
"val": ("val2017", Path("annotations") / f'{mode}_val2017.json'),
}
img_folder, ann_file = PATHS[image_set]
img_folder_path = img_folder_root / img_folder
ann_folder = ann_folder_root / f'{mode}_{img_folder}'
ann_file = ann_folder_root / ann_file
dataset = CocoPanoptic(img_folder_path, ann_folder, ann_file,
transforms=make_coco_transforms(image_set), return_masks=args.masks)
return dataset
================================================
FILE: batchformer-v2/deformable-detr/datasets/data_prefetcher.py
================================================
# ------------------------------------------------------------------------
# Deformable DETR
# Copyright (c) 2020 SenseTime. All Rights Reserved.
# Licensed under the Apache License, Version 2.0 [see LICENSE for details]
# ------------------------------------------------------------------------
import torch
def to_cuda(samples, targets, device):
samples = samples.to(device, non_blocking=True)
targets = [{k: v.to(device, non_blocking=True) for k, v in t.items()} for t in targets]
return samples, targets
class data_prefetcher():
def __init__(self, loader, device, prefetch=True):
self.loader = iter(loader)
self.prefetch = prefetch
self.device = device
if prefetch:
self.stream = torch.cuda.Stream()
self.preload()
def preload(self):
try:
self.next_samples, self.next_targets = next(self.loader)
except StopIteration:
self.next_samples = None
self.next_targets = None
return
# if record_stream() doesn't work, another option is to make sure device inputs are created
# on the main stream.
# self.next_input_gpu = torch.empty_like(self.next_input, device='cuda')
# self.next_target_gpu = torch.empty_like(self.next_target, device='cuda')
# Need to make sure the memory allocated for next_* is not still in use by the main stream
# at the time we start copying to next_*:
# self.stream.wait_stream(torch.cuda.current_stream())
with torch.cuda.stream(self.stream):
self.next_samples, self.next_targets = to_cuda(self.next_samples, self.next_targets, self.device)
# more code for the alternative if record_stream() doesn't work:
# copy_ will record the use of the pinned source tensor in this side stream.
# self.next_input_gpu.copy_(self.next_input, non_blocking=True)
# self.next_target_gpu.copy_(self.next_target, non_blocking=True)
# self.next_input = self.next_input_gpu
# self.next_target = self.next_target_gpu
# With Amp, it isn't necessary to manually convert data to half.
# if args.fp16:
# self.next_input = self.next_input.half()
# else:
def next(self):
if self.prefetch:
torch.cuda.current_stream().wait_stream(self.stream)
samples = self.next_samples
targets = self.next_targets
if samples is not None:
samples.record_stream(torch.cuda.current_stream())
if targets is not None:
for t in targets:
for k, v in t.items():
v.record_stream(torch.cuda.current_stream())
self.preload()
else:
try:
samples, targets = next(self.loader)
samples, targets = to_cuda(samples, targets, self.device)
except StopIteration:
samples = None
targets = None
return samples, targets
================================================
FILE: batchformer-v2/deformable-detr/datasets/panoptic_eval.py
================================================
# ------------------------------------------------------------------------
# Deformable DETR
# Copyright (c) 2020 SenseTime. All Rights Reserved.
# Licensed under the Apache License, Version 2.0 [see LICENSE for details]
# ------------------------------------------------------------------------
# Modified from DETR (https://github.com/facebookresearch/detr)
# Copyright (c) Facebook, Inc. and its affiliates. All Rights Reserved
# ------------------------------------------------------------------------
import json
import os
import util.misc as utils
try:
from panopticapi.evaluation import pq_compute
except ImportError:
pass
class PanopticEvaluator(object):
def __init__(self, ann_file, ann_folder, output_dir="panoptic_eval"):
self.gt_json = ann_file
self.gt_folder = ann_folder
if utils.is_main_process():
if not os.path.exists(output_dir):
os.mkdir(output_dir)
self.output_dir = output_dir
self.predictions = []
def update(self, predictions):
for p in predictions:
with open(os.path.join(self.output_dir, p["file_name"]), "wb") as f:
f.write(p.pop("png_string"))
self.predictions += predictions
def synchronize_between_processes(self):
all_predictions = utils.all_gather(self.predictions)
merged_predictions = []
for p in all_predictions:
merged_predictions += p
self.predictions = merged_predictions
def summarize(self):
if utils.is_main_process():
json_data = {"annotations": self.predictions}
predictions_json = os.path.join(self.output_dir, "predictions.json")
with open(predictions_json, "w") as f:
f.write(json.dumps(json_data))
return pq_compute(self.gt_json, predictions_json, gt_folder=self.gt_folder, pred_folder=self.output_dir)
return None
================================================
FILE: batchformer-v2/deformable-detr/datasets/samplers.py
================================================
# ------------------------------------------------------------------------
# Deformable DETR
# Copyright (c) 2020 SenseTime. All Rights Reserved.
# Licensed under the Apache License, Version 2.0 [see LICENSE for details]
# ------------------------------------------------------------------------
# Modified from codes in torch.utils.data.distributed
# ------------------------------------------------------------------------
import os
import math
import torch
import torch.distributed as dist
from torch.utils.data.sampler import Sampler
class DistributedSampler(Sampler):
"""Sampler that restricts data loading to a subset of the dataset.
It is especially useful in conjunction with
:class:`torch.nn.parallel.DistributedDataParallel`. In such case, each
process can pass a DistributedSampler instance as a DataLoader sampler,
and load a subset of the original dataset that is exclusive to it.
.. note::
Dataset is assumed to be of constant size.
Arguments:
dataset: Dataset used for sampling.
num_replicas (optional): Number of processes participating in
distributed training.
rank (optional): Rank of the current process within num_replicas.
"""
def __init__(self, dataset, num_replicas=None, rank=None, local_rank=None, local_size=None, shuffle=True):
if num_replicas is None:
if not dist.is_available():
raise RuntimeError("Requires distributed package to be available")
num_replicas = dist.get_world_size()
if rank is None:
if not dist.is_available():
raise RuntimeError("Requires distributed package to be available")
rank = dist.get_rank()
self.dataset = dataset
self.num_replicas = num_replicas
self.rank = rank
self.epoch = 0
self.num_samples = int(math.ceil(len(self.dataset) * 1.0 / self.num_replicas))
self.total_size = self.num_samples * self.num_replicas
self.shuffle = shuffle
def __iter__(self):
if self.shuffle:
# deterministically shuffle based on epoch
g = torch.Generator()
g.manual_seed(self.epoch)
indices = torch.randperm(len(self.dataset), generator=g).tolist()
else:
indices = torch.arange(len(self.dataset)).tolist()
# add extra samples to make it evenly divisible
indices += indices[: (self.total_size - len(indices))]
assert len(indices) == self.total_size
# subsample
offset = self.num_samples * self.rank
indices = indices[offset : offset + self.num_samples]
assert len(indices) == self.num_samples
return iter(indices)
def __len__(self):
return self.num_samples
def set_epoch(self, epoch):
self.epoch = epoch
class NodeDistributedSampler(Sampler):
"""Sampler that restricts data loading to a subset of the dataset.
It is especially useful in conjunction with
:class:`torch.nn.parallel.DistributedDataParallel`. In such case, each
process can pass a DistributedSampler instance as a DataLoader sampler,
and load a subset of the original dataset that is exclusive to it.
.. note::
Dataset is assumed to be of constant size.
Arguments:
dataset: Dataset used for sampling.
num_replicas (optional): Number of processes participating in
distributed training.
rank (optional): Rank of the current process within num_replicas.
"""
def __init__(self, dataset, num_replicas=None, rank=None, local_rank=None, local_size=None, shuffle=True):
if num_replicas is None:
if not dist.is_available():
raise RuntimeError("Requires distributed package to be available")
num_replicas = dist.get_world_size()
if rank is None:
if not dist.is_available():
raise RuntimeError("Requires distributed package to be available")
rank = dist.get_rank()
if local_rank is None:
local_rank = int(os.environ.get('LOCAL_RANK', 0))
if local_size is None:
local_size = int(os.environ.get('LOCAL_SIZE', 1))
self.dataset = dataset
self.shuffle = shuffle
self.num_replicas = num_replicas
self.num_parts = local_size
self.rank = rank
self.local_rank = local_rank
self.epoch = 0
self.num_samples = int(math.ceil(len(self.dataset) * 1.0 / self.num_replicas))
self.total_size = self.num_samples * self.num_replicas
self.total_size_parts = self.num_samples * self.num_replicas // self.num_parts
def __iter__(self):
if self.shuffle:
# deterministically shuffle based on epoch
g = torch.Generator()
g.manual_seed(self.epoch)
indices = torch.randperm(len(self.dataset), generator=g).tolist()
else:
indices = torch.arange(len(self.dataset)).tolist()
indices = [i for i in indices if i % self.num_parts == self.local_rank]
# add extra samples to make it evenly divisible
indices += indices[:(self.total_size_parts - len(indices))]
assert len(indices) == self.total_size_parts
# subsample
indices = indices[self.rank // self.num_parts:self.total_size_parts:self.num_replicas // self.num_parts]
assert len(indices) == self.num_samples
return iter(indices)
def __len__(self):
return self.num_samples
def set_epoch(self, epoch):
self.epoch = epoch
================================================
FILE: batchformer-v2/deformable-detr/datasets/torchvision_datasets/__init__.py
================================================
# ------------------------------------------------------------------------
# Deformable DETR
# Copyright (c) 2020 SenseTime. All Rights Reserved.
# Licensed under the Apache License, Version 2.0 [see LICENSE for details]
# ------------------------------------------------------------------------
from .coco import CocoDetection
================================================
FILE: batchformer-v2/deformable-detr/datasets/torchvision_datasets/coco.py
================================================
# ------------------------------------------------------------------------
# Deformable DETR
# Copyright (c) 2020 SenseTime. All Rights Reserved.
# Licensed under the Apache License, Version 2.0 [see LICENSE for details]
# ------------------------------------------------------------------------
# Modified from torchvision
# ------------------------------------------------------------------------
"""
Copy-Paste from torchvision, but add utility of caching images on memory
"""
from torchvision.datasets.vision import VisionDataset
from PIL import Image
import os
import os.path
import tqdm
from io import BytesIO
class CocoDetection(VisionDataset):
"""`MS Coco Detection `_ Dataset.
Args:
root (string): Root directory where images are downloaded to.
annFile (string): Path to json annotation file.
transform (callable, optional): A function/transform that takes in an PIL image
and returns a transformed version. E.g, ``transforms.ToTensor``
target_transform (callable, optional): A function/transform that takes in the
target and transforms it.
transforms (callable, optional): A function/transform that takes input sample and its target as entry
and returns a transformed version.
"""
def __init__(self, root, annFile, transform=None, target_transform=None, transforms=None,
cache_mode=False, local_rank=0, local_size=1):
super(CocoDetection, self).__init__(root, transforms, transform, target_transform)
from pycocotools.coco import COCO
self.coco = COCO(annFile)
self.ids = list(sorted(self.coco.imgs.keys()))
self.cache_mode = cache_mode
self.local_rank = local_rank
self.local_size = local_size
if cache_mode:
self.cache = {}
self.cache_images()
def cache_images(self):
self.cache = {}
for index, img_id in zip(tqdm.trange(len(self.ids)), self.ids):
if index % self.local_size != self.local_rank:
continue
path = self.coco.loadImgs(img_id)[0]['file_name']
with open(os.path.join(self.root, path), 'rb') as f:
self.cache[path] = f.read()
def get_image(self, path):
if self.cache_mode:
if path not in self.cache.keys():
with open(os.path.join(self.root, path), 'rb') as f:
self.cache[path] = f.read()
return Image.open(BytesIO(self.cache[path])).convert('RGB')
return Image.open(os.path.join(self.root, path)).convert('RGB')
def __getitem__(self, index):
"""
Args:
index (int): Index
Returns:
tuple: Tuple (image, target). target is the object returned by ``coco.loadAnns``.
"""
coco = self.coco
img_id = self.ids[index]
ann_ids = coco.getAnnIds(imgIds=img_id)
target = coco.loadAnns(ann_ids)
path = coco.loadImgs(img_id)[0]['file_name']
img = self.get_image(path)
if self.transforms is not None:
img, target = self.transforms(img, target)
return img, target
def __len__(self):
return len(self.ids)
================================================
FILE: batchformer-v2/deformable-detr/datasets/transforms.py
================================================
# ------------------------------------------------------------------------
# Deformable DETR
# Copyright (c) 2020 SenseTime. All Rights Reserved.
# Licensed under the Apache License, Version 2.0 [see LICENSE for details]
# ------------------------------------------------------------------------
# Modified from DETR (https://github.com/facebookresearch/detr)
# Copyright (c) Facebook, Inc. and its affiliates. All Rights Reserved
# ------------------------------------------------------------------------
"""
Transforms and data augmentation for both image + bbox.
"""
import random
import PIL
import torch
import torchvision.transforms as T
import torchvision.transforms.functional as F
from util.box_ops import box_xyxy_to_cxcywh
from util.misc import interpolate
def crop(image, target, region):
cropped_image = F.crop(image, *region)
target = target.copy()
i, j, h, w = region
# should we do something wrt the original size?
target["size"] = torch.tensor([h, w])
fields = ["labels", "area", "iscrowd"]
if "boxes" in target:
boxes = target["boxes"]
max_size = torch.as_tensor([w, h], dtype=torch.float32)
cropped_boxes = boxes - torch.as_tensor([j, i, j, i])
cropped_boxes = torch.min(cropped_boxes.reshape(-1, 2, 2), max_size)
cropped_boxes = cropped_boxes.clamp(min=0)
area = (cropped_boxes[:, 1, :] - cropped_boxes[:, 0, :]).prod(dim=1)
target["boxes"] = cropped_boxes.reshape(-1, 4)
target["area"] = area
fields.append("boxes")
if "masks" in target:
# FIXME should we update the area here if there are no boxes?
target['masks'] = target['masks'][:, i:i + h, j:j + w]
fields.append("masks")
# remove elements for which the boxes or masks that have zero area
if "boxes" in target or "masks" in target:
# favor boxes selection when defining which elements to keep
# this is compatible with previous implementation
if "boxes" in target:
cropped_boxes = target['boxes'].reshape(-1, 2, 2)
keep = torch.all(cropped_boxes[:, 1, :] > cropped_boxes[:, 0, :], dim=1)
else:
keep = target['masks'].flatten(1).any(1)
for field in fields:
target[field] = target[field][keep]
return cropped_image, target
def hflip(image, target):
flipped_image = F.hflip(image)
w, h = image.size
target = target.copy()
if "boxes" in target:
boxes = target["boxes"]
boxes = boxes[:, [2, 1, 0, 3]] * torch.as_tensor([-1, 1, -1, 1]) + torch.as_tensor([w, 0, w, 0])
target["boxes"] = boxes
if "masks" in target:
target['masks'] = target['masks'].flip(-1)
return flipped_image, target
def resize(image, target, size, max_size=None):
# size can be min_size (scalar) or (w, h) tuple
def get_size_with_aspect_ratio(image_size, size, max_size=None):
w, h = image_size
if max_size is not None:
min_original_size = float(min((w, h)))
max_original_size = float(max((w, h)))
if max_original_size / min_original_size * size > max_size:
size = int(round(max_size * min_original_size / max_original_size))
if (w <= h and w == size) or (h <= w and h == size):
return (h, w)
if w < h:
ow = size
oh = int(size * h / w)
else:
oh = size
ow = int(size * w / h)
return (oh, ow)
def get_size(image_size, size, max_size=None):
if isinstance(size, (list, tuple)):
return size[::-1]
else:
return get_size_with_aspect_ratio(image_size, size, max_size)
size = get_size(image.size, size, max_size)
rescaled_image = F.resize(image, size)
if target is None:
return rescaled_image, None
ratios = tuple(float(s) / float(s_orig) for s, s_orig in zip(rescaled_image.size, image.size))
ratio_width, ratio_height = ratios
target = target.copy()
if "boxes" in target:
boxes = target["boxes"]
scaled_boxes = boxes * torch.as_tensor([ratio_width, ratio_height, ratio_width, ratio_height])
target["boxes"] = scaled_boxes
if "area" in target:
area = target["area"]
scaled_area = area * (ratio_width * ratio_height)
target["area"] = scaled_area
h, w = size
target["size"] = torch.tensor([h, w])
if "masks" in target:
target['masks'] = interpolate(
target['masks'][:, None].float(), size, mode="nearest")[:, 0] > 0.5
return rescaled_image, target
def pad(image, target, padding):
# assumes that we only pad on the bottom right corners
padded_image = F.pad(image, (0, 0, padding[0], padding[1]))
if target is None:
return padded_image, None
target = target.copy()
# should we do something wrt the original size?
target["size"] = torch.tensor(padded_image[::-1])
if "masks" in target:
target['masks'] = torch.nn.functional.pad(target['masks'], (0, padding[0], 0, padding[1]))
return padded_image, target
class RandomCrop(object):
def __init__(self, size):
self.size = size
def __call__(self, img, target):
region = T.RandomCrop.get_params(img, self.size)
return crop(img, target, region)
class RandomSizeCrop(object):
def __init__(self, min_size: int, max_size: int):
self.min_size = min_size
self.max_size = max_size
def __call__(self, img: PIL.Image.Image, target: dict):
w = random.randint(self.min_size, min(img.width, self.max_size))
h = random.randint(self.min_size, min(img.height, self.max_size))
region = T.RandomCrop.get_params(img, [h, w])
return crop(img, target, region)
class CenterCrop(object):
def __init__(self, size):
self.size = size
def __call__(self, img, target):
image_width, image_height = img.size
crop_height, crop_width = self.size
crop_top = int(round((image_height - crop_height) / 2.))
crop_left = int(round((image_width - crop_width) / 2.))
return crop(img, target, (crop_top, crop_left, crop_height, crop_width))
class RandomHorizontalFlip(object):
def __init__(self, p=0.5):
self.p = p
def __call__(self, img, target):
if random.random() < self.p:
return hflip(img, target)
return img, target
class RandomResize(object):
def __init__(self, sizes, max_size=None):
assert isinstance(sizes, (list, tuple))
self.sizes = sizes
self.max_size = max_size
def __call__(self, img, target=None):
size = random.choice(self.sizes)
return resize(img, target, size, self.max_size)
class RandomPad(object):
def __init__(self, max_pad):
self.max_pad = max_pad
def __call__(self, img, target):
pad_x = random.randint(0, self.max_pad)
pad_y = random.randint(0, self.max_pad)
return pad(img, target, (pad_x, pad_y))
class RandomSelect(object):
"""
Randomly selects between transforms1 and transforms2,
with probability p for transforms1 and (1 - p) for transforms2
"""
def __init__(self, transforms1, transforms2, p=0.5):
self.transforms1 = transforms1
self.transforms2 = transforms2
self.p = p
def __call__(self, img, target):
if random.random() < self.p:
return self.transforms1(img, target)
return self.transforms2(img, target)
class ToTensor(object):
def __call__(self, img, target):
return F.to_tensor(img), target
class RandomErasing(object):
def __init__(self, *args, **kwargs):
self.eraser = T.RandomErasing(*args, **kwargs)
def __call__(self, img, target):
return self.eraser(img), target
class Normalize(object):
def __init__(self, mean, std):
self.mean = mean
self.std = std
def __call__(self, image, target=None):
image = F.normalize(image, mean=self.mean, std=self.std)
if target is None:
return image, None
target = target.copy()
h, w = image.shape[-2:]
if "boxes" in target:
boxes = target["boxes"]
boxes = box_xyxy_to_cxcywh(boxes)
boxes = boxes / torch.tensor([w, h, w, h], dtype=torch.float32)
target["boxes"] = boxes
return image, target
class Compose(object):
def __init__(self, transforms):
self.transforms = transforms
def __call__(self, image, target):
for t in self.transforms:
image, target = t(image, target)
return image, target
def __repr__(self):
format_string = self.__class__.__name__ + "("
for t in self.transforms:
format_string += "\n"
format_string += " {0}".format(t)
format_string += "\n)"
return format_string
================================================
FILE: batchformer-v2/deformable-detr/docs/changelog.md
================================================
## Changelog
**[2020.12.07]** Fix a bug of sampling offset normalization (see [this issue](https://github.com/fundamentalvision/Deformable-DETR/issues/6)) in the MSDeformAttn module. The final accuracy on COCO is slightly improved. Code and pre-trained models have been updated. This bug only occurs in this released version but not in the original implementation used in our paper.
================================================
FILE: batchformer-v2/deformable-detr/engine.py
================================================
# ------------------------------------------------------------------------
# Deformable DETR
# Copyright (c) 2020 SenseTime. All Rights Reserved.
# Licensed under the Apache License, Version 2.0 [see LICENSE for details]
# ------------------------------------------------------------------------
# Modified from DETR (https://github.com/facebookresearch/detr)
# Copyright (c) Facebook, Inc. and its affiliates. All Rights Reserved
# ------------------------------------------------------------------------
"""
Train and eval functions used in main.py
"""
import math
import os
import sys
from typing import Iterable
import torch
import util.misc as utils
from datasets.coco_eval import CocoEvaluator
from datasets.panoptic_eval import PanopticEvaluator
from datasets.data_prefetcher import data_prefetcher
def train_one_epoch(model: torch.nn.Module, criterion: torch.nn.Module,
data_loader: Iterable, optimizer: torch.optim.Optimizer,
device: torch.device, epoch: int, max_norm: float = 0):
model.train()
criterion.train()
metric_logger = utils.MetricLogger(delimiter=" ")
metric_logger.add_meter('lr', utils.SmoothedValue(window_size=1, fmt='{value:.6f}'))
metric_logger.add_meter('class_error', utils.SmoothedValue(window_size=1, fmt='{value:.2f}'))
metric_logger.add_meter('grad_norm', utils.SmoothedValue(window_size=1, fmt='{value:.2f}'))
header = 'Epoch: [{}]'.format(epoch)
print_freq = 10
prefetcher = data_prefetcher(data_loader, device, prefetch=True)
samples, targets = prefetcher.next()
# for samples, targets in metric_logger.log_every(data_loader, print_freq, header):
for _ in metric_logger.log_every(range(len(data_loader)), print_freq, header):
outputs = model(samples)
if len(outputs['pred_logits']) > len(targets): # Here, we need to set the labels for the batchformerv2 stream/batch
targets = targets + targets # targets is a list
loss_dict = criterion(outputs, targets)
weight_dict = criterion.weight_dict
losses = sum(loss_dict[k] * weight_dict[k] for k in loss_dict.keys() if k in weight_dict)
# reduce losses over all GPUs for logging purposes
loss_dict_reduced = utils.reduce_dict(loss_dict)
loss_dict_reduced_unscaled = {f'{k}_unscaled': v
for k, v in loss_dict_reduced.items()}
loss_dict_reduced_scaled = {k: v * weight_dict[k]
for k, v in loss_dict_reduced.items() if k in weight_dict}
losses_reduced_scaled = sum(loss_dict_reduced_scaled.values())
loss_value = losses_reduced_scaled.item()
if not math.isfinite(loss_value):
print("Loss is {}, stopping training".format(loss_value))
print(loss_dict_reduced)
sys.exit(1)
optimizer.zero_grad()
losses.backward()
if max_norm > 0:
grad_total_norm = torch.nn.utils.clip_grad_norm_(model.parameters(), max_norm)
else:
grad_total_norm = utils.get_total_grad_norm(model.parameters(), max_norm)
optimizer.step()
metric_logger.update(loss=loss_value, **loss_dict_reduced_scaled, **loss_dict_reduced_unscaled)
metric_logger.update(class_error=loss_dict_reduced['class_error'])
metric_logger.update(lr=optimizer.param_groups[0]["lr"])
metric_logger.update(grad_norm=grad_total_norm)
samples, targets = prefetcher.next()
# gather the stats from all processes
metric_logger.synchronize_between_processes()
print("Averaged stats:", metric_logger)
return {k: meter.global_avg for k, meter in metric_logger.meters.items()}
@torch.no_grad()
def evaluate(model, criterion, postprocessors, data_loader, base_ds, device, output_dir):
model.eval()
criterion.eval()
metric_logger = utils.MetricLogger(delimiter=" ")
metric_logger.add_meter('class_error', utils.SmoothedValue(window_size=1, fmt='{value:.2f}'))
header = 'Test:'
iou_types = tuple(k for k in ('segm', 'bbox') if k in postprocessors.keys())
coco_evaluator = CocoEvaluator(base_ds, iou_types)
# coco_evaluator.coco_eval[iou_types[0]].params.iouThrs = [0, 0.1, 0.5, 0.75]
panoptic_evaluator = None
if 'panoptic' in postprocessors.keys():
panoptic_evaluator = PanopticEvaluator(
data_loader.dataset.ann_file,
data_loader.dataset.ann_folder,
output_dir=os.path.join(output_dir, "panoptic_eval"),
)
for samples, targets in metric_logger.log_every(data_loader, 10, header):
samples = samples.to(device)
targets = [{k: v.to(device) for k, v in t.items()} for t in targets]
outputs = model(samples)
loss_dict = criterion(outputs, targets)
weight_dict = criterion.weight_dict
# reduce losses over all GPUs for logging purposes
loss_dict_reduced = utils.reduce_dict(loss_dict)
loss_dict_reduced_scaled = {k: v * weight_dict[k]
for k, v in loss_dict_reduced.items() if k in weight_dict}
loss_dict_reduced_unscaled = {f'{k}_unscaled': v
for k, v in loss_dict_reduced.items()}
metric_logger.update(loss=sum(loss_dict_reduced_scaled.values()),
**loss_dict_reduced_scaled,
**loss_dict_reduced_unscaled)
metric_logger.update(class_error=loss_dict_reduced['class_error'])
orig_target_sizes = torch.stack([t["orig_size"] for t in targets], dim=0)
results = postprocessors['bbox'](outputs, orig_target_sizes)
if 'segm' in postprocessors.keys():
target_sizes = torch.stack([t["size"] for t in targets], dim=0)
results = postprocessors['segm'](results, outputs, orig_target_sizes, target_sizes)
res = {target['image_id'].item(): output for target, output in zip(targets, results)}
if coco_evaluator is not None:
coco_evaluator.update(res)
if panoptic_evaluator is not None:
res_pano = postprocessors["panoptic"](outputs, target_sizes, orig_target_sizes)
for i, target in enumerate(targets):
image_id = target["image_id"].item()
file_name = f"{image_id:012d}.png"
res_pano[i]["image_id"] = image_id
res_pano[i]["file_name"] = file_name
panoptic_evaluator.update(res_pano)
# gather the stats from all processes
metric_logger.synchronize_between_processes()
print("Averaged stats:", metric_logger)
if coco_evaluator is not None:
coco_evaluator.synchronize_between_processes()
if panoptic_evaluator is not None:
panoptic_evaluator.synchronize_between_processes()
# accumulate predictions from all images
if coco_evaluator is not None:
coco_evaluator.accumulate()
coco_evaluator.summarize()
panoptic_res = None
if panoptic_evaluator is not None:
panoptic_res = panoptic_evaluator.summarize()
stats = {k: meter.global_avg for k, meter in metric_logger.meters.items()}
if coco_evaluator is not None:
if 'bbox' in postprocessors.keys():
stats['coco_eval_bbox'] = coco_evaluator.coco_eval['bbox'].stats.tolist()
if 'segm' in postprocessors.keys():
stats['coco_eval_masks'] = coco_evaluator.coco_eval['segm'].stats.tolist()
if panoptic_res is not None:
stats['PQ_all'] = panoptic_res["All"]
stats['PQ_th'] = panoptic_res["Things"]
stats['PQ_st'] = panoptic_res["Stuff"]
return stats, coco_evaluator
================================================
FILE: batchformer-v2/deformable-detr/main.py
================================================
# ------------------------------------------------------------------------
# Deformable DETR
# Copyright (c) 2020 SenseTime. All Rights Reserved.
# Licensed under the Apache License, Version 2.0 [see LICENSE for details]
# ------------------------------------------------------------------------
# Modified from DETR (https://github.com/facebookresearch/detr)
# Copyright (c) Facebook, Inc. and its affiliates. All Rights Reserved
# ------------------------------------------------------------------------
import argparse
import datetime
import json
import random
import time
from pathlib import Path
import numpy as np
import torch
from torch.utils.data import DataLoader
import datasets
import util.misc as utils
import datasets.samplers as samplers
from datasets import build_dataset, get_coco_api_from_dataset
from engine import evaluate, train_one_epoch
from models import build_model
def get_args_parser():
parser = argparse.ArgumentParser('Deformable DETR Detector', add_help=False)
parser.add_argument('--lr', default=2e-4, type=float)
parser.add_argument('--lr_backbone_names', default=["backbone.0"], type=str, nargs='+')
parser.add_argument('--lr_backbone', default=2e-5, type=float)
parser.add_argument('--lr_linear_proj_names', default=['reference_points', 'sampling_offsets'], type=str, nargs='+')
parser.add_argument('--lr_linear_proj_mult', default=0.1, type=float)
parser.add_argument('--batch_size', default=2, type=int)
parser.add_argument('--weight_decay', default=1e-4, type=float)
parser.add_argument('--epochs', default=50, type=int)
parser.add_argument('--lr_drop', default=40, type=int)
parser.add_argument('--lr_drop_epochs', default=None, type=int, nargs='+')
parser.add_argument('--clip_max_norm', default=0.1, type=float,
help='gradient clipping max norm')
parser.add_argument('--sgd', action='store_true')
# Variants of Deformable DETR
parser.add_argument('--with_box_refine', default=False, action='store_true')
parser.add_argument('--two_stage', default=False, action='store_true')
# Model parameters
parser.add_argument('--frozen_weights', type=str, default=None,
help="Path to the pretrained model. If set, only the mask head will be trained")
# * Backbone
parser.add_argument('--backbone', default='resnet50', type=str,
help="Name of the convolutional backbone to use")
parser.add_argument('--dilation', action='store_true',
help="If true, we replace stride with dilation in the last convolutional block (DC5)")
parser.add_argument('--position_embedding', default='sine', type=str, choices=('sine', 'learned'),
help="Type of positional embedding to use on top of the image features")
parser.add_argument('--position_embedding_scale', default=2 * np.pi, type=float,
help="position / size * scale")
parser.add_argument('--num_feature_levels', default=4, type=int, help='number of feature levels')
# * Transformer
parser.add_argument('--enc_layers', default=6, type=int,
help="Number of encoding layers in the transformer")
parser.add_argument('--dec_layers', default=6, type=int,
help="Number of decoding layers in the transformer")
parser.add_argument('--dim_feedforward', default=1024, type=int,
help="Intermediate size of the feedforward layers in the transformer blocks")
parser.add_argument('--hidden_dim', default=256, type=int,
help="Size of the embeddings (dimension of the transformer)")
parser.add_argument('--dropout', default=0.1, type=float,
help="Dropout applied in the transformer")
parser.add_argument('--nheads', default=8, type=int,
help="Number of attention heads inside the transformer's attentions")
parser.add_argument('--num_queries', default=300, type=int,
help="Number of query slots")
parser.add_argument('--dec_n_points', default=4, type=int)
parser.add_argument('--enc_n_points', default=4, type=int)
# * Segmentation
parser.add_argument('--masks', action='store_true',
help="Train segmentation head if the flag is provided")
# Loss
parser.add_argument('--no_aux_loss', dest='aux_loss', action='store_false',
help="Disables auxiliary decoding losses (loss at each layer)")
# * Matcher
parser.add_argument('--set_cost_class', default=2, type=float,
help="Class coefficient in the matching cost")
parser.add_argument('--set_cost_bbox', default=5, type=float,
help="L1 box coefficient in the matching cost")
parser.add_argument('--set_cost_giou', default=2, type=float,
help="giou box coefficient in the matching cost")
# * Loss coefficients
parser.add_argument('--mask_loss_coef', default=1, type=float)
parser.add_argument('--dice_loss_coef', default=1, type=float)
parser.add_argument('--cls_loss_coef', default=2, type=float)
parser.add_argument('--bbox_loss_coef', default=5, type=float)
parser.add_argument('--giou_loss_coef', default=2, type=float)
parser.add_argument('--focal_alpha', default=0.25, type=float)
# dataset parameters
parser.add_argument('--dataset_file', default='coco')
parser.add_argument('--coco_path', default='./data/coco', type=str)
parser.add_argument('--coco_panoptic_path', type=str)
parser.add_argument('--remove_difficult', action='store_true')
parser.add_argument('--output_dir', default='',
help='path where to save, empty for no saving')
parser.add_argument('--device', default='cuda',
help='device to use for training / testing')
parser.add_argument('--seed', default=42, type=int)
parser.add_argument('--resume', default='', help='resume from checkpoint')
parser.add_argument('--start_epoch', default=0, type=int, metavar='N',
help='start epoch')
parser.add_argument('--eval', action='store_true')
parser.add_argument('--num_workers', default=2, type=int)
parser.add_argument('--bf', default=0, type=int)
parser.add_argument('--share_bf', default=1, type=int)
parser.add_argument('--bt_num_layers', default=1, type=int)
parser.add_argument('--insert_idx', action='append', type=int,
help='insert idx list')
parser.add_argument('--eval_bf', action='store_true', help='batch inference with batchformerv2')
parser.add_argument('--drop_path', default=0., type=float)
parser.add_argument('--cache_mode', default=False, action='store_true', help='whether to cache images on memory')
return parser
def main(args):
utils.init_distributed_mode(args)
print("git:\n {}\n".format(utils.get_sha()))
if args.frozen_weights is not None:
assert args.masks, "Frozen training is meant for segmentation only"
print(args)
device = torch.device(args.device)
# fix the seed for reproducibility
seed = args.seed + utils.get_rank()
torch.manual_seed(seed)
np.random.seed(seed)
random.seed(seed)
model, criterion, postprocessors = build_model(args)
model.to(device)
model_without_ddp = model
n_parameters = sum(p.numel() for p in model.parameters() if p.requires_grad)
print('number of params:', n_parameters)
dataset_train = build_dataset(image_set='train', args=args)
dataset_val = build_dataset(image_set='val', args=args)
if args.distributed:
if args.cache_mode:
sampler_train = samplers.NodeDistributedSampler(dataset_train)
sampler_val = samplers.NodeDistributedSampler(dataset_val, shuffle=False)
else:
sampler_train = samplers.DistributedSampler(dataset_train)
sampler_val = samplers.DistributedSampler(dataset_val, shuffle=False)
else:
sampler_train = torch.utils.data.RandomSampler(dataset_train)
sampler_val = torch.utils.data.SequentialSampler(dataset_val)
batch_sampler_train = torch.utils.data.BatchSampler(
sampler_train, args.batch_size, drop_last=True)
data_loader_train = DataLoader(dataset_train, batch_sampler=batch_sampler_train,
collate_fn=utils.collate_fn, num_workers=args.num_workers,
pin_memory=True)
data_loader_val = DataLoader(dataset_val, args.batch_size, sampler=sampler_val,
drop_last=False, collate_fn=utils.collate_fn, num_workers=args.num_workers,
pin_memory=True)
# lr_backbone_names = ["backbone.0", "backbone.neck", "input_proj", "transformer.encoder"]
def match_name_keywords(n, name_keywords):
out = False
for b in name_keywords:
if b in n:
out = True
break
return out
param_dicts = [
{
"params":
[p for n, p in model_without_ddp.named_parameters()
if not match_name_keywords(n, args.lr_backbone_names) and not match_name_keywords(n, args.lr_linear_proj_names) and p.requires_grad],
"lr": args.lr,
},
{
"params": [p for n, p in model_without_ddp.named_parameters() if match_name_keywords(n, args.lr_backbone_names) and p.requires_grad],
"lr": args.lr_backbone,
},
{
"params": [p for n, p in model_without_ddp.named_parameters() if match_name_keywords(n, args.lr_linear_proj_names) and p.requires_grad],
"lr": args.lr * args.lr_linear_proj_mult,
}
]
if args.sgd:
optimizer = torch.optim.SGD(param_dicts, lr=args.lr, momentum=0.9,
weight_decay=args.weight_decay)
else:
optimizer = torch.optim.AdamW(param_dicts, lr=args.lr,
weight_decay=args.weight_decay)
lr_scheduler = torch.optim.lr_scheduler.StepLR(optimizer, args.lr_drop)
if args.distributed:
model = torch.nn.parallel.DistributedDataParallel(model, device_ids=[args.gpu])
model_without_ddp = model.module
if args.dataset_file == "coco_panoptic":
# We also evaluate AP during panoptic training, on original coco DS
coco_val = datasets.coco.build("val", args)
base_ds = get_coco_api_from_dataset(coco_val)
else:
base_ds = get_coco_api_from_dataset(dataset_val)
if args.frozen_weights is not None:
checkpoint = torch.load(args.frozen_weights, map_location='cpu')
model_without_ddp.detr.load_state_dict(checkpoint['model'])
output_dir = Path(args.output_dir)
if args.resume:
if args.resume.startswith('https'):
checkpoint = torch.hub.load_state_dict_from_url(
args.resume, map_location='cpu', check_hash=True)
else:
checkpoint = torch.load(args.resume, map_location='cpu')
missing_keys, unexpected_keys = model_without_ddp.load_state_dict(checkpoint['model'], strict=False)
unexpected_keys = [k for k in unexpected_keys if not (k.endswith('total_params') or k.endswith('total_ops'))]
if len(missing_keys) > 0:
print('Missing Keys: {}'.format(missing_keys))
if len(unexpected_keys) > 0:
print('Unexpected Keys: {}'.format(unexpected_keys))
if not args.eval and 'optimizer' in checkpoint and 'lr_scheduler' in checkpoint and 'epoch' in checkpoint:
import copy
p_groups = copy.deepcopy(optimizer.param_groups)
optimizer.load_state_dict(checkpoint['optimizer'])
for pg, pg_old in zip(optimizer.param_groups, p_groups):
pg['lr'] = pg_old['lr']
pg['initial_lr'] = pg_old['initial_lr']
print(optimizer.param_groups)
lr_scheduler.load_state_dict(checkpoint['lr_scheduler'])
# todo: this is a hack for doing experiment that resume from checkpoint and also modify lr scheduler (e.g., decrease lr in advance).
args.override_resumed_lr_drop = True
if args.override_resumed_lr_drop:
print('Warning: (hack) args.override_resumed_lr_drop is set to True, so args.lr_drop would override lr_drop in resumed lr_scheduler.')
lr_scheduler.step_size = args.lr_drop
lr_scheduler.base_lrs = list(map(lambda group: group['initial_lr'], optimizer.param_groups))
lr_scheduler.step(lr_scheduler.last_epoch)
args.start_epoch = checkpoint['epoch'] + 1
# check the resumed model
if not args.eval:
test_stats, coco_evaluator = evaluate(
model, criterion, postprocessors, data_loader_val, base_ds, device, args.output_dir
)
if args.eval:
test_stats, coco_evaluator = evaluate(model, criterion, postprocessors,
data_loader_val, base_ds, device, args.output_dir)
if args.output_dir:
utils.save_on_master(coco_evaluator.coco_eval["bbox"].eval, output_dir / "eval.pth")
return
print("Start training")
start_time = time.time()
for epoch in range(args.start_epoch, args.epochs):
if args.distributed:
sampler_train.set_epoch(epoch)
train_stats = train_one_epoch(
model, criterion, data_loader_train, optimizer, device, epoch, args.clip_max_norm)
lr_scheduler.step()
if args.output_dir:
checkpoint_paths = [output_dir / 'checkpoint.pth']
# extra checkpoint before LR drop and every 5 epochs
if (epoch + 1) % args.lr_drop == 0 or (epoch + 1) % 5 == 0:
checkpoint_paths.append(output_dir / f'checkpoint{epoch:04}.pth')
for checkpoint_path in checkpoint_paths:
utils.save_on_master({
'model': model_without_ddp.state_dict(),
'optimizer': optimizer.state_dict(),
'lr_scheduler': lr_scheduler.state_dict(),
'epoch': epoch,
'args': args,
}, checkpoint_path)
test_stats, coco_evaluator = evaluate(
model, criterion, postprocessors, data_loader_val, base_ds, device, args.output_dir
)
log_stats = {**{f'train_{k}': v for k, v in train_stats.items()},
**{f'test_{k}': v for k, v in test_stats.items()},
'epoch': epoch,
'n_parameters': n_parameters}
if args.output_dir and utils.is_main_process():
with (output_dir / "log.txt").open("a") as f:
f.write(json.dumps(log_stats) + "\n")
# for evaluation logs
if coco_evaluator is not None:
(output_dir / 'eval').mkdir(exist_ok=True)
if "bbox" in coco_evaluator.coco_eval:
filenames = ['latest.pth']
if epoch % 50 == 0:
filenames.append(f'{epoch:03}.pth')
for name in filenames:
torch.save(coco_evaluator.coco_eval["bbox"].eval,
output_dir / "eval" / name)
total_time = time.time() - start_time
total_time_str = str(datetime.timedelta(seconds=int(total_time)))
print('Training time {}'.format(total_time_str))
if __name__ == '__main__':
parser = argparse.ArgumentParser('Deformable DETR training and evaluation script', parents=[get_args_parser()])
args = parser.parse_args()
if args.output_dir:
Path(args.output_dir).mkdir(parents=True, exist_ok=True)
main(args)
================================================
FILE: batchformer-v2/deformable-detr/models/__init__.py
================================================
# ------------------------------------------------------------------------
# Deformable DETR
# Copyright (c) 2020 SenseTime. All Rights Reserved.
# Licensed under the Apache License, Version 2.0 [see LICENSE for details]
# ------------------------------------------------------------------------
# Modified from DETR (https://github.com/facebookresearch/detr)
# Copyright (c) Facebook, Inc. and its affiliates. All Rights Reserved
# ------------------------------------------------------------------------
from .deformable_detr import build
def build_model(args):
return build(args)
================================================
FILE: batchformer-v2/deformable-detr/models/backbone.py
================================================
# ------------------------------------------------------------------------
# Deformable DETR
# Copyright (c) 2020 SenseTime. All Rights Reserved.
# Licensed under the Apache License, Version 2.0 [see LICENSE for details]
# ------------------------------------------------------------------------
# Modified from DETR (https://github.com/facebookresearch/detr)
# Copyright (c) Facebook, Inc. and its affiliates. All Rights Reserved
# ------------------------------------------------------------------------
"""
Backbone modules.
"""
from collections import OrderedDict
import torch
import torch.nn.functional as F
import torchvision
from torch import nn
from torchvision.models._utils import IntermediateLayerGetter
from typing import Dict, List
from util.misc import NestedTensor, is_main_process
from .position_encoding import build_position_encoding
class FrozenBatchNorm2d(torch.nn.Module):
"""
BatchNorm2d where the batch statistics and the affine parameters are fixed.
Copy-paste from torchvision.misc.ops with added eps before rqsrt,
without which any other models than torchvision.models.resnet[18,34,50,101]
produce nans.
"""
def __init__(self, n, eps=1e-5):
super(FrozenBatchNorm2d, self).__init__()
self.register_buffer("weight", torch.ones(n))
self.register_buffer("bias", torch.zeros(n))
self.register_buffer("running_mean", torch.zeros(n))
self.register_buffer("running_var", torch.ones(n))
self.eps = eps
def _load_from_state_dict(self, state_dict, prefix, local_metadata, strict,
missing_keys, unexpected_keys, error_msgs):
num_batches_tracked_key = prefix + 'num_batches_tracked'
if num_batches_tracked_key in state_dict:
del state_dict[num_batches_tracked_key]
super(FrozenBatchNorm2d, self)._load_from_state_dict(
state_dict, prefix, local_metadata, strict,
missing_keys, unexpected_keys, error_msgs)
def forward(self, x):
# move reshapes to the beginning
# to make it fuser-friendly
w = self.weight.reshape(1, -1, 1, 1)
b = self.bias.reshape(1, -1, 1, 1)
rv = self.running_var.reshape(1, -1, 1, 1)
rm = self.running_mean.reshape(1, -1, 1, 1)
eps = self.eps
scale = w * (rv + eps).rsqrt()
bias = b - rm * scale
return x * scale + bias
class BackboneBase(nn.Module):
def __init__(self, backbone: nn.Module, train_backbone: bool, return_interm_layers: bool):
super().__init__()
for name, parameter in backbone.named_parameters():
if not train_backbone or 'layer2' not in name and 'layer3' not in name and 'layer4' not in name:
parameter.requires_grad_(False)
if return_interm_layers:
# return_layers = {"layer1": "0", "layer2": "1", "layer3": "2", "layer4": "3"}
return_layers = {"layer2": "0", "layer3": "1", "layer4": "2"}
self.strides = [8, 16, 32]
self.num_channels = [512, 1024, 2048]
else:
return_layers = {'layer4': "0"}
self.strides = [32]
self.num_channels = [2048]
self.body = IntermediateLayerGetter(backbone, return_layers=return_layers)
def forward(self, tensor_list: NestedTensor):
xs = self.body(tensor_list.tensors)
out: Dict[str, NestedTensor] = {}
for name, x in xs.items():
m = tensor_list.mask
assert m is not None
mask = F.interpolate(m[None].float(), size=x.shape[-2:]).to(torch.bool)[0]
out[name] = NestedTensor(x, mask)
return out
class Backbone(BackboneBase):
"""ResNet backbone with frozen BatchNorm."""
def __init__(self, name: str,
train_backbone: bool,
return_interm_layers: bool,
dilation: bool):
norm_layer = FrozenBatchNorm2d
backbone = getattr(torchvision.models, name)(
replace_stride_with_dilation=[False, False, dilation],
pretrained=is_main_process(), norm_layer=norm_layer)
assert name not in ('resnet18', 'resnet34'), "number of channels are hard coded"
super().__init__(backbone, train_backbone, return_interm_layers)
if dilation:
self.strides[-1] = self.strides[-1] // 2
class Joiner(nn.Sequential):
def __init__(self, backbone, position_embedding):
super().__init__(backbone, position_embedding)
self.strides = backbone.strides
self.num_channels = backbone.num_channels
def forward(self, tensor_list: NestedTensor):
xs = self[0](tensor_list)
out: List[NestedTensor] = []
pos = []
for name, x in sorted(xs.items()):
out.append(x)
# position encoding
for x in out:
pos.append(self[1](x).to(x.tensors.dtype))
return out, pos
def build_backbone(args):
position_embedding = build_position_encoding(args)
train_backbone = args.lr_backbone > 0
return_interm_layers = args.masks or (args.num_feature_levels > 1)
backbone = Backbone(args.backbone, train_backbone, return_interm_layers, args.dilation)
model = Joiner(backbone, position_embedding)
return model
================================================
FILE: batchformer-v2/deformable-detr/models/deformable_detr.py
================================================
# ------------------------------------------------------------------------
# Deformable DETR
# Copyright (c) 2020 SenseTime. All Rights Reserved.
# Licensed under the Apache License, Version 2.0 [see LICENSE for details]
# ------------------------------------------------------------------------
# Modified from DETR (https://github.com/facebookresearch/detr)
# Copyright (c) Facebook, Inc. and its affiliates. All Rights Reserved
# ------------------------------------------------------------------------
"""
Deformable DETR model and criterion classes.
"""
import torch
import torch.nn.functional as F
from torch import nn
import math
from util import box_ops
from util.misc import (NestedTensor, nested_tensor_from_tensor_list,
accuracy, get_world_size, interpolate,
is_dist_avail_and_initialized, inverse_sigmoid)
from .backbone import build_backbone
from .matcher import build_matcher
from .segmentation import (DETRsegm, PostProcessPanoptic, PostProcessSegm,
dice_loss, sigmoid_focal_loss)
from .deformable_transformer import build_deforamble_transformer
import copy
def _get_clones(module, N):
return nn.ModuleList([copy.deepcopy(module) for i in range(N)])
class DeformableDETR(nn.Module):
""" This is the Deformable DETR module that performs object detection """
def __init__(self, backbone, transformer, num_classes, num_queries, num_feature_levels,
aux_loss=True, with_box_refine=False, two_stage=False):
""" Initializes the model.
Parameters:
backbone: torch module of the backbone to be used. See backbone.py
transformer: torch module of the transformer architecture. See transformer.py
num_classes: number of object classes
num_queries: number of object queries, ie detection slot. This is the maximal number of objects
DETR can detect in a single image. For COCO, we recommend 100 queries.
aux_loss: True if auxiliary decoding losses (loss at each decoder layer) are to be used.
with_box_refine: iterative bounding box refinement
two_stage: two-stage Deformable DETR
"""
super().__init__()
self.num_queries = num_queries
self.transformer = transformer
hidden_dim = transformer.d_model
self.class_embed = nn.Linear(hidden_dim, num_classes)
self.bbox_embed = MLP(hidden_dim, hidden_dim, 4, 3)
self.num_feature_levels = num_feature_levels
if not two_stage:
self.query_embed = nn.Embedding(num_queries, hidden_dim*2)
if num_feature_levels > 1:
num_backbone_outs = len(backbone.strides)
input_proj_list = []
for _ in range(num_backbone_outs):
in_channels = backbone.num_channels[_]
input_proj_list.append(nn.Sequential(
nn.Conv2d(in_channels, hidden_dim, kernel_size=1),
nn.GroupNorm(32, hidden_dim),
))
for _ in range(num_feature_levels - num_backbone_outs):
input_proj_list.append(nn.Sequential(
nn.Conv2d(in_channels, hidden_dim, kernel_size=3, stride=2, padding=1),
nn.GroupNorm(32, hidden_dim),
))
in_channels = hidden_dim
self.input_proj = nn.ModuleList(input_proj_list)
else:
self.input_proj = nn.ModuleList([
nn.Sequential(
nn.Conv2d(backbone.num_channels[0], hidden_dim, kernel_size=1),
nn.GroupNorm(32, hidden_dim),
)])
self.backbone = backbone
self.aux_loss = aux_loss
self.with_box_refine = with_box_refine
self.two_stage = two_stage
prior_prob = 0.01
bias_value = -math.log((1 - prior_prob) / prior_prob)
self.class_embed.bias.data = torch.ones(num_classes) * bias_value
nn.init.constant_(self.bbox_embed.layers[-1].weight.data, 0)
nn.init.constant_(self.bbox_embed.layers[-1].bias.data, 0)
for proj in self.input_proj:
nn.init.xavier_uniform_(proj[0].weight, gain=1)
nn.init.constant_(proj[0].bias, 0)
# if two-stage, the last class_embed and bbox_embed is for region proposal generation
num_pred = (transformer.decoder.num_layers + 1) if two_stage else transformer.decoder.num_layers
if with_box_refine:
self.class_embed = _get_clones(self.class_embed, num_pred)
self.bbox_embed = _get_clones(self.bbox_embed, num_pred)
nn.init.constant_(self.bbox_embed[0].layers[-1].bias.data[2:], -2.0)
# hack implementation for iterative bounding box refinement
self.transformer.decoder.bbox_embed = self.bbox_embed
else:
nn.init.constant_(self.bbox_embed.layers[-1].bias.data[2:], -2.0)
self.class_embed = nn.ModuleList([self.class_embed for _ in range(num_pred)])
self.bbox_embed = nn.ModuleList([self.bbox_embed for _ in range(num_pred)])
self.transformer.decoder.bbox_embed = None
if two_stage:
# hack implementation for two-stage
self.transformer.decoder.class_embed = self.class_embed
for box_embed in self.bbox_embed:
nn.init.constant_(box_embed.layers[-1].bias.data[2:], 0.0)
def forward(self, samples: NestedTensor):
""" The forward expects a NestedTensor, which consists of:
- samples.tensor: batched images, of shape [batch_size x 3 x H x W]
- samples.mask: a binary mask of shape [batch_size x H x W], containing 1 on padded pixels
It returns a dict with the following elements:
- "pred_logits": the classification logits (including no-object) for all queries.
Shape= [batch_size x num_queries x (num_classes + 1)]
- "pred_boxes": The normalized boxes coordinates for all queries, represented as
(center_x, center_y, height, width). These values are normalized in [0, 1],
relative to the size of each individual image (disregarding possible padding).
See PostProcess for information on how to retrieve the unnormalized bounding box.
- "aux_outputs": Optional, only returned when auxilary losses are activated. It is a list of
dictionnaries containing the two above keys for each decoder layer.
"""
if not isinstance(samples, NestedTensor):
samples = nested_tensor_from_tensor_list(samples)
features, pos = self.backbone(samples)
srcs = []
masks = []
for l, feat in enumerate(features):
src, mask = feat.decompose()
srcs.append(self.input_proj[l](src))
masks.append(mask)
assert mask is not None
if self.num_feature_levels > len(srcs):
_len_srcs = len(srcs)
for l in range(_len_srcs, self.num_feature_levels):
if l == _len_srcs:
src = self.input_proj[l](features[-1].tensors)
else:
src = self.input_proj[l](srcs[-1])
m = samples.mask
mask = F.interpolate(m[None].float(), size=src.shape[-2:]).to(torch.bool)[0]
pos_l = self.backbone[1](NestedTensor(src, mask)).to(src.dtype)
srcs.append(src)
masks.append(mask)
pos.append(pos_l)
query_embeds = None
if not self.two_stage:
query_embeds = self.query_embed.weight
hs, init_reference, inter_references, enc_outputs_class, enc_outputs_coord_unact = self.transformer(srcs, masks, pos, query_embeds)
outputs_classes = []
outputs_coords = []
for lvl in range(hs.shape[0]):
if lvl == 0:
reference = init_reference
else:
reference = inter_references[lvl - 1]
reference = inverse_sigmoid(reference)
outputs_class = self.class_embed[lvl](hs[lvl])
tmp = self.bbox_embed[lvl](hs[lvl])
if reference.shape[-1] == 4:
tmp += reference
else:
assert reference.shape[-1] == 2
tmp[..., :2] += reference
outputs_coord = tmp.sigmoid()
outputs_classes.append(outputs_class)
outputs_coords.append(outputs_coord)
outputs_class = torch.stack(outputs_classes)
outputs_coord = torch.stack(outputs_coords)
out = {'pred_logits': outputs_class[-1], 'pred_boxes': outputs_coord[-1]}
if self.aux_loss:
out['aux_outputs'] = self._set_aux_loss(outputs_class, outputs_coord)
if self.two_stage:
enc_outputs_coord = enc_outputs_coord_unact.sigmoid()
out['enc_outputs'] = {'pred_logits': enc_outputs_class, 'pred_boxes': enc_outputs_coord}
return out
@torch.jit.unused
def _set_aux_loss(self, outputs_class, outputs_coord):
# this is a workaround to make torchscript happy, as torchscript
# doesn't support dictionary with non-homogeneous values, such
# as a dict having both a Tensor and a list.
return [{'pred_logits': a, 'pred_boxes': b}
for a, b in zip(outputs_class[:-1], outputs_coord[:-1])]
class SetCriterion(nn.Module):
""" This class computes the loss for DETR.
The process happens in two steps:
1) we compute hungarian assignment between ground truth boxes and the outputs of the model
2) we supervise each pair of matched ground-truth / prediction (supervise class and box)
"""
def __init__(self, num_classes, matcher, weight_dict, losses, focal_alpha=0.25):
""" Create the criterion.
Parameters:
num_classes: number of object categories, omitting the special no-object category
matcher: module able to compute a matching between targets and proposals
weight_dict: dict containing as key the names of the losses and as values their relative weight.
losses: list of all the losses to be applied. See get_loss for list of available losses.
focal_alpha: alpha in Focal Loss
"""
super().__init__()
self.num_classes = num_classes
self.matcher = matcher
self.weight_dict = weight_dict
self.losses = losses
self.focal_alpha = focal_alpha
def loss_labels(self, outputs, targets, indices, num_boxes, log=True):
"""Classification loss (NLL)
targets dicts must contain the key "labels" containing a tensor of dim [nb_target_boxes]
"""
assert 'pred_logits' in outputs
src_logits = outputs['pred_logits']
idx = self._get_src_permutation_idx(indices)
target_classes_o = torch.cat([t["labels"][J] for t, (_, J) in zip(targets, indices)])
target_classes = torch.full(src_logits.shape[:2], self.num_classes,
dtype=torch.int64, device=src_logits.device)
target_classes[idx] = target_classes_o
target_classes_onehot = torch.zeros([src_logits.shape[0], src_logits.shape[1], src_logits.shape[2] + 1],
dtype=src_logits.dtype, layout=src_logits.layout, device=src_logits.device)
target_classes_onehot.scatter_(2, target_classes.unsqueeze(-1), 1)
target_classes_onehot = target_classes_onehot[:,:,:-1]
loss_ce = sigmoid_focal_loss(src_logits, target_classes_onehot, num_boxes, alpha=self.focal_alpha, gamma=2) * src_logits.shape[1]
losses = {'loss_ce': loss_ce}
if log:
# TODO this should probably be a separate loss, not hacked in this one here
losses['class_error'] = 100 - accuracy(src_logits[idx], target_classes_o)[0]
return losses
@torch.no_grad()
def loss_cardinality(self, outputs, targets, indices, num_boxes):
""" Compute the cardinality error, ie the absolute error in the number of predicted non-empty boxes
This is not really a loss, it is intended for logging purposes only. It doesn't propagate gradients
"""
pred_logits = outputs['pred_logits']
device = pred_logits.device
tgt_lengths = torch.as_tensor([len(v["labels"]) for v in targets], device=device)
# Count the number of predictions that are NOT "no-object" (which is the last class)
card_pred = (pred_logits.argmax(-1) != pred_logits.shape[-1] - 1).sum(1)
card_err = F.l1_loss(card_pred.float(), tgt_lengths.float())
losses = {'cardinality_error': card_err}
return losses
def loss_boxes(self, outputs, targets, indices, num_boxes):
"""Compute the losses related to the bounding boxes, the L1 regression loss and the GIoU loss
targets dicts must contain the key "boxes" containing a tensor of dim [nb_target_boxes, 4]
The target boxes are expected in format (center_x, center_y, h, w), normalized by the image size.
"""
assert 'pred_boxes' in outputs
idx = self._get_src_permutation_idx(indices)
src_boxes = outputs['pred_boxes'][idx]
target_boxes = torch.cat([t['boxes'][i] for t, (_, i) in zip(targets, indices)], dim=0)
loss_bbox = F.l1_loss(src_boxes, target_boxes, reduction='none')
losses = {}
losses['loss_bbox'] = loss_bbox.sum() / num_boxes
loss_giou = 1 - torch.diag(box_ops.generalized_box_iou(
box_ops.box_cxcywh_to_xyxy(src_boxes),
box_ops.box_cxcywh_to_xyxy(target_boxes)))
losses['loss_giou'] = loss_giou.sum() / num_boxes
return losses
def loss_masks(self, outputs, targets, indices, num_boxes):
"""Compute the losses related to the masks: the focal loss and the dice loss.
targets dicts must contain the key "masks" containing a tensor of dim [nb_target_boxes, h, w]
"""
assert "pred_masks" in outputs
src_idx = self._get_src_permutation_idx(indices)
tgt_idx = self._get_tgt_permutation_idx(indices)
src_masks = outputs["pred_masks"]
# TODO use valid to mask invalid areas due to padding in loss
target_masks, valid = nested_tensor_from_tensor_list([t["masks"] for t in targets]).decompose()
target_masks = target_masks.to(src_masks)
src_masks = src_masks[src_idx]
# upsample predictions to the target size
src_masks = interpolate(src_masks[:, None], size=target_masks.shape[-2:],
mode="bilinear", align_corners=False)
src_masks = src_masks[:, 0].flatten(1)
target_masks = target_masks[tgt_idx].flatten(1)
losses = {
"loss_mask": sigmoid_focal_loss(src_masks, target_masks, num_boxes),
"loss_dice": dice_loss(src_masks, target_masks, num_boxes),
}
return losses
def _get_src_permutation_idx(self, indices):
# permute predictions following indices
batch_idx = torch.cat([torch.full_like(src, i) for i, (src, _) in enumerate(indices)])
src_idx = torch.cat([src for (src, _) in indices])
return batch_idx, src_idx
def _get_tgt_permutation_idx(self, indices):
# permute targets following indices
batch_idx = torch.cat([torch.full_like(tgt, i) for i, (_, tgt) in enumerate(indices)])
tgt_idx = torch.cat([tgt for (_, tgt) in indices])
return batch_idx, tgt_idx
def get_loss(self, loss, outputs, targets, indices, num_boxes, **kwargs):
loss_map = {
'labels': self.loss_labels,
'cardinality': self.loss_cardinality,
'boxes': self.loss_boxes,
'masks': self.loss_masks
}
assert loss in loss_map, f'do you really want to compute {loss} loss?'
return loss_map[loss](outputs, targets, indices, num_boxes, **kwargs)
def forward(self, outputs, targets):
""" This performs the loss computation.
Parameters:
outputs: dict of tensors, see the output specification of the model for the format
targets: list of dicts, such that len(targets) == batch_size.
The expected keys in each dict depends on the losses applied, see each loss' doc
"""
outputs_without_aux = {k: v for k, v in outputs.items() if k != 'aux_outputs' and k != 'enc_outputs'}
# Retrieve the matching between the outputs of the last layer and the targets
indices = self.matcher(outputs_without_aux, targets)
# Compute the average number of target boxes accross all nodes, for normalization purposes
num_boxes = sum(len(t["labels"]) for t in targets)
num_boxes = torch.as_tensor([num_boxes], dtype=torch.float, device=next(iter(outputs.values())).device)
if is_dist_avail_and_initialized():
torch.distributed.all_reduce(num_boxes)
num_boxes = torch.clamp(num_boxes / get_world_size(), min=1).item()
# Compute all the requested losses
losses = {}
for loss in self.losses:
kwargs = {}
losses.update(self.get_loss(loss, outputs, targets, indices, num_boxes, **kwargs))
# In case of auxiliary losses, we repeat this process with the output of each intermediate layer.
if 'aux_outputs' in outputs:
for i, aux_outputs in enumerate(outputs['aux_outputs']):
indices = self.matcher(aux_outputs, targets)
for loss in self.losses:
if loss == 'masks':
# Intermediate masks losses are too costly to compute, we ignore them.
continue
kwargs = {}
if loss == 'labels':
# Logging is enabled only for the last layer
kwargs['log'] = False
l_dict = self.get_loss(loss, aux_outputs, targets, indices, num_boxes, **kwargs)
l_dict = {k + f'_{i}': v for k, v in l_dict.items()}
losses.update(l_dict)
if 'enc_outputs' in outputs:
enc_outputs = outputs['enc_outputs']
bin_targets = copy.deepcopy(targets)
for bt in bin_targets:
bt['labels'] = torch.zeros_like(bt['labels'])
indices = self.matcher(enc_outputs, bin_targets)
for loss in self.losses:
if loss == 'masks':
# Intermediate masks losses are too costly to compute, we ignore them.
continue
kwargs = {}
if loss == 'labels':
# Logging is enabled only for the last layer
kwargs['log'] = False
l_dict = self.get_loss(loss, enc_outputs, bin_targets, indices, num_boxes, **kwargs)
l_dict = {k + f'_enc': v for k, v in l_dict.items()}
losses.update(l_dict)
return losses
class PostProcess(nn.Module):
""" This module converts the model's output into the format expected by the coco api"""
@torch.no_grad()
def forward(self, outputs, target_sizes):
""" Perform the computation
Parameters:
outputs: raw outputs of the model
target_sizes: tensor of dimension [batch_size x 2] containing the size of each images of the batch
For evaluation, this must be the original image size (before any data augmentation)
For visualization, this should be the image size after data augment, but before padding
"""
out_logits, out_bbox = outputs['pred_logits'], outputs['pred_boxes']
assert len(out_logits) == len(target_sizes)
assert target_sizes.shape[1] == 2
prob = out_logits.sigmoid()
topk_values, topk_indexes = torch.topk(prob.view(out_logits.shape[0], -1), 100, dim=1)
scores = topk_values
topk_boxes = topk_indexes // out_logits.shape[2]
labels = topk_indexes % out_logits.shape[2]
boxes = box_ops.box_cxcywh_to_xyxy(out_bbox)
boxes = torch.gather(boxes, 1, topk_boxes.unsqueeze(-1).repeat(1,1,4))
# and from relative [0, 1] to absolute [0, height] coordinates
img_h, img_w = target_sizes.unbind(1)
scale_fct = torch.stack([img_w, img_h, img_w, img_h], dim=1)
boxes = boxes * scale_fct[:, None, :]
results = [{'scores': s, 'labels': l, 'boxes': b} for s, l, b in zip(scores, labels, boxes)]
return results
class MLP(nn.Module):
""" Very simple multi-layer perceptron (also called FFN)"""
def __init__(self, input_dim, hidden_dim, output_dim, num_layers):
super().__init__()
self.num_layers = num_layers
h = [hidden_dim] * (num_layers - 1)
self.layers = nn.ModuleList(nn.Linear(n, k) for n, k in zip([input_dim] + h, h + [output_dim]))
def forward(self, x):
for i, layer in enumerate(self.layers):
x = F.relu(layer(x)) if i < self.num_layers - 1 else layer(x)
return x
def build(args):
num_classes = 20 if args.dataset_file != 'coco' else 91
if args.dataset_file == "coco_panoptic":
num_classes = 250
device = torch.device(args.device)
backbone = build_backbone(args)
transformer = build_deforamble_transformer(args)
model = DeformableDETR(
backbone,
transformer,
num_classes=num_classes,
num_queries=args.num_queries,
num_feature_levels=args.num_feature_levels,
aux_loss=args.aux_loss,
with_box_refine=args.with_box_refine,
two_stage=args.two_stage,
)
if args.masks:
model = DETRsegm(model, freeze_detr=(args.frozen_weights is not None))
matcher = build_matcher(args)
weight_dict = {'loss_ce': args.cls_loss_coef, 'loss_bbox': args.bbox_loss_coef}
weight_dict['loss_giou'] = args.giou_loss_coef
if args.masks:
weight_dict["loss_mask"] = args.mask_loss_coef
weight_dict["loss_dice"] = args.dice_loss_coef
# TODO this is a hack
if args.aux_loss:
aux_weight_dict = {}
for i in range(args.dec_layers - 1):
aux_weight_dict.update({k + f'_{i}': v for k, v in weight_dict.items()})
aux_weight_dict.update({k + f'_enc': v for k, v in weight_dict.items()})
weight_dict.update(aux_weight_dict)
losses = ['labels', 'boxes', 'cardinality']
if args.masks:
losses += ["masks"]
# num_classes, matcher, weight_dict, losses, focal_alpha=0.25
criterion = SetCriterion(num_classes, matcher, weight_dict, losses, focal_alpha=args.focal_alpha)
criterion.to(device)
postprocessors = {'bbox': PostProcess()}
if args.masks:
postprocessors['segm'] = PostProcessSegm()
if args.dataset_file == "coco_panoptic":
is_thing_map = {i: i <= 90 for i in range(201)}
postprocessors["panoptic"] = PostProcessPanoptic(is_thing_map, threshold=0.85)
return model, criterion, postprocessors
================================================
FILE: batchformer-v2/deformable-detr/models/deformable_transformer.py
================================================
import copy
from typing import Optional, List
import math
import torch
import torch.nn.functional as F
from torch import nn, Tensor
from torch.nn.init import xavier_uniform_, constant_, uniform_, normal_
from util.misc import inverse_sigmoid
from models.ops.modules import MSDeformAttn
class DeformableTransformer(nn.Module):
def __init__(self, d_model=256, nhead=8,
num_encoder_layers=6, num_decoder_layers=6, dim_feedforward=1024, dropout=0.1,
activation="relu", return_intermediate_dec=False,
num_feature_levels=4, dec_n_points=4, enc_n_points=4,
two_stage=False, two_stage_num_proposals=300, args=None):
super().__init__()
assert args is not None
self.d_model = d_model
self.nhead = nhead
self.two_stage = two_stage
self.two_stage_num_proposals = two_stage_num_proposals
encoder_layer = DeformableTransformerEncoderLayer(d_model, dim_feedforward,
dropout, activation,
num_feature_levels, nhead, enc_n_points)
self.bf = None
insert_idx = []
if args is not None and args.bf:
def generate_bf(type, args):
if type == 1:
encoder = torch.nn.TransformerEncoderLayer(d_model, 4, d_model, dropout=0.5)
elif type >= 2:
encoder = torch.nn.TransformerEncoder(torch.nn.TransformerEncoderLayer(d_model, 4, d_model, dropout=0.5), type)
else:
encoder = torch.nn.TransformerEncoderLayer(d_model, 4, d_model, dropout=0.5)
return encoder
self.bf = generate_bf(args.bt_num_layers, args)
insert_idx = args.insert_idx
if not args.share_bf:
self.bf = torch.nn.ModuleList([generate_bf(args.bt_num_layers, args) if i in insert_idx else torch.nn.Identity() for i in range(0, num_encoder_layers)])
self.encoder = DeformableTransformerEncoder(encoder_layer, num_encoder_layers, bf=self.bf, bf_idx = args.bf, insert_idx=insert_idx,
eval_bf=args.eval_bf)
decoder_layer = DeformableTransformerDecoderLayer(d_model, dim_feedforward,
dropout, activation,
num_feature_levels, nhead, dec_n_points)
self.decoder = DeformableTransformerDecoder(decoder_layer, num_decoder_layers, return_intermediate_dec)
self.level_embed = nn.Parameter(torch.Tensor(num_feature_levels, d_model))
if two_stage:
self.enc_output = nn.Linear(d_model, d_model)
self.enc_output_norm = nn.LayerNorm(d_model)
self.pos_trans = nn.Linear(d_model * 2, d_model * 2)
self.pos_trans_norm = nn.LayerNorm(d_model * 2)
else:
self.reference_points = nn.Linear(d_model, 2)
self._reset_parameters()
def _reset_parameters(self):
for p in self.parameters():
if p.dim() > 1:
nn.init.xavier_uniform_(p)
for m in self.modules():
if isinstance(m, MSDeformAttn):
m._reset_parameters()
if not self.two_stage:
xavier_uniform_(self.reference_points.weight.data, gain=1.0)
constant_(self.reference_points.bias.data, 0.)
normal_(self.level_embed)
def get_proposal_pos_embed(self, proposals):
num_pos_feats = 128
temperature = 10000
scale = 2 * math.pi
dim_t = torch.arange(num_pos_feats, dtype=torch.float32, device=proposals.device)
dim_t = temperature ** (2 * (dim_t // 2) / num_pos_feats)
# N, L, 4
proposals = proposals.sigmoid() * scale
# N, L, 4, 128
pos = proposals[:, :, :, None] / dim_t
# N, L, 4, 64, 2
pos = torch.stack((pos[:, :, :, 0::2].sin(), pos[:, :, :, 1::2].cos()), dim=4).flatten(2)
return pos
def gen_encoder_output_proposals(self, memory, memory_padding_mask, spatial_shapes):
N_, S_, C_ = memory.shape
base_scale = 4.0
proposals = []
_cur = 0
for lvl, (H_, W_) in enumerate(spatial_shapes):
mask_flatten_ = memory_padding_mask[:, _cur:(_cur + H_ * W_)].view(N_, H_, W_, 1)
valid_H = torch.sum(~mask_flatten_[:, :, 0, 0], 1)
valid_W = torch.sum(~mask_flatten_[:, 0, :, 0], 1)
grid_y, grid_x = torch.meshgrid(torch.linspace(0, H_ - 1, H_, dtype=torch.float32, device=memory.device),
torch.linspace(0, W_ - 1, W_, dtype=torch.float32, device=memory.device))
grid = torch.cat([grid_x.unsqueeze(-1), grid_y.unsqueeze(-1)], -1)
scale = torch.cat([valid_W.unsqueeze(-1), valid_H.unsqueeze(-1)], 1).view(N_, 1, 1, 2)
grid = (grid.unsqueeze(0).expand(N_, -1, -1, -1) + 0.5) / scale
wh = torch.ones_like(grid) * 0.05 * (2.0 ** lvl)
proposal = torch.cat((grid, wh), -1).view(N_, -1, 4)
proposals.append(proposal)
_cur += (H_ * W_)
output_proposals = torch.cat(proposals, 1)
output_proposals_valid = ((output_proposals > 0.01) & (output_proposals < 0.99)).all(-1, keepdim=True)
output_proposals = torch.log(output_proposals / (1 - output_proposals))
output_proposals = output_proposals.masked_fill(memory_padding_mask.unsqueeze(-1), float('inf'))
output_proposals = output_proposals.masked_fill(~output_proposals_valid, float('inf'))
output_memory = memory
output_memory = output_memory.masked_fill(memory_padding_mask.unsqueeze(-1), float(0))
output_memory = output_memory.masked_fill(~output_proposals_valid, float(0))
output_memory = self.enc_output_norm(self.enc_output(output_memory))
return output_memory, output_proposals
def get_valid_ratio(self, mask):
_, H, W = mask.shape
valid_H = torch.sum(~mask[:, :, 0], 1)
valid_W = torch.sum(~mask[:, 0, :], 1)
valid_ratio_h = valid_H.float() / H
valid_ratio_w = valid_W.float() / W
valid_ratio = torch.stack([valid_ratio_w, valid_ratio_h], -1)
return valid_ratio
def forward(self, srcs, masks, pos_embeds, query_embed=None):
assert self.two_stage or query_embed is not None
# prepare input for encoder
src_flatten = []
mask_flatten = []
lvl_pos_embed_flatten = []
spatial_shapes = []
for lvl, (src, mask, pos_embed) in enumerate(zip(srcs, masks, pos_embeds)):
bs, c, h, w = src.shape
spatial_shape = (h, w)
spatial_shapes.append(spatial_shape)
src = src.flatten(2).transpose(1, 2)
mask = mask.flatten(1)
pos_embed = pos_embed.flatten(2).transpose(1, 2)
lvl_pos_embed = pos_embed + self.level_embed[lvl].view(1, 1, -1)
lvl_pos_embed_flatten.append(lvl_pos_embed)
src_flatten.append(src)
mask_flatten.append(mask)
src_flatten = torch.cat(src_flatten, 1)
mask_flatten = torch.cat(mask_flatten, 1)
lvl_pos_embed_flatten = torch.cat(lvl_pos_embed_flatten, 1)
spatial_shapes = torch.as_tensor(spatial_shapes, dtype=torch.long, device=src_flatten.device)
level_start_index = torch.cat((spatial_shapes.new_zeros((1, )), spatial_shapes.prod(1).cumsum(0)[:-1]))
valid_ratios = torch.stack([self.get_valid_ratio(m) for m in masks], 1)
# encoder
memory = self.encoder(src_flatten, spatial_shapes, level_start_index, valid_ratios, lvl_pos_embed_flatten, mask_flatten)
if len(memory) > len(mask_flatten):
mask_flatten = torch.cat([mask_flatten, mask_flatten], dim=0)
valid_ratios = torch.cat([valid_ratios, valid_ratios], dim=0)
# prepare input for decoder
bs, _, c = memory.shape
if self.two_stage:
output_memory, output_proposals = self.gen_encoder_output_proposals(memory, mask_flatten, spatial_shapes)
# hack implementation for two-stage Deformable DETR
enc_outputs_class = self.decoder.class_embed[self.decoder.num_layers](output_memory)
enc_outputs_coord_unact = self.decoder.bbox_embed[self.decoder.num_layers](output_memory) + output_proposals
topk = self.two_stage_num_proposals
topk_proposals = torch.topk(enc_outputs_class[..., 0], topk, dim=1)[1]
topk_coords_unact = torch.gather(enc_outputs_coord_unact, 1, topk_proposals.unsqueeze(-1).repeat(1, 1, 4))
topk_coords_unact = topk_coords_unact.detach()
reference_points = topk_coords_unact.sigmoid()
init_reference_out = reference_points
pos_trans_out = self.pos_trans_norm(self.pos_trans(self.get_proposal_pos_embed(topk_coords_unact)))
query_embed, tgt = torch.split(pos_trans_out, c, dim=2)
else:
query_embed, tgt = torch.split(query_embed, c, dim=1)
query_embed = query_embed.unsqueeze(0).expand(bs, -1, -1)
tgt = tgt.unsqueeze(0).expand(bs, -1, -1)
reference_points = self.reference_points(query_embed).sigmoid()
init_reference_out = reference_points
# decoder
hs, inter_references = self.decoder(tgt, reference_points, memory,
spatial_shapes, level_start_index, valid_ratios, query_embed, mask_flatten)
inter_references_out = inter_references
if self.two_stage:
return hs, init_reference_out, inter_references_out, enc_outputs_class, enc_outputs_coord_unact
return hs, init_reference_out, inter_references_out, None, None
class DeformableTransformerEncoderLayer(nn.Module):
def __init__(self,
d_model=256, d_ffn=1024,
dropout=0.1, activation="relu",
n_levels=4, n_heads=8, n_points=4):
super().__init__()
# self attention
self.self_attn = MSDeformAttn(d_model, n_levels, n_heads, n_points)
self.dropout1 = nn.Dropout(dropout)
self.norm1 = nn.LayerNorm(d_model)
# ffn
self.linear1 = nn.Linear(d_model, d_ffn)
self.activation = _get_activation_fn(activation)
self.dropout2 = nn.Dropout(dropout)
self.linear2 = nn.Linear(d_ffn, d_model)
self.dropout3 = nn.Dropout(dropout)
self.norm2 = nn.LayerNorm(d_model)
@staticmethod
def with_pos_embed(tensor, pos):
return tensor if pos is None else tensor + pos
def forward_ffn(self, src):
src2 = self.linear2(self.dropout2(self.activation(self.linear1(src))))
src = src + self.dropout3(src2)
src = self.norm2(src)
return src
def forward(self, src, pos, reference_points, spatial_shapes, level_start_index, padding_mask=None):
# self attention
src2 = self.self_attn(self.with_pos_embed(src, pos), reference_points, src, spatial_shapes, level_start_index, padding_mask)
src = src + self.dropout1(src2)
src = self.norm1(src)
# ffn
src = self.forward_ffn(src)
return src
class DeformableTransformerEncoder(nn.Module):
def __init__(self, encoder_layer, num_layers, bf=None, bf_idx = 0, insert_idx=2, eval_bf=0):
super().__init__()
self.layers = _get_clones(encoder_layer, num_layers)
self.num_layers = num_layers
self.insert_idx = insert_idx
if type(bf) != torch.nn.ModuleList and bf is not None:
self.bf = [bf]*num_layers
else:
self.bf = bf
self.bf_idx = bf_idx
self.eval_bf = eval_bf
@staticmethod
def get_reference_points(spatial_shapes, valid_ratios, device):
reference_points_list = []
for lvl, (H_, W_) in enumerate(spatial_shapes):
ref_y, ref_x = torch.meshgrid(torch.linspace(0.5, H_ - 0.5, H_, dtype=torch.float32, device=device),
torch.linspace(0.5, W_ - 0.5, W_, dtype=torch.float32, device=device))
ref_y = ref_y.reshape(-1)[None] / (valid_ratios[:, None, lvl, 1] * H_)
ref_x = ref_x.reshape(-1)[None] / (valid_ratios[:, None, lvl, 0] * W_)
ref = torch.stack((ref_x, ref_y), -1)
reference_points_list.append(ref)
reference_points = torch.cat(reference_points_list, 1)
reference_points = reference_points[:, :, None] * valid_ratios[:, None]
return reference_points
def forward(self, src, spatial_shapes, level_start_index, valid_ratios, pos=None, padding_mask=None):
output = src
reference_points = self.get_reference_points(spatial_shapes, valid_ratios, device=src.device)
for i, layer in enumerate(self.layers):
output = layer(output, pos, reference_points, spatial_shapes, level_start_index, padding_mask)
if i in self.insert_idx and self.bf is not None and self.bf_idx in [1] and (self.training or self.eval_bf):
old_output = output
if i != self.insert_idx[0]: # If this is not the first layer, we split the batch into two streams because we have concatenate the two streams.
old_output = output[ :len(output)//2, :, :] # stream without batchformerv2
output = output[len(output)//2:, :, :] # strea with batchformerv2
output = self.bf[i](output) # if this is th first layer, we apply the batchformerv2 on the batch. then concatenate old_output and output
if self.training:
output = torch.cat([old_output, output], dim=0)
if i == self.insert_idx[0] and self.training: # for other variables, we just share it i the next modules. We thus repeat it.
pos = torch.cat([pos, pos], dim=0)
reference_points = torch.cat([reference_points, reference_points], dim=0)
padding_mask = torch.cat([padding_mask, padding_mask], dim=0)
elif i in self.insert_idx and self.bf is not None and self.bf_idx == 7 and self.training:
# single stream
output = self.bf[i](output)
return output
class DeformableTransformerDecoderLayer(nn.Module):
def __init__(self, d_model=256, d_ffn=1024,
dropout=0.1, activation="relu",
n_levels=4, n_heads=8, n_points=4):
super().__init__()
# cross attention
self.cross_attn = MSDeformAttn(d_model, n_levels, n_heads, n_points)
self.dropout1 = nn.Dropout(dropout)
self.norm1 = nn.LayerNorm(d_model)
# self attention
self.self_attn = nn.MultiheadAttention(d_model, n_heads, dropout=dropout)
self.dropout2 = nn.Dropout(dropout)
self.norm2 = nn.LayerNorm(d_model)
# ffn
self.linear1 = nn.Linear(d_model, d_ffn)
self.activation = _get_activation_fn(activation)
self.dropout3 = nn.Dropout(dropout)
self.linear2 = nn.Linear(d_ffn, d_model)
self.dropout4 = nn.Dropout(dropout)
self.norm3 = nn.LayerNorm(d_model)
@staticmethod
def with_pos_embed(tensor, pos):
return tensor if pos is None else tensor + pos
def forward_ffn(self, tgt):
tgt2 = self.linear2(self.dropout3(self.activation(self.linear1(tgt))))
tgt = tgt + self.dropout4(tgt2)
tgt = self.norm3(tgt)
return tgt
def forward(self, tgt, query_pos, reference_points, src, src_spatial_shapes, level_start_index, src_padding_mask=None):
# self attention
q = k = self.with_pos_embed(tgt, query_pos)
tgt2 = self.self_attn(q.transpose(0, 1), k.transpose(0, 1), tgt.transpose(0, 1))[0].transpose(0, 1)
tgt = tgt + self.dropout2(tgt2)
tgt = self.norm2(tgt)
# cross attention
tgt2 = self.cross_attn(self.with_pos_embed(tgt, query_pos),
reference_points,
src, src_spatial_shapes, level_start_index, src_padding_mask)
tgt = tgt + self.dropout1(tgt2)
tgt = self.norm1(tgt)
# ffn
tgt = self.forward_ffn(tgt)
return tgt
class DeformableTransformerDecoder(nn.Module):
def __init__(self, decoder_layer, num_layers, return_intermediate=False):
super().__init__()
self.layers = _get_clones(decoder_layer, num_layers)
self.num_layers = num_layers
self.return_intermediate = return_intermediate
# hack implementation for iterative bounding box refinement and two-stage Deformable DETR
self.bbox_embed = None
self.class_embed = None
def forward(self, tgt, reference_points, src, src_spatial_shapes, src_level_start_index, src_valid_ratios,
query_pos=None, src_padding_mask=None):
output = tgt
intermediate = []
intermediate_reference_points = []
for lid, layer in enumerate(self.layers):
if reference_points.shape[-1] == 4:
reference_points_input = reference_points[:, :, None] \
* torch.cat([src_valid_ratios, src_valid_ratios], -1)[:, None]
else:
assert reference_points.shape[-1] == 2
reference_points_input = reference_points[:, :, None] * src_valid_ratios[:, None]
output = layer(output, query_pos, reference_points_input, src, src_spatial_shapes, src_level_start_index, src_padding_mask)
# hack implementation for iterative bounding box refinement
if self.bbox_embed is not None:
tmp = self.bbox_embed[lid](output)
if reference_points.shape[-1] == 4:
new_reference_points = tmp + inverse_sigmoid(reference_points)
new_reference_points = new_reference_points.sigmoid()
else:
assert reference_points.shape[-1] == 2
new_reference_points = tmp
new_reference_points[..., :2] = tmp[..., :2] + inverse_sigmoid(reference_points)
new_reference_points = new_reference_points.sigmoid()
reference_points = new_reference_points.detach()
if self.return_intermediate:
intermediate.append(output)
intermediate_reference_points.append(reference_points)
if self.return_intermediate:
return torch.stack(intermediate), torch.stack(intermediate_reference_points)
return output, reference_points
def _get_clones(module, N):
return nn.ModuleList([copy.deepcopy(module) for i in range(N)])
def _get_activation_fn(activation):
"""Return an activation function given a string"""
if activation == "relu":
return F.relu
if activation == "gelu":
return F.gelu
if activation == "glu":
return F.glu
raise RuntimeError(F"activation should be relu/gelu, not {activation}.")
def build_deforamble_transformer(args):
return DeformableTransformer(
d_model=args.hidden_dim,
nhead=args.nheads,
num_encoder_layers=args.enc_layers,
num_decoder_layers=args.dec_layers,
dim_feedforward=args.dim_feedforward,
dropout=args.dropout,
activation="relu",
return_intermediate_dec=True,
num_feature_levels=args.num_feature_levels,
dec_n_points=args.dec_n_points,
enc_n_points=args.enc_n_points,
two_stage=args.two_stage,
two_stage_num_proposals=args.num_queries,
args=args)
================================================
FILE: batchformer-v2/deformable-detr/models/matcher.py
================================================
# ------------------------------------------------------------------------
# Deformable DETR
# Copyright (c) 2020 SenseTime. All Rights Reserved.
# Licensed under the Apache License, Version 2.0 [see LICENSE for details]
# ------------------------------------------------------------------------
# Modified from DETR (https://github.com/facebookresearch/detr)
# Copyright (c) Facebook, Inc. and its affiliates. All Rights Reserved
# ------------------------------------------------------------------------
"""
Modules to compute the matching cost and solve the corresponding LSAP.
"""
import torch
from scipy.optimize import linear_sum_assignment
from torch import nn
from util.box_ops import box_cxcywh_to_xyxy, generalized_box_iou
class HungarianMatcher(nn.Module):
"""This class computes an assignment between the targets and the predictions of the network
For efficiency reasons, the targets don't include the no_object. Because of this, in general,
there are more predictions than targets. In this case, we do a 1-to-1 matching of the best predictions,
while the others are un-matched (and thus treated as non-objects).
"""
def __init__(self,
cost_class: float = 1,
cost_bbox: float = 1,
cost_giou: float = 1):
"""Creates the matcher
Params:
cost_class: This is the relative weight of the classification error in the matching cost
cost_bbox: This is the relative weight of the L1 error of the bounding box coordinates in the matching cost
cost_giou: This is the relative weight of the giou loss of the bounding box in the matching cost
"""
super().__init__()
self.cost_class = cost_class
self.cost_bbox = cost_bbox
self.cost_giou = cost_giou
assert cost_class != 0 or cost_bbox != 0 or cost_giou != 0, "all costs cant be 0"
def forward(self, outputs, targets):
""" Performs the matching
Params:
outputs: This is a dict that contains at least these entries:
"pred_logits": Tensor of dim [batch_size, num_queries, num_classes] with the classification logits
"pred_boxes": Tensor of dim [batch_size, num_queries, 4] with the predicted box coordinates
targets: This is a list of targets (len(targets) = batch_size), where each target is a dict containing:
"labels": Tensor of dim [num_target_boxes] (where num_target_boxes is the number of ground-truth
objects in the target) containing the class labels
"boxes": Tensor of dim [num_target_boxes, 4] containing the target box coordinates
Returns:
A list of size batch_size, containing tuples of (index_i, index_j) where:
- index_i is the indices of the selected predictions (in order)
- index_j is the indices of the corresponding selected targets (in order)
For each batch element, it holds:
len(index_i) = len(index_j) = min(num_queries, num_target_boxes)
"""
with torch.no_grad():
bs, num_queries = outputs["pred_logits"].shape[:2]
# We flatten to compute the cost matrices in a batch
out_prob = outputs["pred_logits"].flatten(0, 1).sigmoid()
out_bbox = outputs["pred_boxes"].flatten(0, 1) # [batch_size * num_queries, 4]
# Also concat the target labels and boxes
tgt_ids = torch.cat([v["labels"] for v in targets])
tgt_bbox = torch.cat([v["boxes"] for v in targets])
# Compute the classification cost.
alpha = 0.25
gamma = 2.0
neg_cost_class = (1 - alpha) * (out_prob ** gamma) * (-(1 - out_prob + 1e-8).log())
pos_cost_class = alpha * ((1 - out_prob) ** gamma) * (-(out_prob + 1e-8).log())
cost_class = pos_cost_class[:, tgt_ids] - neg_cost_class[:, tgt_ids]
# Compute the L1 cost between boxes
cost_bbox = torch.cdist(out_bbox, tgt_bbox, p=1)
# Compute the giou cost betwen boxes
cost_giou = -generalized_box_iou(box_cxcywh_to_xyxy(out_bbox),
box_cxcywh_to_xyxy(tgt_bbox))
# Final cost matrix
C = self.cost_bbox * cost_bbox + self.cost_class * cost_class + self.cost_giou * cost_giou
C = C.view(bs, num_queries, -1).cpu()
sizes = [len(v["boxes"]) for v in targets]
indices = [linear_sum_assignment(c[i]) for i, c in enumerate(C.split(sizes, -1))]
return [(torch.as_tensor(i, dtype=torch.int64), torch.as_tensor(j, dtype=torch.int64)) for i, j in indices]
def build_matcher(args):
return HungarianMatcher(cost_class=args.set_cost_class,
cost_bbox=args.set_cost_bbox,
cost_giou=args.set_cost_giou)
================================================
FILE: batchformer-v2/deformable-detr/models/ops/functions/__init__.py
================================================
# ------------------------------------------------------------------------------------------------
# Deformable DETR
# Copyright (c) 2020 SenseTime. All Rights Reserved.
# Licensed under the Apache License, Version 2.0 [see LICENSE for details]
# ------------------------------------------------------------------------------------------------
# Modified from https://github.com/chengdazhi/Deformable-Convolution-V2-PyTorch/tree/pytorch_1.0.0
# ------------------------------------------------------------------------------------------------
from .ms_deform_attn_func import MSDeformAttnFunction
================================================
FILE: batchformer-v2/deformable-detr/models/ops/functions/ms_deform_attn_func.py
================================================
# ------------------------------------------------------------------------------------------------
# Deformable DETR
# Copyright (c) 2020 SenseTime. All Rights Reserved.
# Licensed under the Apache License, Version 2.0 [see LICENSE for details]
# ------------------------------------------------------------------------------------------------
# Modified from https://github.com/chengdazhi/Deformable-Convolution-V2-PyTorch/tree/pytorch_1.0.0
# ------------------------------------------------------------------------------------------------
from __future__ import absolute_import
from __future__ import print_function
from __future__ import division
import torch
import torch.nn.functional as F
from torch.autograd import Function
from torch.autograd.function import once_differentiable
import MultiScaleDeformableAttention as MSDA
class MSDeformAttnFunction(Function):
@staticmethod
def forward(ctx, value, value_spatial_shapes, value_level_start_index, sampling_locations, attention_weights, im2col_step):
ctx.im2col_step = im2col_step
output = MSDA.ms_deform_attn_forward(
value, value_spatial_shapes, value_level_start_index, sampling_locations, attention_weights, ctx.im2col_step)
ctx.save_for_backward(value, value_spatial_shapes, value_level_start_index, sampling_locations, attention_weights)
return output
@staticmethod
@once_differentiable
def backward(ctx, grad_output):
value, value_spatial_shapes, value_level_start_index, sampling_locations, attention_weights = ctx.saved_tensors
grad_value, grad_sampling_loc, grad_attn_weight = \
MSDA.ms_deform_attn_backward(
value, value_spatial_shapes, value_level_start_index, sampling_locations, attention_weights, grad_output, ctx.im2col_step)
return grad_value, None, None, grad_sampling_loc, grad_attn_weight, None
def ms_deform_attn_core_pytorch(value, value_spatial_shapes, sampling_locations, attention_weights):
# for debug and test only,
# need to use cuda version instead
N_, S_, M_, D_ = value.shape
_, Lq_, M_, L_, P_, _ = sampling_locations.shape
value_list = value.split([H_ * W_ for H_, W_ in value_spatial_shapes], dim=1)
sampling_grids = 2 * sampling_locations - 1
sampling_value_list = []
for lid_, (H_, W_) in enumerate(value_spatial_shapes):
# N_, H_*W_, M_, D_ -> N_, H_*W_, M_*D_ -> N_, M_*D_, H_*W_ -> N_*M_, D_, H_, W_
value_l_ = value_list[lid_].flatten(2).transpose(1, 2).reshape(N_*M_, D_, H_, W_)
# N_, Lq_, M_, P_, 2 -> N_, M_, Lq_, P_, 2 -> N_*M_, Lq_, P_, 2
sampling_grid_l_ = sampling_grids[:, :, :, lid_].transpose(1, 2).flatten(0, 1)
# N_*M_, D_, Lq_, P_
sampling_value_l_ = F.grid_sample(value_l_, sampling_grid_l_,
mode='bilinear', padding_mode='zeros', align_corners=False)
sampling_value_list.append(sampling_value_l_)
# (N_, Lq_, M_, L_, P_) -> (N_, M_, Lq_, L_, P_) -> (N_, M_, 1, Lq_, L_*P_)
attention_weights = attention_weights.transpose(1, 2).reshape(N_*M_, 1, Lq_, L_*P_)
output = (torch.stack(sampling_value_list, dim=-2).flatten(-2) * attention_weights).sum(-1).view(N_, M_*D_, Lq_)
return output.transpose(1, 2).contiguous()
================================================
FILE: batchformer-v2/deformable-detr/models/ops/make.sh
================================================
#!/usr/bin/env bash
# ------------------------------------------------------------------------------------------------
# Deformable DETR
# Copyright (c) 2020 SenseTime. All Rights Reserved.
# Licensed under the Apache License, Version 2.0 [see LICENSE for details]
# ------------------------------------------------------------------------------------------------
# Modified from https://github.com/chengdazhi/Deformable-Convolution-V2-PyTorch/tree/pytorch_1.0.0
# ------------------------------------------------------------------------------------------------
python setup.py build install
================================================
FILE: batchformer-v2/deformable-detr/models/ops/modules/__init__.py
================================================
# ------------------------------------------------------------------------------------------------
# Deformable DETR
# Copyright (c) 2020 SenseTime. All Rights Reserved.
# Licensed under the Apache License, Version 2.0 [see LICENSE for details]
# ------------------------------------------------------------------------------------------------
# Modified from https://github.com/chengdazhi/Deformable-Convolution-V2-PyTorch/tree/pytorch_1.0.0
# ------------------------------------------------------------------------------------------------
from .ms_deform_attn import MSDeformAttn
================================================
FILE: batchformer-v2/deformable-detr/models/ops/modules/ms_deform_attn.py
================================================
# ------------------------------------------------------------------------------------------------
# Deformable DETR
# Copyright (c) 2020 SenseTime. All Rights Reserved.
# Licensed under the Apache License, Version 2.0 [see LICENSE for details]
# ------------------------------------------------------------------------------------------------
# Modified from https://github.com/chengdazhi/Deformable-Convolution-V2-PyTorch/tree/pytorch_1.0.0
# ------------------------------------------------------------------------------------------------
from __future__ import absolute_import
from __future__ import print_function
from __future__ import division
import warnings
import math
import torch
from torch import nn
import torch.nn.functional as F
from torch.nn.init import xavier_uniform_, constant_
from ..functions import MSDeformAttnFunction
def _is_power_of_2(n):
if (not isinstance(n, int)) or (n < 0):
raise ValueError("invalid input for _is_power_of_2: {} (type: {})".format(n, type(n)))
return (n & (n-1) == 0) and n != 0
class MSDeformAttn(nn.Module):
def __init__(self, d_model=256, n_levels=4, n_heads=8, n_points=4):
"""
Multi-Scale Deformable Attention Module
:param d_model hidden dimension
:param n_levels number of feature levels
:param n_heads number of attention heads
:param n_points number of sampling points per attention head per feature level
"""
super().__init__()
if d_model % n_heads != 0:
raise ValueError('d_model must be divisible by n_heads, but got {} and {}'.format(d_model, n_heads))
_d_per_head = d_model // n_heads
# you'd better set _d_per_head to a power of 2 which is more efficient in our CUDA implementation
if not _is_power_of_2(_d_per_head):
warnings.warn("You'd better set d_model in MSDeformAttn to make the dimension of each attention head a power of 2 "
"which is more efficient in our CUDA implementation.")
self.im2col_step = 64
self.d_model = d_model
self.n_levels = n_levels
self.n_heads = n_heads
self.n_points = n_points
self.sampling_offsets = nn.Linear(d_model, n_heads * n_levels * n_points * 2)
self.attention_weights = nn.Linear(d_model, n_heads * n_levels * n_points)
self.value_proj = nn.Linear(d_model, d_model)
self.output_proj = nn.Linear(d_model, d_model)
self._reset_parameters()
def _reset_parameters(self):
constant_(self.sampling_offsets.weight.data, 0.)
thetas = torch.arange(self.n_heads, dtype=torch.float32) * (2.0 * math.pi / self.n_heads)
grid_init = torch.stack([thetas.cos(), thetas.sin()], -1)
grid_init = (grid_init / grid_init.abs().max(-1, keepdim=True)[0]).view(self.n_heads, 1, 1, 2).repeat(1, self.n_levels, self.n_points, 1)
for i in range(self.n_points):
grid_init[:, :, i, :] *= i + 1
with torch.no_grad():
self.sampling_offsets.bias = nn.Parameter(grid_init.view(-1))
constant_(self.attention_weights.weight.data, 0.)
constant_(self.attention_weights.bias.data, 0.)
xavier_uniform_(self.value_proj.weight.data)
constant_(self.value_proj.bias.data, 0.)
xavier_uniform_(self.output_proj.weight.data)
constant_(self.output_proj.bias.data, 0.)
def forward(self, query, reference_points, input_flatten, input_spatial_shapes, input_level_start_index, input_padding_mask=None):
"""
:param query (N, Length_{query}, C)
:param reference_points (N, Length_{query}, n_levels, 2), range in [0, 1], top-left (0,0), bottom-right (1, 1), including padding area
or (N, Length_{query}, n_levels, 4), add additional (w, h) to form reference boxes
:param input_flatten (N, \sum_{l=0}^{L-1} H_l \cdot W_l, C)
:param input_spatial_shapes (n_levels, 2), [(H_0, W_0), (H_1, W_1), ..., (H_{L-1}, W_{L-1})]
:param input_level_start_index (n_levels, ), [0, H_0*W_0, H_0*W_0+H_1*W_1, H_0*W_0+H_1*W_1+H_2*W_2, ..., H_0*W_0+H_1*W_1+...+H_{L-1}*W_{L-1}]
:param input_padding_mask (N, \sum_{l=0}^{L-1} H_l \cdot W_l), True for padding elements, False for non-padding elements
:return output (N, Length_{query}, C)
"""
N, Len_q, _ = query.shape
N, Len_in, _ = input_flatten.shape
assert (input_spatial_shapes[:, 0] * input_spatial_shapes[:, 1]).sum() == Len_in
value = self.value_proj(input_flatten)
if input_padding_mask is not None:
value = value.masked_fill(input_padding_mask[..., None], float(0))
value = value.view(N, Len_in, self.n_heads, self.d_model // self.n_heads)
sampling_offsets = self.sampling_offsets(query).view(N, Len_q, self.n_heads, self.n_levels, self.n_points, 2)
attention_weights = self.attention_weights(query).view(N, Len_q, self.n_heads, self.n_levels * self.n_points)
attention_weights = F.softmax(attention_weights, -1).view(N, Len_q, self.n_heads, self.n_levels, self.n_points)
# N, Len_q, n_heads, n_levels, n_points, 2
if reference_points.shape[-1] == 2:
offset_normalizer = torch.stack([input_spatial_shapes[..., 1], input_spatial_shapes[..., 0]], -1)
sampling_locations = reference_points[:, :, None, :, None, :] \
+ sampling_offsets / offset_normalizer[None, None, None, :, None, :]
elif reference_points.shape[-1] == 4:
sampling_locations = reference_points[:, :, None, :, None, :2] \
+ sampling_offsets / self.n_points * reference_points[:, :, None, :, None, 2:] * 0.5
else:
raise ValueError(
'Last dim of reference_points must be 2 or 4, but get {} instead.'.format(reference_points.shape[-1]))
output = MSDeformAttnFunction.apply(
value, input_spatial_shapes, input_level_start_index, sampling_locations, attention_weights, self.im2col_step)
output = self.output_proj(output)
return output
================================================
FILE: batchformer-v2/deformable-detr/models/ops/setup.py
================================================
# ------------------------------------------------------------------------------------------------
# Deformable DETR
# Copyright (c) 2020 SenseTime. All Rights Reserved.
# Licensed under the Apache License, Version 2.0 [see LICENSE for details]
# ------------------------------------------------------------------------------------------------
# Modified from https://github.com/chengdazhi/Deformable-Convolution-V2-PyTorch/tree/pytorch_1.0.0
# ------------------------------------------------------------------------------------------------
import os
import glob
import torch
from torch.utils.cpp_extension import CUDA_HOME
from torch.utils.cpp_extension import CppExtension
from torch.utils.cpp_extension import CUDAExtension
from setuptools import find_packages
from setuptools import setup
requirements = ["torch", "torchvision"]
def get_extensions():
this_dir = os.path.dirname(os.path.abspath(__file__))
extensions_dir = os.path.join(this_dir, "src")
main_file = glob.glob(os.path.join(extensions_dir, "*.cpp"))
source_cpu = glob.glob(os.path.join(extensions_dir, "cpu", "*.cpp"))
source_cuda = glob.glob(os.path.join(extensions_dir, "cuda", "*.cu"))
sources = main_file + source_cpu
extension = CppExtension
extra_compile_args = {"cxx": []}
define_macros = []
if torch.cuda.is_available() and CUDA_HOME is not None:
extension = CUDAExtension
sources += source_cuda
define_macros += [("WITH_CUDA", None)]
extra_compile_args["nvcc"] = [
"-DCUDA_HAS_FP16=1",
"-D__CUDA_NO_HALF_OPERATORS__",
"-D__CUDA_NO_HALF_CONVERSIONS__",
"-D__CUDA_NO_HALF2_OPERATORS__",
]
else:
raise NotImplementedError('Cuda is not availabel')
sources = [os.path.join(extensions_dir, s) for s in sources]
include_dirs = [extensions_dir]
ext_modules = [
extension(
"MultiScaleDeformableAttention",
sources,
include_dirs=include_dirs,
define_macros=define_macros,
extra_compile_args=extra_compile_args,
)
]
return ext_modules
setup(
name="MultiScaleDeformableAttention",
version="1.0",
author="Weijie Su",
url="https://github.com/fundamentalvision/Deformable-DETR",
description="PyTorch Wrapper for CUDA Functions of Multi-Scale Deformable Attention",
packages=find_packages(exclude=("configs", "tests",)),
ext_modules=get_extensions(),
cmdclass={"build_ext": torch.utils.cpp_extension.BuildExtension},
)
================================================
FILE: batchformer-v2/deformable-detr/models/ops/src/cpu/ms_deform_attn_cpu.cpp
================================================
/*!
**************************************************************************************************
* Deformable DETR
* Copyright (c) 2020 SenseTime. All Rights Reserved.
* Licensed under the Apache License, Version 2.0 [see LICENSE for details]
**************************************************************************************************
* Modified from https://github.com/chengdazhi/Deformable-Convolution-V2-PyTorch/tree/pytorch_1.0.0
**************************************************************************************************
*/
#include
#include
#include
at::Tensor
ms_deform_attn_cpu_forward(
const at::Tensor &value,
const at::Tensor &spatial_shapes,
const at::Tensor &level_start_index,
const at::Tensor &sampling_loc,
const at::Tensor &attn_weight,
const int im2col_step)
{
AT_ERROR("Not implement on cpu");
}
std::vector
ms_deform_attn_cpu_backward(
const at::Tensor &value,
const at::Tensor &spatial_shapes,
const at::Tensor &level_start_index,
const at::Tensor &sampling_loc,
const at::Tensor &attn_weight,
const at::Tensor &grad_output,
const int im2col_step)
{
AT_ERROR("Not implement on cpu");
}
================================================
FILE: batchformer-v2/deformable-detr/models/ops/src/cpu/ms_deform_attn_cpu.h
================================================
/*!
**************************************************************************************************
* Deformable DETR
* Copyright (c) 2020 SenseTime. All Rights Reserved.
* Licensed under the Apache License, Version 2.0 [see LICENSE for details]
**************************************************************************************************
* Modified from https://github.com/chengdazhi/Deformable-Convolution-V2-PyTorch/tree/pytorch_1.0.0
**************************************************************************************************
*/
#pragma once
#include
at::Tensor
ms_deform_attn_cpu_forward(
const at::Tensor &value,
const at::Tensor &spatial_shapes,
const at::Tensor &level_start_index,
const at::Tensor &sampling_loc,
const at::Tensor &attn_weight,
const int im2col_step);
std::vector
ms_deform_attn_cpu_backward(
const at::Tensor &value,
const at::Tensor &spatial_shapes,
const at::Tensor &level_start_index,
const at::Tensor &sampling_loc,
const at::Tensor &attn_weight,
const at::Tensor &grad_output,
const int im2col_step);
================================================
FILE: batchformer-v2/deformable-detr/models/ops/src/cuda/ms_deform_attn_cuda.cu
================================================
/*!
**************************************************************************************************
* Deformable DETR
* Copyright (c) 2020 SenseTime. All Rights Reserved.
* Licensed under the Apache License, Version 2.0 [see LICENSE for details]
**************************************************************************************************
* Modified from https://github.com/chengdazhi/Deformable-Convolution-V2-PyTorch/tree/pytorch_1.0.0
**************************************************************************************************
*/
#include
#include "cuda/ms_deform_im2col_cuda.cuh"
#include
#include
#include
#include
at::Tensor ms_deform_attn_cuda_forward(
const at::Tensor &value,
const at::Tensor &spatial_shapes,
const at::Tensor &level_start_index,
const at::Tensor &sampling_loc,
const at::Tensor &attn_weight,
const int im2col_step)
{
AT_ASSERTM(value.is_contiguous(), "value tensor has to be contiguous");
AT_ASSERTM(spatial_shapes.is_contiguous(), "spatial_shapes tensor has to be contiguous");
AT_ASSERTM(level_start_index.is_contiguous(), "level_start_index tensor has to be contiguous");
AT_ASSERTM(sampling_loc.is_contiguous(), "sampling_loc tensor has to be contiguous");
AT_ASSERTM(attn_weight.is_contiguous(), "attn_weight tensor has to be contiguous");
AT_ASSERTM(value.type().is_cuda(), "value must be a CUDA tensor");
AT_ASSERTM(spatial_shapes.type().is_cuda(), "spatial_shapes must be a CUDA tensor");
AT_ASSERTM(level_start_index.type().is_cuda(), "level_start_index must be a CUDA tensor");
AT_ASSERTM(sampling_loc.type().is_cuda(), "sampling_loc must be a CUDA tensor");
AT_ASSERTM(attn_weight.type().is_cuda(), "attn_weight must be a CUDA tensor");
const int batch = value.size(0);
const int spatial_size = value.size(1);
const int num_heads = value.size(2);
const int channels = value.size(3);
const int num_levels = spatial_shapes.size(0);
const int num_query = sampling_loc.size(1);
const int num_point = sampling_loc.size(4);
const int im2col_step_ = std::min(batch, im2col_step);
AT_ASSERTM(batch % im2col_step_ == 0, "batch(%d) must divide im2col_step(%d)", batch, im2col_step_);
auto output = at::zeros({batch, num_query, num_heads, channels}, value.options());
const int batch_n = im2col_step_;
auto output_n = output.view({batch/im2col_step_, batch_n, num_query, num_heads, channels});
auto per_value_size = spatial_size * num_heads * channels;
auto per_sample_loc_size = num_query * num_heads * num_levels * num_point * 2;
auto per_attn_weight_size = num_query * num_heads * num_levels * num_point;
for (int n = 0; n < batch/im2col_step_; ++n)
{
auto columns = output_n.select(0, n);
AT_DISPATCH_FLOATING_TYPES(value.type(), "ms_deform_attn_forward_cuda", ([&] {
ms_deformable_im2col_cuda(at::cuda::getCurrentCUDAStream(),
value.data() + n * im2col_step_ * per_value_size,
spatial_shapes.data(),
level_start_index.data(),
sampling_loc.data() + n * im2col_step_ * per_sample_loc_size,
attn_weight.data() + n * im2col_step_ * per_attn_weight_size,
batch_n, spatial_size, num_heads, channels, num_levels, num_query, num_point,
columns.data());
}));
}
output = output.view({batch, num_query, num_heads*channels});
return output;
}
std::vector ms_deform_attn_cuda_backward(
const at::Tensor &value,
const at::Tensor &spatial_shapes,
const at::Tensor &level_start_index,
const at::Tensor &sampling_loc,
const at::Tensor &attn_weight,
const at::Tensor &grad_output,
const int im2col_step)
{
AT_ASSERTM(value.is_contiguous(), "value tensor has to be contiguous");
AT_ASSERTM(spatial_shapes.is_contiguous(), "spatial_shapes tensor has to be contiguous");
AT_ASSERTM(level_start_index.is_contiguous(), "level_start_index tensor has to be contiguous");
AT_ASSERTM(sampling_loc.is_contiguous(), "sampling_loc tensor has to be contiguous");
AT_ASSERTM(attn_weight.is_contiguous(), "attn_weight tensor has to be contiguous");
AT_ASSERTM(grad_output.is_contiguous(), "grad_output tensor has to be contiguous");
AT_ASSERTM(value.type().is_cuda(), "value must be a CUDA tensor");
AT_ASSERTM(spatial_shapes.type().is_cuda(), "spatial_shapes must be a CUDA tensor");
AT_ASSERTM(level_start_index.type().is_cuda(), "level_start_index must be a CUDA tensor");
AT_ASSERTM(sampling_loc.type().is_cuda(), "sampling_loc must be a CUDA tensor");
AT_ASSERTM(attn_weight.type().is_cuda(), "attn_weight must be a CUDA tensor");
AT_ASSERTM(grad_output.type().is_cuda(), "grad_output must be a CUDA tensor");
const int batch = value.size(0);
const int spatial_size = value.size(1);
const int num_heads = value.size(2);
const int channels = value.size(3);
const int num_levels = spatial_shapes.size(0);
const int num_query = sampling_loc.size(1);
const int num_point = sampling_loc.size(4);
const int im2col_step_ = std::min(batch, im2col_step);
AT_ASSERTM(batch % im2col_step_ == 0, "batch(%d) must divide im2col_step(%d)", batch, im2col_step_);
auto grad_value = at::zeros_like(value);
auto grad_sampling_loc = at::zeros_like(sampling_loc);
auto grad_attn_weight = at::zeros_like(attn_weight);
const int batch_n = im2col_step_;
auto per_value_size = spatial_size * num_heads * channels;
auto per_sample_loc_size = num_query * num_heads * num_levels * num_point * 2;
auto per_attn_weight_size = num_query * num_heads * num_levels * num_point;
auto grad_output_n = grad_output.view({batch/im2col_step_, batch_n, num_query, num_heads, channels});
for (int n = 0; n < batch/im2col_step_; ++n)
{
auto grad_output_g = grad_output_n.select(0, n);
AT_DISPATCH_FLOATING_TYPES(value.type(), "ms_deform_attn_backward_cuda", ([&] {
ms_deformable_col2im_cuda(at::cuda::getCurrentCUDAStream(),
grad_output_g.data(),
value.data() + n * im2col_step_ * per_value_size,
spatial_shapes.data(),
level_start_index.data(),
sampling_loc.data() + n * im2col_step_ * per_sample_loc_size,
attn_weight.data() + n * im2col_step_ * per_attn_weight_size,
batch_n, spatial_size, num_heads, channels, num_levels, num_query, num_point,
grad_value.data() + n * im2col_step_ * per_value_size,
grad_sampling_loc.data() + n * im2col_step_ * per_sample_loc_size,
grad_attn_weight.data() + n * im2col_step_ * per_attn_weight_size);
}));
}
return {
grad_value, grad_sampling_loc, grad_attn_weight
};
}
================================================
FILE: batchformer-v2/deformable-detr/models/ops/src/cuda/ms_deform_attn_cuda.h
================================================
/*!
**************************************************************************************************
* Deformable DETR
* Copyright (c) 2020 SenseTime. All Rights Reserved.
* Licensed under the Apache License, Version 2.0 [see LICENSE for details]
**************************************************************************************************
* Modified from https://github.com/chengdazhi/Deformable-Convolution-V2-PyTorch/tree/pytorch_1.0.0
**************************************************************************************************
*/
#pragma once
#include
at::Tensor ms_deform_attn_cuda_forward(
const at::Tensor &value,
const at::Tensor &spatial_shapes,
const at::Tensor &level_start_index,
const at::Tensor &sampling_loc,
const at::Tensor &attn_weight,
const int im2col_step);
std::vector ms_deform_attn_cuda_backward(
const at::Tensor &value,
const at::Tensor &spatial_shapes,
const at::Tensor &level_start_index,
const at::Tensor &sampling_loc,
const at::Tensor &attn_weight,
const at::Tensor &grad_output,
const int im2col_step);
================================================
FILE: batchformer-v2/deformable-detr/models/ops/src/cuda/ms_deform_im2col_cuda.cuh
================================================
/*!
**************************************************************************
* Deformable DETR
* Copyright (c) 2020 SenseTime. All Rights Reserved.
* Licensed under the Apache License, Version 2.0 [see LICENSE for details]
**************************************************************************
* Modified from DCN (https://github.com/msracver/Deformable-ConvNets)
* Copyright (c) 2018 Microsoft
**************************************************************************
*/
#include
#include
#include
#include
#include
#include
#define CUDA_KERNEL_LOOP(i, n) \
for (int i = blockIdx.x * blockDim.x + threadIdx.x; \
i < (n); \
i += blockDim.x * gridDim.x)
const int CUDA_NUM_THREADS = 1024;
inline int GET_BLOCKS(const int N, const int num_threads)
{
return (N + num_threads - 1) / num_threads;
}
template
__device__ scalar_t ms_deform_attn_im2col_bilinear(const scalar_t* &bottom_data,
const int &height, const int &width, const int &nheads, const int &channels,
const scalar_t &h, const scalar_t &w, const int &m, const int &c)
{
const int h_low = floor(h);
const int w_low = floor(w);
const int h_high = h_low + 1;
const int w_high = w_low + 1;
const scalar_t lh = h - h_low;
const scalar_t lw = w - w_low;
const scalar_t hh = 1 - lh, hw = 1 - lw;
const int w_stride = nheads * channels;
const int h_stride = width * w_stride;
const int h_low_ptr_offset = h_low * h_stride;
const int h_high_ptr_offset = h_low_ptr_offset + h_stride;
const int w_low_ptr_offset = w_low * w_stride;
const int w_high_ptr_offset = w_low_ptr_offset + w_stride;
const int base_ptr = m * channels + c;
scalar_t v1 = 0;
if (h_low >= 0 && w_low >= 0)
{
const int ptr1 = h_low_ptr_offset + w_low_ptr_offset + base_ptr;
v1 = bottom_data[ptr1];
}
scalar_t v2 = 0;
if (h_low >= 0 && w_high <= width - 1)
{
const int ptr2 = h_low_ptr_offset + w_high_ptr_offset + base_ptr;
v2 = bottom_data[ptr2];
}
scalar_t v3 = 0;
if (h_high <= height - 1 && w_low >= 0)
{
const int ptr3 = h_high_ptr_offset + w_low_ptr_offset + base_ptr;
v3 = bottom_data[ptr3];
}
scalar_t v4 = 0;
if (h_high <= height - 1 && w_high <= width - 1)
{
const int ptr4 = h_high_ptr_offset + w_high_ptr_offset + base_ptr;
v4 = bottom_data[ptr4];
}
const scalar_t w1 = hh * hw, w2 = hh * lw, w3 = lh * hw, w4 = lh * lw;
const scalar_t val = (w1 * v1 + w2 * v2 + w3 * v3 + w4 * v4);
return val;
}
template
__device__ void ms_deform_attn_col2im_bilinear(const scalar_t* &bottom_data,
const int &height, const int &width, const int &nheads, const int &channels,
const scalar_t &h, const scalar_t &w, const int &m, const int &c,
const scalar_t &top_grad,
const scalar_t &attn_weight,
scalar_t* &grad_value,
scalar_t* grad_sampling_loc,
scalar_t* grad_attn_weight)
{
const int h_low = floor(h);
const int w_low = floor(w);
const int h_high = h_low + 1;
const int w_high = w_low + 1;
const scalar_t lh = h - h_low;
const scalar_t lw = w - w_low;
const scalar_t hh = 1 - lh, hw = 1 - lw;
const int w_stride = nheads * channels;
const int h_stride = width * w_stride;
const int h_low_ptr_offset = h_low * h_stride;
const int h_high_ptr_offset = h_low_ptr_offset + h_stride;
const int w_low_ptr_offset = w_low * w_stride;
const int w_high_ptr_offset = w_low_ptr_offset + w_stride;
const int base_ptr = m * channels + c;
const scalar_t w1 = hh * hw, w2 = hh * lw, w3 = lh * hw, w4 = lh * lw;
const scalar_t top_grad_value = top_grad * attn_weight;
scalar_t grad_h_weight = 0, grad_w_weight = 0;
scalar_t v1 = 0;
if (h_low >= 0 && w_low >= 0)
{
const int ptr1 = h_low_ptr_offset + w_low_ptr_offset + base_ptr;
v1 = bottom_data[ptr1];
grad_h_weight -= hw * v1;
grad_w_weight -= hh * v1;
atomicAdd(grad_value+ptr1, w1*top_grad_value);
}
scalar_t v2 = 0;
if (h_low >= 0 && w_high <= width - 1)
{
const int ptr2 = h_low_ptr_offset + w_high_ptr_offset + base_ptr;
v2 = bottom_data[ptr2];
grad_h_weight -= lw * v2;
grad_w_weight += hh * v2;
atomicAdd(grad_value+ptr2, w2*top_grad_value);
}
scalar_t v3 = 0;
if (h_high <= height - 1 && w_low >= 0)
{
const int ptr3 = h_high_ptr_offset + w_low_ptr_offset + base_ptr;
v3 = bottom_data[ptr3];
grad_h_weight += hw * v3;
grad_w_weight -= lh * v3;
atomicAdd(grad_value+ptr3, w3*top_grad_value);
}
scalar_t v4 = 0;
if (h_high <= height - 1 && w_high <= width - 1)
{
const int ptr4 = h_high_ptr_offset + w_high_ptr_offset + base_ptr;
v4 = bottom_data[ptr4];
grad_h_weight += lw * v4;
grad_w_weight += lh * v4;
atomicAdd(grad_value+ptr4, w4*top_grad_value);
}
const scalar_t val = (w1 * v1 + w2 * v2 + w3 * v3 + w4 * v4);
*grad_attn_weight = top_grad * val;
*grad_sampling_loc = width * grad_w_weight * top_grad_value;
*(grad_sampling_loc + 1) = height * grad_h_weight * top_grad_value;
}
template
__device__ void ms_deform_attn_col2im_bilinear_gm(const scalar_t* &bottom_data,
const int &height, const int &width, const int &nheads, const int &channels,
const scalar_t &h, const scalar_t &w, const int &m, const int &c,
const scalar_t &top_grad,
const scalar_t &attn_weight,
scalar_t* &grad_value,
scalar_t* grad_sampling_loc,
scalar_t* grad_attn_weight)
{
const int h_low = floor(h);
const int w_low = floor(w);
const int h_high = h_low + 1;
const int w_high = w_low + 1;
const scalar_t lh = h - h_low;
const scalar_t lw = w - w_low;
const scalar_t hh = 1 - lh, hw = 1 - lw;
const int w_stride = nheads * channels;
const int h_stride = width * w_stride;
const int h_low_ptr_offset = h_low * h_stride;
const int h_high_ptr_offset = h_low_ptr_offset + h_stride;
const int w_low_ptr_offset = w_low * w_stride;
const int w_high_ptr_offset = w_low_ptr_offset + w_stride;
const int base_ptr = m * channels + c;
const scalar_t w1 = hh * hw, w2 = hh * lw, w3 = lh * hw, w4 = lh * lw;
const scalar_t top_grad_value = top_grad * attn_weight;
scalar_t grad_h_weight = 0, grad_w_weight = 0;
scalar_t v1 = 0;
if (h_low >= 0 && w_low >= 0)
{
const int ptr1 = h_low_ptr_offset + w_low_ptr_offset + base_ptr;
v1 = bottom_data[ptr1];
grad_h_weight -= hw * v1;
grad_w_weight -= hh * v1;
atomicAdd(grad_value+ptr1, w1*top_grad_value);
}
scalar_t v2 = 0;
if (h_low >= 0 && w_high <= width - 1)
{
const int ptr2 = h_low_ptr_offset + w_high_ptr_offset + base_ptr;
v2 = bottom_data[ptr2];
grad_h_weight -= lw * v2;
grad_w_weight += hh * v2;
atomicAdd(grad_value+ptr2, w2*top_grad_value);
}
scalar_t v3 = 0;
if (h_high <= height - 1 && w_low >= 0)
{
const int ptr3 = h_high_ptr_offset + w_low_ptr_offset + base_ptr;
v3 = bottom_data[ptr3];
grad_h_weight += hw * v3;
grad_w_weight -= lh * v3;
atomicAdd(grad_value+ptr3, w3*top_grad_value);
}
scalar_t v4 = 0;
if (h_high <= height - 1 && w_high <= width - 1)
{
const int ptr4 = h_high_ptr_offset + w_high_ptr_offset + base_ptr;
v4 = bottom_data[ptr4];
grad_h_weight += lw * v4;
grad_w_weight += lh * v4;
atomicAdd(grad_value+ptr4, w4*top_grad_value);
}
const scalar_t val = (w1 * v1 + w2 * v2 + w3 * v3 + w4 * v4);
atomicAdd(grad_attn_weight, top_grad * val);
atomicAdd(grad_sampling_loc, width * grad_w_weight * top_grad_value);
atomicAdd(grad_sampling_loc + 1, height * grad_h_weight * top_grad_value);
}
template
__global__ void ms_deformable_im2col_gpu_kernel(const int n,
const scalar_t *data_value,
const int64_t *data_spatial_shapes,
const int64_t *data_level_start_index,
const scalar_t *data_sampling_loc,
const scalar_t *data_attn_weight,
const int batch_size,
const int spatial_size,
const int num_heads,
const int channels,
const int num_levels,
const int num_query,
const int num_point,
scalar_t *data_col)
{
CUDA_KERNEL_LOOP(index, n)
{
int _temp = index;
const int c_col = _temp % channels;
_temp /= channels;
const int sampling_index = _temp;
const int m_col = _temp % num_heads;
_temp /= num_heads;
const int q_col = _temp % num_query;
_temp /= num_query;
const int b_col = _temp;
scalar_t *data_col_ptr = data_col + index;
int data_weight_ptr = sampling_index * num_levels * num_point;
int data_loc_w_ptr = data_weight_ptr << 1;
const int qid_stride = num_heads * channels;
const int data_value_ptr_init_offset = b_col * spatial_size * qid_stride;
scalar_t col = 0;
for (int l_col=0; l_col < num_levels; ++l_col)
{
const int level_start_id = data_level_start_index[l_col];
const int spatial_h_ptr = l_col << 1;
const int spatial_h = data_spatial_shapes[spatial_h_ptr];
const int spatial_w = data_spatial_shapes[spatial_h_ptr + 1];
const scalar_t *data_value_ptr = data_value + (data_value_ptr_init_offset + level_start_id * qid_stride);
for (int p_col=0; p_col < num_point; ++p_col)
{
const scalar_t loc_w = data_sampling_loc[data_loc_w_ptr];
const scalar_t loc_h = data_sampling_loc[data_loc_w_ptr + 1];
const scalar_t weight = data_attn_weight[data_weight_ptr];
const scalar_t h_im = loc_h * spatial_h - 0.5;
const scalar_t w_im = loc_w * spatial_w - 0.5;
if (h_im > -1 && w_im > -1 && h_im < spatial_h && w_im < spatial_w)
{
col += ms_deform_attn_im2col_bilinear(data_value_ptr, spatial_h, spatial_w, num_heads, channels, h_im, w_im, m_col, c_col) * weight;
}
data_weight_ptr += 1;
data_loc_w_ptr += 2;
}
}
*data_col_ptr = col;
}
}
template
__global__ void ms_deformable_col2im_gpu_kernel_shm_blocksize_aware_reduce_v1(const int n,
const scalar_t *grad_col,
const scalar_t *data_value,
const int64_t *data_spatial_shapes,
const int64_t *data_level_start_index,
const scalar_t *data_sampling_loc,
const scalar_t *data_attn_weight,
const int batch_size,
const int spatial_size,
const int num_heads,
const int channels,
const int num_levels,
const int num_query,
const int num_point,
scalar_t *grad_value,
scalar_t *grad_sampling_loc,
scalar_t *grad_attn_weight)
{
CUDA_KERNEL_LOOP(index, n)
{
__shared__ scalar_t cache_grad_sampling_loc[blockSize * 2];
__shared__ scalar_t cache_grad_attn_weight[blockSize];
unsigned int tid = threadIdx.x;
int _temp = index;
const int c_col = _temp % channels;
_temp /= channels;
const int sampling_index = _temp;
const int m_col = _temp % num_heads;
_temp /= num_heads;
const int q_col = _temp % num_query;
_temp /= num_query;
const int b_col = _temp;
const scalar_t top_grad = grad_col[index];
int data_weight_ptr = sampling_index * num_levels * num_point;
int data_loc_w_ptr = data_weight_ptr << 1;
const int grad_sampling_ptr = data_weight_ptr;
grad_sampling_loc += grad_sampling_ptr << 1;
grad_attn_weight += grad_sampling_ptr;
const int grad_weight_stride = 1;
const int grad_loc_stride = 2;
const int qid_stride = num_heads * channels;
const int data_value_ptr_init_offset = b_col * spatial_size * qid_stride;
for (int l_col=0; l_col < num_levels; ++l_col)
{
const int level_start_id = data_level_start_index[l_col];
const int spatial_h_ptr = l_col << 1;
const int spatial_h = data_spatial_shapes[spatial_h_ptr];
const int spatial_w = data_spatial_shapes[spatial_h_ptr + 1];
const int value_ptr_offset = data_value_ptr_init_offset + level_start_id * qid_stride;
const scalar_t *data_value_ptr = data_value + value_ptr_offset;
scalar_t *grad_value_ptr = grad_value + value_ptr_offset;
for (int p_col=0; p_col < num_point; ++p_col)
{
const scalar_t loc_w = data_sampling_loc[data_loc_w_ptr];
const scalar_t loc_h = data_sampling_loc[data_loc_w_ptr + 1];
const scalar_t weight = data_attn_weight[data_weight_ptr];
const scalar_t h_im = loc_h * spatial_h - 0.5;
const scalar_t w_im = loc_w * spatial_w - 0.5;
*(cache_grad_sampling_loc+(threadIdx.x << 1)) = 0;
*(cache_grad_sampling_loc+((threadIdx.x << 1) + 1)) = 0;
*(cache_grad_attn_weight+threadIdx.x)=0;
if (h_im > -1 && w_im > -1 && h_im < spatial_h && w_im < spatial_w)
{
ms_deform_attn_col2im_bilinear(
data_value_ptr, spatial_h, spatial_w, num_heads, channels, h_im, w_im, m_col, c_col,
top_grad, weight, grad_value_ptr,
cache_grad_sampling_loc+(threadIdx.x << 1), cache_grad_attn_weight+threadIdx.x);
}
__syncthreads();
if (tid == 0)
{
scalar_t _grad_w=cache_grad_sampling_loc[0], _grad_h=cache_grad_sampling_loc[1], _grad_a=cache_grad_attn_weight[0];
int sid=2;
for (unsigned int tid = 1; tid < blockSize; ++tid)
{
_grad_w += cache_grad_sampling_loc[sid];
_grad_h += cache_grad_sampling_loc[sid + 1];
_grad_a += cache_grad_attn_weight[tid];
sid += 2;
}
*grad_sampling_loc = _grad_w;
*(grad_sampling_loc + 1) = _grad_h;
*grad_attn_weight = _grad_a;
}
__syncthreads();
data_weight_ptr += 1;
data_loc_w_ptr += 2;
grad_attn_weight += grad_weight_stride;
grad_sampling_loc += grad_loc_stride;
}
}
}
}
template
__global__ void ms_deformable_col2im_gpu_kernel_shm_blocksize_aware_reduce_v2(const int n,
const scalar_t *grad_col,
const scalar_t *data_value,
const int64_t *data_spatial_shapes,
const int64_t *data_level_start_index,
const scalar_t *data_sampling_loc,
const scalar_t *data_attn_weight,
const int batch_size,
const int spatial_size,
const int num_heads,
const int channels,
const int num_levels,
const int num_query,
const int num_point,
scalar_t *grad_value,
scalar_t *grad_sampling_loc,
scalar_t *grad_attn_weight)
{
CUDA_KERNEL_LOOP(index, n)
{
__shared__ scalar_t cache_grad_sampling_loc[blockSize * 2];
__shared__ scalar_t cache_grad_attn_weight[blockSize];
unsigned int tid = threadIdx.x;
int _temp = index;
const int c_col = _temp % channels;
_temp /= channels;
const int sampling_index = _temp;
const int m_col = _temp % num_heads;
_temp /= num_heads;
const int q_col = _temp % num_query;
_temp /= num_query;
const int b_col = _temp;
const scalar_t top_grad = grad_col[index];
int data_weight_ptr = sampling_index * num_levels * num_point;
int data_loc_w_ptr = data_weight_ptr << 1;
const int grad_sampling_ptr = data_weight_ptr;
grad_sampling_loc += grad_sampling_ptr << 1;
grad_attn_weight += grad_sampling_ptr;
const int grad_weight_stride = 1;
const int grad_loc_stride = 2;
const int qid_stride = num_heads * channels;
const int data_value_ptr_init_offset = b_col * spatial_size * qid_stride;
for (int l_col=0; l_col < num_levels; ++l_col)
{
const int level_start_id = data_level_start_index[l_col];
const int spatial_h_ptr = l_col << 1;
const int spatial_h = data_spatial_shapes[spatial_h_ptr];
const int spatial_w = data_spatial_shapes[spatial_h_ptr + 1];
const int value_ptr_offset = data_value_ptr_init_offset + level_start_id * qid_stride;
const scalar_t *data_value_ptr = data_value + value_ptr_offset;
scalar_t *grad_value_ptr = grad_value + value_ptr_offset;
for (int p_col=0; p_col < num_point; ++p_col)
{
const scalar_t loc_w = data_sampling_loc[data_loc_w_ptr];
const scalar_t loc_h = data_sampling_loc[data_loc_w_ptr + 1];
const scalar_t weight = data_attn_weight[data_weight_ptr];
const scalar_t h_im = loc_h * spatial_h - 0.5;
const scalar_t w_im = loc_w * spatial_w - 0.5;
*(cache_grad_sampling_loc+(threadIdx.x << 1)) = 0;
*(cache_grad_sampling_loc+((threadIdx.x << 1) + 1)) = 0;
*(cache_grad_attn_weight+threadIdx.x)=0;
if (h_im > -1 && w_im > -1 && h_im < spatial_h && w_im < spatial_w)
{
ms_deform_attn_col2im_bilinear(
data_value_ptr, spatial_h, spatial_w, num_heads, channels, h_im, w_im, m_col, c_col,
top_grad, weight, grad_value_ptr,
cache_grad_sampling_loc+(threadIdx.x << 1), cache_grad_attn_weight+threadIdx.x);
}
__syncthreads();
for (unsigned int s=blockSize/2; s>0; s>>=1)
{
if (tid < s) {
const unsigned int xid1 = tid << 1;
const unsigned int xid2 = (tid + s) << 1;
cache_grad_attn_weight[tid] += cache_grad_attn_weight[tid + s];
cache_grad_sampling_loc[xid1] += cache_grad_sampling_loc[xid2];
cache_grad_sampling_loc[xid1 + 1] += cache_grad_sampling_loc[xid2 + 1];
}
__syncthreads();
}
if (tid == 0)
{
*grad_sampling_loc = cache_grad_sampling_loc[0];
*(grad_sampling_loc + 1) = cache_grad_sampling_loc[1];
*grad_attn_weight = cache_grad_attn_weight[0];
}
__syncthreads();
data_weight_ptr += 1;
data_loc_w_ptr += 2;
grad_attn_weight += grad_weight_stride;
grad_sampling_loc += grad_loc_stride;
}
}
}
}
template
__global__ void ms_deformable_col2im_gpu_kernel_shm_reduce_v1(const int n,
const scalar_t *grad_col,
const scalar_t *data_value,
const int64_t *data_spatial_shapes,
const int64_t *data_level_start_index,
const scalar_t *data_sampling_loc,
const scalar_t *data_attn_weight,
const int batch_size,
const int spatial_size,
const int num_heads,
const int channels,
const int num_levels,
const int num_query,
const int num_point,
scalar_t *grad_value,
scalar_t *grad_sampling_loc,
scalar_t *grad_attn_weight)
{
CUDA_KERNEL_LOOP(index, n)
{
extern __shared__ int _s[];
scalar_t* cache_grad_sampling_loc = (scalar_t*)_s;
scalar_t* cache_grad_attn_weight = cache_grad_sampling_loc + 2 * blockDim.x;
unsigned int tid = threadIdx.x;
int _temp = index;
const int c_col = _temp % channels;
_temp /= channels;
const int sampling_index = _temp;
const int m_col = _temp % num_heads;
_temp /= num_heads;
const int q_col = _temp % num_query;
_temp /= num_query;
const int b_col = _temp;
const scalar_t top_grad = grad_col[index];
int data_weight_ptr = sampling_index * num_levels * num_point;
int data_loc_w_ptr = data_weight_ptr << 1;
const int grad_sampling_ptr = data_weight_ptr;
grad_sampling_loc += grad_sampling_ptr << 1;
grad_attn_weight += grad_sampling_ptr;
const int grad_weight_stride = 1;
const int grad_loc_stride = 2;
const int qid_stride = num_heads * channels;
const int data_value_ptr_init_offset = b_col * spatial_size * qid_stride;
for (int l_col=0; l_col < num_levels; ++l_col)
{
const int level_start_id = data_level_start_index[l_col];
const int spatial_h_ptr = l_col << 1;
const int spatial_h = data_spatial_shapes[spatial_h_ptr];
const int spatial_w = data_spatial_shapes[spatial_h_ptr + 1];
const int value_ptr_offset = data_value_ptr_init_offset + level_start_id * qid_stride;
const scalar_t *data_value_ptr = data_value + value_ptr_offset;
scalar_t *grad_value_ptr = grad_value + value_ptr_offset;
for (int p_col=0; p_col < num_point; ++p_col)
{
const scalar_t loc_w = data_sampling_loc[data_loc_w_ptr];
const scalar_t loc_h = data_sampling_loc[data_loc_w_ptr + 1];
const scalar_t weight = data_attn_weight[data_weight_ptr];
const scalar_t h_im = loc_h * spatial_h - 0.5;
const scalar_t w_im = loc_w * spatial_w - 0.5;
*(cache_grad_sampling_loc+(threadIdx.x << 1)) = 0;
*(cache_grad_sampling_loc+((threadIdx.x << 1) + 1)) = 0;
*(cache_grad_attn_weight+threadIdx.x)=0;
if (h_im > -1 && w_im > -1 && h_im < spatial_h && w_im < spatial_w)
{
ms_deform_attn_col2im_bilinear(
data_value_ptr, spatial_h, spatial_w, num_heads, channels, h_im, w_im, m_col, c_col,
top_grad, weight, grad_value_ptr,
cache_grad_sampling_loc+(threadIdx.x << 1), cache_grad_attn_weight+threadIdx.x);
}
__syncthreads();
if (tid == 0)
{
scalar_t _grad_w=cache_grad_sampling_loc[0], _grad_h=cache_grad_sampling_loc[1], _grad_a=cache_grad_attn_weight[0];
int sid=2;
for (unsigned int tid = 1; tid < blockDim.x; ++tid)
{
_grad_w += cache_grad_sampling_loc[sid];
_grad_h += cache_grad_sampling_loc[sid + 1];
_grad_a += cache_grad_attn_weight[tid];
sid += 2;
}
*grad_sampling_loc = _grad_w;
*(grad_sampling_loc + 1) = _grad_h;
*grad_attn_weight = _grad_a;
}
__syncthreads();
data_weight_ptr += 1;
data_loc_w_ptr += 2;
grad_attn_weight += grad_weight_stride;
grad_sampling_loc += grad_loc_stride;
}
}
}
}
template
__global__ void ms_deformable_col2im_gpu_kernel_shm_reduce_v2(const int n,
const scalar_t *grad_col,
const scalar_t *data_value,
const int64_t *data_spatial_shapes,
const int64_t *data_level_start_index,
const scalar_t *data_sampling_loc,
const scalar_t *data_attn_weight,
const int batch_size,
const int spatial_size,
const int num_heads,
const int channels,
const int num_levels,
const int num_query,
const int num_point,
scalar_t *grad_value,
scalar_t *grad_sampling_loc,
scalar_t *grad_attn_weight)
{
CUDA_KERNEL_LOOP(index, n)
{
extern __shared__ int _s[];
scalar_t* cache_grad_sampling_loc = (scalar_t*)_s;
scalar_t* cache_grad_attn_weight = cache_grad_sampling_loc + 2 * blockDim.x;
unsigned int tid = threadIdx.x;
int _temp = index;
const int c_col = _temp % channels;
_temp /= channels;
const int sampling_index = _temp;
const int m_col = _temp % num_heads;
_temp /= num_heads;
const int q_col = _temp % num_query;
_temp /= num_query;
const int b_col = _temp;
const scalar_t top_grad = grad_col[index];
int data_weight_ptr = sampling_index * num_levels * num_point;
int data_loc_w_ptr = data_weight_ptr << 1;
const int grad_sampling_ptr = data_weight_ptr;
grad_sampling_loc += grad_sampling_ptr << 1;
grad_attn_weight += grad_sampling_ptr;
const int grad_weight_stride = 1;
const int grad_loc_stride = 2;
const int qid_stride = num_heads * channels;
const int data_value_ptr_init_offset = b_col * spatial_size * qid_stride;
for (int l_col=0; l_col < num_levels; ++l_col)
{
const int level_start_id = data_level_start_index[l_col];
const int spatial_h_ptr = l_col << 1;
const int spatial_h = data_spatial_shapes[spatial_h_ptr];
const int spatial_w = data_spatial_shapes[spatial_h_ptr + 1];
const int value_ptr_offset = data_value_ptr_init_offset + level_start_id * qid_stride;
const scalar_t *data_value_ptr = data_value + value_ptr_offset;
scalar_t *grad_value_ptr = grad_value + value_ptr_offset;
for (int p_col=0; p_col < num_point; ++p_col)
{
const scalar_t loc_w = data_sampling_loc[data_loc_w_ptr];
const scalar_t loc_h = data_sampling_loc[data_loc_w_ptr + 1];
const scalar_t weight = data_attn_weight[data_weight_ptr];
const scalar_t h_im = loc_h * spatial_h - 0.5;
const scalar_t w_im = loc_w * spatial_w - 0.5;
*(cache_grad_sampling_loc+(threadIdx.x << 1)) = 0;
*(cache_grad_sampling_loc+((threadIdx.x << 1) + 1)) = 0;
*(cache_grad_attn_weight+threadIdx.x)=0;
if (h_im > -1 && w_im > -1 && h_im < spatial_h && w_im < spatial_w)
{
ms_deform_attn_col2im_bilinear(
data_value_ptr, spatial_h, spatial_w, num_heads, channels, h_im, w_im, m_col, c_col,
top_grad, weight, grad_value_ptr,
cache_grad_sampling_loc+(threadIdx.x << 1), cache_grad_attn_weight+threadIdx.x);
}
__syncthreads();
for (unsigned int s=blockDim.x/2, spre=blockDim.x; s>0; s>>=1, spre>>=1)
{
if (tid < s) {
const unsigned int xid1 = tid << 1;
const unsigned int xid2 = (tid + s) << 1;
cache_grad_attn_weight[tid] += cache_grad_attn_weight[tid + s];
cache_grad_sampling_loc[xid1] += cache_grad_sampling_loc[xid2];
cache_grad_sampling_loc[xid1 + 1] += cache_grad_sampling_loc[xid2 + 1];
if (tid + (s << 1) < spre)
{
cache_grad_attn_weight[tid] += cache_grad_attn_weight[tid + (s << 1)];
cache_grad_sampling_loc[xid1] += cache_grad_sampling_loc[xid2 + (s << 1)];
cache_grad_sampling_loc[xid1 + 1] += cache_grad_sampling_loc[xid2 + 1 + (s << 1)];
}
}
__syncthreads();
}
if (tid == 0)
{
*grad_sampling_loc = cache_grad_sampling_loc[0];
*(grad_sampling_loc + 1) = cache_grad_sampling_loc[1];
*grad_attn_weight = cache_grad_attn_weight[0];
}
__syncthreads();
data_weight_ptr += 1;
data_loc_w_ptr += 2;
grad_attn_weight += grad_weight_stride;
grad_sampling_loc += grad_loc_stride;
}
}
}
}
template
__global__ void ms_deformable_col2im_gpu_kernel_shm_reduce_v2_multi_blocks(const int n,
const scalar_t *grad_col,
const scalar_t *data_value,
const int64_t *data_spatial_shapes,
const int64_t *data_level_start_index,
const scalar_t *data_sampling_loc,
const scalar_t *data_attn_weight,
const int batch_size,
const int spatial_size,
const int num_heads,
const int channels,
const int num_levels,
const int num_query,
const int num_point,
scalar_t *grad_value,
scalar_t *grad_sampling_loc,
scalar_t *grad_attn_weight)
{
CUDA_KERNEL_LOOP(index, n)
{
extern __shared__ int _s[];
scalar_t* cache_grad_sampling_loc = (scalar_t*)_s;
scalar_t* cache_grad_attn_weight = cache_grad_sampling_loc + 2 * blockDim.x;
unsigned int tid = threadIdx.x;
int _temp = index;
const int c_col = _temp % channels;
_temp /= channels;
const int sampling_index = _temp;
const int m_col = _temp % num_heads;
_temp /= num_heads;
const int q_col = _temp % num_query;
_temp /= num_query;
const int b_col = _temp;
const scalar_t top_grad = grad_col[index];
int data_weight_ptr = sampling_index * num_levels * num_point;
int data_loc_w_ptr = data_weight_ptr << 1;
const int grad_sampling_ptr = data_weight_ptr;
grad_sampling_loc += grad_sampling_ptr << 1;
grad_attn_weight += grad_sampling_ptr;
const int grad_weight_stride = 1;
const int grad_loc_stride = 2;
const int qid_stride = num_heads * channels;
const int data_value_ptr_init_offset = b_col * spatial_size * qid_stride;
for (int l_col=0; l_col < num_levels; ++l_col)
{
const int level_start_id = data_level_start_index[l_col];
const int spatial_h_ptr = l_col << 1;
const int spatial_h = data_spatial_shapes[spatial_h_ptr];
const int spatial_w = data_spatial_shapes[spatial_h_ptr + 1];
const int value_ptr_offset = data_value_ptr_init_offset + level_start_id * qid_stride;
const scalar_t *data_value_ptr = data_value + value_ptr_offset;
scalar_t *grad_value_ptr = grad_value + value_ptr_offset;
for (int p_col=0; p_col < num_point; ++p_col)
{
const scalar_t loc_w = data_sampling_loc[data_loc_w_ptr];
const scalar_t loc_h = data_sampling_loc[data_loc_w_ptr + 1];
const scalar_t weight = data_attn_weight[data_weight_ptr];
const scalar_t h_im = loc_h * spatial_h - 0.5;
const scalar_t w_im = loc_w * spatial_w - 0.5;
*(cache_grad_sampling_loc+(threadIdx.x << 1)) = 0;
*(cache_grad_sampling_loc+((threadIdx.x << 1) + 1)) = 0;
*(cache_grad_attn_weight+threadIdx.x)=0;
if (h_im > -1 && w_im > -1 && h_im < spatial_h && w_im < spatial_w)
{
ms_deform_attn_col2im_bilinear(
data_value_ptr, spatial_h, spatial_w, num_heads, channels, h_im, w_im, m_col, c_col,
top_grad, weight, grad_value_ptr,
cache_grad_sampling_loc+(threadIdx.x << 1), cache_grad_attn_weight+threadIdx.x);
}
__syncthreads();
for (unsigned int s=blockDim.x/2, spre=blockDim.x; s>0; s>>=1, spre>>=1)
{
if (tid < s) {
const unsigned int xid1 = tid << 1;
const unsigned int xid2 = (tid + s) << 1;
cache_grad_attn_weight[tid] += cache_grad_attn_weight[tid + s];
cache_grad_sampling_loc[xid1] += cache_grad_sampling_loc[xid2];
cache_grad_sampling_loc[xid1 + 1] += cache_grad_sampling_loc[xid2 + 1];
if (tid + (s << 1) < spre)
{
cache_grad_attn_weight[tid] += cache_grad_attn_weight[tid + (s << 1)];
cache_grad_sampling_loc[xid1] += cache_grad_sampling_loc[xid2 + (s << 1)];
cache_grad_sampling_loc[xid1 + 1] += cache_grad_sampling_loc[xid2 + 1 + (s << 1)];
}
}
__syncthreads();
}
if (tid == 0)
{
atomicAdd(grad_sampling_loc, cache_grad_sampling_loc[0]);
atomicAdd(grad_sampling_loc + 1, cache_grad_sampling_loc[1]);
atomicAdd(grad_attn_weight, cache_grad_attn_weight[0]);
}
__syncthreads();
data_weight_ptr += 1;
data_loc_w_ptr += 2;
grad_attn_weight += grad_weight_stride;
grad_sampling_loc += grad_loc_stride;
}
}
}
}
template
__global__ void ms_deformable_col2im_gpu_kernel_gm(const int n,
const scalar_t *grad_col,
const scalar_t *data_value,
const int64_t *data_spatial_shapes,
const int64_t *data_level_start_index,
const scalar_t *data_sampling_loc,
const scalar_t *data_attn_weight,
const int batch_size,
const int spatial_size,
const int num_heads,
const int channels,
const int num_levels,
const int num_query,
const int num_point,
scalar_t *grad_value,
scalar_t *grad_sampling_loc,
scalar_t *grad_attn_weight)
{
CUDA_KERNEL_LOOP(index, n)
{
int _temp = index;
const int c_col = _temp % channels;
_temp /= channels;
const int sampling_index = _temp;
const int m_col = _temp % num_heads;
_temp /= num_heads;
const int q_col = _temp % num_query;
_temp /= num_query;
const int b_col = _temp;
const scalar_t top_grad = grad_col[index];
int data_weight_ptr = sampling_index * num_levels * num_point;
int data_loc_w_ptr = data_weight_ptr << 1;
const int grad_sampling_ptr = data_weight_ptr;
grad_sampling_loc += grad_sampling_ptr << 1;
grad_attn_weight += grad_sampling_ptr;
const int grad_weight_stride = 1;
const int grad_loc_stride = 2;
const int qid_stride = num_heads * channels;
const int data_value_ptr_init_offset = b_col * spatial_size * qid_stride;
for (int l_col=0; l_col < num_levels; ++l_col)
{
const int level_start_id = data_level_start_index[l_col];
const int spatial_h_ptr = l_col << 1;
const int spatial_h = data_spatial_shapes[spatial_h_ptr];
const int spatial_w = data_spatial_shapes[spatial_h_ptr + 1];
const int value_ptr_offset = data_value_ptr_init_offset + level_start_id * qid_stride;
const scalar_t *data_value_ptr = data_value + value_ptr_offset;
scalar_t *grad_value_ptr = grad_value + value_ptr_offset;
for (int p_col=0; p_col < num_point; ++p_col)
{
const scalar_t loc_w = data_sampling_loc[data_loc_w_ptr];
const scalar_t loc_h = data_sampling_loc[data_loc_w_ptr + 1];
const scalar_t weight = data_attn_weight[data_weight_ptr];
const scalar_t h_im = loc_h * spatial_h - 0.5;
const scalar_t w_im = loc_w * spatial_w - 0.5;
if (h_im > -1 && w_im > -1 && h_im < spatial_h && w_im < spatial_w)
{
ms_deform_attn_col2im_bilinear_gm(
data_value_ptr, spatial_h, spatial_w, num_heads, channels, h_im, w_im, m_col, c_col,
top_grad, weight, grad_value_ptr,
grad_sampling_loc, grad_attn_weight);
}
data_weight_ptr += 1;
data_loc_w_ptr += 2;
grad_attn_weight += grad_weight_stride;
grad_sampling_loc += grad_loc_stride;
}
}
}
}
template
void ms_deformable_im2col_cuda(cudaStream_t stream,
const scalar_t* data_value,
const int64_t* data_spatial_shapes,
const int64_t* data_level_start_index,
const scalar_t* data_sampling_loc,
const scalar_t* data_attn_weight,
const int batch_size,
const int spatial_size,
const int num_heads,
const int channels,
const int num_levels,
const int num_query,
const int num_point,
scalar_t* data_col)
{
const int num_kernels = batch_size * num_query * num_heads * channels;
const int num_actual_kernels = batch_size * num_query * num_heads * channels;
const int num_threads = CUDA_NUM_THREADS;
ms_deformable_im2col_gpu_kernel
<<>>(
num_kernels, data_value, data_spatial_shapes, data_level_start_index, data_sampling_loc, data_attn_weight,
batch_size, spatial_size, num_heads, channels, num_levels, num_query, num_point, data_col);
cudaError_t err = cudaGetLastError();
if (err != cudaSuccess)
{
printf("error in ms_deformable_im2col_cuda: %s\n", cudaGetErrorString(err));
}
}
template
void ms_deformable_col2im_cuda(cudaStream_t stream,
const scalar_t* grad_col,
const scalar_t* data_value,
const int64_t * data_spatial_shapes,
const int64_t * data_level_start_index,
const scalar_t * data_sampling_loc,
const scalar_t * data_attn_weight,
const int batch_size,
const int spatial_size,
const int num_heads,
const int channels,
const int num_levels,
const int num_query,
const int num_point,
scalar_t* grad_value,
scalar_t* grad_sampling_loc,
scalar_t* grad_attn_weight)
{
const int num_threads = (channels > CUDA_NUM_THREADS)?CUDA_NUM_THREADS:channels;
const int num_kernels = batch_size * num_query * num_heads * channels;
const int num_actual_kernels = batch_size * num_query * num_heads * channels;
if (channels > 1024)
{
if ((channels & 1023) == 0)
{
ms_deformable_col2im_gpu_kernel_shm_reduce_v2_multi_blocks
<<>>(
num_kernels,
grad_col,
data_value,
data_spatial_shapes,
data_level_start_index,
data_sampling_loc,
data_attn_weight,
batch_size,
spatial_size,
num_heads,
channels,
num_levels,
num_query,
num_point,
grad_value,
grad_sampling_loc,
grad_attn_weight);
}
else
{
ms_deformable_col2im_gpu_kernel_gm
<<>>(
num_kernels,
grad_col,
data_value,
data_spatial_shapes,
data_level_start_index,
data_sampling_loc,
data_attn_weight,
batch_size,
spatial_size,
num_heads,
channels,
num_levels,
num_query,
num_point,
grad_value,
grad_sampling_loc,
grad_attn_weight);
}
}
else{
switch(channels)
{
case 1:
ms_deformable_col2im_gpu_kernel_shm_blocksize_aware_reduce_v1
<<>>(
num_kernels,
grad_col,
data_value,
data_spatial_shapes,
data_level_start_index,
data_sampling_loc,
data_attn_weight,
batch_size,
spatial_size,
num_heads,
channels,
num_levels,
num_query,
num_point,
grad_value,
grad_sampling_loc,
grad_attn_weight);
break;
case 2:
ms_deformable_col2im_gpu_kernel_shm_blocksize_aware_reduce_v1
<<>>(
num_kernels,
grad_col,
data_value,
data_spatial_shapes,
data_level_start_index,
data_sampling_loc,
data_attn_weight,
batch_size,
spatial_size,
num_heads,
channels,
num_levels,
num_query,
num_point,
grad_value,
grad_sampling_loc,
grad_attn_weight);
break;
case 4:
ms_deformable_col2im_gpu_kernel_shm_blocksize_aware_reduce_v1
<<>>(
num_kernels,
grad_col,
data_value,
data_spatial_shapes,
data_level_start_index,
data_sampling_loc,
data_attn_weight,
batch_size,
spatial_size,
num_heads,
channels,
num_levels,
num_query,
num_point,
grad_value,
grad_sampling_loc,
grad_attn_weight);
break;
case 8:
ms_deformable_col2im_gpu_kernel_shm_blocksize_aware_reduce_v1
<<>>(
num_kernels,
grad_col,
data_value,
data_spatial_shapes,
data_level_start_index,
data_sampling_loc,
data_attn_weight,
batch_size,
spatial_size,
num_heads,
channels,
num_levels,
num_query,
num_point,
grad_value,
grad_sampling_loc,
grad_attn_weight);
break;
case 16:
ms_deformable_col2im_gpu_kernel_shm_blocksize_aware_reduce_v1
<<>>(
num_kernels,
grad_col,
data_value,
data_spatial_shapes,
data_level_start_index,
data_sampling_loc,
data_attn_weight,
batch_size,
spatial_size,
num_heads,
channels,
num_levels,
num_query,
num_point,
grad_value,
grad_sampling_loc,
grad_attn_weight);
break;
case 32:
ms_deformable_col2im_gpu_kernel_shm_blocksize_aware_reduce_v1
<<>>(
num_kernels,
grad_col,
data_value,
data_spatial_shapes,
data_level_start_index,
data_sampling_loc,
data_attn_weight,
batch_size,
spatial_size,
num_heads,
channels,
num_levels,
num_query,
num_point,
grad_value,
grad_sampling_loc,
grad_attn_weight);
break;
case 64:
ms_deformable_col2im_gpu_kernel_shm_blocksize_aware_reduce_v2
<<>>(
num_kernels,
grad_col,
data_value,
data_spatial_shapes,
data_level_start_index,
data_sampling_loc,
data_attn_weight,
batch_size,
spatial_size,
num_heads,
channels,
num_levels,
num_query,
num_point,
grad_value,
grad_sampling_loc,
grad_attn_weight);
break;
case 128:
ms_deformable_col2im_gpu_kernel_shm_blocksize_aware_reduce_v2
<<>>(
num_kernels,
grad_col,
data_value,
data_spatial_shapes,
data_level_start_index,
data_sampling_loc,
data_attn_weight,
batch_size,
spatial_size,
num_heads,
channels,
num_levels,
num_query,
num_point,
grad_value,
grad_sampling_loc,
grad_attn_weight);
break;
case 256:
ms_deformable_col2im_gpu_kernel_shm_blocksize_aware_reduce_v2
<<>>(
num_kernels,
grad_col,
data_value,
data_spatial_shapes,
data_level_start_index,
data_sampling_loc,
data_attn_weight,
batch_size,
spatial_size,
num_heads,
channels,
num_levels,
num_query,
num_point,
grad_value,
grad_sampling_loc,
grad_attn_weight);
break;
case 512:
ms_deformable_col2im_gpu_kernel_shm_blocksize_aware_reduce_v2
<<>>(
num_kernels,
grad_col,
data_value,
data_spatial_shapes,
data_level_start_index,
data_sampling_loc,
data_attn_weight,
batch_size,
spatial_size,
num_heads,
channels,
num_levels,
num_query,
num_point,
grad_value,
grad_sampling_loc,
grad_attn_weight);
break;
case 1024:
ms_deformable_col2im_gpu_kernel_shm_blocksize_aware_reduce_v2
<<>>(
num_kernels,
grad_col,
data_value,
data_spatial_shapes,
data_level_start_index,
data_sampling_loc,
data_attn_weight,
batch_size,
spatial_size,
num_heads,
channels,
num_levels,
num_query,
num_point,
grad_value,
grad_sampling_loc,
grad_attn_weight);
break;
default:
if (channels < 64)
{
ms_deformable_col2im_gpu_kernel_shm_reduce_v1
<<>>(
num_kernels,
grad_col,
data_value,
data_spatial_shapes,
data_level_start_index,
data_sampling_loc,
data_attn_weight,
batch_size,
spatial_size,
num_heads,
channels,
num_levels,
num_query,
num_point,
grad_value,
grad_sampling_loc,
grad_attn_weight);
}
else
{
ms_deformable_col2im_gpu_kernel_shm_reduce_v2
<<>>(
num_kernels,
grad_col,
data_value,
data_spatial_shapes,
data_level_start_index,
data_sampling_loc,
data_attn_weight,
batch_size,
spatial_size,
num_heads,
channels,
num_levels,
num_query,
num_point,
grad_value,
grad_sampling_loc,
grad_attn_weight);
}
}
}
cudaError_t err = cudaGetLastError();
if (err != cudaSuccess)
{
printf("error in ms_deformable_col2im_cuda: %s\n", cudaGetErrorString(err));
}
}
================================================
FILE: batchformer-v2/deformable-detr/models/ops/src/ms_deform_attn.h
================================================
/*!
**************************************************************************************************
* Deformable DETR
* Copyright (c) 2020 SenseTime. All Rights Reserved.
* Licensed under the Apache License, Version 2.0 [see LICENSE for details]
**************************************************************************************************
* Modified from https://github.com/chengdazhi/Deformable-Convolution-V2-PyTorch/tree/pytorch_1.0.0
**************************************************************************************************
*/
#pragma once
#include "cpu/ms_deform_attn_cpu.h"
#ifdef WITH_CUDA
#include "cuda/ms_deform_attn_cuda.h"
#endif
at::Tensor
ms_deform_attn_forward(
const at::Tensor &value,
const at::Tensor &spatial_shapes,
const at::Tensor &level_start_index,
const at::Tensor &sampling_loc,
const at::Tensor &attn_weight,
const int im2col_step)
{
if (value.type().is_cuda())
{
#ifdef WITH_CUDA
return ms_deform_attn_cuda_forward(
value, spatial_shapes, level_start_index, sampling_loc, attn_weight, im2col_step);
#else
AT_ERROR("Not compiled with GPU support");
#endif
}
AT_ERROR("Not implemented on the CPU");
}
std::vector
ms_deform_attn_backward(
const at::Tensor &value,
const at::Tensor &spatial_shapes,
const at::Tensor &level_start_index,
const at::Tensor &sampling_loc,
const at::Tensor &attn_weight,
const at::Tensor &grad_output,
const int im2col_step)
{
if (value.type().is_cuda())
{
#ifdef WITH_CUDA
return ms_deform_attn_cuda_backward(
value, spatial_shapes, level_start_index, sampling_loc, attn_weight, grad_output, im2col_step);
#else
AT_ERROR("Not compiled with GPU support");
#endif
}
AT_ERROR("Not implemented on the CPU");
}
================================================
FILE: batchformer-v2/deformable-detr/models/ops/src/vision.cpp
================================================
/*!
**************************************************************************************************
* Deformable DETR
* Copyright (c) 2020 SenseTime. All Rights Reserved.
* Licensed under the Apache License, Version 2.0 [see LICENSE for details]
**************************************************************************************************
* Modified from https://github.com/chengdazhi/Deformable-Convolution-V2-PyTorch/tree/pytorch_1.0.0
**************************************************************************************************
*/
#include "ms_deform_attn.h"
PYBIND11_MODULE(TORCH_EXTENSION_NAME, m) {
m.def("ms_deform_attn_forward", &ms_deform_attn_forward, "ms_deform_attn_forward");
m.def("ms_deform_attn_backward", &ms_deform_attn_backward, "ms_deform_attn_backward");
}
================================================
FILE: batchformer-v2/deformable-detr/models/ops/test.py
================================================
# ------------------------------------------------------------------------------------------------
# Deformable DETR
# Copyright (c) 2020 SenseTime. All Rights Reserved.
# Licensed under the Apache License, Version 2.0 [see LICENSE for details]
# ------------------------------------------------------------------------------------------------
# Modified from https://github.com/chengdazhi/Deformable-Convolution-V2-PyTorch/tree/pytorch_1.0.0
# ------------------------------------------------------------------------------------------------
from __future__ import absolute_import
from __future__ import print_function
from __future__ import division
import time
import torch
import torch.nn as nn
from torch.autograd import gradcheck
from functions.ms_deform_attn_func import MSDeformAttnFunction, ms_deform_attn_core_pytorch
N, M, D = 1, 2, 2
Lq, L, P = 2, 2, 2
shapes = torch.as_tensor([(6, 4), (3, 2)], dtype=torch.long).cuda()
level_start_index = torch.cat((shapes.new_zeros((1, )), shapes.prod(1).cumsum(0)[:-1]))
S = sum([(H*W).item() for H, W in shapes])
torch.manual_seed(3)
@torch.no_grad()
def check_forward_equal_with_pytorch_double():
value = torch.rand(N, S, M, D).cuda() * 0.01
sampling_locations = torch.rand(N, Lq, M, L, P, 2).cuda()
attention_weights = torch.rand(N, Lq, M, L, P).cuda() + 1e-5
attention_weights /= attention_weights.sum(-1, keepdim=True).sum(-2, keepdim=True)
im2col_step = 2
output_pytorch = ms_deform_attn_core_pytorch(value.double(), shapes, sampling_locations.double(), attention_weights.double()).detach().cpu()
output_cuda = MSDeformAttnFunction.apply(value.double(), shapes, level_start_index, sampling_locations.double(), attention_weights.double(), im2col_step).detach().cpu()
fwdok = torch.allclose(output_cuda, output_pytorch)
max_abs_err = (output_cuda - output_pytorch).abs().max()
max_rel_err = ((output_cuda - output_pytorch).abs() / output_pytorch.abs()).max()
print(f'* {fwdok} check_forward_equal_with_pytorch_double: max_abs_err {max_abs_err:.2e} max_rel_err {max_rel_err:.2e}')
@torch.no_grad()
def check_forward_equal_with_pytorch_float():
value = torch.rand(N, S, M, D).cuda() * 0.01
sampling_locations = torch.rand(N, Lq, M, L, P, 2).cuda()
attention_weights = torch.rand(N, Lq, M, L, P).cuda() + 1e-5
attention_weights /= attention_weights.sum(-1, keepdim=True).sum(-2, keepdim=True)
im2col_step = 2
output_pytorch = ms_deform_attn_core_pytorch(value, shapes, sampling_locations, attention_weights).detach().cpu()
output_cuda = MSDeformAttnFunction.apply(value, shapes, level_start_index, sampling_locations, attention_weights, im2col_step).detach().cpu()
fwdok = torch.allclose(output_cuda, output_pytorch, rtol=1e-2, atol=1e-3)
max_abs_err = (output_cuda - output_pytorch).abs().max()
max_rel_err = ((output_cuda - output_pytorch).abs() / output_pytorch.abs()).max()
print(f'* {fwdok} check_forward_equal_with_pytorch_float: max_abs_err {max_abs_err:.2e} max_rel_err {max_rel_err:.2e}')
def check_gradient_numerical(channels=4, grad_value=True, grad_sampling_loc=True, grad_attn_weight=True):
value = torch.rand(N, S, M, channels).cuda() * 0.01
sampling_locations = torch.rand(N, Lq, M, L, P, 2).cuda()
attention_weights = torch.rand(N, Lq, M, L, P).cuda() + 1e-5
attention_weights /= attention_weights.sum(-1, keepdim=True).sum(-2, keepdim=True)
im2col_step = 2
func = MSDeformAttnFunction.apply
value.requires_grad = grad_value
sampling_locations.requires_grad = grad_sampling_loc
attention_weights.requires_grad = grad_attn_weight
gradok = gradcheck(func, (value.double(), shapes, level_start_index, sampling_locations.double(), attention_weights.double(), im2col_step))
print(f'* {gradok} check_gradient_numerical(D={channels})')
if __name__ == '__main__':
check_forward_equal_with_pytorch_double()
check_forward_equal_with_pytorch_float()
for channels in [30, 32, 64, 71, 1025, 2048, 3096]:
check_gradient_numerical(channels, True, True, True)
================================================
FILE: batchformer-v2/deformable-detr/models/position_encoding.py
================================================
# ------------------------------------------------------------------------
# Deformable DETR
# Copyright (c) 2020 SenseTime. All Rights Reserved.
# Licensed under the Apache License, Version 2.0 [see LICENSE for details]
# ------------------------------------------------------------------------
# Modified from DETR (https://github.com/facebookresearch/detr)
# Copyright (c) Facebook, Inc. and its affiliates. All Rights Reserved
# ------------------------------------------------------------------------
"""
Various positional encodings for the transformer.
"""
import math
import torch
from torch import nn
from util.misc import NestedTensor
class PositionEmbeddingSine(nn.Module):
"""
This is a more standard version of the position embedding, very similar to the one
used by the Attention is all you need paper, generalized to work on images.
"""
def __init__(self, num_pos_feats=64, temperature=10000, normalize=False, scale=None):
super().__init__()
self.num_pos_feats = num_pos_feats
self.temperature = temperature
self.normalize = normalize
if scale is not None and normalize is False:
raise ValueError("normalize should be True if scale is passed")
if scale is None:
scale = 2 * math.pi
self.scale = scale
def forward(self, tensor_list: NestedTensor):
x = tensor_list.tensors
mask = tensor_list.mask
assert mask is not None
not_mask = ~mask
y_embed = not_mask.cumsum(1, dtype=torch.float32)
x_embed = not_mask.cumsum(2, dtype=torch.float32)
if self.normalize:
eps = 1e-6
y_embed = (y_embed - 0.5) / (y_embed[:, -1:, :] + eps) * self.scale
x_embed = (x_embed - 0.5) / (x_embed[:, :, -1:] + eps) * self.scale
dim_t = torch.arange(self.num_pos_feats, dtype=torch.float32, device=x.device)
dim_t = self.temperature ** (2 * (dim_t // 2) / self.num_pos_feats)
pos_x = x_embed[:, :, :, None] / dim_t
pos_y = y_embed[:, :, :, None] / dim_t
pos_x = torch.stack((pos_x[:, :, :, 0::2].sin(), pos_x[:, :, :, 1::2].cos()), dim=4).flatten(3)
pos_y = torch.stack((pos_y[:, :, :, 0::2].sin(), pos_y[:, :, :, 1::2].cos()), dim=4).flatten(3)
pos = torch.cat((pos_y, pos_x), dim=3).permute(0, 3, 1, 2)
return pos
class PositionEmbeddingLearned(nn.Module):
"""
Absolute pos embedding, learned.
"""
def __init__(self, num_pos_feats=256):
super().__init__()
self.row_embed = nn.Embedding(50, num_pos_feats)
self.col_embed = nn.Embedding(50, num_pos_feats)
self.reset_parameters()
def reset_parameters(self):
nn.init.uniform_(self.row_embed.weight)
nn.init.uniform_(self.col_embed.weight)
def forward(self, tensor_list: NestedTensor):
x = tensor_list.tensors
h, w = x.shape[-2:]
i = torch.arange(w, device=x.device)
j = torch.arange(h, device=x.device)
x_emb = self.col_embed(i)
y_emb = self.row_embed(j)
pos = torch.cat([
x_emb.unsqueeze(0).repeat(h, 1, 1),
y_emb.unsqueeze(1).repeat(1, w, 1),
], dim=-1).permute(2, 0, 1).unsqueeze(0).repeat(x.shape[0], 1, 1, 1)
return pos
def build_position_encoding(args):
N_steps = args.hidden_dim // 2
if args.position_embedding in ('v2', 'sine'):
# TODO find a better way of exposing other arguments
position_embedding = PositionEmbeddingSine(N_steps, normalize=True)
elif args.position_embedding in ('v3', 'learned'):
position_embedding = PositionEmbeddingLearned(N_steps)
else:
raise ValueError(f"not supported {args.position_embedding}")
return position_embedding
================================================
FILE: batchformer-v2/deformable-detr/models/segmentation.py
================================================
# ------------------------------------------------------------------------
# Deformable DETR
# Copyright (c) 2020 SenseTime. All Rights Reserved.
# Licensed under the Apache License, Version 2.0 [see LICENSE for details]
# ------------------------------------------------------------------------
# Modified from DETR (https://github.com/facebookresearch/detr)
# Copyright (c) Facebook, Inc. and its affiliates. All Rights Reserved
# ------------------------------------------------------------------------
"""
This file provides the definition of the convolutional heads used to predict masks, as well as the losses
"""
import io
from collections import defaultdict
import torch
import torch.nn as nn
import torch.nn.functional as F
from PIL import Image
import util.box_ops as box_ops
from util.misc import NestedTensor, interpolate, nested_tensor_from_tensor_list
try:
from panopticapi.utils import id2rgb, rgb2id
except ImportError:
pass
class DETRsegm(nn.Module):
def __init__(self, detr, freeze_detr=False):
super().__init__()
self.detr = detr
if freeze_detr:
for p in self.parameters():
p.requires_grad_(False)
hidden_dim, nheads = detr.transformer.d_model, detr.transformer.nhead
self.bbox_attention = MHAttentionMap(hidden_dim, hidden_dim, nheads, dropout=0)
self.mask_head = MaskHeadSmallConv(hidden_dim + nheads, [1024, 512, 256], hidden_dim)
def forward(self, samples: NestedTensor):
if not isinstance(samples, NestedTensor):
samples = nested_tensor_from_tensor_list(samples)
features, pos = self.detr.backbone(samples)
bs = features[-1].tensors.shape[0]
src, mask = features[-1].decompose()
src_proj = self.detr.input_proj(src)
hs, memory = self.detr.transformer(src_proj, mask, self.detr.query_embed.weight, pos[-1])
outputs_class = self.detr.class_embed(hs)
outputs_coord = self.detr.bbox_embed(hs).sigmoid()
out = {"pred_logits": outputs_class[-1], "pred_boxes": outputs_coord[-1]}
if self.detr.aux_loss:
out["aux_outputs"] = [
{"pred_logits": a, "pred_boxes": b} for a, b in zip(outputs_class[:-1], outputs_coord[:-1])
]
# FIXME h_boxes takes the last one computed, keep this in mind
bbox_mask = self.bbox_attention(hs[-1], memory, mask=mask)
seg_masks = self.mask_head(src_proj, bbox_mask, [features[2].tensors, features[1].tensors, features[0].tensors])
outputs_seg_masks = seg_masks.view(bs, self.detr.num_queries, seg_masks.shape[-2], seg_masks.shape[-1])
out["pred_masks"] = outputs_seg_masks
return out
class MaskHeadSmallConv(nn.Module):
"""
Simple convolutional head, using group norm.
Upsampling is done using a FPN approach
"""
def __init__(self, dim, fpn_dims, context_dim):
super().__init__()
inter_dims = [dim, context_dim // 2, context_dim // 4, context_dim // 8, context_dim // 16, context_dim // 64]
self.lay1 = torch.nn.Conv2d(dim, dim, 3, padding=1)
self.gn1 = torch.nn.GroupNorm(8, dim)
self.lay2 = torch.nn.Conv2d(dim, inter_dims[1], 3, padding=1)
self.gn2 = torch.nn.GroupNorm(8, inter_dims[1])
self.lay3 = torch.nn.Conv2d(inter_dims[1], inter_dims[2], 3, padding=1)
self.gn3 = torch.nn.GroupNorm(8, inter_dims[2])
self.lay4 = torch.nn.Conv2d(inter_dims[2], inter_dims[3], 3, padding=1)
self.gn4 = torch.nn.GroupNorm(8, inter_dims[3])
self.lay5 = torch.nn.Conv2d(inter_dims[3], inter_dims[4], 3, padding=1)
self.gn5 = torch.nn.GroupNorm(8, inter_dims[4])
self.out_lay = torch.nn.Conv2d(inter_dims[4], 1, 3, padding=1)
self.dim = dim
self.adapter1 = torch.nn.Conv2d(fpn_dims[0], inter_dims[1], 1)
self.adapter2 = torch.nn.Conv2d(fpn_dims[1], inter_dims[2], 1)
self.adapter3 = torch.nn.Conv2d(fpn_dims[2], inter_dims[3], 1)
for m in self.modules():
if isinstance(m, nn.Conv2d):
nn.init.kaiming_uniform_(m.weight, a=1)
nn.init.constant_(m.bias, 0)
def forward(self, x, bbox_mask, fpns):
def expand(tensor, length):
return tensor.unsqueeze(1).repeat(1, int(length), 1, 1, 1).flatten(0, 1)
x = torch.cat([expand(x, bbox_mask.shape[1]), bbox_mask.flatten(0, 1)], 1)
x = self.lay1(x)
x = self.gn1(x)
x = F.relu(x)
x = self.lay2(x)
x = self.gn2(x)
x = F.relu(x)
cur_fpn = self.adapter1(fpns[0])
if cur_fpn.size(0) != x.size(0):
cur_fpn = expand(cur_fpn, x.size(0) / cur_fpn.size(0))
x = cur_fpn + F.interpolate(x, size=cur_fpn.shape[-2:], mode="nearest")
x = self.lay3(x)
x = self.gn3(x)
x = F.relu(x)
cur_fpn = self.adapter2(fpns[1])
if cur_fpn.size(0) != x.size(0):
cur_fpn = expand(cur_fpn, x.size(0) / cur_fpn.size(0))
x = cur_fpn + F.interpolate(x, size=cur_fpn.shape[-2:], mode="nearest")
x = self.lay4(x)
x = self.gn4(x)
x = F.relu(x)
cur_fpn = self.adapter3(fpns[2])
if cur_fpn.size(0) != x.size(0):
cur_fpn = expand(cur_fpn, x.size(0) / cur_fpn.size(0))
x = cur_fpn + F.interpolate(x, size=cur_fpn.shape[-2:], mode="nearest")
x = self.lay5(x)
x = self.gn5(x)
x = F.relu(x)
x = self.out_lay(x)
return x
class MHAttentionMap(nn.Module):
"""This is a 2D attention module, which only returns the attention softmax (no multiplication by value)"""
def __init__(self, query_dim, hidden_dim, num_heads, dropout=0, bias=True):
super().__init__()
self.num_heads = num_heads
self.hidden_dim = hidden_dim
self.dropout = nn.Dropout(dropout)
self.q_linear = nn.Linear(query_dim, hidden_dim, bias=bias)
self.k_linear = nn.Linear(query_dim, hidden_dim, bias=bias)
nn.init.zeros_(self.k_linear.bias)
nn.init.zeros_(self.q_linear.bias)
nn.init.xavier_uniform_(self.k_linear.weight)
nn.init.xavier_uniform_(self.q_linear.weight)
self.normalize_fact = float(hidden_dim / self.num_heads) ** -0.5
def forward(self, q, k, mask=None):
q = self.q_linear(q)
k = F.conv2d(k, self.k_linear.weight.unsqueeze(-1).unsqueeze(-1), self.k_linear.bias)
qh = q.view(q.shape[0], q.shape[1], self.num_heads, self.hidden_dim // self.num_heads)
kh = k.view(k.shape[0], self.num_heads, self.hidden_dim // self.num_heads, k.shape[-2], k.shape[-1])
weights = torch.einsum("bqnc,bnchw->bqnhw", qh * self.normalize_fact, kh)
if mask is not None:
weights.masked_fill_(mask.unsqueeze(1).unsqueeze(1), float("-inf"))
weights = F.softmax(weights.flatten(2), dim=-1).view_as(weights)
weights = self.dropout(weights)
return weights
def dice_loss(inputs, targets, num_boxes):
"""
Compute the DICE loss, similar to generalized IOU for masks
Args:
inputs: A float tensor of arbitrary shape.
The predictions for each example.
targets: A float tensor with the same shape as inputs. Stores the binary
classification label for each element in inputs
(0 for the negative class and 1 for the positive class).
"""
inputs = inputs.sigmoid()
inputs = inputs.flatten(1)
numerator = 2 * (inputs * targets).sum(1)
denominator = inputs.sum(-1) + targets.sum(-1)
loss = 1 - (numerator + 1) / (denominator + 1)
return loss.sum() / num_boxes
def sigmoid_focal_loss(inputs, targets, num_boxes, alpha: float = 0.25, gamma: float = 2):
"""
Loss used in RetinaNet for dense detection: https://arxiv.org/abs/1708.02002.
Args:
inputs: A float tensor of arbitrary shape.
The predictions for each example.
targets: A float tensor with the same shape as inputs. Stores the binary
classification label for each element in inputs
(0 for the negative class and 1 for the positive class).
alpha: (optional) Weighting factor in range (0,1) to balance
positive vs negative examples. Default = -1 (no weighting).
gamma: Exponent of the modulating factor (1 - p_t) to
balance easy vs hard examples.
Returns:
Loss tensor
"""
prob = inputs.sigmoid()
ce_loss = F.binary_cross_entropy_with_logits(inputs, targets, reduction="none")
p_t = prob * targets + (1 - prob) * (1 - targets)
loss = ce_loss * ((1 - p_t) ** gamma)
if alpha >= 0:
alpha_t = alpha * targets + (1 - alpha) * (1 - targets)
loss = alpha_t * loss
return loss.mean(1).sum() / num_boxes
class PostProcessSegm(nn.Module):
def __init__(self, threshold=0.5):
super().__init__()
self.threshold = threshold
@torch.no_grad()
def forward(self, results, outputs, orig_target_sizes, max_target_sizes):
assert len(orig_target_sizes) == len(max_target_sizes)
max_h, max_w = max_target_sizes.max(0)[0].tolist()
outputs_masks = outputs["pred_masks"].squeeze(2)
outputs_masks = F.interpolate(outputs_masks, size=(max_h, max_w), mode="bilinear", align_corners=False)
outputs_masks = (outputs_masks.sigmoid() > self.threshold).cpu()
for i, (cur_mask, t, tt) in enumerate(zip(outputs_masks, max_target_sizes, orig_target_sizes)):
img_h, img_w = t[0], t[1]
results[i]["masks"] = cur_mask[:, :img_h, :img_w].unsqueeze(1)
results[i]["masks"] = F.interpolate(
results[i]["masks"].float(), size=tuple(tt.tolist()), mode="nearest"
).byte()
return results
class PostProcessPanoptic(nn.Module):
"""This class converts the output of the model to the final panoptic result, in the format expected by the
coco panoptic API """
def __init__(self, is_thing_map, threshold=0.85):
"""
Parameters:
is_thing_map: This is a whose keys are the class ids, and the values a boolean indicating whether
the class is a thing (True) or a stuff (False) class
threshold: confidence threshold: segments with confidence lower than this will be deleted
"""
super().__init__()
self.threshold = threshold
self.is_thing_map = is_thing_map
def forward(self, outputs, processed_sizes, target_sizes=None):
""" This function computes the panoptic prediction from the model's predictions.
Parameters:
outputs: This is a dict coming directly from the model. See the model doc for the content.
processed_sizes: This is a list of tuples (or torch tensors) of sizes of the images that were passed to the
model, ie the size after data augmentation but before batching.
target_sizes: This is a list of tuples (or torch tensors) corresponding to the requested final size
of each prediction. If left to None, it will default to the processed_sizes
"""
if target_sizes is None:
target_sizes = processed_sizes
assert len(processed_sizes) == len(target_sizes)
out_logits, raw_masks, raw_boxes = outputs["pred_logits"], outputs["pred_masks"], outputs["pred_boxes"]
assert len(out_logits) == len(raw_masks) == len(target_sizes)
preds = []
def to_tuple(tup):
if isinstance(tup, tuple):
return tup
return tuple(tup.cpu().tolist())
for cur_logits, cur_masks, cur_boxes, size, target_size in zip(
out_logits, raw_masks, raw_boxes, processed_sizes, target_sizes
):
# we filter empty queries and detection below threshold
scores, labels = cur_logits.softmax(-1).max(-1)
keep = labels.ne(outputs["pred_logits"].shape[-1] - 1) & (scores > self.threshold)
cur_scores, cur_classes = cur_logits.softmax(-1).max(-1)
cur_scores = cur_scores[keep]
cur_classes = cur_classes[keep]
cur_masks = cur_masks[keep]
cur_masks = interpolate(cur_masks[None], to_tuple(size), mode="bilinear").squeeze(0)
cur_boxes = box_ops.box_cxcywh_to_xyxy(cur_boxes[keep])
h, w = cur_masks.shape[-2:]
assert len(cur_boxes) == len(cur_classes)
# It may be that we have several predicted masks for the same stuff class.
# In the following, we track the list of masks ids for each stuff class (they are merged later on)
cur_masks = cur_masks.flatten(1)
stuff_equiv_classes = defaultdict(lambda: [])
for k, label in enumerate(cur_classes):
if not self.is_thing_map[label.item()]:
stuff_equiv_classes[label.item()].append(k)
def get_ids_area(masks, scores, dedup=False):
# This helper function creates the final panoptic segmentation image
# It also returns the area of the masks that appears on the image
m_id = masks.transpose(0, 1).softmax(-1)
if m_id.shape[-1] == 0:
# We didn't detect any mask :(
m_id = torch.zeros((h, w), dtype=torch.long, device=m_id.device)
else:
m_id = m_id.argmax(-1).view(h, w)
if dedup:
# Merge the masks corresponding to the same stuff class
for equiv in stuff_equiv_classes.values():
if len(equiv) > 1:
for eq_id in equiv:
m_id.masked_fill_(m_id.eq(eq_id), equiv[0])
final_h, final_w = to_tuple(target_size)
seg_img = Image.fromarray(id2rgb(m_id.view(h, w).cpu().numpy()))
seg_img = seg_img.resize(size=(final_w, final_h), resample=Image.NEAREST)
np_seg_img = (
torch.ByteTensor(torch.ByteStorage.from_buffer(seg_img.tobytes())).view(final_h, final_w, 3).numpy()
)
m_id = torch.from_numpy(rgb2id(np_seg_img))
area = []
for i in range(len(scores)):
area.append(m_id.eq(i).sum().item())
return area, seg_img
area, seg_img = get_ids_area(cur_masks, cur_scores, dedup=True)
if cur_classes.numel() > 0:
# We know filter empty masks as long as we find some
while True:
filtered_small = torch.as_tensor(
[area[i] <= 4 for i, c in enumerate(cur_classes)], dtype=torch.bool, device=keep.device
)
if filtered_small.any().item():
cur_scores = cur_scores[~filtered_small]
cur_classes = cur_classes[~filtered_small]
cur_masks = cur_masks[~filtered_small]
area, seg_img = get_ids_area(cur_masks, cur_scores)
else:
break
else:
cur_classes = torch.ones(1, dtype=torch.long, device=cur_classes.device)
segments_info = []
for i, a in enumerate(area):
cat = cur_classes[i].item()
segments_info.append({"id": i, "isthing": self.is_thing_map[cat], "category_id": cat, "area": a})
del cur_classes
with io.BytesIO() as out:
seg_img.save(out, format="PNG")
predictions = {"png_string": out.getvalue(), "segments_info": segments_info}
preds.append(predictions)
return preds
================================================
FILE: batchformer-v2/deformable-detr/requirements.txt
================================================
pycocotools
tqdm
cython
scipy
================================================
FILE: batchformer-v2/deformable-detr/tools/launch.py
================================================
# --------------------------------------------------------------------------------------------------------------------------
# Deformable DETR
# Copyright (c) 2020 SenseTime. All Rights Reserved.
# Licensed under the Apache License, Version 2.0 [see LICENSE for details]
# --------------------------------------------------------------------------------------------------------------------------
# Modified from https://github.com/pytorch/pytorch/blob/173f224570017b4b1a3a1a13d0bff280a54d9cd9/torch/distributed/launch.py
# --------------------------------------------------------------------------------------------------------------------------
r"""
`torch.distributed.launch` is a module that spawns up multiple distributed
training processes on each of the training nodes.
The utility can be used for single-node distributed training, in which one or
more processes per node will be spawned. The utility can be used for either
CPU training or GPU training. If the utility is used for GPU training,
each distributed process will be operating on a single GPU. This can achieve
well-improved single-node training performance. It can also be used in
multi-node distributed training, by spawning up multiple processes on each node
for well-improved multi-node distributed training performance as well.
This will especially be benefitial for systems with multiple Infiniband
interfaces that have direct-GPU support, since all of them can be utilized for
aggregated communication bandwidth.
In both cases of single-node distributed training or multi-node distributed
training, this utility will launch the given number of processes per node
(``--nproc_per_node``). If used for GPU training, this number needs to be less
or euqal to the number of GPUs on the current system (``nproc_per_node``),
and each process will be operating on a single GPU from *GPU 0 to
GPU (nproc_per_node - 1)*.
**How to use this module:**
1. Single-Node multi-process distributed training
::
>>> python -m torch.distributed.launch --nproc_per_node=NUM_GPUS_YOU_HAVE
YOUR_TRAINING_SCRIPT.py (--arg1 --arg2 --arg3 and all other
arguments of your training script)
2. Multi-Node multi-process distributed training: (e.g. two nodes)
Node 1: *(IP: 192.168.1.1, and has a free port: 1234)*
::
>>> python -m torch.distributed.launch --nproc_per_node=NUM_GPUS_YOU_HAVE
--nnodes=2 --node_rank=0 --master_addr="192.168.1.1"
--master_port=1234 YOUR_TRAINING_SCRIPT.py (--arg1 --arg2 --arg3
and all other arguments of your training script)
Node 2:
::
>>> python -m torch.distributed.launch --nproc_per_node=NUM_GPUS_YOU_HAVE
--nnodes=2 --node_rank=1 --master_addr="192.168.1.1"
--master_port=1234 YOUR_TRAINING_SCRIPT.py (--arg1 --arg2 --arg3
and all other arguments of your training script)
3. To look up what optional arguments this module offers:
::
>>> python -m torch.distributed.launch --help
**Important Notices:**
1. This utilty and multi-process distributed (single-node or
multi-node) GPU training currently only achieves the best performance using
the NCCL distributed backend. Thus NCCL backend is the recommended backend to
use for GPU training.
2. In your training program, you must parse the command-line argument:
``--local_rank=LOCAL_PROCESS_RANK``, which will be provided by this module.
If your training program uses GPUs, you should ensure that your code only
runs on the GPU device of LOCAL_PROCESS_RANK. This can be done by:
Parsing the local_rank argument
::
>>> import argparse
>>> parser = argparse.ArgumentParser()
>>> parser.add_argument("--local_rank", type=int)
>>> args = parser.parse_args()
Set your device to local rank using either
::
>>> torch.cuda.set_device(arg.local_rank) # before your code runs
or
::
>>> with torch.cuda.device(arg.local_rank):
>>> # your code to run
3. In your training program, you are supposed to call the following function
at the beginning to start the distributed backend. You need to make sure that
the init_method uses ``env://``, which is the only supported ``init_method``
by this module.
::
torch.distributed.init_process_group(backend='YOUR BACKEND',
init_method='env://')
4. In your training program, you can either use regular distributed functions
or use :func:`torch.nn.parallel.DistributedDataParallel` module. If your
training program uses GPUs for training and you would like to use
:func:`torch.nn.parallel.DistributedDataParallel` module,
here is how to configure it.
::
model = torch.nn.parallel.DistributedDataParallel(model,
device_ids=[arg.local_rank],
output_device=arg.local_rank)
Please ensure that ``device_ids`` argument is set to be the only GPU device id
that your code will be operating on. This is generally the local rank of the
process. In other words, the ``device_ids`` needs to be ``[args.local_rank]``,
and ``output_device`` needs to be ``args.local_rank`` in order to use this
utility
5. Another way to pass ``local_rank`` to the subprocesses via environment variable
``LOCAL_RANK``. This behavior is enabled when you launch the script with
``--use_env=True``. You must adjust the subprocess example above to replace
``args.local_rank`` with ``os.environ['LOCAL_RANK']``; the launcher
will not pass ``--local_rank`` when you specify this flag.
.. warning::
``local_rank`` is NOT globally unique: it is only unique per process
on a machine. Thus, don't use it to decide if you should, e.g.,
write to a networked filesystem. See
https://github.com/pytorch/pytorch/issues/12042 for an example of
how things can go wrong if you don't do this correctly.
"""
import sys
import subprocess
import os
import socket
from argparse import ArgumentParser, REMAINDER
import torch
def parse_args():
"""
Helper function parsing the command line options
@retval ArgumentParser
"""
parser = ArgumentParser(description="PyTorch distributed training launch "
"helper utilty that will spawn up "
"multiple distributed processes")
# Optional arguments for the launch helper
parser.add_argument("--nnodes", type=int, default=1,
help="The number of nodes to use for distributed "
"training")
parser.add_argument("--node_rank", type=int, default=0,
help="The rank of the node for multi-node distributed "
"training")
parser.add_argument("--nproc_per_node", type=int, default=1,
help="The number of processes to launch on each node, "
"for GPU training, this is recommended to be set "
"to the number of GPUs in your system so that "
"each process can be bound to a single GPU.")
parser.add_argument("--master_addr", default="127.0.0.1", type=str,
help="Master node (rank 0)'s address, should be either "
"the IP address or the hostname of node 0, for "
"single node multi-proc training, the "
"--master_addr can simply be 127.0.0.1")
parser.add_argument("--master_port", default=29500, type=int,
help="Master node (rank 0)'s free port that needs to "
"be used for communciation during distributed "
"training")
# positional
parser.add_argument("training_script", type=str,
help="The full path to the single GPU training "
"program/script to be launched in parallel, "
"followed by all the arguments for the "
"training script")
# rest from the training program
parser.add_argument('training_script_args', nargs=REMAINDER)
return parser.parse_args()
def main():
args = parse_args()
# world size in terms of number of processes
dist_world_size = args.nproc_per_node * args.nnodes
# set PyTorch distributed related environmental variables
current_env = os.environ.copy()
current_env["MASTER_ADDR"] = args.master_addr
current_env["MASTER_PORT"] = str(args.master_port)
current_env["WORLD_SIZE"] = str(dist_world_size)
processes = []
for local_rank in range(0, args.nproc_per_node):
# each process's rank
dist_rank = args.nproc_per_node * args.node_rank + local_rank
current_env["RANK"] = str(dist_rank)
current_env["LOCAL_RANK"] = str(local_rank)
cmd = [args.training_script] + args.training_script_args
process = subprocess.Popen(cmd, env=current_env)
processes.append(process)
for process in processes:
process.wait()
if process.returncode != 0:
raise subprocess.CalledProcessError(returncode=process.returncode,
cmd=process.args)
if __name__ == "__main__":
main()
================================================
FILE: batchformer-v2/deformable-detr/tools/run_dist_launch.sh
================================================
#!/usr/bin/env bash
# ------------------------------------------------------------------------
# Deformable DETR
# Copyright (c) 2020 SenseTime. All Rights Reserved.
# Licensed under the Apache License, Version 2.0 [see LICENSE for details]
# ------------------------------------------------------------------------
set -x
GPUS=$1
RUN_COMMAND=${@:2}
if [ $GPUS -lt 8 ]; then
GPUS_PER_NODE=${GPUS_PER_NODE:-$GPUS}
else
GPUS_PER_NODE=${GPUS_PER_NODE:-8}
fi
MASTER_ADDR=${MASTER_ADDR:-"127.0.0.1"}
MASTER_PORT=${MASTER_PORT:-"29500"}
NODE_RANK=${NODE_RANK:-0}
let "NNODES=GPUS/GPUS_PER_NODE"
python ./tools/launch.py \
--nnodes ${NNODES} \
--node_rank ${NODE_RANK} \
--master_addr ${MASTER_ADDR} \
--master_port ${MASTER_PORT} \
--nproc_per_node ${GPUS_PER_NODE} \
${RUN_COMMAND}
================================================
FILE: batchformer-v2/deformable-detr/tools/run_dist_slurm.sh
================================================
#!/usr/bin/env bash
# --------------------------------------------------------------------------------------------------------------------------
# Deformable DETR
# Copyright (c) 2020 SenseTime. All Rights Reserved.
# Licensed under the Apache License, Version 2.0 [see LICENSE for details]
# --------------------------------------------------------------------------------------------------------------------------
# Modified from https://github.com/open-mmlab/mmdetection/blob/3b53fe15d87860c6941f3dda63c0f27422da6266/tools/slurm_train.sh
# --------------------------------------------------------------------------------------------------------------------------
set -x
PARTITION=$1
JOB_NAME=$2
GPUS=$3
RUN_COMMAND=${@:4}
if [ $GPUS -lt 8 ]; then
GPUS_PER_NODE=${GPUS_PER_NODE:-$GPUS}
else
GPUS_PER_NODE=${GPUS_PER_NODE:-8}
fi
CPUS_PER_TASK=${CPUS_PER_TASK:-4}
SRUN_ARGS=${SRUN_ARGS:-""}
srun -p ${PARTITION} \
--job-name=${JOB_NAME} \
--gres=gpu:${GPUS_PER_NODE} \
--ntasks=${GPUS} \
--ntasks-per-node=${GPUS_PER_NODE} \
--cpus-per-task=${CPUS_PER_TASK} \
--kill-on-bad-exit=1 \
${SRUN_ARGS} \
${RUN_COMMAND}
================================================
FILE: batchformer-v2/deformable-detr/util/__init__.py
================================================
# ------------------------------------------------------------------------
# Deformable DETR
# Copyright (c) 2020 SenseTime. All Rights Reserved.
# Licensed under the Apache License, Version 2.0 [see LICENSE for details]
# ------------------------------------------------------------------------
# Modified from DETR (https://github.com/facebookresearch/detr)
# Copyright (c) Facebook, Inc. and its affiliates. All Rights Reserved
# ------------------------------------------------------------------------
================================================
FILE: batchformer-v2/deformable-detr/util/box_ops.py
================================================
# ------------------------------------------------------------------------
# Deformable DETR
# Copyright (c) 2020 SenseTime. All Rights Reserved.
# Licensed under the Apache License, Version 2.0 [see LICENSE for details]
# ------------------------------------------------------------------------
# Modified from DETR (https://github.com/facebookresearch/detr)
# Copyright (c) Facebook, Inc. and its affiliates. All Rights Reserved
# ------------------------------------------------------------------------
"""
Utilities for bounding box manipulation and GIoU.
"""
import torch
from torchvision.ops.boxes import box_area
def box_cxcywh_to_xyxy(x):
x_c, y_c, w, h = x.unbind(-1)
b = [(x_c - 0.5 * w), (y_c - 0.5 * h),
(x_c + 0.5 * w), (y_c + 0.5 * h)]
return torch.stack(b, dim=-1)
def box_xyxy_to_cxcywh(x):
x0, y0, x1, y1 = x.unbind(-1)
b = [(x0 + x1) / 2, (y0 + y1) / 2,
(x1 - x0), (y1 - y0)]
return torch.stack(b, dim=-1)
# modified from torchvision to also return the union
def box_iou(boxes1, boxes2):
area1 = box_area(boxes1)
area2 = box_area(boxes2)
lt = torch.max(boxes1[:, None, :2], boxes2[:, :2]) # [N,M,2]
rb = torch.min(boxes1[:, None, 2:], boxes2[:, 2:]) # [N,M,2]
wh = (rb - lt).clamp(min=0) # [N,M,2]
inter = wh[:, :, 0] * wh[:, :, 1] # [N,M]
union = area1[:, None] + area2 - inter
iou = inter / union
return iou, union
def generalized_box_iou(boxes1, boxes2):
"""
Generalized IoU from https://giou.stanford.edu/
The boxes should be in [x0, y0, x1, y1] format
Returns a [N, M] pairwise matrix, where N = len(boxes1)
and M = len(boxes2)
"""
# degenerate boxes gives inf / nan results
# so do an early check
assert (boxes1[:, 2:] >= boxes1[:, :2]).all()
assert (boxes2[:, 2:] >= boxes2[:, :2]).all()
iou, union = box_iou(boxes1, boxes2)
lt = torch.min(boxes1[:, None, :2], boxes2[:, :2])
rb = torch.max(boxes1[:, None, 2:], boxes2[:, 2:])
wh = (rb - lt).clamp(min=0) # [N,M,2]
area = wh[:, :, 0] * wh[:, :, 1]
return iou - (area - union) / area
def masks_to_boxes(masks):
"""Compute the bounding boxes around the provided masks
The masks should be in format [N, H, W] where N is the number of masks, (H, W) are the spatial dimensions.
Returns a [N, 4] tensors, with the boxes in xyxy format
"""
if masks.numel() == 0:
return torch.zeros((0, 4), device=masks.device)
h, w = masks.shape[-2:]
y = torch.arange(0, h, dtype=torch.float)
x = torch.arange(0, w, dtype=torch.float)
y, x = torch.meshgrid(y, x)
x_mask = (masks * x.unsqueeze(0))
x_max = x_mask.flatten(1).max(-1)[0]
x_min = x_mask.masked_fill(~(masks.bool()), 1e8).flatten(1).min(-1)[0]
y_mask = (masks * y.unsqueeze(0))
y_max = y_mask.flatten(1).max(-1)[0]
y_min = y_mask.masked_fill(~(masks.bool()), 1e8).flatten(1).min(-1)[0]
return torch.stack([x_min, y_min, x_max, y_max], 1)
================================================
FILE: batchformer-v2/deformable-detr/util/misc.py
================================================
# ------------------------------------------------------------------------
# Deformable DETR
# Copyright (c) 2020 SenseTime. All Rights Reserved.
# Licensed under the Apache License, Version 2.0 [see LICENSE for details]
# ------------------------------------------------------------------------
# Modified from DETR (https://github.com/facebookresearch/detr)
# Copyright (c) Facebook, Inc. and its affiliates. All Rights Reserved
# ------------------------------------------------------------------------
"""
Misc functions, including distributed helpers.
Mostly copy-paste from torchvision references.
"""
import os
import subprocess
import time
from collections import defaultdict, deque
import datetime
import pickle
from typing import Optional, List
import torch
import torch.nn as nn
import torch.distributed as dist
from torch import Tensor
# needed due to empty tensor bug in pytorch and torchvision 0.5
import torchvision
if float(torchvision.__version__[:3]) < 0.5 and float(torchvision.__version__[:3]) > 0.1:
import math
from torchvision.ops.misc import _NewEmptyTensorOp
def _check_size_scale_factor(dim, size, scale_factor):
# type: (int, Optional[List[int]], Optional[float]) -> None
if size is None and scale_factor is None:
raise ValueError("either size or scale_factor should be defined")
if size is not None and scale_factor is not None:
raise ValueError("only one of size or scale_factor should be defined")
if not (scale_factor is not None and len(scale_factor) != dim):
raise ValueError(
"scale_factor shape must match input shape. "
"Input is {}D, scale_factor size is {}".format(dim, len(scale_factor))
)
def _output_size(dim, input, size, scale_factor):
# type: (int, Tensor, Optional[List[int]], Optional[float]) -> List[int]
assert dim == 2
_check_size_scale_factor(dim, size, scale_factor)
if size is not None:
return size
# if dim is not 2 or scale_factor is iterable use _ntuple instead of concat
assert scale_factor is not None and isinstance(scale_factor, (int, float))
scale_factors = [scale_factor, scale_factor]
# math.floor might return float in py2.7
return [
int(math.floor(input.size(i + 2) * scale_factors[i])) for i in range(dim)
]
elif float(torchvision.__version__[:3]) < 0.7 and float(torchvision.__version__[:3]) > 0.1:
from torchvision.ops import _new_empty_tensor
from torchvision.ops.misc import _output_size
class SmoothedValue(object):
"""Track a series of values and provide access to smoothed values over a
window or the global series average.
"""
def __init__(self, window_size=20, fmt=None):
if fmt is None:
fmt = "{median:.4f} ({global_avg:.4f})"
self.deque = deque(maxlen=window_size)
self.total = 0.0
self.count = 0
self.fmt = fmt
def update(self, value, n=1):
self.deque.append(value)
self.count += n
self.total += value * n
def synchronize_between_processes(self):
"""
Warning: does not synchronize the deque!
"""
if not is_dist_avail_and_initialized():
return
t = torch.tensor([self.count, self.total], dtype=torch.float64, device='cuda')
dist.barrier()
dist.all_reduce(t)
t = t.tolist()
self.count = int(t[0])
self.total = t[1]
@property
def median(self):
d = torch.tensor(list(self.deque))
return d.median().item()
@property
def avg(self):
d = torch.tensor(list(self.deque), dtype=torch.float32)
return d.mean().item()
@property
def global_avg(self):
return self.total / self.count
@property
def max(self):
return max(self.deque)
@property
def value(self):
return self.deque[-1]
def __str__(self):
return self.fmt.format(
median=self.median,
avg=self.avg,
global_avg=self.global_avg,
max=self.max,
value=self.value)
def all_gather(data):
"""
Run all_gather on arbitrary picklable data (not necessarily tensors)
Args:
data: any picklable object
Returns:
list[data]: list of data gathered from each rank
"""
world_size = get_world_size()
if world_size == 1:
return [data]
# serialized to a Tensor
buffer = pickle.dumps(data)
storage = torch.ByteStorage.from_buffer(buffer)
tensor = torch.ByteTensor(storage).to("cuda")
# obtain Tensor size of each rank
local_size = torch.tensor([tensor.numel()], device="cuda")
size_list = [torch.tensor([0], device="cuda") for _ in range(world_size)]
dist.all_gather(size_list, local_size)
size_list = [int(size.item()) for size in size_list]
max_size = max(size_list)
# receiving Tensor from all ranks
# we pad the tensor because torch all_gather does not support
# gathering tensors of different shapes
tensor_list = []
for _ in size_list:
tensor_list.append(torch.empty((max_size,), dtype=torch.uint8, device="cuda"))
if local_size != max_size:
padding = torch.empty(size=(max_size - local_size,), dtype=torch.uint8, device="cuda")
tensor = torch.cat((tensor, padding), dim=0)
dist.all_gather(tensor_list, tensor)
data_list = []
for size, tensor in zip(size_list, tensor_list):
buffer = tensor.cpu().numpy().tobytes()[:size]
data_list.append(pickle.loads(buffer))
return data_list
def reduce_dict(input_dict, average=True):
"""
Args:
input_dict (dict): all the values will be reduced
average (bool): whether to do average or sum
Reduce the values in the dictionary from all processes so that all processes
have the averaged results. Returns a dict with the same fields as
input_dict, after reduction.
"""
world_size = get_world_size()
if world_size < 2:
return input_dict
with torch.no_grad():
names = []
values = []
# sort the keys so that they are consistent across processes
for k in sorted(input_dict.keys()):
names.append(k)
values.append(input_dict[k])
values = torch.stack(values, dim=0)
dist.all_reduce(values)
if average:
values /= world_size
reduced_dict = {k: v for k, v in zip(names, values)}
return reduced_dict
class MetricLogger(object):
def __init__(self, delimiter="\t"):
self.meters = defaultdict(SmoothedValue)
self.delimiter = delimiter
def update(self, **kwargs):
for k, v in kwargs.items():
if isinstance(v, torch.Tensor):
v = v.item()
assert isinstance(v, (float, int))
self.meters[k].update(v)
def __getattr__(self, attr):
if attr in self.meters:
return self.meters[attr]
if attr in self.__dict__:
return self.__dict__[attr]
raise AttributeError("'{}' object has no attribute '{}'".format(
type(self).__name__, attr))
def __str__(self):
loss_str = []
for name, meter in self.meters.items():
loss_str.append(
"{}: {}".format(name, str(meter))
)
return self.delimiter.join(loss_str)
def synchronize_between_processes(self):
for meter in self.meters.values():
meter.synchronize_between_processes()
def add_meter(self, name, meter):
self.meters[name] = meter
def log_every(self, iterable, print_freq, header=None):
i = 0
if not header:
header = ''
start_time = time.time()
end = time.time()
iter_time = SmoothedValue(fmt='{avg:.4f}')
data_time = SmoothedValue(fmt='{avg:.4f}')
space_fmt = ':' + str(len(str(len(iterable)))) + 'd'
if torch.cuda.is_available():
log_msg = self.delimiter.join([
header,
'[{0' + space_fmt + '}/{1}]',
'eta: {eta}',
'{meters}',
'time: {time}',
'data: {data}',
'max mem: {memory:.0f}'
])
else:
log_msg = self.delimiter.join([
header,
'[{0' + space_fmt + '}/{1}]',
'eta: {eta}',
'{meters}',
'time: {time}',
'data: {data}'
])
MB = 1024.0 * 1024.0
for obj in iterable:
data_time.update(time.time() - end)
yield obj
iter_time.update(time.time() - end)
if i % print_freq == 0 or i == len(iterable) - 1:
eta_seconds = iter_time.global_avg * (len(iterable) - i)
eta_string = str(datetime.timedelta(seconds=int(eta_seconds)))
if torch.cuda.is_available():
print(log_msg.format(
i, len(iterable), eta=eta_string,
meters=str(self),
time=str(iter_time), data=str(data_time),
memory=torch.cuda.max_memory_allocated() / MB))
else:
print(log_msg.format(
i, len(iterable), eta=eta_string,
meters=str(self),
time=str(iter_time), data=str(data_time)))
i += 1
end = time.time()
total_time = time.time() - start_time
total_time_str = str(datetime.timedelta(seconds=int(total_time)))
print('{} Total time: {} ({:.4f} s / it)'.format(
header, total_time_str, total_time / len(iterable)))
def get_sha():
cwd = os.path.dirname(os.path.abspath(__file__))
def _run(command):
return subprocess.check_output(command, cwd=cwd).decode('ascii').strip()
sha = 'N/A'
diff = "clean"
branch = 'N/A'
try:
sha = _run(['git', 'rev-parse', 'HEAD'])
subprocess.check_output(['git', 'diff'], cwd=cwd)
diff = _run(['git', 'diff-index', 'HEAD'])
diff = "has uncommited changes" if diff else "clean"
branch = _run(['git', 'rev-parse', '--abbrev-ref', 'HEAD'])
except Exception:
pass
message = f"sha: {sha}, status: {diff}, branch: {branch}"
return message
def collate_fn(batch):
batch = list(zip(*batch))
batch[0] = nested_tensor_from_tensor_list(batch[0])
return tuple(batch)
def _max_by_axis(the_list):
# type: (List[List[int]]) -> List[int]
maxes = the_list[0]
for sublist in the_list[1:]:
for index, item in enumerate(sublist):
maxes[index] = max(maxes[index], item)
return maxes
def nested_tensor_from_tensor_list(tensor_list: List[Tensor]):
# TODO make this more general
if tensor_list[0].ndim == 3:
# TODO make it support different-sized images
max_size = _max_by_axis([list(img.shape) for img in tensor_list])
# min_size = tuple(min(s) for s in zip(*[img.shape for img in tensor_list]))
batch_shape = [len(tensor_list)] + max_size
b, c, h, w = batch_shape
dtype = tensor_list[0].dtype
device = tensor_list[0].device
tensor = torch.zeros(batch_shape, dtype=dtype, device=device)
mask = torch.ones((b, h, w), dtype=torch.bool, device=device)
for img, pad_img, m in zip(tensor_list, tensor, mask):
pad_img[: img.shape[0], : img.shape[1], : img.shape[2]].copy_(img)
m[: img.shape[1], :img.shape[2]] = False
else:
raise ValueError('not supported')
return NestedTensor(tensor, mask)
class NestedTensor(object):
def __init__(self, tensors, mask: Optional[Tensor]):
self.tensors = tensors
self.mask = mask
def to(self, device, non_blocking=False):
# type: (Device) -> NestedTensor # noqa
cast_tensor = self.tensors.to(device, non_blocking=non_blocking)
mask = self.mask
if mask is not None:
assert mask is not None
cast_mask = mask.to(device, non_blocking=non_blocking)
else:
cast_mask = None
return NestedTensor(cast_tensor, cast_mask)
def record_stream(self, *args, **kwargs):
self.tensors.record_stream(*args, **kwargs)
if self.mask is not None:
self.mask.record_stream(*args, **kwargs)
def decompose(self):
return self.tensors, self.mask
def __repr__(self):
return str(self.tensors)
def setup_for_distributed(is_master):
"""
This function disables printing when not in master process
"""
import builtins as __builtin__
builtin_print = __builtin__.print
def print(*args, **kwargs):
force = kwargs.pop('force', False)
if is_master or force:
builtin_print(*args, **kwargs)
__builtin__.print = print
def is_dist_avail_and_initialized():
if not dist.is_available():
return False
if not dist.is_initialized():
return False
return True
def get_world_size():
if not is_dist_avail_and_initialized():
return 1
return dist.get_world_size()
def get_rank():
if not is_dist_avail_and_initialized():
return 0
return dist.get_rank()
def get_local_size():
if not is_dist_avail_and_initialized():
return 1
return int(os.environ['LOCAL_SIZE'])
def get_local_rank():
if not is_dist_avail_and_initialized():
return 0
return int(os.environ['LOCAL_RANK'])
def is_main_process():
return get_rank() == 0
def save_on_master(*args, **kwargs):
if is_main_process():
torch.save(*args, **kwargs)
def init_distributed_mode(args):
if 'RANK' in os.environ and 'WORLD_SIZE' in os.environ:
args.rank = int(os.environ["RANK"])
args.world_size = int(os.environ['WORLD_SIZE'])
args.gpu = int(os.environ['LOCAL_RANK'])
args.dist_url = 'env://'
os.environ['LOCAL_SIZE'] = str(torch.cuda.device_count())
elif 'SLURM_PROCID' in os.environ:
proc_id = int(os.environ['SLURM_PROCID'])
ntasks = int(os.environ['SLURM_NTASKS'])
node_list = os.environ['SLURM_NODELIST']
num_gpus = torch.cuda.device_count()
addr = subprocess.getoutput(
'scontrol show hostname {} | head -n1'.format(node_list))
os.environ['MASTER_PORT'] = os.environ.get('MASTER_PORT', '29500')
os.environ['MASTER_ADDR'] = addr
os.environ['WORLD_SIZE'] = str(ntasks)
os.environ['RANK'] = str(proc_id)
os.environ['LOCAL_RANK'] = str(proc_id % num_gpus)
os.environ['LOCAL_SIZE'] = str(num_gpus)
args.dist_url = 'env://'
args.world_size = ntasks
args.rank = proc_id
args.gpu = proc_id % num_gpus
else:
print('Not using distributed mode')
args.distributed = False
return
args.distributed = True
torch.cuda.set_device(args.gpu)
args.dist_backend = 'nccl'
print('| distributed init (rank {}): {}'.format(
args.rank, args.dist_url), flush=True)
torch.distributed.init_process_group(backend=args.dist_backend, init_method=args.dist_url,
world_size=args.world_size, rank=args.rank)
torch.distributed.barrier()
setup_for_distributed(args.rank == 0)
@torch.no_grad()
def accuracy(output, target, topk=(1,)):
"""Computes the precision@k for the specified values of k"""
if target.numel() == 0:
return [torch.zeros([], device=output.device)]
maxk = max(topk)
batch_size = target.size(0)
_, pred = output.topk(maxk, 1, True, True)
pred = pred.t()
correct = pred.eq(target.view(1, -1).expand_as(pred))
res = []
for k in topk:
correct_k = correct[:k].view(-1).float().sum(0)
res.append(correct_k.mul_(100.0 / batch_size))
return res
def interpolate(input, size=None, scale_factor=None, mode="nearest", align_corners=None):
# type: (Tensor, Optional[List[int]], Optional[float], str, Optional[bool]) -> Tensor
"""
Equivalent to nn.functional.interpolate, but with support for empty batch sizes.
This will eventually be supported natively by PyTorch, and this
class can go away.
"""
if float(torchvision.__version__[:3]) < 0.7:
if input.numel() > 0:
return torch.nn.functional.interpolate(
input, size, scale_factor, mode, align_corners
)
output_shape = _output_size(2, input, size, scale_factor)
output_shape = list(input.shape[:-2]) + list(output_shape)
if float(torchvision.__version__[:3]) < 0.5:
return _NewEmptyTensorOp.apply(input, output_shape)
return _new_empty_tensor(input, output_shape)
else:
return torchvision.ops.misc.interpolate(input, size, scale_factor, mode, align_corners)
def get_total_grad_norm(parameters, norm_type=2):
parameters = list(filter(lambda p: p.grad is not None, parameters))
norm_type = float(norm_type)
device = parameters[0].grad.device
total_norm = torch.norm(torch.stack([torch.norm(p.grad.detach(), norm_type).to(device) for p in parameters]),
norm_type)
return total_norm
def inverse_sigmoid(x, eps=1e-5):
x = x.clamp(min=0, max=1)
x1 = x.clamp(min=eps)
x2 = (1 - x).clamp(min=eps)
return torch.log(x1/x2)
================================================
FILE: batchformer-v2/deformable-detr/util/plot_utils.py
================================================
# ------------------------------------------------------------------------
# Deformable DETR
# Copyright (c) 2020 SenseTime. All Rights Reserved.
# Licensed under the Apache License, Version 2.0 [see LICENSE for details]
# ------------------------------------------------------------------------
# Modified from DETR (https://github.com/facebookresearch/detr)
# Copyright (c) Facebook, Inc. and its affiliates. All Rights Reserved
# ------------------------------------------------------------------------
"""
Plotting utilities to visualize training logs.
"""
import torch
import pandas as pd
import seaborn as sns
import matplotlib.pyplot as plt
from pathlib import Path, PurePath
def plot_logs(logs, fields=('class_error', 'loss_bbox_unscaled', 'mAP'), ewm_col=0, log_name='log.txt'):
'''
Function to plot specific fields from training log(s). Plots both training and test results.
:: Inputs - logs = list containing Path objects, each pointing to individual dir with a log file
- fields = which results to plot from each log file - plots both training and test for each field.
- ewm_col = optional, which column to use as the exponential weighted smoothing of the plots
- log_name = optional, name of log file if different than default 'log.txt'.
:: Outputs - matplotlib plots of results in fields, color coded for each log file.
- solid lines are training results, dashed lines are test results.
'''
func_name = "plot_utils.py::plot_logs"
# verify logs is a list of Paths (list[Paths]) or single Pathlib object Path,
# convert single Path to list to avoid 'not iterable' error
if not isinstance(logs, list):
if isinstance(logs, PurePath):
logs = [logs]
print(f"{func_name} info: logs param expects a list argument, converted to list[Path].")
else:
raise ValueError(f"{func_name} - invalid argument for logs parameter.\n \
Expect list[Path] or single Path obj, received {type(logs)}")
# verify valid dir(s) and that every item in list is Path object
for i, dir in enumerate(logs):
if not isinstance(dir, PurePath):
raise ValueError(f"{func_name} - non-Path object in logs argument of {type(dir)}: \n{dir}")
if dir.exists():
continue
raise ValueError(f"{func_name} - invalid directory in logs argument:\n{dir}")
# load log file(s) and plot
dfs = [pd.read_json(Path(p) / log_name, lines=True) for p in logs]
fig, axs = plt.subplots(ncols=len(fields), figsize=(16, 5))
for df, color in zip(dfs, sns.color_palette(n_colors=len(logs))):
for j, field in enumerate(fields):
if field == 'mAP':
coco_eval = pd.DataFrame(pd.np.stack(df.test_coco_eval.dropna().values)[:, 1]).ewm(com=ewm_col).mean()
axs[j].plot(coco_eval, c=color)
else:
df.interpolate().ewm(com=ewm_col).mean().plot(
y=[f'train_{field}', f'test_{field}'],
ax=axs[j],
color=[color] * 2,
style=['-', '--']
)
for ax, field in zip(axs, fields):
ax.legend([Path(p).name for p in logs])
ax.set_title(field)
def plot_precision_recall(files, naming_scheme='iter'):
if naming_scheme == 'exp_id':
# name becomes exp_id
names = [f.parts[-3] for f in files]
elif naming_scheme == 'iter':
names = [f.stem for f in files]
else:
raise ValueError(f'not supported {naming_scheme}')
fig, axs = plt.subplots(ncols=2, figsize=(16, 5))
for f, color, name in zip(files, sns.color_palette("Blues", n_colors=len(files)), names):
data = torch.load(f)
# precision is n_iou, n_points, n_cat, n_area, max_det
precision = data['precision']
recall = data['params'].recThrs
scores = data['scores']
# take precision for all classes, all areas and 100 detections
precision = precision[0, :, :, 0, -1].mean(1)
scores = scores[0, :, :, 0, -1].mean(1)
prec = precision.mean()
rec = data['recall'][0, :, 0, -1].mean()
print(f'{naming_scheme} {name}: mAP@50={prec * 100: 05.1f}, ' +
f'score={scores.mean():0.3f}, ' +
f'f1={2 * prec * rec / (prec + rec + 1e-8):0.3f}'
)
axs[0].plot(recall, precision, c=color)
axs[1].plot(recall, scores, c=color)
axs[0].set_title('Precision / Recall')
axs[0].legend(names)
axs[1].set_title('Scores / Recall')
axs[1].legend(names)
return fig, axs
================================================
FILE: batchformer-v2/deformable-detr/util/visualizer.py
================================================
import argparse
import cv2
import numpy as np
import re
from detectron2 import model_zoo
from detectron2.config import get_cfg, CfgNode
from detectron2.data import MetadataCatalog
from detectron2.engine import DefaultPredictor
from detectron2.structures import Instances
from detectron2.utils.visualizer import Visualizer, VisImage
if __name__ == "__main__":
image_file: str
for image_file in args.images:
img: np.ndarray = cv2.imread(image_file)
output: Instances = predictor(img)["instances"]
v = Visualizer(img[:, :, ::-1],
MetadataCatalog.get(cfg.DATASETS.TRAIN[0]),
scale=1.0)
result: VisImage = v.draw_instance_predictions(output.to("cpu"))
result_image: np.ndarray = result.get_image()[:, :, ::-1]
# get file name without extension, -1 to remove "." at the end
out_file_name: str = re.search(r"(.*)\.", image_file).group(0)[:-1]
out_file_name += "_processed.png"
cv2.imwrite(out_file_name, result_image)
================================================
FILE: batchformer-v2/deit_share/.circleci/config.yml
================================================
version: 2.1
jobs:
python_lint:
docker:
- image: circleci/python:3.7
steps:
- checkout
- run:
command: |
pip install --user --progress-bar off flake8 typing
flake8 .
test:
docker:
- image: circleci/python:3.7
steps:
- checkout
- run:
command: |
pip install --user --progress-bar off pytest
pip install --user --progress-bar off torch torchvision
pip install --user --progress-bar off timm==0.3.2
pytest .
workflows:
build:
jobs:
- python_lint
================================================
FILE: batchformer-v2/deit_share/.github/CODE_OF_CONDUCT.md
================================================
# Code of Conduct
Facebook has adopted a Code of Conduct that we expect project participants to adhere to.
Please read the [full text](https://code.fb.com/codeofconduct/)
so that you can understand what actions will and will not be tolerated.
================================================
FILE: batchformer-v2/deit_share/.github/CONTRIBUTING.md
================================================
# Contributing to DeiT
We want to make contributing to this project as easy and transparent as
possible.
## Our Development Process
Minor changes and improvements will be released on an ongoing basis. Larger changes (e.g., changesets implementing a new paper) will be released on a more periodic basis.
## Pull Requests
We actively welcome your pull requests.
1. Fork the repo and create your branch from `main`.
2. If you've added code that should be tested, add tests.
3. If you've changed APIs, update the documentation.
4. Ensure the test suite passes.
5. Make sure your code lints.
6. If you haven't already, complete the Contributor License Agreement ("CLA").
## Contributor License Agreement ("CLA")
In order to accept your pull request, we need you to submit a CLA. You only need
to do this once to work on any of Facebook's open source projects.
Complete your CLA here:
## Issues
We use GitHub issues to track public bugs. Please ensure your description is
clear and has sufficient instructions to be able to reproduce the issue.
Facebook has a [bounty program](https://www.facebook.com/whitehat/) for the safe
disclosure of security bugs. In those cases, please go through the process
outlined on that page and do not file a public issue.
## Coding Style
* 4 spaces for indentation rather than tabs
* 80 character line length
* PEP8 formatting following [Black](https://black.readthedocs.io/en/stable/)
## License
By contributing to DeiT, you agree that your contributions will be licensed
under the LICENSE file in the root directory of this source tree.
================================================
FILE: batchformer-v2/deit_share/.gitignore
================================================
*.swp
**/__pycache__/**
imnet_resnet50_scratch/timm_temp/
.dumbo.json
checkpoints/
================================================
FILE: batchformer-v2/deit_share/.idea/.gitignore
================================================
# Default ignored files
/shelf/
/workspace.xml
# Datasource local storage ignored files
/dataSources/
/dataSources.local.xml
# Editor-based HTTP Client requests
/httpRequests/
================================================
FILE: batchformer-v2/deit_share/.idea/deit_share.iml
================================================
================================================
FILE: batchformer-v2/deit_share/.idea/inspectionProfiles/profiles_settings.xml
================================================
================================================
FILE: batchformer-v2/deit_share/.idea/misc.xml
================================================
================================================
FILE: batchformer-v2/deit_share/.idea/modules.xml
================================================
================================================
FILE: batchformer-v2/deit_share/.idea/vcs.xml
================================================
================================================
FILE: batchformer-v2/deit_share/LICENSE
================================================
Apache License
Version 2.0, January 2004
http://www.apache.org/licenses/
TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
1. Definitions.
"License" shall mean the terms and conditions for use, reproduction,
and distribution as defined by Sections 1 through 9 of this document.
"Licensor" shall mean the copyright owner or entity authorized by
the copyright owner that is granting the License.
"Legal Entity" shall mean the union of the acting entity and all
other entities that control, are controlled by, or are under common
control with that entity. For the purposes of this definition,
"control" means (i) the power, direct or indirect, to cause the
direction or management of such entity, whether by contract or
otherwise, or (ii) ownership of fifty percent (50%) or more of the
outstanding shares, or (iii) beneficial ownership of such entity.
"You" (or "Your") shall mean an individual or Legal Entity
exercising permissions granted by this License.
"Source" form shall mean the preferred form for making modifications,
including but not limited to software source code, documentation
source, and configuration files.
"Object" form shall mean any form resulting from mechanical
transformation or translation of a Source form, including but
not limited to compiled object code, generated documentation,
and conversions to other media types.
"Work" shall mean the work of authorship, whether in Source or
Object form, made available under the License, as indicated by a
copyright notice that is included in or attached to the work
(an example is provided in the Appendix below).
"Derivative Works" shall mean any work, whether in Source or Object
form, that is based on (or derived from) the Work and for which the
editorial revisions, annotations, elaborations, or other modifications
represent, as a whole, an original work of authorship. For the purposes
of this License, Derivative Works shall not include works that remain
separable from, or merely link (or bind by name) to the interfaces of,
the Work and Derivative Works thereof.
"Contribution" shall mean any work of authorship, including
the original version of the Work and any modifications or additions
to that Work or Derivative Works thereof, that is intentionally
submitted to Licensor for inclusion in the Work by the copyright owner
or by an individual or Legal Entity authorized to submit on behalf of
the copyright owner. For the purposes of this definition, "submitted"
means any form of electronic, verbal, or written communication sent
to the Licensor or its representatives, including but not limited to
communication on electronic mailing lists, source code control systems,
and issue tracking systems that are managed by, or on behalf of, the
Licensor for the purpose of discussing and improving the Work, but
excluding communication that is conspicuously marked or otherwise
designated in writing by the copyright owner as "Not a Contribution."
"Contributor" shall mean Licensor and any individual or Legal Entity
on behalf of whom a Contribution has been received by Licensor and
subsequently incorporated within the Work.
2. Grant of Copyright License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
copyright license to reproduce, prepare Derivative Works of,
publicly display, publicly perform, sublicense, and distribute the
Work and such Derivative Works in Source or Object form.
3. Grant of Patent License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
(except as stated in this section) patent license to make, have made,
use, offer to sell, sell, import, and otherwise transfer the Work,
where such license applies only to those patent claims licensable
by such Contributor that are necessarily infringed by their
Contribution(s) alone or by combination of their Contribution(s)
with the Work to which such Contribution(s) was submitted. If You
institute patent litigation against any entity (including a
cross-claim or counterclaim in a lawsuit) alleging that the Work
or a Contribution incorporated within the Work constitutes direct
or contributory patent infringement, then any patent licenses
granted to You under this License for that Work shall terminate
as of the date such litigation is filed.
4. Redistribution. You may reproduce and distribute copies of the
Work or Derivative Works thereof in any medium, with or without
modifications, and in Source or Object form, provided that You
meet the following conditions:
(a) You must give any other recipients of the Work or
Derivative Works a copy of this License; and
(b) You must cause any modified files to carry prominent notices
stating that You changed the files; and
(c) You must retain, in the Source form of any Derivative Works
that You distribute, all copyright, patent, trademark, and
attribution notices from the Source form of the Work,
excluding those notices that do not pertain to any part of
the Derivative Works; and
(d) If the Work includes a "NOTICE" text file as part of its
distribution, then any Derivative Works that You distribute must
include a readable copy of the attribution notices contained
within such NOTICE file, excluding those notices that do not
pertain to any part of the Derivative Works, in at least one
of the following places: within a NOTICE text file distributed
as part of the Derivative Works; within the Source form or
documentation, if provided along with the Derivative Works; or,
within a display generated by the Derivative Works, if and
wherever such third-party notices normally appear. The contents
of the NOTICE file are for informational purposes only and
do not modify the License. You may add Your own attribution
notices within Derivative Works that You distribute, alongside
or as an addendum to the NOTICE text from the Work, provided
that such additional attribution notices cannot be construed
as modifying the License.
You may add Your own copyright statement to Your modifications and
may provide additional or different license terms and conditions
for use, reproduction, or distribution of Your modifications, or
for any such Derivative Works as a whole, provided Your use,
reproduction, and distribution of the Work otherwise complies with
the conditions stated in this License.
5. Submission of Contributions. Unless You explicitly state otherwise,
any Contribution intentionally submitted for inclusion in the Work
by You to the Licensor shall be under the terms and conditions of
this License, without any additional terms or conditions.
Notwithstanding the above, nothing herein shall supersede or modify
the terms of any separate license agreement you may have executed
with Licensor regarding such Contributions.
6. Trademarks. This License does not grant permission to use the trade
names, trademarks, service marks, or product names of the Licensor,
except as required for reasonable and customary use in describing the
origin of the Work and reproducing the content of the NOTICE file.
7. Disclaimer of Warranty. Unless required by applicable law or
agreed to in writing, Licensor provides the Work (and each
Contributor provides its Contributions) on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
implied, including, without limitation, any warranties or conditions
of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
PARTICULAR PURPOSE. You are solely responsible for determining the
appropriateness of using or redistributing the Work and assume any
risks associated with Your exercise of permissions under this License.
8. Limitation of Liability. In no event and under no legal theory,
whether in tort (including negligence), contract, or otherwise,
unless required by applicable law (such as deliberate and grossly
negligent acts) or agreed to in writing, shall any Contributor be
liable to You for damages, including any direct, indirect, special,
incidental, or consequential damages of any character arising as a
result of this License or out of the use or inability to use the
Work (including but not limited to damages for loss of goodwill,
work stoppage, computer failure or malfunction, or any and all
other commercial damages or losses), even if such Contributor
has been advised of the possibility of such damages.
9. Accepting Warranty or Additional Liability. While redistributing
the Work or Derivative Works thereof, You may choose to offer,
and charge a fee for, acceptance of support, warranty, indemnity,
or other liability obligations and/or rights consistent with this
License. However, in accepting such obligations, You may act only
on Your own behalf and on Your sole responsibility, not on behalf
of any other Contributor, and only if You agree to indemnify,
defend, and hold each Contributor harmless for any liability
incurred by, or claims asserted against, such Contributor by reason
of your accepting any such warranty or additional liability.
END OF TERMS AND CONDITIONS
APPENDIX: How to apply the Apache License to your work.
To apply the Apache License to your work, attach the following
boilerplate notice, with the fields enclosed by brackets "[]"
replaced with your own identifying information. (Don't include
the brackets!) The text should be enclosed in the appropriate
comment syntax for the file format. We also recommend that a
file or class name and description of purpose be included on the
same "printed page" as the copyright notice for easier
identification within third-party archives.
Copyright 2020 - present, Facebook, Inc
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
================================================
FILE: batchformer-v2/deit_share/README.md
================================================
#BatchFormerV2
The hyperparameters
--add_global 2 --insert_idx 8 --bt_lr 0.5 # for larger model, maybe, it requires to avoid overfitting. e.g. a smaller lr for bt
--add_global 2 --start_idx 0 # CIFAR
================================================
FILE: batchformer-v2/deit_share/README_cait.md
================================================
# CaiT: Going deeper with Image Transformers
This repository contains PyTorch evaluation code, training code and pretrained models for DeiT (Data-Efficient Image Transformers) and CaiT (Going deeper with Image Transformers). All models are trained during 400 epochs.
CaiT obtain competitive tradeoffs in terms of flops / precision:
For details see [Going deeper with Image Transformers](https://arxiv.org/abs/2103.17239) by Hugo Touvron, Matthieu Cord, Alexandre Sablayrolles, Gabriel Synnaeve and Hervé Jégou
If you use this code for a paper please cite:
```
@article{touvron2021cait,
title={Going deeper with Image Transformers},
author={Hugo Touvron and Matthieu Cord and Alexandre Sablayrolles and Gabriel Synnaeve and Herv\'e J\'egou},
journal={arXiv preprint arXiv:2103.17239},
year={2021}
}
```
# Model Zoo
We provide baseline CaiT models pretrained on ImageNet1k 2012 only, using the distilled version of our method.
| name | acc@1 | res | FLOPs| #params | url |
| --- | --- | --- | --- | --- | --- |
| S24 | 83.5 | 224 |9.4B| 47M| [model](https://dl.fbaipublicfiles.com/deit/S24_224.pth) |
| XS24| 84.1 | 384 | 19.3B |27M | [model](https://dl.fbaipublicfiles.com/deit/XS24_384.pth) |
| S24 | 85.1 | 384 | 32.2B |47M | [model](https://dl.fbaipublicfiles.com/deit/S24_384.pth) |
| S36 | 85.4 | 384 | 48.0B| 68M| [model](https://dl.fbaipublicfiles.com/deit/S36_384.pth) |
| M36 | 86.1 | 384 | 173.3B| 271M | [model](https://dl.fbaipublicfiles.com/deit/M36_384.pth) |
| M48 | 86.5 | 448 | 329.6B| 356M | [model](https://dl.fbaipublicfiles.com/deit/M48_448.pth) |
The models are also available via torch hub.
Before using it, make sure you have the pytorch-image-models package [`timm==0.3.2`](https://github.com/rwightman/pytorch-image-models) by [Ross Wightman](https://github.com/rwightman) installed.
# Evaluation transforms
CaiT employs a slightly different pre-processing, in particular a crop-ratio of 1.0 at test time. To reproduce the results of our paper please use the following pre-processing:
```
def get_test_transforms(input_size):
mean, std = [0.485, 0.456, 0.406],[0.229, 0.224, 0.225]
transformations = {}
transformations= transforms.Compose(
[transforms.Resize(input_size, interpolation=3),
transforms.CenterCrop(input_size),
transforms.ToTensor(),
transforms.Normalize(mean, std)])
return transformations
```
Remark: for CaiT M48 it is best to evaluate with FP32 precision
# License
This repository is released under the Apache 2.0 license as found in the [LICENSE](LICENSE) file.
# Contributing
We actively welcome your pull requests! Please see [CONTRIBUTING.md](.github/CONTRIBUTING.md) and [CODE_OF_CONDUCT.md](.github/CODE_OF_CONDUCT.md) for more info.
================================================
FILE: batchformer-v2/deit_share/README_resmlp.md
================================================
# ResMLP: Feedforward networks for image classification with data-efficient training
This repository contains PyTorch evaluation code, training code and pretrained models for the following projects:
* [DeiT](README.md) (Data-Efficient Image Transformers)
* [CaiT](README_cait.md) (Going deeper with Image Transformers)
* [ResMLP](README_resmlp.md) (ResMLP: Feedforward networks for image classification with data-efficient training)
ResMLP obtain good performance given its simplicity:
For details see [ResMLP: Feedforward networks for image classification with data-efficient training](https://arxiv.org/abs/2105.03404) by Hugo Touvron, Piotr Bojanowski, Mathilde Caron, Matthieu Cord, Alaaeldin El-Nouby, Edouard Grave, Armand Joulin, Gabriel Synnaeve, Jakob Verbeek and Hervé Jégou.
If you use this code for a paper please cite:
```
@article{touvron2021resmlp,
title={ResMLP: Feedforward networks for image classification with data-efficient training},
author={Hugo Touvron and Piotr Bojanowski and Mathilde Caron and M. Cord and Alaaeldin El-Nouby and Edouard Grave and Armand Joulin and Gabriel Synnaeve and Jakob Verbeek and Herv'e J'egou},
journal={arXiv preprint arXiv:2105.03404},
year={2021},
}
```
# Model Zoo
We provide baseline ResMLP models pretrained on ImageNet1k 2012, using the distilled version of our method:
| name | acc@1 | res | FLOPs| #params | url |
| --- | --- | --- | --- | --- | --- |
| ResMLP-S12 | 77.8 | 224 |3B| 15M| [model](https://dl.fbaipublicfiles.com/deit/resmlp_12_dist.pth) |
| ResMLP-S24| 80.8 | 224 | 6B |30M | [model](https://dl.fbaipublicfiles.com/deit/resmlp_24_dist.pth) |
| ResMLP-S36 | 81.1 | 224 | 23B |116M | [model](https://dl.fbaipublicfiles.com/deit/resmlp_36_dist.pth) |
| ResMLP-B24 |83.6 | 224 | 100B |129M | [model](https://dl.fbaipublicfiles.com/deit/resmlpB_24_dist.pth) |
Model pretrained on ImageNet-22k with finetuning on ImageNet1k 2012:
| name | acc@1 | res | FLOPs| #params | url |
| --- | --- | --- | --- | --- | --- |
| ResMLP-B24 |84.4 | 224 | 100B |129M | [model](https://dl.fbaipublicfiles.com/deit/resmlpB_24_22k.pth) |
Models pretrained with DINO without finetuning:
| name | acc@1 (knn)| res | FLOPs| #params | url |
| --- | --- | --- | --- | --- | --- |
| ResMLP-S12 | 62.6 | 224 |3B| 15M| [model](https://dl.fbaipublicfiles.com/deit/resmlp_12_dino.pth) |
| ResMLP-S24| 69.4 | 224 | 6B |30M | [model](https://dl.fbaipublicfiles.com/deit/resmlp_24_dino.pth) |
The models are also available via torch hub.
Before using it, make sure you have the pytorch-image-models package [`timm==0.3.2`](https://github.com/rwightman/pytorch-image-models) by [Ross Wightman](https://github.com/rwightman) installed.
# Evaluation transforms
ResMLP employs a slightly different pre-processing, in particular a crop-ratio of 0.9 at test time. To reproduce the results of our paper please use the following pre-processing:
```
def get_test_transforms(input_size):
mean, std = [0.485, 0.456, 0.406],[0.229, 0.224, 0.225]
transformations = {}
Rs_size=int(input_size/0.9)
transformations= transforms.Compose(
[transforms.Resize(Rs_size, interpolation=3),
transforms.CenterCrop(input_size),
transforms.ToTensor(),
transforms.Normalize(mean, std)])
return transformations
```
# License
This repository is released under the Apache 2.0 license as found in the [LICENSE](LICENSE) file.
# Contributing
We actively welcome your pull requests! Please see [CONTRIBUTING.md](.github/CONTRIBUTING.md) and [CODE_OF_CONDUCT.md](.github/CODE_OF_CONDUCT.md) for more info.
================================================
FILE: batchformer-v2/deit_share/bt.py
================================================
import torch
from timm.models.vision_transformer import Attention, Block
class BlockBF(torch.nn.Module):
def __init__(self, block: Block, fp16_type=0):
super().__init__()
self.block = block
self.fp16_type = fp16_type
def forward(self, x):
from torch.cuda.amp import autocast
if self.fp16_type == 2:
with autocast(enabled=False):
old_x = self.block.norm1(x)
x = x + torch.transpose(self.block.drop_path(torch.transpose(self.block.attn(old_x.float()), 1, 0)), 1, 0)
old_x = self.block.norm2(x)
x = x + torch.transpose(self.block.drop_path(torch.transpose(self.block.mlp(old_x), 1, 0)), 1, 0)
elif self.fp16_type == 0:
x = x + torch.transpose(self.block.drop_path(torch.transpose(self.block.attn(self.block.norm1(x)), 1, 0)), 1, 0)
x = x + torch.transpose(self.block.drop_path(torch.transpose(self.block.mlp(self.block.norm2(x)), 1, 0)), 1, 0)
else:
old_x = self.block.norm1(x)
with autocast(enabled=False):
x = x + torch.transpose(self.block.drop_path(torch.transpose(self.block.attn(old_x.float()), 1, 0)), 1, 0)
x = x + torch.transpose(self.block.drop_path(torch.transpose(self.block.mlp(self.block.norm2(x)), 1, 0)), 1, 0)
return x
class BlockWrap32(torch.nn.Module):
def __init__(self, block: Block, fp16_type: int):
super().__init__()
self.block = block
self.fp16_type = fp16_type
def forward(self, x):
from torch.cuda.amp import autocast
if self.fp16_type == 2:
with autocast(enabled=False):
old_x = self.block.norm1(x)
x = x + self.block.drop_path(self.block.attn(old_x.float()))
old_x = self.block.norm2(x)
x = x + self.block.drop_path(self.block.mlp(old_x))
else:
old_x = self.block.norm1(x)
with autocast(enabled=False):
x = x + self.block.drop_path(self.block.attn(old_x.float()))
x = x + self.block.drop_path(self.block.mlp(self.block.norm2(x)))
return x
class BlockWrapDebug(torch.nn.Module):
def __init__(self, block: Block):
super().__init__()
self.block = block
def forward(self, x):
tmp_x = self.block.drop_path(self.block.attn(self.block.norm1(x)))
for n,p in self.block.named_parameters():
if n.__contains__('attn.qkv.weight'):
print(n, p.detach().cpu().numpy(), p.grad)
x = x + tmp_x
tmp_x = self.block.drop_path(self.block.mlp(self.block.norm2(x)))
x = x + tmp_x
return x
class AttentionOnly(torch.nn.Module):
def __init__(self, block: Block, drop_path = 0. , add_bt=70):
super().__init__()
self.block = block
from timm.models.layers import DropPath
self.add_bt=add_bt
self.drop_path = DropPath(drop_path) if drop_path > 0. else torch.nn.Identity()
self.block.mlp = None
self.block.norm2 = None
def forward(self, x):
if self.add_bt==70:
x = x + torch.transpose(self.drop_path(torch.transpose(self.block.attn(self.block.norm1(x)), 0, 1)), 0, 1)
else:
x = x + self.drop_path(self.block.attn(self.block.norm1(x)))
# x = x + self.block.drop_path(self.block.mlp(self.block.norm2(x)))
return x
class MLPDecorder(torch.nn.Module):
def __init__(self, dim, mlp_dim, skip_mlp=False):
super().__init__()
self.skip_mlp = skip_mlp
self.mlp = torch.nn.Sequential(torch.nn.Linear(dim, mlp_dim), torch.nn.BatchNorm1d(mlp_dim), torch.nn.ReLU(),
torch.nn.Linear(mlp_dim, dim), torch.nn.BatchNorm1d(dim), )
def forward(self, x):
if self.training and not self.skip_mlp :
bt_x = x[len(x)//2:]
old_x = x[:len(x)//2]
bt_x = self.mlp(bt_x)
x = torch.cat([old_x, bt_x], dim=0)
return x
class MLPEncoder(torch.nn.Module):
def __init__(self, d_model, batch_size, dim_feedforward=2048, dropout=0.1, activation="relu"):
super(MLPEncoder, self).__init__()
# self.self_attn = MultiheadAttention(d_model, nhead, dropout=dropout)
# Implementation of Feedforward model
self.linear1 = torch.nn.Linear(batch_size, batch_size)
self.norm1 = torch.nn.LayerNorm(d_model)
self.dropout1 = torch.nn.Dropout(dropout)
import torch.nn.functional as F
self.activation = F.relu
def __setstate__(self, state):
if 'activation' not in state:
import torch.nn.functional as F
state['activation'] = F.relu
super(MLPEncoder, self).__setstate__(state)
def forward(self, src):
r"""Pass the input through the encoder layer.
Args:
src: the sequence to the encoder layer (required).
src_mask: the mask for the src sequence (optional).
src_key_padding_mask: the mask for the src keys per batch (optional).
Shape:
see the docs in Transformer class.
"""
src2 = torch.transpose(src, 0, 2)
src2 = self.linear1(src2)
src2 = torch.transpose(src2, 0, 2)
src = src + self.dropout1(src2)
src = self.norm1(src)
return src
class TransformerDecorator1(torch.nn.Module):
def __init__(self, add_bt=3, dim=2048, eval_global=0, args=None, first_layer=False, drop_path=0.1):
super(TransformerDecorator1, self).__init__()
self.not_cls_token = False
self.no_fp16_bt = 0
self.shuffle_patch=False
self.cls_token_only=False
heads = 4
self.empty_bt = 0
self.mlp_enc = 0
self.batch_size = 128
self.skip_bt = False
if args is not None:
self.not_cls_token = args.not_cls_token
self.cls_token_only = args.cls_token_only
self.no_fp16_bt = args.no_fp16_bt
self.mlp_enc = args.mlp_enc
heads = args.num_heads
self.shuffle_patch = args.shuffle_patch
self.empty_bt = args.empty_bt
self.batch_size = args.batch_size
self.all_patches = args.all_patches
self.skip_bt = args.skip_bt
self.atten_weights = None
self.encoder_layers = torch.nn.TransformerEncoderLayer(dim, heads, dim, args.bt_atten_drop)
self.eval_global = eval_global
self.add_bt = add_bt
self.first_layer = first_layer
if self.mlp_enc == 1:
self.encoder_layers = MLPEncoder(dim, args.batch_size)
if self.empty_bt:
self.encoder_layers = torch.nn.Identity()
self.idx = 0
self.noise = None
self.layer_num = 1
if self.first_layer:
self.add_bt = 1
def forward(self, feature):
if self.training and self.add_bt > 0 and not self.skip_bt:
old_feature = feature
if self.add_bt in [1, 2, 3] and not self.first_layer:
# split
old_feature = feature[:len(feature)//2]
feature = feature[-len(feature)//2:]
if self.shuffle_patch: # We do not use this
B, L, C = feature.shape
idx_orig_list = []
idx_shuffle_list = []
for i in range(len(feature)):
idx = torch.arange(0, L).type(torch.LongTensor)
idx_orig = torch.zeros(L).type(torch.LongTensor)
idx_shuffle = torch.cat([torch.randperm(1).type(torch.LongTensor),torch.randperm(L -1 ).type(torch.LongTensor)], 0)
idx_orig[idx_shuffle] = idx + i*L
idx_shuffle_list.append(idx_shuffle+ i * L)
# new_features.append(feature[i][idx_shuffle])
idx_orig_list.append(idx_orig)
idx_shuffle = torch.stack(idx_shuffle_list, 0).reshape(-1).to(feature.device)
idx_orig_list = torch.stack(idx_orig_list, 0).reshape(-1).to(feature.device)
feature = feature.view(B*L, C)[idx_shuffle].view(B, L, C)
pass
size = feature.shape
if len(size) == 3: #vit
if self.all_patches:
stride = 4
# N L C
size = feature.shape
feature = torch.transpose(feature, 1, 0) # L N C
feature = torch.reshape(feature, (stride * size[1], size[0] // stride, size[2]))
if isinstance(self.encoder_layers, Block) or \
isinstance(self.encoder_layers, Attention):
feature = feature.transpose(0, 1)
elif self.not_cls_token: # do not use in the paper
feature1 = self.encoder_layers(feature[:, 1:, :])
feature = torch.cat([feature[:, :1, :], feature1], dim=1)
elif self.cls_token_only: # do not use in the paper
feature1 = self.encoder_layers(feature[:, :1, :])
feature = torch.cat([feature1, feature[:, 1:, :]], dim=1)
elif self.no_fp16_bt:
from torch.cuda.amp import autocast
with autocast(enabled=False):
feature = self.encoder_layers(feature.float())
else:
feature = self.encoder_layers(feature)
if isinstance(self.encoder_layers, Block) or \
isinstance(self.encoder_layers, Attention): # The dimension is different from torch.nn.Transformer
feature = feature.transpose(0, 1)
if self.all_patches:
# recover
feature = torch.reshape(feature, (size[1], size[0], size[2]))
feature = torch.transpose(feature, 1, 0)
else:
feature = feature.view(feature.size(0), feature.size(1), -1)
feature = torch.transpose(feature, 2, 1)
feature = self.encoder_layers(feature)
feature = torch.transpose(feature, 2, 1)
feature = feature.view(size)
if self.shuffle_patch:
feature = feature.view(B*L, C)[idx_orig_list].view(B, L, C)
if self.add_bt not in [1]:
feature = torch.cat([old_feature, feature], dim=0)
# print(self.noise, self.idx)
# print(feature.norm())
return feature
elif self.add_bt and self.eval_global:
if isinstance(self.encoder_layers, Block) or \
isinstance(self.encoder_layers, Attention):
feature = feature.transpose(0, 1)
if self.add_bt and self.eval_global == 2:
size = feature.shape
old_feature = feature
feature = torch.reshape(feature, (size[0] * size[1], 1, size[2]))
feature = self.encoder_layers(feature)
feature = torch.reshape(feature, (size[0], size[1], size[2])) # Acc@1 80.152 Acc@5 95.160 loss 0.849
# feature = torch.cat([old_feature, feature], dim=0)
elif self.add_bt and self.eval_global == 1:
feature = self.encoder_layers(feature)
elif self.add_bt and self.eval_global == 3:
old_feature = feature
feature = self.encoder_layers(feature)
# feature = old_feature + feature # 79.628
feature = torch.cat([old_feature, feature], dim=0)
if isinstance(self.encoder_layers, Block) or \
isinstance(self.encoder_layers, Attention):
feature = feature.transpose(0, 1)
return feature
================================================
FILE: batchformer-v2/deit_share/cait_models.py
================================================
# Copyright (c) 2015-present, Facebook, Inc.
# All rights reserved.
import torch
import torch.nn as nn
from functools import partial
from timm.models.vision_transformer import Mlp, PatchEmbed , _cfg
from timm.models.registry import register_model
from timm.models.layers import trunc_normal_, DropPath
__all__ = [
'cait_M48', 'cait_M36', 'cait_M4',
'cait_S36', 'cait_S24','cait_S24_224',
'cait_XS24','cait_XXS24','cait_XXS24_224',
'cait_XXS36','cait_XXS36_224'
]
class Class_Attention(nn.Module):
# taken from https://github.com/rwightman/pytorch-image-models/blob/master/timm/models/vision_transformer.py
# with slight modifications to do CA
def __init__(self, dim, num_heads=8, qkv_bias=False, qk_scale=None, attn_drop=0., proj_drop=0.):
super().__init__()
self.num_heads = num_heads
head_dim = dim // num_heads
self.scale = qk_scale or head_dim ** -0.5
self.q = nn.Linear(dim, dim, bias=qkv_bias)
self.k = nn.Linear(dim, dim, bias=qkv_bias)
self.v = nn.Linear(dim, dim, bias=qkv_bias)
self.attn_drop = nn.Dropout(attn_drop)
self.proj = nn.Linear(dim, dim)
self.proj_drop = nn.Dropout(proj_drop)
def forward(self, x ):
B, N, C = x.shape
q = self.q(x[:,0]).unsqueeze(1).reshape(B, 1, self.num_heads, C // self.num_heads).permute(0, 2, 1, 3)
k = self.k(x).reshape(B, N, self.num_heads, C // self.num_heads).permute(0, 2, 1, 3)
q = q * self.scale
v = self.v(x).reshape(B, N, self.num_heads, C // self.num_heads).permute(0, 2, 1, 3)
attn = (q @ k.transpose(-2, -1))
attn = attn.softmax(dim=-1)
attn = self.attn_drop(attn)
x_cls = (attn @ v).transpose(1, 2).reshape(B, 1, C)
x_cls = self.proj(x_cls)
x_cls = self.proj_drop(x_cls)
return x_cls
class LayerScale_Block_CA(nn.Module):
# taken from https://github.com/rwightman/pytorch-image-models/blob/master/timm/models/vision_transformer.py
# with slight modifications to add CA and LayerScale
def __init__(self, dim, num_heads, mlp_ratio=4., qkv_bias=False, qk_scale=None, drop=0., attn_drop=0.,
drop_path=0., act_layer=nn.GELU, norm_layer=nn.LayerNorm, Attention_block = Class_Attention,
Mlp_block=Mlp,init_values=1e-4):
super().__init__()
self.norm1 = norm_layer(dim)
self.attn = Attention_block(
dim, num_heads=num_heads, qkv_bias=qkv_bias, qk_scale=qk_scale, attn_drop=attn_drop, proj_drop=drop)
self.drop_path = DropPath(drop_path) if drop_path > 0. else nn.Identity()
self.norm2 = norm_layer(dim)
mlp_hidden_dim = int(dim * mlp_ratio)
self.mlp = Mlp_block(in_features=dim, hidden_features=mlp_hidden_dim, act_layer=act_layer, drop=drop)
self.gamma_1 = nn.Parameter(init_values * torch.ones((dim)),requires_grad=True)
self.gamma_2 = nn.Parameter(init_values * torch.ones((dim)),requires_grad=True)
def forward(self, x, x_cls):
u = torch.cat((x_cls,x),dim=1)
x_cls = x_cls + self.drop_path(self.gamma_1 * self.attn(self.norm1(u)))
x_cls = x_cls + self.drop_path(self.gamma_2 * self.mlp(self.norm2(x_cls)))
return x_cls
class Attention_talking_head(nn.Module):
# taken from https://github.com/rwightman/pytorch-image-models/blob/master/timm/models/vision_transformer.py
# with slight modifications to add Talking Heads Attention (https://arxiv.org/pdf/2003.02436v1.pdf)
def __init__(self, dim, num_heads=8, qkv_bias=False, qk_scale=None, attn_drop=0., proj_drop=0.):
super().__init__()
self.num_heads = num_heads
head_dim = dim // num_heads
self.scale = qk_scale or head_dim ** -0.5
self.qkv = nn.Linear(dim, dim * 3, bias=qkv_bias)
self.attn_drop = nn.Dropout(attn_drop)
self.proj = nn.Linear(dim, dim)
self.proj_l = nn.Linear(num_heads, num_heads)
self.proj_w = nn.Linear(num_heads, num_heads)
self.proj_drop = nn.Dropout(proj_drop)
def forward(self, x):
B, N, C = x.shape
qkv = self.qkv(x).reshape(B, N, 3, self.num_heads, C // self.num_heads).permute(2, 0, 3, 1, 4)
q, k, v = qkv[0] * self.scale , qkv[1], qkv[2]
attn = (q @ k.transpose(-2, -1))
attn = self.proj_l(attn.permute(0,2,3,1)).permute(0,3,1,2)
attn = attn.softmax(dim=-1)
attn = self.proj_w(attn.permute(0,2,3,1)).permute(0,3,1,2)
attn = self.attn_drop(attn)
x = (attn @ v).transpose(1, 2).reshape(B, N, C)
x = self.proj(x)
x = self.proj_drop(x)
return x
class LayerScale_Block(nn.Module):
# taken from https://github.com/rwightman/pytorch-image-models/blob/master/timm/models/vision_transformer.py
# with slight modifications to add layerScale
def __init__(self, dim, num_heads, mlp_ratio=4., qkv_bias=False, qk_scale=None, drop=0., attn_drop=0.,
drop_path=0., act_layer=nn.GELU, norm_layer=nn.LayerNorm,Attention_block = Attention_talking_head,
Mlp_block=Mlp,init_values=1e-4):
super().__init__()
self.norm1 = norm_layer(dim)
self.attn = Attention_block(
dim, num_heads=num_heads, qkv_bias=qkv_bias, qk_scale=qk_scale, attn_drop=attn_drop, proj_drop=drop)
self.drop_path = DropPath(drop_path) if drop_path > 0. else nn.Identity()
self.norm2 = norm_layer(dim)
mlp_hidden_dim = int(dim * mlp_ratio)
self.mlp = Mlp_block(in_features=dim, hidden_features=mlp_hidden_dim, act_layer=act_layer, drop=drop)
self.gamma_1 = nn.Parameter(init_values * torch.ones((dim)),requires_grad=True)
self.gamma_2 = nn.Parameter(init_values * torch.ones((dim)),requires_grad=True)
def forward(self, x):
x = x + self.drop_path(self.gamma_1 * self.attn(self.norm1(x)))
x = x + self.drop_path(self.gamma_2 * self.mlp(self.norm2(x)))
return x
class cait_models(nn.Module):
# taken from https://github.com/rwightman/pytorch-image-models/blob/master/timm/models/vision_transformer.py
# with slight modifications to adapt to our cait models
def __init__(self, img_size=224, patch_size=16, in_chans=3, num_classes=1000, embed_dim=768, depth=12,
num_heads=12, mlp_ratio=4., qkv_bias=False, qk_scale=None, drop_rate=0., attn_drop_rate=0.,
drop_path_rate=0., norm_layer=nn.LayerNorm, global_pool=None,
block_layers = LayerScale_Block,
block_layers_token = LayerScale_Block_CA,
Patch_layer=PatchEmbed,act_layer=nn.GELU,
Attention_block = Attention_talking_head,Mlp_block=Mlp,
init_scale=1e-4,
Attention_block_token_only=Class_Attention,
Mlp_block_token_only= Mlp,
depth_token_only=2,
mlp_ratio_clstk = 4.0):
super().__init__()
self.num_classes = num_classes
self.num_features = self.embed_dim = embed_dim
self.patch_embed = Patch_layer(
img_size=img_size, patch_size=patch_size, in_chans=in_chans, embed_dim=embed_dim)
num_patches = self.patch_embed.num_patches
self.cls_token = nn.Parameter(torch.zeros(1, 1, embed_dim))
self.pos_embed = nn.Parameter(torch.zeros(1, num_patches, embed_dim))
self.pos_drop = nn.Dropout(p=drop_rate)
dpr = [drop_path_rate for i in range(depth)]
self.blocks = nn.ModuleList([
block_layers(
dim=embed_dim, num_heads=num_heads, mlp_ratio=mlp_ratio, qkv_bias=qkv_bias, qk_scale=qk_scale,
drop=drop_rate, attn_drop=attn_drop_rate, drop_path=dpr[i], norm_layer=norm_layer,
act_layer=act_layer,Attention_block=Attention_block,Mlp_block=Mlp_block,init_values=init_scale)
for i in range(depth)])
self.blocks_token_only = nn.ModuleList([
block_layers_token(
dim=embed_dim, num_heads=num_heads, mlp_ratio=mlp_ratio_clstk, qkv_bias=qkv_bias, qk_scale=qk_scale,
drop=0.0, attn_drop=0.0, drop_path=0.0, norm_layer=norm_layer,
act_layer=act_layer,Attention_block=Attention_block_token_only,
Mlp_block=Mlp_block_token_only,init_values=init_scale)
for i in range(depth_token_only)])
self.norm = norm_layer(embed_dim)
self.feature_info = [dict(num_chs=embed_dim, reduction=0, module='head')]
self.head = nn.Linear(embed_dim, num_classes) if num_classes > 0 else nn.Identity()
trunc_normal_(self.pos_embed, std=.02)
trunc_normal_(self.cls_token, std=.02)
self.apply(self._init_weights)
def _init_weights(self, m):
if isinstance(m, nn.Linear):
trunc_normal_(m.weight, std=.02)
if isinstance(m, nn.Linear) and m.bias is not None:
nn.init.constant_(m.bias, 0)
elif isinstance(m, nn.LayerNorm):
nn.init.constant_(m.bias, 0)
nn.init.constant_(m.weight, 1.0)
@torch.jit.ignore
def no_weight_decay(self):
return {'pos_embed', 'cls_token'}
def forward_features(self, x):
B = x.shape[0]
x = self.patch_embed(x)
cls_tokens = self.cls_token.expand(B, -1, -1)
x = x + self.pos_embed
x = self.pos_drop(x)
for i , blk in enumerate(self.blocks):
x = blk(x)
for i , blk in enumerate(self.blocks_token_only):
cls_tokens = blk(x,cls_tokens)
x = torch.cat((cls_tokens, x), dim=1)
x = self.norm(x)
return x[:, 0]
def forward(self, x):
x = self.forward_features(x)
x = self.head(x)
return x
@register_model
def cait_XXS24_224(pretrained=False, **kwargs):
model = cait_models(
img_size= 224,patch_size=16, embed_dim=192, depth=24, num_heads=4, mlp_ratio=4, qkv_bias=True,
norm_layer=partial(nn.LayerNorm, eps=1e-6),
init_scale=1e-5,
depth_token_only=2,**kwargs)
model.default_cfg = _cfg()
if pretrained:
checkpoint = torch.hub.load_state_dict_from_url(
url="https://dl.fbaipublicfiles.com/deit/XXS24_224.pth",
map_location="cpu", check_hash=True
)
checkpoint_no_module = {}
for k in model.state_dict().keys():
checkpoint_no_module[k] = checkpoint["model"]['module.'+k]
model.load_state_dict(checkpoint_no_module)
return model
@register_model
def cait_XXS24(pretrained=False, **kwargs):
model = cait_models(
img_size= 384,patch_size=16, embed_dim=192, depth=24, num_heads=4, mlp_ratio=4, qkv_bias=True,
norm_layer=partial(nn.LayerNorm, eps=1e-6),
init_scale=1e-5,
depth_token_only=2,**kwargs)
model.default_cfg = _cfg()
if pretrained:
checkpoint = torch.hub.load_state_dict_from_url(
url="https://dl.fbaipublicfiles.com/deit/XXS24_384.pth",
map_location="cpu", check_hash=True
)
checkpoint_no_module = {}
for k in model.state_dict().keys():
checkpoint_no_module[k] = checkpoint["model"]['module.'+k]
model.load_state_dict(checkpoint_no_module)
return model
@register_model
def cait_XXS36_224(pretrained=False, **kwargs):
model = cait_models(
img_size= 224,patch_size=16, embed_dim=192, depth=36, num_heads=4, mlp_ratio=4, qkv_bias=True,
norm_layer=partial(nn.LayerNorm, eps=1e-6),
init_scale=1e-5,
depth_token_only=2,**kwargs)
model.default_cfg = _cfg()
if pretrained:
checkpoint = torch.hub.load_state_dict_from_url(
url="https://dl.fbaipublicfiles.com/deit/XXS36_224.pth",
map_location="cpu", check_hash=True
)
checkpoint_no_module = {}
for k in model.state_dict().keys():
checkpoint_no_module[k] = checkpoint["model"]['module.'+k]
model.load_state_dict(checkpoint_no_module)
return model
@register_model
def cait_XXS36(pretrained=False, **kwargs):
model = cait_models(
img_size= 384,patch_size=16, embed_dim=192, depth=36, num_heads=4, mlp_ratio=4, qkv_bias=True,
norm_layer=partial(nn.LayerNorm, eps=1e-6),
init_scale=1e-5,
depth_token_only=2,**kwargs)
model.default_cfg = _cfg()
if pretrained:
checkpoint = torch.hub.load_state_dict_from_url(
url="https://dl.fbaipublicfiles.com/deit/XXS36_384.pth",
map_location="cpu", check_hash=True
)
checkpoint_no_module = {}
for k in model.state_dict().keys():
checkpoint_no_module[k] = checkpoint["model"]['module.'+k]
model.load_state_dict(checkpoint_no_module)
return model
@register_model
def cait_XS24(pretrained=False, **kwargs):
model = cait_models(
img_size= 384,patch_size=16, embed_dim=288, depth=24, num_heads=6, mlp_ratio=4, qkv_bias=True,
norm_layer=partial(nn.LayerNorm, eps=1e-6),
init_scale=1e-5,
depth_token_only=2,**kwargs)
model.default_cfg = _cfg()
if pretrained:
checkpoint = torch.hub.load_state_dict_from_url(
url="https://dl.fbaipublicfiles.com/deit/XS24_384.pth",
map_location="cpu", check_hash=True
)
checkpoint_no_module = {}
for k in model.state_dict().keys():
checkpoint_no_module[k] = checkpoint["model"]['module.'+k]
model.load_state_dict(checkpoint_no_module)
return model
@register_model
def cait_S24_224(pretrained=False, **kwargs):
model = cait_models(
img_size= 224,patch_size=16, embed_dim=384, depth=24, num_heads=8, mlp_ratio=4, qkv_bias=True,
norm_layer=partial(nn.LayerNorm, eps=1e-6),
init_scale=1e-5,
depth_token_only=2,**kwargs)
model.default_cfg = _cfg()
if pretrained:
checkpoint = torch.hub.load_state_dict_from_url(
url="https://dl.fbaipublicfiles.com/deit/S24_224.pth",
map_location="cpu", check_hash=True
)
checkpoint_no_module = {}
for k in model.state_dict().keys():
checkpoint_no_module[k] = checkpoint["model"]['module.'+k]
model.load_state_dict(checkpoint_no_module)
return model
@register_model
def cait_S24(pretrained=False, **kwargs):
model = cait_models(
img_size= 384,patch_size=16, embed_dim=384, depth=24, num_heads=8, mlp_ratio=4, qkv_bias=True,
norm_layer=partial(nn.LayerNorm, eps=1e-6),
init_scale=1e-5,
depth_token_only=2,**kwargs)
model.default_cfg = _cfg()
if pretrained:
checkpoint = torch.hub.load_state_dict_from_url(
url="https://dl.fbaipublicfiles.com/deit/S24_384.pth",
map_location="cpu", check_hash=True
)
checkpoint_no_module = {}
for k in model.state_dict().keys():
checkpoint_no_module[k] = checkpoint["model"]['module.'+k]
model.load_state_dict(checkpoint_no_module)
return model
@register_model
def cait_S36(pretrained=False, **kwargs):
model = cait_models(
img_size= 384,patch_size=16, embed_dim=384, depth=36, num_heads=8, mlp_ratio=4, qkv_bias=True,
norm_layer=partial(nn.LayerNorm, eps=1e-6),
init_scale=1e-6,
depth_token_only=2,**kwargs)
model.default_cfg = _cfg()
if pretrained:
checkpoint = torch.hub.load_state_dict_from_url(
url="https://dl.fbaipublicfiles.com/deit/S36_384.pth",
map_location="cpu", check_hash=True
)
checkpoint_no_module = {}
for k in model.state_dict().keys():
checkpoint_no_module[k] = checkpoint["model"]['module.'+k]
model.load_state_dict(checkpoint_no_module)
return model
@register_model
def cait_M36(pretrained=False, **kwargs):
model = cait_models(
img_size= 384, patch_size=16, embed_dim=768, depth=36, num_heads=16, mlp_ratio=4, qkv_bias=True,
norm_layer=partial(nn.LayerNorm, eps=1e-6),
init_scale=1e-6,
depth_token_only=2,**kwargs)
model.default_cfg = _cfg()
if pretrained:
checkpoint = torch.hub.load_state_dict_from_url(
url="https://dl.fbaipublicfiles.com/deit/M36_384.pth",
map_location="cpu", check_hash=True
)
checkpoint_no_module = {}
for k in model.state_dict().keys():
checkpoint_no_module[k] = checkpoint["model"]['module.'+k]
model.load_state_dict(checkpoint_no_module)
return model
@register_model
def cait_M48(pretrained=False, **kwargs):
model = cait_models(
img_size= 448 , patch_size=16, embed_dim=768, depth=48, num_heads=16, mlp_ratio=4, qkv_bias=True,
norm_layer=partial(nn.LayerNorm, eps=1e-6),
init_scale=1e-6,
depth_token_only=2,**kwargs)
model.default_cfg = _cfg()
if pretrained:
checkpoint = torch.hub.load_state_dict_from_url(
url="https://dl.fbaipublicfiles.com/deit/M48_448.pth",
map_location="cpu", check_hash=True
)
checkpoint_no_module = {}
for k in model.state_dict().keys():
checkpoint_no_module[k] = checkpoint["model"]['module.'+k]
model.load_state_dict(checkpoint_no_module)
return model
================================================
FILE: batchformer-v2/deit_share/datasets.py
================================================
# Copyright (c) 2015-present, Facebook, Inc.
# All rights reserved.
import os
import json
from torchvision import datasets, transforms
from torchvision.datasets.folder import ImageFolder, default_loader
from timm.data.constants import IMAGENET_DEFAULT_MEAN, IMAGENET_DEFAULT_STD
from timm.data import create_transform
class INatDataset(ImageFolder):
def __init__(self, root, train=True, year=2018, transform=None, target_transform=None,
category='name', loader=default_loader):
self.transform = transform
self.loader = loader
self.target_transform = target_transform
self.year = year
# assert category in ['kingdom','phylum','class','order','supercategory','family','genus','name']
path_json = os.path.join(root, f'{"train" if train else "val"}{year}.json')
with open(path_json) as json_file:
data = json.load(json_file)
with open(os.path.join(root, 'categories.json')) as json_file:
data_catg = json.load(json_file)
path_json_for_targeter = os.path.join(root, f"train{year}.json")
with open(path_json_for_targeter) as json_file:
data_for_targeter = json.load(json_file)
targeter = {}
indexer = 0
for elem in data_for_targeter['annotations']:
king = []
king.append(data_catg[int(elem['category_id'])][category])
if king[0] not in targeter.keys():
targeter[king[0]] = indexer
indexer += 1
self.nb_classes = len(targeter)
self.samples = []
for elem in data['images']:
cut = elem['file_name'].split('/')
target_current = int(cut[2])
path_current = os.path.join(root, cut[0], cut[2], cut[3])
categors = data_catg[target_current]
target_current_true = targeter[categors[category]]
self.samples.append((path_current, target_current_true))
# __getitem__ and __len__ inherited from ImageFolder
def build_dataset(is_train, args):
transform = build_transform(is_train, args)
if args.data_set == 'CIFAR':
dataset = datasets.CIFAR100(args.data_path, train=is_train, transform=transform, download=True)
nb_classes = 100
elif args.data_set == 'IMNET':
root = os.path.join(args.data_path, 'train' if is_train else 'val')
dataset = datasets.ImageFolder(root, transform=transform)
nb_classes = len(dataset.classes)
elif args.data_set == 'INAT':
dataset = INatDataset(args.data_path, train=is_train, year=2018,
category=args.inat_category, transform=transform)
nb_classes = dataset.nb_classes
elif args.data_set == 'INAT19':
dataset = INatDataset(args.data_path, train=is_train, year=2019,
category=args.inat_category, transform=transform)
nb_classes = dataset.nb_classes
return dataset, nb_classes
def build_transform(is_train, args):
resize_im = args.input_size > 32
if is_train:
# this should always dispatch to transforms_imagenet_train
transform = create_transform(
input_size=args.input_size,
is_training=True,
color_jitter=args.color_jitter,
auto_augment=args.aa,
interpolation=args.train_interpolation,
re_prob=args.reprob,
re_mode=args.remode,
re_count=args.recount,
)
if not resize_im:
# replace RandomResizedCropAndInterpolation with
# RandomCrop
transform.transforms[0] = transforms.RandomCrop(
args.input_size, padding=4)
return transform
t = []
if resize_im:
size = int((256 / 224) * args.input_size)
t.append(
transforms.Resize(size, interpolation=3), # to maintain same ratio w.r.t. 224 images
)
t.append(transforms.CenterCrop(args.input_size))
t.append(transforms.ToTensor())
t.append(transforms.Normalize(IMAGENET_DEFAULT_MEAN, IMAGENET_DEFAULT_STD))
return transforms.Compose(t)
================================================
FILE: batchformer-v2/deit_share/engine.py
================================================
# Copyright (c) 2015-present, Facebook, Inc.
# All rights reserved.
"""
Train and eval functions used in main.py
"""
import math
import os
import sys
from typing import Iterable, Optional
import torch
from timm.data import Mixup
from timm.utils import accuracy, ModelEma
from losses import DistillationLoss
import utils
def train_one_epoch(model: torch.nn.Module, criterion: DistillationLoss,
data_loader: Iterable, optimizer: torch.optim.Optimizer,
device: torch.device, epoch: int, loss_scaler, max_norm: float = 0,
model_ema: Optional[ModelEma] = None, mixup_fn: Optional[Mixup] = None,
set_training_mode=True, no_fp16_bt=0,
model_without_ddp=None, num_classes=1000):
metric_logger = utils.MetricLogger(delimiter=" ")
metric_logger.add_meter('lr', utils.SmoothedValue(window_size=1, fmt='{value:.6f}'))
header = 'Epoch: [{}]'.format(epoch)
print_freq = 10
for samples, targets in metric_logger.log_every(data_loader, print_freq, header):
samples = samples.to(device, non_blocking=True)
targets = targets.to(device, non_blocking=True)
if mixup_fn is not None:
samples, targets = mixup_fn(samples, targets)
if no_fp16_bt in [4]:
# remove fp16
outputs = model(samples)
real_loss = criterion(samples[:len(targets)], outputs[:len(targets)], targets[:len(targets)])
while len(outputs) > len(targets):
targets = torch.cat([targets, targets], dim=0)
loss = criterion(samples, outputs, targets)
else:
with torch.cuda.amp.autocast():
outputs = model(samples)
real_loss = criterion(samples[:len(targets)], outputs[:len(targets)], targets[:len(targets)])
while len(outputs) > len(targets):
targets = torch.cat([targets, targets], dim=0)
loss = criterion(samples, outputs, targets)
loss_value = loss.item()
real_loss_value = real_loss.item()
if not math.isfinite(loss_value):
print(loss_value)
# open("/checkpoints/debug.log", 'a').write('loss nan').close()
print("Loss is {}, stopping training".format(loss_value))
# os.system("ps -ux | grep slurm | awk '{print $2}' | xargs kill")
# sys.exit(1)
optimizer.zero_grad()
continue
# continue
optimizer.zero_grad()
# this attribute is added by timm on one optimizer (adahessian)
is_second_order = hasattr(optimizer, 'is_second_order') and optimizer.is_second_order
if no_fp16_bt in [3, 4]:
loss.backward()
optimizer.step()
else:
loss_scaler(loss, optimizer, clip_grad=max_norm,
parameters=model.parameters(), create_graph=is_second_order)
torch.cuda.synchronize()
if model_ema is not None:
model_ema.update(model)
metric_logger.update(loss=loss_value)
metric_logger.update(rloss=real_loss_value)
metric_logger.update(lr=optimizer.param_groups[0]["lr"])
# gather the stats from all processes
metric_logger.synchronize_between_processes()
print("Averaged stats:", metric_logger)
return {k: meter.global_avg for k, meter in metric_logger.meters.items()}
@torch.no_grad()
def evaluate(data_loader, model, device):
criterion = torch.nn.CrossEntropyLoss()
metric_logger = utils.MetricLogger(delimiter=" ")
header = 'Test:'
# switch to evaluation mode
model.eval()
for images, target in metric_logger.log_every(data_loader, 10, header):
images = images.to(device, non_blocking=True)
target = target.to(device, non_blocking=True)
# compute output
with torch.cuda.amp.autocast():
output = model(images)
if len(output) > len(target):
output = (output[:len(target)] + output[len(target):])
# output = output[:len(target)]
# output = output[len(target):]
loss = criterion(output, target)
acc1, acc5 = accuracy(output, target, topk=(1, 5))
batch_size = images.shape[0]
metric_logger.update(loss=loss.item())
metric_logger.meters['acc1'].update(acc1.item(), n=batch_size)
metric_logger.meters['acc5'].update(acc5.item(), n=batch_size)
# gather the stats from all processes
metric_logger.synchronize_between_processes()
print('* Acc@1 {top1.global_avg:.3f} Acc@5 {top5.global_avg:.3f} loss {losses.global_avg:.3f}'
.format(top1=metric_logger.acc1, top5=metric_logger.acc5, losses=metric_logger.loss))
return {k: meter.global_avg for k, meter in metric_logger.meters.items()}
def reduce_tensor(acc1, args):
import torch.distributed as dist
rt = acc1.clone()
dist.all_reduce(rt, op=dist.ReduceOp.SUM)
rt = rt / args.world_size
return rt
def validate(val_loader, training_labels, model, device, num_classes, rank):
# switch to evaluate mode
model.eval()
total_logits = torch.empty((0, num_classes)).cuda()
total_labels = torch.empty(0, dtype=torch.long).cuda()
with torch.no_grad():
for i, (images, target) in enumerate(val_loader):
images = images.to(device, non_blocking=True)
target = target.to(device, non_blocking=True)
# compute output
output = model(images)
total_logits = torch.cat((total_logits, output))
total_labels = torch.cat((total_labels, target))
# measure accuracy and record loss
acc1, acc5 = accuracy(output, target, topk=(1, 5))
# if args.wor
import torch.nn.functional as F
probs, preds = F.softmax(total_logits.detach(), dim=1).max(dim=1)
if rank <= 0:
many_acc_top1, median_acc_top1, low_acc_top1, cls_accs = shot_acc(preds, total_labels, training_labels, acc_per_cls=True)
print('Many_acc: %.5f, Medium_acc: %.5f Low_acc: %.5f'%(many_acc_top1, median_acc_top1, low_acc_top1))
def shot_acc(preds, labels, train_data, many_shot_thr=100, low_shot_thr=20, acc_per_cls=False):
import numpy as np
if isinstance(train_data, np.ndarray):
training_labels = np.array(train_data).astype(int)
else:
training_labels = np.array(train_data.dataset.labels).astype(int)
if isinstance(preds, torch.Tensor):
preds = preds.detach().cpu().numpy()
labels = labels.detach().cpu().numpy()
elif isinstance(preds, np.ndarray):
pass
else:
raise TypeError('Type ({}) of preds not supported'.format(type(preds)))
train_class_count = []
test_class_count = []
class_correct = []
for l in np.unique(labels):
train_class_count.append(len(training_labels[training_labels == l]))
test_class_count.append(len(labels[labels == l]))
class_correct.append((preds[labels == l] == labels[labels == l]).sum())
many_shot = []
median_shot = []
low_shot = []
for i in range(len(train_class_count)):
if train_class_count[i] > many_shot_thr:
many_shot.append((class_correct[i] / test_class_count[i]))
elif train_class_count[i] < low_shot_thr:
low_shot.append((class_correct[i] / test_class_count[i]))
else:
median_shot.append((class_correct[i] / test_class_count[i]))
if len(many_shot) == 0:
many_shot.append(0)
if len(median_shot) == 0:
median_shot.append(0)
if len(low_shot) == 0:
low_shot.append(0)
if acc_per_cls:
class_accs = [c / cnt for c, cnt in zip(class_correct, test_class_count)]
return np.mean(many_shot), np.mean(median_shot), np.mean(low_shot), class_accs
else:
return np.mean(many_shot), np.mean(median_shot), np.mean(low_shot)
================================================
FILE: batchformer-v2/deit_share/gpu.pbs
================================================
#!/bin/bash
#PBS -P ZHIHOU
#PBS -l select=4:ngpus=1:ncpus=6:mem=24GB
#PBS -l walltime=72:00:00
#PBS -q alloc-dt
source activate pyt
#env
cd "$PBS_O_WORKDIR"
================================================
FILE: batchformer-v2/deit_share/hubconf.py
================================================
# Copyright (c) 2015-present, Facebook, Inc.
# All rights reserved.
from models import *
dependencies = ["torch", "torchvision", "timm"]
================================================
FILE: batchformer-v2/deit_share/losses.py
================================================
# Copyright (c) 2015-present, Facebook, Inc.
# All rights reserved.
"""
Implements the knowledge distillation loss
"""
import torch
from torch.nn import functional as F
import torch
from torch.nn.modules.loss import _Loss
import torch.nn.functional as F
import json
class DistillationLoss(torch.nn.Module):
"""
This module wraps a standard criterion and adds an extra knowledge distillation loss by
taking a teacher model prediction and using it as additional supervision.
"""
def __init__(self, base_criterion: torch.nn.Module, teacher_model: torch.nn.Module,
distillation_type: str, alpha: float, tau: float):
super().__init__()
self.base_criterion = base_criterion
self.teacher_model = teacher_model
assert distillation_type in ['none', 'soft', 'hard']
self.distillation_type = distillation_type
self.alpha = alpha
self.tau = tau
def forward(self, inputs, outputs, labels):
"""
Args:
inputs: The original inputs that are feed to the teacher model
outputs: the outputs of the model to be trained. It is expected to be
either a Tensor, or a Tuple[Tensor, Tensor], with the original output
in the first position and the distillation predictions as the second output
labels: the labels for the base criterion
"""
outputs_kd = None
if not isinstance(outputs, torch.Tensor):
# assume that the model outputs a tuple of [outputs, outputs_kd]
outputs, outputs_kd = outputs
base_loss = self.base_criterion(outputs, labels)
if self.distillation_type == 'none':
return base_loss
if outputs_kd is None:
raise ValueError("When knowledge distillation is enabled, the model is "
"expected to return a Tuple[Tensor, Tensor] with the output of the "
"class_token and the dist_token")
# don't backprop throught the teacher
with torch.no_grad():
teacher_outputs = self.teacher_model(inputs)
if self.distillation_type == 'soft':
T = self.tau
# taken from https://github.com/peterliht/knowledge-distillation-pytorch/blob/master/model/net.py#L100
# with slight modifications
distillation_loss = F.kl_div(
F.log_softmax(outputs_kd / T, dim=1),
#We provide the teacher's targets in log probability because we use log_target=True
#(as recommended in pytorch https://github.com/pytorch/pytorch/blob/9324181d0ac7b4f7949a574dbc3e8be30abe7041/torch/nn/functional.py#L2719)
#but it is possible to give just the probabilities and set log_target=False. In our experiments we tried both.
F.log_softmax(teacher_outputs / T, dim=1),
reduction='sum',
log_target=True
) * (T * T) / outputs_kd.numel()
#We divide by outputs_kd.numel() to have the legacy PyTorch behavior.
#But we also experiments output_kd.size(0)
#see issue 61(https://github.com/facebookresearch/deit/issues/61) for more details
elif self.distillation_type == 'hard':
distillation_loss = F.cross_entropy(outputs_kd, teacher_outputs.argmax(dim=1))
loss = base_loss * (1 - self.alpha) + distillation_loss * self.alpha
return loss
class BalancedSoftmax(_Loss):
"""
Balanced Softmax Loss
"""
def __init__(self, freq_path):
super(BalancedSoftmax, self).__init__()
with open(freq_path, 'r') as fd:
freq = json.load(fd)
freq = torch.tensor(freq)
self.sample_per_class = freq
def forward(self, inputs, outputs, labels, reduction='mean'):
return balanced_softmax_loss(labels, outputs, self.sample_per_class, reduction)
def balanced_softmax_loss(labels, logits, sample_per_class, reduction):
"""Compute the Balanced Softmax Loss between `logits` and the ground truth `labels`.
Args:
labels: A int tensor of size [batch].
logits: A float tensor of size [batch, no_of_classes].
sample_per_class: A int tensor of size [no of classes].
reduction: string. One of "none", "mean", "sum"
Returns:
loss: A float tensor. Balanced Softmax Loss.
"""
spc = sample_per_class.type_as(logits)
spc = spc.unsqueeze(0).expand(logits.shape[0], -1)
logits = logits + spc.log()
loss = F.binary_cross_entropy_with_logits(input=logits, target=labels, reduction=reduction)
return loss
def create_loss(freq_path):
print('Loading Balanced Softmax Loss.')
return BalancedSoftmax(freq_path)
================================================
FILE: batchformer-v2/deit_share/main.py
================================================
# Copyright (c) 2015-present, Facebook, Inc.
# All rights reserved.
import argparse
import datetime
import numpy as np
import time
import torch
import torch.backends.cudnn as cudnn
import json
from pathlib import Path
from timm.data import Mixup
from timm.models import create_model
from timm.loss import LabelSmoothingCrossEntropy, SoftTargetCrossEntropy
from timm.scheduler import create_scheduler
from timm.optim import create_optimizer
from timm.utils import NativeScaler, get_state_dict, ModelEma
from datasets import build_dataset
from engine import train_one_epoch, evaluate
from losses import DistillationLoss
from samplers import RASampler
import models
import utils
def get_args_parser():
parser = argparse.ArgumentParser('DeiT training and evaluation script', add_help=False)
parser.add_argument('--batch-size', default=64, type=int)
parser.add_argument('--epochs', default=300, type=int)
# Model parameters
parser.add_argument('--model', default='deit_small_patch16_224', type=str, metavar='MODEL',
help='Name of model to train')
parser.add_argument('--input-size', default=224, type=int, help='images input size')
parser.add_argument('--drop', type=float, default=0.0, metavar='PCT',
help='Dropout rate (default: 0.)')
parser.add_argument('--drop-path', type=float, default=0.1, metavar='PCT',
help='Drop path rate (default: 0.1)')
parser.add_argument('--model-ema', action='store_true')
parser.add_argument('--no-model-ema', action='store_false', dest='model_ema')
parser.set_defaults(model_ema=True)
parser.add_argument('--model-ema-decay', type=float, default=0.99996, help='')
parser.add_argument('--model-ema-force-cpu', action='store_true', default=False, help='')
# Optimizer parameters
parser.add_argument('--opt', default='adamw', type=str, metavar='OPTIMIZER',
help='Optimizer (default: "adamw"')
parser.add_argument('--opt-eps', default=1e-8, type=float, metavar='EPSILON',
help='Optimizer Epsilon (default: 1e-8)')
parser.add_argument('--opt-betas', default=None, type=float, nargs='+', metavar='BETA',
help='Optimizer Betas (default: None, use opt default)')
parser.add_argument('--clip-grad', type=float, default=None, metavar='NORM',
help='Clip gradient norm (default: None, no clipping)')
parser.add_argument('--momentum', type=float, default=0.9, metavar='M',
help='SGD momentum (default: 0.9)')
parser.add_argument('--weight-decay', type=float, default=0.05,
help='weight decay (default: 0.05)')
# Learning rate schedule parameters
parser.add_argument('--sched', default='cosine', type=str, metavar='SCHEDULER',
help='LR scheduler (default: "cosine"')
parser.add_argument('--lr', type=float, default=5e-4, metavar='LR',
help='learning rate (default: 5e-4)')
parser.add_argument('--lr-noise', type=float, nargs='+', default=None, metavar='pct, pct',
help='learning rate noise on/off epoch percentages')
parser.add_argument('--lr-noise-pct', type=float, default=0.67, metavar='PERCENT',
help='learning rate noise limit percent (default: 0.67)')
parser.add_argument('--lr-noise-std', type=float, default=1.0, metavar='STDDEV',
help='learning rate noise std-dev (default: 1.0)')
parser.add_argument('--warmup-lr', type=float, default=1e-6, metavar='LR',
help='warmup learning rate (default: 1e-6)')
parser.add_argument('--min-lr', type=float, default=1e-5, metavar='LR',
help='lower lr bound for cyclic schedulers that hit 0 (1e-5)')
parser.add_argument('--decay-epochs', type=float, default=30, metavar='N',
help='epoch interval to decay LR')
parser.add_argument('--warmup-epochs', type=int, default=5, metavar='N',
help='epochs to warmup LR, if scheduler supports')
parser.add_argument('--cooldown-epochs', type=int, default=10, metavar='N',
help='epochs to cooldown LR at min_lr, after cyclic schedule ends')
parser.add_argument('--patience-epochs', type=int, default=10, metavar='N',
help='patience epochs for Plateau LR scheduler (default: 10')
parser.add_argument('--decay-rate', '--dr', type=float, default=0.1, metavar='RATE',
help='LR decay rate (default: 0.1)')
# Augmentation parameters
parser.add_argument('--color-jitter', type=float, default=0.4, metavar='PCT',
help='Color jitter factor (default: 0.4)')
parser.add_argument('--aa', type=str, default='rand-m9-mstd0.5-inc1', metavar='NAME',
help='Use AutoAugment policy. "v0" or "original". " + \
"(default: rand-m9-mstd0.5-inc1)'),
parser.add_argument('--smoothing', type=float, default=0.1, help='Label smoothing (default: 0.1)')
parser.add_argument('--train-interpolation', type=str, default='bicubic',
help='Training interpolation (random, bilinear, bicubic default: "bicubic")')
parser.add_argument('--repeated-aug', action='store_true')
parser.add_argument('--no-repeated-aug', action='store_false', dest='repeated_aug')
parser.set_defaults(repeated_aug=True)
# * Random Erase params
parser.add_argument('--reprob', type=float, default=0.25, metavar='PCT',
help='Random erase prob (default: 0.25)')
parser.add_argument('--remode', type=str, default='pixel',
help='Random erase mode (default: "pixel")')
parser.add_argument('--recount', type=int, default=1,
help='Random erase count (default: 1)')
parser.add_argument('--resplit', action='store_true', default=False,
help='Do not random erase first (clean) augmentation split')
# * Mixup params
parser.add_argument('--mixup', type=float, default=0.8,
help='mixup alpha, mixup enabled if > 0. (default: 0.8)')
parser.add_argument('--cutmix', type=float, default=1.0,
help='cutmix alpha, cutmix enabled if > 0. (default: 1.0)')
parser.add_argument('--cutmix-minmax', type=float, nargs='+', default=None,
help='cutmix min/max ratio, overrides alpha and enables cutmix if set (default: None)')
parser.add_argument('--mixup-prob', type=float, default=1.0,
help='Probability of performing mixup or cutmix when either/both is enabled')
parser.add_argument('--mixup-switch-prob', type=float, default=0.5,
help='Probability of switching to cutmix when both mixup and cutmix enabled')
parser.add_argument('--mixup-mode', type=str, default='batch',
help='How to apply mixup/cutmix params. Per "batch", "pair", or "elem"')
# Distillation parameters
parser.add_argument('--teacher-model', default='regnety_160', type=str, metavar='MODEL',
help='Name of teacher model to train (default: "regnety_160"')
parser.add_argument('--teacher-path', type=str, default='')
parser.add_argument('--distillation-type', default='none', choices=['none', 'soft', 'hard'], type=str, help="")
parser.add_argument('--distillation-alpha', default=0.5, type=float, help="")
parser.add_argument('--distillation-tau', default=1.0, type=float, help="")
# * Finetuning params
parser.add_argument('--finetune', default='', help='finetune from checkpoint')
# Dataset parameters
parser.add_argument('--data-path', default='//project/ZHIHOUDATA/Code/RIDE-LongTailRecognition/data/ImageNet_LT/', type=str,
help='dataset path')
parser.add_argument('--data-set', default='IMNET', choices=['CIFAR', 'IMNET', 'INAT', 'INAT19'],
type=str, help='Image Net dataset path')
parser.add_argument('--inat-category', default='name',
choices=['kingdom', 'phylum', 'class', 'order', 'supercategory', 'family', 'genus', 'name'],
type=str, help='semantic granularity')
parser.add_argument('--output_dir', default='',
help='path where to save, empty for no saving')
parser.add_argument('--device', default='cuda',
help='device to use for training / testing')
parser.add_argument('--seed', default=0, type=int)
parser.add_argument('--resume', default='', help='resume from checkpoint')
parser.add_argument('--start_epoch', default=0, type=int, metavar='N',
help='start epoch')
parser.add_argument('--eval', action='store_true', help='Perform evaluation only')
parser.add_argument('--dist-eval', action='store_true', default=False, help='Enabling distributed evaluation')
parser.add_argument('--num_workers', default=10, type=int)
parser.add_argument('--pin-mem', action='store_true',
help='Pin CPU memory in DataLoader for more efficient (sometimes) transfer to GPU.')
parser.add_argument('--no-pin-mem', action='store_false', dest='pin_mem',
help='')
parser.set_defaults(pin_mem=True)
# distributed training parameters
parser.add_argument('--world_size', default=1, type=int,
help='number of distributed processes')
parser.add_argument('--add_bt', default=0, type=int,
help='add_bt')
parser.add_argument('--eval_global', default=0, type=int,
help='eval_global') # This is for evaluation with a mini-batch
parser.add_argument('--dropout', default=0., type=float,
help='dropout')
parser.add_argument('--exp_id', default="", type=str,
help='exp_id')
# For BT
parser.add_argument('--decay_drop_bt', default=0., type=float,
help='drop_patch')
parser.add_argument('--drop_patch', default=0., type=float,
help='drop_patch')
parser.add_argument('--shuffle_patch', action='store_true', )
parser.add_argument('--no_fp16_bt', default=0, type=int,
help='no_fp16_bt')
parser.add_argument('--num_heads', default=4, type=int,
help='num_heads')
parser.add_argument('--start_idx', default=8, type=int,
help='start_idx: start bt layer index')
parser.add_argument('--start_bt_epoch', default=0, type=int,
help='start_bt_epoch: start bt layer index')
parser.add_argument('--insert_idx', action='append', type=int,
help='insert idx list')
parser.add_argument('--all_patches', action='store_true',)
parser.add_argument('--drop_path_bt', default=0., type=float)
parser.add_argument('--not_cls_token', action='store_true'),
parser.add_argument('--cls_token_only', action='store_true'),
parser.add_argument('--shared_bt', default=1, type=int,),
parser.add_argument('--empty_bt', default=0, type=int,),
parser.add_argument('--add_norm_bt', default=0, type=int,), # This is to add BT in the last layer (normalization)
parser.add_argument('--add_mlp_bt', default=0, type=int,),
parser.add_argument('--mlp_enc', default=0, type=int,),
parser.add_argument('--no_grad_bt', default=0, type=int,),
parser.add_argument('--bt_decay', default=0., type=float,
help='drop_patch')
parser.add_argument('--mlp_decay', default=0., type=float,
help='drop_patch')
parser.add_argument('--bt_lr', default=0.5, type=float,)
parser.add_argument('--balanced_softmax', action='store_true',) # This is for the experiments on Long-Tailed Recognition
parser.add_argument('--bt_atten_drop', default=0.5, type=float)
parser.add_argument('--skip_bt', action='store_true',)
parser.add_argument('--pretrained', action='store_true',
help='pretrained')
parser.add_argument('--dist_url', default='env://', help='url used to set up distributed training')
return parser
def main(args):
# torch.autograd.set_detect_anomaly(True)
utils.init_distributed_mode(args)
print(args)
if args.distillation_type != 'none' and args.finetune and not args.eval:
raise NotImplementedError("Finetuning with distillation not yet supported")
device = torch.device(args.device)
# fix the seed for reproducibility
seed = args.seed + utils.get_rank()
torch.manual_seed(seed)
np.random.seed(seed)
# random.seed(seed)
cudnn.benchmark = True
dataset_train, args.nb_classes = build_dataset(is_train=True, args=args)
dataset_val, _ = build_dataset(is_train=False, args=args)
if True: # args.distributed:
num_tasks = utils.get_world_size()
global_rank = utils.get_rank()
if args.repeated_aug:
sampler_train = RASampler(
dataset_train, num_replicas=num_tasks, rank=global_rank, shuffle=True
)
else:
sampler_train = torch.utils.data.DistributedSampler(
dataset_train, num_replicas=num_tasks, rank=global_rank, shuffle=True
)
if args.dist_eval:
if len(dataset_val) % num_tasks != 0:
print('Warning: Enabling distributed evaluation with an eval dataset not divisible by process number. '
'This will slightly alter validation results as extra duplicate entries are added to achieve '
'equal num of samples per-process.')
sampler_val = torch.utils.data.DistributedSampler(
dataset_val, num_replicas=num_tasks, rank=global_rank, shuffle=False)
else:
sampler_val = torch.utils.data.SequentialSampler(dataset_val)
else:
sampler_train = torch.utils.data.RandomSampler(dataset_train)
sampler_val = torch.utils.data.SequentialSampler(dataset_val)
data_loader_train = torch.utils.data.DataLoader(
dataset_train, sampler=sampler_train,
batch_size=args.batch_size,
num_workers=args.num_workers,
pin_memory=args.pin_mem,
drop_last=True,
)
eval_batch_size = int(1.5 * args.batch_size)
import os
if os.environ['HOME'].endswith('root'):
eval_batch_size = int(4 * args.batch_size)
data_loader_val = torch.utils.data.DataLoader(
dataset_val, sampler=sampler_val,
batch_size=eval_batch_size,
num_workers=args.num_workers,
pin_memory=args.pin_mem,
drop_last=False
)
mixup_fn = None
mixup_active = args.mixup > 0 or args.cutmix > 0. or args.cutmix_minmax is not None
if mixup_active:
mixup_fn = Mixup(
mixup_alpha=args.mixup, cutmix_alpha=args.cutmix, cutmix_minmax=args.cutmix_minmax,
prob=args.mixup_prob, switch_prob=args.mixup_switch_prob, mode=args.mixup_mode,
label_smoothing=args.smoothing, num_classes=args.nb_classes)
print(f"Creating model: {args.model}")
model = create_model(
args.model,
pretrained=args.pretrained,
num_classes=args.nb_classes,
drop_rate=args.drop,
drop_path_rate=args.drop_path,
drop_block_rate=None,
)
linear_scaled_lr = args.lr * args.batch_size * utils.get_world_size() / 512.0
args.lr = linear_scaled_lr
args.bt_lr=linear_scaled_lr*args.bt_lr
from optimizer import create_bt_optimizer
model_o = model
optimizer1 = create_bt_optimizer(args, model, bt_decay=args.bt_decay)
lr_scheduler1, _ = create_scheduler(args, optimizer1)
if args.add_bt:
from bt import TransformerDecorator1, BlockBF, BlockWrap32
encoder_global = TransformerDecorator1(args.add_bt, model.num_features, args.eval_global,
args=args, drop_path=args.drop_path_bt)
if args.add_bt in [1]:
# This is for the last layer
res_blocks = []
for i, block in enumerate(model.blocks):
if args.no_fp16_bt and args.no_fp16_bt not in [4]:
# This is for ease nan
res_blocks.append(BlockWrap32(block, args.no_fp16_bt))
else:
res_blocks.append(block)
model.blocks = torch.nn.Sequential(*res_blocks)
model.norm = torch.nn.Sequential(model.norm, encoder_global)
elif args.add_bt in [3]:
# for insert into a single layer
res_blocks = []
drop_rate_list = [args.drop_path_bt]*(len(model.blocks)+1)
if args.insert_idx is not None and len(args.insert_idx) > 0:
insert_list = args.insert_idx
else:
insert_list = list(range(args.start_idx, len(model.blocks)))
for i, block in enumerate(model.blocks):
if args.no_fp16_bt and args.no_fp16_bt not in [4]:
res_blocks.append(BlockWrap32(block, args.no_fp16_bt))
else:
res_blocks.append(block)
if i in insert_list:
if not args.shared_bt:
encoder_global = TransformerDecorator1(args.add_bt, model.num_features, args.eval_global,
args=args, drop_path=drop_rate_list[i])
res_blocks.append(encoder_global)
model.blocks = torch.nn.Sequential(*res_blocks)
if args.add_norm_bt:
if not args.shared_bt:
encoder_global = TransformerDecorator1(args.add_bt, model.num_features, args.eval_global,
args=args, drop_path=args.drop_path_bt)
model.norm = torch.nn.Sequential(model.norm, encoder_global)
elif args.add_bt in [2]:
# This is for multiple layers
# 61, 62 is for half batch
# model.norm = torch.nn.Sequential(model.norm, encoder_global)
res_blocks = []
first_enc = TransformerDecorator1(args.add_bt, model.num_features, args.eval_global,
args=args, first_layer=True, drop_path=args.drop_path_bt)
old_enc = first_enc.encoder_layers
del old_enc
first_enc.encoder_layers = encoder_global.encoder_layers
nums = 0
if args.insert_idx is not None and len(args.insert_idx) > 0:
insert_list = args.insert_idx
else:
insert_list = list(range(args.start_idx, len(model.blocks)))
for i, block in enumerate(model.blocks):
if args.no_fp16_bt and args.no_fp16_bt not in [4]:
res_blocks.append(BlockWrap32(block, args.no_fp16_bt))
else:
res_blocks.append(block)
if i in insert_list:
if insert_list[0] == i: # first layer
res_blocks.append(first_enc)
first_enc = None
else:
if not args.shared_bt:
encoder_global = TransformerDecorator1(args.add_bt, model.num_features, args.eval_global,
args=args, drop_path=args.drop_path_bt)
res_blocks.append(encoder_global)
# model.norm = torch.nn.Identity()
if args.add_norm_bt:
if not args.shared_bt:
encoder_global = TransformerDecorator1(args.add_bt, model.num_features, args.eval_global,
args=args, drop_path=args.drop_path_bt)
model.norm = torch.nn.Sequential(model.norm, encoder_global)
model.blocks = torch.nn.Sequential(*res_blocks)
if args.drop_patch != 0:
# what's the relationship between drop_patch and shared prediction modules
# This is not for the paper.
res_blocks = []
for i, block in enumerate(model.blocks):
if i > args.start_idx:
if args.drop_patch > 0:
from timm.models.layers import DropPath
block.drop_path = DropPath(args.drop_patch)
res_blocks.append(BlockBF(block, args.no_fp16_bt))
else:
res_blocks.append(BlockBF(block, args.no_fp16_bt))
else:
res_blocks.append(block)
model.blocks = torch.nn.Sequential(*res_blocks)
print(model)
if args.finetune:
if args.finetune.startswith('https'):
checkpoint = torch.hub.load_state_dict_from_url(
args.finetune, map_location='cpu', check_hash=True)
else:
checkpoint = torch.load(args.finetune, map_location='cpu')
checkpoint_model = checkpoint['model']
state_dict = model.state_dict()
for k in ['head.weight', 'head.bias', 'head_dist.weight', 'head_dist.bias']:
if k in checkpoint_model and checkpoint_model[k].shape != state_dict[k].shape:
print(f"Removing key {k} from pretrained checkpoint")
del checkpoint_model[k]
# interpolate position embedding
pos_embed_checkpoint = checkpoint_model['pos_embed']
embedding_size = pos_embed_checkpoint.shape[-1]
num_patches = model.patch_embed.num_patches
num_extra_tokens = model.pos_embed.shape[-2] - num_patches
# height (== width) for the checkpoint position embedding
orig_size = int((pos_embed_checkpoint.shape[-2] - num_extra_tokens) ** 0.5)
# height (== width) for the new position embedding
new_size = int(num_patches ** 0.5)
# class_token and dist_token are kept unchanged
extra_tokens = pos_embed_checkpoint[:, :num_extra_tokens]
# only the position tokens are interpolated
pos_tokens = pos_embed_checkpoint[:, num_extra_tokens:]
pos_tokens = pos_tokens.reshape(-1, orig_size, orig_size, embedding_size).permute(0, 3, 1, 2)
pos_tokens = torch.nn.functional.interpolate(
pos_tokens, size=(new_size, new_size), mode='bicubic', align_corners=False)
pos_tokens = pos_tokens.permute(0, 2, 3, 1).flatten(1, 2)
new_pos_embed = torch.cat((extra_tokens, pos_tokens), dim=1)
checkpoint_model['pos_embed'] = new_pos_embed
model.load_state_dict(checkpoint_model, strict=False)
if args.add_mlp_bt:
from bt import MLPDecorder
model.pre_logits = MLPDecorder(model.num_features, 4096, skip_mlp=args.skip_bt)
print(model)
model.to(device)
model_ema = None
if args.model_ema:
# Important to create EMA model after cuda(), DP wrapper, and AMP but before SyncBN and DDP wrapper
model_ema = ModelEma(
model,
decay=args.model_ema_decay,
device='cpu' if args.model_ema_force_cpu else '',
resume='')
model_without_ddp = model
if args.distributed:
model = torch.nn.parallel.DistributedDataParallel(model, device_ids=[args.gpu], find_unused_parameters=True if args.skip_bt else False)
model_without_ddp = model.module
n_parameters = sum(p.numel() for p in model.parameters() if p.requires_grad)
print('number of params:', n_parameters)
# linear_scaled_lr = args.lr * args.batch_size * utils.get_world_size() / 512.0
args.lr = linear_scaled_lr
from optimizer import create_bt_optimizer
optimizer = create_bt_optimizer(args, model_without_ddp, bt_decay=args.bt_decay)
loss_scaler = NativeScaler()
lr_scheduler, _ = create_scheduler(args, optimizer)
criterion = LabelSmoothingCrossEntropy()
if mixup_active:
# smoothing is handled with mixup label transform
criterion = SoftTargetCrossEntropy()
elif args.smoothing:
criterion = LabelSmoothingCrossEntropy(smoothing=args.smoothing)
else:
criterion = torch.nn.CrossEntropyLoss()
teacher_model = None
if args.distillation_type != 'none':
assert args.teacher_path, 'need to specify teacher-path when using distillation'
print(f"Creating teacher model: {args.teacher_model}")
teacher_model = create_model(
args.teacher_model,
pretrained=args.pretrained,
num_classes=args.nb_classes,
global_pool='avg',
)
if args.teacher_path.startswith('https'):
checkpoint = torch.hub.load_state_dict_from_url(
args.teacher_path, map_location='cpu', check_hash=True)
else:
checkpoint = torch.load(args.teacher_path, map_location='cpu')
teacher_model.load_state_dict(checkpoint['model'])
teacher_model.to(device)
teacher_model.eval()
# wrap the criterion in our custom DistillationLoss, which
# just dispatches to the original criterion if args.distillation_type is 'none'
criterion = DistillationLoss(
criterion, teacher_model, args.distillation_type, args.distillation_alpha, args.distillation_tau
)
if args.balanced_softmax:
from losses import BalancedSoftmax
criterion = BalancedSoftmax('./cls_freq/ImageNet_LT.json')
output_dir = Path(args.output_dir)
if args.resume:
if args.resume.startswith('https'):
checkpoint = torch.hub.load_state_dict_from_url(
args.resume, map_location='cpu', check_hash=True)
else:
checkpoint = torch.load(args.resume, map_location='cpu')
if args.skip_bt:
model_without_ddp.load_state_dict(checkpoint['model'], strict=False)
else:
model_without_ddp.load_state_dict(checkpoint['model'], strict=True)
if not args.eval and 'optimizer' in checkpoint and 'lr_scheduler' in checkpoint and 'epoch' in checkpoint:
optimizer.load_state_dict(checkpoint['optimizer'])
lr_scheduler.load_state_dict(checkpoint['lr_scheduler'])
args.start_epoch = checkpoint['epoch'] + 1
if args.model_ema:
utils._load_checkpoint_for_ema(model_ema, checkpoint['model_ema'])
if 'scaler' in checkpoint:
loss_scaler.load_state_dict(checkpoint['scaler'])
if args.skip_bt:
optimizer.param_groups.pop(1)
if args.add_mlp_bt:
logits_list = [item for item in model_without_ddp.pre_logits.parameters()]
print(len(optimizer.param_groups), len(logits_list))
new_list = []
if len(optimizer.param_groups) == 1:
for item in optimizer.param_groups[0]['params']:
is_logits=False
for item1 in logits_list:
if item.size == item1.size and torch.all(torch.eq(item, item1)):
is_logits = True
break
if not is_logits:
new_list.append(item)
optimizer.param_groups[0]['params'] = new_list
else:
for item in optimizer.param_groups[1]['params']:
is_logits=False
for item1 in logits_list:
if item.size == item1.size and torch.all(torch.eq(item, item1)):
is_logits = True
break
if not is_logits:
new_list.append(item)
optimizer.param_groups[1]['params'] = new_list
if args.eval:
# test_stats = evaluate(data_loader_val, model, device)
training_labels = dataset_train.targets
training_labels = np.array(training_labels).astype(int)
from engine import validate
validate(data_loader_val, training_labels, model, device, args.nb_classes, utils.get_rank())
print(f"Accuracy of the network on the {len(dataset_val)} test images: {test_stats['acc1']:.1f}%")
return
print(f"Start training for {args.epochs} epochs")
start_time = time.time()
max_accuracy = 0.0
opt2=optimizer
model2 = model
lr_sched = lr_scheduler
for epoch in range(args.start_epoch, args.epochs):
if args.distributed:
data_loader_train.sampler.set_epoch(epoch)
if epoch < args.start_bt_epoch:
for item in model.modules():
if isinstance(item, TransformerDecorator1):
item.eval()
for n,p in item.named_parameters():
p.requires_grad=False
optimizer=optimizer1
model = model_o
lr_scheduler = lr_scheduler1
elif epoch == args.start_bt_epoch:
for n,p in model.named_parameters():
p.requires_grad=True
optimizer=opt2
model = model2
lr_scheduler = lr_sched
model.train(args.finetune == '')
train_stats = train_one_epoch(
model, criterion, data_loader_train,
optimizer, device, epoch, loss_scaler,
args.clip_grad, model_ema, mixup_fn,
set_training_mode=args.finetune == '', # keep in eval mode during finetuning
no_fp16_bt=args.no_fp16_bt, model_without_ddp = model_without_ddp,
num_classes=args.nb_classes
)
if epoch < args.start_bt_epoch:
lr_sched.step(epoch)
lr_scheduler.step(epoch)
if args.output_dir:
checkpoint_paths = [output_dir / str(args.exp_id+'checkpoint.pth')]
for checkpoint_path in checkpoint_paths:
utils.save_on_master({
'model': model_without_ddp.state_dict(),
'optimizer': optimizer.state_dict() if not args.skip_bt else 0,
'lr_scheduler': lr_scheduler.state_dict(),
'epoch': epoch,
'model_ema': get_state_dict(model_ema),
'scaler': loss_scaler.state_dict(),
'args': args,
}, checkpoint_path)
test_stats = evaluate(data_loader_val, model, device)
print(f"Accuracy of the network on the {len(dataset_val)} test images: {test_stats['acc1']:.1f}%")
max_accuracy = max(max_accuracy, test_stats["acc1"])
print(f'Max accuracy: {max_accuracy:.2f}%')
log_stats = {**{f'train_{k}': v for k, v in train_stats.items()},
**{f'test_{k}': v for k, v in test_stats.items()},
'epoch': epoch,
'n_parameters': n_parameters}
if args.output_dir and utils.is_main_process():
with (output_dir / str(args.exp_id+"log.txt")).open("a") as f:
f.write(json.dumps(log_stats) + "\n")
total_time = time.time() - start_time
total_time_str = str(datetime.timedelta(seconds=int(total_time)))
print('Training time {}'.format(total_time_str))
if __name__ == '__main__':
parser = argparse.ArgumentParser('DeiT training and evaluation script', parents=[get_args_parser()])
args = parser.parse_args()
if args.output_dir:
Path(args.output_dir).mkdir(parents=True, exist_ok=True)
main(args)
================================================
FILE: batchformer-v2/deit_share/models.py
================================================
# Copyright (c) 2015-present, Facebook, Inc.
# All rights reserved.
import os
import torch
import torch.nn as nn
from functools import partial
from timm.models.vision_transformer import VisionTransformer, _cfg
from timm.models.registry import register_model
from timm.models.layers import trunc_normal_
__all__ = [
'deit_tiny_patch16_224', 'deit_small_patch16_224', 'deit_small_patch16_224_12', 'deit_base_patch16_224',
'deit_tiny_distilled_patch16_224', 'deit_small_distilled_patch16_224',
'deit_base_distilled_patch16_224', 'deit_base_patch16_384',
'deit_base_distilled_patch16_384',
]
class DistilledVisionTransformer(VisionTransformer):
def __init__(self, *args, **kwargs):
super().__init__(*args, **kwargs)
self.dist_token = nn.Parameter(torch.zeros(1, 1, self.embed_dim))
num_patches = self.patch_embed.num_patches
self.pos_embed = nn.Parameter(torch.zeros(1, num_patches + 2, self.embed_dim))
self.head_dist = nn.Linear(self.embed_dim, self.num_classes) if self.num_classes > 0 else nn.Identity()
trunc_normal_(self.dist_token, std=.02)
trunc_normal_(self.pos_embed, std=.02)
self.head_dist.apply(self._init_weights)
def forward_features(self, x):
# taken from https://github.com/rwightman/pytorch-image-models/blob/master/timm/models/vision_transformer.py
# with slight modifications to add the dist_token
B = x.shape[0]
x = self.patch_embed(x)
cls_tokens = self.cls_token.expand(B, -1, -1) # stole cls_tokens impl from Phil Wang, thanks
dist_token = self.dist_token.expand(B, -1, -1)
x = torch.cat((cls_tokens, dist_token, x), dim=1)
x = x + self.pos_embed
x = self.pos_drop(x)
for blk in self.blocks:
x = blk(x)
x = self.norm(x)
return x[:, 0], x[:, 1]
def forward(self, x):
x, x_dist = self.forward_features(x)
x = self.head(x)
x_dist = self.head_dist(x_dist)
if self.training:
return x, x_dist
else:
# during inference, return the average of both classifier predictions
return (x + x_dist) / 2
@register_model
def deit_tiny_patch16_224(pretrained=False, **kwargs):
model = VisionTransformer(
patch_size=16, embed_dim=192, depth=12, num_heads=3, mlp_ratio=4, qkv_bias=True,
norm_layer=partial(nn.LayerNorm, eps=1e-6), **kwargs)
model.default_cfg = _cfg()
if pretrained:
checkpoint = torch.hub.load_state_dict_from_url(
url="https://dl.fbaipublicfiles.com/deit/deit_tiny_patch16_224-a1311bcf.pth",
map_location="cpu", check_hash=True
)
model.load_state_dict(checkpoint["model"])
return model
@register_model
def deit_small_patch16_224(pretrained=False, **kwargs):
model = VisionTransformer(
patch_size=16, embed_dim=384, depth=12, num_heads=6, mlp_ratio=4, qkv_bias=True,
norm_layer=partial(nn.LayerNorm, eps=1e-6), **kwargs)
model.default_cfg = _cfg()
if pretrained:
checkpoint = torch.hub.load_state_dict_from_url(
url="https://dl.fbaipublicfiles.com/deit/deit_small_patch16_224-cd65a155.pth",
map_location="cpu", check_hash=True
)
state_dict = model.state_dict()
for k in ['head.weight', 'head.bias']:
if k in checkpoint['model'] and checkpoint['model'][k].shape != state_dict[k].shape:
print(f"Removing key {k} from pretrained checkpoint")
del checkpoint['model'][k]
res = model.load_state_dict(checkpoint["model"], strict=False)
print(res)
return model
@register_model
def deit_small_patch16_224_12(pretrained=False, **kwargs):
model = VisionTransformer(
patch_size=16, embed_dim=384, depth=12, num_heads=12, mlp_ratio=4, qkv_bias=True,
norm_layer=partial(nn.LayerNorm, eps=1e-6), **kwargs)
model.default_cfg = _cfg()
if pretrained:
if os.environ['HOME'].endswith('zhou9878'):
checkpoint = torch.load("deit_small_patch16_224-cd65a155.pth")
else:
checkpoint = torch.load("/checkpoints/deit_small_patch16_224-cd65a155.pth")
# checkpoint = torch.hub.load_state_dict_from_url(
# url="https://dl.fbaipublicfiles.com/deit/deit_small_patch16_224-cd65a155.pth",
# map_location="cpu", check_hash=True
# )
model.load_state_dict(checkpoint["model"])
return model
@register_model
def deit_base_patch16_224(pretrained=False, **kwargs):
model = VisionTransformer(
patch_size=16, embed_dim=768, depth=12, num_heads=12, mlp_ratio=4, qkv_bias=True,
norm_layer=partial(nn.LayerNorm, eps=1e-6), **kwargs)
model.default_cfg = _cfg()
if pretrained:
# checkpoint = torch.hub.load_state_dict_from_url(
# url="https://dl.fbaipublicfiles.com/deit/deit_base_patch16_224-b5f2ef4d.pth",
# map_location="cpu", check_hash=True
# )
if os.environ['HOME'].endswith('zhou9878'):
checkpoint = torch.load("mae_pretrain_vit_base.pth")
else:
checkpoint = torch.load("/checkpoints/deit_base_patch16_224-b5f2ef4d.pth")
print("load mae")
state_dict = model.state_dict()
for k in ['head.weight', 'head.bias']:
if k in checkpoint['model'] and checkpoint['model'][k].shape != state_dict[k].shape:
print(f"Removing key {k} from pretrained checkpoint")
del checkpoint['model'][k]
res = model.load_state_dict(checkpoint["model"], strict=False)
print(res)
return model
@register_model
def deit_tiny_distilled_patch16_224(pretrained=False, **kwargs):
model = DistilledVisionTransformer(
patch_size=16, embed_dim=192, depth=12, num_heads=3, mlp_ratio=4, qkv_bias=True,
norm_layer=partial(nn.LayerNorm, eps=1e-6), **kwargs)
model.default_cfg = _cfg()
if pretrained:
checkpoint = torch.hub.load_state_dict_from_url(
url="https://dl.fbaipublicfiles.com/deit/deit_tiny_distilled_patch16_224-b40b3cf7.pth",
map_location="cpu", check_hash=True
)
model.load_state_dict(checkpoint["model"])
return model
@register_model
def deit_small_distilled_patch16_224(pretrained=False, **kwargs):
model = DistilledVisionTransformer(
patch_size=16, embed_dim=384, depth=12, num_heads=6, mlp_ratio=4, qkv_bias=True,
norm_layer=partial(nn.LayerNorm, eps=1e-6), **kwargs)
model.default_cfg = _cfg()
if pretrained:
checkpoint = torch.hub.load_state_dict_from_url(
url="https://dl.fbaipublicfiles.com/deit/deit_small_distilled_patch16_224-649709d9.pth",
map_location="cpu", check_hash=True
)
model.load_state_dict(checkpoint["model"])
return model
@register_model
def deit_base_distilled_patch16_224(pretrained=False, **kwargs):
model = DistilledVisionTransformer(
patch_size=16, embed_dim=768, depth=12, num_heads=12, mlp_ratio=4, qkv_bias=True,
norm_layer=partial(nn.LayerNorm, eps=1e-6), **kwargs)
model.default_cfg = _cfg()
if pretrained:
checkpoint = torch.hub.load_state_dict_from_url(
url="https://dl.fbaipublicfiles.com/deit/deit_base_distilled_patch16_224-df68dfff.pth",
map_location="cpu", check_hash=True
)
model.load_state_dict(checkpoint["model"])
return model
@register_model
def deit_base_patch16_384(pretrained=False, **kwargs):
model = VisionTransformer(
img_size=384, patch_size=16, embed_dim=768, depth=12, num_heads=12, mlp_ratio=4, qkv_bias=True,
norm_layer=partial(nn.LayerNorm, eps=1e-6), **kwargs)
model.default_cfg = _cfg()
if pretrained:
checkpoint = torch.hub.load_state_dict_from_url(
url="https://dl.fbaipublicfiles.com/deit/deit_base_patch16_384-8de9b5d1.pth",
map_location="cpu", check_hash=True
)
model.load_state_dict(checkpoint["model"])
return model
@register_model
def deit_base_distilled_patch16_384(pretrained=False, **kwargs):
model = DistilledVisionTransformer(
img_size=384, patch_size=16, embed_dim=768, depth=12, num_heads=12, mlp_ratio=4, qkv_bias=True,
norm_layer=partial(nn.LayerNorm, eps=1e-6), **kwargs)
model.default_cfg = _cfg()
if pretrained:
checkpoint = torch.hub.load_state_dict_from_url(
url="https://dl.fbaipublicfiles.com/deit/deit_base_distilled_patch16_384-d0272ac0.pth",
map_location="cpu", check_hash=True
)
model.load_state_dict(checkpoint["model"])
return model
================================================
FILE: batchformer-v2/deit_share/optimizer.py
================================================
import torch
from timm.optim import optimizer_kwargs
from timm.optim.optim_factory import add_weight_decay
from torch import nn
from typing import Optional
import torch.optim as optim
from timm.optim.adafactor import Adafactor
from timm.optim.adahessian import Adahessian
from timm.optim.adamp import AdamP
from timm.optim.lookahead import Lookahead
from timm.optim.nadam import Nadam
from timm.optim.novograd import NovoGrad
from timm.optim.nvnovograd import NvNovoGrad
from timm.optim.radam import RAdam
from timm.optim.rmsprop_tf import RMSpropTF
from timm.optim.sgdp import SGDP
from timm.optim.adabelief import AdaBelief
try:
from apex.optimizers import FusedNovoGrad, FusedAdam, FusedLAMB, FusedSGD
has_apex = True
except ImportError:
has_apex = False
def add_bt_weight_decay(model, weight_decay=1e-5, skip_list=(), second_group_tag=None, second_decay=1e-5, lr_bt=1e-4, mlp_decay=0):
decay = []
decay2 = []
decay3 = []
no_decay = []
for name, param in model.named_parameters():
if not param.requires_grad:
continue # frozen weights
if len(param.shape) == 1 or name.endswith(".bias") or name in skip_list:
no_decay.append(param)
elif second_group_tag and name.__contains__(second_group_tag):
print(name, 'bt group')
decay2.append(param)
elif mlp_decay:
decay3.append(param)
else:
decay.append(param)
return [
{'params': no_decay, 'weight_decay': 0., },
{'params': decay2, 'weight_decay': second_decay, 'lr': lr_bt},
{'params': decay3, 'weight_decay': mlp_decay},
{'params': decay, 'weight_decay': weight_decay}]
def create_bt_optimizer(args, model, bt_decay, filter_bias_and_bn=True):
""" Legacy optimizer factory for backwards compatibility.
NOTE: Use create_optimizer_v2 for new code.
"""
return create_optimizer_v2(
model,
**optimizer_kwargs(cfg=args),
filter_bias_and_bn=filter_bias_and_bn,
bt_decay=bt_decay,
bt_lr=args.bt_lr,
mlp_decay=args.mlp_decay,
)
def create_optimizer_v2(
model: nn.Module,
optimizer_name: str = 'sgd',
learning_rate: Optional[float] = None,
weight_decay: float = 0.,
momentum: float = 0.9,
filter_bias_and_bn: bool = True,
bt_decay = 0,
bt_lr = 1e-4,
mlp_decay=0,
**kwargs):
""" Create an optimizer.
For more general use an interface that allows selection of parameters to optimize and lr groups, one of:
* a filter fn interface that further breaks params into groups in a weight_decay compatible fashion
* expose the parameters interface and leave it up to caller
Args:
model (nn.Module): model containing parameters to optimize
optimizer_name: name of optimizer to create
learning_rate: initial learning rate
weight_decay: weight decay to apply in optimizer
momentum: momentum for momentum based optimizers (others may use betas via kwargs)
filter_bias_and_bn: filter out bias, bn and other 1d params from weight decay
**kwargs: extra optimizer specific kwargs to pass through
Returns:
Optimizer
"""
opt_lower = optimizer_name.lower()
if weight_decay and filter_bias_and_bn:
skip = {}
if hasattr(model, 'no_weight_decay'):
skip = model.no_weight_decay()
if bt_decay > 0:
parameters = add_bt_weight_decay(model, weight_decay, skip, 'encoder' if bt_decay > 0 else None, bt_decay, bt_lr, mlp_decay)
# if skip_bt:
# parameters = [parameters[0], parameters[2]]
for item in parameters:
print(item['weight_decay'])
else:
parameters = add_weight_decay(model, weight_decay, skip)
weight_decay = 0.
else:
parameters = model.parameters()
if 'fused' in opt_lower:
assert has_apex and torch.cuda.is_available(), 'APEX and CUDA required for fused optimizers'
opt_args = dict(lr=learning_rate, weight_decay=weight_decay, **kwargs)
opt_split = opt_lower.split('_')
opt_lower = opt_split[-1]
if opt_lower == 'sgd' or opt_lower == 'nesterov':
opt_args.pop('eps', None)
optimizer = optim.SGD(parameters, momentum=momentum, nesterov=True, **opt_args)
elif opt_lower == 'momentum':
opt_args.pop('eps', None)
optimizer = optim.SGD(parameters, momentum=momentum, nesterov=False, **opt_args)
elif opt_lower == 'adam':
optimizer = optim.Adam(parameters, **opt_args)
elif opt_lower == 'adabelief':
optimizer = AdaBelief(parameters, rectify=False, **opt_args)
elif opt_lower == 'adamw':
optimizer = optim.AdamW(parameters, **opt_args)
elif opt_lower == 'nadam':
optimizer = Nadam(parameters, **opt_args)
elif opt_lower == 'radam':
optimizer = RAdam(parameters, **opt_args)
elif opt_lower == 'adamp':
optimizer = AdamP(parameters, wd_ratio=0.01, nesterov=True, **opt_args)
elif opt_lower == 'sgdp':
optimizer = SGDP(parameters, momentum=momentum, nesterov=True, **opt_args)
elif opt_lower == 'adadelta':
optimizer = optim.Adadelta(parameters, **opt_args)
elif opt_lower == 'adafactor':
if not learning_rate:
opt_args['lr'] = None
optimizer = Adafactor(parameters, **opt_args)
elif opt_lower == 'adahessian':
optimizer = Adahessian(parameters, **opt_args)
elif opt_lower == 'rmsprop':
optimizer = optim.RMSprop(parameters, alpha=0.9, momentum=momentum, **opt_args)
elif opt_lower == 'rmsproptf':
optimizer = RMSpropTF(parameters, alpha=0.9, momentum=momentum, **opt_args)
elif opt_lower == 'novograd':
optimizer = NovoGrad(parameters, **opt_args)
elif opt_lower == 'nvnovograd':
optimizer = NvNovoGrad(parameters, **opt_args)
elif opt_lower == 'fusedsgd':
opt_args.pop('eps', None)
optimizer = FusedSGD(parameters, momentum=momentum, nesterov=True, **opt_args)
elif opt_lower == 'fusedmomentum':
opt_args.pop('eps', None)
optimizer = FusedSGD(parameters, momentum=momentum, nesterov=False, **opt_args)
elif opt_lower == 'fusedadam':
optimizer = FusedAdam(parameters, adam_w_mode=False, **opt_args)
elif opt_lower == 'fusedadamw':
optimizer = FusedAdam(parameters, adam_w_mode=True, **opt_args)
elif opt_lower == 'fusedlamb':
optimizer = FusedLAMB(parameters, **opt_args)
elif opt_lower == 'fusednovograd':
opt_args.setdefault('betas', (0.95, 0.98))
optimizer = FusedNovoGrad(parameters, **opt_args)
else:
assert False and "Invalid optimizer"
raise ValueError
if len(opt_split) > 1:
if opt_split[0] == 'lookahead':
optimizer = Lookahead(optimizer)
return optimizer
================================================
FILE: batchformer-v2/deit_share/requirements.txt
================================================
torch==1.7.0
torchvision==0.8.1
timm==0.3.2
================================================
FILE: batchformer-v2/deit_share/resmlp_models.py
================================================
# Copyright (c) 2015-present, Facebook, Inc.
# All rights reserved.
import torch
import torch.nn as nn
from functools import partial
from timm.models.vision_transformer import Mlp, PatchEmbed , _cfg
from timm.models.registry import register_model
from timm.models.layers import trunc_normal_, DropPath
__all__ = [
'resMLP_12', 'resMLP_24', 'resMLP_36', 'resmlpB_24'
]
class Affine(nn.Module):
def __init__(self, dim):
super().__init__()
self.alpha = nn.Parameter(torch.ones(dim))
self.beta = nn.Parameter(torch.zeros(dim))
def forward(self, x):
return self.alpha * x + self.beta
class layers_scale_mlp_blocks(nn.Module):
def __init__(self, dim, drop=0., drop_path=0., act_layer=nn.GELU,init_values=1e-4,num_patches = 196):
super().__init__()
self.norm1 = Affine(dim)
self.attn = nn.Linear(num_patches, num_patches)
self.drop_path = DropPath(drop_path) if drop_path > 0. else nn.Identity()
self.norm2 = Affine(dim)
self.mlp = Mlp(in_features=dim, hidden_features=int(4.0 * dim), act_layer=act_layer, drop=drop)
self.gamma_1 = nn.Parameter(init_values * torch.ones((dim)),requires_grad=True)
self.gamma_2 = nn.Parameter(init_values * torch.ones((dim)),requires_grad=True)
def forward(self, x):
x = x + self.drop_path(self.gamma_1 * self.attn(self.norm1(x).transpose(1,2)).transpose(1,2))
x = x + self.drop_path(self.gamma_2 * self.mlp(self.norm2(x)))
return x
class resmlp_models(nn.Module):
def __init__(self, img_size=224, patch_size=16, in_chans=3, num_classes=1000, embed_dim=768, depth=12,drop_rate=0.,
Patch_layer=PatchEmbed,act_layer=nn.GELU,
drop_path_rate=0.0,init_scale=1e-4):
super().__init__()
self.num_classes = num_classes
self.num_features = self.embed_dim = embed_dim
self.patch_embed = Patch_layer(
img_size=img_size, patch_size=patch_size, in_chans=int(in_chans), embed_dim=embed_dim)
num_patches = self.patch_embed.num_patches
dpr = [drop_path_rate for i in range(depth)]
self.blocks = nn.ModuleList([
layers_scale_mlp_blocks(
dim=embed_dim,drop=drop_rate,drop_path=dpr[i],
act_layer=act_layer,init_values=init_scale,
num_patches=num_patches)
for i in range(depth)])
self.norm = Affine(embed_dim)
self.feature_info = [dict(num_chs=embed_dim, reduction=0, module='head')]
self.head = nn.Linear(embed_dim, num_classes) if num_classes > 0 else nn.Identity()
self.apply(self._init_weights)
def _init_weights(self, m):
if isinstance(m, nn.Linear):
trunc_normal_(m.weight, std=0.02)
if isinstance(m, nn.Linear) and m.bias is not None:
nn.init.constant_(m.bias, 0)
elif isinstance(m, nn.LayerNorm):
nn.init.constant_(m.bias, 0)
nn.init.constant_(m.weight, 1.0)
def get_classifier(self):
return self.head
def reset_classifier(self, num_classes, global_pool=''):
self.num_classes = num_classes
self.head = nn.Linear(self.embed_dim, num_classes) if num_classes > 0 else nn.Identity()
def forward_features(self, x):
B = x.shape[0]
x = self.patch_embed(x)
for i , blk in enumerate(self.blocks):
x = blk(x)
x = self.norm(x)
x = x.mean(dim=1).reshape(B,1,-1)
return x[:, 0]
def forward(self, x):
x = self.forward_features(x)
x = self.head(x)
return x
@register_model
def resmlp_12(pretrained=False,dist=False, **kwargs):
model = resmlp_models(
patch_size=16, embed_dim=384, depth=12,
Patch_layer=PatchEmbed,
init_scale=0.1,**kwargs)
model.default_cfg = _cfg()
if pretrained:
if dist:
url_path = "https://dl.fbaipublicfiles.com/deit/resmlp_12_dist.pth"
else:
url_path = "https://dl.fbaipublicfiles.com/deit/resmlp_12_no_dist.pth"
checkpoint = torch.hub.load_state_dict_from_url(
url=url_path,
map_location="cpu", check_hash=True
)
model.load_state_dict(checkpoint)
return model
@register_model
def resmlp_24(pretrained=False,dist=False,dino=False, **kwargs):
model = resmlp_models(
patch_size=16, embed_dim=384, depth=24,
Patch_layer=PatchEmbed,
init_scale=1e-5,**kwargs)
model.default_cfg = _cfg()
if pretrained:
if dist:
url_path = "https://dl.fbaipublicfiles.com/deit/resmlp_24_dist.pth"
elif dino:
url_path = "https://dl.fbaipublicfiles.com/deit/resmlp_24_dino.pth"
else:
url_path = "https://dl.fbaipublicfiles.com/deit/resmlp_24_no_dist.pth"
checkpoint = torch.hub.load_state_dict_from_url(
url=url_path,
map_location="cpu", check_hash=True
)
model.load_state_dict(checkpoint)
return model
@register_model
def resmlp_36(pretrained=False,dist=False, **kwargs):
model = resmlp_models(
patch_size=16, embed_dim=384, depth=36,
Patch_layer=PatchEmbed,
init_scale=1e-6,**kwargs)
model.default_cfg = _cfg()
if pretrained:
if dist:
url_path = "https://dl.fbaipublicfiles.com/deit/resmlp_36_dist.pth"
else:
url_path = "https://dl.fbaipublicfiles.com/deit/resmlp_36_no_dist.pth"
checkpoint = torch.hub.load_state_dict_from_url(
url=url_path,
map_location="cpu", check_hash=True
)
model.load_state_dict(checkpoint)
return model
@register_model
def resmlpB_24(pretrained=False,dist=False, in_22k = False, **kwargs):
model = resmlp_models(
patch_size=8, embed_dim=768, depth=24,
Patch_layer=PatchEmbed,
init_scale=1e-6,**kwargs)
model.default_cfg = _cfg()
if pretrained:
if dist:
url_path = "https://dl.fbaipublicfiles.com/deit/resmlpB_24_dist.pth"
elif in_22k:
url_path = "https://dl.fbaipublicfiles.com/deit/resmlpB_24_22k.pth"
else:
url_path = "https://dl.fbaipublicfiles.com/deit/resmlpB_24_no_dist.pth"
checkpoint = torch.hub.load_state_dict_from_url(
url=url_path,
map_location="cpu", check_hash=True
)
model.load_state_dict(checkpoint)
return model
================================================
FILE: batchformer-v2/deit_share/run_with_submitit.py
================================================
# Copyright (c) 2015-present, Facebook, Inc.
# All rights reserved.
"""
A script to run multinode training with submitit.
"""
import argparse
import os
import uuid
from pathlib import Path
import main as classification
import submitit
def parse_args():
classification_parser = classification.get_args_parser()
parser = argparse.ArgumentParser("Submitit for DeiT", parents=[classification_parser])
parser.add_argument("--ngpus", default=8, type=int, help="Number of gpus to request on each node")
parser.add_argument("--nodes", default=2, type=int, help="Number of nodes to request")
parser.add_argument("--timeout", default=2800, type=int, help="Duration of the job")
parser.add_argument("--job_dir", default="", type=str, help="Job dir. Leave empty for automatic.")
parser.add_argument("--partition", default="learnfair", type=str, help="Partition where to submit")
parser.add_argument("--use_volta32", action='store_true', help="Big models? Use this")
parser.add_argument('--comment', default="", type=str,
help='Comment to pass to scheduler, e.g. priority message')
return parser.parse_args()
def get_shared_folder() -> Path:
user = os.getenv("USER")
if Path("/checkpoint/").is_dir():
p = Path(f"/checkpoint/{user}/experiments")
p.mkdir(exist_ok=True)
return p
raise RuntimeError("No shared folder available")
def get_init_file():
# Init file must not exist, but it's parent dir must exist.
os.makedirs(str(get_shared_folder()), exist_ok=True)
init_file = get_shared_folder() / f"{uuid.uuid4().hex}_init"
if init_file.exists():
os.remove(str(init_file))
return init_file
class Trainer(object):
def __init__(self, args):
self.args = args
def __call__(self):
import main as classification
self._setup_gpu_args()
classification.main(self.args)
def checkpoint(self):
import os
import submitit
self.args.dist_url = get_init_file().as_uri()
checkpoint_file = os.path.join(self.args.output_dir, "checkpoint.pth")
if os.path.exists(checkpoint_file):
self.args.resume = checkpoint_file
print("Requeuing ", self.args)
empty_trainer = type(self)(self.args)
return submitit.helpers.DelayedSubmission(empty_trainer)
def _setup_gpu_args(self):
import submitit
from pathlib import Path
job_env = submitit.JobEnvironment()
self.args.output_dir = Path(str(self.args.output_dir).replace("%j", str(job_env.job_id)))
self.args.gpu = job_env.local_rank
self.args.rank = job_env.global_rank
self.args.world_size = job_env.num_tasks
print(f"Process group: {job_env.num_tasks} tasks, rank: {job_env.global_rank}")
def main():
args = parse_args()
if args.job_dir == "":
args.job_dir = get_shared_folder() / "%j"
# Note that the folder will depend on the job_id, to easily track experiments
executor = submitit.AutoExecutor(folder=args.job_dir, slurm_max_num_timeout=30)
num_gpus_per_node = args.ngpus
nodes = args.nodes
timeout_min = args.timeout
partition = args.partition
kwargs = {}
if args.use_volta32:
kwargs['slurm_constraint'] = 'volta32gb'
if args.comment:
kwargs['slurm_comment'] = args.comment
executor.update_parameters(
mem_gb=40 * num_gpus_per_node,
gpus_per_node=num_gpus_per_node,
tasks_per_node=num_gpus_per_node, # one task per GPU
cpus_per_task=10,
nodes=nodes,
timeout_min=timeout_min, # max is 60 * 72
# Below are cluster dependent parameters
slurm_partition=partition,
slurm_signal_delay_s=120,
**kwargs
)
executor.update_parameters(name="deit")
args.dist_url = get_init_file().as_uri()
args.output_dir = args.job_dir
trainer = Trainer(args)
job = executor.submit(trainer)
print("Submitted job_id:", job.job_id)
if __name__ == "__main__":
main()
================================================
FILE: batchformer-v2/deit_share/samplers.py
================================================
# Copyright (c) 2015-present, Facebook, Inc.
# All rights reserved.
import torch
import torch.distributed as dist
import math
class RASampler(torch.utils.data.Sampler):
"""Sampler that restricts data loading to a subset of the dataset for distributed,
with repeated augmentation.
It ensures that different each augmented version of a sample will be visible to a
different process (GPU)
Heavily based on torch.utils.data.DistributedSampler
"""
def __init__(self, dataset, num_replicas=None, rank=None, shuffle=True):
if num_replicas is None:
if not dist.is_available():
raise RuntimeError("Requires distributed package to be available")
num_replicas = dist.get_world_size()
if rank is None:
if not dist.is_available():
raise RuntimeError("Requires distributed package to be available")
rank = dist.get_rank()
self.dataset = dataset
self.num_replicas = num_replicas
self.rank = rank
self.epoch = 0
self.num_samples = int(math.ceil(len(self.dataset) * 3.0 / self.num_replicas))
self.total_size = self.num_samples * self.num_replicas
# self.num_selected_samples = int(math.ceil(len(self.dataset) / self.num_replicas))
self.num_selected_samples = int(math.floor(len(self.dataset) // 256 * 256 / self.num_replicas))
self.shuffle = shuffle
def __iter__(self):
# deterministically shuffle based on epoch
g = torch.Generator()
g.manual_seed(self.epoch)
if self.shuffle:
indices = torch.randperm(len(self.dataset), generator=g).tolist()
else:
indices = list(range(len(self.dataset)))
# add extra samples to make it evenly divisible
indices = [ele for ele in indices for i in range(3)]
indices += indices[:(self.total_size - len(indices))]
assert len(indices) == self.total_size
# subsample
indices = indices[self.rank:self.total_size:self.num_replicas]
assert len(indices) == self.num_samples
return iter(indices[:self.num_selected_samples])
def __len__(self):
return self.num_selected_samples
def set_epoch(self, epoch):
self.epoch = epoch
================================================
FILE: batchformer-v2/deit_share/temp.txt
================================================
n01443537
n01629819
n01641577
n01644900
n01698640
n01742172
n01768244
n01770393
n01774384
n01774750
n01784675
n01855672
n01882714
n01910747
n01917289
n01944390
n01945685
n01950731
n01983481
n01984695
n02002724
n02056570
n02058221
n02074367
n02085620
n02094433
n02099601
n02099712
n02106662
n02113799
n02123045
n02123394
n02124075
n02125311
n02129165
n02132136
n02165456
n02190166
n02206856
n02226429
n02231487
n02233338
n02236044
n02268443
n02279972
n02281406
n02321529
n02364673
n02395406
n02403003
n02410509
n02415577
n02423022
n02437312
n02480495
n02481823
n02486410
n02504458
n02509815
n02666196
n02669723
n02699494
n02730930
n02769748
n02788148
n02791270
n02793495
n02795169
n02802426
n02808440
n02814533
n02814860
n02815834
n02823428
n02837789
n02841315
n02843684
n02883205
n02892201
n02906734
n02909870
n02917067
n02927161
n02948072
n02950826
n02963159
n02977058
n02988304
n02999410
n03014705
n03026506
n03042490
n03085013
n03089624
n03100240
n03126707
n03160309
n03179701
n03201208
n03250847
n03255030
n03355925
n03388043
n03393912
n03400231
n03404251
n03424325
n03444034
n03447447
n03544143
n03584254
n03599486
n03617480
n03637318
n03649909
n03662601
n03670208
n03706229
n03733131
n03763968
n03770439
n03796401
n03804744
n03814639
n03837869
n03838899
n03854065
n03891332
n03902125
n03930313
n03937543
n03970156
n03976657
n03977966
n03980874
n03983396
n03992509
n04008634
n04023962
n04067472
n04070727
n04074963
n04099969
n04118538
n04133789
n04146614
n04149813
n04179913
n04251144
n04254777
n04259630
n04265275
n04275548
n04285008
n04311004
n04328186
n04356056
n04366367
n04371430
n04376876
n04398044
n04399382
n04417672
n04456115
n04465501
n04486054
n04487081
n04501370
n04507155
n04532106
n04532670
n04540053
n04560804
n04562935
n04596742
n04597913
n06596364
n07579787
n07583066
n07614500
n07615774
n07695742
n07711569
n07715103
n07720875
n07734744
n07747607
n07749582
n07753592
n07768694
n07871810
n07873807
n07875152
n07920052
n09193705
n09246464
n09256479
n09332890
n09428293
n12267677
================================================
FILE: batchformer-v2/deit_share/tiny_img.py
================================================
import os
import sys
os.makedirs('/ImageNetTiny')
os.makedirs('/ImageNetTiny/train')
os.makedirs('/ImageNetTiny/val')
with open('temp.txt') as f:
for item in f.readlines():
if len(item) < 3:
continue
os.system('ln -s /data/train/{} /ImageNetTiny/train/'.format(item.strip()))
os.system('ln -s /data/val/{} /ImageNetTiny/val/'.format(item.strip()))
================================================
FILE: batchformer-v2/deit_share/tox.ini
================================================
[flake8]
max-line-length = 120
ignore = F401,E402,F403,W503,W504
================================================
FILE: batchformer-v2/deit_share/utils.py
================================================
# Copyright (c) 2015-present, Facebook, Inc.
# All rights reserved.
"""
Misc functions, including distributed helpers.
Mostly copy-paste from torchvision references.
"""
import io
import os
import time
from collections import defaultdict, deque
import datetime
import torch
import torch.distributed as dist
class SmoothedValue(object):
"""Track a series of values and provide access to smoothed values over a
window or the global series average.
"""
def __init__(self, window_size=20, fmt=None):
if fmt is None:
fmt = "{median:.4f} ({global_avg:.4f})"
self.deque = deque(maxlen=window_size)
self.total = 0.0
self.count = 0
self.fmt = fmt
def update(self, value, n=1):
self.deque.append(value)
self.count += n
self.total += value * n
def synchronize_between_processes(self):
"""
Warning: does not synchronize the deque!
"""
if not is_dist_avail_and_initialized():
return
t = torch.tensor([self.count, self.total], dtype=torch.float64, device='cuda')
dist.barrier()
dist.all_reduce(t)
t = t.tolist()
self.count = int(t[0])
self.total = t[1]
@property
def median(self):
d = torch.tensor(list(self.deque))
return d.median().item()
@property
def avg(self):
d = torch.tensor(list(self.deque), dtype=torch.float32)
return d.mean().item()
@property
def global_avg(self):
return self.total / self.count
@property
def max(self):
return max(self.deque)
@property
def value(self):
return self.deque[-1]
def __str__(self):
return self.fmt.format(
median=self.median,
avg=self.avg,
global_avg=self.global_avg,
max=self.max,
value=self.value)
class MetricLogger(object):
def __init__(self, delimiter="\t"):
self.meters = defaultdict(SmoothedValue)
self.delimiter = delimiter
def update(self, **kwargs):
for k, v in kwargs.items():
if isinstance(v, torch.Tensor):
v = v.item()
assert isinstance(v, (float, int))
self.meters[k].update(v)
def __getattr__(self, attr):
if attr in self.meters:
return self.meters[attr]
if attr in self.__dict__:
return self.__dict__[attr]
raise AttributeError("'{}' object has no attribute '{}'".format(
type(self).__name__, attr))
def __str__(self):
loss_str = []
for name, meter in self.meters.items():
loss_str.append(
"{}: {}".format(name, str(meter))
)
return self.delimiter.join(loss_str)
def synchronize_between_processes(self):
for meter in self.meters.values():
meter.synchronize_between_processes()
def add_meter(self, name, meter):
self.meters[name] = meter
def log_every(self, iterable, print_freq, header=None):
i = 0
if not header:
header = ''
start_time = time.time()
end = time.time()
iter_time = SmoothedValue(fmt='{avg:.4f}')
data_time = SmoothedValue(fmt='{avg:.4f}')
space_fmt = ':' + str(len(str(len(iterable)))) + 'd'
log_msg = [
header,
'[{0' + space_fmt + '}/{1}]',
'eta: {eta}',
'{meters}',
'time: {time}',
'data: {data}'
]
if torch.cuda.is_available():
log_msg.append('max mem: {memory:.0f}')
log_msg = self.delimiter.join(log_msg)
MB = 1024.0 * 1024.0
for obj in iterable:
data_time.update(time.time() - end)
yield obj
iter_time.update(time.time() - end)
if i % print_freq == 0 or i == len(iterable) - 1:
eta_seconds = iter_time.global_avg * (len(iterable) - i)
eta_string = str(datetime.timedelta(seconds=int(eta_seconds)))
if torch.cuda.is_available():
print(log_msg.format(
i, len(iterable), eta=eta_string,
meters=str(self),
time=str(iter_time), data=str(data_time),
memory=torch.cuda.max_memory_allocated() / MB))
else:
print(log_msg.format(
i, len(iterable), eta=eta_string,
meters=str(self),
time=str(iter_time), data=str(data_time)))
i += 1
end = time.time()
total_time = time.time() - start_time
total_time_str = str(datetime.timedelta(seconds=int(total_time)))
print('{} Total time: {} ({:.4f} s / it)'.format(
header, total_time_str, total_time / len(iterable)))
def _load_checkpoint_for_ema(model_ema, checkpoint):
"""
Workaround for ModelEma._load_checkpoint to accept an already-loaded object
"""
mem_file = io.BytesIO()
torch.save(checkpoint, mem_file)
mem_file.seek(0)
model_ema._load_checkpoint(mem_file)
def setup_for_distributed(is_master):
"""
This function disables printing when not in master process
"""
import builtins as __builtin__
builtin_print = __builtin__.print
def print(*args, **kwargs):
force = kwargs.pop('force', False)
if is_master or force:
builtin_print(*args, **kwargs)
__builtin__.print = print
def is_dist_avail_and_initialized():
if not dist.is_available():
return False
if not dist.is_initialized():
return False
return True
def get_world_size():
if not is_dist_avail_and_initialized():
return 1
return dist.get_world_size()
def get_rank():
if not is_dist_avail_and_initialized():
return 0
return dist.get_rank()
def is_main_process():
return get_rank() == 0
def save_on_master(*args, **kwargs):
if is_main_process():
torch.save(*args, **kwargs)
def init_distributed_mode(args):
if 'RANK' in os.environ and 'WORLD_SIZE' in os.environ:
args.rank = int(os.environ["RANK"])
args.world_size = int(os.environ['WORLD_SIZE'])
args.gpu = int(os.environ['LOCAL_RANK'])
elif 'SLURM_PROCID' in os.environ:
args.rank = int(os.environ['SLURM_PROCID'])
args.gpu = args.rank % torch.cuda.device_count()
else:
print('Not using distributed mode')
args.distributed = False
return
args.distributed = True
torch.cuda.set_device(args.gpu)
args.dist_backend = 'nccl'
print('| distributed init (rank {}): {}'.format(
args.rank, args.dist_url), flush=True)
torch.distributed.init_process_group(backend=args.dist_backend, init_method=args.dist_url,
world_size=args.world_size, rank=args.rank)
torch.distributed.barrier()
setup_for_distributed(args.rank == 0)
================================================
FILE: batchformer-v2/detr/.circleci/config.yml
================================================
version: 2.1
jobs:
python_lint:
docker:
- image: circleci/python:3.7
steps:
- checkout
- run:
command: |
pip install --user --progress-bar off flake8 typing
flake8 .
test:
docker:
- image: circleci/python:3.7
steps:
- checkout
- run:
command: |
pip install --user --progress-bar off scipy pytest
pip install --user --progress-bar off --pre torch torchvision -f https://download.pytorch.org/whl/nightly/cpu/torch_nightly.html
pip install --user --progress-bar off onnx onnxruntime
pytest .
workflows:
build:
jobs:
- python_lint
- test
================================================
FILE: batchformer-v2/detr/.github/CODE_OF_CONDUCT.md
================================================
# Code of Conduct
Facebook has adopted a Code of Conduct that we expect project participants to adhere to.
Please read the [full text](https://code.fb.com/codeofconduct/)
so that you can understand what actions will and will not be tolerated.
================================================
FILE: batchformer-v2/detr/.github/CONTRIBUTING.md
================================================
# Contributing to DETR
We want to make contributing to this project as easy and transparent as
possible.
## Our Development Process
Minor changes and improvements will be released on an ongoing basis. Larger changes (e.g., changesets implementing a new paper) will be released on a more periodic basis.
## Pull Requests
We actively welcome your pull requests.
1. Fork the repo and create your branch from `master`.
2. If you've added code that should be tested, add tests.
3. If you've changed APIs, update the documentation.
4. Ensure the test suite passes.
5. Make sure your code lints.
6. If you haven't already, complete the Contributor License Agreement ("CLA").
## Contributor License Agreement ("CLA")
In order to accept your pull request, we need you to submit a CLA. You only need
to do this once to work on any of Facebook's open source projects.
Complete your CLA here:
## Issues
We use GitHub issues to track public bugs. Please ensure your description is
clear and has sufficient instructions to be able to reproduce the issue.
Facebook has a [bounty program](https://www.facebook.com/whitehat/) for the safe
disclosure of security bugs. In those cases, please go through the process
outlined on that page and do not file a public issue.
## Coding Style
* 4 spaces for indentation rather than tabs
* 80 character line length
* PEP8 formatting following [Black](https://black.readthedocs.io/en/stable/)
## License
By contributing to DETR, you agree that your contributions will be licensed
under the LICENSE file in the root directory of this source tree.
================================================
FILE: batchformer-v2/detr/.github/ISSUE_TEMPLATE/bugs.md
================================================
---
name: "🐛 Bugs"
about: Report bugs in DETR
title: Please read & provide the following
---
## Instructions To Reproduce the 🐛 Bug:
1. what changes you made (`git diff`) or what code you wrote
```
```
2. what exact command you run:
3. what you observed (including __full logs__):
```
```
4. please simplify the steps as much as possible so they do not require additional resources to
run, such as a private dataset.
## Expected behavior:
If there are no obvious error in "what you observed" provided above,
please tell us the expected behavior.
## Environment:
Provide your environment information using the following command:
```
python -m torch.utils.collect_env
```
================================================
FILE: batchformer-v2/detr/.github/ISSUE_TEMPLATE/questions-help-support.md
================================================
---
name: "How to do something❓"
about: How to do something using DETR?
---
## ❓ How to do something using DETR
Describe what you want to do, including:
1. what inputs you will provide, if any:
2. what outputs you are expecting:
NOTE:
1. Only general answers are provided.
If you want to ask about "why X did not work", please use the
[Unexpected behaviors](https://github.com/facebookresearch/detr/issues/new/choose) issue template.
2. About how to implement new models / new dataloader / new training logic, etc., check documentation first.
3. We do not answer general machine learning / computer vision questions that are not specific to DETR, such as how a model works, how to improve your training/make it converge, or what algorithm/methods can be used to achieve X.
================================================
FILE: batchformer-v2/detr/.github/ISSUE_TEMPLATE/unexpected-problems-bugs.md
================================================
---
name: "Unexpected behaviors"
about: Run into unexpected behaviors when using DETR
title: Please read & provide the following
---
If you do not know the root cause of the problem, and wish someone to help you, please
post according to this template:
## Instructions To Reproduce the Issue:
1. what changes you made (`git diff`) or what code you wrote
```
```
2. what exact command you run:
3. what you observed (including __full logs__):
```
```
4. please simplify the steps as much as possible so they do not require additional resources to
run, such as a private dataset.
## Expected behavior:
If there are no obvious error in "what you observed" provided above,
please tell us the expected behavior.
If you expect the model to converge / work better, note that we do not give suggestions
on how to train a new model.
Only in one of the two conditions we will help with it:
(1) You're unable to reproduce the results in DETR model zoo.
(2) It indicates a DETR bug.
## Environment:
Provide your environment information using the following command:
```
python -m torch.utils.collect_env
```
================================================
FILE: batchformer-v2/detr/.gitignore
================================================
.nfs*
*.ipynb
*.pyc
.dumbo.json
.DS_Store
.*.swp
*.pth
**/__pycache__/**
.ipynb_checkpoints/
datasets/data/
experiment-*
*.tmp
*.pkl
**/.mypy_cache/*
.mypy_cache/*
not_tracked_dir/
.vscode
================================================
FILE: batchformer-v2/detr/Dockerfile
================================================
FROM pytorch/pytorch:1.5-cuda10.1-cudnn7-runtime
ENV DEBIAN_FRONTEND=noninteractive
RUN apt-get update -qq && \
apt-get install -y git vim libgtk2.0-dev && \
rm -rf /var/cache/apk/*
RUN pip --no-cache-dir install Cython
COPY requirements.txt /workspace
RUN pip --no-cache-dir install -r /workspace/requirements.txt
================================================
FILE: batchformer-v2/detr/LICENSE
================================================
Apache License
Version 2.0, January 2004
http://www.apache.org/licenses/
TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
1. Definitions.
"License" shall mean the terms and conditions for use, reproduction,
and distribution as defined by Sections 1 through 9 of this document.
"Licensor" shall mean the copyright owner or entity authorized by
the copyright owner that is granting the License.
"Legal Entity" shall mean the union of the acting entity and all
other entities that control, are controlled by, or are under common
control with that entity. For the purposes of this definition,
"control" means (i) the power, direct or indirect, to cause the
direction or management of such entity, whether by contract or
otherwise, or (ii) ownership of fifty percent (50%) or more of the
outstanding shares, or (iii) beneficial ownership of such entity.
"You" (or "Your") shall mean an individual or Legal Entity
exercising permissions granted by this License.
"Source" form shall mean the preferred form for making modifications,
including but not limited to software source code, documentation
source, and configuration files.
"Object" form shall mean any form resulting from mechanical
transformation or translation of a Source form, including but
not limited to compiled object code, generated documentation,
and conversions to other media types.
"Work" shall mean the work of authorship, whether in Source or
Object form, made available under the License, as indicated by a
copyright notice that is included in or attached to the work
(an example is provided in the Appendix below).
"Derivative Works" shall mean any work, whether in Source or Object
form, that is based on (or derived from) the Work and for which the
editorial revisions, annotations, elaborations, or other modifications
represent, as a whole, an original work of authorship. For the purposes
of this License, Derivative Works shall not include works that remain
separable from, or merely link (or bind by name) to the interfaces of,
the Work and Derivative Works thereof.
"Contribution" shall mean any work of authorship, including
the original version of the Work and any modifications or additions
to that Work or Derivative Works thereof, that is intentionally
submitted to Licensor for inclusion in the Work by the copyright owner
or by an individual or Legal Entity authorized to submit on behalf of
the copyright owner. For the purposes of this definition, "submitted"
means any form of electronic, verbal, or written communication sent
to the Licensor or its representatives, including but not limited to
communication on electronic mailing lists, source code control systems,
and issue tracking systems that are managed by, or on behalf of, the
Licensor for the purpose of discussing and improving the Work, but
excluding communication that is conspicuously marked or otherwise
designated in writing by the copyright owner as "Not a Contribution."
"Contributor" shall mean Licensor and any individual or Legal Entity
on behalf of whom a Contribution has been received by Licensor and
subsequently incorporated within the Work.
2. Grant of Copyright License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
copyright license to reproduce, prepare Derivative Works of,
publicly display, publicly perform, sublicense, and distribute the
Work and such Derivative Works in Source or Object form.
3. Grant of Patent License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
(except as stated in this section) patent license to make, have made,
use, offer to sell, sell, import, and otherwise transfer the Work,
where such license applies only to those patent claims licensable
by such Contributor that are necessarily infringed by their
Contribution(s) alone or by combination of their Contribution(s)
with the Work to which such Contribution(s) was submitted. If You
institute patent litigation against any entity (including a
cross-claim or counterclaim in a lawsuit) alleging that the Work
or a Contribution incorporated within the Work constitutes direct
or contributory patent infringement, then any patent licenses
granted to You under this License for that Work shall terminate
as of the date such litigation is filed.
4. Redistribution. You may reproduce and distribute copies of the
Work or Derivative Works thereof in any medium, with or without
modifications, and in Source or Object form, provided that You
meet the following conditions:
(a) You must give any other recipients of the Work or
Derivative Works a copy of this License; and
(b) You must cause any modified files to carry prominent notices
stating that You changed the files; and
(c) You must retain, in the Source form of any Derivative Works
that You distribute, all copyright, patent, trademark, and
attribution notices from the Source form of the Work,
excluding those notices that do not pertain to any part of
the Derivative Works; and
(d) If the Work includes a "NOTICE" text file as part of its
distribution, then any Derivative Works that You distribute must
include a readable copy of the attribution notices contained
within such NOTICE file, excluding those notices that do not
pertain to any part of the Derivative Works, in at least one
of the following places: within a NOTICE text file distributed
as part of the Derivative Works; within the Source form or
documentation, if provided along with the Derivative Works; or,
within a display generated by the Derivative Works, if and
wherever such third-party notices normally appear. The contents
of the NOTICE file are for informational purposes only and
do not modify the License. You may add Your own attribution
notices within Derivative Works that You distribute, alongside
or as an addendum to the NOTICE text from the Work, provided
that such additional attribution notices cannot be construed
as modifying the License.
You may add Your own copyright statement to Your modifications and
may provide additional or different license terms and conditions
for use, reproduction, or distribution of Your modifications, or
for any such Derivative Works as a whole, provided Your use,
reproduction, and distribution of the Work otherwise complies with
the conditions stated in this License.
5. Submission of Contributions. Unless You explicitly state otherwise,
any Contribution intentionally submitted for inclusion in the Work
by You to the Licensor shall be under the terms and conditions of
this License, without any additional terms or conditions.
Notwithstanding the above, nothing herein shall supersede or modify
the terms of any separate license agreement you may have executed
with Licensor regarding such Contributions.
6. Trademarks. This License does not grant permission to use the trade
names, trademarks, service marks, or product names of the Licensor,
except as required for reasonable and customary use in describing the
origin of the Work and reproducing the content of the NOTICE file.
7. Disclaimer of Warranty. Unless required by applicable law or
agreed to in writing, Licensor provides the Work (and each
Contributor provides its Contributions) on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
implied, including, without limitation, any warranties or conditions
of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
PARTICULAR PURPOSE. You are solely responsible for determining the
appropriateness of using or redistributing the Work and assume any
risks associated with Your exercise of permissions under this License.
8. Limitation of Liability. In no event and under no legal theory,
whether in tort (including negligence), contract, or otherwise,
unless required by applicable law (such as deliberate and grossly
negligent acts) or agreed to in writing, shall any Contributor be
liable to You for damages, including any direct, indirect, special,
incidental, or consequential damages of any character arising as a
result of this License or out of the use or inability to use the
Work (including but not limited to damages for loss of goodwill,
work stoppage, computer failure or malfunction, or any and all
other commercial damages or losses), even if such Contributor
has been advised of the possibility of such damages.
9. Accepting Warranty or Additional Liability. While redistributing
the Work or Derivative Works thereof, You may choose to offer,
and charge a fee for, acceptance of support, warranty, indemnity,
or other liability obligations and/or rights consistent with this
License. However, in accepting such obligations, You may act only
on Your own behalf and on Your sole responsibility, not on behalf
of any other Contributor, and only if You agree to indemnify,
defend, and hold each Contributor harmless for any liability
incurred by, or claims asserted against, such Contributor by reason
of your accepting any such warranty or additional liability.
END OF TERMS AND CONDITIONS
APPENDIX: How to apply the Apache License to your work.
To apply the Apache License to your work, attach the following
boilerplate notice, with the fields enclosed by brackets "[]"
replaced with your own identifying information. (Don't include
the brackets!) The text should be enclosed in the appropriate
comment syntax for the file format. We also recommend that a
file or class name and description of purpose be included on the
same "printed page" as the copyright notice for easier
identification within third-party archives.
Copyright 2020 - present, Facebook, Inc
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
================================================
FILE: batchformer-v2/detr/README.md
================================================
This is the repository based on DETR. This repository includes the implementation of batchformer based on DETR. The corresponding code starts from here [https://github.com/zhihou7/detr/blob/449d425a8ae8465512234bc3f8750ff07f5c473b/models/transformer.py#L33](https://github.com/zhihou7/detr/blob/449d425a8ae8465512234bc3f8750ff07f5c473b/models/transformer.py#L33).
Please ignore the "checkpoint" that I actually do not use it.
Core part of the code is
[https://github.com/zhihou7/detr/blob/449d425a8ae8465512234bc3f8750ff07f5c473b/models/transformer.py#L121](https://github.com/zhihou7/detr/blob/449d425a8ae8465512234bc3f8750ff07f5c473b/models/transformer.py#L121)
However, this repository might include some codes for my other projects. For the redundant code, you can just ignore it.
For this batchformerv2 experiment, I insert the batchformerv2 module in the first layer (i.e. insert_index=0 and bf_idx=3). Meanwhile, I run the experiments on 16G V100 and set the batch size as 8 with 4 gpus.
================================================
FILE: batchformer-v2/detr/configs/base.sh
================================================
#!/usr/bin/env bash
source /usr/local/anaconda/4.2.0/etc/profile.d/conda.sh
conda activate pyt
set -x
python main.py --coco_path /project/ZHIHOU/Dataset/coco/ --bf 1 --batch_size 2 --output_dir bf1 --resume bf1/checkpoint.pth
================================================
FILE: batchformer-v2/detr/configs/panoptic.sh
================================================
#!/usr/bin/env bash
source /usr/local/anaconda/4.2.0/etc/profile.d/conda.sh
conda activate pyt
set -x
#python main.py --coco_path /project/ZHIHOU/Dataset/coco/ --bf 1 --batch_size 2 --output_dir bf1
#python -m torch.distributed.launch --nproc_per_node=8 --use_env
python main.py --coco_path /project/ZHIHOU/Dataset/coco/ \
--coco_panoptic_path /project/ZHIHOU/Dataset/coco/ \
--dataset_file coco_panoptic \
--output_dir panoptic_bf3 --bf 3 --resume panoptic_bf3/checkpoint.pth
================================================
FILE: batchformer-v2/detr/configs/panoptic_1.sh
================================================
#!/usr/bin/env bash
source /usr/local/anaconda/4.2.0/etc/profile.d/conda.sh
conda activate pyt
set -x
python main.py --masks --epochs 25 --lr_drop 15 --coco_path /project/ZHIHOU/Dataset/coco/ \
--coco_panoptic_path /project/ZHIHOU/Dataset/coco/ \
--dataset_file coco_panoptic \
--frozen_weights panoptic_bf3/checkpoint0249.pth \
--output_dir panoptic_bf0_segm --batch_size 1
================================================
FILE: batchformer-v2/detr/configs/panoptic_2.sh
================================================
#!/usr/bin/env bash
source /usr/local/anaconda/4.2.0/etc/profile.d/conda.sh
conda activate pyt
set -x
python main.py --masks --epochs 25 --lr_drop 15 --coco_path /project/ZHIHOU/Dataset/coco/ \
--coco_panoptic_path /project/ZHIHOU/Dataset/coco/ \
--dataset_file coco_panoptic \
--frozen_weights panoptic_bf3/checkpoint0249.pth \
--use_checkpoint \
--output_dir panoptic_bf3_segm --bf 3 --batch_size 2
================================================
FILE: batchformer-v2/detr/configs/swin.sh
================================================
#!/usr/bin/env bash
source /usr/local/anaconda/4.2.0/etc/profile.d/conda.sh
conda activate pyt
set -x
python main.py --coco_path /project/ZHIHOU/Dataset/coco/ --bf 3 --batch_size 2 --output_dir output_bf3_7 --resume output_bf3_7/checkpoint.pth
================================================
FILE: batchformer-v2/detr/d2/README.md
================================================
Detectron2 wrapper for DETR
=======
We provide a Detectron2 wrapper for DETR, thus providing a way to better integrate it in the existing detection ecosystem. It can be used for example to easily leverage datasets or backbones provided in Detectron2.
This wrapper currently supports only box detection, and is intended to be as close as possible to the original implementation, and we checked that it indeed match the results. Some notable facts and caveats:
- The data augmentation matches DETR's original data augmentation. This required patching the RandomCrop augmentation from Detectron2, so you'll need a version from the master branch from June 24th 2020 or more recent.
- To match DETR's original backbone initialization, we use the weights of a ResNet50 trained on imagenet using torchvision. This network uses a different pixel mean and std than most of the backbones available in Detectron2 by default, so extra care must be taken when switching to another one. Note that no other torchvision models are available in Detectron2 as of now, though it may change in the future.
- The gradient clipping mode is "full_model", which is not the default in Detectron2.
# Usage
To install Detectron2, please follow the [official installation instructions](https://github.com/facebookresearch/detectron2/blob/master/INSTALL.md).
## Evaluating a model
For convenience, we provide a conversion script to convert models trained by the main DETR training loop into the format of this wrapper. To download and convert the main Resnet50 model, simply do:
```
python converter.py --source_model https://dl.fbaipublicfiles.com/detr/detr-r50-e632da11.pth --output_model converted_model.pth
```
You can then evaluate it using:
```
python train_net.py --eval-only --config configs/detr_256_6_6_torchvision.yaml MODEL.WEIGHTS "converted_model.pth"
```
## Training
To train DETR on a single node with 8 gpus, simply use:
```
python train_net.py --config configs/detr_256_6_6_torchvision.yaml --num-gpus 8
```
To fine-tune DETR for instance segmentation on a single node with 8 gpus, simply use:
```
python train_net.py --config configs/detr_segm_256_6_6_torchvision.yaml --num-gpus 8 MODEL.DETR.FROZEN_WEIGHTS
```
================================================
FILE: batchformer-v2/detr/d2/configs/detr_256_6_6_torchvision.yaml
================================================
MODEL:
META_ARCHITECTURE: "Detr"
WEIGHTS: "detectron2://ImageNetPretrained/torchvision/R-50.pkl"
PIXEL_MEAN: [123.675, 116.280, 103.530]
PIXEL_STD: [58.395, 57.120, 57.375]
MASK_ON: False
RESNETS:
DEPTH: 50
STRIDE_IN_1X1: False
OUT_FEATURES: ["res2", "res3", "res4", "res5"]
DETR:
GIOU_WEIGHT: 2.0
L1_WEIGHT: 5.0
NUM_OBJECT_QUERIES: 100
DATASETS:
TRAIN: ("coco_2017_train",)
TEST: ("coco_2017_val",)
SOLVER:
IMS_PER_BATCH: 64
BASE_LR: 0.0001
STEPS: (369600,)
MAX_ITER: 554400
WARMUP_FACTOR: 1.0
WARMUP_ITERS: 10
WEIGHT_DECAY: 0.0001
OPTIMIZER: "ADAMW"
BACKBONE_MULTIPLIER: 0.1
CLIP_GRADIENTS:
ENABLED: True
CLIP_TYPE: "full_model"
CLIP_VALUE: 0.01
NORM_TYPE: 2.0
INPUT:
MIN_SIZE_TRAIN: (480, 512, 544, 576, 608, 640, 672, 704, 736, 768, 800)
CROP:
ENABLED: True
TYPE: "absolute_range"
SIZE: (384, 600)
FORMAT: "RGB"
TEST:
EVAL_PERIOD: 4000
DATALOADER:
FILTER_EMPTY_ANNOTATIONS: False
NUM_WORKERS: 4
VERSION: 2
================================================
FILE: batchformer-v2/detr/d2/configs/detr_segm_256_6_6_torchvision.yaml
================================================
MODEL:
META_ARCHITECTURE: "Detr"
# WEIGHTS: "detectron2://ImageNetPretrained/torchvision/R-50.pkl"
PIXEL_MEAN: [123.675, 116.280, 103.530]
PIXEL_STD: [58.395, 57.120, 57.375]
MASK_ON: True
RESNETS:
DEPTH: 50
STRIDE_IN_1X1: False
OUT_FEATURES: ["res2", "res3", "res4", "res5"]
DETR:
GIOU_WEIGHT: 2.0
L1_WEIGHT: 5.0
NUM_OBJECT_QUERIES: 100
FROZEN_WEIGHTS: ''
DATASETS:
TRAIN: ("coco_2017_train",)
TEST: ("coco_2017_val",)
SOLVER:
IMS_PER_BATCH: 64
BASE_LR: 0.0001
STEPS: (55440,)
MAX_ITER: 92400
WARMUP_FACTOR: 1.0
WARMUP_ITERS: 10
WEIGHT_DECAY: 0.0001
OPTIMIZER: "ADAMW"
BACKBONE_MULTIPLIER: 0.1
CLIP_GRADIENTS:
ENABLED: True
CLIP_TYPE: "full_model"
CLIP_VALUE: 0.01
NORM_TYPE: 2.0
INPUT:
MIN_SIZE_TRAIN: (480, 512, 544, 576, 608, 640, 672, 704, 736, 768, 800)
CROP:
ENABLED: True
TYPE: "absolute_range"
SIZE: (384, 600)
FORMAT: "RGB"
TEST:
EVAL_PERIOD: 4000
DATALOADER:
FILTER_EMPTY_ANNOTATIONS: False
NUM_WORKERS: 4
VERSION: 2
================================================
FILE: batchformer-v2/detr/d2/converter.py
================================================
# Copyright (c) Facebook, Inc. and its affiliates. All Rights Reserved
"""
Helper script to convert models trained with the main version of DETR to be used with the Detectron2 version.
"""
import json
import argparse
import numpy as np
import torch
def parse_args():
parser = argparse.ArgumentParser("D2 model converter")
parser.add_argument("--source_model", default="", type=str, help="Path or url to the DETR model to convert")
parser.add_argument("--output_model", default="", type=str, help="Path where to save the converted model")
return parser.parse_args()
def main():
args = parse_args()
# D2 expects contiguous classes, so we need to remap the 92 classes from DETR
# fmt: off
coco_idx = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25,
27, 28, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 46, 47, 48, 49, 50, 51,
52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 67, 70, 72, 73, 74, 75, 76, 77,
78, 79, 80, 81, 82, 84, 85, 86, 87, 88, 89, 90, 91]
# fmt: on
coco_idx = np.array(coco_idx)
if args.source_model.startswith("https"):
checkpoint = torch.hub.load_state_dict_from_url(args.source_model, map_location="cpu", check_hash=True)
else:
checkpoint = torch.load(args.source_model, map_location="cpu")
model_to_convert = checkpoint["model"]
model_converted = {}
for k in model_to_convert.keys():
old_k = k
if "backbone" in k:
k = k.replace("backbone.0.body.", "")
if "layer" not in k:
k = "stem." + k
for t in [1, 2, 3, 4]:
k = k.replace(f"layer{t}", f"res{t + 1}")
for t in [1, 2, 3]:
k = k.replace(f"bn{t}", f"conv{t}.norm")
k = k.replace("downsample.0", "shortcut")
k = k.replace("downsample.1", "shortcut.norm")
k = "backbone.0.backbone." + k
k = "detr." + k
print(old_k, "->", k)
if "class_embed" in old_k:
v = model_to_convert[old_k].detach()
if v.shape[0] == 92:
shape_old = v.shape
model_converted[k] = v[coco_idx]
print("Head conversion: changing shape from {} to {}".format(shape_old, model_converted[k].shape))
continue
model_converted[k] = model_to_convert[old_k].detach()
model_to_save = {"model": model_converted}
torch.save(model_to_save, args.output_model)
if __name__ == "__main__":
main()
================================================
FILE: batchformer-v2/detr/d2/detr/__init__.py
================================================
# Copyright (c) Facebook, Inc. and its affiliates. All Rights Reserved
from .config import add_detr_config
from .detr import Detr
from .dataset_mapper import DetrDatasetMapper
================================================
FILE: batchformer-v2/detr/d2/detr/config.py
================================================
# -*- coding: utf-8 -*-
# Copyright (c) Facebook, Inc. and its affiliates. All Rights Reserved
from detectron2.config import CfgNode as CN
def add_detr_config(cfg):
"""
Add config for DETR.
"""
cfg.MODEL.DETR = CN()
cfg.MODEL.DETR.NUM_CLASSES = 80
# For Segmentation
cfg.MODEL.DETR.FROZEN_WEIGHTS = ''
# LOSS
cfg.MODEL.DETR.GIOU_WEIGHT = 2.0
cfg.MODEL.DETR.L1_WEIGHT = 5.0
cfg.MODEL.DETR.DEEP_SUPERVISION = True
cfg.MODEL.DETR.NO_OBJECT_WEIGHT = 0.1
# TRANSFORMER
cfg.MODEL.DETR.NHEADS = 8
cfg.MODEL.DETR.DROPOUT = 0.1
cfg.MODEL.DETR.DIM_FEEDFORWARD = 2048
cfg.MODEL.DETR.ENC_LAYERS = 6
cfg.MODEL.DETR.DEC_LAYERS = 6
cfg.MODEL.DETR.PRE_NORM = False
cfg.MODEL.DETR.HIDDEN_DIM = 256
cfg.MODEL.DETR.NUM_OBJECT_QUERIES = 100
cfg.SOLVER.OPTIMIZER = "ADAMW"
cfg.SOLVER.BACKBONE_MULTIPLIER = 0.1
================================================
FILE: batchformer-v2/detr/d2/detr/dataset_mapper.py
================================================
# Copyright (c) Facebook, Inc. and its affiliates. All Rights Reserved
import copy
import logging
import numpy as np
import torch
from detectron2.data import detection_utils as utils
from detectron2.data import transforms as T
from detectron2.data.transforms import TransformGen
__all__ = ["DetrDatasetMapper"]
def build_transform_gen(cfg, is_train):
"""
Create a list of :class:`TransformGen` from config.
Returns:
list[TransformGen]
"""
if is_train:
min_size = cfg.INPUT.MIN_SIZE_TRAIN
max_size = cfg.INPUT.MAX_SIZE_TRAIN
sample_style = cfg.INPUT.MIN_SIZE_TRAIN_SAMPLING
else:
min_size = cfg.INPUT.MIN_SIZE_TEST
max_size = cfg.INPUT.MAX_SIZE_TEST
sample_style = "choice"
if sample_style == "range":
assert len(min_size) == 2, "more than 2 ({}) min_size(s) are provided for ranges".format(len(min_size))
logger = logging.getLogger(__name__)
tfm_gens = []
if is_train:
tfm_gens.append(T.RandomFlip())
tfm_gens.append(T.ResizeShortestEdge(min_size, max_size, sample_style))
if is_train:
logger.info("TransformGens used in training: " + str(tfm_gens))
return tfm_gens
class DetrDatasetMapper:
"""
A callable which takes a dataset dict in Detectron2 Dataset format,
and map it into a format used by DETR.
The callable currently does the following:
1. Read the image from "file_name"
2. Applies geometric transforms to the image and annotation
3. Find and applies suitable cropping to the image and annotation
4. Prepare image and annotation to Tensors
"""
def __init__(self, cfg, is_train=True):
if cfg.INPUT.CROP.ENABLED and is_train:
self.crop_gen = [
T.ResizeShortestEdge([400, 500, 600], sample_style="choice"),
T.RandomCrop(cfg.INPUT.CROP.TYPE, cfg.INPUT.CROP.SIZE),
]
else:
self.crop_gen = None
self.mask_on = cfg.MODEL.MASK_ON
self.tfm_gens = build_transform_gen(cfg, is_train)
logging.getLogger(__name__).info(
"Full TransformGens used in training: {}, crop: {}".format(str(self.tfm_gens), str(self.crop_gen))
)
self.img_format = cfg.INPUT.FORMAT
self.is_train = is_train
def __call__(self, dataset_dict):
"""
Args:
dataset_dict (dict): Metadata of one image, in Detectron2 Dataset format.
Returns:
dict: a format that builtin models in detectron2 accept
"""
dataset_dict = copy.deepcopy(dataset_dict) # it will be modified by code below
image = utils.read_image(dataset_dict["file_name"], format=self.img_format)
utils.check_image_size(dataset_dict, image)
if self.crop_gen is None:
image, transforms = T.apply_transform_gens(self.tfm_gens, image)
else:
if np.random.rand() > 0.5:
image, transforms = T.apply_transform_gens(self.tfm_gens, image)
else:
image, transforms = T.apply_transform_gens(
self.tfm_gens[:-1] + self.crop_gen + self.tfm_gens[-1:], image
)
image_shape = image.shape[:2] # h, w
# Pytorch's dataloader is efficient on torch.Tensor due to shared-memory,
# but not efficient on large generic data structures due to the use of pickle & mp.Queue.
# Therefore it's important to use torch.Tensor.
dataset_dict["image"] = torch.as_tensor(np.ascontiguousarray(image.transpose(2, 0, 1)))
if not self.is_train:
# USER: Modify this if you want to keep them for some reason.
dataset_dict.pop("annotations", None)
return dataset_dict
if "annotations" in dataset_dict:
# USER: Modify this if you want to keep them for some reason.
for anno in dataset_dict["annotations"]:
if not self.mask_on:
anno.pop("segmentation", None)
anno.pop("keypoints", None)
# USER: Implement additional transformations if you have other types of data
annos = [
utils.transform_instance_annotations(obj, transforms, image_shape)
for obj in dataset_dict.pop("annotations")
if obj.get("iscrowd", 0) == 0
]
instances = utils.annotations_to_instances(annos, image_shape)
dataset_dict["instances"] = utils.filter_empty_instances(instances)
return dataset_dict
================================================
FILE: batchformer-v2/detr/d2/detr/detr.py
================================================
# Copyright (c) Facebook, Inc. and its affiliates. All Rights Reserved
import logging
import math
from typing import List
import numpy as np
import torch
import torch.distributed as dist
import torch.nn.functional as F
from scipy.optimize import linear_sum_assignment
from torch import nn
from detectron2.layers import ShapeSpec
from detectron2.modeling import META_ARCH_REGISTRY, build_backbone, detector_postprocess
from detectron2.structures import Boxes, ImageList, Instances, BitMasks, PolygonMasks
from detectron2.utils.logger import log_first_n
from fvcore.nn import giou_loss, smooth_l1_loss
from models.backbone import Joiner
from models.detr import DETR, SetCriterion
from models.matcher import HungarianMatcher
from models.position_encoding import PositionEmbeddingSine
from models.transformer import Transformer
from models.segmentation import DETRsegm, PostProcessPanoptic, PostProcessSegm
from util.box_ops import box_cxcywh_to_xyxy, box_xyxy_to_cxcywh
from util.misc import NestedTensor
from datasets.coco import convert_coco_poly_to_mask
__all__ = ["Detr"]
class MaskedBackbone(nn.Module):
""" This is a thin wrapper around D2's backbone to provide padding masking"""
def __init__(self, cfg):
super().__init__()
self.backbone = build_backbone(cfg)
backbone_shape = self.backbone.output_shape()
self.feature_strides = [backbone_shape[f].stride for f in backbone_shape.keys()]
self.num_channels = backbone_shape[list(backbone_shape.keys())[-1]].channels
def forward(self, images):
features = self.backbone(images.tensor)
masks = self.mask_out_padding(
[features_per_level.shape for features_per_level in features.values()],
images.image_sizes,
images.tensor.device,
)
assert len(features) == len(masks)
for i, k in enumerate(features.keys()):
features[k] = NestedTensor(features[k], masks[i])
return features
def mask_out_padding(self, feature_shapes, image_sizes, device):
masks = []
assert len(feature_shapes) == len(self.feature_strides)
for idx, shape in enumerate(feature_shapes):
N, _, H, W = shape
masks_per_feature_level = torch.ones((N, H, W), dtype=torch.bool, device=device)
for img_idx, (h, w) in enumerate(image_sizes):
masks_per_feature_level[
img_idx,
: int(np.ceil(float(h) / self.feature_strides[idx])),
: int(np.ceil(float(w) / self.feature_strides[idx])),
] = 0
masks.append(masks_per_feature_level)
return masks
@META_ARCH_REGISTRY.register()
class Detr(nn.Module):
"""
Implement Detr
"""
def __init__(self, cfg):
super().__init__()
self.device = torch.device(cfg.MODEL.DEVICE)
self.num_classes = cfg.MODEL.DETR.NUM_CLASSES
self.mask_on = cfg.MODEL.MASK_ON
hidden_dim = cfg.MODEL.DETR.HIDDEN_DIM
num_queries = cfg.MODEL.DETR.NUM_OBJECT_QUERIES
# Transformer parameters:
nheads = cfg.MODEL.DETR.NHEADS
dropout = cfg.MODEL.DETR.DROPOUT
dim_feedforward = cfg.MODEL.DETR.DIM_FEEDFORWARD
enc_layers = cfg.MODEL.DETR.ENC_LAYERS
dec_layers = cfg.MODEL.DETR.DEC_LAYERS
pre_norm = cfg.MODEL.DETR.PRE_NORM
# Loss parameters:
giou_weight = cfg.MODEL.DETR.GIOU_WEIGHT
l1_weight = cfg.MODEL.DETR.L1_WEIGHT
deep_supervision = cfg.MODEL.DETR.DEEP_SUPERVISION
no_object_weight = cfg.MODEL.DETR.NO_OBJECT_WEIGHT
N_steps = hidden_dim // 2
d2_backbone = MaskedBackbone(cfg)
backbone = Joiner(d2_backbone, PositionEmbeddingSine(N_steps, normalize=True))
backbone.num_channels = d2_backbone.num_channels
transformer = Transformer(
d_model=hidden_dim,
dropout=dropout,
nhead=nheads,
dim_feedforward=dim_feedforward,
num_encoder_layers=enc_layers,
num_decoder_layers=dec_layers,
normalize_before=pre_norm,
return_intermediate_dec=deep_supervision,
)
self.detr = DETR(
backbone, transformer, num_classes=self.num_classes, num_queries=num_queries, aux_loss=deep_supervision
)
if self.mask_on:
frozen_weights = cfg.MODEL.DETR.FROZEN_WEIGHTS
if frozen_weights != '':
print("LOAD pre-trained weights")
weight = torch.load(frozen_weights, map_location=lambda storage, loc: storage)['model']
new_weight = {}
for k, v in weight.items():
if 'detr.' in k:
new_weight[k.replace('detr.', '')] = v
else:
print(f"Skipping loading weight {k} from frozen model")
del weight
self.detr.load_state_dict(new_weight)
del new_weight
self.detr = DETRsegm(self.detr, freeze_detr=(frozen_weights != ''))
self.seg_postprocess = PostProcessSegm
self.detr.to(self.device)
# building criterion
matcher = HungarianMatcher(cost_class=1, cost_bbox=l1_weight, cost_giou=giou_weight)
weight_dict = {"loss_ce": 1, "loss_bbox": l1_weight}
weight_dict["loss_giou"] = giou_weight
if deep_supervision:
aux_weight_dict = {}
for i in range(dec_layers - 1):
aux_weight_dict.update({k + f"_{i}": v for k, v in weight_dict.items()})
weight_dict.update(aux_weight_dict)
losses = ["labels", "boxes", "cardinality"]
if self.mask_on:
losses += ["masks"]
self.criterion = SetCriterion(
self.num_classes, matcher=matcher, weight_dict=weight_dict, eos_coef=no_object_weight, losses=losses,
)
self.criterion.to(self.device)
pixel_mean = torch.Tensor(cfg.MODEL.PIXEL_MEAN).to(self.device).view(3, 1, 1)
pixel_std = torch.Tensor(cfg.MODEL.PIXEL_STD).to(self.device).view(3, 1, 1)
self.normalizer = lambda x: (x - pixel_mean) / pixel_std
self.to(self.device)
def forward(self, batched_inputs):
"""
Args:
batched_inputs: a list, batched outputs of :class:`DatasetMapper` .
Each item in the list contains the inputs for one image.
For now, each item in the list is a dict that contains:
* image: Tensor, image in (C, H, W) format.
* instances: Instances
Other information that's included in the original dicts, such as:
* "height", "width" (int): the output resolution of the model, used in inference.
See :meth:`postprocess` for details.
Returns:
dict[str: Tensor]:
mapping from a named loss to a tensor storing the loss. Used during training only.
"""
images = self.preprocess_image(batched_inputs)
output = self.detr(images)
if self.training:
gt_instances = [x["instances"].to(self.device) for x in batched_inputs]
targets = self.prepare_targets(gt_instances)
loss_dict = self.criterion(output, targets)
weight_dict = self.criterion.weight_dict
for k in loss_dict.keys():
if k in weight_dict:
loss_dict[k] *= weight_dict[k]
return loss_dict
else:
box_cls = output["pred_logits"]
box_pred = output["pred_boxes"]
mask_pred = output["pred_masks"] if self.mask_on else None
results = self.inference(box_cls, box_pred, mask_pred, images.image_sizes)
processed_results = []
for results_per_image, input_per_image, image_size in zip(results, batched_inputs, images.image_sizes):
height = input_per_image.get("height", image_size[0])
width = input_per_image.get("width", image_size[1])
r = detector_postprocess(results_per_image, height, width)
processed_results.append({"instances": r})
return processed_results
def prepare_targets(self, targets):
new_targets = []
for targets_per_image in targets:
h, w = targets_per_image.image_size
image_size_xyxy = torch.as_tensor([w, h, w, h], dtype=torch.float, device=self.device)
gt_classes = targets_per_image.gt_classes
gt_boxes = targets_per_image.gt_boxes.tensor / image_size_xyxy
gt_boxes = box_xyxy_to_cxcywh(gt_boxes)
new_targets.append({"labels": gt_classes, "boxes": gt_boxes})
if self.mask_on and hasattr(targets_per_image, 'gt_masks'):
gt_masks = targets_per_image.gt_masks
gt_masks = convert_coco_poly_to_mask(gt_masks.polygons, h, w)
new_targets[-1].update({'masks': gt_masks})
return new_targets
def inference(self, box_cls, box_pred, mask_pred, image_sizes):
"""
Arguments:
box_cls (Tensor): tensor of shape (batch_size, num_queries, K).
The tensor predicts the classification probability for each query.
box_pred (Tensor): tensors of shape (batch_size, num_queries, 4).
The tensor predicts 4-vector (x,y,w,h) box
regression values for every queryx
image_sizes (List[torch.Size]): the input image sizes
Returns:
results (List[Instances]): a list of #images elements.
"""
assert len(box_cls) == len(image_sizes)
results = []
# For each box we assign the best class or the second best if the best on is `no_object`.
scores, labels = F.softmax(box_cls, dim=-1)[:, :, :-1].max(-1)
for i, (scores_per_image, labels_per_image, box_pred_per_image, image_size) in enumerate(zip(
scores, labels, box_pred, image_sizes
)):
result = Instances(image_size)
result.pred_boxes = Boxes(box_cxcywh_to_xyxy(box_pred_per_image))
result.pred_boxes.scale(scale_x=image_size[1], scale_y=image_size[0])
if self.mask_on:
mask = F.interpolate(mask_pred[i].unsqueeze(0), size=image_size, mode='bilinear', align_corners=False)
mask = mask[0].sigmoid() > 0.5
B, N, H, W = mask_pred.shape
mask = BitMasks(mask.cpu()).crop_and_resize(result.pred_boxes.tensor.cpu(), 32)
result.pred_masks = mask.unsqueeze(1).to(mask_pred[0].device)
result.scores = scores_per_image
result.pred_classes = labels_per_image
results.append(result)
return results
def preprocess_image(self, batched_inputs):
"""
Normalize, pad and batch the input images.
"""
images = [self.normalizer(x["image"].to(self.device)) for x in batched_inputs]
images = ImageList.from_tensors(images)
return images
================================================
FILE: batchformer-v2/detr/d2/train_net.py
================================================
# Copyright (c) Facebook, Inc. and its affiliates. All Rights Reserved
"""
DETR Training Script.
This script is a simplified version of the training script in detectron2/tools.
"""
import os
import sys
import itertools
# fmt: off
sys.path.insert(1, os.path.join(sys.path[0], '..'))
# fmt: on
import time
from typing import Any, Dict, List, Set
import torch
import detectron2.utils.comm as comm
from d2.detr import DetrDatasetMapper, add_detr_config
from detectron2.checkpoint import DetectionCheckpointer
from detectron2.config import get_cfg
from detectron2.data import MetadataCatalog, build_detection_train_loader
from detectron2.engine import DefaultTrainer, default_argument_parser, default_setup, launch
from detectron2.evaluation import COCOEvaluator, verify_results
from detectron2.solver.build import maybe_add_gradient_clipping
class Trainer(DefaultTrainer):
"""
Extension of the Trainer class adapted to DETR.
"""
@classmethod
def build_evaluator(cls, cfg, dataset_name, output_folder=None):
"""
Create evaluator(s) for a given dataset.
This uses the special metadata "evaluator_type" associated with each builtin dataset.
For your own dataset, you can simply create an evaluator manually in your
script and do not have to worry about the hacky if-else logic here.
"""
if output_folder is None:
output_folder = os.path.join(cfg.OUTPUT_DIR, "inference")
return COCOEvaluator(dataset_name, cfg, True, output_folder)
@classmethod
def build_train_loader(cls, cfg):
if "Detr" == cfg.MODEL.META_ARCHITECTURE:
mapper = DetrDatasetMapper(cfg, True)
else:
mapper = None
return build_detection_train_loader(cfg, mapper=mapper)
@classmethod
def build_optimizer(cls, cfg, model):
params: List[Dict[str, Any]] = []
memo: Set[torch.nn.parameter.Parameter] = set()
for key, value in model.named_parameters(recurse=True):
if not value.requires_grad:
continue
# Avoid duplicating parameters
if value in memo:
continue
memo.add(value)
lr = cfg.SOLVER.BASE_LR
weight_decay = cfg.SOLVER.WEIGHT_DECAY
if "backbone" in key:
lr = lr * cfg.SOLVER.BACKBONE_MULTIPLIER
params += [{"params": [value], "lr": lr, "weight_decay": weight_decay}]
def maybe_add_full_model_gradient_clipping(optim): # optim: the optimizer class
# detectron2 doesn't have full model gradient clipping now
clip_norm_val = cfg.SOLVER.CLIP_GRADIENTS.CLIP_VALUE
enable = (
cfg.SOLVER.CLIP_GRADIENTS.ENABLED
and cfg.SOLVER.CLIP_GRADIENTS.CLIP_TYPE == "full_model"
and clip_norm_val > 0.0
)
class FullModelGradientClippingOptimizer(optim):
def step(self, closure=None):
all_params = itertools.chain(*[x["params"] for x in self.param_groups])
torch.nn.utils.clip_grad_norm_(all_params, clip_norm_val)
super().step(closure=closure)
return FullModelGradientClippingOptimizer if enable else optim
optimizer_type = cfg.SOLVER.OPTIMIZER
if optimizer_type == "SGD":
optimizer = maybe_add_full_model_gradient_clipping(torch.optim.SGD)(
params, cfg.SOLVER.BASE_LR, momentum=cfg.SOLVER.MOMENTUM
)
elif optimizer_type == "ADAMW":
optimizer = maybe_add_full_model_gradient_clipping(torch.optim.AdamW)(
params, cfg.SOLVER.BASE_LR
)
else:
raise NotImplementedError(f"no optimizer type {optimizer_type}")
if not cfg.SOLVER.CLIP_GRADIENTS.CLIP_TYPE == "full_model":
optimizer = maybe_add_gradient_clipping(cfg, optimizer)
return optimizer
def setup(args):
"""
Create configs and perform basic setups.
"""
cfg = get_cfg()
add_detr_config(cfg)
cfg.merge_from_file(args.config_file)
cfg.merge_from_list(args.opts)
cfg.freeze()
default_setup(cfg, args)
return cfg
def main(args):
cfg = setup(args)
if args.eval_only:
model = Trainer.build_model(cfg)
DetectionCheckpointer(model, save_dir=cfg.OUTPUT_DIR).resume_or_load(cfg.MODEL.WEIGHTS, resume=args.resume)
res = Trainer.test(cfg, model)
if comm.is_main_process():
verify_results(cfg, res)
return res
trainer = Trainer(cfg)
trainer.resume_or_load(resume=args.resume)
return trainer.train()
if __name__ == "__main__":
args = default_argument_parser().parse_args()
print("Command Line Args:", args)
launch(
main,
args.num_gpus,
num_machines=args.num_machines,
machine_rank=args.machine_rank,
dist_url=args.dist_url,
args=(args,),
)
================================================
FILE: batchformer-v2/detr/datasets/__init__.py
================================================
# Copyright (c) Facebook, Inc. and its affiliates. All Rights Reserved
import torch.utils.data
import torchvision
from .coco import build as build_coco
from .coco import build_hico, build_coco2014
def get_coco_api_from_dataset(dataset):
for _ in range(10):
# if isinstance(dataset, torchvision.datasets.CocoDetection):
# break
if isinstance(dataset, torch.utils.data.Subset):
dataset = dataset.dataset
if isinstance(dataset, torchvision.datasets.CocoDetection):
return dataset.coco
def build_dataset(image_set, args):
if args.dataset_file == 'coco':
return build_coco(image_set, args)
elif args.dataset_file == 'hico':
return build_hico(image_set, args)
elif args.dataset_file == 'coco2014':
return build_coco2014(image_set, args)
if args.dataset_file == 'coco_panoptic':
# to avoid making panopticapi required for coco
from .coco_panoptic import build as build_coco_panoptic
return build_coco_panoptic(image_set, args)
raise ValueError(f'dataset {args.dataset_file} not supported')
================================================
FILE: batchformer-v2/detr/datasets/coco.py
================================================
# Copyright (c) Facebook, Inc. and its affiliates. All Rights Reserved
"""
COCO dataset which returns image_id for evaluation.
Mostly copy-paste from https://github.com/pytorch/vision/blob/13b35ff/references/detection/coco_utils.py
"""
from pathlib import Path
import torch
import torch.utils.data
import torchvision
from pycocotools import mask as coco_mask
import datasets.transforms as T
class CocoDetection(torchvision.datasets.CocoDetection):
def __init__(self, img_folder, ann_file, transforms, return_masks):
super(CocoDetection, self).__init__(img_folder, ann_file)
self._transforms = transforms
self.prepare = ConvertCocoPolysToMask(return_masks)
def __getitem__(self, idx):
img, target = super(CocoDetection, self).__getitem__(idx)
image_id = self.ids[idx]
target = {'image_id': image_id, 'annotations': target}
img, target = self.prepare(img, target)
if self._transforms is not None:
img, target = self._transforms(img, target)
return img, target
def convert_coco_poly_to_mask(segmentations, height, width):
masks = []
for polygons in segmentations:
rles = coco_mask.frPyObjects(polygons, height, width)
mask = coco_mask.decode(rles)
if len(mask.shape) < 3:
mask = mask[..., None]
mask = torch.as_tensor(mask, dtype=torch.uint8)
mask = mask.any(dim=2)
masks.append(mask)
if masks:
masks = torch.stack(masks, dim=0)
else:
masks = torch.zeros((0, height, width), dtype=torch.uint8)
return masks
class ConvertCocoPolysToMask(object):
def __init__(self, return_masks=False):
self.return_masks = return_masks
def __call__(self, image, target):
w, h = image.size
image_id = target["image_id"]
image_id = torch.tensor([image_id])
anno = target["annotations"]
anno = [obj for obj in anno if 'iscrowd' not in obj or obj['iscrowd'] == 0]
boxes = [obj["bbox"] for obj in anno]
# guard against no boxes via resizing
boxes = torch.as_tensor(boxes, dtype=torch.float32).reshape(-1, 4)
boxes[:, 2:] += boxes[:, :2]
boxes[:, 0::2].clamp_(min=0, max=w)
boxes[:, 1::2].clamp_(min=0, max=h)
classes = [obj["category_id"] for obj in anno]
classes = torch.tensor(classes, dtype=torch.int64)
if self.return_masks:
segmentations = [obj["segmentation"] for obj in anno]
masks = convert_coco_poly_to_mask(segmentations, h, w)
keypoints = None
if anno and "keypoints" in anno[0]:
keypoints = [obj["keypoints"] for obj in anno]
keypoints = torch.as_tensor(keypoints, dtype=torch.float32)
num_keypoints = keypoints.shape[0]
if num_keypoints:
keypoints = keypoints.view(num_keypoints, -1, 3)
keep = (boxes[:, 3] > boxes[:, 1]) & (boxes[:, 2] > boxes[:, 0])
boxes = boxes[keep]
classes = classes[keep]
if self.return_masks:
masks = masks[keep]
if keypoints is not None:
keypoints = keypoints[keep]
target = {}
target["boxes"] = boxes
target["labels"] = classes
if self.return_masks:
target["masks"] = masks
target["image_id"] = image_id
if keypoints is not None:
target["keypoints"] = keypoints
# for conversion to coco api
area = torch.tensor([obj["area"] for obj in anno])
iscrowd = torch.tensor([obj["iscrowd"] if "iscrowd" in obj else 0 for obj in anno])
target["area"] = area[keep]
target["iscrowd"] = iscrowd[keep]
target["orig_size"] = torch.as_tensor([int(h), int(w)])
target["size"] = torch.as_tensor([int(h), int(w)])
return image, target
def make_coco_transforms(image_set):
normalize = T.Compose([
T.ToTensor(),
T.Normalize([0.485, 0.456, 0.406], [0.229, 0.224, 0.225])
])
scales = [480, 512, 544, 576, 608, 640, 672, 704, 736, 768, 800]
if image_set == 'train':
return T.Compose([
T.RandomHorizontalFlip(),
T.RandomSelect(
T.RandomResize(scales, max_size=1333),
T.Compose([
T.RandomResize([400, 500, 600]),
T.RandomSizeCrop(384, 600),
T.RandomResize(scales, max_size=1333),
])
),
normalize,
])
if image_set == 'val':
return T.Compose([
T.RandomResize([800], max_size=1333),
normalize,
])
raise ValueError(f'unknown {image_set}')
def build(image_set, args):
root = Path(args.coco_path)
assert root.exists(), f'provided COCO path {root} does not exist'
mode = 'instances'
PATHS = {
"train": (root / "train2017", root / "annotations" / f'{mode}_train2017.json'),
"val": (root / "val2017", root / "annotations" / f'{mode}_val2017.json'),
}
img_folder, ann_file = PATHS[image_set]
dataset = CocoDetection(img_folder, ann_file, transforms=make_coco_transforms(image_set), return_masks=args.masks)
return dataset
def build_coco2014(image_set, args):
root = Path(args.coco_path)
assert root.exists(), f'provided COCO path {root} does not exist'
mode = 'instances'
PATHS = {
"train": (root / "train2014", root / "annotations" / f'{mode}_train2014.json'),
"val": (root / "val2014", root / "annotations" / f'{mode}_val2014.json'),
}
img_folder, ann_file = PATHS[image_set]
dataset = CocoDetection(img_folder, ann_file, transforms=make_coco_transforms(image_set), return_masks=args.masks)
return dataset
def build_hico(image_set, args):
root = Path(args.coco_path)
assert root.exists(), f'provided COCO path {root} does not exist'
mode = 'instances'
PATHS = {
"train": (root / "train2015", root / "annotations" / 'hico_train.json'),
"val": (root / "test2015", root / "annotations" / 'hico_test.json'),
}
img_folder, ann_file = PATHS[image_set]
dataset = CocoDetection(img_folder, ann_file, transforms=make_coco_transforms(image_set), return_masks=args.masks)
return dataset
================================================
FILE: batchformer-v2/detr/datasets/coco_eval.py
================================================
# Copyright (c) Facebook, Inc. and its affiliates. All Rights Reserved
"""
COCO evaluator that works in distributed mode.
Mostly copy-paste from https://github.com/pytorch/vision/blob/edfd5a7/references/detection/coco_eval.py
The difference is that there is less copy-pasting from pycocotools
in the end of the file, as python3 can suppress prints with contextlib
"""
import os
import contextlib
import copy
import numpy as np
import torch
from pycocotools.cocoeval import COCOeval
from pycocotools.coco import COCO
import pycocotools.mask as mask_util
from util.misc import all_gather
class CocoEvaluator(object):
def __init__(self, coco_gt, iou_types):
assert isinstance(iou_types, (list, tuple))
coco_gt = copy.deepcopy(coco_gt)
self.coco_gt = coco_gt
self.iou_types = iou_types
self.coco_eval = {}
for iou_type in iou_types:
self.coco_eval[iou_type] = COCOeval(coco_gt, iouType=iou_type)
self.img_ids = []
self.eval_imgs = {k: [] for k in iou_types}
def update(self, predictions):
img_ids = list(np.unique(list(predictions.keys())))
self.img_ids.extend(img_ids)
for iou_type in self.iou_types:
results = self.prepare(predictions, iou_type)
# suppress pycocotools prints
with open(os.devnull, 'w') as devnull:
with contextlib.redirect_stdout(devnull):
coco_dt = COCO.loadRes(self.coco_gt, results) if results else COCO()
coco_eval = self.coco_eval[iou_type]
coco_eval.cocoDt = coco_dt
coco_eval.params.imgIds = list(img_ids)
img_ids, eval_imgs = evaluate(coco_eval)
self.eval_imgs[iou_type].append(eval_imgs)
def synchronize_between_processes(self):
for iou_type in self.iou_types:
self.eval_imgs[iou_type] = np.concatenate(self.eval_imgs[iou_type], 2)
create_common_coco_eval(self.coco_eval[iou_type], self.img_ids, self.eval_imgs[iou_type])
def accumulate(self):
for coco_eval in self.coco_eval.values():
coco_eval.accumulate()
def summarize(self):
for iou_type, coco_eval in self.coco_eval.items():
print("IoU metric: {}".format(iou_type))
coco_eval.summarize()
def prepare(self, predictions, iou_type):
if iou_type == "bbox":
return self.prepare_for_coco_detection(predictions)
elif iou_type == "segm":
return self.prepare_for_coco_segmentation(predictions)
elif iou_type == "keypoints":
return self.prepare_for_coco_keypoint(predictions)
else:
raise ValueError("Unknown iou type {}".format(iou_type))
def prepare_for_coco_detection(self, predictions):
coco_results = []
for original_id, prediction in predictions.items():
if len(prediction) == 0:
continue
boxes = prediction["boxes"]
boxes = convert_to_xywh(boxes).tolist()
scores = prediction["scores"].tolist()
labels = prediction["labels"].tolist()
coco_results.extend(
[
{
"image_id": original_id,
"category_id": labels[k],
"bbox": box,
"score": scores[k],
}
for k, box in enumerate(boxes)
]
)
return coco_results
def prepare_for_coco_segmentation(self, predictions):
coco_results = []
for original_id, prediction in predictions.items():
if len(prediction) == 0:
continue
scores = prediction["scores"]
labels = prediction["labels"]
masks = prediction["masks"]
masks = masks > 0.5
scores = prediction["scores"].tolist()
labels = prediction["labels"].tolist()
rles = [
mask_util.encode(np.array(mask[0, :, :, np.newaxis], dtype=np.uint8, order="F"))[0]
for mask in masks
]
for rle in rles:
rle["counts"] = rle["counts"].decode("utf-8")
coco_results.extend(
[
{
"image_id": original_id,
"category_id": labels[k],
"segmentation": rle,
"score": scores[k],
}
for k, rle in enumerate(rles)
]
)
return coco_results
def prepare_for_coco_keypoint(self, predictions):
coco_results = []
for original_id, prediction in predictions.items():
if len(prediction) == 0:
continue
boxes = prediction["boxes"]
boxes = convert_to_xywh(boxes).tolist()
scores = prediction["scores"].tolist()
labels = prediction["labels"].tolist()
keypoints = prediction["keypoints"]
keypoints = keypoints.flatten(start_dim=1).tolist()
coco_results.extend(
[
{
"image_id": original_id,
"category_id": labels[k],
'keypoints': keypoint,
"score": scores[k],
}
for k, keypoint in enumerate(keypoints)
]
)
return coco_results
def convert_to_xywh(boxes):
xmin, ymin, xmax, ymax = boxes.unbind(1)
return torch.stack((xmin, ymin, xmax - xmin, ymax - ymin), dim=1)
def merge(img_ids, eval_imgs):
all_img_ids = all_gather(img_ids)
all_eval_imgs = all_gather(eval_imgs)
merged_img_ids = []
for p in all_img_ids:
merged_img_ids.extend(p)
merged_eval_imgs = []
for p in all_eval_imgs:
merged_eval_imgs.append(p)
merged_img_ids = np.array(merged_img_ids)
merged_eval_imgs = np.concatenate(merged_eval_imgs, 2)
# keep only unique (and in sorted order) images
merged_img_ids, idx = np.unique(merged_img_ids, return_index=True)
merged_eval_imgs = merged_eval_imgs[..., idx]
return merged_img_ids, merged_eval_imgs
def create_common_coco_eval(coco_eval, img_ids, eval_imgs):
img_ids, eval_imgs = merge(img_ids, eval_imgs)
img_ids = list(img_ids)
eval_imgs = list(eval_imgs.flatten())
coco_eval.evalImgs = eval_imgs
coco_eval.params.imgIds = img_ids
coco_eval._paramsEval = copy.deepcopy(coco_eval.params)
#################################################################
# From pycocotools, just removed the prints and fixed
# a Python3 bug about unicode not defined
#################################################################
def evaluate(self):
'''
Run per image evaluation on given images and store results (a list of dict) in self.evalImgs
:return: None
'''
# tic = time.time()
# print('Running per image evaluation...')
p = self.params
# add backward compatibility if useSegm is specified in params
if p.useSegm is not None:
p.iouType = 'segm' if p.useSegm == 1 else 'bbox'
print('useSegm (deprecated) is not None. Running {} evaluation'.format(p.iouType))
# print('Evaluate annotation type *{}*'.format(p.iouType))
p.imgIds = list(np.unique(p.imgIds))
if p.useCats:
p.catIds = list(np.unique(p.catIds))
p.maxDets = sorted(p.maxDets)
self.params = p
self._prepare()
# loop through images, area range, max detection number
catIds = p.catIds if p.useCats else [-1]
if p.iouType == 'segm' or p.iouType == 'bbox':
computeIoU = self.computeIoU
elif p.iouType == 'keypoints':
computeIoU = self.computeOks
self.ious = {
(imgId, catId): computeIoU(imgId, catId)
for imgId in p.imgIds
for catId in catIds}
evaluateImg = self.evaluateImg
maxDet = p.maxDets[-1]
evalImgs = [
evaluateImg(imgId, catId, areaRng, maxDet)
for catId in catIds
for areaRng in p.areaRng
for imgId in p.imgIds
]
# this is NOT in the pycocotools code, but could be done outside
evalImgs = np.asarray(evalImgs).reshape(len(catIds), len(p.areaRng), len(p.imgIds))
self._paramsEval = copy.deepcopy(self.params)
# toc = time.time()
# print('DONE (t={:0.2f}s).'.format(toc-tic))
return p.imgIds, evalImgs
#################################################################
# end of straight copy from pycocotools, just removing the prints
#################################################################
================================================
FILE: batchformer-v2/detr/datasets/coco_panoptic.py
================================================
# Copyright (c) Facebook, Inc. and its affiliates. All Rights Reserved
import json
from pathlib import Path
import numpy as np
import torch
from PIL import Image
from panopticapi.utils import rgb2id
from util.box_ops import masks_to_boxes
from .coco import make_coco_transforms
class CocoPanoptic:
def __init__(self, img_folder, ann_folder, ann_file, transforms=None, return_masks=True):
with open(ann_file, 'r') as f:
self.coco = json.load(f)
# sort 'images' field so that they are aligned with 'annotations'
# i.e., in alphabetical order
self.coco['images'] = sorted(self.coco['images'], key=lambda x: x['id'])
# sanity check
if "annotations" in self.coco:
for img, ann in zip(self.coco['images'], self.coco['annotations']):
assert img['file_name'][:-4] == ann['file_name'][:-4]
self.img_folder = img_folder
self.ann_folder = ann_folder
self.ann_file = ann_file
self.transforms = transforms
self.return_masks = return_masks
def __getitem__(self, idx):
ann_info = self.coco['annotations'][idx] if "annotations" in self.coco else self.coco['images'][idx]
img_path = Path(self.img_folder) / ann_info['file_name'].replace('.png', '.jpg')
ann_path = Path(self.ann_folder) / ann_info['file_name']
img = Image.open(img_path).convert('RGB')
w, h = img.size
if "segments_info" in ann_info:
masks = np.asarray(Image.open(ann_path), dtype=np.uint32)
masks = rgb2id(masks)
ids = np.array([ann['id'] for ann in ann_info['segments_info']])
masks = masks == ids[:, None, None]
masks = torch.as_tensor(masks, dtype=torch.uint8)
labels = torch.tensor([ann['category_id'] for ann in ann_info['segments_info']], dtype=torch.int64)
target = {}
target['image_id'] = torch.tensor([ann_info['image_id'] if "image_id" in ann_info else ann_info["id"]])
if self.return_masks:
target['masks'] = masks
target['labels'] = labels
target["boxes"] = masks_to_boxes(masks)
target['size'] = torch.as_tensor([int(h), int(w)])
target['orig_size'] = torch.as_tensor([int(h), int(w)])
if "segments_info" in ann_info:
for name in ['iscrowd', 'area']:
target[name] = torch.tensor([ann[name] for ann in ann_info['segments_info']])
if self.transforms is not None:
img, target = self.transforms(img, target)
return img, target
def __len__(self):
return len(self.coco['images'])
def get_height_and_width(self, idx):
img_info = self.coco['images'][idx]
height = img_info['height']
width = img_info['width']
return height, width
def build(image_set, args):
img_folder_root = Path(args.coco_path)
ann_folder_root = Path(args.coco_panoptic_path)
assert img_folder_root.exists(), f'provided COCO path {img_folder_root} does not exist'
assert ann_folder_root.exists(), f'provided COCO path {ann_folder_root} does not exist'
mode = 'panoptic'
PATHS = {
"train": ("train2017", Path("annotations") / f'{mode}_train2017.json'),
"val": ("val2017", Path("annotations") / f'{mode}_val2017.json'),
}
img_folder, ann_file = PATHS[image_set]
img_folder_path = img_folder_root / img_folder
ann_folder = ann_folder_root / f'{mode}_{img_folder}'
ann_file = ann_folder_root / ann_file
dataset = CocoPanoptic(img_folder_path, ann_folder, ann_file,
transforms=make_coco_transforms(image_set), return_masks=args.masks)
return dataset
================================================
FILE: batchformer-v2/detr/datasets/panoptic_eval.py
================================================
# Copyright (c) Facebook, Inc. and its affiliates. All Rights Reserved
import json
import os
import util.misc as utils
try:
from panopticapi.evaluation import pq_compute
except ImportError:
pass
class PanopticEvaluator(object):
def __init__(self, ann_file, ann_folder, output_dir="panoptic_eval"):
self.gt_json = ann_file
self.gt_folder = ann_folder
if utils.is_main_process():
if not os.path.exists(output_dir):
os.mkdir(output_dir)
self.output_dir = output_dir
self.predictions = []
def update(self, predictions):
for p in predictions:
with open(os.path.join(self.output_dir, p["file_name"]), "wb") as f:
f.write(p.pop("png_string"))
self.predictions += predictions
def synchronize_between_processes(self):
all_predictions = utils.all_gather(self.predictions)
merged_predictions = []
for p in all_predictions:
merged_predictions += p
self.predictions = merged_predictions
def summarize(self):
if utils.is_main_process():
json_data = {"annotations": self.predictions}
predictions_json = os.path.join(self.output_dir, "predictions.json")
with open(predictions_json, "w") as f:
f.write(json.dumps(json_data))
return pq_compute(self.gt_json, predictions_json, gt_folder=self.gt_folder, pred_folder=self.output_dir)
return None
================================================
FILE: batchformer-v2/detr/datasets/transforms.py
================================================
# Copyright (c) Facebook, Inc. and its affiliates. All Rights Reserved
"""
Transforms and data augmentation for both image + bbox.
"""
import random
import PIL
import torch
import torchvision.transforms as T
import torchvision.transforms.functional as F
from util.box_ops import box_xyxy_to_cxcywh
from util.misc import interpolate
def crop(image, target, region):
cropped_image = F.crop(image, *region)
target = target.copy()
i, j, h, w = region
# should we do something wrt the original size?
target["size"] = torch.tensor([h, w])
fields = ["labels", "area", "iscrowd"]
if "boxes" in target:
boxes = target["boxes"]
max_size = torch.as_tensor([w, h], dtype=torch.float32)
cropped_boxes = boxes - torch.as_tensor([j, i, j, i])
cropped_boxes = torch.min(cropped_boxes.reshape(-1, 2, 2), max_size)
cropped_boxes = cropped_boxes.clamp(min=0)
area = (cropped_boxes[:, 1, :] - cropped_boxes[:, 0, :]).prod(dim=1)
target["boxes"] = cropped_boxes.reshape(-1, 4)
target["area"] = area
fields.append("boxes")
if "masks" in target:
# FIXME should we update the area here if there are no boxes?
target['masks'] = target['masks'][:, i:i + h, j:j + w]
fields.append("masks")
# remove elements for which the boxes or masks that have zero area
if "boxes" in target or "masks" in target:
# favor boxes selection when defining which elements to keep
# this is compatible with previous implementation
if "boxes" in target:
cropped_boxes = target['boxes'].reshape(-1, 2, 2)
keep = torch.all(cropped_boxes[:, 1, :] > cropped_boxes[:, 0, :], dim=1)
else:
keep = target['masks'].flatten(1).any(1)
for field in fields:
target[field] = target[field][keep]
return cropped_image, target
def hflip(image, target):
flipped_image = F.hflip(image)
w, h = image.size
target = target.copy()
if "boxes" in target:
boxes = target["boxes"]
boxes = boxes[:, [2, 1, 0, 3]] * torch.as_tensor([-1, 1, -1, 1]) + torch.as_tensor([w, 0, w, 0])
target["boxes"] = boxes
if "masks" in target:
target['masks'] = target['masks'].flip(-1)
return flipped_image, target
def resize(image, target, size, max_size=None):
# size can be min_size (scalar) or (w, h) tuple
def get_size_with_aspect_ratio(image_size, size, max_size=None):
w, h = image_size
if max_size is not None:
min_original_size = float(min((w, h)))
max_original_size = float(max((w, h)))
if max_original_size / min_original_size * size > max_size:
size = int(round(max_size * min_original_size / max_original_size))
if (w <= h and w == size) or (h <= w and h == size):
return (h, w)
if w < h:
ow = size
oh = int(size * h / w)
else:
oh = size
ow = int(size * w / h)
return (oh, ow)
def get_size(image_size, size, max_size=None):
if isinstance(size, (list, tuple)):
return size[::-1]
else:
return get_size_with_aspect_ratio(image_size, size, max_size)
size = get_size(image.size, size, max_size)
rescaled_image = F.resize(image, size)
if target is None:
return rescaled_image, None
ratios = tuple(float(s) / float(s_orig) for s, s_orig in zip(rescaled_image.size, image.size))
ratio_width, ratio_height = ratios
target = target.copy()
if "boxes" in target:
boxes = target["boxes"]
scaled_boxes = boxes * torch.as_tensor([ratio_width, ratio_height, ratio_width, ratio_height])
target["boxes"] = scaled_boxes
if "area" in target:
area = target["area"]
scaled_area = area * (ratio_width * ratio_height)
target["area"] = scaled_area
h, w = size
target["size"] = torch.tensor([h, w])
if "masks" in target:
target['masks'] = interpolate(
target['masks'][:, None].float(), size, mode="nearest")[:, 0] > 0.5
return rescaled_image, target
def pad(image, target, padding):
# assumes that we only pad on the bottom right corners
padded_image = F.pad(image, (0, 0, padding[0], padding[1]))
if target is None:
return padded_image, None
target = target.copy()
# should we do something wrt the original size?
target["size"] = torch.tensor(padded_image.size[::-1])
if "masks" in target:
target['masks'] = torch.nn.functional.pad(target['masks'], (0, padding[0], 0, padding[1]))
return padded_image, target
class RandomCrop(object):
def __init__(self, size):
self.size = size
def __call__(self, img, target):
region = T.RandomCrop.get_params(img, self.size)
return crop(img, target, region)
class RandomSizeCrop(object):
def __init__(self, min_size: int, max_size: int):
self.min_size = min_size
self.max_size = max_size
def __call__(self, img: PIL.Image.Image, target: dict):
w = random.randint(self.min_size, min(img.width, self.max_size))
h = random.randint(self.min_size, min(img.height, self.max_size))
region = T.RandomCrop.get_params(img, [h, w])
return crop(img, target, region)
class CenterCrop(object):
def __init__(self, size):
self.size = size
def __call__(self, img, target):
image_width, image_height = img.size
crop_height, crop_width = self.size
crop_top = int(round((image_height - crop_height) / 2.))
crop_left = int(round((image_width - crop_width) / 2.))
return crop(img, target, (crop_top, crop_left, crop_height, crop_width))
class RandomHorizontalFlip(object):
def __init__(self, p=0.5):
self.p = p
def __call__(self, img, target):
if random.random() < self.p:
return hflip(img, target)
return img, target
class RandomResize(object):
def __init__(self, sizes, max_size=None):
assert isinstance(sizes, (list, tuple))
self.sizes = sizes
self.max_size = max_size
def __call__(self, img, target=None):
size = random.choice(self.sizes)
return resize(img, target, size, self.max_size)
class RandomPad(object):
def __init__(self, max_pad):
self.max_pad = max_pad
def __call__(self, img, target):
pad_x = random.randint(0, self.max_pad)
pad_y = random.randint(0, self.max_pad)
return pad(img, target, (pad_x, pad_y))
class RandomSelect(object):
"""
Randomly selects between transforms1 and transforms2,
with probability p for transforms1 and (1 - p) for transforms2
"""
def __init__(self, transforms1, transforms2, p=0.5):
self.transforms1 = transforms1
self.transforms2 = transforms2
self.p = p
def __call__(self, img, target):
if random.random() < self.p:
return self.transforms1(img, target)
return self.transforms2(img, target)
class ToTensor(object):
def __call__(self, img, target):
return F.to_tensor(img), target
class RandomErasing(object):
def __init__(self, *args, **kwargs):
self.eraser = T.RandomErasing(*args, **kwargs)
def __call__(self, img, target):
return self.eraser(img), target
class Normalize(object):
def __init__(self, mean, std):
self.mean = mean
self.std = std
def __call__(self, image, target=None):
image = F.normalize(image, mean=self.mean, std=self.std)
if target is None:
return image, None
target = target.copy()
h, w = image.shape[-2:]
if "boxes" in target:
boxes = target["boxes"]
boxes = box_xyxy_to_cxcywh(boxes)
boxes = boxes / torch.tensor([w, h, w, h], dtype=torch.float32)
target["boxes"] = boxes
return image, target
class Compose(object):
def __init__(self, transforms):
self.transforms = transforms
def __call__(self, image, target):
for t in self.transforms:
image, target = t(image, target)
return image, target
def __repr__(self):
format_string = self.__class__.__name__ + "("
for t in self.transforms:
format_string += "\n"
format_string += " {0}".format(t)
format_string += "\n)"
return format_string
================================================
FILE: batchformer-v2/detr/detr.pbs
================================================
#!/bin/bash
#PBS -P ZHIHOU
#PBS -l select=8:ngpus=1:ncpus=6:mem=24GB
#PBS -l walltime=48:00:00
##PBS -q alloc-dt
source activate pyt
#env
cd "$PBS_O_WORKDIR"
#python run.py --nproc_per_node 1 --master_port 29543 --nnodes 8 ./configs/base.sh &>> basebf1.out
python run.py --nproc_per_node 1 --master_port 29547 --nnodes 8 ./configs/panoptic_2.sh &>> panoptic_bf3_segm.out
#python run.py --nproc_per_node 1 --master_port 29545 --nnodes 8 ./configs/panoptic.sh &>> panoptic_bf3.out
#python run.py --nproc_per_node 1 --master_port 29546 --nnodes 7 ./configs/swin.sh &>> output_bf3_7.out
================================================
FILE: batchformer-v2/detr/engine.py
================================================
# Copyright (c) Facebook, Inc. and its affiliates. All Rights Reserved
"""
Train and eval functions used in main.py
"""
import math
import os
import sys
from typing import Iterable
import torch
import util.misc as utils
from datasets.coco_eval import CocoEvaluator
from datasets.panoptic_eval import PanopticEvaluator
def train_one_epoch(model: torch.nn.Module, criterion: torch.nn.Module,
data_loader: Iterable, optimizer: torch.optim.Optimizer,
device: torch.device, epoch: int, max_norm: float = 0):
model.train()
criterion.train()
metric_logger = utils.MetricLogger(delimiter=" ")
metric_logger.add_meter('lr', utils.SmoothedValue(window_size=1, fmt='{value:.6f}'))
metric_logger.add_meter('class_error', utils.SmoothedValue(window_size=1, fmt='{value:.2f}'))
header = 'Epoch: [{}]'.format(epoch)
print_freq = 10
for samples, targets in metric_logger.log_every(data_loader, print_freq, header):
samples = samples.to(device)
targets = [{k: v.to(device) for k, v in t.items()} for t in targets]
outputs = model(samples)
if len(targets) < len(outputs['pred_logits']):
targets = targets + targets
loss_dict = criterion(outputs, targets)
weight_dict = criterion.weight_dict
losses = sum(loss_dict[k] * weight_dict[k] for k in loss_dict.keys() if k in weight_dict)
# reduce losses over all GPUs for logging purposes
loss_dict_reduced = utils.reduce_dict(loss_dict)
loss_dict_reduced_unscaled = {f'{k}_unscaled': v
for k, v in loss_dict_reduced.items()}
loss_dict_reduced_scaled = {k: v * weight_dict[k]
for k, v in loss_dict_reduced.items() if k in weight_dict}
losses_reduced_scaled = sum(loss_dict_reduced_scaled.values())
loss_value = losses_reduced_scaled.item()
if not math.isfinite(loss_value):
print("Loss is {}, stopping training".format(loss_value))
print(loss_dict_reduced)
sys.exit(1)
optimizer.zero_grad()
losses.backward()
if max_norm > 0:
torch.nn.utils.clip_grad_norm_(model.parameters(), max_norm)
optimizer.step()
metric_logger.update(loss=loss_value, **loss_dict_reduced_scaled, **loss_dict_reduced_unscaled)
metric_logger.update(class_error=loss_dict_reduced['class_error'])
metric_logger.update(lr=optimizer.param_groups[0]["lr"])
# gather the stats from all processes
metric_logger.synchronize_between_processes()
print("Averaged stats:", metric_logger)
return {k: meter.global_avg for k, meter in metric_logger.meters.items()}
@torch.no_grad()
def evaluate(model, criterion, postprocessors, data_loader, base_ds, device, output_dir):
model.eval()
criterion.eval()
metric_logger = utils.MetricLogger(delimiter=" ")
metric_logger.add_meter('class_error', utils.SmoothedValue(window_size=1, fmt='{value:.2f}'))
header = 'Test:'
iou_types = tuple(k for k in ('segm', 'bbox') if k in postprocessors.keys())
coco_evaluator = CocoEvaluator(base_ds, iou_types)
# coco_evaluator.coco_eval[iou_types[0]].params.iouThrs = [0, 0.1, 0.5, 0.75]
panoptic_evaluator = None
if 'panoptic' in postprocessors.keys():
panoptic_evaluator = PanopticEvaluator(
data_loader.dataset.ann_file,
data_loader.dataset.ann_folder,
output_dir=os.path.join(output_dir, "panoptic_eval"),
)
for samples, targets in metric_logger.log_every(data_loader, 10, header):
samples = samples.to(device)
targets = [{k: v.to(device) for k, v in t.items()} for t in targets]
outputs = model(samples)
loss_dict = criterion(outputs, targets)
weight_dict = criterion.weight_dict
# reduce losses over all GPUs for logging purposes
loss_dict_reduced = utils.reduce_dict(loss_dict)
loss_dict_reduced_scaled = {k: v * weight_dict[k]
for k, v in loss_dict_reduced.items() if k in weight_dict}
loss_dict_reduced_unscaled = {f'{k}_unscaled': v
for k, v in loss_dict_reduced.items()}
metric_logger.update(loss=sum(loss_dict_reduced_scaled.values()),
**loss_dict_reduced_scaled,
**loss_dict_reduced_unscaled)
metric_logger.update(class_error=loss_dict_reduced['class_error'])
orig_target_sizes = torch.stack([t["orig_size"] for t in targets], dim=0)
results = postprocessors['bbox'](outputs, orig_target_sizes)
if 'segm' in postprocessors.keys():
target_sizes = torch.stack([t["size"] for t in targets], dim=0)
results = postprocessors['segm'](results, outputs, orig_target_sizes, target_sizes)
res = {target['image_id'].item(): output for target, output in zip(targets, results)}
if coco_evaluator is not None:
coco_evaluator.update(res)
if panoptic_evaluator is not None:
res_pano = postprocessors["panoptic"](outputs, target_sizes, orig_target_sizes)
for i, target in enumerate(targets):
image_id = target["image_id"].item()
file_name = f"{image_id:012d}.png"
res_pano[i]["image_id"] = image_id
res_pano[i]["file_name"] = file_name
panoptic_evaluator.update(res_pano)
# gather the stats from all processes
metric_logger.synchronize_between_processes()
print("Averaged stats:", metric_logger)
if coco_evaluator is not None:
coco_evaluator.synchronize_between_processes()
if panoptic_evaluator is not None:
panoptic_evaluator.synchronize_between_processes()
# accumulate predictions from all images
if coco_evaluator is not None:
coco_evaluator.accumulate()
coco_evaluator.summarize()
panoptic_res = None
if panoptic_evaluator is not None:
panoptic_res = panoptic_evaluator.summarize()
stats = {k: meter.global_avg for k, meter in metric_logger.meters.items()}
if coco_evaluator is not None:
if 'bbox' in postprocessors.keys():
stats['coco_eval_bbox'] = coco_evaluator.coco_eval['bbox'].stats.tolist()
if 'segm' in postprocessors.keys():
stats['coco_eval_masks'] = coco_evaluator.coco_eval['segm'].stats.tolist()
if panoptic_res is not None:
stats['PQ_all'] = panoptic_res["All"]
stats['PQ_th'] = panoptic_res["Things"]
stats['PQ_st'] = panoptic_res["Stuff"]
return stats, coco_evaluator
================================================
FILE: batchformer-v2/detr/hubconf.py
================================================
# Copyright (c) Facebook, Inc. and its affiliates. All Rights Reserved
import torch
from models.backbone import Backbone, Joiner
from models.detr import DETR, PostProcess
from models.position_encoding import PositionEmbeddingSine
from models.segmentation import DETRsegm, PostProcessPanoptic
from models.transformer import Transformer
dependencies = ["torch", "torchvision"]
def _make_detr(backbone_name: str, dilation=False, num_classes=91, mask=False):
hidden_dim = 256
backbone = Backbone(backbone_name, train_backbone=True, return_interm_layers=mask, dilation=dilation)
pos_enc = PositionEmbeddingSine(hidden_dim // 2, normalize=True)
backbone_with_pos_enc = Joiner(backbone, pos_enc)
backbone_with_pos_enc.num_channels = backbone.num_channels
transformer = Transformer(d_model=hidden_dim, return_intermediate_dec=True)
detr = DETR(backbone_with_pos_enc, transformer, num_classes=num_classes, num_queries=100)
if mask:
return DETRsegm(detr)
return detr
def detr_resnet50(pretrained=False, num_classes=91, return_postprocessor=False):
"""
DETR R50 with 6 encoder and 6 decoder layers.
Achieves 42/62.4 AP/AP50 on COCO val5k.
"""
model = _make_detr("resnet50", dilation=False, num_classes=num_classes)
if pretrained:
checkpoint = torch.hub.load_state_dict_from_url(
url="https://dl.fbaipublicfiles.com/detr/detr-r50-e632da11.pth", map_location="cpu", check_hash=True
)
model.load_state_dict(checkpoint["model"])
if return_postprocessor:
return model, PostProcess()
return model
def detr_resnet50_dc5(pretrained=False, num_classes=91, return_postprocessor=False):
"""
DETR-DC5 R50 with 6 encoder and 6 decoder layers.
The last block of ResNet-50 has dilation to increase
output resolution.
Achieves 43.3/63.1 AP/AP50 on COCO val5k.
"""
model = _make_detr("resnet50", dilation=True, num_classes=num_classes)
if pretrained:
checkpoint = torch.hub.load_state_dict_from_url(
url="https://dl.fbaipublicfiles.com/detr/detr-r50-dc5-f0fb7ef5.pth", map_location="cpu", check_hash=True
)
model.load_state_dict(checkpoint["model"])
if return_postprocessor:
return model, PostProcess()
return model
def detr_resnet101(pretrained=False, num_classes=91, return_postprocessor=False):
"""
DETR-DC5 R101 with 6 encoder and 6 decoder layers.
Achieves 43.5/63.8 AP/AP50 on COCO val5k.
"""
model = _make_detr("resnet101", dilation=False, num_classes=num_classes)
if pretrained:
checkpoint = torch.hub.load_state_dict_from_url(
url="https://dl.fbaipublicfiles.com/detr/detr-r101-2c7b67e5.pth", map_location="cpu", check_hash=True
)
model.load_state_dict(checkpoint["model"])
if return_postprocessor:
return model, PostProcess()
return model
def detr_resnet101_dc5(pretrained=False, num_classes=91, return_postprocessor=False):
"""
DETR-DC5 R101 with 6 encoder and 6 decoder layers.
The last block of ResNet-101 has dilation to increase
output resolution.
Achieves 44.9/64.7 AP/AP50 on COCO val5k.
"""
model = _make_detr("resnet101", dilation=True, num_classes=num_classes)
if pretrained:
checkpoint = torch.hub.load_state_dict_from_url(
url="https://dl.fbaipublicfiles.com/detr/detr-r101-dc5-a2e86def.pth", map_location="cpu", check_hash=True
)
model.load_state_dict(checkpoint["model"])
if return_postprocessor:
return model, PostProcess()
return model
def detr_resnet50_panoptic(
pretrained=False, num_classes=250, threshold=0.85, return_postprocessor=False
):
"""
DETR R50 with 6 encoder and 6 decoder layers.
Achieves 43.4 PQ on COCO val5k.
threshold is the minimum confidence required for keeping segments in the prediction
"""
model = _make_detr("resnet50", dilation=False, num_classes=num_classes, mask=True)
is_thing_map = {i: i <= 90 for i in range(250)}
if pretrained:
checkpoint = torch.hub.load_state_dict_from_url(
url="https://dl.fbaipublicfiles.com/detr/detr-r50-panoptic-00ce5173.pth",
map_location="cpu",
check_hash=True,
)
model.load_state_dict(checkpoint["model"])
if return_postprocessor:
return model, PostProcessPanoptic(is_thing_map, threshold=threshold)
return model
def detr_resnet50_dc5_panoptic(
pretrained=False, num_classes=250, threshold=0.85, return_postprocessor=False
):
"""
DETR-DC5 R50 with 6 encoder and 6 decoder layers.
The last block of ResNet-50 has dilation to increase
output resolution.
Achieves 44.6 on COCO val5k.
threshold is the minimum confidence required for keeping segments in the prediction
"""
model = _make_detr("resnet50", dilation=True, num_classes=num_classes, mask=True)
is_thing_map = {i: i <= 90 for i in range(250)}
if pretrained:
checkpoint = torch.hub.load_state_dict_from_url(
url="https://dl.fbaipublicfiles.com/detr/detr-r50-dc5-panoptic-da08f1b1.pth",
map_location="cpu",
check_hash=True,
)
model.load_state_dict(checkpoint["model"])
if return_postprocessor:
return model, PostProcessPanoptic(is_thing_map, threshold=threshold)
return model
def detr_resnet101_panoptic(
pretrained=False, num_classes=250, threshold=0.85, return_postprocessor=False
):
"""
DETR-DC5 R101 with 6 encoder and 6 decoder layers.
Achieves 45.1 PQ on COCO val5k.
threshold is the minimum confidence required for keeping segments in the prediction
"""
model = _make_detr("resnet101", dilation=False, num_classes=num_classes, mask=True)
is_thing_map = {i: i <= 90 for i in range(250)}
if pretrained:
checkpoint = torch.hub.load_state_dict_from_url(
url="https://dl.fbaipublicfiles.com/detr/detr-r101-panoptic-40021d53.pth",
map_location="cpu",
check_hash=True,
)
model.load_state_dict(checkpoint["model"])
if return_postprocessor:
return model, PostProcessPanoptic(is_thing_map, threshold=threshold)
return model
================================================
FILE: batchformer-v2/detr/main.py
================================================
# Copyright (c) Facebook, Inc. and its affiliates. All Rights Reserved
import argparse
import datetime
import json
import random
import time
from pathlib import Path
import numpy as np
import torch
from torch.utils.data import DataLoader, DistributedSampler
import datasets
import util.misc as utils
from datasets import build_dataset, get_coco_api_from_dataset
from engine import evaluate, train_one_epoch
from models import build_model
def get_args_parser():
parser = argparse.ArgumentParser('Set transformer detector', add_help=False)
parser.add_argument('--lr', default=1e-4, type=float)
parser.add_argument('--lr_backbone', default=1e-5, type=float)
parser.add_argument('--batch_size', default=2, type=int)
parser.add_argument('--weight_decay', default=1e-4, type=float)
parser.add_argument('--epochs', default=300, type=int)
parser.add_argument('--lr_drop', default=200, type=int)
parser.add_argument('--clip_max_norm', default=0.1, type=float,
help='gradient clipping max norm')
# Model parameters
parser.add_argument('--frozen_weights', type=str, default=None,
help="Path to the pretrained model. If set, only the mask head will be trained")
parser.add_argument('--pretrained', type=str, default=None,
help="Path to the pretrained model. If set, only the mask head will be trained")
# * Backbone
parser.add_argument('--backbone', default='resnet50', type=str,
help="Name of the convolutional backbone to use")
parser.add_argument('--dilation', action='store_true',
help="If true, we replace stride with dilation in the last convolutional block (DC5)")
parser.add_argument('--position_embedding', default='sine', type=str, choices=('sine', 'learned'),
help="Type of positional embedding to use on top of the image features")
# * Transformer
parser.add_argument('--enc_layers', default=6, type=int,
help="Number of encoding layers in the transformer")
parser.add_argument('--dec_layers', default=6, type=int,
help="Number of decoding layers in the transformer")
parser.add_argument('--dim_feedforward', default=2048, type=int,
help="Intermediate size of the feedforward layers in the transformer blocks")
parser.add_argument('--hidden_dim', default=256, type=int,
help="Size of the embeddings (dimension of the transformer)")
parser.add_argument('--dropout', default=0.1, type=float,
help="Dropout applied in the transformer")
parser.add_argument('--nheads', default=8, type=int,
help="Number of attention heads inside the transformer's attentions")
parser.add_argument('--num_queries', default=100, type=int,
help="Number of query slots")
parser.add_argument('--pre_norm', action='store_true')
# * Segmentation
parser.add_argument('--masks', action='store_true',
help="Train segmentation head if the flag is provided")
# Loss
parser.add_argument('--no_aux_loss', dest='aux_loss', action='store_false',
help="Disables auxiliary decoding losses (loss at each layer)")
# * Matcher
parser.add_argument('--set_cost_class', default=1, type=float,
help="Class coefficient in the matching cost")
parser.add_argument('--set_cost_bbox', default=5, type=float,
help="L1 box coefficient in the matching cost")
parser.add_argument('--set_cost_giou', default=2, type=float,
help="giou box coefficient in the matching cost")
# * Loss coefficients
parser.add_argument('--mask_loss_coef', default=1, type=float)
parser.add_argument('--dice_loss_coef', default=1, type=float)
parser.add_argument('--bbox_loss_coef', default=5, type=float)
parser.add_argument('--giou_loss_coef', default=2, type=float)
parser.add_argument('--eos_coef', default=0.1, type=float,
help="Relative classification weight of the no-object class")
parser.add_argument('--focal_alpha', default=0., type=float)
# dataset parameters
parser.add_argument('--dataset_file', default='coco')
parser.add_argument('--coco_path', type=str)
parser.add_argument('--coco_panoptic_path', type=str)
parser.add_argument('--remove_difficult', action='store_true')
parser.add_argument('--output_dir', default='',
help='path where to save, empty for no saving')
parser.add_argument('--device', default='cuda',
help='device to use for training / testing')
parser.add_argument('--seed', default=42, type=int)
parser.add_argument('--resume', default='', help='resume from checkpoint')
parser.add_argument('--start_epoch', default=0, type=int, metavar='N',
help='start epoch')
parser.add_argument('--eval', action='store_true')
parser.add_argument('--num_workers', default=2, type=int)
parser.add_argument('--bf', default=0, type=int)
parser.add_argument('--base_bf', default=0, type=int)
parser.add_argument('--bf_lr', default=1., type=float)
parser.add_argument('--bt_decay', default=1e-4, type=float)
parser.add_argument('--share_bf', default=1, type=int)
parser.add_argument('--start_idx', default=0, type=int)
parser.add_argument('--insert_idx', action='append', type=int,
help='insert idx list')
parser.add_argument('--use_checkpoint', action='store_true')
# distributed training parameters
parser.add_argument('--world_size', default=1, type=int,
help='number of distributed processes')
parser.add_argument('--dist_url', default='env://', help='url used to set up distributed training')
return parser
def main(args):
utils.init_distributed_mode(args)
if args.insert_idx is None or len(args.insert_idx) == 0:
args.insert_idx = [i for i in range(args.start_idx, args.enc_layers)]
print("git:\n {}\n".format(utils.get_sha()))
if args.frozen_weights is not None:
assert args.masks, "Frozen training is meant for segmentation only"
print(args)
device = torch.device(args.device)
# fix the seed for reproducibility
seed = args.seed + utils.get_rank()
torch.manual_seed(seed)
np.random.seed(seed)
random.seed(seed)
model, criterion, postprocessors = build_model(args)
model.to(device)
model_without_ddp = model
if args.distributed:
model = torch.nn.parallel.DistributedDataParallel(model, device_ids=[args.gpu], find_unused_parameters=False)
model_without_ddp = model.module
n_parameters = sum(p.numel() for p in model.parameters() if p.requires_grad)
print('number of params:', n_parameters)
param_dicts = [
{"params": [p for n, p in model_without_ddp.named_parameters() if "backbone" not in n and p.requires_grad and "bf" not in n]},
{"params": [p for n, p in model_without_ddp.named_parameters() if "backbone" not in n and "bf" in n and p.requires_grad],
"lr": args.lr*args.bf_lr, 'weight_decay': args.bt_decay},
{
"params": [p for n, p in model_without_ddp.named_parameters() if "backbone" in n and p.requires_grad],
"lr": args.lr_backbone,
},
]
print([n for n, p in model_without_ddp.named_parameters() if "backbone" not in n and "bf" in n and p.requires_grad])
optimizer = torch.optim.AdamW(param_dicts, lr=args.lr,
weight_decay=args.weight_decay)
lr_scheduler = torch.optim.lr_scheduler.StepLR(optimizer, args.lr_drop)
dataset_train = build_dataset(image_set='train', args=args)
dataset_val = build_dataset(image_set='val', args=args)
if args.distributed:
sampler_train = DistributedSampler(dataset_train)
sampler_val = DistributedSampler(dataset_val, shuffle=False)
else:
sampler_train = torch.utils.data.RandomSampler(dataset_train)
sampler_val = torch.utils.data.SequentialSampler(dataset_val)
batch_sampler_train = torch.utils.data.BatchSampler(
sampler_train, args.batch_size, drop_last=True)
data_loader_train = DataLoader(dataset_train, batch_sampler=batch_sampler_train,
collate_fn=utils.collate_fn, num_workers=args.num_workers)
data_loader_val = DataLoader(dataset_val, args.batch_size, sampler=sampler_val,
drop_last=False, collate_fn=utils.collate_fn, num_workers=args.num_workers)
if args.dataset_file == "coco_panoptic":
# We also evaluate AP during panoptic training, on original coco DS
coco_val = datasets.coco.build("val", args)
base_ds = get_coco_api_from_dataset(coco_val)
else:
base_ds = get_coco_api_from_dataset(dataset_val)
if args.frozen_weights is not None:
checkpoint = torch.load(args.frozen_weights, map_location='cpu')
model_without_ddp.detr.load_state_dict(checkpoint['model'], False)
output_dir = Path(args.output_dir)
if args.resume:
if args.resume.startswith('https'):
checkpoint = torch.hub.load_state_dict_from_url(
args.resume, map_location='cpu', check_hash=True)
else:
checkpoint = torch.load(args.resume, map_location='cpu')
model_without_ddp.load_state_dict(checkpoint['model'])
if not args.eval and 'optimizer' in checkpoint and 'lr_scheduler' in checkpoint and 'epoch' in checkpoint:
optimizer.load_state_dict(checkpoint['optimizer'])
lr_scheduler.load_state_dict(checkpoint['lr_scheduler'])
lr_scheduler.step_size = args.lr_drop
lr_scheduler.base_lrs = list(map(lambda group: group['initial_lr'], optimizer.param_groups))
lr_scheduler.step(lr_scheduler.last_epoch)
args.start_epoch = checkpoint['epoch'] + 1
elif args.pretrained:
checkpoint = torch.load(args.pretrained, map_location='cpu')
if 'model' in checkpoint:
checkpoint = checkpoint['model']
if args.eval:
model_without_ddp.load_state_dict(checkpoint, strict=True)
else:
print(type(checkpoint))
new_checkpoint = {}
for k in checkpoint:
k1 = k.replace('backbone.bottom_up.', 'backbone.0.body.')
if not k1.__contains__('backbone.0.body.'):
k1 = 'backbone.0.body.'+k1
new_checkpoint[k1] = checkpoint[k]
res = model_without_ddp.load_state_dict(new_checkpoint, strict=False)
# print(res)
for k in res.missing_keys:
if not k.__contains__('transformer.encoder') and not k.__contains__('transformer.decoder'):
print(k)
print('unexcep======')
for k in res.unexpected_keys:
if not k.__contains__('transformer.encoder') and not k.__contains__('transformer.decoder'):
print(k)
if args.eval:
test_stats, coco_evaluator = evaluate(model, criterion, postprocessors,
data_loader_val, base_ds, device, args.output_dir)
if args.output_dir:
utils.save_on_master(coco_evaluator.coco_eval["bbox"].eval, output_dir / "eval.pth")
return
print("Start training")
start_time = time.time()
for epoch in range(args.start_epoch, args.epochs):
for n,p in model_without_ddp.named_parameters():
print(n, p.requires_grad)
if args.distributed:
sampler_train.set_epoch(epoch)
train_stats = train_one_epoch(
model, criterion, data_loader_train, optimizer, device, epoch,
args.clip_max_norm)
lr_scheduler.step()
if args.output_dir:
checkpoint_paths = [output_dir / 'checkpoint.pth']
# extra checkpoint before LR drop and every 100 epochs
if (epoch + 1) % args.lr_drop == 0 or (epoch + 1) % 10 == 0:
checkpoint_paths.append(output_dir / f'checkpoint{epoch:04}.pth')
for checkpoint_path in checkpoint_paths:
utils.save_on_master({
'model': model_without_ddp.state_dict(),
'optimizer': optimizer.state_dict(),
'lr_scheduler': lr_scheduler.state_dict(),
'epoch': epoch,
'args': args,
}, checkpoint_path)
test_stats, coco_evaluator = evaluate(
model, criterion, postprocessors, data_loader_val, base_ds, device, args.output_dir
)
log_stats = {**{f'train_{k}': v for k, v in train_stats.items()},
**{f'test_{k}': v for k, v in test_stats.items()},
'epoch': epoch,
'n_parameters': n_parameters}
if args.output_dir and utils.is_main_process():
with (output_dir / "log.txt").open("a") as f:
f.write(json.dumps(log_stats) + "\n")
# for evaluation logs
if coco_evaluator is not None:
(output_dir / 'eval').mkdir(exist_ok=True)
if "bbox" in coco_evaluator.coco_eval:
filenames = ['latest.pth']
if epoch % 50 == 0:
filenames.append(f'{epoch:03}.pth')
for name in filenames:
torch.save(coco_evaluator.coco_eval["bbox"].eval,
output_dir / "eval" / name)
total_time = time.time() - start_time
total_time_str = str(datetime.timedelta(seconds=int(total_time)))
print('Training time {}'.format(total_time_str))
if __name__ == '__main__':
parser = argparse.ArgumentParser('DETR training and evaluation script', parents=[get_args_parser()])
args = parser.parse_args()
if args.output_dir:
Path(args.output_dir).mkdir(parents=True, exist_ok=True)
main(args)
================================================
FILE: batchformer-v2/detr/models/__init__.py
================================================
# Copyright (c) Facebook, Inc. and its affiliates. All Rights Reserved
from .detr import build
def build_model(args):
return build(args)
================================================
FILE: batchformer-v2/detr/models/backbone.py
================================================
# Copyright (c) Facebook, Inc. and its affiliates. All Rights Reserved
"""
Backbone modules.
"""
from collections import OrderedDict
import torch
import torch.nn.functional as F
import torchvision
from torch import nn
from torchvision.models._utils import IntermediateLayerGetter
from typing import Dict, List
from util.misc import NestedTensor, is_main_process
from .position_encoding import build_position_encoding
class FrozenBatchNorm2d(torch.nn.Module):
"""
BatchNorm2d where the batch statistics and the affine parameters are fixed.
Copy-paste from torchvision.misc.ops with added eps before rqsrt,
without which any other models than torchvision.models.resnet[18,34,50,101]
produce nans.
"""
def __init__(self, n):
super(FrozenBatchNorm2d, self).__init__()
self.register_buffer("weight", torch.ones(n))
self.register_buffer("bias", torch.zeros(n))
self.register_buffer("running_mean", torch.zeros(n))
self.register_buffer("running_var", torch.ones(n))
def _load_from_state_dict(self, state_dict, prefix, local_metadata, strict,
missing_keys, unexpected_keys, error_msgs):
num_batches_tracked_key = prefix + 'num_batches_tracked'
if num_batches_tracked_key in state_dict:
del state_dict[num_batches_tracked_key]
super(FrozenBatchNorm2d, self)._load_from_state_dict(
state_dict, prefix, local_metadata, strict,
missing_keys, unexpected_keys, error_msgs)
def forward(self, x):
# move reshapes to the beginning
# to make it fuser-friendly
w = self.weight.reshape(1, -1, 1, 1)
b = self.bias.reshape(1, -1, 1, 1)
rv = self.running_var.reshape(1, -1, 1, 1)
rm = self.running_mean.reshape(1, -1, 1, 1)
eps = 1e-5
scale = w * (rv + eps).rsqrt()
bias = b - rm * scale
return x * scale + bias
class BackboneBase(nn.Module):
def __init__(self, backbone: nn.Module, train_backbone: bool, num_channels: int, return_interm_layers: bool,
args=None):
super().__init__()
for name, parameter in backbone.named_parameters():
if not train_backbone or 'layer2' not in name and 'layer3' not in name and 'layer4' not in name:
parameter.requires_grad_(False)
if return_interm_layers:
return_layers = {"layer1": "0", "layer2": "1", "layer3": "2", "layer4": "3"}
else:
return_layers = {'layer4': "0"}
self.body = IntermediateLayerGetter(backbone, return_layers=return_layers)
self.num_channels = num_channels
self.use_checkpoint = args.use_checkpoint
def forward(self, tensor_list: NestedTensor):
xs = self.body(tensor_list.tensors)
out: Dict[str, NestedTensor] = {}
for name, x in xs.items():
m = tensor_list.mask
assert m is not None
mask = F.interpolate(m[None].float(), size=x.shape[-2:]).to(torch.bool)[0]
out[name] = NestedTensor(x, mask)
return out
class Backbone(BackboneBase):
"""ResNet backbone with frozen BatchNorm."""
def __init__(self, name: str,
train_backbone: bool,
return_interm_layers: bool,
dilation: bool,
args=None):
backbone = getattr(torchvision.models, name)(
replace_stride_with_dilation=[False, False, dilation],
pretrained=is_main_process(), norm_layer=FrozenBatchNorm2d)
num_channels = 512 if name in ('resnet18', 'resnet34') else 2048
super().__init__(backbone, train_backbone, num_channels, return_interm_layers, args)
class Joiner(nn.Sequential):
def __init__(self, backbone, position_embedding):
super().__init__(backbone, position_embedding)
def forward(self, tensor_list: NestedTensor):
xs = self[0](tensor_list)
out: List[NestedTensor] = []
pos = []
for name, x in xs.items():
out.append(x)
# position encoding
pos.append(self[1](x).to(x.tensors.dtype))
return out, pos
def build_backbone(args):
position_embedding = build_position_encoding(args)
train_backbone = args.lr_backbone > 0
return_interm_layers = args.masks
backbone = Backbone(args.backbone, train_backbone, return_interm_layers, args.dilation, args)
model = Joiner(backbone, position_embedding)
model.num_channels = backbone.num_channels
return model
class SwinBackbone(nn.Module):
def __init__(self, name: str,
train_backbone: bool,
return_interm_layers: bool,
dilation: bool):
super().__init__()
from models.swin_transformer1 import SwinTransformer
self.body = SwinTransformer(
patch_size=4,
in_chans=3,
embed_dim=96,
depths=[2, 2, 6, 2],
num_heads=[3, 6, 12, 24],
window_size=7,
mlp_ratio=4,
qkv_bias=True,
qk_scale=None,
drop_rate=0.,
attn_drop_rate=0.,
drop_path_rate=0.2,
norm_layer=nn.LayerNorm,
ape=False,
patch_norm=True,
frozen_stages=-1,
out_features=["stage2", "stage3", "stage4", "stage5"]
)
if return_interm_layers:
return_layers = {"stage2": "0", "stage3": "1", "stage4": "2", "stage5": "3"}
else:
return_layers = {'stage5': "0"}
num_channels = 512 if name in ('resnet18', 'resnet34') else 2048
self.num_channels = 768
def forward(self, tensor_list: NestedTensor):
xs = self.body(tensor_list.tensors)
out: Dict[str, NestedTensor] = {}
for name, x in xs.items():
m = tensor_list.mask
assert m is not None
mask = F.interpolate(m[None].float(), size=x.shape[-2:]).to(torch.bool)[0]
out[name] = NestedTensor(x, mask)
return out
def build_swin_backbone(args):
from models.position_encoding import build_position_encoding
position_embedding = build_position_encoding(args)
train_backbone = args.lr_backbone > 0
return_interm_layers = args.masks
backbone = SwinBackbone(args.backbone, train_backbone, return_interm_layers, args.dilation)
model = Joiner(backbone, position_embedding)
model.num_channels = backbone.num_channels
return model
================================================
FILE: batchformer-v2/detr/models/detr.py
================================================
# Copyright (c) Facebook, Inc. and its affiliates. All Rights Reserved
"""
DETR model and criterion classes.
"""
import torch
import torch.nn.functional as F
from torch import nn
from util import box_ops
from util.misc import (NestedTensor, nested_tensor_from_tensor_list,
accuracy, get_world_size, interpolate,
is_dist_avail_and_initialized)
from .backbone import build_backbone
from .matcher import build_matcher
from .segmentation import (DETRsegm, PostProcessPanoptic, PostProcessSegm,
dice_loss, sigmoid_focal_loss)
from .transformer import build_transformer
class DETR(nn.Module):
""" This is the DETR module that performs object detection """
def __init__(self, backbone, transformer, num_classes, num_queries, aux_loss=False, args=None):
""" Initializes the model.
Parameters:
backbone: torch module of the backbone to be used. See backbone.py
transformer: torch module of the transformer architecture. See transformer.py
num_classes: number of object classes
num_queries: number of object queries, ie detection slot. This is the maximal number of objects
DETR can detect in a single image. For COCO, we recommend 100 queries.
aux_loss: True if auxiliary decoding losses (loss at each decoder layer) are to be used.
"""
super().__init__()
self.num_queries = num_queries
self.transformer = transformer
hidden_dim = transformer.d_model
self.class_embed = nn.Linear(hidden_dim, num_classes + 1)
self.bbox_embed = MLP(hidden_dim, hidden_dim, 4, 3)
self.query_embed = nn.Embedding(num_queries, hidden_dim)
self.input_proj = nn.Conv2d(backbone.num_channels, hidden_dim, kernel_size=1)
self.backbone = backbone
self.aux_loss = aux_loss
self.use_checkpoint=args.use_checkpoint
def forward(self, samples: NestedTensor):
""" The forward expects a NestedTensor, which consists of:
- samples.tensor: batched images, of shape [batch_size x 3 x H x W]
- samples.mask: a binary mask of shape [batch_size x H x W], containing 1 on padded pixels
It returns a dict with the following elements:
- "pred_logits": the classification logits (including no-object) for all queries.
Shape= [batch_size x num_queries x (num_classes + 1)]
- "pred_boxes": The normalized boxes coordinates for all queries, represented as
(center_x, center_y, height, width). These values are normalized in [0, 1],
relative to the size of each individual image (disregarding possible padding).
See PostProcess for information on how to retrieve the unnormalized bounding box.
- "aux_outputs": Optional, only returned when auxilary losses are activated. It is a list of
dictionnaries containing the two above keys for each decoder layer.
"""
if isinstance(samples, (list, torch.Tensor)):
samples = nested_tensor_from_tensor_list(samples)
features, pos = self.backbone(samples)
src, mask = features[-1].decompose()
assert mask is not None
hs = self.transformer(self.input_proj(src), mask, self.query_embed.weight, pos[-1])[0]
outputs_class = self.class_embed(hs)
outputs_coord = self.bbox_embed(hs).sigmoid()
out = {'pred_logits': outputs_class[-1], 'pred_boxes': outputs_coord[-1]}
if self.aux_loss:
out['aux_outputs'] = self._set_aux_loss(outputs_class, outputs_coord)
return out
@torch.jit.unused
def _set_aux_loss(self, outputs_class, outputs_coord):
# this is a workaround to make torchscript happy, as torchscript
# doesn't support dictionary with non-homogeneous values, such
# as a dict having both a Tensor and a list.
return [{'pred_logits': a, 'pred_boxes': b}
for a, b in zip(outputs_class[:-1], outputs_coord[:-1])]
class SetCriterion(nn.Module):
""" This class computes the loss for DETR.
The process happens in two steps:
1) we compute hungarian assignment between ground truth boxes and the outputs of the model
2) we supervise each pair of matched ground-truth / prediction (supervise class and box)
"""
def __init__(self, num_classes, matcher, weight_dict, eos_coef, losses, focal_alpha=0.25):
""" Create the criterion.
Parameters:
num_classes: number of object categories, omitting the special no-object category
matcher: module able to compute a matching between targets and proposals
weight_dict: dict containing as key the names of the losses and as values their relative weight.
eos_coef: relative classification weight applied to the no-object category
losses: list of all the losses to be applied. See get_loss for list of available losses.
"""
super().__init__()
self.num_classes = num_classes
self.matcher = matcher
self.weight_dict = weight_dict
self.eos_coef = eos_coef
self.losses = losses
self.focal_alpha = focal_alpha
empty_weight = torch.ones(self.num_classes + 1)
empty_weight[-1] = self.eos_coef
self.register_buffer('empty_weight', empty_weight)
def loss_labels(self, outputs, targets, indices, num_boxes, log=True):
"""Classification loss (NLL)
targets dicts must contain the key "labels" containing a tensor of dim [nb_target_boxes]
"""
assert 'pred_logits' in outputs
src_logits = outputs['pred_logits']
idx = self._get_src_permutation_idx(indices)
target_classes_o = torch.cat([t["labels"][J] for t, (_, J) in zip(targets, indices)])
target_classes = torch.full(src_logits.shape[:2], self.num_classes,
dtype=torch.int64, device=src_logits.device)
target_classes[idx] = target_classes_o
if self.focal_alpha > 0:
target_classes_onehot = torch.zeros([src_logits.shape[0], src_logits.shape[1], src_logits.shape[2] + 1],
dtype=src_logits.dtype, layout=src_logits.layout, device=src_logits.device)
target_classes_onehot.scatter_(2, target_classes.unsqueeze(-1), 1)
target_classes_onehot = target_classes_onehot[:,:,:-1]
loss_ce = sigmoid_focal_loss(src_logits, target_classes_onehot, num_boxes, alpha=self.focal_alpha, gamma=2) * src_logits.shape[1]
else:
loss_ce = F.cross_entropy(src_logits.transpose(1, 2), target_classes, self.empty_weight)
losses = {'loss_ce': loss_ce}
if log:
# TODO this should probably be a separate loss, not hacked in this one here
losses['class_error'] = 100 - accuracy(src_logits[idx], target_classes_o)[0]
return losses
@torch.no_grad()
def loss_cardinality(self, outputs, targets, indices, num_boxes):
""" Compute the cardinality error, ie the absolute error in the number of predicted non-empty boxes
This is not really a loss, it is intended for logging purposes only. It doesn't propagate gradients
"""
pred_logits = outputs['pred_logits']
device = pred_logits.device
tgt_lengths = torch.as_tensor([len(v["labels"]) for v in targets], device=device)
# Count the number of predictions that are NOT "no-object" (which is the last class)
card_pred = (pred_logits.argmax(-1) != pred_logits.shape[-1] - 1).sum(1)
card_err = F.l1_loss(card_pred.float(), tgt_lengths.float())
losses = {'cardinality_error': card_err}
return losses
def loss_boxes(self, outputs, targets, indices, num_boxes):
"""Compute the losses related to the bounding boxes, the L1 regression loss and the GIoU loss
targets dicts must contain the key "boxes" containing a tensor of dim [nb_target_boxes, 4]
The target boxes are expected in format (center_x, center_y, w, h), normalized by the image size.
"""
assert 'pred_boxes' in outputs
idx = self._get_src_permutation_idx(indices)
src_boxes = outputs['pred_boxes'][idx]
target_boxes = torch.cat([t['boxes'][i] for t, (_, i) in zip(targets, indices)], dim=0)
loss_bbox = F.l1_loss(src_boxes, target_boxes, reduction='none')
losses = {}
losses['loss_bbox'] = loss_bbox.sum() / num_boxes
loss_giou = 1 - torch.diag(box_ops.generalized_box_iou(
box_ops.box_cxcywh_to_xyxy(src_boxes),
box_ops.box_cxcywh_to_xyxy(target_boxes)))
losses['loss_giou'] = loss_giou.sum() / num_boxes
return losses
def loss_masks(self, outputs, targets, indices, num_boxes):
"""Compute the losses related to the masks: the focal loss and the dice loss.
targets dicts must contain the key "masks" containing a tensor of dim [nb_target_boxes, h, w]
"""
assert "pred_masks" in outputs
src_idx = self._get_src_permutation_idx(indices)
tgt_idx = self._get_tgt_permutation_idx(indices)
src_masks = outputs["pred_masks"]
src_masks = src_masks[src_idx]
masks = [t["masks"] for t in targets]
# TODO use valid to mask invalid areas due to padding in loss
target_masks, valid = nested_tensor_from_tensor_list(masks).decompose()
target_masks = target_masks.to(src_masks)
target_masks = target_masks[tgt_idx]
# upsample predictions to the target size
src_masks = interpolate(src_masks[:, None], size=target_masks.shape[-2:],
mode="bilinear", align_corners=False)
src_masks = src_masks[:, 0].flatten(1)
target_masks = target_masks.flatten(1)
target_masks = target_masks.view(src_masks.shape)
losses = {
"loss_mask": sigmoid_focal_loss(src_masks, target_masks, num_boxes),
"loss_dice": dice_loss(src_masks, target_masks, num_boxes),
}
return losses
def _get_src_permutation_idx(self, indices):
# permute predictions following indices
batch_idx = torch.cat([torch.full_like(src, i) for i, (src, _) in enumerate(indices)])
src_idx = torch.cat([src for (src, _) in indices])
return batch_idx, src_idx
def _get_tgt_permutation_idx(self, indices):
# permute targets following indices
batch_idx = torch.cat([torch.full_like(tgt, i) for i, (_, tgt) in enumerate(indices)])
tgt_idx = torch.cat([tgt for (_, tgt) in indices])
return batch_idx, tgt_idx
def get_loss(self, loss, outputs, targets, indices, num_boxes, **kwargs):
loss_map = {
'labels': self.loss_labels,
'cardinality': self.loss_cardinality,
'boxes': self.loss_boxes,
'masks': self.loss_masks
}
assert loss in loss_map, f'do you really want to compute {loss} loss?'
return loss_map[loss](outputs, targets, indices, num_boxes, **kwargs)
def forward(self, outputs, targets):
""" This performs the loss computation.
Parameters:
outputs: dict of tensors, see the output specification of the model for the format
targets: list of dicts, such that len(targets) == batch_size.
The expected keys in each dict depends on the losses applied, see each loss' doc
"""
outputs_without_aux = {k: v for k, v in outputs.items() if k != 'aux_outputs'}
# Retrieve the matching between the outputs of the last layer and the targets
indices = self.matcher(outputs_without_aux, targets)
# Compute the average number of target boxes accross all nodes, for normalization purposes
num_boxes = sum(len(t["labels"]) for t in targets)
num_boxes = torch.as_tensor([num_boxes], dtype=torch.float, device=next(iter(outputs.values())).device)
if is_dist_avail_and_initialized():
torch.distributed.all_reduce(num_boxes)
num_boxes = torch.clamp(num_boxes / get_world_size(), min=1).item()
# Compute all the requested losses
losses = {}
for loss in self.losses:
losses.update(self.get_loss(loss, outputs, targets, indices, num_boxes))
# In case of auxiliary losses, we repeat this process with the output of each intermediate layer.
if 'aux_outputs' in outputs:
for i, aux_outputs in enumerate(outputs['aux_outputs']):
indices = self.matcher(aux_outputs, targets)
for loss in self.losses:
if loss == 'masks':
# Intermediate masks losses are too costly to compute, we ignore them.
continue
kwargs = {}
if loss == 'labels':
# Logging is enabled only for the last layer
kwargs = {'log': False}
l_dict = self.get_loss(loss, aux_outputs, targets, indices, num_boxes, **kwargs)
l_dict = {k + f'_{i}': v for k, v in l_dict.items()}
losses.update(l_dict)
return losses
class PostProcess(nn.Module):
""" This module converts the model's output into the format expected by the coco api"""
@torch.no_grad()
def forward(self, outputs, target_sizes):
""" Perform the computation
Parameters:
outputs: raw outputs of the model
target_sizes: tensor of dimension [batch_size x 2] containing the size of each images of the batch
For evaluation, this must be the original image size (before any data augmentation)
For visualization, this should be the image size after data augment, but before padding
"""
out_logits, out_bbox = outputs['pred_logits'], outputs['pred_boxes']
assert len(out_logits) == len(target_sizes)
assert target_sizes.shape[1] == 2
prob = F.softmax(out_logits, -1)
scores, labels = prob[..., :-1].max(-1)
# convert to [x0, y0, x1, y1] format
boxes = box_ops.box_cxcywh_to_xyxy(out_bbox)
# and from relative [0, 1] to absolute [0, height] coordinates
img_h, img_w = target_sizes.unbind(1)
scale_fct = torch.stack([img_w, img_h, img_w, img_h], dim=1)
boxes = boxes * scale_fct[:, None, :]
results = [{'scores': s, 'labels': l, 'boxes': b} for s, l, b in zip(scores, labels, boxes)]
return results
class MLP(nn.Module):
""" Very simple multi-layer perceptron (also called FFN)"""
def __init__(self, input_dim, hidden_dim, output_dim, num_layers):
super().__init__()
self.num_layers = num_layers
h = [hidden_dim] * (num_layers - 1)
self.layers = nn.ModuleList(nn.Linear(n, k) for n, k in zip([input_dim] + h, h + [output_dim]))
def forward(self, x):
for i, layer in enumerate(self.layers):
x = F.relu(layer(x)) if i < self.num_layers - 1 else layer(x)
return x
def build(args):
# the `num_classes` naming here is somewhat misleading.
# it indeed corresponds to `max_obj_id + 1`, where max_obj_id
# is the maximum id for a class in your dataset. For example,
# COCO has a max_obj_id of 90, so we pass `num_classes` to be 91.
# As another example, for a dataset that has a single class with id 1,
# you should pass `num_classes` to be 2 (max_obj_id + 1).
# For more details on this, check the following discussion
# https://github.com/facebookresearch/detr/issues/108#issuecomment-650269223
num_classes = 20 if args.dataset_file != 'coco' else 91
if args.dataset_file == 'hico':
num_classes = 91
if args.dataset_file == "coco_panoptic":
# for panoptic, we just add a num_classes that is large enough to hold
# max_obj_id + 1, but the exact value doesn't really matter
num_classes = 250
device = torch.device(args.device)
if args.backbone.__contains__('swin'):
from .backbone import build_swin_backbone
backbone = build_swin_backbone(args)
else:
backbone = build_backbone(args)
transformer = build_transformer(args)
model = DETR(
backbone,
transformer,
num_classes=num_classes,
num_queries=args.num_queries,
aux_loss=args.aux_loss,
args=args,
)
if args.masks:
model = DETRsegm(model, freeze_detr=(args.frozen_weights is not None))
matcher = build_matcher(args)
weight_dict = {'loss_ce': 1, 'loss_bbox': args.bbox_loss_coef}
weight_dict['loss_giou'] = args.giou_loss_coef
if args.masks:
weight_dict["loss_mask"] = args.mask_loss_coef
weight_dict["loss_dice"] = args.dice_loss_coef
# TODO this is a hack
if args.aux_loss:
aux_weight_dict = {}
for i in range(args.dec_layers - 1):
aux_weight_dict.update({k + f'_{i}': v for k, v in weight_dict.items()})
weight_dict.update(aux_weight_dict)
losses = ['labels', 'boxes', 'cardinality']
if args.masks:
losses += ["masks"]
criterion = SetCriterion(num_classes, matcher=matcher, weight_dict=weight_dict,
eos_coef=args.eos_coef, losses=losses, focal_alpha=args.focal_alpha)
criterion.to(device)
postprocessors = {'bbox': PostProcess()}
if args.masks:
postprocessors['segm'] = PostProcessSegm()
if args.dataset_file == "coco_panoptic":
is_thing_map = {i: i <= 90 for i in range(201)}
postprocessors["panoptic"] = PostProcessPanoptic(is_thing_map, threshold=0.85)
return model, criterion, postprocessors
================================================
FILE: batchformer-v2/detr/models/matcher.py
================================================
# Copyright (c) Facebook, Inc. and its affiliates. All Rights Reserved
"""
Modules to compute the matching cost and solve the corresponding LSAP.
"""
import torch
from scipy.optimize import linear_sum_assignment
from torch import nn
from util.box_ops import box_cxcywh_to_xyxy, generalized_box_iou
class HungarianMatcher(nn.Module):
"""This class computes an assignment between the targets and the predictions of the network
For efficiency reasons, the targets don't include the no_object. Because of this, in general,
there are more predictions than targets. In this case, we do a 1-to-1 matching of the best predictions,
while the others are un-matched (and thus treated as non-objects).
"""
def __init__(self, cost_class: float = 1, cost_bbox: float = 1, cost_giou: float = 1):
"""Creates the matcher
Params:
cost_class: This is the relative weight of the classification error in the matching cost
cost_bbox: This is the relative weight of the L1 error of the bounding box coordinates in the matching cost
cost_giou: This is the relative weight of the giou loss of the bounding box in the matching cost
"""
super().__init__()
self.cost_class = cost_class
self.cost_bbox = cost_bbox
self.cost_giou = cost_giou
assert cost_class != 0 or cost_bbox != 0 or cost_giou != 0, "all costs cant be 0"
@torch.no_grad()
def forward(self, outputs, targets):
""" Performs the matching
Params:
outputs: This is a dict that contains at least these entries:
"pred_logits": Tensor of dim [batch_size, num_queries, num_classes] with the classification logits
"pred_boxes": Tensor of dim [batch_size, num_queries, 4] with the predicted box coordinates
targets: This is a list of targets (len(targets) = batch_size), where each target is a dict containing:
"labels": Tensor of dim [num_target_boxes] (where num_target_boxes is the number of ground-truth
objects in the target) containing the class labels
"boxes": Tensor of dim [num_target_boxes, 4] containing the target box coordinates
Returns:
A list of size batch_size, containing tuples of (index_i, index_j) where:
- index_i is the indices of the selected predictions (in order)
- index_j is the indices of the corresponding selected targets (in order)
For each batch element, it holds:
len(index_i) = len(index_j) = min(num_queries, num_target_boxes)
"""
bs, num_queries = outputs["pred_logits"].shape[:2]
# We flatten to compute the cost matrices in a batch
out_prob = outputs["pred_logits"].flatten(0, 1).softmax(-1) # [batch_size * num_queries, num_classes]
out_bbox = outputs["pred_boxes"].flatten(0, 1) # [batch_size * num_queries, 4]
# Also concat the target labels and boxes
tgt_ids = torch.cat([v["labels"] for v in targets])
tgt_bbox = torch.cat([v["boxes"] for v in targets])
# Compute the classification cost. Contrary to the loss, we don't use the NLL,
# but approximate it in 1 - proba[target class].
# The 1 is a constant that doesn't change the matching, it can be ommitted.
cost_class = -out_prob[:, tgt_ids]
# Compute the L1 cost between boxes
cost_bbox = torch.cdist(out_bbox, tgt_bbox, p=1)
# Compute the giou cost betwen boxes
cost_giou = -generalized_box_iou(box_cxcywh_to_xyxy(out_bbox), box_cxcywh_to_xyxy(tgt_bbox))
# Final cost matrix
C = self.cost_bbox * cost_bbox + self.cost_class * cost_class + self.cost_giou * cost_giou
C = C.view(bs, num_queries, -1).cpu()
sizes = [len(v["boxes"]) for v in targets]
indices = [linear_sum_assignment(c[i]) for i, c in enumerate(C.split(sizes, -1))]
return [(torch.as_tensor(i, dtype=torch.int64), torch.as_tensor(j, dtype=torch.int64)) for i, j in indices]
def build_matcher(args):
return HungarianMatcher(cost_class=args.set_cost_class, cost_bbox=args.set_cost_bbox, cost_giou=args.set_cost_giou)
================================================
FILE: batchformer-v2/detr/models/position_encoding.py
================================================
# Copyright (c) Facebook, Inc. and its affiliates. All Rights Reserved
"""
Various positional encodings for the transformer.
"""
import math
import torch
from torch import nn
from util.misc import NestedTensor
class PositionEmbeddingSine(nn.Module):
"""
This is a more standard version of the position embedding, very similar to the one
used by the Attention is all you need paper, generalized to work on images.
"""
def __init__(self, num_pos_feats=64, temperature=10000, normalize=False, scale=None):
super().__init__()
self.num_pos_feats = num_pos_feats
self.temperature = temperature
self.normalize = normalize
if scale is not None and normalize is False:
raise ValueError("normalize should be True if scale is passed")
if scale is None:
scale = 2 * math.pi
self.scale = scale
def forward(self, tensor_list: NestedTensor):
x = tensor_list.tensors
mask = tensor_list.mask
assert mask is not None
not_mask = ~mask
y_embed = not_mask.cumsum(1, dtype=torch.float32)
x_embed = not_mask.cumsum(2, dtype=torch.float32)
if self.normalize:
eps = 1e-6
y_embed = y_embed / (y_embed[:, -1:, :] + eps) * self.scale
x_embed = x_embed / (x_embed[:, :, -1:] + eps) * self.scale
dim_t = torch.arange(self.num_pos_feats, dtype=torch.float32, device=x.device)
dim_t = self.temperature ** (2 * (dim_t // 2) / self.num_pos_feats)
pos_x = x_embed[:, :, :, None] / dim_t
pos_y = y_embed[:, :, :, None] / dim_t
pos_x = torch.stack((pos_x[:, :, :, 0::2].sin(), pos_x[:, :, :, 1::2].cos()), dim=4).flatten(3)
pos_y = torch.stack((pos_y[:, :, :, 0::2].sin(), pos_y[:, :, :, 1::2].cos()), dim=4).flatten(3)
pos = torch.cat((pos_y, pos_x), dim=3).permute(0, 3, 1, 2)
return pos
class PositionEmbeddingLearned(nn.Module):
"""
Absolute pos embedding, learned.
"""
def __init__(self, num_pos_feats=256):
super().__init__()
self.row_embed = nn.Embedding(50, num_pos_feats)
self.col_embed = nn.Embedding(50, num_pos_feats)
self.reset_parameters()
def reset_parameters(self):
nn.init.uniform_(self.row_embed.weight)
nn.init.uniform_(self.col_embed.weight)
def forward(self, tensor_list: NestedTensor):
x = tensor_list.tensors
h, w = x.shape[-2:]
i = torch.arange(w, device=x.device)
j = torch.arange(h, device=x.device)
x_emb = self.col_embed(i)
y_emb = self.row_embed(j)
pos = torch.cat([
x_emb.unsqueeze(0).repeat(h, 1, 1),
y_emb.unsqueeze(1).repeat(1, w, 1),
], dim=-1).permute(2, 0, 1).unsqueeze(0).repeat(x.shape[0], 1, 1, 1)
return pos
def build_position_encoding(args):
N_steps = args.hidden_dim // 2
if args.position_embedding in ('v2', 'sine'):
# TODO find a better way of exposing other arguments
position_embedding = PositionEmbeddingSine(N_steps, normalize=True)
elif args.position_embedding in ('v3', 'learned'):
position_embedding = PositionEmbeddingLearned(N_steps)
else:
raise ValueError(f"not supported {args.position_embedding}")
return position_embedding
================================================
FILE: batchformer-v2/detr/models/segmentation.py
================================================
# Copyright (c) Facebook, Inc. and its affiliates. All Rights Reserved
"""
This file provides the definition of the convolutional heads used to predict masks, as well as the losses
"""
import io
from collections import defaultdict
from typing import List, Optional
import torch
import torch.nn as nn
import torch.nn.functional as F
from torch import Tensor
from PIL import Image
import util.box_ops as box_ops
from util.misc import NestedTensor, interpolate, nested_tensor_from_tensor_list
try:
from panopticapi.utils import id2rgb, rgb2id
except ImportError:
pass
class DETRsegm(nn.Module):
def __init__(self, detr, freeze_detr=False):
super().__init__()
self.detr = detr
if freeze_detr:
for p in self.parameters():
p.requires_grad_(False)
hidden_dim, nheads = detr.transformer.d_model, detr.transformer.nhead
self.bbox_attention = MHAttentionMap(hidden_dim, hidden_dim, nheads, dropout=0.0)
self.mask_head = MaskHeadSmallConv(hidden_dim + nheads, [1024, 512, 256], hidden_dim)
def forward(self, samples: NestedTensor):
if isinstance(samples, (list, torch.Tensor)):
samples = nested_tensor_from_tensor_list(samples)
features, pos = self.detr.backbone(samples)
bs = features[-1].tensors.shape[0]
src, mask = features[-1].decompose()
assert mask is not None
src_proj = self.detr.input_proj(src)
hs, memory = self.detr.transformer(src_proj, mask, self.detr.query_embed.weight, pos[-1])
feat2 = features[2].tensors
feat1 = features[1].tensors
feat0 = features[0].tensors
if len(memory) > len(src_proj):
mask = torch.cat([mask, mask], dim=0)
bs = bs + bs
src_proj = torch.cat([src_proj, src_proj], dim=0)
feat2 = torch.cat([feat2, feat2], dim=0)
feat1 = torch.cat([feat1, feat1], dim=0)
feat0 = torch.cat([feat0, feat0], dim=0)
# import ipdb;ipdb.set_trace()
outputs_class = self.detr.class_embed(hs)
outputs_coord = self.detr.bbox_embed(hs).sigmoid()
out = {"pred_logits": outputs_class[-1], "pred_boxes": outputs_coord[-1]}
if self.detr.aux_loss:
out['aux_outputs'] = self.detr._set_aux_loss(outputs_class, outputs_coord)
# FIXME h_boxes takes the last one computed, keep this in mind
bbox_mask = self.bbox_attention(hs[-1], memory, mask=mask)
seg_masks = self.mask_head(src_proj, bbox_mask, [feat2, feat1, feat0])
outputs_seg_masks = seg_masks.view(bs, self.detr.num_queries, seg_masks.shape[-2], seg_masks.shape[-1])
out["pred_masks"] = outputs_seg_masks
return out
def _expand(tensor, length: int):
return tensor.unsqueeze(1).repeat(1, int(length), 1, 1, 1).flatten(0, 1)
class MaskHeadSmallConv(nn.Module):
"""
Simple convolutional head, using group norm.
Upsampling is done using a FPN approach
"""
def __init__(self, dim, fpn_dims, context_dim):
super().__init__()
inter_dims = [dim, context_dim // 2, context_dim // 4, context_dim // 8, context_dim // 16, context_dim // 64]
self.lay1 = torch.nn.Conv2d(dim, dim, 3, padding=1)
self.gn1 = torch.nn.GroupNorm(8, dim)
self.lay2 = torch.nn.Conv2d(dim, inter_dims[1], 3, padding=1)
self.gn2 = torch.nn.GroupNorm(8, inter_dims[1])
self.lay3 = torch.nn.Conv2d(inter_dims[1], inter_dims[2], 3, padding=1)
self.gn3 = torch.nn.GroupNorm(8, inter_dims[2])
self.lay4 = torch.nn.Conv2d(inter_dims[2], inter_dims[3], 3, padding=1)
self.gn4 = torch.nn.GroupNorm(8, inter_dims[3])
self.lay5 = torch.nn.Conv2d(inter_dims[3], inter_dims[4], 3, padding=1)
self.gn5 = torch.nn.GroupNorm(8, inter_dims[4])
self.out_lay = torch.nn.Conv2d(inter_dims[4], 1, 3, padding=1)
self.dim = dim
self.adapter1 = torch.nn.Conv2d(fpn_dims[0], inter_dims[1], 1)
self.adapter2 = torch.nn.Conv2d(fpn_dims[1], inter_dims[2], 1)
self.adapter3 = torch.nn.Conv2d(fpn_dims[2], inter_dims[3], 1)
for m in self.modules():
if isinstance(m, nn.Conv2d):
nn.init.kaiming_uniform_(m.weight, a=1)
nn.init.constant_(m.bias, 0)
def forward(self, x: Tensor, bbox_mask: Tensor, fpns: List[Tensor]):
x = torch.cat([_expand(x, bbox_mask.shape[1]), bbox_mask.flatten(0, 1)], 1)
x = self.lay1(x)
x = self.gn1(x)
x = F.relu(x)
x = self.lay2(x)
x = self.gn2(x)
x = F.relu(x)
cur_fpn = self.adapter1(fpns[0])
if cur_fpn.size(0) != x.size(0):
cur_fpn = _expand(cur_fpn, x.size(0) // cur_fpn.size(0))
x = cur_fpn + F.interpolate(x, size=cur_fpn.shape[-2:], mode="nearest")
x = self.lay3(x)
x = self.gn3(x)
x = F.relu(x)
cur_fpn = self.adapter2(fpns[1])
if cur_fpn.size(0) != x.size(0):
cur_fpn = _expand(cur_fpn, x.size(0) // cur_fpn.size(0))
x = cur_fpn + F.interpolate(x, size=cur_fpn.shape[-2:], mode="nearest")
x = self.lay4(x)
x = self.gn4(x)
x = F.relu(x)
cur_fpn = self.adapter3(fpns[2])
if cur_fpn.size(0) != x.size(0):
cur_fpn = _expand(cur_fpn, x.size(0) // cur_fpn.size(0))
x = cur_fpn + F.interpolate(x, size=cur_fpn.shape[-2:], mode="nearest")
x = self.lay5(x)
x = self.gn5(x)
x = F.relu(x)
x = self.out_lay(x)
return x
class MHAttentionMap(nn.Module):
"""This is a 2D attention module, which only returns the attention softmax (no multiplication by value)"""
def __init__(self, query_dim, hidden_dim, num_heads, dropout=0.0, bias=True):
super().__init__()
self.num_heads = num_heads
self.hidden_dim = hidden_dim
self.dropout = nn.Dropout(dropout)
self.q_linear = nn.Linear(query_dim, hidden_dim, bias=bias)
self.k_linear = nn.Linear(query_dim, hidden_dim, bias=bias)
nn.init.zeros_(self.k_linear.bias)
nn.init.zeros_(self.q_linear.bias)
nn.init.xavier_uniform_(self.k_linear.weight)
nn.init.xavier_uniform_(self.q_linear.weight)
self.normalize_fact = float(hidden_dim / self.num_heads) ** -0.5
def forward(self, q, k, mask: Optional[Tensor] = None):
q = self.q_linear(q)
k = F.conv2d(k, self.k_linear.weight.unsqueeze(-1).unsqueeze(-1), self.k_linear.bias)
qh = q.view(q.shape[0], q.shape[1], self.num_heads, self.hidden_dim // self.num_heads)
kh = k.view(k.shape[0], self.num_heads, self.hidden_dim // self.num_heads, k.shape[-2], k.shape[-1])
weights = torch.einsum("bqnc,bnchw->bqnhw", qh * self.normalize_fact, kh)
if mask is not None:
weights.masked_fill_(mask.unsqueeze(1).unsqueeze(1), float("-inf"))
weights = F.softmax(weights.flatten(2), dim=-1).view(weights.size())
weights = self.dropout(weights)
return weights
def dice_loss(inputs, targets, num_boxes):
"""
Compute the DICE loss, similar to generalized IOU for masks
Args:
inputs: A float tensor of arbitrary shape.
The predictions for each example.
targets: A float tensor with the same shape as inputs. Stores the binary
classification label for each element in inputs
(0 for the negative class and 1 for the positive class).
"""
inputs = inputs.sigmoid()
inputs = inputs.flatten(1)
numerator = 2 * (inputs * targets).sum(1)
denominator = inputs.sum(-1) + targets.sum(-1)
loss = 1 - (numerator + 1) / (denominator + 1)
return loss.sum() / num_boxes
def sigmoid_focal_loss(inputs, targets, num_boxes, alpha: float = 0.25, gamma: float = 2):
"""
Loss used in RetinaNet for dense detection: https://arxiv.org/abs/1708.02002.
Args:
inputs: A float tensor of arbitrary shape.
The predictions for each example.
targets: A float tensor with the same shape as inputs. Stores the binary
classification label for each element in inputs
(0 for the negative class and 1 for the positive class).
alpha: (optional) Weighting factor in range (0,1) to balance
positive vs negative examples. Default = -1 (no weighting).
gamma: Exponent of the modulating factor (1 - p_t) to
balance easy vs hard examples.
Returns:
Loss tensor
"""
prob = inputs.sigmoid()
ce_loss = F.binary_cross_entropy_with_logits(inputs, targets, reduction="none")
p_t = prob * targets + (1 - prob) * (1 - targets)
loss = ce_loss * ((1 - p_t) ** gamma)
if alpha >= 0:
alpha_t = alpha * targets + (1 - alpha) * (1 - targets)
loss = alpha_t * loss
return loss.mean(1).sum() / num_boxes
class PostProcessSegm(nn.Module):
def __init__(self, threshold=0.5):
super().__init__()
self.threshold = threshold
@torch.no_grad()
def forward(self, results, outputs, orig_target_sizes, max_target_sizes):
assert len(orig_target_sizes) == len(max_target_sizes)
max_h, max_w = max_target_sizes.max(0)[0].tolist()
outputs_masks = outputs["pred_masks"].squeeze(2)
outputs_masks = F.interpolate(outputs_masks, size=(max_h, max_w), mode="bilinear", align_corners=False)
outputs_masks = (outputs_masks.sigmoid() > self.threshold).cpu()
for i, (cur_mask, t, tt) in enumerate(zip(outputs_masks, max_target_sizes, orig_target_sizes)):
img_h, img_w = t[0], t[1]
results[i]["masks"] = cur_mask[:, :img_h, :img_w].unsqueeze(1)
results[i]["masks"] = F.interpolate(
results[i]["masks"].float(), size=tuple(tt.tolist()), mode="nearest"
).byte()
return results
class PostProcessPanoptic(nn.Module):
"""This class converts the output of the model to the final panoptic result, in the format expected by the
coco panoptic API """
def __init__(self, is_thing_map, threshold=0.85):
"""
Parameters:
is_thing_map: This is a whose keys are the class ids, and the values a boolean indicating whether
the class is a thing (True) or a stuff (False) class
threshold: confidence threshold: segments with confidence lower than this will be deleted
"""
super().__init__()
self.threshold = threshold
self.is_thing_map = is_thing_map
def forward(self, outputs, processed_sizes, target_sizes=None):
""" This function computes the panoptic prediction from the model's predictions.
Parameters:
outputs: This is a dict coming directly from the model. See the model doc for the content.
processed_sizes: This is a list of tuples (or torch tensors) of sizes of the images that were passed to the
model, ie the size after data augmentation but before batching.
target_sizes: This is a list of tuples (or torch tensors) corresponding to the requested final size
of each prediction. If left to None, it will default to the processed_sizes
"""
if target_sizes is None:
target_sizes = processed_sizes
assert len(processed_sizes) == len(target_sizes)
out_logits, raw_masks, raw_boxes = outputs["pred_logits"], outputs["pred_masks"], outputs["pred_boxes"]
assert len(out_logits) == len(raw_masks) == len(target_sizes)
preds = []
def to_tuple(tup):
if isinstance(tup, tuple):
return tup
return tuple(tup.cpu().tolist())
for cur_logits, cur_masks, cur_boxes, size, target_size in zip(
out_logits, raw_masks, raw_boxes, processed_sizes, target_sizes
):
# we filter empty queries and detection below threshold
scores, labels = cur_logits.softmax(-1).max(-1)
keep = labels.ne(outputs["pred_logits"].shape[-1] - 1) & (scores > self.threshold)
cur_scores, cur_classes = cur_logits.softmax(-1).max(-1)
cur_scores = cur_scores[keep]
cur_classes = cur_classes[keep]
cur_masks = cur_masks[keep]
cur_masks = interpolate(cur_masks[:, None], to_tuple(size), mode="bilinear").squeeze(1)
cur_boxes = box_ops.box_cxcywh_to_xyxy(cur_boxes[keep])
h, w = cur_masks.shape[-2:]
assert len(cur_boxes) == len(cur_classes)
# It may be that we have several predicted masks for the same stuff class.
# In the following, we track the list of masks ids for each stuff class (they are merged later on)
cur_masks = cur_masks.flatten(1)
stuff_equiv_classes = defaultdict(lambda: [])
for k, label in enumerate(cur_classes):
if not self.is_thing_map[label.item()]:
stuff_equiv_classes[label.item()].append(k)
def get_ids_area(masks, scores, dedup=False):
# This helper function creates the final panoptic segmentation image
# It also returns the area of the masks that appears on the image
m_id = masks.transpose(0, 1).softmax(-1)
if m_id.shape[-1] == 0:
# We didn't detect any mask :(
m_id = torch.zeros((h, w), dtype=torch.long, device=m_id.device)
else:
m_id = m_id.argmax(-1).view(h, w)
if dedup:
# Merge the masks corresponding to the same stuff class
for equiv in stuff_equiv_classes.values():
if len(equiv) > 1:
for eq_id in equiv:
m_id.masked_fill_(m_id.eq(eq_id), equiv[0])
final_h, final_w = to_tuple(target_size)
seg_img = Image.fromarray(id2rgb(m_id.view(h, w).cpu().numpy()))
seg_img = seg_img.resize(size=(final_w, final_h), resample=Image.NEAREST)
np_seg_img = (
torch.ByteTensor(torch.ByteStorage.from_buffer(seg_img.tobytes())).view(final_h, final_w, 3).numpy()
)
m_id = torch.from_numpy(rgb2id(np_seg_img))
area = []
for i in range(len(scores)):
area.append(m_id.eq(i).sum().item())
return area, seg_img
area, seg_img = get_ids_area(cur_masks, cur_scores, dedup=True)
if cur_classes.numel() > 0:
# We know filter empty masks as long as we find some
while True:
filtered_small = torch.as_tensor(
[area[i] <= 4 for i, c in enumerate(cur_classes)], dtype=torch.bool, device=keep.device
)
if filtered_small.any().item():
cur_scores = cur_scores[~filtered_small]
cur_classes = cur_classes[~filtered_small]
cur_masks = cur_masks[~filtered_small]
area, seg_img = get_ids_area(cur_masks, cur_scores)
else:
break
else:
cur_classes = torch.ones(1, dtype=torch.long, device=cur_classes.device)
segments_info = []
for i, a in enumerate(area):
cat = cur_classes[i].item()
segments_info.append({"id": i, "isthing": self.is_thing_map[cat], "category_id": cat, "area": a})
del cur_classes
with io.BytesIO() as out:
seg_img.save(out, format="PNG")
predictions = {"png_string": out.getvalue(), "segments_info": segments_info}
preds.append(predictions)
return preds
================================================
FILE: batchformer-v2/detr/models/transformer.py
================================================
# Copyright (c) Facebook, Inc. and its affiliates. All Rights Reserved
"""
DETR Transformer class.
Copy-paste from torch.nn.Transformer with modifications:
* positional encodings are passed in MHattention
* extra LN at the end of encoder is removed
* decoder returns a stack of activations from all decoding layers
"""
import copy
from typing import Optional, List
import torch
import torch.nn.functional as F
from torch import nn, Tensor
class Transformer(nn.Module):
def __init__(self, d_model=512, nhead=8, num_encoder_layers=6,
num_decoder_layers=6, dim_feedforward=2048, dropout=0.1,
activation="relu", normalize_before=False,
return_intermediate_dec=False, args=None):
super().__init__()
encoder_layer = TransformerEncoderLayer(d_model, nhead, dim_feedforward,
dropout, activation, normalize_before)
encoder_norm = nn.LayerNorm(d_model) if normalize_before else None
self.bf = None
self.use_checkpoint=False
self.base_bf = 0
insert_idx = []
if args is not None and args.bf:
self.use_checkpoint = args.use_checkpoint
self.base_bf = args.base_bf
insert_idx = args.insert_idx
self.bf = torch.nn.TransformerEncoderLayer(d_model, 4, d_model, dropout=0.5)
if not args.share_bf:
self.bf = torch.nn.ModuleList([torch.nn.TransformerEncoderLayer(d_model, 4, d_model, dropout=0.5) if i in insert_idx else torch.nn.Identity() for i in range(0, num_encoder_layers)])
elif self.base_bf:
self.bf = torch.nn.Identity()
args.bf = 3
self.encoder = TransformerEncoder(encoder_layer, num_encoder_layers, encoder_norm, bf=self.bf, bf_idx=args.bf, insert_idx=insert_idx,
use_checkpoint=args.use_checkpoint)
decoder_layer = TransformerDecoderLayer(d_model, nhead, dim_feedforward,
dropout, activation, normalize_before)
decoder_norm = nn.LayerNorm(d_model)
self.decoder = TransformerDecoder(decoder_layer, num_decoder_layers, decoder_norm,
return_intermediate=return_intermediate_dec, use_checkpoint=args.use_checkpoint)
self._reset_parameters()
self.d_model = d_model
self.nhead = nhead
def _reset_parameters(self):
for p in self.parameters():
if p.dim() > 1:
nn.init.xavier_uniform_(p)
def forward(self, src, mask, query_embed, pos_embed):
# flatten NxCxHxW to HWxNxC
bs, c, h, w = src.shape
src = src.flatten(2).permute(2, 0, 1)
pos_embed = pos_embed.flatten(2).permute(2, 0, 1)
query_embed = query_embed.unsqueeze(1).repeat(1, bs, 1)
mask = mask.flatten(1)
tgt = torch.zeros_like(query_embed)
memory = self.encoder(src, src_key_padding_mask=mask, pos=pos_embed)
if memory.shape[1] > bs:
tgt = torch.cat([tgt,tgt], dim=1)
mask = torch.cat([mask, mask], dim=0)
pos_embed = torch.cat([pos_embed, pos_embed], dim=1)
query_embed = torch.cat([query_embed, query_embed], dim=1)
bs = bs * 2
if self.use_checkpoint:
print('use checkpoint')
from torch.utils import checkpoint
# output = checkpoint.checkpoint(layer, output, mask, src_key_padding_mask, pos)
hs = checkpoint.checkpoint(self.decoder, tgt, memory, None, None, None, mask,
pos_embed, query_embed)
# tgt, memory,
# tgt_mask: Optional[Tensor] = None,
# memory_mask: Optional[Tensor] = None,
# tgt_key_padding_mask: Optional[Tensor] = None,
# memory_key_padding_mask: Optional[Tensor] = None,
# pos: Optional[Tensor] = None,
# query_pos: Optional[Tensor] = None):
else:
hs = self.decoder(tgt, memory, memory_key_padding_mask=mask,
pos=pos_embed, query_pos=query_embed)
return hs.transpose(1, 2), memory.permute(1, 2, 0).view(bs, c, h, w)
class TransformerEncoder(nn.Module):
def __init__(self, encoder_layer, num_layers, norm=None, bf=None, bf_idx =0, insert_idx=[], use_checkpoint=False):
super().__init__()
self.layers = _get_clones(encoder_layer, num_layers)
self.num_layers = num_layers
self.norm = norm
if type(bf) != torch.nn.ModuleList and bf is not None:
self.bf = [bf]*num_layers
else:
self.bf = bf
self.insert_idx = insert_idx
self.bf_idx = bf_idx
self.use_checkpoint = use_checkpoint
def forward(self, src,
mask: Optional[Tensor] = None,
src_key_padding_mask: Optional[Tensor] = None,
pos: Optional[Tensor] = None):
output = src
for i, layer in enumerate(self.layers):
output = layer(output, src_mask=mask,
src_key_padding_mask=src_key_padding_mask, pos=pos)
if i in self.insert_idx and self.bf is not None and self.bf_idx == 3 and self.training:
L, B, C = output.shape
old_output = output
if i != self.insert_idx[0]:
old_output = output[:, :B//2, :]
output = output[:, B//2:, :]
# old_output = output[:, :len(output)//2, :]
# output = output[:, len(output)//2:, :]
# the original batches
output = self.bf[i](torch.transpose(output, 1, 0))
output = torch.transpose(output, 1, 0)
output = torch.cat([old_output, output], dim=1)
if i == self.insert_idx[0]:
pos = torch.cat([pos, pos], dim=1)
src_key_padding_mask = torch.cat([src_key_padding_mask, src_key_padding_mask], dim=0)
elif i == 4 and self.bf is not None and self.bf_idx == 2 and self.training:
old_output = output
output = self.bf[i](torch.transpose(output, 1, 0))
output = torch.transpose(output, 1, 0)
output = torch.cat([old_output, output], dim=1)
pos = torch.cat([pos, pos], dim=1)
src_key_padding_mask = torch.cat([src_key_padding_mask, src_key_padding_mask], dim=0)
if self.norm is not None:
output = self.norm(output)
if self.bf is not None and self.bf_idx == 1 and self.training:
# print(output.shape, self.norm)
# import ipdb;ipdb.set_trace()
old_output = output
output = self.bf[i](torch.transpose(output, 1, 0))
output = torch.transpose(output, 1, 0)
output = torch.cat([old_output, output], dim=1)
pos = torch.cat([pos, pos], dim=1)
src_key_padding_mask = torch.cat([src_key_padding_mask, src_key_padding_mask], dim=0)
return output
class TransformerDecoder(nn.Module):
def __init__(self, decoder_layer, num_layers, norm=None, return_intermediate=False, use_checkpoint=False):
super().__init__()
self.layers = _get_clones(decoder_layer, num_layers)
self.num_layers = num_layers
self.norm = norm
self.return_intermediate = return_intermediate
self.use_checkpoint = use_checkpoint
def forward(self, tgt, memory,
tgt_mask: Optional[Tensor] = None,
memory_mask: Optional[Tensor] = None,
tgt_key_padding_mask: Optional[Tensor] = None,
memory_key_padding_mask: Optional[Tensor] = None,
pos: Optional[Tensor] = None,
query_pos: Optional[Tensor] = None):
output = tgt
intermediate = []
for layer in self.layers:
if self.use_checkpoint:
from torch.utils import checkpoint
output = checkpoint.checkpoint(layer, output, memory, tgt_mask,
memory_mask,
tgt_key_padding_mask,
memory_key_padding_mask,
pos, query_pos)
else:
output = layer(output, memory, tgt_mask=tgt_mask,
memory_mask=memory_mask,
tgt_key_padding_mask=tgt_key_padding_mask,
memory_key_padding_mask=memory_key_padding_mask,
pos=pos, query_pos=query_pos)
if self.return_intermediate:
intermediate.append(self.norm(output))
if self.norm is not None:
output = self.norm(output)
if self.return_intermediate:
intermediate.pop()
intermediate.append(output)
if self.return_intermediate:
return torch.stack(intermediate)
return output.unsqueeze(0)
class TransformerEncoderLayer(nn.Module):
def __init__(self, d_model, nhead, dim_feedforward=2048, dropout=0.1,
activation="relu", normalize_before=False):
super().__init__()
self.self_attn = nn.MultiheadAttention(d_model, nhead, dropout=dropout)
# Implementation of Feedforward model
self.linear1 = nn.Linear(d_model, dim_feedforward)
self.dropout = nn.Dropout(dropout)
self.linear2 = nn.Linear(dim_feedforward, d_model)
self.norm1 = nn.LayerNorm(d_model)
self.norm2 = nn.LayerNorm(d_model)
self.dropout1 = nn.Dropout(dropout)
self.dropout2 = nn.Dropout(dropout)
self.activation = _get_activation_fn(activation)
self.normalize_before = normalize_before
def with_pos_embed(self, tensor, pos: Optional[Tensor]):
return tensor if pos is None else tensor + pos
def forward_post(self,
src,
src_mask: Optional[Tensor] = None,
src_key_padding_mask: Optional[Tensor] = None,
pos: Optional[Tensor] = None):
q = k = self.with_pos_embed(src, pos)
src2 = self.self_attn(q, k, value=src, attn_mask=src_mask,
key_padding_mask=src_key_padding_mask)[0]
src = src + self.dropout1(src2)
src = self.norm1(src)
src2 = self.linear2(self.dropout(self.activation(self.linear1(src))))
src = src + self.dropout2(src2)
src = self.norm2(src)
return src
def forward_pre(self, src,
src_mask: Optional[Tensor] = None,
src_key_padding_mask: Optional[Tensor] = None,
pos: Optional[Tensor] = None):
src2 = self.norm1(src)
q = k = self.with_pos_embed(src2, pos)
src2 = self.self_attn(q, k, value=src2, attn_mask=src_mask,
key_padding_mask=src_key_padding_mask)[0]
src = src + self.dropout1(src2)
src2 = self.norm2(src)
src2 = self.linear2(self.dropout(self.activation(self.linear1(src2))))
src = src + self.dropout2(src2)
return src
def forward(self, src,
src_mask: Optional[Tensor] = None,
src_key_padding_mask: Optional[Tensor] = None,
pos: Optional[Tensor] = None):
if self.normalize_before:
return self.forward_pre(src, src_mask, src_key_padding_mask, pos)
return self.forward_post(src, src_mask, src_key_padding_mask, pos)
class TransformerDecoderLayer(nn.Module):
def __init__(self, d_model, nhead, dim_feedforward=2048, dropout=0.1,
activation="relu", normalize_before=False):
super().__init__()
self.self_attn = nn.MultiheadAttention(d_model, nhead, dropout=dropout)
self.multihead_attn = nn.MultiheadAttention(d_model, nhead, dropout=dropout)
# Implementation of Feedforward model
self.linear1 = nn.Linear(d_model, dim_feedforward)
self.dropout = nn.Dropout(dropout)
self.linear2 = nn.Linear(dim_feedforward, d_model)
self.norm1 = nn.LayerNorm(d_model)
self.norm2 = nn.LayerNorm(d_model)
self.norm3 = nn.LayerNorm(d_model)
self.dropout1 = nn.Dropout(dropout)
self.dropout2 = nn.Dropout(dropout)
self.dropout3 = nn.Dropout(dropout)
self.activation = _get_activation_fn(activation)
self.normalize_before = normalize_before
def with_pos_embed(self, tensor, pos: Optional[Tensor]):
return tensor if pos is None else tensor + pos
def forward_post(self, tgt, memory,
tgt_mask: Optional[Tensor] = None,
memory_mask: Optional[Tensor] = None,
tgt_key_padding_mask: Optional[Tensor] = None,
memory_key_padding_mask: Optional[Tensor] = None,
pos: Optional[Tensor] = None,
query_pos: Optional[Tensor] = None):
q = k = self.with_pos_embed(tgt, query_pos)
tgt2 = self.self_attn(q, k, value=tgt, attn_mask=tgt_mask,
key_padding_mask=tgt_key_padding_mask)[0]
tgt = tgt + self.dropout1(tgt2)
tgt = self.norm1(tgt)
tgt2 = self.multihead_attn(query=self.with_pos_embed(tgt, query_pos),
key=self.with_pos_embed(memory, pos),
value=memory, attn_mask=memory_mask,
key_padding_mask=memory_key_padding_mask)[0]
tgt = tgt + self.dropout2(tgt2)
tgt = self.norm2(tgt)
tgt2 = self.linear2(self.dropout(self.activation(self.linear1(tgt))))
tgt = tgt + self.dropout3(tgt2)
tgt = self.norm3(tgt)
return tgt
def forward_pre(self, tgt, memory,
tgt_mask: Optional[Tensor] = None,
memory_mask: Optional[Tensor] = None,
tgt_key_padding_mask: Optional[Tensor] = None,
memory_key_padding_mask: Optional[Tensor] = None,
pos: Optional[Tensor] = None,
query_pos: Optional[Tensor] = None):
tgt2 = self.norm1(tgt)
q = k = self.with_pos_embed(tgt2, query_pos)
tgt2 = self.self_attn(q, k, value=tgt2, attn_mask=tgt_mask,
key_padding_mask=tgt_key_padding_mask)[0]
tgt = tgt + self.dropout1(tgt2)
tgt2 = self.norm2(tgt)
tgt2 = self.multihead_attn(query=self.with_pos_embed(tgt2, query_pos),
key=self.with_pos_embed(memory, pos),
value=memory, attn_mask=memory_mask,
key_padding_mask=memory_key_padding_mask)[0]
tgt = tgt + self.dropout2(tgt2)
tgt2 = self.norm3(tgt)
tgt2 = self.linear2(self.dropout(self.activation(self.linear1(tgt2))))
tgt = tgt + self.dropout3(tgt2)
return tgt
def forward(self, tgt, memory,
tgt_mask: Optional[Tensor] = None,
memory_mask: Optional[Tensor] = None,
tgt_key_padding_mask: Optional[Tensor] = None,
memory_key_padding_mask: Optional[Tensor] = None,
pos: Optional[Tensor] = None,
query_pos: Optional[Tensor] = None):
if self.normalize_before:
return self.forward_pre(tgt, memory, tgt_mask, memory_mask,
tgt_key_padding_mask, memory_key_padding_mask, pos, query_pos)
return self.forward_post(tgt, memory, tgt_mask, memory_mask,
tgt_key_padding_mask, memory_key_padding_mask, pos, query_pos)
def _get_clones(module, N):
return nn.ModuleList([copy.deepcopy(module) for i in range(N)])
def build_transformer(args):
return Transformer(
d_model=args.hidden_dim,
dropout=args.dropout,
nhead=args.nheads,
dim_feedforward=args.dim_feedforward,
num_encoder_layers=args.enc_layers,
num_decoder_layers=args.dec_layers,
normalize_before=args.pre_norm,
return_intermediate_dec=True,
args=args
)
def _get_activation_fn(activation):
"""Return an activation function given a string"""
if activation == "relu":
return F.relu
if activation == "gelu":
return F.gelu
if activation == "glu":
return F.glu
raise RuntimeError(F"activation should be relu/gelu, not {activation}.")
================================================
FILE: batchformer-v2/detr/requirements.txt
================================================
cython
git+https://github.com/cocodataset/cocoapi.git#subdirectory=PythonAPI&egg=pycocotools
submitit
torch>=1.5.0
torchvision>=0.6.0
git+https://github.com/cocodataset/panopticapi.git#egg=panopticapi
scipy
onnx
onnxruntime
================================================
FILE: batchformer-v2/detr/run.py
================================================
#!/usr/local/anaconda/4.2.0/bin/python
##################################
'''
Code is based on hpc_check_py36_withquotawarning_pbs.py
You can put this file into ~/bin/ and rename filename to ssub (without suffix .py).
Meanwhile chmod a+x ~/bin/ssub
Then, you can submit job by ssub xxx.pbs
This script will search suitable node to submit job with host=xxxx
'''
#################################
import subprocess
from argparse import ArgumentParser
import numpy as np
import re
import time
import json
import os
MEM_PER_NODE = 187
CPU_PER_NODE = 36
GPU_PER_NODE = 4
# from user_quota_config import *
HPC_NODE_INFO_ALLOC_DT = [ 'hpc223', 'hpc224', 'hpc225', 'hpc226', 'hpc227', 'hpc228', 'hpc229', 'hpc230', 'hpc231',
'hpc232',
'hpc233', 'hpc234', 'hpc235', 'hpc236', 'hpc237', 'hpc238', 'hpc239', 'hpc246', 'hpc247',
'hpc248']
HPC_NODE_INFO_ALLOC_DT = ['hpc'+str(i) for i in range(216, 223, 1)] + HPC_NODE_INFO_ALLOC_DT
print(HPC_NODE_INFO_ALLOC_DT )
HOST_IP = {}
for host in HPC_NODE_INFO_ALLOC_DT:
ip = host.replace('hpc', '')
ip = '192.168.66.'+ip
HOST_IP[host+'.hpc.sydney.edu.au'] = ip
print(HOST_IP)
def parse_args():
"""
Helper function parsing the command line options
@retval ArgumentParser
"""
parser = ArgumentParser(description="PyTorch distributed training launch "
"helper utilty that will spawn up "
"multiple distributed processes")
# Optional arguments for the launch helper
parser.add_argument("--nnodes", type=int, default=1,
help="The number of nodes to use for distributed "
"training")
parser.add_argument("--nproc_per_node", type=int, default=1,
help="The number of processes to launch on each node, "
"for GPU training, this is recommended to be set "
"to the number of GPUs in your system so that "
"each process can be bound to a single GPU.")
parser.add_argument("--master_addr", default="127.0.0.1", type=str,
help="Master node (rank 0)'s address, should be either "
"the IP address or the hostname of node 0, for "
"single node multi-proc training, the "
"--master_addr can simply be 127.0.0.1")
parser.add_argument("--master_port", default=29500, type=int,
help="Master node (rank 0)'s free port that needs to "
"be used for communciation during distributed "
"training")
# positional
parser.add_argument("training_script", type=str, default='./configs/r50_deformable_detr.sh',
help="The full path to the single GPU training "
"program/script to be launched in parallel, "
"followed by all the arguments for the "
"training script")
# rest from the training program
# parser.add_argument('training_script_args', nargs=REMAINDER)
return parser.parse_args()
if __name__ == "__main__":
import sys
import os
args = parse_args()
os.system('chmod a+x {}'.format(args.training_script))
fpbs = open(os.environ['PBS_NODEFILE'])
hosts = [item.strip() for item in fpbs.readlines()]
uniq_hosts = set(hosts)
host_gpus = {k:0 for k in uniq_hosts}
host_rank = {k:0 for k in uniq_hosts}
current_gpus = {}
for host in hosts:
host_gpus[host] += args.nproc_per_node
# host_rank[host] += sum()
current_gpus[host] = 0
print(uniq_hosts, hosts)
uniq_hosts = hosts
pbs_jobid = os.environ['PBS_JOBID']
pbs_jobid = pbs_jobid.split('.')[0]
current_env = os.environ.copy()
current_env["MASTER_ADDR"] = HOST_IP[hosts[0]]
current_env["MASTER_PORT"] = str(args.master_port)
current_env["WORLD_SIZE"] = str(args.nnodes*args.nproc_per_node)
current_env["GPUS_PER_NODE"] = str(args.nproc_per_node)
current_env["NNODES"] = str(len(uniq_hosts))
processes = []
alloc_procs = 0
tmp_i = 0
for ni in range(len(uniq_hosts)-1, -1, -1):
# each process's rank
current_env["GPUS_PER_NODE"] = str(args.nproc_per_node)
command_remote = 'NNODES={} MASTER_ADDR={} MASTER_PORT={} NODE_RANK={} GPUS_PER_NODE={} ./tools/run_dist_launch.sh {} {}'.format(
args.nnodes, HOST_IP[hosts[0]], str(args.master_port), ni, args.nproc_per_node, str(args.nnodes*args.nproc_per_node), args.training_script)
current_env["NODE_RANK"] = str(ni)
command = './tools/run_dist_launch.sh {} {}'.format(str(args.nnodes*args.nproc_per_node), args.training_script)
if ni == 0:
print('cur', ni, command)
# print(host_gpus)
# print(current_env["GPUS_PER_NODE"], args.nproc_per_node, 'test')
process = subprocess.Popen(command.split(' '), env=current_env)
# processes.append(process)
else:
# print('ssh', ni)
cmd = 'ssh {} cd {} & {}'.format(hosts[ni], os.path.abspath('.'), command)
cmd = "ssh {} 'cd {} ; {}'".format(hosts[ni], os.path.abspath('.'), command_remote)
# print(cmd)
# os.system(cmd)
# continue
if host_gpus[hosts[ni]] == 4 and args.nproc_per_node == 2:
tmp_i += 1
current_gpus[hosts[ni]] += args.nproc_per_node
if current_gpus[hosts[ni]] == 4:
arr = ['ssh', hosts[ni], 'cd {} ;CUDA_VISIBLE_DEVICES=2,3 {} '.format(os.path.abspath('.'), command_remote)]
else:
arr = ['ssh', hosts[ni], 'cd {} ;CUDA_VISIBLE_DEVICES=0,1 {} '.format(os.path.abspath('.'), command_remote)]
elif host_gpus[hosts[ni]] > args.nproc_per_node:
arr = ['ssh', hosts[ni],
'cd {} ;CUDA_VISIBLE_DEVICES={} {} '.format(os.path.abspath('.'), current_gpus[hosts[ni]], command_remote)]
current_gpus[hosts[ni]] += args.nproc_per_node
else:
arr = ['ssh', hosts[ni], 'cd {} ; {} '.format(os.path.abspath('.'), command_remote)]
print(' '.join(arr))
process = subprocess.Popen(arr,
env=current_env)
processes.append(process)
time.sleep(3)
for process in processes:
process.wait()
if process.returncode != 0:
raise subprocess.CalledProcessError(returncode=process.returncode,
cmd=process.args)
================================================
FILE: batchformer-v2/detr/run_with_submitit.py
================================================
# Copyright (c) Facebook, Inc. and its affiliates. All Rights Reserved
"""
A script to run multinode training with submitit.
"""
import argparse
import os
import uuid
from pathlib import Path
import main as detection
import submitit
def parse_args():
detection_parser = detection.get_args_parser()
parser = argparse.ArgumentParser("Submitit for detection", parents=[detection_parser])
parser.add_argument("--ngpus", default=8, type=int, help="Number of gpus to request on each node")
parser.add_argument("--nodes", default=4, type=int, help="Number of nodes to request")
parser.add_argument("--timeout", default=60, type=int, help="Duration of the job")
parser.add_argument("--job_dir", default="", type=str, help="Job dir. Leave empty for automatic.")
return parser.parse_args()
def get_shared_folder() -> Path:
user = os.getenv("USER")
if Path("/checkpoint/").is_dir():
p = Path(f"/checkpoint/{user}/experiments")
p.mkdir(exist_ok=True)
return p
raise RuntimeError("No shared folder available")
def get_init_file():
# Init file must not exist, but it's parent dir must exist.
os.makedirs(str(get_shared_folder()), exist_ok=True)
init_file = get_shared_folder() / f"{uuid.uuid4().hex}_init"
if init_file.exists():
os.remove(str(init_file))
return init_file
class Trainer(object):
def __init__(self, args):
self.args = args
def __call__(self):
import main as detection
self._setup_gpu_args()
detection.main(self.args)
def checkpoint(self):
import os
import submitit
from pathlib import Path
self.args.dist_url = get_init_file().as_uri()
checkpoint_file = os.path.join(self.args.output_dir, "checkpoint.pth")
if os.path.exists(checkpoint_file):
self.args.resume = checkpoint_file
print("Requeuing ", self.args)
empty_trainer = type(self)(self.args)
return submitit.helpers.DelayedSubmission(empty_trainer)
def _setup_gpu_args(self):
import submitit
from pathlib import Path
job_env = submitit.JobEnvironment()
self.args.output_dir = Path(str(self.args.output_dir).replace("%j", str(job_env.job_id)))
self.args.gpu = job_env.local_rank
self.args.rank = job_env.global_rank
self.args.world_size = job_env.num_tasks
print(f"Process group: {job_env.num_tasks} tasks, rank: {job_env.global_rank}")
def main():
args = parse_args()
if args.job_dir == "":
args.job_dir = get_shared_folder() / "%j"
# Note that the folder will depend on the job_id, to easily track experiments
executor = submitit.AutoExecutor(folder=args.job_dir, slurm_max_num_timeout=30)
# cluster setup is defined by environment variables
num_gpus_per_node = args.ngpus
nodes = args.nodes
timeout_min = args.timeout
executor.update_parameters(
mem_gb=40 * num_gpus_per_node,
gpus_per_node=num_gpus_per_node,
tasks_per_node=num_gpus_per_node, # one task per GPU
cpus_per_task=10,
nodes=nodes,
timeout_min=timeout_min, # max is 60 * 72
)
executor.update_parameters(name="detr")
args.dist_url = get_init_file().as_uri()
args.output_dir = args.job_dir
trainer = Trainer(args)
job = executor.submit(trainer)
print("Submitted job_id:", job.job_id)
if __name__ == "__main__":
main()
================================================
FILE: batchformer-v2/detr/test_all.py
================================================
# Copyright (c) Facebook, Inc. and its affiliates. All Rights Reserved
import io
import unittest
import torch
from torch import nn, Tensor
from typing import List
from models.matcher import HungarianMatcher
from models.position_encoding import PositionEmbeddingSine, PositionEmbeddingLearned
from models.backbone import Backbone, Joiner, BackboneBase
from util import box_ops
from util.misc import nested_tensor_from_tensor_list
from hubconf import detr_resnet50, detr_resnet50_panoptic
# onnxruntime requires python 3.5 or above
try:
import onnxruntime
except ImportError:
onnxruntime = None
class Tester(unittest.TestCase):
def test_box_cxcywh_to_xyxy(self):
t = torch.rand(10, 4)
r = box_ops.box_xyxy_to_cxcywh(box_ops.box_cxcywh_to_xyxy(t))
self.assertLess((t - r).abs().max(), 1e-5)
@staticmethod
def indices_torch2python(indices):
return [(i.tolist(), j.tolist()) for i, j in indices]
def test_hungarian(self):
n_queries, n_targets, n_classes = 100, 15, 91
logits = torch.rand(1, n_queries, n_classes + 1)
boxes = torch.rand(1, n_queries, 4)
tgt_labels = torch.randint(high=n_classes, size=(n_targets,))
tgt_boxes = torch.rand(n_targets, 4)
matcher = HungarianMatcher()
targets = [{'labels': tgt_labels, 'boxes': tgt_boxes}]
indices_single = matcher({'pred_logits': logits, 'pred_boxes': boxes}, targets)
indices_batched = matcher({'pred_logits': logits.repeat(2, 1, 1),
'pred_boxes': boxes.repeat(2, 1, 1)}, targets * 2)
self.assertEqual(len(indices_single[0][0]), n_targets)
self.assertEqual(len(indices_single[0][1]), n_targets)
self.assertEqual(self.indices_torch2python(indices_single),
self.indices_torch2python([indices_batched[0]]))
self.assertEqual(self.indices_torch2python(indices_single),
self.indices_torch2python([indices_batched[1]]))
# test with empty targets
tgt_labels_empty = torch.randint(high=n_classes, size=(0,))
tgt_boxes_empty = torch.rand(0, 4)
targets_empty = [{'labels': tgt_labels_empty, 'boxes': tgt_boxes_empty}]
indices = matcher({'pred_logits': logits.repeat(2, 1, 1),
'pred_boxes': boxes.repeat(2, 1, 1)}, targets + targets_empty)
self.assertEqual(len(indices[1][0]), 0)
indices = matcher({'pred_logits': logits.repeat(2, 1, 1),
'pred_boxes': boxes.repeat(2, 1, 1)}, targets_empty * 2)
self.assertEqual(len(indices[0][0]), 0)
def test_position_encoding_script(self):
m1, m2 = PositionEmbeddingSine(), PositionEmbeddingLearned()
mm1, mm2 = torch.jit.script(m1), torch.jit.script(m2) # noqa
def test_backbone_script(self):
backbone = Backbone('resnet50', True, False, False)
torch.jit.script(backbone) # noqa
def test_model_script_detection(self):
model = detr_resnet50(pretrained=False).eval()
scripted_model = torch.jit.script(model)
x = nested_tensor_from_tensor_list([torch.rand(3, 200, 200), torch.rand(3, 200, 250)])
out = model(x)
out_script = scripted_model(x)
self.assertTrue(out["pred_logits"].equal(out_script["pred_logits"]))
self.assertTrue(out["pred_boxes"].equal(out_script["pred_boxes"]))
def test_model_script_panoptic(self):
model = detr_resnet50_panoptic(pretrained=False).eval()
scripted_model = torch.jit.script(model)
x = nested_tensor_from_tensor_list([torch.rand(3, 200, 200), torch.rand(3, 200, 250)])
out = model(x)
out_script = scripted_model(x)
self.assertTrue(out["pred_logits"].equal(out_script["pred_logits"]))
self.assertTrue(out["pred_boxes"].equal(out_script["pred_boxes"]))
self.assertTrue(out["pred_masks"].equal(out_script["pred_masks"]))
def test_model_detection_different_inputs(self):
model = detr_resnet50(pretrained=False).eval()
# support NestedTensor
x = nested_tensor_from_tensor_list([torch.rand(3, 200, 200), torch.rand(3, 200, 250)])
out = model(x)
self.assertIn('pred_logits', out)
# and 4d Tensor
x = torch.rand(1, 3, 200, 200)
out = model(x)
self.assertIn('pred_logits', out)
# and List[Tensor[C, H, W]]
x = torch.rand(3, 200, 200)
out = model([x])
self.assertIn('pred_logits', out)
def test_warpped_model_script_detection(self):
class WrappedDETR(nn.Module):
def __init__(self, model):
super().__init__()
self.model = model
def forward(self, inputs: List[Tensor]):
sample = nested_tensor_from_tensor_list(inputs)
return self.model(sample)
model = detr_resnet50(pretrained=False)
wrapped_model = WrappedDETR(model)
wrapped_model.eval()
scripted_model = torch.jit.script(wrapped_model)
x = [torch.rand(3, 200, 200), torch.rand(3, 200, 250)]
out = wrapped_model(x)
out_script = scripted_model(x)
self.assertTrue(out["pred_logits"].equal(out_script["pred_logits"]))
self.assertTrue(out["pred_boxes"].equal(out_script["pred_boxes"]))
@unittest.skipIf(onnxruntime is None, 'ONNX Runtime unavailable')
class ONNXExporterTester(unittest.TestCase):
@classmethod
def setUpClass(cls):
torch.manual_seed(123)
def run_model(self, model, inputs_list, tolerate_small_mismatch=False, do_constant_folding=True, dynamic_axes=None,
output_names=None, input_names=None):
model.eval()
onnx_io = io.BytesIO()
# export to onnx with the first input
torch.onnx.export(model, inputs_list[0], onnx_io,
do_constant_folding=do_constant_folding, opset_version=12,
dynamic_axes=dynamic_axes, input_names=input_names, output_names=output_names)
# validate the exported model with onnx runtime
for test_inputs in inputs_list:
with torch.no_grad():
if isinstance(test_inputs, torch.Tensor) or isinstance(test_inputs, list):
test_inputs = (nested_tensor_from_tensor_list(test_inputs),)
test_ouputs = model(*test_inputs)
if isinstance(test_ouputs, torch.Tensor):
test_ouputs = (test_ouputs,)
self.ort_validate(onnx_io, test_inputs, test_ouputs, tolerate_small_mismatch)
def ort_validate(self, onnx_io, inputs, outputs, tolerate_small_mismatch=False):
inputs, _ = torch.jit._flatten(inputs)
outputs, _ = torch.jit._flatten(outputs)
def to_numpy(tensor):
if tensor.requires_grad:
return tensor.detach().cpu().numpy()
else:
return tensor.cpu().numpy()
inputs = list(map(to_numpy, inputs))
outputs = list(map(to_numpy, outputs))
ort_session = onnxruntime.InferenceSession(onnx_io.getvalue())
# compute onnxruntime output prediction
ort_inputs = dict((ort_session.get_inputs()[i].name, inpt) for i, inpt in enumerate(inputs))
ort_outs = ort_session.run(None, ort_inputs)
for i in range(0, len(outputs)):
try:
torch.testing.assert_allclose(outputs[i], ort_outs[i], rtol=1e-03, atol=1e-05)
except AssertionError as error:
if tolerate_small_mismatch:
self.assertIn("(0.00%)", str(error), str(error))
else:
raise
def test_model_onnx_detection(self):
model = detr_resnet50(pretrained=False).eval()
dummy_image = torch.ones(1, 3, 800, 800) * 0.3
model(dummy_image)
# Test exported model on images of different size, or dummy input
self.run_model(
model,
[(torch.rand(1, 3, 750, 800),)],
input_names=["inputs"],
output_names=["pred_logits", "pred_boxes"],
tolerate_small_mismatch=True,
)
@unittest.skip("CI doesn't have enough memory")
def test_model_onnx_detection_panoptic(self):
model = detr_resnet50_panoptic(pretrained=False).eval()
dummy_image = torch.ones(1, 3, 800, 800) * 0.3
model(dummy_image)
# Test exported model on images of different size, or dummy input
self.run_model(
model,
[(torch.rand(1, 3, 750, 800),)],
input_names=["inputs"],
output_names=["pred_logits", "pred_boxes", "pred_masks"],
tolerate_small_mismatch=True,
)
if __name__ == '__main__':
unittest.main()
================================================
FILE: batchformer-v2/detr/tools/launch.py
================================================
# --------------------------------------------------------------------------------------------------------------------------
# Deformable DETR
# Copyright (c) 2020 SenseTime. All Rights Reserved.
# Licensed under the Apache License, Version 2.0 [see LICENSE for details]
# --------------------------------------------------------------------------------------------------------------------------
# Modified from https://github.com/pytorch/pytorch/blob/173f224570017b4b1a3a1a13d0bff280a54d9cd9/torch/distributed/launch.py
# --------------------------------------------------------------------------------------------------------------------------
r"""
`torch.distributed.launch` is a module that spawns up multiple distributed
training processes on each of the training nodes.
The utility can be used for single-node distributed training, in which one or
more processes per node will be spawned. The utility can be used for either
CPU training or GPU training. If the utility is used for GPU training,
each distributed process will be operating on a single GPU. This can achieve
well-improved single-node training performance. It can also be used in
multi-node distributed training, by spawning up multiple processes on each node
for well-improved multi-node distributed training performance as well.
This will especially be benefitial for systems with multiple Infiniband
interfaces that have direct-GPU support, since all of them can be utilized for
aggregated communication bandwidth.
In both cases of single-node distributed training or multi-node distributed
training, this utility will launch the given number of processes per node
(``--nproc_per_node``). If used for GPU training, this number needs to be less
or euqal to the number of GPUs on the current system (``nproc_per_node``),
and each process will be operating on a single GPU from *GPU 0 to
GPU (nproc_per_node - 1)*.
**How to use this module:**
1. Single-Node multi-process distributed training
::
>>> python -m torch.distributed.launch --nproc_per_node=NUM_GPUS_YOU_HAVE
YOUR_TRAINING_SCRIPT.py (--arg1 --arg2 --arg3 and all other
arguments of your training script)
2. Multi-Node multi-process distributed training: (e.g. two nodes)
Node 1: *(IP: 192.168.1.1, and has a free port: 1234)*
::
>>> python -m torch.distributed.launch --nproc_per_node=NUM_GPUS_YOU_HAVE
--nnodes=2 --node_rank=0 --master_addr="192.168.1.1"
--master_port=1234 YOUR_TRAINING_SCRIPT.py (--arg1 --arg2 --arg3
and all other arguments of your training script)
Node 2:
::
>>> python -m torch.distributed.launch --nproc_per_node=NUM_GPUS_YOU_HAVE
--nnodes=2 --node_rank=1 --master_addr="192.168.1.1"
--master_port=1234 YOUR_TRAINING_SCRIPT.py (--arg1 --arg2 --arg3
and all other arguments of your training script)
3. To look up what optional arguments this module offers:
::
>>> python -m torch.distributed.launch --help
**Important Notices:**
1. This utilty and multi-process distributed (single-node or
multi-node) GPU training currently only achieves the best performance using
the NCCL distributed backend. Thus NCCL backend is the recommended backend to
use for GPU training.
2. In your training program, you must parse the command-line argument:
``--local_rank=LOCAL_PROCESS_RANK``, which will be provided by this module.
If your training program uses GPUs, you should ensure that your code only
runs on the GPU device of LOCAL_PROCESS_RANK. This can be done by:
Parsing the local_rank argument
::
>>> import argparse
>>> parser = argparse.ArgumentParser()
>>> parser.add_argument("--local_rank", type=int)
>>> args = parser.parse_args()
Set your device to local rank using either
::
>>> torch.cuda.set_device(arg.local_rank) # before your code runs
or
::
>>> with torch.cuda.device(arg.local_rank):
>>> # your code to run
3. In your training program, you are supposed to call the following function
at the beginning to start the distributed backend. You need to make sure that
the init_method uses ``env://``, which is the only supported ``init_method``
by this module.
::
torch.distributed.init_process_group(backend='YOUR BACKEND',
init_method='env://')
4. In your training program, you can either use regular distributed functions
or use :func:`torch.nn.parallel.DistributedDataParallel` module. If your
training program uses GPUs for training and you would like to use
:func:`torch.nn.parallel.DistributedDataParallel` module,
here is how to configure it.
::
model = torch.nn.parallel.DistributedDataParallel(model,
device_ids=[arg.local_rank],
output_device=arg.local_rank)
Please ensure that ``device_ids`` argument is set to be the only GPU device id
that your code will be operating on. This is generally the local rank of the
process. In other words, the ``device_ids`` needs to be ``[args.local_rank]``,
and ``output_device`` needs to be ``args.local_rank`` in order to use this
utility
5. Another way to pass ``local_rank`` to the subprocesses via environment variable
``LOCAL_RANK``. This behavior is enabled when you launch the script with
``--use_env=True``. You must adjust the subprocess example above to replace
``args.local_rank`` with ``os.environ['LOCAL_RANK']``; the launcher
will not pass ``--local_rank`` when you specify this flag.
.. warning::
``local_rank`` is NOT globally unique: it is only unique per process
on a machine. Thus, don't use it to decide if you should, e.g.,
write to a networked filesystem. See
https://github.com/pytorch/pytorch/issues/12042 for an example of
how things can go wrong if you don't do this correctly.
"""
import sys
import subprocess
import os
import socket
from argparse import ArgumentParser, REMAINDER
import torch
def parse_args():
"""
Helper function parsing the command line options
@retval ArgumentParser
"""
parser = ArgumentParser(description="PyTorch distributed training launch "
"helper utilty that will spawn up "
"multiple distributed processes")
# Optional arguments for the launch helper
parser.add_argument("--nnodes", type=int, default=1,
help="The number of nodes to use for distributed "
"training")
parser.add_argument("--node_rank", type=int, default=0,
help="The rank of the node for multi-node distributed "
"training")
parser.add_argument("--base_rank", type=int, default=0,
help="The rank of the node for multi-node distributed "
"training")
parser.add_argument("--nproc_per_node", type=int, default=1,
help="The number of processes to launch on each node, "
"for GPU training, this is recommended to be set "
"to the number of GPUs in your system so that "
"each process can be bound to a single GPU.")
parser.add_argument("--master_addr", default="127.0.0.1", type=str,
help="Master node (rank 0)'s address, should be either "
"the IP address or the hostname of node 0, for "
"single node multi-proc training, the "
"--master_addr can simply be 127.0.0.1")
parser.add_argument("--master_port", default=29500, type=int,
help="Master node (rank 0)'s free port that needs to "
"be used for communciation during distributed "
"training")
# positional
parser.add_argument("training_script", type=str,
help="The full path to the single GPU training "
"program/script to be launched in parallel, "
"followed by all the arguments for the "
"training script")
# rest from the training program
parser.add_argument('training_script_args', nargs=REMAINDER)
return parser.parse_args()
def main():
args = parse_args()
# world size in terms of number of processes
dist_world_size = args.nproc_per_node * args.nnodes
# set PyTorch distributed related environmental variables
current_env = os.environ.copy()
current_env["MASTER_ADDR"] = args.master_addr
current_env["MASTER_PORT"] = str(args.master_port)
current_env["WORLD_SIZE"] = str(dist_world_size)
processes = []
for local_rank in range(0, args.nproc_per_node):
# each process's rank
dist_rank = args.nproc_per_node * args.node_rank + local_rank
current_env["RANK"] = str(dist_rank)
current_env["LOCAL_RANK"] = str(local_rank)
cmd = [args.training_script] + args.training_script_args
print(cmd)
process = subprocess.Popen(cmd, env=current_env)
processes.append(process)
for process in processes:
process.wait()
if process.returncode != 0:
raise subprocess.CalledProcessError(returncode=process.returncode,
cmd=process.args)
if __name__ == "__main__":
main()
================================================
FILE: batchformer-v2/detr/tools/run_dist_launch.sh
================================================
#!/usr/bin/env bash
# ------------------------------------------------------------------------
# Deformable DETR
# Copyright (c) 2020 SenseTime. All Rights Reserved.
# Licensed under the Apache License, Version 2.0 [see LICENSE for details]
# ------------------------------------------------------------------------
#source ~/.bashrc
#conda activate torch150
source /usr/local/anaconda/4.2.0/etc/profile.d/conda.sh
conda activate pyt
#module load gcc
set -x
GPUS=$1
RUN_COMMAND=${@:2}
#if [ $GPUS -lt 8 ]; then
# GPUS_PER_NODE=${GPUS_PER_NODE:-$GPUS}
#else
# GPUS_PER_NODE=${GPUS_PER_NODE:-8}
#fi
MASTER_ADDR=${MASTER_ADDR:-"127.0.0.1"}
MASTER_PORT=${MASTER_PORT:-"29500"}
NODE_RANK=${NODE_RANK:-0}
BASE_RANK=${BASE_RANK:-0}
#env
echo $GPUS
echo $GPUS_PER_NODE
#let "NNODES=GPUS/GPUS_PER_NODE"
python ./tools/launch.py \
--nnodes ${NNODES} \
--node_rank ${NODE_RANK} \
--base_rank ${BASE_RANK} \
--master_addr ${MASTER_ADDR} \
--master_port ${MASTER_PORT} \
--nproc_per_node ${GPUS_PER_NODE} \
${RUN_COMMAND}
================================================
FILE: batchformer-v2/detr/tools/run_dist_slurm.sh
================================================
#!/usr/bin/env bash
# --------------------------------------------------------------------------------------------------------------------------
# Deformable DETR
# Copyright (c) 2020 SenseTime. All Rights Reserved.
# Licensed under the Apache License, Version 2.0 [see LICENSE for details]
# --------------------------------------------------------------------------------------------------------------------------
# Modified from https://github.com/open-mmlab/mmdetection/blob/3b53fe15d87860c6941f3dda63c0f27422da6266/tools/slurm_train.sh
# --------------------------------------------------------------------------------------------------------------------------
set -x
PARTITION=$1
JOB_NAME=$2
GPUS=$3
RUN_COMMAND=${@:4}
if [ $GPUS -lt 8 ]; then
GPUS_PER_NODE=${GPUS_PER_NODE:-$GPUS}
else
GPUS_PER_NODE=${GPUS_PER_NODE:-8}
fi
CPUS_PER_TASK=${CPUS_PER_TASK:-4}
SRUN_ARGS=${SRUN_ARGS:-""}
srun -p ${PARTITION} \
--job-name=${JOB_NAME} \
--gres=gpu:${GPUS_PER_NODE} \
--ntasks=${GPUS} \
--ntasks-per-node=${GPUS_PER_NODE} \
--cpus-per-task=${CPUS_PER_TASK} \
--kill-on-bad-exit=1 \
${SRUN_ARGS} \
${RUN_COMMAND}
================================================
FILE: batchformer-v2/detr/tox.ini
================================================
[flake8]
max-line-length = 120
ignore = F401,E402,F403,W503,W504
================================================
FILE: batchformer-v2/detr/util/__init__.py
================================================
# Copyright (c) Facebook, Inc. and its affiliates. All Rights Reserved
================================================
FILE: batchformer-v2/detr/util/box_ops.py
================================================
# Copyright (c) Facebook, Inc. and its affiliates. All Rights Reserved
"""
Utilities for bounding box manipulation and GIoU.
"""
import torch
from torchvision.ops.boxes import box_area
def box_cxcywh_to_xyxy(x):
x_c, y_c, w, h = x.unbind(-1)
b = [(x_c - 0.5 * w), (y_c - 0.5 * h),
(x_c + 0.5 * w), (y_c + 0.5 * h)]
return torch.stack(b, dim=-1)
def box_xyxy_to_cxcywh(x):
x0, y0, x1, y1 = x.unbind(-1)
b = [(x0 + x1) / 2, (y0 + y1) / 2,
(x1 - x0), (y1 - y0)]
return torch.stack(b, dim=-1)
# modified from torchvision to also return the union
def box_iou(boxes1, boxes2):
area1 = box_area(boxes1)
area2 = box_area(boxes2)
lt = torch.max(boxes1[:, None, :2], boxes2[:, :2]) # [N,M,2]
rb = torch.min(boxes1[:, None, 2:], boxes2[:, 2:]) # [N,M,2]
wh = (rb - lt).clamp(min=0) # [N,M,2]
inter = wh[:, :, 0] * wh[:, :, 1] # [N,M]
union = area1[:, None] + area2 - inter
iou = inter / union
return iou, union
def generalized_box_iou(boxes1, boxes2):
"""
Generalized IoU from https://giou.stanford.edu/
The boxes should be in [x0, y0, x1, y1] format
Returns a [N, M] pairwise matrix, where N = len(boxes1)
and M = len(boxes2)
"""
# degenerate boxes gives inf / nan results
# so do an early check
assert (boxes1[:, 2:] >= boxes1[:, :2]).all()
assert (boxes2[:, 2:] >= boxes2[:, :2]).all()
iou, union = box_iou(boxes1, boxes2)
lt = torch.min(boxes1[:, None, :2], boxes2[:, :2])
rb = torch.max(boxes1[:, None, 2:], boxes2[:, 2:])
wh = (rb - lt).clamp(min=0) # [N,M,2]
area = wh[:, :, 0] * wh[:, :, 1]
return iou - (area - union) / area
def masks_to_boxes(masks):
"""Compute the bounding boxes around the provided masks
The masks should be in format [N, H, W] where N is the number of masks, (H, W) are the spatial dimensions.
Returns a [N, 4] tensors, with the boxes in xyxy format
"""
if masks.numel() == 0:
return torch.zeros((0, 4), device=masks.device)
h, w = masks.shape[-2:]
y = torch.arange(0, h, dtype=torch.float)
x = torch.arange(0, w, dtype=torch.float)
y, x = torch.meshgrid(y, x)
x_mask = (masks * x.unsqueeze(0))
x_max = x_mask.flatten(1).max(-1)[0]
x_min = x_mask.masked_fill(~(masks.bool()), 1e8).flatten(1).min(-1)[0]
y_mask = (masks * y.unsqueeze(0))
y_max = y_mask.flatten(1).max(-1)[0]
y_min = y_mask.masked_fill(~(masks.bool()), 1e8).flatten(1).min(-1)[0]
return torch.stack([x_min, y_min, x_max, y_max], 1)
================================================
FILE: batchformer-v2/detr/util/misc.py
================================================
# Copyright (c) Facebook, Inc. and its affiliates. All Rights Reserved
"""
Misc functions, including distributed helpers.
Mostly copy-paste from torchvision references.
"""
import os
import subprocess
import time
from collections import defaultdict, deque
import datetime
import pickle
from typing import Optional, List
import torch
import torch.distributed as dist
from torch import Tensor
# needed due to empty tensor bug in pytorch and torchvision 0.5
import torchvision
print(torchvision.__version__)
import math
if float(torchvision.__version__[:3]) < 0.5 and float(torchvision.__version__[:3]) > 0.1:
import math
from torchvision.ops.misc import _NewEmptyTensorOp
def _check_size_scale_factor(dim, size, scale_factor):
# type: (int, Optional[List[int]], Optional[float]) -> None
if size is None and scale_factor is None:
raise ValueError("either size or scale_factor should be defined")
if size is not None and scale_factor is not None:
raise ValueError("only one of size or scale_factor should be defined")
if not (scale_factor is not None and len(scale_factor) != dim):
raise ValueError(
"scale_factor shape must match input shape. "
"Input is {}D, scale_factor size is {}".format(dim, len(scale_factor))
)
def _output_size(dim, input, size, scale_factor):
# type: (int, Tensor, Optional[List[int]], Optional[float]) -> List[int]
assert dim == 2
_check_size_scale_factor(dim, size, scale_factor)
if size is not None:
return size
# if dim is not 2 or scale_factor is iterable use _ntuple instead of concat
assert scale_factor is not None and isinstance(scale_factor, (int, float))
scale_factors = [scale_factor, scale_factor]
# math.floor might return float in py2.7
return [
int(math.floor(input.size(i + 2) * scale_factors[i])) for i in range(dim)
]
elif float(torchvision.__version__[:3]) < 0.7 and float(torchvision.__version__[:3]) > 0.1:
from torchvision.ops import _new_empty_tensor
from torchvision.ops.misc import _output_size
class SmoothedValue(object):
"""Track a series of values and provide access to smoothed values over a
window or the global series average.
"""
def __init__(self, window_size=20, fmt=None):
if fmt is None:
fmt = "{median:.4f} ({global_avg:.4f})"
self.deque = deque(maxlen=window_size)
self.total = 0.0
self.count = 0
self.fmt = fmt
def update(self, value, n=1):
self.deque.append(value)
self.count += n
self.total += value * n
def synchronize_between_processes(self):
"""
Warning: does not synchronize the deque!
"""
if not is_dist_avail_and_initialized():
return
t = torch.tensor([self.count, self.total], dtype=torch.float64, device='cuda')
dist.barrier()
dist.all_reduce(t)
t = t.tolist()
self.count = int(t[0])
self.total = t[1]
@property
def median(self):
d = torch.tensor(list(self.deque))
return d.median().item()
@property
def avg(self):
d = torch.tensor(list(self.deque), dtype=torch.float32)
return d.mean().item()
@property
def global_avg(self):
return self.total / self.count
@property
def max(self):
return max(self.deque)
@property
def value(self):
return self.deque[-1]
def __str__(self):
return self.fmt.format(
median=self.median,
avg=self.avg,
global_avg=self.global_avg,
max=self.max,
value=self.value)
def all_gather(data):
"""
Run all_gather on arbitrary picklable data (not necessarily tensors)
Args:
data: any picklable object
Returns:
list[data]: list of data gathered from each rank
"""
world_size = get_world_size()
if world_size == 1:
return [data]
# serialized to a Tensor
buffer = pickle.dumps(data)
storage = torch.ByteStorage.from_buffer(buffer)
tensor = torch.ByteTensor(storage).to("cuda")
# obtain Tensor size of each rank
local_size = torch.tensor([tensor.numel()], device="cuda")
size_list = [torch.tensor([0], device="cuda") for _ in range(world_size)]
dist.all_gather(size_list, local_size)
size_list = [int(size.item()) for size in size_list]
max_size = max(size_list)
# receiving Tensor from all ranks
# we pad the tensor because torch all_gather does not support
# gathering tensors of different shapes
tensor_list = []
for _ in size_list:
tensor_list.append(torch.empty((max_size,), dtype=torch.uint8, device="cuda"))
if local_size != max_size:
padding = torch.empty(size=(max_size - local_size,), dtype=torch.uint8, device="cuda")
tensor = torch.cat((tensor, padding), dim=0)
dist.all_gather(tensor_list, tensor)
data_list = []
for size, tensor in zip(size_list, tensor_list):
buffer = tensor.cpu().numpy().tobytes()[:size]
data_list.append(pickle.loads(buffer))
return data_list
def reduce_dict(input_dict, average=True):
"""
Args:
input_dict (dict): all the values will be reduced
average (bool): whether to do average or sum
Reduce the values in the dictionary from all processes so that all processes
have the averaged results. Returns a dict with the same fields as
input_dict, after reduction.
"""
world_size = get_world_size()
if world_size < 2:
return input_dict
with torch.no_grad():
names = []
values = []
# sort the keys so that they are consistent across processes
for k in sorted(input_dict.keys()):
names.append(k)
values.append(input_dict[k])
values = torch.stack(values, dim=0)
dist.all_reduce(values)
if average:
values /= world_size
reduced_dict = {k: v for k, v in zip(names, values)}
return reduced_dict
class MetricLogger(object):
def __init__(self, delimiter="\t"):
self.meters = defaultdict(SmoothedValue)
self.delimiter = delimiter
def update(self, **kwargs):
for k, v in kwargs.items():
if isinstance(v, torch.Tensor):
v = v.item()
assert isinstance(v, (float, int))
self.meters[k].update(v)
def __getattr__(self, attr):
if attr in self.meters:
return self.meters[attr]
if attr in self.__dict__:
return self.__dict__[attr]
raise AttributeError("'{}' object has no attribute '{}'".format(
type(self).__name__, attr))
def __str__(self):
loss_str = []
for name, meter in self.meters.items():
loss_str.append(
"{}: {}".format(name, str(meter))
)
return self.delimiter.join(loss_str)
def synchronize_between_processes(self):
for meter in self.meters.values():
meter.synchronize_between_processes()
def add_meter(self, name, meter):
self.meters[name] = meter
def log_every(self, iterable, print_freq, header=None):
i = 0
if not header:
header = ''
start_time = time.time()
end = time.time()
iter_time = SmoothedValue(fmt='{avg:.4f}')
data_time = SmoothedValue(fmt='{avg:.4f}')
space_fmt = ':' + str(len(str(len(iterable)))) + 'd'
if torch.cuda.is_available():
log_msg = self.delimiter.join([
header,
'[{0' + space_fmt + '}/{1}]',
'eta: {eta}',
'{meters}',
'time: {time}',
'data: {data}',
'max mem: {memory:.0f}'
])
else:
log_msg = self.delimiter.join([
header,
'[{0' + space_fmt + '}/{1}]',
'eta: {eta}',
'{meters}',
'time: {time}',
'data: {data}'
])
MB = 1024.0 * 1024.0
for obj in iterable:
data_time.update(time.time() - end)
yield obj
iter_time.update(time.time() - end)
if i % print_freq == 0 or i == len(iterable) - 1:
eta_seconds = iter_time.global_avg * (len(iterable) - i)
eta_string = str(datetime.timedelta(seconds=int(eta_seconds)))
if torch.cuda.is_available():
print(log_msg.format(
i, len(iterable), eta=eta_string,
meters=str(self),
time=str(iter_time), data=str(data_time),
memory=torch.cuda.max_memory_allocated() / MB))
else:
print(log_msg.format(
i, len(iterable), eta=eta_string,
meters=str(self),
time=str(iter_time), data=str(data_time)))
i += 1
end = time.time()
total_time = time.time() - start_time
total_time_str = str(datetime.timedelta(seconds=int(total_time)))
print('{} Total time: {} ({:.4f} s / it)'.format(
header, total_time_str, total_time / len(iterable)))
def get_sha():
cwd = os.path.dirname(os.path.abspath(__file__))
def _run(command):
return subprocess.check_output(command, cwd=cwd).decode('ascii').strip()
sha = 'N/A'
diff = "clean"
branch = 'N/A'
try:
sha = _run(['git', 'rev-parse', 'HEAD'])
subprocess.check_output(['git', 'diff'], cwd=cwd)
diff = _run(['git', 'diff-index', 'HEAD'])
diff = "has uncommited changes" if diff else "clean"
branch = _run(['git', 'rev-parse', '--abbrev-ref', 'HEAD'])
except Exception:
pass
message = f"sha: {sha}, status: {diff}, branch: {branch}"
return message
def collate_fn(batch):
batch = list(zip(*batch))
batch[0] = nested_tensor_from_tensor_list(batch[0])
return tuple(batch)
def _max_by_axis(the_list):
# type: (List[List[int]]) -> List[int]
maxes = the_list[0]
for sublist in the_list[1:]:
for index, item in enumerate(sublist):
maxes[index] = max(maxes[index], item)
return maxes
class NestedTensor(object):
def __init__(self, tensors, mask: Optional[Tensor]):
self.tensors = tensors
self.mask = mask
def to(self, device):
# type: (Device) -> NestedTensor # noqa
cast_tensor = self.tensors.to(device)
mask = self.mask
if mask is not None:
assert mask is not None
cast_mask = mask.to(device)
else:
cast_mask = None
return NestedTensor(cast_tensor, cast_mask)
def decompose(self):
return self.tensors, self.mask
def __repr__(self):
return str(self.tensors)
def nested_tensor_from_tensor_list(tensor_list: List[Tensor]):
# TODO make this more general
if tensor_list[0].ndim == 3:
if torchvision._is_tracing():
# nested_tensor_from_tensor_list() does not export well to ONNX
# call _onnx_nested_tensor_from_tensor_list() instead
return _onnx_nested_tensor_from_tensor_list(tensor_list)
# TODO make it support different-sized images
max_size = _max_by_axis([list(img.shape) for img in tensor_list])
# min_size = tuple(min(s) for s in zip(*[img.shape for img in tensor_list]))
batch_shape = [len(tensor_list)] + max_size
b, c, h, w = batch_shape
dtype = tensor_list[0].dtype
device = tensor_list[0].device
tensor = torch.zeros(batch_shape, dtype=dtype, device=device)
mask = torch.ones((b, h, w), dtype=torch.bool, device=device)
for img, pad_img, m in zip(tensor_list, tensor, mask):
pad_img[: img.shape[0], : img.shape[1], : img.shape[2]].copy_(img)
m[: img.shape[1], :img.shape[2]] = False
else:
raise ValueError('not supported')
return NestedTensor(tensor, mask)
# _onnx_nested_tensor_from_tensor_list() is an implementation of
# nested_tensor_from_tensor_list() that is supported by ONNX tracing.
@torch.jit.unused
def _onnx_nested_tensor_from_tensor_list(tensor_list: List[Tensor]) -> NestedTensor:
max_size = []
for i in range(tensor_list[0].dim()):
max_size_i = torch.max(torch.stack([img.shape[i] for img in tensor_list]).to(torch.float32)).to(torch.int64)
max_size.append(max_size_i)
max_size = tuple(max_size)
# work around for
# pad_img[: img.shape[0], : img.shape[1], : img.shape[2]].copy_(img)
# m[: img.shape[1], :img.shape[2]] = False
# which is not yet supported in onnx
padded_imgs = []
padded_masks = []
for img in tensor_list:
padding = [(s1 - s2) for s1, s2 in zip(max_size, tuple(img.shape))]
padded_img = torch.nn.functional.pad(img, (0, padding[2], 0, padding[1], 0, padding[0]))
padded_imgs.append(padded_img)
m = torch.zeros_like(img[0], dtype=torch.int, device=img.device)
padded_mask = torch.nn.functional.pad(m, (0, padding[2], 0, padding[1]), "constant", 1)
padded_masks.append(padded_mask.to(torch.bool))
tensor = torch.stack(padded_imgs)
mask = torch.stack(padded_masks)
return NestedTensor(tensor, mask=mask)
def setup_for_distributed(is_master):
"""
This function disables printing when not in master process
"""
import builtins as __builtin__
builtin_print = __builtin__.print
def print(*args, **kwargs):
force = kwargs.pop('force', False)
if is_master or force:
builtin_print(*args, **kwargs)
__builtin__.print = print
def is_dist_avail_and_initialized():
if not dist.is_available():
return False
if not dist.is_initialized():
return False
return True
def get_world_size():
if not is_dist_avail_and_initialized():
return 1
return dist.get_world_size()
def get_rank():
if not is_dist_avail_and_initialized():
return 0
return dist.get_rank()
def is_main_process():
return get_rank() == 0
def save_on_master(*args, **kwargs):
if is_main_process():
torch.save(*args, **kwargs)
def init_distributed_mode(args):
if 'RANK' in os.environ and 'WORLD_SIZE' in os.environ:
args.rank = int(os.environ["RANK"])
args.world_size = int(os.environ['WORLD_SIZE'])
args.gpu = int(os.environ['LOCAL_RANK'])
elif 'SLURM_PROCID' in os.environ:
args.rank = int(os.environ['SLURM_PROCID'])
args.gpu = args.rank % torch.cuda.device_count()
else:
print('Not using distributed mode')
args.distributed = False
return
args.distributed = True
torch.cuda.set_device(args.gpu)
args.dist_backend = 'nccl'
print('| distributed init (rank {}): {}'.format(
args.rank, args.dist_url), flush=True)
torch.distributed.init_process_group(backend=args.dist_backend, init_method=args.dist_url,
world_size=args.world_size, rank=args.rank)
torch.distributed.barrier()
setup_for_distributed(args.rank == 0)
@torch.no_grad()
def accuracy(output, target, topk=(1,)):
"""Computes the precision@k for the specified values of k"""
if target.numel() == 0:
return [torch.zeros([], device=output.device)]
maxk = max(topk)
batch_size = target.size(0)
_, pred = output.topk(maxk, 1, True, True)
pred = pred.t()
correct = pred.eq(target.view(1, -1).expand_as(pred))
res = []
for k in topk:
correct_k = correct[:k].view(-1).float().sum(0)
res.append(correct_k.mul_(100.0 / batch_size))
return res
def interpolate(input, size=None, scale_factor=None, mode="nearest", align_corners=None):
# type: (Tensor, Optional[List[int]], Optional[float], str, Optional[bool]) -> Tensor
"""
Equivalent to nn.functional.interpolate, but with support for empty batch sizes.
This will eventually be supported natively by PyTorch, and this
class can go away.
"""
if float(torchvision.__version__[:3]) < 0.7 and not torchvision.__version__.startswith('0.10'):
if input.numel() > 0:
return torch.nn.functional.interpolate(
input, size, scale_factor, mode, align_corners
)
output_shape = _output_size(2, input, size, scale_factor)
output_shape = list(input.shape[:-2]) + list(output_shape)
return _new_empty_tensor(input, output_shape)
else:
return torchvision.ops.misc.interpolate(input, size, scale_factor, mode, align_corners)
================================================
FILE: batchformer-v2/detr/util/plot_utils.py
================================================
"""
Plotting utilities to visualize training logs.
"""
import torch
import pandas as pd
import numpy as np
import seaborn as sns
import matplotlib.pyplot as plt
from pathlib import Path, PurePath
def plot_logs(logs, fields=('class_error', 'loss_bbox_unscaled', 'mAP'), ewm_col=0, log_name='log.txt'):
'''
Function to plot specific fields from training log(s). Plots both training and test results.
:: Inputs - logs = list containing Path objects, each pointing to individual dir with a log file
- fields = which results to plot from each log file - plots both training and test for each field.
- ewm_col = optional, which column to use as the exponential weighted smoothing of the plots
- log_name = optional, name of log file if different than default 'log.txt'.
:: Outputs - matplotlib plots of results in fields, color coded for each log file.
- solid lines are training results, dashed lines are test results.
'''
func_name = "plot_utils.py::plot_logs"
# verify logs is a list of Paths (list[Paths]) or single Pathlib object Path,
# convert single Path to list to avoid 'not iterable' error
if not isinstance(logs, list):
if isinstance(logs, PurePath):
logs = [logs]
print(f"{func_name} info: logs param expects a list argument, converted to list[Path].")
else:
raise ValueError(f"{func_name} - invalid argument for logs parameter.\n \
Expect list[Path] or single Path obj, received {type(logs)}")
# Quality checks - verify valid dir(s), that every item in list is Path object, and that log_name exists in each dir
for i, dir in enumerate(logs):
if not isinstance(dir, PurePath):
raise ValueError(f"{func_name} - non-Path object in logs argument of {type(dir)}: \n{dir}")
if not dir.exists():
raise ValueError(f"{func_name} - invalid directory in logs argument:\n{dir}")
# verify log_name exists
fn = Path(dir / log_name)
if not fn.exists():
print(f"-> missing {log_name}. Have you gotten to Epoch 1 in training?")
print(f"--> full path of missing log file: {fn}")
return
# load log file(s) and plot
dfs = [pd.read_json(Path(p) / log_name, lines=True) for p in logs]
fig, axs = plt.subplots(ncols=len(fields), figsize=(16, 5))
for df, color in zip(dfs, sns.color_palette(n_colors=len(logs))):
for j, field in enumerate(fields):
if field == 'mAP':
coco_eval = pd.DataFrame(
np.stack(df.test_coco_eval_bbox.dropna().values)[:, 1]
).ewm(com=ewm_col).mean()
axs[j].plot(coco_eval, c=color)
else:
df.interpolate().ewm(com=ewm_col).mean().plot(
y=[f'train_{field}', f'test_{field}'],
ax=axs[j],
color=[color] * 2,
style=['-', '--']
)
for ax, field in zip(axs, fields):
ax.legend([Path(p).name for p in logs])
ax.set_title(field)
def plot_precision_recall(files, naming_scheme='iter'):
if naming_scheme == 'exp_id':
# name becomes exp_id
names = [f.parts[-3] for f in files]
elif naming_scheme == 'iter':
names = [f.stem for f in files]
else:
raise ValueError(f'not supported {naming_scheme}')
fig, axs = plt.subplots(ncols=2, figsize=(16, 5))
for f, color, name in zip(files, sns.color_palette("Blues", n_colors=len(files)), names):
data = torch.load(f)
# precision is n_iou, n_points, n_cat, n_area, max_det
precision = data['precision']
recall = data['params'].recThrs
scores = data['scores']
# take precision for all classes, all areas and 100 detections
precision = precision[0, :, :, 0, -1].mean(1)
scores = scores[0, :, :, 0, -1].mean(1)
prec = precision.mean()
rec = data['recall'][0, :, 0, -1].mean()
print(f'{naming_scheme} {name}: mAP@50={prec * 100: 05.1f}, ' +
f'score={scores.mean():0.3f}, ' +
f'f1={2 * prec * rec / (prec + rec + 1e-8):0.3f}'
)
axs[0].plot(recall, precision, c=color)
axs[1].plot(recall, scores, c=color)
axs[0].set_title('Precision / Recall')
axs[0].legend(names)
axs[1].set_title('Scores / Recall')
axs[1].legend(names)
return fig, axs
================================================
FILE: czsl/LICENSE
================================================
GNU GENERAL PUBLIC LICENSE
Version 3, 29 June 2007
Copyright (C) 2007 Free Software Foundation, Inc.
Everyone is permitted to copy and distribute verbatim copies
of this license document, but changing it is not allowed.
Preamble
The GNU General Public License is a free, copyleft license for
software and other kinds of works.
The licenses for most software and other practical works are designed
to take away your freedom to share and change the works. By contrast,
the GNU General Public License is intended to guarantee your freedom to
share and change all versions of a program--to make sure it remains free
software for all its users. We, the Free Software Foundation, use the
GNU General Public License for most of our software; it applies also to
any other work released this way by its authors. You can apply it to
your programs, too.
When we speak of free software, we are referring to freedom, not
price. Our General Public Licenses are designed to make sure that you
have the freedom to distribute copies of free software (and charge for
them if you wish), that you receive source code or can get it if you
want it, that you can change the software or use pieces of it in new
free programs, and that you know you can do these things.
To protect your rights, we need to prevent others from denying you
these rights or asking you to surrender the rights. Therefore, you have
certain responsibilities if you distribute copies of the software, or if
you modify it: responsibilities to respect the freedom of others.
For example, if you distribute copies of such a program, whether
gratis or for a fee, you must pass on to the recipients the same
freedoms that you received. You must make sure that they, too, receive
or can get the source code. And you must show them these terms so they
know their rights.
Developers that use the GNU GPL protect your rights with two steps:
(1) assert copyright on the software, and (2) offer you this License
giving you legal permission to copy, distribute and/or modify it.
For the developers' and authors' protection, the GPL clearly explains
that there is no warranty for this free software. For both users' and
authors' sake, the GPL requires that modified versions be marked as
changed, so that their problems will not be attributed erroneously to
authors of previous versions.
Some devices are designed to deny users access to install or run
modified versions of the software inside them, although the manufacturer
can do so. This is fundamentally incompatible with the aim of
protecting users' freedom to change the software. The systematic
pattern of such abuse occurs in the area of products for individuals to
use, which is precisely where it is most unacceptable. Therefore, we
have designed this version of the GPL to prohibit the practice for those
products. If such problems arise substantially in other domains, we
stand ready to extend this provision to those domains in future versions
of the GPL, as needed to protect the freedom of users.
Finally, every program is threatened constantly by software patents.
States should not allow patents to restrict development and use of
software on general-purpose computers, but in those that do, we wish to
avoid the special danger that patents applied to a free program could
make it effectively proprietary. To prevent this, the GPL assures that
patents cannot be used to render the program non-free.
The precise terms and conditions for copying, distribution and
modification follow.
TERMS AND CONDITIONS
0. Definitions.
"This License" refers to version 3 of the GNU General Public License.
"Copyright" also means copyright-like laws that apply to other kinds of
works, such as semiconductor masks.
"The Program" refers to any copyrightable work licensed under this
License. Each licensee is addressed as "you". "Licensees" and
"recipients" may be individuals or organizations.
To "modify" a work means to copy from or adapt all or part of the work
in a fashion requiring copyright permission, other than the making of an
exact copy. The resulting work is called a "modified version" of the
earlier work or a work "based on" the earlier work.
A "covered work" means either the unmodified Program or a work based
on the Program.
To "propagate" a work means to do anything with it that, without
permission, would make you directly or secondarily liable for
infringement under applicable copyright law, except executing it on a
computer or modifying a private copy. Propagation includes copying,
distribution (with or without modification), making available to the
public, and in some countries other activities as well.
To "convey" a work means any kind of propagation that enables other
parties to make or receive copies. Mere interaction with a user through
a computer network, with no transfer of a copy, is not conveying.
An interactive user interface displays "Appropriate Legal Notices"
to the extent that it includes a convenient and prominently visible
feature that (1) displays an appropriate copyright notice, and (2)
tells the user that there is no warranty for the work (except to the
extent that warranties are provided), that licensees may convey the
work under this License, and how to view a copy of this License. If
the interface presents a list of user commands or options, such as a
menu, a prominent item in the list meets this criterion.
1. Source Code.
The "source code" for a work means the preferred form of the work
for making modifications to it. "Object code" means any non-source
form of a work.
A "Standard Interface" means an interface that either is an official
standard defined by a recognized standards body, or, in the case of
interfaces specified for a particular programming language, one that
is widely used among developers working in that language.
The "System Libraries" of an executable work include anything, other
than the work as a whole, that (a) is included in the normal form of
packaging a Major Component, but which is not part of that Major
Component, and (b) serves only to enable use of the work with that
Major Component, or to implement a Standard Interface for which an
implementation is available to the public in source code form. A
"Major Component", in this context, means a major essential component
(kernel, window system, and so on) of the specific operating system
(if any) on which the executable work runs, or a compiler used to
produce the work, or an object code interpreter used to run it.
The "Corresponding Source" for a work in object code form means all
the source code needed to generate, install, and (for an executable
work) run the object code and to modify the work, including scripts to
control those activities. However, it does not include the work's
System Libraries, or general-purpose tools or generally available free
programs which are used unmodified in performing those activities but
which are not part of the work. For example, Corresponding Source
includes interface definition files associated with source files for
the work, and the source code for shared libraries and dynamically
linked subprograms that the work is specifically designed to require,
such as by intimate data communication or control flow between those
subprograms and other parts of the work.
The Corresponding Source need not include anything that users
can regenerate automatically from other parts of the Corresponding
Source.
The Corresponding Source for a work in source code form is that
same work.
2. Basic Permissions.
All rights granted under this License are granted for the term of
copyright on the Program, and are irrevocable provided the stated
conditions are met. This License explicitly affirms your unlimited
permission to run the unmodified Program. The output from running a
covered work is covered by this License only if the output, given its
content, constitutes a covered work. This License acknowledges your
rights of fair use or other equivalent, as provided by copyright law.
You may make, run and propagate covered works that you do not
convey, without conditions so long as your license otherwise remains
in force. You may convey covered works to others for the sole purpose
of having them make modifications exclusively for you, or provide you
with facilities for running those works, provided that you comply with
the terms of this License in conveying all material for which you do
not control copyright. Those thus making or running the covered works
for you must do so exclusively on your behalf, under your direction
and control, on terms that prohibit them from making any copies of
your copyrighted material outside their relationship with you.
Conveying under any other circumstances is permitted solely under
the conditions stated below. Sublicensing is not allowed; section 10
makes it unnecessary.
3. Protecting Users' Legal Rights From Anti-Circumvention Law.
No covered work shall be deemed part of an effective technological
measure under any applicable law fulfilling obligations under article
11 of the WIPO copyright treaty adopted on 20 December 1996, or
similar laws prohibiting or restricting circumvention of such
measures.
When you convey a covered work, you waive any legal power to forbid
circumvention of technological measures to the extent such circumvention
is effected by exercising rights under this License with respect to
the covered work, and you disclaim any intention to limit operation or
modification of the work as a means of enforcing, against the work's
users, your or third parties' legal rights to forbid circumvention of
technological measures.
4. Conveying Verbatim Copies.
You may convey verbatim copies of the Program's source code as you
receive it, in any medium, provided that you conspicuously and
appropriately publish on each copy an appropriate copyright notice;
keep intact all notices stating that this License and any
non-permissive terms added in accord with section 7 apply to the code;
keep intact all notices of the absence of any warranty; and give all
recipients a copy of this License along with the Program.
You may charge any price or no price for each copy that you convey,
and you may offer support or warranty protection for a fee.
5. Conveying Modified Source Versions.
You may convey a work based on the Program, or the modifications to
produce it from the Program, in the form of source code under the
terms of section 4, provided that you also meet all of these conditions:
a) The work must carry prominent notices stating that you modified
it, and giving a relevant date.
b) The work must carry prominent notices stating that it is
released under this License and any conditions added under section
7. This requirement modifies the requirement in section 4 to
"keep intact all notices".
c) You must license the entire work, as a whole, under this
License to anyone who comes into possession of a copy. This
License will therefore apply, along with any applicable section 7
additional terms, to the whole of the work, and all its parts,
regardless of how they are packaged. This License gives no
permission to license the work in any other way, but it does not
invalidate such permission if you have separately received it.
d) If the work has interactive user interfaces, each must display
Appropriate Legal Notices; however, if the Program has interactive
interfaces that do not display Appropriate Legal Notices, your
work need not make them do so.
A compilation of a covered work with other separate and independent
works, which are not by their nature extensions of the covered work,
and which are not combined with it such as to form a larger program,
in or on a volume of a storage or distribution medium, is called an
"aggregate" if the compilation and its resulting copyright are not
used to limit the access or legal rights of the compilation's users
beyond what the individual works permit. Inclusion of a covered work
in an aggregate does not cause this License to apply to the other
parts of the aggregate.
6. Conveying Non-Source Forms.
You may convey a covered work in object code form under the terms
of sections 4 and 5, provided that you also convey the
machine-readable Corresponding Source under the terms of this License,
in one of these ways:
a) Convey the object code in, or embodied in, a physical product
(including a physical distribution medium), accompanied by the
Corresponding Source fixed on a durable physical medium
customarily used for software interchange.
b) Convey the object code in, or embodied in, a physical product
(including a physical distribution medium), accompanied by a
written offer, valid for at least three years and valid for as
long as you offer spare parts or customer support for that product
model, to give anyone who possesses the object code either (1) a
copy of the Corresponding Source for all the software in the
product that is covered by this License, on a durable physical
medium customarily used for software interchange, for a price no
more than your reasonable cost of physically performing this
conveying of source, or (2) access to copy the
Corresponding Source from a network server at no charge.
c) Convey individual copies of the object code with a copy of the
written offer to provide the Corresponding Source. This
alternative is allowed only occasionally and noncommercially, and
only if you received the object code with such an offer, in accord
with subsection 6b.
d) Convey the object code by offering access from a designated
place (gratis or for a charge), and offer equivalent access to the
Corresponding Source in the same way through the same place at no
further charge. You need not require recipients to copy the
Corresponding Source along with the object code. If the place to
copy the object code is a network server, the Corresponding Source
may be on a different server (operated by you or a third party)
that supports equivalent copying facilities, provided you maintain
clear directions next to the object code saying where to find the
Corresponding Source. Regardless of what server hosts the
Corresponding Source, you remain obligated to ensure that it is
available for as long as needed to satisfy these requirements.
e) Convey the object code using peer-to-peer transmission, provided
you inform other peers where the object code and Corresponding
Source of the work are being offered to the general public at no
charge under subsection 6d.
A separable portion of the object code, whose source code is excluded
from the Corresponding Source as a System Library, need not be
included in conveying the object code work.
A "User Product" is either (1) a "consumer product", which means any
tangible personal property which is normally used for personal, family,
or household purposes, or (2) anything designed or sold for incorporation
into a dwelling. In determining whether a product is a consumer product,
doubtful cases shall be resolved in favor of coverage. For a particular
product received by a particular user, "normally used" refers to a
typical or common use of that class of product, regardless of the status
of the particular user or of the way in which the particular user
actually uses, or expects or is expected to use, the product. A product
is a consumer product regardless of whether the product has substantial
commercial, industrial or non-consumer uses, unless such uses represent
the only significant mode of use of the product.
"Installation Information" for a User Product means any methods,
procedures, authorization keys, or other information required to install
and execute modified versions of a covered work in that User Product from
a modified version of its Corresponding Source. The information must
suffice to ensure that the continued functioning of the modified object
code is in no case prevented or interfered with solely because
modification has been made.
If you convey an object code work under this section in, or with, or
specifically for use in, a User Product, and the conveying occurs as
part of a transaction in which the right of possession and use of the
User Product is transferred to the recipient in perpetuity or for a
fixed term (regardless of how the transaction is characterized), the
Corresponding Source conveyed under this section must be accompanied
by the Installation Information. But this requirement does not apply
if neither you nor any third party retains the ability to install
modified object code on the User Product (for example, the work has
been installed in ROM).
The requirement to provide Installation Information does not include a
requirement to continue to provide support service, warranty, or updates
for a work that has been modified or installed by the recipient, or for
the User Product in which it has been modified or installed. Access to a
network may be denied when the modification itself materially and
adversely affects the operation of the network or violates the rules and
protocols for communication across the network.
Corresponding Source conveyed, and Installation Information provided,
in accord with this section must be in a format that is publicly
documented (and with an implementation available to the public in
source code form), and must require no special password or key for
unpacking, reading or copying.
7. Additional Terms.
"Additional permissions" are terms that supplement the terms of this
License by making exceptions from one or more of its conditions.
Additional permissions that are applicable to the entire Program shall
be treated as though they were included in this License, to the extent
that they are valid under applicable law. If additional permissions
apply only to part of the Program, that part may be used separately
under those permissions, but the entire Program remains governed by
this License without regard to the additional permissions.
When you convey a copy of a covered work, you may at your option
remove any additional permissions from that copy, or from any part of
it. (Additional permissions may be written to require their own
removal in certain cases when you modify the work.) You may place
additional permissions on material, added by you to a covered work,
for which you have or can give appropriate copyright permission.
Notwithstanding any other provision of this License, for material you
add to a covered work, you may (if authorized by the copyright holders of
that material) supplement the terms of this License with terms:
a) Disclaiming warranty or limiting liability differently from the
terms of sections 15 and 16 of this License; or
b) Requiring preservation of specified reasonable legal notices or
author attributions in that material or in the Appropriate Legal
Notices displayed by works containing it; or
c) Prohibiting misrepresentation of the origin of that material, or
requiring that modified versions of such material be marked in
reasonable ways as different from the original version; or
d) Limiting the use for publicity purposes of names of licensors or
authors of the material; or
e) Declining to grant rights under trademark law for use of some
trade names, trademarks, or service marks; or
f) Requiring indemnification of licensors and authors of that
material by anyone who conveys the material (or modified versions of
it) with contractual assumptions of liability to the recipient, for
any liability that these contractual assumptions directly impose on
those licensors and authors.
All other non-permissive additional terms are considered "further
restrictions" within the meaning of section 10. If the Program as you
received it, or any part of it, contains a notice stating that it is
governed by this License along with a term that is a further
restriction, you may remove that term. If a license document contains
a further restriction but permits relicensing or conveying under this
License, you may add to a covered work material governed by the terms
of that license document, provided that the further restriction does
not survive such relicensing or conveying.
If you add terms to a covered work in accord with this section, you
must place, in the relevant source files, a statement of the
additional terms that apply to those files, or a notice indicating
where to find the applicable terms.
Additional terms, permissive or non-permissive, may be stated in the
form of a separately written license, or stated as exceptions;
the above requirements apply either way.
8. Termination.
You may not propagate or modify a covered work except as expressly
provided under this License. Any attempt otherwise to propagate or
modify it is void, and will automatically terminate your rights under
this License (including any patent licenses granted under the third
paragraph of section 11).
However, if you cease all violation of this License, then your
license from a particular copyright holder is reinstated (a)
provisionally, unless and until the copyright holder explicitly and
finally terminates your license, and (b) permanently, if the copyright
holder fails to notify you of the violation by some reasonable means
prior to 60 days after the cessation.
Moreover, your license from a particular copyright holder is
reinstated permanently if the copyright holder notifies you of the
violation by some reasonable means, this is the first time you have
received notice of violation of this License (for any work) from that
copyright holder, and you cure the violation prior to 30 days after
your receipt of the notice.
Termination of your rights under this section does not terminate the
licenses of parties who have received copies or rights from you under
this License. If your rights have been terminated and not permanently
reinstated, you do not qualify to receive new licenses for the same
material under section 10.
9. Acceptance Not Required for Having Copies.
You are not required to accept this License in order to receive or
run a copy of the Program. Ancillary propagation of a covered work
occurring solely as a consequence of using peer-to-peer transmission
to receive a copy likewise does not require acceptance. However,
nothing other than this License grants you permission to propagate or
modify any covered work. These actions infringe copyright if you do
not accept this License. Therefore, by modifying or propagating a
covered work, you indicate your acceptance of this License to do so.
10. Automatic Licensing of Downstream Recipients.
Each time you convey a covered work, the recipient automatically
receives a license from the original licensors, to run, modify and
propagate that work, subject to this License. You are not responsible
for enforcing compliance by third parties with this License.
An "entity transaction" is a transaction transferring control of an
organization, or substantially all assets of one, or subdividing an
organization, or merging organizations. If propagation of a covered
work results from an entity transaction, each party to that
transaction who receives a copy of the work also receives whatever
licenses to the work the party's predecessor in interest had or could
give under the previous paragraph, plus a right to possession of the
Corresponding Source of the work from the predecessor in interest, if
the predecessor has it or can get it with reasonable efforts.
You may not impose any further restrictions on the exercise of the
rights granted or affirmed under this License. For example, you may
not impose a license fee, royalty, or other charge for exercise of
rights granted under this License, and you may not initiate litigation
(including a cross-claim or counterclaim in a lawsuit) alleging that
any patent claim is infringed by making, using, selling, offering for
sale, or importing the Program or any portion of it.
11. Patents.
A "contributor" is a copyright holder who authorizes use under this
License of the Program or a work on which the Program is based. The
work thus licensed is called the contributor's "contributor version".
A contributor's "essential patent claims" are all patent claims
owned or controlled by the contributor, whether already acquired or
hereafter acquired, that would be infringed by some manner, permitted
by this License, of making, using, or selling its contributor version,
but do not include claims that would be infringed only as a
consequence of further modification of the contributor version. For
purposes of this definition, "control" includes the right to grant
patent sublicenses in a manner consistent with the requirements of
this License.
Each contributor grants you a non-exclusive, worldwide, royalty-free
patent license under the contributor's essential patent claims, to
make, use, sell, offer for sale, import and otherwise run, modify and
propagate the contents of its contributor version.
In the following three paragraphs, a "patent license" is any express
agreement or commitment, however denominated, not to enforce a patent
(such as an express permission to practice a patent or covenant not to
sue for patent infringement). To "grant" such a patent license to a
party means to make such an agreement or commitment not to enforce a
patent against the party.
If you convey a covered work, knowingly relying on a patent license,
and the Corresponding Source of the work is not available for anyone
to copy, free of charge and under the terms of this License, through a
publicly available network server or other readily accessible means,
then you must either (1) cause the Corresponding Source to be so
available, or (2) arrange to deprive yourself of the benefit of the
patent license for this particular work, or (3) arrange, in a manner
consistent with the requirements of this License, to extend the patent
license to downstream recipients. "Knowingly relying" means you have
actual knowledge that, but for the patent license, your conveying the
covered work in a country, or your recipient's use of the covered work
in a country, would infringe one or more identifiable patents in that
country that you have reason to believe are valid.
If, pursuant to or in connection with a single transaction or
arrangement, you convey, or propagate by procuring conveyance of, a
covered work, and grant a patent license to some of the parties
receiving the covered work authorizing them to use, propagate, modify
or convey a specific copy of the covered work, then the patent license
you grant is automatically extended to all recipients of the covered
work and works based on it.
A patent license is "discriminatory" if it does not include within
the scope of its coverage, prohibits the exercise of, or is
conditioned on the non-exercise of one or more of the rights that are
specifically granted under this License. You may not convey a covered
work if you are a party to an arrangement with a third party that is
in the business of distributing software, under which you make payment
to the third party based on the extent of your activity of conveying
the work, and under which the third party grants, to any of the
parties who would receive the covered work from you, a discriminatory
patent license (a) in connection with copies of the covered work
conveyed by you (or copies made from those copies), or (b) primarily
for and in connection with specific products or compilations that
contain the covered work, unless you entered into that arrangement,
or that patent license was granted, prior to 28 March 2007.
Nothing in this License shall be construed as excluding or limiting
any implied license or other defenses to infringement that may
otherwise be available to you under applicable patent law.
12. No Surrender of Others' Freedom.
If conditions are imposed on you (whether by court order, agreement or
otherwise) that contradict the conditions of this License, they do not
excuse you from the conditions of this License. If you cannot convey a
covered work so as to satisfy simultaneously your obligations under this
License and any other pertinent obligations, then as a consequence you may
not convey it at all. For example, if you agree to terms that obligate you
to collect a royalty for further conveying from those to whom you convey
the Program, the only way you could satisfy both those terms and this
License would be to refrain entirely from conveying the Program.
13. Use with the GNU Affero General Public License.
Notwithstanding any other provision of this License, you have
permission to link or combine any covered work with a work licensed
under version 3 of the GNU Affero General Public License into a single
combined work, and to convey the resulting work. The terms of this
License will continue to apply to the part which is the covered work,
but the special requirements of the GNU Affero General Public License,
section 13, concerning interaction through a network will apply to the
combination as such.
14. Revised Versions of this License.
The Free Software Foundation may publish revised and/or new versions of
the GNU General Public License from time to time. Such new versions will
be similar in spirit to the present version, but may differ in detail to
address new problems or concerns.
Each version is given a distinguishing version number. If the
Program specifies that a certain numbered version of the GNU General
Public License "or any later version" applies to it, you have the
option of following the terms and conditions either of that numbered
version or of any later version published by the Free Software
Foundation. If the Program does not specify a version number of the
GNU General Public License, you may choose any version ever published
by the Free Software Foundation.
If the Program specifies that a proxy can decide which future
versions of the GNU General Public License can be used, that proxy's
public statement of acceptance of a version permanently authorizes you
to choose that version for the Program.
Later license versions may give you additional or different
permissions. However, no additional obligations are imposed on any
author or copyright holder as a result of your choosing to follow a
later version.
15. Disclaimer of Warranty.
THERE IS NO WARRANTY FOR THE PROGRAM, TO THE EXTENT PERMITTED BY
APPLICABLE LAW. EXCEPT WHEN OTHERWISE STATED IN WRITING THE COPYRIGHT
HOLDERS AND/OR OTHER PARTIES PROVIDE THE PROGRAM "AS IS" WITHOUT WARRANTY
OF ANY KIND, EITHER EXPRESSED OR IMPLIED, INCLUDING, BUT NOT LIMITED TO,
THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
PURPOSE. THE ENTIRE RISK AS TO THE QUALITY AND PERFORMANCE OF THE PROGRAM
IS WITH YOU. SHOULD THE PROGRAM PROVE DEFECTIVE, YOU ASSUME THE COST OF
ALL NECESSARY SERVICING, REPAIR OR CORRECTION.
16. Limitation of Liability.
IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING
WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MODIFIES AND/OR CONVEYS
THE PROGRAM AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES, INCLUDING ANY
GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING OUT OF THE
USE OR INABILITY TO USE THE PROGRAM (INCLUDING BUT NOT LIMITED TO LOSS OF
DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY YOU OR THIRD
PARTIES OR A FAILURE OF THE PROGRAM TO OPERATE WITH ANY OTHER PROGRAMS),
EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE POSSIBILITY OF
SUCH DAMAGES.
17. Interpretation of Sections 15 and 16.
If the disclaimer of warranty and limitation of liability provided
above cannot be given local legal effect according to their terms,
reviewing courts shall apply local law that most closely approximates
an absolute waiver of all civil liability in connection with the
Program, unless a warranty or assumption of liability accompanies a
copy of the Program in return for a fee.
END OF TERMS AND CONDITIONS
How to Apply These Terms to Your New Programs
If you develop a new program, and you want it to be of the greatest
possible use to the public, the best way to achieve this is to make it
free software which everyone can redistribute and change under these terms.
To do so, attach the following notices to the program. It is safest
to attach them to the start of each source file to most effectively
state the exclusion of warranty; and each file should have at least
the "copyright" line and a pointer to where the full notice is found.
Copyright (C)
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program. If not, see .
Also add information on how to contact you by electronic and paper mail.
If the program does terminal interaction, make it output a short
notice like this when it starts in an interactive mode:
Copyright (C)
This program comes with ABSOLUTELY NO WARRANTY; for details type `show w'.
This is free software, and you are welcome to redistribute it
under certain conditions; type `show c' for details.
The hypothetical commands `show w' and `show c' should show the appropriate
parts of the General Public License. Of course, your program's commands
might be different; for a GUI interface, you would use an "about box".
You should also get your employer (if you work as a programmer) or school,
if any, to sign a "copyright disclaimer" for the program, if necessary.
For more information on this, and how to apply and follow the GNU GPL, see
.
The GNU General Public License does not permit incorporating your program
into proprietary programs. If your program is a subroutine library, you
may consider it more useful to permit linking proprietary applications with
the library. If this is what you want to do, use the GNU Lesser General
Public License instead of this License. But first, please read
.
================================================
FILE: czsl/README.md
================================================
# BatchFormer for Compositional Zero-Shot Learning
## Setup
1. Clone the repo and
```
cd czsl
```
2. We recommend using Anaconda for environment setup. To create the environment and activate it, please run:
```
conda env create --file environment.yml
conda activate czsl
```
4. Go to the cloned repo and open a terminal. Download the datasets and embeddings, specifying the desired path (e.g. `DATA_ROOT` in the example):
```
bash ./utils/download_data.sh DATA_ROOT
mkdir logs
```
## Training
**Closed World.** To train a model, the command is simply:
```
python train.py --config CONFIG_FILE
```
where `CONFIG_FILE` is the path to the configuration file of the model.
The folder `configs` contains configuration files for all methods, i.e. CGE in `configs/cge`, CompCos in `configs/compcos`, and the other methods in `configs/baselines`.
To run CGE on MitStates, the command is just:
```
python train.py --config configs/cge/mit.yml --add_bt 1
```
On UT-Zappos, the command is:
```
python train.py --config configs/cge/utzappos.yml --add_bt 1
```
## Test
**Closed World.** To test a model, the code is simple:
```
python test.py --logpath LOG_DIR
```
where `LOG_DIR` is the directory containing the logs of a model.
## References
If you use this code, please cite
```
@inproceedings{hou2022batch,
title={BatchFormer: Learning to Explore Sample Relationships for Robust Representation Learning},
author={Hou, Zhi and Yu, Baosheng and Tao, Dacheng},
booktitle={CVPR},
year={2022}
}
```
**Note**: Code are based on czsl:
```
@inproceedings{naeem2021learning,
title={Learning Graph Embeddings for Compositional Zero-shot Learning},
author={Naeem, MF and Xian, Y and Tombari, F and Akata, Zeynep},
booktitle={34th IEEE Conference on Computer Vision and Pattern Recognition},
year={2021},
organization={IEEE}
}
```
================================================
FILE: czsl/configs/baselines/cgqa/aopp.yml
================================================
---
experiment:
name: aopp/cgqa/
dataset:
data_dir: cgqa
dataset: cgqa
splitname: compositional-split-natural
model_params:
model: attributeop
emb_dim: 300
emb_init: glove
eval_type: dist_fast
image_extractor: resnet18
train_only: true
static_inp: false
composition: add
loss:
lambda_aux: 1.0
lambda_comm: 1.0
training:
batch_size: 128
eval_val_every: 2
load:
lr: 5.0e-05
lrg: 0.001
margin: 0.5
max_epochs: 1000
norm_family: imagenet
save_every: 10000
test_batch_size: 32
test_set: val
topk: 1
wd: 5.0e-05
workers: 8
update_features: false
freeze_features: false
extra:
lambda_attr: 0
lambda_obj: 0
lambda_sub: 0
graph: false
hardk: null
================================================
FILE: czsl/configs/baselines/cgqa/le+.yml
================================================
---
experiment:
name: labelembed/cgqa/
dataset:
data_dir: cgqa
dataset: cgqa
splitname: compositional-split-natural
model_params:
model: labelembed+
dropout: false
norm: false
nlayers: 2
fc_emb: 512,512
emb_dim: 300
emb_init: glove
eval_type: dist_fast
image_extractor: resnet18
train_only: true
static_inp: false
composition: add
loss:
lambda_ce: 1.0
lambda_trip_smart: 0
training:
batch_size: 512
eval_val_every: 2
load:
lr: 5.0e-05
margin: 0.5
max_epochs: 2000
norm_family: imagenet
save_every: 10000
test_batch_size: 32
test_set: val
topk: 1
wd: 5.0e-05
workers: 8
update_features: false
freeze_features: false
================================================
FILE: czsl/configs/baselines/cgqa/symnet.yml
================================================
---
experiment:
name: symnet/cgqa/
dataset:
data_dir: cgqa
dataset: cgqa
splitname: compositional-split-natural
model_params:
model: symnet
dropout: true
norm: true
nlayers: 3
fc_emb: 768,1024,1200
emb_dim: 300
emb_init: glove
image_extractor: resnet18
loss:
lambda_cls_attr: 1.0
lambda_cls_obj: 1.0
lambda_trip: 0.5
lambda_sym: 0.01
lambda_axiom: 0.03
training:
batch_size: 512
eval_val_every: 2
load:
lr: 5.0e-05
lrg: 0.001
margin: 0.5
max_epochs: 2000
norm_family: imagenet
save_every: 10000
test_batch_size: 32
test_set: val
topk: 1
wd: 5.0e-05
workers: 8
update_features: false
freeze_features: false
================================================
FILE: czsl/configs/baselines/cgqa/tmn.yml
================================================
---
experiment:
name: tmn/cgqa/
dataset:
data_dir: cgqa
dataset: cgqa
splitname: compositional-split-natural
model_params:
model: tmn
emb_init: glove
image_extractor: resnet18
num_negs: 600
embed_rank: 64
emb_dim: 16
nmods: 24
training:
batch_size: 256
eval_val_every: 2
load:
lr: 0.0001
lrg: 0.01
margin: 0.5
max_epochs: 100
norm_family: imagenet
save_every: 10000
test_batch_size: 32
test_set: val
topk: 1
wd: 5.0e-05
workers: 8
update_features: false
freeze_features: false
================================================
FILE: czsl/configs/baselines/mit/aopp.yml
================================================
---
experiment:
name: aopp/mitstates/
dataset:
data_dir: mit-states
dataset: mitstates
splitname: compositional-split-natural
model_params:
model: attributeop
emb_dim: 300
emb_init: glove
eval_type: dist_fast
image_extractor: resnet18
train_only: true
static_inp: false
composition: add
loss:
lambda_aux: 1000.0
lambda_comm: 1.0
training:
batch_size: 512
eval_val_every: 2
load:
lr: 5.0e-05
lrg: 0.001
margin: 0.5
max_epochs: 1000
norm_family: imagenet
save_every: 10000
test_batch_size: 32
test_set: val
topk: 1
wd: 5.0e-05
workers: 8
update_features: false
freeze_features: false
extra:
lambda_attr: 0
lambda_obj: 0
lambda_sub: 0
graph: false
hardk: null
================================================
FILE: czsl/configs/baselines/mit/le+.yml
================================================
---
experiment:
name: labelembed/mitstates/
dataset:
data_dir: mit-states
dataset: mitstates
splitname: compositional-split-natural
model_params:
model: labelembed+
dropout: false
norm: false
nlayers: 2
fc_emb: 512,512
emb_dim: 300
emb_init: glove
eval_type: dist_fast
image_extractor: resnet18
train_only: true
static_inp: false
composition: add
training:
batch_size: 512
eval_val_every: 2
load:
lr: 5.0e-05
margin: 0.5
max_epochs: 2000
norm_family: imagenet
save_every: 10000
test_batch_size: 32
test_set: val
topk: 1
wd: 5.0e-05
workers: 8
update_features: false
freeze_features: false
================================================
FILE: czsl/configs/baselines/mit/symnet.yml
================================================
---
experiment:
name: symnet/mit-states/
dataset:
data_dir: mit-states
dataset: mitstates
splitname: compositional-split-natural
model_params:
model: symnet
dropout: true
norm: true
nlayers: 3
fc_emb: 768,1024,1200
emb_dim: 300
emb_init: glove
image_extractor: resnet18
obj_fixed: True
loss:
lambda_cls_attr: 1.0
lambda_cls_obj: 0.1
lambda_trip: 0.5
lambda_sym: 0.01
lambda_axiom: 0.03
training:
batch_size: 450
eval_val_every: 2
load:
lr: 5.0e-05
lrg: 5.0e-06
margin: 0.5
max_epochs: 2000
norm_family: imagenet
save_every: 10000
test_batch_size: 32
test_set: val
topk: 1
wd: 5.0e-05
workers: 8
update_features: false
freeze_features: false
================================================
FILE: czsl/configs/baselines/mit/tmn.yml
================================================
---
experiment:
name: tmn/mitstates/
dataset:
data_dir: mit-states
dataset: mitstates
splitname: compositional-split-natural
model_params:
model: tmn
emb_init: glove
image_extractor: resnet18
num_negs: 600
embed_rank: 64
emb_dim: 16
nmods: 24
training:
batch_size: 256
eval_val_every: 2
load:
lr: 0.0001
lrg: 1.0e-06
margin: 0.5
max_epochs: 100
norm_family: imagenet
save_every: 10000
test_batch_size: 32
test_set: val
topk: 1
wd: 5.0e-05
workers: 8
update_features: true
freeze_features: false
================================================
FILE: czsl/configs/baselines/utzppos/aopp.yml
================================================
---
experiment:
name: aopp/utzappos/
dataset:
data_dir: ut-zap50k
dataset: utzappos
splitname: compositional-split-natural
model_params:
model: attributeop
emb_dim: 300
emb_init: glove
eval_type: dist_fast
image_extractor: resnet18
train_only: true
static_inp: false
composition: add
loss:
lambda_aux: 1.0
lambda_comm: 1.0
training:
batch_size: 512
eval_val_every: 2
load:
lr: 5.0e-05
lrg: 0.001
margin: 0.5
max_epochs: 1000
norm_family: imagenet
save_every: 10000
test_batch_size: 32
test_set: val
topk: 1
wd: 5.0e-05
workers: 8
update_features: false
freeze_features: false
extra:
lambda_attr: 0
lambda_obj: 0
lambda_sub: 0
graph: false
hardk: null
================================================
FILE: czsl/configs/baselines/utzppos/le+.yml
================================================
---
experiment:
name: labelembed/utzappos/
dataset:
data_dir: ut-zap50k
dataset: utzappos
splitname: compositional-split-natural
model_params:
model: labelembed+
dropout: false
norm: false
nlayers: 2
fc_emb: 512,512
emb_dim: 300
emb_init: glove
eval_type: dist_fast
image_extractor: resnet18
train_only: true
static_inp: false
composition: add
training:
batch_size: 512
eval_val_every: 2
load:
lr: 5.0e-05
margin: 0.5
max_epochs: 2000
norm_family: imagenet
save_every: 10000
test_batch_size: 32
test_set: val
topk: 1
wd: 5.0e-05
workers: 8
update_features: false
freeze_features: false
================================================
FILE: czsl/configs/baselines/utzppos/symnet.yml
================================================
---
experiment:
name: symnet/utzappos/
dataset:
data_dir: ut-zap50k
dataset: utzappos
splitname: compositional-split-natural
model_params:
model: symnet
dropout: true
norm: true
nlayers: 3
fc_emb: 768,1024,1200
emb_dim: 300
emb_init: fasttext
image_extractor: resnet18
loss:
lambda_cls_attr: 1.0
lambda_cls_obj: 1.0
lambda_trip: 0.5
lambda_sym: 0.01
lambda_axiom: 0.03
training:
batch_size: 512
eval_val_every: 2
load:
lr: 5.0e-05
lrg: 0.001
margin: 0.5
max_epochs: 2000
norm_family: imagenet
save_every: 10000
test_batch_size: 32
test_set: val
topk: 1
wd: 5.0e-05
workers: 8
update_features: false
freeze_features: false
================================================
FILE: czsl/configs/baselines/utzppos/tmn.yml
================================================
---
experiment:
name: tmn/utzappos/
dataset:
data_dir: ut-zap50k
dataset: utzappos
splitname: compositional-split-natural
model_params:
model: tmn
emb_init: word2vec
emb_dim: 300
image_extractor: resnet18
train_only: true
static_inp: true
num_negs: 600
embed_rank: 64
nmods: 24
training:
batch_size: 256
eval_val_every: 10
load:
lr: 0.0001
lrg: 1.0e-06
margin: 0.5
max_epochs: 100
norm_family: imagenet
save_every: 10000
test_batch_size: 32
test_set: val
topk: 1
wd: 5.0e-05
workers: 8
update_features: false
freeze_features: false
(czsl) mmancini68@EML-Jupi
================================================
FILE: czsl/configs/cge/cgqa.yml
================================================
experiment:
name: graphembed/cgqa
dataset:
data_dir: cgqa
dataset: cgqa
splitname: compositional-split-natural
model_params:
model: graphfull
dropout: true
norm: true
nlayers:
fc_emb: 768,1024,1200
gr_emb: d4096,d
emb_dim: 512
emb_init: null
graph_init: utils/cgqa-graph.t7
eval_type: otherdist
image_extractor: resnet18
train_only: true
static_inp: false
gcnii:
hidden_dim: 1024
gcn_type: gcn
gcn_nlayers: 10
training:
batch_size: 256
eval_val_every: 2
load:
lr: 5.0e-05
wd: 5.0e-05
lrg: 5.0e-06
margin: 2
max_epochs: 200
norm_family: imagenet
save_every: 10000
test_batch_size: 32
test_set: val
topk: 1
workers: 8
update_features: true
freeze_featues: false
================================================
FILE: czsl/configs/cge/mit.yml
================================================
---
experiment:
name: graphembed/mitstates/base
dataset:
data_dir: mit-states
dataset: mitstates
splitname: compositional-split-natural
model_params:
model: graphfull
dropout: true
norm: true
nlayers:
gr_emb: d4096,d
emb_dim: 512
graph_init: utils/mitstates-graph.t7
image_extractor: resnet18
train_only: true
static_inp: false
training:
batch_size: 128
eval_val_every: 2
load:
lr: 5.0e-05
wd: 5.0e-05
lrg: 5.0e-6
margin: 0.5
max_epochs: 100
norm_family: imagenet
save_every: 10000
test_batch_size: 32
test_set: val
topk: 1
workers: 8
update_features: true
freeze_features: false
================================================
FILE: czsl/configs/cge/utzappos.yml
================================================
---
experiment:
name: graphembed/utzappos/base
dataset:
data_dir: ut-zap50k
dataset: utzappos
splitname: compositional-split-natural
model_params:
model: graphfull
dropout: true
norm: true
nlayers: 1
fc_emb: 768,1024,1200
gr_emb: d4096,d
emb_dim: 300
emb_init: null
graph_init: utils/utzappos-graph.t7
eval_type: otherdist
image_extractor: resnet18
train_only: true
static_inp: false
composition: mlp_add
gcnii:
hidden_dim: 1024
gcn_type: gcn
gcn_nlayers: 10
training:
batch_size: 128
eval_val_every: 2
load:
lr: 5.0e-05
wd: 5.0e-05
lrg: 5.0e-6
margin: 0.5
max_epochs: 100
norm_family: imagenet
save_every: 10000
test_batch_size: 32
test_set: val
topk: 1
workers: 8
update_features: true
freeze_features: false
extra:
lambda_attr: 0
lambda_obj: 0
lambda_sub: 0
graph: false
hardk: null
cutmix: false
cutmix_prob: 1.0
beta: 1.0
================================================
FILE: czsl/configs/compcos/cgqa/compcos.yml
================================================
---
experiment:
name: compcos/cgqa
dataset:
data_dir: cgqa
dataset: cgqa
splitname: compositional-split-natural
open_world: true
model_params:
model: compcos
dropout: true
norm: true
nlayers: 2
relu: false
fc_emb: 768,1024,1200
emb_dim: 300
emb_init: word2vec
image_extractor: resnet18
train_only: false
static_inp: false
training:
batch_size: 128
load:
lr: 5.0e-05
lrg: 0.001
margin: 0.4
cosine_scale: 100
max_epochs: 300
norm_family: imagenet
save_every: 10000
test_batch_size: 64
test_set: val
topk: 1
wd: 5.0e-05
workers: 8
update_features: false
freeze_features: false
epoch_max_margin: 15
================================================
FILE: czsl/configs/compcos/cgqa/compcos_cw.yml
================================================
---
experiment:
name: compcos_cw/cgqa/
dataset:
data_dir: cgqa
dataset: cgqa
splitname: compositional-split-natural
model_params:
model: compcos
dropout: true
norm: true
nlayers: 2
relu: false
fc_emb: 768,1024,1200
emb_dim: 300
emb_init: word2vec
image_extractor: resnet18
train_only: true
static_inp: false
training:
batch_size: 128
load:
lr: 5.0e-05
lrg: 0.001
margin: 0.4
cosine_scale: 100
max_epochs: 300
norm_family: imagenet
save_every: 10000
test_batch_size: 64
test_set: val
topk: 1
wd: 5.0e-05
workers: 8
update_features: false
freeze_features: false
epoch_max_margin: 15
================================================
FILE: czsl/configs/compcos/mit/compcos.yml
================================================
---
experiment:
name: compcos/mitstates
dataset:
data_dir: mit-states
dataset: mitstates
splitname: compositional-split-natural
open_world: true
model_params:
model: compcos
dropout: true
norm: true
nlayers: 2
relu: false
fc_emb: 768,1024,1200
emb_dim: 600
emb_init: ft+w2v
image_extractor: resnet18
train_only: false
static_inp: false
training:
batch_size: 128
load:
lr: 5.0e-05
lrg: 0.001
margin: 0.4
cosine_scale: 20
max_epochs: 300
norm_family: imagenet
save_every: 10000
test_batch_size: 64
test_set: val
topk: 1
wd: 5.0e-05
workers: 8
update_features: false
freeze_features: false
epoch_max_margin: 15
================================================
FILE: czsl/configs/compcos/mit/compcos_cw.yml
================================================
---
experiment:
name: compcos_cw/mitstates
dataset:
data_dir: mit-states
dataset: mitstates
splitname: compositional-split-natural
model_params:
model: compcos
dropout: true
norm: true
nlayers: 2
relu: false
fc_emb: 768,1024,1200
emb_dim: 600
emb_init: ft+w2v
image_extractor: resnet18
train_only: true
static_inp: false
training:
batch_size: 128
load:
lr: 5.0e-05
lrg: 0.001
margin: 0.4
cosine_scale: 20
max_epochs: 300
norm_family: imagenet
save_every: 10000
test_batch_size: 64
test_set: val
topk: 1
wd: 5.0e-05
workers: 8
update_features: false
freeze_features: false
epoch_max_margin: 15
================================================
FILE: czsl/configs/compcos/utzppos/compcos.yml
================================================
---
experiment:
name: compcos/utzappos
dataset:
data_dir: ut-zap50k
dataset: utzappos
splitname: compositional-split-natural
open_world: true
model_params:
model: compcos
dropout: true
norm: true
nlayers: 2
relu: false
fc_emb: 768,1024
emb_dim: 300
emb_init: word2vec
image_extractor: resnet18
train_only: false
static_inp: false
training:
batch_size: 128
load:
lr: 5.0e-05
lrg: 0.001
margin: 1.0
cosine_scale: 50
max_epochs: 300
norm_family: imagenet
save_every: 10000
test_batch_size: 64
test_set: val
topk: 1
wd: 5.0e-05
workers: 8
update_features: false
freeze_features: false
epoch_max_margin: 100
================================================
FILE: czsl/configs/compcos/utzppos/compcos_cw.yml
================================================
---
experiment:
name: compcos/utzappos
dataset:
data_dir: ut-zap50k
dataset: utzappos
splitname: compositional-split-natural
model_params:
model: compcos
dropout: true
norm: true
nlayers: 2
relu: false
fc_emb: 768,1024
emb_dim: 300
emb_init: word2vec
image_extractor: resnet18
train_only: true
static_inp: false
training:
batch_size: 128
load:
lr: 5.0e-05
lrg: 0.001
margin: 1.0
cosine_scale: 50
max_epochs: 300
norm_family: imagenet
save_every: 10000
test_batch_size: 64
test_set: val
topk: 1
wd: 5.0e-05
workers: 8
update_features: false
freeze_features: false
epoch_max_margin: 100
================================================
FILE: czsl/data/__init__.py
================================================
================================================
FILE: czsl/data/dataset.py
================================================
#external libs
import numpy as np
from tqdm import tqdm
from PIL import Image
import os
import random
from os.path import join as ospj
from glob import glob
#torch libs
from torch.utils.data import Dataset
import torch
import torchvision.transforms as transforms
#local libs
from utils.utils import get_norm_values, chunks
from models.image_extractor import get_image_extractor
from itertools import product
device = 'cuda' if torch.cuda.is_available() else 'cpu'
class ImageLoader:
def __init__(self, root):
self.root_dir = root
def __call__(self, img):
img = Image.open(ospj(self.root_dir,img)).convert('RGB') #We don't want alpha
return img
def dataset_transform(phase, norm_family = 'imagenet'):
'''
Inputs
phase: String controlling which set of transforms to use
norm_family: String controlling which normaliztion values to use
Returns
transform: A list of pytorch transforms
'''
mean, std = get_norm_values(norm_family=norm_family)
if phase == 'train':
transform = transforms.Compose([
transforms.RandomResizedCrop(224),
transforms.RandomHorizontalFlip(),
transforms.ToTensor(),
transforms.Normalize(mean, std)
])
elif phase == 'val' or phase == 'test':
transform = transforms.Compose([
transforms.Resize(256),
transforms.CenterCrop(224),
transforms.ToTensor(),
transforms.Normalize(mean, std)
])
elif phase == 'all':
transform = transforms.Compose([
transforms.Resize(256),
transforms.CenterCrop(224),
transforms.ToTensor(),
transforms.Normalize(mean, std)
])
else:
raise ValueError('Invalid transform')
return transform
def filter_data(all_data, pairs_gt, topk = 5):
'''
Helper function to clean data
'''
valid_files = []
with open('/home/ubuntu/workspace/top'+str(topk)+'.txt') as f:
for line in f:
valid_files.append(line.strip())
data, pairs, attr, obj = [], [], [], []
for current in all_data:
if current[0] in valid_files:
data.append(current)
pairs.append((current[1],current[2]))
attr.append(current[1])
obj.append(current[2])
counter = 0
for current in pairs_gt:
if current in pairs:
counter+=1
print('Matches ', counter, ' out of ', len(pairs_gt))
print('Samples ', len(data), ' out of ', len(all_data))
return data, sorted(list(set(pairs))), sorted(list(set(attr))), sorted(list(set(obj)))
# Dataset class now
class CompositionDataset(Dataset):
'''
Inputs
root: String of base dir of dataset
phase: String train, val, test
split: String dataset split
subset: Boolean if true uses a subset of train at each epoch
num_negs: Int, numbers of negative pairs per batch
pair_dropout: Percentage of pairs to leave in current epoch
'''
def __init__(
self,
root,
phase,
split = 'compositional-split',
model = 'resnet18',
norm_family = 'imagenet',
subset = False,
num_negs = 1,
pair_dropout = 0.0,
update_features = False,
return_images = False,
train_only = False,
open_world=False
):
self.root = root
self.phase = phase
self.split = split
self.num_negs = num_negs
self.pair_dropout = pair_dropout
self.norm_family = norm_family
self.return_images = return_images
self.update_features = update_features
self.feat_dim = 512 if 'resnet18' in model else 2048 # todo, unify this with models
self.open_world = open_world
self.attrs, self.objs, self.pairs, self.train_pairs, \
self.val_pairs, self.test_pairs = self.parse_split()
self.train_data, self.val_data, self.test_data = self.get_split_info()
self.full_pairs = list(product(self.attrs,self.objs))
# Clean only was here
self.obj2idx = {obj: idx for idx, obj in enumerate(self.objs)}
self.attr2idx = {attr : idx for idx, attr in enumerate(self.attrs)}
if self.open_world:
self.pairs = self.full_pairs
self.all_pair2idx = {pair: idx for idx, pair in enumerate(self.pairs)}
if train_only and self.phase == 'train':
print('Using only train pairs')
self.pair2idx = {pair : idx for idx, pair in enumerate(self.train_pairs)}
else:
print('Using all pairs')
self.pair2idx = {pair : idx for idx, pair in enumerate(self.pairs)}
if self.phase == 'train':
self.data = self.train_data
elif self.phase == 'val':
self.data = self.val_data
elif self.phase == 'test':
self.data = self.test_data
elif self.phase == 'all':
print('Using all data')
self.data = self.train_data + self.val_data + self.test_data
else:
raise ValueError('Invalid training phase')
self.all_data = self.train_data + self.val_data + self.test_data
print('Dataset loaded')
print('Train pairs: {}, Validation pairs: {}, Test Pairs: {}'.format(
len(self.train_pairs), len(self.val_pairs), len(self.test_pairs)))
print('Train images: {}, Validation images: {}, Test images: {}'.format(
len(self.train_data), len(self.val_data), len(self.test_data)))
if subset:
ind = np.arange(len(self.data))
ind = ind[::len(ind) // 1000]
self.data = [self.data[i] for i in ind]
# Keeping a list of all pairs that occur with each object
self.obj_affordance = {}
self.train_obj_affordance = {}
for _obj in self.objs:
candidates = [attr for (_, attr, obj) in self.train_data+self.test_data if obj==_obj]
self.obj_affordance[_obj] = list(set(candidates))
candidates = [attr for (_, attr, obj) in self.train_data if obj==_obj]
self.train_obj_affordance[_obj] = list(set(candidates))
self.sample_indices = list(range(len(self.data)))
self.sample_pairs = self.train_pairs
# Load based on what to output
self.transform = dataset_transform(self.phase, self.norm_family)
self.loader = ImageLoader(ospj(self.root, 'images'))
if not self.update_features:
feat_file = ospj(root, model+'_featurers.t7')
print(f'Using {model} and feature file {feat_file}')
if not os.path.exists(feat_file):
with torch.no_grad():
self.generate_features(feat_file, model)
self.phase = phase
activation_data = torch.load(feat_file)
self.activations = dict(
zip(activation_data['files'], activation_data['features']))
self.feat_dim = activation_data['features'].size(1)
print('{} activations loaded'.format(len(self.activations)))
def parse_split(self):
'''
Helper function to read splits of object atrribute pair
Returns
all_attrs: List of all attributes
all_objs: List of all objects
all_pairs: List of all combination of attrs and objs
tr_pairs: List of train pairs of attrs and objs
vl_pairs: List of validation pairs of attrs and objs
ts_pairs: List of test pairs of attrs and objs
'''
def parse_pairs(pair_list):
'''
Helper function to parse each phase to object attrribute vectors
Inputs
pair_list: path to textfile
'''
with open(pair_list, 'r') as f:
pairs = f.read().strip().split('\n')
pairs = [line.split() for line in pairs]
pairs = list(map(tuple, pairs))
attrs, objs = zip(*pairs)
return attrs, objs, pairs
tr_attrs, tr_objs, tr_pairs = parse_pairs(
ospj(self.root, self.split, 'train_pairs.txt')
)
vl_attrs, vl_objs, vl_pairs = parse_pairs(
ospj(self.root, self.split, 'val_pairs.txt')
)
ts_attrs, ts_objs, ts_pairs = parse_pairs(
ospj(self.root, self.split, 'test_pairs.txt')
)
#now we compose all objs, attrs and pairs
all_attrs, all_objs = sorted(
list(set(tr_attrs + vl_attrs + ts_attrs))), sorted(
list(set(tr_objs + vl_objs + ts_objs)))
all_pairs = sorted(list(set(tr_pairs + vl_pairs + ts_pairs)))
return all_attrs, all_objs, all_pairs, tr_pairs, vl_pairs, ts_pairs
def get_split_info(self):
'''
Helper method to read image, attrs, objs samples
Returns
train_data, val_data, test_data: List of tuple of image, attrs, obj
'''
data = torch.load(ospj(self.root, 'metadata_{}.t7'.format(self.split)))
train_data, val_data, test_data = [], [], []
for instance in data:
image, attr, obj, settype = instance['image'], instance['attr'], \
instance['obj'], instance['set']
curr_data = [image, attr, obj]
if attr == 'NA' or (attr, obj) not in self.pairs or settype == 'NA':
# Skip incomplete pairs, unknown pairs and unknown set
continue
if settype == 'train':
train_data.append(curr_data)
elif settype == 'val':
val_data.append(curr_data)
else:
test_data.append(curr_data)
return train_data, val_data, test_data
def get_dict_data(self, data, pairs):
data_dict = {}
for current in pairs:
data_dict[current] = []
for current in data:
image, attr, obj = current
data_dict[(attr, obj)].append(image)
return data_dict
def reset_dropout(self):
'''
Helper function to sample new subset of data containing a subset of pairs of objs and attrs
'''
self.sample_indices = list(range(len(self.data)))
self.sample_pairs = self.train_pairs
# Using sampling from random instead of 2 step numpy
n_pairs = int((1 - self.pair_dropout) * len(self.train_pairs))
self.sample_pairs = random.sample(self.train_pairs, n_pairs)
print('Sampled new subset')
print('Using {} pairs out of {} pairs right now'.format(
n_pairs, len(self.train_pairs)))
self.sample_indices = [ i for i in range(len(self.data))
if (self.data[i][1], self.data[i][2]) in self.sample_pairs
]
print('Using {} images out of {} images right now'.format(
len(self.sample_indices), len(self.data)))
def sample_negative(self, attr, obj):
'''
Inputs
attr: String of valid attribute
obj: String of valid object
Returns
Tuple of a different attribute, object indexes
'''
new_attr, new_obj = self.sample_pairs[np.random.choice(
len(self.sample_pairs))]
while new_attr == attr and new_obj == obj:
new_attr, new_obj = self.sample_pairs[np.random.choice(
len(self.sample_pairs))]
return (self.attr2idx[new_attr], self.obj2idx[new_obj])
def sample_affordance(self, attr, obj):
'''
Inputs
attr: String of valid attribute
obj: String of valid object
Return
Idx of a different attribute for the same object
'''
new_attr = np.random.choice(self.obj_affordance[obj])
while new_attr == attr:
new_attr = np.random.choice(self.obj_affordance[obj])
return self.attr2idx[new_attr]
def sample_train_affordance(self, attr, obj):
'''
Inputs
attr: String of valid attribute
obj: String of valid object
Return
Idx of a different attribute for the same object from the training pairs
'''
new_attr = np.random.choice(self.train_obj_affordance[obj])
while new_attr == attr:
new_attr = np.random.choice(self.train_obj_affordance[obj])
return self.attr2idx[new_attr]
def generate_features(self, out_file, model):
'''
Inputs
out_file: Path to save features
model: String of extraction model
'''
# data = self.all_data
data = ospj(self.root,'images')
files_before = glob(ospj(data, '**', '*.jpg'), recursive=True)
files_all = []
for current in files_before:
parts = current.split('/')
if "cgqa" in self.root:
files_all.append(parts[-1])
else:
files_all.append(os.path.join(parts[-2],parts[-1]))
transform = dataset_transform('test', self.norm_family)
feat_extractor = get_image_extractor(arch = model).eval()
feat_extractor = feat_extractor.to(device)
image_feats = []
image_files = []
for chunk in tqdm(
chunks(files_all, 512), total=len(files_all) // 512, desc=f'Extracting features {model}'):
files = chunk
imgs = list(map(self.loader, files))
imgs = list(map(transform, imgs))
feats = feat_extractor(torch.stack(imgs, 0).to(device))
image_feats.append(feats.data.cpu())
image_files += files
image_feats = torch.cat(image_feats, 0)
print('features for %d images generated' % (len(image_files)))
torch.save({'features': image_feats, 'files': image_files}, out_file)
def __getitem__(self, index):
'''
Call for getting samples
'''
index = self.sample_indices[index]
image, attr, obj = self.data[index]
# Decide what to output
if not self.update_features:
img = self.activations[image]
else:
img = self.loader(image)
img = self.transform(img)
data = [img, self.attr2idx[attr], self.obj2idx[obj], self.pair2idx[(attr, obj)]]
if self.phase == 'train':
all_neg_attrs = []
all_neg_objs = []
for curr in range(self.num_negs):
neg_attr, neg_obj = self.sample_negative(attr, obj) # negative for triplet lose
all_neg_attrs.append(neg_attr)
all_neg_objs.append(neg_obj)
neg_attr, neg_obj = torch.LongTensor(all_neg_attrs), torch.LongTensor(all_neg_objs)
#note here
if len(self.train_obj_affordance[obj])>1:
inv_attr = self.sample_train_affordance(attr, obj) # attribute for inverse regularizer
else:
inv_attr = (all_neg_attrs[0])
comm_attr = self.sample_affordance(inv_attr, obj) # attribute for commutative regularizer
data += [neg_attr, neg_obj, inv_attr, comm_attr]
# Return image paths if requested as the last element of the list
if self.return_images and self.phase != 'train':
data.append(image)
return data
def __len__(self):
'''
Call for length
'''
return len(self.sample_indices)
================================================
FILE: czsl/environment.yml
================================================
name: czsl
channels:
- pytorch
- defaults
dependencies:
- _libgcc_mutex=0.1=main
- blas=1.0=mkl
- ca-certificates=2021.1.19=h06a4308_0
- certifi=2020.12.5=py38h06a4308_0
- cudatoolkit=11.0.221=h6bb024c_0
- freetype=2.10.4=h5ab3b9f_0
- intel-openmp=2020.2=254
- jpeg=9b=h024ee3a_2
- lcms2=2.11=h396b838_0
- ld_impl_linux-64=2.33.1=h53a641e_7
- libedit=3.1.20191231=h14c3975_1
- libffi=3.3=he6710b0_2
- libgcc-ng=9.1.0=hdf63c60_0
- libgfortran-ng=7.3.0=hdf63c60_0
- libpng=1.6.37=hbc83047_0
- libstdcxx-ng=9.1.0=hdf63c60_0
- libtiff=4.1.0=h2733197_1
- libuv=1.40.0=h7b6447c_0
- lz4-c=1.9.3=h2531618_0
- mkl=2020.2=256
- mkl-service=2.3.0=py38he904b0f_0
- mkl_fft=1.2.0=py38h23d657b_0
- mkl_random=1.1.1=py38h0573a6f_0
- ncurses=6.2=he6710b0_1
- ninja=1.10.2=py38hff7bd54_0
- numpy=1.19.2=py38h54aff64_0
- numpy-base=1.19.2=py38hfa32c7d_0
- olefile=0.46=py_0
- openssl=1.1.1i=h27cfd23_0
- pillow=8.1.0=py38he98fc37_0
- pip=20.3.3=py38h06a4308_0
- python=3.8.5=h7579374_1
- pytorch=1.7.1=py3.8_cuda11.0.221_cudnn8.0.5_0
- readline=8.1=h27cfd23_0
- scipy=1.5.2=py38h0b6359f_0
- setuptools=52.0.0=py38h06a4308_0
- six=1.15.0=py38h06a4308_0
- sqlite=3.33.0=h62c20be_0
- tk=8.6.10=hbc83047_0
- torchaudio=0.7.2=py38
- torchvision=0.8.2=py38_cu110
- typing_extensions=3.7.4.3=pyh06a4308_0
- wheel=0.36.2=pyhd3eb1b0_0
- xz=5.2.5=h7b6447c_0
- yaml=0.2.5=h7b6447c_0
- zlib=1.2.11=h7b6447c_3
- zstd=1.4.5=h9ceee32_0
- pip:
- absl-py==0.11.0
- cachetools==4.2.1
- chardet==4.0.0
- fasttext==0.9.2
- gensim==3.8.3
- google-auth==1.24.0
- google-auth-oauthlib==0.4.2
- grpcio==1.35.0
- idna==2.10
- markdown==3.3.3
- oauthlib==3.1.0
- protobuf==3.14.0
- pyasn1==0.4.8
- pyasn1-modules==0.2.8
- pybind11==2.6.2
- pyyaml==5.4.1
- requests==2.25.1
- requests-oauthlib==1.3.0
- rsa==4.7
- smart-open==4.1.2
- tensorboard==2.4.1
- tensorboard-plugin-wit==1.8.0
- tqdm==4.56.0
- urllib3==1.26.3
- werkzeug==1.0.1
================================================
FILE: czsl/flags.py
================================================
import argparse
DATA_FOLDER = "/project/ZHIHOUDATA/CZSL"
parser = argparse.ArgumentParser(formatter_class=argparse.ArgumentDefaultsHelpFormatter)
parser.add_argument('--config', default='configs/args.yml', help='path of the config file (training only)')
parser.add_argument('--dataset', default='mitstates', help='mitstates|zappos')
parser.add_argument('--data_dir', default='mit-states', help='local path to data root dir from ' + DATA_FOLDER)
parser.add_argument('--logpath', default=None, help='Path to dir where to logs are stored (test only)')
parser.add_argument('--splitname', default='compositional-split-natural', help="dataset split")
parser.add_argument('--cv_dir', default='logs/', help='dir to save checkpoints and logs to')
parser.add_argument('--name', default='temp', help='Name of exp used to name models')
parser.add_argument('--load', default=None, help='path to checkpoint to load from')
parser.add_argument('--image_extractor', default = 'resnet18', help = 'Feature extractor model')
parser.add_argument('--norm_family', default = 'imagenet', help = 'Normalization values from dataset')
parser.add_argument('--num_negs', type=int, default=1, help='Number of negatives to sample per positive (triplet loss)')
parser.add_argument('--pair_dropout', type=float, default=0.0, help='Each epoch drop this fraction of train pairs')
parser.add_argument('--test_set', default='val', help='val|test mode')
parser.add_argument('--clean_only', action='store_true', default=False, help='use only clean subset of data (mitstates)')
parser.add_argument('--subset', action='store_true', default=False, help='test on a 1000 image subset (debug purpose)')
parser.add_argument('--open_world', action='store_true', default=False, help='perform open world experiment')
parser.add_argument('--test_batch_size', type=int, default=32, help="Batch size at test/eval time")
# Model parameters
parser.add_argument('--model', default='graphfull', help='visprodNN|redwine|labelembed+|attributeop|tmn|compcos')
parser.add_argument('--emb_dim', type=int, default=300, help='dimension of share embedding space')
parser.add_argument('--nlayers', type=int, default=3, help='Layers in the image embedder')
parser.add_argument('--nmods', type=int, default=24, help='number of mods per layer for TMN')
parser.add_argument('--embed_rank', type=int, default=64, help='intermediate dimension in the gating model for TMN')
parser.add_argument('--bias', type=float, default=1e3, help='Bias value for unseen concepts')
parser.add_argument('--update_features', action = 'store_true', default=False, help='If specified, train feature extractor')
parser.add_argument('--freeze_features', action = 'store_true', default=False, help='If specified, put extractor in eval mode')
parser.add_argument('--emb_init', default=None, help='w2v|ft|gl|glove|word2vec|fasttext, name of embeddings to use for initializing the primitives')
parser.add_argument('--clf_init', action='store_true', default=False, help='initialize inputs with SVM weights')
parser.add_argument('--static_inp', action='store_true', default=False, help='do not optimize primitives representations')
parser.add_argument('--composition', default='mlp_add', help='add|mul|mlp|mlp_add, how to compose primitives')
parser.add_argument('--relu', action='store_true', default=False, help='Use relu in image embedder')
parser.add_argument('--dropout', action='store_true', default=False, help='Use dropout in image embedder')
parser.add_argument('--norm', action='store_true', default=False, help='Use normalization in image embedder')
parser.add_argument('--train_only', action='store_true', default=False, help='Optimize only for train pairs')
parser.add_argument('--add_bt', type=int, default=0.0, help='global images')
parser.add_argument('--eval_bt', type=int, default=0, help='global images')
#CGE
parser.add_argument('--graph', action='store_true', default=False, help='graph l2 distance triplet') # Do we need this
parser.add_argument('--graph_init', default='graph', help='filename, file from which initializing the nodes and adjacency matrix of the graph')
parser.add_argument('--gcn_type', default='gcn', help='GCN Version')
# Forward
parser.add_argument('--eval_type', default='dist', help='dist|prod|direct, function for computing the predictions')
# Primitive-based loss
parser.add_argument('--lambda_aux', type=float, default=0.0, help='weight of auxiliar losses on the primitives')
# AoP
parser.add_argument('--lambda_inv', type=float, default=0.0, help='weight of inverse consistency loss')
parser.add_argument('--lambda_comm', type=float, default=0.0, help='weight of commutative loss')
parser.add_argument('--lambda_ant', type=float, default=0.0, help='weight of antonyms loss')
# SymNet
parser.add_argument("--lambda_trip", type=float, default=0, help='weight of triplet loss')
parser.add_argument("--lambda_sym", type=float, default=0, help='weight of symmetry loss')
parser.add_argument("--lambda_axiom", type=float, default=0, help='weight of axiom loss')
parser.add_argument("--lambda_cls_attr", type=float, default=0, help='weight of classification loss on attributes')
parser.add_argument("--lambda_cls_obj", type=float, default=0, help='weight of classification loss on objects')
# CompCos (for the margin, see below)
parser.add_argument('--cosine_scale', type=float, default=20,help="Scale for cosine similarity")
parser.add_argument('--epoch_max_margin', type=float, default=15,help="Epoch of max margin")
parser.add_argument('--update_feasibility_every', type=float, default=1,help="Frequency of feasibility scores update in epochs")
parser.add_argument('--hard_masking', action='store_true', default=False, help='hard masking during test time')
parser.add_argument('--threshold', default=None,help="Apply a specific threshold at test time for the hard masking")
parser.add_argument('--threshold_trials', type=int, default=50,help="how many threshold values to try")
# Hyperparameters
parser.add_argument('--topk', type=int, default=1,help="Compute topk accuracy")
parser.add_argument('--margin', type=float, default=2,help="Margin for triplet loss or feasibility scores in CompCos")
parser.add_argument('--workers', type=int, default=8,help="Number of workers")
parser.add_argument('--batch_size', type=int, default=512,help="Training batch size")
parser.add_argument('--lr', type=float, default=5e-5,help="Learning rate")
parser.add_argument('--lrg', type=float, default=1e-3,help="Learning rate feature extractor")
parser.add_argument('--lrb', type=float, default=5e-5,help="Learning rate encoder")
parser.add_argument('--wd', type=float, default=5e-5,help="Weight decay")
parser.add_argument('--save_every', type=int, default=10000,help="Frequency of snapshots in epochs")
parser.add_argument('--eval_val_every', type=int, default=1,help="Frequency of eval in epochs")
parser.add_argument('--max_epochs', type=int, default=800,help="Max number of epochs")
parser.add_argument("--fc_emb", default='768,1024,1200',help="Image embedder layer config")
parser.add_argument("--gr_emb", default='d4096,d',help="graph layers config")
================================================
FILE: czsl/models/__init__.py
================================================
================================================
FILE: czsl/models/common.py
================================================
import torch
import torch.nn as nn
import torch.nn.functional as F
import numpy as np
import copy
from scipy.stats import hmean
device = 'cuda' if torch.cuda.is_available() else 'cpu'
class MLP(nn.Module):
'''
Baseclass to create a simple MLP
Inputs
inp_dim: Int, Input dimension
out-dim: Int, Output dimension
num_layer: Number of hidden layers
relu: Bool, Use non linear function at output
bias: Bool, Use bias
'''
def __init__(self, inp_dim, out_dim, num_layers = 1, relu = True, bias = True, dropout = False, norm = False, layers = []):
super(MLP, self).__init__()
mod = []
incoming = inp_dim
for layer in range(num_layers - 1):
if len(layers) == 0:
outgoing = incoming
else:
outgoing = layers.pop(0)
mod.append(nn.Linear(incoming, outgoing, bias = bias))
incoming = outgoing
if norm:
mod.append(nn.LayerNorm(outgoing))
# mod.append(nn.BatchNorm1d(outgoing))
mod.append(nn.ReLU(inplace = True))
# mod.append(nn.LeakyReLU(inplace=True, negative_slope=0.2))
if dropout:
mod.append(nn.Dropout(p = 0.5))
mod.append(nn.Linear(incoming, out_dim, bias = bias))
if relu:
mod.append(nn.ReLU(inplace = True))
# mod.append(nn.LeakyReLU(inplace=True, negative_slope=0.2))
self.mod = nn.Sequential(*mod)
def forward(self, x):
return self.mod(x)
class Reshape(nn.Module):
def __init__(self, *args):
super(Reshape, self).__init__()
self.shape = args
def forward(self, x):
return x.view(self.shape)
def calculate_margines(domain_embedding, gt, margin_range = 5):
'''
domain_embedding: pairs * feats
gt: batch * feats
'''
batch_size, pairs, features = gt.shape[0], domain_embedding.shape[0], domain_embedding.shape[1]
gt_expanded = gt[:,None,:].expand(-1, pairs, -1)
domain_embedding_expanded = domain_embedding[None, :, :].expand(batch_size, -1, -1)
margin = (gt_expanded - domain_embedding_expanded)**2
margin = margin.sum(2)
max_margin, _ = torch.max(margin, dim = 0)
margin /= max_margin
margin *= margin_range
return margin
def l2_all_batched(image_embedding, domain_embedding):
'''
Image Embedding: Tensor of Batch_size * pairs * Feature_dim
domain_embedding: Tensor of pairs * Feature_dim
'''
pairs = image_embedding.shape[1]
domain_embedding_extended = image_embedding[:,None,:].expand(-1,pairs,-1)
l2_loss = (image_embedding - domain_embedding_extended) ** 2
l2_loss = l2_loss.sum(2)
l2_loss = l2_loss.sum() / l2_loss.numel()
return l2_loss
def same_domain_triplet_loss(image_embedding, trip_images, gt, hard_k = None, margin = 2):
'''
Image Embedding: Tensor of Batch_size * Feature_dim
Triplet Images: Tensor of Batch_size * num_pairs * Feature_dim
GT: Tensor of Batch_size
'''
batch_size, pairs, features = trip_images.shape
batch_iterator = torch.arange(batch_size).to(device)
image_embedding_expanded = image_embedding[:,None,:].expand(-1, pairs, -1)
diff = (image_embedding_expanded - trip_images)**2
diff = diff.sum(2)
positive_anchor = diff[batch_iterator, gt][:,None]
positive_anchor = positive_anchor.expand(-1, pairs)
# Calculating triplet loss
triplet_loss = positive_anchor - diff + margin
# Setting positive anchor loss to 0
triplet_loss[batch_iterator, gt] = 0
# Removing easy triplets
triplet_loss[triplet_loss < 0] = 0
# If only mining hard triplets
if hard_k:
triplet_loss, _ = triplet_loss.topk(hard_k)
# Counting number of valid pairs
num_positive_triplets = triplet_loss[triplet_loss > 1e-16].size(0)
# Calculating the final loss
triplet_loss = triplet_loss.sum() / (num_positive_triplets + 1e-16)
return triplet_loss
def cross_domain_triplet_loss(image_embedding, domain_embedding, gt, hard_k = None, margin = 2):
'''
Image Embedding: Tensor of Batch_size * Feature_dim
Domain Embedding: Tensor of Num_pairs * Feature_dim
gt: Tensor of Batch_size with ground truth labels
margin: Float of margin
Returns:
Triplet loss of all valid triplets
'''
batch_size, pairs, features = image_embedding.shape[0], domain_embedding.shape[0], domain_embedding.shape[1]
batch_iterator = torch.arange(batch_size).to(device)
# Now dimensions will be Batch_size * Num_pairs * Feature_dim
image_embedding = image_embedding[:,None,:].expand(-1, pairs, -1)
domain_embedding = domain_embedding[None,:,:].expand(batch_size, -1, -1)
# Calculating difference
diff = (image_embedding - domain_embedding)**2
diff = diff.sum(2)
# Getting the positive pair
positive_anchor = diff[batch_iterator, gt][:,None]
positive_anchor = positive_anchor.expand(-1, pairs)
# Calculating triplet loss
triplet_loss = positive_anchor - diff + margin
# Setting positive anchor loss to 0
triplet_loss[batch_iterator, gt] = 0
# Removing easy triplets
triplet_loss[triplet_loss < 0] = 0
# If only mining hard triplets
if hard_k:
triplet_loss, _ = triplet_loss.topk(hard_k)
# Counting number of valid pairs
num_positive_triplets = triplet_loss[triplet_loss > 1e-16].size(0)
# Calculating the final loss
triplet_loss = triplet_loss.sum() / (num_positive_triplets + 1e-16)
return triplet_loss
def same_domain_triplet_loss_old(image_embedding, positive_anchor, negative_anchor, margin = 2):
'''
Image Embedding: Tensor of Batch_size * Feature_dim
Positive anchor: Tensor of Batch_size * Feature_dim
negative anchor: Tensor of Batch_size * negs *Feature_dim
'''
batch_size, negs, features = negative_anchor.shape
dist_pos = (image_embedding - positive_anchor)**2
dist_pos = dist_pos.sum(1)
dist_pos = dist_pos[:, None].expand(-1, negs)
image_embedding_expanded = image_embedding[:,None,:].expand(-1, negs, -1)
dist_neg = (image_embedding_expanded - negative_anchor)**2
dist_neg = dist_neg.sum(2)
triplet_loss = dist_pos - dist_neg + margin
triplet_loss[triplet_loss < 0] = 0
num_positive_triplets = triplet_loss[triplet_loss > 1e-16].size(0)
triplet_loss = triplet_loss.sum() / (num_positive_triplets + 1e-16)
return triplet_loss
def pairwise_distances(x, y=None):
'''
Input: x is a Nxd matrix
y is an optional Mxd matirx
Output: dist is a NxM matrix where dist[i,j] is the square norm between x[i,:] and y[j,:]
if y is not given then use 'y=x'.
i.e. dist[i,j] = ||x[i,:]-y[j,:]||^2
'''
x_norm = (x**2).sum(1).view(-1, 1)
if y is not None:
y_t = torch.transpose(y, 0, 1)
y_norm = (y**2).sum(1).view(1, -1)
else:
y_t = torch.transpose(x, 0, 1)
y_norm = x_norm.view(1, -1)
dist = x_norm + y_norm - 2.0 * torch.mm(x, y_t)
# Ensure diagonal is zero if x=y
# if y is None:
# dist = dist - torch.diag(dist.diag)
return torch.clamp(dist, 0.0, np.inf)
class Evaluator:
def __init__(self, dset, model):
self.dset = dset
# Convert text pairs to idx tensors: [('sliced', 'apple'), ('ripe', 'apple'), ...] --> torch.LongTensor([[0,1],[1,1], ...])
pairs = [(dset.attr2idx[attr], dset.obj2idx[obj]) for attr, obj in dset.pairs]
self.train_pairs = [(dset.attr2idx[attr], dset.obj2idx[obj]) for attr, obj in dset.train_pairs]
self.pairs = torch.LongTensor(pairs)
# Mask over pairs that occur in closed world
# Select set based on phase
if dset.phase == 'train':
print('Evaluating with train pairs')
test_pair_set = set(dset.train_pairs)
test_pair_gt = set(dset.train_pairs)
elif dset.phase == 'val':
print('Evaluating with validation pairs')
test_pair_set = set(dset.val_pairs + dset.train_pairs)
test_pair_gt = set(dset.val_pairs)
else:
print('Evaluating with test pairs')
test_pair_set = set(dset.test_pairs + dset.train_pairs)
test_pair_gt = set(dset.test_pairs)
self.test_pair_dict = [(dset.attr2idx[attr], dset.obj2idx[obj]) for attr, obj in test_pair_gt]
self.test_pair_dict = dict.fromkeys(self.test_pair_dict, 0)
# dict values are pair val, score, total
for attr, obj in test_pair_gt:
pair_val = dset.pair2idx[(attr,obj)]
key = (dset.attr2idx[attr], dset.obj2idx[obj])
self.test_pair_dict[key] = [pair_val, 0, 0]
if dset.open_world:
masks = [1 for _ in dset.pairs]
else:
masks = [1 if pair in test_pair_set else 0 for pair in dset.pairs]
self.closed_mask = torch.BoolTensor(masks)
# Mask of seen concepts
seen_pair_set = set(dset.train_pairs)
mask = [1 if pair in seen_pair_set else 0 for pair in dset.pairs]
self.seen_mask = torch.BoolTensor(mask)
# Object specific mask over which pairs occur in the object oracle setting
oracle_obj_mask = []
for _obj in dset.objs:
mask = [1 if _obj == obj else 0 for attr, obj in dset.pairs]
oracle_obj_mask.append(torch.BoolTensor(mask))
self.oracle_obj_mask = torch.stack(oracle_obj_mask, 0)
# Decide if the model under evaluation is a manifold model or not
self.score_model = self.score_manifold_model
# Generate mask for each settings, mask scores, and get prediction labels
def generate_predictions(self, scores, obj_truth, bias = 0.0, topk = 5): # (Batch, #pairs)
'''
Inputs
scores: Output scores
obj_truth: Ground truth object
Returns
results: dict of results in 3 settings
'''
def get_pred_from_scores(_scores, topk):
'''
Given list of scores, returns top 10 attr and obj predictions
Check later
'''
_, pair_pred = _scores.topk(topk, dim = 1) #sort returns indices of k largest values
pair_pred = pair_pred.contiguous().view(-1)
attr_pred, obj_pred = self.pairs[pair_pred][:, 0].view(-1, topk), \
self.pairs[pair_pred][:, 1].view(-1, topk)
return (attr_pred, obj_pred)
results = {}
orig_scores = scores.clone()
mask = self.seen_mask.repeat(scores.shape[0],1) # Repeat mask along pairs dimension
scores[~mask] += bias # Add bias to test pairs
# Unbiased setting
# Open world setting --no mask, all pairs of the dataset
results.update({'open': get_pred_from_scores(scores, topk)})
results.update({'unbiased_open': get_pred_from_scores(orig_scores, topk)})
# Closed world setting - set the score for all Non test pairs to -1e10,
# this excludes the pairs from set not in evaluation
mask = self.closed_mask.repeat(scores.shape[0], 1)
closed_scores = scores.clone()
closed_scores[~mask] = -1e10
closed_orig_scores = orig_scores.clone()
closed_orig_scores[~mask] = -1e10
results.update({'closed': get_pred_from_scores(closed_scores, topk)})
results.update({'unbiased_closed': get_pred_from_scores(closed_orig_scores, topk)})
# Object_oracle setting - set the score to -1e10 for all pairs where the true object does Not participate, can also use the closed score
mask = self.oracle_obj_mask[obj_truth]
oracle_obj_scores = scores.clone()
oracle_obj_scores[~mask] = -1e10
oracle_obj_scores_unbiased = orig_scores.clone()
oracle_obj_scores_unbiased[~mask] = -1e10
results.update({'object_oracle': get_pred_from_scores(oracle_obj_scores, 1)})
results.update({'object_oracle_unbiased': get_pred_from_scores(oracle_obj_scores_unbiased, 1)})
return results
def score_clf_model(self, scores, obj_truth, topk = 5):
'''
Wrapper function to call generate_predictions for CLF models
'''
attr_pred, obj_pred = scores
# Go to CPU
attr_pred, obj_pred, obj_truth = attr_pred.to('cpu'), obj_pred.to('cpu'), obj_truth.to('cpu')
# Gather scores (P(a), P(o)) for all relevant (a,o) pairs
# Multiply P(a) * P(o) to get P(pair)
attr_subset = attr_pred.index_select(1, self.pairs[:,0]) # Return only attributes that are in our pairs
obj_subset = obj_pred.index_select(1, self.pairs[:, 1])
scores = (attr_subset * obj_subset) # (Batch, #pairs)
results = self.generate_predictions(scores, obj_truth)
results['biased_scores'] = scores
return results
def score_manifold_model(self, scores, obj_truth, bias = 0.0, topk = 5):
'''
Wrapper function to call generate_predictions for manifold models
'''
# Go to CPU
scores = {k: v.to('cpu') for k, v in scores.items()}
obj_truth = obj_truth.to(device)
# Gather scores for all relevant (a,o) pairs
scores = torch.stack(
[scores[(attr,obj)] for attr, obj in self.dset.pairs], 1
) # (Batch, #pairs)
orig_scores = scores.clone()
results = self.generate_predictions(scores, obj_truth, bias, topk)
results['scores'] = orig_scores
return results
def score_fast_model(self, scores, obj_truth, bias = 0.0, topk = 5):
'''
Wrapper function to call generate_predictions for manifold models
'''
results = {}
mask = self.seen_mask.repeat(scores.shape[0],1) # Repeat mask along pairs dimension
scores[~mask] += bias # Add bias to test pairs
mask = self.closed_mask.repeat(scores.shape[0], 1)
closed_scores = scores.clone()
closed_scores[~mask] = -1e10
_, pair_pred = closed_scores.topk(topk, dim = 1) #sort returns indices of k largest values
pair_pred = pair_pred.contiguous().view(-1)
attr_pred, obj_pred = self.pairs[pair_pred][:, 0].view(-1, topk), \
self.pairs[pair_pred][:, 1].view(-1, topk)
results.update({'closed': (attr_pred, obj_pred)})
return results
def evaluate_predictions(self, predictions, attr_truth, obj_truth, pair_truth, allpred, topk = 1):
# Go to CPU
attr_truth, obj_truth, pair_truth = attr_truth.to('cpu'), obj_truth.to('cpu'), pair_truth.to('cpu')
pairs = list(
zip(list(attr_truth.numpy()), list(obj_truth.numpy())))
seen_ind, unseen_ind = [], []
for i in range(len(attr_truth)):
if pairs[i] in self.train_pairs:
seen_ind.append(i)
else:
unseen_ind.append(i)
seen_ind, unseen_ind = torch.LongTensor(seen_ind), torch.LongTensor(unseen_ind)
def _process(_scores):
# Top k pair accuracy
# Attribute, object and pair
attr_match = (attr_truth.unsqueeze(1).repeat(1, topk) == _scores[0][:, :topk])
obj_match = (obj_truth.unsqueeze(1).repeat(1, topk) == _scores[1][:, :topk])
# Match of object pair
match = (attr_match * obj_match).any(1).float()
attr_match = attr_match.any(1).float()
obj_match = obj_match.any(1).float()
# Match of seen and unseen pairs
seen_match = match[seen_ind]
unseen_match = match[unseen_ind]
### Calculating class average accuracy
# local_score_dict = copy.deepcopy(self.test_pair_dict)
# for pair_gt, pair_pred in zip(pairs, match):
# # print(pair_gt)
# local_score_dict[pair_gt][2] += 1.0 #increase counter
# if int(pair_pred) == 1:
# local_score_dict[pair_gt][1] += 1.0
# # Now we have hits and totals for classes in evaluation set
# seen_score, unseen_score = [], []
# for key, (idx, hits, total) in local_score_dict.items():
# score = hits/total
# if bool(self.seen_mask[idx]) == True:
# seen_score.append(score)
# else:
# unseen_score.append(score)
seen_score, unseen_score = torch.ones(512,5), torch.ones(512,5)
return attr_match, obj_match, match, seen_match, unseen_match, \
torch.Tensor(seen_score+unseen_score), torch.Tensor(seen_score), torch.Tensor(unseen_score)
def _add_to_dict(_scores, type_name, stats):
base = ['_attr_match', '_obj_match', '_match', '_seen_match', '_unseen_match', '_ca', '_seen_ca', '_unseen_ca']
for val, name in zip(_scores, base):
stats[type_name + name] = val
##################### Match in places where corrent object
obj_oracle_match = (attr_truth == predictions['object_oracle'][0][:, 0]).float() #object is already conditioned
obj_oracle_match_unbiased = (attr_truth == predictions['object_oracle_unbiased'][0][:, 0]).float()
stats = dict(obj_oracle_match = obj_oracle_match, obj_oracle_match_unbiased = obj_oracle_match_unbiased)
#################### Closed world
closed_scores = _process(predictions['closed'])
unbiased_closed = _process(predictions['unbiased_closed'])
_add_to_dict(closed_scores, 'closed', stats)
_add_to_dict(unbiased_closed, 'closed_ub', stats)
#################### Calculating AUC
scores = predictions['scores']
# getting score for each ground truth class
correct_scores = scores[torch.arange(scores.shape[0]), pair_truth][unseen_ind]
# Getting top predicted score for these unseen classes
max_seen_scores = predictions['scores'][unseen_ind][:, self.seen_mask].topk(topk, dim=1)[0][:, topk - 1]
# Getting difference between these scores
unseen_score_diff = max_seen_scores - correct_scores
# Getting matched classes at max bias for diff
unseen_matches = stats['closed_unseen_match'].bool()
correct_unseen_score_diff = unseen_score_diff[unseen_matches] - 1e-4
# sorting these diffs
correct_unseen_score_diff = torch.sort(correct_unseen_score_diff)[0]
magic_binsize = 20
# getting step size for these bias values
bias_skip = max(len(correct_unseen_score_diff) // magic_binsize, 1)
# Getting list
biaslist = correct_unseen_score_diff[::bias_skip]
seen_match_max = float(stats['closed_seen_match'].mean())
unseen_match_max = float(stats['closed_unseen_match'].mean())
seen_accuracy, unseen_accuracy = [], []
# Go to CPU
base_scores = {k: v.to('cpu') for k, v in allpred.items()}
obj_truth = obj_truth.to('cpu')
# Gather scores for all relevant (a,o) pairs
base_scores = torch.stack(
[allpred[(attr,obj)] for attr, obj in self.dset.pairs], 1
) # (Batch, #pairs)
for bias in biaslist:
scores = base_scores.clone()
results = self.score_fast_model(scores, obj_truth, bias = bias, topk = topk)
results = results['closed'] # we only need biased
results = _process(results)
seen_match = float(results[3].mean())
unseen_match = float(results[4].mean())
seen_accuracy.append(seen_match)
unseen_accuracy.append(unseen_match)
seen_accuracy.append(seen_match_max)
unseen_accuracy.append(unseen_match_max)
seen_accuracy, unseen_accuracy = np.array(seen_accuracy), np.array(unseen_accuracy)
area = np.trapz(seen_accuracy, unseen_accuracy)
for key in stats:
stats[key] = float(stats[key].mean())
harmonic_mean = hmean([seen_accuracy, unseen_accuracy], axis = 0)
max_hm = np.max(harmonic_mean)
idx = np.argmax(harmonic_mean)
if idx == len(biaslist):
bias_term = 1e3
else:
bias_term = biaslist[idx]
stats['biasterm'] = float(bias_term)
stats['best_unseen'] = np.max(unseen_accuracy)
stats['best_seen'] = np.max(seen_accuracy)
stats['AUC'] = area
stats['hm_unseen'] = unseen_accuracy[idx]
stats['hm_seen'] = seen_accuracy[idx]
stats['best_hm'] = max_hm
return stats
================================================
FILE: czsl/models/compcos.py
================================================
import torch
import torch.nn as nn
import torch.nn.functional as F
from .word_embedding import load_word_embeddings
from .common import MLP
from itertools import product
device = 'cuda' if torch.cuda.is_available() else 'cpu'
def compute_cosine_similarity(names, weights, return_dict=True):
pairing_names = list(product(names, names))
normed_weights = F.normalize(weights,dim=1)
similarity = torch.mm(normed_weights, normed_weights.t())
if return_dict:
dict_sim = {}
for i,n in enumerate(names):
for j,m in enumerate(names):
dict_sim[(n,m)]=similarity[i,j].item()
return dict_sim
return pairing_names, similarity.to('cpu')
class CompCos(nn.Module):
def __init__(self, dset, args):
super(CompCos, self).__init__()
self.args = args
self.dset = dset
def get_all_ids(relevant_pairs):
# Precompute validation pairs
attrs, objs = zip(*relevant_pairs)
attrs = [dset.attr2idx[attr] for attr in attrs]
objs = [dset.obj2idx[obj] for obj in objs]
pairs = [a for a in range(len(relevant_pairs))]
attrs = torch.LongTensor(attrs).to(device)
objs = torch.LongTensor(objs).to(device)
pairs = torch.LongTensor(pairs).to(device)
return attrs, objs, pairs
# Validation
self.val_attrs, self.val_objs, self.val_pairs = get_all_ids(self.dset.pairs)
# for indivual projections
self.uniq_attrs, self.uniq_objs = torch.arange(len(self.dset.attrs)).long().to(device), \
torch.arange(len(self.dset.objs)).long().to(device)
self.factor = 2
self.scale = self.args.cosine_scale
self.add_bt = args.add_bt
if dset.open_world:
self.train_forward = self.train_forward_open
self.known_pairs = dset.train_pairs
seen_pair_set = set(self.known_pairs)
mask = [1 if pair in seen_pair_set else 0 for pair in dset.pairs]
self.seen_mask = torch.BoolTensor(mask).to(device) * 1.
self.activated = False
# Init feasibility-related variables
self.attrs = dset.attrs
self.objs = dset.objs
self.possible_pairs = dset.pairs
self.validation_pairs = dset.val_pairs
self.feasibility_margin = (1-self.seen_mask).float()
self.epoch_max_margin = self.args.epoch_max_margin
self.cosine_margin_factor = -args.margin
# Intantiate attribut-object relations, needed just to evaluate mined pairs
self.obj_by_attrs_train = {k: [] for k in self.attrs}
for (a, o) in self.known_pairs:
self.obj_by_attrs_train[a].append(o)
# Intantiate attribut-object relations, needed just to evaluate mined pairs
self.attrs_by_obj_train = {k: [] for k in self.objs}
for (a, o) in self.known_pairs:
self.attrs_by_obj_train[o].append(a)
else:
self.train_forward = self.train_forward_closed
# Precompute training compositions
if args.train_only:
self.train_attrs, self.train_objs, self.train_pairs = get_all_ids(self.dset.train_pairs)
else:
self.train_attrs, self.train_objs, self.train_pairs = self.val_attrs, self.val_objs, self.val_pairs
try:
self.args.fc_emb = self.args.fc_emb.split(',')
except:
self.args.fc_emb = [self.args.fc_emb]
layers = []
for a in self.args.fc_emb:
a = int(a)
layers.append(a)
self.image_embedder = MLP(dset.feat_dim, int(args.emb_dim), relu=args.relu, num_layers=args.nlayers,
dropout=self.args.dropout,
norm=self.args.norm, layers=layers)
# Fixed
self.composition = args.composition
input_dim = args.emb_dim
self.attr_embedder = nn.Embedding(len(dset.attrs), input_dim)
self.obj_embedder = nn.Embedding(len(dset.objs), input_dim)
# init with word embeddings
if args.emb_init:
pretrained_weight = load_word_embeddings(args.emb_init, dset.attrs)
self.attr_embedder.weight.data.copy_(pretrained_weight)
pretrained_weight = load_word_embeddings(args.emb_init, dset.objs)
self.obj_embedder.weight.data.copy_(pretrained_weight)
# static inputs
if args.static_inp:
for param in self.attr_embedder.parameters():
param.requires_grad = False
for param in self.obj_embedder.parameters():
param.requires_grad = False
# Composition MLP
self.projection = nn.Linear(input_dim * 2, args.emb_dim)
def freeze_representations(self):
print('Freezing representations')
for param in self.image_embedder.parameters():
param.requires_grad = False
for param in self.attr_embedder.parameters():
param.requires_grad = False
for param in self.obj_embedder.parameters():
param.requires_grad = False
def compose(self, attrs, objs):
attrs, objs = self.attr_embedder(attrs), self.obj_embedder(objs)
inputs = torch.cat([attrs, objs], 1)
output = self.projection(inputs)
output = F.normalize(output, dim=1)
return output
def compute_feasibility(self):
obj_embeddings = self.obj_embedder(torch.arange(len(self.objs)).long().to('cuda'))
obj_embedding_sim = compute_cosine_similarity(self.objs, obj_embeddings,
return_dict=True)
attr_embeddings = self.attr_embedder(torch.arange(len(self.attrs)).long().to('cuda'))
attr_embedding_sim = compute_cosine_similarity(self.attrs, attr_embeddings,
return_dict=True)
feasibility_scores = self.seen_mask.clone().float()
for a in self.attrs:
for o in self.objs:
if (a, o) not in self.known_pairs:
idx = self.dset.all_pair2idx[(a, o)]
score_obj = self.get_pair_scores_objs(a, o, obj_embedding_sim)
score_attr = self.get_pair_scores_attrs(a, o, attr_embedding_sim)
score = (score_obj + score_attr) / 2
feasibility_scores[idx] = score
self.feasibility_scores = feasibility_scores
return feasibility_scores * (1 - self.seen_mask.float())
def get_pair_scores_objs(self, attr, obj, obj_embedding_sim):
score = -1.
for o in self.objs:
if o!=obj and attr in self.attrs_by_obj_train[o]:
temp_score = obj_embedding_sim[(obj,o)]
if temp_score>score:
score=temp_score
return score
def get_pair_scores_attrs(self, attr, obj, attr_embedding_sim):
score = -1.
for a in self.attrs:
if a != attr and obj in self.obj_by_attrs_train[a]:
temp_score = attr_embedding_sim[(attr, a)]
if temp_score > score:
score = temp_score
return score
def update_feasibility(self,epoch):
self.activated = True
feasibility_scores = self.compute_feasibility()
self.feasibility_margin = min(1.,epoch/self.epoch_max_margin) * \
(self.cosine_margin_factor*feasibility_scores.float().to(device))
def val_forward(self, x):
img = x[0]
img_feats = self.image_embedder(img)
img_feats_normed = F.normalize(img_feats, dim=1)
pair_embeds = self.compose(self.val_attrs, self.val_objs).permute(1, 0) # Evaluate all pairs
score = torch.matmul(img_feats_normed, pair_embeds)
scores = {}
for itr, pair in enumerate(self.dset.pairs):
scores[pair] = score[:, self.dset.all_pair2idx[pair]]
return None, scores
def val_forward_with_threshold(self, x, th=0.):
img = x[0]
img_feats = self.image_embedder(img)
img_feats_normed = F.normalize(img_feats, dim=1)
pair_embeds = self.compose(self.val_attrs, self.val_objs).permute(1, 0) # Evaluate all pairs
score = torch.matmul(img_feats_normed, pair_embeds)
# Note: Pairs are already aligned here
mask = (self.feasibility_scores>=th).float()
score = score*mask + (1.-mask)*(-1.)
scores = {}
for itr, pair in enumerate(self.dset.pairs):
scores[pair] = score[:, self.dset.all_pair2idx[pair]]
return None, scores
def train_forward_open(self, x):
img, attrs, objs, pairs = x[0], x[1], x[2], x[3]
img_feats = self.image_embedder(img)
pair_embed = self.compose(self.train_attrs, self.train_objs).permute(1, 0)
img_feats_normed = F.normalize(img_feats, dim=1)
pair_pred = torch.matmul(img_feats_normed, pair_embed)
if self.add_bt > 1:
pairs = torch.cat([pairs, pairs], dim=0)
if self.activated:
pair_pred += (1 - self.seen_mask) * self.feasibility_margin
loss_cos = F.cross_entropy(self.scale * pair_pred, pairs)
else:
pair_pred = pair_pred * self.seen_mask + (1 - self.seen_mask) * (-10)
loss_cos = F.cross_entropy(self.scale * pair_pred, pairs)
return loss_cos.mean(), None
def train_forward_closed(self, x):
img, attrs, objs, pairs = x[0], x[1], x[2], x[3]
img_feats = self.image_embedder(img)
pair_embed = self.compose(self.train_attrs, self.train_objs).permute(1, 0)
img_feats_normed = F.normalize(img_feats, dim=1)
pair_pred = torch.matmul(img_feats_normed, pair_embed)
loss_cos = F.cross_entropy(self.scale * pair_pred, pairs)
return loss_cos.mean(), None
def forward(self, x):
if self.training:
loss, pred = self.train_forward(x)
else:
with torch.no_grad():
loss, pred = self.val_forward(x)
return loss, pred
================================================
FILE: czsl/models/gcn.py
================================================
import numpy as np
import scipy.sparse as sp
import math
import torch
import torch.nn as nn
import torch.nn.functional as F
from torch.nn.init import xavier_uniform_
device = 'cuda' if torch.cuda.is_available() else 'cpu'
def normt_spm(mx, method='in'):
if method == 'in':
mx = mx.transpose()
rowsum = np.array(mx.sum(1))
r_inv = np.power(rowsum, -1).flatten()
r_inv[np.isinf(r_inv)] = 0.
r_mat_inv = sp.diags(r_inv)
mx = r_mat_inv.dot(mx)
return mx
if method == 'sym':
rowsum = np.array(mx.sum(1))
r_inv = np.power(rowsum, -0.5).flatten()
r_inv[np.isinf(r_inv)] = 0.
r_mat_inv = sp.diags(r_inv)
mx = mx.dot(r_mat_inv).transpose().dot(r_mat_inv)
return mx
def spm_to_tensor(sparse_mx):
sparse_mx = sparse_mx.tocoo().astype(np.float32)
indices = torch.from_numpy(np.vstack(
(sparse_mx.row, sparse_mx.col))).long()
values = torch.from_numpy(sparse_mx.data)
shape = torch.Size(sparse_mx.shape)
return torch.sparse.FloatTensor(indices, values, shape)
class GraphConv(nn.Module):
def __init__(self, in_channels, out_channels, dropout=False, relu=True):
super().__init__()
if dropout:
self.dropout = nn.Dropout(p=0.5)
else:
self.dropout = None
self.layer = nn.Linear(in_channels, out_channels)
if relu:
# self.relu = nn.LeakyReLU(negative_slope=0.2)
self.relu = nn.ReLU()
else:
self.relu = None
def forward(self, inputs, adj):
if self.dropout is not None:
inputs = self.dropout(inputs)
outputs = torch.mm(adj, torch.mm(inputs, self.layer.weight.T)) + self.layer.bias
if self.relu is not None:
outputs = self.relu(outputs)
return outputs
class GCN(nn.Module):
def __init__(self, adj, in_channels, out_channels, hidden_layers):
super().__init__()
adj = normt_spm(adj, method='in')
adj = spm_to_tensor(adj)
self.adj = adj.to(device)
self.train_adj = self.adj
hl = hidden_layers.split(',')
if hl[-1] == 'd':
dropout_last = True
hl = hl[:-1]
else:
dropout_last = False
i = 0
layers = []
last_c = in_channels
for c in hl:
if c[0] == 'd':
dropout = True
c = c[1:]
else:
dropout = False
c = int(c)
i += 1
conv = GraphConv(last_c, c, dropout=dropout)
self.add_module('conv{}'.format(i), conv)
layers.append(conv)
last_c = c
conv = GraphConv(last_c, out_channels, relu=False, dropout=dropout_last)
self.add_module('conv-last', conv)
layers.append(conv)
self.layers = layers
def forward(self, x):
if self.training:
for conv in self.layers:
x = conv(x, self.train_adj)
else:
for conv in self.layers:
x = conv(x, self.adj)
return F.normalize(x)
### GCNII
class GraphConvolution(nn.Module):
def __init__(self, in_features, out_features, dropout=False, relu=True, residual=False, variant=False):
super(GraphConvolution, self).__init__()
self.variant = variant
if self.variant:
self.in_features = 2*in_features
else:
self.in_features = in_features
if dropout:
self.dropout = nn.Dropout(p=0.5)
else:
self.dropout = None
if relu:
self.relu = nn.ReLU()
else:
self.relu = None
self.out_features = out_features
self.residual = residual
self.layer = nn.Linear(self.in_features, self.out_features, bias = False)
def reset_parameters(self):
stdv = 1. / math.sqrt(self.out_features)
self.weight.data.uniform_(-stdv, stdv)
def forward(self, input, adj , h0 , lamda, alpha, l):
if self.dropout is not None:
input = self.dropout(input)
theta = math.log(lamda/l+1)
hi = torch.spmm(adj, input)
if self.variant:
support = torch.cat([hi,h0],1)
r = (1-alpha)*hi+alpha*h0
else:
support = (1-alpha)*hi+alpha*h0
r = support
mm_term = torch.mm(support, self.layer.weight.T)
output = theta*mm_term+(1-theta)*r
if self.residual:
output = output+input
if self.relu is not None:
output = self.relu(output)
return output
class GCNII(nn.Module):
def __init__(self, adj, in_channels , out_channels, hidden_dim, hidden_layers, lamda, alpha, variant, dropout = True):
super(GCNII, self).__init__()
self.alpha = alpha
self.lamda = lamda
adj = normt_spm(adj, method='in')
adj = spm_to_tensor(adj)
self.adj = adj.to(device)
i = 0
layers = nn.ModuleList()
self.fc_dim = nn.Linear(in_channels, hidden_dim)
self.relu = nn.ReLU()
self.dropout = nn.Dropout()
for i, c in enumerate(range(hidden_layers)):
conv = GraphConvolution(hidden_dim, hidden_dim, variant=variant, dropout=dropout)
layers.append(conv)
self.layers = layers
self.fc_out = nn.Linear(hidden_dim, out_channels)
def forward(self, x):
_layers = []
layer_inner = self.relu(self.fc_dim(self.dropout(x)))
# layer_inner = x
_layers.append(layer_inner)
for i,con in enumerate(self.layers):
layer_inner = con(layer_inner,self.adj,_layers[0],self.lamda,self.alpha,i+1)
layer_inner = self.fc_out(self.dropout(layer_inner))
return layer_inner
================================================
FILE: czsl/models/graph_method.py
================================================
import torch
import torch.nn as nn
import torch.nn.functional as F
import numpy as np
from .common import MLP, cross_domain_triplet_loss
from .gcn import GCN, GCNII
device = 'cuda' if torch.cuda.is_available() else 'cpu'
class GraphFull(nn.Module):
def __init__(self, dset, args):
super(GraphFull, self).__init__()
self.args = args
self.dset = dset
self.val_forward = self.val_forward_dotpr
self.train_forward = self.train_forward_normal
self.add_bt = args.add_bt
# Image Embedder
self.num_attrs, self.num_objs, self.num_pairs = len(dset.attrs), len(dset.objs), len(dset.pairs)
if self.args.train_only:
train_idx = []
for current in dset.train_pairs:
train_idx.append(dset.all_pair2idx[current]+self.num_attrs+self.num_objs)
self.train_idx = torch.LongTensor(train_idx).to(device)
if self.args.fc_emb.__contains__(','):
self.args.fc_emb = self.args.fc_emb.split(',')
layers = []
for a in self.args.fc_emb:
a = int(a)
layers.append(a)
if args.nlayers:
self.image_embedder = MLP(dset.feat_dim, args.emb_dim, num_layers= args.nlayers, dropout = self.args.dropout,
norm = self.args.norm, layers = layers, relu = True)
path = args.graph_init
graph = torch.load(path)
self.embeddings = graph['embeddings'].to(device)
adj = graph['adj']
hidden_layers = self.args.gr_emb
if args.gcn_type == 'gcn':
self.gcn = GCN(adj, self.embeddings.shape[1], args.emb_dim, hidden_layers)
else:
self.gcn = GCNII(adj, self.embeddings.shape[1], args.emb_dim, args.hidden_dim, args.gcn_nlayers, lamda = 0.5, alpha = 0.1, variant = False)
if args.static_inp:
for param in self.parameters():
param.requires_grad = False
def train_forward_normal(self, x):
img, _, _, pairs = x[0], x[1], x[2], x[3]
if self.args.nlayers:
img_feats = self.image_embedder(img)
else:
img_feats = (img)
current_embeddings = self.gcn(self.embeddings)
if self.args.train_only:
pair_embed = current_embeddings[self.train_idx]
else:
pair_embed = current_embeddings[self.num_attrs+self.num_objs:self.num_attrs+self.num_objs+self.num_pairs,:]
pair_embed = pair_embed.permute(1,0)
pair_pred = torch.matmul(img_feats, pair_embed)
if self.add_bt >= 1:
pairs = torch.cat([pairs, pairs], dim=0)
loss = F.cross_entropy(pair_pred, pairs)
return loss, None
def val_forward_dotpr(self, x):
img = x[0]
if self.args.nlayers:
img_feats = self.image_embedder(img)
else:
img_feats = (img)
current_embedddings = self.gcn(self.embeddings)
pair_embeds = current_embedddings[self.num_attrs+self.num_objs:self.num_attrs+self.num_objs+self.num_pairs,:].permute(1,0)
score = torch.matmul(img_feats, pair_embeds)
scores = {}
for itr, pair in enumerate(self.dset.pairs):
scores[pair] = score[:,self.dset.all_pair2idx[pair]]
return None, scores
def val_forward_distance_fast(self, x):
img = x[0]
img_feats = (self.image_embedder(img))
current_embeddings = self.gcn(self.embeddings)
pair_embeds = current_embeddings[self.num_attrs+self.num_objs:,:]
batch_size, pairs, features = img_feats.shape[0], pair_embeds.shape[0], pair_embeds.shape[1]
img_feats = img_feats[:,None,:].expand(-1, pairs, -1)
pair_embeds = pair_embeds[None,:,:].expand(batch_size, -1, -1)
diff = (img_feats - pair_embeds)**2
score = diff.sum(2) * -1
scores = {}
for itr, pair in enumerate(self.dset.pairs):
scores[pair] = score[:,self.dset.all_pair2idx[pair]]
return None, scores
def forward(self, x):
if self.training:
loss, pred = self.train_forward(x)
else:
with torch.no_grad():
loss, pred = self.val_forward(x)
return loss, pred
================================================
FILE: czsl/models/image_extractor.py
================================================
import torch
import torch.nn as nn
import torch.nn.functional as F
from torchvision import models
from torchvision.models.resnet import ResNet, BasicBlock
class ResNet18_conv(ResNet):
def __init__(self):
super(ResNet18_conv, self).__init__(BasicBlock, [2, 2, 2, 2])
def forward(self, x):
# change forward here
x = self.conv1(x)
x = self.bn1(x)
x = self.relu(x)
x = self.maxpool(x)
x = self.layer1(x)
x = self.layer2(x)
x = self.layer3(x)
x = self.layer4(x)
return x
def get_image_extractor(arch = 'resnet18', pretrained = True, feature_dim = None, checkpoint = ''):
'''
Inputs
arch: Base architecture
pretrained: Bool, Imagenet weights
feature_dim: Int, output feature dimension
checkpoint: String, not implemented
Returns
Pytorch model
'''
if arch == 'resnet18':
model = models.resnet18(pretrained = pretrained)
if feature_dim is None:
model.fc = nn.Sequential()
else:
model.fc = nn.Linear(512, feature_dim)
if arch == 'resnet18_conv':
model = ResNet18_conv()
model.load_state_dict(models.resnet18(pretrained=True).state_dict())
elif arch == 'resnet50':
model = models.resnet50(pretrained = pretrained)
if feature_dim is None:
model.fc = nn.Sequential()
else:
model.fc = nn.Linear(2048, feature_dim)
elif arch == 'resnet50_cutmix':
model = models.resnet50(pretrained = pretrained)
checkpoint = torch.load('/home/ubuntu/workspace/pretrained/resnet50_cutmix.tar')
model.load_state_dict(checkpoint['state_dict'], strict=False)
if feature_dim is None:
model.fc = nn.Sequential()
else:
model.fc = nn.Linear(2048, feature_dim)
elif arch == 'resnet152':
model = models.resnet152(pretrained = pretrained)
if feature_dim is None:
model.fc = nn.Sequential()
else:
model.fc = nn.Linear(2048, feature_dim)
elif arch == 'vgg16':
model = models.vgg16(pretrained = pretrained)
modules = list(model.classifier.children())[:-3]
model.classifier=torch.nn.Sequential(*modules)
if feature_dim is not None:
model.classifier[3]=torch.nn.Linear(4096,feature_dim)
return model
================================================
FILE: czsl/models/manifold_methods.py
================================================
import torch
import torch.nn as nn
import torch.nn.functional as F
import numpy as np
from .word_embedding import load_word_embeddings
from .common import MLP, Reshape
from flags import DATA_FOLDER
device = 'cuda' if torch.cuda.is_available() else 'cpu'
class ManifoldModel(nn.Module):
def __init__(self, dset, args):
super(ManifoldModel, self).__init__()
self.args = args
self.dset = dset
def get_all_ids(relevant_pairs):
# Precompute validation pairs
attrs, objs = zip(*relevant_pairs)
attrs = [dset.attr2idx[attr] for attr in attrs]
objs = [dset.obj2idx[obj] for obj in objs]
pairs = [a for a in range(len(relevant_pairs))]
attrs = torch.LongTensor(attrs).to(device)
objs = torch.LongTensor(objs).to(device)
pairs = torch.LongTensor(pairs).to(device)
return attrs, objs, pairs
#Validation
self.val_attrs, self.val_objs, self.val_pairs = get_all_ids(self.dset.pairs)
# for indivual projections
self.uniq_attrs, self.uniq_objs = torch.arange(len(self.dset.attrs)).long().to(device), \
torch.arange(len(self.dset.objs)).long().to(device)
self.factor = 2
# Precompute training compositions
if args.train_only:
self.train_attrs, self.train_objs, self.train_pairs = get_all_ids(self.dset.train_pairs)
else:
self.train_attrs, self.train_objs = self.val_subs, self.val_attrs, self.val_objs
if args.lambda_aux > 0 or args.lambda_cls_attr > 0 or args.lambda_cls_obj > 0:
print('Initializing classifiers')
self.obj_clf = nn.Linear(args.emb_dim, len(dset.objs))
self.attr_clf = nn.Linear(args.emb_dim, len(dset.attrs))
def train_forward_bce(self, x):
img, attrs, objs = x[0], x[1], x[2]
neg_attrs, neg_objs = x[4][:,0],x[5][:,0] #todo: do a less hacky version
img_feat = self.image_embedder(img)
# Sample 25% positive and 75% negative pairs
labels = np.random.binomial(1, 0.25, attrs.shape[0])
labels = torch.from_numpy(labels).bool().to(device)
sampled_attrs, sampled_objs = neg_attrs.clone(), neg_objs.clone()
sampled_attrs[labels] = attrs[labels]
sampled_objs[labels] = objs[labels]
labels = labels.float()
composed_clf = self.compose(attrs, objs)
p = torch.sigmoid((img_feat*composed_clf).sum(1))
loss = F.binary_cross_entropy(p, labels)
return loss, None
def train_forward_triplet(self, x):
img, attrs, objs = x[0], x[1], x[2]
neg_attrs, neg_objs = x[4][:,0], x[5][:,0] #todo:do a less hacky version
img_feats = self.image_embedder(img)
positive = self.compose(attrs, objs)
negative = self.compose(neg_attrs, neg_objs)
loss = F.triplet_margin_loss(img_feats, positive, negative, margin = self.args.margin)
# Auxiliary object/ attribute prediction loss both need to be correct
if self.args.lambda_aux > 0:
obj_pred = self.obj_clf(positive)
attr_pred = self.attr_clf(positive)
loss_aux = F.cross_entropy(attr_pred, attrs) + F.cross_entropy(obj_pred, objs)
loss += self.args.lambda_aux * loss_aux
return loss, None
def val_forward_distance(self, x):
img = x[0]
batch_size = img.shape[0]
img_feats = self.image_embedder(img)
scores = {}
pair_embeds = self.compose(self.val_attrs, self.val_objs)
for itr, pair in enumerate(self.dset.pairs):
pair_embed = pair_embeds[itr, None].expand(batch_size, pair_embeds.size(1))
score = self.compare_metric(img_feats, pair_embed)
scores[pair] = score
return None, scores
def val_forward_distance_fast(self, x):
img = x[0]
batch_size = img.shape[0]
img_feats = self.image_embedder(img)
pair_embeds = self.compose(self.val_attrs, self.val_objs) # Evaluate all pairs
batch_size, pairs, features = img_feats.shape[0], pair_embeds.shape[0], pair_embeds.shape[1]
img_feats = img_feats[:,None,:].expand(-1, pairs, -1)
pair_embeds = pair_embeds[None,:,:].expand(batch_size, -1, -1)
diff = (img_feats - pair_embeds)**2
score = diff.sum(2) * -1
scores = {}
for itr, pair in enumerate(self.dset.pairs):
scores[pair] = score[:,self.dset.all_pair2idx[pair]]
return None, scores
def val_forward_direct(self, x):
img = x[0]
batch_size = img.shape[0]
img_feats = self.image_embedder(img)
pair_embeds = self.compose(self.val_attrs, self.val_objs).permute(1,0) # Evaluate all pairs
score = torch.matmul(img_feats, pair_embeds)
scores = {}
for itr, pair in enumerate(self.dset.pairs):
scores[pair] = score[:,self.dset.all_pair2idx[pair]]
return None, scores
def forward(self, x):
if self.training:
loss, pred = self.train_forward(x)
else:
with torch.no_grad():
loss, pred = self.val_forward(x)
return loss, pred
#########################################
class RedWine(ManifoldModel):
def __init__(self, dset, args):
super(RedWine, self).__init__(dset, args)
self.image_embedder = lambda img: img
self.compare_metric = lambda img_feats, pair_embed: torch.sigmoid((img_feats*pair_embed).sum(1))
self.train_forward = self.train_forward_bce
self.val_forward = self.val_forward_distance
in_dim = dset.feat_dim if not args.glove_init else 300
self.T = nn.Sequential(
nn.Linear(2*in_dim, 3*in_dim),
nn.LeakyReLU(0.1, True),
nn.Linear(3*in_dim, 3*in_dim//2),
nn.LeakyReLU(0.1, True),
nn.Linear(3*in_dim//2, dset.feat_dim))
self.attr_embedder = nn.Embedding(len(dset.attrs), in_dim)
self.obj_embedder = nn.Embedding(len(dset.objs), in_dim)
# initialize the weights of the embedders with the svm weights
if args.glove_init:
pretrained_weight = load_word_embeddings(DATA_FOLDER+'/glove/glove.6B.300d.txt', dset.attrs)
self.attr_embedder.weight.data.copy_(pretrained_weight)
pretrained_weight = load_word_embeddings(DATA_FOLDER+'/glove/glove.6B.300d.txt', dset.objs)
self.obj_embedder.weight.data.copy_(pretrained_weight)
elif args.clf_init:
for idx, attr in enumerate(dset.attrs):
at_id = self.dset.attr2idx[attr]
weight = torch.load('%s/svm/attr_%d'%(args.data_dir, at_id)).coef_.squeeze()
self.attr_embedder.weight[idx].data.copy_(torch.from_numpy(weight))
for idx, obj in enumerate(dset.objs):
obj_id = self.dset.obj2idx[obj]
weight = torch.load('%s/svm/obj_%d'%(args.data_dir, obj_id)).coef_.squeeze()
self.obj_embedder.weight[idx].data.copy_(torch.from_numpy(weight))
else:
print ('init must be either glove or clf')
return
if args.static_inp:
for param in self.attr_embedder.parameters():
param.requires_grad = False
for param in self.obj_embedder.parameters():
param.requires_grad = False
def compose(self, attrs, objs):
attr_wt = self.attr_embedder(attrs)
obj_wt = self.obj_embedder(objs)
inp_wts = torch.cat([attr_wt, obj_wt], 1) # 2D
composed_clf = self.T(inp_wts)
return composed_clf
class LabelEmbedPlus(ManifoldModel):
def __init__(self, dset, args):
super(LabelEmbedPlus, self).__init__(dset, args)
if 'conv' in args.image_extractor:
self.image_embedder = torch.nn.Sequential(torch.nn.Conv2d(dset.feat_dim,args.emb_dim,7),
torch.nn.ReLU(True),
Reshape(-1,args.emb_dim)
)
else:
self.image_embedder = MLP(dset.feat_dim, args.emb_dim)
self.compare_metric = lambda img_feats, pair_embed: -F.pairwise_distance(img_feats, pair_embed)
self.train_forward = self.train_forward_triplet
self.val_forward = self.val_forward_distance_fast
input_dim = dset.feat_dim if args.clf_init else args.emb_dim
self.attr_embedder = nn.Embedding(len(dset.attrs), input_dim)
self.obj_embedder = nn.Embedding(len(dset.objs), input_dim)
self.T = MLP(2*input_dim, args.emb_dim, num_layers= args.nlayers)
# init with word embeddings
if args.emb_init:
pretrained_weight = load_word_embeddings(args.emb_init, dset.attrs)
self.attr_embedder.weight.data.copy_(pretrained_weight)
pretrained_weight = load_word_embeddings(args.emb_init, dset.objs)
self.obj_embedder.weight.data.copy_(pretrained_weight)
# init with classifier weights
elif args.clf_init:
for idx, attr in enumerate(dset.attrs):
at_id = dset.attrs.index(attr)
weight = torch.load('%s/svm/attr_%d'%(args.data_dir, at_id)).coef_.squeeze()
self.attr_embedder.weight[idx].data.copy_(torch.from_numpy(weight))
for idx, obj in enumerate(dset.objs):
obj_id = dset.objs.index(obj)
weight = torch.load('%s/svm/obj_%d'%(args.data_dir, obj_id)).coef_.squeeze()
self.obj_emb.weight[idx].data.copy_(torch.from_numpy(weight))
# static inputs
if args.static_inp:
for param in self.attr_embedder.parameters():
param.requires_grad = False
for param in self.obj_embedder.parameters():
param.requires_grad = False
def compose(self, attrs, objs):
inputs = [self.attr_embedder(attrs), self.obj_embedder(objs)]
inputs = torch.cat(inputs, 1)
output = self.T(inputs)
return output
class AttributeOperator(ManifoldModel):
def __init__(self, dset, args):
super(AttributeOperator, self).__init__(dset, args)
self.image_embedder = MLP(dset.feat_dim, args.emb_dim)
self.compare_metric = lambda img_feats, pair_embed: -F.pairwise_distance(img_feats, pair_embed)
self.val_forward = self.val_forward_distance_fast
self.attr_ops = nn.ParameterList([nn.Parameter(torch.eye(args.emb_dim)) for _ in range(len(self.dset.attrs))])
self.obj_embedder = nn.Embedding(len(dset.objs), args.emb_dim)
if args.emb_init:
pretrained_weight = load_word_embeddings('glove', dset.objs)
self.obj_embedder.weight.data.copy_(pretrained_weight)
self.inverse_cache = {}
if args.lambda_ant>0 and args.dataset=='mitstates':
antonym_list = open(DATA_FOLDER+'/data/antonyms.txt').read().strip().split('\n')
antonym_list = [l.split() for l in antonym_list]
antonym_list = [[self.dset.attrs.index(a1), self.dset.attrs.index(a2)] for a1, a2 in antonym_list]
antonyms = {}
antonyms.update({a1:a2 for a1, a2 in antonym_list})
antonyms.update({a2:a1 for a1, a2 in antonym_list})
self.antonyms, self.antonym_list = antonyms, antonym_list
if args.static_inp:
for param in self.obj_embedder.parameters():
param.requires_grad = False
def apply_ops(self, ops, rep):
out = torch.bmm(ops, rep.unsqueeze(2)).squeeze(2)
out = F.relu(out)
return out
def compose(self, attrs, objs):
obj_rep = self.obj_embedder(objs)
attr_ops = torch.stack([self.attr_ops[attr.item()] for attr in attrs])
embedded_reps = self.apply_ops(attr_ops, obj_rep)
return embedded_reps
def apply_inverse(self, img_rep, attrs):
inverse_ops = []
for i in range(img_rep.size(0)):
attr = attrs[i]
if attr not in self.inverse_cache:
self.inverse_cache[attr] = self.attr_ops[attr].inverse()
inverse_ops.append(self.inverse_cache[attr])
inverse_ops = torch.stack(inverse_ops) # (B,512,512)
obj_rep = self.apply_ops(inverse_ops, img_rep)
return obj_rep
def train_forward(self, x):
img, attrs, objs = x[0], x[1], x[2]
neg_attrs, neg_objs, inv_attrs, comm_attrs = x[4][:,0], x[5][:,0], x[6], x[7]
batch_size = img.size(0)
loss = []
anchor = self.image_embedder(img)
obj_emb = self.obj_embedder(objs)
pos_ops = torch.stack([self.attr_ops[attr.item()] for attr in attrs])
positive = self.apply_ops(pos_ops, obj_emb)
neg_obj_emb = self.obj_embedder(neg_objs)
neg_ops = torch.stack([self.attr_ops[attr.item()] for attr in neg_attrs])
negative = self.apply_ops(neg_ops, neg_obj_emb)
loss_triplet = F.triplet_margin_loss(anchor, positive, negative, margin=self.args.margin)
loss.append(loss_triplet)
# Auxiliary object/attribute loss
if self.args.lambda_aux>0:
obj_pred = self.obj_clf(positive)
attr_pred = self.attr_clf(positive)
loss_aux = F.cross_entropy(attr_pred, attrs) + F.cross_entropy(obj_pred, objs)
loss.append(self.args.lambda_aux*loss_aux)
# Inverse Consistency
if self.args.lambda_inv>0:
obj_rep = self.apply_inverse(anchor, attrs)
new_ops = torch.stack([self.attr_ops[attr.item()] for attr in inv_attrs])
new_rep = self.apply_ops(new_ops, obj_rep)
new_positive = self.apply_ops(new_ops, obj_emb)
loss_inv = F.triplet_margin_loss(new_rep, new_positive, positive, margin=self.args.margin)
loss.append(self.args.lambda_inv*loss_inv)
# Commutative Operators
if self.args.lambda_comm>0:
B = torch.stack([self.attr_ops[attr.item()] for attr in comm_attrs])
BA = self.apply_ops(B, positive)
AB = self.apply_ops(pos_ops, self.apply_ops(B, obj_emb))
loss_comm = ((AB-BA)**2).sum(1).mean()
loss.append(self.args.lambda_comm*loss_comm)
# Antonym Consistency
if self.args.lambda_ant>0:
select_idx = [i for i in range(batch_size) if attrs[i].item() in self.antonyms]
if len(select_idx)>0:
select_idx = torch.LongTensor(select_idx).cuda()
attr_subset = attrs[select_idx]
antonym_ops = torch.stack([self.attr_ops[self.antonyms[attr.item()]] for attr in attr_subset])
Ao = anchor[select_idx]
if self.args.lambda_inv>0:
o = obj_rep[select_idx]
else:
o = self.apply_inverse(Ao, attr_subset)
BAo = self.apply_ops(antonym_ops, Ao)
loss_cycle = ((BAo-o)**2).sum(1).mean()
loss.append(self.args.lambda_ant*loss_cycle)
loss = sum(loss)
return loss, None
def forward(self, x):
loss, pred = super(AttributeOperator, self).forward(x)
self.inverse_cache = {}
return loss, pred
================================================
FILE: czsl/models/modular_methods.py
================================================
# Copyright (c) Facebook, Inc. and its affiliates.
# All rights reserved.
#
# This source code is licensed under the license found in the
# LICENSE file in the root directory of this source tree.
#
import torch
import torch.nn as nn
import torch.nn.functional as F
import torchvision.models as tmodels
import numpy as np
from .word_embedding import load_word_embeddings
import itertools
import math
import collections
from torch.distributions.bernoulli import Bernoulli
import pdb
import sys
class Flatten(nn.Module):
def forward(self, input):
return input.view(input.size(0), -1)
class GatingSampler(nn.Module):
"""Docstring for GatingSampler. """
def __init__(self, gater, stoch_sample=True, temperature=1.0):
"""TODO: to be defined1.
"""
nn.Module.__init__(self)
self.gater = gater
self._stoch_sample = stoch_sample
self._temperature = temperature
def disable_stochastic_sampling(self):
self._stoch_sample = False
def enable_stochastic_sampling(self):
self._stoch_sample = True
def forward(self, tdesc=None, return_additional=False, gating_wt=None):
if self.gater is None and return_additional:
return None, None
elif self.gater is None:
return None
if gating_wt is not None:
return_wts = gating_wt
gating_g = self.gater(tdesc, gating_wt=gating_wt)
else:
gating_g = self.gater(tdesc)
return_wts = None
if isinstance(gating_g, tuple):
return_wts = gating_g[1]
gating_g = gating_g[0]
if not self._stoch_sample:
sampled_g = gating_g
else:
raise (NotImplementedError)
if return_additional:
return sampled_g, return_wts
return sampled_g
class GatedModularNet(nn.Module):
"""
An interface for creating modular nets
"""
def __init__(self,
module_list,
start_modules=None,
end_modules=None,
single_head=False,
chain=False):
"""TODO: to be defined1.
:module_list: TODO
:g: TODO
"""
nn.Module.__init__(self)
self._module_list = nn.ModuleList(
[nn.ModuleList(m) for m in module_list])
self.num_layers = len(self._module_list)
if start_modules is not None:
self._start_modules = nn.ModuleList(start_modules)
else:
self._start_modules = None
if end_modules is not None:
self._end_modules = nn.ModuleList(end_modules)
self.num_layers += 1
else:
self._end_modules = None
self.sampled_g = None
self.single_head = single_head
self._chain = chain
def forward(self, x, sampled_g=None, t=None, return_feat=False):
"""TODO: Docstring for forward.
:x: Input data
:g: Gating tensor (#Task x )#num_layer x #num_mods x #num_mods
:t: task ID
:returns: TODO
"""
if t is None:
t_not_set = True
t = torch.tensor([0] * x.shape[0], dtype=x.dtype).long()
else:
t_not_set = False
t = t.squeeze()
if self._start_modules is not None:
prev_out = [mod(x) for mod in self._start_modules]
else:
prev_out = [x]
if sampled_g is None:
# NON-Gated Module network
prev_out = sum(prev_out) / float(len(prev_out))
#prev_out = torch.mean(prev_out, 0)
for li in range(len(self._module_list)):
prev_out = sum([
mod(prev_out) for mod in self._module_list[li]
]) / float(len(self._module_list[li]))
features = prev_out
if self._end_modules is not None:
if t_not_set or self.single_head:
prev_out = self._end_modules[0](prev_out)
else:
prev_out = torch.cat([
self._end_modules[tid](prev_out[bi:bi + 1])
for bi, tid in enumerate(t)
], 0)
if return_feat:
return prev_out, features
return prev_out
else:
# Forward prop with sampled Gs
for li in range(len(self._module_list)):
curr_out = []
for j in range(len(self._module_list[li])):
gind = j if not self._chain else 0
# Dim: #Batch x C
module_in_wt = sampled_g[li + 1][gind]
# Module input weights rearranged to match inputs
module_in_wt = module_in_wt.transpose(0, 1)
add_dims = prev_out[0].dim() + 1 - module_in_wt.dim()
module_in_wt = module_in_wt.view(*module_in_wt.shape,
*([1] * add_dims))
module_in_wt = module_in_wt.expand(
len(prev_out), *prev_out[0].shape)
module_in = sum([
module_in_wt[i] * prev_out[i]
for i in range(len(prev_out))
])
mod = self._module_list[li][j]
curr_out.append(mod(module_in))
prev_out = curr_out
# Output modules (with sampled Gs)
if self._end_modules is not None:
li = self.num_layers - 1
if t_not_set or self.single_head:
# Dim: #Batch x C
module_in_wt = sampled_g[li + 1][0]
# Module input weights rearranged to match inputs
module_in_wt = module_in_wt.transpose(0, 1)
add_dims = prev_out[0].dim() + 1 - module_in_wt.dim()
module_in_wt = module_in_wt.view(*module_in_wt.shape,
*([1] * add_dims))
module_in_wt = module_in_wt.expand(
len(prev_out), *prev_out[0].shape)
module_in = sum([
module_in_wt[i] * prev_out[i]
for i in range(len(prev_out))
])
features = module_in
prev_out = self._end_modules[0](module_in)
else:
curr_out = []
for bi, tid in enumerate(t):
# Dim: #Batch x C
gind = tid if not self._chain else 0
module_in_wt = sampled_g[li + 1][gind]
# Module input weights rearranged to match inputs
module_in_wt = module_in_wt.transpose(0, 1)
add_dims = prev_out[0].dim() + 1 - module_in_wt.dim()
module_in_wt = module_in_wt.view(
*module_in_wt.shape, *([1] * add_dims))
module_in_wt = module_in_wt.expand(
len(prev_out), *prev_out[0].shape)
module_in = sum([
module_in_wt[i] * prev_out[i]
for i in range(len(prev_out))
])
features = module_in
mod = self._end_modules[tid]
curr_out.append(mod(module_in[bi:bi + 1]))
prev_out = curr_out
prev_out = torch.cat(prev_out, 0)
if return_feat:
return prev_out, features
return prev_out
class CompositionalModel(nn.Module):
def __init__(self, dset, args):
super(CompositionalModel, self).__init__()
self.args = args
self.dset = dset
# precompute validation pairs
attrs, objs = zip(*self.dset.pairs)
attrs = [dset.attr2idx[attr] for attr in attrs]
objs = [dset.obj2idx[obj] for obj in objs]
self.val_attrs = torch.LongTensor(attrs).cuda()
self.val_objs = torch.LongTensor(objs).cuda()
def train_forward_softmax(self, x):
img, attrs, objs = x[0], x[1], x[2]
neg_attrs, neg_objs = x[4], x[5]
inv_attrs, comm_attrs = x[6], x[7]
sampled_attrs = torch.cat((attrs.unsqueeze(1), neg_attrs), 1)
sampled_objs = torch.cat((objs.unsqueeze(1), neg_objs), 1)
img_ind = torch.arange(sampled_objs.shape[0]).unsqueeze(1).repeat(
1, sampled_attrs.shape[1])
flat_sampled_attrs = sampled_attrs.view(-1)
flat_sampled_objs = sampled_objs.view(-1)
flat_img_ind = img_ind.view(-1)
labels = torch.zeros_like(sampled_attrs[:, 0]).long()
self.composed_g = self.compose(flat_sampled_attrs, flat_sampled_objs)
cls_scores, feat = self.comp_network(
img[flat_img_ind], self.composed_g, return_feat=True)
pair_scores = cls_scores[:, :1]
pair_scores = pair_scores.view(*sampled_attrs.shape)
loss = 0
loss_cls = F.cross_entropy(pair_scores, labels)
loss += loss_cls
loss_obj = torch.FloatTensor([0])
loss_attr = torch.FloatTensor([0])
loss_sparse = torch.FloatTensor([0])
loss_unif = torch.FloatTensor([0])
loss_aux = torch.FloatTensor([0])
acc = (pair_scores.argmax(1) == labels).sum().float() / float(
len(labels))
all_losses = {}
all_losses['total_loss'] = loss
all_losses['main_loss'] = loss_cls
all_losses['aux_loss'] = loss_aux
all_losses['obj_loss'] = loss_obj
all_losses['attr_loss'] = loss_attr
all_losses['sparse_loss'] = loss_sparse
all_losses['unif_loss'] = loss_unif
return loss, all_losses, acc, (pair_scores, feat)
def val_forward(self, x):
img = x[0]
batch_size = img.shape[0]
pair_scores = torch.zeros(batch_size, len(self.val_attrs))
pair_feats = torch.zeros(batch_size, len(self.val_attrs),
self.args.emb_dim)
pair_bs = len(self.val_attrs)
for pi in range(math.ceil(len(self.val_attrs) / pair_bs)):
self.compose_g = self.compose(
self.val_attrs[pi * pair_bs:(pi + 1) * pair_bs],
self.val_objs[pi * pair_bs:(pi + 1) * pair_bs])
compose_g = self.compose_g
expanded_im = img.unsqueeze(1).repeat(
1, compose_g[0][0].shape[0],
*tuple([1] * (img.dim() - 1))).view(-1, *img.shape[1:])
expanded_compose_g = [[
g.unsqueeze(0).repeat(batch_size, *tuple([1] * g.dim())).view(
-1, *g.shape[1:]) for g in layer_g
] for layer_g in compose_g]
this_pair_scores, this_feat = self.comp_network(
expanded_im, expanded_compose_g, return_feat=True)
featnorm = torch.norm(this_feat, p=2, dim=-1)
this_feat = this_feat.div(
featnorm.unsqueeze(-1).expand_as(this_feat))
this_pair_scores = this_pair_scores[:, :1].view(batch_size, -1)
this_feat = this_feat.view(batch_size, -1, self.args.emb_dim)
pair_scores[:, pi * pair_bs:pi * pair_bs +
this_pair_scores.shape[1]] = this_pair_scores[:, :]
pair_feats[:, pi * pair_bs:pi * pair_bs +
this_pair_scores.shape[1], :] = this_feat[:]
scores = {}
feats = {}
for i, (attr, obj) in enumerate(self.dset.pairs):
scores[(attr, obj)] = pair_scores[:, i]
feats[(attr, obj)] = pair_feats[:, i]
# return None, (scores, feats)
return None, scores
def forward(self, x, with_grad=False):
if self.training:
loss, loss_aux, acc, pred = self.train_forward(x)
else:
loss_aux = torch.Tensor([0])
loss = torch.Tensor([0])
if not with_grad:
with torch.no_grad():
acc, pred = self.val_forward(x)
else:
acc, pred = self.val_forward(x)
# return loss, loss_aux, acc, pred
return loss, pred
class GatedGeneralNN(CompositionalModel):
"""Docstring for GatedCompositionalModel. """
def __init__(self,
dset,
args,
num_layers=2,
num_modules_per_layer=3,
stoch_sample=False,
use_full_model=False,
num_classes=[2],
gater_type='general'):
"""TODO: to be defined1.
:dset: TODO
:args: TODO
"""
CompositionalModel.__init__(self, dset, args)
self.train_forward = self.train_forward_softmax
self.compose_type = 'nn' #todo: could be different
gating_in_dim = 128
if args.emb_init:
gating_in_dim = 300
elif args.clf_init:
gating_in_dim = 512
if self.compose_type == 'nn':
tdim = gating_in_dim * 2
inter_tdim = self.args.embed_rank
# Change this to allow only obj, only attr gatings
self.attr_embedder = nn.Embedding(
len(dset.attrs) + 1,
gating_in_dim,
padding_idx=len(dset.attrs),
)
self.obj_embedder = nn.Embedding(
len(dset.objs) + 1,
gating_in_dim,
padding_idx=len(dset.objs),
)
# initialize the weights of the embedders with the svm weights
if args.emb_init:
pretrained_weight = load_word_embeddings(
args.emb_init, dset.attrs)
self.attr_embedder.weight[:-1, :].data.copy_(pretrained_weight)
pretrained_weight = load_word_embeddings(
args.emb_init, dset.objs)
self.obj_embedder.weight.data[:-1, :].copy_(pretrained_weight)
elif args.clf_init:
for idx, attr in enumerate(dset.attrs):
at_id = self.dset.attr2idx[attr]
weight = torch.load(
'%s/svm/attr_%d' % (args.data_dir,
at_id)).coef_.squeeze()
self.attr_embedder.weight[idx].data.copy_(
torch.from_numpy(weight))
for idx, obj in enumerate(dset.objs):
obj_id = self.dset.obj2idx[obj]
weight = torch.load(
'%s/svm/obj_%d' % (args.data_dir,
obj_id)).coef_.squeeze()
self.obj_embedder.weight[idx].data.copy_(
torch.from_numpy(weight))
else:
n_attr = len(dset.attrs)
gating_in_dim = 300
tdim = gating_in_dim * 2 + n_attr
self.attr_embedder = nn.Embedding(
n_attr,
n_attr,
)
self.attr_embedder.weight.data.copy_(
torch.from_numpy(np.eye(n_attr)))
self.obj_embedder = nn.Embedding(
len(dset.objs) + 1,
gating_in_dim,
padding_idx=len(dset.objs),
)
pretrained_weight = load_word_embeddings(
'/home/ubuntu/workspace/czsl/data/glove/glove.6B.300d.txt', dset.objs)
self.obj_embedder.weight.data[:-1, :].copy_(pretrained_weight)
else:
raise (NotImplementedError)
self.comp_network, self.gating_network, self.nummods, _ = modular_general(
num_layers=num_layers,
num_modules_per_layer=num_modules_per_layer,
feat_dim=dset.feat_dim,
inter_dim=args.emb_dim,
stoch_sample=stoch_sample,
use_full_model=use_full_model,
tdim=tdim,
inter_tdim=inter_tdim,
gater_type=gater_type,
)
if args.static_inp:
for param in self.attr_embedder.parameters():
param.requires_grad = False
for param in self.obj_embedder.parameters():
param.requires_grad = False
def compose(self, attrs, objs):
obj_wt = self.obj_embedder(objs)
if self.compose_type == 'nn':
attr_wt = self.attr_embedder(attrs)
inp_wts = torch.cat([attr_wt, obj_wt], 1) # 2D
else:
raise (NotImplementedError)
composed_g, composed_g_wt = self.gating_network(
inp_wts, return_additional=True)
return composed_g
class GeneralNormalizedNN(nn.Module):
"""Docstring for GatedCompositionalModel. """
def __init__(self, num_layers, num_modules_per_layer, in_dim, inter_dim):
"""TODO: to be defined1. """
nn.Module.__init__(self)
self.start_modules = [nn.Sequential()]
self.layer1 = [[
nn.Sequential(
nn.Linear(in_dim, inter_dim), nn.BatchNorm1d(inter_dim),
nn.ReLU()) for _ in range(num_modules_per_layer)
]]
if num_layers > 1:
self.layer2 = [[
nn.Sequential(
nn.BatchNorm1d(inter_dim), nn.Linear(inter_dim, inter_dim),
nn.BatchNorm1d(inter_dim), nn.ReLU())
for _m in range(num_modules_per_layer)
] for _l in range(num_layers - 1)]
self.avgpool = nn.Sequential()
self.fc = [
nn.Sequential(nn.BatchNorm1d(inter_dim), nn.Linear(inter_dim, 1))
]
class GeneralGatingNN(nn.Module):
def __init__(
self,
num_mods,
tdim,
inter_tdim,
randinit=False,
):
"""TODO: to be defined1.
:num_mods: TODO
:tdim: TODO
"""
nn.Module.__init__(self)
self._num_mods = num_mods
self._tdim = tdim
self._inter_tdim = inter_tdim
task_outdim = self._inter_tdim
self.task_linear1 = nn.Linear(self._tdim, task_outdim, bias=False)
self.task_bn1 = nn.BatchNorm1d(task_outdim)
self.task_linear2 = nn.Linear(task_outdim, task_outdim, bias=False)
self.task_bn2 = nn.BatchNorm1d(task_outdim)
self.joint_linear1 = nn.Linear(task_outdim, task_outdim, bias=False)
self.joint_bn1 = nn.BatchNorm1d(task_outdim)
num_out = [[1]] + [[
self._num_mods[i - 1] for _ in range(self._num_mods[i])
] for i in range(1, len(self._num_mods))]
count = 0
out_ind = []
for i in range(len(num_out)):
this_out_ind = []
for j in range(len(num_out[i])):
this_out_ind.append([count, count + num_out[i][j]])
count += num_out[i][j]
out_ind.append(this_out_ind)
self.out_ind = out_ind
self.out_count = count
self.joint_linear2 = nn.Linear(task_outdim, count, bias=False)
def apply_init(m):
if isinstance(m, nn.Linear):
nn.init.normal_(m.weight, 0, 0.1)
if m.bias is not None:
nn.init.constant_(m.bias, 0)
elif isinstance(m, nn.BatchNorm1d):
nn.init.constant_(m.weight, 1)
nn.init.constant_(m.bias, 0)
for m in self.modules():
if isinstance(m, nn.ModuleList):
for subm in m:
if isinstance(subm, nn.ModuleList):
for subsubm in subm:
apply_init(subsubm)
else:
apply_init(subm)
else:
apply_init(m)
if not randinit:
self.joint_linear2.weight.data.zero_()
def forward(self, tdesc=None):
"""TODO: Docstring for function.
:arg1: TODO
:returns: TODO
"""
if tdesc is None:
return None
x = tdesc
task_embeds1 = F.relu(self.task_bn1(self.task_linear1(x)))
joint_embed = task_embeds1
joint_embed = self.joint_linear2(joint_embed)
joint_embed = [[
joint_embed[:, self.out_ind[i][j][0]:self.out_ind[i][j][1]]
for j in range(len(self.out_ind[i]))
] for i in range(len(self.out_ind))]
gating_wt = joint_embed
prob_g = [[F.softmax(wt, -1) for wt in gating_wt[i]]
for i in range(len(gating_wt))]
return prob_g, gating_wt
def modularize_network(
model,
stoch_sample=False,
use_full_model=False,
tdim=200,
inter_tdim=200,
gater_type='general',
single_head=True,
num_classes=[2],
num_lookup_gating=10,
):
# Copy start modules and end modules
start_modules = model.start_modules
end_modules = [
nn.Sequential(model.avgpool, Flatten(), fci) for fci in model.fc
]
# Create module_list as list of lists [[layer1 modules], [layer2 modules], ...]
module_list = []
li = 1
while True:
if hasattr(model, 'layer{}'.format(li)):
module_list.extend(getattr(model, 'layer{}'.format(li)))
li += 1
else:
break
num_module_list = [len(start_modules)] + [len(layer) for layer in module_list] \
+ [len(end_modules)]
gated_model_func = GatedModularNet
gated_net = gated_model_func(
module_list,
start_modules=start_modules,
end_modules=end_modules,
single_head=single_head)
gater_func = GeneralGatingNN
gater = gater_func(
num_mods=num_module_list, tdim=tdim, inter_tdim=inter_tdim)
fan_in = num_module_list
if use_full_model:
gater = None
# Create Gating Sampler
gating_sampler = GatingSampler(gater=gater, stoch_sample=stoch_sample)
return gated_net, gating_sampler, num_module_list, fan_in
def modular_general(
num_layers,
num_modules_per_layer,
feat_dim,
inter_dim,
stoch_sample=False,
use_full_model=False,
num_classes=[2],
single_head=True,
gater_type='general',
tdim=300,
inter_tdim=300,
num_lookup_gating=10,
):
# First create a ResNext model
model = GeneralNormalizedNN(
num_layers,
num_modules_per_layer,
feat_dim,
inter_dim,
)
# Modularize the model and create gating funcs
gated_net, gating_sampler, num_module_list, fan_in = modularize_network(
model,
stoch_sample,
use_full_model=use_full_model,
gater_type=gater_type,
tdim=tdim,
inter_tdim=inter_tdim,
single_head=single_head,
num_classes=num_classes,
num_lookup_gating=num_lookup_gating,
)
return gated_net, gating_sampler, num_module_list, fan_in
================================================
FILE: czsl/models/svm.py
================================================
import numpy as np
import tqdm
from data import dataset as dset
import os
from utils import utils
import torch
from torch.autograd import Variable
import h5py
from sklearn.svm import LinearSVC
from sklearn.model_selection import GridSearchCV
import torch.nn.functional as F
from joblib import Parallel, delayed
import glob
import scipy.io
from sklearn.calibration import CalibratedClassifierCV
import argparse
parser = argparse.ArgumentParser()
parser.add_argument('--batch_size', type=int, default=512)
parser.add_argument('--dataset', default='mitstates', help='mitstates|zappos')
parser.add_argument('--data_dir', default='data/mit-states/')
parser.add_argument('--generate', action ='store_true', default=False)
parser.add_argument('--evalsvm', action ='store_true', default=False)
parser.add_argument('--evaltf', action ='store_true', default=False)
parser.add_argument('--completed', default='tensor-completion/completed/complete.mat')
args = parser.parse_args()
#----------------------------------------------------------------------------------------#
search_params = {'C': np.logspace(-5,5,11)}
def train_svm(Y_train, sample_weight=None):
try:
clf = GridSearchCV(LinearSVC(class_weight='balanced', fit_intercept=False), search_params, scoring='f1', cv=4)
clf.fit(X_train, Y_train, sample_weight=sample_weight)
except:
clf = LinearSVC(C=0.1, class_weight='balanced', fit_intercept=False)
if Y_train.sum()==len(Y_train) or Y_train.sum()==0:
return None
clf.fit(X_train, Y_train, sample_weight=sample_weight)
return clf
def generate_svms():
# train an SVM for every attribute, object and pair primitive
Y = [(train_attrs==attr).astype(np.int) for attr in range(len(dataset.attrs))]
attr_clfs = Parallel(n_jobs=32, verbose=16)(delayed(train_svm)(Y[attr]) for attr in range(len(dataset.attrs)))
for attr, clf in enumerate(attr_clfs):
print (attr, dataset.attrs[attr])
print ('params:', clf.best_params_)
print ('-'*30)
torch.save(clf.best_estimator_, '%s/svm/attr_%d'%(args.data_dir, attr))
Y = [(train_objs==obj).astype(np.int) for obj in range(len(dataset.objs))]
obj_clfs = Parallel(n_jobs=32, verbose=16)(delayed(train_svm)(Y[obj]) for obj in range(len(dataset.objs)))
for obj, clf in enumerate(obj_clfs):
print (obj, dataset.objs[obj])
print ('params:', clf.best_params_)
print ('-'*30)
torch.save(clf.best_estimator_, '%s/svm/obj_%d'%(args.data_dir, obj))
Y, Y_attr, sample_weight = [], [], []
for idx, (attr, obj) in enumerate(dataset.train_pairs):
Y_train = ((train_attrs==attr)*(train_objs==obj)).astype(np.int)
# reweight instances to get a little more training data
Y_train_attr = (train_attrs==attr).astype(np.int)
instance_weights = 0.1*Y_train_attr
instance_weights[Y_train.nonzero()[0]] = 1.0
Y.append(Y_train)
Y_attr.append(Y_train_attr)
sample_weight.append(instance_weights)
pair_clfs = Parallel(n_jobs=32, verbose=16)(delayed(train_svm)(Y[pair]) for pair in range(len(dataset.train_pairs)))
for idx, (attr, obj) in enumerate(dataset.train_pairs):
clf = pair_clfs[idx]
print (dataset.attrs[attr], dataset.objs[obj])
try:
print ('params:', clf.best_params_)
torch.save(clf.best_estimator_, '%s/svm/pair_%d_%d'%(args.data_dir, attr, obj))
except:
print ('FAILED! #positive:', Y[idx].sum(), len(Y[idx]))
return
def make_svm_tensor():
subs, vals, size = [], [], (len(dataset.attrs), len(dataset.objs), X.shape[1])
fullsubs, fullvals = [], []
composite_clfs = glob.glob('%s/svm/pair*'%args.data_dir)
print ('%d composite classifiers found'%(len(composite_clfs)))
for clf in tqdm.tqdm(composite_clfs):
_, attr, obj = os.path.basename(clf).split('_')
attr, obj = int(attr), int(obj)
clf = torch.load(clf)
weight = clf.coef_.squeeze()
for i in range(len(weight)):
subs.append((attr, obj, i))
vals.append(weight[i])
for attr, obj in dataset.pairs:
for i in range(X.shape[1]):
fullsubs.append((attr, obj, i))
subs, vals = np.array(subs), np.array(vals).reshape(-1,1)
fullsubs, fullvals = np.array(fullsubs), np.ones(len(fullsubs)).reshape(-1,1)
savedat = {'subs':subs, 'vals':vals, 'size':size, 'fullsubs':fullsubs, 'fullvals':fullvals}
scipy.io.savemat('tensor-completion/incomplete/%s.mat'%args.dataset, savedat)
print (subs.shape, vals.shape, size)
def evaluate_svms():
attr_clfs = [torch.load('%s/svm/attr_%d'%(args.data_dir, attr)) for attr in range(len(dataset.attrs))]
obj_clfs = [torch.load('%s/svm/obj_%d'%(args.data_dir, obj)) for obj in range(len(dataset.objs))]
# Calibrate all classifiers first
Y = [(train_attrs==attr).astype(np.int) for attr in range(len(dataset.attrs))]
for attr in tqdm.tqdm(range(len(dataset.attrs))):
clf = attr_clfs[attr]
calibrated = CalibratedClassifierCV(clf, method='sigmoid', cv='prefit')
calibrated.fit(X_train, Y[attr])
attr_clfs[attr] = calibrated
Y = [(train_objs==obj).astype(np.int) for obj in range(len(dataset.objs))]
for obj in tqdm.tqdm(range(len(dataset.objs))):
clf = obj_clfs[obj]
calibrated = CalibratedClassifierCV(clf, method='sigmoid', cv='prefit')
calibrated.fit(X_train, Y[obj])
obj_clfs[obj] = calibrated
# Generate all the scores
attr_scores, obj_scores = [], []
for attr in tqdm.tqdm(range(len(dataset.attrs))):
clf = attr_clfs[attr]
score = clf.predict_proba(X_test)[:,1]
attr_scores.append(score)
attr_scores = np.vstack(attr_scores)
for obj in tqdm.tqdm(range(len(dataset.objs))):
clf = obj_clfs[obj]
score = clf.predict_proba(X_test)[:,1]
obj_scores.append(score)
obj_scores = np.vstack(obj_scores)
attr_pred = torch.from_numpy(attr_scores).transpose(0,1)
obj_pred = torch.from_numpy(obj_scores).transpose(0,1)
x = [None, Variable(torch.from_numpy(test_attrs)).long(), Variable(torch.from_numpy(test_objs)).long(), Variable(torch.from_numpy(test_pairs)).long()]
attr_pred, obj_pred, _ = utils.generate_prediction_tensors([attr_pred, obj_pred], dataset, x[2].data, source='classification')
attr_match, obj_match, zsl_match, gzsl_match, fixobj_match = utils.performance_stats(attr_pred, obj_pred, x)
print (attr_match.mean(), obj_match.mean(), zsl_match.mean(), gzsl_match.mean(), fixobj_match.mean())
def evaluate_tensorcompletion():
def parse_tensor(fl):
tensor = scipy.io.loadmat(fl)
nz_idx = zip(*(tensor['subs']))
composite_clfs = np.zeros((len(dataset.attrs), len(dataset.objs), X.shape[1]))
composite_clfs[nz_idx[0], nz_idx[1], nz_idx[2]] = tensor['vals'].squeeze()
return composite_clfs, nz_idx, tensor['vals'].squeeze()
# see recon error
tr_file = 'tensor-completion/incomplete/%s.mat'%args.dataset
ts_file = args.completed
tr_clfs, tr_nz_idx, tr_vals = parse_tensor(tr_file)
ts_clfs, ts_nz_idx, ts_vals = parse_tensor(ts_file)
print (tr_vals.min(), tr_vals.max(), tr_vals.mean())
print (ts_vals.min(), ts_vals.max(), ts_vals.mean())
print ('Completed Tensor: %s'%args.completed)
# see train recon error
err = 1.0*((tr_clfs[tr_nz_idx[0], tr_nz_idx[1], tr_nz_idx[2]]-ts_clfs[tr_nz_idx[0], tr_nz_idx[1], tr_nz_idx[2]])**2).sum()/(len(tr_vals))
print ('recon error:', err)
# Create and scale classifiers for each pair
clfs = {}
test_pair_set = set(map(tuple, dataset.test_pairs.numpy().tolist()))
for idx, (attr, obj) in tqdm.tqdm(enumerate(dataset.pairs), total=len(dataset.pairs)):
clf = LinearSVC(fit_intercept=False)
clf.fit(np.eye(2), [0,1])
if (attr, obj) in test_pair_set:
X_ = X_test
Y_ = (test_attrs==attr).astype(np.int)*(test_objs==obj).astype(np.int)
clf.coef_ = ts_clfs[attr, obj][None,:]
else:
X_ = X_train
Y_ = (train_attrs==attr).astype(np.int)*(train_objs==obj).astype(np.int)
clf.coef_ = tr_clfs[attr, obj][None,:]
calibrated = CalibratedClassifierCV(clf, method='sigmoid', cv='prefit')
calibrated.fit(X_, Y_)
clfs[(attr, obj)] = calibrated
scores = {}
for attr, obj in tqdm.tqdm(dataset.pairs):
score = clfs[(attr, obj)].predict_proba(X_test)[:,1]
scores[(attr, obj)] = torch.from_numpy(score).float().unsqueeze(1)
x = [None, Variable(torch.from_numpy(test_attrs)).long(), Variable(torch.from_numpy(test_objs)).long(), Variable(torch.from_numpy(test_pairs)).long()]
attr_pred, obj_pred, _ = utils.generate_prediction_tensors(scores, dataset, x[2].data, source='manifold')
attr_match, obj_match, zsl_match, gzsl_match, fixobj_match = utils.performance_stats(attr_pred, obj_pred, x)
print (attr_match.mean(), obj_match.mean(), zsl_match.mean(), gzsl_match.mean(), fixobj_match.mean())
#----------------------------------------------------------------------------------------#
if args.dataset == 'mitstates':
DSet = dset.MITStatesActivations
elif args.dataset == 'zappos':
DSet = dset.UTZapposActivations
dataset = DSet(root=args.data_dir, phase='train')
train_idx, train_attrs, train_objs, train_pairs = map(np.array, zip(*dataset.train_data))
test_idx, test_attrs, test_objs, test_pairs = map(np.array, zip(*dataset.test_data))
X = dataset.activations.numpy()
X_train, X_test = X[train_idx,:], X[test_idx,:]
print (len(dataset.attrs), len(dataset.objs), len(dataset.pairs))
print (X_train.shape, X_test.shape)
if args.generate:
generate_svms()
make_svm_tensor()
if args.evalsvm:
evaluate_svms()
if args.evaltf:
evaluate_tensorcompletion()
================================================
FILE: czsl/models/symnet.py
================================================
import torch
import torch.nn as nn
import torch.nn.functional as F
import numpy as np
from .word_embedding import load_word_embeddings
from .common import MLP
device = 'cuda' if torch.cuda.is_available() else 'cpu'
class Symnet(nn.Module):
def __init__(self, dset, args):
super(Symnet, self).__init__()
self.dset = dset
self.args = args
self.num_attrs = len(dset.attrs)
self.num_objs = len(dset.objs)
self.image_embedder = MLP(dset.feat_dim, args.emb_dim, relu = False)
self.ce_loss = nn.CrossEntropyLoss()
self.mse_loss = nn.MSELoss()
self.nll_loss = nn.NLLLoss()
self.softmax = nn.Softmax(dim=1)
# Attribute embedder and object classifier
self.attr_embedder = nn.Embedding(len(dset.attrs), args.emb_dim)
self.obj_classifier = MLP(args.emb_dim, len(dset.objs), num_layers = 2, relu = False, dropout = True, layers = [512]) # original paper uses a second fc classifier for final product
self.attr_classifier = MLP(args.emb_dim, len(dset.attrs), num_layers = 2, relu = False, dropout = True, layers = [512])
# CoN and DecoN
self.CoN_fc_attention = MLP(args.emb_dim, args.emb_dim, num_layers = 2, relu = False, dropout = True, layers = [512])
self.CoN_emb = MLP(args.emb_dim + args.emb_dim, args.emb_dim, num_layers = 2, relu = False, dropout = True, layers = [768])
self.DecoN_fc_attention = MLP(args.emb_dim, args.emb_dim, num_layers = 2, relu = False, dropout = True, layers = [512])
self.DecoN_emb = MLP(args.emb_dim + args.emb_dim, args.emb_dim, num_layers = 2, relu = False, dropout = True, layers = [768])
# if args.glove_init:
pretrained_weight = load_word_embeddings(args.emb_init, dset.attrs)
self.attr_embedder.weight.data.copy_(pretrained_weight)
for param in self.attr_embedder.parameters():
param.requires_grad = False
def CoN(self, img_embedding, attr_embedding):
attention = torch.sigmoid(self.CoN_fc_attention(attr_embedding))
img_embedding = attention*img_embedding + img_embedding
hidden = torch.cat([img_embedding, attr_embedding], dim = 1)
output = self.CoN_emb(hidden)
return output
def DeCoN(self, img_embedding, attr_embedding):
attention = torch.sigmoid(self.DecoN_fc_attention(attr_embedding))
img_embedding = attention*img_embedding + img_embedding
hidden = torch.cat([img_embedding, attr_embedding], dim = 1)
output = self.DecoN_emb(hidden)
return output
def distance_metric(self, a, b):
return torch.norm(a-b, dim = -1)
def RMD_prob(self, feat_plus, feat_minus, repeat_img_feat):
"""return attribute classification probability with our RMD"""
# feat_plus, feat_minus: shape=(bz, #attr, dim_emb)
# d_plus: distance between feature before&after CoN
# d_minus: distance between feature before&after DecoN
d_plus = self.distance_metric(feat_plus, repeat_img_feat).reshape(-1, self.num_attrs)
d_minus = self.distance_metric(feat_minus, repeat_img_feat).reshape(-1, self.num_attrs)
# not adding softmax because it is part of cross entropy loss
return d_minus - d_plus
def train_forward(self, x):
pos_image_feat, pos_attr_id, pos_obj_id = x[0], x[1], x[2]
neg_attr_id = x[4][:,0]
batch_size = pos_image_feat.size(0)
loss = []
pos_attr_emb = self.attr_embedder(pos_attr_id)
neg_attr_emb = self.attr_embedder(neg_attr_id)
pos_img = self.image_embedder(pos_image_feat)
# rA = remove positive attribute A
# aA = add positive attribute A
# rB = remove negative attribute B
# aB = add negative attribute B
pos_rA = self.DeCoN(pos_img, pos_attr_emb)
pos_aA = self.CoN(pos_img, pos_attr_emb)
pos_rB = self.DeCoN(pos_img, neg_attr_emb)
pos_aB = self.CoN(pos_img, neg_attr_emb)
# get all attr embedding #attr, embedding
attr_emb = torch.LongTensor(np.arange(self.num_attrs)).to(device)
attr_emb = self.attr_embedder(attr_emb)
tile_attr_emb = attr_emb.repeat(batch_size, 1) # (batch*attr, dim_emb)
# Now we calculate all the losses
if self.args.lambda_cls_attr > 0:
# Original image
score_pos_A = self.attr_classifier(pos_img)
loss_cls_pos_a = self.ce_loss(score_pos_A, pos_attr_id)
# After removing pos_attr
score_pos_rA_A = self.attr_classifier(pos_rA)
total = sum(score_pos_rA_A)
# prob_pos_rA_A = 1 - self.softmax(score_pos_rA_A)
# loss_cls_pos_rA_a = self.nll_loss(torch.log(prob_pos_rA_A), pos_attr_id)
loss_cls_pos_rA_a = self.ce_loss(total - score_pos_rA_A, pos_attr_id) #should be maximum for the gt label
# rmd time
repeat_img_feat = torch.repeat_interleave(pos_img, self.num_attrs, 0) #(batch*attr, dim_rep)
feat_plus = self.CoN(repeat_img_feat, tile_attr_emb)
feat_minus = self.DeCoN(repeat_img_feat, tile_attr_emb)
score_cls_rmd = self.RMD_prob(feat_plus, feat_minus, repeat_img_feat)
loss_cls_rmd = self.ce_loss(score_cls_rmd, pos_attr_id)
loss_cls_attr = self.args.lambda_cls_attr*sum([loss_cls_pos_a, loss_cls_pos_rA_a, loss_cls_rmd])
loss.append(loss_cls_attr)
if self.args.lambda_cls_obj > 0:
# Original image
score_pos_O = self.obj_classifier(pos_img)
loss_cls_pos_o = self.ce_loss(score_pos_O, pos_obj_id)
# After removing pos attr
score_pos_rA_O = self.obj_classifier(pos_rA)
loss_cls_pos_rA_o = self.ce_loss(score_pos_rA_O, pos_obj_id)
# After adding neg attr
score_pos_aB_O = self.obj_classifier(pos_aB)
loss_cls_pos_aB_o = self.ce_loss(score_pos_aB_O, pos_obj_id)
loss_cls_obj = self.args.lambda_cls_obj * sum([loss_cls_pos_o, loss_cls_pos_rA_o, loss_cls_pos_aB_o])
loss.append(loss_cls_obj)
if self.args.lambda_sym > 0:
loss_sys_pos = self.mse_loss(pos_aA, pos_img)
loss_sys_neg = self.mse_loss(pos_rB, pos_img)
loss_sym = self.args.lambda_sym * (loss_sys_pos + loss_sys_neg)
loss.append(loss_sym)
##### Axiom losses
if self.args.lambda_axiom > 0:
loss_clo = loss_inv = loss_com = 0
# closure
pos_aA_rA = self.DeCoN(pos_aA, pos_attr_emb)
pos_rB_aB = self.CoN(pos_rB, neg_attr_emb)
loss_clo = self.mse_loss(pos_aA_rA, pos_rA) + self.mse_loss(pos_rB_aB, pos_aB)
# invertibility
pos_rA_aA = self.CoN(pos_rA, pos_attr_emb)
pos_aB_rB = self.DeCoN(pos_aB, neg_attr_emb)
loss_inv = self.mse_loss(pos_rA_aA, pos_img) + self.mse_loss(pos_aB_rB, pos_img)
# commutative
pos_aA_rB = self.DeCoN(pos_aA, neg_attr_emb)
pos_rB_aA = self.DeCoN(pos_rB, pos_attr_emb)
loss_com = self.mse_loss(pos_aA_rB, pos_rB_aA)
loss_axiom = self.args.lambda_axiom * (loss_clo + loss_inv + loss_com)
loss.append(loss_axiom)
# triplet loss
if self.args.lambda_trip > 0:
pos_triplet = F.triplet_margin_loss(pos_img, pos_aA, pos_rA)
neg_triplet = F.triplet_margin_loss(pos_img, pos_rB, pos_aB)
loss_triplet = self.args.lambda_trip * (pos_triplet + neg_triplet)
loss.append(loss_triplet)
loss = sum(loss)
return loss, None
def val_forward(self, x):
pos_image_feat, pos_attr_id, pos_obj_id = x[0], x[1], x[2]
batch_size = pos_image_feat.shape[0]
pos_img = self.image_embedder(pos_image_feat)
repeat_img_feat = torch.repeat_interleave(pos_img, self.num_attrs, 0) #(batch*attr, dim_rep)
# get all attr embedding #attr, embedding
attr_emb = torch.LongTensor(np.arange(self.num_attrs)).to(device)
attr_emb = self.attr_embedder(attr_emb)
tile_attr_emb = attr_emb.repeat(batch_size, 1) # (batch*attr, dim_emb)
feat_plus = self.CoN(repeat_img_feat, tile_attr_emb)
feat_minus = self.DeCoN(repeat_img_feat, tile_attr_emb)
score_cls_rmd = self.RMD_prob(feat_plus, feat_minus, repeat_img_feat)
prob_A_rmd = F.softmax(score_cls_rmd, dim = 1)
score_obj = self.obj_classifier(pos_img)
prob_O = F.softmax(score_obj, dim = 1)
scores = {}
for itr, (attr, obj) in enumerate(self.dset.pairs):
attr_id, obj_id = self.dset.attr2idx[attr], self.dset.obj2idx[obj]
score = prob_A_rmd[:,attr_id] * prob_O[:, obj_id]
scores[(attr, obj)] = score
return None, scores
def forward(self, x):
if self.training:
loss, pred = self.train_forward(x)
else:
with torch.no_grad():
loss, pred = self.val_forward(x)
return loss, pred
================================================
FILE: czsl/models/visual_product.py
================================================
import torch
import torch.nn as nn
import torch.nn.functional as F
import torchvision.models as models
import numpy as np
from .common import MLP
class VisualProductNN(nn.Module):
def __init__(self, dset, args):
super(VisualProductNN, self).__init__()
self.attr_clf = MLP(dset.feat_dim, len(dset.attrs), 2, relu = False)
self.obj_clf = MLP(dset.feat_dim, len(dset.objs), 2, relu = False)
self.dset = dset
def train_forward(self, x):
img, attrs, objs = x[0],x[1], x[2]
attr_pred = self.attr_clf(img)
obj_pred = self.obj_clf(img)
attr_loss = F.cross_entropy(attr_pred, attrs)
obj_loss = F.cross_entropy(obj_pred, objs)
loss = attr_loss + obj_loss
return loss, None
def val_forward(self, x):
img = x[0]
attr_pred = F.softmax(self.attr_clf(img), dim =1)
obj_pred = F.softmax(self.obj_clf(img), dim = 1)
scores = {}
for itr, (attr, obj) in enumerate(self.dset.pairs):
attr_id, obj_id = self.dset.attr2idx[attr], self.dset.obj2idx[obj]
score = attr_pred[:,attr_id] * obj_pred[:, obj_id]
scores[(attr, obj)] = score
return None, scores
def forward(self, x):
if self.training:
loss, pred = self.train_forward(x)
else:
with torch.no_grad():
loss, pred = self.val_forward(x)
return loss, pred
================================================
FILE: czsl/models/word_embedding.py
================================================
import torch
import numpy as np
from flags import DATA_FOLDER
def load_word_embeddings(emb_type, vocab):
if emb_type == 'glove':
embeds = load_glove_embeddings(vocab)
elif emb_type == 'fasttext':
embeds = load_fasttext_embeddings(vocab)
elif emb_type == 'word2vec':
embeds = load_word2vec_embeddings(vocab)
elif emb_type == 'ft+w2v':
embeds1 = load_fasttext_embeddings(vocab)
embeds2 = load_word2vec_embeddings(vocab)
embeds = torch.cat([embeds1, embeds2], dim = 1)
print('Combined embeddings are ',embeds.shape)
elif emb_type == 'ft+gl':
embeds1 = load_fasttext_embeddings(vocab)
embeds2 = load_glove_embeddings(vocab)
embeds = torch.cat([embeds1, embeds2], dim = 1)
print('Combined embeddings are ',embeds.shape)
elif emb_type == 'ft+ft':
embeds1 = load_fasttext_embeddings(vocab)
embeds = torch.cat([embeds1, embeds1], dim = 1)
print('Combined embeddings are ',embeds.shape)
elif emb_type == 'gl+w2v':
embeds1 = load_glove_embeddings(vocab)
embeds2 = load_word2vec_embeddings(vocab)
embeds = torch.cat([embeds1, embeds2], dim = 1)
print('Combined embeddings are ',embeds.shape)
elif emb_type == 'ft+w2v+gl':
embeds1 = load_fasttext_embeddings(vocab)
embeds2 = load_word2vec_embeddings(vocab)
embeds3 = load_glove_embeddings(vocab)
embeds = torch.cat([embeds1, embeds2, embeds3], dim = 1)
print('Combined embeddings are ',embeds.shape)
else:
raise ValueError('Invalid embedding')
return embeds
def load_fasttext_embeddings(vocab):
custom_map = {
'Faux.Fur': 'fake fur',
'Faux.Leather': 'fake leather',
'Full.grain.leather': 'thick leather',
'Hair.Calf': 'hairy leather',
'Patent.Leather': 'shiny leather',
'Boots.Ankle': 'ankle boots',
'Boots.Knee.High': 'kneehigh boots',
'Boots.Mid-Calf': 'midcalf boots',
'Shoes.Boat.Shoes': 'boatshoes',
'Shoes.Clogs.and.Mules': 'clogs shoes',
'Shoes.Flats': 'flats shoes',
'Shoes.Heels': 'heels',
'Shoes.Loafers': 'loafers',
'Shoes.Oxfords': 'oxford shoes',
'Shoes.Sneakers.and.Athletic.Shoes': 'sneakers',
'traffic_light': 'traficlight',
'trash_can': 'trashcan',
'dry-erase_board' : 'dry_erase_board',
'black_and_white' : 'black_white',
'eiffel_tower' : 'tower'
}
vocab_lower = [v.lower() for v in vocab]
vocab = []
for current in vocab_lower:
if current in custom_map:
vocab.append(custom_map[current])
else:
vocab.append(current)
import fasttext.util
ft = fasttext.load_model(DATA_FOLDER+'/fast/cc.en.300.bin')
embeds = []
for k in vocab:
if '_' in k:
ks = k.split('_')
emb = np.stack([ft.get_word_vector(it) for it in ks]).mean(axis=0)
else:
emb = ft.get_word_vector(k)
embeds.append(emb)
embeds = torch.Tensor(np.stack(embeds))
print('Fasttext Embeddings loaded, total embeddings: {}'.format(embeds.size()))
return embeds
def load_word2vec_embeddings(vocab):
# vocab = [v.lower() for v in vocab]
from gensim import models
model = models.KeyedVectors.load_word2vec_format(
DATA_FOLDER+'/w2v/GoogleNews-vectors-negative300.bin', binary=True)
custom_map = {
'Faux.Fur': 'fake_fur',
'Faux.Leather': 'fake_leather',
'Full.grain.leather': 'thick_leather',
'Hair.Calf': 'hair_leather',
'Patent.Leather': 'shiny_leather',
'Boots.Ankle': 'ankle_boots',
'Boots.Knee.High': 'knee_high_boots',
'Boots.Mid-Calf': 'midcalf_boots',
'Shoes.Boat.Shoes': 'boat_shoes',
'Shoes.Clogs.and.Mules': 'clogs_shoes',
'Shoes.Flats': 'flats_shoes',
'Shoes.Heels': 'heels',
'Shoes.Loafers': 'loafers',
'Shoes.Oxfords': 'oxford_shoes',
'Shoes.Sneakers.and.Athletic.Shoes': 'sneakers',
'traffic_light': 'traffic_light',
'trash_can': 'trashcan',
'dry-erase_board' : 'dry_erase_board',
'black_and_white' : 'black_white',
'eiffel_tower' : 'tower'
}
embeds = []
for k in vocab:
if k in custom_map:
k = custom_map[k]
if '_' in k and k not in model:
ks = k.split('_')
emb = np.stack([model[it] for it in ks]).mean(axis=0)
else:
emb = model[k]
embeds.append(emb)
embeds = torch.Tensor(np.stack(embeds))
print('Word2Vec Embeddings loaded, total embeddings: {}'.format(embeds.size()))
return embeds
def load_glove_embeddings(vocab):
'''
Inputs
emb_file: Text file with word embedding pairs e.g. Glove, Processed in lower case.
vocab: List of words
Returns
Embedding Matrix
'''
vocab = [v.lower() for v in vocab]
emb_file = DATA_FOLDER+'/glove/glove.6B.300d.txt'
model = {} # populating a dictionary of word and embeddings
for line in open(emb_file, 'r'):
line = line.strip().split(' ') # Word-embedding
wvec = torch.FloatTensor(list(map(float, line[1:])))
model[line[0]] = wvec
# Adding some vectors for UT Zappos
custom_map = {
'faux.fur': 'fake_fur',
'faux.leather': 'fake_leather',
'full.grain.leather': 'thick_leather',
'hair.calf': 'hair_leather',
'patent.leather': 'shiny_leather',
'boots.ankle': 'ankle_boots',
'boots.knee.high': 'knee_high_boots',
'boots.mid-calf': 'midcalf_boots',
'shoes.boat.shoes': 'boat_shoes',
'shoes.clogs.and.mules': 'clogs_shoes',
'shoes.flats': 'flats_shoes',
'shoes.heels': 'heels',
'shoes.loafers': 'loafers',
'shoes.oxfords': 'oxford_shoes',
'shoes.sneakers.and.athletic.shoes': 'sneakers',
'traffic_light': 'traffic_light',
'trash_can': 'trashcan',
'dry-erase_board' : 'dry_erase_board',
'black_and_white' : 'black_white',
'eiffel_tower' : 'tower',
'nubuck' : 'grainy_leather',
}
embeds = []
for k in vocab:
if k in custom_map:
k = custom_map[k]
if '_' in k:
ks = k.split('_')
emb = torch.stack([model[it] for it in ks]).mean(dim=0)
else:
emb = model[k]
embeds.append(emb)
embeds = torch.stack(embeds)
print('Glove Embeddings loaded, total embeddings: {}'.format(embeds.size()))
return embeds
================================================
FILE: czsl/notebooks/analysis.ipynb
================================================
{
"cells": [
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"import sys\n",
"sys.path.append('../')\n",
"# Torch imports\n",
"import torch\n",
"import torch.nn as nn\n",
"import torch.optim as optim\n",
"import torch.nn.functional as F\n",
"from torch.utils.tensorboard import SummaryWriter\n",
"import torch.backends.cudnn as cudnn\n",
"cudnn.benchmark = True\n",
"\n",
"# Python imports\n",
"import numpy as np\n",
"import tqdm\n",
"import torchvision.models as tmodels\n",
"from tqdm import tqdm\n",
"import os\n",
"from os.path import join as ospj\n",
"import itertools\n",
"import glob\n",
"import random\n",
"\n",
"#Local imports\n",
"from data import dataset as dset\n",
"from models.common import Evaluator\n",
"from models.image_extractor import get_image_extractor\n",
"from models.manifold_methods import RedWine, LabelEmbedPlus, AttributeOperator\n",
"from models.modular_methods import GatedGeneralNN\n",
"from models.symnet import Symnet\n",
"from utils.utils import save_args, UnNormalizer, load_args\n",
"from utils.config_model import configure_model\n",
"from flags import parser\n",
"from PIL import Image\n",
"import matplotlib.pyplot as plt\n",
"import importlib\n",
"\n",
"device = 'cuda' if torch.cuda.is_available() else 'cpu'\n",
"args, unknown = parser.parse_known_args()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Run one of the cells to load the dataset you want to run test for and move to the next section"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"best_mit = '../logs/graphembed/mitstates/base/mit.yml'\n",
"load_args(best_mit,args)\n",
"args.graph_init = '../'+args.graph_init\n",
"args.load = best_mit[:-7] + 'ckpt_best_auc.t7'"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"best_ut = '../logs/graphembed/utzappos/base/utzappos.yml'\n",
"load_args(best_ut,args)\n",
"args.graph_init = '../'+args.graph_init\n",
"args.load = best_ut[:-12] + 'ckpt_best_auc.t7'"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Loading arguments and dataset"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"args.data_dir = '../'+args.data_dir\n",
"args.test_set = 'test'\n",
"testset = dset.CompositionDataset(\n",
" root= args.data_dir,\n",
" phase=args.test_set,\n",
" split=args.splitname,\n",
" model =args.image_extractor,\n",
" subset=args.subset,\n",
" return_images = True,\n",
" update_features = args.update_features,\n",
" clean_only = args.clean_only\n",
" )\n",
"testloader = torch.utils.data.DataLoader(\n",
" testset,\n",
" batch_size=args.test_batch_size,\n",
" shuffle=True,\n",
" num_workers=args.workers)\n",
"\n",
"print('Objs ', len(testset.objs), ' Attrs ', len(testset.attrs))"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"image_extractor, model, optimizer = configure_model(args, testset)\n",
"evaluator = Evaluator(testset, model)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"if args.load is not None:\n",
" checkpoint = torch.load(args.load)\n",
" if image_extractor:\n",
" try:\n",
" image_extractor.load_state_dict(checkpoint['image_extractor'])\n",
" image_extractor.eval()\n",
" except:\n",
" print('No Image extractor in checkpoint')\n",
" model.load_state_dict(checkpoint['net'])\n",
" model.eval()\n",
" print('Loaded model from ', args.load)\n",
" print('Best AUC: ', checkpoint['AUC'])"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"def print_results(scores, exp):\n",
" print(exp)\n",
" result = scores[exp]\n",
" attr = [evaluator.dset.attrs[result[0][idx,a]] for a in range(topk)]\n",
" obj = [evaluator.dset.objs[result[1][idx,a]] for a in range(topk)]\n",
" attr_gt, obj_gt = evaluator.dset.attrs[data[1][idx]], evaluator.dset.objs[data[2][idx]]\n",
" print(f'Ground truth: {attr_gt} {obj_gt}')\n",
" prediction = ''\n",
" for a,o in zip(attr, obj):\n",
" prediction += a + ' ' + o + '| '\n",
" print('Predictions: ', prediction)\n",
" print('__'*50)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### An example of predictions\n",
"closed -> Biased for unseen classes\n",
"\n",
"unbiiased -> Biased against unseen classes"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"data = next(iter(testloader))\n",
"images = data[-1]\n",
"data = [d.to(device) for d in data[:-1]]\n",
"if image_extractor:\n",
" data[0] = image_extractor(data[0])\n",
"_, predictions = model(data)\n",
"data = [d.to('cpu') for d in data]\n",
"topk = 5\n",
"results = evaluator.score_model(predictions, data[2], bias = 1000, topk=topk)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"scrolled": true
},
"outputs": [],
"source": [
"for idx in range(len(images)):\n",
" seen = bool(evaluator.seen_mask[data[3][idx]])\n",
" if seen:\n",
" continue\n",
" image = Image.open(ospj( args.data_dir,'images', images[idx]))\n",
" \n",
" plt.figure(dpi=300)\n",
" plt.imshow(image)\n",
" plt.axis('off')\n",
" plt.show()\n",
" \n",
" print(f'GT pair seen: {seen}')\n",
" print_results(results, 'closed')\n",
" print_results(results, 'unbiased_closed')"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Run Evaluation"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"model.eval()\n",
"args.bias = 1e3\n",
"accuracies, all_attr_gt, all_obj_gt, all_pair_gt, all_pred = [], [], [], [], []\n",
"\n",
"for idx, data in tqdm(enumerate(testloader), total=len(testloader), desc = 'Testing'):\n",
" data.pop()\n",
" data = [d.to(device) for d in data]\n",
" if image_extractor:\n",
" data[0] = image_extractor(data[0])\n",
"\n",
" _, predictions = model(data) # todo: Unify outputs across models\n",
"\n",
" attr_truth, obj_truth, pair_truth = data[1], data[2], data[3]\n",
" all_pred.append(predictions)\n",
" all_attr_gt.append(attr_truth)\n",
" all_obj_gt.append(obj_truth)\n",
" all_pair_gt.append(pair_truth)\n",
"\n",
"all_attr_gt, all_obj_gt, all_pair_gt = torch.cat(all_attr_gt), torch.cat(all_obj_gt), torch.cat(all_pair_gt)\n",
"\n",
"all_pred_dict = {}\n",
"# Gather values as dict of (attr, obj) as key and list of predictions as values\n",
"for k in all_pred[0].keys():\n",
" all_pred_dict[k] = torch.cat(\n",
" [all_pred[i][k] for i in range(len(all_pred))])\n",
"\n",
"# Calculate best unseen accuracy\n",
"attr_truth, obj_truth = all_attr_gt.to('cpu'), all_obj_gt.to('cpu')\n",
"pairs = list(\n",
" zip(list(attr_truth.numpy()), list(obj_truth.numpy())))"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"topk = 1 ### For topk results\n",
"our_results = evaluator.score_model(all_pred_dict, all_obj_gt, bias = 1e3, topk = topk)\n",
"stats = evaluator.evaluate_predictions(our_results, all_attr_gt, all_obj_gt, all_pair_gt, all_pred_dict, topk = topk)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"for k, v in stats.items():\n",
" print(k, v)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": []
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.8.5"
}
},
"nbformat": 4,
"nbformat_minor": 4
}
================================================
FILE: czsl/test.py
================================================
# Torch imports
import torch
from torch.utils.tensorboard import SummaryWriter
import torch.backends.cudnn as cudnn
import numpy as np
from flags import DATA_FOLDER
cudnn.benchmark = True
# Python imports
import tqdm
from tqdm import tqdm
import os
from os.path import join as ospj
# Local imports
from data import dataset as dset
from models.common import Evaluator
from utils.utils import load_args
from utils.config_model import configure_model
from flags import parser
device = 'cuda' if torch.cuda.is_available() else 'cpu'
def main():
# Get arguments and start logging
args = parser.parse_args()
logpath = args.logpath
config = [os.path.join(logpath, _) for _ in os.listdir(logpath) if _.endswith('yml')][0]
load_args(config, args)
# Get dataset
trainset = dset.CompositionDataset(
root=os.path.join(DATA_FOLDER,args.data_dir),
phase='train',
split=args.splitname,
model=args.image_extractor,
update_features=args.update_features,
train_only=args.train_only,
subset=args.subset,
open_world=args.open_world
)
valset = dset.CompositionDataset(
root=os.path.join(DATA_FOLDER,args.data_dir),
phase='val',
split=args.splitname,
model=args.image_extractor,
subset=args.subset,
update_features=args.update_features,
open_world=args.open_world
)
valoader = torch.utils.data.DataLoader(
valset,
batch_size=args.test_batch_size,
shuffle=False,
num_workers=8)
testset = dset.CompositionDataset(
root=os.path.join(DATA_FOLDER,args.data_dir),
phase='test',
split=args.splitname,
model =args.image_extractor,
subset=args.subset,
update_features = args.update_features,
open_world=args.open_world
)
testloader = torch.utils.data.DataLoader(
testset,
batch_size=args.test_batch_size,
shuffle=False,
num_workers=args.workers)
# Get model and optimizer
image_extractor, model, optimizer = configure_model(args, trainset)
args.extractor = image_extractor
if args.add_bt:
args.load = ospj(logpath,'ckpt_global{}1e-06_best_auc.t7'.format(args.add_bt))
else:
args.load = ospj(logpath,'ckpt_best_auc.t7')
print(args.load)
checkpoint = torch.load(args.load)
print(checkpoint['epoch'])
if image_extractor:
try:
image_extractor.load_state_dict(checkpoint['image_extractor'])
image_extractor.eval()
except:
print('No Image extractor in checkpoint')
net_params = {}
for k in checkpoint['net'].keys():
nk = k
# if k.startswith('gcn.'):
# nk = 'model.' + k
net_params[nk] = checkpoint['net'][k]
print(net_params.keys())
model.load_state_dict(net_params, False)
model.eval()
threshold = None
if args.open_world and args.hard_masking:
assert args.model == 'compcos', args.model + ' does not have hard masking.'
if args.threshold is not None:
threshold = args.threshold
else:
evaluator_val = Evaluator(valset, model)
unseen_scores = model.compute_feasibility().to('cpu')
seen_mask = model.seen_mask.to('cpu')
min_feasibility = (unseen_scores+seen_mask*10.).min()
max_feasibility = (unseen_scores-seen_mask*10.).max()
thresholds = np.linspace(min_feasibility,max_feasibility, num=args.threshold_trials)
best_auc = 0.
best_th = -10
with torch.no_grad():
for th in thresholds:
results = test(image_extractor,model,valoader,evaluator_val,args,threshold=th,print_results=False)
auc = results['AUC']
if auc > best_auc:
best_auc = auc
best_th = th
print('New best AUC',best_auc)
print('Threshold',best_th)
threshold = best_th
evaluator = Evaluator(testset, model)
with torch.no_grad():
test(image_extractor, model, testloader, evaluator, args, threshold)
def test(image_extractor, model, testloader, evaluator, args, threshold=None, print_results=True):
if image_extractor:
image_extractor.eval()
model.eval()
accuracies, all_sub_gt, all_attr_gt, all_obj_gt, all_pair_gt, all_pred = [], [], [], [], [], []
for idx, data in tqdm(enumerate(testloader), total=len(testloader), desc='Testing'):
data = [d.to(device) for d in data]
if image_extractor:
data[0] = image_extractor(data[0])
if threshold is None:
_, predictions = model(data)
else:
_, predictions = model.val_forward_with_threshold(data,threshold)
attr_truth, obj_truth, pair_truth = data[1], data[2], data[3]
all_pred.append(predictions)
all_attr_gt.append(attr_truth)
all_obj_gt.append(obj_truth)
all_pair_gt.append(pair_truth)
all_attr_gt, all_obj_gt, all_pair_gt = torch.cat(all_attr_gt), torch.cat(all_obj_gt), torch.cat(all_pair_gt)
all_pred_dict = {}
# Gather values as dict of (attr, obj) as key and list of predictions as values
for k in all_pred[0].keys():
all_pred_dict[k] = torch.cat(
[all_pred[i][k] for i in range(len(all_pred))])
# Calculate best unseen accuracy
results = evaluator.score_model(all_pred_dict, all_obj_gt, bias=args.bias, topk=args.topk)
stats = evaluator.evaluate_predictions(results, all_attr_gt, all_obj_gt, all_pair_gt, all_pred_dict,
topk=args.topk)
result = ''
for key in stats:
result = result + key + ' ' + str(round(stats[key], 4)) + '| '
result = result + args.name
if print_results:
print(f'Results')
print(result)
return results
if __name__ == '__main__':
main()
================================================
FILE: czsl/train.py
================================================
# Torch imports
import torch
import torch.nn as nn
import torch.nn.functional as F
from torch.utils.tensorboard import SummaryWriter
import torch.backends.cudnn as cudnn
cudnn.benchmark = True
# Python imports
import tqdm
from tqdm import tqdm
import os
from os.path import join as ospj
import csv
#Local imports
from data import dataset as dset
from models.common import Evaluator
from utils.utils import save_args, load_args
from utils.config_model import configure_model
from flags import parser, DATA_FOLDER
best_auc = 0
best_hm = 0
compose_switch = True
device = 'cuda' if torch.cuda.is_available() else 'cpu'
def main():
# Get arguments and start logging
args = parser.parse_args()
load_args(args.config, args)
logpath = os.path.join(args.cv_dir, args.name)
os.makedirs(logpath, exist_ok=True)
save_args(args, logpath, args.config)
writer = SummaryWriter(log_dir = logpath, flush_secs = 30)
# Get dataset
trainset = dset.CompositionDataset(
root=os.path.join(DATA_FOLDER,args.data_dir),
phase='train',
split=args.splitname,
model =args.image_extractor,
num_negs=args.num_negs,
pair_dropout=args.pair_dropout,
update_features = args.update_features,
train_only= args.train_only,
open_world=args.open_world
)
trainloader = torch.utils.data.DataLoader(
trainset,
batch_size=args.batch_size,
shuffle=True,
num_workers=args.workers)
testset = dset.CompositionDataset(
root=os.path.join(DATA_FOLDER,args.data_dir),
phase=args.test_set,
split=args.splitname,
model =args.image_extractor,
subset=args.subset,
update_features = args.update_features,
open_world=args.open_world
)
testloader = torch.utils.data.DataLoader(
testset,
batch_size=args.test_batch_size,
shuffle=False,
num_workers=args.workers)
# Get model and optimizer
image_extractor, model, optimizer = configure_model(args, trainset)
args.extractor = image_extractor
train = train_normal
evaluator_val = Evaluator(testset, model)
print(model)
start_epoch = 0
# Load checkpoint
if args.load is not None:
checkpoint = torch.load(args.load)
if image_extractor:
try:
image_extractor.load_state_dict(checkpoint['image_extractor'])
if args.freeze_features:
print('Freezing image extractor')
image_extractor.eval()
for param in image_extractor.parameters():
param.requires_grad = False
except:
print('No Image extractor in checkpoint')
model.load_state_dict(checkpoint['net'])
start_epoch = checkpoint['epoch']
print('Loaded model from ', args.load)
for epoch in tqdm(range(start_epoch, args.max_epochs + 1), desc = 'Current epoch'):
train(epoch, image_extractor, model, trainloader, optimizer, writer)
if model.is_open and args.model=='compcos' and ((epoch+1)%args.update_feasibility_every)==0 :
print('Updating feasibility scores')
model.update_feasibility(epoch+1.)
if epoch % args.eval_val_every == 0:
with torch.no_grad(): # todo: might not be needed
test(epoch, image_extractor, model, testloader, evaluator_val, writer, args, logpath)
print('Best AUC achieved is ', best_auc)
print('Best HM achieved is ', best_hm)
def train_normal(epoch, image_extractor, model, trainloader, optimizer, writer):
'''
Runs training for an epoch
'''
if image_extractor:
image_extractor.train()
model.train() # Let's switch to training
train_loss = 0.0
for idx, data in tqdm(enumerate(trainloader), total=len(trainloader), desc = 'Training'):
data = [d.to(device) for d in data]
if image_extractor:
data[0] = image_extractor(data[0])
loss, _ = model(data)
optimizer.zero_grad()
loss.backward()
optimizer.step()
train_loss += loss.item()
train_loss = train_loss/len(trainloader)
writer.add_scalar('Loss/train_total', train_loss, epoch)
print('Epoch: {}| Loss: {}'.format(epoch, round(train_loss, 2)))
def test(epoch, image_extractor, model, testloader, evaluator, writer, args, logpath):
'''
Runs testing for an epoch
'''
global best_auc, best_hm
def save_checkpoint(filename):
state = {
'net': model.state_dict(),
'epoch': epoch,
'AUC': stats['AUC']
}
if image_extractor:
state['image_extractor'] = image_extractor.state_dict()
filename = 'global{}_'.format(str(args.add_bt)+str(args.lrb)) + filename
torch.save(state, os.path.join(logpath, 'ckpt_{}.t7'.format(filename)))
if image_extractor:
image_extractor.eval()
model.eval()
accuracies, all_sub_gt, all_attr_gt, all_obj_gt, all_pair_gt, all_pred = [], [], [], [], [], []
for idx, data in tqdm(enumerate(testloader), total=len(testloader), desc='Testing'):
data = [d.to(device) for d in data]
if image_extractor:
data[0] = image_extractor(data[0])
_, predictions = model(data)
attr_truth, obj_truth, pair_truth = data[1], data[2], data[3]
all_pred.append({k: v.to('cpu') for k, v in predictions.items()})
all_attr_gt.append(attr_truth)
all_obj_gt.append(obj_truth)
all_pair_gt.append(pair_truth)
all_attr_gt, all_obj_gt, all_pair_gt = torch.cat(all_attr_gt), torch.cat(all_obj_gt), torch.cat(all_pair_gt)
all_pred_dict = {}
# Gather values as dict of (attr, obj) as key and list of predictions as values
for k in all_pred[0].keys():
all_pred_dict[k] = torch.cat(
[all_pred[i][k] for i in range(len(all_pred))])
# Calculate best unseen accuracy
results = evaluator.score_model(all_pred_dict, all_obj_gt, bias=args.bias, topk=args.topk)
stats = evaluator.evaluate_predictions(results, all_attr_gt, all_obj_gt, all_pair_gt, all_pred_dict, topk=args.topk)
stats['a_epoch'] = epoch
result = ''
# write to Tensorboard
for key in stats:
writer.add_scalar(key, stats[key], epoch)
result = result + key + ' ' + str(round(stats[key], 4)) + '| '
result = result + args.name
print(f'Test Epoch: {epoch}')
print(result)
if epoch > 0 and epoch % args.save_every == 0:
save_checkpoint(epoch)
if stats['AUC'] > best_auc:
best_auc = stats['AUC']
print('New best AUC ', best_auc)
save_checkpoint('best_auc')
if stats['best_hm'] > best_hm:
best_hm = stats['best_hm']
print('New best HM ', best_hm)
save_checkpoint('best_hm')
# Logs
with open(ospj(logpath, 'logs.csv'), 'a') as f:
w = csv.DictWriter(f, stats.keys())
if epoch == 0:
w.writeheader()
w.writerow(stats)
if __name__ == '__main__':
try:
main()
except KeyboardInterrupt:
print('Best AUC achieved is ', best_auc)
print('Best HM achieved is ', best_hm)
================================================
FILE: czsl/utils/__init__.py
================================================
================================================
FILE: czsl/utils/config_model.py
================================================
import torch
import torch.optim as optim
from models.image_extractor import get_image_extractor
from models.visual_product import VisualProductNN
from models.manifold_methods import RedWine, LabelEmbedPlus, AttributeOperator
from models.modular_methods import GatedGeneralNN
from models.graph_method import GraphFull
from models.symnet import Symnet
from models.compcos import CompCos
device = 'cuda' if torch.cuda.is_available() else 'cpu'
class TransformerDecorator(torch.nn.Module):
def __init__(self, model, model_name, add_bt=0, eval_bt=0):
super(TransformerDecorator, self).__init__()
self.model = model
self.model_name = model_name
self.add_bt = add_bt
self.eval_bt = eval_bt
if self.add_bt:
self.encoder_layers = torch.nn.TransformerEncoderLayer(512, 4, 512, 0.5)
@property
def is_open(self):
return self.model.is_open
@is_open.setter
def is_open(self, value):
self.model.is_open = value
def update_feasibility(self, value):
self.model.update_feasibility(value)
def forward_feats(self, feature):
feature[0] = feature[0].view(feature[0].size(0), -1)
old_feat = feature[0]
if self.model.training and self.add_bt or self.eval_bt:
feature[0] = feature[0].unsqueeze(1)
feature[0] = self.encoder_layers(feature[0])
feature[0] = feature[0].squeeze(1)
if self.add_bt > 1:
feature[0] = torch.cat([old_feat, feature[0]], dim=0)
return feature
def forward(self, feature):
feature = self.forward_feats(feature)
feature[0] = self.model(feature)
return feature[0]
def configure_model(args, dataset):
image_extractor = None
is_open = False
if args.model == 'visprodNN':
model = VisualProductNN(dataset, args)
elif args.model == 'redwine':
model = RedWine(dataset, args)
elif args.model == 'labelembed+':
model = LabelEmbedPlus(dataset, args)
elif args.model == 'attributeop':
model = AttributeOperator(dataset, args)
elif args.model == 'tmn':
model = GatedGeneralNN(dataset, args, num_layers=args.nlayers, num_modules_per_layer=args.nmods)
elif args.model == 'symnet':
model = Symnet(dataset, args)
elif args.model == 'graphfull':
model = GraphFull(dataset, args)
elif args.model == 'compcos':
model = CompCos(dataset, args)
if dataset.open_world and not args.train_only:
is_open = True
else:
raise NotImplementedError
# add Transformer
model = TransformerDecorator(model, args.model, args.add_bt)
model = model.to(device)
if args.update_features:
print('Learnable image_embeddings')
image_extractor = get_image_extractor(arch = args.image_extractor, pretrained = True)
image_extractor = image_extractor.to(device)
# configuring optimizer
if args.model=='redwine':
optim_params = filter(lambda p: p.requires_grad, model.parameters())
elif args.model=='attributeop':
attr_params = [param for name, param in model.named_parameters() if 'attr_op' in name and param.requires_grad]
other_params = [param for name, param in model.named_parameters() if 'attr_op' not in name and param.requires_grad]
optim_params = [{'params':attr_params, 'lr':0.1*args.lr}, {'params':other_params}]
elif args.model=='tmn':
gating_params = [
param for name, param in model.named_parameters()
if 'gating_network' in name and param.requires_grad
]
network_params = [
param for name, param in model.named_parameters()
if 'gating_network' not in name and param.requires_grad
]
optim_params = [
{
'params': network_params,
},
{
'params': gating_params,
'lr': args.lrg
},
]
else:
model_params = [param for name, param in model.named_parameters() if param.requires_grad and not name.__contains__('encoder_layers')]
optim_params = [{'params':model_params}]
if args.add_bt:
model_params = [param for name, param in model.named_parameters() if param.requires_grad and name.__contains__('encoder_layers')]
optim_params.append({'params':model_params, 'lr': args.lrb})
if args.update_features:
ie_parameters = [param for name, param in image_extractor.named_parameters()]
optim_params.append({'params': ie_parameters,
'lr': args.lrg})
optimizer = optim.Adam(optim_params, lr=args.lr, weight_decay=args.wd)
model.is_open = is_open
return image_extractor, model, optimizer
================================================
FILE: czsl/utils/download_data.sh
================================================
# Copyright (c) Facebook, Inc. and its affiliates.
# All rights reserved.
#
# This source code is licensed under the license found in the
# LICENSE file in the root directory of this source tree.
#
CURRENT_DIR=$(pwd)
FOLDER=$1
mkdir $FOLDER
mkdir $FOLDER/fast
mkdir $FOLDER/glove
mkdir $FOLDER/w2v
wget --show-progress -O utils/cgqa-graph.t7 https://raw.githubusercontent.com/ExplainableML/czsl/main/utils/cgqa-graph.t7
wget --show-progress -O utils/mitstates-graph.t7 https://raw.githubusercontent.com/ExplainableML/czsl/main/utils/mitstates-graph.t7
wget --show-progress -O utils/utzappos-graph.t7 https://raw.githubusercontent.com/ExplainableML/czsl/main/utils/utzappos-graph.t7
cp utils/download_embeddings.py $FOLDER/fast
# Download everything
wget --show-progress -O $FOLDER/attr-ops-data.tar.gz https://www.cs.utexas.edu/~tushar/attribute-ops/attr-ops-data.tar.gz
wget --show-progress -O $FOLDER/mitstates.zip http://wednesday.csail.mit.edu/joseph_result/state_and_transformation/release_dataset.zip
wget --show-progress -O $FOLDER/utzap.zip http://vision.cs.utexas.edu/projects/finegrained/utzap50k/ut-zap50k-images.zip
wget --show-progress -O $FOLDER/splits.tar.gz https://www.senthilpurushwalkam.com/publication/compositional/compositional_split_natural.tar.gz
wget --show-progress -O $FOLDER/cgqa.zip https://s3.mlcloud.uni-tuebingen.de/czsl/cgqa.zip
echo "Data downloaded. Extracting files..."
sed -i "s|ROOT_FOLDER|$FOLDER|g" utils/reorganize_utzap.py
sed -i "s|ROOT_FOLDER|$FOLDER|g" flags.py
# Dataset metadata, pretrained SVMs and features, tensor completion data
cd $FOLDER
tar -zxvf attr-ops-data.tar.gz --strip 1
# MIT-States
unzip mitstates.zip 'release_dataset/images/*' -d mit-states/
mv mit-states/release_dataset/images mit-states/images/
rm -r mit-states/release_dataset
rename "s/ /_/g" mit-states/images/*
# UT-Zappos50k
unzip utzap.zip -d ut-zap50k/
mv ut-zap50k/ut-zap50k-images ut-zap50k/_images/
# C-GQA
unzip cgqa.zip -d cgqa/
# Download new splits for Purushwalkam et. al
tar -zxvf splits.tar.gz
# remove all zip files and temporary files
rm -r attr-ops-data.tar.gz mitstates.zip utzap.zip splits.tar.gz cgqa.zip
# Download embeddings
# Glove (from attribute as operators)
mv data/glove/* glove/
# FastText
cd fast
python download_embeddings.py
rm cc.en.300.bin.gz
# Word2Vec
cd ../w2v
wget -c "https://s3.amazonaws.com/dl4j-distribution/GoogleNews-vectors-negative300.bin.gz"
gzip -d GoogleNews-vectors-negative300.bin.gz
rm GoogleNews-vectors-negative300.bin.gz
cd ..
rm -r data
cd $CURRENT_DIR
python utils/reorganize_utzap.py
================================================
FILE: czsl/utils/download_embeddings.py
================================================
import fasttext.util
fasttext.util.download_model('en', if_exists='ignore') # English
================================================
FILE: czsl/utils/reorganize_utzap.py
================================================
# Copyright (c) Facebook, Inc. and its affiliates.
# All rights reserved.
#
# This source code is licensed under the license found in the
# LICENSE file in the root directory of this source tree.
#
"""
Reorganize the UT-Zappos dataset to resemble the MIT-States dataset
root/attr_obj/img1.jpg
root/attr_obj/img2.jpg
root/attr_obj/img3.jpg
...
"""
import os
import torch
import shutil
import tqdm
DATA_FOLDER= "/scratch/ZHIHOU/CZSL"
root = DATA_FOLDER+'/ut-zap50k/'
os.makedirs(root+'/images',exist_ok=True)
data = torch.load(root+'/metadata_compositional-split-natural.t7')
for instance in tqdm.tqdm(data):
image, attr, obj = instance['_image'], instance['attr'], instance['obj']
old_file = '%s/_images/%s'%(root, image)
new_dir = '%s/images/%s_%s/'%(root, attr, obj)
os.makedirs(new_dir, exist_ok=True)
shutil.copy(old_file, new_dir)
================================================
FILE: czsl/utils/utils.py
================================================
import os
from os.path import join as ospj
import torch
import random
import copy
import shutil
import sys
import yaml
def chunks(l, n):
"""Yield successive n-sized chunks from l."""
for i in range(0, len(l), n):
yield l[i:i + n]
def get_norm_values(norm_family = 'imagenet'):
'''
Inputs
norm_family: String of norm_family
Returns
mean, std : tuple of 3 channel values
'''
if norm_family == 'imagenet':
mean, std = [0.485, 0.456, 0.406], [0.229, 0.224, 0.225]
else:
raise ValueError('Incorrect normalization family')
return mean, std
def save_args(args, log_path, argfile):
shutil.copy('train.py', log_path)
modelfiles = ospj(log_path, 'models')
try:
shutil.copy(argfile, log_path)
except:
print('Config exists')
try:
shutil.copytree('models/', modelfiles)
except:
print('Already exists')
with open(ospj(log_path,'args_all.yaml'),'w') as f:
yaml.dump(args, f, default_flow_style=False, allow_unicode=True)
with open(ospj(log_path, 'args.txt'), 'w') as f:
f.write('\n'.join(sys.argv[1:]))
class UnNormalizer:
'''
Unnormalize a given tensor using mean and std of a dataset family
Inputs
norm_family: String, dataset
tensor: Torch tensor
Outputs
tensor: Unnormalized tensor
'''
def __init__(self, norm_family = 'imagenet'):
self.mean, self.std = get_norm_values(norm_family=norm_family)
self.mean, self.std = torch.Tensor(self.mean).view(1, 3, 1, 1), torch.Tensor(self.std).view(1, 3, 1, 1)
def __call__(self, tensor):
return (tensor * self.std) + self.mean
def load_args(filename, args):
with open(filename, 'r') as stream:
data_loaded = yaml.safe_load(stream)
for key, group in data_loaded.items():
for key, val in group.items():
setattr(args, key, val)
================================================
FILE: domain_generalization/README.md
================================================
# BatchFormer for Domain Generalization
## Installation
Code is based on Transfer-Learning-Library (https://github.com/thuml/Transfer-Learning-Library.git). One shoud first install this library, or copy the files the files in this directory into Transfer-Learning-Library/examples/domain_generalization/image_classification/
This repository has changed a few lines of code based on Transfer-Learning-Library. Check the modifications by:
```
diff baseline.py <(curl https://raw.githubusercontent.com/thuml/Transfer-Learning-Library/master/examples/domain_generalization/image_classification/baseline.py)
```
I have also included Transfer-Learning-Library in this directory. You can find the examples in Transfer-Learning-Library/master/examples/domain_generalization/image_classification/ (e.g. baseline.py, irm.py)
In Transfer-Learning-Library/master/examples/domain_adaption/image_classification/, you can find corresponding code for Domain Adaption.
BatchFormer based on SWAD is provided in swad.
## Experiment and Results
For example, if you want to reproduce IRM on Office-Home, use the following script
```shell script
CUDA_VISIBLE_DEVICES=0 python irm.py data/office-home -d OfficeHome -s Ar Cl Rw -t Pr -a resnet50 --seed 0 --log logs/irm/OfficeHome_Pr --add_bt 1
```
You can find the corresponding *.pbs files in Transfer-Learning-Library/master/examples/domain_generalization/image_classification/ to get the running scripts for other methods. Please refer to Transfer-Learning-Library for More suggestions.
if you find this repository helpful, please consider cite:
@inproceedings{hou2022batch,
title={BatchFormer: Learning to Explore Sample Relationships for Robust Representation Learning},
author={Hou, Zhi and Yu, Baosheng and Tao, Dacheng},
booktitle={CVPR},
year={2022}
}
================================================
FILE: domain_generalization/Transfer-Learning-Library/.github/ISSUE_TEMPLATE/bug_report.md
================================================
---
name: Bug report
about: Create a report to help us improve
title: ''
labels: ''
assignees: ''
---
**Describe the bug**
A clear and concise description of what the bug is.
**To Reproduce**
Steps to reproduce the behavior:
1. Go to '...'
2. Click on '....'
3. Scroll down to '....'
4. See error
**Expected behavior**
A clear and concise description of what you expected to happen.
**Screenshots**
If applicable, add screenshots to help explain your problem.
**Desktop (please complete the following information):**
- OS: [e.g. iOS]
- Browser [e.g. chrome, safari]
- Version [e.g. 22]
**Smartphone (please complete the following information):**
- Device: [e.g. iPhone6]
- OS: [e.g. iOS8.1]
- Browser [e.g. stock browser, safari]
- Version [e.g. 22]
**Additional context**
Add any other context about the problem here.
================================================
FILE: domain_generalization/Transfer-Learning-Library/.github/ISSUE_TEMPLATE/custom.md
================================================
---
name: Custom issue template
about: Describe this issue template's purpose here.
title: ''
labels: ''
assignees: ''
---
================================================
FILE: domain_generalization/Transfer-Learning-Library/.github/ISSUE_TEMPLATE/feature_request.md
================================================
---
name: Feature request
about: Suggest an idea for this project
title: ''
labels: ''
assignees: ''
---
**Is your feature request related to a problem? Please describe.**
A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]
**Describe the solution you'd like**
A clear and concise description of what you want to happen.
**Describe alternatives you've considered**
A clear and concise description of any alternative solutions or features you've considered.
**Additional context**
Add any other context or screenshots about the feature request here.
================================================
FILE: domain_generalization/Transfer-Learning-Library/.gitignore
================================================
# Byte-compiled / optimized / DLL files
__pycache__/
*.py[cod]
*$py.class
# C extensions
*.so
# Distribution / packaging
.Python
build/
develop-eggs/
dist/
downloads/
eggs/
.eggs/
lib/
lib64/
parts/
sdist/
var/
wheels/
pip-wheel-metadata/
share/python-wheels/
*.egg-info/
.installed.cfg
*.egg
MANIFEST
# PyInstaller
# Usually these files are written by a python script from a template
# before PyInstaller builds the exe, so as to inject date/other infos into it.
*.manifest
*.spec
# Installer logs
pip-log.txt
pip-delete-this-directory.txt
# Unit test / coverage reports
htmlcov/
.tox/
.nox/
.coverage
.coverage.*
.cache
nosetests.xml
coverage.xml
*.cover
.hypothesis/
.pytest_cache/
# Translations
*.mo
*.pot
# Django stuff:
*.log
local_settings.py
db.sqlite3
db.sqlite3-journal
# Flask stuff:
instance/
.webassets-cache
# Scrapy stuff:
.scrapy
# Sphinx documentation
docs/_build/
docs/build/*
docs/pytorch_sphinx_theme/*
# PyBuilder
target/
# Jupyter Notebook
.ipynb_checkpoints
# IPython
profile_default/
ipython_config.py
# pyenv
.python-version
# pipenv
# According to pypa/pipenv#598, it is recommended to include Pipfile.lock in version control.
# However, in case of collaboration, if having platform-specific dependencies or dependencies
# having no cross-platform support, pipenv may install dependencies that don't work, or not
# install all needed dependencies.
#Pipfile.lock
# celery beat schedule file
celerybeat-schedule
# SageMath parsed files
*.sage.py
# Environments
.env
.venv
env/
venv/
ENV/
env.bak/
venv.bak/
# Spyder project settings
.spyderproject
.spyproject
# Rope project settings
.ropeproject
# mkdocs documentation
/site
# mypy
.mypy_cache/
.dmypy.json
dmypy.json
# Pyre type checker
.pyre/
.idea/
exp/*
trash/*
examples/domain_adaptation/digits/logs/*
examples/domain_adaptation/digits/data/*
================================================
FILE: domain_generalization/Transfer-Learning-Library/CONTRIBUTING.md
================================================
## Contributing to Transfer-Learning-Library
All kinds of contributions are welcome, including but not limited to the following.
- Fix typo or bugs
- Add documentation
- Add new features and components
### Workflow
1. fork and pull the latest Transfer-Learning-Library repository
2. checkout a new branch (do not use master branch for PRs)
3. commit your changes
4. create a PR
```{note}
If you plan to add some new features that involve large changes, it is encouraged to open an issue for discussion first.
```
================================================
FILE: domain_generalization/Transfer-Learning-Library/LICENSE
================================================
Copyright (c) 2018 The Python Packaging Authority
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
================================================
FILE: domain_generalization/Transfer-Learning-Library/README.md
================================================
## Introduction
*Transfer-Learn* is an open-source and well-documented library for Transfer Learning. It is based on pure PyTorch with high performance and friendly API. Our code is pythonic, and the design is consistent with torchvision. You can easily develop new algorithms, or readily apply existing algorithms.
The currently supported algorithms include:
##### [Domain Adaptation for Classification](/examples/domain_adaptation/image_classification)
- [Domain Adversarial Neural Network (DANN)](https://arxiv.org/abs/1505.07818)
- [Deep Adaptation Network (DAN)](https://arxiv.org/pdf/1502.02791)
- [Joint Adaptation Network (JAN)](https://arxiv.org/abs/1605.06636)
- [Adversarial Discriminative Domain Adaptation (ADDA)](https://arxiv.org/pdf/1702.05464.pdf)
- [Conditional Domain Adversarial Network (CDAN)](https://arxiv.org/abs/1705.10667)
- [Maximum Classifier Discrepancy (MCD)](https://arxiv.org/abs/1712.02560)
- [Adaptive Feature Norm (AFN)](https://arxiv.org/pdf/1811.07456v2.pdf)
- [Margin Disparity Discrepancy (MDD)](https://arxiv.org/abs/1904.05801)
- [Batch Spectral Penalization (BSP)](http://ise.thss.tsinghua.edu.cn/~mlong/doc/batch-spectral-penalization-icml19.pdf)
- [Minimum Class Confusion (MCC)](https://arxiv.org/abs/1912.03699)
##### [Partial Domain Adaptation](/examples/domain_adaptation/partial_domain_adaptation/)
- [Domain Adversarial Neural Network (DANN)](https://arxiv.org/abs/1505.07818)
- [Partial Adversarial Domain Adaptation (PADA)](https://arxiv.org/abs/1808.04205)
- [Importance Weighted Adversarial Nets (IWAN)](https://arxiv.org/abs/1803.09210)
- [Adaptive Feature Norm (AFN)](https://arxiv.org/pdf/1811.07456v2.pdf)
##### [Open-set Domain Adaptation](/examples/domain_adaptation/openset_domain_adaptation/)
- [Open Set Domain Adaptation (OSBP)](https://arxiv.org/abs/1804.10427)
##### [Domain Adaptation for Semantic Segmentation](/examples/domain_adaptation/semantic_segmentation/)
- [Cycle-Consistent Adversarial Networks (CycleGAN)](https://arxiv.org/pdf/1703.10593.pdf)
- [CyCADA: Cycle-Consistent Adversarial Domain Adaptation](https://arxiv.org/abs/1711.03213)
- [Adversarial Entropy Minimization (ADVENT)](https://arxiv.org/abs/1811.12833)
- [Fourier Domain Adaptation (FDA)](https://arxiv.org/abs/2004.05498)
##### [Domain Adaptation for Keypoint Detection](/examples/domain_adaptation/keypoint_detection)
- [Regressive Domain Adaptation for Unsupervised Keypoint Detection (RegDA, CVPR 2021)](http://ise.thss.tsinghua.edu.cn/~mlong/doc/regressive-domain-adaptation-cvpr21.pdf)
##### [Domain Adaptation for Person Re-identification](/examples/domain_adaptation/re_identification/)
- [Two at Once: Enhancing Learning and Generalization Capacities via IBN-Net (IBN-Net, ECCV 2018)](https://openaccess.thecvf.com/content_ECCV_2018/papers/Xingang_Pan_Two_at_Once_ECCV_2018_paper.pdf)
- [Mutual Mean-Teaching: Pseudo Label Refinery for Unsupervised Domain Adaptation on Person Re-identification (MMT, ICLR 2020)](https://arxiv.org/abs/2001.01526)
- [Similarity Preserving Generative Adversarial Network (SPGAN, CVPR 2018)](https://arxiv.org/pdf/1811.10551.pdf)
##### [Task Adaptation for Image Classification](/examples/task_adaptation/image_classification/)
- [Explicit inductive bias for transfer learning with convolutional networks
(L2-SP, ICML 2018)](https://arxiv.org/abs/1802.01483)
- [Catastrophic Forgetting Meets Negative Transfer: Batch Spectral Shrinkage for Safe Transfer Learning (BSS, NIPS 2019)](https://proceedings.neurips.cc/paper/2019/file/c6bff625bdb0393992c9d4db0c6bbe45-Paper.pdf)
- [DEep Learning Transfer using Fea- ture Map with Attention for convolutional networks (DELTA, ICLR 2019)](https://openreview.net/pdf?id=rkgbwsAcYm)
- [Co-Tuning for Transfer Learning (Co-Tuning, NIPS 2020)](http://ise.thss.tsinghua.edu.cn/~mlong/doc/co-tuning-for-transfer-learning-nips20.pdf)
- [Stochastic Normalization (StochNorm, NIPS 2020)](https://papers.nips.cc/paper/2020/file/bc573864331a9e42e4511de6f678aa83-Paper.pdf)
- [Learning Without Forgetting (LWF, ECCV 2016)](https://arxiv.org/abs/1606.09282)
- [Bi-tuning of Pre-trained Representations (Bi-Tuning)](https://arxiv.org/abs/2011.06182?utm_source=feedburner&utm_medium=feed&utm_campaign=Feed%3A+arxiv%2FQSXk+%28ExcitingAds%21+cs+updates+on+arXiv.org%29)
##### [Domain Generalization for Classification](/examples/domain_generalization/image_classification/)
- [Two at Once: Enhancing Learning and Generalization Capacities via IBN-Net (IBN-Net, ECCV 2018)](https://openaccess.thecvf.com/content_ECCV_2018/papers/Xingang_Pan_Two_at_Once_ECCV_2018_paper.pdf)
- [Domain Generalization with MixStyle (MixStyle, ICLR 2021)](https://arxiv.org/abs/2104.02008)
- [Learning to Generalize: Meta-Learning for Domain Generalization (MLDG, AAAI 2018)](https://arxiv.org/pdf/1710.03463.pdf)
- [Invariant Risk Minimization (IRM)](https://arxiv.org/abs/1907.02893)
- [Out-of-Distribution Generalization via Risk Extrapolation (VREx, ICML 2021)](https://arxiv.org/abs/2003.00688)
- [Distributionally Robust Neural Networks for Group Shifts: On the Importance of Regularization for Worst-Case Generalization(GroupDRO)](https://arxiv.org/abs/1911.08731)
- [Deep CORAL: Correlation Alignment for Deep Domain Adaptation (Deep Coral, ECCV 2016)](https://arxiv.org/abs/1607.01719)
##### [Domain Generalization for Person Re-identification](/examples/domain_generalization/re_identification/)
- [Two at Once: Enhancing Learning and Generalization Capacities via IBN-Net (IBN-Net, ECCV 2018)](https://openaccess.thecvf.com/content_ECCV_2018/papers/Xingang_Pan_Two_at_Once_ECCV_2018_paper.pdf)
- [Domain Generalization with MixStyle (MixStyle, ICLR 2021)](https://arxiv.org/abs/2104.02008)
We are planning to add
- DA for Object Detection
- TA for text classification
## Installation
To use ``dalib``, ``talib``, ``dglib``, and ``common`` in other places, you need to install Transfer-Learn,
```shell
python setup.py install
```
Note that we do not support *pip install* currently.
For flexible use and modification of Transfer-Learn, please git clone the library.
## Documentation
You can find the tutorial and API documentation on the website: [Documentation (please open in Firefox or Safari)](http://170.106.108.162/index.html). Note that this link is only for temporary use. You can also build the doc by yourself following the instructions in http://170.106.108.162/get_started/faq.html.
Also, we have examples in the directory `examples`. A typical usage is
```shell script
# Train a DANN on Office-31 Amazon -> Webcam task using ResNet 50.
# Assume you have put the datasets under the path `data/office-31`,
# or you are glad to download the datasets automatically from the Internet to this path
python dann.py data/office31 -d Office31 -s A -t W -a resnet50 --epochs 20
```
In the directory `examples`, you can find all the necessary running scripts to reproduce the benchmarks with specified hyper-parameters.
## Contributing
We appreciate all contributions. If you are planning to contribute back bug-fixes, please do so without any further discussion. If you plan to contribute new features, utility functions or extensions, please first open an issue and discuss the feature with us.
## Disclaimer on Datasets
This is a utility library that downloads and prepares public datasets. We do not host or distribute these datasets, vouch for their quality or fairness, or claim that you have licenses to use the dataset. It is your responsibility to determine whether you have permission to use the dataset under the dataset's license.
If you're a dataset owner and wish to update any part of it (description, citation, etc.), or do not want your dataset to be included in this library, please get in touch through a GitHub issue. Thanks for your contribution to the ML community!
## Contact
If you have any problem with our code or have some suggestions, including the future feature, feel free to contact
- Junguang Jiang (JiangJunguang1123@outlook.com)
- Baixu Chen (cbx_99_hasta@outlook.com)
- Mingsheng Long (longmingsheng@gmail.com)
or describe it in Issues.
For Q&A in Chinese, you can choose to ask questions here before sending an email. [迁移学习算法库答疑专区](https://zhuanlan.zhihu.com/p/248104070)
## Citation
If you use this toolbox or benchmark in your research, please cite this project.
```latex
@misc{dalib,
author = {Junguang Jiang, Baixu Chen, Bo Fu, Mingsheng Long},
title = {Transfer-Learning-library},
year = {2020},
publisher = {GitHub},
journal = {GitHub repository},
howpublished = {\url{https://github.com/thuml/Transfer-Learning-Library}},
}
```
## Acknowledgment
We would like to thank School of Software, Tsinghua University and The National Engineering Laboratory for Big Data Software for providing such an excellent ML research platform.
================================================
FILE: domain_generalization/Transfer-Learning-Library/common/__init__.py
================================================
__all__ = ['modules', 'utils', 'vision']
================================================
FILE: domain_generalization/Transfer-Learning-Library/common/loss/__init__.py
================================================
import torch.nn as nn
import torch.nn.functional as F
class KnowledgeDistillationLoss(nn.Module):
"""Knowledge Distillation Loss.
Args:
T (double): Temperature. Default: 1.
reduction (str, optional): Specifies the reduction to apply to the output:
``'none'`` | ``'mean'`` | ``'sum'``. ``'none'``: no reduction will be applied,
``'mean'``: the sum of the output will be divided by the number of
elements in the output, ``'sum'``: the output will be summed. Default: ``'batchmean'``
Inputs:
- y_student (tensor): logits output of the student
- y_teacher (tensor): logits output of the teacher
Shape:
- y_student: (minibatch, `num_classes`)
- y_teacher: (minibatch, `num_classes`)
"""
def __init__(self, T=1., reduction='batchmean'):
super(KnowledgeDistillationLoss, self).__init__()
self.T = T
self.kl = nn.KLDivLoss(reduction=reduction)
def forward(self, y_student, y_teacher):
""""""
return self.kl(F.log_softmax(y_student / self.T, dim=-1), F.softmax(y_teacher / self.T, dim=-1))
================================================
FILE: domain_generalization/Transfer-Learning-Library/common/modules/__init__.py
================================================
from .classifier import *
from .regressor import *
__all__ = ['classifier', 'regressor']
================================================
FILE: domain_generalization/Transfer-Learning-Library/common/modules/classifier.py
================================================
"""
@author: Junguang Jiang
@contact: JiangJunguang1123@outlook.com
"""
from typing import Tuple, Optional, List, Dict
import torch.nn as nn
import torch
__all__ = ['Classifier']
class Classifier(nn.Module):
"""A generic Classifier class for domain adaptation.
Args:
backbone (torch.nn.Module): Any backbone to extract 2-d features from data
num_classes (int): Number of classes
bottleneck (torch.nn.Module, optional): Any bottleneck layer. Use no bottleneck by default
bottleneck_dim (int, optional): Feature dimension of the bottleneck layer. Default: -1
head (torch.nn.Module, optional): Any classifier head. Use :class:`torch.nn.Linear` by default
finetune (bool): Whether finetune the classifier or train from scratch. Default: True
.. note::
Different classifiers are used in different domain adaptation algorithms to achieve better accuracy
respectively, and we provide a suggested `Classifier` for different algorithms.
Remember they are not the core of algorithms. You can implement your own `Classifier` and combine it with
the domain adaptation algorithm in this algorithm library.
.. note::
The learning rate of this classifier is set 10 times to that of the feature extractor for better accuracy
by default. If you have other optimization strategies, please over-ride :meth:`~Classifier.get_parameters`.
Inputs:
- x (tensor): input data fed to `backbone`
Outputs:
- predictions: classifier's predictions
- features: features after `bottleneck` layer and before `head` layer
Shape:
- Inputs: (minibatch, *) where * means, any number of additional dimensions
- predictions: (minibatch, `num_classes`)
- features: (minibatch, `features_dim`)
"""
def __init__(self, backbone: nn.Module, num_classes: int, bottleneck: Optional[nn.Module] = None,
bottleneck_dim: Optional[int] = -1, head: Optional[nn.Module] = None, finetune=True, pool_layer=None):
super(Classifier, self).__init__()
self.backbone = backbone
self.num_classes = num_classes
if pool_layer is None:
self.pool_layer = nn.Sequential(
nn.AdaptiveAvgPool2d(output_size=(1, 1)),
nn.Flatten()
)
else:
self.pool_layer = pool_layer
if bottleneck is None:
self.bottleneck = nn.Identity()
self._features_dim = backbone.out_features
else:
self.bottleneck = bottleneck
assert bottleneck_dim > 0
self._features_dim = bottleneck_dim
if head is None:
self.head = nn.Linear(self._features_dim, num_classes)
else:
self.head = head
self.finetune = finetune
@property
def features_dim(self) -> int:
"""The dimension of features before the final `head` layer"""
return self._features_dim
def forward(self, x: torch.Tensor) -> Tuple[torch.Tensor, torch.Tensor]:
""""""
f = self.pool_layer(self.backbone(x))
f = self.bottleneck(f)
predictions = self.head(f)
if self.training:
return predictions, f
else:
return predictions
def get_parameters(self, base_lr=1.0) -> List[Dict]:
"""A parameter list which decides optimization hyper-parameters,
such as the relative learning rate of each layer
"""
params = [
{"params": self.backbone.parameters(), "lr": 0.1 * base_lr if self.finetune else 1.0 * base_lr},
{"params": self.bottleneck.parameters(), "lr": 1.0 * base_lr},
{"params": self.head.parameters(), "lr": 1.0 * base_lr},
]
return params
class ImageClassifier(Classifier):
pass
================================================
FILE: domain_generalization/Transfer-Learning-Library/common/modules/regressor.py
================================================
"""
@author: Junguang Jiang
@contact: JiangJunguang1123@outlook.com
"""
from typing import Tuple, Optional, List, Dict
import torch.nn as nn
import torch
__all__ = ['Regressor']
class Regressor(nn.Module):
"""A generic Regressor class for domain adaptation.
Args:
backbone (torch.nn.Module): Any backbone to extract 2-d features from data
num_factors (int): Number of factors
bottleneck (torch.nn.Module, optional): Any bottleneck layer. Use no bottleneck by default
bottleneck_dim (int, optional): Feature dimension of the bottleneck layer. Default: -1
head (torch.nn.Module, optional): Any classifier head. Use `nn.Linear` by default
finetune (bool): Whether finetune the classifier or train from scratch. Default: True
.. note::
The learning rate of this regressor is set 10 times to that of the feature extractor for better accuracy
by default. If you have other optimization strategies, please over-ride :meth:`~Regressor.get_parameters`.
Inputs:
- x (tensor): input data fed to `backbone`
Outputs:
- predictions: regressor's predictions
- features: features after `bottleneck` layer and before `head` layer
Shape:
- Inputs: (minibatch, *) where * means, any number of additional dimensions
- predictions: (minibatch, `num_factors`)
- features: (minibatch, `features_dim`)
"""
def __init__(self, backbone: nn.Module, num_factors: int, bottleneck: Optional[nn.Module] = None,
bottleneck_dim=-1, head: Optional[nn.Module] = None, finetune=True):
super(Regressor, self).__init__()
self.backbone = backbone
self.num_factors = num_factors
if bottleneck is None:
feature_dim = backbone.out_features
self.bottleneck = nn.Sequential(
nn.Conv2d(feature_dim, feature_dim, kernel_size=3, stride=1, padding=1),
nn.BatchNorm2d(feature_dim, feature_dim),
nn.ReLU(),
nn.AdaptiveAvgPool2d(output_size=(1, 1)),
nn.Flatten()
)
self._features_dim = feature_dim
else:
self.bottleneck = bottleneck
assert bottleneck_dim > 0
self._features_dim = bottleneck_dim
if head is None:
self.head = nn.Sequential(
nn.Linear(self._features_dim, num_factors),
nn.Sigmoid()
)
else:
self.head = head
self.finetune = finetune
@property
def features_dim(self) -> int:
"""The dimension of features before the final `head` layer"""
return self._features_dim
def forward(self, x: torch.Tensor) -> Tuple[torch.Tensor, torch.Tensor]:
""""""
f = self.backbone(x)
f = self.bottleneck(f)
predictions = self.head(f)
if self.training:
return predictions, f
else:
return predictions
def get_parameters(self, base_lr=1.0) -> List[Dict]:
"""A parameter list which decides optimization hyper-parameters,
such as the relative learning rate of each layer
"""
params = [
{"params": self.backbone.parameters(), "lr": 0.1 * base_lr if self.finetune else 1.0 * base_lr},
{"params": self.bottleneck.parameters(), "lr": 1.0 * base_lr},
{"params": self.head.parameters(), "lr": 1.0 * base_lr},
]
return params
================================================
FILE: domain_generalization/Transfer-Learning-Library/common/utils/__init__.py
================================================
from .logger import CompleteLogger
from .meter import *
from .data import ForeverDataIterator
__all__ = ['metric', 'analysis', 'meter', 'data', 'logger']
================================================
FILE: domain_generalization/Transfer-Learning-Library/common/utils/analysis/__init__.py
================================================
import torch
from torch.utils.data import DataLoader
import torch.nn as nn
import tqdm
def collect_feature(data_loader: DataLoader, feature_extractor: nn.Module,
device: torch.device, max_num_features=None) -> torch.Tensor:
"""
Fetch data from `data_loader`, and then use `feature_extractor` to collect features
Args:
data_loader (torch.utils.data.DataLoader): Data loader.
feature_extractor (torch.nn.Module): A feature extractor.
device (torch.device)
max_num_features (int): The max number of features to return
Returns:
Features in shape (min(len(data_loader), max_num_features * mini-batch size), :math:`|\mathcal{F}|`).
"""
feature_extractor.eval()
all_features = []
with torch.no_grad():
for i, (images, target) in enumerate(tqdm.tqdm(data_loader)):
if max_num_features is not None and i >= max_num_features:
break
images = images.to(device)
feature = feature_extractor(images).cpu()
all_features.append(feature)
return torch.cat(all_features, dim=0)
================================================
FILE: domain_generalization/Transfer-Learning-Library/common/utils/analysis/a_distance.py
================================================
"""
@author: Junguang Jiang
@contact: JiangJunguang1123@outlook.com
"""
from torch.utils.data import TensorDataset
import torch
import torch.nn as nn
import torch.nn.functional as F
from torch.utils.data import DataLoader
from torch.optim import SGD
from ..meter import AverageMeter
from ..metric import binary_accuracy
class ANet(nn.Module):
def __init__(self, in_feature):
super(ANet, self).__init__()
self.layer = nn.Linear(in_feature, 1)
self.sigmoid = nn.Sigmoid()
def forward(self, x):
x = self.layer(x)
x = self.sigmoid(x)
return x
def calculate(source_feature: torch.Tensor, target_feature: torch.Tensor,
device, progress=True, training_epochs=10):
"""
Calculate the :math:`\mathcal{A}`-distance, which is a measure for distribution discrepancy.
The definition is :math:`dist_\mathcal{A} = 2 (1-2\epsilon)`, where :math:`\epsilon` is the
test error of a classifier trained to discriminate the source from the target.
Args:
source_feature (tensor): features from source domain in shape :math:`(minibatch, F)`
target_feature (tensor): features from target domain in shape :math:`(minibatch, F)`
device (torch.device)
progress (bool): if True, displays a the progress of training A-Net
training_epochs (int): the number of epochs when training the classifier
Returns:
:math:`\mathcal{A}`-distance
"""
source_label = torch.ones((source_feature.shape[0], 1))
target_label = torch.zeros((target_feature.shape[0], 1))
feature = torch.cat([source_feature, target_feature], dim=0)
label = torch.cat([source_label, target_label], dim=0)
dataset = TensorDataset(feature, label)
length = len(dataset)
train_size = int(0.8 * length)
val_size = length - train_size
train_set, val_set = torch.utils.data.random_split(dataset, [train_size, val_size])
train_loader = DataLoader(train_set, batch_size=2, shuffle=True)
val_loader = DataLoader(val_set, batch_size=8, shuffle=False)
anet = ANet(feature.shape[1]).to(device)
optimizer = SGD(anet.parameters(), lr=0.01)
a_distance = 2.0
for epoch in range(training_epochs):
anet.train()
for (x, label) in train_loader:
x = x.to(device)
label = label.to(device)
anet.zero_grad()
y = anet(x)
loss = F.binary_cross_entropy(y, label)
loss.backward()
optimizer.step()
anet.eval()
meter = AverageMeter("accuracy", ":4.2f")
with torch.no_grad():
for (x, label) in val_loader:
x = x.to(device)
label = label.to(device)
y = anet(x)
acc = binary_accuracy(y, label)
meter.update(acc, x.shape[0])
error = 1 - meter.avg / 100
a_distance = 2 * (1 - 2 * error)
if progress:
print("epoch {} accuracy: {} A-dist: {}".format(epoch, meter.avg, a_distance))
return a_distance
================================================
FILE: domain_generalization/Transfer-Learning-Library/common/utils/analysis/tsne.py
================================================
"""
@author: Junguang Jiang
@contact: JiangJunguang1123@outlook.com
"""
import torch
import matplotlib
matplotlib.use('Agg')
from sklearn.manifold import TSNE
import numpy as np
import matplotlib.pyplot as plt
import matplotlib.colors as col
def visualize(source_feature: torch.Tensor, target_feature: torch.Tensor,
filename: str, source_color='r', target_color='b'):
"""
Visualize features from different domains using t-SNE.
Args:
source_feature (tensor): features from source domain in shape :math:`(minibatch, F)`
target_feature (tensor): features from target domain in shape :math:`(minibatch, F)`
filename (str): the file name to save t-SNE
source_color (str): the color of the source features. Default: 'r'
target_color (str): the color of the target features. Default: 'b'
"""
source_feature = source_feature.numpy()
target_feature = target_feature.numpy()
features = np.concatenate([source_feature, target_feature], axis=0)
# map features to 2-d using TSNE
X_tsne = TSNE(n_components=2, random_state=33).fit_transform(features)
# domain labels, 1 represents source while 0 represents target
domains = np.concatenate((np.ones(len(source_feature)), np.zeros(len(target_feature))))
# visualize using matplotlib
fig, ax = plt.subplots(figsize=(10, 10))
ax.spines['top'].set_visible(False)
ax.spines['right'].set_visible(False)
ax.spines['bottom'].set_visible(False)
ax.spines['left'].set_visible(False)
plt.scatter(X_tsne[:, 0], X_tsne[:, 1], c=domains, cmap=col.ListedColormap([target_color, source_color]), s=20)
plt.xticks([])
plt.yticks([])
plt.savefig(filename)
================================================
FILE: domain_generalization/Transfer-Learning-Library/common/utils/data.py
================================================
"""
@author: Junguang Jiang
@contact: JiangJunguang1123@outlook.com
"""
import itertools
import random
import numpy as np
import torch
from torch.utils.data import Sampler
from torch.utils.data import DataLoader, Dataset
from typing import TypeVar, Iterable, Dict, List
T_co = TypeVar('T_co', covariant=True)
T = TypeVar('T')
def send_to_device(tensor, device):
"""
Recursively sends the elements in a nested list/tuple/dictionary of tensors to a given device.
Args:
tensor (nested list/tuple/dictionary of :obj:`torch.Tensor`):
The data to send to a given device.
device (:obj:`torch.device`):
The device to send the data to
Returns:
The same data structure as :obj:`tensor` with all tensors sent to the proper device.
"""
if isinstance(tensor, (list, tuple)):
return type(tensor)(send_to_device(t, device) for t in tensor)
elif isinstance(tensor, dict):
return type(tensor)({k: send_to_device(v, device) for k, v in tensor.items()})
elif not hasattr(tensor, "to"):
return tensor
return tensor.to(device)
class ForeverDataIterator:
r"""A data iterator that will never stop producing data"""
def __init__(self, data_loader: DataLoader, device=None):
self.data_loader = data_loader
self.iter = iter(self.data_loader)
self.device = device
def __next__(self):
try:
data = next(self.iter)
if self.device is not None:
data = send_to_device(data, self.device)
except StopIteration:
self.iter = iter(self.data_loader)
data = next(self.iter)
if self.device is not None:
data = send_to_device(data, self.device)
return data
def __len__(self):
return len(self.data_loader)
class RandomMultipleGallerySampler(Sampler):
r"""Sampler from `In defense of the Triplet Loss for Person Re-Identification
(ICCV 2017) `_. Assume there are :math:`N` identities in the dataset, this
implementation simply samples :math:`K` images for every identity to form an iter of size :math:`N\times K`. During
training, we will call ``__iter__`` method of pytorch dataloader once we reach a ``StopIteration``, this guarantees
every image in the dataset will eventually be selected and we are not wasting any training data.
Args:
dataset(list): each element of this list is a tuple (image_path, person_id, camera_id)
num_instances(int, optional): number of images to sample for every identity (:math:`K` here)
"""
def __init__(self, dataset, num_instances=4):
super(RandomMultipleGallerySampler, self).__init__(dataset)
self.dataset = dataset
self.num_instances = num_instances
self.idx_to_pid = {}
self.cid_list_per_pid = {}
self.idx_list_per_pid = {}
for idx, (_, pid, cid) in enumerate(dataset):
if pid not in self.cid_list_per_pid:
self.cid_list_per_pid[pid] = []
self.idx_list_per_pid[pid] = []
self.idx_to_pid[idx] = pid
self.cid_list_per_pid[pid].append(cid)
self.idx_list_per_pid[pid].append(idx)
self.pid_list = list(self.idx_list_per_pid.keys())
self.num_samples = len(self.pid_list)
def __len__(self):
return self.num_samples * self.num_instances
def __iter__(self):
def select_idxes(element_list, target_element):
assert isinstance(element_list, list)
return [i for i, element in enumerate(element_list) if element != target_element]
pid_idxes = torch.randperm(len(self.pid_list)).tolist()
final_idxes = []
for perm_id in pid_idxes:
i = random.choice(self.idx_list_per_pid[self.pid_list[perm_id]])
_, _, cid = self.dataset[i]
final_idxes.append(i)
pid_i = self.idx_to_pid[i]
cid_list = self.cid_list_per_pid[pid_i]
idx_list = self.idx_list_per_pid[pid_i]
selected_cid_list = select_idxes(cid_list, cid)
if selected_cid_list:
if len(selected_cid_list) >= self.num_instances:
cid_idxes = np.random.choice(selected_cid_list, size=self.num_instances - 1, replace=False)
else:
cid_idxes = np.random.choice(selected_cid_list, size=self.num_instances - 1, replace=True)
for cid_idx in cid_idxes:
final_idxes.append(idx_list[cid_idx])
else:
selected_idxes = select_idxes(idx_list, i)
if not selected_idxes:
continue
if len(selected_idxes) >= self.num_instances:
pid_idxes = np.random.choice(selected_idxes, size=self.num_instances - 1, replace=False)
else:
pid_idxes = np.random.choice(selected_idxes, size=self.num_instances - 1, replace=True)
for pid_idx in pid_idxes:
final_idxes.append(idx_list[pid_idx])
return iter(final_idxes)
class CombineDataset(Dataset[T_co]):
r"""Dataset as a combination of multiple datasets.
The element of each dataset must be a list, and the i-th element of the combined dataset
is a list splicing of the i-th element of each sub dataset.
The length of the combined dataset is the minimum of the lengths of all sub datasets.
Arguments:
datasets (sequence): List of datasets to be concatenated
"""
def __init__(self, datasets: Iterable[Dataset]) -> None:
super(CombineDataset, self).__init__()
# Cannot verify that datasets is Sized
assert len(datasets) > 0, 'datasets should not be an empty iterable' # type: ignore
self.datasets = list(datasets)
def __len__(self):
return min([len(d) for d in self.datasets])
def __getitem__(self, idx):
return list(itertools.chain(*[d[idx] for d in self.datasets]))
def concatenate(tensors):
"""concatenate multiple batches into one batch.
``tensors`` can be :class:`torch.Tensor`, List or Dict, but they must be the same data format.
"""
if isinstance(tensors[0], torch.Tensor):
return torch.cat(tensors, dim=0)
elif isinstance(tensors[0], List):
ret = []
for i in range(len(tensors[0])):
ret.append(concatenate([t[i] for t in tensors]))
return ret
elif isinstance(tensors[0], Dict):
ret = dict()
for k in tensors[0].keys():
ret[k] = concatenate([t[k] for t in tensors])
return ret
================================================
FILE: domain_generalization/Transfer-Learning-Library/common/utils/logger.py
================================================
"""
@author: Junguang Jiang
@contact: JiangJunguang1123@outlook.com
"""
import os
import sys
import time
class TextLogger(object):
"""Writes stream output to external text file.
Args:
filename (str): the file to write stream output
stream: the stream to read from. Default: sys.stdout
"""
def __init__(self, filename, stream=sys.stdout):
self.terminal = stream
self.log = open(filename, 'a')
def write(self, message):
self.terminal.write(message)
self.log.write(message)
self.flush()
def flush(self):
self.terminal.flush()
self.log.flush()
def close(self):
self.terminal.close()
self.log.close()
class CompleteLogger:
"""
A useful logger that
- writes outputs to files and displays them on the console at the same time.
- manages the directory of checkpoints and debugging images.
Args:
root (str): the root directory of logger
phase (str): the phase of training.
"""
def __init__(self, root, phase='train'):
self.root = root
self.phase = phase
self.visualize_directory = os.path.join(self.root, "visualize")
self.checkpoint_directory = os.path.join(self.root, "checkpoints")
self.epoch = 0
os.makedirs(self.root, exist_ok=True)
os.makedirs(self.visualize_directory, exist_ok=True)
os.makedirs(self.checkpoint_directory, exist_ok=True)
# redirect std out
now = time.strftime("%Y-%m-%d-%H_%M_%S", time.localtime(time.time()))
log_filename = os.path.join(self.root, "{}-{}.txt".format(phase, now))
if os.path.exists(log_filename):
os.remove(log_filename)
self.logger = TextLogger(log_filename)
sys.stdout = self.logger
sys.stderr = self.logger
if phase != 'train':
self.set_epoch(phase)
def set_epoch(self, epoch):
"""Set the epoch number. Please use it during training."""
os.makedirs(os.path.join(self.visualize_directory, str(epoch)), exist_ok=True)
self.epoch = epoch
def _get_phase_or_epoch(self):
if self.phase == 'train':
return str(self.epoch)
else:
return self.phase
def get_image_path(self, filename: str):
"""
Get the full image path for a specific filename
"""
return os.path.join(self.visualize_directory, self._get_phase_or_epoch(), filename)
def get_checkpoint_path(self, name=None):
"""
Get the full checkpoint path.
Args:
name (optional): the filename (without file extension) to save checkpoint.
If None, when the phase is ``train``, checkpoint will be saved to ``{epoch}.pth``.
Otherwise, will be saved to ``{phase}.pth``.
"""
if name is None:
name = self._get_phase_or_epoch()
name = str(name)
return os.path.join(self.checkpoint_directory, name + ".pth")
def close(self):
self.logger.close()
================================================
FILE: domain_generalization/Transfer-Learning-Library/common/utils/meter.py
================================================
"""
@author: Junguang Jiang
@contact: JiangJunguang1123@outlook.com
"""
from typing import Optional, List
class AverageMeter(object):
r"""Computes and stores the average and current value.
Examples::
>>> # Initialize a meter to record loss
>>> losses = AverageMeter()
>>> # Update meter after every minibatch update
>>> losses.update(loss_value, batch_size)
"""
def __init__(self, name: str, fmt: Optional[str] = ':f'):
self.name = name
self.fmt = fmt
self.reset()
def reset(self):
self.val = 0
self.avg = 0
self.sum = 0
self.count = 0
def update(self, val, n=1):
self.val = val
self.sum += val * n
self.count += n
if self.count > 0:
self.avg = self.sum / self.count
def __str__(self):
fmtstr = '{name} {val' + self.fmt + '} ({avg' + self.fmt + '})'
return fmtstr.format(**self.__dict__)
class AverageMeterDict(object):
def __init__(self, names: List, fmt: Optional[str] = ':f'):
self.dict = {
name: AverageMeter(name, fmt) for name in names
}
def reset(self):
for meter in self.dict.values():
meter.reset()
def update(self, accuracies, n=1):
for name, acc in accuracies.items():
self.dict[name].update(acc, n)
def average(self):
return {
name: meter.avg for name, meter in self.dict.items()
}
def __getitem__(self, item):
return self.dict[item]
class Meter(object):
"""Computes and stores the current value."""
def __init__(self, name: str, fmt: Optional[str] = ':f'):
self.name = name
self.fmt = fmt
self.reset()
def reset(self):
self.val = 0
def update(self, val):
self.val = val
def __str__(self):
fmtstr = '{name} {val' + self.fmt + '}'
return fmtstr.format(**self.__dict__)
class ProgressMeter(object):
def __init__(self, num_batches, meters, prefix=""):
self.batch_fmtstr = self._get_batch_fmtstr(num_batches)
self.meters = meters
self.prefix = prefix
def display(self, batch):
entries = [self.prefix + self.batch_fmtstr.format(batch)]
entries += [str(meter) for meter in self.meters]
print('\t'.join(entries))
def _get_batch_fmtstr(self, num_batches):
num_digits = len(str(num_batches // 1))
fmt = '{:' + str(num_digits) + 'd}'
return '[' + fmt + '/' + fmt.format(num_batches) + ']'
================================================
FILE: domain_generalization/Transfer-Learning-Library/common/utils/metric/__init__.py
================================================
import torch
import prettytable
__all__ = ['keypoint_detection']
def binary_accuracy(output: torch.Tensor, target: torch.Tensor) -> float:
"""Computes the accuracy for binary classification"""
with torch.no_grad():
batch_size = target.size(0)
pred = (output >= 0.5).float().t().view(-1)
correct = pred.eq(target.view(-1)).float().sum()
correct.mul_(100. / batch_size)
return correct
def accuracy(output, target, topk=(1,)):
r"""
Computes the accuracy over the k top predictions for the specified values of k
Args:
output (tensor): Classification outputs, :math:`(N, C)` where `C = number of classes`
target (tensor): :math:`(N)` where each value is :math:`0 \leq \text{targets}[i] \leq C-1`
topk (sequence[int]): A list of top-N number.
Returns:
Top-N accuracies (N :math:`\in` topK).
"""
with torch.no_grad():
maxk = max(topk)
batch_size = target.size(0)
_, pred = output.topk(maxk, 1, True, True)
pred = pred.t()
correct = pred.eq(target[None])
res = []
for k in topk:
correct_k = correct[:k].flatten().sum(dtype=torch.float32)
res.append(correct_k * (100.0 / batch_size))
return res
class ConfusionMatrix(object):
def __init__(self, num_classes):
self.num_classes = num_classes
self.mat = None
def update(self, target, output):
"""
Update confusion matrix.
Args:
target: ground truth
output: predictions of models
Shape:
- target: :math:`(minibatch, C)` where C means the number of classes.
- output: :math:`(minibatch, C)` where C means the number of classes.
"""
n = self.num_classes
if self.mat is None:
self.mat = torch.zeros((n, n), dtype=torch.int64, device=target.device)
with torch.no_grad():
k = (target >= 0) & (target < n)
inds = n * target[k].to(torch.int64) + output[k]
self.mat += torch.bincount(inds, minlength=n**2).reshape(n, n)
def reset(self):
self.mat.zero_()
def compute(self):
"""compute global accuracy, per-class accuracy and per-class IoU"""
h = self.mat.float()
acc_global = torch.diag(h).sum() / h.sum()
acc = torch.diag(h) / h.sum(1)
iu = torch.diag(h) / (h.sum(1) + h.sum(0) - torch.diag(h))
return acc_global, acc, iu
# def reduce_from_all_processes(self):
# if not torch.distributed.is_available():
# return
# if not torch.distributed.is_initialized():
# return
# torch.distributed.barrier()
# torch.distributed.all_reduce(self.mat)
def __str__(self):
acc_global, acc, iu = self.compute()
return (
'global correct: {:.1f}\n'
'average row correct: {}\n'
'IoU: {}\n'
'mean IoU: {:.1f}').format(
acc_global.item() * 100,
['{:.1f}'.format(i) for i in (acc * 100).tolist()],
['{:.1f}'.format(i) for i in (iu * 100).tolist()],
iu.mean().item() * 100)
def format(self, classes: list):
"""Get the accuracy and IoU for each class in the table format"""
acc_global, acc, iu = self.compute()
table = prettytable.PrettyTable(["class", "acc", "iou"])
for i, class_name, per_acc, per_iu in zip(range(len(classes)), classes, (acc * 100).tolist(), (iu * 100).tolist()):
table.add_row([class_name, per_acc, per_iu])
return 'global correct: {:.1f}\nmean correct:{:.1f}\nmean IoU: {:.1f}\n{}'.format(
acc_global.item() * 100, acc.mean().item() * 100, iu.mean().item() * 100, table.get_string())
================================================
FILE: domain_generalization/Transfer-Learning-Library/common/utils/metric/keypoint_detection.py
================================================
"""
@author: Junguang Jiang
@contact: JiangJunguang1123@outlook.com
"""
# TODO: add documentation
import numpy as np
def get_max_preds(batch_heatmaps):
'''
get predictions from score maps
heatmaps: numpy.ndarray([batch_size, num_joints, height, width])
'''
assert isinstance(batch_heatmaps, np.ndarray), \
'batch_heatmaps should be numpy.ndarray'
assert batch_heatmaps.ndim == 4, 'batch_images should be 4-ndim'
batch_size = batch_heatmaps.shape[0]
num_joints = batch_heatmaps.shape[1]
width = batch_heatmaps.shape[3]
heatmaps_reshaped = batch_heatmaps.reshape((batch_size, num_joints, -1))
idx = np.argmax(heatmaps_reshaped, 2)
maxvals = np.amax(heatmaps_reshaped, 2)
maxvals = maxvals.reshape((batch_size, num_joints, 1))
idx = idx.reshape((batch_size, num_joints, 1))
preds = np.tile(idx, (1, 1, 2)).astype(np.float32)
preds[:, :, 0] = (preds[:, :, 0]) % width
preds[:, :, 1] = np.floor((preds[:, :, 1]) / width)
pred_mask = np.tile(np.greater(maxvals, 0.0), (1, 1, 2))
pred_mask = pred_mask.astype(np.float32)
preds *= pred_mask
return preds, maxvals
def calc_dists(preds, target, normalize):
preds = preds.astype(np.float32)
target = target.astype(np.float32)
dists = np.zeros((preds.shape[1], preds.shape[0]))
for n in range(preds.shape[0]):
for c in range(preds.shape[1]):
if target[n, c, 0] > 1 and target[n, c, 1] > 1:
normed_preds = preds[n, c, :] / normalize[n]
normed_targets = target[n, c, :] / normalize[n]
dists[c, n] = np.linalg.norm(normed_preds - normed_targets)
else:
dists[c, n] = -1
return dists
def dist_acc(dists, thr=0.5):
''' Return percentage below threshold while ignoring values with a -1 '''
dist_cal = np.not_equal(dists, -1)
num_dist_cal = dist_cal.sum()
if num_dist_cal > 0:
return np.less(dists[dist_cal], thr).sum() * 1.0 / num_dist_cal
else:
return -1
def accuracy(output, target, hm_type='gaussian', thr=0.5):
'''
Calculate accuracy according to PCK,
but uses ground truth heatmap rather than x,y locations
First value to be returned is average accuracy across 'idxs',
followed by individual accuracies
'''
idx = list(range(output.shape[1]))
norm = 1.0
if hm_type == 'gaussian':
pred, _ = get_max_preds(output)
target, _ = get_max_preds(target)
h = output.shape[2]
w = output.shape[3]
norm = np.ones((pred.shape[0], 2)) * np.array([h, w]) / 10
dists = calc_dists(pred, target, norm)
acc = np.zeros(len(idx))
avg_acc = 0
cnt = 0
for i in range(len(idx)):
acc[i] = dist_acc(dists[idx[i]], thr)
if acc[i] >= 0:
avg_acc = avg_acc + acc[i]
cnt += 1
avg_acc = avg_acc / cnt if cnt != 0 else 0
return acc, avg_acc, cnt, pred
================================================
FILE: domain_generalization/Transfer-Learning-Library/common/utils/metric/reid.py
================================================
# TODO: add documentation
"""
Modified from https://github.com/yxgeee/MMT
@author: Baixu Chen
@contact: cbx_99_hasta@outlook.com
"""
import os
import os.path as osp
from collections import defaultdict
import time
import numpy as np
import torch
import torch.nn.functional as F
from sklearn.metrics import average_precision_score
from common.utils.meter import AverageMeter, ProgressMeter
def unique_sample(ids_dict, num):
"""Randomly choose one instance for each person id, these instances will not be selected again"""
mask = np.zeros(num, dtype=np.bool)
for _, indices in ids_dict.items():
i = np.random.choice(indices)
mask[i] = True
return mask
def cmc(dist_mat, query_ids, gallery_ids, query_cams, gallery_cams, topk=100, separate_camera_set=False,
single_gallery_shot=False, first_match_break=False):
"""Compute Cumulative Matching Characteristics (CMC)"""
dist_mat = dist_mat.cpu().numpy()
m, n = dist_mat.shape
query_ids = np.asarray(query_ids)
gallery_ids = np.asarray(gallery_ids)
query_cams = np.asarray(query_cams)
gallery_cams = np.asarray(gallery_cams)
# Sort and find correct matches
indices = np.argsort(dist_mat, axis=1)
matches = (gallery_ids[indices] == query_ids[:, np.newaxis])
# Compute CMC for each query
ret = np.zeros(topk)
num_valid_queries = 0
for i in range(m):
# Filter out the same id and same camera
valid = ((gallery_ids[indices[i]] != query_ids[i]) |
(gallery_cams[indices[i]] != query_cams[i]))
if separate_camera_set:
# Filter out samples from same camera
valid &= (gallery_cams[indices[i]] != query_cams[i])
if not np.any(matches[i, valid]): continue
if single_gallery_shot:
repeat = 10
gids = gallery_ids[indices[i][valid]]
inds = np.where(valid)[0]
ids_dict = defaultdict(list)
for j, x in zip(inds, gids):
ids_dict[x].append(j)
else:
repeat = 1
for _ in range(repeat):
if single_gallery_shot:
# Randomly choose one instance for each id
sampled = (valid & unique_sample(ids_dict, len(valid)))
index = np.nonzero(matches[i, sampled])[0]
else:
index = np.nonzero(matches[i, valid])[0]
delta = 1. / (len(index) * repeat)
for j, k in enumerate(index):
if k - j >= topk: break
if first_match_break:
ret[k - j] += 1
break
ret[k - j] += delta
num_valid_queries += 1
if num_valid_queries == 0:
raise RuntimeError("No valid query")
return ret.cumsum() / num_valid_queries
def mean_ap(dist_mat, query_ids, gallery_ids, query_cams, gallery_cams):
"""Compute mean average precision (mAP)"""
dist_mat = dist_mat.cpu().numpy()
m, n = dist_mat.shape
query_ids = np.asarray(query_ids)
gallery_ids = np.asarray(gallery_ids)
query_cams = np.asarray(query_cams)
gallery_cams = np.asarray(gallery_cams)
# Sort and find correct matches
indices = np.argsort(dist_mat, axis=1)
matches = (gallery_ids[indices] == query_ids[:, np.newaxis])
# Compute AP for each query
aps = []
for i in range(m):
# Filter out the same id and same camera
valid = ((gallery_ids[indices[i]] != query_ids[i]) |
(gallery_cams[indices[i]] != query_cams[i]))
y_true = matches[i, valid]
y_score = -dist_mat[i][indices[i]][valid]
if not np.any(y_true): continue
aps.append(average_precision_score(y_true, y_score))
if len(aps) == 0:
raise RuntimeError("No valid query")
return np.mean(aps)
def re_ranking(q_g_dist, q_q_dist, g_g_dist, k1=20, k2=6, lambda_value=0.3):
"""Perform re-ranking with distance matrix between query and gallery images `q_g_dist`, distance matrix between
query and query images `q_q_dist` and distance matrix between gallery and gallery images `g_g_dist`.
"""
q_g_dist = q_g_dist.cpu().numpy()
q_q_dist = q_q_dist.cpu().numpy()
g_g_dist = g_g_dist.cpu().numpy()
original_dist = np.concatenate(
[np.concatenate([q_q_dist, q_g_dist], axis=1),
np.concatenate([q_g_dist.T, g_g_dist], axis=1)],
axis=0)
original_dist = np.power(original_dist, 2).astype(np.float32)
original_dist = np.transpose(1. * original_dist / np.max(original_dist, axis=0))
V = np.zeros_like(original_dist).astype(np.float32)
initial_rank = np.argsort(original_dist).astype(np.int32)
query_num = q_g_dist.shape[0]
gallery_num = q_g_dist.shape[0] + q_g_dist.shape[1]
all_num = gallery_num
for i in range(all_num):
# k-reciprocal neighbors
forward_k_neigh_index = initial_rank[i, :k1 + 1]
backward_k_neigh_index = initial_rank[forward_k_neigh_index, :k1 + 1]
fi = np.where(backward_k_neigh_index == i)[0]
k_reciprocal_index = forward_k_neigh_index[fi]
k_reciprocal_expansion_index = k_reciprocal_index
for j in range(len(k_reciprocal_index)):
candidate = k_reciprocal_index[j]
candidate_forward_k_neigh_index = initial_rank[candidate, :int(np.around(k1 / 2.)) + 1]
candidate_backward_k_neigh_index = initial_rank[candidate_forward_k_neigh_index,
:int(np.around(k1 / 2.)) + 1]
fi_candidate = np.where(candidate_backward_k_neigh_index == candidate)[0]
candidate_k_reciprocal_index = candidate_forward_k_neigh_index[fi_candidate]
if len(np.intersect1d(candidate_k_reciprocal_index, k_reciprocal_index)) > 2. / 3 * len(
candidate_k_reciprocal_index):
k_reciprocal_expansion_index = np.append(k_reciprocal_expansion_index, candidate_k_reciprocal_index)
k_reciprocal_expansion_index = np.unique(k_reciprocal_expansion_index)
weight = np.exp(-original_dist[i, k_reciprocal_expansion_index])
V[i, k_reciprocal_expansion_index] = 1. * weight / np.sum(weight)
original_dist = original_dist[:query_num, ]
if k2 != 1:
V_qe = np.zeros_like(V, dtype=np.float32)
for i in range(all_num):
V_qe[i, :] = np.mean(V[initial_rank[i, :k2], :], axis=0)
V = V_qe
del V_qe
del initial_rank
invIndex = []
for i in range(gallery_num):
invIndex.append(np.where(V[:, i] != 0)[0])
jaccard_dist = np.zeros_like(original_dist, dtype=np.float32)
for i in range(query_num):
temp_min = np.zeros(shape=[1, gallery_num], dtype=np.float32)
indNonZero = np.where(V[i, :] != 0)[0]
indImages = [invIndex[ind] for ind in indNonZero]
for j in range(len(indNonZero)):
temp_min[0, indImages[j]] = temp_min[0, indImages[j]] + np.minimum(V[i, indNonZero[j]],
V[indImages[j], indNonZero[j]])
jaccard_dist[i] = 1 - temp_min / (2. - temp_min)
final_dist = jaccard_dist * (1 - lambda_value) + original_dist * lambda_value
del original_dist
del V
del jaccard_dist
final_dist = final_dist[:query_num, query_num:]
return final_dist
def extract_reid_feature(data_loader, model, device, normalize, print_freq=200):
"""Extract feature for person ReID. If `normalize` is True, `cosine` distance will be employed as distance
metric, otherwise `euclidean` distance.
"""
batch_time = AverageMeter('Time', ':6.3f')
progress = ProgressMeter(
len(data_loader),
[batch_time],
prefix='Collect feature: ')
# switch to eval mode
model.eval()
feature_dict = dict()
with torch.no_grad():
end = time.time()
for i, (images_batch, filenames_batch, _, _) in enumerate(data_loader):
images_batch = images_batch.to(device)
features_batch = model(images_batch)
if normalize:
features_batch = F.normalize(features_batch)
for filename, feature in zip(filenames_batch, features_batch):
feature_dict[filename] = feature
# measure elapsed time
batch_time.update(time.time() - end)
end = time.time()
if i % print_freq == 0:
progress.display(i)
return feature_dict
def pairwise_distance(feature_dict, query, gallery):
"""Compute pairwise distance between two sets of features"""
# concat features and convert to pytorch tensor
# we compute pairwise distance metric on cpu because it may require a large amount of GPU memory, if you are using
# gpu with a larger capacity, it's faster to calculate on gpu
x = torch.cat([feature_dict[f].unsqueeze(0) for f, _, _ in query], dim=0).cpu()
y = torch.cat([feature_dict[f].unsqueeze(0) for f, _, _ in gallery], dim=0).cpu()
m, n = x.size(0), y.size(0)
# flatten
x = x.view(m, -1)
y = y.view(n, -1)
# compute dist_mat
dist_mat = torch.pow(x, 2).sum(dim=1, keepdim=True).expand(m, n) + \
torch.pow(y, 2).sum(dim=1, keepdim=True).expand(n, m).t() - \
2 * torch.matmul(x, y.t())
return dist_mat
def evaluate_all(dist_mat, query, gallery, cmc_topk=(1, 5, 10), cmc_flag=False):
"""Compute CMC score, mAP and return"""
query_ids = [pid for _, pid, _ in query]
gallery_ids = [pid for _, pid, _ in gallery]
query_cams = [cid for _, _, cid in query]
gallery_cams = [cid for _, _, cid in gallery]
# Compute mean AP
mAP = mean_ap(dist_mat, query_ids, gallery_ids, query_cams, gallery_cams)
print('Mean AP: {:4.1%}'.format(mAP))
if not cmc_flag:
return mAP
cmc_configs = {
'config': dict(separate_camera_set=False, single_gallery_shot=False, first_match_break=True)
}
cmc_scores = {name: cmc(dist_mat, query_ids, gallery_ids, query_cams, gallery_cams, **params) for name, params in
cmc_configs.items()}
print('CMC Scores:')
for k in cmc_topk:
print(' top-{:<4}{:12.1%}'.format(k, cmc_scores['config'][k - 1]))
return cmc_scores['config'][0], mAP
def validate(val_loader, model, query, gallery, device, criterion='cosine', cmc_flag=False, rerank=False):
assert criterion in ['cosine', 'euclidean']
# when criterion == 'cosine', normalize feature of single image into unit norm
normalize = (criterion == 'cosine')
feature_dict = extract_reid_feature(val_loader, model, device, normalize)
dist_mat = pairwise_distance(feature_dict, query, gallery)
results = evaluate_all(dist_mat, query=query, gallery=gallery, cmc_flag=cmc_flag)
if not rerank:
return results
# apply person re-ranking
print('Applying person re-ranking')
dist_mat_query = pairwise_distance(feature_dict, query, query)
dist_mat_gallery = pairwise_distance(feature_dict, gallery, gallery)
dist_mat = re_ranking(dist_mat, dist_mat_query, dist_mat_gallery)
return evaluate_all(dist_mat, query=query, gallery=gallery, cmc_flag=cmc_flag)
# location parameters for visualization
GRID_SPACING = 10
QUERY_EXTRA_SPACING = 90
# border width
BW = 5
GREEN = (0, 255, 0)
RED = (0, 0, 255)
def visualize_ranked_results(data_loader, model, query, gallery, device, visualize_dir, criterion='cosine',
rerank=False, width=128, height=256, topk=10):
"""Visualize ranker results. We first compute pair-wise distance between query images and gallery images. Then for
every query image, `topk` gallery images with least distance between given query image are selected. We plot the
query image and selected gallery images together. A green border denotes a match, and a red one denotes a mis-match.
"""
assert criterion in ['cosine', 'euclidean']
normalize = (criterion == 'cosine')
# compute pairwise distance matrix
feature_dict = extract_reid_feature(data_loader, model, device, normalize)
dist_mat = pairwise_distance(feature_dict, query, gallery)
if rerank:
dist_mat_query = pairwise_distance(feature_dict, query, query)
dist_mat_gallery = pairwise_distance(feature_dict, gallery, gallery)
dist_mat = re_ranking(dist_mat, dist_mat_query, dist_mat_gallery)
# make dir if not exists
os.makedirs(visualize_dir, exist_ok=True)
dist_mat = dist_mat.numpy()
num_q, num_g = dist_mat.shape
print('query images: {}'.format(num_q))
print('gallery images: {}'.format(num_g))
assert num_q == len(query)
assert num_g == len(gallery)
# start visualizing
import cv2
sorted_idxes = np.argsort(dist_mat, axis=1)
for q_idx in range(num_q):
q_img_path, q_pid, q_cid = query[q_idx]
q_img = cv2.imread(q_img_path)
q_img = cv2.resize(q_img, (width, height))
# use black border to denote query image
q_img = cv2.copyMakeBorder(
q_img, BW, BW, BW, BW, cv2.BORDER_CONSTANT, value=(0, 0, 0)
)
q_img = cv2.resize(q_img, (width, height))
num_cols = topk + 1
grid_img = 255 * np.ones(
(height, num_cols * width + topk * GRID_SPACING + QUERY_EXTRA_SPACING, 3), dtype=np.uint8
)
grid_img[:, :width, :] = q_img
# collect top-k gallery images with smallest distance
rank_idx = 1
for g_idx in sorted_idxes[q_idx, :]:
g_img_path, g_pid, g_cid = gallery[g_idx]
invalid = (q_pid == g_pid) & (q_cid == g_cid)
if not invalid:
matched = (g_pid == q_pid)
border_color = GREEN if matched else RED
g_img = cv2.imread(g_img_path)
g_img = cv2.resize(g_img, (width, height))
g_img = cv2.copyMakeBorder(
g_img, BW, BW, BW, BW, cv2.BORDER_CONSTANT, value=border_color
)
g_img = cv2.resize(g_img, (width, height))
start = rank_idx * width + rank_idx * GRID_SPACING + QUERY_EXTRA_SPACING
end = (rank_idx + 1) * width + rank_idx * GRID_SPACING + QUERY_EXTRA_SPACING
grid_img[:, start:end, :] = g_img
rank_idx += 1
if rank_idx > topk:
break
save_path = osp.basename(osp.splitext(q_img_path)[0])
cv2.imwrite(osp.join(visualize_dir, save_path + '.jpg'), grid_img)
if (q_idx + 1) % 100 == 0:
print('Visualize {}/{}'.format(q_idx + 1, num_q))
print('Visualization process is done, ranked results are saved to {}'.format(visualize_dir))
================================================
FILE: domain_generalization/Transfer-Learning-Library/common/utils/scheduler.py
================================================
"""
Modified from https://github.com/yxgeee/MMT
@author: Baixu Chen
@contact: cbx_99_hasta@outlook.com
"""
import torch
from bisect import bisect_right
class WarmupMultiStepLR(torch.optim.lr_scheduler._LRScheduler):
r"""Starts with a warm-up phase, then decays the learning rate of each parameter group by gamma once the
number of epoch reaches one of the milestones. When last_epoch=-1, sets initial lr as lr.
Args:
optimizer (Optimizer): Wrapped optimizer.
milestones (list): List of epoch indices. Must be increasing.
gamma (float): Multiplicative factor of learning rate decay.
Default: 0.1.
warmup_factor (float): a float number :math:`k` between 0 and 1, the start learning rate of warmup phase
will be set to :math:`k*initial\_lr`
warmup_steps (int): number of warm-up steps.
warmup_method (str): "constant" denotes a constant learning rate during warm-up phase and "linear" denotes a
linear-increasing learning rate during warm-up phase.
last_epoch (int): The index of last epoch. Default: -1.
"""
def __init__(
self,
optimizer,
milestones,
gamma=0.1,
warmup_factor=1.0 / 3,
warmup_steps=500,
warmup_method="linear",
last_epoch=-1,
):
if not list(milestones) == sorted(milestones):
raise ValueError(
"Milestones should be a list of" " increasing integers. Got {}",
milestones,
)
if warmup_method not in ("constant", "linear"):
raise ValueError(
"Only 'constant' or 'linear' warmup_method accepted"
"got {}".format(warmup_method)
)
self.milestones = milestones
self.gamma = gamma
self.warmup_factor = warmup_factor
self.warmup_steps = warmup_steps
self.warmup_method = warmup_method
super(WarmupMultiStepLR, self).__init__(optimizer, last_epoch)
def get_lr(self):
warmup_factor = 1
if self.last_epoch < self.warmup_steps:
if self.warmup_method == "constant":
warmup_factor = self.warmup_factor
elif self.warmup_method == "linear":
alpha = float(self.last_epoch) / float(self.warmup_steps)
warmup_factor = self.warmup_factor * (1 - alpha) + alpha
return [
base_lr
* warmup_factor
* self.gamma ** bisect_right(self.milestones, self.last_epoch)
for base_lr in self.base_lrs
]
================================================
FILE: domain_generalization/Transfer-Learning-Library/common/vision/__init__.py
================================================
__all__ = ['datasets', 'models', 'transforms']
================================================
FILE: domain_generalization/Transfer-Learning-Library/common/vision/datasets/__init__.py
================================================
from .imagelist import ImageList
from .office31 import Office31
from .officehome import OfficeHome
from .visda2017 import VisDA2017
from .officecaltech import OfficeCaltech
from .domainnet import DomainNet
from .imagenet_r import ImageNetR
from .imagenet_sketch import ImageNetSketch
from .pacs import PACS
from .digits import *
from .aircrafts import Aircraft
from .cub200 import CUB200
from .stanford_cars import StanfordCars
from .stanford_dogs import StanfordDogs
from .coco70 import COCO70
from .oxfordpet import OxfordIIITPet
from .dtd import DTD
from .oxfordflowers import OxfordFlowers102
from .patchcamelyon import PatchCamelyon
from .retinopathy import Retinopathy
from .eurosat import EuroSAT
from .resisc45 import Resisc45
__all__ = ['ImageList', 'Office31', 'OfficeHome', "VisDA2017", "OfficeCaltech", "DomainNet", "ImageNetR", "ImageNetSketch",
"Aircraft", "cub200", "StanfordCars", "StanfordDogs", "COCO70", "OxfordIIITPet", "PACS", "DTD",
"OxfordFlowers102", "PatchCamelyon", "Retinopathy", "EuroSAT", "Resisc45"]
================================================
FILE: domain_generalization/Transfer-Learning-Library/common/vision/datasets/_util.py
================================================
"""
@author: Junguang Jiang
@contact: JiangJunguang1123@outlook.com
"""
import os
from typing import List
from torchvision.datasets.utils import download_and_extract_archive
def download(root: str, file_name: str, archive_name: str, url_link: str):
"""
Download file from internet url link.
Args:
root (str) The directory to put downloaded files.
file_name: (str) The name of the unzipped file.
archive_name: (str) The name of archive(zipped file) downloaded.
url_link: (str) The url link to download data.
.. note::
If `file_name` already exists under path `root`, then it is not downloaded again.
Else `archive_name` will be downloaded from `url_link` and extracted to `file_name`.
"""
if not os.path.exists(os.path.join(root, file_name)):
print("Downloading {}".format(file_name))
# if os.path.exists(os.path.join(root, archive_name)):
# os.remove(os.path.join(root, archive_name))
try:
download_and_extract_archive(url_link, download_root=root, filename=archive_name, remove_finished=False)
except Exception:
print("Fail to download {} from url link {}".format(archive_name, url_link))
print('Please check you internet connection.'
"Simply trying again may be fine.")
exit(0)
def check_exits(root: str, file_name: str):
"""Check whether `file_name` exists under directory `root`. """
if not os.path.exists(os.path.join(root, file_name)):
print("Dataset directory {} not found under {}".format(file_name, root))
exit(-1)
def read_list_from_file(file_name: str) -> List[str]:
"""Read data from file and convert each line into an element in the list"""
result = []
with open(file_name, "r") as f:
for line in f.readlines():
result.append(line.strip())
return result
================================================
FILE: domain_generalization/Transfer-Learning-Library/common/vision/datasets/aircrafts.py
================================================
"""
@author: Yifei Ji
@contact: jiyf990330@163.com
"""
import os
from typing import Optional
from .imagelist import ImageList
from ._util import download as download_data, check_exits
class Aircraft(ImageList):
"""`FVGC-Aircraft `_ \
is a benchmark for the fine-grained visual categorization of aircraft. \
The dataset contains 10,200 images of aircraft, with 100 images for each \
of the 102 different aircraft variants.
Args:
root (str): Root directory of dataset
split (str, optional): The dataset split, supports ``train``, or ``test``.
sample_rate (int): The sampling rates to sample random ``training`` images for each category.
Choices include 100, 50, 30, 15. Default: 100.
download (bool, optional): If true, downloads the dataset from the internet and puts it \
in root directory. If dataset is already downloaded, it is not downloaded again.
transform (callable, optional): A function/transform that takes in an PIL image and returns a \
transformed version. E.g, :class:`torchvision.transforms.RandomCrop`.
target_transform (callable, optional): A function/transform that takes in the target and transforms it.
.. note:: In `root`, there will exist following files after downloading.
::
train/
test/
image_list/
train_100.txt
train_50.txt
train_30.txt
train_15.txt
test.txt
"""
download_list = [
("image_list", "image_list.zip", "https://cloud.tsinghua.edu.cn/f/04356d49d0054092b07e/?dl=1"),
("train", "train.tgz", "https://cloud.tsinghua.edu.cn/f/9fed22eba03046d69012/?dl=1"),
("test", "test.tgz", "https://cloud.tsinghua.edu.cn/f/8d5e4c1b031a4a608c68/?dl=1"),
]
image_list = {
"train": "image_list/train_100.txt",
"train100": "image_list/train_100.txt",
"train50": "image_list/train_50.txt",
"train30": "image_list/train_30.txt",
"train15": "image_list/train_15.txt",
"test": "image_list/test.txt",
"test100": "image_list/test.txt",
}
CLASSES = ['707-320', '727-200', '737-200', '737-300', '737-400', '737-500', '737-600', '737-700', '737-800', '737-900', '747-100',
'747-200', '747-300', '747-400', '757-200', '757-300', '767-200', '767-300', '767-400', '777-200', '777-300', 'A300B4',
'A310', 'A318', 'A319', 'A320', 'A321', 'A330-200', 'A330-300', 'A340-200', 'A340-300', 'A340-500', 'A340-600', 'A380',
'ATR-42', 'ATR-72', 'An-12', 'BAE 146-200', 'BAE 146-300', 'BAE-125', 'Beechcraft 1900', 'Boeing 717', 'C-130', 'C-47',
'CRJ-200', 'CRJ-700', 'CRJ-900', 'Cessna 172', 'Cessna 208', 'Cessna 525', 'Cessna 560', 'Challenger 600', 'DC-10',
'DC-3', 'DC-6', 'DC-8', 'DC-9-30', 'DH-82', 'DHC-1', 'DHC-6', 'DHC-8-100', 'DHC-8-300', 'DR-400', 'Dornier 328', 'E-170',
'E-190', 'E-195', 'EMB-120', 'ERJ 135', 'ERJ 145', 'Embraer Legacy 600', 'Eurofighter Typhoon', 'F-16A-B', 'F-A-18',
'Falcon 2000', 'Falcon 900', 'Fokker 100', 'Fokker 50', 'Fokker 70', 'Global Express', 'Gulfstream IV', 'Gulfstream V',
'Hawk T1', 'Il-76', 'L-1011', 'MD-11', 'MD-80', 'MD-87', 'MD-90', 'Metroliner', 'Model B200', 'PA-28', 'SR-20',
'Saab 2000', 'Saab 340', 'Spitfire', 'Tornado', 'Tu-134', 'Tu-154', 'Yak-42']
def __init__(self, root: str, split: str, sample_rate: Optional[int]=100, download: Optional[bool] = False, **kwargs):
if split == 'train':
list_name = 'train' + str(sample_rate)
assert list_name in self.image_list
data_list_file = os.path.join(root, self.image_list[list_name])
else:
data_list_file = os.path.join(root, self.image_list['test'])
if download:
list(map(lambda args: download_data(root, *args), self.download_list))
else:
list(map(lambda file_name, _: check_exits(root, file_name), self.download_list))
super(Aircraft, self).__init__(root, Aircraft.CLASSES, data_list_file=data_list_file, **kwargs)
================================================
FILE: domain_generalization/Transfer-Learning-Library/common/vision/datasets/coco70.py
================================================
"""
@author: Yifei Ji
@contact: jiyf990330@163.com
"""
import os
from typing import Optional
from .imagelist import ImageList
from ._util import download as download_data, check_exits
class COCO70(ImageList):
"""COCO-70 dataset is a large-scale classification dataset (1000 images per class) created from
`COCO `_ Dataset.
It is used to explore the effect of fine-tuning with a large amount of data.
Args:
root (str): Root directory of dataset
split (str, optional): The dataset split, supports ``train``, or ``test``.
sample_rate (int): The sampling rates to sample random ``training`` images for each category.
Choices include 100, 50, 30, 15. Default: 100.
download (bool, optional): If true, downloads the dataset from the internet and puts it \
in root directory. If dataset is already downloaded, it is not downloaded again.
transform (callable, optional): A function/transform that takes in an PIL image and returns a \
transformed version. E.g, :class:`torchvision.transforms.RandomCrop`.
target_transform (callable, optional): A function/transform that takes in the target and transforms it.
.. note:: In `root`, there will exist following files after downloading.
::
train/
test/
image_list/
train_100.txt
train_50.txt
train_30.txt
train_15.txt
test.txt
"""
download_list = [
("image_list", "image_list.zip", "https://cloud.tsinghua.edu.cn/f/d2ffb62fe3d140f1a73c/?dl=1"),
("train", "train.tgz", "https://cloud.tsinghua.edu.cn/f/e0dc4368342948c5bb2a/?dl=1"),
("test", "test.tgz", "https://cloud.tsinghua.edu.cn/f/59393a55c818429fb8d1/?dl=1"),
]
image_list = {
"train": "image_list/train_100.txt",
"train100": "image_list/train_100.txt",
"train50": "image_list/train_50.txt",
"train30": "image_list/train_30.txt",
"train15": "image_list/train_15.txt",
"test": "image_list/test.txt",
"test100": "image_list/test.txt",
}
CLASSES =['person', 'bicycle', 'car', 'motorcycle', 'airplane', 'bus', 'train', 'truck',
'boat', 'traffic_light', 'fire_hydrant', 'stop_sign', 'bench', 'bird', 'cat', 'dog',
'horse', 'sheep', 'cow', 'elephant', 'bear', 'zebra', 'giraffe', 'backpack', 'umbrella',
'handbag', 'tie', 'suitcase', 'skis', 'kite', 'baseball_bat', 'skateboard', 'surfboard',
'tennis_racket', 'bottle', 'wine_glass', 'cup', 'fork', 'knife', 'spoon', 'bowl', 'banana',
'apple', 'sandwich', 'orange', 'broccoli', 'carrot', 'hot_dog', 'pizza', 'donut', 'cake',
'chair', 'couch', 'potted_plant', 'bed', 'dining_table', 'toilet', 'tv', 'laptop',
'remote', 'keyboard', 'cell_phone', 'microwave', 'oven', 'sink', 'refrigerator', 'book', 'clock', 'vase', 'teddy_bear']
def __init__(self, root: str, split: str, sample_rate: Optional[int] =100, download: Optional[bool] = False, **kwargs):
if split == 'train':
list_name = 'train' + str(sample_rate)
assert list_name in self.image_list
data_list_file = os.path.join(root, self.image_list[list_name])
else:
data_list_file = os.path.join(root, self.image_list['test'])
if download:
list(map(lambda args: download_data(root, *args), self.download_list))
else:
list(map(lambda file_name, _: check_exits(root, file_name), self.download_list))
super(COCO70, self).__init__(root, COCO70.CLASSES, data_list_file=data_list_file, **kwargs)
================================================
FILE: domain_generalization/Transfer-Learning-Library/common/vision/datasets/cub200.py
================================================
"""
@author: Yifei Ji
@contact: jiyf990330@163.com
"""
import os
from typing import Optional
from .imagelist import ImageList
from ._util import download as download_data, check_exits
class CUB200(ImageList):
"""`Caltech-UCSD Birds-200-2011 `_ \
is a dataset for fine-grained visual recognition with 11,788 images in 200 bird species. \
It is an extended version of the CUB-200 dataset, roughly doubling the number of images.
Args:
root (str): Root directory of dataset
split (str, optional): The dataset split, supports ``train``, or ``test``.
sample_rate (int): The sampling rates to sample random ``training`` images for each category.
Choices include 100, 50, 30, 15. Default: 100.
download (bool, optional): If true, downloads the dataset from the internet and puts it \
in root directory. If dataset is already downloaded, it is not downloaded again.
transform (callable, optional): A function/transform that takes in an PIL image and returns a \
transformed version. E.g, :class:`torchvision.transforms.RandomCrop`.
target_transform (callable, optional): A function/transform that takes in the target and transforms it.
.. note:: In `root`, there will exist following files after downloading.
::
train/
test/
image_list/
train_100.txt
train_50.txt
train_30.txt
train_15.txt
test.txt
"""
download_list = [
("image_list", "image_list.zip", "https://cloud.tsinghua.edu.cn/f/fa398a321b094a24a347/?dl=1"),
("train", "train.tgz", "https://cloud.tsinghua.edu.cn/f/521ba92bafc04ee69c20/?dl=1"),
("test", "test.tgz", "https://cloud.tsinghua.edu.cn/f/cc7ef72081e64bc7a218/?dl=1"),
]
image_list = {
"train": "image_list/train_100.txt",
"train100": "image_list/train_100.txt",
"train50": "image_list/train_50.txt",
"train30": "image_list/train_30.txt",
"train15": "image_list/train_15.txt",
"test": "image_list/test.txt",
"test100": "image_list/test.txt",
}
CLASSES = ['001.Black_footed_Albatross', '002.Laysan_Albatross', '003.Sooty_Albatross', '004.Groove_billed_Ani', '005.Crested_Auklet', '006.Least_Auklet', '007.Parakeet_Auklet', '008.Rhinoceros_Auklet', '009.Brewer_Blackbird', '010.Red_winged_Blackbird', '011.Rusty_Blackbird', '012.Yellow_headed_Blackbird', '013.Bobolink', '014.Indigo_Bunting', '015.Lazuli_Bunting', '016.Painted_Bunting', '017.Cardinal', '018.Spotted_Catbird', '019.Gray_Catbird', '020.Yellow_breasted_Chat', '021.Eastern_Towhee', '022.Chuck_will_Widow', '023.Brandt_Cormorant', '024.Red_faced_Cormorant', '025.Pelagic_Cormorant', '026.Bronzed_Cowbird', '027.Shiny_Cowbird', '028.Brown_Creeper', '029.American_Crow', '030.Fish_Crow', '031.Black_billed_Cuckoo', '032.Mangrove_Cuckoo', '033.Yellow_billed_Cuckoo', '034.Gray_crowned_Rosy_Finch', '035.Purple_Finch', '036.Northern_Flicker', '037.Acadian_Flycatcher', '038.Great_Crested_Flycatcher', '039.Least_Flycatcher', '040.Olive_sided_Flycatcher', '041.Scissor_tailed_Flycatcher', '042.Vermilion_Flycatcher', '043.Yellow_bellied_Flycatcher', '044.Frigatebird', '045.Northern_Fulmar', '046.Gadwall', '047.American_Goldfinch', '048.European_Goldfinch', '049.Boat_tailed_Grackle', '050.Eared_Grebe',
'051.Horned_Grebe', '052.Pied_billed_Grebe', '053.Western_Grebe', '054.Blue_Grosbeak', '055.Evening_Grosbeak', '056.Pine_Grosbeak', '057.Rose_breasted_Grosbeak', '058.Pigeon_Guillemot', '059.California_Gull', '060.Glaucous_winged_Gull', '061.Heermann_Gull', '062.Herring_Gull', '063.Ivory_Gull', '064.Ring_billed_Gull', '065.Slaty_backed_Gull', '066.Western_Gull', '067.Anna_Hummingbird', '068.Ruby_throated_Hummingbird', '069.Rufous_Hummingbird', '070.Green_Violetear', '071.Long_tailed_Jaeger', '072.Pomarine_Jaeger', '073.Blue_Jay', '074.Florida_Jay', '075.Green_Jay', '076.Dark_eyed_Junco', '077.Tropical_Kingbird', '078.Gray_Kingbird', '079.Belted_Kingfisher', '080.Green_Kingfisher', '081.Pied_Kingfisher', '082.Ringed_Kingfisher', '083.White_breasted_Kingfisher', '084.Red_legged_Kittiwake', '085.Horned_Lark', '086.Pacific_Loon', '087.Mallard', '088.Western_Meadowlark', '089.Hooded_Merganser', '090.Red_breasted_Merganser', '091.Mockingbird', '092.Nighthawk', '093.Clark_Nutcracker', '094.White_breasted_Nuthatch', '095.Baltimore_Oriole', '096.Hooded_Oriole', '097.Orchard_Oriole', '098.Scott_Oriole', '099.Ovenbird', '100.Brown_Pelican',
'101.White_Pelican', '102.Western_Wood_Pewee', '103.Sayornis', '104.American_Pipit', '105.Whip_poor_Will', '106.Horned_Puffin', '107.Common_Raven', '108.White_necked_Raven', '109.American_Redstart', '110.Geococcyx', '111.Loggerhead_Shrike', '112.Great_Grey_Shrike', '113.Baird_Sparrow', '114.Black_throated_Sparrow', '115.Brewer_Sparrow', '116.Chipping_Sparrow', '117.Clay_colored_Sparrow', '118.House_Sparrow', '119.Field_Sparrow', '120.Fox_Sparrow', '121.Grasshopper_Sparrow', '122.Harris_Sparrow', '123.Henslow_Sparrow', '124.Le_Conte_Sparrow', '125.Lincoln_Sparrow', '126.Nelson_Sharp_tailed_Sparrow', '127.Savannah_Sparrow', '128.Seaside_Sparrow', '129.Song_Sparrow', '130.Tree_Sparrow', '131.Vesper_Sparrow', '132.White_crowned_Sparrow', '133.White_throated_Sparrow', '134.Cape_Glossy_Starling', '135.Bank_Swallow', '136.Barn_Swallow', '137.Cliff_Swallow', '138.Tree_Swallow', '139.Scarlet_Tanager', '140.Summer_Tanager', '141.Artic_Tern', '142.Black_Tern', '143.Caspian_Tern', '144.Common_Tern', '145.Elegant_Tern', '146.Forsters_Tern', '147.Least_Tern', '148.Green_tailed_Towhee', '149.Brown_Thrasher', '150.Sage_Thrasher',
'151.Black_capped_Vireo', '152.Blue_headed_Vireo', '153.Philadelphia_Vireo', '154.Red_eyed_Vireo', '155.Warbling_Vireo', '156.White_eyed_Vireo', '157.Yellow_throated_Vireo', '158.Bay_breasted_Warbler', '159.Black_and_white_Warbler', '160.Black_throated_Blue_Warbler', '161.Blue_winged_Warbler', '162.Canada_Warbler', '163.Cape_May_Warbler', '164.Cerulean_Warbler', '165.Chestnut_sided_Warbler', '166.Golden_winged_Warbler', '167.Hooded_Warbler', '168.Kentucky_Warbler', '169.Magnolia_Warbler', '170.Mourning_Warbler', '171.Myrtle_Warbler', '172.Nashville_Warbler', '173.Orange_crowned_Warbler', '174.Palm_Warbler', '175.Pine_Warbler', '176.Prairie_Warbler', '177.Prothonotary_Warbler', '178.Swainson_Warbler', '179.Tennessee_Warbler', '180.Wilson_Warbler', '181.Worm_eating_Warbler', '182.Yellow_Warbler', '183.Northern_Waterthrush', '184.Louisiana_Waterthrush', '185.Bohemian_Waxwing', '186.Cedar_Waxwing', '187.American_Three_toed_Woodpecker', '188.Pileated_Woodpecker', '189.Red_bellied_Woodpecker', '190.Red_cockaded_Woodpecker', '191.Red_headed_Woodpecker', '192.Downy_Woodpecker', '193.Bewick_Wren', '194.Cactus_Wren', '195.Carolina_Wren', '196.House_Wren', '197.Marsh_Wren', '198.Rock_Wren', '199.Winter_Wren', '200.Common_Yellowthroat']
def __init__(self, root: str, split: str, sample_rate: Optional[int] =100, download: Optional[bool] = False, **kwargs):
if split == 'train':
list_name = 'train' + str(sample_rate)
assert list_name in self.image_list
data_list_file = os.path.join(root, self.image_list[list_name])
else:
data_list_file = os.path.join(root, self.image_list['test'])
if download:
list(map(lambda args: download_data(root, *args), self.download_list))
else:
list(map(lambda file_name, _: check_exits(root, file_name), self.download_list))
super(CUB200, self).__init__(root, CUB200.CLASSES, data_list_file=data_list_file, **kwargs)
================================================
FILE: domain_generalization/Transfer-Learning-Library/common/vision/datasets/digits.py
================================================
"""
@author: Junguang Jiang, Baixu Chen
@contact: JiangJunguang1123@outlook.com, cbx_99_hasta@outlook.com
"""
import os
from typing import Optional, Tuple, Any
from .imagelist import ImageList
from ._util import download as download_data, check_exits
class MNIST(ImageList):
"""`MNIST `_ Dataset.
Args:
root (str): Root directory of dataset where ``MNIST/processed/training.pt``
and ``MNIST/processed/test.pt`` exist.
mode (str): The channel mode for image. Choices includes ``"L"```, ``"RGB"``.
Default: ``"L"```
split (str, optional): The dataset split, supports ``train``, or ``test``.
download (bool, optional): If true, downloads the dataset from the internet and
puts it in root directory. If dataset is already downloaded, it is not
downloaded again.
transform (callable, optional): A function/transform that takes in an PIL image
and returns a transformed version. E.g, ``transforms.RandomCrop``
"""
download_list = [
("image_list", "image_list.zip", "https://cloud.tsinghua.edu.cn/f/f18f1e115de94644b900/?dl=1"),
("mnist_train_image", "mnist_image.tar.gz", "https://cloud.tsinghua.edu.cn/f/fdf45c75d2e746acba93/?dl=1"),
# ("mnist_test_image", "mnist_image.tar.gz", "https://cloud.tsinghua.edu.cn/f/fdf45c75d2e746acba93/?dl=1")
]
image_list = {
"train": "image_list/mnist_train.txt",
"test": "image_list/mnist_test.txt"
}
CLASSES = ['0 - zero', '1 - one', '2 - two', '3 - three', '4 - four',
'5 - five', '6 - six', '7 - seven', '8 - eight', '9 - nine']
def __init__(self, root, mode="L", split='train', download: Optional[bool] = True, **kwargs):
assert split in ['train', 'test']
data_list_file = os.path.join(root, self.image_list[split])
if download:
list(map(lambda args: download_data(root, *args), self.download_list))
else:
list(map(lambda file_name, _: check_exits(root, file_name), self.download_list))
assert mode in ['L', 'RGB']
self.mode = mode
super(MNIST, self).__init__(root, MNIST.CLASSES, data_list_file=data_list_file, **kwargs)
def __getitem__(self, index: int) -> Tuple[Any, int]:
"""
Args:
index (int): Index
return (tuple): (image, target) where target is index of the target class.
"""
path, target = self.samples[index]
img = self.loader(path).convert(self.mode)
if self.transform is not None:
img = self.transform(img)
if self.target_transform is not None and target is not None:
target = self.target_transform(target)
return img, target
@classmethod
def get_classes(self):
return MNIST.CLASSES
class USPS(ImageList):
"""`USPS `_ Dataset.
The data-format is : [label [index:value ]*256 \\n] * num_lines, where ``label`` lies in ``[1, 10]``.
The value for each pixel lies in ``[-1, 1]``. Here we transform the ``label`` into ``[0, 9]``
and make pixel values in ``[0, 255]``.
Args:
root (str): Root directory of dataset to store``USPS`` data files.
mode (str): The channel mode for image. Choices includes ``"L"```, ``"RGB"``.
Default: ``"L"```
split (str, optional): The dataset split, supports ``train``, or ``test``.
transform (callable, optional): A function/transform that takes in an PIL image
and returns a transformed version. E.g, ``transforms.RandomCrop``
download (bool, optional): If true, downloads the dataset from the internet and
puts it in root directory. If dataset is already downloaded, it is not
downloaded again.
"""
download_list = [
("image_list", "image_list.zip", "https://cloud.tsinghua.edu.cn/f/10ddb319c24e40a08e58/?dl=1"),
("usps_train_image", "usps_image.tar.gz", "https://cloud.tsinghua.edu.cn/f/1d3d7e2540bd4392b715/?dl=1"),
# ("usps_test_image", "usps_image.tar.gz", "https://cloud.tsinghua.edu.cn/f/1d3d7e2540bd4392b715/?dl=1")
]
image_list = {
"train": "image_list/usps_train.txt",
"test": "image_list/usps_test.txt"
}
CLASSES = ['0 - zero', '1 - one', '2 - two', '3 - three', '4 - four',
'5 - five', '6 - six', '7 - seven', '8 - eight', '9 - nine']
def __init__(self, root, mode="L", split='train', download: Optional[bool] = True, **kwargs):
assert split in ['train', 'test']
data_list_file = os.path.join(root, self.image_list[split])
if download:
list(map(lambda args: download_data(root, *args), self.download_list))
else:
list(map(lambda file_name, _: check_exits(root, file_name), self.download_list))
assert mode in ['L', 'RGB']
self.mode = mode
super(USPS, self).__init__(root, USPS.CLASSES, data_list_file=data_list_file, **kwargs)
def __getitem__(self, index: int) -> Tuple[Any, int]:
"""
Args:
index (int): Index
return (tuple): (image, target) where target is index of the target class.
"""
path, target = self.samples[index]
img = self.loader(path).convert(self.mode)
if self.transform is not None:
img = self.transform(img)
if self.target_transform is not None and target is not None:
target = self.target_transform(target)
return img, target
class SVHN(ImageList):
"""`SVHN `_ Dataset.
Note: The SVHN dataset assigns the label `10` to the digit `0`. However, in this Dataset,
we assign the label `0` to the digit `0` to be compatible with PyTorch loss functions which
expect the class labels to be in the range `[0, C-1]`
.. warning::
This class needs `scipy `_ to load data from `.mat` format.
Args:
root (str): Root directory of dataset where directory
``SVHN`` exists.
mode (str): The channel mode for image. Choices includes ``"L"```, ``"RGB"``.
Default: ``"RGB"```
split (str, optional): The dataset split, supports ``train``, or ``test``.
transform (callable, optional): A function/transform that takes in an PIL image
and returns a transformed version. E.g, ``transforms.RandomCrop``
download (bool, optional): If true, downloads the dataset from the internet and
puts it in root directory. If dataset is already downloaded, it is not
downloaded again.
"""
download_list = [
("image_list", "image_list.zip", "https://cloud.tsinghua.edu.cn/f/80a8a06c4a324c59a5e4/?dl=1"),
("svhn_image", "svhn_image.tar.gz", "https://cloud.tsinghua.edu.cn/f/0e48a871e00345eb91a9/?dl=1")
]
image_list = "image_list/svhn_balanced.txt"
# image_list = "image_list/svhn.txt"
CLASSES = ['0 - zero', '1 - one', '2 - two', '3 - three', '4 - four',
'5 - five', '6 - six', '7 - seven', '8 - eight', '9 - nine']
def __init__(self, root, mode="L", download: Optional[bool] = True, **kwargs):
data_list_file = os.path.join(root, self.image_list)
if download:
list(map(lambda args: download_data(root, *args), self.download_list))
else:
list(map(lambda file_name, _: check_exits(root, file_name), self.download_list))
assert mode in ['L', 'RGB']
self.mode = mode
super(SVHN, self).__init__(root, SVHN.CLASSES, data_list_file=data_list_file, **kwargs)
def __getitem__(self, index: int) -> Tuple[Any, int]:
"""
Args:
index (int): Index
return (tuple): (image, target) where target is index of the target class.
"""
path, target = self.samples[index]
img = self.loader(path).convert(self.mode)
if self.transform is not None:
img = self.transform(img)
if self.target_transform is not None and target is not None:
target = self.target_transform(target)
return img, target
class MNISTRGB(MNIST):
def __init__(self, root, **kwargs):
super(MNISTRGB, self).__init__(root, mode='RGB', **kwargs)
class USPSRGB(USPS):
def __init__(self, root, **kwargs):
super(USPSRGB, self).__init__(root, mode='RGB', **kwargs)
class SVHNRGB(SVHN):
def __init__(self, root, **kwargs):
super(SVHNRGB, self).__init__(root, mode='RGB', **kwargs)
================================================
FILE: domain_generalization/Transfer-Learning-Library/common/vision/datasets/domainnet.py
================================================
"""
@author: Junguang Jiang
@contact: JiangJunguang1123@outlook.com
"""
import os
from typing import Optional
from .imagelist import ImageList
from ._util import download as download_data, check_exits
class DomainNet(ImageList):
"""`DomainNet `_ (cleaned version, recommended)
See `Moment Matching for Multi-Source Domain Adaptation `_ for details.
Args:
root (str): Root directory of dataset
task (str): The task (domain) to create dataset. Choices include ``'c'``:clipart, \
``'i'``: infograph, ``'p'``: painting, ``'q'``: quickdraw, ``'r'``: real, ``'s'``: sketch
split (str, optional): The dataset split, supports ``train``, or ``test``.
download (bool, optional): If true, downloads the dataset from the internet and puts it \
in root directory. If dataset is already downloaded, it is not downloaded again.
transform (callable, optional): A function/transform that takes in an PIL image and returns a \
transformed version. E.g, :class:`torchvision.transforms.RandomCrop`.
target_transform (callable, optional): A function/transform that takes in the target and transforms it.
.. note:: In `root`, there will exist following files after downloading.
::
clipart/
infograph/
painting/
quickdraw/
real/
sketch/
image_list/
clipart.txt
...
"""
download_list = [
("image_list", "image_list.zip", "https://cloud.tsinghua.edu.cn/f/90ecb35bbd374e5e8c41/?dl=1"),
("clipart", "clipart.zip", "http://csr.bu.edu/ftp/visda/2019/multi-source/groundtruth/clipart.zip"),
("infograph", "infograph.zip", "http://csr.bu.edu/ftp/visda/2019/multi-source/infograph.zip"),
("painting", "painting.zip", "http://csr.bu.edu/ftp/visda/2019/multi-source/groundtruth/painting.zip"),
("quickdraw", "quickdraw.zip", "http://csr.bu.edu/ftp/visda/2019/multi-source/quickdraw.zip"),
("real", "real.zip", "http://csr.bu.edu/ftp/visda/2019/multi-source/real.zip"),
("sketch", "sketch.zip", "http://csr.bu.edu/ftp/visda/2019/multi-source/sketch.zip"),
]
image_list = {
"c": "clipart",
"i": "infograph",
"p": "painting",
"q": "quickdraw",
"r": "real",
"s": "sketch",
}
CLASSES = ['aircraft_carrier', 'airplane', 'alarm_clock', 'ambulance', 'angel', 'animal_migration', 'ant', 'anvil',
'apple', 'arm', 'asparagus', 'axe', 'backpack', 'banana', 'bandage', 'barn', 'baseball', 'baseball_bat',
'basket', 'basketball', 'bat', 'bathtub', 'beach', 'bear', 'beard', 'bed', 'bee', 'belt', 'bench',
'bicycle', 'binoculars', 'bird', 'birthday_cake', 'blackberry', 'blueberry', 'book', 'boomerang',
'bottlecap', 'bowtie', 'bracelet', 'brain', 'bread', 'bridge', 'broccoli', 'broom', 'bucket',
'bulldozer', 'bus', 'bush', 'butterfly', 'cactus', 'cake', 'calculator', 'calendar', 'camel', 'camera',
'camouflage', 'campfire', 'candle', 'cannon', 'canoe', 'car', 'carrot', 'castle', 'cat', 'ceiling_fan',
'cello', 'cell_phone', 'chair', 'chandelier', 'church', 'circle', 'clarinet', 'clock', 'cloud',
'coffee_cup', 'compass', 'computer', 'cookie', 'cooler', 'couch', 'cow', 'crab', 'crayon', 'crocodile',
'crown', 'cruise_ship', 'cup', 'diamond', 'dishwasher', 'diving_board', 'dog', 'dolphin', 'donut',
'door', 'dragon', 'dresser', 'drill', 'drums', 'duck', 'dumbbell', 'ear', 'elbow', 'elephant',
'envelope', 'eraser', 'eye', 'eyeglasses', 'face', 'fan', 'feather', 'fence', 'finger', 'fire_hydrant',
'fireplace', 'firetruck', 'fish', 'flamingo', 'flashlight', 'flip_flops', 'floor_lamp', 'flower',
'flying_saucer', 'foot', 'fork', 'frog', 'frying_pan', 'garden', 'garden_hose', 'giraffe', 'goatee',
'golf_club', 'grapes', 'grass', 'guitar', 'hamburger', 'hammer', 'hand', 'harp', 'hat', 'headphones',
'hedgehog', 'helicopter', 'helmet', 'hexagon', 'hockey_puck', 'hockey_stick', 'horse', 'hospital',
'hot_air_balloon', 'hot_dog', 'hot_tub', 'hourglass', 'house', 'house_plant', 'hurricane', 'ice_cream',
'jacket', 'jail', 'kangaroo', 'key', 'keyboard', 'knee', 'knife', 'ladder', 'lantern', 'laptop', 'leaf',
'leg', 'light_bulb', 'lighter', 'lighthouse', 'lightning', 'line', 'lion', 'lipstick', 'lobster',
'lollipop', 'mailbox', 'map', 'marker', 'matches', 'megaphone', 'mermaid', 'microphone', 'microwave',
'monkey', 'moon', 'mosquito', 'motorbike', 'mountain', 'mouse', 'moustache', 'mouth', 'mug', 'mushroom',
'nail', 'necklace', 'nose', 'ocean', 'octagon', 'octopus', 'onion', 'oven', 'owl', 'paintbrush',
'paint_can', 'palm_tree', 'panda', 'pants', 'paper_clip', 'parachute', 'parrot', 'passport', 'peanut',
'pear', 'peas', 'pencil', 'penguin', 'piano', 'pickup_truck', 'picture_frame', 'pig', 'pillow',
'pineapple', 'pizza', 'pliers', 'police_car', 'pond', 'pool', 'popsicle', 'postcard', 'potato',
'power_outlet', 'purse', 'rabbit', 'raccoon', 'radio', 'rain', 'rainbow', 'rake', 'remote_control',
'rhinoceros', 'rifle', 'river', 'roller_coaster', 'rollerskates', 'sailboat', 'sandwich', 'saw',
'saxophone', 'school_bus', 'scissors', 'scorpion', 'screwdriver', 'sea_turtle', 'see_saw', 'shark',
'sheep', 'shoe', 'shorts', 'shovel', 'sink', 'skateboard', 'skull', 'skyscraper', 'sleeping_bag',
'smiley_face', 'snail', 'snake', 'snorkel', 'snowflake', 'snowman', 'soccer_ball', 'sock', 'speedboat',
'spider', 'spoon', 'spreadsheet', 'square', 'squiggle', 'squirrel', 'stairs', 'star', 'steak', 'stereo',
'stethoscope', 'stitches', 'stop_sign', 'stove', 'strawberry', 'streetlight', 'string_bean', 'submarine',
'suitcase', 'sun', 'swan', 'sweater', 'swing_set', 'sword', 'syringe', 'table', 'teapot', 'teddy-bear',
'telephone', 'television', 'tennis_racquet', 'tent', 'The_Eiffel_Tower', 'The_Great_Wall_of_China',
'The_Mona_Lisa', 'tiger', 'toaster', 'toe', 'toilet', 'tooth', 'toothbrush', 'toothpaste', 'tornado',
'tractor', 'traffic_light', 'train', 'tree', 'triangle', 'trombone', 'truck', 'trumpet', 't-shirt',
'umbrella', 'underwear', 'van', 'vase', 'violin', 'washing_machine', 'watermelon', 'waterslide',
'whale', 'wheel', 'windmill', 'wine_bottle', 'wine_glass', 'wristwatch', 'yoga', 'zebra', 'zigzag']
def __init__(self, root: str, task: str, split: Optional[str] = 'train', download: Optional[float] = False, **kwargs):
assert task in self.image_list
assert split in ['train', 'test']
data_list_file = os.path.join(root, "image_list", "{}_{}.txt".format(self.image_list[task], split))
print("loading {}".format(data_list_file))
if download:
list(map(lambda args: download_data(root, *args), self.download_list))
else:
list(map(lambda args: check_exits(root, args[0]), self.download_list))
super(DomainNet, self).__init__(root, DomainNet.CLASSES, data_list_file=data_list_file, **kwargs)
@classmethod
def domains(cls):
return list(cls.image_list.keys())
================================================
FILE: domain_generalization/Transfer-Learning-Library/common/vision/datasets/dtd.py
================================================
"""
@author: Junguang Jiang
@contact: JiangJunguang1123@outlook.com
"""
import os
from common.vision.datasets import ImageList
from ._util import download as download_data, check_exits
class DTD(ImageList):
"""
`The Describable Textures Dataset (DTD) `_ is an \
evolving collection of textural images in the wild, annotated with a series of human-centric attributes, \
inspired by the perceptual properties of textures. \
The task consists in classifying images of textural patterns (47 classes, with 120 training images each). \
Some of the textures are banded, bubbly, meshed, lined, or porous. \
The image size ranges between 300x300 and 640x640 pixels.
Args:
root (str): Root directory of dataset
split (str, optional): The dataset split, supports ``train``, or ``test``.
download (bool, optional): If true, downloads the dataset from the internet and puts it \
in root directory. If dataset is already downloaded, it is not downloaded again.
transform (callable, optional): A function/transform that takes in an PIL image and returns a \
transformed version. E.g, :class:`torchvision.transforms.RandomCrop`.
target_transform (callable, optional): A function/transform that takes in the target and transforms it.
"""
CLASSES =['banded', 'blotchy', 'braided', 'bubbly', 'bumpy', 'chequered', 'cobwebbed', 'cracked',
'crosshatched', 'crystalline', 'dotted', 'fibrous', 'flecked', 'freckled', 'frilly', 'gauzy',
'grid', 'grooved', 'honeycombed', 'interlaced', 'knitted', 'lacelike', 'lined', 'marbled',
'matted', 'meshed', 'paisley', 'perforated', 'pitted', 'pleated', 'polka-dotted', 'porous',
'potholed', 'scaly', 'smeared', 'spiralled', 'sprinkled', 'stained', 'stratified', 'striped',
'studded', 'swirly', 'veined', 'waffled', 'woven', 'wrinkled', 'zigzagged']
def __init__(self, root, split, download=False, **kwargs):
if download:
download_data(root, "dtd", "dtd.tgz", "https://cloud.tsinghua.edu.cn/f/97e7d188f0d74b5c9d36/?dl=1")
else:
check_exits(root, "dtd")
root = os.path.join(root, "dtd")
super(DTD, self).__init__(root, DTD.CLASSES, os.path.join(root, "imagelist", "{}.txt".format(split)), **kwargs)
================================================
FILE: domain_generalization/Transfer-Learning-Library/common/vision/datasets/eurosat.py
================================================
"""
@author: Junguang Jiang
@contact: JiangJunguang1123@outlook.com
"""
import os
from common.vision.datasets import ImageList
from ._util import download as download_data, check_exits
class EuroSAT(ImageList):
"""
`EuroSAT `_ dataset consists in classifying \
Sentinel-2 satellite images into 10 different types of land use (Residential, \
Industrial, River, Highway, etc). \
The spatial resolution corresponds to 10 meters per pixel, and the image size \
is 64x64 pixels.
Args:
root (str): Root directory of dataset
split (str, optional): The dataset split, supports ``train``, or ``test``.
download (bool, optional): If true, downloads the dataset from the internet and puts it \
in root directory. If dataset is already downloaded, it is not downloaded again.
transform (callable, optional): A function/transform that takes in an PIL image and returns a \
transformed version. E.g, :class:`torchvision.transforms.RandomCrop`.
target_transform (callable, optional): A function/transform that takes in the target and transforms it.
"""
CLASSES =['AnnualCrop', 'Forest', 'HerbaceousVegetation', 'Highway', 'Industrial', 'Pasture',
'PermanentCrop', 'Residential', 'River', 'SeaLake']
def __init__(self, root, split='train', download=False, **kwargs):
if download:
download_data(root, "eurosat", "eurosat.tgz", "https://cloud.tsinghua.edu.cn/f/9983d7ab86184d74bb17/?dl=1")
else:
check_exits(root, "eurosat")
split = 'train[:21600]' if split == 'train' else 'train[21600:]'
root = os.path.join(root, "eurosat")
super(EuroSAT, self).__init__(root, EuroSAT.CLASSES, os.path.join(root, "imagelist", "{}.txt".format(split)), **kwargs)
================================================
FILE: domain_generalization/Transfer-Learning-Library/common/vision/datasets/imagelist.py
================================================
"""
@author: Junguang Jiang
@contact: JiangJunguang1123@outlook.com
"""
import os
from typing import Optional, Callable, Tuple, Any, List
import torchvision.datasets as datasets
from torchvision.datasets.folder import default_loader
class ImageList(datasets.VisionDataset):
"""A generic Dataset class for image classification
Args:
root (str): Root directory of dataset
classes (list[str]): The names of all the classes
data_list_file (str): File to read the image list from.
transform (callable, optional): A function/transform that takes in an PIL image \
and returns a transformed version. E.g, :class:`torchvision.transforms.RandomCrop`.
target_transform (callable, optional): A function/transform that takes in the target and transforms it.
.. note:: In `data_list_file`, each line has 2 values in the following format.
::
source_dir/dog_xxx.png 0
source_dir/cat_123.png 1
target_dir/dog_xxy.png 0
target_dir/cat_nsdf3.png 1
The first value is the relative path of an image, and the second value is the label of the corresponding image.
If your data_list_file has different formats, please over-ride :meth:`~ImageList.parse_data_file`.
"""
def __init__(self, root: str, classes: List[str], data_list_file: str,
transform: Optional[Callable] = None, target_transform: Optional[Callable] = None):
super().__init__(root, transform=transform, target_transform=target_transform)
self.samples = self.parse_data_file(data_list_file)
self.classes = classes
self.class_to_idx = {cls: idx
for idx, cls in enumerate(self.classes)}
self.loader = default_loader
self.data_list_file = data_list_file
def __getitem__(self, index: int) -> Tuple[Any, int]:
"""
Args:
index (int): Index
return (tuple): (image, target) where target is index of the target class.
"""
path, target = self.samples[index]
img = self.loader(path)
if self.transform is not None:
img = self.transform(img)
if self.target_transform is not None and target is not None:
target = self.target_transform(target)
return img, target
def __len__(self) -> int:
return len(self.samples)
def parse_data_file(self, file_name: str) -> List[Tuple[str, int]]:
"""Parse file to data list
Args:
file_name (str): The path of data file
return (list): List of (image path, class_index) tuples
"""
with open(file_name, "r") as f:
data_list = []
for line in f.readlines():
split_line = line.split()
target = split_line[-1]
path = ' '.join(split_line[:-1])
if not os.path.isabs(path):
path = os.path.join(self.root, path)
target = int(target)
data_list.append((path, target))
return data_list
@property
def num_classes(self) -> int:
"""Number of classes"""
return len(self.classes)
@classmethod
def domains(cls):
"""All possible domain in this dataset"""
raise NotImplemented
================================================
FILE: domain_generalization/Transfer-Learning-Library/common/vision/datasets/imagenet_r.py
================================================
"""
@author: Junguang Jiang
@contact: JiangJunguang1123@outlook.com
"""
from typing import Optional
import os
from .imagelist import ImageList
from ._util import download as download_data, check_exits
class ImageNetR(ImageList):
"""ImageNet-R Dataset.
Args:
root (str): Root directory of dataset
task (str): The task (domain) to create dataset. Choices include ``'A'``: amazon, \
``'D'``: dslr and ``'W'``: webcam.
download (bool, optional): If true, downloads the dataset from the internet and puts it \
in root directory. If dataset is already downloaded, it is not downloaded again.
transform (callable, optional): A function/transform that takes in an PIL image and returns a \
transformed version. E.g, :class:`torchvision.transforms.RandomCrop`.
target_transform (callable, optional): A function/transform that takes in the target and transforms it.
.. note:: You need to put ``train`` directory of ImageNet-1K and ``imagenet_r`` directory of ImageNet-R
manually in `root` directory.
DALIB will only download ImageList automatically.
In `root`, there will exist following files after preparing.
::
train/
n02128385/
...
val/
imagenet-r/
n02128385/
image_list/
imagenet-train.txt
imagenet-r.txt
art.txt
...
"""
download_list = [
("image_list", "image_list.zip", "https://cloud.tsinghua.edu.cn/f/8066e6c5a8974be6a702/?dl=1"),
]
image_list = {
"IN": "image_list/imagenet-train.txt",
"IN-val": "image_list/imagenet-val.txt",
"INR": "image_list/imagenet-r.txt",
"art": "art.txt",
"embroidery": "embroidery.txt",
"misc": "misc.txt",
"sculpture": "sculpture.txt",
"tattoo": "tattoo.txt",
"cartoon": "cartoon.txt",
"graffiti": "graffiti.txt",
"origami": "origami.txt",
"sketch": "sketch.txt",
"toy": "toy.txt",
"deviantart": "deviantart.txt",
"graphic": "graphic.txt",
"painting": "painting.txt",
"sticker": "sticker.txt",
"videogame": "videogame.txt"
}
CLASSES = ['n01443537', 'n01484850', 'n01494475', 'n01498041', 'n01514859', 'n01518878', 'n01531178', 'n01534433', 'n01614925', 'n01616318', 'n01630670', 'n01632777', 'n01644373', 'n01677366', 'n01694178', 'n01748264', 'n01770393', 'n01774750', 'n01784675', 'n01806143', 'n01820546', 'n01833805', 'n01843383', 'n01847000', 'n01855672', 'n01860187', 'n01882714', 'n01910747', 'n01944390', 'n01983481', 'n01986214'
, 'n02007558', 'n02009912', 'n02051845', 'n02056570', 'n02066245', 'n02071294', 'n02077923', 'n02085620', 'n02086240', 'n02088094', 'n02088238', 'n02088364', 'n02088466', 'n02091032', 'n02091134', 'n02092339', 'n02094433', 'n02096585', 'n02097298', 'n02098286', 'n02099601', 'n02099712', 'n02102318', 'n02106030', 'n02106166', 'n02106550', 'n02106662', 'n02108089', 'n02108915', 'n02109525', 'n02110185', 'n02110341', 'n02110958', 'n02112018', 'n02112137', 'n02113023', 'n02113624', 'n02113799', 'n02114367', 'n02117135', 'n02119022', 'n02123045', 'n02128385', 'n02128757', 'n02129165', 'n02129604', 'n02130308', 'n02134084', 'n02138441', 'n02165456', 'n02190166', 'n02206856', 'n02219486', 'n02226429', 'n02233338', 'n02236044', 'n02268443', 'n02279972', 'n02317335', 'n02325366', 'n02346627', 'n02356798', 'n02363005', 'n02364673', 'n02391049', 'n02395406', 'n02398521', 'n02410509', 'n02423022', 'n02437616', 'n02445715', 'n02447366', 'n02480495', 'n02480855', 'n02481823', 'n02483362', 'n02486410', 'n02510455', 'n02526121', 'n02607072', 'n02655020', 'n02672831', 'n02701002', 'n02749479', 'n02769748', 'n02793495', 'n02797295', 'n02802426', 'n02808440', 'n02814860', 'n02823750', 'n02841315', 'n02843684', 'n02883205', 'n02906734', 'n02909870', 'n02939185', 'n02948072', 'n02950826', 'n02951358', 'n02966193', 'n02980441', 'n02992529', 'n03124170', 'n03272010', 'n03345487', 'n03372029', 'n03424325', 'n03452741', 'n03467068', 'n03481172', 'n03494278', 'n03495258', 'n03498962', 'n03594945', 'n03602883', 'n03630383', 'n03649909', 'n03676483', 'n03710193', 'n03773504', 'n03775071', 'n03888257', 'n03930630', 'n03947888', 'n04086273', 'n04118538', 'n04133789', 'n04141076', 'n04146614', 'n04147183', 'n04192698', 'n04254680', 'n04266014', 'n04275548', 'n04310018', 'n04325704', 'n04347754', 'n04389033', 'n04409515', 'n04465501', 'n04487394', 'n04522168', 'n04536866', 'n04552348', 'n04591713', 'n07614500', 'n07693725', 'n07695742', 'n07697313', 'n07697537', 'n07714571', 'n07714990', 'n07718472', 'n07720875', 'n07734744', 'n07742313', 'n07745940', 'n07749582', 'n07753275', 'n07753592', 'n07768694', 'n07873807', 'n07880968', 'n07920052', 'n09472597', 'n09835506', 'n10565667', 'n12267677']
def __init__(self, root: str, task: str, split: Optional[str] = 'all', download: Optional[bool] = True, **kwargs):
assert task in self.image_list
assert split in ["train", "val", "all"]
if task == "IN" and split == "val":
task = "IN-val"
data_list_file = os.path.join(root, self.image_list[task])
if download:
list(map(lambda args: download_data(root, *args), self.download_list))
else:
list(map(lambda file_name, _: check_exits(root, file_name), self.download_list))
super(ImageNetR, self).__init__(root, ImageNetR.CLASSES, data_list_file=data_list_file, **kwargs)
@classmethod
def domains(cls):
return list(cls.image_list.keys())
================================================
FILE: domain_generalization/Transfer-Learning-Library/common/vision/datasets/imagenet_sketch.py
================================================
"""
@author: Junguang Jiang
@contact: JiangJunguang1123@outlook.com
"""
from typing import Optional
import os
from torchvision.datasets.imagenet import ImageNet
from .imagelist import ImageList
from ._util import download as download_data, check_exits
class ImageNetSketch(ImageList):
"""ImageNet-Sketch Dataset.
Args:
root (str): Root directory of dataset
task (str): The task (domain) to create dataset. Choices include ``'A'``: amazon, \
``'D'``: dslr and ``'W'``: webcam.
download (bool, optional): If true, downloads the dataset from the internet and puts it \
in root directory. If dataset is already downloaded, it is not downloaded again.
transform (callable, optional): A function/transform that takes in an PIL image and returns a \
transformed version. E.g, :class:`torchvision.transforms.RandomCrop`.
target_transform (callable, optional): A function/transform that takes in the target and transforms it.
.. note:: You need to put ``train`` directory, ``metabin`` of ImageNet-1K and ``sketch`` directory of ImageNet-Sketch
manually in `root` directory.
DALIB will only download ImageList automatically.
In `root`, there will exist following files after preparing.
::
metabin (from ImageNet)
train/
n02128385/
...
val/
sketch/
n02128385/
image_list/
imagenet-train.txt
sketch.txt
...
"""
# TODO
download_list = [
("image_list", "image_list.zip", "https://cloud.tsinghua.edu.cn/f/8e12fff7b6224e5fa62b/?dl=1"),
]
image_list = {
"IN": "image_list/imagenet-train.txt",
"IN-val": "image_list/imagenet-val.txt",
"sketch": "image_list/sketch.txt",
}
def __init__(self, root: str, task: str, split: Optional[str] = 'all', download: Optional[bool] = True, **kwargs):
assert task in self.image_list
assert split in ["train", "val", "all"]
if task == "IN" and split == "val":
task = "IN-val"
data_list_file = os.path.join(root, self.image_list[task])
if download:
list(map(lambda args: download_data(root, *args), self.download_list))
else:
list(map(lambda file_name, _: check_exits(root, file_name), self.download_list))
super(ImageNetSketch, self).__init__(root, ImageNet(root).classes, data_list_file=data_list_file, **kwargs)
@classmethod
def domains(cls):
return list(cls.image_list.keys())
================================================
FILE: domain_generalization/Transfer-Learning-Library/common/vision/datasets/keypoint_detection/__init__.py
================================================
from .rendered_hand_pose import RenderedHandPose
from .hand_3d_studio import Hand3DStudio, Hand3DStudioAll
from .freihand import FreiHand
from .surreal import SURREAL
from .lsp import LSP
from .human36m import Human36M
================================================
FILE: domain_generalization/Transfer-Learning-Library/common/vision/datasets/keypoint_detection/freihand.py
================================================
"""
@author: Junguang Jiang
@contact: JiangJunguang1123@outlook.com
"""
import json
import time
import torch
import os
import os.path as osp
from torchvision.datasets.utils import download_and_extract_archive
from ...transforms.keypoint_detection import *
from .keypoint_dataset import Hand21KeypointDataset
from .util import *
""" General util functions. """
def _assert_exist(p):
msg = 'File does not exists: %s' % p
assert os.path.exists(p), msg
def json_load(p):
_assert_exist(p)
with open(p, 'r') as fi:
d = json.load(fi)
return d
def load_db_annotation(base_path, set_name=None):
if set_name is None:
# only training set annotations are released so this is a valid default choice
set_name = 'training'
print('Loading FreiHAND dataset index ...')
t = time.time()
# assumed paths to data containers
k_path = os.path.join(base_path, '%s_K.json' % set_name)
mano_path = os.path.join(base_path, '%s_mano.json' % set_name)
xyz_path = os.path.join(base_path, '%s_xyz.json' % set_name)
# load if exist
K_list = json_load(k_path)
mano_list = json_load(mano_path)
xyz_list = json_load(xyz_path)
# should have all the same length
assert len(K_list) == len(mano_list), 'Size mismatch.'
assert len(K_list) == len(xyz_list), 'Size mismatch.'
print('Loading of %d samples done in %.2f seconds' % (len(K_list), time.time()-t))
return list(zip(K_list, mano_list, xyz_list))
def projectPoints(xyz, K):
""" Project 3D coordinates into image space. """
xyz = np.array(xyz)
K = np.array(K)
uv = np.matmul(K, xyz.T).T
return uv[:, :2] / uv[:, -1:]
""" Dataset related functions. """
def db_size(set_name):
""" Hardcoded size of the datasets. """
if set_name == 'training':
return 32560 # number of unique samples (they exists in multiple 'versions')
elif set_name == 'evaluation':
return 3960
else:
assert 0, 'Invalid choice.'
class sample_version:
gs = 'gs' # green screen
hom = 'hom' # homogenized
sample = 'sample' # auto colorization with sample points
auto = 'auto' # auto colorization without sample points: automatic color hallucination
db_size = db_size('training')
@classmethod
def valid_options(cls):
return [cls.gs, cls.hom, cls.sample, cls.auto]
@classmethod
def check_valid(cls, version):
msg = 'Invalid choice: "%s" (must be in %s)' % (version, cls.valid_options())
assert version in cls.valid_options(), msg
@classmethod
def map_id(cls, id, version):
cls.check_valid(version)
return id + cls.db_size*cls.valid_options().index(version)
class FreiHand(Hand21KeypointDataset):
"""`FreiHand Dataset `_
Args:
root (str): Root directory of dataset
split (str, optional): The dataset split, supports ``train``, ``test``, or ``all``.
task (str, optional): The post-processing option to create dataset. Choices include ``'gs'``: green screen \
recording, ``'auto'``: auto colorization without sample points: automatic color hallucination, \
``'sample'``: auto colorization with sample points, ``'hom'``: homogenized, \
and ``'all'``: all hands. Default: 'all'.
download (bool, optional): If true, downloads the dataset from the internet and puts it \
in root directory. If dataset is already downloaded, it is not downloaded again.
transforms (callable, optional): A function/transform that takes in a dict (which contains PIL image and
its labels) and returns a transformed version. E.g, :class:`~common.vision.transforms.keypoint_detection.Resize`.
image_size (tuple): (width, height) of the image. Default: (256, 256)
heatmap_size (tuple): (width, height) of the heatmap. Default: (64, 64)
sigma (int): sigma parameter when generate the heatmap. Default: 2
.. note:: In `root`, there will exist following files after downloading.
::
*.json
training/
evaluation/
"""
def __init__(self, root, split='train', task='all', download=True, **kwargs):
if download:
if not osp.exists(osp.join(root, "training")) or not osp.exists(osp.join(root, "evaluation")):
download_and_extract_archive("https://lmb.informatik.uni-freiburg.de/data/freihand/FreiHAND_pub_v2.zip",
download_root=root, filename="FreiHAND_pub_v2.zip", remove_finished=False,
extract_root=root)
assert split in ['train', 'test', 'all']
self.split = split
assert task in ['all', 'gs', 'auto', 'sample', 'hom']
self.task = task
if task == 'all':
samples = self.get_samples(root, 'gs') + self.get_samples(root, 'auto') + self.get_samples(root, 'sample') + self.get_samples(root, 'hom')
else:
samples = self.get_samples(root, task)
random.seed(42)
random.shuffle(samples)
samples_len = len(samples)
samples_split = min(int(samples_len * 0.2), 3200)
if self.split == 'train':
samples = samples[samples_split:]
elif self.split == 'test':
samples = samples[:samples_split]
super(FreiHand, self).__init__(root, samples, **kwargs)
def __getitem__(self, index):
sample = self.samples[index]
image_name = sample['name']
image_path = os.path.join(self.root, image_name)
image = Image.open(image_path)
keypoint3d_camera = np.array(sample['keypoint3d']) # NUM_KEYPOINTS x 3
keypoint2d = np.array(sample['keypoint2d']) # NUM_KEYPOINTS x 2
intrinsic_matrix = np.array(sample['intrinsic_matrix'])
Zc = keypoint3d_camera[:, 2]
# Crop the images such that the hand is at the center of the image
# The images will be 1.5 times larger than the hand
# The crop process will change Xc and Yc, leaving Zc with no changes
bounding_box = get_bounding_box(keypoint2d)
w, h = image.size
left, upper, right, lower = scale_box(bounding_box, w, h, 1.5)
image, keypoint2d = crop(image, upper, left, lower - upper, right - left, keypoint2d)
# Change all hands to right hands
if sample['left'] is False:
image, keypoint2d = hflip(image, keypoint2d)
image, data = self.transforms(image, keypoint2d=keypoint2d, intrinsic_matrix=intrinsic_matrix)
keypoint2d = data['keypoint2d']
intrinsic_matrix = data['intrinsic_matrix']
keypoint3d_camera = keypoint2d_to_3d(keypoint2d, intrinsic_matrix, Zc)
# noramlize 2D pose:
visible = np.ones((self.num_keypoints, ), dtype=np.float32)
visible = visible[:, np.newaxis]
# 2D heatmap
target, target_weight = generate_target(keypoint2d, visible, self.heatmap_size, self.sigma, self.image_size)
target = torch.from_numpy(target)
target_weight = torch.from_numpy(target_weight)
# normalize 3D pose:
# put middle finger metacarpophalangeal (MCP) joint in the center of the coordinate system
# and make distance between wrist and middle finger MCP joint to be of length 1
keypoint3d_n = keypoint3d_camera - keypoint3d_camera[9:10, :]
keypoint3d_n = keypoint3d_n / np.sqrt(np.sum(keypoint3d_n[0, :] ** 2))
z = keypoint3d_n[:, 2]
meta = {
'image': image_name,
'keypoint2d': keypoint2d, # （NUM_KEYPOINTS x 2）
'keypoint3d': keypoint3d_n, # （NUM_KEYPOINTS x 3）
'z': z,
}
return image, target, target_weight, meta
def get_samples(self, root, version='gs'):
set = 'training'
# load annotations of this set
db_data_anno = load_db_annotation(root, set)
version_map = {
'gs': sample_version.gs,
'hom': sample_version.hom,
'sample': sample_version.sample,
'auto': sample_version.auto
}
samples = []
for idx in range(db_size(set)):
image_name = os.path.join(set, 'rgb',
'%08d.jpg' % sample_version.map_id(idx, version_map[version]))
mask_name = os.path.join(set, 'mask', '%08d.jpg' % idx)
intrinsic_matrix, mano, keypoint3d = db_data_anno[idx]
keypoint2d = projectPoints(keypoint3d, intrinsic_matrix)
sample = {
'name': image_name,
'mask_name': mask_name,
'keypoint2d': keypoint2d,
'keypoint3d': keypoint3d,
'intrinsic_matrix': intrinsic_matrix,
'left': False
}
samples.append(sample)
return samples
================================================
FILE: domain_generalization/Transfer-Learning-Library/common/vision/datasets/keypoint_detection/hand_3d_studio.py
================================================
"""
@author: Junguang Jiang
@contact: JiangJunguang1123@outlook.com
"""
import os
import json
import random
from PIL import ImageFile, Image
import torch
import os.path as osp
from .._util import download as download_data, check_exits
from .keypoint_dataset import Hand21KeypointDataset
from .util import *
ImageFile.LOAD_TRUNCATED_IMAGES = True
class Hand3DStudio(Hand21KeypointDataset):
"""`Hand-3d-Studio Dataset `_
Args:
root (str): Root directory of dataset
split (str, optional): The dataset split, supports ``train``, ``test``, or ``all``.
task (str, optional): The task to create dataset. Choices include ``'noobject'``: only hands without objects, \
``'object'``: only hands interacting with hands, and ``'all'``: all hands. Default: 'noobject'.
download (bool, optional): If true, downloads the dataset from the internet and puts it \
in root directory. If dataset is already downloaded, it is not downloaded again.
transforms (callable, optional): A function/transform that takes in a dict (which contains PIL image and
its labels) and returns a transformed version. E.g, :class:`~common.vision.transforms.keypoint_detection.Resize`.
image_size (tuple): (width, height) of the image. Default: (256, 256)
heatmap_size (tuple): (width, height) of the heatmap. Default: (64, 64)
sigma (int): sigma parameter when generate the heatmap. Default: 2
.. note::
We found that the original H3D image is in high resolution while most part in an image is background,
thus we crop the image and keep only the surrounding area of hands (1.5x bigger than hands) to speed up training.
.. note:: In `root`, there will exist following files after downloading.
::
H3D_crop/
annotation.json
part1/
part2/
part3/
part4/
part5/
"""
def __init__(self, root, split='train', task='noobject', download=True, **kwargs):
assert split in ['train', 'test', 'all']
self.split = split
assert task in ['noobject', 'object', 'all']
self.task = task
if download:
download_data(root, "H3D_crop", "H3D_crop.tar", "https://cloud.tsinghua.edu.cn/f/d4e612e44dc04d8eb01f/?dl=1")
else:
check_exits(root, "H3D_crop")
root = osp.join(root, "H3D_crop")
# load labels
annotation_file = os.path.join(root, 'annotation.json')
print("loading from {}".format(annotation_file))
with open(annotation_file) as f:
samples = list(json.load(f))
if task == 'noobject':
samples = [sample for sample in samples if int(sample['without_object']) == 1]
elif task == 'object':
samples = [sample for sample in samples if int(sample['without_object']) == 0]
random.seed(42)
random.shuffle(samples)
samples_len = len(samples)
samples_split = min(int(samples_len * 0.2), 3200)
if split == 'train':
samples = samples[samples_split:]
elif split == 'test':
samples = samples[:samples_split]
super(Hand3DStudio, self).__init__(root, samples, **kwargs)
def __getitem__(self, index):
sample = self.samples[index]
image_name = sample['name']
image_path = os.path.join(self.root, image_name)
image = Image.open(image_path)
keypoint3d_camera = np.array(sample['keypoint3d']) # NUM_KEYPOINTS x 3
keypoint2d = np.array(sample['keypoint2d']) # NUM_KEYPOINTS x 2
intrinsic_matrix = np.array(sample['intrinsic_matrix'])
Zc = keypoint3d_camera[:, 2]
image, data = self.transforms(image, keypoint2d=keypoint2d, intrinsic_matrix=intrinsic_matrix)
keypoint2d = data['keypoint2d']
intrinsic_matrix = data['intrinsic_matrix']
keypoint3d_camera = keypoint2d_to_3d(keypoint2d, intrinsic_matrix, Zc)
# noramlize 2D pose:
visible = np.ones((self.num_keypoints, ), dtype=np.float32)
visible = visible[:, np.newaxis]
# 2D heatmap
target, target_weight = generate_target(keypoint2d, visible, self.heatmap_size, self.sigma, self.image_size)
target = torch.from_numpy(target)
target_weight = torch.from_numpy(target_weight)
# normalize 3D pose:
# put middle finger metacarpophalangeal (MCP) joint in the center of the coordinate system
# and make distance between wrist and middle finger MCP joint to be of length 1
keypoint3d_n = keypoint3d_camera - keypoint3d_camera[9:10, :]
keypoint3d_n = keypoint3d_n / np.sqrt(np.sum(keypoint3d_n[0, :] ** 2))
meta = {
'image': image_name,
'keypoint2d': keypoint2d, # （NUM_KEYPOINTS x 2）
'keypoint3d': keypoint3d_n, # （NUM_KEYPOINTS x 3）
}
return image, target, target_weight, meta
class Hand3DStudioAll(Hand3DStudio):
"""
`Hand-3d-Studio Dataset `_
"""
def __init__(self, root, task='all', **kwargs):
super(Hand3DStudioAll, self).__init__(root, task=task, **kwargs)
================================================
FILE: domain_generalization/Transfer-Learning-Library/common/vision/datasets/keypoint_detection/human36m.py
================================================
"""
@author: Junguang Jiang
@contact: JiangJunguang1123@outlook.com
"""
import os
import json
import tqdm
from PIL import ImageFile
import torch
from .keypoint_dataset import Body16KeypointDataset
from ...transforms.keypoint_detection import *
from .util import *
ImageFile.LOAD_TRUNCATED_IMAGES = True
class Human36M(Body16KeypointDataset):
"""`Human3.6M Dataset `_
Args:
root (str): Root directory of dataset
split (str, optional): The dataset split, supports ``train``, ``test``, or ``all``.
Default: ``train``.
task (str, optional): Placeholder.
download (bool, optional): Placeholder.
transforms (callable, optional): A function/transform that takes in a dict (which contains PIL image and
its labels) and returns a transformed version. E.g, :class:`~common.vision.transforms.keypoint_detection.Resize`.
image_size (tuple): (width, height) of the image. Default: (256, 256)
heatmap_size (tuple): (width, height) of the heatmap. Default: (64, 64)
sigma (int): sigma parameter when generate the heatmap. Default: 2
.. note:: You need to download Human36M manually.
Ensure that there exist following files in the `root` directory before you using this class.
::
annotations/
Human36M_subject11_joint_3d.json
...
images/
.. note::
We found that the original Human3.6M image is in high resolution while most part in an image is background,
thus we crop the image and keep only the surrounding area of hands (1.5x bigger than hands) to speed up training.
In `root`, there will exist following files after crop.
::
Human36M_crop/
annotations/
keypoints2d_11.json
...
"""
def __init__(self, root, split='train', task='all', download=True, **kwargs):
assert split in ['train', 'test', 'all']
self.split = split
samples = []
if self.split == 'train':
parts = [1, 5, 6, 7, 8]
elif self.split == 'test':
parts = [9, 11]
else:
parts = [1, 5, 6, 7, 8, 9, 11]
for part in parts:
annotation_file = os.path.join(root, 'annotations/keypoints2d_{}.json'.format(part))
if not os.path.exists(annotation_file):
self.preprocess(part, root)
print("loading", annotation_file)
with open(annotation_file) as f:
samples.extend(json.load(f))
# decrease the number of test samples to decrease the time spent on test
random.seed(42)
if self.split == 'test':
samples = random.choices(samples, k=3200)
super(Human36M, self).__init__(root, samples, **kwargs)
def __getitem__(self, index):
sample = self.samples[index]
image_name = sample['name']
image_path = os.path.join(self.root, "crop_images", image_name)
image = Image.open(image_path)
keypoint3d_camera = np.array(sample['keypoint3d']) # NUM_KEYPOINTS x 3
keypoint2d = np.array(sample['keypoint2d']) # NUM_KEYPOINTS x 2
intrinsic_matrix = np.array(sample['intrinsic_matrix'])
Zc = keypoint3d_camera[:, 2]
image, data = self.transforms(image, keypoint2d=keypoint2d, intrinsic_matrix=intrinsic_matrix)
keypoint2d = data['keypoint2d']
intrinsic_matrix = data['intrinsic_matrix']
keypoint3d_camera = keypoint2d_to_3d(keypoint2d, intrinsic_matrix, Zc)
# noramlize 2D pose:
visible = np.ones((self.num_keypoints, ), dtype=np.float32)
visible = visible[:, np.newaxis]
# 2D heatmap
target, target_weight = generate_target(keypoint2d, visible, self.heatmap_size, self.sigma, self.image_size)
target = torch.from_numpy(target)
target_weight = torch.from_numpy(target_weight)
# normalize 3D pose:
# put middle finger metacarpophalangeal (MCP) joint in the center of the coordinate system
# and make distance between wrist and middle finger MCP joint to be of length 1
keypoint3d_n = keypoint3d_camera - keypoint3d_camera[9:10, :]
keypoint3d_n = keypoint3d_n / np.sqrt(np.sum(keypoint3d_n[0, :] ** 2))
meta = {
'image': image_name,
'keypoint2d': keypoint2d, # （NUM_KEYPOINTS x 2）
'keypoint3d': keypoint3d_n, # （NUM_KEYPOINTS x 3）
}
return image, target, target_weight, meta
def preprocess(self, part, root):
body_index = [3, 2, 1, 4, 5, 6, 0, 11, 8, 10, 16, 15, 14, 11, 12, 13]
image_size = 512
print("preprocessing part", part)
camera_json = os.path.join(root, "annotations", "Human36M_subject{}_camera.json".format(part))
data_json = os.path.join(root, "annotations", "Human36M_subject{}_data.json".format(part))
joint_3d_json = os.path.join(root, "annotations", "Human36M_subject{}_joint_3d.json".format(part))
with open(camera_json, "r") as f:
cameras = json.load(f)
with open(data_json, "r") as f:
data = json.load(f)
images = data['images']
with open(joint_3d_json, "r") as f:
joints_3d = json.load(f)
data = []
for i, image_data in enumerate(tqdm.tqdm(images)):
# downsample
if i % 5 == 0:
keypoint3d = np.array(joints_3d[str(image_data["action_idx"])][str(image_data["subaction_idx"])][
str(image_data["frame_idx"])])
keypoint3d = keypoint3d[body_index, :]
keypoint3d[7, :] = 0.5 * (keypoint3d[12, :] + keypoint3d[13, :])
camera = cameras[str(image_data["cam_idx"])]
R, T = np.array(camera["R"]), np.array(camera['t'])[:, np.newaxis]
extrinsic_matrix = np.concatenate([R, T], axis=1)
keypoint3d_camera = np.matmul(extrinsic_matrix, np.hstack(
(keypoint3d, np.ones((keypoint3d.shape[0], 1)))).T) # (3 x NUM_KEYPOINTS)
Z_c = keypoint3d_camera[2:3, :] # 1 x NUM_KEYPOINTS
f, c = np.array(camera["f"]), np.array(camera['c'])
intrinsic_matrix = np.zeros((3, 3))
intrinsic_matrix[0, 0] = f[0]
intrinsic_matrix[1, 1] = f[1]
intrinsic_matrix[0, 2] = c[0]
intrinsic_matrix[1, 2] = c[1]
intrinsic_matrix[2, 2] = 1
keypoint2d = np.matmul(intrinsic_matrix, keypoint3d_camera) # (3 x NUM_KEYPOINTS)
keypoint2d = keypoint2d[0: 2, :] / Z_c
keypoint2d = keypoint2d.T
src_image_path = os.path.join(root, "images", image_data['file_name'])
tgt_image_path = os.path.join(root, "crop_images", image_data['file_name'])
os.makedirs(os.path.dirname(tgt_image_path), exist_ok=True)
image = Image.open(src_image_path)
bounding_box = get_bounding_box(keypoint2d)
w, h = image.size
left, upper, right, lower = scale_box(bounding_box, w, h, 1.5)
image, keypoint2d = crop(image, upper, left, lower-upper+1, right-left+1, keypoint2d)
Z_c = Z_c.T
# Calculate XYZ from uvz
uv1 = np.concatenate([np.copy(keypoint2d), np.ones((16, 1))],
axis=1) # NUM_KEYPOINTS x 3
uv1 = uv1 * Z_c # NUM_KEYPOINTS x 3
keypoint3d_camera = np.matmul(np.linalg.inv(intrinsic_matrix), uv1.T).T
# resize image will change camera intrinsic matrix
w, h = image.size
image = image.resize((image_size, image_size))
image.save(tgt_image_path)
zoom_factor = float(w) / float(image_size)
keypoint2d /= zoom_factor
intrinsic_matrix[0, 0] /= zoom_factor
intrinsic_matrix[1, 1] /= zoom_factor
intrinsic_matrix[0, 2] /= zoom_factor
intrinsic_matrix[1, 2] /= zoom_factor
data.append({
"name": image_data['file_name'],
'keypoint2d': keypoint2d.tolist(),
'keypoint3d': keypoint3d_camera.tolist(),
'intrinsic_matrix': intrinsic_matrix.tolist(),
})
with open(os.path.join(root, "annotations", "keypoints2d_{}.json".format(part)), "w") as f:
json.dump(data, f)
================================================
FILE: domain_generalization/Transfer-Learning-Library/common/vision/datasets/keypoint_detection/keypoint_dataset.py
================================================
"""
@author: Junguang Jiang
@contact: JiangJunguang1123@outlook.com
"""
from abc import ABC
import numpy as np
from torch.utils.data.dataset import Dataset
from webcolors import name_to_rgb
import cv2
class KeypointDataset(Dataset, ABC):
"""A generic dataset class for image keypoint detection
Args:
root (str): Root directory of dataset
num_keypoints (int): Number of keypoints
samples (list): list of data
transforms (callable, optional): A function/transform that takes in a dict (which contains PIL image and
its labels) and returns a transformed version. E.g, :class:`~common.vision.transforms.keypoint_detection.Resize`.
image_size (tuple): (width, height) of the image. Default: (256, 256)
heatmap_size (tuple): (width, height) of the heatmap. Default: (64, 64)
sigma (int): sigma parameter when generate the heatmap. Default: 2
keypoints_group (dict): a dict that stores the index of different types of keypoints
colored_skeleton (dict): a dict that stores the index and color of different skeleton
"""
def __init__(self, root, num_keypoints, samples, transforms=None, image_size=(256, 256), heatmap_size=(64, 64),
sigma=2, keypoints_group=None, colored_skeleton=None):
self.root = root
self.num_keypoints = num_keypoints
self.samples = samples
self.transforms = transforms
self.image_size = image_size
self.heatmap_size = heatmap_size
self.sigma = sigma
self.keypoints_group = keypoints_group
self.colored_skeleton = colored_skeleton
def __len__(self):
return len(self.samples)
def visualize(self, image, keypoints, filename):
"""Visualize an image with its keypoints, and store the result into a file
Args:
image (PIL.Image):
keypoints (torch.Tensor): keypoints in shape K x 2
filename (str): the name of file to store
"""
assert self.colored_skeleton is not None
image = cv2.cvtColor(np.asarray(image), cv2.COLOR_RGB2BGR).copy()
for (_, (line, color)) in self.colored_skeleton.items():
for i in range(len(line) - 1):
start, end = keypoints[line[i]], keypoints[line[i + 1]]
cv2.line(image, (int(start[0]), int(start[1])), (int(end[0]), int(end[1])), color=name_to_rgb(color),
thickness=3)
for keypoint in keypoints:
cv2.circle(image, (int(keypoint[0]), int(keypoint[1])), 3, name_to_rgb('black'), 1)
cv2.imwrite(filename, image)
def group_accuracy(self, accuracies):
""" Group the accuracy of K keypoints into different kinds.
Args:
accuracies (list): accuracy of the K keypoints
Returns:
accuracy of ``N=len(keypoints_group)`` kinds of keypoints
"""
grouped_accuracies = dict()
for name, keypoints in self.keypoints_group.items():
grouped_accuracies[name] = sum([accuracies[idx] for idx in keypoints]) / len(keypoints)
return grouped_accuracies
class Body16KeypointDataset(KeypointDataset, ABC):
"""
Dataset with 16 body keypoints.
"""
# TODO: add image
head = (9,)
shoulder = (12, 13)
elbow = (11, 14)
wrist = (10, 15)
hip = (2, 3)
knee = (1, 4)
ankle = (0, 5)
all = (12, 13, 11, 14, 10, 15, 2, 3, 1, 4, 0, 5)
right_leg = (0, 1, 2, 8)
left_leg = (5, 4, 3, 8)
backbone = (8, 9)
right_arm = (10, 11, 12, 8)
left_arm = (15, 14, 13, 8)
def __init__(self, root, samples, **kwargs):
colored_skeleton = {
"right_leg": (self.right_leg, 'yellow'),
"left_leg": (self.left_leg, 'green'),
"backbone": (self.backbone, 'blue'),
"right_arm": (self.right_arm, 'purple'),
"left_arm": (self.left_arm, 'red'),
}
keypoints_group = {
"head": self.head,
"shoulder": self.shoulder,
"elbow": self.elbow,
"wrist": self.wrist,
"hip": self.hip,
"knee": self.knee,
"ankle": self.ankle,
"all": self.all
}
super(Body16KeypointDataset, self).__init__(root, 16, samples, keypoints_group=keypoints_group,
colored_skeleton=colored_skeleton, **kwargs)
class Hand21KeypointDataset(KeypointDataset, ABC):
"""
Dataset with 21 hand keypoints.
"""
# TODO: add image
MCP = (1, 5, 9, 13, 17)
PIP = (2, 6, 10, 14, 18)
DIP = (3, 7, 11, 15, 19)
fingertip = (4, 8, 12, 16, 20)
all = tuple(range(21))
thumb = (0, 1, 2, 3, 4)
index_finger = (0, 5, 6, 7, 8)
middle_finger = (0, 9, 10, 11, 12)
ring_finger = (0, 13, 14, 15, 16)
little_finger = (0, 17, 18, 19, 20)
def __init__(self, root, samples, **kwargs):
colored_skeleton = {
"thumb": (self.thumb, 'yellow'),
"index_finger": (self.index_finger, 'green'),
"middle_finger": (self.middle_finger, 'blue'),
"ring_finger": (self.ring_finger, 'purple'),
"little_finger": (self.little_finger, 'red'),
}
keypoints_group = {
"MCP": self.MCP,
"PIP": self.PIP,
"DIP": self.DIP,
"fingertip": self.fingertip,
"all": self.all
}
super(Hand21KeypointDataset, self).__init__(root, 21, samples, keypoints_group=keypoints_group,
colored_skeleton=colored_skeleton, **kwargs)
================================================
FILE: domain_generalization/Transfer-Learning-Library/common/vision/datasets/keypoint_detection/lsp.py
================================================
"""
@author: Junguang Jiang
@contact: JiangJunguang1123@outlook.com
"""
import scipy.io as scio
import os
from PIL import ImageFile
import torch
from .keypoint_dataset import Body16KeypointDataset
from ...transforms.keypoint_detection import *
from .util import *
from .._util import download as download_data, check_exits
ImageFile.LOAD_TRUNCATED_IMAGES = True
class LSP(Body16KeypointDataset):
"""`Leeds Sports Pose Dataset `_
Args:
root (str): Root directory of dataset
split (str, optional): PlaceHolder.
task (str, optional): Placeholder.
download (bool, optional): If true, downloads the dataset from the internet and puts it \
in root directory. If dataset is already downloaded, it is not downloaded again.
transforms (callable, optional): PlaceHolder.
heatmap_size (tuple): (width, height) of the heatmap. Default: (64, 64)
sigma (int): sigma parameter when generate the heatmap. Default: 2
.. note:: In `root`, there will exist following files after downloading.
::
lsp/
images/
joints.mat
.. note::
LSP is only used for target domain. Due to the small dataset size, the whole dataset is used
no matter what ``split`` is. Also, the transform is fixed.
"""
def __init__(self, root, split='train', task='all', download=True, image_size=(256, 256), transforms=None, **kwargs):
if download:
download_data(root, "images", "lsp_dataset.zip",
"https://cloud.tsinghua.edu.cn/f/46ea73c89abc46bfb125/?dl=1")
else:
check_exits(root, "lsp")
assert split in ['train', 'test', 'all']
self.split = split
samples = []
annotations = scio.loadmat(os.path.join(root, "joints.mat"))['joints'].transpose((2, 1, 0))
for i in range(0, 2000):
image = "im{0:04d}.jpg".format(i+1)
annotation = annotations[i]
samples.append((image, annotation))
self.joints_index = (0, 1, 2, 3, 4, 5, 13, 13, 12, 13, 6, 7, 8, 9, 10, 11)
self.visible = np.array([1.] * 6 + [0, 0] + [1.] * 8, dtype=np.float32)
normalize = Normalize([0.485, 0.456, 0.406], [0.229, 0.224, 0.225])
transforms = Compose([
ResizePad(image_size[0]),
ToTensor(),
normalize
])
super(LSP, self).__init__(root, samples, transforms=transforms, image_size=image_size, **kwargs)
def __getitem__(self, index):
sample = self.samples[index]
image_name = sample[0]
image = Image.open(os.path.join(self.root, "images", image_name))
keypoint2d = sample[1][self.joints_index, :2]
image, data = self.transforms(image, keypoint2d=keypoint2d)
keypoint2d = data['keypoint2d']
visible = self.visible * (1-sample[1][self.joints_index, 2])
visible = visible[:, np.newaxis]
# 2D heatmap
target, target_weight = generate_target(keypoint2d, visible, self.heatmap_size, self.sigma, self.image_size)
target = torch.from_numpy(target)
target_weight = torch.from_numpy(target_weight)
meta = {
'image': image_name,
'keypoint2d': keypoint2d, # （NUM_KEYPOINTS x 2）
'keypoint3d': np.zeros((self.num_keypoints, 3)).astype(keypoint2d.dtype), # （NUM_KEYPOINTS x 3）
}
return image, target, target_weight, meta
================================================
FILE: domain_generalization/Transfer-Learning-Library/common/vision/datasets/keypoint_detection/rendered_hand_pose.py
================================================
"""
@author: Junguang Jiang
@contact: JiangJunguang1123@outlook.com
"""
import torch
import os
import pickle
from .._util import download as download_data, check_exits
from ...transforms.keypoint_detection import *
from .keypoint_dataset import Hand21KeypointDataset
from .util import *
class RenderedHandPose(Hand21KeypointDataset):
"""`Rendered Handpose Dataset `_
Args:
root (str): Root directory of dataset
split (str, optional): The dataset split, supports ``train``, ``test``, or ``all``.
task (str, optional): Placeholder.
download (bool, optional): If true, downloads the dataset from the internet and puts it \
in root directory. If dataset is already downloaded, it is not downloaded again.
transforms (callable, optional): A function/transform that takes in a dict (which contains PIL image and
its labels) and returns a transformed version. E.g, :class:`~common.vision.transforms.keypoint_detection.Resize`.
image_size (tuple): (width, height) of the image. Default: (256, 256)
heatmap_size (tuple): (width, height) of the heatmap. Default: (64, 64)
sigma (int): sigma parameter when generate the heatmap. Default: 2
.. note:: In `root`, there will exist following files after downloading.
::
RHD_published_v2/
training/
evaluation/
"""
def __init__(self, root, split='train', task='all', download=True, **kwargs):
if download:
download_data(root, "RHD_published_v2", "RHD_v1-1.zip", "https://lmb.informatik.uni-freiburg.de/data/RenderedHandpose/RHD_v1-1.zip")
else:
check_exits(root, "RHD_published_v2")
root = os.path.join(root, "RHD_published_v2")
assert split in ['train', 'test', 'all']
self.split = split
if split == 'all':
samples = self.get_samples(root, 'train') + self.get_samples(root, 'test')
else:
samples = self.get_samples(root, split)
super(RenderedHandPose, self).__init__(
root, samples, **kwargs)
def __getitem__(self, index):
sample = self.samples[index]
image_name = sample['name']
image_path = os.path.join(self.root, image_name)
image = Image.open(image_path)
keypoint3d_camera = np.array(sample['keypoint3d']) # NUM_KEYPOINTS x 3
keypoint2d = np.array(sample['keypoint2d']) # NUM_KEYPOINTS x 2
intrinsic_matrix = np.array(sample['intrinsic_matrix'])
Zc = keypoint3d_camera[:, 2]
# Crop the images such that the hand is at the center of the image
# The images will be 1.5 times larger than the hand
# The crop process will change Xc and Yc, leaving Zc with no changes
bounding_box = get_bounding_box(keypoint2d)
w, h = image.size
left, upper, right, lower = scale_box(bounding_box, w, h, 1.5)
image, keypoint2d = crop(image, upper, left, lower - upper, right - left, keypoint2d)
# Change all hands to right hands
if sample['left'] is False:
image, keypoint2d = hflip(image, keypoint2d)
image, data = self.transforms(image, keypoint2d=keypoint2d, intrinsic_matrix=intrinsic_matrix)
keypoint2d = data['keypoint2d']
intrinsic_matrix = data['intrinsic_matrix']
keypoint3d_camera = keypoint2d_to_3d(keypoint2d, intrinsic_matrix, Zc)
# noramlize 2D pose:
visible = np.array(sample['visible'], dtype=np.float32)
visible = visible[:, np.newaxis]
# 2D heatmap
target, target_weight = generate_target(keypoint2d, visible, self.heatmap_size, self.sigma, self.image_size)
target = torch.from_numpy(target)
target_weight = torch.from_numpy(target_weight)
# normalize 3D pose:
# put middle finger metacarpophalangeal (MCP) joint in the center of the coordinate system
# and make distance between wrist and middle finger MCP joint to be of length 1
keypoint3d_n = keypoint3d_camera - keypoint3d_camera[9:10, :]
keypoint3d_n = keypoint3d_n / np.sqrt(np.sum(keypoint3d_n[0, :] ** 2))
z = keypoint3d_n[:, 2]
meta = {
'image': image_name,
'keypoint2d': keypoint2d, # （NUM_KEYPOINTS x 2）
'keypoint3d': keypoint3d_n, # （NUM_KEYPOINTS x 3）
'z': z,
}
return image, target, target_weight, meta
def get_samples(self, root, task, min_size=64):
if task == 'train':
set = 'training'
else:
set = 'evaluation'
# load annotations of this set
with open(os.path.join(root, set, 'anno_%s.pickle' % set), 'rb') as fi:
anno_all = pickle.load(fi)
samples = []
left_hand_index = [0, 4, 3, 2, 1, 8, 7, 6, 5, 12, 11, 10, 9, 16, 15, 14, 13, 20, 19, 18, 17]
right_hand_index = [i+21 for i in left_hand_index]
for sample_id, anno in anno_all.items():
image_name = os.path.join(set, 'color', '%.5d.png' % sample_id)
mask_name = os.path.join(set, 'mask', '%.5d.png' % sample_id)
keypoint2d = anno['uv_vis'][:, :2]
keypoint3d = anno['xyz']
intrinsic_matrix = anno['K']
visible = anno['uv_vis'][:, 2]
left_hand_keypoint2d = keypoint2d[left_hand_index] # NUM_KEYPOINTS x 2
left_box = get_bounding_box(left_hand_keypoint2d)
right_hand_keypoint2d = keypoint2d[right_hand_index] # NUM_KEYPOINTS x 2
right_box = get_bounding_box(right_hand_keypoint2d)
w, h = 320, 320
scaled_left_box = scale_box(left_box, w, h, 1.5)
left, upper, right, lower = scaled_left_box
size = max(right - left, lower - upper)
if size > min_size and np.sum(visible[left_hand_index]) > 16 and area(*intersection(scaled_left_box, right_box)) / area(*scaled_left_box) < 0.3:
sample = {
'name': image_name,
'mask_name': mask_name,
'keypoint2d': left_hand_keypoint2d,
'visible': visible[left_hand_index],
'keypoint3d': keypoint3d[left_hand_index],
'intrinsic_matrix': intrinsic_matrix,
'left': True
}
samples.append(sample)
scaled_right_box = scale_box(right_box, w, h, 1.5)
left, upper, right, lower = scaled_right_box
size = max(right - left, lower - upper)
if size > min_size and np.sum(visible[right_hand_index]) > 16 and area(*intersection(scaled_right_box, left_box)) / area(*scaled_right_box) < 0.3:
sample = {
'name': image_name,
'mask_name': mask_name,
'keypoint2d': right_hand_keypoint2d,
'visible': visible[right_hand_index],
'keypoint3d': keypoint3d[right_hand_index],
'intrinsic_matrix': intrinsic_matrix,
'left': False
}
samples.append(sample)
return samples
================================================
FILE: domain_generalization/Transfer-Learning-Library/common/vision/datasets/keypoint_detection/surreal.py
================================================
"""
@author: Junguang Jiang
@contact: JiangJunguang1123@outlook.com
"""
import os
import json
from PIL import ImageFile
import torch
from ...transforms.keypoint_detection import *
from .util import *
from .._util import download as download_data, check_exits
from .keypoint_dataset import Body16KeypointDataset
ImageFile.LOAD_TRUNCATED_IMAGES = True
class SURREAL(Body16KeypointDataset):
"""`Surreal Dataset `_
Args:
root (str): Root directory of dataset
split (str, optional): The dataset split, supports ``train``, ``test``, or ``all``.
Default: ``train``.
task (str, optional): Placeholder.
download (bool, optional): If true, downloads the dataset from the internet and puts it \
in root directory. If dataset is already downloaded, it is not downloaded again.
transforms (callable, optional): A function/transform that takes in a dict (which contains PIL image and
its labels) and returns a transformed version. E.g, :class:`~common.vision.transforms.keypoint_detection.Resize`.
image_size (tuple): (width, height) of the image. Default: (256, 256)
heatmap_size (tuple): (width, height) of the heatmap. Default: (64, 64)
sigma (int): sigma parameter when generate the heatmap. Default: 2
.. note::
We found that the original Surreal image is in high resolution while most part in an image is background,
thus we crop the image and keep only the surrounding area of hands (1.5x bigger than hands) to speed up training.
.. note:: In `root`, there will exist following files after downloading.
::
train/
test/
val/
"""
def __init__(self, root, split='train', task='all', download=True, **kwargs):
assert split in ['train', 'test', 'val']
self.split = split
if download:
download_data(root, "train/run0", "train0.tgz", "https://cloud.tsinghua.edu.cn/f/b13604f06ff1445c830a/?dl=1")
download_data(root, "train/run1", "train1.tgz", "https://cloud.tsinghua.edu.cn/f/919aefe2de3541c3b940/?dl=1")
download_data(root, "train/run1", "train2.tgz", "https://cloud.tsinghua.edu.cn/f/34864760ad4945b9bcd6/?dl=1")
download_data(root, "val", "val.tgz", "https://cloud.tsinghua.edu.cn/f/16b20f2e76684f848dc1/?dl=1")
download_data(root, "test", "test.tgz", "https://cloud.tsinghua.edu.cn/f/36c72d86e43540e0a913/?dl=1")
else:
check_exits(root, "train/run0")
check_exits(root, "train/run1")
check_exits(root, "train/run2")
check_exits(root, "val")
check_exits(root, "test")
all_samples = []
for part in [0, 1, 2]:
annotation_file = os.path.join(root, split, 'run{}.json'.format(part))
print("loading", annotation_file)
with open(annotation_file) as f:
samples = json.load(f)
for sample in samples:
sample["image_path"] = os.path.join(root, self.split, 'run{}'.format(part), sample['name'])
all_samples.extend(samples)
random.seed(42)
random.shuffle(all_samples)
samples_len = len(all_samples)
samples_split = min(int(samples_len * 0.2), 3200)
if self.split == 'train':
all_samples = all_samples[samples_split:]
elif self.split == 'test':
all_samples = all_samples[:samples_split]
self.joints_index = (7, 4, 1, 2, 5, 8, 0, 9, 12, 15, 20, 18, 13, 14, 19, 21)
super(SURREAL, self).__init__(root, all_samples, **kwargs)
def __getitem__(self, index):
sample = self.samples[index]
image_name = sample['name']
image_path = sample['image_path']
image = Image.open(image_path)
keypoint3d_camera = np.array(sample['keypoint3d'])[self.joints_index, :] # NUM_KEYPOINTS x 3
keypoint2d = np.array(sample['keypoint2d'])[self.joints_index, :] # NUM_KEYPOINTS x 2
intrinsic_matrix = np.array(sample['intrinsic_matrix'])
Zc = keypoint3d_camera[:, 2]
image, data = self.transforms(image, keypoint2d=keypoint2d, intrinsic_matrix=intrinsic_matrix)
keypoint2d = data['keypoint2d']
intrinsic_matrix = data['intrinsic_matrix']
keypoint3d_camera = keypoint2d_to_3d(keypoint2d, intrinsic_matrix, Zc)
# noramlize 2D pose:
visible = np.array([1.] * 16, dtype=np.float32)
visible = visible[:, np.newaxis]
# 2D heatmap
target, target_weight = generate_target(keypoint2d, visible, self.heatmap_size, self.sigma, self.image_size)
target = torch.from_numpy(target)
target_weight = torch.from_numpy(target_weight)
# normalize 3D pose:
# put middle finger metacarpophalangeal (MCP) joint in the center of the coordinate system
# and make distance between wrist and middle finger MCP joint to be of length 1
keypoint3d_n = keypoint3d_camera - keypoint3d_camera[9:10, :]
keypoint3d_n = keypoint3d_n / np.sqrt(np.sum(keypoint3d_n[0, :] ** 2))
meta = {
'image': image_name,
'keypoint2d': keypoint2d, # （NUM_KEYPOINTS x 2）
'keypoint3d': keypoint3d_n, # （NUM_KEYPOINTS x 3）
}
return image, target, target_weight, meta
def __len__(self):
return len(self.samples)
================================================
FILE: domain_generalization/Transfer-Learning-Library/common/vision/datasets/keypoint_detection/util.py
================================================
"""
@author: Junguang Jiang
@contact: JiangJunguang1123@outlook.com
"""
import numpy as np
import cv2
def generate_target(joints, joints_vis, heatmap_size, sigma, image_size):
"""Generate heatamap for joints.
Args:
joints: (K, 2)
joints_vis: (K, 1)
heatmap_size: W, H
sigma:
image_size:
Returns:
"""
num_joints = joints.shape[0]
target_weight = np.ones((num_joints, 1), dtype=np.float32)
target_weight[:, 0] = joints_vis[:, 0]
target = np.zeros((num_joints,
heatmap_size[1],
heatmap_size[0]),
dtype=np.float32)
tmp_size = sigma * 3
image_size = np.array(image_size)
heatmap_size = np.array(heatmap_size)
for joint_id in range(num_joints):
feat_stride = image_size / heatmap_size
mu_x = int(joints[joint_id][0] / feat_stride[0] + 0.5)
mu_y = int(joints[joint_id][1] / feat_stride[1] + 0.5)
# Check that any part of the gaussian is in-bounds
ul = [int(mu_x - tmp_size), int(mu_y - tmp_size)]
br = [int(mu_x + tmp_size + 1), int(mu_y + tmp_size + 1)]
if mu_x >= heatmap_size[0] or mu_y >= heatmap_size[1] \
or mu_x < 0 or mu_y < 0:
# If not, just return the image as is
target_weight[joint_id] = 0
continue
# Generate gaussian
size = 2 * tmp_size + 1
x = np.arange(0, size, 1, np.float32)
y = x[:, np.newaxis]
x0 = y0 = size // 2
# The gaussian is not normalized, we want the center value to equal 1
g = np.exp(- ((x - x0) ** 2 + (y - y0) ** 2) / (2 * sigma ** 2))
# Usable gaussian range
g_x = max(0, -ul[0]), min(br[0], heatmap_size[0]) - ul[0]
g_y = max(0, -ul[1]), min(br[1], heatmap_size[1]) - ul[1]
# Image range
img_x = max(0, ul[0]), min(br[0], heatmap_size[0])
img_y = max(0, ul[1]), min(br[1], heatmap_size[1])
v = target_weight[joint_id]
if v > 0.5:
target[joint_id][img_y[0]:img_y[1], img_x[0]:img_x[1]] = \
g[g_y[0]:g_y[1], g_x[0]:g_x[1]]
return target, target_weight
def keypoint2d_to_3d(keypoint2d: np.ndarray, intrinsic_matrix: np.ndarray, Zc: np.ndarray):
"""Convert 2D keypoints to 3D keypoints"""
uv1 = np.concatenate([np.copy(keypoint2d), np.ones((keypoint2d.shape[0], 1))], axis=1).T * Zc # 3 x NUM_KEYPOINTS
xyz = np.matmul(np.linalg.inv(intrinsic_matrix), uv1).T # NUM_KEYPOINTS x 3
return xyz
def keypoint3d_to_2d(keypoint3d: np.ndarray, intrinsic_matrix: np.ndarray):
"""Convert 3D keypoints to 2D keypoints"""
keypoint2d = np.matmul(intrinsic_matrix, keypoint3d.T).T # NUM_KEYPOINTS x 3
keypoint2d = keypoint2d[:, :2] / keypoint2d[:, 2:3] # NUM_KEYPOINTS x 2
return keypoint2d
def scale_box(box, image_width, image_height, scale):
"""
Change `box` to a square box.
The side with of the square box will be `scale` * max(w, h)
where w and h is the width and height of the origin box
"""
left, upper, right, lower = box
center_x, center_y = (left + right) / 2, (upper + lower) / 2
w, h = right - left, lower - upper
side_with = min(round(scale * max(w, h)), min(image_width, image_height))
left = round(center_x - side_with / 2)
right = left + side_with - 1
upper = round(center_y - side_with / 2)
lower = upper + side_with - 1
if left < 0:
left = 0
right = side_with - 1
if right >= image_width:
right = image_width - 1
left = image_width - side_with
if upper < 0:
upper = 0
lower = side_with -1
if lower >= image_height:
lower = image_height - 1
upper = image_height - side_with
return left, upper, right, lower
def get_bounding_box(keypoint2d: np.array):
"""Get the bounding box for keypoints"""
left = np.min(keypoint2d[:, 0])
right = np.max(keypoint2d[:, 0])
upper = np.min(keypoint2d[:, 1])
lower = np.max(keypoint2d[:, 1])
return left, upper, right, lower
def visualize_heatmap(image, heatmaps, filename):
image = cv2.cvtColor(np.asarray(image), cv2.COLOR_RGB2BGR).copy()
H, W = heatmaps.shape[1], heatmaps.shape[2]
resized_image = cv2.resize(image, (int(W), int(H)))
heatmaps = heatmaps.mul(255).clamp(0, 255).byte().cpu().numpy()
for k in range(heatmaps.shape[0]):
heatmap = heatmaps[k]
colored_heatmap = cv2.applyColorMap(heatmap, cv2.COLORMAP_JET)
masked_image = colored_heatmap * 0.7 + resized_image * 0.3
cv2.imwrite(filename.format(k), masked_image)
def area(left, upper, right, lower):
return max(right - left + 1, 0) * max(lower - upper + 1, 0)
def intersection(box_a, box_b):
left_a, upper_a, right_a, lower_a = box_a
left_b, upper_b, right_b, lower_b = box_b
return max(left_a, left_b), max(upper_a, upper_b), min(right_a, right_b), min(lower_a, lower_b)
================================================
FILE: domain_generalization/Transfer-Learning-Library/common/vision/datasets/office31.py
================================================
"""
@author: Junguang Jiang
@contact: JiangJunguang1123@outlook.com
"""
from typing import Optional
import os
from .imagelist import ImageList
from ._util import download as download_data, check_exits
class Office31(ImageList):
"""Office31 Dataset.
Args:
root (str): Root directory of dataset
task (str): The task (domain) to create dataset. Choices include ``'A'``: amazon, \
``'D'``: dslr and ``'W'``: webcam.
download (bool, optional): If true, downloads the dataset from the internet and puts it \
in root directory. If dataset is already downloaded, it is not downloaded again.
transform (callable, optional): A function/transform that takes in an PIL image and returns a \
transformed version. E.g, :class:`torchvision.transforms.RandomCrop`.
target_transform (callable, optional): A function/transform that takes in the target and transforms it.
.. note:: In `root`, there will exist following files after downloading.
::
amazon/
images/
backpack/
*.jpg
...
dslr/
webcam/
image_list/
amazon.txt
dslr.txt
webcam.txt
"""
download_list = [
("image_list", "image_list.zip", "https://cloud.tsinghua.edu.cn/f/d9bca681c71249f19da2/?dl=1"),
("amazon", "amazon.tgz", "https://cloud.tsinghua.edu.cn/f/edc8d1bba1c740dc821c/?dl=1"),
("dslr", "dslr.tgz", "https://cloud.tsinghua.edu.cn/f/ca6df562b7e64850ad7f/?dl=1"),
("webcam", "webcam.tgz", "https://cloud.tsinghua.edu.cn/f/82b24ed2e08f4a3c8888/?dl=1"),
]
image_list = {
"A": "image_list/amazon.txt",
"D": "image_list/dslr.txt",
"W": "image_list/webcam.txt"
}
CLASSES = ['back_pack', 'bike', 'bike_helmet', 'bookcase', 'bottle', 'calculator', 'desk_chair', 'desk_lamp',
'desktop_computer', 'file_cabinet', 'headphones', 'keyboard', 'laptop_computer', 'letter_tray',
'mobile_phone', 'monitor', 'mouse', 'mug', 'paper_notebook', 'pen', 'phone', 'printer', 'projector',
'punchers', 'ring_binder', 'ruler', 'scissors', 'speaker', 'stapler', 'tape_dispenser', 'trash_can']
def __init__(self, root: str, task: str, download: Optional[bool] = True, **kwargs):
assert task in self.image_list
data_list_file = os.path.join(root, self.image_list[task])
if download:
list(map(lambda args: download_data(root, *args), self.download_list))
else:
list(map(lambda file_name, _: check_exits(root, file_name), self.download_list))
super(Office31, self).__init__(root, Office31.CLASSES, data_list_file=data_list_file, **kwargs)
@classmethod
def domains(cls):
return list(cls.image_list.keys())
================================================
FILE: domain_generalization/Transfer-Learning-Library/common/vision/datasets/officecaltech.py
================================================
"""
@author: Junguang Jiang
@contact: JiangJunguang1123@outlook.com
"""
import os
from typing import Optional
from torchvision.datasets.folder import DatasetFolder, IMG_EXTENSIONS, default_loader
from torchvision.datasets.utils import download_and_extract_archive
from ._util import check_exits
class OfficeCaltech(DatasetFolder):
"""Office+Caltech Dataset.
Args:
root (str): Root directory of dataset
task (str): The task (domain) to create dataset. Choices include ``'A'``: amazon, \
``'D'``: dslr, ``'W'``:webcam and ``'C'``: caltech.
download (bool, optional): If true, downloads the dataset from the internet and puts it \
in root directory. If dataset is already downloaded, it is not downloaded again.
transform (callable, optional): A function/transform that takes in an PIL image and returns a \
transformed version. E.g, :class:`torchvision.transforms.RandomCrop`.
target_transform (callable, optional): A function/transform that takes in the target and transforms it.
.. note:: In `root`, there will exist following files after downloading.
::
amazon/
images/
backpack/
*.jpg
...
dslr/
webcam/
caltech/
image_list/
amazon.txt
dslr.txt
webcam.txt
caltech.txt
"""
directories = {
"A": "amazon",
"D": "dslr",
"W": "webcam",
"C": "caltech"
}
CLASSES = ['back_pack', 'bike', 'calculator', 'headphones', 'keyboard',
'laptop_computer', 'monitor', 'mouse', 'mug', 'projector']
def __init__(self, root: str, task: str, download: Optional[bool] = False, **kwargs):
if download:
for dir in self.directories.values():
if not os.path.exists(os.path.join(root, dir)):
download_and_extract_archive(url="https://cloud.tsinghua.edu.cn/f/e93f2e07d93243d6b57e/?dl=1",
download_root=os.path.join(root, 'download'),
filename="officecaltech.tgz", remove_finished=False, extract_root=root)
break
else:
list(map(lambda dir, _: check_exits(root, dir), self.directories.values()))
super(OfficeCaltech, self).__init__(
os.path.join(root, self.directories[task]), default_loader, extensions=IMG_EXTENSIONS, **kwargs)
self.classes = OfficeCaltech.CLASSES
self.class_to_idx = {cls: idx
for idx, clss in enumerate(self.classes)
for cls in clss}
@property
def num_classes(self):
"""Number of classes"""
return len(self.classes)
@classmethod
def domains(cls):
return list(cls.directories.keys())
================================================
FILE: domain_generalization/Transfer-Learning-Library/common/vision/datasets/officehome.py
================================================
"""
@author: Junguang Jiang
@contact: JiangJunguang1123@outlook.com
"""
import os
from typing import Optional
from .imagelist import ImageList
from ._util import download as download_data, check_exits
class OfficeHome(ImageList):
"""`OfficeHome `_ Dataset.
Args:
root (str): Root directory of dataset
task (str): The task (domain) to create dataset. Choices include ``'Ar'``: Art, \
``'Cl'``: Clipart, ``'Pr'``: Product and ``'Rw'``: Real_World.
download (bool, optional): If true, downloads the dataset from the internet and puts it \
in root directory. If dataset is already downloaded, it is not downloaded again.
transform (callable, optional): A function/transform that takes in an PIL image and returns a \
transformed version. E.g, :class:`torchvision.transforms.RandomCrop`.
target_transform (callable, optional): A function/transform that takes in the target and transforms it.
.. note:: In `root`, there will exist following files after downloading.
::
Art/
Alarm_Clock/*.jpg
...
Clipart/
Product/
Real_World/
image_list/
Art.txt
Clipart.txt
Product.txt
Real_World.txt
"""
download_list = [
("image_list", "image_list.zip", "https://cloud.tsinghua.edu.cn/f/ca3a3b6a8d554905b4cd/?dl=1"),
("Art", "Art.tgz", "https://cloud.tsinghua.edu.cn/f/4691878067d04755beab/?dl=1"),
("Clipart", "Clipart.tgz", "https://cloud.tsinghua.edu.cn/f/0d41e7da4558408ea5aa/?dl=1"),
("Product", "Product.tgz", "https://cloud.tsinghua.edu.cn/f/76186deacd7c4fa0a679/?dl=1"),
("Real_World", "Real_World.tgz", "https://cloud.tsinghua.edu.cn/f/dee961894cc64b1da1d7/?dl=1")
]
image_list = {
"Ar": "image_list/Art.txt",
"Cl": "image_list/Clipart.txt",
"Pr": "image_list/Product.txt",
"Rw": "image_list/Real_World.txt",
}
CLASSES = ['Drill', 'Exit_Sign', 'Bottle', 'Glasses', 'Computer', 'File_Cabinet', 'Shelf', 'Toys', 'Sink',
'Laptop', 'Kettle', 'Folder', 'Keyboard', 'Flipflops', 'Pencil', 'Bed', 'Hammer', 'ToothBrush', 'Couch',
'Bike', 'Postit_Notes', 'Mug', 'Webcam', 'Desk_Lamp', 'Telephone', 'Helmet', 'Mouse', 'Pen', 'Monitor',
'Mop', 'Sneakers', 'Notebook', 'Backpack', 'Alarm_Clock', 'Push_Pin', 'Paper_Clip', 'Batteries', 'Radio',
'Fan', 'Ruler', 'Pan', 'Screwdriver', 'Trash_Can', 'Printer', 'Speaker', 'Eraser', 'Bucket', 'Chair',
'Calendar', 'Calculator', 'Flowers', 'Lamp_Shade', 'Spoon', 'Candles', 'Clipboards', 'Scissors', 'TV',
'Curtains', 'Fork', 'Soda', 'Table', 'Knives', 'Oven', 'Refrigerator', 'Marker']
def __init__(self, root: str, task: str, download: Optional[bool] = False, **kwargs):
assert task in self.image_list
data_list_file = os.path.join(root, self.image_list[task])
if download:
list(map(lambda args: download_data(root, *args), self.download_list))
else:
list(map(lambda file_name, _: check_exits(root, file_name), self.download_list))
super(OfficeHome, self).__init__(root, OfficeHome.CLASSES, data_list_file=data_list_file, **kwargs)
@classmethod
def domains(cls):
return list(cls.image_list.keys())
================================================
FILE: domain_generalization/Transfer-Learning-Library/common/vision/datasets/openset/__init__.py
================================================
"""
@author: Junguang Jiang
@contact: JiangJunguang1123@outlook.com
"""
from ..imagelist import ImageList
from ..office31 import Office31
from ..officehome import OfficeHome
from ..visda2017 import VisDA2017
from typing import Optional, ClassVar, Sequence
from copy import deepcopy
__all__ = ['Office31', 'OfficeHome', "VisDA2017"]
def open_set(dataset_class: ClassVar, public_classes: Sequence[str],
private_classes: Optional[Sequence[str]] = ()) -> ClassVar:
"""
Convert a dataset into its open-set version.
In other words, those samples which doesn't belong to `private_classes` will be marked as "unknown".
Be aware that `open_set` will change the label number of each category.
Args:
dataset_class (class): Dataset class. Only subclass of ``ImageList`` can be open-set.
public_classes (sequence[str]): A sequence of which categories need to be kept in the open-set dataset.\
Each element of `public_classes` must belong to the `classes` list of `dataset_class`.
private_classes (sequence[str], optional): A sequence of which categories need to be marked as "unknown" \
in the open-set dataset. Each element of `private_classes` must belong to the `classes` list of \
`dataset_class`. Default: ().
Examples::
>>> public_classes = ['back_pack', 'bike', 'calculator', 'headphones', 'keyboard']
>>> private_classes = ['laptop_computer', 'monitor', 'mouse', 'mug', 'projector']
>>> # create a open-set dataset class which has classes
>>> # 'back_pack', 'bike', 'calculator', 'headphones', 'keyboard' and 'unknown'.
>>> OpenSetOffice31 = open_set(Office31, public_classes, private_classes)
>>> # create an instance of the open-set dataset
>>> dataset = OpenSetDataset(root="data/office31", task="A")
"""
if not (issubclass(dataset_class, ImageList)):
raise Exception("Only subclass of ImageList can be openset")
class OpenSetDataset(dataset_class):
def __init__(self, **kwargs):
super(OpenSetDataset, self).__init__(**kwargs)
samples = []
all_classes = list(deepcopy(public_classes)) + ["unknown"]
for (path, label) in self.samples:
class_name = self.classes[label]
if class_name in public_classes:
samples.append((path, all_classes.index(class_name)))
elif class_name in private_classes:
samples.append((path, all_classes.index("unknown")))
self.samples = samples
self.classes = all_classes
self.class_to_idx = {cls: idx
for idx, cls in enumerate(self.classes)}
return OpenSetDataset
def default_open_set(dataset_class: ClassVar, source: bool) -> ClassVar:
"""
Default open-set used in some paper.
Args:
dataset_class (class): Dataset class. Currently, dataset_class must be one of
:class:`~common.vision.datasets.office31.Office31`, :class:`~common.vision.datasets.officehome.OfficeHome`,
:class:`~common.vision.datasets.visda2017.VisDA2017`,
source (bool): Whether the dataset is used for source domain or not.
"""
if dataset_class == Office31:
public_classes = Office31.CLASSES[:20]
if source:
private_classes = ()
else:
private_classes = Office31.CLASSES[20:]
elif dataset_class == OfficeHome:
public_classes = sorted(OfficeHome.CLASSES)[:25]
if source:
private_classes = ()
else:
private_classes = sorted(OfficeHome.CLASSES)[25:]
elif dataset_class == VisDA2017:
public_classes = ('bicycle', 'bus', 'car', 'motorcycle', 'train', 'truck')
if source:
private_classes = ()
else:
private_classes = ('aeroplane', 'horse', 'knife', 'person', 'plant', 'skateboard')
else:
raise NotImplementedError("Unknown openset domain adaptation dataset: {}".format(dataset_class.__name__))
return open_set(dataset_class, public_classes, private_classes)
================================================
FILE: domain_generalization/Transfer-Learning-Library/common/vision/datasets/oxfordflowers.py
================================================
"""
@author: Junguang Jiang
@contact: JiangJunguang1123@outlook.com
"""
import os
from common.vision.datasets import ImageList
from ._util import download as download_data, check_exits
class OxfordFlowers102(ImageList):
"""
`The Oxford Flowers 102 `_ is a \
consistent of 102 flower categories commonly occurring in the United Kingdom. \
Each class consists of between 40 and 258 images. The images have large scale, \
pose and light variations. In addition, there are categories that have large \
variations within the category and several very similar categories. \
The dataset is divided into a training set, a validation set and a test set. \
The training set and validation set each consist of 10 images per class \
(totalling 1020 images each). \
The test set consists of the remaining 6149 images (minimum 20 per class).
Args:
root (str): Root directory of dataset
split (str, optional): The dataset split, supports ``train``, or ``test``.
download (bool, optional): If true, downloads the dataset from the internet and puts it \
in root directory. If dataset is already downloaded, it is not downloaded again.
transform (callable, optional): A function/transform that takes in an PIL image and returns a \
transformed version. E.g, :class:`torchvision.transforms.RandomCrop`.
target_transform (callable, optional): A function/transform that takes in the target and transforms it.
"""
CLASSES = ['pink primrose', 'hard-leaved pocket orchid', 'canterbury bells', 'sweet pea', 'english marigold',
'tiger lily', 'moon orchid', 'bird of paradise', 'monkshood', 'globe thistle', 'snapdragon',
"colt's foot", 'king protea', 'spear thistle', 'yellow iris', 'globe-flower', 'purple coneflower',
'peruvian lily', 'balloon flower', 'giant white arum lily', 'fire lily', 'pincushion flower',
'fritillary', 'red ginger', 'grape hyacinth', 'corn poppy', 'prince of wales feathers',
'stemless gentian', 'artichoke', 'sweet william', 'carnation', 'garden phlox', 'love in the mist',
'mexican aster', 'alpine sea holly', 'ruby-lipped cattleya', 'cape flower', 'great masterwort',
'siam tulip', 'lenten rose', 'barbeton daisy', 'daffodil', 'sword lily', 'poinsettia',
'bolero deep blue', 'wallflower', 'marigold', 'buttercup', 'oxeye daisy', 'common dandelion',
'petunia', 'wild pansy', 'primula', 'sunflower', 'pelargonium', 'bishop of llandaff', 'gaura',
'geranium', 'orange dahlia', 'pink-yellow dahlia?', 'cautleya spicata', 'japanese anemone',
'black-eyed susan', 'silverbush', 'californian poppy', 'osteospermum', 'spring crocus',
'bearded iris', 'windflower', 'tree poppy', 'gazania', 'azalea', 'water lily', 'rose', 'thorn apple',
'morning glory', 'passion flower', 'lotus', 'toad lily', 'anthurium', 'frangipani', 'clematis',
'hibiscus', 'columbine', 'desert-rose', 'tree mallow', 'magnolia', 'cyclamen', 'watercress',
'canna lily', 'hippeastrum', 'bee balm', 'ball moss', 'foxglove', 'bougainvillea', 'camellia',
'mallow', 'mexican petunia', 'bromelia', 'blanket flower', 'trumpet creeper', 'blackberry lily']
def __init__(self, root, split='train', download=False, **kwargs):
if download:
download_data(root, "oxford_flowers102", "oxford_flowers102.tgz", "https://cloud.tsinghua.edu.cn/f/9910751054764d568281/?dl=1")
else:
check_exits(root, "oxford_flowers102")
root = os.path.join(root, "oxford_flowers102")
super(OxfordFlowers102, self).__init__(root, OxfordFlowers102.CLASSES, os.path.join(root, 'imagelist', '{}.txt'.format(split)), **kwargs)
================================================
FILE: domain_generalization/Transfer-Learning-Library/common/vision/datasets/oxfordpet.py
================================================
"""
@author: Yifei Ji
@contact: jiyf990330@163.com
"""
import os
from typing import Optional
from .imagelist import ImageList
from ._util import download as download_data, check_exits
class OxfordIIITPet(ImageList):
"""`The Oxford-IIIT Pet `_ \
is a 37-category pet dataset with roughly 200 images for each class.
Args:
root (str): Root directory of dataset
split (str, optional): The dataset split, supports ``train``, or ``test``.
sample_rate (int): The sampling rates to sample random ``training`` images for each category.
Choices include 100, 50, 30, 15. Default: 100.
download (bool, optional): If true, downloads the dataset from the internet and puts it \
in root directory. If dataset is already downloaded, it is not downloaded again.
transform (callable, optional): A function/transform that takes in an PIL image and returns a \
transformed version. E.g, :class:`torchvision.transforms.RandomCrop`.
target_transform (callable, optional): A function/transform that takes in the target and transforms it.
.. note:: In `root`, there will exist following files after downloading.
::
train/
test/
image_list/
train_100.txt
train_50.txt
train_30.txt
train_15.txt
test.txt
"""
download_list = [
("image_list", "image_list.zip", "https://cloud.tsinghua.edu.cn/f/738d75de56844bd0951b/?dl=1"),
("train", "train.tgz", "https://cloud.tsinghua.edu.cn/f/f40cf9f8c9ac4b04ba9b/?dl=1"),
("test", "test.tgz", "https://cloud.tsinghua.edu.cn/f/72b4bff8b5c84f4ba240/?dl=1"),
]
image_list = {
"train": "image_list/train_100.txt",
"train100": "image_list/train_100.txt",
"train50": "image_list/train_50.txt",
"train30": "image_list/train_30.txt",
"train15": "image_list/train_15.txt",
"test": "image_list/test.txt",
"test100": "image_list/test.txt",
}
CLASSES = ['Abyssinian', 'american_bulldog', 'american_pit_bull_terrier', 'basset_hound', 'beagle', 'Bengal',
'Birman', 'Bombay', 'boxer', 'British_Shorthair', 'chihuahua', 'Egyptian_Mau', 'english_cocker_spaniel',
'english_setter', 'german_shorthaired', 'great_pyrenees', 'havanese', 'japanese_chin', 'keeshond', 'leonberger',
'Maine_Coon', 'miniature_pinscher', 'newfoundland', 'Persian', 'pomeranian', 'pug', 'Ragdoll',
'Russian_Blue', 'saint_bernard', 'samoyed', 'scottish_terrier', 'shiba_inu', 'Siamese', 'Sphynx',
'staffordshire_bull_terrier', 'wheaten_terrier', 'yorkshire_terrier']
def __init__(self, root: str, split: str, sample_rate: Optional[int] =100, download: Optional[bool] = False, **kwargs):
if split == 'train':
list_name = 'train' + str(sample_rate)
assert list_name in self.image_list
data_list_file = os.path.join(root, self.image_list[list_name])
else:
data_list_file = os.path.join(root, self.image_list['test'])
if download:
list(map(lambda args: download_data(root, *args), self.download_list))
else:
list(map(lambda file_name, _: check_exits(root, file_name), self.download_list))
super(OxfordIIITPet, self).__init__(root, OxfordIIITPet.CLASSES, data_list_file=data_list_file, **kwargs)
================================================
FILE: domain_generalization/Transfer-Learning-Library/common/vision/datasets/pacs.py
================================================
from typing import Optional
import os
from .imagelist import ImageList
from ._util import download as download_data, check_exits
class PACS(ImageList):
"""`PACS Dataset `_.
Args:
root (str): Root directory of dataset
task (str): The task (domain) to create dataset. Choices include ``'A'``: amazon, \
``'D'``: dslr and ``'W'``: webcam.
download (bool, optional): If true, downloads the dataset from the internet and puts it \
in root directory. If dataset is already downloaded, it is not downloaded again.
transform (callable, optional): A function/transform that takes in an PIL image and returns a \
transformed version. E.g, :class:`torchvision.transforms.RandomCrop`.
target_transform (callable, optional): A function/transform that takes in the target and transforms it.
.. note:: In `root`, there will exist following files after downloading.
::
art_painting/
dog/
*.jpg
...
cartoon/
photo/
sketch
image_list/
art_painting.txt
cartoon.txt
photo.txt
sketch.txt
"""
download_list = [
("image_list", "image_list.zip", "https://cloud.tsinghua.edu.cn/f/add42cc3859847bc988c/?dl=1"),
("art_painting", "art_painting.tgz", "https://cloud.tsinghua.edu.cn/f/4eb7db4f3eec41719856/?dl=1"),
("cartoon", "cartoon.tgz", "https://cloud.tsinghua.edu.cn/f/d847ac22497b4826889f/?dl=1"),
("photo", "photo.tgz", "https://cloud.tsinghua.edu.cn/f/458ad21483da4a45935b/?dl=1"),
("sketch", "sketch.tgz", "https://cloud.tsinghua.edu.cn/f/c892ac2d94a44b1196b8/?dl=1"),
]
image_list = {
"A": "image_list/art_painting_{}.txt",
"C": "image_list/cartoon_{}.txt",
"P": "image_list/photo_{}.txt",
"S": "image_list/sketch_{}.txt"
}
CLASSES = ['dog', 'elephant', 'giraffe', 'guitar', 'horse', 'house', 'person']
def __init__(self, root: str, task: str, split='all', download: Optional[bool] = True, **kwargs):
assert task in self.image_list
assert split in ["train", "val", "all", "test"]
if split == "test":
split = "all"
data_list_file = os.path.join(root, self.image_list[task].format(split))
if download:
list(map(lambda args: download_data(root, *args), self.download_list))
else:
list(map(lambda file_name, _: check_exits(root, file_name), self.download_list))
super(PACS, self).__init__(root, PACS.CLASSES, data_list_file=data_list_file, target_transform=lambda x: x - 1,
**kwargs)
@classmethod
def domains(cls):
return list(cls.image_list.keys())
================================================
FILE: domain_generalization/Transfer-Learning-Library/common/vision/datasets/partial/__init__.py
================================================
"""
@author: Junguang Jiang
@contact: JiangJunguang1123@outlook.com
"""
from ..imagelist import ImageList
from ..office31 import Office31
from ..officehome import OfficeHome
from ..visda2017 import VisDA2017
from ..officecaltech import OfficeCaltech
from .imagenet_caltech import ImageNetCaltech
from .caltech_imagenet import CaltechImageNet
from common.vision.datasets.partial.imagenet_caltech import ImageNetCaltech
from typing import Sequence, ClassVar
__all__ = ['Office31', 'OfficeHome', "VisDA2017", "CaltechImageNet", "ImageNetCaltech"]
def partial(dataset_class: ClassVar, partial_classes: Sequence[str]) -> ClassVar:
"""
Convert a dataset into its partial version.
In other words, those samples which doesn't belong to `partial_classes` will be discarded.
Yet `partial` will not change the label space of `dataset_class`.
Args:
dataset_class (class): Dataset class. Only subclass of ``ImageList`` can be partial.
partial_classes (sequence[str]): A sequence of which categories need to be kept in the partial dataset.\
Each element of `partial_classes` must belong to the `classes` list of `dataset_class`.
Examples::
>>> partial_classes = ['back_pack', 'bike', 'calculator', 'headphones', 'keyboard']
>>> # create a partial dataset class
>>> PartialOffice31 = partial(Office31, partial_classes)
>>> # create an instance of the partial dataset
>>> dataset = PartialDataset(root="data/office31", task="A")
"""
if not (issubclass(dataset_class, ImageList)):
raise Exception("Only subclass of ImageList can be partial")
class PartialDataset(dataset_class):
def __init__(self, **kwargs):
super(PartialDataset, self).__init__(**kwargs)
assert all([c in self.classes for c in partial_classes])
samples = []
for (path, label) in self.samples:
class_name = self.classes[label]
if class_name in partial_classes:
samples.append((path, label))
self.samples = samples
self.partial_classes = partial_classes
self.partial_classes_idx = [self.class_to_idx[c] for c in partial_classes]
return PartialDataset
def default_partial(dataset_class: ClassVar) -> ClassVar:
"""
Default partial used in some paper.
Args:
dataset_class (class): Dataset class. Currently, dataset_class must be one of
:class:`~common.vision.datasets.office31.Office31`, :class:`~common.vision.datasets.officehome.OfficeHome`,
:class:`~common.vision.datasets.visda2017.VisDA2017`,
:class:`~common.vision.datasets.partial.imagenet_caltech.ImageNetCaltech`
and :class:`~common.vision.datasets.partial.caltech_imagenet.CaltechImageNet`.
"""
if dataset_class == Office31:
kept_classes = OfficeCaltech.CLASSES
elif dataset_class == OfficeHome:
kept_classes = sorted(OfficeHome.CLASSES)[:25]
elif dataset_class == VisDA2017:
kept_classes = sorted(VisDA2017.CLASSES)[:6]
elif dataset_class in [ImageNetCaltech, CaltechImageNet]:
kept_classes = dataset_class.CLASSES
else:
raise NotImplementedError("Unknown partial domain adaptation dataset: {}".format(dataset_class.__name__))
return partial(dataset_class, kept_classes)
================================================
FILE: domain_generalization/Transfer-Learning-Library/common/vision/datasets/partial/caltech_imagenet.py
================================================
"""
@author: Junguang Jiang
@contact: JiangJunguang1123@outlook.com
"""
from typing import Optional
import os
from ..imagelist import ImageList
from .._util import download as download_data, check_exits
_CLASSES = ['ak47', 'american flag', 'backpack', 'baseball bat', 'baseball glove', 'basketball hoop', 'bat',
'bathtub', 'bear', 'beer mug', 'billiards', 'binoculars', 'birdbath', 'blimp', 'bonsai 101',
'boom box', 'bowling ball', 'bowling pin', 'boxing glove', 'brain 101', 'breadmaker', 'buddha 101',
'bulldozer', 'butterfly', 'cactus', 'cake', 'calculator', 'camel', 'cannon', 'canoe', 'car tire',
'cartman', 'cd', 'centipede', 'cereal box', 'chandelier 101', 'chess board', 'chimp', 'chopsticks',
'cockroach', 'coffee mug', 'coffin', 'coin', 'comet', 'computer keyboard', 'computer monitor',
'computer mouse', 'conch', 'cormorant', 'covered wagon', 'cowboy hat', 'crab 101', 'desk globe',
'diamond ring', 'dice', 'dog', 'dolphin 101', 'doorknob', 'drinking straw', 'duck', 'dumb bell',
'eiffel tower', 'electric guitar 101', 'elephant 101', 'elk', 'ewer 101', 'eyeglasses', 'fern',
'fighter jet', 'fire extinguisher', 'fire hydrant', 'fire truck', 'fireworks', 'flashlight',
'floppy disk', 'football helmet', 'french horn', 'fried egg', 'frisbee', 'frog', 'frying pan',
'galaxy', 'gas pump', 'giraffe', 'goat', 'golden gate bridge', 'goldfish', 'golf ball', 'goose',
'gorilla', 'grand piano 101', 'grapes', 'grasshopper', 'guitar pick', 'hamburger', 'hammock',
'harmonica', 'harp', 'harpsichord', 'hawksbill 101', 'head phones', 'helicopter 101', 'hibiscus',
'homer simpson', 'horse', 'horseshoe crab', 'hot air balloon', 'hot dog', 'hot tub', 'hourglass',
'house fly', 'human skeleton', 'hummingbird', 'ibis 101', 'ice cream cone', 'iguana', 'ipod',
'iris', 'jesus christ', 'joy stick', 'kangaroo 101', 'kayak', 'ketch 101', 'killer whale', 'knife',
'ladder', 'laptop 101', 'lathe', 'leopards 101', 'license plate', 'lightbulb', 'light house',
'lightning', 'llama 101', 'mailbox', 'mandolin', 'mars', 'mattress', 'megaphone', 'menorah 101',
'microscope', 'microwave', 'minaret', 'minotaur', 'motorbikes 101', 'mountain bike', 'mushroom',
'mussels', 'necktie', 'octopus', 'ostrich', 'owl', 'palm pilot', 'palm tree', 'paperclip',
'paper shredder', 'pci card', 'penguin', 'people', 'pez dispenser', 'photocopier', 'picnic table',
'playing card', 'porcupine', 'pram', 'praying mantis', 'pyramid', 'raccoon', 'radio telescope',
'rainbow', 'refrigerator', 'revolver 101', 'rifle', 'rotary phone', 'roulette wheel', 'saddle',
'saturn', 'school bus', 'scorpion 101', 'screwdriver', 'segway', 'self propelled lawn mower',
'sextant', 'sheet music', 'skateboard', 'skunk', 'skyscraper', 'smokestack', 'snail', 'snake',
'sneaker', 'snowmobile', 'soccer ball', 'socks', 'soda can', 'spaghetti', 'speed boat', 'spider',
'spoon', 'stained glass', 'starfish 101', 'steering wheel', 'stirrups', 'sunflower 101', 'superman',
'sushi', 'swan', 'swiss army knife', 'sword', 'syringe', 'tambourine', 'teapot', 'teddy bear',
'teepee', 'telephone box', 'tennis ball', 'tennis court', 'tennis racket', 'theodolite', 'toaster',
'tomato', 'tombstone', 'top hat', 'touring bike', 'tower pisa', 'traffic light', 'treadmill',
'triceratops', 'tricycle', 'trilobite 101', 'tripod', 't shirt', 'tuning fork', 'tweezer',
'umbrella 101', 'unicorn', 'vcr', 'video projector', 'washing machine', 'watch 101', 'waterfall',
'watermelon', 'welding mask', 'wheelbarrow', 'windmill', 'wine bottle', 'xylophone', 'yarmulke',
'yo yo', 'zebra', 'airplanes 101', 'car side 101', 'faces easy 101', 'greyhound', 'tennis shoes',
'toad']
class CaltechImageNet(ImageList):
"""Caltech-ImageNet is constructed from `Caltech-256 `_ and
`ImageNet-1K `_ .
They share 84 common classes. Caltech-ImageNet keeps all classes of Caltech-256.
The label is based on the Caltech256 (class 0-255) . The private classes of ImageNet-1K is discarded.
Args:
root (str): Root directory of dataset
task (str): The task (domain) to create dataset. Choices include ``'C'``:Caltech-256, \
``'I'``: ImageNet-1K validation set.
download (bool, optional): If true, downloads the dataset from the internet and puts it \
in root directory. If dataset is already downloaded, it is not downloaded again.
transform (callable, optional): A function/transform that takes in an PIL image and returns a \
transformed version. E.g, :class:`torchvision.transforms.RandomCrop`.
target_transform (callable, optional): A function/transform that takes in the target and transforms it.
.. note:: You need to put ``train`` and ``val`` directory of ImageNet-1K manually in `root` directory
since ImageNet-1K is no longer publicly accessible. DALIB will only download Caltech-256 and ImageList automatically.
In `root`, there will exist following files after downloading.
::
train/
n01440764/
...
val/
256_ObjectCategories/
001.ak47/
...
image_list/
caltech_256_list.txt
...
"""
image_list = {
"C": "image_list/caltech_256_list.txt",
"I": "image_list/imagenet_val_84_list.txt",
}
CLASSES = _CLASSES
def __init__(self, root: str, task: str, download: Optional[bool] = True, **kwargs):
assert task in self.image_list
data_list_file = os.path.join(root, self.image_list[task])
if download:
list(map(lambda args: download_data(root, *args), download_list))
else:
list(map(lambda file_name, _: check_exits(root, file_name), download_list))
if not os.path.exists(os.path.join(root, 'val')):
print("Please put train and val directory of ImageNet-1K manually under {} "
"since ImageNet-1K is no longer publicly accessible.".format(root))
exit(-1)
super(CaltechImageNet, self).__init__(root, CaltechImageNet.CLASSES, data_list_file=data_list_file, **kwargs)
class CaltechImageNetUniversal(ImageList):
"""Caltech-ImageNet-Universal is constructed from `Caltech-256 `_
and `ImageNet-1K `_ .
They share 84 common classes. Caltech-ImageNet keeps all classes of Caltech-256.
The label is based on the Caltech256 (class 0-255) . The private classes of ImageNet-1K is grouped into class 256 ("unknown").
Thus, CaltechImageNetUniversal has 257 classes in total.
Args:
root (str): Root directory of dataset
task (str): The task (domain) to create dataset. Choices include ``'C'``:Caltech-256, \
``'I'``: ImageNet-1K validation set.
download (bool, optional): If true, downloads the dataset from the internet and puts it \
in root directory. If dataset is already downloaded, it is not downloaded again.
transform (callable, optional): A function/transform that takes in an PIL image and returns a \
transformed version. E.g, :class:`torchvision.transforms.RandomCrop`.
target_transform (callable, optional): A function/transform that takes in the target and transforms it.
.. note:: You need to put ``train`` and ``val`` directory of ImageNet-1K manually in `root` directory
since ImageNet-1K is no longer publicly accessible. DALIB will only download Caltech-256 and ImageList automatically.
In `root`, there will exist following files after downloading.
::
train/
n01440764/
...
val/
256_ObjectCategories/
001.ak47/
...
image_list/
caltech_256_list.txt
...
"""
image_list = {
"C": "image_list/caltech_256_list.txt",
"I": "image_list/imagenet_val_85_list.txt",
}
CLASSES = _CLASSES + ['unknown']
def __init__(self, root: str, task: str, download: Optional[bool] = True, **kwargs):
assert task in self.image_list
data_list_file = os.path.join(root, self.image_list[task])
if download:
list(map(lambda args: download_data(root, *args), download_list))
else:
list(map(lambda file_name, _: check_exits(root, file_name), download_list))
if not os.path.exists(os.path.join(root, 'val')):
print("Please put train and val directory of ImageNet-1K manually under {} "
"since ImageNet-1K is no longer publicly accessible.".format(root))
exit(-1)
super(CaltechImageNetUniversal, self).__init__(root, CaltechImageNetUniversal.CLASSES,
data_list_file=data_list_file, **kwargs)
download_list = [
("image_list", "image_list.zip", "https://cloud.tsinghua.edu.cn/f/a0d7ea37026946f98965/?dl=1"),
("256_ObjectCategories", "256_ObjectCategories.tar",
"http://www.vision.caltech.edu/Image_Datasets/Caltech256/256_ObjectCategories.tar"),
]
================================================
FILE: domain_generalization/Transfer-Learning-Library/common/vision/datasets/partial/imagenet_caltech.py
================================================
"""
@author: Junguang Jiang
@contact: JiangJunguang1123@outlook.com
"""
import os
from typing import Optional
from ..imagelist import ImageList
from .._util import download as download_data, check_exits
_CLASSES = [c[0] for c in [('tench', 'Tinca tinca'), ('goldfish', 'Carassius auratus'),
('great white shark', 'white shark', 'man-eater', 'man-eating shark', 'Carcharodon carcharias'),
('tiger shark', 'Galeocerdo cuvieri'), ('hammerhead', 'hammerhead shark'),
('electric ray', 'crampfish', 'numbfish', 'torpedo'), ('stingray',), ('cock',), ('hen',),
('ostrich', 'Struthio camelus'), ('brambling', 'Fringilla montifringilla'),
('goldfinch', 'Carduelis carduelis'), ('house finch', 'linnet', 'Carpodacus mexicanus'),
('junco', 'snowbird'), ('indigo bunting', 'indigo finch', 'indigo bird', 'Passerina cyanea'),
('robin', 'American robin', 'Turdus migratorius'), ('bulbul',), ('jay',), ('magpie',),
('chickadee',), ('water ouzel', 'dipper'), ('kite',),
('bald eagle', 'American eagle', 'Haliaeetus leucocephalus'), ('vulture',),
('great grey owl', 'great gray owl', 'Strix nebulosa'),
('European fire salamander', 'Salamandra salamandra'), ('common newt', 'Triturus vulgaris'),
('eft',), ('spotted salamander', 'Ambystoma maculatum'),
('axolotl', 'mud puppy', 'Ambystoma mexicanum'), ('bullfrog', 'Rana catesbeiana'),
('tree frog', 'tree-frog'),
('tailed frog', 'bell toad', 'ribbed toad', 'tailed toad', 'Ascaphus trui'),
('loggerhead', 'loggerhead turtle', 'Caretta caretta'),
('leatherback turtle', 'leatherback', 'leathery turtle', 'Dermochelys coriacea'),
('mud turtle',), ('terrapin',), ('box turtle', 'box tortoise'), ('banded gecko',),
('common iguana', 'iguana', 'Iguana iguana'),
('American chameleon', 'anole', 'Anolis carolinensis'), ('whiptail', 'whiptail lizard'),
('agama',), ('frilled lizard', 'Chlamydosaurus kingi'), ('alligator lizard',),
('Gila monster', 'Heloderma suspectum'), ('green lizard', 'Lacerta viridis'),
('African chameleon', 'Chamaeleo chamaeleon'),
('Komodo dragon', 'Komodo lizard', 'dragon lizard', 'giant lizard', 'Varanus komodoensis'),
('African crocodile', 'Nile crocodile', 'Crocodylus niloticus'),
('American alligator', 'Alligator mississipiensis'), ('triceratops',),
('thunder snake', 'worm snake', 'Carphophis amoenus'),
('ringneck snake', 'ring-necked snake', 'ring snake'),
('hognose snake', 'puff adder', 'sand viper'), ('green snake', 'grass snake'),
('king snake', 'kingsnake'), ('garter snake', 'grass snake'), ('water snake',),
('vine snake',), ('night snake', 'Hypsiglena torquata'),
('boa constrictor', 'Constrictor constrictor'), ('rock python', 'rock snake', 'Python sebae'),
('Indian cobra', 'Naja naja'), ('green mamba',), ('sea snake',),
('horned viper', 'cerastes', 'sand viper', 'horned asp', 'Cerastes cornutus'),
('diamondback', 'diamondback rattlesnake', 'Crotalus adamanteus'),
('sidewinder', 'horned rattlesnake', 'Crotalus cerastes'), ('trilobite',),
('harvestman', 'daddy longlegs', 'Phalangium opilio'), ('scorpion',),
('black and gold garden spider', 'Argiope aurantia'), ('barn spider', 'Araneus cavaticus'),
('garden spider', 'Aranea diademata'), ('black widow', 'Latrodectus mactans'), ('tarantula',),
('wolf spider', 'hunting spider'), ('tick',), ('centipede',), ('black grouse',),
('ptarmigan',), ('ruffed grouse', 'partridge', 'Bonasa umbellus'),
('prairie chicken', 'prairie grouse', 'prairie fowl'), ('peacock',), ('quail',),
('partridge',), ('African grey', 'African gray', 'Psittacus erithacus'), ('macaw',),
('sulphur-crested cockatoo', 'Kakatoe galerita', 'Cacatua galerita'), ('lorikeet',),
('coucal',), ('bee eater',), ('hornbill',), ('hummingbird',), ('jacamar',), ('toucan',),
('drake',), ('red-breasted merganser', 'Mergus serrator'), ('goose',),
('black swan', 'Cygnus atratus'), ('tusker',), ('echidna', 'spiny anteater', 'anteater'), (
'platypus', 'duckbill', 'duckbilled platypus', 'duck-billed platypus',
'Ornithorhynchus anatinus'), ('wallaby', 'brush kangaroo'),
('koala', 'koalabear', 'kangaroo bear', 'native bear', 'Phascolarctos cinereus'), ('wombat',),
('jellyfish',), ('sea anemone', 'anemone'), ('brain coral',), ('flatworm', 'platyhelminth'),
('nematode', 'nematode worm', 'roundworm'), ('conch',), ('snail',), ('slug',),
('sea slug', 'nudibranch'), ('chiton', 'coat-of-mail shell', 'sea cradle', 'polyplacophore'),
('chambered nautilus', 'pearly nautilus', 'nautilus'), ('Dungeness crab', 'Cancer magister'),
('rock crab', 'Cancer irroratus'), ('fiddler crab',), (
'king crab', 'Alaska crab', 'Alaskan king crab', 'Alaska king crab',
'Paralithodes camtschatica'),
('American lobster', 'Northernlobster', 'Maine lobster', 'Homarus americanus'),
('spiny lobster', 'langouste', 'rock lobster', 'crawfish', 'crayfish', 'sea crawfish'),
('crayfish', 'crawfish', 'crawdad', 'crawdaddy'), ('hermit crab',), ('isopod',),
('white stork', 'Ciconia ciconia'), ('black stork', 'Ciconia nigra'), ('spoonbill',),
('flamingo',), ('little blue heron', 'Egretta caerulea'),
('American egret', 'great white heron', 'Egretta albus'), ('bittern',), ('crane',),
('limpkin', 'Aramus pictus'), ('European gallinule', 'Porphyrio porphyrio'),
('American coot', 'marsh hen', 'mud hen', 'water hen', 'Fulica americana'), ('bustard',),
('ruddy turnstone', 'Arenaria interpres'),
('red-backed sandpiper', 'dunlin', 'Erolia alpina'), ('redshank', 'Tringa totanus'),
('dowitcher',), ('oystercatcher', 'oyster catcher'), ('pelican',),
('king penguin', 'Aptenodytes patagonica'), ('albatross', 'mollymawk'),
('grey whale', 'gray whale', 'devilfish', 'Eschrichtius gibbosus', 'Eschrichtius robustus'),
('killer whale', 'killer', 'orca', 'grampus', 'sea wolf', 'Orcinusorca'),
('dugong', 'Dugong dugon'), ('sea lion',), ('Chihuahua',), ('Japanese spaniel',),
('Maltese dog', 'Maltese terrier', 'Maltese'), ('Pekinese', 'Pekingese', 'Peke'),
('Shih-Tzu',), ('Blenheim spaniel',), ('papillon',), ('toy terrier',),
('Rhodesian ridgeback',), ('Afghan hound', 'Afghan'), ('basset', 'basset hound'), ('beagle',),
('bloodhound', 'sleuthhound'), ('bluetick',), ('black-and-tan coonhound',),
('Walker hound', 'Walker foxhound'), ('English foxhound',), ('redbone',),
('borzoi', 'Russian wolfhound'), ('Irish wolfhound',), ('Italian greyhound',), ('whippet',),
('Ibizan hound', 'Ibizan Podenco'), ('Norwegian elkhound', 'elkhound'),
('otterhound', 'otter hound'), ('Saluki', 'gazelle hound'),
('Scottish deerhound', 'deerhound'), ('Weimaraner',),
('Staffordshire bullterrier', 'Staffordshire bull terrier'), (
'American Staffordshire terrier', 'Staffordshire terrier', 'American pit bull terrier',
'pit bull terrier'), ('Bedlington terrier',), ('Border terrier',),
('Kerry blue terrier',),
('Irish terrier',), ('Norfolkterrier',), ('Norwich terrier',), ('Yorkshire terrier',),
('wire-haired fox terrier',), ('Lakeland terrier',), ('Sealyham terrier', 'Sealyham'),
('Airedale', 'Airedale terrier'), ('cairn', 'cairn terrier'), ('Australian terrier',),
('Dandie Dinmont', 'Dandie Dinmont terrier'), ('Boston bull', 'Boston terrier'),
('miniature schnauzer',), ('giant schnauzer',), ('standard schnauzer',),
('Scotch terrier', 'Scottish terrier', 'Scottie'), ('Tibetan terrier', 'chrysanthemum dog'),
('silky terrier', 'Sydney silky'), ('soft-coated wheaten terrier',),
('West Highland white terrier',), ('Lhasa', 'Lhasa apso'), ('flat-coated retriever',),
('curly-coated retriever',), ('golden retriever',), ('Labrador retriever',),
('Chesapeake Bay retriever',), ('German short-haired pointer',),
('vizsla', 'Hungarian pointer'), ('English setter',), ('Irish setter', 'red setter'),
('Gordon setter',), ('Brittany spaniel',), ('clumber', 'clumber spaniel'),
('English springer', 'English springer spaniel'), ('Welsh springer spaniel',),
('cocker spaniel', 'English cocker spaniel', 'cocker'), ('Sussex spaniel',),
('Irish water spaniel',), ('kuvasz',), ('schipperke',), ('groenendael',), ('malinois',),
('briard',), ('kelpie',), ('komondor',), ('Old English sheepdog', 'bobtail'),
('Shetland sheepdog', 'Shetland sheep dog', 'Shetland'), ('collie',), ('Border collie',),
('Bouvier des Flandres', 'Bouviers des Flandres'), ('Rottweiler',),
('German shepherd', 'German shepherd dog', 'German police dog', 'alsatian'),
('Doberman', 'Doberman pinscher'), ('miniature pinscher',), ('Greater Swiss Mountain dog',),
('Bernese mountain dog',), ('Appenzeller',), ('EntleBucher',), ('boxer',), ('bull mastiff',),
('Tibetan mastiff',), ('French bulldog',), ('Great Dane',), ('Saint Bernard', 'St Bernard'),
('Eskimo dog', 'husky'), ('malamute', 'malemute', 'Alaskan malamute'), ('Siberian husky',),
('dalmatian', 'coach dog', 'carriage dog'),
('affenpinscher', 'monkey pinscher', 'monkey dog'), ('basenji',), ('pug', 'pug-dog'),
('Leonberg',), ('Newfoundland', 'Newfoundland dog'), ('Great Pyrenees',),
('Samoyed', 'Samoyede'), ('Pomeranian',), ('chow', 'chow chow'), ('keeshond',),
('Brabancon griffon',), ('Pembroke', 'Pembroke Welsh corgi'),
('Cardigan', 'Cardigan Welsh corgi'), ('toy poodle',), ('miniature poodle',),
('standard poodle',), ('Mexican hairless',),
('timber wolf', 'grey wolf', 'gray wolf', 'Canis lupus'),
('white wolf', 'Arctic wolf', 'Canis lupus tundrarum'),
('red wolf', 'maned wolf', 'Canis rufus', 'Canis niger'),
('coyote', 'prairie wolf', 'brush wolf', 'Canis latrans'),
('dingo', 'warrigal', 'warragal', 'Canis dingo'), ('dhole', 'Cuon alpinus'),
('African hunting dog', 'hyena dog', 'Cape hunting dog', 'Lycaon pictus'),
('hyena', 'hyaena'), ('red fox', 'Vulpes vulpes'), ('kit fox', 'Vulpes macrotis'),
('Arctic fox', 'white fox', 'Alopex lagopus'),
('grey fox', 'gray fox', 'Urocyon cinereoargenteus'), ('tabby', 'tabby cat'), ('tiger cat',),
('Persian cat',), ('Siamese cat', 'Siamese'), ('Egyptian cat',),
('cougar', 'puma', 'catamount', 'mountain lion', 'painter', 'panther', 'Felis concolor'),
('lynx', 'catamount'), ('leopard', 'Panthera pardus'),
('snow leopard', 'ounce', 'Panthera uncia'),
('jaguar', 'panther', 'Panthera onca', 'Felis onca'),
('lion', 'king of beasts', 'Panthera leo'), ('tiger', 'Panthera tigris'),
('cheetah', 'chetah', 'Acinonyx jubatus'), ('brown bear', 'bruin', 'Ursus arctos'),
('American black bear', 'black bear', 'Ursus americanus', 'Euarctos americanus'),
('ice bear', 'polar bear', 'Ursus Maritimus', 'Thalarctos maritimus'),
('sloth bear', 'Melursus ursinus', 'Ursus ursinus'), ('mongoose',), ('meerkat', 'mierkat'),
('tiger beetle',), ('ladybug', 'ladybeetle', 'lady beetle', 'ladybird', 'ladybird beetle'),
('ground beetle', 'carabid beetle'), ('long-horned beetle', 'longicorn', 'longicorn beetle'),
('leaf beetle', 'chrysomelid'), ('dung beetle',), ('rhinoceros beetle',), ('weevil',),
('fly',), ('bee',), ('ant', 'emmet', 'pismire'), ('grasshopper', 'hopper'), ('cricket',),
('walking stick', 'walkingstick', 'stick insect'), ('cockroach', 'roach'),
('mantis', 'mantid'), ('cicada', 'cicala'), ('leafhopper',), ('lacewing', 'lacewing fly'), (
'dragonfly', 'darning needle', "devil's darning needle", 'sewing needle', 'snake feeder',
'snake doctor', 'mosquito hawk', 'skeeter hawk'), ('damselfly',), ('admiral',),
('ringlet', 'ringlet butterfly'),
('monarch', 'monarch butterfly', 'milkweed butterfly', 'Danaus plexippus'),
('cabbage butterfly',), ('sulphur butterfly', 'sulfur butterfly'),
('lycaenid', 'lycaenid butterfly'), ('starfish', 'sea star'), ('sea urchin',),
('sea cucumber', 'holothurian'), ('wood rabbit', 'cottontail', 'cottontail rabbit'),
('hare',), ('Angora', 'Angora rabbit'), ('hamster',), ('porcupine', 'hedgehog'),
('fox squirrel', 'eastern fox squirrel', 'Sciurus niger'), ('marmot',), ('beaver',),
('guinea pig', 'Cavia cobaya'), ('sorrel',), ('zebra',),
('hog', 'pig', 'grunter', 'squealer', 'Sus scrofa'), ('wild boar', 'boar', 'Sus scrofa'),
('warthog',), ('hippopotamus', 'hippo', 'river horse', 'Hippopotamus amphibius'), ('ox',),
('water buffalo', 'water ox', 'Asiatic buffalo', 'Bubalus bubalis'), ('bison',),
('ram', 'tup'), (
'bighorn', 'bighorn sheep', 'cimarron', 'Rocky Mountain bighorn', 'Rocky Mountain sheep',
'Ovis canadensis'), ('ibex', 'Capra ibex'), ('hartebeest',),
('impala', 'Aepyceros melampus'),
('gazelle',), ('Arabian camel', 'dromedary', 'Camelus dromedarius'), ('llama',), ('weasel',),
('mink',), ('polecat', 'fitch', 'foulmart', 'foumart', 'Mustela putorius'),
('black-footed ferret', 'ferret', 'Mustela nigripes'), ('otter',),
('skunk', 'polecat', 'wood pussy'), ('badger',), ('armadillo',),
('three-toed sloth', 'ai', 'Bradypus tridactylus'),
('orangutan', 'orang', 'orangutang', 'Pongo pygmaeus'), ('gorilla', 'Gorilla gorilla'),
('chimpanzee', 'chimp', 'Pan troglodytes'), ('gibbon', 'Hylobates lar'),
('siamang', 'Hylobates syndactylus', 'Symphalangus syndactylus'), ('guenon', 'guenon monkey'),
('patas', 'hussar monkey', 'Erythrocebus patas'), ('baboon',), ('macaque',), ('langur',),
('colobus', 'colobus monkey'), ('proboscis monkey', 'Nasalis larvatus'), ('marmoset',),
('capuchin', 'ringtail', 'Cebus capucinus'), ('howler monkey', 'howler'),
('titi', 'titi monkey'), ('spider monkey', 'Ateles geoffroyi'),
('squirrel monkey', 'Saimiri sciureus'),
('Madagascar cat', 'ring-tailed lemur', 'Lemur catta'),
('indri', 'indris', 'Indri indri', 'Indri brevicaudatus'),
('Indian elephant', 'Elephas maximus'), ('African elephant', 'Loxodonta africana'),
('lesser panda', 'red panda', 'panda', 'bear cat', 'cat bear', 'Ailurus fulgens'),
('giant panda', 'panda', 'panda bear', 'coon bear', 'Ailuropoda melanoleuca'),
('barracouta', 'snoek'), ('eel',),
('coho', 'cohoe', 'coho salmon', 'blue jack', 'silver salmon', 'Oncorhynchus kisutch'),
('rock beauty', 'Holocanthus tricolor'), ('anemone fish',), ('sturgeon',),
('gar', 'garfish', 'garpike', 'billfish', 'Lepisosteus osseus'), ('lionfish',),
('puffer', 'pufferfish', 'blowfish', 'globefish'), ('abacus',), ('abaya',),
('academic gown', 'academic robe', "judge's robe"),
('accordion', 'piano accordion', 'squeeze box'), ('acoustic guitar',),
('aircraft carrier', 'carrier', 'flattop', 'attack aircraft carrier'), ('airliner',),
('airship', 'dirigible'), ('altar',), ('ambulance',), ('amphibian', 'amphibious vehicle'),
('analog clock',), ('apiary', 'bee house'), ('apron',), (
'ashcan', 'trash can', 'garbage can', 'wastebin', 'ash bin', 'ash-bin', 'ashbin',
'dustbin',
'trash barrel', 'trash bin'), ('assault rifle', 'assault gun'),
('backpack', 'back pack', 'knapsack', 'packsack', 'rucksack', 'haversack'),
('bakery', 'bakeshop', 'bakehouse'), ('balance beam', 'beam'), ('balloon',),
('ballpoint', 'ballpoint pen', 'ballpen', 'Biro'), ('Band Aid',), ('banjo',),
('bannister', 'banister', 'balustrade', 'balusters', 'handrail'), ('barbell',),
('barber chair',), ('barbershop',), ('barn',), ('barometer',), ('barrel', 'cask'),
('barrow', 'garden cart', 'lawn cart', 'wheelbarrow'), ('baseball',), ('basketball',),
('bassinet',), ('bassoon',), ('bathing cap', 'swimming cap'), ('bath towel',),
('bathtub', 'bathing tub', 'bath', 'tub'), (
'beach wagon', 'station wagon', 'wagon', 'estate car', 'beach waggon', 'station waggon',
'waggon'), ('beacon', 'lighthouse', 'beacon light', 'pharos'), ('beaker',),
('bearskin', 'busby', 'shako'), ('beer bottle',), ('beer glass',), ('bell cote', 'bell cot'),
('bib',), ('bicycle-built-for-two', 'tandem bicycle', 'tandem'), ('bikini', 'two-piece'),
('binder', 'ring-binder'), ('binoculars', 'field glasses', 'opera glasses'), ('birdhouse',),
('boathouse',), ('bobsled', 'bobsleigh', 'bob'), ('bolo tie', 'bolo', 'bola tie', 'bola'),
('bonnet', 'poke bonnet'), ('bookcase',), ('bookshop', 'bookstore', 'bookstall'),
('bottlecap',), ('bow',), ('bow tie', 'bow-tie', 'bowtie'),
('brass', 'memorial tablet', 'plaque'), ('brassiere', 'bra', 'bandeau'),
('breakwater', 'groin', 'groyne', 'mole', 'bulwark', 'seawall', 'jetty'),
('breastplate', 'aegis', 'egis'), ('broom',), ('bucket', 'pail'), ('buckle',),
('bulletproof vest',), ('bullet train', 'bullet'), ('butcher shop', 'meat market'),
('cab', 'hack', 'taxi', 'taxicab'), ('caldron', 'cauldron'), ('candle', 'taper', 'wax light'),
('cannon',), ('canoe',), ('can opener', 'tin opener'), ('cardigan',), ('car mirror',),
('carousel', 'carrousel', 'merry-go-round', 'roundabout', 'whirligig'),
("carpenter's kit", 'tool kit'), ('carton',), ('car wheel',), (
'cash machine', 'cash dispenser', 'automated teller machine', 'automatic teller machine',
'automated teller', 'automatic teller', 'ATM'), ('cassette',), ('cassette player',),
('castle',), ('catamaran',), ('CD player',), ('cello', 'violoncello'),
('cellular telephone', 'cellular phone', 'cellphone', 'cell', 'mobile phone'), ('chain',),
('chainlink fence',), (
'chain mail', 'ring mail', 'mail', 'chain armor', 'chain armour', 'ring armor',
'ring armour'), ('chain saw', 'chainsaw'), ('chest',), ('chiffonier', 'commode'),
('chime', 'bell', 'gong'), ('china cabinet', 'china closet'), ('Christmas stocking',),
('church', 'church building'),
('cinema', 'movie theater', 'movie theatre', 'movie house', 'picture palace'),
('cleaver', 'meat cleaver', 'chopper'), ('cliff dwelling',), ('cloak',),
('clog', 'geta', 'patten', 'sabot'), ('cocktail shaker',), ('coffee mug',), ('coffeepot',),
('coil', 'spiral', 'volute', 'whorl', 'helix'), ('combination lock',),
('computer keyboard', 'keypad'), ('confectionery', 'confectionary', 'candy store'),
('container ship', 'containership', 'container vessel'), ('convertible',),
('corkscrew', 'bottle screw'), ('cornet', 'horn', 'trumpet', 'trump'), ('cowboy boot',),
('cowboy hat', 'ten-gallon hat'), ('cradle',), ('crane',), ('crash helmet',), ('crate',),
('crib', 'cot'), ('Crock Pot',), ('croquet ball',), ('crutch',), ('cuirass',),
('dam', 'dike', 'dyke'), ('desk',), ('desktop computer',), ('dial telephone', 'dial phone'),
('diaper', 'nappy', 'napkin'), ('digital clock',), ('digital watch',),
('dining table', 'board'), ('dishrag', 'dishcloth'),
('dishwasher', 'dish washer', 'dishwashing machine'), ('disk brake', 'disc brake'),
('dock', 'dockage', 'docking facility'), ('dogsled', 'dog sled', 'dog sleigh'), ('dome',),
('doormat', 'welcome mat'), ('drilling platform', 'offshore rig'),
('drum', 'membranophone', 'tympan'), ('drumstick',), ('dumbbell',), ('Dutch oven',),
('electric fan', 'blower'), ('electric guitar',), ('electric locomotive',),
('entertainment center',), ('envelope',), ('espresso maker',), ('face powder',),
('feather boa', 'boa'), ('file', 'file cabinet', 'filing cabinet'), ('fireboat',),
('fire engine', 'fire truck'), ('fire screen', 'fireguard'), ('flagpole', 'flagstaff'),
('flute', 'transverse flute'), ('folding chair',), ('football helmet',), ('forklift',),
('fountain',), ('fountain pen',), ('four-poster',), ('freight car',), ('French horn', 'horn'),
('frying pan', 'frypan', 'skillet'), ('fur coat',), ('garbage truck', 'dustcart'),
('gasmask', 'respirator', 'gas helmet'),
('gas pump', 'gasoline pump', 'petrol pump', 'island dispenser'), ('goblet',), ('go-kart',),
('golf ball',), ('golfcart', 'golf cart'), ('gondola',), ('gong', 'tam-tam'), ('gown',),
('grand piano', 'grand'), ('greenhouse', 'nursery', 'glasshouse'),
('grille', 'radiator grille'), ('grocery store', 'grocery', 'food market', 'market'),
('guillotine',), ('hair slide',), ('hair spray',), ('half track',), ('hammer',), ('hamper',),
('hand blower', 'blow dryer', 'blow drier', 'hair dryer', 'hair drier'),
('hand-held computer', 'hand-held microcomputer'),
('handkerchief', 'hankie', 'hanky', 'hankey'), ('hard disc', 'hard disk', 'fixed disk'),
('harmonica', 'mouth organ', 'harp', 'mouth harp'), ('harp',), ('harvester', 'reaper'),
('hatchet',), ('holster',), ('home theater', 'home theatre'), ('honeycomb',),
('hook', 'claw'), ('hoopskirt', 'crinoline'), ('horizontal bar', 'high bar'),
('horse cart', 'horse-cart'), ('hourglass',), ('iPod',), ('iron', 'smoothing iron'),
("jack-o'-lantern",), ('jean', 'blue jean', 'denim'), ('jeep', 'landrover'),
('jersey', 'T-shirt', 'tee shirt'), ('jigsaw puzzle',), ('jinrikisha', 'ricksha', 'rickshaw'),
('joystick',), ('kimono',), ('knee pad',), ('knot',), ('lab coat', 'laboratory coat'),
('ladle',), ('lampshade', 'lamp shade'), ('laptop', 'laptop computer'),
('lawn mower', 'mower'), ('lens cap', 'lens cover'),
('letter opener', 'paper knife', 'paperknife'), ('library',), ('lifeboat',),
('lighter', 'light', 'igniter', 'ignitor'), ('limousine', 'limo'), ('liner', 'ocean liner'),
('lipstick', 'lip rouge'), ('Loafer',), ('lotion',),
('loudspeaker', 'speaker', 'speaker unit', 'loudspeaker system', 'speaker system'),
('loupe', "jeweler's loupe"), ('lumbermill', 'sawmill'), ('magnetic compass',),
('mailbag', 'postbag'), ('mailbox', 'letter box'), ('maillot',), ('maillot', 'tank suit'),
('manhole cover',), ('maraca',), ('marimba', 'xylophone'), ('mask',), ('matchstick',),
('maypole',), ('maze', 'labyrinth'), ('measuring cup',),
('medicine chest', 'medicine cabinet'), ('megalith', 'megalithic structure'),
('microphone', 'mike'), ('microwave', 'microwave oven'), ('military uniform',), ('milk can',),
('minibus',), ('miniskirt', 'mini'), ('minivan',), ('missile',), ('mitten',),
('mixing bowl',), ('mobile home', 'manufactured home'), ('Model T',), ('modem',),
('monastery',), ('monitor',), ('moped',), ('mortar',), ('mortarboard',), ('mosque',),
('mosquito net',), ('motor scooter', 'scooter'),
('mountain bike', 'all-terrain bike', 'off-roader'), ('mountain tent',),
('mouse', 'computer mouse'), ('mousetrap',), ('moving van',), ('muzzle',), ('nail',),
('neck brace',), ('necklace',), ('nipple',), ('notebook', 'notebook computer'), ('obelisk',),
('oboe', 'hautboy', 'hautbois'), ('ocarina', 'sweet potato'),
('odometer', 'hodometer', 'mileometer', 'milometer'), ('oil filter',),
('organ', 'pipe organ'), ('oscilloscope', 'scope', 'cathode-ray oscilloscope', 'CRO'),
('overskirt',), ('oxcart',), ('oxygen mask',), ('packet',), ('paddle', 'boat paddle'),
('paddlewheel', 'paddle wheel'), ('padlock',), ('paintbrush',),
('pajama', 'pyjama', "pj's", 'jammies'), ('palace',), ('panpipe', 'pandean pipe', 'syrinx'),
('paper towel',), ('parachute', 'chute'), ('parallel bars', 'bars'), ('park bench',),
('parking meter',), ('passenger car', 'coach', 'carriage'), ('patio', 'terrace'),
('pay-phone', 'pay-station'), ('pedestal', 'plinth', 'footstall'),
('pencil box', 'pencil case'), ('pencil sharpener',), ('perfume', 'essence'), ('Petri dish',),
('photocopier',), ('pick', 'plectrum', 'plectron'), ('pickelhaube',),
('picket fence', 'paling'), ('pickup', 'pickup truck'), ('pier',),
('piggy bank', 'penny bank'), ('pill bottle',), ('pillow',), ('ping-pong ball',),
('pinwheel',), ('pirate', 'pirate ship'), ('pitcher', 'ewer'),
('plane', "carpenter's plane", 'woodworking plane'), ('planetarium',), ('plastic bag',),
('plate rack',), ('plow', 'plough'), ('plunger', "plumber's helper"),
('Polaroid camera', 'Polaroid Land camera'), ('pole',),
('police van', 'police wagon', 'paddy wagon', 'patrol wagon', 'wagon', 'black Maria'),
('poncho',), ('pool table', 'billiard table', 'snooker table'), ('pop bottle', 'soda bottle'),
('pot', 'flowerpot'), ("potter's wheel",), ('power drill',), ('prayer rug', 'prayer mat'),
('printer',), ('prison', 'prison house'), ('projectile', 'missile'), ('projector',),
('puck', 'hockey puck'), ('punching bag', 'punch bag', 'punching ball', 'punchball'),
('purse',), ('quill', 'quill pen'), ('quilt', 'comforter', 'comfort', 'puff'),
('racer', 'race car', 'racing car'), ('racket', 'racquet'), ('radiator',),
('radio', 'wireless'), ('radio telescope', 'radio reflector'), ('rain barrel',),
('recreational vehicle', 'RV', 'R.V.'), ('reel',), ('reflex camera',),
('refrigerator', 'icebox'), ('remote control', 'remote'),
('restaurant', 'eating house', 'eating place', 'eatery'),
('revolver', 'six-gun', 'six-shooter'), ('rifle',), ('rocking chair', 'rocker'),
('rotisserie',), ('rubber eraser', 'rubber', 'pencil eraser'), ('rugby ball',),
('rule', 'ruler'), ('running shoe',), ('safe',), ('safety pin',),
('saltshaker', 'salt shaker'), ('sandal',), ('sarong',), ('sax', 'saxophone'), ('scabbard',),
('scale', 'weighing machine'), ('school bus',), ('schooner',), ('scoreboard',),
('screen', 'CRT screen'), ('screw',), ('screwdriver',), ('seat belt', 'seatbelt'),
('sewing machine',), ('shield', 'buckler'), ('shoe shop', 'shoe-shop', 'shoe store'),
('shoji',), ('shopping basket',), ('shopping cart',), ('shovel',), ('shower cap',),
('shower curtain',), ('ski',), ('ski mask',), ('sleeping bag',), ('slide rule', 'slipstick'),
('sliding door',), ('slot', 'one-armed bandit'), ('snorkel',), ('snowmobile',),
('snowplow', 'snowplough'), ('soap dispenser',), ('soccer ball',), ('sock',),
('solar dish', 'solar collector', 'solar furnace'), ('sombrero',), ('soup bowl',),
('space bar',), ('space heater',), ('space shuttle',), ('spatula',), ('speedboat',),
('spider web', "spider's web"), ('spindle',), ('sports car', 'sport car'),
('spotlight', 'spot'), ('stage',), ('steam locomotive',), ('steel arch bridge',),
('steel drum',), ('stethoscope',), ('stole',), ('stone wall',), ('stopwatch', 'stop watch'),
('stove',), ('strainer',), ('streetcar', 'tram', 'tramcar', 'trolley', 'trolley car'),
('stretcher',), ('studio couch', 'day bed'), ('stupa', 'tope'),
('submarine', 'pigboat', 'sub', 'U-boat'), ('suit', 'suit of clothes'), ('sundial',),
('sunglass',), ('sunglasses', 'dark glasses', 'shades'),
('sunscreen', 'sunblock', 'sun blocker'), ('suspension bridge',), ('swab', 'swob', 'mop'),
('sweatshirt',), ('swimming trunks', 'bathing trunks'), ('swing',),
('switch', 'electric switch', 'electrical switch'), ('syringe',), ('table lamp',),
('tank', 'army tank', 'armored combat vehicle', 'armoured combat vehicle'), ('tape player',),
('teapot',), ('teddy', 'teddy bear'), ('television', 'television system'), ('tennis ball',),
('thatch', 'thatched roof'), ('theater curtain', 'theatre curtain'), ('thimble',),
('thresher', 'thrasher', 'threshing machine'), ('throne',), ('tile roof',), ('toaster',),
('tobacco shop', 'tobacconist shop', 'tobacconist'), ('toilet seat',), ('torch',),
('totem pole',), ('tow truck', 'tow car', 'wrecker'), ('toyshop',), ('tractor',),
('trailer truck', 'tractor trailer', 'trucking rig', 'rig', 'articulated lorry', 'semi'),
('tray',), ('trench coat',), ('tricycle', 'trike', 'velocipede'), ('trimaran',), ('tripod',),
('triumphal arch',), ('trolleybus', 'trolley coach', 'trackless trolley'), ('trombone',),
('tub', 'vat'), ('turnstile',), ('typewriter keyboard',), ('umbrella',),
('unicycle', 'monocycle'), ('upright', 'upright piano'), ('vacuum', 'vacuum cleaner'),
('vase',), ('vault',), ('velvet',), ('vending machine',), ('vestment',), ('viaduct',),
('violin', 'fiddle'), ('volleyball',), ('waffle iron',), ('wall clock',),
('wallet', 'billfold', 'notecase', 'pocketbook'), ('wardrobe', 'closet', 'press'),
('warplane', 'military plane'),
('washbasin', 'handbasin', 'washbowl', 'lavabo', 'wash-hand basin'),
('washer', 'automatic washer', 'washing machine'), ('water bottle',), ('water jug',),
('water tower',), ('whiskey jug',), ('whistle',), ('wig',), ('window screen',),
('window shade',), ('Windsor tie',), ('wine bottle',), ('wing',), ('wok',), ('wooden spoon',),
('wool', 'woolen', 'woollen'),
('worm fence', 'snake fence', 'snake-rail fence', 'Virginia fence'), ('wreck',), ('yawl',),
('yurt',), ('web site', 'website', 'internet site', 'site'), ('comic book',),
('crossword puzzle', 'crossword'), ('street sign',),
('traffic light', 'traffic signal', 'stoplight'),
('book jacket', 'dust cover', 'dust jacket', 'dust wrapper'), ('menu',), ('plate',),
('guacamole',), ('consomme',), ('hot pot', 'hotpot'), ('trifle',), ('ice cream', 'icecream'),
('ice lolly', 'lolly', 'lollipop', 'popsicle'), ('French loaf',), ('bagel', 'beigel'),
('pretzel',), ('cheeseburger',), ('hotdog', 'hot dog', 'red hot'), ('mashed potato',),
('head cabbage',), ('broccoli',), ('cauliflower',), ('zucchini', 'courgette'),
('spaghetti squash',), ('acorn squash',), ('butternut squash',), ('cucumber', 'cuke'),
('artichoke', 'globe artichoke'), ('bell pepper',), ('cardoon',), ('mushroom',),
('Granny Smith',), ('strawberry',), ('orange',), ('lemon',), ('fig',),
('pineapple', 'ananas'), ('banana',), ('jackfruit', 'jak', 'jack'), ('custard apple',),
('pomegranate',), ('hay',), ('carbonara',), ('chocolate sauce', 'chocolate syrup'),
('dough',), ('meat loaf', 'meatloaf'), ('pizza', 'pizza pie'), ('potpie',), ('burrito',),
('red wine',), ('espresso',), ('cup',), ('eggnog',), ('alp',), ('bubble',),
('cliff', 'drop', 'drop-off'), ('coral reef',), ('geyser',), ('lakeside', 'lakeshore'),
('promontory', 'headland', 'head', 'foreland'), ('sandbar', 'sand bar'),
('seashore', 'coast', 'seacoast', 'sea-coast'), ('valley', 'vale'), ('volcano',),
('ballplayer', 'baseball player'), ('groom', 'bridegroom'), ('scuba diver',), ('rapeseed',),
('daisy',), ("yellow lady's slipper", 'yellow lady-slipper', 'Cypripedium calceolus',
'Cypripedium parviflorum'), ('corn',), ('acorn',),
('hip', 'rose hip', 'rosehip'), ('buckeye', 'horse chestnut', 'conker'), ('coral fungus',),
('agaric',), ('gyromitra',), ('stinkhorn', 'carrion fungus'), ('earthstar',),
('hen-of-the-woods', 'hen of the woods', 'Polyporus frondosus', 'Grifola frondosa'),
('bolete',), ('ear', 'spike', 'capitulum'),
('toilet tissue', 'toilet paper', 'bathroom tissue')]]
class ImageNetCaltech(ImageList):
"""ImageNet-Caltech is constructed from `Caltech-256 `_ and
`ImageNet-1K `_ .
They share 84 common classes. ImageNet-Caltech keeps all classes of ImageNet-1K.
The label is based on the ImageNet-1K (class 0-999) . The private classes of Caltech-256 is discarded.
Args:
root (str): Root directory of dataset
task (str): The task (domain) to create dataset. Choices include ``'C'``:Caltech-256, \
``'I'``: ImageNet-1K training set.
download (bool, optional): If true, downloads the dataset from the internet and puts it \
in root directory. If dataset is already downloaded, it is not downloaded again.
transform (callable, optional): A function/transform that takes in an PIL image and returns a \
transformed version. E.g, :class:`torchvision.transforms.RandomCrop`.
target_transform (callable, optional): A function/transform that takes in the target and transforms it.
.. note:: You need to put ``train`` and ``val`` directory of ImageNet-1K manually in `root` directory
since ImageNet-1K is no longer publicly accessible. DALIB will only download Caltech-256 and ImageList automatically.
In `root`, there will exist following files after downloading.
::
train/
n01440764/
...
val/
256_ObjectCategories/
001.ak47/
...
image_list/
caltech_256_list.txt
...
"""
image_list = {
"I": "image_list/imagenet_train_1000_list.txt",
"C": "image_list/caltech_84_list.txt",
}
CLASSES = _CLASSES
def __init__(self, root: str, task: str, download: Optional[bool] = True, **kwargs):
assert task in self.image_list
data_list_file = os.path.join(root, self.image_list[task])
if download:
list(map(lambda args: download_data(root, *args), download_list))
else:
list(map(lambda file_name, _: check_exits(root, file_name), download_list))
if not os.path.exists(os.path.join(root, 'train')):
print("Please put train and val directory of ImageNet-1K manually under {} "
"since ImageNet-1K is no longer publicly accessible.".format(root))
exit(-1)
super(ImageNetCaltech, self).__init__(root, ImageNetCaltech.CLASSES, data_list_file=data_list_file, **kwargs)
class ImageNetCaltechUniversal(ImageList):
"""ImageNet-Caltech-Universal is constructed from `Caltech-256 `_
and `ImageNet-1K `_ .
They share 84 common classes. ImageNet-Caltech keeps all classes of ImageNet-1K.
The label is based on the ImageNet-1K (class 0-999) . The private classes of Caltech-256 is grouped into class 1000 ("unknown").
Thus, ImageNetCaltechUniversal has 1001 classes in total.
Args:
root (str): Root directory of dataset
task (str): The task (domain) to create dataset. Choices include ``'C'``:Caltech-256, \
``'I'``: ImageNet-1K training set.
download (bool, optional): If true, downloads the dataset from the internet and puts it \
in root directory. If dataset is already downloaded, it is not downloaded again.
transform (callable, optional): A function/transform that takes in an PIL image and returns a \
transformed version. E.g, :class:`torchvision.transforms.RandomCrop`.
target_transform (callable, optional): A function/transform that takes in the target and transforms it.
.. note:: You need to put ``train`` and ``val`` directory of ImageNet-1K manually in `root` directory
since ImageNet-1K is no longer publicly accessible. DALIB will only download Caltech-256 and ImageList automatically.
In `root`, there will exist following files after downloading.
::
train/
n01440764/
...
val/
256_ObjectCategories/
001.ak47/
...
image_list/
caltech_256_list.txt
...
"""
image_list = {
"I": "image_list/imagenet_train_1000_list.txt",
"C": "image_list/caltech_85_list.txt",
}
CLASSES = _CLASSES + ["unknown"]
def __init__(self, root: str, task: str, download: Optional[bool] = True, **kwargs):
assert task in self.image_list
data_list_file = os.path.join(root, self.image_list[task])
if download:
list(map(lambda args: download_data(root, *args), download_list))
else:
list(map(lambda file_name, _: check_exits(root, file_name), download_list))
if not os.path.exists(os.path.join(root, 'train')):
print("Please put train and val directory of ImageNet-1K manually under {} "
"since ImageNet-1K is no longer publicly accessible.".format(root))
exit(-1)
super(ImageNetCaltechUniversal, self).__init__(root, ImageNetCaltechUniversal.CLASSES, data_list_file=data_list_file, **kwargs)
download_list = [
("image_list", "image_list.zip", "https://cloud.tsinghua.edu.cn/f/a0d7ea37026946f98965/?dl=1"),
("256_ObjectCategories", "256_ObjectCategories.tar",
"http://www.vision.caltech.edu/Image_Datasets/Caltech256/256_ObjectCategories.tar"),
]
================================================
FILE: domain_generalization/Transfer-Learning-Library/common/vision/datasets/patchcamelyon.py
================================================
"""
@author: Junguang Jiang
@contact: JiangJunguang1123@outlook.com
"""
import os
from common.vision.datasets import ImageList
from ._util import download as download_data, check_exits
class PatchCamelyon(ImageList):
"""
The `PatchCamelyon `_ dataset contains \
327680 images of histopathologic scans of lymph node sections. \
The classification task consists in predicting the presence of metastatic tissue \
in given image (i.e., two classes). All images are 96x96 pixels
Args:
root (str): Root directory of dataset
split (str, optional): The dataset split, supports ``train``, or ``test``.
download (bool, optional): If true, downloads the dataset from the internet and puts it \
in root directory. If dataset is already downloaded, it is not downloaded again.
transform (callable, optional): A function/transform that takes in an PIL image and returns a \
transformed version. E.g, :class:`torchvision.transforms.RandomCrop`.
target_transform (callable, optional): A function/transform that takes in the target and transforms it.
"""
CLASSES = ['0', '1']
def __init__(self, root, split, download=False, **kwargs):
if download:
download_data(root, "patch_camelyon", "patch_camelyon.tgz", "https://cloud.tsinghua.edu.cn/f/21360b3441a54274b843/?dl=1")
else:
check_exits(root, "patch_camelyon")
root = os.path.join(root, "patch_camelyon")
super(PatchCamelyon, self).__init__(root, PatchCamelyon.CLASSES, os.path.join(root, "imagelist", "{}.txt".format(split)), **kwargs)
================================================
FILE: domain_generalization/Transfer-Learning-Library/common/vision/datasets/regression/__init__.py
================================================
from .image_regression import ImageRegression
from .dsprites import DSprites
from .mpi3d import MPI3D
================================================
FILE: domain_generalization/Transfer-Learning-Library/common/vision/datasets/regression/dsprites.py
================================================
"""
@author: Junguang Jiang
@contact: JiangJunguang1123@outlook.com
"""
from typing import Optional, Sequence
import os
from .._util import download as download_data, check_exits
from .image_regression import ImageRegression
class DSprites(ImageRegression):
"""`DSprites `_ Dataset.
Args:
root (str): Root directory of dataset
task (str): The task (domain) to create dataset. Choices include ``'C'``: Color, \
``'N'``: Noisy and ``'S'``: Scream.
split (str, optional): The dataset split, supports ``train``, or ``test``.
factors (sequence[str]): Factors selected. Default: ('scale', 'position x', 'position y').
download (bool, optional): If true, downloads the dataset from the internet and puts it \
in root directory. If dataset is already downloaded, it is not downloaded again.
transform (callable, optional): A function/transform that takes in an PIL image and returns a \
transformed version. E.g, :class:`torchvision.transforms.RandomCrop`.
target_transform (callable, optional): A function/transform that takes in the target and transforms it.
.. note:: In `root`, there will exist following files after downloading.
::
color/
...
noisy/
scream/
image_list/
color_train.txt
noisy_train.txt
scream_train.txt
color_test.txt
noisy_test.txt
scream_test.txt
"""
download_list = [
("image_list", "image_list.zip", "https://cloud.tsinghua.edu.cn/f/fbbb6b1a43034712b34d/?dl=1"),
("color", "color.tgz", "https://cloud.tsinghua.edu.cn/f/9ce9f2abc61f49ed995a/?dl=1"),
("noisy", "noisy.tgz", "https://cloud.tsinghua.edu.cn/f/674435c8cb914ca0ad10/?dl=1"),
("scream", "scream.tgz", "https://cloud.tsinghua.edu.cn/f/0613675916ac4c3bb6bd/?dl=1"),
]
image_list = {
"C": "color",
"N": "noisy",
"S": "scream"
}
FACTORS = ('none', 'shape', 'scale', 'orientation', 'position x', 'position y')
def __init__(self, root: str, task: str, split: Optional[str] = 'train',
factors: Sequence[str] = ('scale', 'position x', 'position y'),
download: Optional[bool] = True, target_transform=None, **kwargs):
assert task in self.image_list
assert split in ['train', 'test']
for factor in factors:
assert factor in self.FACTORS
factor_index = [self.FACTORS.index(factor) for factor in factors]
if target_transform is None:
target_transform = lambda x: x[list(factor_index)]
else:
target_transform = lambda x: target_transform(x[list(factor_index)])
data_list_file = os.path.join(root, "image_list", "{}_{}.txt".format(self.image_list[task], split))
if download:
list(map(lambda args: download_data(root, *args), self.download_list))
else:
list(map(lambda file_name, _: check_exits(root, file_name), self.download_list))
super(DSprites, self).__init__(root, factors, data_list_file=data_list_file, target_transform=target_transform, **kwargs)
================================================
FILE: domain_generalization/Transfer-Learning-Library/common/vision/datasets/regression/image_regression.py
================================================
"""
@author: Junguang Jiang
@contact: JiangJunguang1123@outlook.com
"""
import os
from typing import Optional, Callable, Tuple, Any, List, Sequence
import torchvision.datasets as datasets
from torchvision.datasets.folder import default_loader
import numpy as np
class ImageRegression(datasets.VisionDataset):
"""A generic Dataset class for domain adaptation in image regression
Args:
root (str): Root directory of dataset
factors (sequence[str]): Factors selected. Default: ('scale', 'position x', 'position y').
data_list_file (str): File to read the image list from.
transform (callable, optional): A function/transform that takes in an PIL image and returns a \
transformed version. E.g, :class:`torchvision.transforms.RandomCrop`.
target_transform (callable, optional): A function/transform that takes in the target and transforms it.
.. note::
In `data_list_file`, each line has `1+len(factors)` values in the following format.
::
source_dir/dog_xxx.png x11, x12, ...
source_dir/cat_123.png x21, x22, ...
target_dir/dog_xxy.png x31, x32, ...
target_dir/cat_nsdf3.png x41, x42, ...
The first value is the relative path of an image, and the rest values are the ground truth of the corresponding factors.
If your data_list_file has different formats, please over-ride :meth:`ImageRegression.parse_data_file`.
"""
def __init__(self, root: str, factors: Sequence[str], data_list_file: str,
transform: Optional[Callable] = None, target_transform: Optional[Callable] = None):
super().__init__(root, transform=transform, target_transform=target_transform)
self.samples = self.parse_data_file(data_list_file)
self.factors = factors
self.loader = default_loader
self.data_list_file = data_list_file
def __getitem__(self, index: int) -> Tuple[Any, Tuple[float]]:
"""
Args:
index (int): Index
Returns:
(image, target) where target is a numpy float array.
"""
path, target = self.samples[index]
img = self.loader(path)
if self.transform is not None:
img = self.transform(img)
if self.target_transform is not None and target is not None:
target = self.target_transform(target)
return img, target
def __len__(self) -> int:
return len(self.samples)
def parse_data_file(self, file_name: str) -> List[Tuple[str, Any]]:
"""Parse file to data list
Args:
file_name (str): The path of data file
Returns:
List of (image path, (factors)) tuples
"""
with open(file_name, "r") as f:
data_list = []
for line in f.readlines():
data = line.split()
path = str(data[0])
target = np.array([float(d) for d in data[1:]], dtype=np.float)
if not os.path.isabs(path):
path = os.path.join(self.root, path)
data_list.append((path, target))
return data_list
@property
def num_factors(self) -> int:
return len(self.factors)
================================================
FILE: domain_generalization/Transfer-Learning-Library/common/vision/datasets/regression/mpi3d.py
================================================
"""
@author: Junguang Jiang
@contact: JiangJunguang1123@outlook.com
"""
from typing import Optional, Sequence
import os
from .._util import download as download_data, check_exits
from .image_regression import ImageRegression
class MPI3D(ImageRegression):
"""`MPI3D `_ Dataset.
Args:
root (str): Root directory of dataset
task (str): The task (domain) to create dataset. Choices include ``'C'``: Color, \
``'N'``: Noisy and ``'S'``: Scream.
split (str, optional): The dataset split, supports ``train``, or ``test``.
factors (sequence[str]): Factors selected. Default: ('horizontal axis', 'vertical axis').
download (bool, optional): If true, downloads the dataset from the internet and puts it \
in root directory. If dataset is already downloaded, it is not downloaded again.
transform (callable, optional): A function/transform that takes in an PIL image and returns a \
transformed version. E.g, :class:`torchvision.transforms.RandomCrop`.
target_transform (callable, optional): A function/transform that takes in the target and transforms it.
.. note:: In `root`, there will exist following files after downloading.
::
real/
...
realistic/
toy/
image_list/
real_train.txt
realistic_train.txt
toy_train.txt
real_test.txt
realistic_test.txt
toy_test.txt
"""
download_list = [
("image_list", "image_list.zip", "https://cloud.tsinghua.edu.cn/f/f0ff24df967b42479d9e/?dl=1"),
("real", "real.tgz", "https://cloud.tsinghua.edu.cn/f/04c1318555fc4283862b/?dl=1"),
("realistic", "realistic.tgz", "https://cloud.tsinghua.edu.cn/f/2c0f7dacc73148cea593/?dl=1"),
("toy", "toy.tgz", "https://cloud.tsinghua.edu.cn/f/6327912a50374e20af95/?dl=1"),
]
image_list = {
"RL": "real",
"RC": "realistic",
"T": "toy"
}
FACTORS = ('horizontal axis', 'vertical axis')
def __init__(self, root: str, task: str, split: Optional[str] = 'train',
factors: Sequence[str] = ('horizontal axis', 'vertical axis'),
download: Optional[bool] = True, target_transform=None, **kwargs):
assert task in self.image_list
assert split in ['train', 'test']
for factor in factors:
assert factor in self.FACTORS
factor_index = [self.FACTORS.index(factor) for factor in factors]
if target_transform is None:
target_transform = lambda x: x[list(factor_index)] / 39.
else:
target_transform = lambda x: target_transform(x[list(factor_index)]) / 39.
data_list_file = os.path.join(root, "image_list", "{}_{}.txt".format(self.image_list[task], split))
if download:
list(map(lambda args: download_data(root, *args), self.download_list))
else:
list(map(lambda file_name, _: check_exits(root, file_name), self.download_list))
super(MPI3D, self).__init__(root, factors, data_list_file=data_list_file, target_transform=target_transform, **kwargs)
================================================
FILE: domain_generalization/Transfer-Learning-Library/common/vision/datasets/reid/__init__.py
================================================
from .market1501 import Market1501
from .dukemtmc import DukeMTMC
from .msmt17 import MSMT17
from .personx import PersonX
from .unreal import UnrealPerson
__all__ = ['Market1501', 'DukeMTMC', 'MSMT17', 'PersonX', 'UnrealPerson']
================================================
FILE: domain_generalization/Transfer-Learning-Library/common/vision/datasets/reid/basedataset.py
================================================
"""
Modified from https://github.com/yxgeee/MMT
@author: Baixu Chen
@contact: cbx_99_hasta@outlook.com
"""
import os.path as osp
import numpy as np
class BaseDataset(object):
"""
Base class of reid dataset
"""
def get_imagedata_info(self, data):
pids, cams = [], []
for _, pid, camid in data:
pids += [pid]
cams += [camid]
pids = set(pids)
cams = set(cams)
num_pids = len(pids)
num_cams = len(cams)
num_imgs = len(data)
return num_pids, num_imgs, num_cams
def get_videodata_info(self, data, return_tracklet_stats=False):
pids, cams, tracklet_stats = [], [], []
for img_paths, pid, camid in data:
pids += [pid]
cams += [camid]
tracklet_stats += [len(img_paths)]
pids = set(pids)
cams = set(cams)
num_pids = len(pids)
num_cams = len(cams)
num_tracklets = len(data)
if return_tracklet_stats:
return num_pids, num_tracklets, num_cams, tracklet_stats
return num_pids, num_tracklets, num_cams
def print_dataset_statistics(self, train, query, galler):
raise NotImplementedError
def check_before_run(self, required_files):
"""Checks if required files exist before going deeper.
Args:
required_files (str or list): string file name(s).
"""
if isinstance(required_files, str):
required_files = [required_files]
for fpath in required_files:
if not osp.exists(fpath):
raise RuntimeError('"{}" is not found'.format(fpath))
@property
def images_dir(self):
return None
class BaseImageDataset(BaseDataset):
"""
Base class of image reid dataset
"""
def print_dataset_statistics(self, train, query, gallery):
num_train_pids, num_train_imgs, num_train_cams = self.get_imagedata_info(train)
num_query_pids, num_query_imgs, num_query_cams = self.get_imagedata_info(query)
num_gallery_pids, num_gallery_imgs, num_gallery_cams = self.get_imagedata_info(gallery)
print("Dataset statistics:")
print(" ----------------------------------------")
print(" subset | # ids | # images | # cameras")
print(" ----------------------------------------")
print(" train | {:5d} | {:8d} | {:9d}".format(num_train_pids, num_train_imgs, num_train_cams))
print(" query | {:5d} | {:8d} | {:9d}".format(num_query_pids, num_query_imgs, num_query_cams))
print(" gallery | {:5d} | {:8d} | {:9d}".format(num_gallery_pids, num_gallery_imgs, num_gallery_cams))
print(" ----------------------------------------")
class BaseVideoDataset(BaseDataset):
"""
Base class of video reid dataset
"""
def print_dataset_statistics(self, train, query, gallery):
num_train_pids, num_train_tracklets, num_train_cams, train_tracklet_stats = \
self.get_videodata_info(train, return_tracklet_stats=True)
num_query_pids, num_query_tracklets, num_query_cams, query_tracklet_stats = \
self.get_videodata_info(query, return_tracklet_stats=True)
num_gallery_pids, num_gallery_tracklets, num_gallery_cams, gallery_tracklet_stats = \
self.get_videodata_info(gallery, return_tracklet_stats=True)
tracklet_stats = train_tracklet_stats + query_tracklet_stats + gallery_tracklet_stats
min_num = np.min(tracklet_stats)
max_num = np.max(tracklet_stats)
avg_num = np.mean(tracklet_stats)
print("Dataset statistics:")
print(" -------------------------------------------")
print(" subset | # ids | # tracklets | # cameras")
print(" -------------------------------------------")
print(" train | {:5d} | {:11d} | {:9d}".format(num_train_pids, num_train_tracklets, num_train_cams))
print(" query | {:5d} | {:11d} | {:9d}".format(num_query_pids, num_query_tracklets, num_query_cams))
print(" gallery | {:5d} | {:11d} | {:9d}".format(num_gallery_pids, num_gallery_tracklets, num_gallery_cams))
print(" -------------------------------------------")
print(" number of images per tracklet: {} ~ {}, average {:.2f}".format(min_num, max_num, avg_num))
print(" -------------------------------------------")
================================================
FILE: domain_generalization/Transfer-Learning-Library/common/vision/datasets/reid/convert.py
================================================
"""
@author: Baixu Chen
@contact: cbx_99_hasta@outlook.com
"""
import os.path as osp
from torch.utils.data import Dataset
from PIL import Image
def convert_to_pytorch_dataset(dataset, root=None, transform=None, return_idxes=False):
class ReidDataset(Dataset):
def __init__(self, dataset, root, transform):
super(ReidDataset, self).__init__()
self.dataset = dataset
self.root = root
self.transform = transform
self.return_idxes = return_idxes
def __len__(self):
return len(self.dataset)
def __getitem__(self, index):
fname, pid, cid = self.dataset[index]
fpath = fname
if self.root is not None:
fpath = osp.join(self.root, fname)
img = Image.open(fpath).convert('RGB')
if self.transform is not None:
img = self.transform(img)
if not self.return_idxes:
return img, fname, pid, cid
else:
return img, fname, pid, cid, index
return ReidDataset(dataset, root, transform)
================================================
FILE: domain_generalization/Transfer-Learning-Library/common/vision/datasets/reid/dukemtmc.py
================================================
"""
@author: Baixu Chen
@contact: cbx_99_hasta@outlook.com
"""
from .basedataset import BaseImageDataset
from typing import Callable
from PIL import Image
import os
import os.path as osp
import glob
import re
from common.vision.datasets._util import download
class DukeMTMC(BaseImageDataset):
"""DukeMTMC-reID dataset from `Performance Measures and a Data Set for Multi-Target, Multi-Camera Tracking
(ECCV 2016) `_.
Dataset statistics:
- identities: 1404 (train + query)
- images:16522 (train) + 2228 (query) + 17661 (gallery)
- cameras: 8
Args:
root (str): Root directory of dataset
verbose (bool, optional): If true, print dataset statistics after loading the dataset. Default: True
"""
dataset_dir = '.'
archive_name = 'DukeMTMC-reID.tgz'
dataset_url = 'https://cloud.tsinghua.edu.cn/f/89f1edaf0f83434f8070/?dl=1'
def __init__(self, root, verbose=True):
super(DukeMTMC, self).__init__()
download(root, self.dataset_dir, self.archive_name, self.dataset_url)
self.relative_dataset_dir = self.dataset_dir
self.dataset_dir = osp.join(root, self.dataset_dir)
self.train_dir = osp.join(self.dataset_dir, 'DukeMTMC-reID/bounding_box_train')
self.query_dir = osp.join(self.dataset_dir, 'DukeMTMC-reID/query')
self.gallery_dir = osp.join(self.dataset_dir, 'DukeMTMC-reID/bounding_box_test')
required_files = [self.dataset_dir, self.train_dir, self.query_dir, self.gallery_dir]
self.check_before_run(required_files)
train = self.process_dir(self.train_dir, relabel=True)
query = self.process_dir(self.query_dir, relabel=False)
gallery = self.process_dir(self.gallery_dir, relabel=False)
if verbose:
print("=> DukeMTMC-reID loaded")
self.print_dataset_statistics(train, query, gallery)
self.train = train
self.query = query
self.gallery = gallery
self.num_train_pids, self.num_train_imgs, self.num_train_cams = self.get_imagedata_info(self.train)
self.num_query_pids, self.num_query_imgs, self.num_query_cams = self.get_imagedata_info(self.query)
self.num_gallery_pids, self.num_gallery_imgs, self.num_gallery_cams = self.get_imagedata_info(self.gallery)
def process_dir(self, dir_path, relabel=False):
img_paths = glob.glob(osp.join(dir_path, '*.jpg'))
pattern = re.compile(r'([-\d]+)_c(\d)')
pid_container = set()
for img_path in img_paths:
pid, _ = map(int, pattern.search(img_path).groups())
pid_container.add(pid)
pid2label = {pid: label for label, pid in enumerate(pid_container)}
dataset = []
for img_path in img_paths:
pid, cid = map(int, pattern.search(img_path).groups())
assert 1 <= cid <= 8
cid -= 1 # index starts from 0
if relabel:
pid = pid2label[pid]
dataset.append((img_path, pid, cid))
return dataset
def translate(self, transform: Callable, target_root: str):
""" Translate an image and save it into a specified directory
Args:
transform (callable): a transform function that maps images from one domain to another domain
target_root (str): the root directory to save images
"""
os.makedirs(target_root, exist_ok=True)
translated_dataset_dir = osp.join(target_root, self.relative_dataset_dir)
translated_train_dir = osp.join(translated_dataset_dir, 'DukeMTMC-reID/bounding_box_train')
translated_query_dir = osp.join(translated_dataset_dir, 'DukeMTMC-reID/query')
translated_gallery_dir = osp.join(translated_dataset_dir, 'DukeMTMC-reID/bounding_box_test')
print("Translating dataset with image to image transform...")
self.translate_dir(transform, self.train_dir, translated_train_dir)
self.translate_dir(None, self.query_dir, translated_query_dir)
self.translate_dir(None, self.gallery_dir, translated_gallery_dir)
print("Translation process is done, save dataset to {}".format(translated_dataset_dir))
def translate_dir(self, transform, origin_dir: str, target_dir: str):
image_list = os.listdir(origin_dir)
for image_name in image_list:
if not image_name.endswith(".jpg"):
continue
image_path = osp.join(origin_dir, image_name)
image = Image.open(image_path)
translated_image_path = osp.join(target_dir, image_name)
translated_image = image
if transform:
translated_image = transform(image)
os.makedirs(os.path.dirname(translated_image_path), exist_ok=True)
translated_image.save(translated_image_path)
================================================
FILE: domain_generalization/Transfer-Learning-Library/common/vision/datasets/reid/market1501.py
================================================
"""
@author: Baixu Chen
@contact: cbx_99_hasta@outlook.com
"""
from .basedataset import BaseImageDataset
from typing import Callable
from PIL import Image
import os
import os.path as osp
import glob
import re
from common.vision.datasets._util import download
class Market1501(BaseImageDataset):
"""Market1501 dataset from `Scalable Person Re-identification: A Benchmark (ICCV 2015)
`_.
Dataset statistics:
- identities: 1501 (+1 for background)
- images: 12936 (train) + 3368 (query) + 15913 (gallery)
- cameras: 6
Args:
root (str): Root directory of dataset
verbose (bool, optional): If true, print dataset statistics after loading the dataset. Default: True
"""
dataset_dir = 'Market-1501-v15.09.15'
archive_name = 'Market-1501-v15.09.15.tgz'
dataset_url = 'https://cloud.tsinghua.edu.cn/f/aad1f51c221f40c7b2a7/?dl=1'
def __init__(self, root, verbose=True):
super(Market1501, self).__init__()
download(root, self.dataset_dir, self.archive_name, self.dataset_url)
self.relative_dataset_dir = self.dataset_dir
self.dataset_dir = osp.join(root, self.dataset_dir)
self.train_dir = osp.join(self.dataset_dir, 'bounding_box_train')
self.query_dir = osp.join(self.dataset_dir, 'query')
self.gallery_dir = osp.join(self.dataset_dir, 'bounding_box_test')
required_files = [self.dataset_dir, self.train_dir, self.query_dir, self.gallery_dir]
self.check_before_run(required_files)
train = self.process_dir(self.train_dir, relabel=True)
query = self.process_dir(self.query_dir, relabel=False)
gallery = self.process_dir(self.gallery_dir, relabel=False)
if verbose:
print("=> Market1501 loaded")
self.print_dataset_statistics(train, query, gallery)
self.train = train
self.query = query
self.gallery = gallery
self.num_train_pids, self.num_train_imgs, self.num_train_cams = self.get_imagedata_info(self.train)
self.num_query_pids, self.num_query_imgs, self.num_query_cams = self.get_imagedata_info(self.query)
self.num_gallery_pids, self.num_gallery_imgs, self.num_gallery_cams = self.get_imagedata_info(self.gallery)
def process_dir(self, dir_path, relabel=False):
img_paths = glob.glob(osp.join(dir_path, '*.jpg'))
pattern = re.compile(r'([-\d]+)_c(\d)')
pid_container = set()
for img_path in img_paths:
pid, _ = map(int, pattern.search(img_path).groups())
if pid == -1:
continue # junk images are just ignored
pid_container.add(pid)
pid2label = {pid: label for label, pid in enumerate(pid_container)}
dataset = []
for img_path in img_paths:
pid, cid = map(int, pattern.search(img_path).groups())
if pid == -1:
continue # junk images are just ignored
assert 0 <= pid <= 1501 # pid == 0 means background
assert 1 <= cid <= 6
cid -= 1 # index starts from 0
if relabel:
pid = pid2label[pid]
dataset.append((img_path, pid, cid))
return dataset
def translate(self, transform: Callable, target_root: str):
""" Translate an image and save it into a specified directory
Args:
transform (callable): a transform function that maps images from one domain to another domain
target_root (str): the root directory to save images
"""
os.makedirs(target_root, exist_ok=True)
translated_dataset_dir = osp.join(target_root, self.relative_dataset_dir)
translated_train_dir = osp.join(translated_dataset_dir, 'bounding_box_train')
translated_query_dir = osp.join(translated_dataset_dir, 'query')
translated_gallery_dir = osp.join(translated_dataset_dir, 'bounding_box_test')
print("Translating dataset with image to image transform...")
self.translate_dir(transform, self.train_dir, translated_train_dir)
self.translate_dir(None, self.query_dir, translated_query_dir)
self.translate_dir(None, self.gallery_dir, translated_gallery_dir)
print("Translation process is done, save dataset to {}".format(translated_dataset_dir))
def translate_dir(self, transform, origin_dir: str, target_dir: str):
image_list = os.listdir(origin_dir)
for image_name in image_list:
if not image_name.endswith(".jpg"):
continue
image_path = osp.join(origin_dir, image_name)
image = Image.open(image_path)
translated_image_path = osp.join(target_dir, image_name)
translated_image = image
if transform:
translated_image = transform(image)
os.makedirs(os.path.dirname(translated_image_path), exist_ok=True)
translated_image.save(translated_image_path)
================================================
FILE: domain_generalization/Transfer-Learning-Library/common/vision/datasets/reid/msmt17.py
================================================
"""
@author: Baixu Chen
@contact: cbx_99_hasta@outlook.com
"""
from .basedataset import BaseImageDataset
from typing import Callable
from PIL import Image
import os
import os.path as osp
from common.vision.datasets._util import download
class MSMT17(BaseImageDataset):
"""MSMT17 dataset from `Person Transfer GAN to Bridge Domain Gap for Person Re-Identification (CVPR 2018)
`_.
Dataset statistics:
- identities: 4101
- images: 32621 (train) + 11659 (query) + 82161 (gallery)
- cameras: 15
Args:
root (str): Root directory of dataset
verbose (bool, optional): If true, print dataset statistics after loading the dataset. Default: True
"""
dataset_dir = '.'
archive_name = 'MSMT17_V1.zip'
dataset_url = 'https://cloud.tsinghua.edu.cn/f/55d7e5aa3c224f49b908/?dl=1'
def __init__(self, root, verbose=True):
super(MSMT17, self).__init__()
download(root, self.dataset_dir, self.archive_name, self.dataset_url)
self.relative_dataset_dir = self.dataset_dir
self.dataset_dir = osp.join(root, self.dataset_dir)
self.train_dir = osp.join(self.dataset_dir, 'bounding_box_train')
self.query_dir = osp.join(self.dataset_dir, 'query')
self.gallery_dir = osp.join(self.dataset_dir, 'bounding_box_test')
required_files = [self.dataset_dir, self.train_dir, self.query_dir, self.gallery_dir]
self.check_before_run(required_files)
self.train = self.process_dir(self.train_dir)
self.query = self.process_dir(self.query_dir)
self.gallery = self.process_dir(self.gallery_dir)
if verbose:
print("=> MSMT17 loaded")
self.print_dataset_statistics(self.train, self.query, self.gallery)
self.num_train_pids, self.num_train_imgs, self.num_train_cams = self.get_imagedata_info(self.train)
self.num_query_pids, self.num_query_imgs, self.num_query_cams = self.get_imagedata_info(self.query)
self.num_gallery_pids, self.num_gallery_imgs, self.num_gallery_cams = self.get_imagedata_info(self.gallery)
def process_dir(self, dir_path):
image_list = os.listdir(dir_path)
dataset = []
pid_container = set()
for image_path in image_list:
pid, cid, _ = image_path.split('_')
pid = int(pid)
cid = int(cid[1:]) - 1 # index starts from 0
full_image_path = osp.join(dir_path, image_path)
dataset.append((full_image_path, pid, cid))
pid_container.add(pid)
# check if pid starts from 0 and increments with 1
for idx, pid in enumerate(pid_container):
assert idx == pid, "See code comment for explanation"
return dataset
def translate(self, transform: Callable, target_root: str):
""" Translate an image and save it into a specified directory
Args:
transform (callable): a transform function that maps images from one domain to another domain
target_root (str): the root directory to save images
"""
os.makedirs(target_root, exist_ok=True)
translated_dataset_dir = osp.join(target_root, self.relative_dataset_dir)
translated_train_dir = osp.join(translated_dataset_dir, 'bounding_box_train')
translated_query_dir = osp.join(translated_dataset_dir, 'query')
translated_gallery_dir = osp.join(translated_dataset_dir, 'bounding_box_test')
print("Translating dataset with image to image transform...")
self.translate_dir(transform, self.train_dir, translated_train_dir)
self.translate_dir(None, self.query_dir, translated_query_dir)
self.translate_dir(None, self.gallery_dir, translated_gallery_dir)
print("Translation process is done, save dataset to {}".format(translated_dataset_dir))
def translate_dir(self, transform, origin_dir: str, target_dir: str):
image_list = os.listdir(origin_dir)
for image_name in image_list:
if not image_name.endswith(".jpg"):
continue
image_path = osp.join(origin_dir, image_name)
image = Image.open(image_path)
translated_image_path = osp.join(target_dir, image_name)
translated_image = image
if transform:
translated_image = transform(image)
os.makedirs(os.path.dirname(translated_image_path), exist_ok=True)
translated_image.save(translated_image_path)
================================================
FILE: domain_generalization/Transfer-Learning-Library/common/vision/datasets/reid/personx.py
================================================
"""
Modified from https://github.com/yxgeee/SpCL
@author: Baixu Chen
@contact: cbx_99_hasta@outlook.com
"""
from .basedataset import BaseImageDataset
from typing import Callable
from PIL import Image
import os
import os.path as osp
import glob
import re
from common.vision.datasets._util import download
class PersonX(BaseImageDataset):
"""PersonX dataset from `Dissecting Person Re-identification from the Viewpoint of Viewpoint (CVPR 2019)
`_.
Dataset statistics:
- identities: 1266
- images: 9840 (train) + 5136 (query) + 30816 (gallery)
- cameras: 6
Args:
root (str): Root directory of dataset
verbose (bool, optional): If true, print dataset statistics after loading the dataset. Default: True
"""
dataset_dir = '.'
archive_name = 'PersonX.zip'
dataset_url = 'https://cloud.tsinghua.edu.cn/f/f506cd11d6b646729bd1/?dl=1'
def __init__(self, root, verbose=True):
super(PersonX, self).__init__()
download(root, self.dataset_dir, self.archive_name, self.dataset_url)
self.relative_dataset_dir = self.dataset_dir
self.dataset_dir = osp.join(root, self.dataset_dir)
self.train_dir = osp.join(self.dataset_dir, 'bounding_box_train')
self.query_dir = osp.join(self.dataset_dir, 'query')
self.gallery_dir = osp.join(self.dataset_dir, 'bounding_box_test')
required_files = [self.dataset_dir, self.train_dir, self.query_dir, self.gallery_dir]
self.check_before_run(required_files)
train = self.process_dir(self.train_dir, relabel=True)
query = self.process_dir(self.query_dir, relabel=False)
gallery = self.process_dir(self.gallery_dir, relabel=False)
if verbose:
print("=> PersonX loaded")
self.print_dataset_statistics(train, query, gallery)
self.train = train
self.query = query
self.gallery = gallery
self.num_train_pids, self.num_train_imgs, self.num_train_cams = self.get_imagedata_info(self.train)
self.num_query_pids, self.num_query_imgs, self.num_query_cams = self.get_imagedata_info(self.query)
self.num_gallery_pids, self.num_gallery_imgs, self.num_gallery_cams = self.get_imagedata_info(self.gallery)
def process_dir(self, dir_path, relabel=False):
img_paths = glob.glob(osp.join(dir_path, '*.jpg'))
pattern = re.compile(r'([-\d]+)_c([-\d]+)')
cam2label = {3: 1, 4: 2, 8: 3, 10: 4, 11: 5, 12: 6}
pid_container = set()
for img_path in img_paths:
pid, _ = map(int, pattern.search(img_path).groups())
pid_container.add(pid)
pid2label = {pid: label for label, pid in enumerate(pid_container)}
dataset = []
for img_path in img_paths:
pid, cid = map(int, pattern.search(img_path).groups())
assert (cid in cam2label.keys())
cid = cam2label[cid]
cid -= 1 # index starts from 0
if relabel:
pid = pid2label[pid]
dataset.append((img_path, pid, cid))
return dataset
def translate(self, transform: Callable, target_root: str):
""" Translate an image and save it into a specified directory
Args:
transform (callable): a transform function that maps images from one domain to another domain
target_root (str): the root directory to save images
"""
os.makedirs(target_root, exist_ok=True)
translated_dataset_dir = osp.join(target_root, self.relative_dataset_dir)
translated_train_dir = osp.join(translated_dataset_dir, 'bounding_box_train')
translated_query_dir = osp.join(translated_dataset_dir, 'query')
translated_gallery_dir = osp.join(translated_dataset_dir, 'bounding_box_test')
print("Translating dataset with image to image transform...")
self.translate_dir(transform, self.train_dir, translated_train_dir)
self.translate_dir(None, self.query_dir, translated_query_dir)
self.translate_dir(None, self.gallery_dir, translated_gallery_dir)
print("Translation process is done, save dataset to {}".format(translated_dataset_dir))
def translate_dir(self, transform, origin_dir: str, target_dir: str):
image_list = os.listdir(origin_dir)
for image_name in image_list:
if not image_name.endswith(".jpg"):
continue
image_path = osp.join(origin_dir, image_name)
image = Image.open(image_path)
translated_image_path = osp.join(target_dir, image_name)
translated_image = image
if transform:
translated_image = transform(image)
os.makedirs(os.path.dirname(translated_image_path), exist_ok=True)
translated_image.save(translated_image_path)
================================================
FILE: domain_generalization/Transfer-Learning-Library/common/vision/datasets/reid/unreal.py
================================================
"""
Modified from https://github.com/SikaStar/IDM
@author: Baixu Chen
@contact: cbx_99_hasta@outlook.com
"""
from .basedataset import BaseImageDataset
from typing import Callable
import os.path as osp
from common.vision.datasets._util import download
class UnrealPerson(BaseImageDataset):
"""UnrealPerson dataset from `UnrealPerson: An Adaptive Pipeline towards Costless Person Re-identification
(CVPR 2021) `_.
Dataset statistics:
- identities: 3000
- images: 120,000
- cameras: 34
Args:
root (str): Root directory of dataset
verbose (bool, optional): If true, print dataset statistics after loading the dataset. Default: True
"""
dataset_dir = '.'
download_list = [
("list_unreal_train.txt", "image_list.zip", "https://cloud.tsinghua.edu.cn/f/a51b22fd760743e7bca6/?dl=1"),
("unreal_v1.1", "unreal_v1.1.tar", "https://cloud.tsinghua.edu.cn/f/a8806bb3bf1744dda5b1/?dl=1"),
("unreal_v1.2", "unreal_v1.2.tar", "https://cloud.tsinghua.edu.cn/f/449224485a654c5baa8f/?dl=1"),
("unreal_v1.3", "unreal_v1.3.tar", "https://cloud.tsinghua.edu.cn/f/069f3162f74849c09c10/?dl=1"),
("unreal_v2.1", "unreal_v2.1.tar", "https://cloud.tsinghua.edu.cn/f/a791aaa42674466eb183/?dl=1"),
("unreal_v2.2", "unreal_v2.2.tar", "https://cloud.tsinghua.edu.cn/f/b601d9f54f964248bd0e/?dl=1"),
("unreal_v2.3", "unreal_v2.3.tar", "https://cloud.tsinghua.edu.cn/f/311ec60e810b42d48d12/?dl=1"),
("unreal_v3.1", "unreal_v3.1.tar", "https://cloud.tsinghua.edu.cn/f/d51b7c1d125e4632bcf9/?dl=1"),
("unreal_v3.2", "unreal_v3.2.tar", "https://cloud.tsinghua.edu.cn/f/4efbd969ea2f4e8197e8/?dl=1"),
("unreal_v3.3", "unreal_v3.3.tar", "https://cloud.tsinghua.edu.cn/f/a3cc3d9c460247848fb7/?dl=1"),
("unreal_v4.1", "unreal_v4.1.tar", "https://cloud.tsinghua.edu.cn/f/ca05183ac9cd4be5a53b/?dl=1"),
("unreal_v4.2", "unreal_v4.2.tar", "https://cloud.tsinghua.edu.cn/f/b90722cbd754496f9f40/?dl=1"),
("unreal_v4.3", "unreal_v4.3.tar", "https://cloud.tsinghua.edu.cn/f/547ae646c3d346038297/?dl=1"),
]
def __init__(self, root, verbose=True):
super(UnrealPerson, self).__init__()
list(map(lambda args: download(root, *args), self.download_list))
self.dataset_dir = osp.join(root, self.dataset_dir)
self.train_list = osp.join(self.dataset_dir, 'list_unreal_train.txt')
required_files = [self.dataset_dir, self.train_list]
self.check_before_run(required_files)
train = self.process_dir(self.train_list)
self.train = train
self.query = []
self.gallery = []
self.num_train_pids, self.num_train_imgs, self.num_train_cams = self.get_imagedata_info(self.train)
if verbose:
print("=> UnrealPerson loaded")
print(" ----------------------------------------")
print(" subset | # ids | # cams | # images")
print(" ----------------------------------------")
print(" train | {:5d} | {:5d} | {:8d}"
.format(self.num_train_pids, self.num_train_cams, self.num_train_imgs))
print(" ----------------------------------------")
def process_dir(self, list_file):
with open(list_file, 'r') as f:
lines = f.readlines()
dataset = []
pid_container = set()
for line in lines:
line = line.strip()
pid = line.split(' ')[1]
pid_container.add(pid)
pid2label = {pid: label for label, pid in enumerate(sorted(pid_container))}
for line in lines:
line = line.strip()
fname, pid, cid = line.split(' ')[0], line.split(' ')[1], int(line.split(' ')[2])
img_path = osp.join(self.dataset_dir, fname)
dataset.append((img_path, pid2label[pid], cid))
return dataset
def translate(self, transform: Callable, target_root: str):
raise NotImplementedError
================================================
FILE: domain_generalization/Transfer-Learning-Library/common/vision/datasets/resisc45.py
================================================
"""
@author: Junguang Jiang
@contact: JiangJunguang1123@outlook.com
"""
from torchvision.datasets.folder import ImageFolder
import random
class Resisc45(ImageFolder):
"""`Resisc45 `_ dataset \
is a scene classification task from remote sensing images. There are 45 classes, \
containing 700 images each, including tennis court, ship, island, lake, \
parking lot, sparse residential, or stadium. \
The image size is RGB 256x256 pixels.
.. note:: You need to download the source data manually into `root` directory.
Args:
root (str): Root directory of dataset
split (str, optional): The dataset split, supports ``train``, or ``test``.
transform (callable, optional): A function/transform that takes in an PIL image and returns a \
transformed version. E.g, :class:`torchvision.transforms.RandomCrop`.
target_transform (callable, optional): A function/transform that takes in the target and transforms it.
"""
def __init__(self, root, split='train', download=False, **kwargs):
super(Resisc45, self).__init__(root, **kwargs)
random.seed(0)
random.shuffle(self.samples)
if split == 'train':
self.samples = self.samples[:25200]
else:
self.samples = self.samples[25200:]
@property
def num_classes(self) -> int:
"""Number of classes"""
return len(self.classes)
================================================
FILE: domain_generalization/Transfer-Learning-Library/common/vision/datasets/retinopathy.py
================================================
"""
@author: Junguang Jiang
@contact: JiangJunguang1123@outlook.com
"""
import os
from common.vision.datasets import ImageList
class Retinopathy(ImageList):
"""`Retinopathy `_ dataset \
consists of image-label pairs with high-resolution retina images, and labels that indicate \
the presence of Diabetic Retinopahy (DR) in a 0-4 scale (No DR, Mild, Moderate, Severe, \
or Proliferative DR).
.. note:: You need to download the source data manually into `root` directory.
Args:
root (str): Root directory of dataset
split (str, optional): The dataset split, supports ``train``, or ``test``.
transform (callable, optional): A function/transform that takes in an PIL image and returns a \
transformed version. E.g, :class:`torchvision.transforms.RandomCrop`.
target_transform (callable, optional): A function/transform that takes in the target and transforms it.
"""
CLASSES = ['No DR', 'Mild', 'Moderate', 'Severe', 'Proliferative DR']
def __init__(self, root, split, download=False, **kwargs):
super(Retinopathy, self).__init__(os.path.join(root, split), Retinopathy.CLASSES, os.path.join(root, "image_list", "{}.txt".format(split)), **kwargs)
================================================
FILE: domain_generalization/Transfer-Learning-Library/common/vision/datasets/segmentation/__init__.py
================================================
from .segmentation_list import SegmentationList
from .cityscapes import Cityscapes, FoggyCityscapes
from .gta5 import GTA5
from .synthia import Synthia
__all__ = ["SegmentationList", "Cityscapes", "GTA5", "Synthia", "FoggyCityscapes"]
================================================
FILE: domain_generalization/Transfer-Learning-Library/common/vision/datasets/segmentation/cityscapes.py
================================================
"""
@author: Junguang Jiang
@contact: JiangJunguang1123@outlook.com
"""
import os
from .segmentation_list import SegmentationList
from .._util import download as download_data
class Cityscapes(SegmentationList):
"""`Cityscapes `_ is a real-world semantic segmentation dataset collected
in driving scenarios.
Args:
root (str): Root directory of dataset
split (str, optional): The dataset split, supports ``train``, or ``val``.
data_folder (str, optional): Sub-directory of the image. Default: 'leftImg8bit'.
label_folder (str, optional): Sub-directory of the label. Default: 'gtFine'.
mean (seq[float]): mean BGR value. Normalize the image if not None. Default: None.
transforms (callable, optional): A function/transform that takes in (PIL image, label) pair \
and returns a transformed version. E.g, :class:`~common.vision.transforms.segmentation.Resize`.
.. note:: You need to download Cityscapes manually.
Ensure that there exist following files in the `root` directory before you using this class.
::
leftImg8bit/
train/
val/
test/
gtFine/
train/
val/
test/
"""
CLASSES = ['road', 'sidewalk', 'building', 'wall', 'fence', 'pole', 'traffic light', 'traffic sign',
'vegetation', 'terrain', 'sky', 'person', 'rider', 'car', 'truck', 'bus', 'train', 'motorcycle',
'bicycle']
ID_TO_TRAIN_ID = {
7: 0, 8: 1, 11: 2, 12: 3, 13: 4, 17: 5,
19: 6, 20: 7, 21: 8, 22: 9, 23: 10, 24: 11, 25: 12,
26: 13, 27: 14, 28: 15, 31: 16, 32: 17, 33: 18
}
TRAIN_ID_TO_COLOR = [(128, 64, 128), (244, 35, 232), (70, 70, 70), (102, 102, 156),
(190, 153, 153), (153, 153, 153), (250, 170, 30), (220, 220, 0),
(107, 142, 35), (152, 251, 152), (70, 130, 180), (220, 20, 60),
(255, 0, 0), (0, 0, 142), (0, 0, 70), (0, 60, 100), (0, 80, 100),
(0, 0, 230), (119, 11, 32), [0, 0, 0]]
download_list = [
("image_list", "image_list.zip", "https://cloud.tsinghua.edu.cn/f/08745e798b16483db4bf/?dl=1"),
]
EVALUATE_CLASSES = CLASSES
def __init__(self, root, split='train', data_folder='leftImg8bit', label_folder='gtFine', **kwargs):
assert split in ['train', 'val']
# download meta information from Internet
list(map(lambda args: download_data(root, *args), self.download_list))
data_list_file = os.path.join(root, "image_list", "{}.txt".format(split))
self.split = split
super(Cityscapes, self).__init__(root, Cityscapes.CLASSES, data_list_file, data_list_file,
os.path.join(data_folder, split), os.path.join(label_folder, split),
id_to_train_id=Cityscapes.ID_TO_TRAIN_ID,
train_id_to_color=Cityscapes.TRAIN_ID_TO_COLOR, **kwargs)
def parse_label_file(self, label_list_file):
with open(label_list_file, "r") as f:
label_list = [line.strip().replace("leftImg8bit", "gtFine_labelIds") for line in f.readlines()]
return label_list
class FoggyCityscapes(Cityscapes):
"""`Foggy Cityscapes `_ is a real-world semantic segmentation dataset collected
in foggy driving scenarios.
Args:
root (str): Root directory of dataset
split (str, optional): The dataset split, supports ``train``, or ``val``.
data_folder (str, optional): Sub-directory of the image. Default: 'leftImg8bit'.
label_folder (str, optional): Sub-directory of the label. Default: 'gtFine'.
beta (float, optional): The parameter for foggy. Choices includes: 0.005, 0.01, 0.02. Default: 0.02
mean (seq[float]): mean BGR value. Normalize the image if not None. Default: None.
transforms (callable, optional): A function/transform that takes in (PIL image, label) pair \
and returns a transformed version. E.g, :class:`~common.vision.transforms.segmentation.Resize`.
.. note:: You need to download Cityscapes manually.
Ensure that there exist following files in the `root` directory before you using this class.
::
leftImg8bit_foggy/
train/
val/
test/
gtFine/
train/
val/
test/
"""
def __init__(self, root, split='train', data_folder='leftImg8bit_foggy', label_folder='gtFine', beta=0.02, **kwargs):
assert beta in [0.02, 0.01, 0.005]
self.beta = beta
super(FoggyCityscapes, self).__init__(root, split, data_folder, label_folder, **kwargs)
def parse_data_file(self, file_name):
"""Parse file to image list
Args:
file_name (str): The path of data file
Returns:
List of image path
"""
with open(file_name, "r") as f:
data_list = [line.strip().replace("leftImg8bit", "leftImg8bit_foggy_beta_{}".format(self.beta)) for line in f.readlines()]
return data_list
================================================
FILE: domain_generalization/Transfer-Learning-Library/common/vision/datasets/segmentation/gta5.py
================================================
"""
@author: Junguang Jiang
@contact: JiangJunguang1123@outlook.com
"""
import os
from .segmentation_list import SegmentationList
from .cityscapes import Cityscapes
from .._util import download as download_data
class GTA5(SegmentationList):
"""`GTA5 `_
Args:
root (str): Root directory of dataset
split (str, optional): The dataset split, supports ``train``.
data_folder (str, optional): Sub-directory of the image. Default: 'images'.
label_folder (str, optional): Sub-directory of the label. Default: 'labels'.
mean (seq[float]): mean BGR value. Normalize the image if not None. Default: None.
transforms (callable, optional): A function/transform that takes in (PIL image, label) pair \
and returns a transformed version. E.g, :class:`~common.vision.transforms.segmentation.Resize`.
.. note:: You need to download GTA5 manually.
Ensure that there exist following directories in the `root` directory before you using this class.
::
images/
labels/
"""
download_list = [
("image_list", "image_list.zip", "https://cloud.tsinghua.edu.cn/f/c77ff6fc4eea435791f4/?dl=1"),
]
def __init__(self, root, split='train', data_folder='images', label_folder='labels', **kwargs):
assert split in ['train']
# download meta information from Internet
list(map(lambda args: download_data(root, *args), self.download_list))
data_list_file = os.path.join(root, "image_list", "{}.txt".format(split))
self.split = split
super(GTA5, self).__init__(root, Cityscapes.CLASSES, data_list_file, data_list_file, data_folder, label_folder,
id_to_train_id=Cityscapes.ID_TO_TRAIN_ID, train_id_to_color=Cityscapes.TRAIN_ID_TO_COLOR, **kwargs)
================================================
FILE: domain_generalization/Transfer-Learning-Library/common/vision/datasets/segmentation/segmentation_list.py
================================================
"""
@author: Junguang Jiang
@contact: JiangJunguang1123@outlook.com
"""
import os
from typing import Sequence, Optional, Dict, Callable
from PIL import Image
import tqdm
import numpy as np
from torch.utils import data
import torch
class SegmentationList(data.Dataset):
"""A generic Dataset class for domain adaptation in image segmentation
Args:
root (str): Root directory of dataset
classes (seq[str]): The names of all the classes
data_list_file (str): File to read the image list from.
label_list_file (str): File to read the label list from.
data_folder (str): Sub-directory of the image.
label_folder (str): Sub-directory of the label.
mean (seq[float]): mean BGR value. Normalize and convert to the image if not None. Default: None.
id_to_train_id (dict, optional): the map between the id on the label and the actual train id.
train_id_to_color (seq, optional): the map between the train id and the color.
transforms (callable, optional): A function/transform that takes in (PIL Image, label) pair \
and returns a transformed version. E.g, :class:`~common.vision.transforms.segmentation.Resize`.
.. note:: In ``data_list_file``, each line is the relative path of an image.
If your data_list_file has different formats, please over-ride :meth:`~SegmentationList.parse_data_file`.
::
source_dir/dog_xxx.png
target_dir/dog_xxy.png
In ``label_list_file``, each line is the relative path of an label.
If your label_list_file has different formats, please over-ride :meth:`~SegmentationList.parse_label_file`.
.. warning:: When mean is not None, please do not provide Normalize and ToTensor in transforms.
"""
def __init__(self, root: str, classes: Sequence[str], data_list_file: str, label_list_file: str,
data_folder: str, label_folder: str,
id_to_train_id: Optional[Dict] = None, train_id_to_color: Optional[Sequence] = None,
transforms: Optional[Callable] = None):
self.root = root
self.classes = classes
self.data_list_file = data_list_file
self.label_list_file = label_list_file
self.data_folder = data_folder
self.label_folder = label_folder
self.ignore_label = 255
self.id_to_train_id = id_to_train_id
self.train_id_to_color = np.array(train_id_to_color)
self.data_list = self.parse_data_file(self.data_list_file)
self.label_list = self.parse_label_file(self.label_list_file)
self.transforms = transforms
def parse_data_file(self, file_name):
"""Parse file to image list
Args:
file_name (str): The path of data file
Returns:
List of image path
"""
with open(file_name, "r") as f:
data_list = [line.strip() for line in f.readlines()]
return data_list
def parse_label_file(self, file_name):
"""Parse file to label list
Args:
file_name (str): The path of data file
Returns:
List of label path
"""
with open(file_name, "r") as f:
label_list = [line.strip() for line in f.readlines()]
return label_list
def __len__(self):
return len(self.data_list)
def __getitem__(self, index):
image_name = self.data_list[index]
label_name = self.label_list[index]
image = Image.open(os.path.join(self.root, self.data_folder, image_name)).convert('RGB')
label = Image.open(os.path.join(self.root, self.label_folder, label_name))
image, label = self.transforms(image, label)
# remap label
if isinstance(label, torch.Tensor):
label = label.numpy()
label = np.asarray(label, np.int64)
label_copy = self.ignore_label * np.ones(label.shape, dtype=np.int64)
if self.id_to_train_id:
for k, v in self.id_to_train_id.items():
label_copy[label == k] = v
return image, label_copy.copy()
@property
def num_classes(self) -> int:
"""Number of classes"""
return len(self.classes)
def decode_target(self, target):
""" Decode label (each value is integer) into the corresponding RGB value.
Args:
target (numpy.array): label in shape H x W
Returns:
RGB label (PIL Image) in shape H x W x 3
"""
target = target.copy()
target[target == 255] = self.num_classes # unknown label is black on the RGB label
target = self.train_id_to_color[target]
return Image.fromarray(target.astype(np.uint8))
def collect_image_paths(self):
"""Return a list of the absolute path of all the images"""
return [os.path.join(self.root, self.data_folder, image_name) for image_name in self.data_list]
@staticmethod
def _save_pil_image(image, path):
os.makedirs(os.path.dirname(path), exist_ok=True)
image.save(path)
def translate(self, transform: Callable, target_root: str, color=False):
""" Translate an image and save it into a specified directory
Args:
transform (callable): a transform function that maps (image, label) pair from one domain to another domain
target_root (str): the root directory to save images and labels
"""
os.makedirs(target_root, exist_ok=True)
for image_name, label_name in zip(tqdm.tqdm(self.data_list), self.label_list):
image_path = os.path.join(target_root, self.data_folder, image_name)
label_path = os.path.join(target_root, self.label_folder, label_name)
if os.path.exists(image_path) and os.path.exists(label_path):
continue
image = Image.open(os.path.join(self.root, self.data_folder, image_name)).convert('RGB')
label = Image.open(os.path.join(self.root, self.label_folder, label_name))
translated_image, translated_label = transform(image, label)
self._save_pil_image(translated_image, image_path)
self._save_pil_image(translated_label, label_path)
if color:
colored_label = self.decode_target(np.array(translated_label))
file_name, file_ext = os.path.splitext(label_name)
self._save_pil_image(colored_label, os.path.join(target_root, self.label_folder,
"{}_color{}".format(file_name, file_ext)))
@property
def evaluate_classes(self):
"""The name of classes to be evaluated"""
return self.classes
@property
def ignore_classes(self):
"""The name of classes to be ignored"""
return list(set(self.classes) - set(self.evaluate_classes))
================================================
FILE: domain_generalization/Transfer-Learning-Library/common/vision/datasets/segmentation/synthia.py
================================================
"""
@author: Junguang Jiang
@contact: JiangJunguang1123@outlook.com
"""
import os
from .segmentation_list import SegmentationList
from .cityscapes import Cityscapes
from .._util import download as download_data
class Synthia(SegmentationList):
"""`SYNTHIA `_
Args:
root (str): Root directory of dataset
split (str, optional): The dataset split, supports ``train``.
data_folder (str, optional): Sub-directory of the image. Default: 'RGB'.
label_folder (str, optional): Sub-directory of the label. Default: 'synthia_mapped_to_cityscapes'.
mean (seq[float]): mean BGR value. Normalize the image if not None. Default: None.
transforms (callable, optional): A function/transform that takes in (PIL image, label) pair \
and returns a transformed version. E.g, :class:`~common.vision.transforms.segmentation.Resize`.
.. note:: You need to download GTA5 manually.
Ensure that there exist following directories in the `root` directory before you using this class.
::
RGB/
synthia_mapped_to_cityscapes/
"""
ID_TO_TRAIN_ID = {
3: 0, 4: 1, 2: 2, 21: 3, 5: 4, 7: 5,
15: 6, 9: 7, 6: 8, 16: 9, 1: 10, 10: 11, 17: 12,
8: 13, 18: 14, 19: 15, 20: 16, 12: 17, 11: 18
}
download_list = [
("image_list", "image_list.zip", "https://cloud.tsinghua.edu.cn/f/15c4d0f8e62e45d9a6b7/?dl=1"),
]
def __init__(self, root, split='train', data_folder='RGB', label_folder='synthia_mapped_to_cityscapes', **kwargs):
assert split in ['train']
# download meta information from Internet
list(map(lambda args: download_data(root, *args), self.download_list))
data_list_file = os.path.join(root, "image_list", "{}.txt".format(split))
super(Synthia, self).__init__(root, Cityscapes.CLASSES, data_list_file, data_list_file, data_folder,
label_folder, id_to_train_id=Synthia.ID_TO_TRAIN_ID,
train_id_to_color=Cityscapes.TRAIN_ID_TO_COLOR, **kwargs)
@property
def evaluate_classes(self):
return [
'road', 'sidewalk', 'building', 'traffic light', 'traffic sign',
'vegetation', 'sky', 'person', 'rider', 'car', 'bus', 'motorcycle', 'bicycle'
]
================================================
FILE: domain_generalization/Transfer-Learning-Library/common/vision/datasets/stanford_cars.py
================================================
"""
@author: Yifei Ji
@contact: jiyf990330@163.com
"""
import os
from typing import Optional
from .imagelist import ImageList
from ._util import download as download_data, check_exits
class StanfordCars(ImageList):
"""`The Stanford Cars `_ \
contains 16,185 images of 196 classes of cars. \
Each category has been split roughly in a 50-50 split. \
There are 8,144 images for training and 8,041 images for testing.
Args:
root (str): Root directory of dataset
split (str, optional): The dataset split, supports ``train``, or ``test``.
sample_rate (int): The sampling rates to sample random ``training`` images for each category.
Choices include 100, 50, 30, 15. Default: 100.
download (bool, optional): If true, downloads the dataset from the internet and puts it \
in root directory. If dataset is already downloaded, it is not downloaded again.
transform (callable, optional): A function/transform that takes in an PIL image and returns a \
transformed version. E.g, :class:`torchvision.transforms.RandomCrop`.
target_transform (callable, optional): A function/transform that takes in the target and transforms it.
.. note:: In `root`, there will exist following files after downloading.
::
train/
test/
image_list/
train_100.txt
train_50.txt
train_30.txt
train_15.txt
test.txt
"""
download_list = [
("image_list", "image_list.zip", "https://cloud.tsinghua.edu.cn/f/d95c188cc49c404aba70/?dl=1"),
("train", "train.tgz", "https://cloud.tsinghua.edu.cn/f/d5ab63c391a949509db0/?dl=1"),
("test", "test.tgz", "https://cloud.tsinghua.edu.cn/f/04e6fd5222a84d0a8ff5/?dl=1"),
]
image_list = {
"train": "image_list/train_100.txt",
"train100": "image_list/train_100.txt",
"train50": "image_list/train_50.txt",
"train30": "image_list/train_30.txt",
"train15": "image_list/train_15.txt",
"test": "image_list/test.txt",
"test100": "image_list/test.txt",
}
CLASSES = ['1', '2', '3', '4', '5', '6', '7', '8', '9', '10', '11', '12', '13', '14', '15', '16', '17', '18', '19', '20',
'21', '22', '23', '24', '25', '26', '27', '28', '29', '30', '31', '32', '33', '34', '35', '36', '37', '38', '39', '40',
'41', '42', '43', '44', '45', '46', '47', '48', '49', '50', '51', '52', '53', '54', '55', '56', '57', '58', '59', '60',
'61', '62', '63', '64', '65', '66', '67', '68', '69', '70', '71', '72', '73', '74', '75', '76', '77', '78', '79', '80',
'81', '82', '83', '84', '85', '86', '87', '88', '89', '90', '91', '92', '93', '94', '95', '96', '97', '98', '99', '100',
'101', '102', '103', '104', '105', '106', '107', '108', '109', '110', '111', '112', '113', '114', '115', '116', '117', '118', '119', '120',
'121', '122', '123', '124', '125', '126', '127', '128', '129', '130', '131', '132', '133', '134', '135', '136', '137', '138', '139', '140',
'141', '142', '143', '144', '145', '146', '147', '148', '149', '150', '151', '152', '153', '154', '155', '156', '157', '158', '159', '160',
'161', '162', '163', '164', '165', '166', '167', '168', '169', '170', '171', '172', '173', '174', '175', '176', '177', '178', '179', '180',
'181', '182', '183', '184', '185', '186', '187', '188', '189', '190', '191', '192', '193', '194', '195', '196']
def __init__(self, root: str, split: str, sample_rate: Optional[int] =100, download: Optional[bool] = False, **kwargs):
if split == 'train':
list_name = 'train' + str(sample_rate)
assert list_name in self.image_list
data_list_file = os.path.join(root, self.image_list[list_name])
else:
data_list_file = os.path.join(root, self.image_list['test'])
if download:
list(map(lambda args: download_data(root, *args), self.download_list))
else:
list(map(lambda file_name, _: check_exits(root, file_name), self.download_list))
super(StanfordCars, self).__init__(root, StanfordCars.CLASSES, data_list_file=data_list_file, **kwargs)
================================================
FILE: domain_generalization/Transfer-Learning-Library/common/vision/datasets/stanford_dogs.py
================================================
"""
@author: Yifei Ji
@contact: jiyf990330@163.com
"""
import os
from typing import Optional
from .imagelist import ImageList
from ._util import download as download_data, check_exits
class StanfordDogs(ImageList):
"""`The Stanford Dogs `_ \
contains 20,580 images of 120 breeds of dogs from around the world. \
Each category is composed of exactly 100 training examples and around 72 testing examples.
Args:
root (str): Root directory of dataset
split (str, optional): The dataset split, supports ``train``, or ``test``.
sample_rate (int): The sampling rates to sample random ``training`` images for each category.
Choices include 100, 50, 30, 15. Default: 100.
download (bool, optional): If true, downloads the dataset from the internet and puts it \
in root directory. If dataset is already downloaded, it is not downloaded again.
transform (callable, optional): A function/transform that takes in an PIL image and returns a \
transformed version. E.g, :class:`torchvision.transforms.RandomCrop`.
target_transform (callable, optional): A function/transform that takes in the target and transforms it.
.. note:: In `root`, there will exist following files after downloading.
::
train/
test/
image_list/
train_100.txt
train_50.txt
train_30.txt
train_15.txt
test.txt
"""
download_list = [
("image_list", "image_list.zip", "https://cloud.tsinghua.edu.cn/f/b7b3dd297ec54e038238/?dl=1"),
("train", "train.tgz", "https://cloud.tsinghua.edu.cn/f/cd09d9ca88f044bfa5d3/?dl=1"),
("test", "test.tgz", "https://cloud.tsinghua.edu.cn/f/64a67e97069948c7b2c9/?dl=1"),
]
image_list = {
"train": "image_list/train_100.txt",
"train100": "image_list/train_100.txt",
"train50": "image_list/train_50.txt",
"train30": "image_list/train_30.txt",
"train15": "image_list/train_15.txt",
"test": "image_list/test.txt",
"test100": "image_list/test.txt",
}
CLASSES = ['n02085620-Chihuahua', 'n02085782-Japanese_spaniel', 'n02085936-Maltese_dog', 'n02086079-Pekinese', 'n02086240-Shih-Tzu',
'n02086646-Blenheim_spaniel', 'n02086910-papillon', 'n02087046-toy_terrier', 'n02087394-Rhodesian_ridgeback',
'n02088094-Afghan_hound', 'n02088238-basset', 'n02088364-beagle', 'n02088466-bloodhound', 'n02088632-bluetick', 'n02089078-black-and-tan_coonhound',
'n02089867-Walker_hound', 'n02089973-English_foxhound', 'n02090379-redbone', 'n02090622-borzoi', 'n02090721-Irish_wolfhound', 'n02091032-Italian_greyhound',
'n02091134-whippet', 'n02091244-Ibizan_hound', 'n02091467-Norwegian_elkhound', 'n02091635-otterhound', 'n02091831-Saluki', 'n02092002-Scottish_deerhound',
'n02092339-Weimaraner', 'n02093256-Staffordshire_bullterrier', 'n02093428-American_Staffordshire_terrier', 'n02093647-Bedlington_terrier', 'n02093754-Border_terrier',
'n02093859-Kerry_blue_terrier', 'n02093991-Irish_terrier', 'n02094114-Norfolk_terrier', 'n02094258-Norwich_terrier', 'n02094433-Yorkshire_terrier',
'n02095314-wire-haired_fox_terrier', 'n02095570-Lakeland_terrier', 'n02095889-Sealyham_terrier', 'n02096051-Airedale', 'n02096177-cairn', 'n02096294-Australian_terrier',
'n02096437-Dandie_Dinmont', 'n02096585-Boston_bull', 'n02097047-miniature_schnauzer', 'n02097130-giant_schnauzer', 'n02097209-standard_schnauzer',
'n02097298-Scotch_terrier', 'n02097474-Tibetan_terrier', 'n02097658-silky_terrier', 'n02098105-soft-coated_wheaten_terrier', 'n02098286-West_Highland_white_terrier',
'n02098413-Lhasa', 'n02099267-flat-coated_retriever', 'n02099429-curly-coated_retriever', 'n02099601-golden_retriever', 'n02099712-Labrador_retriever',
'n02099849-Chesapeake_Bay_retriever', 'n02100236-German_short-haired_pointer', 'n02100583-vizsla', 'n02100735-English_setter', 'n02100877-Irish_setter',
'n02101006-Gordon_setter', 'n02101388-Brittany_spaniel', 'n02101556-clumber', 'n02102040-English_springer', 'n02102177-Welsh_springer_spaniel', 'n02102318-cocker_spaniel',
'n02102480-Sussex_spaniel', 'n02102973-Irish_water_spaniel', 'n02104029-kuvasz', 'n02104365-schipperke', 'n02105056-groenendael', 'n02105162-malinois', 'n02105251-briard', 'n02105412-kelpie',
'n02105505-komondor', 'n02105641-Old_English_sheepdog', 'n02105855-Shetland_sheepdog', 'n02106030-collie', 'n02106166-Border_collie', 'n02106382-Bouvier_des_Flandres', 'n02106550-Rottweiler',
'n02106662-German_shepherd', 'n02107142-Doberman', 'n02107312-miniature_pinscher', 'n02107574-Greater_Swiss_Mountain_dog',
'n02107683-Bernese_mountain_dog', 'n02107908-Appenzeller', 'n02108000-EntleBucher', 'n02108089-boxer', 'n02108422-bull_mastiff', 'n02108551-Tibetan_mastiff',
'n02108915-French_bulldog', 'n02109047-Great_Dane', 'n02109525-Saint_Bernard', 'n02109961-Eskimo_dog', 'n02110063-malamute', 'n02110185-Siberian_husky',
'n02110627-affenpinscher', 'n02110806-basenji', 'n02110958-pug', 'n02111129-Leonberg', 'n02111277-Newfoundland', 'n02111500-Great_Pyrenees', 'n02111889-Samoyed', 'n02112018-Pomeranian',
'n02112137-chow', 'n02112350-keeshond', 'n02112706-Brabancon_griffon', 'n02113023-Pembroke', 'n02113186-Cardigan',
'n02113624-toy_poodle', 'n02113712-miniature_poodle', 'n02113799-standard_poodle', 'n02113978-Mexican_hairless', 'n02115641-dingo', 'n02115913-dhole', 'n02116738-African_hunting_dog']
def __init__(self, root: str, split: str, sample_rate: Optional[int] =100, download: Optional[bool] = False, **kwargs):
if split == 'train':
list_name = 'train' + str(sample_rate)
assert list_name in self.image_list
data_list_file = os.path.join(root, self.image_list[list_name])
else:
data_list_file = os.path.join(root, self.image_list['test'])
if download:
list(map(lambda args: download_data(root, *args), self.download_list))
else:
list(map(lambda file_name, _: check_exits(root, file_name), self.download_list))
super(StanfordDogs, self).__init__(root, StanfordDogs.CLASSES, data_list_file=data_list_file, **kwargs)
================================================
FILE: domain_generalization/Transfer-Learning-Library/common/vision/datasets/visda2017.py
================================================
"""
@author: Junguang Jiang
@contact: JiangJunguang1123@outlook.com
"""
import os
from typing import Optional
from .imagelist import ImageList
from ._util import download as download_data, check_exits
class VisDA2017(ImageList):
"""`VisDA-2017 `_ Dataset
Args:
root (str): Root directory of dataset
task (str): The task (domain) to create dataset. Choices include ``'Synthetic'``: synthetic images and \
``'Real'``: real-world images.
download (bool, optional): If true, downloads the dataset from the internet and puts it \
in root directory. If dataset is already downloaded, it is not downloaded again.
transform (callable, optional): A function/transform that takes in an PIL image and returns a \
transformed version. E.g, ``transforms.RandomCrop``.
target_transform (callable, optional): A function/transform that takes in the target and transforms it.
.. note:: In `root`, there will exist following files after downloading.
::
train/
aeroplance/
*.png
...
validation/
image_list/
train.txt
validation.txt
"""
download_list = [
("image_list", "image_list.zip", "https://cloud.tsinghua.edu.cn/f/b25b2b990e8f42e691f0/?dl=1"),
("train", "train.tar", "http://csr.bu.edu/ftp/visda17/clf/train.tar"),
("validation", "validation.tar", "http://csr.bu.edu/ftp/visda17/clf/validation.tar")
]
image_list = {
"Synthetic": "image_list/train.txt",
"Real": "image_list/validation.txt"
}
CLASSES = ['aeroplane', 'bicycle', 'bus', 'car', 'horse', 'knife',
'motorcycle', 'person', 'plant', 'skateboard', 'train', 'truck']
def __init__(self, root: str, task: str, download: Optional[bool] = False, **kwargs):
assert task in self.image_list
data_list_file = os.path.join(root, self.image_list[task])
if download:
list(map(lambda args: download_data(root, *args), self.download_list))
else:
list(map(lambda file_name, _: check_exits(root, file_name), self.download_list))
super(VisDA2017, self).__init__(root, VisDA2017.CLASSES, data_list_file=data_list_file, **kwargs)
@classmethod
def domains(cls):
return list(cls.image_list.keys())
================================================
FILE: domain_generalization/Transfer-Learning-Library/common/vision/models/__init__.py
================================================
from .resnet import *
from .ibn import *
from .digits import *
__all__ = ['resnet', 'digits', 'ibn']
================================================
FILE: domain_generalization/Transfer-Learning-Library/common/vision/models/digits.py
================================================
"""
@author: Junguang Jiang
@contact: JiangJunguang1123@outlook.com
"""
import torch.nn as nn
class LeNet(nn.Sequential):
def __init__(self, num_classes=10):
super(LeNet, self).__init__(
nn.Conv2d(1, 20, kernel_size=5),
nn.MaxPool2d(2),
nn.ReLU(),
nn.Conv2d(20, 50, kernel_size=5),
nn.Dropout2d(p=0.5),
nn.MaxPool2d(2),
nn.ReLU(),
nn.Flatten(start_dim=1),
nn.Linear(50 * 4 * 4, 500),
nn.ReLU(),
nn.Dropout(p=0.5),
)
self.num_classes = num_classes
self.out_features = 500
def copy_head(self):
return nn.Linear(500, self.num_classes)
class DTN(nn.Sequential):
def __init__(self, num_classes=10):
super(DTN, self).__init__(
nn.Conv2d(3, 64, kernel_size=5, stride=2, padding=2),
nn.BatchNorm2d(64),
nn.Dropout2d(0.1),
nn.ReLU(),
nn.Conv2d(64, 128, kernel_size=5, stride=2, padding=2),
nn.BatchNorm2d(128),
nn.Dropout2d(0.3),
nn.ReLU(),
nn.Conv2d(128, 256, kernel_size=5, stride=2, padding=2),
nn.BatchNorm2d(256),
nn.Dropout2d(0.5),
nn.ReLU(),
nn.Flatten(start_dim=1),
nn.Linear(256 * 4 * 4, 512),
nn.BatchNorm1d(512),
nn.ReLU(),
nn.Dropout(),
)
self.num_classes = num_classes
self.out_features = 512
def copy_head(self):
return nn.Linear(512, self.num_classes)
def lenet(pretrained=False, **kwargs):
"""LeNet model from
`"Gradient-based learning applied to document recognition" `_
Args:
num_classes (int): number of classes. Default: 10
.. note::
The input image size must be 28 x 28.
"""
return LeNet(**kwargs)
def dtn(pretrained=False, **kwargs):
""" DTN model
Args:
num_classes (int): number of classes. Default: 10
.. note::
The input image size must be 32 x 32.
"""
return DTN(**kwargs)
================================================
FILE: domain_generalization/Transfer-Learning-Library/common/vision/models/ibn.py
================================================
"""
Modified from https://github.com/XingangPan/IBN-Net
@author: Baixu Chen
@contact: cbx_99_hasta@outlook.com
"""
import math
import torch
import torch.nn as nn
__all__ = ['resnet18_ibn_a', 'resnet18_ibn_b', 'resnet34_ibn_a', 'resnet34_ibn_b', 'resnet50_ibn_a', 'resnet50_ibn_b',
'resnet101_ibn_a', 'resnet101_ibn_b']
model_urls = {
'resnet18_ibn_a': 'https://github.com/XingangPan/IBN-Net/releases/download/v1.0/resnet18_ibn_a-2f571257.pth',
'resnet34_ibn_a': 'https://github.com/XingangPan/IBN-Net/releases/download/v1.0/resnet34_ibn_a-94bc1577.pth',
'resnet50_ibn_a': 'https://github.com/XingangPan/IBN-Net/releases/download/v1.0/resnet50_ibn_a-d9d0bb7b.pth',
'resnet101_ibn_a': 'https://github.com/XingangPan/IBN-Net/releases/download/v1.0/resnet101_ibn_a-59ea0ac6.pth',
'resnet18_ibn_b': 'https://github.com/XingangPan/IBN-Net/releases/download/v1.0/resnet18_ibn_b-bc2f3c11.pth',
'resnet34_ibn_b': 'https://github.com/XingangPan/IBN-Net/releases/download/v1.0/resnet34_ibn_b-04134c37.pth',
'resnet50_ibn_b': 'https://github.com/XingangPan/IBN-Net/releases/download/v1.0/resnet50_ibn_b-9ca61e85.pth',
'resnet101_ibn_b': 'https://github.com/XingangPan/IBN-Net/releases/download/v1.0/resnet101_ibn_b-c55f6dba.pth',
}
class IBN(nn.Module):
r"""Instance-Batch Normalization layer from
`Two at Once: Enhancing Learning and Generalization Capacities via IBN-Net (ECCV 2018)
`_.
Given input feature map :math:`f\_input` of dimension :math:`(C,H,W)`, we first split :math:`f\_input` into
two parts along `channel` dimension. They are denoted as :math:`f_1` of dimension :math:`(C_1,H,W)` and
:math:`f_2` of dimension :math:`(C_2,H,W)`, where :math:`C_1+C_2=C`. Then we pass :math:`f_1` and :math:`f_2`
through IN and BN layer, respectively, to get :math:`IN(f_1)` and :math:`BN(f_2)`. Last, we concat them along
`channel` dimension to create :math:`f\_output=concat(IN(f_1), BN(f_2))`.
Args:
planes (int): Number of channels for the input tensor
ratio (float): Ratio of instance normalization in the IBN layer
"""
def __init__(self, planes, ratio=0.5):
super(IBN, self).__init__()
self.half = int(planes * ratio)
self.IN = nn.InstanceNorm2d(self.half, affine=True)
self.BN = nn.BatchNorm2d(planes - self.half)
def forward(self, x):
split = torch.split(x, self.half, 1)
out1 = self.IN(split[0].contiguous())
out2 = self.BN(split[1].contiguous())
out = torch.cat((out1, out2), 1)
return out
class BasicBlock_IBN(nn.Module):
expansion = 1
def __init__(self, inplanes, planes, ibn=None, stride=1, downsample=None):
super(BasicBlock_IBN, self).__init__()
self.conv1 = nn.Conv2d(inplanes, planes, kernel_size=3, stride=stride,
padding=1, bias=False)
if ibn == 'a':
self.bn1 = IBN(planes)
else:
self.bn1 = nn.BatchNorm2d(planes)
self.relu = nn.ReLU(inplace=True)
self.conv2 = nn.Conv2d(planes, planes, kernel_size=3, padding=1, bias=False)
self.bn2 = nn.BatchNorm2d(planes)
self.IN = nn.InstanceNorm2d(planes, affine=True) if ibn == 'b' else None
self.downsample = downsample
self.stride = stride
def forward(self, x):
residual = x
out = self.conv1(x)
out = self.bn1(out)
out = self.relu(out)
out = self.conv2(out)
out = self.bn2(out)
if self.downsample is not None:
residual = self.downsample(x)
out += residual
if self.IN is not None:
out = self.IN(out)
out = self.relu(out)
return out
class Bottleneck_IBN(nn.Module):
expansion = 4
def __init__(self, inplanes, planes, ibn=None, stride=1, downsample=None):
super(Bottleneck_IBN, self).__init__()
self.conv1 = nn.Conv2d(inplanes, planes, kernel_size=1, bias=False)
if ibn == 'a':
self.bn1 = IBN(planes)
else:
self.bn1 = nn.BatchNorm2d(planes)
self.conv2 = nn.Conv2d(planes, planes, kernel_size=3, stride=stride,
padding=1, bias=False)
self.bn2 = nn.BatchNorm2d(planes)
self.conv3 = nn.Conv2d(planes, planes * self.expansion, kernel_size=1, bias=False)
self.bn3 = nn.BatchNorm2d(planes * self.expansion)
self.IN = nn.InstanceNorm2d(planes * 4, affine=True) if ibn == 'b' else None
self.relu = nn.ReLU(inplace=True)
self.downsample = downsample
self.stride = stride
def forward(self, x):
residual = x
out = self.conv1(x)
out = self.bn1(out)
out = self.relu(out)
out = self.conv2(out)
out = self.bn2(out)
out = self.relu(out)
out = self.conv3(out)
out = self.bn3(out)
if self.downsample is not None:
residual = self.downsample(x)
out += residual
if self.IN is not None:
out = self.IN(out)
out = self.relu(out)
return out
class ResNet_IBN(nn.Module):
r"""
ResNets-IBN without fully connected layer
"""
def __init__(self, block, layers, ibn_cfg=('a', 'a', 'a', None)):
self.inplanes = 64
super(ResNet_IBN, self).__init__()
self.conv1 = nn.Conv2d(3, 64, kernel_size=7, stride=2, padding=3,
bias=False)
if ibn_cfg[0] == 'b':
self.bn1 = nn.InstanceNorm2d(64, affine=True)
else:
self.bn1 = nn.BatchNorm2d(64)
self.relu = nn.ReLU(inplace=True)
self.maxpool = nn.MaxPool2d(kernel_size=3, stride=2, padding=1)
self.layer1 = self._make_layer(block, 64, layers[0], ibn=ibn_cfg[0])
self.layer2 = self._make_layer(block, 128, layers[1], stride=2, ibn=ibn_cfg[1])
self.layer3 = self._make_layer(block, 256, layers[2], stride=2, ibn=ibn_cfg[2])
self.layer4 = self._make_layer(block, 512, layers[3], stride=2, ibn=ibn_cfg[3])
self._out_features = 512 * block.expansion
for m in self.modules():
if isinstance(m, nn.Conv2d):
n = m.kernel_size[0] * m.kernel_size[1] * m.out_channels
m.weight.data.normal_(0, math.sqrt(2. / n))
elif isinstance(m, nn.BatchNorm2d) or isinstance(m, nn.InstanceNorm2d):
m.weight.data.fill_(1)
m.bias.data.zero_()
def _make_layer(self, block, planes, blocks, stride=1, ibn=None):
downsample = None
if stride != 1 or self.inplanes != planes * block.expansion:
downsample = nn.Sequential(
nn.Conv2d(self.inplanes, planes * block.expansion,
kernel_size=1, stride=stride, bias=False),
nn.BatchNorm2d(planes * block.expansion),
)
layers = []
layers.append(block(self.inplanes, planes,
None if ibn == 'b' else ibn,
stride, downsample))
self.inplanes = planes * block.expansion
for i in range(1, blocks):
layers.append(block(self.inplanes, planes,
None if (ibn == 'b' and i < blocks - 1) else ibn))
return nn.Sequential(*layers)
def forward(self, x):
""""""
x = self.conv1(x)
x = self.bn1(x)
x = self.relu(x)
x = self.maxpool(x)
x = self.layer1(x)
x = self.layer2(x)
x = self.layer3(x)
x = self.layer4(x)
return x
@property
def out_features(self) -> int:
"""The dimension of output features"""
return self._out_features
def resnet18_ibn_a(pretrained=False):
"""Constructs a ResNet-18-IBN-a model.
Args:
pretrained (bool): If True, returns a model pre-trained on ImageNet
"""
model = ResNet_IBN(block=BasicBlock_IBN,
layers=[2, 2, 2, 2],
ibn_cfg=('a', 'a', 'a', None))
if pretrained:
model.load_state_dict(torch.hub.load_state_dict_from_url(model_urls['resnet18_ibn_a']), strict=False)
return model
def resnet34_ibn_a(pretrained=False):
"""Constructs a ResNet-34-IBN-a model.
Args:
pretrained (bool): If True, returns a model pre-trained on ImageNet
"""
model = ResNet_IBN(block=BasicBlock_IBN,
layers=[3, 4, 6, 3],
ibn_cfg=('a', 'a', 'a', None))
if pretrained:
model.load_state_dict(torch.hub.load_state_dict_from_url(model_urls['resnet34_ibn_a']), strict=False)
return model
def resnet50_ibn_a(pretrained=False):
"""Constructs a ResNet-50-IBN-a model.
Args:
pretrained (bool): If True, returns a model pre-trained on ImageNet
"""
model = ResNet_IBN(block=Bottleneck_IBN,
layers=[3, 4, 6, 3],
ibn_cfg=('a', 'a', 'a', None))
if pretrained:
model.load_state_dict(torch.hub.load_state_dict_from_url(model_urls['resnet50_ibn_a']), strict=False)
return model
def resnet101_ibn_a(pretrained=False):
"""Constructs a ResNet-101-IBN-a model.
Args:
pretrained (bool): If True, returns a model pre-trained on ImageNet
"""
model = ResNet_IBN(block=Bottleneck_IBN,
layers=[3, 4, 23, 3],
ibn_cfg=('a', 'a', 'a', None))
if pretrained:
model.load_state_dict(torch.hub.load_state_dict_from_url(model_urls['resnet101_ibn_a']), strict=False)
return model
def resnet18_ibn_b(pretrained=False):
"""Constructs a ResNet-18-IBN-b model.
Args:
pretrained (bool): If True, returns a model pre-trained on ImageNet
"""
model = ResNet_IBN(block=BasicBlock_IBN,
layers=[2, 2, 2, 2],
ibn_cfg=('b', 'b', None, None))
if pretrained:
model.load_state_dict(torch.hub.load_state_dict_from_url(model_urls['resnet18_ibn_b']), strict=False)
return model
def resnet34_ibn_b(pretrained=False):
"""Constructs a ResNet-34-IBN-b model.
Args:
pretrained (bool): If True, returns a model pre-trained on ImageNet
"""
model = ResNet_IBN(block=BasicBlock_IBN,
layers=[3, 4, 6, 3],
ibn_cfg=('b', 'b', None, None))
if pretrained:
model.load_state_dict(torch.hub.load_state_dict_from_url(model_urls['resnet34_ibn_b']), strict=False)
return model
def resnet50_ibn_b(pretrained=False):
"""Constructs a ResNet-50-IBN-b model.
Args:
pretrained (bool): If True, returns a model pre-trained on ImageNet
"""
model = ResNet_IBN(block=Bottleneck_IBN,
layers=[3, 4, 6, 3],
ibn_cfg=('b', 'b', None, None))
if pretrained:
model.load_state_dict(torch.hub.load_state_dict_from_url(model_urls['resnet50_ibn_b']), strict=False)
return model
def resnet101_ibn_b(pretrained=False):
"""Constructs a ResNet-101-IBN-b model.
Args:
pretrained (bool): If True, returns a model pre-trained on ImageNet
"""
model = ResNet_IBN(block=Bottleneck_IBN,
layers=[3, 4, 23, 3],
ibn_cfg=('b', 'b', None, None))
if pretrained:
model.load_state_dict(torch.hub.load_state_dict_from_url(model_urls['resnet101_ibn_b']), strict=False)
return model
================================================
FILE: domain_generalization/Transfer-Learning-Library/common/vision/models/keypoint_detection/__init__.py
================================================
from .pose_resnet import *
from . import loss
__all__ = ['pose_resnet']
================================================
FILE: domain_generalization/Transfer-Learning-Library/common/vision/models/keypoint_detection/loss.py
================================================
"""
Modified from https://github.com/microsoft/human-pose-estimation.pytorch
@author: Junguang Jiang
@contact: JiangJunguang1123@outlook.com
"""
import torch.nn as nn
import torch.nn.functional as F
class JointsMSELoss(nn.Module):
"""
Typical MSE loss for keypoint detection.
Args:
reduction (str, optional): Specifies the reduction to apply to the output:
``'none'`` | ``'mean'``. ``'none'``: no reduction will be applied,
``'mean'``: the sum of the output will be divided by the number of
elements in the output. Default: ``'mean'``
Inputs:
- output (tensor): heatmap predictions
- target (tensor): heatmap labels
- target_weight (tensor): whether the keypoint is visible. All keypoint is visible if None. Default: None.
Shape:
- output: :math:`(minibatch, K, H, W)` where K means the number of keypoints,
H and W is the height and width of the heatmap respectively.
- target: :math:`(minibatch, K, H, W)`.
- target_weight: :math:`(minibatch, K)`.
- Output: scalar by default. If :attr:`reduction` is ``'none'``, then :math:`(minibatch, K)`.
"""
def __init__(self, reduction='mean'):
super(JointsMSELoss, self).__init__()
self.criterion = nn.MSELoss(reduction='none')
self.reduction = reduction
def forward(self, output, target, target_weight=None):
B, K, _, _ = output.shape
heatmaps_pred = output.reshape((B, K, -1))
heatmaps_gt = target.reshape((B, K, -1))
loss = self.criterion(heatmaps_pred, heatmaps_gt) * 0.5
if target_weight is not None:
loss = loss * target_weight.view((B, K, 1))
if self.reduction == 'mean':
return loss.mean()
elif self.reduction == 'none':
return loss.mean(dim=-1)
class JointsKLLoss(nn.Module):
"""
KL Divergence for keypoint detection proposed by
`Regressive Domain Adaptation for Unsupervised Keypoint Detection `_.
Args:
reduction (str, optional): Specifies the reduction to apply to the output:
``'none'`` | ``'mean'``. ``'none'``: no reduction will be applied,
``'mean'``: the sum of the output will be divided by the number of
elements in the output. Default: ``'mean'``
Inputs:
- output (tensor): heatmap predictions
- target (tensor): heatmap labels
- target_weight (tensor): whether the keypoint is visible. All keypoint is visible if None. Default: None.
Shape:
- output: :math:`(minibatch, K, H, W)` where K means the number of keypoints,
H and W is the height and width of the heatmap respectively.
- target: :math:`(minibatch, K, H, W)`.
- target_weight: :math:`(minibatch, K)`.
- Output: scalar by default. If :attr:`reduction` is ``'none'``, then :math:`(minibatch, K)`.
"""
def __init__(self, reduction='mean', epsilon=0.):
super(JointsKLLoss, self).__init__()
self.criterion = nn.KLDivLoss(reduction='none')
self.reduction = reduction
self.epsilon = epsilon
def forward(self, output, target, target_weight=None):
B, K, _, _ = output.shape
heatmaps_pred = output.reshape((B, K, -1))
heatmaps_pred = F.log_softmax(heatmaps_pred, dim=-1)
heatmaps_gt = target.reshape((B, K, -1))
heatmaps_gt = heatmaps_gt + self.epsilon
heatmaps_gt = heatmaps_gt / heatmaps_gt.sum(dim=-1, keepdims=True)
loss = self.criterion(heatmaps_pred, heatmaps_gt).sum(dim=-1)
if target_weight is not None:
loss = loss * target_weight.view((B, K))
if self.reduction == 'mean':
return loss.mean()
elif self.reduction == 'none':
return loss.mean(dim=-1)
================================================
FILE: domain_generalization/Transfer-Learning-Library/common/vision/models/keypoint_detection/pose_resnet.py
================================================
"""
Modified from https://github.com/microsoft/human-pose-estimation.pytorch
@author: Junguang Jiang
@contact: JiangJunguang1123@outlook.com
"""
import torch.nn as nn
from ..resnet import _resnet, Bottleneck
class Upsampling(nn.Sequential):
"""
3-layers deconvolution used in `Simple Baseline `_.
"""
def __init__(self, in_channel=2048, hidden_dims=(256, 256, 256), kernel_sizes=(4, 4, 4), bias=False):
assert len(hidden_dims) == len(kernel_sizes), \
'ERROR: len(hidden_dims) is different len(kernel_sizes)'
layers = []
for hidden_dim, kernel_size in zip(hidden_dims, kernel_sizes):
if kernel_size == 4:
padding = 1
output_padding = 0
elif kernel_size == 3:
padding = 1
output_padding = 1
elif kernel_size == 2:
padding = 0
output_padding = 0
else:
raise NotImplementedError("kernel_size is {}".format(kernel_size))
layers.append(
nn.ConvTranspose2d(
in_channels=in_channel,
out_channels=hidden_dim,
kernel_size=kernel_size,
stride=2,
padding=padding,
output_padding=output_padding,
bias=bias))
layers.append(nn.BatchNorm2d(hidden_dim))
layers.append(nn.ReLU(inplace=True))
in_channel = hidden_dim
super(Upsampling, self).__init__(*layers)
# init following Simple Baseline
for name, m in self.named_modules():
if isinstance(m, nn.ConvTranspose2d):
nn.init.normal_(m.weight, std=0.001)
if bias:
nn.init.constant_(m.bias, 0)
elif isinstance(m, nn.BatchNorm2d):
nn.init.constant_(m.weight, 1)
nn.init.constant_(m.bias, 0)
class PoseResNet(nn.Module):
"""
`Simple Baseline `_ for keypoint detection.
Args:
backbone (torch.nn.Module): Backbone to extract 2-d features from data
upsampling (torch.nn.Module): Layer to upsample image feature to heatmap size
feature_dim (int): The dimension of the features from upsampling layer.
num_keypoints (int): Number of keypoints
finetune (bool, optional): Whether use 10x smaller learning rate in the backbone. Default: False
"""
def __init__(self, backbone, upsampling, feature_dim, num_keypoints, finetune=False):
super(PoseResNet, self).__init__()
self.backbone = backbone
self.upsampling = upsampling
self.head = nn.Conv2d(in_channels=feature_dim, out_channels=num_keypoints, kernel_size=1, stride=1, padding=0)
self.finetune = finetune
for m in self.head.modules():
nn.init.normal_(m.weight, std=0.001)
nn.init.constant_(m.bias, 0)
def forward(self, x):
x = self.backbone(x)
x = self.upsampling(x)
x = self.head(x)
return x
def get_parameters(self, lr=1.):
return [
{'params': self.backbone.parameters(), 'lr': 0.1 * lr if self.finetune else lr},
{'params': self.upsampling.parameters(), 'lr': lr},
{'params': self.head.parameters(), 'lr': lr},
]
def _pose_resnet(arch, num_keypoints, block, layers, pretrained_backbone, deconv_with_bias, finetune=False, progress=True, **kwargs):
backbone = _resnet(arch, block, layers, pretrained_backbone, progress, **kwargs)
upsampling = Upsampling(backbone.out_features, bias=deconv_with_bias)
model = PoseResNet(backbone, upsampling, 256, num_keypoints, finetune)
return model
def pose_resnet101(num_keypoints, pretrained_backbone=True, deconv_with_bias=False, finetune=False, progress=True, **kwargs):
"""Constructs a Simple Baseline model with a ResNet-101 backbone.
Args:
num_keypoints (int): number of keypoints
pretrained_backbone (bool, optional): If True, returns a model pre-trained on ImageNet. Default: True.
deconv_with_bias (bool, optional): Whether use bias in the deconvolution layer. Default: False
finetune (bool, optional): Whether use 10x smaller learning rate in the backbone. Default: False
progress (bool, optional): If True, displays a progress bar of the download to stderr. Default: True
"""
return _pose_resnet('resnet101', num_keypoints, Bottleneck, [3, 4, 23, 3], pretrained_backbone, deconv_with_bias, finetune, progress, **kwargs)
================================================
FILE: domain_generalization/Transfer-Learning-Library/common/vision/models/reid/__init__.py
================================================
from .resnet import *
from ..ibn import *
__all__ = ['resnet']
================================================
FILE: domain_generalization/Transfer-Learning-Library/common/vision/models/reid/identifier.py
================================================
"""
@author: Baixu Chen
@contact: cbx_99_hasta@outlook.com
"""
from typing import List, Dict, Optional
import torch
import torch.nn as nn
from torch.nn import init
class ReIdentifier(nn.Module):
r"""Person reIdentifier from `Bag of Tricks and A Strong Baseline for Deep Person Re-identification (CVPR 2019)
`_.
Given 2-d features :math:`f` from backbone network, the authors pass :math:`f` through another `BatchNorm1d` layer
and get :math:`bn\_f`, which will then pass through a `Linear` layer to output predictions. During training, we
use :math:`f` to compute triplet loss. While during testing, :math:`bn\_f` is used as feature. This may be a little
confusing. The figures in the origin paper will help you understand better.
"""
def __init__(self, backbone: nn.Module, num_classes: int, bottleneck: Optional[nn.Module] = None,
bottleneck_dim: Optional[int] = -1, finetune=True, pool_layer=None):
super(ReIdentifier, self).__init__()
if pool_layer is None:
self.pool_layer = nn.Sequential(
nn.AdaptiveAvgPool2d(output_size=(1, 1)),
nn.Flatten()
)
else:
self.pool_layer = pool_layer
self.backbone = backbone
self.num_classes = num_classes
if bottleneck is None:
feature_bn = nn.BatchNorm1d(backbone.out_features)
self.bottleneck = feature_bn
self._features_dim = backbone.out_features
else:
feature_bn = nn.BatchNorm1d(bottleneck_dim)
self.bottleneck = nn.Sequential(
bottleneck,
feature_bn
)
self._features_dim = bottleneck_dim
self.head = nn.Linear(self.features_dim, num_classes, bias=False)
self.finetune = finetune
# initialize feature_bn and head
feature_bn.bias.requires_grad_(False)
init.constant_(feature_bn.weight, 1)
init.constant_(feature_bn.bias, 0)
init.normal_(self.head.weight, std=0.001)
@property
def features_dim(self) -> int:
"""The dimension of features before the final `head` layer"""
return self._features_dim
def forward(self, x: torch.Tensor):
""""""
f = self.pool_layer(self.backbone(x))
bn_f = self.bottleneck(f)
if not self.training:
return bn_f
predictions = self.head(bn_f)
return predictions, f
def get_parameters(self, base_lr=1.0, rate=0.1) -> List[Dict]:
"""A parameter list which decides optimization hyper-parameters,
such as the relative learning rate of each layer
"""
params = [
{"params": self.backbone.parameters(), "lr": rate * base_lr if self.finetune else 1.0 * base_lr},
{"params": self.bottleneck.parameters(), "lr": 1.0 * base_lr},
{"params": self.head.parameters(), "lr": 1.0 * base_lr},
]
return params
================================================
FILE: domain_generalization/Transfer-Learning-Library/common/vision/models/reid/loss.py
================================================
"""
Modified from https://github.com/yxgeee/MMT
@author: Baixu Chen
@contact: cbx_99_hasta@outlook.com
"""
import torch
import torch.nn as nn
import torch.nn.functional as F
def pairwise_euclidean_distance(x, y):
"""Compute pairwise euclidean distance between two sets of features"""
m, n = x.size(0), y.size(0)
dist_mat = torch.pow(x, 2).sum(1, keepdim=True).expand(m, n) + \
torch.pow(y, 2).sum(1, keepdim=True).expand(n, m).t() \
- 2 * torch.matmul(x, y.t())
# for numerical stability
dist_mat = dist_mat.clamp(min=1e-12).sqrt()
return dist_mat
def hard_examples_mining(dist_mat, identity_mat, return_idxes=False):
r"""Select hard positives and hard negatives according to `In defense of the Triplet Loss for Person
Re-Identification (ICCV 2017) `_
Args:
dist_mat (tensor): pairwise distance matrix between two sets of features
identity_mat (tensor): a matrix of shape :math:`(N, M)`. If two images :math:`P[i]` of set :math:`P` and
:math:`Q[j]` of set :math:`Q` come from the same person, then :math:`identity\_mat[i, j] = 1`,
otherwise :math:`identity\_mat[i, j] = 0`
return_idxes (bool, optional): if True, also return indexes of hard examples. Default: False
"""
# the implementation here is a little tricky, dist_mat contains pairwise distance between probe image and other
# images in current mini-batch. As we want to select positive examples of the same person, we add a constant
# negative offset on other images before sorting. As a result, images of the **same** person will rank first.
sorted_dist_mat, sorted_idxes = torch.sort(dist_mat + (-1e7) * (1 - identity_mat), dim=1,
descending=True)
dist_ap = sorted_dist_mat[:, 0]
hard_positive_idxes = sorted_idxes[:, 0]
# the implementation here is similar to above code, we add a constant positive offset on images of same person
# before sorting. Besides, we sort in ascending order. As a result, images of **different** persons will rank first.
sorted_dist_mat, sorted_idxes = torch.sort(dist_mat + 1e7 * identity_mat, dim=1,
descending=False)
dist_an = sorted_dist_mat[:, 0]
hard_negative_idxes = sorted_idxes[:, 0]
if return_idxes:
return dist_ap, dist_an, hard_positive_idxes, hard_negative_idxes
return dist_ap, dist_an
class CrossEntropyLossWithLabelSmooth(nn.Module):
r"""Cross entropy loss with label smooth from `Rethinking the Inception Architecture for Computer Vision
(CVPR 2016) `_.
Given one-hot labels :math:`labels \in R^C`, where :math:`C` is the number of classes,
smoothed labels are calculated as
.. math::
smoothed\_labels = (1 - \epsilon) \times labels + \epsilon \times \frac{1}{C}
We use smoothed labels when calculating cross entropy loss and this can be helpful for preventing over-fitting.
Args:
num_classes (int): number of classes.
epsilon (float): a float number that controls the smoothness.
Inputs:
- y (tensor): unnormalized classifier predictions, :math:`y`
- labels (tensor): ground truth labels, :math:`labels`
Shape:
- y: :math:`(minibatch, C)`, where :math:`C` is the number of classes
- labels: :math:`(minibatch, )`
"""
def __init__(self, num_classes, epsilon=0.1):
super(CrossEntropyLossWithLabelSmooth, self).__init__()
self.num_classes = num_classes
self.epsilon = epsilon
self.log_softmax = nn.LogSoftmax(dim=1).cuda()
def forward(self, y, labels):
log_prob = self.log_softmax(y)
labels = torch.zeros_like(log_prob).scatter_(1, labels.unsqueeze(1), 1)
labels = (1 - self.epsilon) * labels + self.epsilon / self.num_classes
loss = (- labels * log_prob).mean(0).sum()
return loss
class TripletLoss(nn.Module):
"""Triplet loss augmented with batch hard from `In defense of the Triplet Loss for Person Re-Identification
(ICCV 2017) `_.
Args:
margin (float): margin of triplet loss
normalize_feature (bool, optional): if True, normalize features into unit norm first before computing loss.
Default: False.
"""
def __init__(self, margin, normalize_feature=False):
super(TripletLoss, self).__init__()
self.margin = margin
self.normalize_feature = normalize_feature
self.margin_loss = nn.MarginRankingLoss(margin=margin).cuda()
def forward(self, f, labels):
if self.normalize_feature:
# equivalent to cosine similarity
f = F.normalize(f)
dist_mat = pairwise_euclidean_distance(f, f)
n = dist_mat.size(0)
identity_mat = labels.expand(n, n).eq(labels.expand(n, n).t()).float()
dist_ap, dist_an = hard_examples_mining(dist_mat, identity_mat)
y = torch.ones_like(dist_ap)
loss = self.margin_loss(dist_an, dist_ap, y)
return loss
class TripletLossXBM(nn.Module):
r"""Triplet loss augmented with batch hard from `In defense of the Triplet Loss for Person Re-Identification
(ICCV 2017) `_. The only difference from triplet loss lies in that
both features from current mini batch and external storage (XBM) are involved.
Args:
margin (float, optional): margin of triplet loss. Default: 0.3
normalize_feature (bool, optional): if True, normalize features into unit norm first before computing loss.
Default: False
Inputs:
- f (tensor): features of current mini batch, :math:`f`
- labels (tensor): identity labels for current mini batch, :math:`labels`
- xbm_f (tensor): features collected from XBM, :math:`xbm\_f`
- xbm_labels (tensor): corresponding identity labels of xbm_f, :math:`xbm\_labels`
Shape:
- f: :math:`(minibatch, F)`, where :math:`F` is the feature dimension
- labels: :math:`(minibatch, )`
- xbm_f: :math:`(minibatch, F)`
- xbm_labels: :math:`(minibatch, )`
"""
def __init__(self, margin=0.3, normalize_feature=False):
super(TripletLossXBM, self).__init__()
self.margin = margin
self.normalize_feature = normalize_feature
self.ranking_loss = nn.MarginRankingLoss(margin=margin)
def forward(self, f, labels, xbm_f, xbm_labels):
if self.normalize_feature:
# equivalent to cosine similarity
f = F.normalize(f)
xbm_f = F.normalize(xbm_f)
dist_mat = pairwise_euclidean_distance(f, xbm_f)
# hard examples mining
n, m = f.size(0), xbm_f.size(0)
identity_mat = labels.expand(m, n).t().eq(xbm_labels.expand(n, m)).float()
dist_ap, dist_an = hard_examples_mining(dist_mat, identity_mat)
# Compute ranking hinge loss
y = torch.ones_like(dist_an)
loss = self.ranking_loss(dist_an, dist_ap, y)
return loss
class SoftTripletLoss(nn.Module):
r"""Soft triplet loss from `Mutual Mean-Teaching: Pseudo Label Refinery for Unsupervised
Domain Adaptation on Person Re-identification (ICLR 2020) `_.
Consider a triplet :math:`x,x_p,x_n` (anchor, positive, negative), corresponding features are :math:`f,f_p,f_n`.
We optimize for a smaller distance between :math:`f` and :math:`f_p` and a larger distance
between :math:`f` and :math:`f_n`. Inner product is adopted as their similarity measure, soft triplet loss is thus
defined as
.. math::
loss = \mathcal{L}_{\text{bce}}(\frac{\text{exp}(f^Tf_p)}{\text{exp}(f^Tf_p)+\text{exp}(f^Tf_n)}, 1)
where :math:`\mathcal{L}_{\text{bce}}` means binary cross entropy loss. We denote the first term in above loss function
as :math:`T`. When features from another teacher network can be obtained, we can calculate :math:`T_{teacher}` as
labels, resulting in the following soft version
.. math::
loss = \mathcal{L}_{\text{bce}}(T, T_{teacher})
Args:
margin (float, optional): margin of triplet loss. If None, soft labels from another network will be adopted when
computing loss. Default: None.
normalize_feature (bool, optional): if True, normalize features into unit norm first before computing loss.
Default: False.
"""
def __init__(self, margin=None, normalize_feature=False):
super(SoftTripletLoss, self).__init__()
self.margin = margin
self.normalize_feature = normalize_feature
def forward(self, features_1, features_2, labels):
if self.normalize_feature:
# equal to cosine similarity
features_1 = F.normalize(features_1)
features_2 = F.normalize(features_2)
dist_mat = pairwise_euclidean_distance(features_1, features_1)
assert dist_mat.size(0) == dist_mat.size(1)
n = dist_mat.size(0)
identity_mat = labels.expand(n, n).eq(labels.expand(n, n).t()).float()
dist_ap, dist_an, ap_idxes, an_idxes = hard_examples_mining(dist_mat, identity_mat, return_idxes=True)
assert dist_an.size(0) == dist_ap.size(0)
triple_dist = torch.stack((dist_ap, dist_an), dim=1)
triple_dist = F.log_softmax(triple_dist, dim=1)
if self.margin is not None:
loss = (- self.margin * triple_dist[:, 0] - (1 - self.margin) * triple_dist[:, 1]).mean()
return loss
dist_mat_ref = pairwise_euclidean_distance(features_2, features_2)
dist_ap_ref = torch.gather(dist_mat_ref, 1, ap_idxes.view(n, 1).expand(n, n))[:, 0]
dist_an_ref = torch.gather(dist_mat_ref, 1, an_idxes.view(n, 1).expand(n, n))[:, 0]
triple_dist_ref = torch.stack((dist_ap_ref, dist_an_ref), dim=1)
triple_dist_ref = F.softmax(triple_dist_ref, dim=1).detach()
loss = (- triple_dist_ref * triple_dist).sum(dim=1).mean()
return loss
class CrossEntropyLoss(nn.Module):
r"""We use :math:`C` to denote the number of classes, :math:`N` to denote mini-batch
size, this criterion expects unnormalized predictions :math:`y\_{logits}` of shape :math:`(N, C)` and
:math:`target\_{logits}` of the same shape :math:`(N, C)`. Then we first normalize them into
probability distributions among classes
.. math::
y = \text{softmax}(y\_{logits})
.. math::
target = \text{softmax}(target\_{logits})
Final objective is calculated as
.. math::
\text{loss} = \frac{1}{N} \sum_{i=1}^{N} \sum_{j=1}^C -target_i^j \times \text{log} (y_i^j)
"""
def __init__(self):
super(CrossEntropyLoss, self).__init__()
self.log_softmax = nn.LogSoftmax(dim=1).cuda()
def forward(self, y, labels):
log_prob = self.log_softmax(y)
loss = (- F.softmax(labels, dim=1).detach() * log_prob).sum(dim=1).mean()
return loss
================================================
FILE: domain_generalization/Transfer-Learning-Library/common/vision/models/reid/resnet.py
================================================
"""
@author: Baixu Chen
@contact: cbx_99_hasta@outlook.com
"""
from common.vision.models.resnet import ResNet, load_state_dict_from_url, model_urls, BasicBlock, Bottleneck
__all__ = ['reid_resnet18', 'reid_resnet34', 'reid_resnet50', 'reid_resnet101']
class ReidResNet(ResNet):
r"""Modified `ResNet` architecture for ReID from `Mutual Mean-Teaching: Pseudo Label Refinery for Unsupervised
Domain Adaptation on Person Re-identification (ICLR 2020) `_. We change stride
of :math:`layer4\_group1\_conv2, layer4\_group1\_downsample1` to 1. During forward pass, we will not activate
`self.relu`. Please refer to source code for details.
"""
def __init__(self, *args, **kwargs):
super(ReidResNet, self).__init__(*args, **kwargs)
self.layer4[0].conv2.stride = (1, 1)
self.layer4[0].downsample[0].stride = (1, 1)
def forward(self, x):
x = self.conv1(x)
x = self.bn1(x)
# x = self.relu(x)
x = self.maxpool(x)
x = self.layer1(x)
x = self.layer2(x)
x = self.layer3(x)
x = self.layer4(x)
return x
def _reid_resnet(arch, block, layers, pretrained, progress, **kwargs):
model = ReidResNet(block, layers, **kwargs)
if pretrained:
model_dict = model.state_dict()
pretrained_dict = load_state_dict_from_url(model_urls[arch],
progress=progress)
# remove keys from pretrained dict that doesn't appear in model dict
pretrained_dict = {k: v for k, v in pretrained_dict.items() if k in model_dict}
model.load_state_dict(pretrained_dict, strict=False)
return model
def reid_resnet18(pretrained=False, progress=True, **kwargs):
r"""Constructs a Reid-ResNet-18 model.
Args:
pretrained (bool): If True, returns a model pre-trained on ImageNet
progress (bool): If True, displays a progress bar of the download to stderr
"""
return _reid_resnet('resnet18', BasicBlock, [2, 2, 2, 2], pretrained, progress,
**kwargs)
def reid_resnet34(pretrained=False, progress=True, **kwargs):
r"""Constructs a Reid-ResNet-34 model.
Args:
pretrained (bool): If True, returns a model pre-trained on ImageNet
progress (bool): If True, displays a progress bar of the download to stderr
"""
return _reid_resnet('resnet34', BasicBlock, [3, 4, 6, 3], pretrained, progress,
**kwargs)
def reid_resnet50(pretrained=False, progress=True, **kwargs):
r"""Constructs a Reid-ResNet-50 model.
Args:
pretrained (bool): If True, returns a model pre-trained on ImageNet
progress (bool): If True, displays a progress bar of the download to stderr
"""
return _reid_resnet('resnet50', Bottleneck, [3, 4, 6, 3], pretrained, progress,
**kwargs)
def reid_resnet101(pretrained=False, progress=True, **kwargs):
r"""Constructs a Reid-ResNet-101 model.
Args:
pretrained (bool): If True, returns a model pre-trained on ImageNet
progress (bool): If True, displays a progress bar of the download to stderr
"""
return _reid_resnet('resnet101', Bottleneck, [3, 4, 23, 3], pretrained, progress,
**kwargs)
================================================
FILE: domain_generalization/Transfer-Learning-Library/common/vision/models/resnet.py
================================================
"""
Modified based on torchvision.models.resnet.
@author: Junguang Jiang
@contact: JiangJunguang1123@outlook.com
"""
import torch.nn as nn
from torchvision import models
from torchvision.models.utils import load_state_dict_from_url
from torchvision.models.resnet import BasicBlock, Bottleneck, model_urls
import copy
__all__ = ['ResNet', 'resnet18', 'resnet34', 'resnet50', 'resnet101',
'resnet152', 'resnext50_32x4d', 'resnext101_32x8d',
'wide_resnet50_2', 'wide_resnet101_2']
class ResNet(models.ResNet):
"""ResNets without fully connected layer"""
def __init__(self, *args, **kwargs):
super(ResNet, self).__init__(*args, **kwargs)
self._out_features = self.fc.in_features
def forward(self, x):
""""""
x = self.conv1(x)
x = self.bn1(x)
x = self.relu(x)
x = self.maxpool(x)
x = self.layer1(x)
x = self.layer2(x)
x = self.layer3(x)
x = self.layer4(x)
# x = self.avgpool(x)
# x = torch.flatten(x, 1)
# x = x.view(-1, self._out_features)
return x
@property
def out_features(self) -> int:
"""The dimension of output features"""
return self._out_features
def copy_head(self) -> nn.Module:
"""Copy the origin fully connected layer"""
return copy.deepcopy(self.fc)
def _resnet(arch, block, layers, pretrained, progress, **kwargs):
model = ResNet(block, layers, **kwargs)
if pretrained:
model_dict = model.state_dict()
pretrained_dict = load_state_dict_from_url(model_urls[arch],
progress=progress)
# remove keys from pretrained dict that doesn't appear in model dict
pretrained_dict = {k: v for k, v in pretrained_dict.items() if k in model_dict}
model.load_state_dict(pretrained_dict, strict=False)
return model
def resnet18(pretrained=False, progress=True, **kwargs):
r"""ResNet-18 model from
`"Deep Residual Learning for Image Recognition" `_
Args:
pretrained (bool): If True, returns a model pre-trained on ImageNet
progress (bool): If True, displays a progress bar of the download to stderr
"""
return _resnet('resnet18', BasicBlock, [2, 2, 2, 2], pretrained, progress,
**kwargs)
def resnet34(pretrained=False, progress=True, **kwargs):
r"""ResNet-34 model from
`"Deep Residual Learning for Image Recognition" `_
Args:
pretrained (bool): If True, returns a model pre-trained on ImageNet
progress (bool): If True, displays a progress bar of the download to stderr
"""
return _resnet('resnet34', BasicBlock, [3, 4, 6, 3], pretrained, progress,
**kwargs)
def resnet50(pretrained=False, progress=True, **kwargs):
r"""ResNet-50 model from
`"Deep Residual Learning for Image Recognition" `_
Args:
pretrained (bool): If True, returns a model pre-trained on ImageNet
progress (bool): If True, displays a progress bar of the download to stderr
"""
return _resnet('resnet50', Bottleneck, [3, 4, 6, 3], pretrained, progress,
**kwargs)
def resnet101(pretrained=False, progress=True, **kwargs):
r"""ResNet-101 model from
`"Deep Residual Learning for Image Recognition" `_
Args:
pretrained (bool): If True, returns a model pre-trained on ImageNet
progress (bool): If True, displays a progress bar of the download to stderr
"""
return _resnet('resnet101', Bottleneck, [3, 4, 23, 3], pretrained, progress,
**kwargs)
def resnet152(pretrained=False, progress=True, **kwargs):
r"""ResNet-152 model from
`"Deep Residual Learning for Image Recognition" `_
Args:
pretrained (bool): If True, returns a model pre-trained on ImageNet
progress (bool): If True, displays a progress bar of the download to stderr
"""
return _resnet('resnet152', Bottleneck, [3, 8, 36, 3], pretrained, progress,
**kwargs)
def resnext50_32x4d(pretrained=False, progress=True, **kwargs):
r"""ResNeXt-50 32x4d model from
`"Aggregated Residual Transformation for Deep Neural Networks" `_
Args:
pretrained (bool): If True, returns a model pre-trained on ImageNet
progress (bool): If True, displays a progress bar of the download to stderr
"""
kwargs['groups'] = 32
kwargs['width_per_group'] = 4
return _resnet('resnext50_32x4d', Bottleneck, [3, 4, 6, 3],
pretrained, progress, **kwargs)
def resnext101_32x8d(pretrained=False, progress=True, **kwargs):
r"""ResNeXt-101 32x8d model from
`"Aggregated Residual Transformation for Deep Neural Networks" `_
Args:
pretrained (bool): If True, returns a model pre-trained on ImageNet
progress (bool): If True, displays a progress bar of the download to stderr
"""
kwargs['groups'] = 32
kwargs['width_per_group'] = 8
return _resnet('resnext101_32x8d', Bottleneck, [3, 4, 23, 3],
pretrained, progress, **kwargs)
def wide_resnet50_2(pretrained=False, progress=True, **kwargs):
r"""Wide ResNet-50-2 model from
`"Wide Residual Networks" `_
The model is the same as ResNet except for the bottleneck number of channels
which is twice larger in every block. The number of channels in outer 1x1
convolutions is the same, e.g. last block in ResNet-50 has 2048-512-2048
channels, and in Wide ResNet-50-2 has 2048-1024-2048.
Args:
pretrained (bool): If True, returns a model pre-trained on ImageNet
progress (bool): If True, displays a progress bar of the download to stderr
"""
kwargs['width_per_group'] = 64 * 2
return _resnet('wide_resnet50_2', Bottleneck, [3, 4, 6, 3],
pretrained, progress, **kwargs)
def wide_resnet101_2(pretrained=False, progress=True, **kwargs):
r"""Wide ResNet-101-2 model from
`"Wide Residual Networks" `_
The model is the same as ResNet except for the bottleneck number of channels
which is twice larger in every block. The number of channels in outer 1x1
convolutions is the same, e.g. last block in ResNet-50 has 2048-512-2048
channels, and in Wide ResNet-50-2 has 2048-1024-2048.
Args:
pretrained (bool): If True, returns a model pre-trained on ImageNet
progress (bool): If True, displays a progress bar of the download to stderr
"""
kwargs['width_per_group'] = 64 * 2
return _resnet('wide_resnet101_2', Bottleneck, [3, 4, 23, 3],
pretrained, progress, **kwargs)
================================================
FILE: domain_generalization/Transfer-Learning-Library/common/vision/models/segmentation/__init__.py
================================================
from .deeplabv2 import *
__all__ = ['deeplabv2']
================================================
FILE: domain_generalization/Transfer-Learning-Library/common/vision/models/segmentation/deeplabv2.py
================================================
"""
@author: Junguang Jiang
@contact: JiangJunguang1123@outlook.com
"""
import torch.nn as nn
from torchvision.models.utils import load_state_dict_from_url
model_urls = {
'deeplabv2_resnet101': 'https://cloud.tsinghua.edu.cn/f/2d9a7fc43ce34f76803a/?dl=1'
}
affine_par = True
class Bottleneck(nn.Module):
expansion = 4
def __init__(self, inplanes, planes, stride=1, dilation=1, downsample=None):
super(Bottleneck, self).__init__()
self.conv1 = nn.Conv2d(inplanes, planes, kernel_size=1, stride=stride, bias=False) # change
self.bn1 = nn.BatchNorm2d(planes, affine=affine_par)
for i in self.bn1.parameters():
i.requires_grad = False
padding = dilation
self.conv2 = nn.Conv2d(planes, planes, kernel_size=3, stride=1, # change
padding=padding, bias=False, dilation=dilation)
self.bn2 = nn.BatchNorm2d(planes, affine=affine_par)
for i in self.bn2.parameters():
i.requires_grad = False
self.conv3 = nn.Conv2d(planes, planes * 4, kernel_size=1, bias=False)
self.bn3 = nn.BatchNorm2d(planes * 4, affine=affine_par)
for i in self.bn3.parameters():
i.requires_grad = False
self.relu = nn.ReLU(inplace=True)
self.downsample = downsample
self.stride = stride
def forward(self, x):
residual = x
out = self.conv1(x)
out = self.bn1(out)
out = self.relu(out)
out = self.conv2(out)
out = self.bn2(out)
out = self.relu(out)
out = self.conv3(out)
out = self.bn3(out)
if self.downsample is not None:
residual = self.downsample(x)
out += residual
out = self.relu(out)
return out
class ASPP_V2(nn.Module):
def __init__(self, inplanes, dilation_series, padding_series, num_classes):
super(ASPP_V2, self).__init__()
self.conv2d_list = nn.ModuleList()
for dilation, padding in zip(dilation_series, padding_series):
self.conv2d_list.append(
nn.Conv2d(inplanes, num_classes, kernel_size=3, stride=1, padding=padding, dilation=dilation,
bias=True))
for m in self.conv2d_list:
m.weight.data.normal_(0, 0.01)
def forward(self, x):
out = self.conv2d_list[0](x)
for i in range(len(self.conv2d_list) - 1):
out += self.conv2d_list[i + 1](x)
return out
class ResNet(nn.Module):
def __init__(self, block, layers):
self.inplanes = 64
super(ResNet, self).__init__()
self.conv1 = nn.Conv2d(3, 64, kernel_size=7, stride=2, padding=3,
bias=False)
self.bn1 = nn.BatchNorm2d(64, affine=affine_par)
for i in self.bn1.parameters():
i.requires_grad = False
self.relu = nn.ReLU(inplace=True)
self.maxpool = nn.MaxPool2d(kernel_size=3, stride=2, padding=1, ceil_mode=True) # change
self.layer1 = self._make_layer(block, 64, layers[0])
self.layer2 = self._make_layer(block, 128, layers[1], stride=2)
self.layer3 = self._make_layer(block, 256, layers[2], stride=1, dilation=2)
self.layer4 = self._make_layer(block, 512, layers[3], stride=1, dilation=4)
for m in self.modules():
if isinstance(m, nn.Conv2d):
m.weight.data.normal_(0, 0.01)
elif isinstance(m, nn.BatchNorm2d):
m.weight.data.fill_(1)
m.bias.data.zero_()
def _make_layer(self, block, planes, blocks, stride=1, dilation=1):
downsample = None
if stride != 1 or self.inplanes != planes * block.expansion or dilation == 2 or dilation == 4:
downsample = nn.Sequential(
nn.Conv2d(self.inplanes, planes * block.expansion,
kernel_size=1, stride=stride, bias=False),
nn.BatchNorm2d(planes * block.expansion, affine=affine_par))
for i in downsample._modules['1'].parameters():
i.requires_grad = False
layers = []
layers.append(block(self.inplanes, planes, stride, dilation=dilation, downsample=downsample))
self.inplanes = planes * block.expansion
for i in range(1, blocks):
layers.append(block(self.inplanes, planes, dilation=dilation))
return nn.Sequential(*layers)
def forward(self, x):
x = self.conv1(x)
x = self.bn1(x)
x = self.relu(x)
x = self.maxpool(x)
x = self.layer1(x)
x = self.layer2(x)
x = self.layer3(x)
x = self.layer4(x)
return x
class Deeplab(nn.Module):
def __init__(self, backbone, classifier, num_classes):
super(Deeplab, self).__init__()
self.backbone = backbone
self.classifier = classifier
self.num_classes = num_classes
def forward(self, x):
x = self.backbone(x)
y = self.classifier(x)
return y
def get_1x_lr_params_NOscale(self):
"""
This generator returns all the parameters of the net except for
the last classification layer. Note that for each batchnorm layer,
requires_grad is set to False in deeplab_resnet.py, therefore this function does not return
any batchnorm parameter
"""
layers = [self.backbone.conv1, self.backbone.bn1,
self.backbone.layer1, self.backbone.layer2, self.backbone.layer3, self.backbone.layer4]
for layer in layers:
for module in layer.modules():
for param in module.parameters():
if param.requires_grad:
yield param
def get_10x_lr_params(self):
"""
This generator returns all the parameters for the last layer of the net,
which does the classification of pixel into classes
"""
for param in self.classifier.parameters():
yield param
def get_parameters(self, lr=1.):
return [
{'params': self.get_1x_lr_params_NOscale(), 'lr': 0.1 * lr},
{'params': self.get_10x_lr_params(), 'lr': lr}
]
def deeplabv2_resnet101(num_classes=19, pretrained_backbone=True):
"""Constructs a DeepLabV2 model with a ResNet-101 backbone.
Args:
num_classes (int, optional): number of classes. Default: 19
pretrained_backbone (bool, optional): If True, returns a model pre-trained on ImageNet. Default: True.
"""
backbone = ResNet(Bottleneck, [3, 4, 23, 3])
if pretrained_backbone:
# download from Internet
saved_state_dict = load_state_dict_from_url(model_urls['deeplabv2_resnet101'], map_location=lambda storage, loc: storage, file_name="deeplabv2_resnet101.pth")
new_params = backbone.state_dict().copy()
for i in saved_state_dict:
i_parts = i.split('.')
if not i_parts[1] == 'layer5':
new_params['.'.join(i_parts[1:])] = saved_state_dict[i]
backbone.load_state_dict(new_params)
classifier = ASPP_V2(2048, [6, 12, 18, 24], [6, 12, 18, 24], num_classes)
return Deeplab(backbone, classifier, num_classes)
================================================
FILE: domain_generalization/Transfer-Learning-Library/common/vision/transforms/__init__.py
================================================
import math
import random
from PIL import Image
import numpy as np
import torch
from torchvision.transforms import Normalize
class ResizeImage(object):
"""Resize the input PIL Image to the given size.
Args:
size (sequence or int): Desired output size. If size is a sequence like
(h, w), output size will be matched to this. If size is an int,
output size will be (size, size)
"""
def __init__(self, size):
if isinstance(size, int):
self.size = (int(size), int(size))
else:
self.size = size
def __call__(self, img):
th, tw = self.size
return img.resize((th, tw))
def __repr__(self):
return self.__class__.__name__ + '(size={0})'.format(self.size)
class MultipleApply:
"""Apply a list of transformations to an image and get multiple transformed images.
Args:
transforms (list or tuple): list of transformations
Example:
>>> transform1 = T.Compose([
... ResizeImage(256),
... T.RandomCrop(224)
... ])
>>> transform2 = T.Compose([
... ResizeImage(256),
... T.RandomCrop(224),
... ])
>>> multiply_transform = MultipleApply([transform1, transform2])
"""
def __init__(self, transforms):
self.transforms = transforms
def __call__(self, image):
return [t(image) for t in self.transforms]
def __repr__(self):
format_string = self.__class__.__name__ + '('
for t in self.transforms:
format_string += '\n'
format_string += ' {0}'.format(t)
format_string += '\n)'
return format_string
class Denormalize(Normalize):
"""DeNormalize a tensor image with mean and standard deviation.
Given mean: ``(mean[1],...,mean[n])`` and std: ``(std[1],..,std[n])`` for ``n``
channels, this transform will denormalize each channel of the input
``torch.*Tensor`` i.e.,
``output[channel] = input[channel] * std[channel] + mean[channel]``
.. note::
This transform acts out of place, i.e., it does not mutate the input tensor.
Args:
mean (sequence): Sequence of means for each channel.
std (sequence): Sequence of standard deviations for each channel.
"""
def __init__(self, mean, std):
mean = np.array(mean)
std = np.array(std)
super().__init__((-mean / std).tolist(), (1 / std).tolist())
class NormalizeAndTranspose:
"""
First, normalize a tensor image with mean and standard deviation.
Then, convert the shape (H x W x C) to shape (C x H x W).
"""
def __init__(self, mean=(104.00698793, 116.66876762, 122.67891434)):
self.mean = np.array(mean, dtype=np.float32)
def __call__(self, image):
if isinstance(image, Image.Image):
image = np.asarray(image, np.float32)
# change to BGR
image = image[:, :, ::-1]
# normalize
image -= self.mean
image = image.transpose((2, 0, 1)).copy()
elif isinstance(image, torch.Tensor):
# change to BGR
image = image[:, :, [2, 1, 0]]
# normalize
image -= torch.from_numpy(self.mean).to(image.device)
image = image.permute((2, 0, 1))
else:
raise NotImplementedError(type(image))
return image
class DeNormalizeAndTranspose:
"""
First, convert a tensor image from the shape (C x H x W ) to shape (H x W x C).
Then, denormalize it with mean and standard deviation.
"""
def __init__(self, mean=(104.00698793, 116.66876762, 122.67891434)):
self.mean = np.array(mean, dtype=np.float32)
def __call__(self, image):
image = image.transpose((1, 2, 0))
# denormalize
image += self.mean
# change to RGB
image = image[:, :, ::-1]
return image
class RandomErasing(object):
"""Random erasing augmentation from `Random Erasing Data Augmentation (CVPR 2017)
`_. This augmentation randomly selects a rectangle region in an image
and erases its pixels.
Args:
probability (float): The probability that the Random Erasing operation will be performed.
sl (float): Minimum proportion of erased area against input image.
sh (float): Maximum proportion of erased area against input image.
r1 (float): Minimum aspect ratio of erased area.
mean (sequence): Value to fill the erased area.
"""
def __init__(self, probability=0.5, sl=0.02, sh=0.4, r1=0.3, mean=(0.4914, 0.4822, 0.4465)):
self.probability = probability
self.mean = mean
self.sl = sl
self.sh = sh
self.r1 = r1
def __call__(self, img):
if random.uniform(0, 1) >= self.probability:
return img
for attempt in range(100):
area = img.size()[1] * img.size()[2]
target_area = random.uniform(self.sl, self.sh) * area
aspect_ratio = random.uniform(self.r1, 1 / self.r1)
h = int(round(math.sqrt(target_area * aspect_ratio)))
w = int(round(math.sqrt(target_area / aspect_ratio)))
if w < img.size()[2] and h < img.size()[1]:
x1 = random.randint(0, img.size()[1] - h)
y1 = random.randint(0, img.size()[2] - w)
if img.size()[0] == 3:
img[0, x1:x1 + h, y1:y1 + w] = self.mean[0]
img[1, x1:x1 + h, y1:y1 + w] = self.mean[1]
img[2, x1:x1 + h, y1:y1 + w] = self.mean[2]
else:
img[0, x1:x1 + h, y1:y1 + w] = self.mean[0]
return img
return img
def __repr__(self):
return self.__class__.__name__ + '(p={})'.format(self.probability)
================================================
FILE: domain_generalization/Transfer-Learning-Library/common/vision/transforms/keypoint_detection.py
================================================
"""
@author: Junguang Jiang
@contact: JiangJunguang1123@outlook.com
"""
# TODO needs better documentation
import numpy as np
from PIL import ImageFilter, Image
import torchvision.transforms.functional as F
import torchvision.transforms.transforms as T
import numbers
import random
import math
import warnings
from typing import ClassVar
def wrapper(transform: ClassVar):
""" Wrap a transform for classification to a transform for keypoint detection.
Note that the keypoint detection label will keep the same before and after wrapper.
Args:
transform (class, callable): transform for classification
Returns:
transform for keypoint detection
"""
class WrapperTransform(transform):
def __call__(self, image, **kwargs):
image = super().__call__(image)
return image, kwargs
return WrapperTransform
ToTensor = wrapper(T.ToTensor)
Normalize = wrapper(T.Normalize)
ColorJitter = wrapper(T.ColorJitter)
def resize(image: Image.Image, size: int, interpolation=Image.BILINEAR,
keypoint2d: np.ndarray=None, intrinsic_matrix: np.ndarray=None):
width, height = image.size
assert width == height
factor = float(size) / float(width)
image = F.resize(image, size, interpolation)
keypoint2d = np.copy(keypoint2d)
keypoint2d *= factor
intrinsic_matrix = np.copy(intrinsic_matrix)
intrinsic_matrix[0][0] *= factor
intrinsic_matrix[0][2] *= factor
intrinsic_matrix[1][1] *= factor
intrinsic_matrix[1][2] *= factor
return image, keypoint2d, intrinsic_matrix
def crop(image: Image.Image, top, left, height, width, keypoint2d: np.ndarray):
image = F.crop(image, top, left, height, width)
keypoint2d = np.copy(keypoint2d)
keypoint2d[:, 0] -= left
keypoint2d[:, 1] -= top
return image, keypoint2d
def resized_crop(img, top, left, height, width, size, interpolation=Image.BILINEAR,
keypoint2d: np.ndarray=None, intrinsic_matrix: np.ndarray=None):
"""Crop the given PIL Image and resize it to desired size.
Notably used in :class:`~torchvision.transforms.RandomResizedCrop`.
Args:
img (PIL Image): Image to be cropped. (0,0) denotes the top left corner of the image.
top (int): Vertical component of the top left corner of the crop box.
left (int): Horizontal component of the top left corner of the crop box.
height (int): Height of the crop box.
width (int): Width of the crop box.
size (sequence or int): Desired output size. Same semantics as ``resize``.
interpolation (int, optional): Desired interpolation. Default is
``PIL.Image.BILINEAR``.
Returns:
PIL Image: Cropped image.
"""
assert isinstance(img, Image.Image), 'img should be PIL Image'
img, keypoint2d = crop(img, top, left, height, width, keypoint2d)
img, keypoint2d, intrinsic_matrix = resize(img, size, interpolation, keypoint2d, intrinsic_matrix)
return img, keypoint2d, intrinsic_matrix
def center_crop(image, output_size, keypoint2d: np.ndarray):
"""Crop the given PIL Image and resize it to desired size.
Args:
img (PIL Image): Image to be cropped. (0,0) denotes the top left corner of the image.
output_size (sequence or int): (height, width) of the crop box. If int,
it is used for both directions
Returns:
PIL Image: Cropped image.
"""
width, height = image.size
crop_height, crop_width = output_size
crop_top = int(round((height - crop_height) / 2.))
crop_left = int(round((width - crop_width) / 2.))
return crop(image, crop_top, crop_left, crop_height, crop_width, keypoint2d)
def hflip(image: Image.Image, keypoint2d: np.ndarray):
width, height = image.size
image = F.hflip(image)
keypoint2d = np.copy(keypoint2d)
keypoint2d[:, 0] = width - 1. - keypoint2d[:, 0]
return image, keypoint2d
def rotate(image: Image.Image, angle, keypoint2d: np.ndarray):
image = F.rotate(image, angle)
angle = -np.deg2rad(angle)
keypoint2d = np.copy(keypoint2d)
rotation_matrix = np.array([
[np.cos(angle), -np.sin(angle)],
[np.sin(angle), np.cos(angle)]
])
width, height = image.size
keypoint2d[:, 0] = keypoint2d[:, 0] - width / 2
keypoint2d[:, 1] = keypoint2d[:, 1] - height / 2
keypoint2d = np.matmul(rotation_matrix, keypoint2d.T).T
keypoint2d[:, 0] = keypoint2d[:, 0] + width / 2
keypoint2d[:, 1] = keypoint2d[:, 1] + height / 2
return image, keypoint2d
def resize_pad(img, keypoint2d, size, interpolation=Image.BILINEAR):
w, h = img.size
if w < h:
oh = size
ow = int(size * w / h)
img = img.resize((ow, oh), interpolation)
pad_top = pad_bottom = 0
pad_left = math.floor((size - ow) / 2)
pad_right = math.ceil((size - ow) / 2)
keypoint2d = keypoint2d * oh / h
keypoint2d[:, 0] += (size - ow) / 2
else:
ow = size
oh = int(size * h / w)
img = img.resize((ow, oh), interpolation)
pad_top = math.floor((size - oh) / 2)
pad_bottom = math.ceil((size - oh) / 2)
pad_left = pad_right = 0
keypoint2d = keypoint2d * ow / w
keypoint2d[:, 1] += (size - oh) / 2
keypoint2d[:, 0] += (size - ow) / 2
img = np.asarray(img)
img = np.pad(img, ((pad_top, pad_bottom), (pad_left, pad_right), (0, 0)), 'constant', constant_values=0)
return Image.fromarray(img), keypoint2d
class Compose(object):
"""Composes several transforms together.
Args:
transforms (list of ``Transform`` objects): list of transforms to compose.
"""
def __init__(self, transforms):
self.transforms = transforms
def __call__(self, image, **kwargs):
for t in self.transforms:
image, kwargs = t(image, **kwargs)
return image, kwargs
class GaussianBlur(object):
def __init__(self, low=0, high=0.8):
self.low = low
self.high = high
def __call__(self, image: Image, **kwargs):
radius = np.random.uniform(low=self.low, high=self.high)
image = image.filter(ImageFilter.GaussianBlur(radius))
return image, kwargs
class Resize(object):
"""Resize the input PIL Image to the given size.
"""
def __init__(self, size, interpolation=Image.BILINEAR):
assert isinstance(size, int)
self.size = size
self.interpolation = interpolation
def __call__(self, image, keypoint2d: np.ndarray, intrinsic_matrix: np.ndarray, **kwargs):
image, keypoint2d, intrinsic_matrix = resize(image, self.size, self.interpolation, keypoint2d, intrinsic_matrix)
kwargs.update(keypoint2d=keypoint2d, intrinsic_matrix=intrinsic_matrix)
if 'depth' in kwargs:
kwargs['depth'] = F.resize(kwargs['depth'], self.size)
return image, kwargs
class ResizePad(object):
"""Pad the given image on all sides with the given "pad" value to resize the image to the given size.
"""
def __init__(self, size, interpolation=Image.BILINEAR):
self.size = size
self.interpolation = interpolation
def __call__(self, img, keypoint2d, **kwargs):
image, keypoint2d = resize_pad(img, keypoint2d, self.size, self.interpolation)
kwargs.update(keypoint2d=keypoint2d)
return image, kwargs
class CenterCrop(object):
"""Crops the given PIL Image at the center.
"""
def __init__(self, size):
if isinstance(size, numbers.Number):
self.size = (int(size), int(size))
else:
self.size = size
def __call__(self, image, keypoint2d, **kwargs):
"""
Args:
img (PIL Image): Image to be cropped.
Returns:
PIL Image: Cropped image.
"""
image, keypoint2d = center_crop(image, self.size, keypoint2d)
kwargs.update(keypoint2d=keypoint2d)
if 'depth' in kwargs:
kwargs['depth'] = F.center_crop(kwargs['depth'], self.size)
return image, kwargs
class RandomRotation(object):
"""Rotate the image by angle.
Args:
degrees (sequence or float or int): Range of degrees to select from.
If degrees is a number instead of sequence like (min, max), the range of degrees
will be (-degrees, +degrees).
"""
def __init__(self, degrees):
if isinstance(degrees, numbers.Number):
if degrees < 0:
raise ValueError("If degrees is a single number, it must be positive.")
self.degrees = (-degrees, degrees)
else:
if len(degrees) != 2:
raise ValueError("If degrees is a sequence, it must be of len 2.")
self.degrees = degrees
@staticmethod
def get_params(degrees):
"""Get parameters for ``rotate`` for a random rotation.
Returns:
sequence: params to be passed to ``rotate`` for random rotation.
"""
angle = random.uniform(degrees[0], degrees[1])
return angle
def __call__(self, image, keypoint2d, **kwargs):
"""
Args:
img (PIL Image): Image to be rotated.
Returns:
PIL Image: Rotated image.
"""
angle = self.get_params(self.degrees)
image, keypoint2d = rotate(image, angle, keypoint2d)
kwargs.update(keypoint2d=keypoint2d)
if 'depth' in kwargs:
kwargs['depth'] = F.rotate(kwargs['depth'], angle)
return image, kwargs
class RandomResizedCrop(object):
"""Crop the given PIL Image to random size and aspect ratio.
A crop of random size (default: of 0.08 to 1.0) of the original size and a random
aspect ratio (default: of 3/4 to 4/3) of the original aspect ratio is made. This crop
is finally resized to given size.
This is popularly used to train the Inception networks.
Args:
size: expected output size of each edge
scale: range of size of the origin size cropped
ratio: range of aspect ratio of the origin aspect ratio cropped
interpolation: Default: PIL.Image.BILINEAR
"""
def __init__(self, size, scale=(0.6, 1.3), interpolation=Image.BILINEAR):
self.size = size
if scale[0] > scale[1]:
warnings.warn("range should be of kind (min, max)")
self.interpolation = interpolation
self.scale = scale
@staticmethod
def get_params(img, scale):
"""Get parameters for ``crop`` for a random sized crop.
Args:
img (PIL Image): Image to be cropped.
scale (tuple): range of size of the origin size cropped
Returns:
tuple: params (i, j, h, w) to be passed to ``crop`` for a random
sized crop.
"""
width, height = img.size
area = height * width
for attempt in range(10):
target_area = random.uniform(*scale) * area
aspect_ratio = 1
w = int(round(math.sqrt(target_area * aspect_ratio)))
h = int(round(math.sqrt(target_area / aspect_ratio)))
if 0 < w <= width and 0 < h <= height:
i = random.randint(0, height - h)
j = random.randint(0, width - w)
return i, j, h, w
# Fallback to whole image
return 0, 0, height, width
def __call__(self, image, keypoint2d: np.ndarray, intrinsic_matrix: np.ndarray, **kwargs):
"""
Args:
img (PIL Image): Image to be cropped and resized.
Returns:
PIL Image: Randomly cropped and resized image.
"""
i, j, h, w = self.get_params(image, self.scale)
image, keypoint2d, intrinsic_matrix = resized_crop(image, i, j, h, w, self.size, self.interpolation, keypoint2d, intrinsic_matrix)
kwargs.update(keypoint2d=keypoint2d, intrinsic_matrix=intrinsic_matrix)
if 'depth' in kwargs:
kwargs['depth'] = F.resized_crop(kwargs['depth'], i, j, h, w, self.size, self.interpolation,)
return image, kwargs
class RandomApply(T.RandomTransforms):
"""Apply randomly a list of transformations with a given probability.
Args:
transforms (list or tuple or torch.nn.Module): list of transformations
p (float): probability
"""
def __init__(self, transforms, p=0.5):
super(RandomApply, self).__init__(transforms)
self.p = p
def __call__(self, image, **kwargs):
if self.p < random.random():
return image, kwargs
for t in self.transforms:
image, kwargs = t(image, **kwargs)
return image, kwargs
================================================
FILE: domain_generalization/Transfer-Learning-Library/common/vision/transforms/segmentation.py
================================================
"""
@author: Junguang Jiang
@contact: JiangJunguang1123@outlook.com
"""
from PIL import Image
import random
import math
from typing import ClassVar, Sequence, List, Tuple
from torch import Tensor
import torch
import torchvision.transforms.functional as F
import torchvision.transforms.transforms as T
import torch.nn as nn
from . import MultipleApply as MultipleApplyBase, NormalizeAndTranspose as NormalizeAndTransposeBase
def wrapper(transform: ClassVar):
""" Wrap a transform for classification to a transform for segmentation.
Note that the segmentation label will keep the same before and after wrapper.
Args:
transform (class, callable): transform for classification
Returns:
transform for segmentation
"""
class WrapperTransform(transform):
def __call__(self, image, label):
image = super().__call__(image)
return image, label
return WrapperTransform
ColorJitter = wrapper(T.ColorJitter)
Normalize = wrapper(T.Normalize)
ToTensor = wrapper(T.ToTensor)
ToPILImage = wrapper(T.ToPILImage)
MultipleApply = wrapper(MultipleApplyBase)
NormalizeAndTranspose = wrapper(NormalizeAndTransposeBase)
class Compose:
"""Composes several transforms together.
Args:
transforms (list): list of transforms to compose.
Example:
>>> Compose([
>>> Resize((512, 512)),
>>> RandomHorizontalFlip()
>>> ])
"""
def __init__(self, transforms):
super(Compose, self).__init__()
self.transforms = transforms
def __call__(self, image, target):
for t in self.transforms:
image, target = t(image, target)
return image, target
class Resize(nn.Module):
"""Resize the input image and the corresponding label to the given size.
The image should be a PIL Image.
Args:
image_size (sequence): The requested image size in pixels, as a 2-tuple:
(width, height).
label_size (sequence, optional): The requested segmentation label size in pixels, as a 2-tuple:
(width, height). The same as image_size if None. Default: None.
"""
def __init__(self, image_size, label_size=None):
super(Resize, self).__init__()
self.image_size = image_size
if label_size is None:
self.label_size = image_size
else:
self.label_size = label_size
def forward(self, image, label):
"""
Args:
image: (PIL Image): Image to be scaled.
label: (PIL Image): Segmentation label to be scaled.
Returns:
Rescaled image, rescaled segmentation label
"""
# resize
image = image.resize(self.image_size, Image.BICUBIC)
label = label.resize(self.label_size, Image.NEAREST)
return image, label
class RandomCrop(nn.Module):
"""Crop the given image at a random location.
The image can be a PIL Image
Args:
size (sequence): Desired output size of the crop.
"""
def __init__(self, size):
super(RandomCrop, self).__init__()
self.size = size
def forward(self, image, label):
"""
Args:
image: (PIL Image): Image to be cropped.
label: (PIL Image): Segmentation label to be cropped.
Returns:
Cropped image, cropped segmentation label.
"""
# random crop
left = image.size[0] - self.size[0]
upper = image.size[1] - self.size[1]
left = random.randint(0, left-1)
upper = random.randint(0, upper-1)
right = left + self.size[0]
lower = upper + self.size[1]
image = image.crop((left, upper, right, lower))
label = label.crop((left, upper, right, lower))
return image, label
class RandomHorizontalFlip(nn.Module):
"""Horizontally flip the given PIL Image randomly with a given probability.
Args:
p (float): probability of the image being flipped. Default value is 0.5
"""
def __init__(self, p=0.5):
super(RandomHorizontalFlip, self).__init__()
self.p = p
def forward(self, image, label):
"""
Args:
image: (PIL Image): Image to be flipped.
label: (PIL Image): Segmentation label to be flipped.
Returns:
Randomly flipped image, randomly flipped segmentation label.
"""
if random.random() < self.p:
return F.hflip(image), F.hflip(label)
return image, label
class RandomResizedCrop(T.RandomResizedCrop):
"""Crop the given image to random size and aspect ratio.
The image can be a PIL Image.
A crop of random size (default: of 0.5 to 1.0) of the original size and a random
aspect ratio (default: of 3/4 to 4/3) of the original aspect ratio is made. This crop
is finally resized to given size.
Args:
size (int or sequence): expected output size of each edge. If size is an
int instead of sequence like (h, w), a square output size ``(size, size)`` is
made. If provided a tuple or list of length 1, it will be interpreted as (size[0], size[0]).
scale (tuple of float): range of size of the origin size cropped
ratio (tuple of float): range of aspect ratio of the origin aspect ratio cropped.
interpolation: Default: PIL.Image.BILINEAR
"""
def __init__(self, size, scale=(0.5, 1.0), ratio=(3. / 4., 4. / 3.), interpolation=Image.BICUBIC):
super(RandomResizedCrop, self).__init__(size, scale, ratio, interpolation)
@staticmethod
def get_params(
img: Tensor, scale: List[float], ratio: List[float]
) -> Tuple[int, int, int, int]:
"""Get parameters for ``crop`` for a random sized crop.
Args:
img (PIL Image): Input image.
scale (list): range of scale of the origin size cropped
ratio (list): range of aspect ratio of the origin aspect ratio cropped
Returns:
params (i, j, h, w) to be passed to ``crop`` for a random sized crop.
"""
width, height = F._get_image_size(img)
area = height * width
for _ in range(10):
target_area = area * random.uniform(scale[0], scale[1])
log_ratio = torch.log(torch.tensor(ratio))
aspect_ratio = math.exp(random.uniform(log_ratio[0], log_ratio[1]))
w = int(round(math.sqrt(target_area * aspect_ratio)))
h = int(round(math.sqrt(target_area / aspect_ratio)))
if 0 < w <= width and 0 < h <= height:
i = random.randint(0, height - h)
j = random.randint(0, width - w)
return i, j, h, w
# Fallback to central crop
in_ratio = float(width) / float(height)
if in_ratio < min(ratio):
w = width
h = int(round(w / min(ratio)))
elif in_ratio > max(ratio):
h = height
w = int(round(h * max(ratio)))
else: # whole image
w = width
h = height
i = (height - h) // 2
j = (width - w) // 2
return i, j, h, w
def forward(self, image, label):
"""
Args:
image: (PIL Image): Image to be cropped and resized.
label: (PIL Image): Segmentation label to be cropped and resized.
Returns:
Randomly cropped and resized image, randomly cropped and resized segmentation label.
"""
top, left, height, width = self.get_params(image, self.scale, self.ratio)
image = image.crop((left, top, left + width, top + height))
image = image.resize(self.size, self.interpolation)
label = label.crop((left, top, left + width, top + height))
label = label.resize(self.size, Image.NEAREST)
return image, label
class RandomChoice(T.RandomTransforms):
"""Apply single transformation randomly picked from a list.
"""
def __call__(self, image, label):
t = random.choice(self.transforms)
return t(image, label)
class RandomApply(T.RandomTransforms):
"""Apply randomly a list of transformations with a given probability.
Args:
transforms (list or tuple or torch.nn.Module): list of transformations
p (float): probability
"""
def __init__(self, transforms, p=0.5):
super(RandomApply, self).__init__(transforms)
self.p = p
def __call__(self, image, label):
if self.p < random.random():
return image
for t in self.transforms:
image, label = t(image, label)
return image
================================================
FILE: domain_generalization/Transfer-Learning-Library/dalib/__init__.py
================================================
__all__ = ['adaptation', 'modules', 'translation']
__version__ = '0.3'
================================================
FILE: domain_generalization/Transfer-Learning-Library/dalib/adaptation/__init__.py
================================================
from . import cdan
from . import dann
from . import mdd
from . import dan
from . import jan
from . import mcd
from . import mcc
from . import pada
from . import osbp
from . import iwan
from . import adda
from . import bsp
__all__ = ["cdan", "dann", "mdd", "dan", "jan", "mcd", "mcc", "pada", "osbp", "iwan", "adda", "bsp"]
================================================
FILE: domain_generalization/Transfer-Learning-Library/dalib/adaptation/adda.py
================================================
"""
@author: Baixu Chen
@contact: cbx_99_hasta@outlook.com
"""
from typing import Optional, List, Dict
import torch
import torch.nn as nn
import torch.nn.functional as F
from common.modules.classifier import Classifier as ClassifierBase
class DomainAdversarialLoss(nn.Module):
r"""Domain adversarial loss from `Adversarial Discriminative Domain Adaptation (CVPR 2017)
`_.
Similar to the original `GAN `_ paper, ADDA argues that replacing
:math:`\text{log}(1-p)` with :math:`-\text{log}(p)` in the adversarial loss provides better gradient qualities. Detailed
optimization process can be found `here
`_.
Inputs:
- domain_pred (tensor): predictions of domain discriminator
- domain_label (str, optional): whether the data comes from source or target.
Must be 'source' or 'target'. Default: 'source'
Shape:
- domain_pred: :math:`(minibatch,)`.
- Outputs: scalar.
"""
def __init__(self):
super(DomainAdversarialLoss, self).__init__()
def forward(self, domain_pred, domain_label='source'):
assert domain_label in ['source', 'target']
if domain_label == 'source':
return F.binary_cross_entropy(domain_pred, torch.ones_like(domain_pred).to(domain_pred.device))
else:
return F.binary_cross_entropy(domain_pred, torch.zeros_like(domain_pred).to(domain_pred.device))
class ImageClassifier(ClassifierBase):
def __init__(self, backbone: nn.Module, num_classes: int, bottleneck_dim: Optional[int] = 256, **kwargs):
bottleneck = nn.Sequential(
# nn.AdaptiveAvgPool2d(output_size=(1, 1)),
# nn.Flatten(),
nn.Linear(backbone.out_features, bottleneck_dim),
nn.BatchNorm1d(bottleneck_dim),
nn.ReLU()
)
super(ImageClassifier, self).__init__(backbone, num_classes, bottleneck, bottleneck_dim, **kwargs)
def freeze_bn(self):
for m in self.modules():
if isinstance(m, nn.BatchNorm2d) or isinstance(m, nn.BatchNorm1d):
m.eval()
def get_parameters(self, base_lr=1.0, optimize_head=True) -> List[Dict]:
params = [
{"params": self.backbone.parameters(), "lr": 0.1 * base_lr if self.finetune else 1.0 * base_lr},
{"params": self.bottleneck.parameters(), "lr": 1.0 * base_lr}
]
if optimize_head:
params.append({"params": self.head.parameters(), "lr": 1.0 * base_lr})
return params
================================================
FILE: domain_generalization/Transfer-Learning-Library/dalib/adaptation/advent.py
================================================
"""
@author: Junguang Jiang
@contact: JiangJunguang1123@outlook.com
"""
from torch import nn
import torch
import torch.nn.functional as F
import numpy as np
class Discriminator(nn.Sequential):
"""
Domain discriminator model from
`ADVENT: Adversarial Entropy Minimization for Domain Adaptation in Semantic Segmentation (CVPR 2019) `_
Distinguish pixel-by-pixel whether the input predictions come from the source domain or the target domain.
The source domain label is 1 and the target domain label is 0.
Args:
num_classes (int): num of classes in the predictions
ndf (int): dimension of the hidden features
Shape:
- Inputs: :math:`(minibatch, C, H, W)` where :math:`C` is the number of classes
- Outputs: :math:`(minibatch, 1, H, W)`
"""
def __init__(self, num_classes, ndf=64):
super(Discriminator, self).__init__(
nn.Conv2d(num_classes, ndf, kernel_size=4, stride=2, padding=1),
nn.LeakyReLU(negative_slope=0.2, inplace=True),
nn.Conv2d(ndf, ndf * 2, kernel_size=4, stride=2, padding=1),
nn.LeakyReLU(negative_slope=0.2, inplace=True),
nn.Conv2d(ndf * 2, ndf * 4, kernel_size=4, stride=2, padding=1),
nn.LeakyReLU(negative_slope=0.2, inplace=True),
nn.Conv2d(ndf * 4, ndf * 8, kernel_size=4, stride=2, padding=1),
nn.LeakyReLU(negative_slope=0.2, inplace=True),
nn.Conv2d(ndf * 8, 1, kernel_size=4, stride=2, padding=1),
)
def prob_2_entropy(prob):
""" convert probabilistic prediction maps to weighted self-information maps
"""
n, c, h, w = prob.size()
return -torch.mul(prob, torch.log2(prob + 1e-30)) / np.log2(c)
def bce_loss(y_pred, y_label):
y_truth_tensor = torch.FloatTensor(y_pred.size())
y_truth_tensor.fill_(y_label)
y_truth_tensor = y_truth_tensor.to(y_pred.get_device())
return F.binary_cross_entropy_with_logits(y_pred, y_truth_tensor)
class DomainAdversarialEntropyLoss(nn.Module):
r"""The `Domain Adversarial Entropy Loss `_
Minimizing entropy with adversarial learning through training a domain discriminator.
Args:
domain_discriminator (torch.nn.Module): A domain discriminator object, which predicts
the domains of predictions. Its input shape is :math:`(minibatch, C, H, W)` and output shape is :math:`(minibatch, 1, H, W)`
Inputs:
- logits (tensor): logits output of segmentation model
- domain_label (str, optional): whether the data comes from source or target.
Choices: ['source', 'target']. Default: 'source'
Shape:
- logits: :math:`(minibatch, C, H, W)` where :math:`C` means the number of classes
- Outputs: scalar.
Examples::
>>> B, C, H, W = 2, 19, 512, 512
>>> discriminator = Discriminator(num_classes=C)
>>> dann = DomainAdversarialEntropyLoss(discriminator)
>>> # logits output on source domain and target domain
>>> y_s, y_t = torch.randn(B, C, H, W), torch.randn(B, C, H, W)
>>> loss = 0.5 * (dann(y_s, "source") + dann(y_t, "target"))
"""
def __init__(self, discriminator: nn.Module):
super(DomainAdversarialEntropyLoss, self).__init__()
self.discriminator = discriminator
def forward(self, logits, domain_label='source'):
"""
"""
assert domain_label in ['source', 'target']
probability = F.softmax(logits, dim=1)
entropy = prob_2_entropy(probability)
domain_prediciton = self.discriminator(entropy)
if domain_label == 'source':
return bce_loss(domain_prediciton, 1)
else:
return bce_loss(domain_prediciton, 0)
def train(self, mode=True):
r"""Sets the discriminator in training mode. In the training mode,
all the parameters in discriminator will be set requires_grad=True.
Args:
mode (bool): whether to set training mode (``True``) or evaluation mode (``False``). Default: ``True``.
"""
self.discriminator.train(mode)
for param in self.discriminator.parameters():
param.requires_grad = mode
return self
def eval(self):
r"""Sets the module in evaluation mode. In the training mode,
all the parameters in discriminator will be set requires_grad=False.
This is equivalent with :meth:`self.train(False) `.
"""
return self.train(False)
================================================
FILE: domain_generalization/Transfer-Learning-Library/dalib/adaptation/afn.py
================================================
"""
Modified from https://github.com/jihanyang/AFN
@author: Baixu Chen
@contact: cbx_99_hasta@outlook.com
"""
from typing import Optional, List, Dict
import torch
import torch.nn as nn
import math
from common.modules.classifier import Classifier as ClassfierBase
class AdaptiveFeatureNorm(nn.Module):
r"""
The `Stepwise Adaptive Feature Norm loss (ICCV 2019) `_
Instead of using restrictive scalar R to match the corresponding feature norm, Stepwise Adaptive Feature Norm
is used in order to learn task-specific features with large norms in a progressive manner.
We denote parameters of backbone :math:`G` as :math:`\theta_g`, parameters of bottleneck :math:`F_f` as :math:`\theta_f`
, parameters of classifier head :math:`F_y` as :math:`\theta_y`, and features extracted from sample :math:`x_i` as
:math:`h(x_i;\theta)`. Full loss is calculated as follows
.. math::
L(\theta_g,\theta_f,\theta_y)=\frac{1}{n_s}\sum_{(x_i,y_i)\in D_s}L_y(x_i,y_i)+\frac{\lambda}{n_s+n_t}
\sum_{x_i\in D_s\cup D_t}L_d(h(x_i;\theta_0)+\Delta_r,h(x_i;\theta))\\
where :math:`L_y` denotes classification loss, :math:`L_d` denotes norm loss, :math:`\theta_0` and :math:`\theta`
represent the updated and updating model parameters in the last and current iterations respectively.
Args:
delta (float): positive residual scalar to control the feature norm enlargement.
Inputs:
- f (tensor): feature representations on source or target domain.
Shape:
- f: :math:`(N, F)` where F means the dimension of input features.
- Outputs: scalar.
Examples::
>>> adaptive_feature_norm = AdaptiveFeatureNorm(delta=1)
>>> f_s = torch.randn(32, 1000)
>>> f_t = torch.randn(32, 1000)
>>> norm_loss = adaptive_feature_norm(f_s) + adaptive_feature_norm(f_t)
"""
def __init__(self, delta):
super(AdaptiveFeatureNorm, self).__init__()
self.delta = delta
def forward(self, f: torch.Tensor) -> torch.Tensor:
radius = f.norm(p=2, dim=1).detach()
assert radius.requires_grad == False
radius = radius + self.delta
loss = ((f.norm(p=2, dim=1) - radius) ** 2).mean()
return loss
class Block(nn.Module):
r"""
Basic building block for Image Classifier with structure: FC-BN-ReLU-Dropout.
We use :math:`L_2` preserved dropout layers.
Given mask probability :math:`p`, input :math:`x_k`, generated mask :math:`a_k`,
vanilla dropout layers calculate
.. math::
\hat{x}_k = a_k\frac{1}{1-p}x_k\\
While in :math:`L_2` preserved dropout layers
.. math::
\hat{x}_k = a_k\frac{1}{\sqrt{1-p}}x_k\\
Args:
in_features (int): Dimension of input features
bottleneck_dim (int, optional): Feature dimension of the bottleneck layer. Default: 1000
dropout_p (float, optional): dropout probability. Default: 0.5
"""
def __init__(self, in_features: int, bottleneck_dim: Optional[int] = 1000, dropout_p: Optional[float] = 0.5):
super(Block, self).__init__()
self.fc = nn.Linear(in_features, bottleneck_dim)
self.bn = nn.BatchNorm1d(bottleneck_dim, affine=True)
self.relu = nn.ReLU(inplace=True)
self.dropout = nn.Dropout(dropout_p)
self.dropout_p = dropout_p
def forward(self, x: torch.Tensor) -> torch.Tensor:
f = self.fc(x)
f = self.bn(f)
f = self.relu(f)
f = self.dropout(f)
if self.training:
f.mul_(math.sqrt(1 - self.dropout_p))
return f
class ImageClassifier(ClassfierBase):
r"""
ImageClassifier for AFN.
Args:
backbone (torch.nn.Module): Any backbone to extract 2-d features from data
num_classes (int): Number of classes
num_blocks (int, optional): Number of basic blocks. Default: 1
bottleneck_dim (int, optional): Feature dimension of the bottleneck layer. Default: 1000
dropout_p (float, optional): dropout probability. Default: 0.5
"""
def __init__(self, backbone: nn.Module, num_classes: int, num_blocks: Optional[int] = 1,
bottleneck_dim: Optional[int] = 1000, dropout_p: Optional[float] = 0.5, **kwargs):
assert num_blocks >= 1
layers = [nn.Sequential(
Block(backbone.out_features, bottleneck_dim, dropout_p)
)]
for _ in range(num_blocks - 1):
layers.append(Block(bottleneck_dim, bottleneck_dim, dropout_p))
bottleneck = nn.Sequential(*layers)
super(ImageClassifier, self).__init__(backbone, num_classes, bottleneck, bottleneck_dim, **kwargs)
# init parameters for bottleneck and head
for m in self.bottleneck.modules():
if isinstance(m, nn.BatchNorm1d):
m.weight.data.normal_(1.0, 0.01)
m.bias.data.fill_(0)
if isinstance(m, nn.Linear):
m.weight.data.normal_(0.0, 0.01)
m.bias.data.normal_(0.0, 0.01)
for m in self.head.modules():
if isinstance(m, nn.Linear):
m.weight.data.normal_(0.0, 0.01)
m.bias.data.normal_(0.0, 0.01)
def get_parameters(self, base_lr=1.0) -> List[Dict]:
params = [
{"params": self.backbone.parameters()},
{"params": self.bottleneck.parameters(), "momentum": 0.9},
{"params": self.head.parameters(), "momentum": 0.9},
]
return params
================================================
FILE: domain_generalization/Transfer-Learning-Library/dalib/adaptation/bsp.py
================================================
"""
@author: Baixu Chen
@contact: cbx_99_hasta@outlook.com
"""
from typing import Optional
import torch
import torch.nn as nn
from common.modules.classifier import Classifier as ClassifierBase
class BatchSpectralPenalizationLoss(nn.Module):
r"""Batch spectral penalization loss from `Transferability vs. Discriminability: Batch
Spectral Penalization for Adversarial Domain Adaptation (ICML 2019)
`_.
Given source features :math:`f_s` and target features :math:`f_t` in current mini batch, singular value
decomposition is first performed
.. math::
f_s = U_s\Sigma_sV_s^T
.. math::
f_t = U_t\Sigma_tV_t^T
Then batch spectral penalization loss is calculated as
.. math::
loss=\sum_{i=1}^k(\sigma_{s,i}^2+\sigma_{t,i}^2)
where :math:`\sigma_{s,i},\sigma_{t,i}` refer to the :math:`i-th` largest singular value of source features
and target features respectively. We empirically set :math:`k=1`.
Inputs:
- f_s (tensor): feature representations on source domain, :math:`f^s`
- f_t (tensor): feature representations on target domain, :math:`f^t`
Shape:
- f_s, f_t: :math:`(N, F)` where F means the dimension of input features.
- Outputs: scalar.
"""
def __init__(self):
super(BatchSpectralPenalizationLoss, self).__init__()
def forward(self, f_s, f_t):
_, s_s, _ = torch.svd(f_s)
_, s_t, _ = torch.svd(f_t)
loss = torch.pow(s_s[0], 2) + torch.pow(s_t[0], 2)
return loss
class ImageClassifier(ClassifierBase):
def __init__(self, backbone: nn.Module, num_classes: int, bottleneck_dim: Optional[int] = 256, **kwargs):
bottleneck = nn.Sequential(
# nn.AdaptiveAvgPool2d(output_size=(1, 1)),
# nn.Flatten(),
nn.Linear(backbone.out_features, bottleneck_dim),
nn.BatchNorm1d(bottleneck_dim),
nn.ReLU(),
)
super(ImageClassifier, self).__init__(backbone, num_classes, bottleneck, bottleneck_dim, **kwargs)
================================================
FILE: domain_generalization/Transfer-Learning-Library/dalib/adaptation/cdan.py
================================================
"""
@author: Junguang Jiang
@contact: JiangJunguang1123@outlook.com
"""
from typing import Optional
import numpy as np
import torch
import torch.nn as nn
import torch.nn.functional as F
from common.modules.classifier import Classifier as ClassifierBase
from common.utils.metric import binary_accuracy
from ..modules.grl import WarmStartGradientReverseLayer
from ..modules.entropy import entropy
__all__ = ['ConditionalDomainAdversarialLoss', 'ImageClassifier']
class ConditionalDomainAdversarialLoss(nn.Module):
r"""The Conditional Domain Adversarial Loss used in `Conditional Adversarial Domain Adaptation (NIPS 2018) `_
Conditional Domain adversarial loss measures the domain discrepancy through training a domain discriminator in a
conditional manner. Given domain discriminator :math:`D`, feature representation :math:`f` and
classifier predictions :math:`g`, the definition of CDAN loss is
.. math::
loss(\mathcal{D}_s, \mathcal{D}_t) &= \mathbb{E}_{x_i^s \sim \mathcal{D}_s} \text{log}[D(T(f_i^s, g_i^s))] \\
&+ \mathbb{E}_{x_j^t \sim \mathcal{D}_t} \text{log}[1-D(T(f_j^t, g_j^t))],\\
where :math:`T` is a :class:`MultiLinearMap` or :class:`RandomizedMultiLinearMap` which convert two tensors to a single tensor.
Args:
domain_discriminator (torch.nn.Module): A domain discriminator object, which predicts the domains of
features. Its input shape is (N, F) and output shape is (N, 1)
entropy_conditioning (bool, optional): If True, use entropy-aware weight to reweight each training example.
Default: False
randomized (bool, optional): If True, use `randomized multi linear map`. Else, use `multi linear map`.
Default: False
num_classes (int, optional): Number of classes. Default: -1
features_dim (int, optional): Dimension of input features. Default: -1
randomized_dim (int, optional): Dimension of features after randomized. Default: 1024
reduction (str, optional): Specifies the reduction to apply to the output:
``'none'`` | ``'mean'`` | ``'sum'``. ``'none'``: no reduction will be applied,
``'mean'``: the sum of the output will be divided by the number of
elements in the output, ``'sum'``: the output will be summed. Default: ``'mean'``
.. note::
You need to provide `num_classes`, `features_dim` and `randomized_dim` **only when** `randomized`
is set True.
Inputs:
- g_s (tensor): unnormalized classifier predictions on source domain, :math:`g^s`
- f_s (tensor): feature representations on source domain, :math:`f^s`
- g_t (tensor): unnormalized classifier predictions on target domain, :math:`g^t`
- f_t (tensor): feature representations on target domain, :math:`f^t`
Shape:
- g_s, g_t: :math:`(minibatch, C)` where C means the number of classes.
- f_s, f_t: :math:`(minibatch, F)` where F means the dimension of input features.
- Output: scalar by default. If :attr:`reduction` is ``'none'``, then :math:`(minibatch, )`.
Examples::
>>> from dalib.modules.domain_discriminator import DomainDiscriminator
>>> from dalib.adaptation.cdan import ConditionalDomainAdversarialLoss
>>> import torch
>>> num_classes = 2
>>> feature_dim = 1024
>>> batch_size = 10
>>> discriminator = DomainDiscriminator(in_feature=feature_dim * num_classes, hidden_size=1024)
>>> loss = ConditionalDomainAdversarialLoss(discriminator, reduction='mean')
>>> # features from source domain and target domain
>>> f_s, f_t = torch.randn(batch_size, feature_dim), torch.randn(batch_size, feature_dim)
>>> # logits output from source domain adn target domain
>>> g_s, g_t = torch.randn(batch_size, num_classes), torch.randn(batch_size, num_classes)
>>> output = loss(g_s, f_s, g_t, f_t)
"""
def __init__(self, domain_discriminator: nn.Module, entropy_conditioning: Optional[bool] = False,
randomized: Optional[bool] = False, num_classes: Optional[int] = -1,
features_dim: Optional[int] = -1, randomized_dim: Optional[int] = 1024,
reduction: Optional[str] = 'mean'):
super(ConditionalDomainAdversarialLoss, self).__init__()
self.domain_discriminator = domain_discriminator
self.grl = WarmStartGradientReverseLayer(alpha=1., lo=0., hi=1., max_iters=1000, auto_step=True)
self.entropy_conditioning = entropy_conditioning
if randomized:
assert num_classes > 0 and features_dim > 0 and randomized_dim > 0
self.map = RandomizedMultiLinearMap(features_dim, num_classes, randomized_dim)
else:
self.map = MultiLinearMap()
self.bce = lambda input, target, weight: F.binary_cross_entropy(input, target, weight,
reduction=reduction) if self.entropy_conditioning \
else F.binary_cross_entropy(input, target, reduction=reduction)
self.domain_discriminator_accuracy = None
def forward(self, g_s: torch.Tensor, f_s: torch.Tensor, g_t: torch.Tensor, f_t: torch.Tensor) -> torch.Tensor:
f = torch.cat((f_s, f_t), dim=0)
g = torch.cat((g_s, g_t), dim=0)
g = F.softmax(g, dim=1).detach()
h = self.grl(self.map(f, g))
d = self.domain_discriminator(h)
d_label = torch.cat((
torch.ones((g_s.size(0), 1)).to(g_s.device),
torch.zeros((g_t.size(0), 1)).to(g_t.device),
))
weight = 1.0 + torch.exp(-entropy(g))
batch_size = f.size(0)
weight = weight / torch.sum(weight) * batch_size
self.domain_discriminator_accuracy = binary_accuracy(d, d_label)
return self.bce(d, d_label, weight.view_as(d))
class RandomizedMultiLinearMap(nn.Module):
"""Random multi linear map
Given two inputs :math:`f` and :math:`g`, the definition is
.. math::
T_{\odot}(f,g) = \dfrac{1}{\sqrt{d}} (R_f f) \odot (R_g g),
where :math:`\odot` is element-wise product, :math:`R_f` and :math:`R_g` are random matrices
sampled only once and fixed in training.
Args:
features_dim (int): dimension of input :math:`f`
num_classes (int): dimension of input :math:`g`
output_dim (int, optional): dimension of output tensor. Default: 1024
Shape:
- f: (minibatch, features_dim)
- g: (minibatch, num_classes)
- Outputs: (minibatch, output_dim)
"""
def __init__(self, features_dim: int, num_classes: int, output_dim: Optional[int] = 1024):
super(RandomizedMultiLinearMap, self).__init__()
self.Rf = torch.randn(features_dim, output_dim)
self.Rg = torch.randn(num_classes, output_dim)
self.output_dim = output_dim
def forward(self, f: torch.Tensor, g: torch.Tensor) -> torch.Tensor:
f = torch.mm(f, self.Rf.to(f.device))
g = torch.mm(g, self.Rg.to(g.device))
output = torch.mul(f, g) / np.sqrt(float(self.output_dim))
return output
class MultiLinearMap(nn.Module):
"""Multi linear map
Shape:
- f: (minibatch, F)
- g: (minibatch, C)
- Outputs: (minibatch, F * C)
"""
def __init__(self):
super(MultiLinearMap, self).__init__()
def forward(self, f: torch.Tensor, g: torch.Tensor) -> torch.Tensor:
batch_size = f.size(0)
output = torch.bmm(g.unsqueeze(2), f.unsqueeze(1))
return output.view(batch_size, -1)
class ImageClassifier(ClassifierBase):
def __init__(self, backbone: nn.Module, num_classes: int, bottleneck_dim: Optional[int] = 256, **kwargs):
bottleneck = nn.Sequential(
# nn.AdaptiveAvgPool2d(output_size=(1, 1)),
# nn.Flatten(),
nn.Linear(backbone.out_features, bottleneck_dim),
nn.BatchNorm1d(bottleneck_dim),
nn.ReLU()
)
super(ImageClassifier, self).__init__(backbone, num_classes, bottleneck, bottleneck_dim, **kwargs)
================================================
FILE: domain_generalization/Transfer-Learning-Library/dalib/adaptation/dan.py
================================================
"""
@author: Junguang Jiang
@contact: JiangJunguang1123@outlook.com
"""
from typing import Optional, Sequence
import torch
import torch.nn as nn
from common.modules.classifier import Classifier as ClassifierBase
__all__ = ['MultipleKernelMaximumMeanDiscrepancy', 'ImageClassifier']
class MultipleKernelMaximumMeanDiscrepancy(nn.Module):
r"""The Multiple Kernel Maximum Mean Discrepancy (MK-MMD) used in
`Learning Transferable Features with Deep Adaptation Networks (ICML 2015) `_
Given source domain :math:`\mathcal{D}_s` of :math:`n_s` labeled points and target domain :math:`\mathcal{D}_t`
of :math:`n_t` unlabeled points drawn i.i.d. from P and Q respectively, the deep networks will generate
activations as :math:`\{z_i^s\}_{i=1}^{n_s}` and :math:`\{z_i^t\}_{i=1}^{n_t}`.
The MK-MMD :math:`D_k (P, Q)` between probability distributions P and Q is defined as
.. math::
D_k(P, Q) \triangleq \| E_p [\phi(z^s)] - E_q [\phi(z^t)] \|^2_{\mathcal{H}_k},
:math:`k` is a kernel function in the function space
.. math::
\mathcal{K} \triangleq \{ k=\sum_{u=1}^{m}\beta_{u} k_{u} \}
where :math:`k_{u}` is a single kernel.
Using kernel trick, MK-MMD can be computed as
.. math::
\hat{D}_k(P, Q) &=
\dfrac{1}{n_s^2} \sum_{i=1}^{n_s}\sum_{j=1}^{n_s} k(z_i^{s}, z_j^{s})\\
&+ \dfrac{1}{n_t^2} \sum_{i=1}^{n_t}\sum_{j=1}^{n_t} k(z_i^{t}, z_j^{t})\\
&- \dfrac{2}{n_s n_t} \sum_{i=1}^{n_s}\sum_{j=1}^{n_t} k(z_i^{s}, z_j^{t}).\\
Args:
kernels (tuple(torch.nn.Module)): kernel functions.
linear (bool): whether use the linear version of DAN. Default: False
Inputs:
- z_s (tensor): activations from the source domain, :math:`z^s`
- z_t (tensor): activations from the target domain, :math:`z^t`
Shape:
- Inputs: :math:`(minibatch, *)` where * means any dimension
- Outputs: scalar
.. note::
Activations :math:`z^{s}` and :math:`z^{t}` must have the same shape.
.. note::
The kernel values will add up when there are multiple kernels.
Examples::
>>> from dalib.modules.kernels import GaussianKernel
>>> feature_dim = 1024
>>> batch_size = 10
>>> kernels = (GaussianKernel(alpha=0.5), GaussianKernel(alpha=1.), GaussianKernel(alpha=2.))
>>> loss = MultipleKernelMaximumMeanDiscrepancy(kernels)
>>> # features from source domain and target domain
>>> z_s, z_t = torch.randn(batch_size, feature_dim), torch.randn(batch_size, feature_dim)
>>> output = loss(z_s, z_t)
"""
def __init__(self, kernels: Sequence[nn.Module], linear: Optional[bool] = False):
super(MultipleKernelMaximumMeanDiscrepancy, self).__init__()
self.kernels = kernels
self.index_matrix = None
self.linear = linear
def forward(self, z_s: torch.Tensor, z_t: torch.Tensor) -> torch.Tensor:
features = torch.cat([z_s, z_t], dim=0)
batch_size = int(z_s.size(0))
self.index_matrix = _update_index_matrix(batch_size, self.index_matrix, self.linear).to(z_s.device)
kernel_matrix = sum([kernel(features) for kernel in self.kernels]) # Add up the matrix of each kernel
# Add 2 / (n-1) to make up for the value on the diagonal
# to ensure loss is positive in the non-linear version
loss = (kernel_matrix * self.index_matrix).sum() + 2. / float(batch_size - 1)
return loss
def _update_index_matrix(batch_size: int, index_matrix: Optional[torch.Tensor] = None,
linear: Optional[bool] = True) -> torch.Tensor:
r"""
Update the `index_matrix` which convert `kernel_matrix` to loss.
If `index_matrix` is a tensor with shape (2 x batch_size, 2 x batch_size), then return `index_matrix`.
Else return a new tensor with shape (2 x batch_size, 2 x batch_size).
"""
if index_matrix is None or index_matrix.size(0) != batch_size * 2:
index_matrix = torch.zeros(2 * batch_size, 2 * batch_size)
if linear:
for i in range(batch_size):
s1, s2 = i, (i + 1) % batch_size
t1, t2 = s1 + batch_size, s2 + batch_size
index_matrix[s1, s2] = 1. / float(batch_size)
index_matrix[t1, t2] = 1. / float(batch_size)
index_matrix[s1, t2] = -1. / float(batch_size)
index_matrix[s2, t1] = -1. / float(batch_size)
else:
for i in range(batch_size):
for j in range(batch_size):
if i != j:
index_matrix[i][j] = 1. / float(batch_size * (batch_size - 1))
index_matrix[i + batch_size][j + batch_size] = 1. / float(batch_size * (batch_size - 1))
for i in range(batch_size):
for j in range(batch_size):
index_matrix[i][j + batch_size] = -1. / float(batch_size * batch_size)
index_matrix[i + batch_size][j] = -1. / float(batch_size * batch_size)
return index_matrix
class ImageClassifier(ClassifierBase):
def __init__(self, backbone: nn.Module, num_classes: int, bottleneck_dim: Optional[int] = 256, **kwargs):
bottleneck = nn.Sequential(
# nn.AdaptiveAvgPool2d(output_size=(1, 1)),
# nn.Flatten(),
nn.Linear(backbone.out_features, bottleneck_dim),
nn.ReLU(),
nn.Dropout(0.5)
)
super(ImageClassifier, self).__init__(backbone, num_classes, bottleneck, bottleneck_dim, **kwargs)
================================================
FILE: domain_generalization/Transfer-Learning-Library/dalib/adaptation/dann.py
================================================
"""
@author: Junguang Jiang
@contact: JiangJunguang1123@outlook.com
"""
from typing import Optional
import torch
import torch.nn as nn
import torch.nn.functional as F
from ..modules.grl import WarmStartGradientReverseLayer
from common.modules.classifier import Classifier as ClassifierBase
from common.utils.metric import binary_accuracy
__all__ = ['DomainAdversarialLoss']
class DomainAdversarialLoss(nn.Module):
r"""
The Domain Adversarial Loss proposed in
`Domain-Adversarial Training of Neural Networks (ICML 2015) `_
Domain adversarial loss measures the domain discrepancy through training a domain discriminator.
Given domain discriminator :math:`D`, feature representation :math:`f`, the definition of DANN loss is
.. math::
loss(\mathcal{D}_s, \mathcal{D}_t) = \mathbb{E}_{x_i^s \sim \mathcal{D}_s} \text{log}[D(f_i^s)]
+ \mathbb{E}_{x_j^t \sim \mathcal{D}_t} \text{log}[1-D(f_j^t)].
Args:
domain_discriminator (torch.nn.Module): A domain discriminator object, which predicts the domains of features. Its input shape is (N, F) and output shape is (N, 1)
reduction (str, optional): Specifies the reduction to apply to the output:
``'none'`` | ``'mean'`` | ``'sum'``. ``'none'``: no reduction will be applied,
``'mean'``: the sum of the output will be divided by the number of
elements in the output, ``'sum'``: the output will be summed. Default: ``'mean'``
grl (WarmStartGradientReverseLayer, optional): Default: None.
Inputs:
- f_s (tensor): feature representations on source domain, :math:`f^s`
- f_t (tensor): feature representations on target domain, :math:`f^t`
- w_s (tensor, optional): a rescaling weight given to each instance from source domain.
- w_t (tensor, optional): a rescaling weight given to each instance from target domain.
Shape:
- f_s, f_t: :math:`(N, F)` where F means the dimension of input features.
- Outputs: scalar by default. If :attr:`reduction` is ``'none'``, then :math:`(N, )`.
Examples::
>>> from dalib.modules.domain_discriminator import DomainDiscriminator
>>> discriminator = DomainDiscriminator(in_feature=1024, hidden_size=1024)
>>> loss = DomainAdversarialLoss(discriminator, reduction='mean')
>>> # features from source domain and target domain
>>> f_s, f_t = torch.randn(20, 1024), torch.randn(20, 1024)
>>> # If you want to assign different weights to each instance, you should pass in w_s and w_t
>>> w_s, w_t = torch.randn(20), torch.randn(20)
>>> output = loss(f_s, f_t, w_s, w_t)
"""
def __init__(self, domain_discriminator: nn.Module, reduction: Optional[str] = 'mean',
grl: Optional = None):
super(DomainAdversarialLoss, self).__init__()
self.grl = WarmStartGradientReverseLayer(alpha=1., lo=0., hi=1., max_iters=1000, auto_step=True) if grl is None else grl
self.domain_discriminator = domain_discriminator
self.bce = lambda input, target, weight: \
F.binary_cross_entropy(input, target, weight=weight, reduction=reduction)
self.domain_discriminator_accuracy = None
def forward(self, f_s: torch.Tensor, f_t: torch.Tensor,
w_s: Optional[torch.Tensor] = None, w_t: Optional[torch.Tensor] = None) -> torch.Tensor:
f = self.grl(torch.cat((f_s, f_t), dim=0))
d = self.domain_discriminator(f)
d_s, d_t = d.chunk(2, dim=0)
d_label_s = torch.ones((f_s.size(0), 1)).to(f_s.device)
d_label_t = torch.zeros((f_t.size(0), 1)).to(f_t.device)
self.domain_discriminator_accuracy = 0.5 * (binary_accuracy(d_s, d_label_s) + binary_accuracy(d_t, d_label_t))
if w_s is None:
w_s = torch.ones_like(d_label_s)
if w_t is None:
w_t = torch.ones_like(d_label_t)
return 0.5 * (self.bce(d_s, d_label_s, w_s.view_as(d_s)) + self.bce(d_t, d_label_t, w_t.view_as(d_t)))
class ImageClassifier(ClassifierBase):
def __init__(self, backbone: nn.Module, num_classes: int, bottleneck_dim: Optional[int] = 256, **kwargs):
bottleneck = nn.Sequential(
# nn.AdaptiveAvgPool2d(output_size=(1, 1)),
# nn.Flatten(),
nn.Linear(backbone.out_features, bottleneck_dim),
nn.BatchNorm1d(bottleneck_dim),
nn.ReLU()
)
super(ImageClassifier, self).__init__(backbone, num_classes, bottleneck, bottleneck_dim, **kwargs)
================================================
FILE: domain_generalization/Transfer-Learning-Library/dalib/adaptation/fda.py
================================================
"""
@author: Junguang Jiang
@contact: JiangJunguang1123@outlook.com
"""
import torch.nn.functional as F
import math
def robust_entropy(y, ita=1.5, num_classes=19, reduction='mean'):
""" Robust entropy proposed in `FDA: Fourier Domain Adaptation for Semantic Segmentation (CVPR 2020) `_
Args:
y (tensor): logits output of segmentation model in shape of :math:`(N, C, H, W)`
ita (float, optional): parameters for robust entropy. Default: 1.5
num_classes (int, optional): number of classes. Default: 19
reduction (string, optional): Specifies the reduction to apply to the output:
``'none'`` | ``'mean'``. ``'none'``: no reduction will be applied,
``'mean'``: the sum of the output will be divided by the number of
elements in the output. Default: ``'mean'``
Returns:
Scalar by default. If :attr:`reduction` is ``'none'``, then :math:`(N, )`.
"""
P = F.softmax(y, dim=1)
logP = F.log_softmax(y, dim=1)
PlogP = P * logP
ent = -1.0 * PlogP.sum(dim=1)
ent = ent / math.log(num_classes)
# compute robust entropy
ent = ent ** 2.0 + 1e-8
ent = ent ** ita
if reduction == 'mean':
return ent.mean()
else:
return ent
================================================
FILE: domain_generalization/Transfer-Learning-Library/dalib/adaptation/idm/__init__.py
================================================
================================================
FILE: domain_generalization/Transfer-Learning-Library/dalib/adaptation/idm/loss.py
================================================
"""
Modified from https://github.com/SikaStar/IDM
@author: Baixu Chen
@contact: cbx_99_hasta@outlook.com
"""
import torch
import torch.nn as nn
class DivLoss(nn.Module):
"""Diversity loss, which is defined as negative of standard deviation.
"""
def __init__(self, ):
super(DivLoss, self).__init__()
def forward(self, lam):
mu = lam.mean(0)
std = ((lam - mu) ** 2).mean(0, keepdim=True).clamp(min=1e-12).sqrt()
loss_std = -std.sum()
return loss_std
class BridgeFeatLoss(nn.Module):
"""Bridge loss on feature space.
"""
def __init__(self):
super(BridgeFeatLoss, self).__init__()
def forward(self, f_s, f_t, f_mixed, lam):
dist_mixed2s = ((f_mixed - f_s) ** 2).sum(1, keepdim=True)
dist_mixed2t = ((f_mixed - f_t) ** 2).sum(1, keepdim=True)
dist_mixed2s = dist_mixed2s.clamp(min=1e-12).sqrt()
dist_mixed2t = dist_mixed2t.clamp(min=1e-12).sqrt()
dist_mixed = torch.cat((dist_mixed2s, dist_mixed2t), 1)
lam_dist_mixed = (lam * dist_mixed).sum(1, keepdim=True)
loss = lam_dist_mixed.mean()
return loss
class BridgeProbLoss(nn.Module):
"""Bridge loss on prediction space.
"""
def __init__(self, num_classes, epsilon=0.1):
super(BridgeProbLoss, self).__init__()
self.num_classes = num_classes
self.epsilon = epsilon
self.log_softmax = nn.LogSoftmax(dim=1).cuda()
def forward(self, y, labels, lam, device_num=4):
y = y.view(device_num, -1, y.size(-1))
y_s, y_t, y_mixed = y.split(y.size(1) // 3, dim=1)
y_ori = torch.cat((y_s, y_t), 1).view(-1, y.size(-1))
y_mixed = y_mixed.contiguous().view(-1, y.size(-1))
log_prob_ori = self.log_softmax(y_ori)
log_prob_mixed = self.log_softmax(y_mixed)
labels = torch.zeros_like(log_prob_ori).scatter_(1, labels.unsqueeze(1), 1)
labels = labels.view(device_num, -1, labels.size(-1))
labels_s, labels_t = labels.split(labels.size(1) // 2, dim=1)
labels_s = labels_s.contiguous().view(-1, labels.size(-1))
labels_t = labels_t.contiguous().view(-1, labels.size(-1))
labels = labels.view(-1, labels.size(-1))
soft_labels = (1 - self.epsilon) * labels + self.epsilon / self.num_classes
lam = lam.view(-1, 1)
soft_labels_mixed = lam * labels_s + (1. - lam) * labels_t
soft_labels_mixed = (1 - self.epsilon) * soft_labels_mixed + self.epsilon / self.num_classes
loss_ori = (- soft_labels * log_prob_ori).sum(dim=1).mean()
loss_bridge_prob = (- soft_labels_mixed * log_prob_mixed).sum(dim=1).mean()
return loss_ori, loss_bridge_prob
================================================
FILE: domain_generalization/Transfer-Learning-Library/dalib/adaptation/idm/models/__init__.py
================================================
from .resnet import *
__all__ = ['resnet']
================================================
FILE: domain_generalization/Transfer-Learning-Library/dalib/adaptation/idm/models/dsbn.py
================================================
"""
Modified from https://github.com/SikaStar/IDM
@author: Baixu Chen
@contact: cbx_99_hasta@outlook.com
"""
import torch
import torch.nn as nn
class DSBN2d(nn.Module):
"""Domain-Specific BatchNorm2d layer in `Domain-Specific Batch Normalization for Unsupervised Domain Adaptation (CVPR 2019)
`_.
"""
def __init__(self, planes):
super(DSBN2d, self).__init__()
self.num_features = planes
self.BN_S = nn.BatchNorm2d(planes)
self.BN_T = nn.BatchNorm2d(planes)
def forward(self, x):
if not self.training:
return self.BN_T(x)
bs = x.size(0)
assert (bs % 2 == 0)
split = torch.split(x, int(bs / 2), 0)
out1 = self.BN_S(split[0].contiguous())
out2 = self.BN_T(split[1].contiguous())
out = torch.cat((out1, out2), 0)
return out
class DSBN1d(nn.Module):
"""Domain-Specific BatchNorm1d layer in `Domain-Specific Batch Normalization for Unsupervised Domain Adaptation (CVPR 2019)
`_.
"""
def __init__(self, planes):
super(DSBN1d, self).__init__()
self.num_features = planes
self.BN_S = nn.BatchNorm1d(planes)
self.BN_T = nn.BatchNorm1d(planes)
def forward(self, x):
if not self.training:
return self.BN_T(x)
bs = x.size(0)
assert (bs % 2 == 0)
split = torch.split(x, int(bs / 2), 0)
out1 = self.BN_S(split[0].contiguous())
out2 = self.BN_T(split[1].contiguous())
out = torch.cat((out1, out2), 0)
return out
class DSBN2d_idm(nn.Module):
"""Domain-Specific BatchNorm2d layer with `IDM` module
in `IDM: An Intermediate Domain Module for Domain Adaptive Person Re-ID (ICCV 2021)
`_.
"""
def __init__(self, planes):
super(DSBN2d_idm, self).__init__()
self.num_features = planes
self.BN_S = nn.BatchNorm2d(planes)
self.BN_T = nn.BatchNorm2d(planes)
self.BN_mix = nn.BatchNorm2d(planes)
def forward(self, x):
if not self.training:
return self.BN_T(x)
bs = x.size(0)
assert (bs % 3 == 0)
split = torch.split(x, int(bs / 3), 0)
out1 = self.BN_S(split[0].contiguous())
out2 = self.BN_T(split[1].contiguous())
out3 = self.BN_mix(split[2].contiguous())
out = torch.cat((out1, out2, out3), 0)
return out
class DSBN1d_idm(nn.Module):
"""Domain-Specific BatchNorm1d layer with `IDM` module
in `IDM: An Intermediate Domain Module for Domain Adaptive Person Re-ID (ICCV 2021)
`_.
"""
def __init__(self, planes):
super(DSBN1d_idm, self).__init__()
self.num_features = planes
self.BN_S = nn.BatchNorm1d(planes)
self.BN_T = nn.BatchNorm1d(planes)
self.BN_mix = nn.BatchNorm1d(planes)
def forward(self, x):
if not self.training:
return self.BN_T(x)
bs = x.size(0)
assert (bs % 3 == 0)
split = torch.split(x, int(bs / 3), 0)
out1 = self.BN_S(split[0].contiguous())
out2 = self.BN_T(split[1].contiguous())
out3 = self.BN_mix(split[2].contiguous())
out = torch.cat((out1, out2, out3), 0)
return out
================================================
FILE: domain_generalization/Transfer-Learning-Library/dalib/adaptation/idm/models/identifier.py
================================================
"""
Modified from https://github.com/SikaStar/IDM
@author: Baixu Chen
@contact: cbx_99_hasta@outlook.com
"""
from common.vision.models.reid.identifier import ReIdentifier as ReIdentifierBase
class ReIdentifier(ReIdentifierBase):
r"""Person reIdentifier in `IDM: An Intermediate Domain Module for Domain Adaptive Person Re-ID (ICCV 2021)
`_. During training, model predictions, extracted features as well as
attention lambda will be returned.
"""
def __init__(self, *args, **kwargs):
super(ReIdentifier, self).__init__(*args, **kwargs)
def forward(self, x, stage=0):
if self.training:
f, attention_lam = self.backbone(x, stage)
else:
f = self.backbone(x, stage)
f = self.pool_layer(f)
bn_f = self.bottleneck(f)
if not self.training:
return bn_f
predictions = self.head(bn_f)
return predictions, f, attention_lam
================================================
FILE: domain_generalization/Transfer-Learning-Library/dalib/adaptation/idm/models/idm.py
================================================
"""
Modified from https://github.com/SikaStar/IDM
@author: Baixu Chen
@contact: cbx_99_hasta@outlook.com
"""
import torch
import torch.nn as nn
class IDM(nn.Module):
"""IDM module in `IDM: An Intermediate Domain Module for Domain Adaptive Person Re-ID (ICCV 2021)
`_. This module automatically generates mixed features
with source and target features.
"""
def __init__(self, channel=64):
super(IDM, self).__init__()
self.channel = channel
self.fc1 = nn.Linear(channel * 2, channel)
self.fc2 = nn.Linear(channel, int(channel / 2))
self.fc3 = nn.Linear(int(channel / 2), 2)
self.softmax = nn.Softmax(dim=1)
self.avgpool = nn.AdaptiveAvgPool2d(1)
self.maxpool = nn.AdaptiveMaxPool2d(1)
def forward(self, x):
if not self.training:
return x
bs = x.size(0)
assert bs % 2 == 0
split = torch.split(x, int(bs / 2), 0)
x_s = split[0].contiguous()
x_t = split[1].contiguous()
x_emb_s = torch.cat((self.avgpool(x_s.detach()).squeeze(), self.maxpool(x_s.detach()).squeeze()), 1)
x_emb_t = torch.cat((self.avgpool(x_t.detach()).squeeze(), self.maxpool(x_t.detach()).squeeze()), 1)
x_emb_s, x_emb_t = self.fc1(x_emb_s), self.fc1(x_emb_t)
x_emb = x_emb_s + x_emb_t
x_emb = self.fc2(x_emb)
lam = self.fc3(x_emb)
lam = self.softmax(lam)
x_inter = lam[:, 0].reshape(-1, 1, 1, 1) * x_s + lam[:, 1].reshape(-1, 1, 1, 1) * x_t
output = torch.cat((x_s, x_t, x_inter), 0)
return output, lam
================================================
FILE: domain_generalization/Transfer-Learning-Library/dalib/adaptation/idm/models/resnet.py
================================================
"""
Modified from https://github.com/SikaStar/IDM
@author: Baixu Chen
@contact: cbx_99_hasta@outlook.com
"""
from .idm import IDM
import torch.nn as nn
from collections import OrderedDict
from common.vision.models.reid.resnet import ReidResNet as ResNetBase
from common.vision.models.reid.resnet import (
load_state_dict_from_url, model_urls, BasicBlock, Bottleneck
)
__all__ = ['reid_resnet18', 'reid_resnet34', 'reid_resnet50', 'reid_resnet101']
class ReidResNet(ResNetBase):
r"""Modified `ResNet` architecture with `IDM` module from `IDM: An Intermediate Domain Module for Domain Adaptive
Person Re-ID (ICCV 2021) `_. Although `IDM` Module can be inserted anywhere,
original paper places `IDM` after layer0-4. Our implementation follows this idea, but you are free to modify this
function to try other possibilities.
"""
def __init__(self, *args, **kwargs):
super(ReidResNet, self).__init__(*args, **kwargs)
self.conv = nn.Sequential(
OrderedDict([
('conv1', self.conv1),
('bn1', self.bn1),
('relu', self.relu),
('maxpool', self.maxpool)
])
)
self.idm_layer1 = IDM(channel=64)
self.idm_layer2 = IDM(channel=256)
self.idm_layer3 = IDM(channel=512)
self.idm_layer4 = IDM(channel=1024)
self.idm_layer5 = IDM(channel=2048)
def forward(self, x, stage=0):
assert stage in range(5)
x = self.conv(x)
if stage == 0 and self.training:
x, attention_lam = self.idm_layer1(x)
x = self.layer1(x)
if stage == 1 and self.training:
x, attention_lam = self.idm_layer2(x)
x = self.layer2(x)
if stage == 2 and self.training:
x, attention_lam = self.idm_layer3(x)
x = self.layer3(x)
if stage == 3 and self.training:
x, attention_lam = self.idm_layer4(x)
x = self.layer4(x)
if stage == 4 and self.training:
x, attention_lam = self.idm_layer5(x)
if self.training:
return x, attention_lam
else:
return x
def _reid_resnet(arch, block, layers, pretrained, progress, **kwargs):
model = ReidResNet(block, layers, **kwargs)
if pretrained:
model_dict = model.state_dict()
pretrained_dict = load_state_dict_from_url(model_urls[arch],
progress=progress)
# remove keys from pretrained dict that doesn't appear in model dict
pretrained_dict = {k: v for k, v in pretrained_dict.items() if k in model_dict}
model.load_state_dict(pretrained_dict, strict=False)
return model
def reid_resnet18(pretrained=False, progress=True, **kwargs):
r"""Constructs a Reid-ResNet-18 model.
Args:
pretrained (bool): If True, returns a model pre-trained on ImageNet
progress (bool): If True, displays a progress bar of the download to stderr
"""
return _reid_resnet('resnet18', BasicBlock, [2, 2, 2, 2], pretrained, progress,
**kwargs)
def reid_resnet34(pretrained=False, progress=True, **kwargs):
r"""Constructs a Reid-ResNet-34 model.
Args:
pretrained (bool): If True, returns a model pre-trained on ImageNet
progress (bool): If True, displays a progress bar of the download to stderr
"""
return _reid_resnet('resnet34', BasicBlock, [3, 4, 6, 3], pretrained, progress,
**kwargs)
def reid_resnet50(pretrained=False, progress=True, **kwargs):
r"""Constructs a Reid-ResNet-50 model.
Args:
pretrained (bool): If True, returns a model pre-trained on ImageNet
progress (bool): If True, displays a progress bar of the download to stderr
"""
return _reid_resnet('resnet50', Bottleneck, [3, 4, 6, 3], pretrained, progress,
**kwargs)
def reid_resnet101(pretrained=False, progress=True, **kwargs):
r"""Constructs a Reid-ResNet-101 model.
Args:
pretrained (bool): If True, returns a model pre-trained on ImageNet
progress (bool): If True, displays a progress bar of the download to stderr
"""
return _reid_resnet('resnet101', Bottleneck, [3, 4, 23, 3], pretrained, progress,
**kwargs)
================================================
FILE: domain_generalization/Transfer-Learning-Library/dalib/adaptation/idm/utils.py
================================================
"""
Modified from https://github.com/SikaStar/IDM
@author: Baixu Chen
@contact: cbx_99_hasta@outlook.com
"""
import torch.nn as nn
from .models.dsbn import DSBN1d, DSBN2d, DSBN1d_idm, DSBN2d_idm
def filter_layers(stage):
layer_names = ['conv', 'layer1', 'layer2', 'layer3', 'layer4', 'bottleneck']
idm_bn_names = []
for i in range(len(layer_names)):
if i >= stage + 1:
idm_bn_names.append(layer_names[i])
return idm_bn_names
def convert_dsbn_idm(model, mixup_bn_names, idm=False):
for _, (child_name, child) in enumerate(model.named_children()):
# print(child_name)
idm_flag = idm
for name in mixup_bn_names:
if name in child_name:
idm_flag = True
if isinstance(child, nn.BatchNorm2d) and not idm_flag:
m = DSBN2d(child.num_features)
m.BN_S.load_state_dict(child.state_dict())
m.BN_T.load_state_dict(child.state_dict())
setattr(model, child_name, m)
elif isinstance(child, nn.BatchNorm2d) and idm_flag:
m = DSBN2d_idm(child.num_features)
m.BN_S.load_state_dict(child.state_dict())
m.BN_T.load_state_dict(child.state_dict())
m.BN_mix.load_state_dict(child.state_dict())
setattr(model, child_name, m)
elif isinstance(child, nn.BatchNorm1d) and not idm_flag:
m = DSBN1d(child.num_features)
m.BN_S.load_state_dict(child.state_dict())
m.BN_T.load_state_dict(child.state_dict())
setattr(model, child_name, m)
elif isinstance(child, nn.BatchNorm1d) and idm_flag:
m = DSBN1d_idm(child.num_features)
m.BN_S.load_state_dict(child.state_dict())
m.BN_T.load_state_dict(child.state_dict())
m.BN_mix.load_state_dict(child.state_dict())
setattr(model, child_name, m)
else:
convert_dsbn_idm(child, mixup_bn_names, idm=idm_flag)
def convert_bn_idm(model, use_target=True):
for _, (child_name, child) in enumerate(model.named_children()):
assert (not next(model.parameters()).is_cuda)
if isinstance(child, DSBN2d):
m = nn.BatchNorm2d(child.num_features)
if use_target:
m.load_state_dict(child.BN_T.state_dict())
else:
m.load_state_dict(child.BN_S.state_dict())
setattr(model, child_name, m)
elif isinstance(child, DSBN2d_idm):
m = nn.BatchNorm2d(child.num_features)
if use_target:
m.load_state_dict(child.BN_T.state_dict())
else:
m.load_state_dict(child.BN_S.state_dict())
setattr(model, child_name, m)
elif isinstance(child, DSBN1d):
m = nn.BatchNorm1d(child.num_features)
if use_target:
m.load_state_dict(child.BN_T.state_dict())
else:
m.load_state_dict(child.BN_S.state_dict())
setattr(model, child_name, m)
elif isinstance(child, DSBN1d_idm):
m = nn.BatchNorm1d(child.num_features)
if use_target:
m.load_state_dict(child.BN_T.state_dict())
else:
m.load_state_dict(child.BN_S.state_dict())
setattr(model, child_name, m)
else:
convert_bn_idm(child, use_target=use_target)
================================================
FILE: domain_generalization/Transfer-Learning-Library/dalib/adaptation/idm/xbm.py
================================================
"""
Modified from https://github.com/SikaStar/IDM
@author: Baixu Chen
@contact: cbx_99_hasta@outlook.com
"""
import torch
class XBM(object):
"""`XBM` module in `Cross-Batch Memory for Embedding Learning (CVPR 2020) `_.
"""
def __init__(self, memory_size, feature_size):
self.K = memory_size
self.D = feature_size
self.feats = torch.zeros(self.K, self.D).cuda()
self.targets = torch.zeros(self.K, dtype=torch.long).cuda()
self.ptr = 0
@property
def is_full(self):
return self.targets[-1].item() != 0
def get(self):
if self.is_full:
return self.feats, self.targets
else:
return self.feats[:self.ptr], self.targets[:self.ptr]
def enqueue_dequeue(self, feats, targets):
q_size = len(targets)
if self.ptr + q_size > self.K:
self.feats[-q_size:] = feats
self.targets[-q_size:] = targets
self.ptr = 0
else:
self.feats[self.ptr: self.ptr + q_size] = feats
self.targets[self.ptr: self.ptr + q_size] = targets
self.ptr += q_size
def clean_target_domain(self, source_classes, target_classes):
empty_feats = torch.zeros(self.K, self.D).cuda()
empty_targets = torch.zeros(self.K, dtype=torch.long).cuda()
j = 0
for i in range(self.K):
if source_classes <= self.targets[i] < source_classes + target_classes:
continue
else:
empty_feats[j] = self.feats[i]
empty_targets[j] = self.targets[i]
j += 1
self.feats = empty_feats
self.targets = empty_targets
if j > 0:
self.ptr = j
================================================
FILE: domain_generalization/Transfer-Learning-Library/dalib/adaptation/iwan.py
================================================
"""
@author: Baixu Chen
@contact: cbx_99_hasta@outlook.com
"""
from typing import Optional, List, Dict
import torch
import torch.nn as nn
from common.modules.classifier import Classifier as ClassifierBase
class ImportanceWeightModule(object):
r"""
Calculating class weight based on the output of discriminator.
Introduced by `Importance Weighted Adversarial Nets for Partial Domain Adaptation (CVPR 2018) `_
Args:
discriminator (torch.nn.Module): A domain discriminator object, which predicts the domains of features.
Its input shape is :math:`(N, F)` and output shape is :math:`(N, 1)`
partial_classes_index (list[int], optional): The index of partial classes. Note that this parameter is \
just for debugging, since in real-world dataset, we have no access to the index of partial classes. \
Default: None.
Examples::
>>> domain_discriminator = DomainDiscriminator(1024, 1024)
>>> importance_weight_module = ImportanceWeightModule(domain_discriminator)
>>> num_iterations = 10000
>>> for _ in range(num_iterations):
>>> # feature from source domain
>>> f_s = torch.randn(32, 1024)
>>> # importance weights for source instance
>>> w_s = importance_weight_module.get_importance_weight(f_s)
"""
def __init__(self, discriminator: nn.Module, partial_classes_index: Optional[List[int]] = None):
self.discriminator = discriminator
self.partial_classes_index = partial_classes_index
def get_importance_weight(self, feature):
"""
Get importance weights for each instance.
Args:
feature (tensor): feature from source domain, in shape :math:`(N, F)`
Returns:
instance weight in shape :math:`(N, 1)`
"""
weight = 1. - self.discriminator(feature)
weight = weight / weight.mean()
weight = weight.detach()
return weight
def get_partial_classes_weight(self, weights: torch.Tensor, labels: torch.Tensor):
"""
Get class weight averaged on the partial classes and non-partial classes respectively.
Args:
weights (tensor): instance weight in shape :math:`(N, 1)`
labels (tensor): ground truth labels in shape :math:`(N, 1)`
.. warning::
This function is just for debugging, since in real-world dataset, we have no access to the index of \
partial classes and this function will throw an error when `partial_classes_index` is None.
"""
assert self.partial_classes_index is not None
weights = weights.squeeze()
is_partial = torch.Tensor([label in self.partial_classes_index for label in labels]).to(weights.device)
if is_partial.sum() > 0:
partial_classes_weight = (weights * is_partial).sum() / is_partial.sum()
else:
partial_classes_weight = torch.tensor(0)
not_partial = 1. - is_partial
if not_partial.sum() > 0:
not_partial_classes_weight = (weights * not_partial).sum() / not_partial.sum()
else:
not_partial_classes_weight = torch.tensor(0)
return partial_classes_weight, not_partial_classes_weight
class ImageClassifier(ClassifierBase):
r"""The Image Classifier for `Importance Weighted Adversarial Nets for Partial Domain Adaptation `_
"""
def __init__(self, backbone: nn.Module, num_classes: int, bottleneck_dim: Optional[int] = 256, **kwargs):
bottleneck = nn.Sequential(
nn.Linear(backbone.out_features, bottleneck_dim),
nn.BatchNorm1d(bottleneck_dim),
nn.ReLU()
)
super(ImageClassifier, self).__init__(backbone, num_classes, bottleneck, bottleneck_dim, **kwargs)
================================================
FILE: domain_generalization/Transfer-Learning-Library/dalib/adaptation/jan.py
================================================
"""
@author: Junguang Jiang
@contact: JiangJunguang1123@outlook.com
"""
from typing import Optional, Sequence
import torch
import torch.nn as nn
from common.modules.classifier import Classifier as ClassifierBase
from ..modules.grl import GradientReverseLayer
from ..modules.kernels import GaussianKernel
from .dan import _update_index_matrix
__all__ = ['JointMultipleKernelMaximumMeanDiscrepancy', 'ImageClassifier']
class JointMultipleKernelMaximumMeanDiscrepancy(nn.Module):
r"""The Joint Multiple Kernel Maximum Mean Discrepancy (JMMD) used in
`Deep Transfer Learning with Joint Adaptation Networks (ICML 2017) `_
Given source domain :math:`\mathcal{D}_s` of :math:`n_s` labeled points and target domain :math:`\mathcal{D}_t`
of :math:`n_t` unlabeled points drawn i.i.d. from P and Q respectively, the deep networks will generate
activations in layers :math:`\mathcal{L}` as :math:`\{(z_i^{s1}, ..., z_i^{s|\mathcal{L}|})\}_{i=1}^{n_s}` and
:math:`\{(z_i^{t1}, ..., z_i^{t|\mathcal{L}|})\}_{i=1}^{n_t}`. The empirical estimate of
:math:`\hat{D}_{\mathcal{L}}(P, Q)` is computed as the squared distance between the empirical kernel mean
embeddings as
.. math::
\hat{D}_{\mathcal{L}}(P, Q) &=
\dfrac{1}{n_s^2} \sum_{i=1}^{n_s}\sum_{j=1}^{n_s} \prod_{l\in\mathcal{L}} k^l(z_i^{sl}, z_j^{sl}) \\
&+ \dfrac{1}{n_t^2} \sum_{i=1}^{n_t}\sum_{j=1}^{n_t} \prod_{l\in\mathcal{L}} k^l(z_i^{tl}, z_j^{tl}) \\
&- \dfrac{2}{n_s n_t} \sum_{i=1}^{n_s}\sum_{j=1}^{n_t} \prod_{l\in\mathcal{L}} k^l(z_i^{sl}, z_j^{tl}). \\
Args:
kernels (tuple(tuple(torch.nn.Module))): kernel functions, where `kernels[r]` corresponds to kernel :math:`k^{\mathcal{L}[r]}`.
linear (bool): whether use the linear version of JAN. Default: False
thetas (list(Theta): use adversarial version JAN if not None. Default: None
Inputs:
- z_s (tuple(tensor)): multiple layers' activations from the source domain, :math:`z^s`
- z_t (tuple(tensor)): multiple layers' activations from the target domain, :math:`z^t`
Shape:
- :math:`z^{sl}` and :math:`z^{tl}`: :math:`(minibatch, *)` where * means any dimension
- Outputs: scalar
.. note::
Activations :math:`z^{sl}` and :math:`z^{tl}` must have the same shape.
.. note::
The kernel values will add up when there are multiple kernels for a certain layer.
Examples::
>>> feature_dim = 1024
>>> batch_size = 10
>>> layer1_kernels = (GaussianKernel(alpha=0.5), GaussianKernel(1.), GaussianKernel(2.))
>>> layer2_kernels = (GaussianKernel(1.), )
>>> loss = JointMultipleKernelMaximumMeanDiscrepancy((layer1_kernels, layer2_kernels))
>>> # layer1 features from source domain and target domain
>>> z1_s, z1_t = torch.randn(batch_size, feature_dim), torch.randn(batch_size, feature_dim)
>>> # layer2 features from source domain and target domain
>>> z2_s, z2_t = torch.randn(batch_size, feature_dim), torch.randn(batch_size, feature_dim)
>>> output = loss((z1_s, z2_s), (z1_t, z2_t))
"""
def __init__(self, kernels: Sequence[Sequence[nn.Module]], linear: Optional[bool] = True, thetas: Sequence[nn.Module] = None):
super(JointMultipleKernelMaximumMeanDiscrepancy, self).__init__()
self.kernels = kernels
self.index_matrix = None
self.linear = linear
if thetas:
self.thetas = thetas
else:
self.thetas = [nn.Identity() for _ in kernels]
def forward(self, z_s: torch.Tensor, z_t: torch.Tensor) -> torch.Tensor:
batch_size = int(z_s[0].size(0))
self.index_matrix = _update_index_matrix(batch_size, self.index_matrix, self.linear).to(z_s[0].device)
kernel_matrix = torch.ones_like(self.index_matrix)
for layer_z_s, layer_z_t, layer_kernels, theta in zip(z_s, z_t, self.kernels, self.thetas):
layer_features = torch.cat([layer_z_s, layer_z_t], dim=0)
layer_features = theta(layer_features)
kernel_matrix *= sum(
[kernel(layer_features) for kernel in layer_kernels]) # Add up the matrix of each kernel
# Add 2 / (n-1) to make up for the value on the diagonal
# to ensure loss is positive in the non-linear version
loss = (kernel_matrix * self.index_matrix).sum() + 2. / float(batch_size - 1)
return loss
class Theta(nn.Module):
"""
maximize loss respect to :math:`\theta`
minimize loss respect to features
"""
def __init__(self, dim: int):
super(Theta, self).__init__()
self.grl1 = GradientReverseLayer()
self.grl2 = GradientReverseLayer()
self.layer1 = nn.Linear(dim, dim)
nn.init.eye_(self.layer1.weight)
nn.init.zeros_(self.layer1.bias)
def forward(self, features: torch.Tensor) -> torch.Tensor:
features = self.grl1(features)
return self.grl2(self.layer1(features))
class ImageClassifier(ClassifierBase):
def __init__(self, backbone: nn.Module, num_classes: int, bottleneck_dim: Optional[int] = 256, **kwargs):
bottleneck = nn.Sequential(
# nn.AdaptiveAvgPool2d(output_size=(1, 1)),
# nn.Flatten(),
nn.Linear(backbone.out_features, bottleneck_dim),
nn.BatchNorm1d(bottleneck_dim),
nn.ReLU(),
nn.Dropout(0.5)
)
super(ImageClassifier, self).__init__(backbone, num_classes, bottleneck, bottleneck_dim, **kwargs)
================================================
FILE: domain_generalization/Transfer-Learning-Library/dalib/adaptation/mcc.py
================================================
"""
@author: Ying Jin
@contact: sherryying003@gmail.com
"""
from typing import Optional
import torch
import torch.nn as nn
import torch.nn.functional as F
from common.modules.classifier import Classifier as ClassifierBase
from ..modules.entropy import entropy
__all__ = ['MinimumClassConfusionLoss', 'ImageClassifier']
class MinimumClassConfusionLoss(nn.Module):
r"""
Minimum Class Confusion loss minimizes the class confusion in the target predictions.
You can see more details in `Minimum Class Confusion for Versatile Domain Adaptation (ECCV 2020) `_
Args:
temperature (float) : The temperature for rescaling, the prediction will shrink to vanilla softmax if
temperature is 1.0.
.. note::
Make sure that temperature is larger than 0.
Inputs: g_t
- g_t (tensor): unnormalized classifier predictions on target domain, :math:`g^t`
Shape:
- g_t: :math:`(minibatch, C)` where C means the number of classes.
- Output: scalar.
Examples::
>>> temperature = 2.0
>>> loss = MinimumClassConfusionLoss(temperature)
>>> # logits output from target domain
>>> g_t = torch.randn(batch_size, num_classes)
>>> output = loss(g_t)
MCC can also serve as a regularizer for existing methods.
Examples::
>>> from dalib.modules.domain_discriminator import DomainDiscriminator
>>> num_classes = 2
>>> feature_dim = 1024
>>> batch_size = 10
>>> temperature = 2.0
>>> discriminator = DomainDiscriminator(in_feature=feature_dim, hidden_size=1024)
>>> cdan_loss = ConditionalDomainAdversarialLoss(discriminator, reduction='mean')
>>> mcc_loss = MinimumClassConfusionLoss(temperature)
>>> # features from source domain and target domain
>>> f_s, f_t = torch.randn(batch_size, feature_dim), torch.randn(batch_size, feature_dim)
>>> # logits output from source domain adn target domain
>>> g_s, g_t = torch.randn(batch_size, num_classes), torch.randn(batch_size, num_classes)
>>> total_loss = cdan_loss(g_s, f_s, g_t, f_t) + mcc_loss(g_t)
"""
def __init__(self, temperature: float):
super(MinimumClassConfusionLoss, self).__init__()
self.temperature = temperature
def forward(self, logits: torch.Tensor) -> torch.Tensor:
batch_size, num_classes = logits.shape
predictions = F.softmax(logits / self.temperature, dim=1) # batch_size x num_classes
entropy_weight = entropy(predictions).detach()
entropy_weight = 1 + torch.exp(-entropy_weight)
entropy_weight = (batch_size * entropy_weight / torch.sum(entropy_weight)).unsqueeze(dim=1) # batch_size x 1
class_confusion_matrix = torch.mm((predictions * entropy_weight).transpose(1, 0), predictions) # num_classes x num_classes
class_confusion_matrix = class_confusion_matrix / torch.sum(class_confusion_matrix, dim=1)
mcc_loss = (torch.sum(class_confusion_matrix) - torch.trace(class_confusion_matrix)) / num_classes
return mcc_loss
class ImageClassifier(ClassifierBase):
def __init__(self, backbone: nn.Module, num_classes: int, bottleneck_dim: Optional[int] = 256, **kwargs):
bottleneck = nn.Sequential(
# nn.AdaptiveAvgPool2d(output_size=(1, 1)),
# nn.Flatten(),
nn.Linear(backbone.out_features, bottleneck_dim),
nn.BatchNorm1d(bottleneck_dim),
nn.ReLU()
)
super(ImageClassifier, self).__init__(backbone, num_classes, bottleneck, bottleneck_dim, **kwargs)
================================================
FILE: domain_generalization/Transfer-Learning-Library/dalib/adaptation/mcd.py
================================================
"""
@author: Junguang Jiang
@contact: JiangJunguang1123@outlook.com
"""
from typing import Optional
import torch.nn as nn
import torch
def classifier_discrepancy(predictions1: torch.Tensor, predictions2: torch.Tensor) -> torch.Tensor:
r"""The `Classifier Discrepancy` in
`Maximum Classifier Discrepancy for Unsupervised Domain Adaptation (CVPR 2018) `_.
The classfier discrepancy between predictions :math:`p_1` and :math:`p_2` can be described as:
.. math::
d(p_1, p_2) = \dfrac{1}{K} \sum_{k=1}^K | p_{1k} - p_{2k} |,
where K is number of classes.
Args:
predictions1 (torch.Tensor): Classifier predictions :math:`p_1`. Expected to contain raw, normalized scores for each class
predictions2 (torch.Tensor): Classifier predictions :math:`p_2`
"""
return torch.mean(torch.abs(predictions1 - predictions2))
def entropy(predictions: torch.Tensor) -> torch.Tensor:
r"""Entropy of N predictions :math:`(p_1, p_2, ..., p_N)`.
The definition is:
.. math::
d(p_1, p_2, ..., p_N) = -\dfrac{1}{K} \sum_{k=1}^K \log \left( \dfrac{1}{N} \sum_{i=1}^N p_{ik} \right)
where K is number of classes.
.. note::
This entropy function is specifically used in MCD and different from the usual :meth:`~dalib.modules.entropy.entropy` function.
Args:
predictions (torch.Tensor): Classifier predictions. Expected to contain raw, normalized scores for each class
"""
return -torch.mean(torch.log(torch.mean(predictions, 0) + 1e-6))
class ImageClassifierHead(nn.Module):
r"""Classifier Head for MCD.
Args:
in_features (int): Dimension of input features
num_classes (int): Number of classes
bottleneck_dim (int, optional): Feature dimension of the bottleneck layer. Default: 1024
Shape:
- Inputs: :math:`(minibatch, F)` where F = `in_features`.
- Output: :math:`(minibatch, C)` where C = `num_classes`.
"""
def __init__(self, in_features: int, num_classes: int, bottleneck_dim: Optional[int] = 1024, pool_layer=None):
super(ImageClassifierHead, self).__init__()
self.num_classes = num_classes
if pool_layer is None:
self.pool_layer = nn.Sequential(
nn.AdaptiveAvgPool2d(output_size=(1, 1)),
nn.Flatten()
)
else:
self.pool_layer = pool_layer
self.head = nn.Sequential(
nn.Dropout(0.5),
nn.Linear(in_features, bottleneck_dim),
nn.BatchNorm1d(bottleneck_dim),
nn.ReLU(),
nn.Dropout(0.5),
nn.Linear(bottleneck_dim, bottleneck_dim),
nn.BatchNorm1d(bottleneck_dim),
nn.ReLU(),
nn.Linear(bottleneck_dim, num_classes)
)
def forward(self, inputs: torch.Tensor) -> torch.Tensor:
return self.head(self.pool_layer(inputs))
================================================
FILE: domain_generalization/Transfer-Learning-Library/dalib/adaptation/mdd.py
================================================
"""
@author: Junguang Jiang
@contact: JiangJunguang1123@outlook.com
"""
from typing import Optional, List, Dict, Tuple, Callable
import torch.nn as nn
import torch.nn.functional as F
import torch
from dalib.modules.grl import WarmStartGradientReverseLayer
class MarginDisparityDiscrepancy(nn.Module):
r"""The margin disparity discrepancy (MDD) proposed in `Bridging Theory and Algorithm for Domain Adaptation (ICML 2019) `_.
MDD can measure the distribution discrepancy in domain adaptation.
The :math:`y^s` and :math:`y^t` are logits output by the main head on the source and target domain respectively.
The :math:`y_{adv}^s` and :math:`y_{adv}^t` are logits output by the adversarial head.
The definition can be described as:
.. math::
\mathcal{D}_{\gamma}(\hat{\mathcal{S}}, \hat{\mathcal{T}}) =
-\gamma \mathbb{E}_{y^s, y_{adv}^s \sim\hat{\mathcal{S}}} L_s (y^s, y_{adv}^s) +
\mathbb{E}_{y^t, y_{adv}^t \sim\hat{\mathcal{T}}} L_t (y^t, y_{adv}^t),
where :math:`\gamma` is a margin hyper-parameter, :math:`L_s` refers to the disparity function defined on the source domain
and :math:`L_t` refers to the disparity function defined on the target domain.
Args:
source_disparity (callable): The disparity function defined on the source domain, :math:`L_s`.
target_disparity (callable): The disparity function defined on the target domain, :math:`L_t`.
margin (float): margin :math:`\gamma`. Default: 4
reduction (str, optional): Specifies the reduction to apply to the output:
``'none'`` | ``'mean'`` | ``'sum'``. ``'none'``: no reduction will be applied,
``'mean'``: the sum of the output will be divided by the number of
elements in the output, ``'sum'``: the output will be summed. Default: ``'mean'``
Inputs:
- y_s: output :math:`y^s` by the main head on the source domain
- y_s_adv: output :math:`y^s` by the adversarial head on the source domain
- y_t: output :math:`y^t` by the main head on the target domain
- y_t_adv: output :math:`y_{adv}^t` by the adversarial head on the target domain
- w_s (optional): instance weights for source domain
- w_t (optional): instance weights for target domain
Examples::
>>> num_outputs = 2
>>> batch_size = 10
>>> loss = MarginDisparityDiscrepancy(margin=4., source_disparity=F.l1_loss, target_disparity=F.l1_loss)
>>> # output from source domain and target domain
>>> y_s, y_t = torch.randn(batch_size, num_outputs), torch.randn(batch_size, num_outputs)
>>> # adversarial output from source domain and target domain
>>> y_s_adv, y_t_adv = torch.randn(batch_size, num_outputs), torch.randn(batch_size, num_outputs)
>>> output = loss(y_s, y_s_adv, y_t, y_t_adv)
"""
def __init__(self, source_disparity: Callable, target_disparity: Callable,
margin: Optional[float] = 4, reduction: Optional[str] = 'mean'):
super(MarginDisparityDiscrepancy, self).__init__()
self.margin = margin
self.reduction = reduction
self.source_disparity = source_disparity
self.target_disparity = target_disparity
def forward(self, y_s: torch.Tensor, y_s_adv: torch.Tensor, y_t: torch.Tensor, y_t_adv: torch.Tensor,
w_s: Optional[torch.Tensor] = None, w_t: Optional[torch.Tensor] = None) -> torch.Tensor:
source_loss = -self.margin * self.source_disparity(y_s, y_s_adv)
target_loss = self.target_disparity(y_t, y_t_adv)
if w_s is None:
w_s = torch.ones_like(source_loss)
source_loss = source_loss * w_s
if w_t is None:
w_t = torch.ones_like(target_loss)
target_loss = target_loss * w_t
loss = source_loss + target_loss
if self.reduction == 'mean':
loss = loss.mean()
elif self.reduction == 'sum':
loss = loss.sum()
return loss
class ClassificationMarginDisparityDiscrepancy(MarginDisparityDiscrepancy):
r"""
The margin disparity discrepancy (MDD) proposed in `Bridging Theory and Algorithm for Domain Adaptation (ICML 2019) `_.
It measures the distribution discrepancy in domain adaptation
for classification.
When margin is equal to 1, it's also called disparity discrepancy (DD).
The :math:`y^s` and :math:`y^t` are logits output by the main classifier on the source and target domain respectively.
The :math:`y_{adv}^s` and :math:`y_{adv}^t` are logits output by the adversarial classifier.
They are expected to contain raw, unnormalized scores for each class.
The definition can be described as:
.. math::
\mathcal{D}_{\gamma}(\hat{\mathcal{S}}, \hat{\mathcal{T}}) =
\gamma \mathbb{E}_{y^s, y_{adv}^s \sim\hat{\mathcal{S}}} \log\left(\frac{\exp(y_{adv}^s[h_{y^s}])}{\sum_j \exp(y_{adv}^s[j])}\right) +
\mathbb{E}_{y^t, y_{adv}^t \sim\hat{\mathcal{T}}} \log\left(1-\frac{\exp(y_{adv}^t[h_{y^t}])}{\sum_j \exp(y_{adv}^t[j])}\right),
where :math:`\gamma` is a margin hyper-parameter and :math:`h_y` refers to the predicted label when the logits output is :math:`y`.
You can see more details in `Bridging Theory and Algorithm for Domain Adaptation `_.
Args:
margin (float): margin :math:`\gamma`. Default: 4
reduction (str, optional): Specifies the reduction to apply to the output:
``'none'`` | ``'mean'`` | ``'sum'``. ``'none'``: no reduction will be applied,
``'mean'``: the sum of the output will be divided by the number of
elements in the output, ``'sum'``: the output will be summed. Default: ``'mean'``
Inputs:
- y_s: logits output :math:`y^s` by the main classifier on the source domain
- y_s_adv: logits output :math:`y^s` by the adversarial classifier on the source domain
- y_t: logits output :math:`y^t` by the main classifier on the target domain
- y_t_adv: logits output :math:`y_{adv}^t` by the adversarial classifier on the target domain
Shape:
- Inputs: :math:`(minibatch, C)` where C = number of classes, or :math:`(minibatch, C, d_1, d_2, ..., d_K)`
with :math:`K \geq 1` in the case of `K`-dimensional loss.
- Output: scalar. If :attr:`reduction` is ``'none'``, then the same size as the target: :math:`(minibatch)`, or
:math:`(minibatch, d_1, d_2, ..., d_K)` with :math:`K \geq 1` in the case of K-dimensional loss.
Examples::
>>> num_classes = 2
>>> batch_size = 10
>>> loss = ClassificationMarginDisparityDiscrepancy(margin=4.)
>>> # logits output from source domain and target domain
>>> y_s, y_t = torch.randn(batch_size, num_classes), torch.randn(batch_size, num_classes)
>>> # adversarial logits output from source domain and target domain
>>> y_s_adv, y_t_adv = torch.randn(batch_size, num_classes), torch.randn(batch_size, num_classes)
>>> output = loss(y_s, y_s_adv, y_t, y_t_adv)
"""
def __init__(self, margin: Optional[float] = 4, **kwargs):
def source_discrepancy(y: torch.Tensor, y_adv: torch.Tensor):
_, prediction = y.max(dim=1)
return F.cross_entropy(y_adv, prediction, reduction='none')
def target_discrepancy(y: torch.Tensor, y_adv: torch.Tensor):
_, prediction = y.max(dim=1)
return -F.nll_loss(shift_log(1. - F.softmax(y_adv, dim=1)), prediction, reduction='none')
super(ClassificationMarginDisparityDiscrepancy, self).__init__(source_discrepancy, target_discrepancy, margin,
**kwargs)
class RegressionMarginDisparityDiscrepancy(MarginDisparityDiscrepancy):
r"""
The margin disparity discrepancy (MDD) proposed in `Bridging Theory and Algorithm for Domain Adaptation (ICML 2019) `_.
It measures the distribution discrepancy in domain adaptation
for regression.
The :math:`y^s` and :math:`y^t` are logits output by the main regressor on the source and target domain respectively.
The :math:`y_{adv}^s` and :math:`y_{adv}^t` are logits output by the adversarial regressor.
They are expected to contain ``normalized`` values for each factors.
The definition can be described as:
.. math::
\mathcal{D}_{\gamma}(\hat{\mathcal{S}}, \hat{\mathcal{T}}) =
-\gamma \mathbb{E}_{y^s, y_{adv}^s \sim\hat{\mathcal{S}}} L (y^s, y_{adv}^s) +
\mathbb{E}_{y^t, y_{adv}^t \sim\hat{\mathcal{T}}} L (y^t, y_{adv}^t),
where :math:`\gamma` is a margin hyper-parameter and :math:`L` refers to the disparity function defined on both domains.
You can see more details in `Bridging Theory and Algorithm for Domain Adaptation `_.
Args:
loss_function (callable): The disparity function defined on both domains, :math:`L`.
margin (float): margin :math:`\gamma`. Default: 1
reduction (str, optional): Specifies the reduction to apply to the output:
``'none'`` | ``'mean'`` | ``'sum'``. ``'none'``: no reduction will be applied,
``'mean'``: the sum of the output will be divided by the number of
elements in the output, ``'sum'``: the output will be summed. Default: ``'mean'``
Inputs:
- y_s: logits output :math:`y^s` by the main regressor on the source domain
- y_s_adv: logits output :math:`y^s` by the adversarial regressor on the source domain
- y_t: logits output :math:`y^t` by the main regressor on the target domain
- y_t_adv: logits output :math:`y_{adv}^t` by the adversarial regressor on the target domain
Shape:
- Inputs: :math:`(minibatch, F)` where F = number of factors, or :math:`(minibatch, F, d_1, d_2, ..., d_K)`
with :math:`K \geq 1` in the case of `K`-dimensional loss.
- Output: scalar. The same size as the target: :math:`(minibatch)`, or
:math:`(minibatch, d_1, d_2, ..., d_K)` with :math:`K \geq 1` in the case of K-dimensional loss.
Examples::
>>> num_outputs = 2
>>> batch_size = 10
>>> loss = RegressionMarginDisparityDiscrepancy(margin=4., loss_function=F.l1_loss)
>>> # output from source domain and target domain
>>> y_s, y_t = torch.randn(batch_size, num_outputs), torch.randn(batch_size, num_outputs)
>>> # adversarial output from source domain and target domain
>>> y_s_adv, y_t_adv = torch.randn(batch_size, num_outputs), torch.randn(batch_size, num_outputs)
>>> output = loss(y_s, y_s_adv, y_t, y_t_adv)
"""
def __init__(self, margin: Optional[float] = 1, loss_function=F.l1_loss, **kwargs):
def source_discrepancy(y: torch.Tensor, y_adv: torch.Tensor):
return loss_function(y_adv, y.detach(), reduction='none')
def target_discrepancy(y: torch.Tensor, y_adv: torch.Tensor):
return loss_function(y_adv, y.detach(), reduction='none')
super(RegressionMarginDisparityDiscrepancy, self).__init__(source_discrepancy, target_discrepancy, margin,
**kwargs)
def shift_log(x: torch.Tensor, offset: Optional[float] = 1e-6) -> torch.Tensor:
r"""
First shift, then calculate log, which can be described as:
.. math::
y = \max(\log(x+\text{offset}), 0)
Used to avoid the gradient explosion problem in log(x) function when x=0.
Args:
x (torch.Tensor): input tensor
offset (float, optional): offset size. Default: 1e-6
.. note::
Input tensor falls in [0., 1.] and the output tensor falls in [-log(offset), 0]
"""
return torch.log(torch.clamp(x + offset, max=1.))
class GeneralModule(nn.Module):
def __init__(self, backbone: nn.Module, num_classes: int, bottleneck: nn.Module,
head: nn.Module, adv_head: nn.Module, grl: Optional[WarmStartGradientReverseLayer] = None,
finetune: Optional[bool] = True):
super(GeneralModule, self).__init__()
self.backbone = backbone
self.num_classes = num_classes
self.bottleneck = bottleneck
self.head = head
self.adv_head = adv_head
self.finetune = finetune
self.grl_layer = WarmStartGradientReverseLayer(alpha=1.0, lo=0.0, hi=0.1, max_iters=1000,
auto_step=False) if grl is None else grl
def forward(self, x: torch.Tensor) -> Tuple[torch.Tensor, torch.Tensor]:
""""""
features = self.backbone(x)
features = self.bottleneck(features)
outputs = self.head(features)
features_adv = self.grl_layer(features)
outputs_adv = self.adv_head(features_adv)
if self.training:
return outputs, outputs_adv
else:
return outputs
def step(self):
"""
Gradually increase :math:`\lambda` in GRL layer.
"""
self.grl_layer.step()
def get_parameters(self, base_lr=1.0) -> List[Dict]:
"""
Return a parameters list which decides optimization hyper-parameters,
such as the relative learning rate of each layer.
"""
params = [
{"params": self.backbone.parameters(), "lr": 0.1 * base_lr if self.finetune else base_lr},
{"params": self.bottleneck.parameters(), "lr": base_lr},
{"params": self.head.parameters(), "lr": base_lr},
{"params": self.adv_head.parameters(), "lr": base_lr}
]
return params
class ImageClassifier(GeneralModule):
r"""Classifier for MDD.
Classifier for MDD has one backbone, one bottleneck, while two classifier heads.
The first classifier head is used for final predictions.
The adversarial classifier head is only used when calculating MarginDisparityDiscrepancy.
Args:
backbone (torch.nn.Module): Any backbone to extract 1-d features from data
num_classes (int): Number of classes
bottleneck_dim (int, optional): Feature dimension of the bottleneck layer. Default: 1024
width (int, optional): Feature dimension of the classifier head. Default: 1024
grl (nn.Module): Gradient reverse layer. Will use default parameters if None. Default: None.
finetune (bool, optional): Whether use 10x smaller learning rate in the backbone. Default: True
Inputs:
- x (tensor): input data
Outputs:
- outputs: logits outputs by the main classifier
- outputs_adv: logits outputs by the adversarial classifier
Shape:
- x: :math:`(minibatch, *)`, same shape as the input of the `backbone`.
- outputs, outputs_adv: :math:`(minibatch, C)`, where C means the number of classes.
.. note::
Remember to call function `step()` after function `forward()` **during training phase**! For instance,
>>> # x is inputs, classifier is an ImageClassifier
>>> outputs, outputs_adv = classifier(x)
>>> classifier.step()
"""
def __init__(self, backbone: nn.Module, num_classes: int,
bottleneck_dim: Optional[int] = 1024, width: Optional[int] = 1024,
grl: Optional[WarmStartGradientReverseLayer] = None, finetune=True, pool_layer=None):
grl_layer = WarmStartGradientReverseLayer(alpha=1.0, lo=0.0, hi=0.1, max_iters=1000,
auto_step=False) if grl is None else grl
if pool_layer is None:
pool_layer = nn.Sequential(
nn.AdaptiveAvgPool2d(output_size=(1, 1)),
nn.Flatten()
)
bottleneck = nn.Sequential(
pool_layer,
nn.Linear(backbone.out_features, bottleneck_dim),
nn.BatchNorm1d(bottleneck_dim),
nn.ReLU(),
nn.Dropout(0.5)
)
bottleneck[1].weight.data.normal_(0, 0.005)
bottleneck[1].bias.data.fill_(0.1)
# The classifier head used for final predictions.
head = nn.Sequential(
nn.Linear(bottleneck_dim, width),
nn.ReLU(),
nn.Dropout(0.5),
nn.Linear(width, num_classes)
)
# The adversarial classifier head
adv_head = nn.Sequential(
nn.Linear(bottleneck_dim, width),
nn.ReLU(),
nn.Dropout(0.5),
nn.Linear(width, num_classes)
)
for dep in range(2):
head[dep * 3].weight.data.normal_(0, 0.01)
head[dep * 3].bias.data.fill_(0.0)
adv_head[dep * 3].weight.data.normal_(0, 0.01)
adv_head[dep * 3].bias.data.fill_(0.0)
super(ImageClassifier, self).__init__(backbone, num_classes, bottleneck,
head, adv_head, grl_layer, finetune)
class ImageRegressor(GeneralModule):
r"""Regressor for MDD.
Regressor for MDD has one backbone, one bottleneck, while two regressor heads.
The first regressor head is used for final predictions.
The adversarial regressor head is only used when calculating MarginDisparityDiscrepancy.
Args:
backbone (torch.nn.Module): Any backbone to extract 1-d features from data
num_factors (int): Number of factors
bottleneck_dim (int, optional): Feature dimension of the bottleneck layer. Default: 1024
width (int, optional): Feature dimension of the classifier head. Default: 1024
finetune (bool, optional): Whether use 10x smaller learning rate in the backbone. Default: True
Inputs:
- x (Tensor): input data
Outputs: (outputs, outputs_adv)
- outputs: outputs by the main regressor
- outputs_adv: outputs by the adversarial regressor
Shape:
- x: :math:`(minibatch, *)`, same shape as the input of the `backbone`.
- outputs, outputs_adv: :math:`(minibatch, F)`, where F means the number of factors.
.. note::
Remember to call function `step()` after function `forward()` **during training phase**! For instance,
>>> # x is inputs, regressor is an ImageRegressor
>>> outputs, outputs_adv = regressor(x)
>>> regressor.step()
"""
def __init__(self, backbone: nn.Module, num_factors: int, bottleneck = None, head=None, adv_head=None,
bottleneck_dim: Optional[int] = 1024, width: Optional[int] = 1024, finetune=True):
grl_layer = WarmStartGradientReverseLayer(alpha=1.0, lo=0.0, hi=0.1, max_iters=1000, auto_step=False)
if bottleneck is None:
bottleneck = nn.Sequential(
nn.Conv2d(backbone.out_features, bottleneck_dim, kernel_size=3, stride=1, padding=1),
nn.BatchNorm2d(bottleneck_dim),
nn.ReLU(),
)
# The regressor head used for final predictions.
if head is None:
head = nn.Sequential(
nn.Conv2d(bottleneck_dim, width, kernel_size=3, stride=1, padding=1),
nn.BatchNorm2d(width),
nn.ReLU(),
nn.Conv2d(width, width, kernel_size=3, stride=1, padding=1),
nn.BatchNorm2d(width),
nn.ReLU(),
nn.AdaptiveAvgPool2d(output_size=(1, 1)),
nn.Flatten(),
nn.Linear(width, num_factors),
nn.Sigmoid()
)
for layer in head:
if isinstance(layer, nn.Conv2d) or isinstance(layer, nn.Linear):
nn.init.normal_(layer.weight, 0, 0.01)
nn.init.constant_(layer.bias, 0)
# The adversarial regressor head
if adv_head is None:
adv_head = nn.Sequential(
nn.Conv2d(bottleneck_dim, width, kernel_size=3, stride=1, padding=1),
nn.BatchNorm2d(width),
nn.ReLU(),
nn.Conv2d(width, width, kernel_size=3, stride=1, padding=1),
nn.BatchNorm2d(width),
nn.ReLU(),
nn.AdaptiveAvgPool2d(output_size=(1, 1)),
nn.Flatten(),
nn.Linear(width, num_factors),
nn.Sigmoid()
)
for layer in adv_head:
if isinstance(layer, nn.Conv2d) or isinstance(layer, nn.Linear):
nn.init.normal_(layer.weight, 0, 0.01)
nn.init.constant_(layer.bias, 0)
super(ImageRegressor, self).__init__(backbone, num_factors, bottleneck,
head, adv_head, grl_layer, finetune)
self.num_factors = num_factors
================================================
FILE: domain_generalization/Transfer-Learning-Library/dalib/adaptation/osbp.py
================================================
"""
@author: Junguang Jiang
@contact: JiangJunguang1123@outlook.com
"""
from typing import Optional
import torch
import torch.nn as nn
import torch.nn.functional as F
from common.modules.classifier import Classifier as ClassifierBase
from ..modules.grl import GradientReverseLayer
class UnknownClassBinaryCrossEntropy(nn.Module):
r"""
Binary cross entropy loss to make a boundary for unknown samples, proposed by
`Open Set Domain Adaptation by Backpropagation (ECCV 2018) `_.
Given a sample on target domain :math:`x_t` and its classifcation outputs :math:`y`, the binary cross entropy
loss is defined as
.. math::
L_{\text{adv}}(x_t) = -t \text{log}(p(y=C+1|x_t)) - (1-t)\text{log}(1-p(y=C+1|x_t))
where t is a hyper-parameter and C is the number of known classes.
Args:
t (float): Predefined hyper-parameter. Default: 0.5
Inputs:
- y (tensor): classification outputs (before softmax).
Shape:
- y: :math:`(minibatch, C+1)` where C is the number of known classes.
- Outputs: scalar
"""
def __init__(self, t: Optional[float]=0.5):
super(UnknownClassBinaryCrossEntropy, self).__init__()
self.t = t
def forward(self, y):
# y : N x (C+1)
softmax_output = F.softmax(y, dim=1)
unknown_class_prob = softmax_output[:, -1].contiguous().view(-1, 1)
known_class_prob = 1. - unknown_class_prob
unknown_target = torch.ones((y.size(0), 1)).to(y.device) * self.t
known_target = 1. - unknown_target
return - torch.mean(unknown_target * torch.log(unknown_class_prob + 1e-6)) \
- torch.mean(known_target * torch.log(known_class_prob + 1e-6))
class ImageClassifier(ClassifierBase):
def __init__(self, backbone: nn.Module, num_classes: int, bottleneck_dim: Optional[int] = 256, **kwargs):
bottleneck = nn.Sequential(
# nn.AdaptiveAvgPool2d(output_size=(1, 1)),
# nn.Flatten(),
nn.Linear(backbone.out_features, bottleneck_dim),
nn.BatchNorm1d(bottleneck_dim),
nn.ReLU(),
nn.Dropout(),
nn.Linear(bottleneck_dim, bottleneck_dim),
nn.BatchNorm1d(bottleneck_dim),
nn.ReLU(),
nn.Dropout()
)
super(ImageClassifier, self).__init__(backbone, num_classes, bottleneck, bottleneck_dim, **kwargs)
self.grl = GradientReverseLayer()
def forward(self, x: torch.Tensor, grad_reverse: Optional[bool] = False):
features = self.pool_layer(self.backbone(x))
features = self.bottleneck(features)
if grad_reverse:
features = self.grl(features)
outputs = self.head(features)
if self.training:
return outputs, features
else:
return outputs
================================================
FILE: domain_generalization/Transfer-Learning-Library/dalib/adaptation/pada.py
================================================
"""
@author: Junguang Jiang
@contact: JiangJunguang1123@outlook.com
"""
from typing import Optional, List, Tuple
from torch.utils.data.dataloader import DataLoader
import torch.nn as nn
import torch
import torch.nn.functional as F
class AutomaticUpdateClassWeightModule(object):
r"""
Calculating class weight based on the output of classifier. See ``ClassWeightModule`` about the details of the calculation.
Every N iterations, the class weight is updated automatically.
Args:
update_steps (int): N, the number of iterations to update class weight.
data_loader (torch.utils.data.DataLoader): The data loader from which we can collect classification outputs.
classifier (torch.nn.Module): Classifier.
num_classes (int): Number of classes.
device (torch.device): The device to run classifier.
temperature (float, optional): T, temperature in ClassWeightModule. Default: 0.1
partial_classes_index (list[int], optional): The index of partial classes. Note that this parameter is \
just for debugging, since in real-world dataset, we have no access to the index of partial classes. \
Default: None.
Examples::
>>> class_weight_module = AutomaticUpdateClassWeightModule(update_steps=500, ...)
>>> num_iterations = 10000
>>> for _ in range(num_iterations):
>>> class_weight_module.step()
>>> # weight for F.cross_entropy
>>> w_c = class_weight_module.get_class_weight_for_cross_entropy_loss()
>>> # weight for dalib.addaptation.dann.DomainAdversarialLoss
>>> w_s, w_t = class_weight_module.get_class_weight_for_adversarial_loss()
"""
def __init__(self, update_steps: int, data_loader: DataLoader,
classifier: nn.Module, num_classes: int,
device: torch.device, temperature: Optional[float] = 0.1,
partial_classes_index: Optional[List[int]] = None):
self.update_steps = update_steps
self.data_loader = data_loader
self.classifier = classifier
self.device = device
self.class_weight_module = ClassWeightModule(temperature)
self.class_weight = torch.ones(num_classes).to(device)
self.num_steps = 0
self.partial_classes_index = partial_classes_index
if partial_classes_index is not None:
self.non_partial_classes_index = [c for c in range(num_classes) if c not in partial_classes_index]
def step(self):
self.num_steps += 1
if self.num_steps % self.update_steps == 0:
all_outputs = collect_classification_results(self.data_loader, self.classifier, self.device)
self.class_weight = self.class_weight_module(all_outputs)
def get_class_weight_for_cross_entropy_loss(self):
"""
Outputs: weight for F.cross_entropy
Shape: :math:`(C, )` where C means the number of classes.
"""
return self.class_weight
def get_class_weight_for_adversarial_loss(self, source_labels: torch.Tensor) -> Tuple[torch.Tensor, torch.Tensor]:
"""
Outputs:
- w_s: source weight for :py:class:`~dalib.adaptation.dann.DomainAdversarialLoss`
- w_t: target weight for :py:class:`~dalib.adaptation.dann.DomainAdversarialLoss`
Shape:
- w_s: :math:`(minibatch, )`
- w_t: :math:`(minibatch, )`
"""
class_weight_adv_source = self.class_weight[source_labels]
class_weight_adv_target = torch.ones_like(class_weight_adv_source) * class_weight_adv_source.mean()
return class_weight_adv_source, class_weight_adv_target
def get_partial_classes_weight(self):
"""
Get class weight averaged on the partial classes and non-partial classes respectively.
.. warning::
This function is just for debugging, since in real-world dataset, we have no access to the index of \
partial classes and this function will throw an error when `partial_classes_index` is None.
"""
assert self.partial_classes_index is not None
return torch.mean(self.class_weight[self.partial_classes_index]), torch.mean(
self.class_weight[self.non_partial_classes_index])
class ClassWeightModule(nn.Module):
r"""
Calculating class weight based on the output of classifier.
Introduced by `Partial Adversarial Domain Adaptation (ECCV 2018) `_
Given classification logits outputs :math:`\{\hat{y}_i\}_{i=1}^n`, where :math:`n` is the dataset size,
the weight indicating the contribution of each class to the training can be calculated as
follows
.. math::
\mathcal{\gamma} = \dfrac{1}{n} \sum_{i=1}^{n}\text{softmax}( \hat{y}_i / T),
where :math:`\mathcal{\gamma}` is a :math:`|\mathcal{C}|`-dimensional weight vector quantifying the contribution
of each class and T is a hyper-parameters called temperature.
In practice, it's possible that some of the weights are very small, thus, we normalize weight :math:`\mathcal{\gamma}`
by dividing its largest element, i.e. :math:`\mathcal{\gamma} \leftarrow \mathcal{\gamma} / max(\mathcal{\gamma})`
Args:
temperature (float, optional): hyper-parameters :math:`T`. Default: 0.1
Shape:
- Inputs: (minibatch, :math:`|\mathcal{C}|`)
- Outputs: (:math:`|\mathcal{C}|`,)
"""
def __init__(self, temperature: Optional[float] = 0.1):
super(ClassWeightModule, self).__init__()
self.temperature = temperature
def forward(self, outputs: torch.Tensor):
outputs.detach_()
softmax_outputs = F.softmax(outputs / self.temperature, dim=1)
class_weight = torch.mean(softmax_outputs, dim=0)
class_weight = class_weight / torch.max(class_weight)
class_weight = class_weight.view(-1)
return class_weight
def collect_classification_results(data_loader: DataLoader, classifier: nn.Module,
device: torch.device) -> torch.Tensor:
"""
Fetch data from `data_loader`, and then use `classifier` to collect classification results
Args:
data_loader (torch.utils.data.DataLoader): Data loader.
classifier (torch.nn.Module): A classifier.
device (torch.device)
Returns:
Classification results in shape (len(data_loader), :math:`|\mathcal{C}|`).
"""
training = classifier.training
classifier.eval()
all_outputs = []
with torch.no_grad():
for i, (images, target) in enumerate(data_loader):
images = images.to(device)
output = classifier(images)
all_outputs.append(output)
classifier.train(training)
return torch.cat(all_outputs, dim=0)
================================================
FILE: domain_generalization/Transfer-Learning-Library/dalib/adaptation/regda.py
================================================
"""
@author: Junguang Jiang
@contact: JiangJunguang1123@outlook.com
"""
from typing import Optional
import torch
import torch.nn as nn
import torch.nn.functional as F
import numpy as np
from dalib.modules.gl import WarmStartGradientLayer
from common.utils.metric.keypoint_detection import get_max_preds
class FastPseudoLabelGenerator2d(nn.Module):
def __init__(self, sigma=2):
super().__init__()
self.sigma = sigma
def forward(self, heatmap: torch.Tensor):
heatmap = heatmap.detach()
height, width = heatmap.shape[-2:]
idx = heatmap.flatten(-2).argmax(dim=-1) # B, K
pred_h, pred_w = idx.div(width, rounding_mode='floor'), idx.remainder(width) # B, K
delta_h = torch.arange(height, device=heatmap.device) - pred_h.unsqueeze(-1) # B, K, H
delta_w = torch.arange(width, device=heatmap.device) - pred_w.unsqueeze(-1) # B, K, W
gaussian = (delta_h.square().unsqueeze(-1) + delta_w.square().unsqueeze(-2)).div(-2 * self.sigma * self.sigma).exp() # B, K, H, W
ground_truth = F.threshold(gaussian, threshold=1e-2, value=0.)
ground_false = (ground_truth.sum(dim=1, keepdim=True) - ground_truth).clamp(0., 1.)
return ground_truth, ground_false
class PseudoLabelGenerator2d(nn.Module):
"""
Generate ground truth heatmap and ground false heatmap from a prediction.
Args:
num_keypoints (int): Number of keypoints
height (int): height of the heatmap. Default: 64
width (int): width of the heatmap. Default: 64
sigma (int): sigma parameter when generate the heatmap. Default: 2
Inputs:
- y: predicted heatmap
Outputs:
- ground_truth: heatmap conforming to Gaussian distribution
- ground_false: ground false heatmap
Shape:
- y: :math:`(minibatch, K, H, W)` where K means the number of keypoints,
H and W is the height and width of the heatmap respectively.
- ground_truth: :math:`(minibatch, K, H, W)`
- ground_false: :math:`(minibatch, K, H, W)`
"""
def __init__(self, num_keypoints, height=64, width=64, sigma=2):
super(PseudoLabelGenerator2d, self).__init__()
self.height = height
self.width = width
self.sigma = sigma
heatmaps = np.zeros((width, height, height, width), dtype=np.float32)
tmp_size = sigma * 3
for mu_x in range(width):
for mu_y in range(height):
# Check that any part of the gaussian is in-bounds
ul = [int(mu_x - tmp_size), int(mu_y - tmp_size)]
br = [int(mu_x + tmp_size + 1), int(mu_y + tmp_size + 1)]
# Generate gaussian
size = 2 * tmp_size + 1
x = np.arange(0, size, 1, np.float32)
y = x[:, np.newaxis]
x0 = y0 = size // 2
# The gaussian is not normalized, we want the center value to equal 1
g = np.exp(- ((x - x0) ** 2 + (y - y0) ** 2) / (2 * sigma ** 2))
# Usable gaussian range
g_x = max(0, -ul[0]), min(br[0], width) - ul[0]
g_y = max(0, -ul[1]), min(br[1], height) - ul[1]
# Image range
img_x = max(0, ul[0]), min(br[0], width)
img_y = max(0, ul[1]), min(br[1], height)
heatmaps[mu_x][mu_y][img_y[0]:img_y[1], img_x[0]:img_x[1]] = \
g[g_y[0]:g_y[1], g_x[0]:g_x[1]]
self.heatmaps = heatmaps
self.false_matrix = 1. - np.eye(num_keypoints, dtype=np.float32)
def forward(self, y):
B, K, H, W = y.shape
y = y.detach()
preds, max_vals = get_max_preds(y.cpu().numpy()) # B x K x (x, y)
preds = preds.reshape(-1, 2).astype(np.int)
ground_truth = self.heatmaps[preds[:, 0], preds[:, 1], :, :].copy().reshape(B, K, H, W).copy()
ground_false = ground_truth.reshape(B, K, -1).transpose((0, 2, 1))
ground_false = ground_false.dot(self.false_matrix).clip(max=1., min=0.).transpose((0, 2, 1)).reshape(B, K, H, W).copy()
return torch.from_numpy(ground_truth).to(y.device), torch.from_numpy(ground_false).to(y.device)
class RegressionDisparity(nn.Module):
"""
Regression Disparity proposed by `Regressive Domain Adaptation for Unsupervised Keypoint Detection (CVPR 2021) `_.
Args:
pseudo_label_generator (PseudoLabelGenerator2d): generate ground truth heatmap and ground false heatmap
from a prediction.
criterion (torch.nn.Module): the loss function to calculate distance between two predictions.
Inputs:
- y: output by the main head
- y_adv: output by the adversarial head
- weight (optional): instance weights
- mode (str): whether minimize the disparity or maximize the disparity. Choices includes ``min``, ``max``.
Default: ``min``.
Shape:
- y: :math:`(minibatch, K, H, W)` where K means the number of keypoints,
H and W is the height and width of the heatmap respectively.
- y_adv: :math:`(minibatch, K, H, W)`
- weight: :math:`(minibatch, K)`.
- Output: depends on the ``criterion``.
Examples::
>>> num_keypoints = 5
>>> batch_size = 10
>>> H = W = 64
>>> pseudo_label_generator = PseudoLabelGenerator2d(num_keypoints)
>>> from common.vision.models.keypoint_detection.loss import JointsKLLoss
>>> loss = RegressionDisparity(pseudo_label_generator, JointsKLLoss())
>>> # output from source domain and target domain
>>> y_s, y_t = torch.randn(batch_size, num_keypoints, H, W), torch.randn(batch_size, num_keypoints, H, W)
>>> # adversarial output from source domain and target domain
>>> y_s_adv, y_t_adv = torch.randn(batch_size, num_keypoints, H, W), torch.randn(batch_size, num_keypoints, H, W)
>>> # minimize regression disparity on source domain
>>> output = loss(y_s, y_s_adv, mode='min')
>>> # maximize regression disparity on target domain
>>> output = loss(y_t, y_t_adv, mode='max')
"""
def __init__(self, pseudo_label_generator: PseudoLabelGenerator2d, criterion: nn.Module):
super(RegressionDisparity, self).__init__()
self.criterion = criterion
self.pseudo_label_generator = pseudo_label_generator
def forward(self, y, y_adv, weight=None, mode='min'):
assert mode in ['min', 'max']
ground_truth, ground_false = self.pseudo_label_generator(y.detach())
self.ground_truth = ground_truth
self.ground_false = ground_false
if mode == 'min':
return self.criterion(y_adv, ground_truth, weight)
else:
return self.criterion(y_adv, ground_false, weight)
class PoseResNet2d(nn.Module):
"""
Pose ResNet for RegDA has one backbone, one upsampling, while two regression heads.
Args:
backbone (torch.nn.Module): Backbone to extract 2-d features from data
upsampling (torch.nn.Module): Layer to upsample image feature to heatmap size
feature_dim (int): The dimension of the features from upsampling layer.
num_keypoints (int): Number of keypoints
gl (WarmStartGradientLayer):
finetune (bool, optional): Whether use 10x smaller learning rate in the backbone. Default: True
num_head_layers (int): Number of head layers. Default: 2
Inputs:
- x (tensor): input data
Outputs:
- outputs: logits outputs by the main regressor
- outputs_adv: logits outputs by the adversarial regressor
Shape:
- x: :math:`(minibatch, *)`, same shape as the input of the `backbone`.
- outputs, outputs_adv: :math:`(minibatch, K, H, W)`, where K means the number of keypoints.
.. note::
Remember to call function `step()` after function `forward()` **during training phase**! For instance,
>>> # x is inputs, model is an PoseResNet
>>> outputs, outputs_adv = model(x)
>>> model.step()
"""
def __init__(self, backbone, upsampling, feature_dim, num_keypoints,
gl: Optional[WarmStartGradientLayer] = None, finetune: Optional[bool] = True, num_head_layers=2):
super(PoseResNet2d, self).__init__()
self.backbone = backbone
self.upsampling = upsampling
self.head = self._make_head(num_head_layers, feature_dim, num_keypoints)
self.head_adv = self._make_head(num_head_layers, feature_dim, num_keypoints)
self.finetune = finetune
self.gl_layer = WarmStartGradientLayer(alpha=1.0, lo=0.0, hi=0.1, max_iters=1000, auto_step=False) if gl is None else gl
@staticmethod
def _make_head(num_layers, channel_dim, num_keypoints):
layers = []
for i in range(num_layers-1):
layers.extend([
nn.Conv2d(channel_dim, channel_dim, 3, 1, 1),
nn.BatchNorm2d(channel_dim),
nn.ReLU(),
])
layers.append(
nn.Conv2d(
in_channels=channel_dim,
out_channels=num_keypoints,
kernel_size=1,
stride=1,
padding=0
)
)
layers = nn.Sequential(*layers)
for m in layers.modules():
if isinstance(m, nn.Conv2d):
nn.init.normal_(m.weight, std=0.001)
nn.init.constant_(m.bias, 0)
return layers
def forward(self, x):
x = self.backbone(x)
f = self.upsampling(x)
f_adv = self.gl_layer(f)
y = self.head(f)
y_adv = self.head_adv(f_adv)
if self.training:
return y, y_adv
else:
return y
def get_parameters(self, lr=1.):
return [
{'params': self.backbone.parameters(), 'lr': 0.1 * lr if self.finetune else lr},
{'params': self.upsampling.parameters(), 'lr': lr},
{'params': self.head.parameters(), 'lr': lr},
{'params': self.head_adv.parameters(), 'lr': lr},
]
def step(self):
"""Call step() each iteration during training.
Will increase :math:`\lambda` in GL layer.
"""
self.gl_layer.step()
================================================
FILE: domain_generalization/Transfer-Learning-Library/dalib/adaptation/rsd.py
================================================
"""
@author: Junguang Jiang
@contact: JiangJunguang1123@outlook.com
"""
import torch.nn as nn
import torch
class RepresentationSubspaceDistance(nn.Module):
"""
`Representation Subspace Distance (ICML 2021) `_
Args:
trade_off (float): The trade-off value between Representation Subspace Distance
and Base Mismatch Penalization. Default: 0.1
Inputs:
- f_s (tensor): feature representations on source domain, :math:`f^s`
- f_t (tensor): feature representations on target domain, :math:`f^t`
"""
def __init__(self, trade_off=0.1):
super(RepresentationSubspaceDistance, self).__init__()
self.trade_off = trade_off
def forward(self, f_s, f_t):
U_s, _, _ = torch.svd(f_s.t())
U_t, _, _ = torch.svd(f_t.t())
P_s, cosine, P_t = torch.svd(torch.mm(U_s.t(), U_t))
sine = torch.sqrt(1 - torch.pow(cosine, 2))
rsd = torch.norm(sine, 1) # Representation Subspace Distance
bmp = torch.norm(torch.abs(P_s) - torch.abs(P_t), 2) # Base Mismatch Penalization
return rsd + self.trade_off * bmp
================================================
FILE: domain_generalization/Transfer-Learning-Library/dalib/adaptation/self_ensemble.py
================================================
"""
@author: Baixu Chen
@contact: cbx_99_hasta@outlook.com
"""
from typing import Optional, Callable
import copy
import torch
import torch.nn as nn
import torch.nn.functional as F
from common.modules.classifier import Classifier as ClassifierBase
from dalib.translation.cyclegan.util import set_requires_grad
class ConsistencyLoss(nn.Module):
r"""
Consistency loss between output of student model and output of teacher model.
Given distance measure :math:`D`, student model's output :math:`y`, teacher
model's output :math:`y_{teacher}`, binary mask :math:`mask`, consistency loss is
.. math::
D(y, y_{teacher}) * mask
Args:
distance_measure (callable): Distance measure function.
reduction (str, optional): Specifies the reduction to apply to the output:
``'none'`` | ``'mean'`` | ``'sum'``. ``'none'``: no reduction will be applied,
``'mean'``: the sum of the output will be divided by the number of
elements in the output, ``'sum'``: the output will be summed. Default: ``'mean'``
Inputs:
- y: predictions from student model
- y_teacher: predictions from teacher model
- mask: binary mask
Shape:
- y, y_teacher: :math:`(N, C)` where C means the number of classes.
- mask: :math:`(N, )` where N means mini-batch size.
"""
def __init__(self, distance_measure: Callable, reduction: Optional[str] = 'mean'):
super(ConsistencyLoss, self).__init__()
self.distance_measure = distance_measure
self.reduction = reduction
def forward(self, y: torch.Tensor, y_teacher: torch.Tensor, mask: torch.Tensor):
cons_loss = self.distance_measure(y, y_teacher)
cons_loss = cons_loss * mask
if self.reduction == 'mean':
return cons_loss.mean()
else:
return cons_loss
class L2ConsistencyLoss(ConsistencyLoss):
r"""
L2 consistency loss. Given student model's output :math:`y`, teacher model's output :math:`y_{teacher}`
and binary mask :math:`mask`, L2 consistency loss is
.. math::
\text{MSELoss}(y, y_{teacher}) * mask
"""
def __init__(self, reduction: Optional[str] = 'mean'):
def l2_distance(y: torch.Tensor, y_teacher: torch.Tensor):
return ((y - y_teacher) ** 2).sum(dim=1)
super(L2ConsistencyLoss, self).__init__(l2_distance, reduction)
class ClassBalanceLoss(nn.Module):
r"""
Class balance loss that penalises the network for making predictions that exhibit large class imbalance.
Given predictions :math:`y` with dimension :math:`(N, C)`, we first calculate mean across mini-batch dimension,
resulting in mini-batch mean per-class probability :math:`y_{mean}` with dimension :math:`(C, )`
.. math::
y_{mean}^j = \frac{1}{N} \sum_{i=1}^N y_i^j
Then we calculate binary cross entropy loss between :math:`y_{mean}` and uniform probability vector :math:`u` with
the same dimension where :math:`u^j` = :math:`\frac{1}{C}`
.. math::
loss = \text{BCELoss}(y_{mean}, u)
Args:
num_classes (int): Number of classes
Inputs:
- y (tensor): predictions from classifier
Shape:
- y: :math:`(N, C)` where C means the number of classes.
"""
def __init__(self, num_classes):
super(ClassBalanceLoss, self).__init__()
self.uniform_distribution = torch.ones(num_classes) / num_classes
def forward(self, y: torch.Tensor):
return F.binary_cross_entropy(y.mean(dim=0), self.uniform_distribution.to(y.device))
class EmaTeacher(object):
r"""
Exponential moving average model used in `Self-ensembling for Visual Domain Adaptation (ICLR 2018) `_
We denote :math:`\theta_t'` as the parameters of teacher model at training step t, :math:`\theta_t` as the
parameters of student model at training step t, :math:`\alpha` as decay rate. Then we update teacher model in an
exponential moving average manner as follows
.. math::
\theta_t'=\alpha \theta_{t-1}' + (1-\alpha)\theta_t
Args:
model (torch.nn.Module): student model
alpha (float): decay rate for EMA.
Inputs:
x (tensor): input data fed to teacher model
Examples::
>>> classifier = ImageClassifier(backbone, num_classes=31, bottleneck_dim=256).to(device)
>>> # initialize teacher model
>>> teacher = EmaTeacher(classifier, 0.9)
>>> num_iterations = 1000
>>> for _ in range(num_iterations):
>>> # x denotes input of one mini-batch
>>> # you can get teacher model's output by teacher(x)
>>> y_teacher = teacher(x)
>>> # when you want to update teacher, you should call teacher.update()
>>> teacher.update()
"""
def __init__(self, model, alpha):
self.model = model
self.alpha = alpha
self.teacher = copy.deepcopy(model)
set_requires_grad(self.teacher, False)
def set_alpha(self, alpha: float):
assert alpha >= 0
self.alpha = alpha
def update(self):
for teacher_param, param in zip(self.teacher.parameters(), self.model.parameters()):
teacher_param.data = self.alpha * teacher_param + (1 - self.alpha) * param
def __call__(self, x: torch.Tensor):
return self.teacher(x)
def train(self, mode: Optional[bool] = True):
self.teacher.train(mode)
def eval(self):
self.train(False)
def state_dict(self):
return self.teacher.state_dict()
def load_state_dict(self, state_dict):
self.teacher.load_state_dict(state_dict)
@property
def module(self):
return self.teacher.module
class ImageClassifier(ClassifierBase):
def __init__(self, backbone: nn.Module, num_classes: int, bottleneck_dim: Optional[int] = 256, **kwargs):
bottleneck = nn.Sequential(
# nn.AdaptiveAvgPool2d(output_size=(1, 1)),
# nn.Flatten(),
nn.Linear(backbone.out_features, bottleneck_dim),
nn.BatchNorm1d(bottleneck_dim),
nn.ReLU()
)
super(ImageClassifier, self).__init__(backbone, num_classes, bottleneck, bottleneck_dim, **kwargs)
================================================
FILE: domain_generalization/Transfer-Learning-Library/dalib/modules/__init__.py
================================================
from .grl import *
from .domain_discriminator import *
from .kernels import *
from .entropy import *
__all__ = ['grl', 'kernels', 'domain_discriminator', 'entropy']
================================================
FILE: domain_generalization/Transfer-Learning-Library/dalib/modules/domain_discriminator.py
================================================
"""
@author: Junguang Jiang
@contact: JiangJunguang1123@outlook.com
"""
from typing import List, Dict
import torch.nn as nn
__all__ = ['DomainDiscriminator']
class DomainDiscriminator(nn.Sequential):
r"""Domain discriminator model from
`Domain-Adversarial Training of Neural Networks (ICML 2015) `_
Distinguish whether the input features come from the source domain or the target domain.
The source domain label is 1 and the target domain label is 0.
Args:
in_feature (int): dimension of the input feature
hidden_size (int): dimension of the hidden features
batch_norm (bool): whether use :class:`~torch.nn.BatchNorm1d`.
Use :class:`~torch.nn.Dropout` if ``batch_norm`` is False. Default: True.
Shape:
- Inputs: (minibatch, `in_feature`)
- Outputs: :math:`(minibatch, 1)`
"""
def __init__(self, in_feature: int, hidden_size: int, batch_norm=True):
if batch_norm:
super(DomainDiscriminator, self).__init__(
nn.Linear(in_feature, hidden_size),
nn.BatchNorm1d(hidden_size),
nn.ReLU(),
nn.Linear(hidden_size, hidden_size),
nn.BatchNorm1d(hidden_size),
nn.ReLU(),
nn.Linear(hidden_size, 1),
nn.Sigmoid()
)
else:
super(DomainDiscriminator, self).__init__(
nn.Linear(in_feature, hidden_size),
nn.ReLU(inplace=True),
nn.Dropout(0.5),
nn.Linear(hidden_size, hidden_size),
nn.ReLU(inplace=True),
nn.Dropout(0.5),
nn.Linear(hidden_size, 1),
nn.Sigmoid()
)
def get_parameters(self) -> List[Dict]:
return [{"params": self.parameters(), "lr": 1.}]
================================================
FILE: domain_generalization/Transfer-Learning-Library/dalib/modules/entropy.py
================================================
"""
@author: Junguang Jiang
@contact: JiangJunguang1123@outlook.com
"""
import torch
def entropy(predictions: torch.Tensor, reduction='none') -> torch.Tensor:
r"""Entropy of prediction.
The definition is:
.. math::
entropy(p) = - \sum_{c=1}^C p_c \log p_c
where C is number of classes.
Args:
predictions (tensor): Classifier predictions. Expected to contain raw, normalized scores for each class
reduction (str, optional): Specifies the reduction to apply to the output:
``'none'`` | ``'mean'``. ``'none'``: no reduction will be applied,
``'mean'``: the sum of the output will be divided by the number of
elements in the output. Default: ``'mean'``
Shape:
- predictions: :math:`(minibatch, C)` where C means the number of classes.
- Output: :math:`(minibatch, )` by default. If :attr:`reduction` is ``'mean'``, then scalar.
"""
epsilon = 1e-5
H = -predictions * torch.log(predictions + epsilon)
H = H.sum(dim=1)
if reduction == 'mean':
return H.mean()
else:
return H
================================================
FILE: domain_generalization/Transfer-Learning-Library/dalib/modules/gl.py
================================================
"""
@author: Junguang Jiang
@contact: JiangJunguang1123@outlook.com
"""
from typing import Optional, Any, Tuple
import numpy as np
import torch.nn as nn
from torch.autograd import Function
import torch
class GradientFunction(Function):
@staticmethod
def forward(ctx: Any, input: torch.Tensor, coeff: Optional[float] = 1.) -> torch.Tensor:
ctx.coeff = coeff
output = input * 1.0
return output
@staticmethod
def backward(ctx: Any, grad_output: torch.Tensor) -> Tuple[torch.Tensor, Any]:
return grad_output * ctx.coeff, None
class WarmStartGradientLayer(nn.Module):
"""Warm Start Gradient Layer :math:`\mathcal{R}(x)` with warm start
The forward and backward behaviours are:
.. math::
\mathcal{R}(x) = x,
\dfrac{ d\mathcal{R}} {dx} = \lambda I.
:math:`\lambda` is initiated at :math:`lo` and is gradually changed to :math:`hi` using the following schedule:
.. math::
\lambda = \dfrac{2(hi-lo)}{1+\exp(- α \dfrac{i}{N})} - (hi-lo) + lo
where :math:`i` is the iteration step.
Parameters:
- **alpha** (float, optional): :math:`α`. Default: 1.0
- **lo** (float, optional): Initial value of :math:`\lambda`. Default: 0.0
- **hi** (float, optional): Final value of :math:`\lambda`. Default: 1.0
- **max_iters** (int, optional): :math:`N`. Default: 1000
- **auto_step** (bool, optional): If True, increase :math:`i` each time `forward` is called.
Otherwise use function `step` to increase :math:`i`. Default: False
"""
def __init__(self, alpha: Optional[float] = 1.0, lo: Optional[float] = 0.0, hi: Optional[float] = 1.,
max_iters: Optional[int] = 1000., auto_step: Optional[bool] = False):
super(WarmStartGradientLayer, self).__init__()
self.alpha = alpha
self.lo = lo
self.hi = hi
self.iter_num = 0
self.max_iters = max_iters
self.auto_step = auto_step
def forward(self, input: torch.Tensor) -> torch.Tensor:
""""""
coeff = np.float(
2.0 * (self.hi - self.lo) / (1.0 + np.exp(-self.alpha * self.iter_num / self.max_iters))
- (self.hi - self.lo) + self.lo
)
if self.auto_step:
self.step()
return GradientFunction.apply(input, coeff)
def step(self):
"""Increase iteration number :math:`i` by 1"""
self.iter_num += 1
================================================
FILE: domain_generalization/Transfer-Learning-Library/dalib/modules/grl.py
================================================
"""
@author: Junguang Jiang
@contact: JiangJunguang1123@outlook.com
"""
from typing import Optional, Any, Tuple
import numpy as np
import torch.nn as nn
from torch.autograd import Function
import torch
class GradientReverseFunction(Function):
@staticmethod
def forward(ctx: Any, input: torch.Tensor, coeff: Optional[float] = 1.) -> torch.Tensor:
ctx.coeff = coeff
output = input * 1.0
return output
@staticmethod
def backward(ctx: Any, grad_output: torch.Tensor) -> Tuple[torch.Tensor, Any]:
return grad_output.neg() * ctx.coeff, None
class GradientReverseLayer(nn.Module):
def __init__(self):
super(GradientReverseLayer, self).__init__()
def forward(self, *input):
return GradientReverseFunction.apply(*input)
class WarmStartGradientReverseLayer(nn.Module):
"""Gradient Reverse Layer :math:`\mathcal{R}(x)` with warm start
The forward and backward behaviours are:
.. math::
\mathcal{R}(x) = x,
\dfrac{ d\mathcal{R}} {dx} = - \lambda I.
:math:`\lambda` is initiated at :math:`lo` and is gradually changed to :math:`hi` using the following schedule:
.. math::
\lambda = \dfrac{2(hi-lo)}{1+\exp(- α \dfrac{i}{N})} - (hi-lo) + lo
where :math:`i` is the iteration step.
Args:
alpha (float, optional): :math:`α`. Default: 1.0
lo (float, optional): Initial value of :math:`\lambda`. Default: 0.0
hi (float, optional): Final value of :math:`\lambda`. Default: 1.0
max_iters (int, optional): :math:`N`. Default: 1000
auto_step (bool, optional): If True, increase :math:`i` each time `forward` is called.
Otherwise use function `step` to increase :math:`i`. Default: False
"""
def __init__(self, alpha: Optional[float] = 1.0, lo: Optional[float] = 0.0, hi: Optional[float] = 1.,
max_iters: Optional[int] = 1000., auto_step: Optional[bool] = False):
super(WarmStartGradientReverseLayer, self).__init__()
self.alpha = alpha
self.lo = lo
self.hi = hi
self.iter_num = 0
self.max_iters = max_iters
self.auto_step = auto_step
def forward(self, input: torch.Tensor) -> torch.Tensor:
""""""
coeff = np.float(
2.0 * (self.hi - self.lo) / (1.0 + np.exp(-self.alpha * self.iter_num / self.max_iters))
- (self.hi - self.lo) + self.lo
)
if self.auto_step:
self.step()
return GradientReverseFunction.apply(input, coeff)
def step(self):
"""Increase iteration number :math:`i` by 1"""
self.iter_num += 1
================================================
FILE: domain_generalization/Transfer-Learning-Library/dalib/modules/kernels.py
================================================
"""
@author: Junguang Jiang
@contact: JiangJunguang1123@outlook.com
"""
from typing import Optional
import torch
import torch.nn as nn
__all__ = ['GaussianKernel']
class GaussianKernel(nn.Module):
r"""Gaussian Kernel Matrix
Gaussian Kernel k is defined by
.. math::
k(x_1, x_2) = \exp \left( - \dfrac{\| x_1 - x_2 \|^2}{2\sigma^2} \right)
where :math:`x_1, x_2 \in R^d` are 1-d tensors.
Gaussian Kernel Matrix K is defined on input group :math:`X=(x_1, x_2, ..., x_m),`
.. math::
K(X)_{i,j} = k(x_i, x_j)
Also by default, during training this layer keeps running estimates of the
mean of L2 distances, which are then used to set hyperparameter :math:`\sigma`.
Mathematically, the estimation is :math:`\sigma^2 = \dfrac{\alpha}{n^2}\sum_{i,j} \| x_i - x_j \|^2`.
If :attr:`track_running_stats` is set to ``False``, this layer then does not
keep running estimates, and use a fixed :math:`\sigma` instead.
Args:
sigma (float, optional): bandwidth :math:`\sigma`. Default: None
track_running_stats (bool, optional): If ``True``, this module tracks the running mean of :math:`\sigma^2`.
Otherwise, it won't track such statistics and always uses fix :math:`\sigma^2`. Default: ``True``
alpha (float, optional): :math:`\alpha` which decides the magnitude of :math:`\sigma^2` when track_running_stats is set to ``True``
Inputs:
- X (tensor): input group :math:`X`
Shape:
- Inputs: :math:`(minibatch, F)` where F means the dimension of input features.
- Outputs: :math:`(minibatch, minibatch)`
"""
def __init__(self, sigma: Optional[float] = None, track_running_stats: Optional[bool] = True,
alpha: Optional[float] = 1.):
super(GaussianKernel, self).__init__()
assert track_running_stats or sigma is not None
self.sigma_square = torch.tensor(sigma * sigma) if sigma is not None else None
self.track_running_stats = track_running_stats
self.alpha = alpha
def forward(self, X: torch.Tensor) -> torch.Tensor:
l2_distance_square = ((X.unsqueeze(0) - X.unsqueeze(1)) ** 2).sum(2)
if self.track_running_stats:
self.sigma_square = self.alpha * torch.mean(l2_distance_square.detach())
return torch.exp(-l2_distance_square / (2 * self.sigma_square))
================================================
FILE: domain_generalization/Transfer-Learning-Library/dalib/translation/__init__.py
================================================
from . import fourier_transform
__all__ = ['fourier_transform']
================================================
FILE: domain_generalization/Transfer-Learning-Library/dalib/translation/cycada.py
================================================
"""
@author: Junguang Jiang
@contact: JiangJunguang1123@outlook.com
"""
import torch.nn as nn
from torch import Tensor
class SemanticConsistency(nn.Module):
"""
Semantic consistency loss is introduced by
`CyCADA: Cycle-Consistent Adversarial Domain Adaptation (ICML 2018) `_
This helps to prevent label flipping during image translation.
Args:
ignore_index (tuple, optional): Specifies target values that are ignored
and do not contribute to the input gradient. When :attr:`size_average` is
``True``, the loss is averaged over non-ignored targets. Default: ().
reduction (string, optional): Specifies the reduction to apply to the output:
``'none'`` | ``'mean'`` | ``'sum'``. ``'none'``: no reduction will
be applied, ``'mean'``: the weighted mean of the output is taken,
``'sum'``: the output will be summed. Note: :attr:`size_average`
and :attr:`reduce` are in the process of being deprecated, and in
the meantime, specifying either of those two args will override
:attr:`reduction`. Default: ``'mean'``
Shape:
- Input: :math:`(N, C)` where `C = number of classes`, or
:math:`(N, C, d_1, d_2, ..., d_K)` with :math:`K \geq 1`
in the case of `K`-dimensional loss.
- Target: :math:`(N)` where each value is :math:`0 \leq \text{targets}[i] \leq C-1`, or
:math:`(N, d_1, d_2, ..., d_K)` with :math:`K \geq 1` in the case of
K-dimensional loss.
- Output: scalar.
If :attr:`reduction` is ``'none'``, then the same size as the target:
:math:`(N)`, or
:math:`(N, d_1, d_2, ..., d_K)` with :math:`K \geq 1` in the case
of K-dimensional loss.
Examples::
>>> loss = SemanticConsistency()
>>> input = torch.randn(3, 5, requires_grad=True)
>>> target = torch.empty(3, dtype=torch.long).random_(5)
>>> output = loss(input, target)
>>> output.backward()
"""
def __init__(self, ignore_index=(), reduction='mean'):
super(SemanticConsistency, self).__init__()
self.ignore_index = ignore_index
self.loss = nn.CrossEntropyLoss(ignore_index=-1, reduction=reduction)
def forward(self, input: Tensor, target: Tensor) -> Tensor:
for class_idx in self.ignore_index:
target[target == class_idx] = -1
return self.loss(input, target)
================================================
FILE: domain_generalization/Transfer-Learning-Library/dalib/translation/cyclegan/__init__.py
================================================
from . import discriminator
from . import generator
from . import loss
from . import transform
from .discriminator import *
from .generator import *
from .loss import *
from .transform import *
================================================
FILE: domain_generalization/Transfer-Learning-Library/dalib/translation/cyclegan/discriminator.py
================================================
"""
Modified from https://github.com/junyanz/pytorch-CycleGAN-and-pix2pix
@author: Junguang Jiang
@contact: JiangJunguang1123@outlook.com
"""
import torch.nn as nn
from torch.nn import init
import functools
from .util import get_norm_layer, init_weights
class NLayerDiscriminator(nn.Module):
"""Construct a PatchGAN discriminator
Args:
input_nc (int): the number of channels in input images.
ndf (int): the number of filters in the last conv layer. Default: 64
n_layers (int): the number of conv layers in the discriminator. Default: 3
norm_layer (torch.nn.Module): normalization layer. Default: :class:`nn.BatchNorm2d`
"""
def __init__(self, input_nc, ndf=64, n_layers=3, norm_layer=nn.BatchNorm2d):
super(NLayerDiscriminator, self).__init__()
if type(norm_layer) == functools.partial: # no need to use bias as BatchNorm2d has affine parameters
use_bias = norm_layer.func == nn.InstanceNorm2d
else:
use_bias = norm_layer == nn.InstanceNorm2d
kw = 4
padw = 1
sequence = [nn.Conv2d(input_nc, ndf, kernel_size=kw, stride=2, padding=padw), nn.LeakyReLU(0.2, True)]
nf_mult = 1
nf_mult_prev = 1
for n in range(1, n_layers): # gradually increase the number of filters
nf_mult_prev = nf_mult
nf_mult = min(2 ** n, 8)
sequence += [
nn.Conv2d(ndf * nf_mult_prev, ndf * nf_mult, kernel_size=kw, stride=2, padding=padw, bias=use_bias),
norm_layer(ndf * nf_mult),
nn.LeakyReLU(0.2, True)
]
nf_mult_prev = nf_mult
nf_mult = min(2 ** n_layers, 8)
sequence += [
nn.Conv2d(ndf * nf_mult_prev, ndf * nf_mult, kernel_size=kw, stride=1, padding=padw, bias=use_bias),
norm_layer(ndf * nf_mult),
nn.LeakyReLU(0.2, True)
]
sequence += [nn.Conv2d(ndf * nf_mult, 1, kernel_size=kw, stride=1, padding=padw)] # output 1 channel prediction map
self.model = nn.Sequential(*sequence)
def forward(self, input):
return self.model(input)
class PixelDiscriminator(nn.Module):
"""Construct a 1x1 PatchGAN discriminator (pixelGAN)
Args:
input_nc (int): the number of channels in input images.
ndf (int): the number of filters in the last conv layer. Default: 64
norm_layer (torch.nn.Module): normalization layer. Default: :class:`nn.BatchNorm2d`
"""
def __init__(self, input_nc, ndf=64, norm_layer=nn.BatchNorm2d):
super(PixelDiscriminator, self).__init__()
if type(norm_layer) == functools.partial: # no need to use bias as BatchNorm2d has affine parameters
use_bias = norm_layer.func == nn.InstanceNorm2d
else:
use_bias = norm_layer == nn.InstanceNorm2d
self.net = [
nn.Conv2d(input_nc, ndf, kernel_size=1, stride=1, padding=0),
nn.LeakyReLU(0.2, True),
nn.Conv2d(ndf, ndf * 2, kernel_size=1, stride=1, padding=0, bias=use_bias),
norm_layer(ndf * 2),
nn.LeakyReLU(0.2, True),
nn.Conv2d(ndf * 2, 1, kernel_size=1, stride=1, padding=0, bias=use_bias)]
self.net = nn.Sequential(*self.net)
def forward(self, input):
return self.net(input)
def patch(ndf, input_nc=3, norm='batch', n_layers=3, init_type='normal', init_gain=0.02):
"""
PatchGAN classifier described in the original pix2pix paper.
It can classify whether 70×70 overlapping patches are real or fake.
Such a patch-level discriminator architecture has fewer parameters
than a full-image discriminator and can work on arbitrarily-sized images
in a fully convolutional fashion.
Args:
ndf (int): the number of filters in the first conv layer
input_nc (int): the number of channels in input images. Default: 3
norm (str): the type of normalization layers used in the network. Default: 'batch'
n_layers (int): the number of conv layers in the discriminator. Default: 3
init_type (str): the name of the initialization method. Choices includes: ``normal`` |
``xavier`` | ``kaiming`` | ``orthogonal``. Default: 'normal'
init_gain (float): scaling factor for normal, xavier and orthogonal. Default: 0.02
"""
norm_layer = get_norm_layer(norm_type=norm)
net = NLayerDiscriminator(input_nc, ndf, n_layers=n_layers, norm_layer=norm_layer)
init_weights(net, init_type, init_gain=init_gain)
return net
def pixel(ndf, input_nc=3, norm='batch', init_type='normal', init_gain=0.02):
"""
1x1 PixelGAN discriminator can classify whether a pixel is real or not.
It encourages greater color diversity but has no effect on spatial statistics.
Args:
ndf (int): the number of filters in the first conv layer
input_nc (int): the number of channels in input images. Default: 3
norm (str): the type of normalization layers used in the network. Default: 'batch'
init_type (str): the name of the initialization method. Choices includes: ``normal`` |
``xavier`` | ``kaiming`` | ``orthogonal``. Default: 'normal'
init_gain (float): scaling factor for normal, xavier and orthogonal. Default: 0.02
"""
norm_layer = get_norm_layer(norm_type=norm)
net = PixelDiscriminator(input_nc, ndf, norm_layer=norm_layer)
init_weights(net, init_type, init_gain=init_gain)
return net
================================================
FILE: domain_generalization/Transfer-Learning-Library/dalib/translation/cyclegan/generator.py
================================================
"""
Modified from https://github.com/junyanz/pytorch-CycleGAN-and-pix2pix
@author: Junguang Jiang
@contact: JiangJunguang1123@outlook.com
"""
import torch
import torch.nn as nn
import functools
from .util import get_norm_layer, init_weights
class ResnetBlock(nn.Module):
"""Define a Resnet block"""
def __init__(self, dim, padding_type, norm_layer, use_dropout, use_bias):
"""Initialize the Resnet block
A resnet block is a conv block with skip connections
We construct a conv block with build_conv_block function,
and implement skip connections in function.
Original Resnet paper: https://arxiv.org/pdf/1512.03385.pdf
"""
super(ResnetBlock, self).__init__()
self.conv_block = self.build_conv_block(dim, padding_type, norm_layer, use_dropout, use_bias)
def build_conv_block(self, dim, padding_type, norm_layer, use_dropout, use_bias):
"""Construct a convolutional block.
Args:
dim (int): the number of channels in the conv layer.
padding_type (str): the name of padding layer: reflect | replicate | zero
norm_layer (torch.nn.Module): normalization layer
use_dropout (bool): if use dropout layers.
use_bias (bool): if the conv layer uses bias or not
Returns a conv block (with a conv layer, a normalization layer, and a non-linearity layer (ReLU))
"""
conv_block = []
p = 0
if padding_type == 'reflect':
conv_block += [nn.ReflectionPad2d(1)]
elif padding_type == 'replicate':
conv_block += [nn.ReplicationPad2d(1)]
elif padding_type == 'zero':
p = 1
else:
raise NotImplementedError('padding [%s] is not implemented' % padding_type)
conv_block += [nn.Conv2d(dim, dim, kernel_size=3, padding=p, bias=use_bias), norm_layer(dim), nn.ReLU(True)]
if use_dropout:
conv_block += [nn.Dropout(0.5)]
p = 0
if padding_type == 'reflect':
conv_block += [nn.ReflectionPad2d(1)]
elif padding_type == 'replicate':
conv_block += [nn.ReplicationPad2d(1)]
elif padding_type == 'zero':
p = 1
else:
raise NotImplementedError('padding [%s] is not implemented' % padding_type)
conv_block += [nn.Conv2d(dim, dim, kernel_size=3, padding=p, bias=use_bias), norm_layer(dim)]
return nn.Sequential(*conv_block)
def forward(self, x):
"""Forward function (with skip connections)"""
out = x + self.conv_block(x) # add skip connections
return out
class ResnetGenerator(nn.Module):
"""Resnet-based generator that consists of Resnet blocks between a few downsampling/upsampling operations.
We adapt Torch code and idea from Justin Johnson's neural style transfer project(https://github.com/jcjohnson/fast-neural-style)
"""
def __init__(self, input_nc, output_nc, ngf=64, norm_layer=nn.BatchNorm2d, use_dropout=False, n_blocks=6, padding_type='reflect'):
"""Construct a Resnet-based generator
Args:
input_nc (int): the number of channels in input images
output_nc (int): the number of channels in output images
ngf (int): the number of filters in the last conv layer
norm_layer (torch.nn.Module): normalization layer
use_dropout (bool): if use dropout layers
n_blocks (int): the number of ResNet blocks
padding_type (str): the name of padding layer in conv layers: reflect | replicate | zero
"""
assert(n_blocks >= 0)
super(ResnetGenerator, self).__init__()
if type(norm_layer) == functools.partial:
use_bias = norm_layer.func == nn.InstanceNorm2d
else:
use_bias = norm_layer == nn.InstanceNorm2d
model = [nn.ReflectionPad2d(3),
nn.Conv2d(input_nc, ngf, kernel_size=7, padding=0, bias=use_bias),
norm_layer(ngf),
nn.ReLU(True)]
n_downsampling = 2
for i in range(n_downsampling): # add downsampling layers
mult = 2 ** i
model += [nn.Conv2d(ngf * mult, ngf * mult * 2, kernel_size=3, stride=2, padding=1, bias=use_bias),
norm_layer(ngf * mult * 2),
nn.ReLU(True)]
mult = 2 ** n_downsampling
for i in range(n_blocks): # add ResNet blocks
model += [ResnetBlock(ngf * mult, padding_type=padding_type, norm_layer=norm_layer, use_dropout=use_dropout, use_bias=use_bias)]
for i in range(n_downsampling): # add upsampling layers
mult = 2 ** (n_downsampling - i)
model += [nn.ConvTranspose2d(ngf * mult, int(ngf * mult / 2),
kernel_size=3, stride=2,
padding=1, output_padding=1,
bias=use_bias),
norm_layer(int(ngf * mult / 2)),
nn.ReLU(True)]
model += [nn.ReflectionPad2d(3)]
model += [nn.Conv2d(ngf, output_nc, kernel_size=7, padding=0)]
model += [nn.Tanh()]
self.model = nn.Sequential(*model)
def forward(self, input):
"""Standard forward"""
return self.model(input)
class UnetGenerator(nn.Module):
"""Create a Unet-based generator"""
def __init__(self, input_nc, output_nc, num_downs, ngf=64, norm_layer=nn.BatchNorm2d, use_dropout=False):
"""Construct a Unet generator
Args:
input_nc (int): the number of channels in input images
output_nc (int): the number of channels in output images
num_downs (int): the number of downsamplings in UNet. For example, # if |num_downs| == 7,
image of size 128x128 will become of size 1x1 # at the bottleneck
ngf (int): the number of filters in the last conv layer
norm_layer(torch.nn.Module): normalization layer
We construct the U-Net from the innermost layer to the outermost layer.
It is a recursive process.
"""
super(UnetGenerator, self).__init__()
# construct unet structure
unet_block = UnetSkipConnectionBlock(ngf * 8, ngf * 8, input_nc=None, submodule=None, norm_layer=norm_layer, innermost=True) # add the innermost layer
for i in range(num_downs - 5): # add intermediate layers with ngf * 8 filters
unet_block = UnetSkipConnectionBlock(ngf * 8, ngf * 8, input_nc=None, submodule=unet_block, norm_layer=norm_layer, use_dropout=use_dropout)
# gradually reduce the number of filters from ngf * 8 to ngf
unet_block = UnetSkipConnectionBlock(ngf * 4, ngf * 8, input_nc=None, submodule=unet_block, norm_layer=norm_layer)
unet_block = UnetSkipConnectionBlock(ngf * 2, ngf * 4, input_nc=None, submodule=unet_block, norm_layer=norm_layer)
unet_block = UnetSkipConnectionBlock(ngf, ngf * 2, input_nc=None, submodule=unet_block, norm_layer=norm_layer)
self.model = UnetSkipConnectionBlock(output_nc, ngf, input_nc=input_nc, submodule=unet_block, outermost=True, norm_layer=norm_layer) # add the outermost layer
def forward(self, input):
"""Standard forward"""
return self.model(input)
class UnetSkipConnectionBlock(nn.Module):
"""Defines the Unet submodule with skip connection.
X -------------------identity----------------------
|-- downsampling -- |submodule| -- upsampling --|
"""
def __init__(self, outer_nc, inner_nc, input_nc=None,
submodule=None, outermost=False, innermost=False, norm_layer=nn.BatchNorm2d, use_dropout=False):
"""Construct a Unet submodule with skip connections.
Args:
outer_nc (int): the number of filters in the outer conv layer
inner_nc (int): the number of filters in the inner conv layer
input_nc (int): the number of channels in input images/features
submodule (UnetSkipConnectionBlock): previously defined submodules
outermost (bool): if this module is the outermost module
innermost (bool): if this module is the innermost module
norm_layer (torch.nn.Module): normalization layer
use_dropout (bool): if use dropout layers.
"""
super(UnetSkipConnectionBlock, self).__init__()
self.outermost = outermost
if type(norm_layer) == functools.partial:
use_bias = norm_layer.func == nn.InstanceNorm2d
else:
use_bias = norm_layer == nn.InstanceNorm2d
if input_nc is None:
input_nc = outer_nc
downconv = nn.Conv2d(input_nc, inner_nc, kernel_size=4,
stride=2, padding=1, bias=use_bias)
downrelu = nn.LeakyReLU(0.2, True)
downnorm = norm_layer(inner_nc)
uprelu = nn.ReLU(True)
upnorm = norm_layer(outer_nc)
if outermost:
upconv = nn.ConvTranspose2d(inner_nc * 2, outer_nc,
kernel_size=4, stride=2,
padding=1)
down = [downconv]
up = [uprelu, upconv, nn.Tanh()]
model = down + [submodule] + up
elif innermost:
upconv = nn.ConvTranspose2d(inner_nc, outer_nc,
kernel_size=4, stride=2,
padding=1, bias=use_bias)
down = [downrelu, downconv]
up = [uprelu, upconv, upnorm]
model = down + up
else:
upconv = nn.ConvTranspose2d(inner_nc * 2, outer_nc,
kernel_size=4, stride=2,
padding=1, bias=use_bias)
down = [downrelu, downconv, downnorm]
up = [uprelu, upconv, upnorm]
if use_dropout:
model = down + [submodule] + up + [nn.Dropout(0.5)]
else:
model = down + [submodule] + up
self.model = nn.Sequential(*model)
def forward(self, x):
if self.outermost:
return self.model(x)
else: # add skip connections
return torch.cat([x, self.model(x)], 1)
def resnet_9(ngf, input_nc=3, output_nc=3, norm='batch', use_dropout=False,
init_type='normal', init_gain=0.02):
"""
Resnet-based generator with 9 Resnet blocks.
Args:
ngf (int): the number of filters in the last conv layer
input_nc (int): the number of channels in input images. Default: 3
output_nc (int): the number of channels in output images. Default: 3
norm (str): the type of normalization layers used in the network. Default: 'batch'
use_dropout (bool): whether use dropout. Default: False
init_type (str): the name of the initialization method. Choices includes: ``normal`` |
``xavier`` | ``kaiming`` | ``orthogonal``. Default: 'normal'
init_gain (float): scaling factor for normal, xavier and orthogonal. Default: 0.02
"""
norm_layer = get_norm_layer(norm_type=norm)
net = ResnetGenerator(input_nc, output_nc, ngf, norm_layer=norm_layer, use_dropout=use_dropout, n_blocks=9)
init_weights(net, init_type, init_gain)
return net
def resnet_6(ngf, input_nc=3, output_nc=3, norm='batch', use_dropout=False,
init_type='normal', init_gain=0.02):
"""
Resnet-based generator with 6 Resnet blocks.
Args:
ngf (int): the number of filters in the last conv layer
input_nc (int): the number of channels in input images. Default: 3
output_nc (int): the number of channels in output images. Default: 3
norm (str): the type of normalization layers used in the network. Default: 'batch'
use_dropout (bool): whether use dropout. Default: False
init_type (str): the name of the initialization method. Choices includes: ``normal`` |
``xavier`` | ``kaiming`` | ``orthogonal``. Default: 'normal'
init_gain (float): scaling factor for normal, xavier and orthogonal. Default: 0.02
"""
norm_layer = get_norm_layer(norm_type=norm)
net = ResnetGenerator(input_nc, output_nc, ngf, norm_layer=norm_layer, use_dropout=use_dropout, n_blocks=6)
init_weights(net, init_type, init_gain)
return net
def unet_256(ngf, input_nc=3, output_nc=3, norm='batch', use_dropout=False,
init_type='normal', init_gain=0.02):
"""
`U-Net `_ generator for 256x256 input images.
The size of the input image should be a multiple of 256.
Args:
ngf (int): the number of filters in the last conv layer
input_nc (int): the number of channels in input images. Default: 3
output_nc (int): the number of channels in output images. Default: 3
norm (str): the type of normalization layers used in the network. Default: 'batch'
use_dropout (bool): whether use dropout. Default: False
init_type (str): the name of the initialization method. Choices includes: ``normal`` |
``xavier`` | ``kaiming`` | ``orthogonal``. Default: 'normal'
init_gain (float): scaling factor for normal, xavier and orthogonal. Default: 0.02
"""
norm_layer = get_norm_layer(norm_type=norm)
net = UnetGenerator(input_nc, output_nc, 8, ngf, norm_layer=norm_layer, use_dropout=use_dropout)
init_weights(net, init_type, init_gain)
return net
def unet_128(ngf, input_nc=3, output_nc=3, norm='batch', use_dropout=False,
init_type='normal', init_gain=0.02):
"""
`U-Net `_ generator for 128x128 input images.
The size of the input image should be a multiple of 128.
Args:
ngf (int): the number of filters in the last conv layer
input_nc (int): the number of channels in input images. Default: 3
output_nc (int): the number of channels in output images. Default: 3
norm (str): the type of normalization layers used in the network. Default: 'batch'
use_dropout (bool): whether use dropout. Default: False
init_type (str): the name of the initialization method. Choices includes: ``normal`` |
``xavier`` | ``kaiming`` | ``orthogonal``. Default: 'normal'
init_gain (float): scaling factor for normal, xavier and orthogonal. Default: 0.02
"""
norm_layer = get_norm_layer(norm_type=norm)
net = UnetGenerator(input_nc, output_nc, 7, ngf, norm_layer=norm_layer, use_dropout=use_dropout)
init_weights(net, init_type, init_gain)
return net
def unet_32(ngf, input_nc=3, output_nc=3, norm='batch', use_dropout=False,
init_type='normal', init_gain=0.02):
"""
`U-Net `_ generator for 32x32 input images
Args:
ngf (int): the number of filters in the last conv layer
input_nc (int): the number of channels in input images. Default: 3
output_nc (int): the number of channels in output images. Default: 3
norm (str): the type of normalization layers used in the network. Default: 'batch'
use_dropout (bool): whether use dropout. Default: False
init_type (str): the name of the initialization method. Choices includes: ``normal`` |
``xavier`` | ``kaiming`` | ``orthogonal``. Default: 'normal'
init_gain (float): scaling factor for normal, xavier and orthogonal. Default: 0.02
"""
norm_layer = get_norm_layer(norm_type=norm)
net = UnetGenerator(input_nc, output_nc, 5, ngf, norm_layer=norm_layer, use_dropout=use_dropout)
init_weights(net, init_type, init_gain)
return net
================================================
FILE: domain_generalization/Transfer-Learning-Library/dalib/translation/cyclegan/loss.py
================================================
"""
Modified from https://github.com/junyanz/pytorch-CycleGAN-and-pix2pix
@author: Junguang Jiang
@contact: JiangJunguang1123@outlook.com
"""
import torch.nn as nn
import torch
class LeastSquaresGenerativeAdversarialLoss(nn.Module):
"""
Loss for `Least Squares Generative Adversarial Network (LSGAN) `_
Args:
reduction (str, optional): Specifies the reduction to apply to the output:
``'none'`` | ``'mean'`` | ``'sum'``. ``'none'``: no reduction will be applied,
``'mean'``: the sum of the output will be divided by the number of
elements in the output, ``'sum'``: the output will be summed. Default: ``'mean'``
Inputs:
- prediction (tensor): unnormalized discriminator predictions
- real (bool): if the ground truth label is for real images or fake images. Default: true
.. warning::
Do not use sigmoid as the last layer of Discriminator.
"""
def __init__(self, reduction='mean'):
super(LeastSquaresGenerativeAdversarialLoss, self).__init__()
self.mse_loss = nn.MSELoss(reduction=reduction)
def forward(self, prediction, real=True):
if real:
label = torch.ones_like(prediction)
else:
label = torch.zeros_like(prediction)
return self.mse_loss(prediction, label)
class VanillaGenerativeAdversarialLoss(nn.Module):
"""
Loss for `Vanilla Generative Adversarial Network `_
Args:
reduction (str, optional): Specifies the reduction to apply to the output:
``'none'`` | ``'mean'`` | ``'sum'``. ``'none'``: no reduction will be applied,
``'mean'``: the sum of the output will be divided by the number of
elements in the output, ``'sum'``: the output will be summed. Default: ``'mean'``
Inputs:
- prediction (tensor): unnormalized discriminator predictions
- real (bool): if the ground truth label is for real images or fake images. Default: true
.. warning::
Do not use sigmoid as the last layer of Discriminator.
"""
def __init__(self, reduction='mean'):
super(VanillaGenerativeAdversarialLoss, self).__init__()
self.bce_loss = nn.BCEWithLogitsLoss(reduction=reduction)
def forward(self, prediction, real=True):
if real:
label = torch.ones_like(prediction)
else:
label = torch.zeros_like(prediction)
return self.bce_loss(prediction, label)
class WassersteinGenerativeAdversarialLoss(nn.Module):
"""
Loss for `Wasserstein Generative Adversarial Network `_
Args:
reduction (str, optional): Specifies the reduction to apply to the output:
``'none'`` | ``'mean'`` | ``'sum'``. ``'none'``: no reduction will be applied,
``'mean'``: the sum of the output will be divided by the number of
elements in the output, ``'sum'``: the output will be summed. Default: ``'mean'``
Inputs:
- prediction (tensor): unnormalized discriminator predictions
- real (bool): if the ground truth label is for real images or fake images. Default: true
.. warning::
Do not use sigmoid as the last layer of Discriminator.
"""
def __init__(self, reduction='mean'):
super(WassersteinGenerativeAdversarialLoss, self).__init__()
self.mse_loss = nn.MSELoss(reduction=reduction)
def forward(self, prediction, real=True):
if real:
return -prediction.mean()
else:
return prediction.mean()
================================================
FILE: domain_generalization/Transfer-Learning-Library/dalib/translation/cyclegan/transform.py
================================================
"""
@author: Junguang Jiang
@contact: JiangJunguang1123@outlook.com
"""
import torch
import torch.nn as nn
import torchvision.transforms as T
from common.vision.transforms import Denormalize
class Translation(nn.Module):
"""
Image Translation Transform Module
Args:
generator (torch.nn.Module): An image generator, e.g. :meth:`~dalib.translation.cyclegan.resnet_9_generator`
device (torch.device): device to put the generator. Default: 'cpu'
mean (tuple): the normalized mean for image
std (tuple): the normalized std for image
Input:
- image (PIL.Image): raw image in shape H x W x C
Output:
raw image in shape H x W x 3
"""
def __init__(self, generator, device=torch.device("cpu"), mean=(0.5, 0.5, 0.5), std=(0.5, 0.5, 0.5)):
super(Translation, self).__init__()
self.generator = generator.to(device)
self.device = device
self.pre_process = T.Compose([
T.ToTensor(),
T.Normalize(mean, std)
])
self.post_process = T.Compose([
Denormalize(mean, std),
T.ToPILImage()
])
def forward(self, image):
image = self.pre_process(image.copy()) # C x H x W
image = image.to(self.device)
generated_image = self.generator(image.unsqueeze(dim=0)).squeeze(dim=0).cpu()
return self.post_process(generated_image)
================================================
FILE: domain_generalization/Transfer-Learning-Library/dalib/translation/cyclegan/util.py
================================================
"""
@author: Junguang Jiang
@contact: JiangJunguang1123@outlook.com
"""
import torch.nn as nn
import functools
import random
import torch
from torch.nn import init
class Identity(nn.Module):
def forward(self, x):
return x
def get_norm_layer(norm_type='instance'):
"""Return a normalization layer
Parameters:
norm_type (str) -- the name of the normalization layer: batch | instance | none
For BatchNorm, we use learnable affine parameters and track running statistics (mean/stddev).
For InstanceNorm, we do not use learnable affine parameters. We do not track running statistics.
"""
if norm_type == 'batch':
norm_layer = functools.partial(nn.BatchNorm2d, affine=True, track_running_stats=True)
elif norm_type == 'instance':
norm_layer = functools.partial(nn.InstanceNorm2d, affine=False, track_running_stats=False)
elif norm_type == 'none':
def norm_layer(x): return Identity()
else:
raise NotImplementedError('normalization layer [%s] is not found' % norm_type)
return norm_layer
def init_weights(net, init_type='normal', init_gain=0.02):
"""Initialize network weights.
Args:
net (torch.nn.Module): network to be initialized
init_type (str): the name of an initialization method. Choices includes: ``normal`` |
``xavier`` | ``kaiming`` | ``orthogonal``
init_gain (float): scaling factor for normal, xavier and orthogonal.
'normal' is used in the original CycleGAN paper. But xavier and kaiming might
work better for some applications.
"""
def init_func(m): # define the initialization function
classname = m.__class__.__name__
if hasattr(m, 'weight') and (classname.find('Conv') != -1 or classname.find('Linear') != -1):
if init_type == 'normal':
init.normal_(m.weight.data, 0.0, init_gain)
elif init_type == 'xavier':
init.xavier_normal_(m.weight.data, gain=init_gain)
elif init_type == 'kaiming':
init.kaiming_normal_(m.weight.data, a=0, mode='fan_in')
elif init_type == 'orthogonal':
init.orthogonal_(m.weight.data, gain=init_gain)
else:
raise NotImplementedError('initialization method [%s] is not implemented' % init_type)
if hasattr(m, 'bias') and m.bias is not None:
init.constant_(m.bias.data, 0.0)
elif classname.find('BatchNorm2d') != -1: # BatchNorm Layer's weight is not a matrix; only normal distribution applies.
init.normal_(m.weight.data, 1.0, init_gain)
init.constant_(m.bias.data, 0.0)
print('initialize network with %s' % init_type)
net.apply(init_func) # apply the initialization function
class ImagePool:
"""An image buffer that stores previously generated images.
This buffer enables us to update discriminators using a history of generated images
rather than the ones produced by the latest generators.
Args:
pool_size (int): the size of image buffer, if pool_size=0, no buffer will be created
"""
def __init__(self, pool_size):
self.pool_size = pool_size
if self.pool_size > 0: # create an empty pool
self.num_imgs = 0
self.images = []
def query(self, images):
"""Return an image from the pool.
Args:
images (torch.Tensor): the latest generated images from the generator
Returns:
By 50/100, the buffer will return input images.
By 50/100, the buffer will return images previously stored in the buffer,
and insert the current images to the buffer.
"""
if self.pool_size == 0: # if the buffer size is 0, do nothing
return images
return_images = []
for image in images:
image = torch.unsqueeze(image.data, 0)
if self.num_imgs < self.pool_size: # if the buffer is not full; keep inserting current images to the buffer
self.num_imgs = self.num_imgs + 1
self.images.append(image)
return_images.append(image)
else:
p = random.uniform(0, 1)
if p > 0.5: # by 50% chance, the buffer will return a previously stored image, and insert the current image into the buffer
random_id = random.randint(0, self.pool_size - 1) # randint is inclusive
tmp = self.images[random_id].clone()
self.images[random_id] = image
return_images.append(tmp)
else: # by another 50% chance, the buffer will return the current image
return_images.append(image)
return_images = torch.cat(return_images, 0) # collect all the images and return
return return_images
def set_requires_grad(net, requires_grad=False):
"""
Set requies_grad=Fasle for all the networks to avoid unnecessary computations
"""
for param in net.parameters():
param.requires_grad = requires_grad
================================================
FILE: domain_generalization/Transfer-Learning-Library/dalib/translation/fourier_transform.py
================================================
"""
@author: Junguang Jiang
@contact: JiangJunguang1123@outlook.com
"""
import numpy as np
import os
import tqdm
import random
from PIL import Image
from typing import Optional, Sequence
import torch.nn as nn
def low_freq_mutate(amp_src: np.ndarray, amp_trg: np.ndarray, beta: Optional[int] = 1):
"""
Args:
amp_src (numpy.ndarray): amplitude component of the Fourier transform of source image
amp_trg (numpy.ndarray): amplitude component of the Fourier transform of target image
beta (int, optional): the size of the center region to be replace. Default: 1
Returns:
amplitude component of the Fourier transform of source image
whose low-frequency component is replaced by that of the target image.
"""
# Shift the zero-frequency component to the center of the spectrum.
a_src = np.fft.fftshift(amp_src, axes=(-2, -1))
a_trg = np.fft.fftshift(amp_trg, axes=(-2, -1))
# The low-frequency component includes
# the area where the horizontal and vertical distance from the center does not exceed beta
_, h, w = a_src.shape
c_h = np.floor(h / 2.0).astype(int)
c_w = np.floor(w / 2.0).astype(int)
h1 = c_h - beta
h2 = c_h + beta + 1
w1 = c_w - beta
w2 = c_w + beta + 1
# The low-frequency component of source amplitude is replaced by the target amplitude
a_src[:, h1:h2, w1:w2] = a_trg[:, h1:h2, w1:w2]
a_src = np.fft.ifftshift(a_src, axes=(-2, -1))
return a_src
class FourierTransform(nn.Module):
"""
Fourier Transform is introduced by `FDA: Fourier Domain Adaptation for Semantic Segmentation (CVPR 2020) `_
Fourier Transform replace the low frequency component of the amplitude of the source image to that of the target image.
Denote with :math:`M_{β}` a mask, whose value is zero except for the center region:
.. math::
M_{β}(h,w) = \mathbb{1}_{(h, w)\in [-β,β, -β, β]}
Given images :math:`x^s` from source domain and :math:`x^t` from target domain, the source image in the target style is
.. math::
x^{s→t} = \mathcal{F}^{-1}([ M_{β}\circ\mathcal{F}^A(x^t) + (1-M_{β})\circ\mathcal{F}^A(x^s), \mathcal{F}^P(x^s) ])
where :math:`\mathcal{F}^A`, :math:`\mathcal{F}^P` are the amplitude and phase component of the Fourier
Transform :math:`\mathcal{F}` of an RGB image.
Args:
image_list (sequence[str]): A sequence of image list from the target domain.
amplitude_dir (str): Specifies the directory to put the amplitude component of the target image.
beta (int, optional): :math:`β`. Default: 1.
rebuild (bool, optional): whether rebuild the amplitude component of the target image in the given directory.
Inputs:
- image (PIL Image): image from the source domain, :math:`x^t`.
Examples:
>>> from dalib.translation.fourier_transform import FourierTransform
>>> image_list = ["target_image_path1", "target_image_path2"]
>>> amplitude_dir = "path/to/amplitude_dir"
>>> fourier_transform = FourierTransform(image_list, amplitude_dir, beta=1, rebuild=False)
>>> source_image = np.array((256, 256, 3)) # image form source domain
>>> source_image_in_target_style = fourier_transform(source_image)
.. note::
The meaning of :math:`β` is different from that of the origin paper. Experimentally, we found that the size of
the center region in the frequency space should be constant when the image size increases. Thus we make the size
of the center region independent of the image size. A recommended value for :math:`β` is 1.
.. note::
The image structure of the source domain and target domain should be as similar as possible,
thus for segemntation tasks, FourierTransform should be used before RandomResizeCrop and other transformations.
.. note::
The image size of the source domain and the target domain need to be the same, thus before FourierTransform,
you should use Resize to convert the source image to the target image size.
Examples:
>>> from dalib.translation.fourier_transform import FourierTransform
>>> import common.vision.datasets.segmentation.transforms as T
>>> from PIL import Image
>>> target_image_list = ["target_image_path1", "target_image_path2"]
>>> amplitude_dir = "path/to/amplitude_dir"
>>> # build a fourier transform that translate source images to the target style
>>> fourier_transform = T.wrapper(FourierTransform)(target_image_list, amplitude_dir)
>>> transforms=T.Compose([
... # convert source image to the size of the target image before fourier transform
... T.Resize((2048, 1024)),
... fourier_transform,
... T.RandomResizedCrop((1024, 512)),
... T.RandomHorizontalFlip(),
... ])
>>> source_image = Image.open("path/to/source_image") # image form source domain
>>> source_image_in_target_style = transforms(source_image)
"""
# TODO add image examples when beta is different
def __init__(self, image_list: Sequence[str], amplitude_dir: str,
beta: Optional[int] = 1, rebuild: Optional[bool] = False):
super(FourierTransform, self).__init__()
self.amplitude_dir = amplitude_dir
if not os.path.exists(amplitude_dir) or rebuild:
os.makedirs(amplitude_dir, exist_ok=True)
self.build_amplitude(image_list, amplitude_dir)
self.beta = beta
self.length = len(image_list)
@staticmethod
def build_amplitude(image_list, amplitude_dir):
# extract amplitudes from target domain
for i, image_name in enumerate(tqdm.tqdm(image_list)):
image = Image.open(image_name).convert('RGB')
image = np.asarray(image, np.float32)
image = image.transpose((2, 0, 1))
fft = np.fft.fft2(image, axes=(-2, -1))
amp = np.abs(fft)
np.save(os.path.join(amplitude_dir, "{}.npy".format(i)), amp)
def forward(self, image):
# randomly sample a target image and load its amplitude component
amp_trg = np.load(os.path.join(self.amplitude_dir, "{}.npy".format(random.randint(0, self.length-1))))
image = np.asarray(image, np.float32)
image = image.transpose((2, 0, 1))
# get fft, amplitude on source domain
fft_src = np.fft.fft2(image, axes=(-2, -1))
amp_src, pha_src = np.abs(fft_src), np.angle(fft_src)
# mutate the amplitude part of source with target
amp_src_ = low_freq_mutate(amp_src, amp_trg, beta=self.beta)
# mutated fft of source
fft_src_ = amp_src_ * np.exp(1j * pha_src)
# get the mutated image
src_in_trg = np.fft.ifft2(fft_src_, axes=(-2, -1))
src_in_trg = np.real(src_in_trg)
src_in_trg = src_in_trg.transpose((1, 2, 0))
src_in_trg = Image.fromarray(src_in_trg.clip(min=0, max=255).astype('uint8')).convert('RGB')
return src_in_trg
================================================
FILE: domain_generalization/Transfer-Learning-Library/dalib/translation/spgan/__init__.py
================================================
from . import siamese
from . import loss
from .siamese import *
from .loss import *
================================================
FILE: domain_generalization/Transfer-Learning-Library/dalib/translation/spgan/loss.py
================================================
"""
Modified from https://github.com/Simon4Yan/eSPGAN
@author: Baixu Chen
@contact: cbx_99_hasta@outlook.com
"""
import torch
import torch.nn.functional as F
class ContrastiveLoss(torch.nn.Module):
r"""Contrastive loss from `Dimensionality Reduction by Learning an Invariant Mapping (CVPR 2006)
`_.
Given output features :math:`f_1, f_2`, we use :math:`D` to denote the pairwise euclidean distance between them,
:math:`Y` to denote the ground truth labels, :math:`m` to denote a pre-defined margin, then contrastive loss is
calculated as
.. math::
(1 - Y)\frac{1}{2}D^2 + (Y)\frac{1}{2}\{\text{max}(0, m-D)^2\}
Args:
margin (float, optional): margin for contrastive loss. Default: 2.0
Inputs:
- output1 (tensor): feature representations of the first set of samples (:math:`f_1` here).
- output2 (tensor): feature representations of the second set of samples (:math:`f_2` here).
- label (tensor): labels (:math:`Y` here).
Shape:
- output1, output2: :math:`(minibatch, F)` where F means the dimension of input features.
- label: :math:`(minibatch, )`
"""
def __init__(self, margin=2.0):
super(ContrastiveLoss, self).__init__()
self.margin = margin
def forward(self, output1, output2, label):
euclidean_distance = F.pairwise_distance(output1, output2)
loss = torch.mean((1 - label) * torch.pow(euclidean_distance, 2) +
label * torch.pow(torch.clamp(self.margin - euclidean_distance, min=0.0), 2))
return loss
================================================
FILE: domain_generalization/Transfer-Learning-Library/dalib/translation/spgan/siamese.py
================================================
"""
Modified from https://github.com/Simon4Yan/eSPGAN
@author: Baixu Chen
@contact: cbx_99_hasta@outlook.com
"""
import torch.nn as nn
import torch.nn.functional as F
class ConvBlock(nn.Module):
"""Basic block with structure Conv-LeakyReLU->Pool"""
def __init__(self, in_dim, out_dim):
super(ConvBlock, self).__init__()
self.conv_block = nn.Sequential(
nn.Conv2d(in_dim, out_dim, kernel_size=4, stride=2, padding=1),
nn.LeakyReLU(0.2),
nn.MaxPool2d(kernel_size=2, stride=2)
)
def forward(self, x):
return self.conv_block(x)
class SiameseNetwork(nn.Module):
"""Siamese network whose input is an image of shape :math:`(3,H,W)` and output is an one-dimensional feature vector.
Args:
nsf (int): dimension of output feature representation.
"""
def __init__(self, nsf=64):
super(SiameseNetwork, self).__init__()
self.conv = nn.Sequential(
nn.Conv2d(3, nsf, kernel_size=4, stride=2, padding=1),
nn.LeakyReLU(0.2),
nn.MaxPool2d(kernel_size=2, stride=2),
ConvBlock(nsf, nsf * 2),
ConvBlock(nsf * 2, nsf * 4),
)
self.flatten = nn.Flatten()
self.fc1 = nn.Linear(2048, nsf * 2, bias=False)
self.leaky_relu = nn.LeakyReLU(0.2)
self.dropout = nn.Dropout(0.5)
self.fc2 = nn.Linear(nsf * 2, nsf, bias=False)
def forward(self, x):
x = self.flatten(self.conv(x))
x = self.fc1(x)
x = self.leaky_relu(x)
x = self.dropout(x)
x = self.fc2(x)
x = F.normalize(x)
return x
================================================
FILE: domain_generalization/Transfer-Learning-Library/dglib/__init__.py
================================================
__all__ = ['generalization']
__version__ = '0.3'
================================================
FILE: domain_generalization/Transfer-Learning-Library/dglib/generalization/__init__.py
================================================
from . import mixstyle
from . import irm
from . import groupdro
from . import coral
__all__ = ['mixstyle', 'irm', 'groupdro', 'coral']
================================================
FILE: domain_generalization/Transfer-Learning-Library/dglib/generalization/coral.py
================================================
"""
@author: Baixu Chen
@contact: cbx_99_hasta@outlook.com
"""
import torch
import torch.nn as nn
class CorrelationAlignmentLoss(nn.Module):
r"""The `Correlation Alignment Loss` in
`Deep CORAL: Correlation Alignment for Deep Domain Adaptation (ECCV 2016) `_.
Given source features :math:`f_S` and target features :math:`f_T`, the covariance matrices are given by
.. math::
C_S = \frac{1}{n_S-1}(f_S^Tf_S-\frac{1}{n_S}(\textbf{1}^Tf_S)^T(\textbf{1}^Tf_S))
.. math::
C_T = \frac{1}{n_T-1}(f_T^Tf_T-\frac{1}{n_T}(\textbf{1}^Tf_T)^T(\textbf{1}^Tf_T))
where :math:`\textbf{1}` denotes a column vector with all elements equal to 1, :math:`n_S, n_T` denotes number of
source and target samples, respectively. We use :math:`d` to denote feature dimension, use
:math:`{\Vert\cdot\Vert}^2_F` to denote the squared matrix `Frobenius norm`. The correlation alignment loss is
given by
.. math::
l_{CORAL} = \frac{1}{4d^2}\Vert C_S-C_T \Vert^2_F
Inputs:
- f_s (tensor): feature representations on source domain, :math:`f^s`
- f_t (tensor): feature representations on target domain, :math:`f^t`
Shape:
- f_s, f_t: :math:`(N, d)` where d means the dimension of input features, :math:`N=n_S=n_T` is mini-batch size.
- Outputs: scalar.
"""
def __init__(self):
super(CorrelationAlignmentLoss, self).__init__()
def forward(self, f_s: torch.Tensor, f_t: torch.Tensor) -> torch.Tensor:
mean_s = f_s.mean(0, keepdim=True)
mean_t = f_t.mean(0, keepdim=True)
cent_s = f_s - mean_s
cent_t = f_t - mean_t
cov_s = torch.mm(cent_s.t(), cent_s) / (len(f_s) - 1)
cov_t = torch.mm(cent_t.t(), cent_t) / (len(f_t) - 1)
mean_diff = (mean_s - mean_t).pow(2).mean()
cov_diff = (cov_s - cov_t).pow(2).mean()
return mean_diff + cov_diff
================================================
FILE: domain_generalization/Transfer-Learning-Library/dglib/generalization/groupdro.py
================================================
"""
Modified from https://github.com/facebookresearch/DomainBed
@author: Baixu Chen
@contact: cbx_99_hasta@outlook.com
"""
import torch
class AutomaticUpdateDomainWeightModule(object):
r"""
Maintaining group weight based on loss history of all domains according
to `Distributionally Robust Neural Networks for Group Shifts: On the Importance of Regularization for Worst-Case
Generalization (ICLR 2020) `_.
Suppose we have :math:`N` domains. During each iteration, we first calculate unweighted loss among all
domains, resulting in :math:`loss\in R^N`. Then we update domain weight by
.. math::
w_k = w_k * \text{exp}(loss_k ^{\eta}), \forall k \in [1, N]
where :math:`\eta` is the hyper parameter which ensures smoother change of weight.
As :math:`w \in R^N` denotes a distribution, we `normalize`
:math:`w` by its sum. At last, weighted loss is calculated as our objective
.. math::
objective = \sum_{k=1}^N w_k * loss_k
Args:
num_domains (int): The number of source domains.
eta (float): Hyper parameter eta.
device (torch.device): The device to run on.
"""
def __init__(self, num_domains: int, eta: float, device):
self.domain_weight = torch.ones(num_domains).to(device) / num_domains
self.eta = eta
def get_domain_weight(self, sampled_domain_idxes):
"""Get domain weight to calculate final objective.
Inputs:
- sampled_domain_idxes (list): sampled domain indexes in current mini-batch
Shape:
- sampled_domain_idxes: :math:`(D, )` where D means the number of sampled domains in current mini-batch
- Outputs: :math:`(D, )`
"""
domain_weight = self.domain_weight[sampled_domain_idxes]
domain_weight = domain_weight / domain_weight.sum()
return domain_weight
def update(self, sampled_domain_losses: torch.Tensor, sampled_domain_idxes):
"""Update domain weight using loss of current mini-batch.
Inputs:
- sampled_domain_losses (tensor): loss of among sampled domains in current mini-batch
- sampled_domain_idxes (list): sampled domain indexes in current mini-batch
Shape:
- sampled_domain_losses: :math:`(D, )` where D means the number of sampled domains in current mini-batch
- sampled_domain_idxes: :math:`(D, )`
"""
sampled_domain_losses = sampled_domain_losses.detach()
for loss, idx in zip(sampled_domain_losses, sampled_domain_idxes):
self.domain_weight[idx] *= (self.eta * loss).exp()
================================================
FILE: domain_generalization/Transfer-Learning-Library/dglib/generalization/irm.py
================================================
"""
Modified from https://github.com/facebookresearch/DomainBed
@author: Baixu Chen
@contact: cbx_99_hasta@outlook.com
"""
import torch
import torch.nn as nn
import torch.nn.functional as F
import torch.autograd as autograd
class InvariancePenaltyLoss(nn.Module):
r"""Invariance Penalty Loss from `Invariant Risk Minimization `_.
We adopt implementation from `DomainBed `_. Given classifier
output :math:`y` and ground truth :math:`labels`, we split :math:`y` into two parts :math:`y_1, y_2`, corresponding
labels are :math:`labels_1, labels_2`. Next we calculate cross entropy loss with respect to a dummy classifier
:math:`w`, resulting in :math:`grad_1, grad_2` . Invariance penalty is then :math:`grad_1*grad_2`.
Inputs:
- y: predictions from model
- labels: ground truth
Shape:
- y: :math:`(N, C)` where C means the number of classes.
- labels: :math:`(N, )` where N mean mini-batch size
"""
def __init__(self):
super(InvariancePenaltyLoss, self).__init__()
self.scale = torch.tensor(1.).requires_grad_()
def forward(self, y: torch.Tensor, labels: torch.Tensor) -> torch.Tensor:
loss_1 = F.cross_entropy(y[::2] * self.scale, labels[::2])
loss_2 = F.cross_entropy(y[1::2] * self.scale, labels[1::2])
grad_1 = autograd.grad(loss_1, [self.scale], create_graph=True)[0]
grad_2 = autograd.grad(loss_2, [self.scale], create_graph=True)[0]
penalty = torch.sum(grad_1 * grad_2)
return penalty
================================================
FILE: domain_generalization/Transfer-Learning-Library/dglib/generalization/mixstyle/__init__.py
================================================
================================================
FILE: domain_generalization/Transfer-Learning-Library/dglib/generalization/mixstyle/models/__init__.py
================================================
from .resnet import *
from .mixstyle import *
__all__ = ['resnet']
================================================
FILE: domain_generalization/Transfer-Learning-Library/dglib/generalization/mixstyle/models/mixstyle.py
================================================
"""
Modified from https://github.com/KaiyangZhou/mixstyle-release
@author: Baixu Chen
@contact: cbx_99_hasta@outlook.com
"""
import random
import torch
import torch.nn as nn
class MixStyle(nn.Module):
r"""MixStyle module from `DOMAIN GENERALIZATION WITH MIXSTYLE (ICLR 2021) `_.
Given input :math:`x`, we first compute mean :math:`\mu(x)` and standard deviation :math:`\sigma(x)` across spatial
dimension. Then we permute :math:`x` and get :math:`\tilde{x}`, corresponding mean :math:`\mu(\tilde{x})` and
standard deviation :math:`\sigma(\tilde{x})`. `MixUp` is performed using mean and standard deviation
.. math::
\gamma_{mix} = \lambda\sigma(x) + (1-\lambda)\sigma(\tilde{x})
.. math::
\beta_{mix} = \lambda\mu(x) + (1-\lambda)\mu(\tilde{x})
where :math:`\lambda` is instance-wise weight sampled from `Beta distribution`. MixStyle is then
.. math::
MixStyle(x) = \gamma_{mix}\frac{x-\mu(x)}{\sigma(x)} + \beta_{mix}
Args:
p (float): probability of using MixStyle.
alpha (float): parameter of the `Beta distribution`.
eps (float): scaling parameter to avoid numerical issues.
"""
def __init__(self, p=0.5, alpha=0.1, eps=1e-6):
super().__init__()
self.p = p
self.beta = torch.distributions.Beta(alpha, alpha)
self.eps = eps
self.alpha = alpha
def forward(self, x):
if not self.training:
return x
if random.random() > self.p:
return x
batch_size = x.size(0)
mu = x.mean(dim=[2, 3], keepdim=True)
var = x.var(dim=[2, 3], keepdim=True)
sigma = (var + self.eps).sqrt()
mu, sigma = mu.detach(), sigma.detach()
x_normed = (x - mu) / sigma
interpolation = self.beta.sample((batch_size, 1, 1, 1))
interpolation = interpolation.to(x.device)
# split into two halves and swap the order
perm = torch.arange(batch_size - 1, -1, -1) # inverse index
perm_b, perm_a = perm.chunk(2)
perm_b = perm_b[torch.randperm(batch_size // 2)]
perm_a = perm_a[torch.randperm(batch_size // 2)]
perm = torch.cat([perm_b, perm_a], 0)
mu_perm, sigma_perm = mu[perm], sigma[perm]
mu_mix = mu * interpolation + mu_perm * (1 - interpolation)
sigma_mix = sigma * interpolation + sigma_perm * (1 - interpolation)
return x_normed * sigma_mix + mu_mix
================================================
FILE: domain_generalization/Transfer-Learning-Library/dglib/generalization/mixstyle/models/resnet.py
================================================
"""
@author: Baixu Chen
@contact: cbx_99_hasta@outlook.com
"""
from .mixstyle import MixStyle
from common.vision.models.reid.resnet import ReidResNet
from common.vision.models.resnet import ResNet, load_state_dict_from_url, model_urls, BasicBlock, Bottleneck
__all__ = ['resnet18', 'resnet34', 'resnet50', 'resnet101']
def _resnet_with_mix_style(arch, block, layers, pretrained, progress, mix_layers=None, mix_p=0.5, mix_alpha=0.1,
resnet_class=ResNet, **kwargs):
"""Construct `ResNet` with MixStyle modules. Given any resnet architecture **resnet_class** that contains conv1,
bn1, relu, maxpool, layer1-4, this function define a new class that inherits from **resnet_class** and inserts
MixStyle module during forward pass. Although MixStyle Module can be inserted anywhere, original paper finds it
better to place MixStyle after layer1-3. Our implementation follows this idea, but you are free to modify this
function to try other possibilities.
Args:
arch (str): resnet architecture (resnet50 for example)
block (class): class of resnet block
layers (list): depth list of each block
pretrained (bool): if True, load imagenet pre-trained model parameters
progress (bool): whether or not to display a progress bar to stderr
mix_layers (list): layers to insert MixStyle module after
mix_p (float): probability to activate MixStyle during forward pass
mix_alpha (float): parameter alpha for beta distribution
resnet_class (class): base resnet class to inherit from
"""
if mix_layers is None:
mix_layers = []
available_resnet_class = [ResNet, ReidResNet]
assert resnet_class in available_resnet_class
class ResNetWithMixStyleModule(resnet_class):
def __init__(self, mix_layers, mix_p=0.5, mix_alpha=0.1, *args, **kwargs):
super(ResNetWithMixStyleModule, self).__init__(*args, **kwargs)
self.mixStyleModule = MixStyle(p=mix_p, alpha=mix_alpha)
for layer in mix_layers:
assert layer in ['layer1', 'layer2', 'layer3']
self.apply_layers = mix_layers
def forward(self, x):
x = self.conv1(x)
x = self.bn1(x)
# turn on relu activation here **except for** reid tasks
if resnet_class != ReidResNet:
x = self.relu(x)
x = self.maxpool(x)
x = self.layer1(x)
if 'layer1' in self.apply_layers:
x = self.mixStyleModule(x)
x = self.layer2(x)
if 'layer2' in self.apply_layers:
x = self.mixStyleModule(x)
x = self.layer3(x)
if 'layer3' in self.apply_layers:
x = self.mixStyleModule(x)
x = self.layer4(x)
return x
model = ResNetWithMixStyleModule(mix_layers=mix_layers, mix_p=mix_p, mix_alpha=mix_alpha, block=block,
layers=layers, **kwargs)
if pretrained:
model_dict = model.state_dict()
pretrained_dict = load_state_dict_from_url(model_urls[arch],
progress=progress)
# remove keys from pretrained dict that doesn't appear in model dict
pretrained_dict = {k: v for k, v in pretrained_dict.items() if k in model_dict}
model.load_state_dict(pretrained_dict, strict=False)
return model
def resnet18(pretrained=False, progress=True, **kwargs):
"""Constructs a ResNet-18 model with MixStyle.
Args:
pretrained (bool): If True, returns a model pre-trained on ImageNet
progress (bool): If True, displays a progress bar of the download to stderr
"""
return _resnet_with_mix_style('resnet18', BasicBlock, [2, 2, 2, 2], pretrained, progress,
**kwargs)
def resnet34(pretrained=False, progress=True, **kwargs):
"""Constructs a ResNet-34 model with MixStyle.
Args:
pretrained (bool): If True, returns a model pre-trained on ImageNet
progress (bool): If True, displays a progress bar of the download to stderr
"""
return _resnet_with_mix_style('resnet34', BasicBlock, [3, 4, 6, 3], pretrained, progress,
**kwargs)
def resnet50(pretrained=False, progress=True, **kwargs):
"""Constructs a ResNet-50 model with MixStyle.
Args:
pretrained (bool): If True, returns a model pre-trained on ImageNet
progress (bool): If True, displays a progress bar of the download to stderr
"""
return _resnet_with_mix_style('resnet50', Bottleneck, [3, 4, 6, 3], pretrained, progress,
**kwargs)
def resnet101(pretrained=False, progress=True, **kwargs):
"""Constructs a ResNet-101 model with MixStyle.
Args:
pretrained (bool): If True, returns a model pre-trained on ImageNet
progress (bool): If True, displays a progress bar of the download to stderr
"""
return _resnet_with_mix_style('resnet101', Bottleneck, [3, 4, 23, 3], pretrained, progress,
**kwargs)
================================================
FILE: domain_generalization/Transfer-Learning-Library/dglib/generalization/mixstyle/sampler.py
================================================
"""
@author: Baixu Chen
@contact: cbx_99_hasta@outlook.com
"""
import random
import copy
from torch.utils.data.dataset import ConcatDataset
from torch.utils.data.sampler import Sampler
class RandomDomainMultiInstanceSampler(Sampler):
r"""Randomly sample :math:`N` domains, then randomly select :math:`P` instances in each domain, for each instance,
randomly select :math:`K` images to form a mini-batch of size :math:`N\times P\times K`.
Args:
dataset (ConcatDataset): dataset that contains data from multiple domains
batch_size (int): mini-batch size (:math:`N\times P\times K` here)
n_domains_per_batch (int): number of domains to select in a single mini-batch (:math:`N` here)
num_instances (int): number of instances to select in each domain (:math:`K` here)
"""
def __init__(self, dataset, batch_size, n_domains_per_batch, num_instances):
super(Sampler, self).__init__()
self.dataset = dataset
self.sample_idxes_per_domain = {}
for idx, (_, _, domain_id) in enumerate(self.dataset):
if domain_id not in self.sample_idxes_per_domain:
self.sample_idxes_per_domain[domain_id] = []
self.sample_idxes_per_domain[domain_id].append(idx)
self.n_domains_in_dataset = len(self.sample_idxes_per_domain)
self.n_domains_per_batch = n_domains_per_batch
assert self.n_domains_in_dataset >= self.n_domains_per_batch
assert batch_size % n_domains_per_batch == 0
self.batch_size_per_domain = batch_size // n_domains_per_batch
assert self.batch_size_per_domain % num_instances == 0
self.num_instances = num_instances
self.num_classes_per_domain = self.batch_size_per_domain // num_instances
self.length = len(list(self.__iter__()))
def __iter__(self):
sample_idxes_per_domain = copy.deepcopy(self.sample_idxes_per_domain)
domain_idxes = [idx for idx in range(self.n_domains_in_dataset)]
final_idxes = []
stop_flag = False
while not stop_flag:
selected_domains = random.sample(domain_idxes, self.n_domains_per_batch)
for domain in selected_domains:
sample_idxes = sample_idxes_per_domain[domain]
selected_idxes = self.sample_multi_instances(sample_idxes)
final_idxes.extend(selected_idxes)
for idx in selected_idxes:
sample_idxes_per_domain[domain].remove(idx)
remaining_size = len(sample_idxes_per_domain[domain])
if remaining_size < self.batch_size_per_domain:
stop_flag = True
return iter(final_idxes)
def sample_multi_instances(self, sample_idxes):
idxes_per_cls = {}
for idx in sample_idxes:
_, cls, _ = self.dataset[idx]
if cls not in idxes_per_cls:
idxes_per_cls[cls] = []
idxes_per_cls[cls].append(idx)
cls_list = [cls for cls in idxes_per_cls if len(idxes_per_cls[cls]) >= self.num_instances]
if len(cls_list) < self.num_classes_per_domain:
return random.sample(sample_idxes, self.batch_size_per_domain)
selected_idxes = []
selected_classes = random.sample(cls_list, self.num_classes_per_domain)
for cls in selected_classes:
selected_idxes.extend(random.sample(idxes_per_cls[cls], self.num_instances))
return selected_idxes
def __len__(self):
return self.length
================================================
FILE: domain_generalization/Transfer-Learning-Library/dglib/modules/__init__.py
================================================
================================================
FILE: domain_generalization/Transfer-Learning-Library/dglib/modules/classifier.py
================================================
"""
@author: Baixu Chen
@contact: cbx_99_hasta@outlook.com
"""
from typing import Optional, Tuple
import torch
import torch.nn as nn
from common.modules.classifier import Classifier as ClassifierBase
class ImageClassifier(ClassifierBase):
"""ImageClassifier specific for reproducing results of `DomainBed `_.
You are free to freeze all `BatchNorm2d` layers and insert one additional `Dropout` layer, this can achieve better
results for some datasets like PACS but may be worse for others.
Args:
backbone (torch.nn.Module): Any backbone to extract features from data
num_classes (int): Number of classes
freeze_bn (bool, optional): whether to freeze all `BatchNorm2d` layers. Default: False
dropout_p (float, optional): dropout ratio for additional `Dropout` layer, this layer is only used when `freeze_bn` is True. Default: 0.1
"""
def __init__(self, backbone: nn.Module, num_classes: int, freeze_bn: Optional[bool] = False,
dropout_p: Optional[float] = 0.1, **kwargs):
super(ImageClassifier, self).__init__(backbone, num_classes, **kwargs)
self.freeze_bn = freeze_bn
if freeze_bn:
self.feature_dropout = nn.Dropout(p=dropout_p)
def forward(self, x: torch.Tensor) -> Tuple[torch.Tensor, torch.Tensor]:
f = self.pool_layer(self.backbone(x))
f = self.bottleneck(f)
if self.freeze_bn:
f = self.feature_dropout(f)
predictions = self.head(f)
if self.training:
return predictions, f
else:
return predictions
def train(self, mode=True):
super(ImageClassifier, self).train(mode)
if self.freeze_bn:
for m in self.modules():
if isinstance(m, nn.BatchNorm2d):
m.eval()
================================================
FILE: domain_generalization/Transfer-Learning-Library/dglib/modules/sampler.py
================================================
"""
@author: Baixu Chen
@contact: cbx_99_hasta@outlook.com
"""
import random
import copy
import numpy as np
from torch.utils.data.dataset import ConcatDataset
from torch.utils.data.sampler import Sampler
class DefaultSampler(Sampler):
r"""Traverse all :math:`N` domains, randomly select :math:`K` samples in each domain to form a mini-batch of size
:math:`N\times K`.
Args:
data_source (ConcatDataset): dataset that contains data from multiple domains
batch_size (int): mini-batch size (:math:`N\times K` here)
"""
def __init__(self, data_source: ConcatDataset, batch_size: int):
super(Sampler, self).__init__()
self.num_all_domains = len(data_source.cumulative_sizes)
self.sample_idxes_per_domain = []
start = 0
for end in data_source.cumulative_sizes:
idxes = [idx for idx in range(start, end)]
self.sample_idxes_per_domain.append(idxes)
start = end
assert batch_size % self.num_all_domains == 0
self.batch_size_per_domain = batch_size // self.num_all_domains
self.length = len(list(self.__iter__()))
def __iter__(self):
sample_idxes_per_domain = copy.deepcopy(self.sample_idxes_per_domain)
final_idxes = []
stop_flag = False
while not stop_flag:
for domain in range(self.num_all_domains):
sample_idxes = sample_idxes_per_domain[domain]
selected_idxes = random.sample(sample_idxes, self.batch_size_per_domain)
final_idxes.extend(selected_idxes)
for idx in selected_idxes:
sample_idxes_per_domain[domain].remove(idx)
remaining_size = len(sample_idxes_per_domain[domain])
if remaining_size < self.batch_size_per_domain:
stop_flag = True
return iter(final_idxes)
def __len__(self):
return self.length
class RandomDomainSampler(Sampler):
r"""Randomly sample :math:`N` domains, then randomly select :math:`K` samples in each domain to form a mini-batch of
size :math:`N\times K`.
Args:
data_source (ConcatDataset): dataset that contains data from multiple domains
batch_size (int): mini-batch size (:math:`N\times K` here)
n_domains_per_batch (int): number of domains to select in a single mini-batch (:math:`N` here)
"""
def __init__(self, data_source: ConcatDataset, batch_size: int, n_domains_per_batch: int):
super(Sampler, self).__init__()
self.n_domains_in_dataset = len(data_source.cumulative_sizes)
self.n_domains_per_batch = n_domains_per_batch
assert self.n_domains_in_dataset >= self.n_domains_per_batch
self.sample_idxes_per_domain = []
start = 0
for end in data_source.cumulative_sizes:
idxes = [idx for idx in range(start, end)]
self.sample_idxes_per_domain.append(idxes)
start = end
assert batch_size % n_domains_per_batch == 0
self.batch_size_per_domain = batch_size // n_domains_per_batch
self.length = len(list(self.__iter__()))
def __iter__(self):
sample_idxes_per_domain = copy.deepcopy(self.sample_idxes_per_domain)
domain_idxes = [idx for idx in range(self.n_domains_in_dataset)]
final_idxes = []
stop_flag = False
while not stop_flag:
selected_domains = random.sample(domain_idxes, self.n_domains_per_batch)
for domain in selected_domains:
sample_idxes = sample_idxes_per_domain[domain]
if len(sample_idxes) < self.batch_size_per_domain:
selected_idxes = np.random.choice(sample_idxes, self.batch_size_per_domain, replace=True)
else:
selected_idxes = random.sample(sample_idxes, self.batch_size_per_domain)
final_idxes.extend(selected_idxes)
for idx in selected_idxes:
if idx in sample_idxes_per_domain[domain]:
sample_idxes_per_domain[domain].remove(idx)
remaining_size = len(sample_idxes_per_domain[domain])
if remaining_size < self.batch_size_per_domain:
stop_flag = True
return iter(final_idxes)
def __len__(self):
return self.length
================================================
FILE: domain_generalization/Transfer-Learning-Library/docs/Makefile
================================================
# Minimal makefile for Sphinx documentation
#
# You can set these variables from the command line.
SPHINXOPTS =
SPHINXBUILD = sphinx-build
SPHINXPROJ = PyTorchSphinxTheme
SOURCEDIR = .
BUILDDIR = build
# Put it first so that "make" without argument is like "make help".
help:
@$(SPHINXBUILD) -M help "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O)
.PHONY: help Makefile
# Catch-all target: route all unknown targets to Sphinx using the new
# "make mode" option. $(O) is meant as a shortcut for $(SPHINXOPTS).
%: Makefile
@$(SPHINXBUILD) -M $@ "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O)
================================================
FILE: domain_generalization/Transfer-Learning-Library/docs/common/loss.rst
================================================
Loss
=====================
Knowledge Distillation
-------------------------------
.. autoclass:: common.loss.KnowledgeDistillationLoss
:members:
Cross Entropy with Label Smooth
----------------------------------------
.. autoclass:: common.vision.models.reid.loss.CrossEntropyLossWithLabelSmooth
================================================
FILE: domain_generalization/Transfer-Learning-Library/docs/common/modules.rst
================================================
Modules
=====================
Classifier
-------------------------------
.. autoclass:: common.modules.classifier.Classifier
:members:
Regressor
-------------------------------
.. autoclass:: common.modules.regressor.Regressor
:members:
================================================
FILE: domain_generalization/Transfer-Learning-Library/docs/common/utils/analysis.rst
================================================
==============
Analysis Tools
==============
.. autofunction:: common.utils.analysis.collect_feature
.. autofunction:: common.utils.analysis.a_distance.calculate
.. autofunction:: common.utils.analysis.tsne.visualize
================================================
FILE: domain_generalization/Transfer-Learning-Library/docs/common/utils/base.rst
================================================
Generic Tools
==============
Average Meter
---------------------------------
.. autoclass:: common.utils.meter.AverageMeter
:members:
Progress Meter
---------------------------------
.. autoclass:: common.utils.meter.ProgressMeter
:members:
Meter
---------------------------------
.. autoclass:: common.utils.meter.Meter
:members:
Data
---------------------------------
.. autoclass:: common.utils.data.ForeverDataIterator
:members:
.. autoclass:: common.utils.data.CombineDataset
:members:
.. autofunction:: common.utils.data.send_to_device
.. autofunction:: common.utils.data.concatenate
Logger
-----------
.. autoclass:: common.utils.logger.TextLogger
:members:
.. autoclass:: common.utils.logger.CompleteLogger
:members:
================================================
FILE: domain_generalization/Transfer-Learning-Library/docs/common/utils/metric.rst
================================================
===========
Metrics
===========
Classification & Segmentation
==============================
Accuracy
---------------------------------
.. autofunction:: common.utils.metric.accuracy
ConfusionMatrix
---------------------------------
.. autoclass:: common.utils.metric.ConfusionMatrix
:members:
================================================
FILE: domain_generalization/Transfer-Learning-Library/docs/common/vision/datasets.rst
================================================
Vision Datasets
=============================
Cross-Domain Classification
---------------------------------------------------------
--------------------------------------
ImageList
--------------------------------------
.. autoclass:: common.vision.datasets.imagelist.ImageList
:members:
-------------------------------------
Office-31
-------------------------------------
.. autoclass:: common.vision.datasets.office31.Office31
:members:
:inherited-members:
---------------------------------------
Office-Caltech
---------------------------------------
.. autoclass:: common.vision.datasets.officecaltech.OfficeCaltech
:members:
:inherited-members:
---------------------------------------
Office-Home
---------------------------------------
.. autoclass:: common.vision.datasets.officehome.OfficeHome
:members:
:inherited-members:
--------------------------------------
VisDA-2017
--------------------------------------
.. autoclass:: common.vision.datasets.visda2017.VisDA2017
:members:
:inherited-members:
--------------------------------------
DomainNet
--------------------------------------
.. autoclass:: common.vision.datasets.domainnet.DomainNet
:members:
:inherited-members:
--------------------------------------
PACS
--------------------------------------
.. autoclass:: common.vision.datasets.pacs.PACS
:members:
--------------------------------------
MNIST
--------------------------------------
.. autoclass:: common.vision.datasets.digits.MNIST
:members:
--------------------------------------
USPS
--------------------------------------
.. autoclass:: common.vision.datasets.digits.USPS
:members:
--------------------------------------
SVHN
--------------------------------------
.. autoclass:: common.vision.datasets.digits.SVHN
:members:
Partial Cross-Domain Classification
----------------------------------------------------
---------------------------------------
Partial Wrapper
---------------------------------------
.. autofunction:: common.vision.datasets.partial.partial
.. autofunction:: common.vision.datasets.partial.default_partial
---------------------------------------
Caltech-256->ImageNet-1k
---------------------------------------
.. autoclass:: common.vision.datasets.partial.caltech_imagenet.CaltechImageNet
:members:
---------------------------------------
ImageNet-1k->Caltech-256
---------------------------------------
.. autoclass:: common.vision.datasets.partial.imagenet_caltech.ImageNetCaltech
:members:
Open Set Cross-Domain Classification
------------------------------------------------------
---------------------------------------
Open Set Wrapper
---------------------------------------
.. autofunction:: common.vision.datasets.openset.open_set
.. autofunction:: common.vision.datasets.openset.default_open_set
Cross-Domain Regression
------------------------------------------------------
---------------------------------------
ImageRegression
---------------------------------------
.. autoclass:: common.vision.datasets.regression.image_regression.ImageRegression
:members:
---------------------------------------
DSprites
---------------------------------------
.. autoclass:: common.vision.datasets.regression.dsprites.DSprites
:members:
---------------------------------------
MPI3D
---------------------------------------
.. autoclass:: common.vision.datasets.regression.mpi3d.MPI3D
:members:
Cross-Domain Segmentation
-----------------------------------------------
---------------------------------------
SegmentationList
---------------------------------------
.. autoclass:: common.vision.datasets.segmentation.segmentation_list.SegmentationList
:members:
---------------------------------------
Cityscapes
---------------------------------------
.. autoclass:: common.vision.datasets.segmentation.cityscapes.Cityscapes
---------------------------------------
GTA5
---------------------------------------
.. autoclass:: common.vision.datasets.segmentation.gta5.GTA5
---------------------------------------
Synthia
---------------------------------------
.. autoclass:: common.vision.datasets.segmentation.synthia.Synthia
---------------------------------------
Foggy Cityscapes
---------------------------------------
.. autoclass:: common.vision.datasets.segmentation.cityscapes.FoggyCityscapes
Cross-Domain Keypoint Detection
-----------------------------------------------
---------------------------------------
Dataset Base for Keypoint Detection
---------------------------------------
.. autoclass:: common.vision.datasets.keypoint_detection.keypoint_dataset.KeypointDataset
:members:
.. autoclass:: common.vision.datasets.keypoint_detection.keypoint_dataset.Body16KeypointDataset
:members:
.. autoclass:: common.vision.datasets.keypoint_detection.keypoint_dataset.Hand21KeypointDataset
:members:
---------------------------------------
Rendered Handpose Dataset
---------------------------------------
.. autoclass:: common.vision.datasets.keypoint_detection.rendered_hand_pose.RenderedHandPose
:members:
---------------------------------------
Hand-3d-Studio Dataset
---------------------------------------
.. autoclass:: common.vision.datasets.keypoint_detection.hand_3d_studio.Hand3DStudio
:members:
---------------------------------------
FreiHAND Dataset
---------------------------------------
.. autoclass:: common.vision.datasets.keypoint_detection.freihand.FreiHand
:members:
---------------------------------------
Surreal Dataset
---------------------------------------
.. autoclass:: common.vision.datasets.keypoint_detection.surreal.SURREAL
:members:
---------------------------------------
LSP Dataset
---------------------------------------
.. autoclass:: common.vision.datasets.keypoint_detection.lsp.LSP
:members:
---------------------------------------
Human3.6M Dataset
---------------------------------------
.. autoclass:: common.vision.datasets.keypoint_detection.human36m.Human36M
:members:
Cross-Domain ReID
------------------------------------------------------
---------------------------------------
Market1501
---------------------------------------
.. autoclass:: common.vision.datasets.reid.market1501.Market1501
:members:
---------------------------------------
DukeMTMC-reID
---------------------------------------
.. autoclass:: common.vision.datasets.reid.dukemtmc.DukeMTMC
:members:
---------------------------------------
MSMT17
---------------------------------------
.. autoclass:: common.vision.datasets.reid.msmt17.MSMT17
:members:
Natural Object Recognition
---------------------------------------------------------
-------------------------------------
Stanford Dogs
-------------------------------------
.. autoclass:: common.vision.datasets.stanford_dogs.StanfordDogs
:members:
-------------------------------------
Stanford Cars
-------------------------------------
.. autoclass:: common.vision.datasets.stanford_cars.StanfordCars
:members:
-------------------------------------
CUB-200-2011
-------------------------------------
.. autoclass:: common.vision.datasets.cub200.CUB200
:members:
-------------------------------------
FVGC Aircraft
-------------------------------------
.. autoclass:: common.vision.datasets.aircrafts.Aircraft
:members:
-------------------------------------
Oxford-IIIT Pet
-------------------------------------
.. autoclass:: common.vision.datasets.oxfordpet.OxfordIIITPet
:members:
-------------------------------------
COCO-70
-------------------------------------
.. autoclass:: common.vision.datasets.coco70.COCO70
:members:
-------------------------------------
DTD
-------------------------------------
.. autoclass:: common.vision.datasets.dtd.DTD
:members:
-------------------------------------
OxfordFlowers102
-------------------------------------
.. autoclass:: common.vision.datasets.oxfordflowers.OxfordFlowers102
:members:
Specialized Image Classification
--------------------------------
-------------------------------------
PatchCamelyon
-------------------------------------
.. autoclass:: common.vision.datasets.patchcamelyon.PatchCamelyon
:members:
-------------------------------------
Retinopathy
-------------------------------------
.. autoclass:: common.vision.datasets.retinopathy.Retinopathy
:members:
-------------------------------------
EuroSAT
-------------------------------------
.. autoclass:: common.vision.datasets.eurosat.EuroSAT
:members:
-------------------------------------
Resisc45
-------------------------------------
.. autoclass:: common.vision.datasets.resisc45.Resisc45
:members:
================================================
FILE: domain_generalization/Transfer-Learning-Library/docs/common/vision/models.rst
================================================
Vision Models
===========================
------------------------------
Image Classification
------------------------------
ResNets
---------------------------------
.. automodule:: common.vision.models.resnet
:members:
LeNet
--------------------------
.. automodule:: common.vision.models.digits.lenet
:members:
DTN
--------------------------
.. automodule:: common.vision.models.digits.dtn
:members:
----------------------------------
Semantic Segmentation
----------------------------------
.. autofunction:: common.vision.models.segmentation.deeplabv2.deeplabv2_resnet101
----------------------------------
Keypoint Detection
----------------------------------
PoseResNet
--------------------------
.. autofunction:: common.vision.models.keypoint_detection.pose_resnet.pose_resnet101
.. autoclass:: common.vision.models.keypoint_detection.pose_resnet.PoseResNet
.. autoclass:: common.vision.models.keypoint_detection.pose_resnet.Upsampling
Joint Loss
----------------------------------
.. autoclass:: common.vision.models.keypoint_detection.loss.JointsMSELoss
.. autoclass:: common.vision.models.keypoint_detection.loss.JointsKLLoss
-----------------------------------
Re-Identification
-----------------------------------
Models
---------------
.. autoclass:: common.vision.models.reid.resnet.ReidResNet
.. automodule:: common.vision.models.reid.resnet
:members:
.. autoclass:: common.vision.models.reid.identifier.ReIdentifier
:members:
Loss
-----------------------------------
.. autoclass:: common.vision.models.reid.loss.TripletLoss
Sampler
-----------------------------------
.. autoclass:: common.utils.data.RandomMultipleGallerySampler
================================================
FILE: domain_generalization/Transfer-Learning-Library/docs/common/vision/transforms.rst
================================================
Vision Transforms
=============================
Classification
---------------------------------
.. automodule:: common.vision.transforms
:members:
Segmentation
---------------------------------
.. automodule:: common.vision.transforms.segmentation
:members:
Keypoint Detection
---------------------------------
.. automodule:: common.vision.transforms.keypoint_detection
:members:
================================================
FILE: domain_generalization/Transfer-Learning-Library/docs/conf.py
================================================
import sys
import os
sys.path.append(os.path.abspath('..'))
sys.path.append(os.path.abspath('./demo/'))
from pytorch_sphinx_theme import __version__
import pytorch_sphinx_theme
# If extensions (or modules to document with autodoc) are in another directory,
# add these directories to sys.path here. If the directory is relative to the
# documentation root, use os.path.abspath to make it absolute, like shown here.
#sys.path.insert(0, os.path.abspath('.'))
# -- General configuration -----------------------------------------------------
# If your documentation needs a minimal Sphinx version, state it here.
#needs_sphinx = '1.0'
# Add any Sphinx extension module names here, as strings. They can be extensions
# coming with Sphinx (named 'sphinx.ext.*') or your custom ones.
extensions = [
'sphinx.ext.intersphinx',
'sphinx.ext.autodoc',
'sphinx.ext.viewcode',
'sphinxcontrib.httpdomain',
'sphinx.ext.autosummary',
'sphinx.ext.autosectionlabel',
'sphinx.ext.napoleon',
]
# build the templated autosummary files
autosummary_generate = True
numpydoc_show_class_members = False
# autosectionlabel throws warnings if section names are duplicated.
# The following tells autosectionlabel to not throw a warning for
# duplicated section names that are in different documents.
autosectionlabel_prefix_document = True
napoleon_use_ivar = True
# Do not warn about external images (status badges in README.rst)
suppress_warnings = ['image.nonlocal_uri']
# Add any paths that contain templates here, relative to this directory.
templates_path = ['_templates']
# The suffix of source filenames.
source_suffix = '.rst'
# The encoding of source files.
#source_encoding = 'utf-8-sig'
# The master toctree document.
master_doc = 'index'
# General information about the project.
project = u'Transfer Learning Library'
copyright = u'THUML Group'
# The version info for the project you're documenting, acts as replacement for
# |version| and |release|, also used in various other places throughout the
# built documents.
#
# The short X.Y version.
version = __version__
# The full version, including alpha/beta/rc tags.
release = __version__
# The language for content autogenerated by Sphinx. Refer to documentation
# for a list of supported languages.
language = 'en'
# There are two options for replacing |today|: either, you set today to some
# non-false value, then it is used:
#today = ''
# Else, today_fmt is used as the format for a strftime call.
#today_fmt = '%B %d, %Y'
# List of patterns, relative to source directory, that match files and
# directories to ignore when looking for source files.
exclude_patterns = []
# The reST default role (used for this markup: `text`) to use for all documents.
#default_role = None
# If true, '()' will be appended to :func: etc. cross-reference text.
#add_function_parentheses = True
# If true, the current module name will be prepended to all description
# unit titles (such as .. function::).
#add_module_names = True
# If true, sectionauthor and moduleauthor directives will be shown in the
# output. They are ignored by default.
#show_authors = False
# The name of the Pygments (syntax highlighting) style to use.
pygments_style = 'sphinx'
# A list of ignored prefixes for module index sorting.
#modindex_common_prefix = []
intersphinx_mapping = {
'rtd': ('https://docs.readthedocs.io/en/latest/', None),
'python': ('https://docs.python.org/3', None),
'numpy': ('https://numpy.org/doc/stable', None),
'torch': ('https://pytorch.org/docs/stable', None),
'torchvision': ('https://pytorch.org/vision/stable', None),
'PIL': ('https://pillow.readthedocs.io/en/stable/', None)
}
# -- Options for HTML output ---------------------------------------------------
# The theme to use for HTML and HTML Help pages. See the documentation for
# a list of builtin themes.
html_theme = 'pytorch_sphinx_theme'
html_theme_path = [pytorch_sphinx_theme.get_html_theme_path()]
# Theme options are theme-specific and customize the look and feel of a theme
# further. For a list of options available for each theme, see the
# documentation.
html_theme_options = {
'canonical_url': '',
'analytics_id': '',
'logo_only': False,
'display_version': False,
'prev_next_buttons_location': 'bottom',
'style_external_links': False,
# Toc options
'collapse_navigation': True,
'sticky_navigation': False,
'navigation_depth': 4,
'includehidden': True,
'titles_only': False
}
# The name for this set of Sphinx documents. If None, it defaults to
# " v documentation".
#html_title = None
# A shorter title for the navigation bar. Default is the same as html_title.
#html_short_title = None
# The name of an image file (relative to this directory) to place at the top
# of the sidebar.
html_logo = "_static/images/TransLearn.png"
# The name of an image file (within the static path) to use as favicon of the
# docs. This file should be a Windows icon file (.ico) being 16x16 or 32x32
# pixels large.
#html_favicon = None
# Add any paths that contain custom static files (such as style sheets) here,
# relative to this directory. They are copied after the builtin static files,
# so a file named "default.css" will overwrite the builtin "default.css".
html_static_path = ['_static']
# If not '', a 'Last updated on:' timestamp is inserted at every page bottom,
# using the given strftime format.
#html_last_updated_fmt = '%b %d, %Y'
# If true, SmartyPants will be used to convert quotes and dashes to
# typographically correct entities.
#html_use_smartypants = True
# Custom sidebar templates, maps document names to template names.
#html_sidebars = {}
# Additional templates that should be rendered to pages, maps page names to
# template names.
#html_additional_pages = {}
# If false, no module index is generated.
#html_domain_indices = True
# If false, no index is generated.
#html_use_index = True
# If true, the index is split into individual pages for each letter.
#html_split_index = False
# If true, links to the reST sources are added to the pages.
html_show_sourcelink = True
# If true, "Created using Sphinx" is shown in the HTML footer. Default is True.
#html_show_sphinx = True
# If true, "(C) Copyright ..." is shown in the HTML footer. Default is True.
#html_show_copyright = True
# If true, an OpenSearch description file will be output, and all pages will
# contain a tag referring to it. The value of this option must be the
# base URL from which the finished HTML is served.
#html_use_opensearch = ''
# This is the file name suffix for HTML files (e.g. ".xhtml").
#html_file_suffix = None
# Disable displaying type annotations, these can be very verbose
autodoc_typehints = 'none'
# Output file base name for HTML help builder.
htmlhelp_basename = 'TransferLearningLibrary'
# -- Options for LaTeX output --------------------------------------------------
latex_elements = {
# The paper size ('letterpaper' or 'a4paper').
#'papersize': 'letterpaper',
# The font size ('10pt', '11pt' or '12pt').
#'pointsize': '10pt',
# Additional stuff for the LaTeX preamble.
#'preamble': '',
}
# Grouping the document tree into LaTeX files. List of tuples
# (source start file, target name, title, author, documentclass [howto/manual]).
latex_documents = [
('index', 'TransferLearningLibrary.tex', u'Transfer Learning Library Documentation',
u'THUML', 'manual'),
]
# The name of an image file (relative to this directory) to place at the top of
# the title page.
#latex_logo = None
# For "manual" documents, if this is true, then toplevel headings are parts,
# not chapters.
#latex_use_parts = False
# If true, show page references after internal links.
#latex_show_pagerefs = False
# If true, show URL addresses after external links.
#latex_show_urls = False
# Documents to append as an appendix to all manuals.
#latex_appendices = []
# If false, no module index is generated.
#latex_domain_indices = True
# -- Options for manual page output --------------------------------------------
# One entry per manual page. List of tuples
# (source start file, name, description, authors, manual section).
man_pages = [
('index', 'Transfer Learning Library', u'Transfer Learning Library Documentation',
[u'THUML'], 1)
]
# If true, show URL addresses after external links.
#man_show_urls = False
# -- Options for Texinfo output ------------------------------------------------
# Grouping the document tree into Texinfo files. List of tuples
# (source start file, target name, title, author,
# dir menu entry, description, category)
texinfo_documents = [
('index', 'Transfer Learning Library', u'Transfer Learning Library Documentation',
u'THUML', 'Transfer Learning Library',
'One line description of project.', 'Miscellaneous'),
]
# Documents to append as an appendix to all manuals.
#texinfo_appendices = []
# If false, no module index is generated.
#texinfo_domain_indices = True
# How to display URL addresses: 'footnote', 'no', or 'inline'.
#texinfo_show_urls = 'footnote'
================================================
FILE: domain_generalization/Transfer-Learning-Library/docs/dalib/adaptation/domain_adversarial.rst
================================================
=======================================
Domain Adversarial Learning
=======================================
Domain Adversarial Training in Close-set DA
===========================================
.. _DANN:
DANN: Domain Adversarial Neural Network
----------------------------------------
.. autoclass:: dalib.adaptation.dann.DomainAdversarialLoss
.. _CDAN:
CDAN: Conditional Domain Adversarial Network
-----------------------------------------------
.. autoclass:: dalib.adaptation.cdan.ConditionalDomainAdversarialLoss
.. autoclass:: dalib.adaptation.cdan.RandomizedMultiLinearMap
.. autoclass:: dalib.adaptation.cdan.MultiLinearMap
.. _ADDA:
ADDA: Adversarial Discriminative Domain Adaptation
-----------------------------------------------------
.. autoclass:: dalib.adaptation.adda.DomainAdversarialLoss
.. note::
ADDAgrl is also implemented and benchmarked. You can find code
`here `_.
.. _BSP:
BSP: Batch Spectral Penalization
-----------------------------------
.. autoclass:: dalib.adaptation.bsp.BatchSpectralPenalizationLoss
Domain Adversarial Training in Open-set DA
===========================================
.. _OSBP:
OSBP: Open Set Domain Adaptation by Backpropagation
----------------------------------------------------
.. autoclass:: dalib.adaptation.osbp.UnknownClassBinaryCrossEntropy
Domain Adversarial Training in Partial DA
===========================================
.. _PADA:
PADA: Partial Adversarial Domain Adaptation
---------------------------------------------
.. autoclass:: dalib.adaptation.pada.ClassWeightModule
.. autoclass:: dalib.adaptation.pada.AutomaticUpdateClassWeightModule
:members:
.. autofunction:: dalib.adaptation.pada.collect_classification_results
.. _IWAN:
IWAN: Importance Weighted Adversarial Nets
---------------------------------------------
.. autoclass:: dalib.adaptation.iwan.ImportanceWeightModule
:members:
Domain Adversarial Training in Segmentation
===========================================
.. _ADVENT:
ADVENT: Adversarial Entropy Minimization
------------------------------------------
.. autoclass:: dalib.adaptation.advent.Discriminator
.. autoclass:: dalib.adaptation.advent.DomainAdversarialEntropyLoss
:members:
================================================
FILE: domain_generalization/Transfer-Learning-Library/docs/dalib/adaptation/domain_translation.rst
================================================
=======================================
Domain Translation
=======================================
.. _CycleGAN:
------------------------------------------------
CycleGAN: Cycle-Consistent Adversarial Networks
------------------------------------------------
Discriminator
--------------
.. autofunction:: dalib.translation.cyclegan.pixel
.. autofunction:: dalib.translation.cyclegan.patch
Generator
--------------
.. autofunction:: dalib.translation.cyclegan.resnet_9
.. autofunction:: dalib.translation.cyclegan.resnet_6
.. autofunction:: dalib.translation.cyclegan.unet_256
.. autofunction:: dalib.translation.cyclegan.unet_128
GAN Loss
--------------
.. autoclass:: dalib.translation.cyclegan.LeastSquaresGenerativeAdversarialLoss
.. autoclass:: dalib.translation.cyclegan.VanillaGenerativeAdversarialLoss
.. autoclass:: dalib.translation.cyclegan.WassersteinGenerativeAdversarialLoss
Translation
--------------
.. autoclass:: dalib.translation.cyclegan.Translation
Util
----------------
.. autoclass:: dalib.translation.cyclegan.util.ImagePool
:members:
.. autofunction:: dalib.translation.cyclegan.util.set_requires_grad
.. _Cycada:
--------------------------------------------------------------
CyCADA: Cycle-Consistent Adversarial Domain Adaptation
--------------------------------------------------------------
.. autoclass:: dalib.translation.cycada.SemanticConsistency
.. _SPGAN:
-----------------------------------------------------------
SPGAN: Similarity Preserving Generative Adversarial Network
-----------------------------------------------------------
`Image-Image Domain Adaptation with Preserved Self-Similarity and Domain-Dissimilarity for Person Re-identification
`_. SPGAN is based on CycleGAN. An additional Siamese network is adopted to force
the generator to produce images different from identities in target dataset.
Siamese Network
-------------------
.. autoclass:: dalib.translation.spgan.siamese.SiameseNetwork
Contrastive Loss
-------------------
.. autoclass:: dalib.translation.spgan.loss.ContrastiveLoss
.. _FDA:
------------------------------------------------
FDA: Fourier Domain Adaptation
------------------------------------------------
.. autoclass:: dalib.translation.fourier_transform.FourierTransform
.. autofunction:: dalib.translation.fourier_transform.low_freq_mutate
.. autofunction:: dalib.adaptation.fda.robust_entropy
================================================
FILE: domain_generalization/Transfer-Learning-Library/docs/dalib/adaptation/hypothesis_adversarial.rst
================================================
=======================================
Hypothesis Adversarial Learning
=======================================
.. _MCD:
MCD: Maximum Classifier Discrepancy
=============================================
.. autofunction:: dalib.adaptation.mcd.classifier_discrepancy
.. autofunction:: dalib.adaptation.mcd.entropy
.. autoclass:: dalib.adaptation.mcd.ImageClassifierHead
.. _MDD:
MDD: Margin Disparity Discrepancy
=============================================
.. autoclass:: dalib.adaptation.mdd.MarginDisparityDiscrepancy
MDD for Classification
----------------------
.. autoclass:: dalib.adaptation.mdd.ClassificationMarginDisparityDiscrepancy
.. autoclass:: dalib.adaptation.mdd.ImageClassifier
:members:
.. autofunction:: dalib.adaptation.mdd.shift_log
MDD for Regression
------------------
.. autoclass:: dalib.adaptation.mdd.RegressionMarginDisparityDiscrepancy
.. autoclass:: dalib.adaptation.mdd.ImageRegressor
.. _RegDA:
RegDA: Regressive Domain Adaptation
=============================================
.. autoclass:: dalib.adaptation.regda.PseudoLabelGenerator2d
.. autoclass:: dalib.adaptation.regda.RegressionDisparity
.. autoclass:: dalib.adaptation.regda.PoseResNet2d
================================================
FILE: domain_generalization/Transfer-Learning-Library/docs/dalib/adaptation/others.rst
================================================
=======================================
Self Training Methods
=======================================
.. _SelfEnsemble:
Self Ensemble
-----------------------------
.. autoclass:: dalib.adaptation.self_ensemble.ConsistencyLoss
.. autoclass:: dalib.adaptation.self_ensemble.L2ConsistencyLoss
.. autoclass:: dalib.adaptation.self_ensemble.ClassBalanceLoss
.. autoclass:: dalib.adaptation.self_ensemble.EmaTeacher
.. _MCC:
MCC: Minimum Class Confusion
-----------------------------
.. autoclass:: dalib.adaptation.mcc.MinimumClassConfusionLoss
.. _MMT:
MMT: Mutual Mean-Teaching
--------------------------
`Mutual Mean-Teaching: Pseudo Label Refinery for Unsupervised
Domain Adaptation on Person Re-identification (ICLR 2020) `_
State of the art unsupervised domain adaptation methods utilize clustering algorithms to generate pseudo labels on target
domain, which are noisy and thus harmful for training. Inspired by the teacher-student approaches, MMT framework
provides robust soft pseudo labels in an on-line peer-teaching manner.
We denote two networks as :math:`f_1,f_2`, their parameters as :math:`\theta_1,\theta_2`. The authors also
propose to use the temporally average model of each network :math:`\text{ensemble}(f_1),\text{ensemble}(f_2)` to generate more reliable
soft pseudo labels for supervising the other network. Specifically, the parameters of the temporally
average models of the two networks at current iteration :math:`T` are denoted as :math:`E^{(T)}[\theta_1]` and
:math:`E^{(T)}[\theta_2]` respectively, which can be calculated as
.. math::
E^{(T)}[\theta_1] = \alpha E^{(T-1)}[\theta_1] + (1-\alpha)\theta_1
.. math::
E^{(T)}[\theta_2] = \alpha E^{(T-1)}[\theta_2] + (1-\alpha)\theta_2
where :math:`E^{(T-1)}[\theta_1],E^{(T-1)}[\theta_2]` indicate the temporal average parameters of the two networks in
the previous iteration :math:`(T-1)`, the initial temporal average parameters are
:math:`E^{(0)}[\theta_1]=\theta_1,E^{(0)}[\theta_2]=\theta_2` and :math:`\alpha` is the momentum.
These two networks cooperate with each other in three ways:
- When running clustering algorithm, we average features produced by :math:`\text{ensemble}(f_1)` and
:math:`\text{ensemble}(f_2)` instead of only considering one of them.
- A **soft triplet loss** is optimized between :math:`f_1` and :math:`\text{ensemble}(f_2)` and vice versa
to force one network to learn from temporally average of another network.
- A **cross entropy loss** is optimized between :math:`f_1` and :math:`\text{ensemble}(f_2)` and vice versa
to force one network to learn from temporally average of another network.
The above mentioned loss functions are listed below, more details can be found in training scripts.
.. autoclass:: common.vision.models.reid.loss.SoftTripletLoss
.. autoclass:: common.vision.models.reid.loss.CrossEntropyLoss
=======================================
Other Methods
=======================================
.. _AFN:
AFN: Adaptive Feature Norm
-----------------------------
.. autoclass:: dalib.adaptation.afn.AdaptiveFeatureNorm
.. autoclass:: dalib.adaptation.afn.Block
.. autoclass:: dalib.adaptation.afn.ImageClassifier
================================================
FILE: domain_generalization/Transfer-Learning-Library/docs/dalib/adaptation/statistics_matching.rst
================================================
Statistics Matching
===================================
.. _DAN:
DAN: Deep Adaptation Network
-----------------------------
.. autoclass:: dalib.adaptation.dan.MultipleKernelMaximumMeanDiscrepancy
.. _JAN:
JAN: Joint Adaptation Network
------------------------------
.. autoclass:: dalib.adaptation.jan.JointMultipleKernelMaximumMeanDiscrepancy
================================================
FILE: domain_generalization/Transfer-Learning-Library/docs/dalib/benchmarks/image_classification.rst
================================================
===============================
Image Classification
===============================
We provide benchmarks of different domain adaptation algorithms on `Digits`_, `Office-31`_ , `Office-Home`_, `VisDA-2017`_ and `DomainNet`_.
Those domain adaptation algorithms includes:
- :ref:`DANN`
- :ref:`DAN`
- :ref:`JAN`
- :ref:`ADDA`
- :ref:`CDAN`
- :ref:`MCD`
- :ref:`AFN`
- :ref:`BSP`
- :ref:`MDD`
- :ref:`MCC`
.. note::
- ``Origin`` means the accuracy reported by the original paper.
- ``Avg`` is the accuracy reported by Trasnfer-Learn.
- ``Source Only`` refers to the model trained with data from the source domain.
- ``Oracle`` refers to the model trained with data from the target domain.
.. note::
We found that the accuracies of adversarial methods (including DANN, ADDA, CDAN, MCD, BSP and MDD) are not stable
even after the random seed is fixed, thus we repeat running adversarial methods on *Office-31* and *VisDA-2017*
for three times and report their average accuracy.
.. note::
`ADDA` with gradient reverse layer is frequently benchmarked in the literature. Therefore, we implement this
baseline and use `ADDAgrl` to denote it below.
.. _Digits:
--------------------------------
Digits accuracy
--------------------------------
=========== =========== =========== ===========
Methods SVHN2MNIST MNIST2USPS USPS2MNIST
Source Only 74.1 82.1 74.6
DANN 90.8 91.7 95.2
DAN 82.1 86.0 89.5
JAN 90.3 84.0 86.8
ADDA 93.3 94.5 98.3
CDAN 93.8 96.0 97.7
MCD 90.6 94.1 97.6
AFN 88.2 88.6 97.2
BSP+DANN 84.2 95.7 97.8
MDD 88.4 94.8 97.8
LDD 93.5 95.5 96.3
MCC 76.6 95.1 94.6
=========== =========== =========== ===========
.. _Office-31:
--------------------------------
Office-31 accuracy on ResNet-50
--------------------------------
=========== ====== ====== ====== ====== ====== ====== ====== ======
Methods Origin Avg A → W D → W W → D A → D D → A W → A
Source Only 76.1 79.5 75.8 95.5 99.0 79.3 63.6 63.8
DANN 82.2 86.1 91.4 97.9 100.0 83.6 73.3 70.4
DAN 80.4 83.7 84.2 98.4 100.0 87.3 66.9 65.2
JAN 84.3 87.0 93.7 98.4 100.0 89.4 69.2 71.0
ADDA / 86.5 91.2 98.5 100.0 84.3 73.7 71.2
ADDAgrl / 87.3 94.6 97.5 99.7 90.0 69.6 72.5
CDAN 87.7 87.7 93.8 98.5 100.0 89.9 73.4 70.4
MCD / 85.4 90.4 98.5 100.0 87.3 68.3 67.6
AFN 85.7 88.6 94.0 98.9 100.0 94.4 72.9 71.1
BSP+DANN 87.7 87.8 92.7 97.9 100.0 88.2 74.1 73.8
MDD 88.9 89.6 95.6 98.6 100.0 94.4 76.6 72.2
MCC 89.4 89.6 94.1 98.4 99.8 95.6 75.5 74.2
=========== ====== ====== ====== ====== ====== ====== ====== ======
.. _Office-Home:
-----------------------------------
Office-Home accuracy on ResNet-50
-----------------------------------
=========== ======= ======= ======= ======= ======= ======= ======= ======= ======= ======= ======= ======= ======= =======
Methods Origin Avg Ar → Cl Ar → Pr Ar → Rw Cl → Ar Cl → Pr Cl → Rw Pr → Ar Pr → Cl Pr → Rw Rw → Ar Rw → Cl Rw → Pr
Source Only 46.1 58.4 41.1 65.9 73.7 53.1 60.1 63.3 52.2 36.7 71.8 64.8 42.6 75.2
DANN 57.6 65.2 53.8 62.6 74.0 55.8 67.3 67.3 55.8 55.1 77.9 71.1 60.7 81.1
DAN 56.3 61.4 45.6 67.7 73.9 57.7 63.8 66.0 54.9 40.0 74.5 66.2 49.1 77.9
JAN 58.3 65.9 50.8 71.9 76.5 60.6 68.3 68.7 60.5 49.6 76.9 71.0 55.9 80.5
ADDA / 62.5 47.4 63.9 72.6 53.1 62.6 64.3 56.0 49.1 76.3 68.1 56.5 80.3
ADDAgrl / 65.6 52.6 62.9 74.0 59.7 68.0 68.8 61.4 52.5 77.6 71.1 58.6 80.2
CDAN 65.8 68.8 55.2 72.4 77.6 62.0 69.7 70.9 62.4 54.3 80.5 75.5 61.0 83.8
MCD / 67.8 51.7 72.2 78.2 63.7 69.5 70.8 61.5 52.8 78.0 74.5 58.4 81.8
AFN 67.3 68.2 53.2 72.7 76.8 65.0 71.3 72.3 65.0 51.4 77.9 72.3 57.8 82.4
BSP+DANN 64.9 67.6 54.7 67.7 76.2 61.0 69.4 70.9 60.9 55.2 80.2 73.4 60.3 81.2
MDD 68.1 69.7 56.2 75.4 79.6 63.5 72.1 73.8 62.5 54.8 79.9 73.5 60.9 84.5
MCC / 72.4 58.4 79.6 83.0 67.5 77.0 78.5 66.6 54.8 81.8 74.4 61.4 85.6
=========== ======= ======= ======= ======= ======= ======= ======= ======= ======= ======= ======= ======= ======= =======
.. _VisDA-2017:
-----------------------------------
VisDA-2017 accuracy ResNet-101
-----------------------------------
.. note::
- ``Origin`` means the accuracy reported by the original paper.
- ``Mean`` refers to the accuracy average over ``classes``
- ``Avg`` refers to accuracy average over ``samples``.
=========== ========== ======= ======= ======= ======= ======= ======= ======= ======= ======= ======= ======= ======= ======= =======
Methods Origin Mean plane bcycl bus car horse knife mcycl person plant sktbrd train truck Avg
Source Only 52.4 51.7 63.6 35.3 50.6 78.2 74.6 18.7 82.1 16.0 84.2 35.5 77.4 4.7 56.9
DANN 57.4 79.5 93.5 74.3 83.4 50.7 87.2 90.2 89.9 76.1 88.1 91.4 89.7 39.8 74.9
DAN 61.1 66.4 89.2 37.2 77.7 61.8 81.7 64.3 90.6 61.4 79.9 37.7 88.1 27.4 67.2
JAN / 73.4 96.3 66.0 82.0 44.1 86.4 70.3 87.9 74.6 83.0 64.6 84.5 41.3 70.3
ADDA / 79.3 93.6 70.8 83.2 63.5 90.6 93.2 89.0 75.3 88.4 79.3 87.4 37.2 76.4
ADDAgrl / 77.5 95.6 70.8 84.4 54.0 87.8 75.8 88.4 69.3 84.1 86.2 85.0 48.0 74.3
CDAN / 80.1 94.0 69.2 78.9 57.0 89.8 94.9 91.9 80.3 86.8 84.9 85.0 48.5 76.5
MCD 71.9 77.7 87.8 75.7 84.2 78.1 91.6 95.3 88.1 78.3 83.4 64.5 84.8 20.9 76.7
AFN 76.1 75.0 95.6 56.2 81.3 69.8 93.0 81.0 93.4 74.1 91.7 55.0 90.6 18.1 74.4
BSP+DANN 75.9 80.5 95.7 75.6 82.8 54.5 89.2 96.5 91.3 72.2 88.9 88.7 88.0 43.4 76.2
MDD / 82.0 88.3 62.8 85.2 69.9 91.9 95.1 94.4 81.2 93.8 89.8 84.1 47.9 79.8
MCC 78.8 83.6 95.3 85.8 77.1 68.0 93.9 92.9 84.5 79.5 93.6 93.7 85.3 53.8 80.4
=========== ========== ======= ======= ======= ======= ======= ======= ======= ======= ======= ======= ======= ======= ======= =======
.. _DomainNet:
-----------------------------------
DomainNet accuracy on ResNet-101
-----------------------------------
=========== ====== ====== ====== ====== ====== ====== ====== ====== ====== ====== ====== ====== ======
Methods c->p c->r c->s p->c p->r p->s r->c r->p r->s s->c s->p s->r Avg
Source Only 32.7 50.6 39.4 41.1 56.8 35.0 48.6 48.8 36.1 49.0 34.8 46.1 43.3
DANN 37.9 54.3 44.4 41.7 55.6 36.8 50.7 50.8 40.1 55.0 45.0 54.5 47.2
DAN 38.8 55.2 43.9 45.9 59.0 40.8 50.8 49.8 38.9 56.1 45.9 55.5 48.4
JAN 40.5 56.7 45.1 47.2 59.9 43.0 54.2 52.6 41.9 56.6 46.2 55.5 50.0
ADDA 38.4 54.1 44.1 43.5 56.7 39.2 52.8 51.3 40.9 55.0 45.4 54.5 48.0
CDAN 40.4 56.8 46.1 45.1 58.4 40.5 55.6 53.6 43.0 57.2 46.4 55.7 49.9
MCD 37.5 52.9 44.0 44.6 54.5 41.6 52.0 51.5 39.7 55.5 44.6 52.0 47.5
MDD 42.9 59.5 47.5 48.6 59.4 42.6 58.3 53.7 46.2 58.7 46.5 57.7 51.8
MCC 37.7 55.7 42.6 45.4 59.8 39.9 54.4 53.1 37.0 58.1 46.3 56.2 48.9
=========== ====== ====== ====== ====== ====== ====== ====== ====== ====== ====== ====== ====== ======
Oracle DomainNet accuracy on ResNet-101
--------------------------------------------
=========== ====== ====== ====== ====== ====== ======
Oracle clp inf pnt real skt Avg
/ 78.2 40.7 71.6 83.8 70.6 69.0
=========== ====== ====== ====== ====== ====== ======
================================================
FILE: domain_generalization/Transfer-Learning-Library/docs/dalib/benchmarks/image_regression.rst
================================================
===============================================
Image Regression
===============================================
We provide benchmarks of different domain adaptation algorithms on `dSprites`_ and `MPI3D`_ .
Those domain adaptation algorithms includes:
- :ref:`MDD`
.. note::
- ``Origin`` means the accuracy reported by the original paper.
- ``Avg`` is the accuracy reported by Transfer-Learn.
- ``Source Only`` refers to the model trained with data from the source domain.
- ``Oracle`` refers to the model trained with data from the target domain.
.. note::
Labels are all normalized to [0, 1] to eliminate the effects of diverse scale in regression values.
We repeat experiments on DD for three times and report the average error of the ``final`` epoch.
.. _dSprites:
dSprites error on ResNet-18
---------------------------------
=========== ====== ====== ====== ====== ====== ====== ======
Methods Avg C → N C → S N → C N → S S → C S → N
Source Only 0.157 0.232 0.271 0.081 0.220 0.038 0.092
DD 0.057 0.047 0.080 0.030 0.095 0.053 0.037
=========== ====== ====== ====== ====== ====== ====== ======
.. _MPI3D:
MPI3D error on ResNet-18
---------------------------------
=========== ======== ======== ======== ======== ======== ======== ========
Methods Avg RL → RC RL → T RC → RL RC → T T → RL T → RC
Source Only 0.176 0.232 0.271 0.081 0.220 0.038 0.092
DD 0.030 0.086 0.029 0.057 0.189 0.131 0.087
=========== ======== ======== ======== ======== ======== ======== ========
================================================
FILE: domain_generalization/Transfer-Learning-Library/docs/dalib/benchmarks/keypoint_detection.rst
================================================
==========================================
Keypoint Detection
==========================================
We provide benchmarks of different keypoint detection domain adaptation algorithms on as follows.
Those domain adaptation algorithms includes:
- :ref:`RegDA`
.. _RHD2H3D:
--------------------------------
RHD->H3D accuracy on ResNet-101
--------------------------------
=========== ====== ====== ====== ========= ======
Methods MCP PIP DIP Fingertip Avg
Source Only 67.4 64.2 63.3 54.8 61.8
RegDA 79.6 74.4 71.2 62.9 72.5
Oracle 97.7 97.2 95.7 92.5 95.8
=========== ====== ====== ====== ========= ======
.. _Surreal2Human36M:
-----------------------------------------
Surreal->Human3.6M accuracy on ResNet-101
-----------------------------------------
=========== ======== ====== ====== ===== ===== ===== =====
Methods Shoulder Elbow Wrist Hip Knee Ankle Avg
Source Only 69.4 75.4 66.4 37.9 77.3 77.7 67.3
RegDA 73.3 86.4 72.8 54.8 82.0 84.4 75.6
Oracle 95.3 91.8 86.9 95.6 94.1 93.6 92.9
=========== ======== ====== ====== ===== ===== ===== =====
.. _Surreal2LSP:
-----------------------------------
Surreal->LSP accuracy on ResNet-101
-----------------------------------
=========== ======== ====== ====== ===== ===== ===== =====
Methods Shoulder Elbow Wrist Hip Knee Ankle Avg
Source Only 51.5 65.0 62.9 68.0 68.7 67.4 63.9
RegDA 62.7 76.7 71.1 81.0 80.3 75.3 74.6
=========== ======== ====== ====== ===== ===== ===== =====
================================================
FILE: domain_generalization/Transfer-Learning-Library/docs/dalib/benchmarks/multi_source_domain_adaptation.rst
================================================
==========================================
Multi Source Domain Adaptation
==========================================
We provide benchmarks of different domain adaptation algorithms on `Office-Home`_ and `DomainNet`_.
Those domain adaptation algorithms includes:
- :ref:`MDD`
.. note::
- ``Origin`` means the accuracy reported by the original paper.
- ``Avg`` is the accuracy reported by Transfer-Learn.
- ``Source Only`` refers to the model trained with data from the source domain.
- ``Oracle`` refers to the model trained with data from the target domain.
.. _Office-Home:
Office-Home accuracy on ResNet-50
---------------------------------
=========== ====== ====== ====== ====== ======
Methods Avg :Ar :Cl :Pr :Rw
Source Only 69.5 67.3 50.8 78.8 81.0
MDD 77.2 74.7 64.6 85.2 84.1
=========== ====== ====== ====== ====== ======
.. _DomainNet:
DomainNet accuracy on ResNet-50
-----------------------------------
=========== ======= ======= ======= ======= ======= ======= ======= =======
Methods Origin Avg :c :i :p :q :r :s
Source Only 32.9 47.0 64.9 25.2 54.4 16.9 68.2 52.3
MDD / 48.8 68.7 29.7 58.2 9.7 69.4 56.9
Oracle 63.0 69.1 78.2 40.7 71.6 69.7 83.8 70.6
=========== ======= ======= ======= ======= ======= ======= ======= =======
================================================
FILE: domain_generalization/Transfer-Learning-Library/docs/dalib/benchmarks/openset_domain_adaptation.rst
================================================
==========================================
Openset Domain Adaptation
==========================================
We provide benchmarks of different domain adaptation algorithms on `Office-31`_ , `Office-Home`_ and `VisDA-2017`_ as follows.
Those domain adaptation algorithms includes:
- :ref:`DANN`
- :ref:`OSBP`
.. note::
- ``Source Only`` refers to the model trained with data from the source domain.
- ``OS`` means normalized accuracy for all classes including the unknown as one class.
- ``OS*`` means normalized accuracy only on known classes.
- ``UNK`` is the accuracy of unknown samples.
In ``OS``, the accuracy of each common class has the same contribution
as the whole ``unknown`` class. Thus we report ``HOS`` used in `ROS (ECCV 2020)`_
to better measure the abilities of different open set domain adaptation algorithms.
.. math::
\textit{HOS} = 2 \cdot \dfrac{ \textit{OS*} \cdot \textit{UNK} }{ \textit{OS*} + \textit{UNK} }
The new evaluation metric is high only when both the ``OS*`` and ``UNK`` are high.
.. note::
We report the best ``HOS`` in all epochs.
DANN (baseline model) will degrade performance as training progresses, thus the
final ``HOS`` will be much lower than reported.
In contrast, OSBP will improve performance stably.
.. _Office-31:
Office-31 H-Score on ResNet-50
---------------------------------
.. note::
We conduct `21` class classification experiments in this setting (follows :ref:`OSBP`).
=========== ===== ====== ====== ====== ====== ====== ======
Methods Avg A → W D → W W → D A → D D → A W → A
Source Only 75.9 67.7 85.7 91.4 72.1 68.4 67.8
DANN 80.4 81.4 89.1 92.0 82.5 66.7 70.4
OSBP 87.8 90.7 96.4 97.5 88.7 77.0 76.7
=========== ===== ====== ====== ====== ====== ====== ======
.. _Office-Home:
.. note::
- ``Origin`` means the accuracy reported by the related paper.
- ``Avg`` is the accuracy reported by DALIB.
Office-Home HOS on ResNet-50
-----------------------------------
=========== ====== ====== ======= ======= ======= ======= ======= ======= ======= ======= ======= ======= ======= =======
Methods Origin Avg Ar → Cl Ar → Pr Ar → Rw Cl → Ar Cl → Pr Cl → Rw Pr → Ar Pr → Cl Pr → Rw Rw → Ar Rw → Cl Rw → Pr
Source Only / 59.8 55.2 65.2 71.4 52.8 59.6 65.2 55.8 44.8 68.0 63.8 49.4 68.0
DANN / 64.8 55.2 65.2 71.4 52.8 59.6 65.2 55.8 44.8 68.0 63.8 49.4 68.0
OSBP 64.7 68.6 62.0 70.8 76.5 66.4 68.8 73.8 65.8 57.1 75.4 70.6 60.6 75.9
=========== ====== ====== ======= ======= ======= ======= ======= ======= ======= ======= ======= ======= ======= =======
.. _VisDA-2017:
VisDA-2017 performance on ResNet-50
-----------------------------------
=========== ======== ====== ===== ==== ======= ======= ======= ======= ======= =======
Methods HOS OS OS* UNK bcycl bus car mcycl train truck
Source Only 42.6 37.6 34.7 55.1 42.6 6.4 30.5 67.1 84.0 0.2
DANN 57.8 50.4 45.6 78.9 20.1 71.4 29.5 74.4 67.8 10.4
OSBP 75.4 67.3 62.9 94.3 63.7 75.9 49.6 74.4 86.2 27.3
=========== ======== ====== ===== ==== ======= ======= ======= ======= ======= =======
.. _ROS (ECCV 2020): http://www.ecva.net/papers/eccv_2020/papers_ECCV/papers/123610409.pdf
================================================
FILE: domain_generalization/Transfer-Learning-Library/docs/dalib/benchmarks/partial_domain_adaptation.rst
================================================
==========================================
Partial Domain Adaptation
==========================================
We provide benchmarks of different domain adaptation algorithms on `Office-31`_ , `Office-Home`_, `VisDA-2017`_ and `ImageNet-Caltech`_.
Those domain adaptation algorithms includes:
- :ref:`DANN`
- :ref:`PADA`
- :ref:`IWAN`
- :ref:`AFN`
.. note::
- ``Origin`` means the accuracy reported by the original paper.
- ``Avg`` is the accuracy reported by Transfer-Learn.
- ``Source Only`` refers to the model trained with data from the source domain.
.. note::
We found that the accuracies of adversarial methods are not stable even after the random seed is fixed, thus
we repeat running adversarial methods on *Office-31* and *VisDA-2017* for three times and report their average accuracy.
.. _Office-31:
Office-31 accuracy on ResNet-50
---------------------------------
=========== ====== ====== ====== ====== ====== ====== ====== ======
Methods Origin Avg A → W D → W W → D A → D D → A W → A
Source Only 75.6 90.1 78.3 98.3 99.4 87.3 88.5 88.8
DANN 43.4 82.4 60.0 94.9 98.1 71.3 84.9 85.0
PADA 92.7 93.8 86.4 100.0 100.0 87.3 93.8 95.4
IWAN 94.7 94.8 91.2 99.7 99.4 89.8 94.2 94.3
AFN / 93.1 87.8 95.6 99.4 87.9 93.9 94.1
=========== ====== ====== ====== ====== ====== ====== ====== ======
.. _Office-Home:
Office-Home accuracy on ResNet-50
-----------------------------------
=========== ======= ======= ======= ======= ======= ======= ======= ======= ======= ======= ======= ======= ======= =======
Methods Origin Avg Ar → Cl Ar → Pr Ar → Rw Cl → Ar Cl → Pr Cl → Rw Pr → Ar Pr → Cl Pr → Rw Rw → Ar Rw → Cl Rw → Pr
Source Only 53.7 60.1 42.0 66.9 78.5 56.4 55.2 65.4 57.9 36.0 75.5 68.7 43.6 74.8
DANN 47.4 57.0 46.2 59.3 76.9 47.0 47.4 56.4 51.6 38.8 72.1 68.0 46.1 74.2
PADA 62.1 65.9 52.9 69.3 82.8 59.0 57.5 66.4 66.0 41.7 82.5 78.0 50.2 84.1
IWAN 63.6 71.3 59.2 76.6 84.0 67.8 66.7 69.2 73.3 55.0 83.9 79.0 58.3 82.2
AFN 71.8 72.6 59.2 76.7 82.8 72.5 74.5 76.8 72.5 56.7 80.8 77.0 60.5 81.6
=========== ======= ======= ======= ======= ======= ======= ======= ======= ======= ======= ======= ======= ======= =======
.. _ImageNet-Caltech:
ImageNet-Caltech accuracy on ResNet-50
--------------------------------------
=========== ======= ======= ==== ====
Methods Origin Avg I→C C→I
Source Only 68.9 73.3 71.8 74.8
DANN 60.8 73.1 71.6 74.5
PADA 72.8 79.2 79.2 79.1
IWAN 75.7 78.9 77.5 75.7
=========== ======= ======= ==== ====
.. _VisDA-2017:
VisDA-2017 accuracy on ResNet-50
-----------------------------------
Note that `Origin` means the accuracy reported by the original paper,
`Mean` refers to the accuracy average over classes, while `Avg` refers to accuracy average over samples.
=========== ========== ======= ======= ======= ======= ======= ======= ======= =======
Methods Origin Mean plane bcycl bus car horse knife Avg
Source Only 45.3 50.9 59.2 31.3 68.7 73.2 69.3 3.4 60.0
DANN 51.0 55.9 88.4 34.1 72.1 50.7 61.9 27.8 57.1
PADA 53.5 60.5 89.4 35.1 72.5 69.2 86.7 10.1 66.8
IWAN / 61.5 89.2 57.0 61.5 55.2 80.1 25.7 66.8
AFN 67.6 61.0 79.1 62.7 73.9 49.6 79.6 21.0 64.1
=========== ========== ======= ======= ======= ======= ======= ======= ======= =======
================================================
FILE: domain_generalization/Transfer-Learning-Library/docs/dalib/benchmarks/re_identification.rst
================================================
===========================================
Re-Identification
===========================================
We provide benchmarks of different domain adaptation algorithms. Currently three datasets are supported:
`Market1501 `_,
`DukeMTMC `_, `MSMT17 `_.
Those domain adaptation algorithms includes:
- :ref:`IBN`
- :ref:`SPGAN`
- :ref:`MMT`
We adopt cross dataset setting (another one is cross camera setting). The model is first trained on source dataset, then we evaluate it on target dataset and report `mAP` (mean average precision) on target dataset.
For a fair comparison, our model is trained with standard cross entropy loss and triplet loss. We adopt modified resnet architecture from `Mutual Mean-Teaching: Pseudo Label Refinery for Unsupervised Domain Adaptation on Person Re-identification (ICLR 2020) `_.
As we are given unlabeled samples from target domain, we can utilize clustering algorithms to produce pseudo labels on target domain and then use them as supervision signals to perform self-training. This simple method turns out to be a strong baseline. We use ``Baseline_Cluster`` to represent this baseline in our results.
.. note::
- ``Avg`` means the average mAP across these tasks reported by Transfer-Learn.
-----------------------------------
Cross dataset mAP on ResNet-50
-----------------------------------
========================= ======= ============= ============= ============= ============= =========== ===========
Methods Avg Market2Duke Duke2Market Market2MSMT MSMT2Market Duke2MSMT MSMT2Duke
Baseline 27.1 32.4 31.4 8.2 36.7 11.0 43.1
IBN 30.0 35.2 36.5 11.3 38.7 14.1 44.3
SPGAN 30.7 34.4 35.4 14.1 40.2 16.1 43.8
Baseline_Cluster(kmeans) 45.1 52.8 59.5 19.0 62.6 20.3 56.2
Baseline_Cluster(dbscan) 54.9 62.5 73.5 25.2 77.9 25.3 65.0
MMT(kmeans) 55.4 63.7 72.5 26.2 75.8 28.0 66.1
MMT(dbscan) 60.0 68.2 80.0 28.2 82.5 31.2 70.0
========================= ======= ============= ============= ============= ============= =========== ===========
================================================
FILE: domain_generalization/Transfer-Learning-Library/docs/dalib/benchmarks/semantic_segmentation.rst
================================================
==========================================
Semantic Segmentation
==========================================
We provide benchmarks of different segmentation domain adaptation algorithms on `GTA5->Cityscapes`_ and `Synthia->Cityscapes`_ as follows.
Those domain adaptation algorithms includes:
- :ref:`CycleGAN`
- :ref:`Cycada`
- :ref:`ADVENT`
- :ref:`FDA`
.. note::
- ``Origin`` means the accuracy reported by the original paper.
- ``mIoU`` is the mean IoU reported by Transfer-Learn.
- ``Src Only`` refers to the model trained with data from the source domain.
- ``Oracle`` refers to the model trained with data from the target domain.
.. _GTA5->Cityscapes:
GTA5->Cityscapes mIoU on deeplabv2 (ResNet-101)
-----------------------------------------------
=========== ====== ======= ======= ======== ======== ======= ======= ======= ======= ======= =======
Methods Origin mIoU road sidewalk building wall fence pole light sign veg
Src Only 27.1 37.3 66.5 17.4 73.3 13.4 21.5 22.8 30.1 17.1 82.2
CycleGAN / 47.0 88.4 41.9 83.6 34.4 23.9 32.9 35.5 26.0 83.1
Cycada 42.7 47.4 87.3 35.7 83.7 31.3 24.0 32.2 35.8 30.3 82.7
ADVENT 43.8 43.8 89.3 33.9 80.3 24.0 25.2 27.8 36.7 18.2 84.3
FDA 44.6 45.6 85.5 31.7 81.8 27.1 24.9 28.9 38.1 23.2 83.7
Oracle 65.1 70.5 97.4 79.7 90.1 53.0 50.0 48.0 55.5 67.2 90.2
=========== ====== ======= ======= ======== ======== ======= ======= ======= ======= ======= =======
=========== ======= ======= ======= ======= ======= ======= ======= ======= ======= =======
Methods terrain sky person rider car truck bus train mbike bike
Src Only 7.1 73.6 57.4 28.4 78.6 36.1 13.4 1.5 31.9 36.2
CycleGAN 36.8 82.3 59.9 27.0 83.4 31.6 42.3 11.0 28.2 40.5
Cycada 32.0 85.7 60.8 31.5 85.6 39.8 43.3 5.4 29.5 44.6
ADVENT 33.9 81.3 59.8 28.4 84.3 34.1 44.4 0.1 33.2 12.9
FDA 40.3 80.6 60.5 30.3 79.1 32.8 45.1 5.0 32.4 35.2
Oracle 60.0 93.0 72.7 55.2 92.7 76.5 78.5 56.0 54.6 68.8
=========== ======= ======= ======= ======= ======= ======= ======= ======= ======= =======
.. _Synthia->Cityscapes:
Synthia->Cityscapes mIoU on deeplabv2 (ResNet-101)
--------------------------------------------------
========= ====== ==== ==== ======== ======== ===== ==== ==== ==== ====== ===== ==== ==== ===== ====
Methods Origin mIoU road sidewalk building light sign veg sky person rider car bus mbike bike
Src Only 22.1 41.5 59.6 21.1 77.4 7.7 17.6 78.0 84.5 53.2 16.9 65.9 24.9 8.5 24.8
ADVENT 47.6 47.9 88.3 44.9 80.5 4.5 9.1 81.3 86.2 52.9 21.0 82.0 30.3 11.9 30.2
FDA / 43.9 62.5 23.7 78.5 9.4 15.7 78.3 81.1 52.3 18.7 79.8 32.5 8.7 29.6
Oracle 71.7 76.6 97.4 79.7 90.1 55.5 67.2 90.2 93.0 72.7 55.2 92.7 78.5 54.6 68.8
========= ====== ==== ==== ======== ======== ===== ==== ==== ==== ====== ===== ==== ==== ===== ====
Cityscapes->Foggy Cityscapes mIoU on deeplabv2 (ResNet-101)
--------------------------------------------------------------
=========== ======= ======= ======== ======== ======= ======= ======= ======= ======= =======
Methods mIoU road sidewalk building wall fence pole light sign veg
Src Only 51.2 95.3 70.2 64.1 31.9 35.2 30.7 33.3 51.1 42.3
CycleGAN 66.0 97.1 77.6 84.3 42.7 46.3 42.8 47.5 61.0 84.0
Cycada 63.3 96.8 75.5 79.1 38.0 40.3 42.1 48.2 61.2 76.9
ADVENT 61.8 96.8 75.1 76.4 46.2 42.6 39.3 43.6 58.9 74.3
FDA 61.9 96.9 77.2 75.3 46.5 42.0 39.8 47.1 61.0 72.7
Oracle 66.9 97.4 78.6 88.1 50.7 50.5 46.2 51.3 64.4 88.1
=========== ======= ======= ======== ======== ======= ======= ======= ======= ======= =======
=========== ======= ======= ======= ======= ======= ======= ======= ======= ======= =======
Methods terrain sky person rider car truck bus train mbike bike
Src Only 44.0 32.1 64.4 47.0 86.0 64.4 56.4 21.1 43.1 60.8
CycleGAN 55.2 83.4 69.4 51.8 90.7 73.7 76.2 54.2 50.7 65.6
Cycada 52.1 77.6 68.6 51.7 90.4 71.7 70.4 43.3 52.6 65.7
ADVENT 50.1 75.9 67.3 51.0 89.4 70.5 64.7 39.9 47.9 65.0
FDA 54.6 63.8 68.4 50.1 90.1 72.8 68.0 35.5 50.8 64.2
Oracle 55.3 87.4 70.9 52.7 91.6 72.4 73.2 31.8 52.2 67.4
=========== ======= ======= ======= ======= ======= ======= ======= ======= ======= =======
================================================
FILE: domain_generalization/Transfer-Learning-Library/docs/dalib/modules.rst
================================================
Common Adaptation Modules
===========================
Domain Discriminator
-------------------------------
.. autoclass:: dalib.modules.domain_discriminator.DomainDiscriminator
:members:
GRL
------------------------
.. autoclass:: dalib.modules.grl.WarmStartGradientReverseLayer
:members:
Kernels
------------------------
.. autoclass:: dalib.modules.kernels.GaussianKernel
Entropy
------------------------
.. autofunction:: dalib.modules.entropy.entropy
================================================
FILE: domain_generalization/Transfer-Learning-Library/docs/dglib/benchmarks/image_classification.rst
================================================
===============================
Image Classification
===============================
We provide benchmarks of different domain generalization algorithms on `PACS`_, `Office-Home`_,
`iWildCam-Wilds`_, `Camelyon17-Wilds`_, `FMoW-Wilds`_.
Those domain generalization algorithms includes:
- :ref:`IBN`
- :ref:`MIXSTYLE`
- :ref:`MLDG`
- :ref:`IRM`
- :ref:`VREX`
- :ref:`GroupDRO`
- :ref:`CORAL`
.. note::
`DomainBed `_ proposed three model selection methods, our hyper
parameter is selected based on model's performance on `training-domain validation set` (first rule proposed).
Concretely, we select model with highest accuracy on `training-domain validation set` during whole training
process and use selected checkpoint to test on target domain.
.. note::
Different from `DomainBed `_, we do not freeze `BatchNorm2d` layers
and do not insert additional `Dropout` layer except for `PACS` dataset. Besides, we use `SGD` with momentum as
default optimizer and find it usually achieves better results than `Adam`. During training, a cosine learning rate
decay strategy is used.
.. note::
- ``ERM`` refers to the model trained with `ERM `_, which is a strong baseline.
- ``Avg`` is the average accuracy.
- ``Acc1`` is the top-1 accuracy on `OOD` test set for Wilds datasets.
.. _PACS:
-----------------------------------
PACS accuracy on ResNet-50
-----------------------------------
======== ===== ===== ===== ===== =====
Methods avg A C P S
ERM 86.4 88.5 78.4 97.2 81.4
IBN 87.8 88.2 84.5 97.1 81.4
MixStyle 87.4 87.8 82.3 95.0 84.5
MLDG 87.2 88.2 81.4 96.6 82.5
IRM 86.9 88.0 82.5 98.0 79.0
VREx 87.0 87.2 82.3 97.4 81.0
GroupDRO 87.3 88.9 81.7 97.8 80.8
CORAL 86.4 89.1 80.0 97.4 79.1
======== ===== ===== ===== ===== =====
.. _Office-Home:
-----------------------------------
Office-Home accuracy on ResNet-50
-----------------------------------
======== ===== ===== ===== ===== =====
Methods avg A C P R
ERM 70.8 68.3 55.9 78.9 80.0
IBN 69.9 67.4 55.2 77.3 79.6
MixStyle 71.7 66.8 58.1 78.0 79.9
MLDG 70.3 65.9 57.6 78.2 79.6
IRM 70.3 66.7 54.8 78.6 80.9
VREx 70.2 66.9 54.9 78.2 80.9
GroupDRO 70.0 66.7 55.2 78.8 79.9
CORAL 70.9 68.3 55.4 78.8 81.0
======== ===== ===== ===== ===== =====
.. _iWildCam-Wilds:
----------------------------------------
iWildCam-Wilds accuracy on ResNet-50
----------------------------------------
======== ======
Methods acc1
ERM 75.4
IBN 77.3
MixStyle 71.0
IRM 75.5
VREx 71.5
GroupDRO 28.0
CORAL 71.0
======== ======
.. _Camelyon17-Wilds:
----------------------------------------
Camelyon17-Wilds accuracy on ResNet-50
----------------------------------------
======== ======
Methods acc1
ERM 94.6
IBN 96.1
MixStyle 94.2
MLDG 91.2
IRM 94.9
VREx 88.2
GroupDRO 93.1
CORAL 90.6
======== ======
.. _FMoW-Wilds:
--------------------------------------
FMoW-Wilds accuracy on DenseNet-121
--------------------------------------
======== ======
Methods acc1
ERM 53.0
MLDG 47.4
IRM 48.1
VREx 50.4
GroupDRO 47.5
CORAL 50.0
======== ======
================================================
FILE: domain_generalization/Transfer-Learning-Library/docs/dglib/benchmarks/re_identification.rst
================================================
===============================
Re-Identification
===============================
We provide benchmarks of different domain generalization algorithms. Currently three datasets are supported:
`Market1501 `_,
`DukeMTMC `_, `MSMT17 `_.
Those domain generalization algorithms includes:
- :ref:`IBN`
- :ref:`MIXSTYLE`
.. note::
We adopt cross dataset setting (another one is cross camera setting). The model is first trained on source dataset,
then we evaluate it on target dataset and report `mAP` (mean average precision) on target dataset.
.. note::
For a fair comparison, our model is trained with standard cross entropy loss and triplet loss. We adopt modified
resnet architecture from `Mutual Mean-Teaching: Pseudo Label Refinery for Unsupervised
Domain Adaptation on Person Re-identification (ICLR 2020) `_.
-----------------------------------
Cross dataset mAP on ResNet-50
-----------------------------------
======== ======= ============= ============= ============= ============= =========== ===========
Methods Avg Market2Duke Duke2Market Market2MSMT MSMT2Market Duke2MSMT MSMT2Duke
Baseline 23.5 25.6 29.6 6.3 31.7 10.1 37.8
IBN 27.0 31.5 33.3 10.4 33.6 13.7 40.0
MixStyle 25.5 27.2 31.6 8.2 33.9 12.4 39.9
======== ======= ============= ============= ============= ============= =========== ===========
================================================
FILE: domain_generalization/Transfer-Learning-Library/docs/dglib/generalization/architecture_design.rst
================================================
=======================================
Architecture Design
=======================================
.. _IBN:
IBN-Net: Instance-Batch Normalization Network
------------------------------------------------
.. autoclass:: common.vision.models.ibn.IBN
.. autoclass:: common.vision.models.ibn.ResNet_IBN
:members:
.. automodule:: common.vision.models.ibn
:members:
================================================
FILE: domain_generalization/Transfer-Learning-Library/docs/dglib/generalization/data_manipulation.rst
================================================
=======================================
Data Manipulation
=======================================
.. _MIXSTYLE:
Domain Generalization with MixStyle (MixStyle)
-------------------------------------------------
.. autoclass:: dglib.generalization.mixstyle.models.mixstyle.MixStyle
.. note::
MixStyle is only activated during `training` stage, with some probability :math:`p`.
.. automodule:: dglib.generalization.mixstyle.models.resnet
:members:
================================================
FILE: domain_generalization/Transfer-Learning-Library/docs/dglib/generalization/learning_strategy.rst
================================================
=======================================
Learning Strategy
=======================================
.. _GroupDRO:
Group Distributionally robust optimization (GroupDRO)
------------------------------------------------------
.. autoclass:: dglib.generalization.groupdro.AutomaticUpdateDomainWeightModule
:members:
.. _IRM:
Invariant Risk Minimization (IRM)
----------------------------------
.. autoclass:: dglib.generalization.irm.InvariancePenaltyLoss
.. _MLDG:
Meta Learning for Domain Generalization (MLDG)
------------------------------------------------
`Learning to Generalize: Meta-Learning for Domain Generalization (AAAI 2018) `_
Consider there are :math:`S` source domains, at each learning iteration MLDG splits the
original :math:`S` source domains into `meta-train` domains :math:`S_1` and `meta-test` domains :math:`S_2`. The `inner`
objective is cross entropy loss on `meta-train` domains :math:`S_1`. The `outer` (meta-optimization)
objective contains two terms. The first one (which is the same as inner objective) is cross entropy loss on `meta-train`
domains :math:`S_1` with current model parameters :math:`\theta`
.. math::
\mathbb{E}_{(x,y) \in S_1} l(f(\theta, x), y)
where :math:`l` denotes cross entropy loss, :math:`f(\theta, x)` denotes predictions from model.
The second term is cross entropy loss on `meta-test` domains :math:`S_2` with inner optimized model parameters
:math:`\theta_{updated}`
.. math::
\mathbb{E}_{(x,y) \in S_2} l(f(\theta_{updated}, x), y)
In this way, MLDG simulates train/test domain shift during training by synthesizing virtual testing domains within
each mini-batch. The outer objective forces that steps to improve training domain performance should
also improve testing domain performance.
.. note::
Because we need to compute second-order gradient, this full optimization process may take a long time and have
heavy budget on GPU resource. A first order approximation implementation can be found at
`DomainBed `_.
.. _VREX:
Variance Risk Extrapolation (VREx)
------------------------------------------------
`Out-of-Distribution Generalization via Risk Extrapolation (ICML 2021) `_
VREx shows that reducing differences in risk across training domains can reduce a model’s sensitivity to a wide range
of extreme distributional shifts. Consider there are :math:`S` source domains. At each learning iteration VREx first
computes cross entropy loss on each source domain separately, producing a loss vector :math:`l \in R^S`. The `ERM`
(vanilla cross entropy) loss can be computed as
.. math::
l_{\text{ERM}} = \frac{1}{S}\sum_{i=1}^S l_i
And the penalty loss is
.. math::
penalty = \frac{1}{S} \sum_{i=1}^S {(l_i - l_{\text{ERM}})}^2
The final objective is then
.. math::
objective = l_{\text{ERM}} + \beta * penalty
where :math:`\beta` is the trade off hyper parameter.
================================================
FILE: domain_generalization/Transfer-Learning-Library/docs/dglib/generalization/representation_learning.rst
================================================
=======================================
Representation Learning
=======================================
.. _CORAL:
Correlation Alignment for Deep Domain Adaptation (Deep CORAL)
--------------------------------------------------------------
.. autoclass:: dglib.generalization.coral.CorrelationAlignmentLoss
.. note::
Under domain generalization setting, we can't access to samples from target domain during training. Therefore,
two different domains from source domains are treated as source domain and target domain respectively to calculate
`Correlation Alignment Loss`.
================================================
FILE: domain_generalization/Transfer-Learning-Library/docs/dglib/modules.rst
================================================
Common Generalization Modules
==============================
Classifier
-----------------------------
.. autoclass:: dglib.modules.classifier.ImageClassifier
Sampler
-----------------------------
.. autoclass:: dglib.modules.sampler.DefaultSampler
.. autoclass:: dglib.modules.sampler.RandomDomainSampler
.. autoclass:: dglib.generalization.mixstyle.sampler.RandomDomainMultiInstanceSampler
================================================
FILE: domain_generalization/Transfer-Learning-Library/docs/get_started/faq.rst
================================================
*************
FAQ
*************
1. How to build the doc?
=========================
Sometimes, the online doc is not consistent with the latest code. In this case, you can build the doc by yourself.
First, you need to install sphinx, which is used to build doc.
.. code-block:: shell
pip install -U sphinx
Second, you need to download a pytorch-style `html.zip `_
into directory ``docs/build`` and unzip it. Some browsers may identify it as malicious file. You can safely ignore those warnings.
Then, in the directory ``docs`` run the following command
.. code-block:: shell
make html
Also, warnings during ``make`` process doesn't matter and can be ignored.
Finally, you can open the docs in ``docs/build/html/index.html``
2. How to customize model?
===============================
A typical classifier has 3 components.
- backbone: usually ``ResNet`` network to extract feature maps.
- bottleneck: random initialized bottleneck layers between ``backbone`` and ``head`` to increase model capacity
- head: classifier head which outputs final predictions.
The way we construct an image classifier is as follows.
.. code-block:: python
import common.modules.Classifier
# define classifier
classifier = Classifier(backbone=your_backbone, num_classes=num_classes, bottleneck=your_bottleneck,
bottleneck_dim=your_bottleneck_dim, finetune=True)
Below are some examples on how to change these components.
2.1 Define a new backbone
'''''''''''''''''''''''''''
.. code-block:: python
import torch.nn as nn
class FeatureExtractor(nn.Module):
def __init__(self):
pass
def forward(self, x):
pass
@property
def out_features(self):
pass
your_backbone = FeatureExtractor()
Notice your ``FeatureExtractor`` should implement `out_features` method, which returns the dimension of output features.
2.2 Define a new bottleneck
'''''''''''''''''''''''''''
.. code-block:: python
import torch.nn as nn
your_bottleneck = nn.Sequential(
nn.AdaptiveAvgPool2d(output_size=(1, 1)), # for 2d images only
nn.Flatten(),
nn.Linear(512, 512),
nn.BatchNorm1d(512),
nn.ReLU()
)
2.3 Override get_parameters method
'''''''''''''''''''''''''''''''''''
In ``DA`` settings, we usually use smaller learning rates for ``backbone`` parameters, typically ``0.1x`` compared to other layers.
If you want to use other strategies such as different momentum, different weight_decay for different parts, you should inherit `Classifier` and
override ``get_parameters`` method.
Below we use same learning rate for ``backbone``, ``bottleneck`` and ``head``.
And momentum factor of ``0.9`` is used for ``bottleneck`` and ``head``.
.. code-block:: python
import torch.nn as nn
import common.modules.Classifier
class ImageClassifier(Classifier):
def __init__(self):
pass
def get_parameters(self):
params = [
{"params": self.backbone.parameters()},
{"params": self.bottleneck.parameters(), "momentum": 0.9},
{"params": self.head.parameters(), "momentum": 0.9},
]
return params
3. How to customize datasets?
==================================
If you want to implement your own vision datasets, you can use ``torchvision.datasets.VisionDataset``
or ``common.vision.datasets.ImageList``.
Before using ``ImageList``, you need to prepare a txt file ``dog_cat.txt``.
In this file, each line should has the following format::
path/to/dog/0.jpg 0
path/to/cat/1.jpg 1
where the first part is a relative file path, and the second part is an integer label.
The way to construct an dog-cat dataset is
.. code-block:: python
import torch.nn as nn
import common.vision.datasets.ImageList
dataset = ImageList(root="your_root", classes=("dog", "cat"), data_list_file="dog_cat.txt")
================================================
FILE: domain_generalization/Transfer-Learning-Library/docs/get_started/installing.rst
================================================
************
Installation
************
Via Git or Download
===================
For flexible use and modification,
please git clone `Transfer Learning Library `_.
Check that you have install all the dependency.
.. code-block::
pip install -r requirements.txt
It's recommended to use pytorch==1.7.1 and torchvision==0.8.2 in order to better reproduce the benchmark results.
================================================
FILE: domain_generalization/Transfer-Learning-Library/docs/get_started/introduction.rst
================================================
************
Introduction
************
Currently, **Trans-Learn** provide algorithms for **domain adaptation (DA)**, **task adaptation (TA)**, and **domain generalization (DG)**.
We implement training codes (in directory ``examples``) in a unified way, which allows quantitative, fair, and reproducible comparisons between different algorithms.
Comparison with Existing Library
===================================
There exist several open-source transfer learning libraries,
but most of them only focus on domain adaptation, especially statistics matching and adversarial adaptation, while ignoring other adaptation methods.
In contrast, Transfer-Learn covers more types of adaptation algorithms.
- For more convenient algorithm selection, we provide the most comprehensive benchmark among all those libraries.
- For faster algorithm reproduction, we provide training scripts for all the results in the benchmark.
- For better application of the library, we provide detailed documentation and examples in many downstream tasks.
As far as we know, Transfer-Learn is currently the only one open source transfer learning library that can meet all the above requirements.
=========================== ================= ==================== =============== ============== ======================
Features Transfer-Learn transferlearning.xyz salad Dassl deep transfer learning
Task Adaptation :math:`\surd` :math:`\times` :math:`\times` :math:`\times` :math:`\times`
Statistics Matching :math:`\surd` :math:`\surd` :math:`\surd` :math:`\surd` :math:`\surd`
Adversarial Adaptation :math:`\surd` :math:`\surd` :math:`\surd` :math:`\surd` :math:`\surd`
Disparity Discrepancy :math:`\surd` :math:`\times` :math:`\times` :math:`\times` :math:`\times`
Translation :math:`\surd` :math:`\times` :math:`\times` :math:`\times` :math:`\times`
Benchmark :math:`\surd` :math:`\surd` :math:`\surd` :math:`\times` :math:`\surd`
Reproducible Scripts :math:`\surd` :math:`\surd` :math:`\surd` :math:`\times` :math:`\times`
Detailed Documentation :math:`\surd` :math:`\times` :math:`\surd` :math:`\times` :math:`\times`
Application :math:`\surd` :math:`\times` :math:`\times` :math:`\times` :math:`\surd`
=========================== ================= ==================== =============== ============== ======================
Library Usage
=============
Here we give a short description on how to use Transfer-Learn using `DANN `_ as an instance.
In the original implementation of DANN, the domain adversarial loss, domain discriminator, feature extractor, and classifier are tightly coupled together in one *nn.Module*, which cause the difficulty of reuse, e.g., the entire algorithm needs re-implementation when the input data is changed from image to text. Yet in this case, the domain adversarial loss and the domain discriminator remains unchanged and can be reused.
Thus, in Transfer-Learn, models and loss functions are decoupled.
When using DANN, users need to initialize a domain discriminator and pass it to the domain adversarial loss module, and then use this module the same way as the cross-entropy loss module defined in the Pytorch,
.. code:: python
>>> # define the domain discriminator
>>> from dalib.modules.domain_discriminator import DomainDiscriminator
>>> discriminator = DomainDiscriminator(in_feature=1024, hidden_size=1024)
>>> # define the domain adversarial loss module
>>> from dalib.adptation.dann import DomainAdversarialLoss
>>> dann = DomainAdversarialLoss(discriminator, reduction='mean')
>>> # features from the source and target domain
>>> f_s, f_t = torch.randn(20, 1024), torch.randn(20, 1024)
>>> # calculate the final loss
>>> loss = dann(f_s, f_t)
Transfer-Learn provide similar API for other transfer learning algorithms. Detailed usages of these algorithms can be found at the documentation.
Library philosophy
=====================
Transfer-Learn is designed to be *extensible* by researchers and *simple* for practitioners.
Currently, there are mainly two types of algorithm implementations.
- One is to encapsulate each algorithm into a Trainer, whose typical representative is *Pytorch-lighting*. The user only needs to feed the training data to it, and does not need to care about the specific training process.
- Another strategy is to encapsulate the core loss function in each algorithm, and the user needs to implement the complete training process themselves. A typical representative is *Pytorch*.
Although the former method is easier to use, it is less extensible. Since it's often necessary to adjust training process in different transfer learning scenarios, Transfer-Learn adopts the latter method for better extensibility.
Following Pytorch, we provide training examples of different transfer algorithms in different scenarios, which allows users to quickly adapt to Transfer-Learn as long as they have learned Pytorch before.
We try our best to make Transfer-Learn easy to start with, e.g., we support the automatic download of most common transfer learning datasets so that users do not need to spend time on data preprocessing.
How to Choose Algorithms?
======================================
Although there are many methods proposed for transfer learning each year, unfortunately, no method is universal.
Each method has its advantages and disadvantages and applicable scenarios.
Therefore, we provide benchmarks under different settings. They can serve as a reference when you choose an algorithm.
You are suggested to read related papers if you want to have a deeper understanding of an algorithm.
After completing the above things, trying is still the most important.
In addition to the final accuracy, you need to pay attention to the output of the program throughout the training process.
You can also use the tools in ``common.utils.analysis`` to visualize the results.
================================================
FILE: domain_generalization/Transfer-Learning-Library/docs/get_started/quickstart.rst
================================================
************
Quick Start
************
In this section, we will use `DANN `_ as an example,
and show you how to reproduce the benchmark results.
Before going deeper, please **make sure** you have installed all the dependency.
Step1: Find it
===================
Our code for domain adaptation is in the directory ``examples/domain_adaptation``.
DANN is designed for close set domain adaptation tasks. You can find the training code in
``examples/domain_adaptation/image_classification``. This directory contains implementations for other algorithms such as ``CDAN``, ``MDD``.
For now, you only need to care about two files: ``dann.py`` and ``dann.sh``.
Step2: Run it
===================
``dann.py`` is an executable python file. And you can find all the necessary running scripts to
reproduce the benchmarks with specified hyper-parameters in ``dann.sh``.
For instance, running the following command will start training on ``Office-31`` dataset.
.. code-block:: shell
CUDA_VISIBLE_DEVICES=0 python dann.py data/office31 -d Office31 -s A -t W -a resnet50 --epochs 20 --seed 1 --log logs/dann/Office31_A2W
Note that ``-s`` specifies the source domain, ``-t`` specifies the target domain,
and ``--log`` specifies where to store results.
After running the above command, it will download ``Office-31`` datasets from the Internet if it's the
first time you run the code. Directory that stores datasets will be named as
``examples/domain_adaptation/image_classification/data/``.
Step3: Analysis it
===================
If everything works fine, you will see results in following format::
Epoch: [1][ 900/1000] Time 0.60 ( 0.69) Data 0.22 ( 0.31) Loss 0.74 ( 0.85) Cls Acc 96.9 (95.1) Domain Acc 64.1 (62.6)
``Trans-Learn`` provides detailed statistics to monitor during training.
Here your algorithm is running in epoch 2 (index starts from 0), has been trained for 900 iterations.
The loss comes down to 0.74, on source domain your classifier achieves 96.9% accuracy
and your domain discriminator has an accuracy of 64.1%.
Different algorithms may show you different statistics.
You can also watch these results in a log file.
They are located in directory ``logs/dann/Office31_A2W/log.txt``.
During training, ``latest`` and ``best`` model checkpoints will be saved in directory ``logs/dann/Office31_A2W/checkpoints``.
Resuming from checkpoint is also **supported**.
After training, you can test your algorithm's performance by passing in ``--phase test``.
.. code-block:: shell
CUDA_VISIBLE_DEVICES=0 python dann.py data/office31 -d Office31 -s A -t W -a resnet50 --epochs 20 --seed 1 --log logs/dann/Office31_A2W --phase test
In next section, we will introduce how to visualize the results in details.
================================================
FILE: domain_generalization/Transfer-Learning-Library/docs/get_started/visualization.rst
================================================
*************
Visualization
*************
How to visualize the representations using t-SNE?
===================================================================
Many algorithms aim at aligning feature representations between ``source`` and ``target`` domain.
Through visualization, you can find and analysis the mis-alignment between different domains.
After training `DANN`, in directory ``examples/domain_adaptation/image_classification``, run the following command
.. code-block:: shell
CUDA_VISIBLE_DEVICES=0 python dann.py data/office31 -d Office31 -s A -t W -a resnet50 --epochs 20 --seed 1 --log logs/dann/Office31_A2W --phase analysis
It may take a while, then in directory ``logs/dann/Office31_A2W/visualize``, you can find
``TSNE.png``.
.. figure:: ../_static/images/resnet_A2W.png
:width: 300
t-SNE of representations from ResNet50 trained on source domain.
.. figure:: ../_static/images/dann_A2W.png
:width: 300
t-SNE of representations from DANN.
How to visualize the segmentation predictions?
===================================================================
For each segmentation algorithms, we've implemented the visualization code. All you need to do is set ``--debug`` during training.
For instance, in the directory ``examples/domain_adaptation/semantic_segmentation``,
.. code-block:: shell
CUDA_VISIBLE_DEVICES=0 python source_only.py data/GTA5 data/Cityscapes -s GTA5 -t Cityscapes --log logs/src_only/gtav2cityscapes --debug
Then you can find visualization images in directory ``logs/src_only/gtav2cityscapes/visualize/``.
.. figure:: ../_static/images/visualization/segmentation_image.png
:width: 400
Cityscapes image.
.. figure:: ../_static/images/visualization/segmentation_pred.png
:width: 400
Segmentation predictions.
.. figure:: ../_static/images/visualization/segmentation_label.png
:width: 400
Segmentation labels.
Translation model such as CycleGAN will save images by default. Here is the translation results from source style to target style.
.. figure:: ../_static/images/visualization/cyclegan_real_S.png
:width: 400
Source images.
.. figure:: ../_static/images/visualization/cyclegan_fake_T.png
:width: 400
Source image in target style.
How to visualize the keypoint detection predictions?
===================================================================
For each keypoint detection algorithms, we've implemented the visualization code. All you need to do is set ``--debug`` during training.
For instance, in the directory ``examples/domain_adaptation/keypoint_detection``,
.. code-block:: shell
CUDA_VISIBLE_DEVICES=0 python source_only.py data/RHD data/H3D_crop -s RenderedHandPose -t Hand3DStudio --log logs/baseline/rhd2h3d --debug --seed 0
Then you can find visualization images in directory ``logs/baseline/rhd2h3d/visualize/``.
.. figure:: ../_static/images/visualization/keypoint_detection.jpg
:width: 300
================================================
FILE: domain_generalization/Transfer-Learning-Library/docs/index.rst
================================================
=====================================
Transfer Learning
=====================================
.. toctree::
:caption: Get Started for Transfer Learning Library
:maxdepth: 2
:titlesonly:
get_started/installing
get_started/introduction
get_started/quickstart
get_started/visualization
get_started/faq
.. toctree::
:maxdepth: 2
:caption: Domain Adaptation Settings
:titlesonly:
dalib/benchmarks/image_classification
dalib/benchmarks/image_regression
dalib/benchmarks/semantic_segmentation
dalib/benchmarks/keypoint_detection
dalib/benchmarks/re_identification
dalib/benchmarks/partial_domain_adaptation
dalib/benchmarks/openset_domain_adaptation
dalib/benchmarks/multi_source_domain_adaptation
.. toctree::
:maxdepth: 2
:caption: Task Adaptation Settings
:titlesonly:
talib/benchmarks/image_classification
.. toctree::
:maxdepth: 2
:caption: Domain Generalization Settings
:titlesonly:
dglib/benchmarks/image_classification
dglib/benchmarks/re_identification
.. toctree::
:maxdepth: 1
:caption: Domain Adaptation Methods
:titlesonly:
dalib/adaptation/statistics_matching.rst
dalib/adaptation/domain_adversarial.rst
dalib/adaptation/hypothesis_adversarial.rst
dalib/adaptation/domain_translation.rst
dalib/adaptation/others.rst
dalib/modules.rst
.. toctree::
:maxdepth: 1
:caption: Task Adaptation Methods
:titlesonly:
talib/finetune.rst
.. toctree::
:maxdepth: 1
:caption: Domain Generalization Methods
:titlesonly:
dglib/generalization/data_manipulation.rst
dglib/generalization/architecture_design.rst
dglib/generalization/representation_learning.rst
dglib/generalization/learning_strategy.rst
dglib/modules.rst
.. toctree::
:maxdepth: 1
:caption: Common Packages
:titlesonly:
common/modules
common/loss
common/vision/datasets
common/vision/models
common/vision/transforms
common/utils/base
common/utils/metric
common/utils/analysis
================================================
FILE: domain_generalization/Transfer-Learning-Library/docs/make.bat
================================================
@ECHO OFF
pushd %~dp0
REM Command file for Sphinx documentation
if "%SPHINXBUILD%" == "" (
set SPHINXBUILD=python -msphinx
)
set SPHINXOPTS=
set SPHINXBUILD=sphinx-build
set SOURCEDIR=.
set BUILDDIR=build
set SPHINXPROJ=PyTorchSphinxTheme
if "%1" == "" goto help
%SPHINXBUILD% >NUL 2>NUL
if errorlevel 9009 (
echo.
echo.The Sphinx module was not found. Make sure you have Sphinx installed,
echo.then set the SPHINXBUILD environment variable to point to the full
echo.path of the 'sphinx-build' executable. Alternatively you may add the
echo.Sphinx directory to PATH.
echo.
echo.If you don't have Sphinx installed, grab it from
echo.http://sphinx-doc.org/
exit /b 1
)
%SPHINXBUILD% -M %1 %SOURCEDIR% %BUILDDIR% %SPHINXOPTS%
goto end
:help
%SPHINXBUILD% -M help %SOURCEDIR% %BUILDDIR% %SPHINXOPTS%
:end
popd
================================================
FILE: domain_generalization/Transfer-Learning-Library/docs/requirements.txt
================================================
sphinxcontrib-httpdomain
sphinx
================================================
FILE: domain_generalization/Transfer-Learning-Library/docs/talib/benchmarks/image_classification.rst
================================================
========================================================
Image Classification
========================================================
We provide benchmarks of different task adaptation algorithms on fine-grained classification datasets `CUB-200-2011`_, `StanfordCars`_,
`Aircraft`_, and specialized datasets `Resisc45`_, `PatchCamelyon`_.
Those task adaptation algorithms includes:
- :ref:`LWF`
- :ref:`L2SP`
- :ref:`BSS`
- :ref:`DELTA`
- :ref:`CoTuning`
- :ref:`StochNorm`
- :ref:`BiTuning`
We follow the common practice in the community as described in :ref:`BSS`.
Training iterations and data augmentations are kept the same for different task-adaptation
methods for a fair comparison.
Hyper-parameters of each method are selected by the performance on target validation data.
.. _CUB-200-2011:
------------------------------------------------------------------------
CUB-200-2011 on ResNet-50 (Supervised Pre-trained)
------------------------------------------------------------------------
=========== ====== ====== ====== ====== ======
Sample rate 15% 30% 50% 100% Avg
Baseline 51.2 64.6 74.6 81.8 68.1
LWF 56.7 66.8 73.4 81.5 69.6
BSS 53.4 66.7 76.0 82.0 69.5
DELTA 54.8 67.3 76.3 82.3 70.2
StochNorm 54.8 66.8 75.8 82.2 69.9
Co-Tuning 57.6 70.1 77.3 82.5 71.9
Bi-Tuning 55.8 69.3 77.2 83.1 71.4
=========== ====== ====== ====== ====== ======
.. _StanfordCars:
------------------------------------------------------------------------
Stanford Cars on ResNet-50 (Supervised Pre-trained)
------------------------------------------------------------------------
=========== ====== ====== ====== ====== ======
Sample rate 15% 30% 50% 100% Avg
Baseline 41.1 65.9 78.4 87.8 68.3
LWF 44.9 67.0 77.6 87.5 69.3
BSS 43.3 67.6 79.6 88.0 69.6
DELTA 45.0 68.4 79.6 88.4 70.4
StochNorm 44.4 68.1 79.3 87.9 69.9
Co-Tuning 49.0 70.6 81.9 89.1 72.7
Bi-Tuning 48.3 72.8 83.3 90.2 73.7
=========== ====== ====== ====== ====== ======
.. _Aircraft:
------------------------------------------------------------------------
FGVC Aircraft on ResNet-50 (Supervised Pre-trained)
------------------------------------------------------------------------
=========== ====== ====== ====== ====== ======
Sample rate 15% 30% 50% 100% Avg
Baseline 41.6 57.8 68.7 80.2 62.1
LWF 44.1 60.6 68.7 82.4 64.0
BSS 43.6 59.5 69.6 81.2 63.5
DELTA 44.4 61.9 71.4 82.7 65.1
StochNorm 44.3 60.6 70.1 81.5 64.1
Co-Tuning 45.9 61.2 71.3 82.2 65.2
Bi-Tuning 47.2 64.3 73.7 84.3 67.4
=========== ====== ====== ====== ====== ======
Resisc45 and PatchCamelyon have very different data distributions from ImageNet,
thus some regularization-based fine-tuning methods lead to negative transfer on these two benchmarks.
.. _Resisc45:
------------------------------------------------------------------------
Resisc45 on ResNet-50 (Supervised Pre-trained)
------------------------------------------------------------------------
================= ====== ====== ====== ====== ======
#samples/#classes 10 20 40 80 Avg
Baseline 74.0 81.9 86.5 90.1 83.1
LWF 74.2 81.7 86.1 89.9 83.0
BSS 73.5 80.9 85.7 90.0 82.5
DELTA 73.9 81.7 86.0 90.1 82.9
StochNorm 73.9 82.1 87.0 90.2 83.3
Co-Tuning 75.0 82.7 87.3 91.4 84.1
Bi-Tuning 75.3 83.2 87.4 91.4 84.3
================= ====== ====== ====== ====== ======
.. _PatchCamelyon:
------------------------------------------------------------------------
PatchCamelyon on ResNet-50 (Supervised Pre-trained)
------------------------------------------------------------------------
================= ====== ====== ====== ====== ======
#samples/#classes 40 80 160 320 Avg
Baseline 75.5 75.9 80.4 81.2 78.3
LWF 75.3 77.7 80.6 82.5 79.0
BSS 78.0 78.2 80.4 80.4 79.3
DELTA 74.1 76.4 80.0 81.9 78.1
StochNorm 75.9 77.1 78.2 81.3 78.1
Co-Tuning 75.1 76.2 80.7 81.8 78.5
Bi-Tuning 75.1 77.6 80.6 81.4 78.7
================= ====== ====== ====== ====== ======
We further evaluate task adaptation algorithms when the downstream tasks are different
from the pre-training tasks. The pre-training task is MoCo unsupervised pre-training, and
the downstream tasks are still fine-grained classification.
In this scenario, some regularization-based fine-tuning methods also lead to negative transfer.
.. _CUB-200-2011_MoCo:
------------------------------------------------------------------------
CUB-200-2011 on ResNet-50 (MoCo Pre-trained)
------------------------------------------------------------------------
=========== ====== ====== ====== ====== ======
Sample rate 15% 30% 50% 100% Avg
Baseline 28.0 48.2 62.7 75.6 53.6
LWF 28.8 50.1 62.8 76.2 54.5
BSS 30.9 50.3 63.7 75.8 55.2
DELTA 27.9 51.4 65.9 74.6 55.0
StochNorm 20.8 44.9 60.1 72.8 49.7
Co-Tuning 29.1 50.1 63.8 75.9 54.7
Bi-Tuning 32.4 51.8 65.7 76.1 56.5
=========== ====== ====== ====== ====== ======
.. _StanfordCars_MoCo:
------------------------------------------------------------------------
Stanford Cars on ResNet-50 (MoCo Pre-trained)
------------------------------------------------------------------------
=========== ====== ====== ====== ====== ======
Sample rate 15% 30% 50% 100% Avg
Baseline 42.5 71.2 83.0 90.1 71.7
LWF 44.2 71.7 82.9 90.5 72.3
BSS 45.0 71.5 83.8 90.1 72.6
DELTA 45.9 72.9 82.5 88.9 72.6
StochNorm 40.3 66.2 78.0 86.2 67.7
Co-Tuning 44.2 72.6 83.3 90.3 72.6
Bi-Tuning 45.6 72.8 83.2 90.8 73.1
=========== ====== ====== ====== ====== ======
.. _Aircraft_MoCo:
------------------------------------------------------------------------
FGVC Aircraft on ResNet-50 (MoCo Pre-trained)
------------------------------------------------------------------------
=========== ====== ====== ====== ====== ======
Sample rate 15% 30% 50% 100% Avg
Baseline 45.8 67.6 78.8 88.0 70.1
LWF 48.5 68.5 78.0 87.9 70.7
BSS 47.7 69.1 79.2 88.0 71.0
DELTA \- \- \- \- \-
StochNorm 45.4 68.8 76.7 86.1 69.3
Co-Tuning 48.2 68.5 78.7 87.3 70.7
Bi-Tuning 46.4 69.6 79.4 87.9 70.8
=========== ====== ====== ====== ====== ======
.. note::
\- indicates that the training cannot converge.
================================================
FILE: domain_generalization/Transfer-Learning-Library/docs/talib/finetune.rst
================================================
=======================================
Fine-tuning
=======================================
Regularization-based Fine-tuning
===========================================
.. _L2:
L2
------
.. autoclass:: talib.finetune.delta.L2Regularization
.. _L2SP:
L2-SP
------
.. autoclass:: talib.finetune.delta.SPRegularization
.. _DELTA:
DELTA: DEep Learning Transfer using Feature Map with Attention
-------------------------------------------------------------------------------------
.. autoclass:: talib.finetune.delta.BehavioralRegularization
.. autoclass:: talib.finetune.delta.AttentionBehavioralRegularization
.. autoclass:: talib.finetune.delta.IntermediateLayerGetter
.. _LWF:
LWF: Learning without Forgetting
------------------------------------------
.. autoclass:: talib.finetune.lwf.Classifier
.. _CoTuning:
Co-Tuning
------------------------------------------
.. autoclass:: talib.finetune.co_tuning.CoTuningLoss
.. autoclass:: talib.finetune.co_tuning.Relationship
.. _StochNorm:
StochNorm: Stochastic Normalization
------------------------------------------
.. autoclass:: talib.finetune.stochnorm.StochNorm1d
.. autoclass:: talib.finetune.stochnorm.StochNorm2d
.. autoclass:: talib.finetune.stochnorm.StochNorm3d
.. autofunction:: talib.finetune.stochnorm.convert_model
Adaptive Fine-tuning
===========================================
.. _BiTuning:
Bi-Tuning
------------------------------------------
.. autoclass:: talib.finetune.bi_tuning.BiTuning
Rejecting Untransferable Information
===========================================
.. _BSS:
BSS: Batch Spectral Shrinkage
------------------------------------------
.. autoclass:: talib.finetune.bss.BatchSpectralShrinkage
================================================
FILE: domain_generalization/Transfer-Learning-Library/examples/domain_adaptation/image_classification/README.md
================================================
# Unsupervised Domain Adaptation for Image Classification
## Installation
Example scripts can deal with [WILDS datasets](https://wilds.stanford.edu/).
You should first install ``wilds`` before using these scripts.
```
pip install wilds
```
Example scripts also support all models in [PyTorch-Image-Models](https://github.com/rwightman/pytorch-image-models).
You also need to install timm to use PyTorch-Image-Models.
```
pip install timm
```
## Dataset
Following datasets can be downloaded automatically:
- [MNIST](http://yann.lecun.com/exdb/mnist/), [SVHN](http://ufldl.stanford.edu/housenumbers/), [USPS](https://www.csie.ntu.edu.tw/~cjlin/libsvmtools/datasets/multiclass.html#usps)
- [Office31](https://www.cc.gatech.edu/~judy/domainadapt/)
- [OfficeCaltech](https://www.cc.gatech.edu/~judy/domainadapt/)
- [OfficeHome](https://www.hemanthdv.org/officeHomeDataset.html)
- [VisDA2017](http://ai.bu.edu/visda-2017/)
- [DomainNet](http://ai.bu.edu/M3SDA/)
- [iwildcam (WILDS)](https://wilds.stanford.edu/datasets/)
- [camelyon17 (WILDS)](https://wilds.stanford.edu/datasets/)
- [fmow (WILDS)](https://wilds.stanford.edu/datasets/)
You need to prepare following datasets manually if you want to use them:
- [ImageNet](https://www.image-net.org/)
- [ImageNetR](https://github.com/hendrycks/imagenet-r)
- [ImageNet-Sketch](https://github.com/HaohanWang/ImageNet-Sketch)
and prepare them following [Documentation for ImageNetR](/common/vision/datasets/imagenet_r.py) and [ImageNet-Sketch](/common/vision/datasets/imagenet_sketch.py).
## Supported Methods
Supported methods include:
- [Domain Adversarial Neural Network (DANN)](https://arxiv.org/abs/1505.07818)
- [Deep Adaptation Network (DAN)](https://arxiv.org/pdf/1502.02791)
- [Joint Adaptation Network (JAN)](https://arxiv.org/abs/1605.06636)
- [Adversarial Discriminative Domain Adaptation (ADDA)](https://arxiv.org/pdf/1702.05464.pdf)
- [Conditional Domain Adversarial Network (CDAN)](https://arxiv.org/abs/1705.10667)
- [Maximum Classifier Discrepancy (MCD)](https://arxiv.org/abs/1712.02560)
- [Adaptive Feature Norm (AFN)](https://arxiv.org/pdf/1811.07456v2.pdf)
- [Batch Spectral Penalization (BSP)](http://ise.thss.tsinghua.edu.cn/~mlong/doc/batch-spectral-penalization-icml19.pdf)
- [Margin Disparity Discrepancy (MDD)](https://arxiv.org/abs/1904.05801)
- [Minimum Class Confusion (MCC)](https://arxiv.org/abs/1912.03699)
## Experiment and Results
The shell files give the script to reproduce the [benchmarks](/docs/dalib/benchmarks/image_classification.rst) with specified hyper-parameters.
For example, if you want to train DANN on Office31, use the following script
```shell script
# Train a DANN on Office-31 Amazon -> Webcam task using ResNet 50.
# Assume you have put the datasets under the path `data/office-31`,
# or you are glad to download the datasets automatically from the Internet to this path
CUDA_VISIBLE_DEVICES=0 python dann.py data/office31 -d Office31 -s A -t W -a resnet50 --epochs 20 --seed 1 --log logs/dann/Office31_A2W
```
For more information please refer to [Get Started](/docs/get_started/quickstart.rst) for help.
## TODO
Support methods: AdaBN/TransNorm, CycleGAN, CyCADA
## Citation
If you use these methods in your research, please consider citing.
```
@inproceedings{DANN,
author = {Ganin, Yaroslav and Lempitsky, Victor},
Booktitle = {ICML},
Title = {Unsupervised domain adaptation by backpropagation},
Year = {2015}
}
@inproceedings{DAN,
author = {Mingsheng Long and
Yue Cao and
Jianmin Wang and
Michael I. Jordan},
title = {Learning Transferable Features with Deep Adaptation Networks},
booktitle = {ICML},
year = {2015},
}
@inproceedings{JAN,
title={Deep transfer learning with joint adaptation networks},
author={Long, Mingsheng and Zhu, Han and Wang, Jianmin and Jordan, Michael I},
booktitle={ICML},
year={2017},
}
@inproceedings{ADDA,
title={Adversarial discriminative domain adaptation},
author={Tzeng, Eric and Hoffman, Judy and Saenko, Kate and Darrell, Trevor},
booktitle={CVPR},
year={2017}
}
@inproceedings{CDAN,
author = {Mingsheng Long and
Zhangjie Cao and
Jianmin Wang and
Michael I. Jordan},
title = {Conditional Adversarial Domain Adaptation},
booktitle = {NeurIPS},
year = {2018}
}
@inproceedings{MCD,
title={Maximum classifier discrepancy for unsupervised domain adaptation},
author={Saito, Kuniaki and Watanabe, Kohei and Ushiku, Yoshitaka and Harada, Tatsuya},
booktitle={CVPR},
year={2018}
}
@InProceedings{AFN,
author = {Xu, Ruijia and Li, Guanbin and Yang, Jihan and Lin, Liang},
title = {Larger Norm More Transferable: An Adaptive Feature Norm Approach for Unsupervised Domain Adaptation},
booktitle = {ICCV},
year = {2019}
}
@inproceedings{MDD,
title={Bridging theory and algorithm for domain adaptation},
author={Zhang, Yuchen and Liu, Tianle and Long, Mingsheng and Jordan, Michael},
booktitle={ICML},
year={2019},
}
@inproceedings{BSP,
title={Transferability vs. discriminability: Batch spectral penalization for adversarial domain adaptation},
author={Chen, Xinyang and Wang, Sinan and Long, Mingsheng and Wang, Jianmin},
booktitle={ICML},
year={2019},
}
@inproceedings{MCC,
author = {Ying Jin and
Ximei Wang and
Mingsheng Long and
Jianmin Wang},
title = {Less Confusion More Transferable: Minimum Class Confusion for Versatile
Domain Adaptation},
year={2020},
booktitle={ECCV},
}
```
================================================
FILE: domain_generalization/Transfer-Learning-Library/examples/domain_adaptation/image_classification/adda.py
================================================
"""
@author: Baixu Chen
@contact: cbx_99_hasta@outlook.com
Note: Our implementation is different from ADDA paper in several respects. We do not use separate networks for
source and target domain, nor fix classifier head. Besides, we do not adopt asymmetric objective loss function
of the feature extractor. We achieve promising results on digits datasets (reported by ADDA paper).
But on other benchmarks, ADDA-grl may achieve better results.
"""
import random
import time
import warnings
import sys
import argparse
import shutil
import os.path as osp
import torch
import torch.nn as nn
import torch.backends.cudnn as cudnn
from torch.optim import SGD
from torch.optim.lr_scheduler import LambdaLR
from torch.utils.data import DataLoader
import torch.nn.functional as F
sys.path.append('../../..')
from dalib.modules.domain_discriminator import DomainDiscriminator
from dalib.adaptation.adda import ImageClassifier, DomainAdversarialLoss
from dalib.modules.gl import WarmStartGradientLayer
from dalib.translation.cyclegan.util import set_requires_grad
from common.utils.data import ForeverDataIterator
from common.utils.metric import accuracy, binary_accuracy
from common.utils.meter import AverageMeter, ProgressMeter
from common.utils.logger import CompleteLogger
from common.utils.analysis import collect_feature, tsne, a_distance
sys.path.append('.')
import utils
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
def main(args: argparse.Namespace):
logger = CompleteLogger(args.log, args.phase)
print(args)
if args.seed is not None:
random.seed(args.seed)
torch.manual_seed(args.seed)
cudnn.deterministic = True
warnings.warn('You have chosen to seed training. '
'This will turn on the CUDNN deterministic setting, '
'which can slow down your training considerably! '
'You may see unexpected behavior when restarting '
'from checkpoints.')
cudnn.benchmark = True
# Data loading code
train_transform = utils.get_train_transform(args.train_resizing, random_horizontal_flip=not args.no_hflip,
random_color_jitter=False, resize_size=args.resize_size,
norm_mean=args.norm_mean, norm_std=args.norm_std)
val_transform = utils.get_val_transform(args.val_resizing, resize_size=args.resize_size,
norm_mean=args.norm_mean, norm_std=args.norm_std)
print("train_transform: ", train_transform)
print("val_transform: ", val_transform)
train_source_dataset, train_target_dataset, val_dataset, test_dataset, num_classes, args.class_names = \
utils.get_dataset(args.data, args.root, args.source, args.target, train_transform, val_transform)
train_source_loader = DataLoader(train_source_dataset, batch_size=args.batch_size,
shuffle=True, num_workers=args.workers, drop_last=True)
train_target_loader = DataLoader(train_target_dataset, batch_size=args.batch_size,
shuffle=True, num_workers=args.workers, drop_last=True)
val_loader = DataLoader(val_dataset, batch_size=args.batch_size, shuffle=False, num_workers=args.workers)
test_loader = DataLoader(test_dataset, batch_size=args.batch_size, shuffle=False, num_workers=args.workers)
train_source_iter = ForeverDataIterator(train_source_loader)
train_target_iter = ForeverDataIterator(train_target_loader)
# create model
print("=> using model '{}'".format(args.arch))
backbone = utils.get_model(args.arch, pretrain=not args.scratch)
pool_layer = nn.Identity() if args.no_pool else None
classifier = ImageClassifier(backbone, num_classes, bottleneck_dim=args.bottleneck_dim,
pool_layer=pool_layer, finetune=not args.scratch).to(device)
domain_discri = DomainDiscriminator(in_feature=classifier.features_dim, hidden_size=1024).to(device)
# define loss function
domain_adv = DomainAdversarialLoss().to(device)
gl = WarmStartGradientLayer(alpha=1., lo=0., hi=1., max_iters=1000, auto_step=True)
# define optimizer and lr scheduler
optimizer = SGD(classifier.get_parameters(), args.lr, momentum=args.momentum, weight_decay=args.weight_decay,
nesterov=True)
optimizer_d = SGD(domain_discri.get_parameters(), args.lr_d, momentum=args.momentum, weight_decay=args.weight_decay,
nesterov=True)
lr_scheduler = LambdaLR(optimizer, lambda x: args.lr * (1. + args.lr_gamma * float(x)) ** (-args.lr_decay))
lr_scheduler_d = LambdaLR(optimizer_d, lambda x: args.lr_d * (1. + args.lr_gamma * float(x)) ** (-args.lr_decay))
# resume from the best checkpoint
if args.phase != 'train':
checkpoint = torch.load(logger.get_checkpoint_path('best'), map_location='cpu')
classifier.load_state_dict(checkpoint)
# analysis the model
if args.phase == 'analysis':
# extract features from both domains
feature_extractor = nn.Sequential(classifier.backbone, classifier.pool_layer, classifier.bottleneck).to(device)
source_feature = collect_feature(train_source_loader, feature_extractor, device)
target_feature = collect_feature(train_target_loader, feature_extractor, device)
# plot t-SNE
tSNE_filename = osp.join(logger.visualize_directory, 'TSNE.pdf')
tsne.visualize(source_feature, target_feature, tSNE_filename)
print("Saving t-SNE to", tSNE_filename)
# calculate A-distance, which is a measure for distribution discrepancy
A_distance = a_distance.calculate(source_feature, target_feature, device)
print("A-distance =", A_distance)
return
if args.phase == 'test':
acc1 = utils.validate(test_loader, classifier, args, device)
print(acc1)
return
# start training
best_acc1 = 0.
for epoch in range(args.epochs):
print("lr classifier:", lr_scheduler.get_lr())
print("lr discriminator:", lr_scheduler_d.get_lr())
# train for one epoch
train(train_source_iter, train_target_iter, classifier, domain_discri, domain_adv, gl, optimizer,
lr_scheduler, optimizer_d, lr_scheduler_d, epoch, args)
# evaluate on validation set
acc1 = utils.validate(val_loader, classifier, args, device)
# remember best acc@1 and save checkpoint
torch.save(classifier.state_dict(), logger.get_checkpoint_path('latest'))
if acc1 > best_acc1:
shutil.copy(logger.get_checkpoint_path('latest'), logger.get_checkpoint_path('best'))
best_acc1 = max(acc1, best_acc1)
print("best_acc1 = {:3.1f}".format(best_acc1))
# evaluate on test set
classifier.load_state_dict(torch.load(logger.get_checkpoint_path('best')))
acc1 = utils.validate(test_loader, classifier, args, device)
print("test_acc1 = {:3.1f}".format(acc1))
logger.close()
def train(train_source_iter: ForeverDataIterator, train_target_iter: ForeverDataIterator, model: ImageClassifier,
domain_discri: DomainDiscriminator, domain_adv: DomainAdversarialLoss, gl,
optimizer: SGD, lr_scheduler: LambdaLR, optimizer_d: SGD, lr_scheduler_d: LambdaLR,
epoch: int, args: argparse.Namespace):
batch_time = AverageMeter('Time', ':5.2f')
data_time = AverageMeter('Data', ':5.2f')
losses_s = AverageMeter('Cls Loss', ':6.2f')
losses_transfer = AverageMeter('Transfer Loss', ':6.2f')
losses_discriminator = AverageMeter('Discriminator Loss', ':6.2f')
cls_accs = AverageMeter('Cls Acc', ':3.1f')
domain_accs = AverageMeter('Domain Acc', ':3.1f')
progress = ProgressMeter(
args.iters_per_epoch,
[batch_time, data_time, losses_s, losses_transfer, losses_discriminator, cls_accs, domain_accs],
prefix="Epoch: [{}]".format(epoch))
end = time.time()
for i in range(args.iters_per_epoch):
x_s, labels_s = next(train_source_iter)
x_t, _ = next(train_target_iter)
x_s = x_s.to(device)
x_t = x_t.to(device)
labels_s = labels_s.to(device)
# measure data loading time
data_time.update(time.time() - end)
# Step 1: Train the classifier, freeze the discriminator
model.train()
domain_discri.eval()
set_requires_grad(model, True)
set_requires_grad(domain_discri, False)
x = torch.cat((x_s, x_t), dim=0)
y, f = model(x)
y_s, y_t = y.chunk(2, dim=0)
loss_s = F.cross_entropy(y_s, labels_s)
# adversarial training to fool the discriminator
d = domain_discri(gl(f))
d_s, d_t = d.chunk(2, dim=0)
loss_transfer = 0.5 * (domain_adv(d_s, 'target') + domain_adv(d_t, 'source'))
optimizer.zero_grad()
(loss_s + loss_transfer * args.trade_off).backward()
optimizer.step()
lr_scheduler.step()
# Step 2: Train the discriminator
model.eval()
domain_discri.train()
set_requires_grad(model, False)
set_requires_grad(domain_discri, True)
d = domain_discri(f.detach())
d_s, d_t = d.chunk(2, dim=0)
loss_discriminator = 0.5 * (domain_adv(d_s, 'source') + domain_adv(d_t, 'target'))
optimizer_d.zero_grad()
loss_discriminator.backward()
optimizer_d.step()
lr_scheduler_d.step()
losses_s.update(loss_s.item(), x_s.size(0))
losses_transfer.update(loss_transfer.item(), x_s.size(0))
losses_discriminator.update(loss_discriminator.item(), x_s.size(0))
cls_acc = accuracy(y_s, labels_s)[0]
cls_accs.update(cls_acc.item(), x_s.size(0))
domain_acc = 0.5 * (binary_accuracy(d_s, torch.ones_like(d_s)) + binary_accuracy(d_t, torch.zeros_like(d_t)))
domain_accs.update(domain_acc.item(), x_s.size(0))
# measure elapsed time
batch_time.update(time.time() - end)
end = time.time()
if i % args.print_freq == 0:
progress.display(i)
if __name__ == '__main__':
parser = argparse.ArgumentParser(description='ADDA for Unsupervised Domain Adaptation')
# dataset parameters
parser.add_argument('root', metavar='DIR',
help='root path of dataset')
parser.add_argument('-d', '--data', metavar='DATA', default='Office31', choices=utils.get_dataset_names(),
help='dataset: ' + ' | '.join(utils.get_dataset_names()) +
' (default: Office31)')
parser.add_argument('-s', '--source', help='source domain(s)', nargs='+')
parser.add_argument('-t', '--target', help='target domain(s)', nargs='+')
parser.add_argument('--train-resizing', type=str, default='default')
parser.add_argument('--val-resizing', type=str, default='default')
parser.add_argument('--resize-size', type=int, default=224,
help='the image size after resizing')
parser.add_argument('--no-hflip', action='store_true',
help='no random horizontal flipping during training')
parser.add_argument('--norm-mean', type=float, nargs='+',
default=(0.485, 0.456, 0.406), help='normalization mean')
parser.add_argument('--norm-std', type=float, nargs='+',
default=(0.229, 0.224, 0.225), help='normalization std')
# model parameters
parser.add_argument('-a', '--arch', metavar='ARCH', default='resnet18',
choices=utils.get_model_names(),
help='backbone architecture: ' +
' | '.join(utils.get_model_names()) +
' (default: resnet18)')
parser.add_argument('--bottleneck-dim', default=256, type=int,
help='Dimension of bottleneck')
parser.add_argument('--no-pool', action='store_true',
help='no pool layer after the feature extractor.')
parser.add_argument('--scratch', action='store_true', help='whether train from scratch.')
parser.add_argument('--trade-off', default=0.1, type=float,
help='the trade-off hyper-parameter for transfer loss')
# training parameters
parser.add_argument('-b', '--batch-size', default=32, type=int,
metavar='N',
help='mini-batch size (default: 32)')
parser.add_argument('--lr', '--learning-rate', default=0.01, type=float,
metavar='LR', help='initial learning rate of the classifier', dest='lr')
parser.add_argument('--lr-d', default=0.01, type=float,
help='initial learning rate of the domain discriminator')
parser.add_argument('--lr-gamma', default=0.001, type=float, help='parameter for lr scheduler')
parser.add_argument('--lr-decay', default=0.75, type=float, help='parameter for lr scheduler')
parser.add_argument('--momentum', default=0.9, type=float, metavar='M',
help='momentum')
parser.add_argument('--wd', '--weight-decay', default=1e-3, type=float,
metavar='W', help='weight decay (default: 1e-3)',
dest='weight_decay')
parser.add_argument('-j', '--workers', default=2, type=int, metavar='N',
help='number of data loading workers (default: 4)')
parser.add_argument('--epochs', default=20, type=int, metavar='N',
help='number of total epochs to run')
parser.add_argument('-i', '--iters-per-epoch', default=1000, type=int,
help='Number of iterations per epoch')
parser.add_argument('-p', '--print-freq', default=100, type=int,
metavar='N', help='print frequency (default: 100)')
parser.add_argument('--seed', default=None, type=int,
help='seed for initializing training. ')
parser.add_argument('--per-class-eval', action='store_true',
help='whether output per-class accuracy during evaluation')
parser.add_argument("--log", type=str, default='adda',
help="Where to save logs, checkpoints and debugging images.")
parser.add_argument("--phase", type=str, default='train', choices=['train', 'test', 'analysis'],
help="When phase is 'test', only test the model."
"When phase is 'analysis', only analysis the model.")
args = parser.parse_args()
main(args)
================================================
FILE: domain_generalization/Transfer-Learning-Library/examples/domain_adaptation/image_classification/adda.sh
================================================
#!/usr/bin/env bash
# ResNet50, Office31, Single Source
CUDA_VISIBLE_DEVICES=0 python adda.py data/office31 -d Office31 -s A -t W -a resnet50 --epochs 20 --seed 1 --log logs/adda/Office31_A2W
CUDA_VISIBLE_DEVICES=0 python adda.py data/office31 -d Office31 -s D -t W -a resnet50 --epochs 20 --seed 1 --log logs/adda/Office31_D2W
CUDA_VISIBLE_DEVICES=0 python adda.py data/office31 -d Office31 -s W -t D -a resnet50 --epochs 20 --seed 1 --log logs/adda/Office31_W2D
CUDA_VISIBLE_DEVICES=0 python adda.py data/office31 -d Office31 -s A -t D -a resnet50 --epochs 20 --seed 1 --log logs/adda/Office31_A2D
CUDA_VISIBLE_DEVICES=0 python adda.py data/office31 -d Office31 -s D -t A -a resnet50 --epochs 20 --seed 1 --log logs/adda/Office31_D2A
CUDA_VISIBLE_DEVICES=0 python adda.py data/office31 -d Office31 -s W -t A -a resnet50 --epochs 20 --seed 1 --log logs/adda/Office31_W2A
# ResNet50, Office-Home, Single Source
CUDA_VISIBLE_DEVICES=0 python adda.py data/office-home -d OfficeHome -s Ar -t Cl -a resnet50 --epochs 30 --seed 0 --log logs/adda/OfficeHome_Ar2Cl
CUDA_VISIBLE_DEVICES=0 python adda.py data/office-home -d OfficeHome -s Ar -t Pr -a resnet50 --epochs 30 --seed 0 --log logs/adda/OfficeHome_Ar2Pr
CUDA_VISIBLE_DEVICES=0 python adda.py data/office-home -d OfficeHome -s Ar -t Rw -a resnet50 --epochs 30 --seed 0 --log logs/adda/OfficeHome_Ar2Rw
CUDA_VISIBLE_DEVICES=0 python adda.py data/office-home -d OfficeHome -s Cl -t Ar -a resnet50 --epochs 30 --seed 0 --log logs/adda/OfficeHome_Cl2Ar
CUDA_VISIBLE_DEVICES=0 python adda.py data/office-home -d OfficeHome -s Cl -t Pr -a resnet50 --epochs 30 --seed 0 --log logs/adda/OfficeHome_Cl2Pr
CUDA_VISIBLE_DEVICES=0 python adda.py data/office-home -d OfficeHome -s Cl -t Rw -a resnet50 --epochs 30 --seed 0 --log logs/adda/OfficeHome_Cl2Rw
CUDA_VISIBLE_DEVICES=0 python adda.py data/office-home -d OfficeHome -s Pr -t Ar -a resnet50 --epochs 30 --seed 0 --log logs/adda/OfficeHome_Pr2Ar
CUDA_VISIBLE_DEVICES=0 python adda.py data/office-home -d OfficeHome -s Pr -t Cl -a resnet50 --epochs 30 --seed 0 --log logs/adda/OfficeHome_Pr2Cl
CUDA_VISIBLE_DEVICES=0 python adda.py data/office-home -d OfficeHome -s Pr -t Rw -a resnet50 --epochs 30 --seed 0 --log logs/adda/OfficeHome_Pr2Rw
CUDA_VISIBLE_DEVICES=0 python adda.py data/office-home -d OfficeHome -s Rw -t Ar -a resnet50 --epochs 30 --seed 0 --log logs/adda/OfficeHome_Rw2Ar
CUDA_VISIBLE_DEVICES=0 python adda.py data/office-home -d OfficeHome -s Rw -t Cl -a resnet50 --epochs 30 --seed 0 --log logs/adda/OfficeHome_Rw2Cl
CUDA_VISIBLE_DEVICES=0 python adda.py data/office-home -d OfficeHome -s Rw -t Pr -a resnet50 --epochs 30 --seed 0 --log logs/adda/OfficeHome_Rw2Pr
# ResNet101, VisDA-2017, Single Source
CUDA_VISIBLE_DEVICES=0 python adda.py data/visda-2017 -d VisDA2017 -s Synthetic -t Real -a resnet101 \
--epochs 30 --seed 0 --per-class-eval --train-resizing cen.crop --log logs/adda/VisDA2017
# ResNet101, DomainNet, Single Source
CUDA_VISIBLE_DEVICES=0 python adda.py data/domainnet -d DomainNet -s c -t p -a resnet101 --bottleneck-dim 1024 --epochs 30 -i 2500 -p 500 --seed 0 --log logs/adda/DomainNet_c2p
CUDA_VISIBLE_DEVICES=0 python adda.py data/domainnet -d DomainNet -s c -t r -a resnet101 --bottleneck-dim 1024 --epochs 30 -i 2500 -p 500 --seed 0 --log logs/adda/DomainNet_c2r
CUDA_VISIBLE_DEVICES=0 python adda.py data/domainnet -d DomainNet -s c -t s -a resnet101 --bottleneck-dim 1024 --epochs 30 -i 2500 -p 500 --seed 0 --log logs/adda/DomainNet_c2s
CUDA_VISIBLE_DEVICES=0 python adda.py data/domainnet -d DomainNet -s p -t c -a resnet101 --bottleneck-dim 1024 --epochs 30 -i 2500 -p 500 --seed 0 --log logs/adda/DomainNet_p2c
CUDA_VISIBLE_DEVICES=0 python adda.py data/domainnet -d DomainNet -s p -t r -a resnet101 --bottleneck-dim 1024 --epochs 30 -i 2500 -p 500 --seed 0 --log logs/adda/DomainNet_p2r
CUDA_VISIBLE_DEVICES=0 python adda.py data/domainnet -d DomainNet -s p -t s -a resnet101 --bottleneck-dim 1024 --epochs 30 -i 2500 -p 500 --seed 0 --log logs/adda/DomainNet_p2s
CUDA_VISIBLE_DEVICES=0 python adda.py data/domainnet -d DomainNet -s r -t c -a resnet101 --bottleneck-dim 1024 --epochs 30 -i 2500 -p 500 --seed 0 --log logs/adda/DomainNet_r2c
CUDA_VISIBLE_DEVICES=0 python adda.py data/domainnet -d DomainNet -s r -t p -a resnet101 --bottleneck-dim 1024 --epochs 30 -i 2500 -p 500 --seed 0 --log logs/adda/DomainNet_r2p
CUDA_VISIBLE_DEVICES=0 python adda.py data/domainnet -d DomainNet -s r -t s -a resnet101 --bottleneck-dim 1024 --epochs 30 -i 2500 -p 500 --seed 0 --log logs/adda/DomainNet_r2s
CUDA_VISIBLE_DEVICES=0 python adda.py data/domainnet -d DomainNet -s s -t c -a resnet101 --bottleneck-dim 1024 --epochs 30 -i 2500 -p 500 --seed 0 --log logs/adda/DomainNet_s2c
CUDA_VISIBLE_DEVICES=0 python adda.py data/domainnet -d DomainNet -s s -t p -a resnet101 --bottleneck-dim 1024 --epochs 30 -i 2500 -p 500 --seed 0 --log logs/adda/DomainNet_s2p
CUDA_VISIBLE_DEVICES=0 python adda.py data/domainnet -d DomainNet -s s -t r -a resnet101 --bottleneck-dim 1024 --epochs 30 -i 2500 -p 500 --seed 0 --log logs/adda/DomainNet_s2r
# ResNet50, ImageNet200 -> ImageNetR
CUDA_VISIBLE_DEVICES=0 python adda.py data/ImageNetR -d ImageNetR -s IN -t INR -a resnet50 --epochs 30 -i 2500 -p 500 --seed 0 --log logs/adda/ImageNet_IN2INR
# ig_resnext101_32x8d, ImageNet -> ImageNetSketch
CUDA_VISIBLE_DEVICES=0 python adda.py data/imagenet-sketch -d ImageNetSketch -s IN -t sketch -a ig_resnext101_32x8d --epochs 30 -i 2500 -p 500 --bottleneck-dim 1024 --log logs/dann_ig_resnext101_32x8d/ImageNet_IN2sketch
# Vision Transformer, Office-Home, Single Source
CUDA_VISIBLE_DEVICES=0 python adda.py data/office-home -d OfficeHome -s Ar -t Cl -a vit_base_patch16_224 --epochs 30 --seed 0 -b 24 --no-pool --log logs/dann_vit/OfficeHome_Ar2Cl
CUDA_VISIBLE_DEVICES=0 python adda.py data/office-home -d OfficeHome -s Ar -t Pr -a vit_base_patch16_224 --epochs 30 --seed 0 -b 24 --no-pool --log logs/dann_vit/OfficeHome_Ar2Pr
CUDA_VISIBLE_DEVICES=0 python adda.py data/office-home -d OfficeHome -s Ar -t Rw -a vit_base_patch16_224 --epochs 30 --seed 0 -b 24 --no-pool --log logs/dann_vit/OfficeHome_Ar2Rw
CUDA_VISIBLE_DEVICES=0 python adda.py data/office-home -d OfficeHome -s Cl -t Ar -a vit_base_patch16_224 --epochs 30 --seed 0 -b 24 --no-pool --log logs/dann_vit/OfficeHome_Cl2Ar
CUDA_VISIBLE_DEVICES=0 python adda.py data/office-home -d OfficeHome -s Cl -t Pr -a vit_base_patch16_224 --epochs 30 --seed 0 -b 24 --no-pool --log logs/dann_vit/OfficeHome_Cl2Pr
CUDA_VISIBLE_DEVICES=0 python adda.py data/office-home -d OfficeHome -s Cl -t Rw -a vit_base_patch16_224 --epochs 30 --seed 0 -b 24 --no-pool --log logs/dann_vit/OfficeHome_Cl2Rw
CUDA_VISIBLE_DEVICES=0 python adda.py data/office-home -d OfficeHome -s Pr -t Ar -a vit_base_patch16_224 --epochs 30 --seed 0 -b 24 --no-pool --log logs/dann_vit/OfficeHome_Pr2Ar
CUDA_VISIBLE_DEVICES=0 python adda.py data/office-home -d OfficeHome -s Pr -t Cl -a vit_base_patch16_224 --epochs 30 --seed 0 -b 24 --no-pool --log logs/dann_vit/OfficeHome_Pr2Cl
CUDA_VISIBLE_DEVICES=0 python adda.py data/office-home -d OfficeHome -s Pr -t Rw -a vit_base_patch16_224 --epochs 30 --seed 0 -b 24 --no-pool --log logs/dann_vit/OfficeHome_Pr2Rw
CUDA_VISIBLE_DEVICES=0 python adda.py data/office-home -d OfficeHome -s Rw -t Ar -a vit_base_patch16_224 --epochs 30 --seed 0 -b 24 --no-pool --log logs/dann_vit/OfficeHome_Rw2Ar
CUDA_VISIBLE_DEVICES=0 python adda.py data/office-home -d OfficeHome -s Rw -t Cl -a vit_base_patch16_224 --epochs 30 --seed 0 -b 24 --no-pool --log logs/dann_vit/OfficeHome_Rw2Cl
CUDA_VISIBLE_DEVICES=0 python adda.py data/office-home -d OfficeHome -s Rw -t Pr -a vit_base_patch16_224 --epochs 30 --seed 0 -b 24 --no-pool --log logs/dann_vit/OfficeHome_Rw2Pr
# ResNet50, Office-Home, Multi Source
CUDA_VISIBLE_DEVICES=0 python adda.py data/office-home -d OfficeHome -s Cl Pr Rw -t Ar -a resnet50 --epochs 30 --seed 0 --log logs/adda/OfficeHome_:2Ar
CUDA_VISIBLE_DEVICES=0 python adda.py data/office-home -d OfficeHome -s Ar Pr Rw -t Cl -a resnet50 --epochs 30 --seed 0 --log logs/adda/OfficeHome_:2Cl
CUDA_VISIBLE_DEVICES=0 python adda.py data/office-home -d OfficeHome -s Ar Cl Rw -t Pr -a resnet50 --epochs 30 --seed 0 --log logs/adda/OfficeHome_:2Pr
CUDA_VISIBLE_DEVICES=0 python adda.py data/office-home -d OfficeHome -s Ar Cl Pr -t Rw -a resnet50 --epochs 30 --seed 0 --log logs/adda/OfficeHome_:2Rw
# ResNet101, DomainNet, Multi Source
CUDA_VISIBLE_DEVICES=0 python adda.py data/domainnet -d DomainNet -s i p q r s -t c -a resnet101 --bottleneck-dim 1024 --epochs 40 -i 5000 -p 500 --seed 0 --log logs/adda/DomainNet_:2c
CUDA_VISIBLE_DEVICES=0 python adda.py data/domainnet -d DomainNet -s c p q r s -t i -a resnet101 --bottleneck-dim 1024 --epochs 40 -i 5000 -p 500 --seed 0 --log logs/adda/DomainNet_:2i
CUDA_VISIBLE_DEVICES=0 python adda.py data/domainnet -d DomainNet -s c i q r s -t p -a resnet101 --bottleneck-dim 1024 --epochs 40 -i 5000 -p 500 --seed 0 --log logs/adda/DomainNet_:2p
CUDA_VISIBLE_DEVICES=0 python adda.py data/domainnet -d DomainNet -s c i p r s -t q -a resnet101 --bottleneck-dim 1024 --epochs 40 -i 5000 -p 500 --seed 0 --log logs/adda/DomainNet_:2q
CUDA_VISIBLE_DEVICES=0 python adda.py data/domainnet -d DomainNet -s c i p q s -t r -a resnet101 --bottleneck-dim 1024 --epochs 40 -i 5000 -p 500 --seed 0 --log logs/adda/DomainNet_:2r
CUDA_VISIBLE_DEVICES=0 python adda.py data/domainnet -d DomainNet -s c i p q r -t s -a resnet101 --bottleneck-dim 1024 --epochs 40 -i 5000 -p 500 --seed 0 --log logs/adda/DomainNet_:2s
# ResNet50, Wilds Dataset
CUDA_VISIBLE_DEVICES=0 python adda.py data/wilds -d iwildcam -a resnet50 --epochs 30 -i 1000 --seed 0 --log logs/adda/iwildcam
# DenseNet121, Wilds Dataset
CUDA_VISIBLE_DEVICES=0 python adda.py data/wilds -d camelyon17 --train-resizing 'res.' --val-resizing 'res.' --resize-size 96 \
-a densenet121 --scratch --epochs 10 -i 1000 --lr 0.01 --seed 0 --log logs/adda/camelyon17
CUDA_VISIBLE_DEVICES=0 python adda.py data/wilds -d fmow --train-resizing 'res.' --val-resizing 'res.' \
-a densenet121 --epochs 10 -i 1000 --lr 0.01 --seed 0 --log logs/adda/fmow
# Digits
CUDA_VISIBLE_DEVICES=0 python adda.py data/digits -d Digits -s MNIST -t USPS --train-resizing 'res.' --val-resizing 'res.' \
--resize-size 28 --no-hflip --norm-mean 0.5 --norm-std 0.5 -a lenet --no-pool --lr 0.01 -b 128 -i 2500 --scratch --seed 0 --log logs/adda/MNIST2USPS
CUDA_VISIBLE_DEVICES=0 python adda.py data/digits -d Digits -s USPS -t MNIST --train-resizing 'res.' --val-resizing 'res.' \
--resize-size 28 --no-hflip --norm-mean 0.5 --norm-std 0.5 -a lenet --no-pool --lr 0.1 -b 128 -i 2500 --scratch --seed 0 --log logs/adda/USPS2MNIST
CUDA_VISIBLE_DEVICES=3 python adda.py data/digits -d Digits -s SVHNRGB -t MNISTRGB --train-resizing 'res.' --val-resizing 'res.' \
--resize-size 32 --no-hflip --norm-mean 0.5 0.5 0.5 --norm-std 0.5 0.5 0.5 -a dtn --no-pool --trade-off 0.3 --lr 0.03 --lr-d 0.03 -b 128 -i 2500 --scratch --seed 0 --log logs/adda/SVHN2MNIST
================================================
FILE: domain_generalization/Transfer-Learning-Library/examples/domain_adaptation/image_classification/addagrl.py
================================================
"""
@author: Baixu Chen
@contact: cbx_99_hasta@outlook.com
Note: ADDA with gradient reverse layer
"""
import random
import time
import warnings
import sys
import copy
import argparse
import shutil
import os.path as osp
import torch
import torch.nn as nn
import torch.backends.cudnn as cudnn
from torch.optim import SGD
from torch.optim.lr_scheduler import LambdaLR
from torch.utils.data import DataLoader
sys.path.append('../../..')
from dalib.modules.domain_discriminator import DomainDiscriminator
from dalib.adaptation.adda import ImageClassifier
from dalib.adaptation.dann import DomainAdversarialLoss
from dalib.translation.cyclegan.util import set_requires_grad
from dalib.modules.grl import WarmStartGradientReverseLayer
from common.utils.data import ForeverDataIterator
from common.utils.meter import AverageMeter, ProgressMeter
from common.utils.logger import CompleteLogger
from common.utils.analysis import collect_feature, tsne, a_distance
sys.path.append('../classification')
import utils
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
def main(args: argparse.Namespace):
logger = CompleteLogger(args.log, args.phase)
print(args)
if args.seed is not None:
random.seed(args.seed)
torch.manual_seed(args.seed)
cudnn.deterministic = True
warnings.warn('You have chosen to seed training. '
'This will turn on the CUDNN deterministic setting, '
'which can slow down your training considerably! '
'You may see unexpected behavior when restarting '
'from checkpoints.')
cudnn.benchmark = True
# Data loading code
train_transform = utils.get_train_transform(args.train_resizing, random_horizontal_flip=not args.no_hflip,
random_color_jitter=False, resize_size=args.resize_size,
norm_mean=args.norm_mean, norm_std=args.norm_std)
val_transform = utils.get_val_transform(args.val_resizing, resize_size=args.resize_size,
norm_mean=args.norm_mean, norm_std=args.norm_std)
print("train_transform: ", train_transform)
print("val_transform: ", val_transform)
train_source_dataset, train_target_dataset, val_dataset, test_dataset, num_classes, args.class_names = \
utils.get_dataset(args.data, args.root, args.source, args.target, train_transform, val_transform)
train_source_loader = DataLoader(train_source_dataset, batch_size=args.batch_size,
shuffle=True, num_workers=args.workers, drop_last=True)
train_target_loader = DataLoader(train_target_dataset, batch_size=args.batch_size,
shuffle=True, num_workers=args.workers, drop_last=True)
val_loader = DataLoader(val_dataset, batch_size=args.batch_size, shuffle=False, num_workers=args.workers)
test_loader = DataLoader(test_dataset, batch_size=args.batch_size, shuffle=False, num_workers=args.workers)
train_source_iter = ForeverDataIterator(train_source_loader)
train_target_iter = ForeverDataIterator(train_target_loader)
# create model
print("=> using model '{}'".format(args.arch))
backbone = utils.get_model(args.arch, pretrain=not args.scratch)
pool_layer = nn.Identity() if args.no_pool else None
source_classifier = ImageClassifier(backbone, num_classes, bottleneck_dim=args.bottleneck_dim,
pool_layer=pool_layer, finetune=not args.scratch).to(device)
if args.phase == 'train' and args.pretrain is None:
# first pretrain the classifier wish source data
print("Pretraining the model on source domain.")
args.pretrain = logger.get_checkpoint_path('pretrain')
pretrain_model = ImageClassifier(backbone, num_classes, bottleneck_dim=args.bottleneck_dim,
pool_layer=pool_layer, finetune=not args.scratch).to(device)
pretrain_optimizer = SGD(pretrain_model.get_parameters(), args.pretrain_lr, momentum=args.momentum,
weight_decay=args.weight_decay, nesterov=True)
pretrain_lr_scheduler = LambdaLR(pretrain_optimizer,
lambda x: args.pretrain_lr * (1. + args.lr_gamma * float(x)) ** (
-args.lr_decay))
# start pretraining
for epoch in range(args.pretrain_epochs):
print("lr:", pretrain_lr_scheduler.get_lr())
# pretrain for one epoch
utils.pretrain(train_source_iter, pretrain_model, pretrain_optimizer, pretrain_lr_scheduler, epoch, args,
device)
# validate to show pretrain process
utils.validate(val_loader, pretrain_model, args, device)
torch.save(pretrain_model.state_dict(), args.pretrain)
print("Pretraining process is done.")
checkpoint = torch.load(args.pretrain, map_location='cpu')
source_classifier.load_state_dict(checkpoint)
target_classifier = copy.deepcopy(source_classifier)
# freeze source classifier
set_requires_grad(source_classifier, False)
source_classifier.freeze_bn()
domain_discri = DomainDiscriminator(in_feature=source_classifier.features_dim, hidden_size=1024).to(device)
# define loss function
grl = WarmStartGradientReverseLayer(alpha=1., lo=0., hi=2., max_iters=1000, auto_step=True)
domain_adv = DomainAdversarialLoss(domain_discri, grl=grl).to(device)
# define optimizer and lr scheduler
# note that we only optimize target feature extractor
optimizer = SGD(target_classifier.get_parameters(optimize_head=False) + domain_discri.get_parameters(), args.lr,
momentum=args.momentum, weight_decay=args.weight_decay, nesterov=True)
lr_scheduler = LambdaLR(optimizer, lambda x: args.lr * (1. + args.lr_gamma * float(x)) ** (-args.lr_decay))
# resume from the best checkpoint
if args.phase != 'train':
checkpoint = torch.load(logger.get_checkpoint_path('best'), map_location='cpu')
target_classifier.load_state_dict(checkpoint)
# analysis the model
if args.phase == 'analysis':
# extract features from both domains
feature_extractor = nn.Sequential(target_classifier.backbone, target_classifier.pool_layer,
target_classifier.bottleneck).to(device)
source_feature = collect_feature(train_source_loader, feature_extractor, device)
target_feature = collect_feature(train_target_loader, feature_extractor, device)
# plot t-SNE
tSNE_filename = osp.join(logger.visualize_directory, 'TSNE.pdf')
tsne.visualize(source_feature, target_feature, tSNE_filename)
print("Saving t-SNE to", tSNE_filename)
# calculate A-distance, which is a measure for distribution discrepancy
A_distance = a_distance.calculate(source_feature, target_feature, device)
print("A-distance =", A_distance)
return
if args.phase == 'test':
acc1 = utils.validate(test_loader, target_classifier, args, device)
print(acc1)
return
# start training
best_acc1 = 0.
for epoch in range(args.epochs):
print(lr_scheduler.get_lr())
# train for one epoch
train(train_source_iter, train_target_iter, source_classifier, target_classifier, domain_adv,
optimizer, lr_scheduler, epoch, args)
# evaluate on validation set
acc1 = utils.validate(val_loader, target_classifier, args, device)
# remember best acc@1 and save checkpoint
torch.save(target_classifier.state_dict(), logger.get_checkpoint_path('latest'))
if acc1 > best_acc1:
shutil.copy(logger.get_checkpoint_path('latest'), logger.get_checkpoint_path('best'))
best_acc1 = max(acc1, best_acc1)
print("best_acc1 = {:3.1f}".format(best_acc1))
# evaluate on test set
target_classifier.load_state_dict(torch.load(logger.get_checkpoint_path('best')))
acc1 = utils.validate(test_loader, target_classifier, args, device)
print("test_acc1 = {:3.1f}".format(acc1))
logger.close()
def train(train_source_iter: ForeverDataIterator, train_target_iter: ForeverDataIterator,
source_model: ImageClassifier, target_model: ImageClassifier, domain_adv: DomainAdversarialLoss,
optimizer: SGD, lr_scheduler: LambdaLR, epoch: int, args: argparse.Namespace):
batch_time = AverageMeter('Time', ':5.2f')
data_time = AverageMeter('Data', ':5.2f')
losses_transfer = AverageMeter('Transfer Loss', ':6.2f')
domain_accs = AverageMeter('Domain Acc', ':3.1f')
progress = ProgressMeter(
args.iters_per_epoch,
[batch_time, data_time, losses_transfer, domain_accs],
prefix="Epoch: [{}]".format(epoch))
# switch to train mode
target_model.train()
domain_adv.train()
end = time.time()
for i in range(args.iters_per_epoch):
x_s, _ = next(train_source_iter)
x_t, _ = next(train_target_iter)
x_s = x_s.to(device)
x_t = x_t.to(device)
# measure data loading time
data_time.update(time.time() - end)
_, f_s = source_model(x_s)
_, f_t = target_model(x_t)
loss_transfer = domain_adv(f_s, f_t)
# Compute gradient and do SGD step
optimizer.zero_grad()
loss_transfer.backward()
optimizer.step()
lr_scheduler.step()
losses_transfer.update(loss_transfer.item(), x_s.size(0))
domain_acc = domain_adv.domain_discriminator_accuracy
domain_accs.update(domain_acc.item(), x_s.size(0))
# measure elapsed time
batch_time.update(time.time() - end)
end = time.time()
if i % args.print_freq == 0:
progress.display(i)
if __name__ == '__main__':
parser = argparse.ArgumentParser(description='ADDA for Unsupervised Domain Adaptation')
# dataset parameters
parser.add_argument('root', metavar='DIR',
help='root path of dataset')
parser.add_argument('-d', '--data', metavar='DATA', default='Office31', choices=utils.get_dataset_names(),
help='dataset: ' + ' | '.join(utils.get_dataset_names()) +
' (default: Office31)')
parser.add_argument('-s', '--source', help='source domain(s)', nargs='+')
parser.add_argument('-t', '--target', help='target domain(s)', nargs='+')
parser.add_argument('--train-resizing', type=str, default='default')
parser.add_argument('--val-resizing', type=str, default='default')
parser.add_argument('--resize-size', type=int, default=224,
help='the image size after resizing')
parser.add_argument('--no-hflip', action='store_true',
help='no random horizontal flipping during training')
parser.add_argument('--norm-mean', type=float, nargs='+',
default=(0.485, 0.456, 0.406), help='normalization mean')
parser.add_argument('--norm-std', type=float, nargs='+',
default=(0.229, 0.224, 0.225), help='normalization std')
# model parameters
parser.add_argument('-a', '--arch', metavar='ARCH', default='resnet18',
choices=utils.get_model_names(),
help='backbone architecture: ' +
' | '.join(utils.get_model_names()) +
' (default: resnet18)')
parser.add_argument('--pretrain', type=str, default=None,
help='pretrain checkpoint for classification model')
parser.add_argument('--bottleneck-dim', default=256, type=int,
help='Dimension of bottleneck')
parser.add_argument('--no-pool', action='store_true',
help='no pool layer after the feature extractor.')
parser.add_argument('--scratch', action='store_true', help='whether train from scratch.')
# training parameters
parser.add_argument('-b', '--batch-size', default=32, type=int,
metavar='N',
help='mini-batch size (default: 32)')
parser.add_argument('--lr', '--learning-rate', default=0.001, type=float,
metavar='LR', help='initial learning rate of the classifier', dest='lr')
parser.add_argument('--pretrain-lr', default=0.001, type=float, help='initial pretrain learning rate')
parser.add_argument('--lr-gamma', default=0.0003, type=float, help='parameter for lr scheduler')
parser.add_argument('--lr-decay', default=0.75, type=float, help='parameter for lr scheduler')
parser.add_argument('--momentum', default=0.9, type=float, metavar='M',
help='momentum')
parser.add_argument('--wd', '--weight-decay', default=1e-3, type=float,
metavar='W', help='weight decay (default: 1e-3)',
dest='weight_decay')
parser.add_argument('-j', '--workers', default=2, type=int, metavar='N',
help='number of data loading workers (default: 4)')
parser.add_argument('--epochs', default=20, type=int, metavar='N',
help='number of total epochs to run')
parser.add_argument('--pretrain-epochs', default=3, type=int, metavar='N',
help='number of total epochs(pretrain) to run')
parser.add_argument('-i', '--iters-per-epoch', default=1000, type=int,
help='Number of iterations per epoch')
parser.add_argument('-p', '--print-freq', default=100, type=int,
metavar='N', help='print frequency (default: 100)')
parser.add_argument('--seed', default=None, type=int,
help='seed for initializing training. ')
parser.add_argument('--per-class-eval', action='store_true',
help='whether output per-class accuracy during evaluation')
parser.add_argument("--log", type=str, default='addagrl',
help="Where to save logs, checkpoints and debugging images.")
parser.add_argument("--phase", type=str, default='train', choices=['train', 'test', 'analysis'],
help="When phase is 'test', only test the model."
"When phase is 'analysis', only analysis the model.")
args = parser.parse_args()
main(args)
================================================
FILE: domain_generalization/Transfer-Learning-Library/examples/domain_adaptation/image_classification/addagrl.sh
================================================
#!/usr/bin/env bash
# ResNet50, Office31, Single Source
CUDA_VISIBLE_DEVICES=0 python addagrl.py data/office31 -d Office31 -s A -t W -a resnet50 --epochs 20 --seed 1 --log logs/addagrl/Office31_A2W
CUDA_VISIBLE_DEVICES=0 python addagrl.py data/office31 -d Office31 -s D -t W -a resnet50 --epochs 20 --seed 1 --log logs/addagrl/Office31_D2W
CUDA_VISIBLE_DEVICES=0 python addagrl.py data/office31 -d Office31 -s W -t D -a resnet50 --epochs 20 --seed 1 --log logs/addagrl/Office31_W2D
CUDA_VISIBLE_DEVICES=0 python addagrl.py data/office31 -d Office31 -s A -t D -a resnet50 --epochs 20 --seed 1 --log logs/addagrl/Office31_A2D
CUDA_VISIBLE_DEVICES=0 python addagrl.py data/office31 -d Office31 -s D -t A -a resnet50 --epochs 20 --seed 1 --log logs/addagrl/Office31_D2A
CUDA_VISIBLE_DEVICES=0 python addagrl.py data/office31 -d Office31 -s W -t A -a resnet50 --epochs 20 --seed 1 --log logs/addagrl/Office31_W2A
# ResNet50, Office-Home, Single Source
CUDA_VISIBLE_DEVICES=0 python addagrl.py data/office-home -d OfficeHome -s Ar -t Cl -a resnet50 --epochs 30 --seed 0 --log logs/addagrl/OfficeHome_Ar2Cl
CUDA_VISIBLE_DEVICES=0 python addagrl.py data/office-home -d OfficeHome -s Ar -t Pr -a resnet50 --epochs 30 --seed 0 --log logs/addagrl/OfficeHome_Ar2Pr
CUDA_VISIBLE_DEVICES=0 python addagrl.py data/office-home -d OfficeHome -s Ar -t Rw -a resnet50 --epochs 30 --seed 0 --log logs/addagrl/OfficeHome_Ar2Rw
CUDA_VISIBLE_DEVICES=0 python addagrl.py data/office-home -d OfficeHome -s Cl -t Ar -a resnet50 --epochs 30 --seed 0 --log logs/addagrl/OfficeHome_Cl2Ar
CUDA_VISIBLE_DEVICES=0 python addagrl.py data/office-home -d OfficeHome -s Cl -t Pr -a resnet50 --epochs 30 --seed 0 --log logs/addagrl/OfficeHome_Cl2Pr
CUDA_VISIBLE_DEVICES=0 python addagrl.py data/office-home -d OfficeHome -s Cl -t Rw -a resnet50 --epochs 30 --seed 0 --log logs/addagrl/OfficeHome_Cl2Rw
CUDA_VISIBLE_DEVICES=0 python addagrl.py data/office-home -d OfficeHome -s Pr -t Ar -a resnet50 --epochs 30 --seed 0 --log logs/addagrl/OfficeHome_Pr2Ar
CUDA_VISIBLE_DEVICES=0 python addagrl.py data/office-home -d OfficeHome -s Pr -t Cl -a resnet50 --epochs 30 --seed 0 --log logs/addagrl/OfficeHome_Pr2Cl
CUDA_VISIBLE_DEVICES=0 python addagrl.py data/office-home -d OfficeHome -s Pr -t Rw -a resnet50 --epochs 30 --seed 0 --log logs/addagrl/OfficeHome_Pr2Rw
CUDA_VISIBLE_DEVICES=0 python addagrl.py data/office-home -d OfficeHome -s Rw -t Ar -a resnet50 --epochs 30 --seed 0 --log logs/addagrl/OfficeHome_Rw2Ar
CUDA_VISIBLE_DEVICES=0 python addagrl.py data/office-home -d OfficeHome -s Rw -t Cl -a resnet50 --epochs 30 --seed 0 --log logs/addagrl/OfficeHome_Rw2Cl
CUDA_VISIBLE_DEVICES=0 python addagrl.py data/office-home -d OfficeHome -s Rw -t Pr -a resnet50 --epochs 30 --seed 0 --log logs/addagrl/OfficeHome_Rw2Pr
# ResNet101, VisDA-2017, Single Source
CUDA_VISIBLE_DEVICES=0 python addagrl.py data/visda-2017 -d VisDA2017 -s Synthetic -t Real -a resnet101 \
--epochs 30 --seed 0 --per-class-eval --train-resizing cen.crop --log logs/addagrl/VisDA2017
# ResNet101, DomainNet, Single Source
CUDA_VISIBLE_DEVICES=0 python addagrl.py data/domainnet -d DomainNet -s c -t p -a resnet101 --bottleneck-dim 1024 --epochs 30 -i 2500 -p 500 --seed 0 --log logs/addagrl/DomainNet_c2p
CUDA_VISIBLE_DEVICES=0 python addagrl.py data/domainnet -d DomainNet -s c -t r -a resnet101 --bottleneck-dim 1024 --epochs 30 -i 2500 -p 500 --seed 0 --log logs/addagrl/DomainNet_c2r
CUDA_VISIBLE_DEVICES=0 python addagrl.py data/domainnet -d DomainNet -s c -t s -a resnet101 --bottleneck-dim 1024 --epochs 30 -i 2500 -p 500 --seed 0 --log logs/addagrl/DomainNet_c2s
CUDA_VISIBLE_DEVICES=0 python addagrl.py data/domainnet -d DomainNet -s p -t c -a resnet101 --bottleneck-dim 1024 --epochs 30 -i 2500 -p 500 --seed 0 --log logs/addagrl/DomainNet_p2c
CUDA_VISIBLE_DEVICES=0 python addagrl.py data/domainnet -d DomainNet -s p -t r -a resnet101 --bottleneck-dim 1024 --epochs 30 -i 2500 -p 500 --seed 0 --log logs/addagrl/DomainNet_p2r
CUDA_VISIBLE_DEVICES=0 python addagrl.py data/domainnet -d DomainNet -s p -t s -a resnet101 --bottleneck-dim 1024 --epochs 30 -i 2500 -p 500 --seed 0 --log logs/addagrl/DomainNet_p2s
CUDA_VISIBLE_DEVICES=0 python addagrl.py data/domainnet -d DomainNet -s r -t c -a resnet101 --bottleneck-dim 1024 --epochs 30 -i 2500 -p 500 --seed 0 --log logs/addagrl/DomainNet_r2c
CUDA_VISIBLE_DEVICES=0 python addagrl.py data/domainnet -d DomainNet -s r -t p -a resnet101 --bottleneck-dim 1024 --epochs 30 -i 2500 -p 500 --seed 0 --log logs/addagrl/DomainNet_r2p
CUDA_VISIBLE_DEVICES=0 python addagrl.py data/domainnet -d DomainNet -s r -t s -a resnet101 --bottleneck-dim 1024 --epochs 30 -i 2500 -p 500 --seed 0 --log logs/addagrl/DomainNet_r2s
CUDA_VISIBLE_DEVICES=0 python addagrl.py data/domainnet -d DomainNet -s s -t c -a resnet101 --bottleneck-dim 1024 --epochs 30 -i 2500 -p 500 --seed 0 --log logs/addagrl/DomainNet_s2c
CUDA_VISIBLE_DEVICES=0 python addagrl.py data/domainnet -d DomainNet -s s -t p -a resnet101 --bottleneck-dim 1024 --epochs 30 -i 2500 -p 500 --seed 0 --log logs/addagrl/DomainNet_s2p
CUDA_VISIBLE_DEVICES=0 python addagrl.py data/domainnet -d DomainNet -s s -t r -a resnet101 --bottleneck-dim 1024 --epochs 30 -i 2500 -p 500 --seed 0 --log logs/addagrl/DomainNet_s2r
# ResNet50, ImageNet200 -> ImageNetR
CUDA_VISIBLE_DEVICES=0 python addagrl.py data/ImageNetR -d ImageNetR -s IN -t INR -a resnet50 --epochs 30 -i 2500 -p 500 --seed 0 --log logs/addagrl/ImageNet_IN2INR
# ig_resnext101_32x8d, ImageNet -> ImageNetSketch
CUDA_VISIBLE_DEVICES=0 python addagrl.py data/imagenet-sketch -d ImageNetSketch -s IN -t sketch -a ig_resnext101_32x8d --epochs 30 -i 2500 -p 500 --bottleneck-dim 1024 --log logs/dann_ig_resnext101_32x8d/ImageNet_IN2sketch
# Vision Transformer, Office-Home, Single Source
CUDA_VISIBLE_DEVICES=0 python addagrl.py data/office-home -d OfficeHome -s Ar -t Cl -a vit_base_patch16_224 --epochs 30 --seed 0 -b 24 --no-pool --log logs/dann_vit/OfficeHome_Ar2Cl
CUDA_VISIBLE_DEVICES=0 python addagrl.py data/office-home -d OfficeHome -s Ar -t Pr -a vit_base_patch16_224 --epochs 30 --seed 0 -b 24 --no-pool --log logs/dann_vit/OfficeHome_Ar2Pr
CUDA_VISIBLE_DEVICES=0 python addagrl.py data/office-home -d OfficeHome -s Ar -t Rw -a vit_base_patch16_224 --epochs 30 --seed 0 -b 24 --no-pool --log logs/dann_vit/OfficeHome_Ar2Rw
CUDA_VISIBLE_DEVICES=0 python addagrl.py data/office-home -d OfficeHome -s Cl -t Ar -a vit_base_patch16_224 --epochs 30 --seed 0 -b 24 --no-pool --log logs/dann_vit/OfficeHome_Cl2Ar
CUDA_VISIBLE_DEVICES=0 python addagrl.py data/office-home -d OfficeHome -s Cl -t Pr -a vit_base_patch16_224 --epochs 30 --seed 0 -b 24 --no-pool --log logs/dann_vit/OfficeHome_Cl2Pr
CUDA_VISIBLE_DEVICES=0 python addagrl.py data/office-home -d OfficeHome -s Cl -t Rw -a vit_base_patch16_224 --epochs 30 --seed 0 -b 24 --no-pool --log logs/dann_vit/OfficeHome_Cl2Rw
CUDA_VISIBLE_DEVICES=0 python addagrl.py data/office-home -d OfficeHome -s Pr -t Ar -a vit_base_patch16_224 --epochs 30 --seed 0 -b 24 --no-pool --log logs/dann_vit/OfficeHome_Pr2Ar
CUDA_VISIBLE_DEVICES=0 python addagrl.py data/office-home -d OfficeHome -s Pr -t Cl -a vit_base_patch16_224 --epochs 30 --seed 0 -b 24 --no-pool --log logs/dann_vit/OfficeHome_Pr2Cl
CUDA_VISIBLE_DEVICES=0 python addagrl.py data/office-home -d OfficeHome -s Pr -t Rw -a vit_base_patch16_224 --epochs 30 --seed 0 -b 24 --no-pool --log logs/dann_vit/OfficeHome_Pr2Rw
CUDA_VISIBLE_DEVICES=0 python addagrl.py data/office-home -d OfficeHome -s Rw -t Ar -a vit_base_patch16_224 --epochs 30 --seed 0 -b 24 --no-pool --log logs/dann_vit/OfficeHome_Rw2Ar
CUDA_VISIBLE_DEVICES=0 python addagrl.py data/office-home -d OfficeHome -s Rw -t Cl -a vit_base_patch16_224 --epochs 30 --seed 0 -b 24 --no-pool --log logs/dann_vit/OfficeHome_Rw2Cl
CUDA_VISIBLE_DEVICES=0 python addagrl.py data/office-home -d OfficeHome -s Rw -t Pr -a vit_base_patch16_224 --epochs 30 --seed 0 -b 24 --no-pool --log logs/dann_vit/OfficeHome_Rw2Pr
# ResNet50, Office-Home, Multi Source
CUDA_VISIBLE_DEVICES=0 python addagrl.py data/office-home -d OfficeHome -s Cl Pr Rw -t Ar -a resnet50 --epochs 30 --seed 0 --log logs/addagrl/OfficeHome_:2Ar
CUDA_VISIBLE_DEVICES=0 python addagrl.py data/office-home -d OfficeHome -s Ar Pr Rw -t Cl -a resnet50 --epochs 30 --seed 0 --log logs/addagrl/OfficeHome_:2Cl
CUDA_VISIBLE_DEVICES=0 python addagrl.py data/office-home -d OfficeHome -s Ar Cl Rw -t Pr -a resnet50 --epochs 30 --seed 0 --log logs/addagrl/OfficeHome_:2Pr
CUDA_VISIBLE_DEVICES=0 python addagrl.py data/office-home -d OfficeHome -s Ar Cl Pr -t Rw -a resnet50 --epochs 30 --seed 0 --log logs/addagrl/OfficeHome_:2Rw
# ResNet101, DomainNet, Multi Source
CUDA_VISIBLE_DEVICES=0 python addagrl.py data/domainnet -d DomainNet -s i p q r s -t c -a resnet101 --bottleneck-dim 1024 --epochs 40 -i 5000 -p 500 --seed 0 --log logs/addagrl/DomainNet_:2c
CUDA_VISIBLE_DEVICES=0 python addagrl.py data/domainnet -d DomainNet -s c p q r s -t i -a resnet101 --bottleneck-dim 1024 --epochs 40 -i 5000 -p 500 --seed 0 --log logs/addagrl/DomainNet_:2i
CUDA_VISIBLE_DEVICES=0 python addagrl.py data/domainnet -d DomainNet -s c i q r s -t p -a resnet101 --bottleneck-dim 1024 --epochs 40 -i 5000 -p 500 --seed 0 --log logs/addagrl/DomainNet_:2p
CUDA_VISIBLE_DEVICES=0 python addagrl.py data/domainnet -d DomainNet -s c i p r s -t q -a resnet101 --bottleneck-dim 1024 --epochs 40 -i 5000 -p 500 --seed 0 --log logs/addagrl/DomainNet_:2q
CUDA_VISIBLE_DEVICES=0 python addagrl.py data/domainnet -d DomainNet -s c i p q s -t r -a resnet101 --bottleneck-dim 1024 --epochs 40 -i 5000 -p 500 --seed 0 --log logs/addagrl/DomainNet_:2r
CUDA_VISIBLE_DEVICES=0 python addagrl.py data/domainnet -d DomainNet -s c i p q r -t s -a resnet101 --bottleneck-dim 1024 --epochs 40 -i 5000 -p 500 --seed 0 --log logs/addagrl/DomainNet_:2s
# ResNet50, Wilds Dataset
CUDA_VISIBLE_DEVICES=0 python addagrl.py data/wilds -d iwildcam -a resnet50 --epochs 30 -i 1000 --seed 0 --log logs/addagrl/iwildcam
# DenseNet121, Wilds Dataset
CUDA_VISIBLE_DEVICES=0 python addagrl.py data/wilds -d camelyon17 --train-resizing 'res.' --val-resizing 'res.' --resize-size 96 \
-a densenet121 --scratch --epochs 10 -i 1000 --lr 0.01 --seed 0 --log logs/addagrl/camelyon17
CUDA_VISIBLE_DEVICES=0 python addagrl.py data/wilds -d fmow --train-resizing 'res.' --val-resizing 'res.' \
-a densenet121 --epochs 10 -i 1000 --lr 0.01 --seed 0 --log logs/addagrl/fmow
# Digits
CUDA_VISIBLE_DEVICES=0 python addagrl.py data/digits -d Digits -s MNIST -t USPS --train-resizing 'res.' --val-resizing 'res.' \
--resize-size 28 --no-hflip --norm-mean 0.5 --norm-std 0.5 -a lenet --no-pool --lr 0.01 -b 128 -i 2500 --scratch --seed 0 --log logs/addagrl/MNIST2USPS
CUDA_VISIBLE_DEVICES=0 python addagrl.py data/digits -d Digits -s USPS -t MNIST --train-resizing 'res.' --val-resizing 'res.' \
--resize-size 28 --no-hflip --norm-mean 0.5 --norm-std 0.5 -a lenet --no-pool --lr 0.1 -b 128 -i 2500 --scratch --seed 0 --log logs/addagrl/USPS2MNIST
CUDA_VISIBLE_DEVICES=0 python addagrl.py data/digits -d Digits -s SVHNRGB -t MNISTRGB --train-resizing 'res.' --val-resizing 'res.' \
--resize-size 32 --no-hflip --norm-mean 0.5 0.5 0.5 --norm-std 0.5 0.5 0.5 -a dtn --no-pool --lr 0.03 --lr-d 0.03 -b 128 -i 2500 --scratch --seed 0 --log logs/addagrl/SVHN2MNIST
================================================
FILE: domain_generalization/Transfer-Learning-Library/examples/domain_adaptation/image_classification/afn.py
================================================
"""
@author: Baixu Chen
@contact: cbx_99_hasta@outlook.com
"""
import random
import time
import warnings
import sys
import argparse
import shutil
import os.path as osp
import torch
import torch.nn as nn
import torch.backends.cudnn as cudnn
from torch.optim import SGD
from torch.utils.data import DataLoader
import torch.nn.functional as F
sys.path.append('../../..')
from dalib.adaptation.afn import AdaptiveFeatureNorm, ImageClassifier
from dalib.modules.entropy import entropy
from common.utils.data import ForeverDataIterator
from common.utils.metric import accuracy
from common.utils.meter import AverageMeter, ProgressMeter
from common.utils.logger import CompleteLogger
from common.utils.analysis import collect_feature, tsne, a_distance
sys.path.append('.')
import utils
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
def main(args: argparse.Namespace):
logger = CompleteLogger(args.log, args.phase)
print(args)
if args.seed is not None:
random.seed(args.seed)
torch.manual_seed(args.seed)
cudnn.deterministic = True
warnings.warn('You have chosen to seed training. '
'This will turn on the CUDNN deterministic setting, '
'which can slow down your training considerably! '
'You may see unexpected behavior when restarting '
'from checkpoints.')
cudnn.benchmark = True
# Data loading code
train_transform = utils.get_train_transform(args.train_resizing, random_horizontal_flip=not args.no_hflip,
random_color_jitter=False, resize_size=args.resize_size,
norm_mean=args.norm_mean, norm_std=args.norm_std)
val_transform = utils.get_val_transform(args.val_resizing, resize_size=args.resize_size,
norm_mean=args.norm_mean, norm_std=args.norm_std)
print("train_transform: ", train_transform)
print("val_transform: ", val_transform)
train_source_dataset, train_target_dataset, val_dataset, test_dataset, num_classes, args.class_names = \
utils.get_dataset(args.data, args.root, args.source, args.target, train_transform, val_transform)
train_source_loader = DataLoader(train_source_dataset, batch_size=args.batch_size,
shuffle=True, num_workers=args.workers, drop_last=True)
train_target_loader = DataLoader(train_target_dataset, batch_size=args.batch_size,
shuffle=True, num_workers=args.workers, drop_last=True)
val_loader = DataLoader(val_dataset, batch_size=args.batch_size, shuffle=False, num_workers=args.workers)
test_loader = DataLoader(test_dataset, batch_size=args.batch_size, shuffle=False, num_workers=args.workers)
train_source_iter = ForeverDataIterator(train_source_loader)
train_target_iter = ForeverDataIterator(train_target_loader)
# create model
print("=> using model '{}'".format(args.arch))
backbone = utils.get_model(args.arch, pretrain=not args.scratch)
pool_layer = nn.Identity() if args.no_pool else None
classifier = ImageClassifier(backbone, num_classes, args.num_blocks,
bottleneck_dim=args.bottleneck_dim, dropout_p=args.dropout_p,
pool_layer=pool_layer, finetune=not args.scratch).to(device)
adaptive_feature_norm = AdaptiveFeatureNorm(args.delta).to(device)
# define optimizer
# the learning rate is fixed according to origin paper
optimizer = SGD(classifier.get_parameters(), args.lr, weight_decay=args.weight_decay)
# resume from the best checkpoint
if args.phase != 'train':
checkpoint = torch.load(logger.get_checkpoint_path('best'), map_location='cpu')
classifier.load_state_dict(checkpoint)
# analysis the model
if args.phase == 'analysis':
# extract features from both domains
feature_extractor = nn.Sequential(classifier.backbone, classifier.pool_layer, classifier.bottleneck).to(device)
source_feature = collect_feature(train_source_loader, feature_extractor, device)
target_feature = collect_feature(train_target_loader, feature_extractor, device)
# plot t-SNE
tSNE_filename = osp.join(logger.visualize_directory, 'TSNE.pdf')
tsne.visualize(source_feature, target_feature, tSNE_filename)
print("Saving t-SNE to", tSNE_filename)
# calculate A-distance, which is a measure for distribution discrepancy
A_distance = a_distance.calculate(source_feature, target_feature, device)
print("A-distance =", A_distance)
return
if args.phase == 'test':
acc1 = utils.validate(test_loader, classifier, args, device)
print(acc1)
return
# start training
best_acc1 = 0.
for epoch in range(args.epochs):
# train for one epoch
train(train_source_iter, train_target_iter, classifier, adaptive_feature_norm, optimizer, epoch, args)
# evaluate on validation set
acc1 = utils.validate(val_loader, classifier, args, device)
# remember best acc@1 and save checkpoint
torch.save(classifier.state_dict(), logger.get_checkpoint_path('latest'))
if acc1 > best_acc1:
shutil.copy(logger.get_checkpoint_path('latest'), logger.get_checkpoint_path('best'))
best_acc1 = max(acc1, best_acc1)
print("best_acc1 = {:3.1f}".format(best_acc1))
# evaluate on test set
classifier.load_state_dict(torch.load(logger.get_checkpoint_path('best')))
acc1 = utils.validate(test_loader, classifier, args, device)
print("test_acc1 = {:3.1f}".format(acc1))
logger.close()
def train(train_source_iter: ForeverDataIterator, train_target_iter: ForeverDataIterator, model: ImageClassifier,
adaptive_feature_norm: AdaptiveFeatureNorm, optimizer: SGD, epoch: int, args: argparse.Namespace):
batch_time = AverageMeter('Time', ':3.1f')
data_time = AverageMeter('Data', ':3.1f')
cls_losses = AverageMeter('Cls Loss', ':3.2f')
norm_losses = AverageMeter('Norm Loss', ':3.2f')
src_feature_norm = AverageMeter('Source Feature Norm', ':3.2f')
tgt_feature_norm = AverageMeter('Target Feature Norm', ':3.2f')
cls_accs = AverageMeter('Cls Acc', ':3.1f')
tgt_accs = AverageMeter('Tgt Acc', ':3.1f')
progress = ProgressMeter(
args.iters_per_epoch,
[batch_time, data_time, cls_losses, norm_losses, src_feature_norm, tgt_feature_norm, cls_accs, tgt_accs],
prefix="Epoch: [{}]".format(epoch))
# switch to train mode
model.train()
end = time.time()
for i in range(args.iters_per_epoch):
x_s, labels_s = next(train_source_iter)
x_t, labels_t = next(train_target_iter)
x_s = x_s.to(device)
x_t = x_t.to(device)
labels_s = labels_s.to(device)
labels_t = labels_t.to(device)
# measure data loading time
data_time.update(time.time() - end)
# compute output
y_s, f_s = model(x_s)
y_t, f_t = model(x_t)
# classification loss
cls_loss = F.cross_entropy(y_s, labels_s)
# norm loss
norm_loss = adaptive_feature_norm(f_s) + adaptive_feature_norm(f_t)
loss = cls_loss + norm_loss * args.trade_off_norm
# using entropy minimization
if args.trade_off_entropy:
y_t = F.softmax(y_t, dim=1)
entropy_loss = entropy(y_t, reduction='mean')
loss += entropy_loss * args.trade_off_entropy
# compute gradient and do SGD step
optimizer.zero_grad()
loss.backward()
optimizer.step()
# update statistics
cls_acc = accuracy(y_s, labels_s)[0]
tgt_acc = accuracy(y_t, labels_t)[0]
cls_losses.update(cls_loss.item(), x_s.size(0))
norm_losses.update(norm_loss.item(), x_s.size(0))
src_feature_norm.update(f_s.norm(p=2, dim=1).mean().item(), x_s.size(0))
tgt_feature_norm.update(f_t.norm(p=2, dim=1).mean().item(), x_s.size(0))
cls_accs.update(cls_acc.item(), x_s.size(0))
tgt_accs.update(tgt_acc.item(), x_s.size(0))
# measure elapsed time
batch_time.update(time.time() - end)
end = time.time()
if i % args.print_freq == 0:
progress.display(i)
if __name__ == '__main__':
parser = argparse.ArgumentParser(description='DANN for Unsupervised Domain Adaptation')
# dataset parameters
parser.add_argument('root', metavar='DIR',
help='root path of dataset')
parser.add_argument('-d', '--data', metavar='DATA', default='Office31', choices=utils.get_dataset_names(),
help='dataset: ' + ' | '.join(utils.get_dataset_names()) +
' (default: Office31)')
parser.add_argument('-s', '--source', help='source domain(s)', nargs='+')
parser.add_argument('-t', '--target', help='target domain(s)', nargs='+')
parser.add_argument('--train-resizing', type=str, default='ran.crop')
parser.add_argument('--val-resizing', type=str, default='default')
parser.add_argument('--resize-size', type=int, default=224,
help='the image size after resizing')
parser.add_argument('--no-hflip', action='store_true',
help='no random horizontal flipping during training')
parser.add_argument('--norm-mean', type=float, nargs='+',
default=(0.485, 0.456, 0.406), help='normalization mean')
parser.add_argument('--norm-std', type=float, nargs='+',
default=(0.229, 0.224, 0.225), help='normalization std')
# model parameters
parser.add_argument('-a', '--arch', metavar='ARCH', default='resnet18',
choices=utils.get_model_names(),
help='backbone architecture: ' +
' | '.join(utils.get_model_names()) +
' (default: resnet18)')
parser.add_argument('--no-pool', action='store_true',
help='no pool layer after the feature extractor.')
parser.add_argument('--scratch', action='store_true', help='whether train from scratch.')
parser.add_argument('-n', '--num-blocks', default=1, type=int, help='Number of basic blocks for classifier')
parser.add_argument('--bottleneck-dim', default=1000, type=int, help='Dimension of bottleneck')
parser.add_argument('--dropout-p', default=0.5, type=float,
help='Dropout probability')
# training parameters
parser.add_argument('-b', '--batch-size', default=32, type=int,
metavar='N',
help='mini-batch size (default: 32)')
parser.add_argument('--lr', '--learning-rate', default=0.001, type=float,
metavar='LR', help='initial learning rate', dest='lr')
parser.add_argument('--momentum', default=0.9, type=float, metavar='M',
help='momentum')
parser.add_argument('--wd', '--weight-decay', default=5e-4, type=float,
metavar='W', help='weight decay (default: 5e-4)',
dest='weight_decay')
parser.add_argument('--trade-off-norm', default=0.05, type=float,
help='the trade-off hyper-parameter for norm loss')
parser.add_argument('--trade-off-entropy', default=None, type=float,
help='the trade-off hyper-parameter for entropy loss')
parser.add_argument('-r', '--delta', default=1, type=float, help='Increment for L2 norm')
parser.add_argument('-j', '--workers', default=2, type=int, metavar='N',
help='number of data loading workers (default: 2)')
parser.add_argument('--epochs', default=20, type=int, metavar='N',
help='number of total epochs to run')
parser.add_argument('-i', '--iters-per-epoch', default=500, type=int,
help='Number of iterations per epoch')
parser.add_argument('-p', '--print-freq', default=100, type=int,
metavar='N', help='print frequency (default: 100)')
parser.add_argument('--seed', default=None, type=int,
help='seed for initializing training. ')
parser.add_argument('--per-class-eval', action='store_true',
help='whether output per-class accuracy during evaluation')
parser.add_argument("--log", type=str, default='afn',
help="Where to save logs, checkpoints and debugging images.")
parser.add_argument("--phase", type=str, default='train', choices=['train', 'test', 'analysis'],
help="When phase is 'test', only test the model."
"When phase is 'analysis', only analysis the model.")
args = parser.parse_args()
main(args)
================================================
FILE: domain_generalization/Transfer-Learning-Library/examples/domain_adaptation/image_classification/afn.sh
================================================
#!/usr/bin/env bash
# ResNet50, Office31, Single Source
CUDA_VISIBLE_DEVICES=0 python afn.py data/office31 -d Office31 -s A -t W -a resnet50 --trade-off-entropy 0.1 --epochs 20 --seed 1 --log logs/afn/Office31_A2W
CUDA_VISIBLE_DEVICES=0 python afn.py data/office31 -d Office31 -s D -t W -a resnet50 --trade-off-entropy 0.1 --epochs 20 --seed 1 --log logs/afn/Office31_D2W
CUDA_VISIBLE_DEVICES=0 python afn.py data/office31 -d Office31 -s W -t D -a resnet50 --trade-off-entropy 0.1 --epochs 20 --seed 1 --log logs/afn/Office31_W2D
CUDA_VISIBLE_DEVICES=0 python afn.py data/office31 -d Office31 -s A -t D -a resnet50 --trade-off-entropy 0.1 --epochs 20 --seed 1 --log logs/afn/Office31_A2D
CUDA_VISIBLE_DEVICES=0 python afn.py data/office31 -d Office31 -s D -t A -a resnet50 --trade-off-entropy 0.1 --epochs 20 --seed 1 --log logs/afn/Office31_D2A
CUDA_VISIBLE_DEVICES=0 python afn.py data/office31 -d Office31 -s W -t A -a resnet50 --trade-off-entropy 0.1 --epochs 20 --seed 1 --log logs/afn/Office31_W2A
# ResNet50, Office-Home, Single Source
CUDA_VISIBLE_DEVICES=0 python afn.py data/office-home -d OfficeHome -s Ar -t Cl -a resnet50 --epochs 20 --seed 0 --log logs/afn/OfficeHome_Ar2Cl
CUDA_VISIBLE_DEVICES=0 python afn.py data/office-home -d OfficeHome -s Ar -t Pr -a resnet50 --epochs 20 --seed 0 --log logs/afn/OfficeHome_Ar2Pr
CUDA_VISIBLE_DEVICES=0 python afn.py data/office-home -d OfficeHome -s Ar -t Rw -a resnet50 --epochs 20 --seed 0 --log logs/afn/OfficeHome_Ar2Rw
CUDA_VISIBLE_DEVICES=0 python afn.py data/office-home -d OfficeHome -s Cl -t Ar -a resnet50 --epochs 20 --seed 0 --log logs/afn/OfficeHome_Cl2Ar
CUDA_VISIBLE_DEVICES=0 python afn.py data/office-home -d OfficeHome -s Cl -t Pr -a resnet50 --epochs 20 --seed 0 --log logs/afn/OfficeHome_Cl2Pr
CUDA_VISIBLE_DEVICES=0 python afn.py data/office-home -d OfficeHome -s Cl -t Rw -a resnet50 --epochs 20 --seed 0 --log logs/afn/OfficeHome_Cl2Rw
CUDA_VISIBLE_DEVICES=0 python afn.py data/office-home -d OfficeHome -s Pr -t Ar -a resnet50 --epochs 20 --seed 0 --log logs/afn/OfficeHome_Pr2Ar
CUDA_VISIBLE_DEVICES=0 python afn.py data/office-home -d OfficeHome -s Pr -t Cl -a resnet50 --epochs 20 --seed 0 --log logs/afn/OfficeHome_Pr2Cl
CUDA_VISIBLE_DEVICES=0 python afn.py data/office-home -d OfficeHome -s Pr -t Rw -a resnet50 --epochs 20 --seed 0 --log logs/afn/OfficeHome_Pr2Rw
CUDA_VISIBLE_DEVICES=0 python afn.py data/office-home -d OfficeHome -s Rw -t Ar -a resnet50 --epochs 20 --seed 0 --log logs/afn/OfficeHome_Rw2Ar
CUDA_VISIBLE_DEVICES=0 python afn.py data/office-home -d OfficeHome -s Rw -t Cl -a resnet50 --epochs 20 --seed 0 --log logs/afn/OfficeHome_Rw2Cl
CUDA_VISIBLE_DEVICES=0 python afn.py data/office-home -d OfficeHome -s Rw -t Pr -a resnet50 --epochs 20 --seed 0 --log logs/afn/OfficeHome_Rw2Pr
# ResNet101, VisDA-2017, Single Source
CUDA_VISIBLE_DEVICES=0 python afn.py data/visda-2017 -d VisDA2017 -s Synthetic -t Real -a resnet101 -r 0.3 -b 36 \
--epochs 10 -i 1000 --seed 0 --per-class-eval --train-resizing cen.crop --log logs/afn/VisDA2017
# ResNet101, DomainNet, Single Source
CUDA_VISIBLE_DEVICES=0 python afn.py data/domainnet -d DomainNet -s c -t p -a resnet101 --bottleneck-dim 1024 --epochs 30 -i 2500 -p 500 --seed 0 --trade-off-norm 0.01 --lr 0.002 --log logs/afn/DomainNet_c2p
CUDA_VISIBLE_DEVICES=0 python afn.py data/domainnet -d DomainNet -s c -t r -a resnet101 --bottleneck-dim 1024 --epochs 30 -i 2500 -p 500 --seed 0 --trade-off-norm 0.01 --lr 0.002 --log logs/afn/DomainNet_c2r
CUDA_VISIBLE_DEVICES=0 python afn.py data/domainnet -d DomainNet -s c -t s -a resnet101 --bottleneck-dim 1024 --epochs 30 -i 2500 -p 500 --seed 0 --trade-off-norm 0.01 --lr 0.002 --log logs/afn/DomainNet_c2s
CUDA_VISIBLE_DEVICES=0 python afn.py data/domainnet -d DomainNet -s p -t c -a resnet101 --bottleneck-dim 1024 --epochs 30 -i 2500 -p 500 --seed 0 --trade-off-norm 0.01 --lr 0.002 --log logs/afn/DomainNet_p2c
CUDA_VISIBLE_DEVICES=0 python afn.py data/domainnet -d DomainNet -s p -t r -a resnet101 --bottleneck-dim 1024 --epochs 30 -i 2500 -p 500 --seed 0 --trade-off-norm 0.01 --lr 0.002 --log logs/afn/DomainNet_p2r
CUDA_VISIBLE_DEVICES=0 python afn.py data/domainnet -d DomainNet -s p -t s -a resnet101 --bottleneck-dim 1024 --epochs 30 -i 2500 -p 500 --seed 0 --trade-off-norm 0.01 --lr 0.002 --log logs/afn/DomainNet_p2s
CUDA_VISIBLE_DEVICES=0 python afn.py data/domainnet -d DomainNet -s r -t c -a resnet101 --bottleneck-dim 1024 --epochs 30 -i 2500 -p 500 --seed 0 --trade-off-norm 0.01 --lr 0.002 --log logs/afn/DomainNet_r2c
CUDA_VISIBLE_DEVICES=0 python afn.py data/domainnet -d DomainNet -s r -t p -a resnet101 --bottleneck-dim 1024 --epochs 30 -i 2500 -p 500 --seed 0 --trade-off-norm 0.01 --lr 0.002 --log logs/afn/DomainNet_r2p
CUDA_VISIBLE_DEVICES=0 python afn.py data/domainnet -d DomainNet -s r -t s -a resnet101 --bottleneck-dim 1024 --epochs 30 -i 2500 -p 500 --seed 0 --trade-off-norm 0.01 --lr 0.002 --log logs/afn/DomainNet_r2s
CUDA_VISIBLE_DEVICES=0 python afn.py data/domainnet -d DomainNet -s s -t c -a resnet101 --bottleneck-dim 1024 --epochs 30 -i 2500 -p 500 --seed 0 --trade-off-norm 0.01 --lr 0.002 --log logs/afn/DomainNet_s2c
CUDA_VISIBLE_DEVICES=0 python afn.py data/domainnet -d DomainNet -s s -t p -a resnet101 --bottleneck-dim 1024 --epochs 30 -i 2500 -p 500 --seed 0 --trade-off-norm 0.01 --lr 0.002 --log logs/afn/DomainNet_s2p
CUDA_VISIBLE_DEVICES=0 python afn.py data/domainnet -d DomainNet -s s -t r -a resnet101 --bottleneck-dim 1024 --epochs 30 -i 2500 -p 500 --seed 0 --trade-off-norm 0.01 --lr 0.002 --log logs/afn/DomainNet_s2r
# ResNet50, ImageNet200 -> ImageNetR
CUDA_VISIBLE_DEVICES=0 python afn.py data/ImageNetR -d ImageNetR -s IN -t INR -a resnet50 --epochs 20 -i 2500 --seed 0 --log logs/afn/ImageNet_IN2INR
# ig_resnext101_32x8d, ImageNet -> ImageNetSketch
CUDA_VISIBLE_DEVICES=0 python afn.py data/imagenet-sketch -d ImageNetSketch -s IN -t sketch -a ig_resnext101_32x8d --epochs 20 -i 2500 --seed 0 --log logs/afn_ig_resnext101_32x8d/ImageNet_IN2sketch
# Vision Transformer, Office-Home, Single Source
CUDA_VISIBLE_DEVICES=0 python afn.py data/office-home -d OfficeHome -s Ar -t Cl -a vit_base_patch16_224 --no-pool --epochs 30 --seed 0 -b 24 --log logs/afn_vit/OfficeHome_Ar2Cl
CUDA_VISIBLE_DEVICES=0 python afn.py data/office-home -d OfficeHome -s Ar -t Pr -a vit_base_patch16_224 --no-pool --epochs 30 --seed 0 -b 24 --log logs/afn_vit/OfficeHome_Ar2Pr
CUDA_VISIBLE_DEVICES=0 python afn.py data/office-home -d OfficeHome -s Ar -t Rw -a vit_base_patch16_224 --no-pool --epochs 30 --seed 0 -b 24 --log logs/afn_vit/OfficeHome_Ar2Rw
CUDA_VISIBLE_DEVICES=0 python afn.py data/office-home -d OfficeHome -s Cl -t Ar -a vit_base_patch16_224 --no-pool --epochs 30 --seed 0 -b 24 --log logs/afn_vit/OfficeHome_Cl2Ar
CUDA_VISIBLE_DEVICES=0 python afn.py data/office-home -d OfficeHome -s Cl -t Pr -a vit_base_patch16_224 --no-pool --epochs 30 --seed 0 -b 24 --log logs/afn_vit/OfficeHome_Cl2Pr
CUDA_VISIBLE_DEVICES=0 python afn.py data/office-home -d OfficeHome -s Cl -t Rw -a vit_base_patch16_224 --no-pool --epochs 30 --seed 0 -b 24 --log logs/afn_vit/OfficeHome_Cl2Rw
CUDA_VISIBLE_DEVICES=0 python afn.py data/office-home -d OfficeHome -s Pr -t Ar -a vit_base_patch16_224 --no-pool --epochs 30 --seed 0 -b 24 --log logs/afn_vit/OfficeHome_Pr2Ar
CUDA_VISIBLE_DEVICES=0 python afn.py data/office-home -d OfficeHome -s Pr -t Cl -a vit_base_patch16_224 --no-pool --epochs 30 --seed 0 -b 24 --log logs/afn_vit/OfficeHome_Pr2Cl
CUDA_VISIBLE_DEVICES=0 python afn.py data/office-home -d OfficeHome -s Pr -t Rw -a vit_base_patch16_224 --no-pool --epochs 30 --seed 0 -b 24 --log logs/afn_vit/OfficeHome_Pr2Rw
CUDA_VISIBLE_DEVICES=0 python afn.py data/office-home -d OfficeHome -s Rw -t Ar -a vit_base_patch16_224 --no-pool --epochs 30 --seed 0 -b 24 --log logs/afn_vit/OfficeHome_Rw2Ar
CUDA_VISIBLE_DEVICES=0 python afn.py data/office-home -d OfficeHome -s Rw -t Cl -a vit_base_patch16_224 --no-pool --epochs 30 --seed 0 -b 24 --log logs/afn_vit/OfficeHome_Rw2Cl
CUDA_VISIBLE_DEVICES=0 python afn.py data/office-home -d OfficeHome -s Rw -t Pr -a vit_base_patch16_224 --no-pool --epochs 30 --seed 0 -b 24 --log logs/afn_vit/OfficeHome_Rw2Pr
# ResNet50, Office-Home, Multi Source
CUDA_VISIBLE_DEVICES=0 python afn.py data/office-home -d OfficeHome -s Cl Pr Rw -t Ar -a resnet50 --epochs 30 --seed 0 --log logs/afn/OfficeHome_:2Ar
CUDA_VISIBLE_DEVICES=0 python afn.py data/office-home -d OfficeHome -s Ar Pr Rw -t Cl -a resnet50 --epochs 30 --seed 0 --log logs/afn/OfficeHome_:2Cl
CUDA_VISIBLE_DEVICES=0 python afn.py data/office-home -d OfficeHome -s Ar Cl Rw -t Pr -a resnet50 --epochs 30 --seed 0 --log logs/afn/OfficeHome_:2Pr
CUDA_VISIBLE_DEVICES=0 python afn.py data/office-home -d OfficeHome -s Ar Cl Pr -t Rw -a resnet50 --epochs 30 --seed 0 --log logs/afn/OfficeHome_:2Rw
# ResNet101, DomainNet, Multi Source
CUDA_VISIBLE_DEVICES=0 python afn.py data/domainnet -d DomainNet -s i p q r s -t c -a resnet101 --bottleneck-dim 1024 --epochs 40 -i 5000 -p 500 --seed 0 --log logs/afn/DomainNet_:2c
CUDA_VISIBLE_DEVICES=0 python afn.py data/domainnet -d DomainNet -s c p q r s -t i -a resnet101 --bottleneck-dim 1024 --epochs 40 -i 5000 -p 500 --seed 0 --log logs/afn/DomainNet_:2i
CUDA_VISIBLE_DEVICES=0 python afn.py data/domainnet -d DomainNet -s c i q r s -t p -a resnet101 --bottleneck-dim 1024 --epochs 40 -i 5000 -p 500 --seed 0 --log logs/afn/DomainNet_:2p
CUDA_VISIBLE_DEVICES=0 python afn.py data/domainnet -d DomainNet -s c i p r s -t q -a resnet101 --bottleneck-dim 1024 --epochs 40 -i 5000 -p 500 --seed 0 --log logs/afn/DomainNet_:2q
CUDA_VISIBLE_DEVICES=0 python afn.py data/domainnet -d DomainNet -s c i p q s -t r -a resnet101 --bottleneck-dim 1024 --epochs 40 -i 5000 -p 500 --seed 0 --log logs/afn/DomainNet_:2r
CUDA_VISIBLE_DEVICES=0 python afn.py data/domainnet -d DomainNet -s c i p q r -t s -a resnet101 --bottleneck-dim 1024 --epochs 40 -i 5000 -p 500 --seed 0 --log logs/afn/DomainNet_:2s
# ResNet50, Wilds Dataset
CUDA_VISIBLE_DEVICES=0 python afn.py data/wilds -d iwildcam -a resnet50 --epochs 30 -i 1000 --seed 0 --log logs/afn/iwildcam
# DenseNet121, Wilds Dataset
CUDA_VISIBLE_DEVICES=0 python afn.py data/wilds -d camelyon17 --train-resizing 'res.' --val-resizing 'res.' --resize-size 96 \
-a densenet121 --scratch --epochs 10 -i 1000 --lr 0.01 --seed 0 --log logs/afn/camelyon17
CUDA_VISIBLE_DEVICES=0 python afn.py data/wilds -d fmow --train-resizing 'res.' --val-resizing 'res.' \
-a densenet121 --epochs 10 -i 1000 --lr 0.01 --seed 0 --log logs/afn/fmow
# Digits
CUDA_VISIBLE_DEVICES=0 python afn.py data/digits -d Digits -s MNIST -t USPS --train-resizing 'res.' --val-resizing 'res.' \
--resize-size 28 --no-hflip --norm-mean 0.5 --norm-std 0.5 -a lenet --no-pool -r 0.3 --lr 0.01 --trade-off-entropy 0.03 -b 128 -i 2500 --scratch --seed 0 --log logs/afn/MNIST2USPS
CUDA_VISIBLE_DEVICES=0 python afn.py data/digits -d Digits -s USPS -t MNIST --train-resizing 'res.' --val-resizing 'res.' \
--resize-size 28 --no-hflip --norm-mean 0.5 --norm-std 0.5 -a lenet --no-pool -r 0.1 --lr 0.03 --trade-off-entropy 0.03 -b 128 -i 2500 --scratch --seed 0 --log logs/afn/USPS2MNIST
CUDA_VISIBLE_DEVICES=0 python afn.py data/digits -d Digits -s SVHNRGB -t MNISTRGB --train-resizing 'res.' --val-resizing 'res.' \
--resize-size 32 --no-hflip --norm-mean 0.5 0.5 0.5 --norm-std 0.5 0.5 0.5 -a dtn --no-pool -r 0.1 --lr 0.1 --trade-off-entropy 0.03 -b 128 -i 2500 --scratch --seed 0 --log logs/afn/SVHN2MNIST
================================================
FILE: domain_generalization/Transfer-Learning-Library/examples/domain_adaptation/image_classification/bsp.py
================================================
"""
@author: Baixu Chen
@contact: cbx_99_hasta@outlook.com
"""
import random
import time
import warnings
import sys
import argparse
import shutil
import os.path as osp
import torch
import torch.nn as nn
import torch.backends.cudnn as cudnn
from torch.optim import SGD
from torch.optim.lr_scheduler import LambdaLR
from torch.utils.data import DataLoader
import torch.nn.functional as F
sys.path.append('../../..')
from dalib.modules.domain_discriminator import DomainDiscriminator
from dalib.adaptation.dann import DomainAdversarialLoss
from dalib.adaptation.bsp import BatchSpectralPenalizationLoss, ImageClassifier
from common.utils.data import ForeverDataIterator
from common.utils.metric import accuracy
from common.utils.meter import AverageMeter, ProgressMeter
from common.utils.logger import CompleteLogger
from common.utils.analysis import collect_feature, tsne, a_distance
sys.path.append('.')
import utils
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
def main(args: argparse.Namespace):
logger = CompleteLogger(args.log, args.phase)
print(args)
if args.seed is not None:
random.seed(args.seed)
torch.manual_seed(args.seed)
cudnn.deterministic = True
warnings.warn('You have chosen to seed training. '
'This will turn on the CUDNN deterministic setting, '
'which can slow down your training considerably! '
'You may see unexpected behavior when restarting '
'from checkpoints.')
cudnn.benchmark = True
# Data loading code
train_transform = utils.get_train_transform(args.train_resizing, random_horizontal_flip=not args.no_hflip,
random_color_jitter=False, resize_size=args.resize_size,
norm_mean=args.norm_mean, norm_std=args.norm_std)
val_transform = utils.get_val_transform(args.val_resizing, resize_size=args.resize_size,
norm_mean=args.norm_mean, norm_std=args.norm_std)
print("train_transform: ", train_transform)
print("val_transform: ", val_transform)
train_source_dataset, train_target_dataset, val_dataset, test_dataset, num_classes, args.class_names = \
utils.get_dataset(args.data, args.root, args.source, args.target, train_transform, val_transform)
train_source_loader = DataLoader(train_source_dataset, batch_size=args.batch_size,
shuffle=True, num_workers=args.workers, drop_last=True)
train_target_loader = DataLoader(train_target_dataset, batch_size=args.batch_size,
shuffle=True, num_workers=args.workers, drop_last=True)
val_loader = DataLoader(val_dataset, batch_size=args.batch_size, shuffle=False, num_workers=args.workers)
test_loader = DataLoader(test_dataset, batch_size=args.batch_size, shuffle=False, num_workers=args.workers)
train_source_iter = ForeverDataIterator(train_source_loader)
train_target_iter = ForeverDataIterator(train_target_loader)
# create model
print("=> using model '{}'".format(args.arch))
backbone = utils.get_model(args.arch, pretrain=not args.scratch)
pool_layer = nn.Identity() if args.no_pool else None
classifier = ImageClassifier(backbone, num_classes, bottleneck_dim=args.bottleneck_dim,
pool_layer=pool_layer, finetune=not args.scratch).to(device)
domain_discri = DomainDiscriminator(in_feature=classifier.features_dim, hidden_size=1024).to(device)
# define optimizer and lr scheduler
optimizer = SGD(classifier.get_parameters() + domain_discri.get_parameters(),
args.lr, momentum=args.momentum, weight_decay=args.weight_decay, nesterov=True)
lr_scheduler = LambdaLR(optimizer, lambda x: args.lr * (1. + args.lr_gamma * float(x)) ** (-args.lr_decay))
# define loss function
domain_adv = DomainAdversarialLoss(domain_discri).to(device)
bsp_penalty = BatchSpectralPenalizationLoss().to(device)
# resume from the best checkpoint
if args.phase != 'train':
checkpoint = torch.load(logger.get_checkpoint_path('best'), map_location='cpu')
classifier.load_state_dict(checkpoint)
# analysis the model
if args.phase == 'analysis':
# extract features from both domains
feature_extractor = nn.Sequential(classifier.backbone, classifier.pool_layer, classifier.bottleneck).to(device)
source_feature = collect_feature(train_source_loader, feature_extractor, device)
target_feature = collect_feature(train_target_loader, feature_extractor, device)
# plot t-SNE
tSNE_filename = osp.join(logger.visualize_directory, 'TSNE.pdf')
tsne.visualize(source_feature, target_feature, tSNE_filename)
print("Saving t-SNE to", tSNE_filename)
# calculate A-distance, which is a measure for distribution discrepancy
A_distance = a_distance.calculate(source_feature, target_feature, device)
print("A-distance =", A_distance)
return
if args.phase == 'test':
acc1 = utils.validate(test_loader, classifier, args, device)
print(acc1)
return
if args.pretrain is None:
# first pretrain the classifier wish source data
print("Pretraining the model on source domain.")
args.pretrain = logger.get_checkpoint_path('pretrain')
pretrain_model = ImageClassifier(backbone, num_classes, bottleneck_dim=args.bottleneck_dim,
pool_layer=pool_layer, finetune=not args.scratch).to(device)
pretrain_optimizer = SGD(pretrain_model.get_parameters(), args.pretrain_lr, momentum=args.momentum,
weight_decay=args.weight_decay, nesterov=True)
pretrain_lr_scheduler = LambdaLR(pretrain_optimizer,
lambda x: args.pretrain_lr * (1. + args.lr_gamma * float(x)) ** (
-args.lr_decay))
# start pretraining
for epoch in range(args.pretrain_epochs):
print("lr:", pretrain_lr_scheduler.get_lr())
# pretrain for one epoch
utils.pretrain(train_source_iter, pretrain_model, pretrain_optimizer, pretrain_lr_scheduler, epoch, args,
device)
# validate to show pretrain process
utils.validate(val_loader, pretrain_model, args, device)
torch.save(pretrain_model.state_dict(), args.pretrain)
print("Pretraining process is done.")
checkpoint = torch.load(args.pretrain, map_location='cpu')
classifier.load_state_dict(checkpoint)
# start training
best_acc1 = 0.
for epoch in range(args.epochs):
print("lr:", lr_scheduler.get_last_lr()[0])
# train for one epoch
train(train_source_iter, train_target_iter, classifier, domain_adv, bsp_penalty, optimizer,
lr_scheduler, epoch, args)
# evaluate on validation set
acc1 = utils.validate(val_loader, classifier, args, device)
# remember best acc@1 and save checkpoint
torch.save(classifier.state_dict(), logger.get_checkpoint_path('latest'))
if acc1 > best_acc1:
shutil.copy(logger.get_checkpoint_path('latest'), logger.get_checkpoint_path('best'))
best_acc1 = max(acc1, best_acc1)
print("best_acc1 = {:3.1f}".format(best_acc1))
# evaluate on test set
classifier.load_state_dict(torch.load(logger.get_checkpoint_path('best')))
acc1 = utils.validate(test_loader, classifier, args, device)
print("test_acc1 = {:3.1f}".format(acc1))
logger.close()
def train(train_source_iter: ForeverDataIterator, train_target_iter: ForeverDataIterator,
model: ImageClassifier, domain_adv: DomainAdversarialLoss, bsp_penalty: BatchSpectralPenalizationLoss,
optimizer: SGD, lr_scheduler: LambdaLR, epoch: int, args: argparse.Namespace):
batch_time = AverageMeter('Time', ':5.2f')
data_time = AverageMeter('Data', ':5.2f')
losses = AverageMeter('Loss', ':6.2f')
cls_accs = AverageMeter('Cls Acc', ':3.1f')
domain_accs = AverageMeter('Domain Acc', ':3.1f')
progress = ProgressMeter(
args.iters_per_epoch,
[batch_time, data_time, losses, cls_accs, domain_accs],
prefix="Epoch: [{}]".format(epoch))
# switch to train mode
model.train()
domain_adv.train()
end = time.time()
for i in range(args.iters_per_epoch):
x_s, labels_s = next(train_source_iter)
x_t, _ = next(train_target_iter)
x_s = x_s.to(device)
x_t = x_t.to(device)
labels_s = labels_s.to(device)
# measure data loading time
data_time.update(time.time() - end)
# compute output
x = torch.cat((x_s, x_t), dim=0)
y, f = model(x)
y_s, y_t = y.chunk(2, dim=0)
f_s, f_t = f.chunk(2, dim=0)
cls_loss = F.cross_entropy(y_s, labels_s)
transfer_loss = domain_adv(f_s, f_t)
bsp_loss = bsp_penalty(f_s, f_t)
domain_acc = domain_adv.domain_discriminator_accuracy
loss = cls_loss + transfer_loss * args.trade_off + bsp_loss * args.trade_off_bsp
cls_acc = accuracy(y_s, labels_s)[0]
losses.update(loss.item(), x_s.size(0))
cls_accs.update(cls_acc.item(), x_s.size(0))
domain_accs.update(domain_acc.item(), x_s.size(0))
# compute gradient and do SGD step
optimizer.zero_grad()
loss.backward()
optimizer.step()
lr_scheduler.step()
# measure elapsed time
batch_time.update(time.time() - end)
end = time.time()
if i % args.print_freq == 0:
progress.display(i)
if __name__ == '__main__':
parser = argparse.ArgumentParser(description='DANN for Unsupervised Domain Adaptation')
# dataset parameters
parser.add_argument('root', metavar='DIR',
help='root path of dataset')
parser.add_argument('-d', '--data', metavar='DATA', default='Office31', choices=utils.get_dataset_names(),
help='dataset: ' + ' | '.join(utils.get_dataset_names()) +
' (default: Office31)')
parser.add_argument('-s', '--source', help='source domain(s)', nargs='+')
parser.add_argument('-t', '--target', help='target domain(s)', nargs='+')
parser.add_argument('--train-resizing', type=str, default='default')
parser.add_argument('--val-resizing', type=str, default='default')
parser.add_argument('--resize-size', type=int, default=224,
help='the image size after resizing')
parser.add_argument('--no-hflip', action='store_true',
help='no random horizontal flipping during training')
parser.add_argument('--norm-mean', type=float, nargs='+',
default=(0.485, 0.456, 0.406), help='normalization mean')
parser.add_argument('--norm-std', type=float, nargs='+',
default=(0.229, 0.224, 0.225), help='normalization std')
# model parameters
parser.add_argument('-a', '--arch', metavar='ARCH', default='resnet18',
choices=utils.get_model_names(),
help='backbone architecture: ' +
' | '.join(utils.get_model_names()) +
' (default: resnet18)')
parser.add_argument('--pretrain', type=str, default=None,
help='pretrain checkpoint for classification model')
parser.add_argument('--bottleneck-dim', default=256, type=int,
help='Dimension of bottleneck')
parser.add_argument('--no-pool', action='store_true',
help='no pool layer after the feature extractor.')
parser.add_argument('--scratch', action='store_true', help='whether train from scratch.')
parser.add_argument('--trade-off', default=1., type=float,
help='the trade-off hyper-parameter for transfer loss')
parser.add_argument('--trade-off-bsp', default=2e-4, type=float,
help='the trade-off hyper-parameter for bsp loss')
# training parameters
parser.add_argument('-b', '--batch-size', default=32, type=int,
metavar='N',
help='mini-batch size (default: 32)')
parser.add_argument('--lr', '--learning-rate', default=0.003, type=float,
metavar='LR', help='initial learning rate', dest='lr')
parser.add_argument('--pretrain-lr', default=0.001, type=float, help='initial pretrain learning rate')
parser.add_argument('--lr-gamma', default=0.001, type=float, help='parameter for lr scheduler')
parser.add_argument('--lr-decay', default=0.75, type=float, help='parameter for lr scheduler')
parser.add_argument('--momentum', default=0.9, type=float, metavar='M',
help='momentum')
parser.add_argument('--wd', '--weight-decay', default=1e-3, type=float,
metavar='W', help='weight decay (default: 1e-3)',
dest='weight_decay')
parser.add_argument('-j', '--workers', default=2, type=int, metavar='N',
help='number of data loading workers (default: 2)')
parser.add_argument('--epochs', default=20, type=int, metavar='N',
help='number of total epochs to run')
parser.add_argument('--pretrain-epochs', default=3, type=int, metavar='N',
help='number of total epochs(pretrain) to run')
parser.add_argument('-i', '--iters-per-epoch', default=1000, type=int,
help='Number of iterations per epoch')
parser.add_argument('-p', '--print-freq', default=100, type=int,
metavar='N', help='print frequency (default: 100)')
parser.add_argument('--seed', default=None, type=int,
help='seed for initializing training. ')
parser.add_argument('--per-class-eval', action='store_true',
help='whether output per-class accuracy during evaluation')
parser.add_argument("--log", type=str, default='bsp',
help="Where to save logs, checkpoints and debugging images.")
parser.add_argument("--phase", type=str, default='train', choices=['train', 'test', 'analysis'],
help="When phase is 'test', only test the model."
"When phase is 'analysis', only analysis the model.")
args = parser.parse_args()
main(args)
================================================
FILE: domain_generalization/Transfer-Learning-Library/examples/domain_adaptation/image_classification/bsp.sh
================================================
#!/usr/bin/env bash
# ResNet50, Office31, Single Source
CUDA_VISIBLE_DEVICES=0 python bsp.py data/office31 -d Office31 -s A -t W -a resnet50 --epochs 20 --seed 1 --log logs/bsp/Office31_A2W
CUDA_VISIBLE_DEVICES=0 python bsp.py data/office31 -d Office31 -s D -t W -a resnet50 --epochs 20 --seed 1 --log logs/bsp/Office31_D2W
CUDA_VISIBLE_DEVICES=0 python bsp.py data/office31 -d Office31 -s W -t D -a resnet50 --epochs 20 --seed 1 --log logs/bsp/Office31_W2D
CUDA_VISIBLE_DEVICES=0 python bsp.py data/office31 -d Office31 -s A -t D -a resnet50 --epochs 20 --seed 1 --log logs/bsp/Office31_A2D
CUDA_VISIBLE_DEVICES=0 python bsp.py data/office31 -d Office31 -s D -t A -a resnet50 --epochs 20 --seed 1 --log logs/bsp/Office31_D2A
CUDA_VISIBLE_DEVICES=0 python bsp.py data/office31 -d Office31 -s W -t A -a resnet50 --epochs 20 --seed 1 --log logs/bsp/Office31_W2A
# ResNet50, Office-Home, Single Source
CUDA_VISIBLE_DEVICES=0 python bsp.py data/office-home -d OfficeHome -s Ar -t Cl -a resnet50 --epochs 30 --seed 0 --log logs/bsp/OfficeHome_Ar2Cl
CUDA_VISIBLE_DEVICES=0 python bsp.py data/office-home -d OfficeHome -s Ar -t Pr -a resnet50 --epochs 30 --seed 0 --log logs/bsp/OfficeHome_Ar2Pr
CUDA_VISIBLE_DEVICES=0 python bsp.py data/office-home -d OfficeHome -s Ar -t Rw -a resnet50 --epochs 30 --seed 0 --log logs/bsp/OfficeHome_Ar2Rw
CUDA_VISIBLE_DEVICES=0 python bsp.py data/office-home -d OfficeHome -s Cl -t Ar -a resnet50 --epochs 30 --seed 0 --log logs/bsp/OfficeHome_Cl2Ar
CUDA_VISIBLE_DEVICES=0 python bsp.py data/office-home -d OfficeHome -s Cl -t Pr -a resnet50 --epochs 30 --seed 0 --log logs/bsp/OfficeHome_Cl2Pr
CUDA_VISIBLE_DEVICES=0 python bsp.py data/office-home -d OfficeHome -s Cl -t Rw -a resnet50 --epochs 30 --seed 0 --log logs/bsp/OfficeHome_Cl2Rw
CUDA_VISIBLE_DEVICES=0 python bsp.py data/office-home -d OfficeHome -s Pr -t Ar -a resnet50 --epochs 30 --seed 0 --log logs/bsp/OfficeHome_Pr2Ar
CUDA_VISIBLE_DEVICES=0 python bsp.py data/office-home -d OfficeHome -s Pr -t Cl -a resnet50 --epochs 30 --seed 0 --log logs/bsp/OfficeHome_Pr2Cl
CUDA_VISIBLE_DEVICES=0 python bsp.py data/office-home -d OfficeHome -s Pr -t Rw -a resnet50 --epochs 30 --seed 0 --log logs/bsp/OfficeHome_Pr2Rw
CUDA_VISIBLE_DEVICES=0 python bsp.py data/office-home -d OfficeHome -s Rw -t Ar -a resnet50 --epochs 30 --seed 0 --log logs/bsp/OfficeHome_Rw2Ar
CUDA_VISIBLE_DEVICES=0 python bsp.py data/office-home -d OfficeHome -s Rw -t Cl -a resnet50 --epochs 30 --seed 0 --log logs/bsp/OfficeHome_Rw2Cl
CUDA_VISIBLE_DEVICES=0 python bsp.py data/office-home -d OfficeHome -s Rw -t Pr -a resnet50 --epochs 30 --seed 0 --log logs/bsp/OfficeHome_Rw2Pr
# ResNet101, VisDA-2017, Single Source
CUDA_VISIBLE_DEVICES=0 python bsp.py data/visda-2017 -d VisDA2017 -s Synthetic -t Real -a resnet101 \
--epochs 30 --seed 0 --per-class-eval --train-resizing cen.crop --log logs/bsp/VisDA2017
# ResNet101, DomainNet, Single Source
CUDA_VISIBLE_DEVICES=0 python bsp.py data/domainnet -d DomainNet -s c -t p -a resnet101 --bottleneck-dim 1024 --epochs 30 -i 2500 -p 500 --seed 0 --log logs/bsp/DomainNet_c2p
CUDA_VISIBLE_DEVICES=0 python bsp.py data/domainnet -d DomainNet -s c -t r -a resnet101 --bottleneck-dim 1024 --epochs 30 -i 2500 -p 500 --seed 0 --log logs/bsp/DomainNet_c2r
CUDA_VISIBLE_DEVICES=0 python bsp.py data/domainnet -d DomainNet -s c -t s -a resnet101 --bottleneck-dim 1024 --epochs 30 -i 2500 -p 500 --seed 0 --log logs/bsp/DomainNet_c2s
CUDA_VISIBLE_DEVICES=0 python bsp.py data/domainnet -d DomainNet -s p -t c -a resnet101 --bottleneck-dim 1024 --epochs 30 -i 2500 -p 500 --seed 0 --log logs/bsp/DomainNet_p2c
CUDA_VISIBLE_DEVICES=0 python bsp.py data/domainnet -d DomainNet -s p -t r -a resnet101 --bottleneck-dim 1024 --epochs 30 -i 2500 -p 500 --seed 0 --log logs/bsp/DomainNet_p2r
CUDA_VISIBLE_DEVICES=0 python bsp.py data/domainnet -d DomainNet -s p -t s -a resnet101 --bottleneck-dim 1024 --epochs 30 -i 2500 -p 500 --seed 0 --log logs/bsp/DomainNet_p2s
CUDA_VISIBLE_DEVICES=0 python bsp.py data/domainnet -d DomainNet -s r -t c -a resnet101 --bottleneck-dim 1024 --epochs 30 -i 2500 -p 500 --seed 0 --log logs/bsp/DomainNet_r2c
CUDA_VISIBLE_DEVICES=0 python bsp.py data/domainnet -d DomainNet -s r -t p -a resnet101 --bottleneck-dim 1024 --epochs 30 -i 2500 -p 500 --seed 0 --log logs/bsp/DomainNet_r2p
CUDA_VISIBLE_DEVICES=0 python bsp.py data/domainnet -d DomainNet -s r -t s -a resnet101 --bottleneck-dim 1024 --epochs 30 -i 2500 -p 500 --seed 0 --log logs/bsp/DomainNet_r2s
CUDA_VISIBLE_DEVICES=0 python bsp.py data/domainnet -d DomainNet -s s -t c -a resnet101 --bottleneck-dim 1024 --epochs 30 -i 2500 -p 500 --seed 0 --log logs/bsp/DomainNet_s2c
CUDA_VISIBLE_DEVICES=0 python bsp.py data/domainnet -d DomainNet -s s -t p -a resnet101 --bottleneck-dim 1024 --epochs 30 -i 2500 -p 500 --seed 0 --log logs/bsp/DomainNet_s2p
CUDA_VISIBLE_DEVICES=0 python bsp.py data/domainnet -d DomainNet -s s -t r -a resnet101 --bottleneck-dim 1024 --epochs 30 -i 2500 -p 500 --seed 0 --log logs/bsp/DomainNet_s2r
# ResNet50, ImageNet200 -> ImageNetR
CUDA_VISIBLE_DEVICES=0 python bsp.py data/ImageNetR -d ImageNetR -s IN -t INR -a resnet50 --epochs 30 -i 2500 -p 500 --seed 0 --log logs/bsp/ImageNet_IN2INR
# ig_resnext101_32x8d, ImageNet -> ImageNetSketch
CUDA_VISIBLE_DEVICES=0 python bsp.py data/imagenet-sketch -d ImageNetSketch -s IN -t sketch -a ig_resnext101_32x8d --epochs 30 -i 2500 -p 500 --bottleneck-dim 1024 --log logs/dann_ig_resnext101_32x8d/ImageNet_IN2sketch
# Vision Transformer, Office-Home, Single Source
CUDA_VISIBLE_DEVICES=0 python bsp.py data/office-home -d OfficeHome -s Ar -t Cl -a vit_base_patch16_224 --epochs 30 --seed 0 -b 24 --no-pool --log logs/dann_vit/OfficeHome_Ar2Cl
CUDA_VISIBLE_DEVICES=0 python bsp.py data/office-home -d OfficeHome -s Ar -t Pr -a vit_base_patch16_224 --epochs 30 --seed 0 -b 24 --no-pool --log logs/dann_vit/OfficeHome_Ar2Pr
CUDA_VISIBLE_DEVICES=0 python bsp.py data/office-home -d OfficeHome -s Ar -t Rw -a vit_base_patch16_224 --epochs 30 --seed 0 -b 24 --no-pool --log logs/dann_vit/OfficeHome_Ar2Rw
CUDA_VISIBLE_DEVICES=0 python bsp.py data/office-home -d OfficeHome -s Cl -t Ar -a vit_base_patch16_224 --epochs 30 --seed 0 -b 24 --no-pool --log logs/dann_vit/OfficeHome_Cl2Ar
CUDA_VISIBLE_DEVICES=0 python bsp.py data/office-home -d OfficeHome -s Cl -t Pr -a vit_base_patch16_224 --epochs 30 --seed 0 -b 24 --no-pool --log logs/dann_vit/OfficeHome_Cl2Pr
CUDA_VISIBLE_DEVICES=0 python bsp.py data/office-home -d OfficeHome -s Cl -t Rw -a vit_base_patch16_224 --epochs 30 --seed 0 -b 24 --no-pool --log logs/dann_vit/OfficeHome_Cl2Rw
CUDA_VISIBLE_DEVICES=0 python bsp.py data/office-home -d OfficeHome -s Pr -t Ar -a vit_base_patch16_224 --epochs 30 --seed 0 -b 24 --no-pool --log logs/dann_vit/OfficeHome_Pr2Ar
CUDA_VISIBLE_DEVICES=0 python bsp.py data/office-home -d OfficeHome -s Pr -t Cl -a vit_base_patch16_224 --epochs 30 --seed 0 -b 24 --no-pool --log logs/dann_vit/OfficeHome_Pr2Cl
CUDA_VISIBLE_DEVICES=0 python bsp.py data/office-home -d OfficeHome -s Pr -t Rw -a vit_base_patch16_224 --epochs 30 --seed 0 -b 24 --no-pool --log logs/dann_vit/OfficeHome_Pr2Rw
CUDA_VISIBLE_DEVICES=0 python bsp.py data/office-home -d OfficeHome -s Rw -t Ar -a vit_base_patch16_224 --epochs 30 --seed 0 -b 24 --no-pool --log logs/dann_vit/OfficeHome_Rw2Ar
CUDA_VISIBLE_DEVICES=0 python bsp.py data/office-home -d OfficeHome -s Rw -t Cl -a vit_base_patch16_224 --epochs 30 --seed 0 -b 24 --no-pool --log logs/dann_vit/OfficeHome_Rw2Cl
CUDA_VISIBLE_DEVICES=0 python bsp.py data/office-home -d OfficeHome -s Rw -t Pr -a vit_base_patch16_224 --epochs 30 --seed 0 -b 24 --no-pool --log logs/dann_vit/OfficeHome_Rw2Pr
# ResNet50, Office-Home, Multi Source
CUDA_VISIBLE_DEVICES=0 python bsp.py data/office-home -d OfficeHome -s Cl Pr Rw -t Ar -a resnet50 --epochs 30 --seed 0 --log logs/bsp/OfficeHome_:2Ar
CUDA_VISIBLE_DEVICES=0 python bsp.py data/office-home -d OfficeHome -s Ar Pr Rw -t Cl -a resnet50 --epochs 30 --seed 0 --log logs/bsp/OfficeHome_:2Cl
CUDA_VISIBLE_DEVICES=0 python bsp.py data/office-home -d OfficeHome -s Ar Cl Rw -t Pr -a resnet50 --epochs 30 --seed 0 --log logs/bsp/OfficeHome_:2Pr
CUDA_VISIBLE_DEVICES=0 python bsp.py data/office-home -d OfficeHome -s Ar Cl Pr -t Rw -a resnet50 --epochs 30 --seed 0 --log logs/bsp/OfficeHome_:2Rw
# ResNet101, DomainNet, Multi Source
CUDA_VISIBLE_DEVICES=0 python bsp.py data/domainnet -d DomainNet -s i p q r s -t c -a resnet101 --bottleneck-dim 1024 --epochs 40 -i 5000 -p 500 --seed 0 --log logs/bsp/DomainNet_:2c
CUDA_VISIBLE_DEVICES=0 python bsp.py data/domainnet -d DomainNet -s c p q r s -t i -a resnet101 --bottleneck-dim 1024 --epochs 40 -i 5000 -p 500 --seed 0 --log logs/bsp/DomainNet_:2i
CUDA_VISIBLE_DEVICES=0 python bsp.py data/domainnet -d DomainNet -s c i q r s -t p -a resnet101 --bottleneck-dim 1024 --epochs 40 -i 5000 -p 500 --seed 0 --log logs/bsp/DomainNet_:2p
CUDA_VISIBLE_DEVICES=0 python bsp.py data/domainnet -d DomainNet -s c i p r s -t q -a resnet101 --bottleneck-dim 1024 --epochs 40 -i 5000 -p 500 --seed 0 --log logs/bsp/DomainNet_:2q
CUDA_VISIBLE_DEVICES=0 python bsp.py data/domainnet -d DomainNet -s c i p q s -t r -a resnet101 --bottleneck-dim 1024 --epochs 40 -i 5000 -p 500 --seed 0 --log logs/bsp/DomainNet_:2r
CUDA_VISIBLE_DEVICES=0 python bsp.py data/domainnet -d DomainNet -s c i p q r -t s -a resnet101 --bottleneck-dim 1024 --epochs 40 -i 5000 -p 500 --seed 0 --log logs/bsp/DomainNet_:2s
# ResNet50, Wilds Dataset
CUDA_VISIBLE_DEVICES=0 python bsp.py data/wilds -d iwildcam -a resnet50 --epochs 30 -i 1000 --seed 0 --log logs/bsp/iwildcam
# DenseNet121, Wilds Dataset
CUDA_VISIBLE_DEVICES=0 python bsp.py data/wilds -d camelyon17 --train-resizing 'res.' --val-resizing 'res.' --resize-size 96 \
-a densenet121 --scratch --epochs 10 -i 1000 --lr 0.01 --seed 0 --log logs/bsp/camelyon17
CUDA_VISIBLE_DEVICES=0 python bsp.py data/wilds -d fmow --train-resizing 'res.' --val-resizing 'res.' \
-a densenet121 --epochs 10 -i 1000 --lr 0.01 --seed 0 --log logs/bsp/fmow
# Digits
CUDA_VISIBLE_DEVICES=0 python bsp.py data/digits -d Digits -s MNIST -t USPS --train-resizing 'res.' --val-resizing 'res.' \
--resize-size 28 --no-hflip --norm-mean 0.5 --norm-std 0.5 -a lenet --no-pool --lr 0.01 --trade-off-bsp 0.0001 -b 128 -i 2500 --scratch --seed 0 --log logs/bsp/MNIST2USPS
CUDA_VISIBLE_DEVICES=0 python bsp.py data/digits -d Digits -s USPS -t MNIST --train-resizing 'res.' --val-resizing 'res.' \
--resize-size 28 --no-hflip --norm-mean 0.5 --norm-std 0.5 -a lenet --no-pool --lr 0.1 --trade-off-bsp 0.0001 -b 128 -i 2500 --scratch --seed 0 --log logs/bsp/USPS2MNIST
CUDA_VISIBLE_DEVICES=0 python bsp.py data/digits -d Digits -s SVHNRGB -t MNISTRGB --train-resizing 'res.' --val-resizing 'res.' \
--resize-size 32 --no-hflip --norm-mean 0.5 0.5 0.5 --norm-std 0.5 0.5 0.5 -a dtn --no-pool --lr 0.1 --trade-off-bsp 0.0001 -b 128 -i 2500 --scratch --seed 0 --log logs/bsp/SVHN2MNIST --pretrain-epochs 1
================================================
FILE: domain_generalization/Transfer-Learning-Library/examples/domain_adaptation/image_classification/cdan.py
================================================
"""
@author: Junguang Jiang
@contact: JiangJunguang1123@outlook.com
"""
import random
import time
import warnings
import sys
import argparse
import shutil
import os.path as osp
import torch
import torch.nn as nn
import torch.backends.cudnn as cudnn
from torch.optim import SGD
from torch.optim.lr_scheduler import LambdaLR
from torch.utils.data import DataLoader
import torch.nn.functional as F
sys.path.append('../../..')
from dalib.modules.domain_discriminator import DomainDiscriminator
from dalib.adaptation.cdan import ConditionalDomainAdversarialLoss, ImageClassifier
from common.utils.data import ForeverDataIterator
from common.utils.metric import accuracy
from common.utils.meter import AverageMeter, ProgressMeter
from common.utils.logger import CompleteLogger
from common.utils.analysis import collect_feature, tsne, a_distance
sys.path.append('.')
import utils
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
def main(args: argparse.Namespace):
logger = CompleteLogger(args.log, args.phase)
print(args)
if args.seed is not None:
random.seed(args.seed)
torch.manual_seed(args.seed)
cudnn.deterministic = True
warnings.warn('You have chosen to seed training. '
'This will turn on the CUDNN deterministic setting, '
'which can slow down your training considerably! '
'You may see unexpected behavior when restarting '
'from checkpoints.')
cudnn.benchmark = True
# Data loading code
train_transform = utils.get_train_transform(args.train_resizing, random_horizontal_flip=not args.no_hflip,
random_color_jitter=False, resize_size=args.resize_size,
norm_mean=args.norm_mean, norm_std=args.norm_std)
val_transform = utils.get_val_transform(args.val_resizing, resize_size=args.resize_size,
norm_mean=args.norm_mean, norm_std=args.norm_std)
print("train_transform: ", train_transform)
print("val_transform: ", val_transform)
train_source_dataset, train_target_dataset, val_dataset, test_dataset, num_classes, args.class_names = \
utils.get_dataset(args.data, args.root, args.source, args.target, train_transform, val_transform)
train_source_loader = DataLoader(train_source_dataset, batch_size=args.batch_size,
shuffle=True, num_workers=args.workers, drop_last=True)
train_target_loader = DataLoader(train_target_dataset, batch_size=args.batch_size,
shuffle=True, num_workers=args.workers, drop_last=True)
val_loader = DataLoader(val_dataset, batch_size=args.batch_size, shuffle=False, num_workers=args.workers)
test_loader = DataLoader(test_dataset, batch_size=args.batch_size, shuffle=False, num_workers=args.workers)
train_source_iter = ForeverDataIterator(train_source_loader)
train_target_iter = ForeverDataIterator(train_target_loader)
# create model
print("=> using model '{}'".format(args.arch))
backbone = utils.get_model(args.arch, pretrain=not args.scratch)
pool_layer = nn.Identity() if args.no_pool else None
classifier = ImageClassifier(backbone, num_classes, bottleneck_dim=args.bottleneck_dim,
pool_layer=pool_layer, finetune=not args.scratch).to(device)
classifier_feature_dim = classifier.features_dim
if args.randomized:
domain_discri = DomainDiscriminator(args.randomized_dim, hidden_size=1024).to(device)
else:
domain_discri = DomainDiscriminator(classifier_feature_dim * num_classes, hidden_size=1024).to(device)
all_parameters = classifier.get_parameters() + domain_discri.get_parameters()
# define optimizer and lr scheduler
optimizer = SGD(all_parameters, args.lr, momentum=args.momentum, weight_decay=args.weight_decay, nesterov=True)
lr_scheduler = LambdaLR(optimizer, lambda x: args.lr * (1. + args.lr_gamma * float(x)) ** (-args.lr_decay))
# define loss function
domain_adv = ConditionalDomainAdversarialLoss(
domain_discri, entropy_conditioning=args.entropy,
num_classes=num_classes, features_dim=classifier_feature_dim, randomized=args.randomized,
randomized_dim=args.randomized_dim
).to(device)
# resume from the best checkpoint
if args.phase != 'train':
checkpoint = torch.load(logger.get_checkpoint_path('best'), map_location='cpu')
classifier.load_state_dict(checkpoint)
# analysis the model
if args.phase == 'analysis':
# extract features from both domains
feature_extractor = nn.Sequential(classifier.backbone, classifier.pool_layer, classifier.bottleneck).to(device)
source_feature = collect_feature(train_source_loader, feature_extractor, device)
target_feature = collect_feature(train_target_loader, feature_extractor, device)
# plot t-SNE
tSNE_filename = osp.join(logger.visualize_directory, 'TSNE.pdf')
tsne.visualize(source_feature, target_feature, tSNE_filename)
print("Saving t-SNE to", tSNE_filename)
# calculate A-distance, which is a measure for distribution discrepancy
A_distance = a_distance.calculate(source_feature, target_feature, device)
print("A-distance =", A_distance)
return
if args.phase == 'test':
acc1 = utils.validate(test_loader, classifier, args, device)
print(acc1)
return
# start training
best_acc1 = 0.
for epoch in range(args.epochs):
print("lr:", lr_scheduler.get_last_lr()[0])
# train for one epoch
train(train_source_iter, train_target_iter, classifier, domain_adv, optimizer,
lr_scheduler, epoch, args)
# evaluate on validation set
acc1 = utils.validate(val_loader, classifier, args, device)
# remember best acc@1 and save checkpoint
torch.save(classifier.state_dict(), logger.get_checkpoint_path('latest'))
if acc1 > best_acc1:
shutil.copy(logger.get_checkpoint_path('latest'), logger.get_checkpoint_path('best'))
best_acc1 = max(acc1, best_acc1)
print("best_acc1 = {:3.1f}".format(best_acc1))
# evaluate on test set
classifier.load_state_dict(torch.load(logger.get_checkpoint_path('best')))
acc1 = utils.validate(test_loader, classifier, args, device)
print("test_acc1 = {:3.1f}".format(acc1))
logger.close()
def train(train_source_iter: ForeverDataIterator, train_target_iter: ForeverDataIterator, model: ImageClassifier,
domain_adv: ConditionalDomainAdversarialLoss, optimizer: SGD,
lr_scheduler: LambdaLR, epoch: int, args: argparse.Namespace):
batch_time = AverageMeter('Time', ':3.1f')
data_time = AverageMeter('Data', ':3.1f')
losses = AverageMeter('Loss', ':3.2f')
trans_losses = AverageMeter('Trans Loss', ':3.2f')
cls_accs = AverageMeter('Cls Acc', ':3.1f')
domain_accs = AverageMeter('Domain Acc', ':3.1f')
progress = ProgressMeter(
args.iters_per_epoch,
[batch_time, data_time, losses, trans_losses, cls_accs, domain_accs],
prefix="Epoch: [{}]".format(epoch))
# switch to train mode
model.train()
domain_adv.train()
end = time.time()
for i in range(args.iters_per_epoch):
x_s, labels_s = next(train_source_iter)
x_t, _ = next(train_target_iter)
x_s = x_s.to(device)
x_t = x_t.to(device)
labels_s = labels_s.to(device)
# measure data loading time
data_time.update(time.time() - end)
# compute output
x = torch.cat((x_s, x_t), dim=0)
y, f = model(x)
y_s, y_t = y.chunk(2, dim=0)
f_s, f_t = f.chunk(2, dim=0)
cls_loss = F.cross_entropy(y_s, labels_s)
transfer_loss = domain_adv(y_s, f_s, y_t, f_t)
domain_acc = domain_adv.domain_discriminator_accuracy
loss = cls_loss + transfer_loss * args.trade_off
cls_acc = accuracy(y_s, labels_s)[0]
losses.update(loss.item(), x_s.size(0))
cls_accs.update(cls_acc, x_s.size(0))
domain_accs.update(domain_acc, x_s.size(0))
trans_losses.update(transfer_loss.item(), x_s.size(0))
# compute gradient and do SGD step
optimizer.zero_grad()
loss.backward()
optimizer.step()
lr_scheduler.step()
# measure elapsed time
batch_time.update(time.time() - end)
end = time.time()
if i % args.print_freq == 0:
progress.display(i)
if __name__ == '__main__':
parser = argparse.ArgumentParser(description='CDAN for Unsupervised Domain Adaptation')
# dataset parameters
parser.add_argument('root', metavar='DIR',
help='root path of dataset')
parser.add_argument('-d', '--data', metavar='DATA', default='Office31', choices=utils.get_dataset_names(),
help='dataset: ' + ' | '.join(utils.get_dataset_names()) +
' (default: Office31)')
parser.add_argument('-s', '--source', help='source domain(s)', nargs='+')
parser.add_argument('-t', '--target', help='target domain(s)', nargs='+')
parser.add_argument('--train-resizing', type=str, default='default')
parser.add_argument('--val-resizing', type=str, default='default')
parser.add_argument('--resize-size', type=int, default=224,
help='the image size after resizing')
parser.add_argument('--no-hflip', action='store_true',
help='no random horizontal flipping during training')
parser.add_argument('--norm-mean', type=float, nargs='+',
default=(0.485, 0.456, 0.406), help='normalization mean')
parser.add_argument('--norm-std', type=float, nargs='+',
default=(0.229, 0.224, 0.225), help='normalization std')
# model parameters
parser.add_argument('-a', '--arch', metavar='ARCH', default='resnet18',
choices=utils.get_model_names(),
help='backbone architecture: ' +
' | '.join(utils.get_model_names()) +
' (default: resnet18)')
parser.add_argument('--bottleneck-dim', default=256, type=int,
help='Dimension of bottleneck')
parser.add_argument('--no-pool', action='store_true',
help='no pool layer after the feature extractor.')
parser.add_argument('--scratch', action='store_true', help='whether train from scratch.')
parser.add_argument('-r', '--randomized', action='store_true',
help='using randomized multi-linear-map (default: False)')
parser.add_argument('-rd', '--randomized-dim', default=1024, type=int,
help='randomized dimension when using randomized multi-linear-map (default: 1024)')
parser.add_argument('--entropy', default=False, action='store_true', help='use entropy conditioning')
parser.add_argument('--trade-off', default=1., type=float,
help='the trade-off hyper-parameter for transfer loss')
# training parameters
parser.add_argument('-b', '--batch-size', default=32, type=int,
metavar='N',
help='mini-batch size (default: 32)')
parser.add_argument('--lr', '--learning-rate', default=0.01, type=float,
metavar='LR', help='initial learning rate', dest='lr')
parser.add_argument('--lr-gamma', default=0.001, type=float, help='parameter for lr scheduler')
parser.add_argument('--lr-decay', default=0.75, type=float, help='parameter for lr scheduler')
parser.add_argument('--momentum', default=0.9, type=float, metavar='M', help='momentum')
parser.add_argument('--wd', '--weight-decay', default=1e-3, type=float,
metavar='W', help='weight decay (default: 1e-3)',
dest='weight_decay')
parser.add_argument('-j', '--workers', default=2, type=int, metavar='N',
help='number of data loading workers (default: 2)')
parser.add_argument('--epochs', default=20, type=int, metavar='N',
help='number of total epochs to run')
parser.add_argument('-i', '--iters-per-epoch', default=1000, type=int,
help='Number of iterations per epoch')
parser.add_argument('-p', '--print-freq', default=100, type=int,
metavar='N', help='print frequency (default: 100)')
parser.add_argument('--seed', default=None, type=int,
help='seed for initializing training. ')
parser.add_argument('--per-class-eval', action='store_true',
help='whether output per-class accuracy during evaluation')
parser.add_argument("--log", type=str, default='cdan',
help="Where to save logs, checkpoints and debugging images.")
parser.add_argument("--phase", type=str, default='train', choices=['train', 'test', 'analysis'],
help="When phase is 'test', only test the model."
"When phase is 'analysis', only analysis the model.")
args = parser.parse_args()
main(args)
================================================
FILE: domain_generalization/Transfer-Learning-Library/examples/domain_adaptation/image_classification/cdan.sh
================================================
#!/usr/bin/env bash
# ResNet50, Office31, Single Source
CUDA_VISIBLE_DEVICES=0 python cdan.py data/office31 -d Office31 -s A -t W -a resnet50 --epochs 20 --seed 2 --log logs/cdan/Office31_A2W
CUDA_VISIBLE_DEVICES=0 python cdan.py data/office31 -d Office31 -s D -t W -a resnet50 --epochs 20 --seed 2 --log logs/cdan/Office31_D2W
CUDA_VISIBLE_DEVICES=0 python cdan.py data/office31 -d Office31 -s W -t D -a resnet50 --epochs 20 --seed 2 --log logs/cdan/Office31_W2D
CUDA_VISIBLE_DEVICES=0 python cdan.py data/office31 -d Office31 -s A -t D -a resnet50 --epochs 20 --seed 2 --log logs/cdan/Office31_A2D
CUDA_VISIBLE_DEVICES=0 python cdan.py data/office31 -d Office31 -s D -t A -a resnet50 --epochs 20 --seed 2 --log logs/cdan/Office31_D2A
CUDA_VISIBLE_DEVICES=0 python cdan.py data/office31 -d Office31 -s W -t A -a resnet50 --epochs 20 --seed 2 --log logs/cdan/Office31_W2A
# ResNet50, Office-Home, Single Source
CUDA_VISIBLE_DEVICES=0 python cdan.py data/office-home -d OfficeHome -s Ar -t Cl -a resnet50 --epochs 30 --seed 0 --log logs/cdan/OfficeHome_Ar2Cl
CUDA_VISIBLE_DEVICES=0 python cdan.py data/office-home -d OfficeHome -s Ar -t Pr -a resnet50 --epochs 30 --seed 0 --log logs/cdan/OfficeHome_Ar2Pr
CUDA_VISIBLE_DEVICES=0 python cdan.py data/office-home -d OfficeHome -s Ar -t Rw -a resnet50 --epochs 30 --seed 0 --log logs/cdan/OfficeHome_Ar2Rw
CUDA_VISIBLE_DEVICES=0 python cdan.py data/office-home -d OfficeHome -s Cl -t Ar -a resnet50 --epochs 30 --seed 0 --log logs/cdan/OfficeHome_Cl2Ar
CUDA_VISIBLE_DEVICES=0 python cdan.py data/office-home -d OfficeHome -s Cl -t Pr -a resnet50 --epochs 30 --seed 0 --log logs/cdan/OfficeHome_Cl2Pr
CUDA_VISIBLE_DEVICES=0 python cdan.py data/office-home -d OfficeHome -s Cl -t Rw -a resnet50 --epochs 30 --seed 0 --log logs/cdan/OfficeHome_Cl2Rw
CUDA_VISIBLE_DEVICES=0 python cdan.py data/office-home -d OfficeHome -s Pr -t Ar -a resnet50 --epochs 30 --seed 0 --log logs/cdan/OfficeHome_Pr2Ar
CUDA_VISIBLE_DEVICES=0 python cdan.py data/office-home -d OfficeHome -s Pr -t Cl -a resnet50 --epochs 30 --seed 0 --log logs/cdan/OfficeHome_Pr2Cl
CUDA_VISIBLE_DEVICES=0 python cdan.py data/office-home -d OfficeHome -s Pr -t Rw -a resnet50 --epochs 30 --seed 0 --log logs/cdan/OfficeHome_Pr2Rw
CUDA_VISIBLE_DEVICES=0 python cdan.py data/office-home -d OfficeHome -s Rw -t Ar -a resnet50 --epochs 30 --seed 0 --log logs/cdan/OfficeHome_Rw2Ar
CUDA_VISIBLE_DEVICES=0 python cdan.py data/office-home -d OfficeHome -s Rw -t Cl -a resnet50 --epochs 30 --seed 0 --log logs/cdan/OfficeHome_Rw2Cl
CUDA_VISIBLE_DEVICES=0 python cdan.py data/office-home -d OfficeHome -s Rw -t Pr -a resnet50 --epochs 30 --seed 0 --log logs/cdan/OfficeHome_Rw2Pr
# ResNet101, VisDA-2017, Single Source
CUDA_VISIBLE_DEVICES=0 python cdan.py data/visda-2017 -d VisDA2017 -s Synthetic -t Real -a resnet101 \
--epochs 30 --seed 0 --per-class-eval --train-resizing cen.crop --log logs/cdan/VisDA2017
# ResNet101, DomainNet, Single Source
# Use randomized multi-linear-map to decrease GPU memory usage
CUDA_VISIBLE_DEVICES=0 python cdan.py data/domainnet -d DomainNet -s c -t p -a resnet101 --bottleneck-dim 1024 -r -rd 51200 --epochs 30 -i 2500 -p 500 --seed 0 --log logs/cdan/DomainNet_c2p
CUDA_VISIBLE_DEVICES=0 python cdan.py data/domainnet -d DomainNet -s c -t r -a resnet101 --bottleneck-dim 1024 -r -rd 51200 --epochs 30 -i 2500 -p 500 --seed 0 --log logs/cdan/DomainNet_c2r
CUDA_VISIBLE_DEVICES=0 python cdan.py data/domainnet -d DomainNet -s c -t s -a resnet101 --bottleneck-dim 1024 -r -rd 51200 --epochs 30 -i 2500 -p 500 --seed 0 --log logs/cdan/DomainNet_c2s
CUDA_VISIBLE_DEVICES=0 python cdan.py data/domainnet -d DomainNet -s p -t c -a resnet101 --bottleneck-dim 1024 -r -rd 51200 --epochs 30 -i 2500 -p 500 --seed 0 --log logs/cdan/DomainNet_p2c
CUDA_VISIBLE_DEVICES=0 python cdan.py data/domainnet -d DomainNet -s p -t r -a resnet101 --bottleneck-dim 1024 -r -rd 51200 --epochs 30 -i 2500 -p 500 --seed 0 --log logs/cdan/DomainNet_p2r
CUDA_VISIBLE_DEVICES=0 python cdan.py data/domainnet -d DomainNet -s p -t s -a resnet101 --bottleneck-dim 1024 -r -rd 51200 --epochs 30 -i 2500 -p 500 --seed 0 --log logs/cdan/DomainNet_p2s
CUDA_VISIBLE_DEVICES=0 python cdan.py data/domainnet -d DomainNet -s r -t c -a resnet101 --bottleneck-dim 1024 -r -rd 51200 --epochs 30 -i 2500 -p 500 --seed 0 --log logs/cdan/DomainNet_r2c
CUDA_VISIBLE_DEVICES=0 python cdan.py data/domainnet -d DomainNet -s r -t p -a resnet101 --bottleneck-dim 1024 -r -rd 51200 --epochs 30 -i 2500 -p 500 --seed 0 --log logs/cdan/DomainNet_r2p
CUDA_VISIBLE_DEVICES=0 python cdan.py data/domainnet -d DomainNet -s r -t s -a resnet101 --bottleneck-dim 1024 -r -rd 51200 --epochs 30 -i 2500 -p 500 --seed 0 --log logs/cdan/DomainNet_r2s
CUDA_VISIBLE_DEVICES=0 python cdan.py data/domainnet -d DomainNet -s s -t c -a resnet101 --bottleneck-dim 1024 -r -rd 51200 --epochs 30 -i 2500 -p 500 --seed 0 --log logs/cdan/DomainNet_s2c
CUDA_VISIBLE_DEVICES=0 python cdan.py data/domainnet -d DomainNet -s s -t p -a resnet101 --bottleneck-dim 1024 -r -rd 51200 --epochs 30 -i 2500 -p 500 --seed 0 --log logs/cdan/DomainNet_s2p
CUDA_VISIBLE_DEVICES=0 python cdan.py data/domainnet -d DomainNet -s s -t r -a resnet101 --bottleneck-dim 1024 -r -rd 51200 --epochs 30 -i 2500 -p 500 --seed 0 --log logs/cdan/DomainNet_s2r
# ResNet50, ImageNet200 -> ImageNetR
CUDA_VISIBLE_DEVICES=0 python cdan.py data/ImageNetR -d ImageNetR -s IN -t INR -a resnet50 --epochs 30 -i 2500 -p 500 --seed 0 --log logs/cdan/ImageNet_IN2INR
# ig_resnext101_32x8d, ImageNet -> ImageNetSketch
CUDA_VISIBLE_DEVICES=0 python cdan.py data/imagenet-sketch -d ImageNetSketch -s IN -t sketch -a ig_resnext101_32x8d --bottleneck-dim 1024 --epochs 30 -i 2500 -p 500 --log logs/cdan_ig_resnext101_32x8d/ImageNet_IN2sketch
# Vision Transformer, Office-Home, Single Source
CUDA_VISIBLE_DEVICES=0 python cdan.py data/office-home -d OfficeHome -s Ar -t Cl -a vit_base_patch16_224 --epochs 30 --seed 0 -b 24 --no-pool --log logs/cdan_vit/OfficeHome_Ar2Cl
CUDA_VISIBLE_DEVICES=0 python cdan.py data/office-home -d OfficeHome -s Ar -t Pr -a vit_base_patch16_224 --epochs 30 --seed 0 -b 24 --no-pool --log logs/cdan_vit/OfficeHome_Ar2Pr
CUDA_VISIBLE_DEVICES=0 python cdan.py data/office-home -d OfficeHome -s Ar -t Rw -a vit_base_patch16_224 --epochs 30 --seed 0 -b 24 --no-pool --log logs/cdan_vit/OfficeHome_Ar2Rw
CUDA_VISIBLE_DEVICES=0 python cdan.py data/office-home -d OfficeHome -s Cl -t Ar -a vit_base_patch16_224 --epochs 30 --seed 0 -b 24 --no-pool --log logs/cdan_vit/OfficeHome_Cl2Ar
CUDA_VISIBLE_DEVICES=0 python cdan.py data/office-home -d OfficeHome -s Cl -t Pr -a vit_base_patch16_224 --epochs 30 --seed 0 -b 24 --no-pool --log logs/cdan_vit/OfficeHome_Cl2Pr
CUDA_VISIBLE_DEVICES=0 python cdan.py data/office-home -d OfficeHome -s Cl -t Rw -a vit_base_patch16_224 --epochs 30 --seed 0 -b 24 --no-pool --log logs/cdan_vit/OfficeHome_Cl2Rw
CUDA_VISIBLE_DEVICES=0 python cdan.py data/office-home -d OfficeHome -s Pr -t Ar -a vit_base_patch16_224 --epochs 30 --seed 0 -b 24 --no-pool --log logs/cdan_vit/OfficeHome_Pr2Ar
CUDA_VISIBLE_DEVICES=0 python cdan.py data/office-home -d OfficeHome -s Pr -t Cl -a vit_base_patch16_224 --epochs 30 --seed 0 -b 24 --no-pool --log logs/cdan_vit/OfficeHome_Pr2Cl
CUDA_VISIBLE_DEVICES=0 python cdan.py data/office-home -d OfficeHome -s Pr -t Rw -a vit_base_patch16_224 --epochs 30 --seed 0 -b 24 --no-pool --log logs/cdan_vit/OfficeHome_Pr2Rw
CUDA_VISIBLE_DEVICES=0 python cdan.py data/office-home -d OfficeHome -s Rw -t Ar -a vit_base_patch16_224 --epochs 30 --seed 0 -b 24 --no-pool --log logs/cdan_vit/OfficeHome_Rw2Ar
CUDA_VISIBLE_DEVICES=0 python cdan.py data/office-home -d OfficeHome -s Rw -t Cl -a vit_base_patch16_224 --epochs 30 --seed 0 -b 24 --no-pool --log logs/cdan_vit/OfficeHome_Rw2Cl
CUDA_VISIBLE_DEVICES=0 python cdan.py data/office-home -d OfficeHome -s Rw -t Pr -a vit_base_patch16_224 --epochs 30 --seed 0 -b 24 --no-pool --log logs/cdan_vit/OfficeHome_Rw2Pr
# ResNet50, Office-Home, Multi Source
CUDA_VISIBLE_DEVICES=0 python cdan.py data/office-home -d OfficeHome -s Cl Pr Rw -t Ar -a resnet50 --epochs 30 --seed 0 --log logs/cdan/OfficeHome_:2Ar
CUDA_VISIBLE_DEVICES=0 python cdan.py data/office-home -d OfficeHome -s Ar Pr Rw -t Cl -a resnet50 --epochs 30 --seed 0 --log logs/cdan/OfficeHome_:2Cl
CUDA_VISIBLE_DEVICES=0 python cdan.py data/office-home -d OfficeHome -s Ar Cl Rw -t Pr -a resnet50 --epochs 30 --seed 0 --log logs/cdan/OfficeHome_:2Pr
CUDA_VISIBLE_DEVICES=0 python cdan.py data/office-home -d OfficeHome -s Ar Cl Pr -t Rw -a resnet50 --epochs 30 --seed 0 --log logs/cdan/OfficeHome_:2Rw
# ResNet101, DomainNet, Multi Source
CUDA_VISIBLE_DEVICES=0 python cdan.py data/domainnet -d DomainNet -s i p q r s -t c -a resnet101 --bottleneck-dim 1024 -r -rd 51200 --epochs 40 -i 5000 -p 500 --seed 0 --log logs/cdan/DomainNet_:2c
CUDA_VISIBLE_DEVICES=0 python cdan.py data/domainnet -d DomainNet -s c p q r s -t i -a resnet101 --bottleneck-dim 1024 -r -rd 51200 --epochs 40 -i 5000 -p 500 --seed 0 --log logs/cdan/DomainNet_:2i
CUDA_VISIBLE_DEVICES=0 python cdan.py data/domainnet -d DomainNet -s c i q r s -t p -a resnet101 --bottleneck-dim 1024 -r -rd 51200 --epochs 40 -i 5000 -p 500 --seed 0 --log logs/cdan/DomainNet_:2p
CUDA_VISIBLE_DEVICES=0 python cdan.py data/domainnet -d DomainNet -s c i p r s -t q -a resnet101 --bottleneck-dim 1024 -r -rd 51200 --epochs 40 -i 5000 -p 500 --seed 0 --log logs/cdan/DomainNet_:2q
CUDA_VISIBLE_DEVICES=0 python cdan.py data/domainnet -d DomainNet -s c i p q s -t r -a resnet101 --bottleneck-dim 1024 -r -rd 51200 --epochs 40 -i 5000 -p 500 --seed 0 --log logs/cdan/DomainNet_:2r
CUDA_VISIBLE_DEVICES=0 python cdan.py data/domainnet -d DomainNet -s c i p q r -t s -a resnet101 --bottleneck-dim 1024 -r -rd 51200 --epochs 40 -i 5000 -p 500 --seed 0 --log logs/cdan/DomainNet_:2s
# ResNet50, Wilds Dataset
CUDA_VISIBLE_DEVICES=0 python cdan.py data/wilds -d iwildcam -a resnet50 --epochs 30 -i 1000 --seed 0 --log logs/cdan/iwildcam
# DenseNet121, Wilds Dataset
CUDA_VISIBLE_DEVICES=0 python cdan.py data/wilds -d camelyon17 --train-resizing 'res.' --val-resizing 'res.' --resize-size 96 \
-a densenet121 --scratch --epochs 10 -i 1000 --lr 0.01 --seed 0 --log logs/cdan/camelyon17
CUDA_VISIBLE_DEVICES=0 python cdan.py data/wilds -d fmow --train-resizing 'res.' --val-resizing 'res.' \
-a densenet121 --epochs 10 -i 1000 --lr 0.01 --seed 0 --log logs/cdan/fmow
# Digits
CUDA_VISIBLE_DEVICES=0 python cdan.py data/digits -d Digits -s MNIST -t USPS --train-resizing 'res.' --val-resizing 'res.' \
--resize-size 28 --no-hflip --norm-mean 0.5 --norm-std 0.5 -a lenet --no-pool --lr 0.01 -b 128 -i 2500 --scratch --seed 0 --log logs/cdan/MNIST2USPS
CUDA_VISIBLE_DEVICES=0 python cdan.py data/digits -d Digits -s USPS -t MNIST --train-resizing 'res.' --val-resizing 'res.' \
--resize-size 28 --no-hflip --norm-mean 0.5 --norm-std 0.5 -a lenet --no-pool --lr 0.1 -b 128 -i 2500 --scratch --seed 0 --log logs/cdan/USPS2MNIST
CUDA_VISIBLE_DEVICES=0 python cdan.py data/digits -d Digits -s SVHNRGB -t MNISTRGB --train-resizing 'res.' --val-resizing 'res.' \
--resize-size 32 --no-hflip --norm-mean 0.5 0.5 0.5 --norm-std 0.5 0.5 0.5 -a dtn --no-pool --lr 0.01 -b 128 -i 2500 --scratch --seed 0 --log logs/cdan/SVHN2MNIST
================================================
FILE: domain_generalization/Transfer-Learning-Library/examples/domain_adaptation/image_classification/dan.py
================================================
"""
@author: Junguang Jiang
@contact: JiangJunguang1123@outlook.com
"""
import random
import time
import warnings
import sys
import argparse
import shutil
import os.path as osp
import torch
import torch.nn as nn
import torch.backends.cudnn as cudnn
from torch.optim import SGD
from torch.optim.lr_scheduler import LambdaLR
from torch.utils.data import DataLoader
import torch.nn.functional as F
sys.path.append('../../..')
from dalib.adaptation.dan import MultipleKernelMaximumMeanDiscrepancy, ImageClassifier
from dalib.modules.kernels import GaussianKernel
from common.utils.data import ForeverDataIterator
from common.utils.metric import accuracy
from common.utils.meter import AverageMeter, ProgressMeter
from common.utils.logger import CompleteLogger
from common.utils.analysis import collect_feature, tsne, a_distance
sys.path.append('.')
import utils
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
class TransformerDecorator(torch.nn.Module):
def __init__(self, backbone=None, add_bt=3, dim=2048, eval_global=0):
super(TransformerDecorator, self).__init__()
self.encoder_layers = torch.nn.TransformerEncoderLayer(dim, 4, dim, 0.5)
self.eval_global = eval_global
self.add_bt = add_bt
self.backbone = backbone
def forward_feats(self, feature):
feature = self.backbone(feature)
if self.training and self.add_bt > 0:
pre_feature = feature
feature = feature.unsqueeze(1)
feature = self.encoder_layers(feature)
# feature = self.encoder_layers(feature, feature, feature)
feature = feature.squeeze(1)
if self.add_bt >= 1:
return torch.cat([pre_feature, feature], dim=0)
return feature
def forward(self, feature):
feature = self.forward_feats(feature)
return feature
def main(args: argparse.Namespace):
logger = CompleteLogger(args.log, args.phase)
print(args)
if args.seed is not None:
random.seed(args.seed)
torch.manual_seed(args.seed)
cudnn.deterministic = True
warnings.warn('You have chosen to seed training. '
'This will turn on the CUDNN deterministic setting, '
'which can slow down your training considerably! '
'You may see unexpected behavior when restarting '
'from checkpoints.')
cudnn.benchmark = True
# Data loading code
train_transform = utils.get_train_transform(args.train_resizing, random_horizontal_flip=not args.no_hflip,
random_color_jitter=False, resize_size=args.resize_size,
norm_mean=args.norm_mean, norm_std=args.norm_std)
val_transform = utils.get_val_transform(args.val_resizing, resize_size=args.resize_size,
norm_mean=args.norm_mean, norm_std=args.norm_std)
print("train_transform: ", train_transform)
print("val_transform: ", val_transform)
train_source_dataset, train_target_dataset, val_dataset, test_dataset, num_classes, args.class_names = \
utils.get_dataset(args.data, args.root, args.source, args.target, train_transform, val_transform)
train_source_loader = DataLoader(train_source_dataset, batch_size=args.batch_size,
shuffle=True, num_workers=args.workers, drop_last=True)
train_target_loader = DataLoader(train_target_dataset, batch_size=args.batch_size,
shuffle=True, num_workers=args.workers, drop_last=True)
val_loader = DataLoader(val_dataset, batch_size=args.batch_size, shuffle=False, num_workers=args.workers)
test_loader = DataLoader(test_dataset, batch_size=args.batch_size, shuffle=False, num_workers=args.workers)
train_source_iter = ForeverDataIterator(train_source_loader)
train_target_iter = ForeverDataIterator(train_target_loader)
# create model
print("=> using model '{}'".format(args.arch))
backbone = utils.get_model(args.arch, pretrain=not args.scratch)
# pool_layer = nn.Identity() if args.no_pool else None
pool_layer = nn.Sequential(
nn.AdaptiveAvgPool2d(output_size=(1, 1)),
nn.Flatten(),
)
pool_layer = TransformerDecorator(pool_layer, add_bt=args.add_bt, dim=backbone.out_features)
classifier = ImageClassifier(backbone, num_classes, bottleneck_dim=args.bottleneck_dim,
pool_layer=pool_layer, finetune=not args.scratch).to(device)
# define optimizer and lr scheduler
optimizer = SGD(classifier.get_parameters(), args.lr, momentum=args.momentum, weight_decay=args.wd, nesterov=True)
lr_scheduler = LambdaLR(optimizer, lambda x: args.lr * (1. + args.lr_gamma * float(x)) ** (-args.lr_decay))
# define loss function
mkmmd_loss = MultipleKernelMaximumMeanDiscrepancy(
kernels=[GaussianKernel(alpha=2 ** k) for k in range(-3, 2)],
linear=not args.non_linear
)
# resume from the best checkpoint
if args.phase != 'train':
checkpoint = torch.load(logger.get_checkpoint_path('best'), map_location='cpu')
classifier.load_state_dict(checkpoint)
# analysis the model
if args.phase == 'analysis':
# extract features from both domains
feature_extractor = nn.Sequential(classifier.backbone, classifier.pool_layer, classifier.bottleneck).to(device)
source_feature = collect_feature(train_source_loader, feature_extractor, device)
target_feature = collect_feature(train_target_loader, feature_extractor, device)
# plot t-SNE
tSNE_filename = osp.join(logger.visualize_directory, 'TSNE.pdf')
tsne.visualize(source_feature, target_feature, tSNE_filename)
print("Saving t-SNE to", tSNE_filename)
# calculate A-distance, which is a measure for distribution discrepancy
A_distance = a_distance.calculate(source_feature, target_feature, device)
print("A-distance =", A_distance)
return
if args.phase == 'test':
acc1 = utils.validate(test_loader, classifier, args, device)
print(acc1)
return
# start training
best_acc1 = 0.
for epoch in range(args.epochs):
# train for one epoch
train(train_source_iter, train_target_iter, classifier, mkmmd_loss, optimizer,
lr_scheduler, epoch, args)
# evaluate on validation set
acc1 = utils.validate(val_loader, classifier, args, device)
# remember best acc@1 and save checkpoint
torch.save(classifier.state_dict(), logger.get_checkpoint_path('latest'))
if acc1 > best_acc1:
shutil.copy(logger.get_checkpoint_path('latest'), logger.get_checkpoint_path('best'))
best_acc1 = max(acc1, best_acc1)
print("best_acc1 = {:3.1f}".format(best_acc1))
# evaluate on test set
classifier.load_state_dict(torch.load(logger.get_checkpoint_path('best')))
acc1 = utils.validate(test_loader, classifier, args, device)
print("test_acc1 = {:3.1f}".format(acc1))
logger.close()
def train(train_source_iter: ForeverDataIterator, train_target_iter: ForeverDataIterator, model: ImageClassifier,
mkmmd_loss: MultipleKernelMaximumMeanDiscrepancy, optimizer: SGD,
lr_scheduler: LambdaLR, epoch: int, args: argparse.Namespace):
batch_time = AverageMeter('Time', ':4.2f')
data_time = AverageMeter('Data', ':3.1f')
losses = AverageMeter('Loss', ':3.2f')
trans_losses = AverageMeter('Trans Loss', ':5.4f')
cls_accs = AverageMeter('Cls Acc', ':3.1f')
tgt_accs = AverageMeter('Tgt Acc', ':3.1f')
progress = ProgressMeter(
args.iters_per_epoch,
[batch_time, data_time, losses, trans_losses, cls_accs, tgt_accs],
prefix="Epoch: [{}]".format(epoch))
# switch to train mode
model.train()
mkmmd_loss.train()
end = time.time()
for i in range(args.iters_per_epoch):
x_s, labels_s = next(train_source_iter)
x_t, labels_t = next(train_target_iter)
x_s = x_s.to(device)
x_t = x_t.to(device)
labels_s = labels_s.to(device)
labels_t = labels_t.to(device)
# measure data loading time
data_time.update(time.time() - end)
# compute output
y_s, f_s = model(x_s)
y_t, f_t = model(x_t)
if len(y_s) > len(labels_s):
labels_s = torch.cat([labels_s, labels_s], dim=0)
if len(y_t) > len(labels_t):
labels_t = torch.cat([labels_t, labels_t], dim=0)
cls_loss = F.cross_entropy(y_s, labels_s)
transfer_loss = mkmmd_loss(f_s, f_t)
loss = cls_loss + transfer_loss * args.trade_off
cls_acc = accuracy(y_s, labels_s)[0]
tgt_acc = accuracy(y_t, labels_t)[0]
losses.update(loss.item(), x_s.size(0))
cls_accs.update(cls_acc.item(), x_s.size(0))
tgt_accs.update(tgt_acc.item(), x_t.size(0))
trans_losses.update(transfer_loss.item(), x_s.size(0))
# compute gradient and do SGD step
optimizer.zero_grad()
loss.backward()
optimizer.step()
lr_scheduler.step()
# measure elapsed time
batch_time.update(time.time() - end)
end = time.time()
if i % args.print_freq == 0:
progress.display(i)
if __name__ == '__main__':
parser = argparse.ArgumentParser(description='DAN for Unsupervised Domain Adaptation')
# dataset parameters
parser.add_argument('root', metavar='DIR',
help='root path of dataset')
parser.add_argument('-d', '--data', metavar='DATA', default='Office31', choices=utils.get_dataset_names(),
help='dataset: ' + ' | '.join(utils.get_dataset_names()) +
' (default: Office31)')
parser.add_argument('-s', '--source', help='source domain(s)', nargs='+')
parser.add_argument('-t', '--target', help='target domain(s)', nargs='+')
parser.add_argument('--train-resizing', type=str, default='default')
parser.add_argument('--val-resizing', type=str, default='default')
parser.add_argument('--resize-size', type=int, default=224,
help='the image size after resizing')
parser.add_argument('--no-hflip', action='store_true',
help='no random horizontal flipping during training')
parser.add_argument('--norm-mean', type=float, nargs='+',
default=(0.485, 0.456, 0.406), help='normalization mean')
parser.add_argument('--norm-std', type=float, nargs='+',
default=(0.229, 0.224, 0.225), help='normalization std')
# model parameters
parser.add_argument('-a', '--arch', metavar='ARCH', default='resnet18',
choices=utils.get_model_names(),
help='backbone architecture: ' +
' | '.join(utils.get_model_names()) +
' (default: resnet18)')
parser.add_argument('--bottleneck-dim', default=256, type=int,
help='Dimension of bottleneck')
parser.add_argument('--no-pool', action='store_true',
help='no pool layer after the feature extractor.')
parser.add_argument('--scratch', action='store_true', help='whether train from scratch.')
parser.add_argument('--non-linear', default=False, action='store_true',
help='whether not use the linear version')
parser.add_argument('--trade-off', default=1., type=float,
help='the trade-off hyper-parameter for transfer loss')
# training parameters
parser.add_argument('-b', '--batch-size', default=32, type=int,
metavar='N',
help='mini-batch size (default: 32)')
parser.add_argument('--lr', '--learning-rate', default=0.003, type=float,
metavar='LR', help='initial learning rate', dest='lr')
parser.add_argument('--lr-gamma', default=0.0003, type=float, help='parameter for lr scheduler')
parser.add_argument('--lr-decay', default=0.75, type=float, help='parameter for lr scheduler')
parser.add_argument('--momentum', default=0.9, type=float, metavar='M',
help='momentum')
parser.add_argument('--wd', '--weight-decay', default=0.0005, type=float,
metavar='W', help='weight decay (default: 5e-4)')
parser.add_argument('-j', '--workers', default=2, type=int, metavar='N',
help='number of data loading workers (default: 2)')
parser.add_argument('--epochs', default=20, type=int, metavar='N',
help='number of total epochs to run')
parser.add_argument('-i', '--iters-per-epoch', default=500, type=int,
help='Number of iterations per epoch')
parser.add_argument('-p', '--print-freq', default=100, type=int,
metavar='N', help='print frequency (default: 100)')
parser.add_argument('--seed', default=None, type=int,
help='seed for initializing training. ')
parser.add_argument('--per-class-eval', action='store_true',
help='whether output per-class accuracy during evaluation')
parser.add_argument("--log", type=str, default='dan',
help="Where to save logs, checkpoints and debugging images.")
parser.add_argument("--phase", type=str, default='train', choices=['train', 'test', 'analysis'],
help="When phase is 'test', only test the model."
"When phase is 'analysis', only analysis the model.")
parser.add_argument("--add_bt", type=int, default=0,
help="add_bt")
args = parser.parse_args()
main(args)
================================================
FILE: domain_generalization/Transfer-Learning-Library/examples/domain_adaptation/image_classification/dan.sh
================================================
#!/usr/bin/env bash
# ResNet50, Office31, Single Source
CUDA_VISIBLE_DEVICES=0 python dan.py data/office31 -d Office31 -s D -t A -a resnet50 --epochs 20 --seed 0 --log logs/dan/Office31_D2A
CUDA_VISIBLE_DEVICES=0 python dan.py data/office31 -d Office31 -s W -t A -a resnet50 --epochs 20 --seed 0 --log logs/dan/Office31_W2A
CUDA_VISIBLE_DEVICES=0 python dan.py data/office31 -d Office31 -s A -t W -a resnet50 --epochs 20 --seed 0 --log logs/dan/Office31_A2W
CUDA_VISIBLE_DEVICES=0 python dan.py data/office31 -d Office31 -s A -t D -a resnet50 --epochs 20 --seed 0 --log logs/dan/Office31_A2D
CUDA_VISIBLE_DEVICES=0 python dan.py data/office31 -d Office31 -s D -t W -a resnet50 --epochs 20 --seed 0 --log logs/dan/Office31_D2W
CUDA_VISIBLE_DEVICES=0 python dan.py data/office31 -d Office31 -s W -t D -a resnet50 --epochs 20 --seed 0 --log logs/dan/Office31_W2D
# ResNet50, Office-Home, Single Source
CUDA_VISIBLE_DEVICES=0 python dan.py data/office-home -d OfficeHome -s Ar -t Cl -a resnet50 --epochs 20 -i 500 --seed 0 --log logs/dan/OfficeHome_Ar2Cl
CUDA_VISIBLE_DEVICES=0 python dan.py data/office-home -d OfficeHome -s Ar -t Pr -a resnet50 --epochs 20 -i 500 --seed 0 --log logs/dan/OfficeHome_Ar2Pr
CUDA_VISIBLE_DEVICES=0 python dan.py data/office-home -d OfficeHome -s Ar -t Rw -a resnet50 --epochs 20 -i 500 --seed 0 --log logs/dan/OfficeHome_Ar2Rw
CUDA_VISIBLE_DEVICES=0 python dan.py data/office-home -d OfficeHome -s Cl -t Ar -a resnet50 --epochs 20 -i 500 --seed 0 --log logs/dan/OfficeHome_Cl2Ar
CUDA_VISIBLE_DEVICES=0 python dan.py data/office-home -d OfficeHome -s Cl -t Pr -a resnet50 --epochs 20 -i 500 --seed 0 --log logs/dan/OfficeHome_Cl2Pr
CUDA_VISIBLE_DEVICES=0 python dan.py data/office-home -d OfficeHome -s Cl -t Rw -a resnet50 --epochs 20 -i 500 --seed 0 --log logs/dan/OfficeHome_Cl2Rw
CUDA_VISIBLE_DEVICES=0 python dan.py data/office-home -d OfficeHome -s Pr -t Ar -a resnet50 --epochs 20 -i 500 --seed 0 --log logs/dan/OfficeHome_Pr2Ar
CUDA_VISIBLE_DEVICES=0 python dan.py data/office-home -d OfficeHome -s Pr -t Cl -a resnet50 --epochs 20 -i 500 --seed 0 --log logs/dan/OfficeHome_Pr2Cl
CUDA_VISIBLE_DEVICES=0 python dan.py data/office-home -d OfficeHome -s Pr -t Rw -a resnet50 --epochs 20 -i 500 --seed 0 --log logs/dan/OfficeHome_Pr2Rw
CUDA_VISIBLE_DEVICES=0 python dan.py data/office-home -d OfficeHome -s Rw -t Ar -a resnet50 --epochs 20 -i 500 --seed 0 --log logs/dan/OfficeHome_Rw2Ar
CUDA_VISIBLE_DEVICES=0 python dan.py data/office-home -d OfficeHome -s Rw -t Cl -a resnet50 --epochs 20 -i 500 --seed 0 --log logs/dan/OfficeHome_Rw2Cl
CUDA_VISIBLE_DEVICES=0 python dan.py data/office-home -d OfficeHome -s Rw -t Pr -a resnet50 --epochs 20 -i 500 --seed 0 --log logs/dan/OfficeHome_Rw2Pr
# ResNet101, VisDA-2017, Single Source
CUDA_VISIBLE_DEVICES=0 python dan.py data/visda-2017 -d VisDA2017 -s Synthetic -t Real -a resnet101 \
--epochs 20 -i 500 --seed 0 --per-class-eval --train-resizing cen.crop --log logs/dan/VisDA2017
# ResNet101, DomainNet, Single Source
CUDA_VISIBLE_DEVICES=0 python dan.py data/domainnet -d DomainNet -s c -t p -a resnet101 --bottleneck-dim 1024 --epochs 30 -i 2500 -p 500 --seed 0 --log logs/dan/DomainNet_c2p
CUDA_VISIBLE_DEVICES=0 python dan.py data/domainnet -d DomainNet -s c -t r -a resnet101 --bottleneck-dim 1024 --epochs 30 -i 2500 -p 500 --seed 0 --log logs/dan/DomainNet_c2r
CUDA_VISIBLE_DEVICES=0 python dan.py data/domainnet -d DomainNet -s c -t s -a resnet101 --bottleneck-dim 1024 --epochs 30 -i 2500 -p 500 --seed 0 --log logs/dan/DomainNet_c2s
CUDA_VISIBLE_DEVICES=0 python dan.py data/domainnet -d DomainNet -s p -t c -a resnet101 --bottleneck-dim 1024 --epochs 30 -i 2500 -p 500 --seed 0 --log logs/dan/DomainNet_p2c
CUDA_VISIBLE_DEVICES=0 python dan.py data/domainnet -d DomainNet -s p -t r -a resnet101 --bottleneck-dim 1024 --epochs 30 -i 2500 -p 500 --seed 0 --log logs/dan/DomainNet_p2r
CUDA_VISIBLE_DEVICES=0 python dan.py data/domainnet -d DomainNet -s p -t s -a resnet101 --bottleneck-dim 1024 --epochs 30 -i 2500 -p 500 --seed 0 --log logs/dan/DomainNet_p2s
CUDA_VISIBLE_DEVICES=0 python dan.py data/domainnet -d DomainNet -s r -t c -a resnet101 --bottleneck-dim 1024 --epochs 30 -i 2500 -p 500 --seed 0 --log logs/dan/DomainNet_r2c
CUDA_VISIBLE_DEVICES=0 python dan.py data/domainnet -d DomainNet -s r -t p -a resnet101 --bottleneck-dim 1024 --epochs 30 -i 2500 -p 500 --seed 0 --log logs/dan/DomainNet_r2p
CUDA_VISIBLE_DEVICES=0 python dan.py data/domainnet -d DomainNet -s r -t s -a resnet101 --bottleneck-dim 1024 --epochs 30 -i 2500 -p 500 --seed 0 --log logs/dan/DomainNet_r2s
CUDA_VISIBLE_DEVICES=0 python dan.py data/domainnet -d DomainNet -s s -t c -a resnet101 --bottleneck-dim 1024 --epochs 30 -i 2500 -p 500 --seed 0 --log logs/dan/DomainNet_s2c
CUDA_VISIBLE_DEVICES=0 python dan.py data/domainnet -d DomainNet -s s -t p -a resnet101 --bottleneck-dim 1024 --epochs 30 -i 2500 -p 500 --seed 0 --log logs/dan/DomainNet_s2p
CUDA_VISIBLE_DEVICES=0 python dan.py data/domainnet -d DomainNet -s s -t r -a resnet101 --bottleneck-dim 1024 --epochs 30 -i 2500 -p 500 --seed 0 --log logs/dan/DomainNet_s2r
# ResNet50, ImageNet200 -> ImageNetR
CUDA_VISIBLE_DEVICES=0 python dan.py data/ImageNetR -d ImageNetR -s IN -t INR -a resnet50 --epochs 30 -i 2500 -p 500 --seed 0 --log logs/dan/ImageNet_IN2INR
# ig_resnext101_32x8d, ImageNet -> ImageNetSketch
CUDA_VISIBLE_DEVICES=0 python dan.py data/imagenet-sketch -d ImageNetSketch -s IN -t sketch -a ig_resnext101_32x8d --bottleneck-dim 1024 --epochs 30 -i 2500 -p 500 --log logs/dan_ig_resnext101_32x8d/ImageNet_IN2sketch
# Vision Transformer, Office-Home, Single Source
CUDA_VISIBLE_DEVICES=0 python dan.py data/office-home -d OfficeHome -s Ar -t Cl -a vit_base_patch16_224 --no-pool --epochs 30 --seed 0 -b 24 --log logs/dan_vit/OfficeHome_Ar2Cl
CUDA_VISIBLE_DEVICES=0 python dan.py data/office-home -d OfficeHome -s Ar -t Pr -a vit_base_patch16_224 --no-pool --epochs 30 --seed 0 -b 24 --log logs/dan_vit/OfficeHome_Ar2Pr
CUDA_VISIBLE_DEVICES=0 python dan.py data/office-home -d OfficeHome -s Ar -t Rw -a vit_base_patch16_224 --no-pool --epochs 30 --seed 0 -b 24 --log logs/dan_vit/OfficeHome_Ar2Rw
CUDA_VISIBLE_DEVICES=0 python dan.py data/office-home -d OfficeHome -s Cl -t Ar -a vit_base_patch16_224 --no-pool --epochs 30 --seed 0 -b 24 --log logs/dan_vit/OfficeHome_Cl2Ar
CUDA_VISIBLE_DEVICES=0 python dan.py data/office-home -d OfficeHome -s Cl -t Pr -a vit_base_patch16_224 --no-pool --epochs 30 --seed 0 -b 24 --log logs/dan_vit/OfficeHome_Cl2Pr
CUDA_VISIBLE_DEVICES=0 python dan.py data/office-home -d OfficeHome -s Cl -t Rw -a vit_base_patch16_224 --no-pool --epochs 30 --seed 0 -b 24 --log logs/dan_vit/OfficeHome_Cl2Rw
CUDA_VISIBLE_DEVICES=0 python dan.py data/office-home -d OfficeHome -s Pr -t Ar -a vit_base_patch16_224 --no-pool --epochs 30 --seed 0 -b 24 --log logs/dan_vit/OfficeHome_Pr2Ar
CUDA_VISIBLE_DEVICES=0 python dan.py data/office-home -d OfficeHome -s Pr -t Cl -a vit_base_patch16_224 --no-pool --epochs 30 --seed 0 -b 24 --log logs/dan_vit/OfficeHome_Pr2Cl
CUDA_VISIBLE_DEVICES=0 python dan.py data/office-home -d OfficeHome -s Pr -t Rw -a vit_base_patch16_224 --no-pool --epochs 30 --seed 0 -b 24 --log logs/dan_vit/OfficeHome_Pr2Rw
CUDA_VISIBLE_DEVICES=0 python dan.py data/office-home -d OfficeHome -s Rw -t Ar -a vit_base_patch16_224 --no-pool --epochs 30 --seed 0 -b 24 --log logs/dan_vit/OfficeHome_Rw2Ar
CUDA_VISIBLE_DEVICES=0 python dan.py data/office-home -d OfficeHome -s Rw -t Cl -a vit_base_patch16_224 --no-pool --epochs 30 --seed 0 -b 24 --log logs/dan_vit/OfficeHome_Rw2Cl
CUDA_VISIBLE_DEVICES=0 python dan.py data/office-home -d OfficeHome -s Rw -t Pr -a vit_base_patch16_224 --no-pool --epochs 30 --seed 0 -b 24 --log logs/dan_vit/OfficeHome_Rw2Pr
# ResNet50, Office-Home, Multi Source
CUDA_VISIBLE_DEVICES=0 python dan.py data/office-home -d OfficeHome -s Cl Pr Rw -t Ar -a resnet50 --epochs 30 --seed 0 --log logs/dan/OfficeHome_:2Ar
CUDA_VISIBLE_DEVICES=0 python dan.py data/office-home -d OfficeHome -s Ar Pr Rw -t Cl -a resnet50 --epochs 30 --seed 0 --log logs/dan/OfficeHome_:2Cl
CUDA_VISIBLE_DEVICES=0 python dan.py data/office-home -d OfficeHome -s Ar Cl Rw -t Pr -a resnet50 --epochs 30 --seed 0 --log logs/dan/OfficeHome_:2Pr
CUDA_VISIBLE_DEVICES=0 python dan.py data/office-home -d OfficeHome -s Ar Cl Pr -t Rw -a resnet50 --epochs 30 --seed 0 --log logs/dan/OfficeHome_:2Rw
# ResNet101, DomainNet, Multi Source
CUDA_VISIBLE_DEVICES=0 python dan.py data/domainnet -d DomainNet -s i p q r s -t c -a resnet101 --bottleneck-dim 1024 --epochs 40 -i 5000 -p 500 --seed 0 --log logs/dan/DomainNet_:2c
CUDA_VISIBLE_DEVICES=0 python dan.py data/domainnet -d DomainNet -s c p q r s -t i -a resnet101 --bottleneck-dim 1024 --epochs 40 -i 5000 -p 500 --seed 0 --log logs/dan/DomainNet_:2i
CUDA_VISIBLE_DEVICES=0 python dan.py data/domainnet -d DomainNet -s c i q r s -t p -a resnet101 --bottleneck-dim 1024 --epochs 40 -i 5000 -p 500 --seed 0 --log logs/dan/DomainNet_:2p
CUDA_VISIBLE_DEVICES=0 python dan.py data/domainnet -d DomainNet -s c i p r s -t q -a resnet101 --bottleneck-dim 1024 --epochs 40 -i 5000 -p 500 --seed 0 --log logs/dan/DomainNet_:2q
CUDA_VISIBLE_DEVICES=0 python dan.py data/domainnet -d DomainNet -s c i p q s -t r -a resnet101 --bottleneck-dim 1024 --epochs 40 -i 5000 -p 500 --seed 0 --log logs/dan/DomainNet_:2r
CUDA_VISIBLE_DEVICES=0 python dan.py data/domainnet -d DomainNet -s c i p q r -t s -a resnet101 --bottleneck-dim 1024 --epochs 40 -i 5000 -p 500 --seed 0 --log logs/dan/DomainNet_:2s
# ResNet50, Wilds Dataset
CUDA_VISIBLE_DEVICES=0 python dan.py data/wilds -d iwildcam -a resnet50 --epochs 30 -i 1000 --seed 0 --log logs/dan/iwildcam
# DenseNet121, Wilds Dataset
CUDA_VISIBLE_DEVICES=0 python dan.py data/wilds -d camelyon17 --train-resizing 'res.' --val-resizing 'res.' --resize-size 96 \
-a densenet121 --scratch --epochs 10 -i 1000 --lr 0.01 --seed 0 --log logs/dan/camelyon17
CUDA_VISIBLE_DEVICES=0 python dan.py data/wilds -d fmow --train-resizing 'res.' --val-resizing 'res.' \
-a densenet121 --epochs 10 -i 1000 --lr 0.01 --seed 0 --log logs/dan/fmow
# Digits
CUDA_VISIBLE_DEVICES=0 python dan.py data/digits -d Digits -s MNIST -t USPS --train-resizing 'res.' --val-resizing 'res.' \
--resize-size 28 --no-hflip --norm-mean 0.5 --norm-std 0.5 -a lenet --no-pool --lr 0.01 -b 128 -i 2500 --scratch --seed 0 --log logs/dan/MNIST2USPS
CUDA_VISIBLE_DEVICES=0 python dan.py data/digits -d Digits -s USPS -t MNIST --train-resizing 'res.' --val-resizing 'res.' \
--resize-size 28 --no-hflip --norm-mean 0.5 --norm-std 0.5 -a lenet --no-pool --lr 0.1 -b 128 -i 2500 --scratch --seed 0 --log logs/dan/USPS2MNIST
CUDA_VISIBLE_DEVICES=0 python dan.py data/digits -d Digits -s SVHNRGB -t MNISTRGB --train-resizing 'res.' --val-resizing 'res.' \
--resize-size 32 --no-hflip --norm-mean 0.5 0.5 0.5 --norm-std 0.5 0.5 0.5 -a dtn --no-pool --lr 0.01 -b 128 -i 2500 --scratch --seed 0 --log logs/dan/SVHN2MNIST
================================================
FILE: domain_generalization/Transfer-Learning-Library/examples/domain_adaptation/image_classification/dann.py
================================================
"""
@author: Junguang Jiang
@contact: JiangJunguang1123@outlook.com
"""
import random
import time
import warnings
import sys
import argparse
import shutil
import os.path as osp
import torch
import torch.nn as nn
import torch.backends.cudnn as cudnn
from torch.optim import SGD
from torch.optim.lr_scheduler import LambdaLR
from torch.utils.data import DataLoader
import torch.nn.functional as F
sys.path.append('../../..')
from dalib.modules.domain_discriminator import DomainDiscriminator
from dalib.adaptation.dann import DomainAdversarialLoss, ImageClassifier
from common.utils.data import ForeverDataIterator
from common.utils.metric import accuracy
from common.utils.meter import AverageMeter, ProgressMeter
from common.utils.logger import CompleteLogger
from common.utils.analysis import collect_feature, tsne, a_distance
sys.path.append('.')
import utils
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
class TransformerDecorator(torch.nn.Module):
def __init__(self, backbone=None, add_bt=3, dim=2048, eval_global=0):
super(TransformerDecorator, self).__init__()
self.encoder_layers = torch.nn.TransformerEncoderLayer(dim, 4, dim, 0.5)
self.eval_global = eval_global
self.add_bt = add_bt
self.backbone = backbone
def forward_feats(self, feature):
feature = self.backbone(feature)
if self.training and self.add_bt > 0:
pre_feature = feature
feature = feature.unsqueeze(1)
feature = self.encoder_layers(feature)
# feature = self.encoder_layers(feature, feature, feature)
feature = feature.squeeze(1)
if self.add_bt >= 1:
return torch.cat([pre_feature, feature], dim=0)
return feature
def forward(self, feature):
feature = self.forward_feats(feature)
return feature
def main(args: argparse.Namespace):
logger = CompleteLogger(args.log, args.phase)
print(args)
if args.seed is not None:
random.seed(args.seed)
torch.manual_seed(args.seed)
cudnn.deterministic = True
warnings.warn('You have chosen to seed training. '
'This will turn on the CUDNN deterministic setting, '
'which can slow down your training considerably! '
'You may see unexpected behavior when restarting '
'from checkpoints.')
cudnn.benchmark = True
# Data loading code
train_transform = utils.get_train_transform(args.train_resizing, random_horizontal_flip=not args.no_hflip,
random_color_jitter=False, resize_size=args.resize_size,
norm_mean=args.norm_mean, norm_std=args.norm_std)
val_transform = utils.get_val_transform(args.val_resizing, resize_size=args.resize_size,
norm_mean=args.norm_mean, norm_std=args.norm_std)
print("train_transform: ", train_transform)
print("val_transform: ", val_transform)
train_source_dataset, train_target_dataset, val_dataset, test_dataset, num_classes, args.class_names = \
utils.get_dataset(args.data, args.root, args.source, args.target, train_transform, val_transform)
train_source_loader = DataLoader(train_source_dataset, batch_size=args.batch_size,
shuffle=True, num_workers=args.workers, drop_last=True)
train_target_loader = DataLoader(train_target_dataset, batch_size=args.batch_size,
shuffle=True, num_workers=args.workers, drop_last=True)
val_loader = DataLoader(val_dataset, batch_size=args.batch_size, shuffle=False, num_workers=args.workers)
test_loader = DataLoader(test_dataset, batch_size=args.batch_size, shuffle=False, num_workers=args.workers)
train_source_iter = ForeverDataIterator(train_source_loader)
train_target_iter = ForeverDataIterator(train_target_loader)
# create model
print("=> using model '{}'".format(args.arch))
backbone = utils.get_model(args.arch, pretrain=not args.scratch)
# pool_layer = nn.Identity() if args.no_pool else None
pool_layer = nn.Sequential(
nn.AdaptiveAvgPool2d(output_size=(1, 1)),
nn.Flatten(),
)
pool_layer = TransformerDecorator(pool_layer, add_bt=args.add_bt, dim=backbone.out_features)
classifier = ImageClassifier(backbone, num_classes, bottleneck_dim=args.bottleneck_dim,
pool_layer=pool_layer, finetune=not args.scratch).to(device)
domain_discri = DomainDiscriminator(in_feature=classifier.features_dim, hidden_size=1024).to(device)
# define optimizer and lr scheduler
optimizer = SGD(classifier.get_parameters() + domain_discri.get_parameters(),
args.lr, momentum=args.momentum, weight_decay=args.weight_decay, nesterov=True)
lr_scheduler = LambdaLR(optimizer, lambda x: args.lr * (1. + args.lr_gamma * float(x)) ** (-args.lr_decay))
# define loss function
domain_adv = DomainAdversarialLoss(domain_discri).to(device)
# resume from the best checkpoint
if args.phase != 'train':
checkpoint = torch.load(logger.get_checkpoint_path('best'), map_location='cpu')
classifier.load_state_dict(checkpoint)
# analysis the model
if args.phase == 'analysis':
# extract features from both domains
feature_extractor = nn.Sequential(classifier.backbone, classifier.pool_layer, classifier.bottleneck).to(device)
source_feature = collect_feature(train_source_loader, feature_extractor, device)
target_feature = collect_feature(train_target_loader, feature_extractor, device)
# plot t-SNE
tSNE_filename = osp.join(logger.visualize_directory, 'TSNE.pdf')
tsne.visualize(source_feature, target_feature, tSNE_filename)
print("Saving t-SNE to", tSNE_filename)
# calculate A-distance, which is a measure for distribution discrepancy
A_distance = a_distance.calculate(source_feature, target_feature, device)
print("A-distance =", A_distance)
return
if args.phase == 'test':
acc1 = utils.validate(test_loader, classifier, args, device)
print(acc1)
return
# start training
best_acc1 = 0.
for epoch in range(args.epochs):
print("lr:", lr_scheduler.get_last_lr()[0])
# train for one epoch
train(train_source_iter, train_target_iter, classifier, domain_adv, optimizer,
lr_scheduler, epoch, args)
# evaluate on validation set
acc1 = utils.validate(val_loader, classifier, args, device)
# remember best acc@1 and save checkpoint
torch.save(classifier.state_dict(), logger.get_checkpoint_path('latest'))
if acc1 > best_acc1:
shutil.copy(logger.get_checkpoint_path('latest'), logger.get_checkpoint_path('best'))
best_acc1 = max(acc1, best_acc1)
print("best_acc1 = {:3.1f}".format(best_acc1))
# evaluate on test set
classifier.load_state_dict(torch.load(logger.get_checkpoint_path('best')))
acc1 = utils.validate(test_loader, classifier, args, device)
print("test_acc1 = {:3.1f}".format(acc1))
logger.close()
def train(train_source_iter: ForeverDataIterator, train_target_iter: ForeverDataIterator,
model: ImageClassifier, domain_adv: DomainAdversarialLoss, optimizer: SGD,
lr_scheduler: LambdaLR, epoch: int, args: argparse.Namespace):
batch_time = AverageMeter('Time', ':5.2f')
data_time = AverageMeter('Data', ':5.2f')
losses = AverageMeter('Loss', ':6.2f')
cls_accs = AverageMeter('Cls Acc', ':3.1f')
domain_accs = AverageMeter('Domain Acc', ':3.1f')
progress = ProgressMeter(
args.iters_per_epoch,
[batch_time, data_time, losses, cls_accs, domain_accs],
prefix="Epoch: [{}]".format(epoch))
# switch to train mode
model.train()
domain_adv.train()
end = time.time()
for i in range(args.iters_per_epoch):
x_s, labels_s = next(train_source_iter)
x_t, _ = next(train_target_iter)
x_s = x_s.to(device)
x_t = x_t.to(device)
labels_s = labels_s.to(device)
# measure data loading time
data_time.update(time.time() - end)
# compute output
x = torch.cat((x_s, x_t), dim=0)
y, f = model(x)
if len(y) > len(x):
y_s, y_t, y_s1, y_t1 = y.chunk(4, dim=0)
y_s = torch.cat([y_s, y_s1], dim=0)
y_t = torch.cat([y_t, y_t1], dim=0)
f_s, f_t, f_s1, f_t1 = f.chunk(4, dim=0)
f_s = torch.cat([f_s, f_s1], dim=0)
f_t = torch.cat([f_t, f_t1], dim=0)
else:
y_s, y_t = y.chunk(2, dim=0)
f_s, f_t = f.chunk(2, dim=0)
if len(y_s) > len(labels_s):
labels_s = torch.cat([labels_s, labels_s], dim=0)
cls_loss = F.cross_entropy(y_s, labels_s)
transfer_loss = domain_adv(f_s, f_t)
domain_acc = domain_adv.domain_discriminator_accuracy
loss = cls_loss + transfer_loss * args.trade_off
cls_acc = accuracy(y_s, labels_s)[0]
losses.update(loss.item(), x_s.size(0))
cls_accs.update(cls_acc.item(), x_s.size(0))
domain_accs.update(domain_acc.item(), x_s.size(0))
# compute gradient and do SGD step
optimizer.zero_grad()
loss.backward()
optimizer.step()
lr_scheduler.step()
# measure elapsed time
batch_time.update(time.time() - end)
end = time.time()
if i % args.print_freq == 0:
progress.display(i)
if __name__ == '__main__':
parser = argparse.ArgumentParser(description='DANN for Unsupervised Domain Adaptation')
# dataset parameters
parser.add_argument('root', metavar='DIR',
help='root path of dataset')
parser.add_argument('-d', '--data', metavar='DATA', default='Office31', choices=utils.get_dataset_names(),
help='dataset: ' + ' | '.join(utils.get_dataset_names()) +
' (default: Office31)')
parser.add_argument('-s', '--source', help='source domain(s)', nargs='+')
parser.add_argument('-t', '--target', help='target domain(s)', nargs='+')
parser.add_argument('--train-resizing', type=str, default='default')
parser.add_argument('--val-resizing', type=str, default='default')
parser.add_argument('--resize-size', type=int, default=224,
help='the image size after resizing')
parser.add_argument('--no-hflip', action='store_true',
help='no random horizontal flipping during training')
parser.add_argument('--norm-mean', type=float, nargs='+',
default=(0.485, 0.456, 0.406), help='normalization mean')
parser.add_argument('--norm-std', type=float, nargs='+',
default=(0.229, 0.224, 0.225), help='normalization std')
# model parameters
parser.add_argument('-a', '--arch', metavar='ARCH', default='resnet18',
choices=utils.get_model_names(),
help='backbone architecture: ' +
' | '.join(utils.get_model_names()) +
' (default: resnet18)')
parser.add_argument('--bottleneck-dim', default=256, type=int,
help='Dimension of bottleneck')
parser.add_argument('--no-pool', action='store_true',
help='no pool layer after the feature extractor.')
parser.add_argument('--scratch', action='store_true', help='whether train from scratch.')
parser.add_argument('--trade-off', default=1., type=float,
help='the trade-off hyper-parameter for transfer loss')
# training parameters
parser.add_argument('-b', '--batch-size', default=32, type=int,
metavar='N',
help='mini-batch size (default: 32)')
parser.add_argument('--lr', '--learning-rate', default=0.01, type=float,
metavar='LR', help='initial learning rate', dest='lr')
parser.add_argument('--lr-gamma', default=0.001, type=float, help='parameter for lr scheduler')
parser.add_argument('--lr-decay', default=0.75, type=float, help='parameter for lr scheduler')
parser.add_argument('--momentum', default=0.9, type=float, metavar='M',
help='momentum')
parser.add_argument('--wd', '--weight-decay',default=1e-3, type=float,
metavar='W', help='weight decay (default: 1e-3)',
dest='weight_decay')
parser.add_argument('-j', '--workers', default=2, type=int, metavar='N',
help='number of data loading workers (default: 2)')
parser.add_argument('--epochs', default=20, type=int, metavar='N',
help='number of total epochs to run')
parser.add_argument('-i', '--iters-per-epoch', default=1000, type=int,
help='Number of iterations per epoch')
parser.add_argument('-p', '--print-freq', default=100, type=int,
metavar='N', help='print frequency (default: 100)')
parser.add_argument('--seed', default=None, type=int,
help='seed for initializing training. ')
parser.add_argument('--per-class-eval', action='store_true',
help='whether output per-class accuracy during evaluation')
parser.add_argument("--log", type=str, default='dann',
help="Where to save logs, checkpoints and debugging images.")
parser.add_argument("--phase", type=str, default='train', choices=['train', 'test', 'analysis'],
help="When phase is 'test', only test the model."
"When phase is 'analysis', only analysis the model.")
parser.add_argument("--add_bt", type=int, default=0,
help="add_bt")
args = parser.parse_args()
main(args)
================================================
FILE: domain_generalization/Transfer-Learning-Library/examples/domain_adaptation/image_classification/dann.sh
================================================
#!/usr/bin/env bash
# ResNet50, Office31, Single Source
CUDA_VISIBLE_DEVICES=0 python dann.py data/office31 -d Office31 -s A -t W -a resnet50 --epochs 20 --seed 1 --log logs/dann/Office31_A2W
CUDA_VISIBLE_DEVICES=0 python dann.py data/office31 -d Office31 -s D -t W -a resnet50 --epochs 20 --seed 1 --log logs/dann/Office31_D2W
CUDA_VISIBLE_DEVICES=0 python dann.py data/office31 -d Office31 -s W -t D -a resnet50 --epochs 20 --seed 1 --log logs/dann/Office31_W2D
CUDA_VISIBLE_DEVICES=0 python dann.py data/office31 -d Office31 -s A -t D -a resnet50 --epochs 20 --seed 1 --log logs/dann/Office31_A2D
CUDA_VISIBLE_DEVICES=0 python dann.py data/office31 -d Office31 -s D -t A -a resnet50 --epochs 20 --seed 1 --log logs/dann/Office31_D2A
CUDA_VISIBLE_DEVICES=0 python dann.py data/office31 -d Office31 -s W -t A -a resnet50 --epochs 20 --seed 1 --log logs/dann/Office31_W2A
# ResNet50, Office-Home, Single Source
CUDA_VISIBLE_DEVICES=0 python dann.py data/office-home -d OfficeHome -s Ar -t Cl -a resnet50 --epochs 30 --seed 0 --log logs/dann/OfficeHome_Ar2Cl
CUDA_VISIBLE_DEVICES=0 python dann.py data/office-home -d OfficeHome -s Ar -t Pr -a resnet50 --epochs 30 --seed 0 --log logs/dann/OfficeHome_Ar2Pr
CUDA_VISIBLE_DEVICES=0 python dann.py data/office-home -d OfficeHome -s Ar -t Rw -a resnet50 --epochs 30 --seed 0 --log logs/dann/OfficeHome_Ar2Rw
CUDA_VISIBLE_DEVICES=0 python dann.py data/office-home -d OfficeHome -s Cl -t Ar -a resnet50 --epochs 30 --seed 0 --log logs/dann/OfficeHome_Cl2Ar
CUDA_VISIBLE_DEVICES=0 python dann.py data/office-home -d OfficeHome -s Cl -t Pr -a resnet50 --epochs 30 --seed 0 --log logs/dann/OfficeHome_Cl2Pr
CUDA_VISIBLE_DEVICES=0 python dann.py data/office-home -d OfficeHome -s Cl -t Rw -a resnet50 --epochs 30 --seed 0 --log logs/dann/OfficeHome_Cl2Rw
CUDA_VISIBLE_DEVICES=0 python dann.py data/office-home -d OfficeHome -s Pr -t Ar -a resnet50 --epochs 30 --seed 0 --log logs/dann/OfficeHome_Pr2Ar
CUDA_VISIBLE_DEVICES=0 python dann.py data/office-home -d OfficeHome -s Pr -t Cl -a resnet50 --epochs 30 --seed 0 --log logs/dann/OfficeHome_Pr2Cl
CUDA_VISIBLE_DEVICES=0 python dann.py data/office-home -d OfficeHome -s Pr -t Rw -a resnet50 --epochs 30 --seed 0 --log logs/dann/OfficeHome_Pr2Rw
CUDA_VISIBLE_DEVICES=0 python dann.py data/office-home -d OfficeHome -s Rw -t Ar -a resnet50 --epochs 30 --seed 0 --log logs/dann/OfficeHome_Rw2Ar
CUDA_VISIBLE_DEVICES=0 python dann.py data/office-home -d OfficeHome -s Rw -t Cl -a resnet50 --epochs 30 --seed 0 --log logs/dann/OfficeHome_Rw2Cl
CUDA_VISIBLE_DEVICES=0 python dann.py data/office-home -d OfficeHome -s Rw -t Pr -a resnet50 --epochs 30 --seed 0 --log logs/dann/OfficeHome_Rw2Pr
# ResNet101, VisDA-2017, Single Source
CUDA_VISIBLE_DEVICES=0 python dann.py data/visda-2017 -d VisDA2017 -s Synthetic -t Real -a resnet101 \
--epochs 30 --seed 0 --per-class-eval --train-resizing cen.crop --log logs/dann/VisDA2017
# ResNet101, DomainNet, Single Source
CUDA_VISIBLE_DEVICES=0 python dann.py data/domainnet -d DomainNet -s c -t p -a resnet101 --bottleneck-dim 1024 --epochs 30 -i 2500 -p 500 --seed 0 --log logs/dann/DomainNet_c2p
CUDA_VISIBLE_DEVICES=0 python dann.py data/domainnet -d DomainNet -s c -t r -a resnet101 --bottleneck-dim 1024 --epochs 30 -i 2500 -p 500 --seed 0 --log logs/dann/DomainNet_c2r
CUDA_VISIBLE_DEVICES=0 python dann.py data/domainnet -d DomainNet -s c -t s -a resnet101 --bottleneck-dim 1024 --epochs 30 -i 2500 -p 500 --seed 0 --log logs/dann/DomainNet_c2s
CUDA_VISIBLE_DEVICES=0 python dann.py data/domainnet -d DomainNet -s p -t c -a resnet101 --bottleneck-dim 1024 --epochs 30 -i 2500 -p 500 --seed 0 --log logs/dann/DomainNet_p2c
CUDA_VISIBLE_DEVICES=0 python dann.py data/domainnet -d DomainNet -s p -t r -a resnet101 --bottleneck-dim 1024 --epochs 30 -i 2500 -p 500 --seed 0 --log logs/dann/DomainNet_p2r
CUDA_VISIBLE_DEVICES=0 python dann.py data/domainnet -d DomainNet -s p -t s -a resnet101 --bottleneck-dim 1024 --epochs 30 -i 2500 -p 500 --seed 0 --log logs/dann/DomainNet_p2s
CUDA_VISIBLE_DEVICES=0 python dann.py data/domainnet -d DomainNet -s r -t c -a resnet101 --bottleneck-dim 1024 --epochs 30 -i 2500 -p 500 --seed 0 --log logs/dann/DomainNet_r2c
CUDA_VISIBLE_DEVICES=0 python dann.py data/domainnet -d DomainNet -s r -t p -a resnet101 --bottleneck-dim 1024 --epochs 30 -i 2500 -p 500 --seed 0 --log logs/dann/DomainNet_r2p
CUDA_VISIBLE_DEVICES=0 python dann.py data/domainnet -d DomainNet -s r -t s -a resnet101 --bottleneck-dim 1024 --epochs 30 -i 2500 -p 500 --seed 0 --log logs/dann/DomainNet_r2s
CUDA_VISIBLE_DEVICES=0 python dann.py data/domainnet -d DomainNet -s s -t c -a resnet101 --bottleneck-dim 1024 --epochs 30 -i 2500 -p 500 --seed 0 --log logs/dann/DomainNet_s2c
CUDA_VISIBLE_DEVICES=0 python dann.py data/domainnet -d DomainNet -s s -t p -a resnet101 --bottleneck-dim 1024 --epochs 30 -i 2500 -p 500 --seed 0 --log logs/dann/DomainNet_s2p
CUDA_VISIBLE_DEVICES=0 python dann.py data/domainnet -d DomainNet -s s -t r -a resnet101 --bottleneck-dim 1024 --epochs 30 -i 2500 -p 500 --seed 0 --log logs/dann/DomainNet_s2r
# ResNet50, ImageNet200 -> ImageNetR
CUDA_VISIBLE_DEVICES=0 python dann.py data/ImageNetR -d ImageNetR -s IN -t INR -a resnet50 --epochs 30 -i 2500 -p 500 --seed 0 --log logs/dann/ImageNet_IN2INR
# ig_resnext101_32x8d, ImageNet -> ImageNetSketch
CUDA_VISIBLE_DEVICES=0 python dann.py data/imagenet-sketch -d ImageNetSketch -s IN -t sketch -a ig_resnext101_32x8d --epochs 30 -i 2500 -p 500 --bottleneck-dim 1024 --log logs/dann_ig_resnext101_32x8d/ImageNet_IN2sketch
# Vision Transformer, Office-Home, Single Source
CUDA_VISIBLE_DEVICES=0 python dann.py data/office-home -d OfficeHome -s Ar -t Cl -a vit_base_patch16_224 --epochs 30 --seed 0 -b 24 --no-pool --log logs/dann_vit/OfficeHome_Ar2Cl
CUDA_VISIBLE_DEVICES=0 python dann.py data/office-home -d OfficeHome -s Ar -t Pr -a vit_base_patch16_224 --epochs 30 --seed 0 -b 24 --no-pool --log logs/dann_vit/OfficeHome_Ar2Pr
CUDA_VISIBLE_DEVICES=0 python dann.py data/office-home -d OfficeHome -s Ar -t Rw -a vit_base_patch16_224 --epochs 30 --seed 0 -b 24 --no-pool --log logs/dann_vit/OfficeHome_Ar2Rw
CUDA_VISIBLE_DEVICES=0 python dann.py data/office-home -d OfficeHome -s Cl -t Ar -a vit_base_patch16_224 --epochs 30 --seed 0 -b 24 --no-pool --log logs/dann_vit/OfficeHome_Cl2Ar
CUDA_VISIBLE_DEVICES=0 python dann.py data/office-home -d OfficeHome -s Cl -t Pr -a vit_base_patch16_224 --epochs 30 --seed 0 -b 24 --no-pool --log logs/dann_vit/OfficeHome_Cl2Pr
CUDA_VISIBLE_DEVICES=0 python dann.py data/office-home -d OfficeHome -s Cl -t Rw -a vit_base_patch16_224 --epochs 30 --seed 0 -b 24 --no-pool --log logs/dann_vit/OfficeHome_Cl2Rw
CUDA_VISIBLE_DEVICES=0 python dann.py data/office-home -d OfficeHome -s Pr -t Ar -a vit_base_patch16_224 --epochs 30 --seed 0 -b 24 --no-pool --log logs/dann_vit/OfficeHome_Pr2Ar
CUDA_VISIBLE_DEVICES=0 python dann.py data/office-home -d OfficeHome -s Pr -t Cl -a vit_base_patch16_224 --epochs 30 --seed 0 -b 24 --no-pool --log logs/dann_vit/OfficeHome_Pr2Cl
CUDA_VISIBLE_DEVICES=0 python dann.py data/office-home -d OfficeHome -s Pr -t Rw -a vit_base_patch16_224 --epochs 30 --seed 0 -b 24 --no-pool --log logs/dann_vit/OfficeHome_Pr2Rw
CUDA_VISIBLE_DEVICES=0 python dann.py data/office-home -d OfficeHome -s Rw -t Ar -a vit_base_patch16_224 --epochs 30 --seed 0 -b 24 --no-pool --log logs/dann_vit/OfficeHome_Rw2Ar
CUDA_VISIBLE_DEVICES=0 python dann.py data/office-home -d OfficeHome -s Rw -t Cl -a vit_base_patch16_224 --epochs 30 --seed 0 -b 24 --no-pool --log logs/dann_vit/OfficeHome_Rw2Cl
CUDA_VISIBLE_DEVICES=0 python dann.py data/office-home -d OfficeHome -s Rw -t Pr -a vit_base_patch16_224 --epochs 30 --seed 0 -b 24 --no-pool --log logs/dann_vit/OfficeHome_Rw2Pr
# ResNet50, Office-Home, Multi Source
CUDA_VISIBLE_DEVICES=0 python dann.py data/office-home -d OfficeHome -s Cl Pr Rw -t Ar -a resnet50 --epochs 30 --seed 0 --log logs/dann/OfficeHome_:2Ar
CUDA_VISIBLE_DEVICES=0 python dann.py data/office-home -d OfficeHome -s Ar Pr Rw -t Cl -a resnet50 --epochs 30 --seed 0 --log logs/dann/OfficeHome_:2Cl
CUDA_VISIBLE_DEVICES=0 python dann.py data/office-home -d OfficeHome -s Ar Cl Rw -t Pr -a resnet50 --epochs 30 --seed 0 --log logs/dann/OfficeHome_:2Pr
CUDA_VISIBLE_DEVICES=0 python dann.py data/office-home -d OfficeHome -s Ar Cl Pr -t Rw -a resnet50 --epochs 30 --seed 0 --log logs/dann/OfficeHome_:2Rw
# ResNet101, DomainNet, Multi Source
CUDA_VISIBLE_DEVICES=0 python dann.py data/domainnet -d DomainNet -s i p q r s -t c -a resnet101 --bottleneck-dim 1024 --epochs 40 -i 5000 -p 500 --seed 0 --log logs/dann/DomainNet_:2c
CUDA_VISIBLE_DEVICES=0 python dann.py data/domainnet -d DomainNet -s c p q r s -t i -a resnet101 --bottleneck-dim 1024 --epochs 40 -i 5000 -p 500 --seed 0 --log logs/dann/DomainNet_:2i
CUDA_VISIBLE_DEVICES=0 python dann.py data/domainnet -d DomainNet -s c i q r s -t p -a resnet101 --bottleneck-dim 1024 --epochs 40 -i 5000 -p 500 --seed 0 --log logs/dann/DomainNet_:2p
CUDA_VISIBLE_DEVICES=0 python dann.py data/domainnet -d DomainNet -s c i p r s -t q -a resnet101 --bottleneck-dim 1024 --epochs 40 -i 5000 -p 500 --seed 0 --log logs/dann/DomainNet_:2q
CUDA_VISIBLE_DEVICES=0 python dann.py data/domainnet -d DomainNet -s c i p q s -t r -a resnet101 --bottleneck-dim 1024 --epochs 40 -i 5000 -p 500 --seed 0 --log logs/dann/DomainNet_:2r
CUDA_VISIBLE_DEVICES=0 python dann.py data/domainnet -d DomainNet -s c i p q r -t s -a resnet101 --bottleneck-dim 1024 --epochs 40 -i 5000 -p 500 --seed 0 --log logs/dann/DomainNet_:2s
# ResNet50, Wilds Dataset
CUDA_VISIBLE_DEVICES=0 python dann.py data/wilds -d iwildcam -a resnet50 --epochs 30 -i 1000 --seed 0 --log logs/dann/iwildcam
# DenseNet121, Wilds Dataset
CUDA_VISIBLE_DEVICES=0 python dann.py data/wilds -d camelyon17 --train-resizing 'res.' --val-resizing 'res.' --resize-size 96 \
-a densenet121 --scratch --epochs 10 -i 1000 --lr 0.01 --seed 0 --log logs/dann/camelyon17
CUDA_VISIBLE_DEVICES=0 python dann.py data/wilds -d fmow --train-resizing 'res.' --val-resizing 'res.' \
-a densenet121 --epochs 10 -i 1000 --lr 0.01 --seed 0 --log logs/dann/fmow
# Digits
CUDA_VISIBLE_DEVICES=0 python dann.py data/digits -d Digits -s MNIST -t USPS --train-resizing 'res.' --val-resizing 'res.' \
--resize-size 28 --no-hflip --norm-mean 0.5 --norm-std 0.5 -a lenet --no-pool --lr 0.01 -b 128 -i 2500 --scratch --seed 0 --log logs/dann/MNIST2USPS
CUDA_VISIBLE_DEVICES=0 python dann.py data/digits -d Digits -s USPS -t MNIST --train-resizing 'res.' --val-resizing 'res.' \
--resize-size 28 --no-hflip --norm-mean 0.5 --norm-std 0.5 -a lenet --no-pool --lr 0.1 -b 128 -i 2500 --scratch --seed 0 --log logs/dann/USPS2MNIST
CUDA_VISIBLE_DEVICES=0 python dann.py data/digits -d Digits -s SVHNRGB -t MNISTRGB --train-resizing 'res.' --val-resizing 'res.' \
--resize-size 32 --no-hflip --norm-mean 0.5 0.5 0.5 --norm-std 0.5 0.5 0.5 -a dtn --no-pool --lr 0.03 -b 128 -i 2500 --scratch --seed 0 --log logs/dann/SVHN2MNIST
================================================
FILE: domain_generalization/Transfer-Learning-Library/examples/domain_adaptation/image_classification/gpu_dann.pbs
================================================
#!/bin/bash
#PBS -P ZHIHOU
#PBS -l select=1:ngpus=1:ncpus=6:mem=28GB
#PBS -l walltime=48:00:00
##PBS -q alloc-dt
source activate pyt
#env
cd "$PBS_O_WORKDIR"
#
## ResNet50, Office31, Single Source
#CUDA_VISIBLE_DEVICES=0 python dann.py data/office31 -d Office31 -s A -t W -a resnet50 --epochs 20 --seed 1 --log logs/dann/Office31_A2W_g16 --add_bt 16
#CUDA_VISIBLE_DEVICES=0 python dann.py data/office31 -d Office31 -s D -t W -a resnet50 --epochs 20 --seed 1 --log logs/dann/Office31_D2W_g16 --add_bt 16
#CUDA_VISIBLE_DEVICES=0 python dann.py data/office31 -d Office31 -s W -t D -a resnet50 --epochs 20 --seed 1 --log logs/dann/Office31_W2D_g16 --add_bt 16
#CUDA_VISIBLE_DEVICES=0 python dann.py data/office31 -d Office31 -s A -t D -a resnet50 --epochs 20 --seed 1 --log logs/dann/Office31_A2D_g16 --add_bt 16
#CUDA_VISIBLE_DEVICES=0 python dann.py data/office31 -d Office31 -s D -t A -a resnet50 --epochs 20 --seed 1 --log logs/dann/Office31_D2A_g16 --add_bt 16
#CUDA_VISIBLE_DEVICES=0 python dann.py data/office31 -d Office31 -s W -t A -a resnet50 --epochs 20 --seed 1 --log logs/dann/Office31_W2A_g16 --add_bt 16
#
## ResNet50, Office-Home, Single Source
#CUDA_VISIBLE_DEVICES=0 python dann.py data/office-home -d OfficeHome -s Ar -t Cl -a resnet50 --epochs 30 --seed 0 --log logs/dann/OfficeHome_Ar2Cl_g16 --add_bt 16
#CUDA_VISIBLE_DEVICES=0 python dann.py data/office-home -d OfficeHome -s Ar -t Pr -a resnet50 --epochs 30 --seed 0 --log logs/dann/OfficeHome_Ar2Pr_g16 --add_bt 16
#CUDA_VISIBLE_DEVICES=0 python dann.py data/office-home -d OfficeHome -s Ar -t Rw -a resnet50 --epochs 30 --seed 0 --log logs/dann/OfficeHome_Ar2Rw_g16 --add_bt 16
#CUDA_VISIBLE_DEVICES=0 python dann.py data/office-home -d OfficeHome -s Cl -t Ar -a resnet50 --epochs 30 --seed 0 --log logs/dann/OfficeHome_Cl2Ar_g16 --add_bt 16
#CUDA_VISIBLE_DEVICES=0 python dann.py data/office-home -d OfficeHome -s Cl -t Pr -a resnet50 --epochs 30 --seed 0 --log logs/dann/OfficeHome_Cl2Pr_g16 --add_bt 16
#CUDA_VISIBLE_DEVICES=0 python dann.py data/office-home -d OfficeHome -s Cl -t Rw -a resnet50 --epochs 30 --seed 0 --log logs/dann/OfficeHome_Cl2Rw_g16 --add_bt 16
CUDA_VISIBLE_DEVICES=0 python dann.py data/office-home -d OfficeHome -s Pr -t Ar -a resnet50 --epochs 30 --seed 0 --log logs/dann/OfficeHome_Pr2Ar_g16 --add_bt 16
CUDA_VISIBLE_DEVICES=0 python dann.py data/office-home -d OfficeHome -s Pr -t Cl -a resnet50 --epochs 30 --seed 0 --log logs/dann/OfficeHome_Pr2Cl_g16 --add_bt 16
CUDA_VISIBLE_DEVICES=0 python dann.py data/office-home -d OfficeHome -s Pr -t Rw -a resnet50 --epochs 30 --seed 0 --log logs/dann/OfficeHome_Pr2Rw_g16 --add_bt 16
CUDA_VISIBLE_DEVICES=0 python dann.py data/office-home -d OfficeHome -s Rw -t Ar -a resnet50 --epochs 30 --seed 0 --log logs/dann/OfficeHome_Rw2Ar_g16 --add_bt 16
CUDA_VISIBLE_DEVICES=0 python dann.py data/office-home -d OfficeHome -s Rw -t Cl -a resnet50 --epochs 30 --seed 0 --log logs/dann/OfficeHome_Rw2Cl_g16 --add_bt 16
CUDA_VISIBLE_DEVICES=0 python dann.py data/office-home -d OfficeHome -s Rw -t Pr -a resnet50 --epochs 30 --seed 0 --log logs/dann/OfficeHome_Rw2Pr_g16 --add_bt 16
##
### ResNet101, VisDA-2017, Single Source
#CUDA_VISIBLE_DEVICES=0 python dann.py data/visda-2017 -d VisDA2017 -s Synthetic -t Real -a resnet101 \
# --epochs 30 --seed 0 --per-class-eval --train-resizing cen.crop --log logs/dann/VisDA2017_g16 --add_bt 16
================================================
FILE: domain_generalization/Transfer-Learning-Library/examples/domain_adaptation/image_classification/gpu_mcc.pbs
================================================
#!/bin/bash
#PBS -P ZHIHOU
#PBS -l select=1:ngpus=1:ncpus=6:mem=28GB
#PBS -l walltime=48:00:00
#PBS -q alloc-dt
source activate pyt
#env
cd "$PBS_O_WORKDIR"
# ResNet50, Office31, Single Source
#CUDA_VISIBLE_DEVICES=0 python mcc.py data/office31 -d Office31 -s A -t W -a resnet18 --epochs 20 -i 500 --seed 1 --bottleneck-dim 1024 --log logs/mcc/Office31_A2W
#CUDA_VISIBLE_DEVICES=0 python mcc.py data/office31 -d Office31 -s D -t W -a resnet18 --epochs 20 -i 500 --seed 1 --bottleneck-dim 1024 --log logs/mcc/Office31_D2W
#CUDA_VISIBLE_DEVICES=0 python mcc.py data/office31 -d Office31 -s W -t D -a resnet18 --epochs 20 -i 500 --seed 1 --bottleneck-dim 1024 --log logs/mcc/Office31_W2D
#CUDA_VISIBLE_DEVICES=0 python mcc.py data/office31 -d Office31 -s A -t D -a resnet18 --epochs 20 -i 500 --seed 1 --bottleneck-dim 1024 --log logs/mcc/Office31_A2D
#CUDA_VISIBLE_DEVICES=0 python mcc.py data/office31 -d Office31 -s D -t A -a resnet18 --epochs 20 -i 500 --seed 1 --bottleneck-dim 1024 --log logs/mcc/Office31_D2A
#CUDA_VISIBLE_DEVICES=0 python mcc.py data/office31 -d Office31 -s W -t A -a resnet18 --epochs 20 -i 500 --seed 1 --bottleneck-dim 1024 --log logs/mcc/Office31_W2A
## ResNet50, Office-Home, Single Source
CUDA_VISIBLE_DEVICES=0 python mcc.py data/office-home -d OfficeHome -s Ar -t Cl -a resnet18 --epochs 30 --seed 2 --bottleneck-dim 2048 --log logs/mcc_r18/OfficeHome_Ar2Cl
CUDA_VISIBLE_DEVICES=0 python mcc.py data/office-home -d OfficeHome -s Ar -t Pr -a resnet18 --epochs 30 --seed 2 --bottleneck-dim 2048 --log logs/mcc_r18/OfficeHome_Ar2Pr
CUDA_VISIBLE_DEVICES=0 python mcc.py data/office-home -d OfficeHome -s Ar -t Rw -a resnet18 --epochs 30 --seed 2 --bottleneck-dim 2048 --log logs/mcc_r18/OfficeHome_Ar2Rw
CUDA_VISIBLE_DEVICES=0 python mcc.py data/office-home -d OfficeHome -s Cl -t Ar -a resnet18 --epochs 30 --seed 2 --bottleneck-dim 2048 --log logs/mcc_r18/OfficeHome_Cl2Ar
CUDA_VISIBLE_DEVICES=0 python mcc.py data/office-home -d OfficeHome -s Cl -t Pr -a resnet18 --epochs 30 --seed 2 --bottleneck-dim 2048 --log logs/mcc_r18/OfficeHome_Cl2Pr
CUDA_VISIBLE_DEVICES=0 python mcc.py data/office-home -d OfficeHome -s Cl -t Rw -a resnet18 --epochs 30 --seed 2 --bottleneck-dim 2048 --log logs/mcc_r18/OfficeHome_Cl2Rw
CUDA_VISIBLE_DEVICES=0 python mcc.py data/office-home -d OfficeHome -s Pr -t Ar -a resnet18 --epochs 30 --seed 2 --bottleneck-dim 2048 --log logs/mcc_r18/OfficeHome_Pr2Ar
CUDA_VISIBLE_DEVICES=0 python mcc.py data/office-home -d OfficeHome -s Pr -t Cl -a resnet18 --epochs 30 --seed 2 --bottleneck-dim 2048 --log logs/mcc_r18/OfficeHome_Pr2Cl
CUDA_VISIBLE_DEVICES=0 python mcc.py data/office-home -d OfficeHome -s Pr -t Rw -a resnet18 --epochs 30 --seed 2 --bottleneck-dim 2048 --log logs/mcc_r18/OfficeHome_Pr2Rw
CUDA_VISIBLE_DEVICES=0 python mcc.py data/office-home -d OfficeHome -s Rw -t Ar -a resnet18 --epochs 30 --seed 2 --bottleneck-dim 2048 --log logs/mcc_r18/OfficeHome_Rw2Ar
CUDA_VISIBLE_DEVICES=0 python mcc.py data/office-home -d OfficeHome -s Rw -t Cl -a resnet18 --epochs 30 --seed 2 --bottleneck-dim 2048 --log logs/mcc_r18/OfficeHome_Rw2Cl
CUDA_VISIBLE_DEVICES=0 python mcc.py data/office-home -d OfficeHome -s Rw -t Pr -a resnet18 --epochs 30 --seed 2 --bottleneck-dim 2048 --log logs/mcc_r18/OfficeHome_Rw2Pr
## ResNet101, VisDA-2017, Single Source
#CUDA_VISIBLE_DEVICES=5 python mcc.py data/visda-2017 -d VisDA2017 -s Synthetic -t Real -a resnet101 \
# --epochs 30 --seed 0 --lr 0.002 --per-class-eval --temperature 3.0 --train-resizing cen.crop --log logs/mcc/VisDA2017
================================================
FILE: domain_generalization/Transfer-Learning-Library/examples/domain_adaptation/image_classification/gpu_mdd.pbs
================================================
#!/bin/bash
#PBS -P ZHIHOU
#PBS -l select=1:ngpus=1:ncpus=6:mem=28GB
#PBS -l walltime=48:00:00
#PBS -q alloc-dt
source activate pyt
#env
cd "$PBS_O_WORKDIR"
# ResNet50, Office31, Single Source
#CUDA_VISIBLE_DEVICES=0 python mdd.py data/office31 -d Office31 -s A -t W -a resnet50 --epochs 20 --bottleneck-dim 1024 --seed 1 --log logs/mdd/Office31_A2W
#CUDA_VISIBLE_DEVICES=0 python mdd.py data/office31 -d Office31 -s D -t W -a resnet50 --epochs 20 --bottleneck-dim 1024 --seed 1 --log logs/mdd/Office31_D2W
#CUDA_VISIBLE_DEVICES=0 python mdd.py data/office31 -d Office31 -s W -t D -a resnet50 --epochs 20 --bottleneck-dim 1024 --seed 1 --log logs/mdd/Office31_W2D
#CUDA_VISIBLE_DEVICES=0 python mdd.py data/office31 -d Office31 -s A -t D -a resnet50 --epochs 20 --bottleneck-dim 1024 --seed 1 --log logs/mdd/Office31_A2D
#CUDA_VISIBLE_DEVICES=0 python mdd.py data/office31 -d Office31 -s D -t A -a resnet50 --epochs 20 --bottleneck-dim 1024 --seed 1 --log logs/mdd/Office31_D2A
#CUDA_VISIBLE_DEVICES=0 python mdd.py data/office31 -d Office31 -s W -t A -a resnet50 --epochs 20 --bottleneck-dim 1024 --seed 1 --log logs/mdd/Office31_W2A
#
## ResNet50, Office-Home, Single Source
#CUDA_VISIBLE_DEVICES=0 python mdd.py data/office-home -d OfficeHome -s Ar -t Cl -a resnet50 --epochs 30 --bottleneck-dim 2048 --seed 0 --log logs/mdd/OfficeHome_Ar2Cl_g16 --add_bt 16
#CUDA_VISIBLE_DEVICES=0 python mdd.py data/office-home -d OfficeHome -s Ar -t Pr -a resnet50 --epochs 30 --bottleneck-dim 2048 --seed 0 --log logs/mdd/OfficeHome_Ar2Pr_g16 --add_bt 16
#CUDA_VISIBLE_DEVICES=0 python mdd.py data/office-home -d OfficeHome -s Ar -t Rw -a resnet50 --epochs 30 --bottleneck-dim 2048 --seed 0 --log logs/mdd/OfficeHome_Ar2Rw_g16 --add_bt 16
#CUDA_VISIBLE_DEVICES=0 python mdd.py data/office-home -d OfficeHome -s Cl -t Ar -a resnet50 --epochs 30 --bottleneck-dim 2048 --seed 0 --log logs/mdd/OfficeHome_Cl2Ar_g16 --add_bt 16
#CUDA_VISIBLE_DEVICES=0 python mdd.py data/office-home -d OfficeHome -s Cl -t Pr -a resnet50 --epochs 30 --bottleneck-dim 2048 --seed 0 --log logs/mdd/OfficeHome_Cl2Pr_g16 --add_bt 16
#CUDA_VISIBLE_DEVICES=0 python mdd.py data/office-home -d OfficeHome -s Cl -t Rw -a resnet50 --epochs 30 --bottleneck-dim 2048 --seed 0 --log logs/mdd/OfficeHome_Cl2Rw_g16 --add_bt 16
#CUDA_VISIBLE_DEVICES=0 python mdd.py data/office-home -d OfficeHome -s Pr -t Ar -a resnet50 --epochs 30 --bottleneck-dim 2048 --seed 0 --log logs/mdd/OfficeHome_Pr2Ar_g16 --add_bt 16
#CUDA_VISIBLE_DEVICES=0 python mdd.py data/office-home -d OfficeHome -s Pr -t Cl -a resnet50 --epochs 30 --bottleneck-dim 2048 --seed 0 --log logs/mdd/OfficeHome_Pr2Cl_g16 --add_bt 16
#CUDA_VISIBLE_DEVICES=0 python mdd.py data/office-home -d OfficeHome -s Pr -t Rw -a resnet50 --epochs 30 --bottleneck-dim 2048 --seed 0 --log logs/mdd/OfficeHome_Pr2Rw_g16 --add_bt 16
#CUDA_VISIBLE_DEVICES=0 python mdd.py data/office-home -d OfficeHome -s Rw -t Ar -a resnet50 --epochs 30 --bottleneck-dim 2048 --seed 0 --log logs/mdd/OfficeHome_Rw2Ar_g16 --add_bt 16
#CUDA_VISIBLE_DEVICES=0 python mdd.py data/office-home -d OfficeHome -s Rw -t Cl -a resnet50 --epochs 30 --bottleneck-dim 2048 --seed 0 --log logs/mdd/OfficeHome_Rw2Cl_g16 --add_bt 16
#CUDA_VISIBLE_DEVICES=0 python mdd.py data/office-home -d OfficeHome -s Rw -t Pr -a resnet50 --epochs 30 --bottleneck-dim 2048 --seed 0 --log logs/mdd/OfficeHome_Rw2Pr_g16 --add_bt 16
#
## ResNet101, VisDA-2017, Single Source
CUDA_VISIBLE_DEVICES=0 python mdd.py data/visda-2017 -d VisDA2017 -s Synthetic -t Real -a resnet101 --epochs 30 \
--bottleneck-dim 1024 --seed 0 --train-resizing cen.crop --per-class-eval -b 36 --log logs/mdd/VisDA2017
CUDA_VISIBLE_DEVICES=0 python mdd.py data/visda-2017 -d VisDA2017 -s Synthetic -t Real -a resnet101 --epochs 30 \
--bottleneck-dim 1024 --seed 0 --train-resizing cen.crop --per-class-eval -b 36 --log logs/mdd/VisDA2017_g16 --add_bt 16
================================================
FILE: domain_generalization/Transfer-Learning-Library/examples/domain_adaptation/image_classification/jan.py
================================================
"""
@author: Junguang Jiang
@contact: JiangJunguang1123@outlook.com
"""
import random
import time
import warnings
import sys
import argparse
import shutil
import os.path as osp
import torch
import torch.nn as nn
import torch.backends.cudnn as cudnn
from torch.optim import SGD
from torch.optim.lr_scheduler import LambdaLR
from torch.utils.data import DataLoader
import torch.nn.functional as F
sys.path.append('../../..')
from dalib.adaptation.jan import JointMultipleKernelMaximumMeanDiscrepancy, ImageClassifier, Theta
from dalib.modules.kernels import GaussianKernel
from common.utils.data import ForeverDataIterator
from common.utils.metric import accuracy
from common.utils.meter import AverageMeter, ProgressMeter
from common.utils.logger import CompleteLogger
from common.utils.analysis import collect_feature, tsne, a_distance
sys.path.append('.')
import utils
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
def main(args: argparse.Namespace):
logger = CompleteLogger(args.log, args.phase)
print(args)
if args.seed is not None:
random.seed(args.seed)
torch.manual_seed(args.seed)
cudnn.deterministic = True
warnings.warn('You have chosen to seed training. '
'This will turn on the CUDNN deterministic setting, '
'which can slow down your training considerably! '
'You may see unexpected behavior when restarting '
'from checkpoints.')
cudnn.benchmark = True
# Data loading code
train_transform = utils.get_train_transform(args.train_resizing, random_horizontal_flip=not args.no_hflip,
random_color_jitter=False, resize_size=args.resize_size,
norm_mean=args.norm_mean, norm_std=args.norm_std)
val_transform = utils.get_val_transform(args.val_resizing, resize_size=args.resize_size,
norm_mean=args.norm_mean, norm_std=args.norm_std)
print("train_transform: ", train_transform)
print("val_transform: ", val_transform)
train_source_dataset, train_target_dataset, val_dataset, test_dataset, num_classes, args.class_names = \
utils.get_dataset(args.data, args.root, args.source, args.target, train_transform, val_transform)
train_source_loader = DataLoader(train_source_dataset, batch_size=args.batch_size,
shuffle=True, num_workers=args.workers, drop_last=True)
train_target_loader = DataLoader(train_target_dataset, batch_size=args.batch_size,
shuffle=True, num_workers=args.workers, drop_last=True)
val_loader = DataLoader(val_dataset, batch_size=args.batch_size, shuffle=False, num_workers=args.workers)
test_loader = DataLoader(test_dataset, batch_size=args.batch_size, shuffle=False, num_workers=args.workers)
train_source_iter = ForeverDataIterator(train_source_loader)
train_target_iter = ForeverDataIterator(train_target_loader)
# create model
print("=> using model '{}'".format(args.arch))
backbone = utils.get_model(args.arch, pretrain=not args.scratch)
pool_layer = nn.Identity() if args.no_pool else None
classifier = ImageClassifier(backbone, num_classes, bottleneck_dim=args.bottleneck_dim,
pool_layer=pool_layer, finetune=not args.scratch).to(device)
# define loss function
if args.adversarial:
thetas = [Theta(dim).to(device) for dim in (classifier.features_dim, num_classes)]
else:
thetas = None
jmmd_loss = JointMultipleKernelMaximumMeanDiscrepancy(
kernels=(
[GaussianKernel(alpha=2 ** k) for k in range(-3, 2)],
(GaussianKernel(sigma=0.92, track_running_stats=False),)
),
linear=args.linear, thetas=thetas
).to(device)
parameters = classifier.get_parameters()
if thetas is not None:
parameters += [{"params": theta.parameters(), 'lr': 0.1} for theta in thetas]
# define optimizer
optimizer = SGD(parameters, args.lr, momentum=args.momentum, weight_decay=args.wd, nesterov=True)
lr_scheduler = LambdaLR(optimizer, lambda x: args.lr * (1. + args.lr_gamma * float(x)) ** (-args.lr_decay))
# resume from the best checkpoint
if args.phase != 'train':
checkpoint = torch.load(logger.get_checkpoint_path('best'), map_location='cpu')
classifier.load_state_dict(checkpoint)
# analysis the model
if args.phase == 'analysis':
# extract features from both domains
feature_extractor = nn.Sequential(classifier.backbone, classifier.pool_layer, classifier.bottleneck).to(device)
source_feature = collect_feature(train_source_loader, feature_extractor, device)
target_feature = collect_feature(train_target_loader, feature_extractor, device)
# plot t-SNE
tSNE_filename = osp.join(logger.visualize_directory, 'TSNE.pdf')
tsne.visualize(source_feature, target_feature, tSNE_filename)
print("Saving t-SNE to", tSNE_filename)
# calculate A-distance, which is a measure for distribution discrepancy
A_distance = a_distance.calculate(source_feature, target_feature, device)
print("A-distance =", A_distance)
return
if args.phase == 'test':
acc1 = utils.validate(test_loader, classifier, args, device)
print(acc1)
return
# start training
best_acc1 = 0.
for epoch in range(args.epochs):
# train for one epoch
train(train_source_iter, train_target_iter, classifier, jmmd_loss, optimizer,
lr_scheduler, epoch, args)
# evaluate on validation set
acc1 = utils.validate(val_loader, classifier, args, device)
# remember best acc@1 and save checkpoint
torch.save(classifier.state_dict(), logger.get_checkpoint_path('latest'))
if acc1 > best_acc1:
shutil.copy(logger.get_checkpoint_path('latest'), logger.get_checkpoint_path('best'))
best_acc1 = max(acc1, best_acc1)
print("best_acc1 = {:3.1f}".format(best_acc1))
# evaluate on test set
classifier.load_state_dict(torch.load(logger.get_checkpoint_path('best')))
acc1 = utils.validate(test_loader, classifier, args, device)
print("test_acc1 = {:3.1f}".format(acc1))
logger.close()
def train(train_source_iter: ForeverDataIterator, train_target_iter: ForeverDataIterator, model: ImageClassifier,
jmmd_loss: JointMultipleKernelMaximumMeanDiscrepancy, optimizer: SGD,
lr_scheduler: LambdaLR, epoch: int, args: argparse.Namespace):
batch_time = AverageMeter('Time', ':4.2f')
data_time = AverageMeter('Data', ':3.1f')
losses = AverageMeter('Loss', ':3.2f')
trans_losses = AverageMeter('Trans Loss', ':5.4f')
cls_accs = AverageMeter('Cls Acc', ':3.1f')
tgt_accs = AverageMeter('Tgt Acc', ':3.1f')
progress = ProgressMeter(
args.iters_per_epoch,
[batch_time, data_time, losses, trans_losses, cls_accs, tgt_accs],
prefix="Epoch: [{}]".format(epoch))
# switch to train mode
model.train()
jmmd_loss.train()
end = time.time()
for i in range(args.iters_per_epoch):
x_s, labels_s = next(train_source_iter)
x_t, labels_t = next(train_target_iter)
x_s = x_s.to(device)
x_t = x_t.to(device)
labels_s = labels_s.to(device)
labels_t = labels_t.to(device)
# measure data loading time
data_time.update(time.time() - end)
# compute output
x = torch.cat((x_s, x_t), dim=0)
y, f = model(x)
y_s, y_t = y.chunk(2, dim=0)
f_s, f_t = f.chunk(2, dim=0)
cls_loss = F.cross_entropy(y_s, labels_s)
transfer_loss = jmmd_loss(
(f_s, F.softmax(y_s, dim=1)),
(f_t, F.softmax(y_t, dim=1))
)
loss = cls_loss + transfer_loss * args.trade_off
cls_acc = accuracy(y_s, labels_s)[0]
tgt_acc = accuracy(y_t, labels_t)[0]
losses.update(loss.item(), x_s.size(0))
cls_accs.update(cls_acc.item(), x_s.size(0))
tgt_accs.update(tgt_acc.item(), x_t.size(0))
trans_losses.update(transfer_loss.item(), x_s.size(0))
# compute gradient and do SGD step
optimizer.zero_grad()
loss.backward()
optimizer.step()
lr_scheduler.step()
# measure elapsed time
batch_time.update(time.time() - end)
end = time.time()
if i % args.print_freq == 0:
progress.display(i)
if __name__ == '__main__':
parser = argparse.ArgumentParser(description='JAN for Unsupervised Domain Adaptation')
# dataset parameters
parser.add_argument('root', metavar='DIR',
help='root path of dataset')
parser.add_argument('-d', '--data', metavar='DATA', default='Office31', choices=utils.get_dataset_names(),
help='dataset: ' + ' | '.join(utils.get_dataset_names()) +
' (default: Office31)')
parser.add_argument('-s', '--source', help='source domain(s)', nargs='+')
parser.add_argument('-t', '--target', help='target domain(s)', nargs='+')
parser.add_argument('--train-resizing', type=str, default='default')
parser.add_argument('--val-resizing', type=str, default='default')
parser.add_argument('--resize-size', type=int, default=224,
help='the image size after resizing')
parser.add_argument('--no-hflip', action='store_true',
help='no random horizontal flipping during training')
parser.add_argument('--norm-mean', type=float, nargs='+',
default=(0.485, 0.456, 0.406), help='normalization mean')
parser.add_argument('--norm-std', type=float, nargs='+',
default=(0.229, 0.224, 0.225), help='normalization std')
# model parameters
parser.add_argument('-a', '--arch', metavar='ARCH', default='resnet18',
choices=utils.get_model_names(),
help='backbone architecture: ' +
' | '.join(utils.get_model_names()) +
' (default: resnet18)')
parser.add_argument('--bottleneck-dim', default=256, type=int,
help='Dimension of bottleneck')
parser.add_argument('--no-pool', action='store_true',
help='no pool layer after the feature extractor.')
parser.add_argument('--scratch', action='store_true', help='whether train from scratch.')
parser.add_argument('--linear', default=False, action='store_true',
help='whether use the linear version')
parser.add_argument('--adversarial', default=False, action='store_true',
help='whether use adversarial theta')
parser.add_argument('--trade-off', default=1., type=float,
help='the trade-off hyper-parameter for transfer loss')
# training parameters
parser.add_argument('-b', '--batch-size', default=32, type=int,
metavar='N',
help='mini-batch size (default: 32)')
parser.add_argument('--lr', '--learning-rate', default=0.003, type=float,
metavar='LR', help='initial learning rate', dest='lr')
parser.add_argument('--lr-gamma', default=0.0003, type=float, help='parameter for lr scheduler')
parser.add_argument('--lr-decay', default=0.75, type=float, help='parameter for lr scheduler')
parser.add_argument('--momentum', default=0.9, type=float, metavar='M',
help='momentum')
parser.add_argument('--wd', '--weight-decay', default=0.0005, type=float,
metavar='W', help='weight decay (default: 5e-4)')
parser.add_argument('-j', '--workers', default=2, type=int, metavar='N',
help='number of data loading workers (default: 2)')
parser.add_argument('--epochs', default=20, type=int, metavar='N',
help='number of total epochs to run')
parser.add_argument('-i', '--iters-per-epoch', default=500, type=int,
help='Number of iterations per epoch')
parser.add_argument('-p', '--print-freq', default=100, type=int,
metavar='N', help='print frequency (default: 100)')
parser.add_argument('--seed', default=None, type=int,
help='seed for initializing training. ')
parser.add_argument('--per-class-eval', action='store_true',
help='whether output per-class accuracy during evaluation')
parser.add_argument("--log", type=str, default='jan',
help="Where to save logs, checkpoints and debugging images.")
parser.add_argument("--phase", type=str, default='train', choices=['train', 'test', 'analysis'],
help="When phase is 'test', only test the model."
"When phase is 'analysis', only analysis the model.")
args = parser.parse_args()
main(args)
================================================
FILE: domain_generalization/Transfer-Learning-Library/examples/domain_adaptation/image_classification/jan.sh
================================================
#!/usr/bin/env bash
# ResNet50, Office31, Single Source
CUDA_VISIBLE_DEVICES=0 python jan.py data/office31 -d Office31 -s D -t A -a resnet50 --epochs 20 --seed 0 --log logs/jan/Office31_D2A
CUDA_VISIBLE_DEVICES=0 python jan.py data/office31 -d Office31 -s W -t A -a resnet50 --epochs 20 --seed 0 --log logs/jan/Office31_W2A
CUDA_VISIBLE_DEVICES=0 python jan.py data/office31 -d Office31 -s A -t W -a resnet50 --epochs 20 --seed 0 --log logs/jan/Office31_A2W
CUDA_VISIBLE_DEVICES=0 python jan.py data/office31 -d Office31 -s A -t D -a resnet50 --epochs 20 --seed 0 --log logs/jan/Office31_A2D
CUDA_VISIBLE_DEVICES=0 python jan.py data/office31 -d Office31 -s D -t W -a resnet50 --epochs 20 --seed 0 --log logs/jan/Office31_D2W
CUDA_VISIBLE_DEVICES=0 python jan.py data/office31 -d Office31 -s W -t D -a resnet50 --epochs 20 --seed 0 --log logs/jan/Office31_W2D
# ResNet50, Office-Home, Single Source
CUDA_VISIBLE_DEVICES=0 python jan.py data/office-home -d OfficeHome -s Ar -t Cl -a resnet50 --epochs 20 --seed 0 --log logs/jan/OfficeHome_Ar2Cl
CUDA_VISIBLE_DEVICES=0 python jan.py data/office-home -d OfficeHome -s Ar -t Pr -a resnet50 --epochs 20 --seed 0 --log logs/jan/OfficeHome_Ar2Pr
CUDA_VISIBLE_DEVICES=0 python jan.py data/office-home -d OfficeHome -s Ar -t Rw -a resnet50 --epochs 20 --seed 0 --log logs/jan/OfficeHome_Ar2Rw
CUDA_VISIBLE_DEVICES=0 python jan.py data/office-home -d OfficeHome -s Cl -t Ar -a resnet50 --epochs 20 --seed 0 --log logs/jan/OfficeHome_Cl2Ar
CUDA_VISIBLE_DEVICES=0 python jan.py data/office-home -d OfficeHome -s Cl -t Pr -a resnet50 --epochs 20 --seed 0 --log logs/jan/OfficeHome_Cl2Pr
CUDA_VISIBLE_DEVICES=0 python jan.py data/office-home -d OfficeHome -s Cl -t Rw -a resnet50 --epochs 20 --seed 0 --log logs/jan/OfficeHome_Cl2Rw
CUDA_VISIBLE_DEVICES=0 python jan.py data/office-home -d OfficeHome -s Pr -t Ar -a resnet50 --epochs 20 --seed 0 --log logs/jan/OfficeHome_Pr2Ar
CUDA_VISIBLE_DEVICES=0 python jan.py data/office-home -d OfficeHome -s Pr -t Cl -a resnet50 --epochs 20 --seed 0 --log logs/jan/OfficeHome_Pr2Cl
CUDA_VISIBLE_DEVICES=0 python jan.py data/office-home -d OfficeHome -s Pr -t Rw -a resnet50 --epochs 20 --seed 0 --log logs/jan/OfficeHome_Pr2Rw
CUDA_VISIBLE_DEVICES=0 python jan.py data/office-home -d OfficeHome -s Rw -t Ar -a resnet50 --epochs 20 --seed 0 --log logs/jan/OfficeHome_Rw2Ar
CUDA_VISIBLE_DEVICES=0 python jan.py data/office-home -d OfficeHome -s Rw -t Cl -a resnet50 --epochs 20 --seed 0 --log logs/jan/OfficeHome_Rw2Cl
CUDA_VISIBLE_DEVICES=0 python jan.py data/office-home -d OfficeHome -s Rw -t Pr -a resnet50 --epochs 20 --seed 0 --log logs/jan/OfficeHome_Rw2Pr
# ResNet101, VisDA-2017, Single Source
CUDA_VISIBLE_DEVICES=0 python jan.py data/visda-2017 -d VisDA2017 -s Synthetic -t Real -a resnet101 \
--epochs 20 -i 500 --seed 0 --per-class-eval --train-resizing cen.crop --log logs/jan/VisDA2017
# ResNet101, DomainNet, Single Source
CUDA_VISIBLE_DEVICES=0 python jan.py data/domainnet -d DomainNet -s c -t p -a resnet101 --bottleneck-dim 1024 --epochs 30 -i 2500 -p 500 --seed 0 --log logs/jan/DomainNet_c2p
CUDA_VISIBLE_DEVICES=0 python jan.py data/domainnet -d DomainNet -s c -t r -a resnet101 --bottleneck-dim 1024 --epochs 30 -i 2500 -p 500 --seed 0 --log logs/jan/DomainNet_c2r
CUDA_VISIBLE_DEVICES=0 python jan.py data/domainnet -d DomainNet -s c -t s -a resnet101 --bottleneck-dim 1024 --epochs 30 -i 2500 -p 500 --seed 0 --log logs/jan/DomainNet_c2s
CUDA_VISIBLE_DEVICES=0 python jan.py data/domainnet -d DomainNet -s p -t c -a resnet101 --bottleneck-dim 1024 --epochs 30 -i 2500 -p 500 --seed 0 --log logs/jan/DomainNet_p2c
CUDA_VISIBLE_DEVICES=0 python jan.py data/domainnet -d DomainNet -s p -t r -a resnet101 --bottleneck-dim 1024 --epochs 30 -i 2500 -p 500 --seed 0 --log logs/jan/DomainNet_p2r
CUDA_VISIBLE_DEVICES=0 python jan.py data/domainnet -d DomainNet -s p -t s -a resnet101 --bottleneck-dim 1024 --epochs 30 -i 2500 -p 500 --seed 0 --log logs/jan/DomainNet_p2s
CUDA_VISIBLE_DEVICES=0 python jan.py data/domainnet -d DomainNet -s r -t c -a resnet101 --bottleneck-dim 1024 --epochs 30 -i 2500 -p 500 --seed 0 --log logs/jan/DomainNet_r2c
CUDA_VISIBLE_DEVICES=0 python jan.py data/domainnet -d DomainNet -s r -t p -a resnet101 --bottleneck-dim 1024 --epochs 30 -i 2500 -p 500 --seed 0 --log logs/jan/DomainNet_r2p
CUDA_VISIBLE_DEVICES=0 python jan.py data/domainnet -d DomainNet -s r -t s -a resnet101 --bottleneck-dim 1024 --epochs 30 -i 2500 -p 500 --seed 0 --log logs/jan/DomainNet_r2s
CUDA_VISIBLE_DEVICES=0 python jan.py data/domainnet -d DomainNet -s s -t c -a resnet101 --bottleneck-dim 1024 --epochs 30 -i 2500 -p 500 --seed 0 --log logs/jan/DomainNet_s2c
CUDA_VISIBLE_DEVICES=0 python jan.py data/domainnet -d DomainNet -s s -t p -a resnet101 --bottleneck-dim 1024 --epochs 30 -i 2500 -p 500 --seed 0 --log logs/jan/DomainNet_s2p
CUDA_VISIBLE_DEVICES=0 python jan.py data/domainnet -d DomainNet -s s -t r -a resnet101 --bottleneck-dim 1024 --epochs 30 -i 2500 -p 500 --seed 0 --log logs/jan/DomainNet_s2r
# ResNet50, ImageNet200 -> ImageNetR
CUDA_VISIBLE_DEVICES=0 python jan.py data/ImageNetR -d ImageNetR -s IN -t INR -a resnet50 --epochs 30 -i 2500 -p 500 --seed 0 --log logs/jan/ImageNet_IN2INR
# ig_resnext101_32x8d, ImageNet -> ImageNetSketch
CUDA_VISIBLE_DEVICES=0 python jan.py data/imagenet-sketch -d ImageNetSketch -s IN -t sketch -a ig_resnext101_32x8d --epochs 30 -i 2500 -p 500 --log logs/jan_ig_resnext101_32x8d/ImageNet_IN2sketch
# Vision Transformer, Office-Home, Single Source
CUDA_VISIBLE_DEVICES=0 python jan.py data/office-home -d OfficeHome -s Ar -t Cl -a vit_base_patch16_224 --no-pool --epochs 30 --seed 0 -b 24 --log logs/jan_vit/OfficeHome_Ar2Cl
CUDA_VISIBLE_DEVICES=0 python jan.py data/office-home -d OfficeHome -s Ar -t Pr -a vit_base_patch16_224 --no-pool --epochs 30 --seed 0 -b 24 --log logs/jan_vit/OfficeHome_Ar2Pr
CUDA_VISIBLE_DEVICES=0 python jan.py data/office-home -d OfficeHome -s Ar -t Rw -a vit_base_patch16_224 --no-pool --epochs 30 --seed 0 -b 24 --log logs/jan_vit/OfficeHome_Ar2Rw
CUDA_VISIBLE_DEVICES=0 python jan.py data/office-home -d OfficeHome -s Cl -t Ar -a vit_base_patch16_224 --no-pool --epochs 30 --seed 0 -b 24 --log logs/jan_vit/OfficeHome_Cl2Ar
CUDA_VISIBLE_DEVICES=0 python jan.py data/office-home -d OfficeHome -s Cl -t Pr -a vit_base_patch16_224 --no-pool --epochs 30 --seed 0 -b 24 --log logs/jan_vit/OfficeHome_Cl2Pr
CUDA_VISIBLE_DEVICES=0 python jan.py data/office-home -d OfficeHome -s Cl -t Rw -a vit_base_patch16_224 --no-pool --epochs 30 --seed 0 -b 24 --log logs/jan_vit/OfficeHome_Cl2Rw
CUDA_VISIBLE_DEVICES=0 python jan.py data/office-home -d OfficeHome -s Pr -t Ar -a vit_base_patch16_224 --no-pool --epochs 30 --seed 0 -b 24 --log logs/jan_vit/OfficeHome_Pr2Ar
CUDA_VISIBLE_DEVICES=0 python jan.py data/office-home -d OfficeHome -s Pr -t Cl -a vit_base_patch16_224 --no-pool --epochs 30 --seed 0 -b 24 --log logs/jan_vit/OfficeHome_Pr2Cl
CUDA_VISIBLE_DEVICES=0 python jan.py data/office-home -d OfficeHome -s Pr -t Rw -a vit_base_patch16_224 --no-pool --epochs 30 --seed 0 -b 24 --log logs/jan_vit/OfficeHome_Pr2Rw
CUDA_VISIBLE_DEVICES=0 python jan.py data/office-home -d OfficeHome -s Rw -t Ar -a vit_base_patch16_224 --no-pool --epochs 30 --seed 0 -b 24 --log logs/jan_vit/OfficeHome_Rw2Ar
CUDA_VISIBLE_DEVICES=0 python jan.py data/office-home -d OfficeHome -s Rw -t Cl -a vit_base_patch16_224 --no-pool --epochs 30 --seed 0 -b 24 --log logs/jan_vit/OfficeHome_Rw2Cl
CUDA_VISIBLE_DEVICES=0 python jan.py data/office-home -d OfficeHome -s Rw -t Pr -a vit_base_patch16_224 --no-pool --epochs 30 --seed 0 -b 24 --log logs/jan_vit/OfficeHome_Rw2Pr
# ResNet50, Office-Home, Multi Source
CUDA_VISIBLE_DEVICES=0 python jan.py data/office-home -d OfficeHome -s Cl Pr Rw -t Ar -a resnet50 --epochs 30 --seed 0 --log logs/jan/OfficeHome_:2Ar
CUDA_VISIBLE_DEVICES=0 python jan.py data/office-home -d OfficeHome -s Ar Pr Rw -t Cl -a resnet50 --epochs 30 --seed 0 --log logs/jan/OfficeHome_:2Cl
CUDA_VISIBLE_DEVICES=0 python jan.py data/office-home -d OfficeHome -s Ar Cl Rw -t Pr -a resnet50 --epochs 30 --seed 0 --log logs/jan/OfficeHome_:2Pr
CUDA_VISIBLE_DEVICES=0 python jan.py data/office-home -d OfficeHome -s Ar Cl Pr -t Rw -a resnet50 --epochs 30 --seed 0 --log logs/jan/OfficeHome_:2Rw
# ResNet101, DomainNet, Multi Source
CUDA_VISIBLE_DEVICES=0 python jan.py data/domainnet -d DomainNet -s i p q r s -t c -a resnet101 --bottleneck-dim 1024 --epochs 40 -i 5000 -p 500 --seed 0 --log logs/jan/DomainNet_:2c
CUDA_VISIBLE_DEVICES=0 python jan.py data/domainnet -d DomainNet -s c p q r s -t i -a resnet101 --bottleneck-dim 1024 --epochs 40 -i 5000 -p 500 --seed 0 --log logs/jan/DomainNet_:2i
CUDA_VISIBLE_DEVICES=0 python jan.py data/domainnet -d DomainNet -s c i q r s -t p -a resnet101 --bottleneck-dim 1024 --epochs 40 -i 5000 -p 500 --seed 0 --log logs/jan/DomainNet_:2p
CUDA_VISIBLE_DEVICES=0 python jan.py data/domainnet -d DomainNet -s c i p r s -t q -a resnet101 --bottleneck-dim 1024 --epochs 40 -i 5000 -p 500 --seed 0 --log logs/jan/DomainNet_:2q
CUDA_VISIBLE_DEVICES=0 python jan.py data/domainnet -d DomainNet -s c i p q s -t r -a resnet101 --bottleneck-dim 1024 --epochs 40 -i 5000 -p 500 --seed 0 --log logs/jan/DomainNet_:2r
CUDA_VISIBLE_DEVICES=0 python jan.py data/domainnet -d DomainNet -s c i p q r -t s -a resnet101 --bottleneck-dim 1024 --epochs 40 -i 5000 -p 500 --seed 0 --log logs/jan/DomainNet_:2s
# ResNet50, Wilds Dataset
CUDA_VISIBLE_DEVICES=0 python jan.py data/wilds -d iwildcam -a resnet50 --epochs 30 -i 1000 --seed 0 --log logs/jan/iwildcam
# DenseNet121, Wilds Dataset
CUDA_VISIBLE_DEVICES=0 python jan.py data/wilds -d camelyon17 --train-resizing 'res.' --val-resizing 'res.' --resize-size 96 \
-a densenet121 --scratch --epochs 10 -i 1000 --lr 0.01 --seed 0 --log logs/jan/camelyon17
CUDA_VISIBLE_DEVICES=0 python jan.py data/wilds -d fmow --train-resizing 'res.' --val-resizing 'res.' \
-a densenet121 --epochs 10 -i 1000 --lr 0.01 --seed 0 --log logs/jan/fmow
# Digits
CUDA_VISIBLE_DEVICES=0 python jan.py data/digits -d Digits -s MNIST -t USPS --train-resizing 'res.' --val-resizing 'res.' \
--resize-size 28 --no-hflip --norm-mean 0.5 --norm-std 0.5 -a lenet --no-pool --lr 0.01 -b 128 -i 2500 --scratch --seed 0 --log logs/jan/MNIST2USPS
CUDA_VISIBLE_DEVICES=0 python jan.py data/digits -d Digits -s USPS -t MNIST --train-resizing 'res.' --val-resizing 'res.' \
--resize-size 28 --no-hflip --norm-mean 0.5 --norm-std 0.5 -a lenet --no-pool --lr 0.1 -b 128 -i 2500 --scratch --seed 0 --log logs/jan/USPS2MNIST
CUDA_VISIBLE_DEVICES=4 python jan.py data/digits -d Digits -s SVHNRGB -t MNISTRGB --train-resizing 'res.' --val-resizing 'res.' \
--resize-size 32 --no-hflip --norm-mean 0.5 0.5 0.5 --norm-std 0.5 0.5 0.5 -a dtn --no-pool --lr 0.03 -b 128 -i 2500 --scratch --seed 0 --log logs/jan/SVHN2MNIST
================================================
FILE: domain_generalization/Transfer-Learning-Library/examples/domain_adaptation/image_classification/mcc.py
================================================
"""
@author: Ying Jin
@contact: sherryying003@gmail.com
"""
import random
import time
import warnings
import sys
import argparse
import shutil
import os.path as osp
import torch
import torch.nn as nn
import torch.backends.cudnn as cudnn
from torch.optim import SGD
from torch.optim.lr_scheduler import LambdaLR
from torch.utils.data import DataLoader
import torch.nn.functional as F
sys.path.append('../../..')
from dalib.adaptation.mcc import MinimumClassConfusionLoss, ImageClassifier
from common.utils.data import ForeverDataIterator
from common.utils.metric import accuracy
from common.utils.meter import AverageMeter, ProgressMeter
from common.utils.logger import CompleteLogger
from common.utils.analysis import collect_feature, tsne, a_distance
sys.path.append('.')
import utils
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
class TransformerDecorator(torch.nn.Module):
def __init__(self, backbone=None, add_bt=3, dim=2048, eval_global=0):
super(TransformerDecorator, self).__init__()
self.encoder_layers = torch.nn.TransformerEncoderLayer(dim, 4, dim, 0.5)
self.eval_global = eval_global
self.add_bt = add_bt
self.backbone = backbone
def forward_feats(self, feature):
feature = self.backbone(feature)
if self.training and self.add_bt > 0:
pre_feature = feature
feature = feature.unsqueeze(1)
feature = self.encoder_layers(feature)
# feature = self.encoder_layers(feature, feature, feature)
feature = feature.squeeze(1)
if self.add_bt >= 1:
return torch.cat([pre_feature, feature], dim=0)
return feature
def forward(self, feature):
feature = self.forward_feats(feature)
return feature
def main(args: argparse.Namespace):
logger = CompleteLogger(args.log, args.phase)
print(args)
if args.seed is not None:
random.seed(args.seed)
torch.manual_seed(args.seed)
cudnn.deterministic = True
warnings.warn('You have chosen to seed training. '
'This will turn on the CUDNN deterministic setting, '
'which can slow down your training considerably! '
'You may see unexpected behavior when restarting '
'from checkpoints.')
cudnn.benchmark = True
# Data loading code
train_transform = utils.get_train_transform(args.train_resizing, random_horizontal_flip=not args.no_hflip,
random_color_jitter=False, resize_size=args.resize_size,
norm_mean=args.norm_mean, norm_std=args.norm_std)
val_transform = utils.get_val_transform(args.val_resizing, resize_size=args.resize_size,
norm_mean=args.norm_mean, norm_std=args.norm_std)
print("train_transform: ", train_transform)
print("val_transform: ", val_transform)
train_source_dataset, train_target_dataset, val_dataset, test_dataset, num_classes, args.class_names = \
utils.get_dataset(args.data, args.root, args.source, args.target, train_transform, val_transform)
train_source_loader = DataLoader(train_source_dataset, batch_size=args.batch_size,
shuffle=True, num_workers=args.workers, drop_last=True)
train_target_loader = DataLoader(train_target_dataset, batch_size=args.batch_size,
shuffle=True, num_workers=args.workers, drop_last=True)
val_loader = DataLoader(val_dataset, batch_size=args.batch_size, shuffle=False, num_workers=args.workers)
test_loader = DataLoader(test_dataset, batch_size=args.batch_size, shuffle=False, num_workers=args.workers)
train_source_iter = ForeverDataIterator(train_source_loader)
train_target_iter = ForeverDataIterator(train_target_loader)
# create model
print("=> using model '{}'".format(args.arch))
backbone = utils.get_model(args.arch, pretrain=not args.scratch)
# pool_layer = nn.Identity() if args.no_pool else None
pool_layer = nn.Sequential(
nn.AdaptiveAvgPool2d(output_size=(1, 1)),
nn.Flatten(),
)
pool_layer = TransformerDecorator(pool_layer, add_bt=args.add_bt, dim=backbone.out_features)
classifier = ImageClassifier(backbone, num_classes, bottleneck_dim=args.bottleneck_dim,
pool_layer=pool_layer, finetune=not args.scratch).to(device)
# define optimizer and lr scheduler
optimizer = SGD(classifier.get_parameters(), args.lr, momentum=args.momentum, weight_decay=args.weight_decay, nesterov=True)
lr_scheduler = LambdaLR(optimizer, lambda x: args.lr * (1. + args.lr_gamma * float(x)) ** (-args.lr_decay))
# define loss function
mcc_loss = MinimumClassConfusionLoss(temperature=args.temperature)
# resume from the best checkpoint
if args.phase != 'train':
checkpoint = torch.load(logger.get_checkpoint_path('best'), map_location='cpu')
classifier.load_state_dict(checkpoint)
# analysis the model
if args.phase == 'analysis':
# extract features from both domains
feature_extractor = nn.Sequential(classifier.backbone, classifier.pool_layer, classifier.bottleneck).to(device)
source_feature = collect_feature(train_source_loader, feature_extractor, device)
target_feature = collect_feature(train_target_loader, feature_extractor, device)
# plot t-SNE
tSNE_filename = osp.join(logger.visualize_directory, 'TSNE.pdf')
tsne.visualize(source_feature, target_feature, tSNE_filename)
print("Saving t-SNE to", tSNE_filename)
# calculate A-distance, which is a measure for distribution discrepancy
A_distance = a_distance.calculate(source_feature, target_feature, device)
print("A-distance =", A_distance)
return
if args.phase == 'test':
acc1 = utils.validate(test_loader, classifier, args, device)
print(acc1)
return
# start training
best_acc1 = 0.
for epoch in range(args.epochs):
print("lr:", lr_scheduler.get_last_lr()[0])
# train for one epoch
train(train_source_iter, train_target_iter, classifier, mcc_loss, optimizer,
lr_scheduler, epoch, args)
# evaluate on validation set
acc1 = utils.validate(val_loader, classifier, args, device)
# remember best acc@1 and save checkpoint
torch.save(classifier.state_dict(), logger.get_checkpoint_path('latest'))
if acc1 > best_acc1:
shutil.copy(logger.get_checkpoint_path('latest'), logger.get_checkpoint_path('best'))
best_acc1 = max(acc1, best_acc1)
print("best_acc1 = {:3.1f}".format(best_acc1))
# evaluate on test set
classifier.load_state_dict(torch.load(logger.get_checkpoint_path('best')))
acc1 = utils.validate(test_loader, classifier, args, device)
print("test_acc1 = {:3.1f}".format(acc1))
logger.close()
def train(train_source_iter: ForeverDataIterator, train_target_iter: ForeverDataIterator,
model: ImageClassifier, mcc: MinimumClassConfusionLoss, optimizer: SGD,
lr_scheduler: LambdaLR, epoch: int, args: argparse.Namespace):
batch_time = AverageMeter('Time', ':3.1f')
data_time = AverageMeter('Data', ':3.1f')
losses = AverageMeter('Loss', ':3.2f')
trans_losses = AverageMeter('Trans Loss', ':3.2f')
cls_accs = AverageMeter('Cls Acc', ':3.1f')
progress = ProgressMeter(
args.iters_per_epoch,
[batch_time, data_time, losses, trans_losses, cls_accs],
prefix="Epoch: [{}]".format(epoch))
# switch to train mode
model.train()
end = time.time()
for i in range(args.iters_per_epoch):
x_s, labels_s = next(train_source_iter)
x_t, _ = next(train_target_iter)
x_s = x_s.to(device)
x_t = x_t.to(device)
labels_s = labels_s.to(device)
# measure data loading time
data_time.update(time.time() - end)
# compute output
x = torch.cat((x_s, x_t), dim=0)
y, f = model(x)
if len(y) > len(x):
y_s, y_t, y_s1, y_t1 = y.chunk(4, dim=0)
y_s = torch.cat([y_s, y_s1], dim=0)
y_t = torch.cat([y_t, y_t1], dim=0)
else:
y_s, y_t = y.chunk(2, dim=0)
if len(y_s) > len(labels_s):
labels_s = torch.cat([labels_s, labels_s], dim=0)
cls_loss = F.cross_entropy(y_s, labels_s)
transfer_loss = mcc(y_t)
loss = cls_loss + transfer_loss * args.trade_off
cls_acc = accuracy(y_s, labels_s)[0]
losses.update(loss.item(), x_s.size(0))
cls_accs.update(cls_acc.item(), x_s.size(0))
trans_losses.update(transfer_loss.item(), x_s.size(0))
# compute gradient and do SGD step
optimizer.zero_grad()
loss.backward()
optimizer.step()
lr_scheduler.step()
# measure elapsed time
batch_time.update(time.time() - end)
end = time.time()
if i % args.print_freq == 0:
progress.display(i)
if __name__ == '__main__':
parser = argparse.ArgumentParser(description='MCC for Unsupervised Domain Adaptation')
# dataset parameters
parser.add_argument('root', metavar='DIR',
help='root path of dataset')
parser.add_argument('-d', '--data', metavar='DATA', default='Office31', choices=utils.get_dataset_names(),
help='dataset: ' + ' | '.join(utils.get_dataset_names()) +
' (default: Office31)')
parser.add_argument('-s', '--source', help='source domain(s)', nargs='+')
parser.add_argument('-t', '--target', help='target domain(s)', nargs='+')
parser.add_argument('--train-resizing', type=str, default='default')
parser.add_argument('--val-resizing', type=str, default='default')
parser.add_argument('--resize-size', type=int, default=224,
help='the image size after resizing')
parser.add_argument('--no-hflip', action='store_true',
help='no random horizontal flipping during training')
parser.add_argument('--norm-mean', type=float, nargs='+',
default=(0.485, 0.456, 0.406), help='normalization mean')
parser.add_argument('--norm-std', type=float, nargs='+',
default=(0.229, 0.224, 0.225), help='normalization std')
# model parameters
parser.add_argument('-a', '--arch', metavar='ARCH', default='resnet18',
choices=utils.get_model_names(),
help='backbone architecture: ' +
' | '.join(utils.get_model_names()) +
' (default: resnet18)')
parser.add_argument('--bottleneck-dim', default=256, type=int,
help='Dimension of bottleneck')
parser.add_argument('--no-pool', action='store_true',
help='no pool layer after the feature extractor.')
parser.add_argument('--scratch', action='store_true', help='whether train from scratch.')
parser.add_argument('--temperature', default=2.5, type=float, help='parameter temperature scaling')
parser.add_argument('--trade-off', default=1., type=float,
help='the trade-off hyper-parameter for transfer loss')
# training parameters
parser.add_argument('-b', '--batch-size', default=36, type=int,
metavar='N',
help='mini-batch size (default: 36)')
parser.add_argument('--lr', '--learning-rate', default=0.005, type=float,
metavar='LR', help='initial learning rate', dest='lr')
parser.add_argument('--lr-gamma', default=0.001, type=float, help='parameter for lr scheduler')
parser.add_argument('--lr-decay', default=0.75, type=float, help='parameter for lr scheduler')
parser.add_argument('--momentum', default=0.9, type=float, metavar='M', help='momentum')
parser.add_argument('--wd', '--weight-decay', default=1e-3, type=float,
metavar='W', help='weight decay (default: 1e-3)',
dest='weight_decay')
parser.add_argument('-j', '--workers', default=2, type=int, metavar='N',
help='number of data loading workers (default: 2)')
parser.add_argument('--epochs', default=20, type=int, metavar='N',
help='number of total epochs to run')
parser.add_argument('-i', '--iters-per-epoch', default=1000, type=int,
help='Number of iterations per epoch')
parser.add_argument('-p', '--print-freq', default=100, type=int,
metavar='N', help='print frequency (default: 100)')
parser.add_argument('--seed', default=None, type=int,
help='seed for initializing training. ')
parser.add_argument('--per-class-eval', action='store_true',
help='whether output per-class accuracy during evaluation')
parser.add_argument("--log", type=str, default='mcc',
help="Where to save logs, checkpoints and debugging images.")
parser.add_argument("--phase", type=str, default='train', choices=['train', 'test', 'analysis'],
help="When phase is 'test', only test the model."
"When phase is 'analysis', only analysis the model.")
parser.add_argument("--add_bt", type=int, default=0,
help="add_bt")
args = parser.parse_args()
main(args)
================================================
FILE: domain_generalization/Transfer-Learning-Library/examples/domain_adaptation/image_classification/mcc.sh
================================================
#!/usr/bin/env bash
# ResNet50, Office31, Single Source
CUDA_VISIBLE_DEVICES=0 python mcc.py data/office31 -d Office31 -s A -t W -a resnet50 --epochs 20 -i 500 --seed 2 --bottleneck-dim 1024 --log logs/mcc/Office31_A2W
CUDA_VISIBLE_DEVICES=0 python mcc.py data/office31 -d Office31 -s D -t W -a resnet50 --epochs 20 -i 500 --seed 2 --bottleneck-dim 1024 --log logs/mcc/Office31_D2W
CUDA_VISIBLE_DEVICES=0 python mcc.py data/office31 -d Office31 -s W -t D -a resnet50 --epochs 20 -i 500 --seed 2 --bottleneck-dim 1024 --log logs/mcc/Office31_W2D
CUDA_VISIBLE_DEVICES=0 python mcc.py data/office31 -d Office31 -s A -t D -a resnet50 --epochs 20 -i 500 --seed 2 --bottleneck-dim 1024 --log logs/mcc/Office31_A2D
CUDA_VISIBLE_DEVICES=0 python mcc.py data/office31 -d Office31 -s D -t A -a resnet50 --epochs 20 -i 500 --seed 2 --bottleneck-dim 1024 --log logs/mcc/Office31_D2A
CUDA_VISIBLE_DEVICES=0 python mcc.py data/office31 -d Office31 -s W -t A -a resnet50 --epochs 20 -i 500 --seed 2 --bottleneck-dim 1024 --log logs/mcc/Office31_W2A
# ResNet50, Office-Home, Single Source
CUDA_VISIBLE_DEVICES=0 python mcc.py data/office-home -d OfficeHome -s Ar -t Cl -a resnet50 --epochs 30 --seed 0 --bottleneck-dim 2048 --log logs/mcc/OfficeHome_Ar2Cl
CUDA_VISIBLE_DEVICES=0 python mcc.py data/office-home -d OfficeHome -s Ar -t Pr -a resnet50 --epochs 30 --seed 0 --bottleneck-dim 2048 --log logs/mcc/OfficeHome_Ar2Pr
CUDA_VISIBLE_DEVICES=0 python mcc.py data/office-home -d OfficeHome -s Ar -t Rw -a resnet50 --epochs 30 --seed 0 --bottleneck-dim 2048 --log logs/mcc/OfficeHome_Ar2Rw
CUDA_VISIBLE_DEVICES=0 python mcc.py data/office-home -d OfficeHome -s Cl -t Ar -a resnet50 --epochs 30 --seed 0 --bottleneck-dim 2048 --log logs/mcc/OfficeHome_Cl2Ar
CUDA_VISIBLE_DEVICES=0 python mcc.py data/office-home -d OfficeHome -s Cl -t Pr -a resnet50 --epochs 30 --seed 0 --bottleneck-dim 2048 --log logs/mcc/OfficeHome_Cl2Pr
CUDA_VISIBLE_DEVICES=0 python mcc.py data/office-home -d OfficeHome -s Cl -t Rw -a resnet50 --epochs 30 --seed 0 --bottleneck-dim 2048 --log logs/mcc/OfficeHome_Cl2Rw
CUDA_VISIBLE_DEVICES=0 python mcc.py data/office-home -d OfficeHome -s Pr -t Ar -a resnet50 --epochs 30 --seed 0 --bottleneck-dim 2048 --log logs/mcc/OfficeHome_Pr2Ar
CUDA_VISIBLE_DEVICES=0 python mcc.py data/office-home -d OfficeHome -s Pr -t Cl -a resnet50 --epochs 30 --seed 0 --bottleneck-dim 2048 --log logs/mcc/OfficeHome_Pr2Cl
CUDA_VISIBLE_DEVICES=0 python mcc.py data/office-home -d OfficeHome -s Pr -t Rw -a resnet50 --epochs 30 --seed 0 --bottleneck-dim 2048 --log logs/mcc/OfficeHome_Pr2Rw
CUDA_VISIBLE_DEVICES=0 python mcc.py data/office-home -d OfficeHome -s Rw -t Ar -a resnet50 --epochs 30 --seed 0 --bottleneck-dim 2048 --log logs/mcc/OfficeHome_Rw2Ar
CUDA_VISIBLE_DEVICES=0 python mcc.py data/office-home -d OfficeHome -s Rw -t Cl -a resnet50 --epochs 30 --seed 0 --bottleneck-dim 2048 --log logs/mcc/OfficeHome_Rw2Cl
CUDA_VISIBLE_DEVICES=0 python mcc.py data/office-home -d OfficeHome -s Rw -t Pr -a resnet50 --epochs 30 --seed 0 --bottleneck-dim 2048 --log logs/mcc/OfficeHome_Rw2Pr
# ResNet101, VisDA-2017, Single Source
CUDA_VISIBLE_DEVICES=5 python mcc.py data/visda-2017 -d VisDA2017 -s Synthetic -t Real -a resnet101 \
--epochs 30 --seed 0 --lr 0.002 --per-class-eval --temperature 3.0 --train-resizing cen.crop --log logs/mcc/VisDA2017
# ResNet101, DomainNet, Single Source
CUDA_VISIBLE_DEVICES=0 python mcc.py data/domainnet -d DomainNet -s c -t p -a resnet101 --epochs 30 -b 32 -i 2500 -p 500 --temperature 2.0 --lr 0.005 --bottleneck-dim 2048 --trade-off 10.0 --seed 0 --log logs/mcc/DomainNet_c2p
CUDA_VISIBLE_DEVICES=0 python mcc.py data/domainnet -d DomainNet -s c -t r -a resnet101 --epochs 30 -b 32 -i 2500 -p 500 --temperature 2.0 --lr 0.005 --bottleneck-dim 2048 --trade-off 10.0 --seed 0 --log logs/mcc/DomainNet_c2r
CUDA_VISIBLE_DEVICES=0 python mcc.py data/domainnet -d DomainNet -s c -t s -a resnet101 --epochs 30 -b 32 -i 2500 -p 500 --temperature 2.0 --lr 0.005 --bottleneck-dim 2048 --trade-off 10.0 --seed 0 --log logs/mcc/DomainNet_c2s
CUDA_VISIBLE_DEVICES=0 python mcc.py data/domainnet -d DomainNet -s p -t c -a resnet101 --epochs 30 -b 32 -i 2500 -p 500 --temperature 2.0 --lr 0.005 --bottleneck-dim 2048 --trade-off 10.0 --seed 0 --log logs/mcc/DomainNet_p2c
CUDA_VISIBLE_DEVICES=0 python mcc.py data/domainnet -d DomainNet -s p -t r -a resnet101 --epochs 30 -b 32 -i 2500 -p 500 --temperature 2.0 --lr 0.005 --bottleneck-dim 2048 --trade-off 10.0 --seed 0 --log logs/mcc/DomainNet_p2r
CUDA_VISIBLE_DEVICES=0 python mcc.py data/domainnet -d DomainNet -s p -t s -a resnet101 --epochs 30 -b 32 -i 2500 -p 500 --temperature 2.0 --lr 0.005 --bottleneck-dim 2048 --trade-off 10.0 --seed 0 --log logs/mcc/DomainNet_p2s
CUDA_VISIBLE_DEVICES=0 python mcc.py data/domainnet -d DomainNet -s r -t c -a resnet101 --epochs 30 -b 32 -i 2500 -p 500 --temperature 2.0 --lr 0.005 --bottleneck-dim 2048 --trade-off 10.0 --seed 0 --log logs/mcc/DomainNet_r2c
CUDA_VISIBLE_DEVICES=0 python mcc.py data/domainnet -d DomainNet -s r -t p -a resnet101 --epochs 30 -b 32 -i 2500 -p 500 --temperature 2.0 --lr 0.005 --bottleneck-dim 2048 --trade-off 10.0 --seed 0 --log logs/mcc/DomainNet_r2p
CUDA_VISIBLE_DEVICES=0 python mcc.py data/domainnet -d DomainNet -s r -t s -a resnet101 --epochs 30 -b 32 -i 2500 -p 500 --temperature 2.0 --lr 0.005 --bottleneck-dim 2048 --trade-off 10.0 --seed 0 --log logs/mcc/DomainNet_r2s
CUDA_VISIBLE_DEVICES=0 python mcc.py data/domainnet -d DomainNet -s s -t c -a resnet101 --epochs 30 -b 32 -i 2500 -p 500 --temperature 2.0 --lr 0.005 --bottleneck-dim 2048 --trade-off 10.0 --seed 0 --log logs/mcc/DomainNet_s2c
CUDA_VISIBLE_DEVICES=0 python mcc.py data/domainnet -d DomainNet -s s -t p -a resnet101 --epochs 30 -b 32 -i 2500 -p 500 --temperature 2.0 --lr 0.005 --bottleneck-dim 2048 --trade-off 10.0 --seed 0 --log logs/mcc/DomainNet_s2p
CUDA_VISIBLE_DEVICES=0 python mcc.py data/domainnet -d DomainNet -s s -t r -a resnet101 --epochs 30 -b 32 -i 2500 -p 500 --temperature 2.0 --lr 0.005 --bottleneck-dim 2048 --trade-off 10.0 --seed 0 --log logs/mcc/DomainNet_s2r
# ResNet50, ImageNet200 -> ImageNetR
CUDA_VISIBLE_DEVICES=0 python mcc.py data/ImageNetR -d ImageNetR -s IN -t INR -a resnet50 --epochs 30 --seed 0 --temperature 2.5 --bottleneck-dim 2048 --log logs/mcc/ImageNet_IN2INR
# ig_resnext101_32x8d, ImageNet -> ImageNetSketch
CUDA_VISIBLE_DEVICES=0 python mcc.py data/imagenet-sketch -d ImageNetSketch -s IN -t sketch -a ig_resnext101_32x8d --epochs 30 -i 2500 -p 500 --log logs/mcc_ig_resnext101_32x8d/ImageNet_IN2sketch
# Vision Transformer, Office-Home, Single Source
CUDA_VISIBLE_DEVICES=0 python mcc.py data/office-home -d OfficeHome -s Ar -t Cl -a vit_base_patch16_224 --no-pool --bottleneck-dim 2048 --epochs 30 --seed 0 -b 24 --log logs/mcc_vit/OfficeHome_Ar2Cl
CUDA_VISIBLE_DEVICES=0 python mcc.py data/office-home -d OfficeHome -s Ar -t Pr -a vit_base_patch16_224 --no-pool --bottleneck-dim 2048 --epochs 30 --seed 0 -b 24 --log logs/mcc_vit/OfficeHome_Ar2Pr
CUDA_VISIBLE_DEVICES=0 python mcc.py data/office-home -d OfficeHome -s Ar -t Rw -a vit_base_patch16_224 --no-pool --bottleneck-dim 2048 --epochs 30 --seed 0 -b 24 --log logs/mcc_vit/OfficeHome_Ar2Rw
CUDA_VISIBLE_DEVICES=0 python mcc.py data/office-home -d OfficeHome -s Cl -t Ar -a vit_base_patch16_224 --no-pool --bottleneck-dim 2048 --epochs 30 --seed 0 -b 24 --log logs/mcc_vit/OfficeHome_Cl2Ar
CUDA_VISIBLE_DEVICES=0 python mcc.py data/office-home -d OfficeHome -s Cl -t Pr -a vit_base_patch16_224 --no-pool --bottleneck-dim 2048 --epochs 30 --seed 0 -b 24 --log logs/mcc_vit/OfficeHome_Cl2Pr
CUDA_VISIBLE_DEVICES=0 python mcc.py data/office-home -d OfficeHome -s Cl -t Rw -a vit_base_patch16_224 --no-pool --bottleneck-dim 2048 --epochs 30 --seed 0 -b 24 --log logs/mcc_vit/OfficeHome_Cl2Rw
CUDA_VISIBLE_DEVICES=0 python mcc.py data/office-home -d OfficeHome -s Pr -t Ar -a vit_base_patch16_224 --no-pool --bottleneck-dim 2048 --epochs 30 --seed 0 -b 24 --log logs/mcc_vit/OfficeHome_Pr2Ar
CUDA_VISIBLE_DEVICES=0 python mcc.py data/office-home -d OfficeHome -s Pr -t Cl -a vit_base_patch16_224 --no-pool --bottleneck-dim 2048 --epochs 30 --seed 0 -b 24 --log logs/mcc_vit/OfficeHome_Pr2Cl
CUDA_VISIBLE_DEVICES=0 python mcc.py data/office-home -d OfficeHome -s Pr -t Rw -a vit_base_patch16_224 --no-pool --bottleneck-dim 2048 --epochs 30 --seed 0 -b 24 --log logs/mcc_vit/OfficeHome_Pr2Rw
CUDA_VISIBLE_DEVICES=0 python mcc.py data/office-home -d OfficeHome -s Rw -t Ar -a vit_base_patch16_224 --no-pool --bottleneck-dim 2048 --epochs 30 --seed 0 -b 24 --log logs/mcc_vit/OfficeHome_Rw2Ar
CUDA_VISIBLE_DEVICES=0 python mcc.py data/office-home -d OfficeHome -s Rw -t Cl -a vit_base_patch16_224 --no-pool --bottleneck-dim 2048 --epochs 30 --seed 0 -b 24 --log logs/mcc_vit/OfficeHome_Rw2Cl
CUDA_VISIBLE_DEVICES=0 python mcc.py data/office-home -d OfficeHome -s Rw -t Pr -a vit_base_patch16_224 --no-pool --bottleneck-dim 2048 --epochs 30 --seed 0 -b 24 --log logs/mcc_vit/OfficeHome_Rw2Pr
# ResNet50, Office-Home, Multi Source
CUDA_VISIBLE_DEVICES=0 python mcc.py data/office-home -d OfficeHome -s Cl Pr Rw -t Ar -a resnet50 --bottleneck-dim 2048 --epochs 30 --seed 0 --log logs/mcc/OfficeHome_:2Ar
CUDA_VISIBLE_DEVICES=0 python mcc.py data/office-home -d OfficeHome -s Ar Pr Rw -t Cl -a resnet50 --bottleneck-dim 2048 --epochs 30 --seed 0 --log logs/mcc/OfficeHome_:2Cl
CUDA_VISIBLE_DEVICES=0 python mcc.py data/office-home -d OfficeHome -s Ar Cl Rw -t Pr -a resnet50 --bottleneck-dim 2048 --epochs 30 --seed 0 --log logs/mcc/OfficeHome_:2Pr
CUDA_VISIBLE_DEVICES=0 python mcc.py data/office-home -d OfficeHome -s Ar Cl Pr -t Rw -a resnet50 --bottleneck-dim 2048 --epochs 30 --seed 0 --log logs/mcc/OfficeHome_:2Rw
# ResNet101, DomainNet, Multi Source
CUDA_VISIBLE_DEVICES=0 python mcc.py data/domainnet -d DomainNet -s i p q r s -t c -a resnet101 --bottleneck-dim 2048 --epochs 40 -i 5000 -p 500 --seed 0 --log logs/mcc/DomainNet_:2c
CUDA_VISIBLE_DEVICES=0 python mcc.py data/domainnet -d DomainNet -s c p q r s -t i -a resnet101 --bottleneck-dim 2048 --epochs 40 -i 5000 -p 500 --seed 0 --log logs/mcc/DomainNet_:2i
CUDA_VISIBLE_DEVICES=0 python mcc.py data/domainnet -d DomainNet -s c i q r s -t p -a resnet101 --bottleneck-dim 2048 --epochs 40 -i 5000 -p 500 --seed 0 --log logs/mcc/DomainNet_:2p
CUDA_VISIBLE_DEVICES=0 python mcc.py data/domainnet -d DomainNet -s c i p r s -t q -a resnet101 --bottleneck-dim 2048 --epochs 40 -i 5000 -p 500 --seed 0 --log logs/mcc/DomainNet_:2q
CUDA_VISIBLE_DEVICES=0 python mcc.py data/domainnet -d DomainNet -s c i p q s -t r -a resnet101 --bottleneck-dim 2048 --epochs 40 -i 5000 -p 500 --seed 0 --log logs/mcc/DomainNet_:2r
CUDA_VISIBLE_DEVICES=0 python mcc.py data/domainnet -d DomainNet -s c i p q r -t s -a resnet101 --bottleneck-dim 2048 --epochs 40 -i 5000 -p 500 --seed 0 --log logs/mcc/DomainNet_:2s
# ResNet50, Wilds Dataset
CUDA_VISIBLE_DEVICES=0 python mcc.py data/wilds -d iwildcam -a resnet50 --epochs 30 -i 1000 --seed 0 --bottleneck-dim 2048 --log logs/mcc/iwildcam
# DenseNet121, Wilds Dataset
CUDA_VISIBLE_DEVICES=0 python mcc.py data/wilds -d camelyon17 --train-resizing 'res.' --val-resizing 'res.' --resize-size 96 \
-a densenet121 --scratch --epochs 10 -i 1000 --lr 0.01 --seed 0 --log logs/mcc/camelyon17
CUDA_VISIBLE_DEVICES=0 python mcc.py data/wilds -d fmow --train-resizing 'res.' --val-resizing 'res.' \
-a densenet121 --epochs 10 -i 1000 --lr 0.01 --seed 0 --bottleneck-dim 2048 --log logs/mcc/fmow
# Digits
CUDA_VISIBLE_DEVICES=0 python mcc.py data/digits -d Digits -s MNIST -t USPS --train-resizing 'res.' --val-resizing 'res.' \
--resize-size 28 --no-hflip --norm-mean 0.5 --norm-std 0.5 -a lenet --no-pool --lr 0.01 -b 128 -i 2500 --scratch --seed 0 --log logs/mcc/MNIST2USPS
CUDA_VISIBLE_DEVICES=0 python mcc.py data/digits -d Digits -s USPS -t MNIST --train-resizing 'res.' --val-resizing 'res.' \
--resize-size 28 --no-hflip --norm-mean 0.5 --norm-std 0.5 -a lenet --no-pool --lr 0.1 -b 128 -i 2500 --scratch --seed 0 --log logs/mcc/USPS2MNIST
CUDA_VISIBLE_DEVICES=0 python mcc.py data/digits -d Digits -s SVHNRGB -t MNISTRGB --train-resizing 'res.' --val-resizing 'res.' \
--resize-size 32 --no-hflip --norm-mean 0.5 0.5 0.5 --norm-std 0.5 0.5 0.5 -a dtn --no-pool --lr 0.01 -b 128 -i 2500 --scratch --seed 0 --log logs/mcc/SVHN2MNIST
================================================
FILE: domain_generalization/Transfer-Learning-Library/examples/domain_adaptation/image_classification/mcd.py
================================================
"""
@author: Junguang Jiang
@contact: JiangJunguang1123@outlook.com
"""
import random
import time
import warnings
import sys
import argparse
import shutil
import os.path as osp
from typing import Tuple
import torch
import torch.nn as nn
import torch.backends.cudnn as cudnn
from torch.optim import SGD
import torch.utils.data
from torch.utils.data import DataLoader
import torch.nn.functional as F
sys.path.append('../../..')
from dalib.adaptation.mcd import ImageClassifierHead, entropy, classifier_discrepancy
from common.utils.data import ForeverDataIterator
from common.utils.metric import accuracy, ConfusionMatrix
from common.utils.meter import AverageMeter, ProgressMeter
from common.utils.logger import CompleteLogger
from common.utils.analysis import collect_feature, tsne, a_distance
sys.path.append('.')
import utils
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
def main(args: argparse.Namespace):
logger = CompleteLogger(args.log, args.phase)
print(args)
if args.seed is not None:
random.seed(args.seed)
torch.manual_seed(args.seed)
cudnn.deterministic = True
warnings.warn('You have chosen to seed training. '
'This will turn on the CUDNN deterministic setting, '
'which can slow down your training considerably! '
'You may see unexpected behavior when restarting '
'from checkpoints.')
cudnn.benchmark = True
# Data loading code
train_transform = utils.get_train_transform(args.train_resizing, random_horizontal_flip=not args.no_hflip,
random_color_jitter=False, resize_size=args.resize_size,
norm_mean=args.norm_mean, norm_std=args.norm_std)
val_transform = utils.get_val_transform(args.val_resizing, resize_size=args.resize_size,
norm_mean=args.norm_mean, norm_std=args.norm_std)
print("train_transform: ", train_transform)
print("val_transform: ", val_transform)
train_source_dataset, train_target_dataset, val_dataset, test_dataset, num_classes, args.class_names = \
utils.get_dataset(args.data, args.root, args.source, args.target, train_transform, val_transform)
train_source_loader = DataLoader(train_source_dataset, batch_size=args.batch_size,
shuffle=True, num_workers=args.workers, drop_last=True)
train_target_loader = DataLoader(train_target_dataset, batch_size=args.batch_size,
shuffle=True, num_workers=args.workers, drop_last=True)
val_loader = DataLoader(val_dataset, batch_size=args.batch_size, shuffle=False, num_workers=args.workers)
test_loader = DataLoader(test_dataset, batch_size=args.batch_size, shuffle=False, num_workers=args.workers)
train_source_iter = ForeverDataIterator(train_source_loader)
train_target_iter = ForeverDataIterator(train_target_loader)
# create model
print("=> using model '{}'".format(args.arch))
G = utils.get_model(args.arch, pretrain=not args.scratch).to(device) # feature extractor
# two image classifier heads
pool_layer = nn.Identity() if args.no_pool else None
F1 = ImageClassifierHead(G.out_features, num_classes, args.bottleneck_dim, pool_layer).to(device)
F2 = ImageClassifierHead(G.out_features, num_classes, args.bottleneck_dim, pool_layer).to(device)
# define optimizer
# the learning rate is fixed according to origin paper
optimizer_g = SGD(G.parameters(), lr=args.lr, weight_decay=0.0005)
optimizer_f = SGD([
{"params": F1.parameters()},
{"params": F2.parameters()},
], momentum=0.9, lr=args.lr, weight_decay=0.0005)
# resume from the best checkpoint
if args.phase != 'train':
checkpoint = torch.load(logger.get_checkpoint_path('best'), map_location='cpu')
G.load_state_dict(checkpoint['G'])
F1.load_state_dict(checkpoint['F1'])
F2.load_state_dict(checkpoint['F2'])
# analysis the model
if args.phase == 'analysis':
# extract features from both domains
feature_extractor = nn.Sequential(G, F1.pool_layer).to(device)
source_feature = collect_feature(train_source_loader, feature_extractor, device)
target_feature = collect_feature(train_target_loader, feature_extractor, device)
# plot t-SNE
tSNE_filename = osp.join(logger.visualize_directory, 'TSNE.pdf')
tsne.visualize(source_feature, target_feature, tSNE_filename)
print("Saving t-SNE to", tSNE_filename)
# calculate A-distance, which is a measure for distribution discrepancy
A_distance = a_distance.calculate(source_feature, target_feature, device)
print("A-distance =", A_distance)
return
if args.phase == 'test':
acc1 = validate(test_loader, G, F1, F2, args)
print(acc1)
return
# start training
best_acc1 = 0.
best_results = None
for epoch in range(args.epochs):
# train for one epoch
train(train_source_iter, train_target_iter, G, F1, F2, optimizer_g, optimizer_f, epoch, args)
# evaluate on validation set
results = validate(val_loader, G, F1, F2, args)
# remember best acc@1 and save checkpoint
torch.save({
'G': G.state_dict(),
'F1': F1.state_dict(),
'F2': F2.state_dict()
}, logger.get_checkpoint_path('latest'))
if max(results) > best_acc1:
shutil.copy(logger.get_checkpoint_path('latest'), logger.get_checkpoint_path('best'))
best_acc1 = max(results)
best_results = results
print("best_acc1 = {:3.1f}, results = {}".format(best_acc1, best_results))
# evaluate on test set
checkpoint = torch.load(logger.get_checkpoint_path('best'), map_location='cpu')
G.load_state_dict(checkpoint['G'])
F1.load_state_dict(checkpoint['F1'])
F2.load_state_dict(checkpoint['F2'])
results = validate(test_loader, G, F1, F2, args)
print("test_acc1 = {:3.1f}".format(max(results)))
logger.close()
def train(train_source_iter: ForeverDataIterator, train_target_iter: ForeverDataIterator,
G: nn.Module, F1: ImageClassifierHead, F2: ImageClassifierHead,
optimizer_g: SGD, optimizer_f: SGD, epoch: int, args: argparse.Namespace):
batch_time = AverageMeter('Time', ':3.1f')
data_time = AverageMeter('Data', ':3.1f')
losses = AverageMeter('Loss', ':3.2f')
trans_losses = AverageMeter('Trans Loss', ':3.2f')
cls_accs = AverageMeter('Cls Acc', ':3.1f')
tgt_accs = AverageMeter('Tgt Acc', ':3.1f')
progress = ProgressMeter(
args.iters_per_epoch,
[batch_time, data_time, losses, trans_losses, cls_accs, tgt_accs],
prefix="Epoch: [{}]".format(epoch))
# switch to train mode
G.train()
F1.train()
F2.train()
end = time.time()
for i in range(args.iters_per_epoch):
x_s, labels_s = next(train_source_iter)
x_t, labels_t = next(train_target_iter)
x_s = x_s.to(device)
x_t = x_t.to(device)
labels_s = labels_s.to(device)
labels_t = labels_t.to(device)
x = torch.cat((x_s, x_t), dim=0)
assert x.requires_grad is False
# measure data loading time
data_time.update(time.time() - end)
# Step A train all networks to minimize loss on source domain
optimizer_g.zero_grad()
optimizer_f.zero_grad()
g = G(x)
y_1 = F1(g)
y_2 = F2(g)
y1_s, y1_t = y_1.chunk(2, dim=0)
y2_s, y2_t = y_2.chunk(2, dim=0)
y1_t, y2_t = F.softmax(y1_t, dim=1), F.softmax(y2_t, dim=1)
loss = F.cross_entropy(y1_s, labels_s) + F.cross_entropy(y2_s, labels_s) + \
(entropy(y1_t) + entropy(y2_t)) * args.trade_off_entropy
loss.backward()
optimizer_g.step()
optimizer_f.step()
# Step B train classifier to maximize discrepancy
optimizer_g.zero_grad()
optimizer_f.zero_grad()
g = G(x)
y_1 = F1(g)
y_2 = F2(g)
y1_s, y1_t = y_1.chunk(2, dim=0)
y2_s, y2_t = y_2.chunk(2, dim=0)
y1_t, y2_t = F.softmax(y1_t, dim=1), F.softmax(y2_t, dim=1)
loss = F.cross_entropy(y1_s, labels_s) + F.cross_entropy(y2_s, labels_s) + \
(entropy(y1_t) + entropy(y2_t)) * args.trade_off_entropy - \
classifier_discrepancy(y1_t, y2_t) * args.trade_off
loss.backward()
optimizer_f.step()
# Step C train genrator to minimize discrepancy
for k in range(args.num_k):
optimizer_g.zero_grad()
g = G(x)
y_1 = F1(g)
y_2 = F2(g)
y1_s, y1_t = y_1.chunk(2, dim=0)
y2_s, y2_t = y_2.chunk(2, dim=0)
y1_t, y2_t = F.softmax(y1_t, dim=1), F.softmax(y2_t, dim=1)
mcd_loss = classifier_discrepancy(y1_t, y2_t) * args.trade_off
mcd_loss.backward()
optimizer_g.step()
cls_acc = accuracy(y1_s, labels_s)[0]
tgt_acc = accuracy(y1_t, labels_t)[0]
losses.update(loss.item(), x_s.size(0))
cls_accs.update(cls_acc.item(), x_s.size(0))
tgt_accs.update(tgt_acc.item(), x_t.size(0))
trans_losses.update(mcd_loss.item(), x_s.size(0))
# measure elapsed time
batch_time.update(time.time() - end)
end = time.time()
if i % args.print_freq == 0:
progress.display(i)
def validate(val_loader: DataLoader, G: nn.Module, F1: ImageClassifierHead,
F2: ImageClassifierHead, args: argparse.Namespace) -> Tuple[float, float]:
batch_time = AverageMeter('Time', ':6.3f')
top1_1 = AverageMeter('Acc_1', ':6.2f')
top1_2 = AverageMeter('Acc_2', ':6.2f')
progress = ProgressMeter(
len(val_loader),
[batch_time, top1_1, top1_2],
prefix='Test: ')
# switch to evaluate mode
G.eval()
F1.eval()
F2.eval()
if args.per_class_eval:
confmat = ConfusionMatrix(len(args.class_names))
else:
confmat = None
with torch.no_grad():
end = time.time()
for i, (images, target) in enumerate(val_loader):
images = images.to(device)
target = target.to(device)
# compute output
g = G(images)
y1, y2 = F1(g), F2(g)
# measure accuracy and record loss
acc1, = accuracy(y1, target)
acc2, = accuracy(y2, target)
if confmat:
confmat.update(target, y1.argmax(1))
top1_1.update(acc1.item(), images.size(0))
top1_2.update(acc2.item(), images.size(0))
# measure elapsed time
batch_time.update(time.time() - end)
end = time.time()
if i % args.print_freq == 0:
progress.display(i)
print(' * Acc1 {top1_1.avg:.3f} Acc2 {top1_2.avg:.3f}'
.format(top1_1=top1_1, top1_2=top1_2))
if confmat:
print(confmat.format(args.class_names))
return top1_1.avg, top1_2.avg
if __name__ == '__main__':
parser = argparse.ArgumentParser(description='MCD for Unsupervised Domain Adaptation')
# dataset parameters
parser.add_argument('root', metavar='DIR',
help='root path of dataset')
parser.add_argument('-d', '--data', metavar='DATA', default='Office31', choices=utils.get_dataset_names(),
help='dataset: ' + ' | '.join(utils.get_dataset_names()) +
' (default: Office31)')
parser.add_argument('-s', '--source', help='source domain(s)', nargs='+')
parser.add_argument('-t', '--target', help='target domain(s)', nargs='+')
parser.add_argument('--train-resizing', type=str, default='default')
parser.add_argument('--val-resizing', type=str, default='default')
parser.add_argument('--resize-size', type=int, default=224,
help='the image size after resizing')
parser.add_argument('--no-hflip', action='store_true',
help='no random horizontal flipping during training')
parser.add_argument('--norm-mean', type=float, nargs='+',
default=(0.485, 0.456, 0.406), help='normalization mean')
parser.add_argument('--norm-std', type=float, nargs='+',
default=(0.229, 0.224, 0.225), help='normalization std')
# model parameters
parser.add_argument('-a', '--arch', metavar='ARCH', default='resnet18',
choices=utils.get_model_names(),
help='backbone architecture: ' +
' | '.join(utils.get_model_names()) +
' (default: resnet18)')
parser.add_argument('--bottleneck-dim', default=1024, type=int)
parser.add_argument('--no-pool', action='store_true',
help='no pool layer after the feature extractor.')
parser.add_argument('--scratch', action='store_true', help='whether train from scratch.')
parser.add_argument('--trade-off', default=1., type=float,
help='the trade-off hyper-parameter for transfer loss')
parser.add_argument('--trade-off-entropy', default=0.01, type=float,
help='the trade-off hyper-parameter for entropy loss')
parser.add_argument('--num-k', type=int, default=4, metavar='K',
help='how many steps to repeat the generator update')
# training parameters
parser.add_argument('-b', '--batch-size', default=32, type=int,
metavar='N',
help='mini-batch size (default: 32)')
parser.add_argument('--lr', '--learning-rate', default=0.001, type=float,
metavar='LR', help='initial learning rate', dest='lr')
parser.add_argument('-j', '--workers', default=2, type=int, metavar='N',
help='number of data loading workers (default: 2)')
parser.add_argument('--epochs', default=20, type=int, metavar='N',
help='number of total epochs to run')
parser.add_argument('-i', '--iters-per-epoch', default=1000, type=int,
help='Number of iterations per epoch')
parser.add_argument('-p', '--print-freq', default=100, type=int,
metavar='N', help='print frequency (default: 100)')
parser.add_argument('--seed', default=None, type=int,
help='seed for initializing training. ')
parser.add_argument('--per-class-eval', action='store_true',
help='whether output per-class accuracy during evaluation')
parser.add_argument("--log", type=str, default='mcd',
help="Where to save logs, checkpoints and debugging images.")
parser.add_argument("--phase", type=str, default='train', choices=['train', 'test', 'analysis'],
help="When phase is 'test', only test the model."
"When phase is 'analysis', only analysis the model.")
args = parser.parse_args()
main(args)
================================================
FILE: domain_generalization/Transfer-Learning-Library/examples/domain_adaptation/image_classification/mcd.sh
================================================
#!/usr/bin/env bash
# ResNet50, Office31, Single Source
# We found MCD loss is sensitive to class number,
# thus, when the class number increase, please increase trade-off correspondingly.
CUDA_VISIBLE_DEVICES=0 python mcd.py data/office31 -d Office31 -s D -t A -a resnet50 --epochs 20 --seed 0 -i 500 --trade-off 10.0 --log logs/mcd/Office31_D2A
CUDA_VISIBLE_DEVICES=0 python mcd.py data/office31 -d Office31 -s W -t A -a resnet50 --epochs 20 --seed 0 -i 500 --trade-off 10.0 --log logs/mcd/Office31_W2A
CUDA_VISIBLE_DEVICES=0 python mcd.py data/office31 -d Office31 -s A -t W -a resnet50 --epochs 20 --seed 0 -i 500 --trade-off 10.0 --log logs/mcd/Office31_A2W
CUDA_VISIBLE_DEVICES=0 python mcd.py data/office31 -d Office31 -s A -t D -a resnet50 --epochs 20 --seed 0 -i 500 --trade-off 10.0 --log logs/mcd/Office31_A2D
CUDA_VISIBLE_DEVICES=0 python mcd.py data/office31 -d Office31 -s D -t W -a resnet50 --epochs 20 --seed 0 -i 500 --trade-off 10.0 --log logs/mcd/Office31_D2W
CUDA_VISIBLE_DEVICES=0 python mcd.py data/office31 -d Office31 -s W -t D -a resnet50 --epochs 20 --seed 0 -i 500 --trade-off 10.0 --log logs/mcd/Office31_W2D
# ResNet50, Office-Home, Single Source
CUDA_VISIBLE_DEVICES=0 python mcd.py data/office-home -d OfficeHome -s Ar -t Cl -a resnet50 --epochs 20 -i 500 --seed 0 --trade-off 30.0 --log logs/mcd/OfficeHome_Ar2Cl
CUDA_VISIBLE_DEVICES=0 python mcd.py data/office-home -d OfficeHome -s Ar -t Pr -a resnet50 --epochs 20 -i 500 --seed 0 --trade-off 30.0 --log logs/mcd/OfficeHome_Ar2Pr
CUDA_VISIBLE_DEVICES=0 python mcd.py data/office-home -d OfficeHome -s Ar -t Rw -a resnet50 --epochs 20 -i 500 --seed 0 --trade-off 30.0 --log logs/mcd/OfficeHome_Ar2Rw
CUDA_VISIBLE_DEVICES=0 python mcd.py data/office-home -d OfficeHome -s Cl -t Ar -a resnet50 --epochs 20 -i 500 --seed 0 --trade-off 30.0 --log logs/mcd/OfficeHome_Cl2Ar
CUDA_VISIBLE_DEVICES=0 python mcd.py data/office-home -d OfficeHome -s Cl -t Pr -a resnet50 --epochs 20 -i 500 --seed 0 --trade-off 30.0 --log logs/mcd/OfficeHome_Cl2Pr
CUDA_VISIBLE_DEVICES=0 python mcd.py data/office-home -d OfficeHome -s Cl -t Rw -a resnet50 --epochs 20 -i 500 --seed 0 --trade-off 30.0 --log logs/mcd/OfficeHome_Cl2Rw
CUDA_VISIBLE_DEVICES=0 python mcd.py data/office-home -d OfficeHome -s Pr -t Ar -a resnet50 --epochs 20 -i 500 --seed 0 --trade-off 30.0 --log logs/mcd/OfficeHome_Pr2Ar
CUDA_VISIBLE_DEVICES=0 python mcd.py data/office-home -d OfficeHome -s Pr -t Cl -a resnet50 --epochs 20 -i 500 --seed 0 --trade-off 30.0 --log logs/mcd/OfficeHome_Pr2Cl
CUDA_VISIBLE_DEVICES=0 python mcd.py data/office-home -d OfficeHome -s Pr -t Rw -a resnet50 --epochs 20 -i 500 --seed 0 --trade-off 30.0 --log logs/mcd/OfficeHome_Pr2Rw
CUDA_VISIBLE_DEVICES=0 python mcd.py data/office-home -d OfficeHome -s Rw -t Ar -a resnet50 --epochs 20 -i 500 --seed 0 --trade-off 30.0 --log logs/mcd/OfficeHome_Rw2Ar
CUDA_VISIBLE_DEVICES=0 python mcd.py data/office-home -d OfficeHome -s Rw -t Cl -a resnet50 --epochs 20 -i 500 --seed 0 --trade-off 30.0 --log logs/mcd/OfficeHome_Rw2Cl
CUDA_VISIBLE_DEVICES=0 python mcd.py data/office-home -d OfficeHome -s Rw -t Pr -a resnet50 --epochs 20 -i 500 --seed 0 --trade-off 30.0 --log logs/mcd/OfficeHome_Rw2Pr
# ResNet101, VisDA-2017, Single Source
CUDA_VISIBLE_DEVICES=0 python mcd.py data/visda-2017 -d VisDA2017 -s Synthetic -t Real -a resnet101 \
--epochs 20 --center-crop --seed 0 -i 500 --per-class-eval --train-resizing cen.crop --log logs/mcd/VisDA2017
# ResNet101, DomainNet, Single Source
CUDA_VISIBLE_DEVICES=0 python mcd.py data/domainnet -d DomainNet -s c -t p -a resnet101 --bottleneck-dim 1024 --epochs 30 -i 2500 -p 500 --seed 0 --trade-off 120.0 --log logs/mcd/DomainNet_c2p
CUDA_VISIBLE_DEVICES=0 python mcd.py data/domainnet -d DomainNet -s c -t r -a resnet101 --bottleneck-dim 1024 --epochs 30 -i 2500 -p 500 --seed 0 --trade-off 120.0 --log logs/mcd/DomainNet_c2r
CUDA_VISIBLE_DEVICES=0 python mcd.py data/domainnet -d DomainNet -s c -t s -a resnet101 --bottleneck-dim 1024 --epochs 30 -i 2500 -p 500 --seed 0 --trade-off 120.0 --log logs/mcd/DomainNet_c2s
CUDA_VISIBLE_DEVICES=0 python mcd.py data/domainnet -d DomainNet -s p -t c -a resnet101 --bottleneck-dim 1024 --epochs 30 -i 2500 -p 500 --seed 0 --trade-off 120.0 --log logs/mcd/DomainNet_p2c
CUDA_VISIBLE_DEVICES=0 python mcd.py data/domainnet -d DomainNet -s p -t i -a resnet101 --bottleneck-dim 1024 --epochs 30 -i 2500 -p 500 --seed 0 --trade-off 120.0 --log logs/mcd/DomainNet_p2i
CUDA_VISIBLE_DEVICES=0 python mcd.py data/domainnet -d DomainNet -s p -t r -a resnet101 --bottleneck-dim 1024 --epochs 30 -i 2500 -p 500 --seed 0 --trade-off 120.0 --log logs/mcd/DomainNet_p2r
CUDA_VISIBLE_DEVICES=0 python mcd.py data/domainnet -d DomainNet -s p -t s -a resnet101 --bottleneck-dim 1024 --epochs 30 -i 2500 -p 500 --seed 0 --trade-off 120.0 --log logs/mcd/DomainNet_p2s
CUDA_VISIBLE_DEVICES=0 python mcd.py data/domainnet -d DomainNet -s r -t c -a resnet101 --bottleneck-dim 1024 --epochs 30 -i 2500 -p 500 --seed 0 --trade-off 120.0 --log logs/mcd/DomainNet_r2c
CUDA_VISIBLE_DEVICES=0 python mcd.py data/domainnet -d DomainNet -s r -t p -a resnet101 --bottleneck-dim 1024 --epochs 30 -i 2500 -p 500 --seed 0 --trade-off 120.0 --log logs/mcd/DomainNet_r2p
CUDA_VISIBLE_DEVICES=0 python mcd.py data/domainnet -d DomainNet -s r -t s -a resnet101 --bottleneck-dim 1024 --epochs 30 -i 2500 -p 500 --seed 0 --trade-off 120.0 --log logs/mcd/DomainNet_r2s
CUDA_VISIBLE_DEVICES=0 python mcd.py data/domainnet -d DomainNet -s s -t c -a resnet101 --bottleneck-dim 1024 --epochs 30 -i 2500 -p 500 --seed 0 --trade-off 120.0 --log logs/mcd/DomainNet_s2c
CUDA_VISIBLE_DEVICES=0 python mcd.py data/domainnet -d DomainNet -s s -t p -a resnet101 --bottleneck-dim 1024 --epochs 30 -i 2500 -p 500 --seed 0 --trade-off 120.0 --log logs/mcd/DomainNet_s2p
CUDA_VISIBLE_DEVICES=0 python mcd.py data/domainnet -d DomainNet -s s -t r -a resnet101 --bottleneck-dim 1024 --epochs 30 -i 2500 -p 500 --seed 0 --trade-off 120.0 --log logs/mcd/DomainNet_s2r
# ResNet50, ImageNet200 -> ImageNetR
CUDA_VISIBLE_DEVICES=0 python mcd.py data/ImageNetR -d ImageNetR -s IN -t INR -a resnet50 --epochs 30 -i 2500 -p 500 --seed 0 --trade-off 100.0 --log logs/mcd/ImageNet_IN2INR
# ig_resnext101_32x8d, ImageNet -> ImageNetSketch
CUDA_VISIBLE_DEVICES=0 python mcd.py data/imagenet-sketch -d ImageNetSketch -s IN -t sketch -a ig_resnext101_32x8d --epochs 30 -i 2500 -p 500 --trade-off 500.0 --log logs/mcd_ig_resnext101_32x8d/ImageNet_IN2sketch
# Vision Transformer, Office-Home, Single Source
CUDA_VISIBLE_DEVICES=0 python mcd.py data/office-home -d OfficeHome -s Ar -t Cl -a vit_base_patch16_224 --epochs 30 --seed 0 -b 24 --no-pool --trade-off 30.0 --log logs/mcd_vit/OfficeHome_Ar2Cl
CUDA_VISIBLE_DEVICES=0 python mcd.py data/office-home -d OfficeHome -s Ar -t Pr -a vit_base_patch16_224 --epochs 30 --seed 0 -b 24 --no-pool --trade-off 30.0 --log logs/mcd_vit/OfficeHome_Ar2Pr
CUDA_VISIBLE_DEVICES=0 python mcd.py data/office-home -d OfficeHome -s Ar -t Rw -a vit_base_patch16_224 --epochs 30 --seed 0 -b 24 --no-pool --trade-off 30.0 --log logs/mcd_vit/OfficeHome_Ar2Rw
CUDA_VISIBLE_DEVICES=0 python mcd.py data/office-home -d OfficeHome -s Cl -t Ar -a vit_base_patch16_224 --epochs 30 --seed 0 -b 24 --no-pool --trade-off 30.0 --log logs/mcd_vit/OfficeHome_Cl2Ar
CUDA_VISIBLE_DEVICES=0 python mcd.py data/office-home -d OfficeHome -s Cl -t Pr -a vit_base_patch16_224 --epochs 30 --seed 0 -b 24 --no-pool --trade-off 30.0 --log logs/mcd_vit/OfficeHome_Cl2Pr
CUDA_VISIBLE_DEVICES=0 python mcd.py data/office-home -d OfficeHome -s Cl -t Rw -a vit_base_patch16_224 --epochs 30 --seed 0 -b 24 --no-pool --trade-off 30.0 --log logs/mcd_vit/OfficeHome_Cl2Rw
CUDA_VISIBLE_DEVICES=0 python mcd.py data/office-home -d OfficeHome -s Pr -t Ar -a vit_base_patch16_224 --epochs 30 --seed 0 -b 24 --no-pool --trade-off 30.0 --log logs/mcd_vit/OfficeHome_Pr2Ar
CUDA_VISIBLE_DEVICES=0 python mcd.py data/office-home -d OfficeHome -s Pr -t Cl -a vit_base_patch16_224 --epochs 30 --seed 0 -b 24 --no-pool --trade-off 30.0 --log logs/mcd_vit/OfficeHome_Pr2Cl
CUDA_VISIBLE_DEVICES=0 python mcd.py data/office-home -d OfficeHome -s Pr -t Rw -a vit_base_patch16_224 --epochs 30 --seed 0 -b 24 --no-pool --trade-off 30.0 --log logs/mcd_vit/OfficeHome_Pr2Rw
CUDA_VISIBLE_DEVICES=0 python mcd.py data/office-home -d OfficeHome -s Rw -t Ar -a vit_base_patch16_224 --epochs 30 --seed 0 -b 24 --no-pool --trade-off 30.0 --log logs/mcd_vit/OfficeHome_Rw2Ar
CUDA_VISIBLE_DEVICES=0 python mcd.py data/office-home -d OfficeHome -s Rw -t Cl -a vit_base_patch16_224 --epochs 30 --seed 0 -b 24 --no-pool --trade-off 30.0 --log logs/mcd_vit/OfficeHome_Rw2Cl
CUDA_VISIBLE_DEVICES=0 python mcd.py data/office-home -d OfficeHome -s Rw -t Pr -a vit_base_patch16_224 --epochs 30 --seed 0 -b 24 --no-pool --trade-off 30.0 --log logs/mcd_vit/OfficeHome_Rw2Pr
# ResNet50, Wilds Dataset
CUDA_VISIBLE_DEVICES=0 python mcd.py data/wilds -d iwildcam -a resnet50 --epochs 30 -i 1000 --seed 0 --trade-off 100.0 --log logs/mcd/iwildcam
# DenseNet121, Wilds Dataset
CUDA_VISIBLE_DEVICES=0 python mcd.py data/wilds -d camelyon17 --train-resizing 'res.' --val-resizing 'res.' --resize-size 96 \
-a densenet121 --scratch --epochs 10 -i 1000 --lr 0.01 --seed 0 --log logs/mcd/camelyon17
CUDA_VISIBLE_DEVICES=0 python mcd.py data/wilds -d fmow --train-resizing 'res.' --val-resizing 'res.' \
-a densenet121 --epochs 10 -i 1000 --lr 0.01 --seed 0 --trade-off 30.0 --log logs/mcd/fmow
# Digits
CUDA_VISIBLE_DEVICES=0 python mcd.py data/digits -d Digits -s MNIST -t USPS --train-resizing 'res.' --val-resizing 'res.' \
--resize-size 28 --no-hflip --norm-mean 0.5 --norm-std 0.5 -a lenet --no-pool --lr 0.01 --trade-off 0.3 --trade-off-entropy 0.03 -b 128 -i 2500 --scratch --seed 0 --log logs/mcd/MNIST2USPS
CUDA_VISIBLE_DEVICES=0 python mcd.py data/digits -d Digits -s USPS -t MNIST --train-resizing 'res.' --val-resizing 'res.' \
--resize-size 28 --no-hflip --norm-mean 0.5 --norm-std 0.5 -a lenet --no-pool --lr 0.1 --trade-off 0.3 --trade-off-entropy 0.03 -b 128 -i 2500 --scratch --seed 0 --log logs/mcd/USPS2MNIST
CUDA_VISIBLE_DEVICES=0 python mcd.py data/digits -d Digits -s SVHNRGB -t MNISTRGB --train-resizing 'res.' --val-resizing 'res.' \
--resize-size 32 --no-hflip --norm-mean 0.5 0.5 0.5 --norm-std 0.5 0.5 0.5 -a dtn --no-pool --lr 0.03 --trade-off 0.3 --trade-off-entropy 0.03 -b 128 -i 2500 --scratch --seed 0 --log logs/mcd/SVHN2MNIST
================================================
FILE: domain_generalization/Transfer-Learning-Library/examples/domain_adaptation/image_classification/mdd.py
================================================
"""
@author: Junguang Jiang
@contact: JiangJunguang1123@outlook.com
"""
import random
import time
import warnings
import sys
import argparse
import os.path as osp
import shutil
import torch
import torch.nn as nn
import torch.backends.cudnn as cudnn
from torch.optim import SGD
from torch.optim.lr_scheduler import LambdaLR
from torch.utils.data import DataLoader
import torch.nn.functional as F
sys.path.append('../../..')
from dalib.adaptation.mdd import ClassificationMarginDisparityDiscrepancy\
as MarginDisparityDiscrepancy, ImageClassifier
from common.utils.data import ForeverDataIterator
from common.utils.metric import accuracy
from common.utils.meter import AverageMeter, ProgressMeter
from common.utils.logger import CompleteLogger
from common.utils.analysis import collect_feature, tsne, a_distance
sys.path.append('.')
import utils
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
class TransformerDecorator(torch.nn.Module):
def __init__(self, backbone=None, add_bt=3, dim=2048, eval_global=0):
super(TransformerDecorator, self).__init__()
self.encoder_layers = torch.nn.TransformerEncoderLayer(dim, 4, dim, 0.5)
self.eval_global = eval_global
self.add_bt = add_bt
self.backbone = backbone
def forward_feats(self, feature):
feature = self.backbone(feature)
if self.training and self.add_bt > 0:
pre_feature = feature
feature = feature.unsqueeze(1)
feature = self.encoder_layers(feature)
# feature = self.encoder_layers(feature, feature, feature)
feature = feature.squeeze(1)
if self.add_bt >= 1:
return torch.cat([pre_feature, feature], dim=0)
return feature
def forward(self, feature):
feature = self.forward_feats(feature)
return feature
def main(args: argparse.Namespace):
logger = CompleteLogger(args.log, args.phase)
print(args)
if args.seed is not None:
random.seed(args.seed)
torch.manual_seed(args.seed)
cudnn.deterministic = True
warnings.warn('You have chosen to seed training. '
'This will turn on the CUDNN deterministic setting, '
'which can slow down your training considerably! '
'You may see unexpected behavior when restarting '
'from checkpoints.')
cudnn.benchmark = True
# Data loading code
train_transform = utils.get_train_transform(args.train_resizing, random_horizontal_flip=not args.no_hflip,
random_color_jitter=False, resize_size=args.resize_size,
norm_mean=args.norm_mean, norm_std=args.norm_std)
val_transform = utils.get_val_transform(args.val_resizing, resize_size=args.resize_size,
norm_mean=args.norm_mean, norm_std=args.norm_std)
print("train_transform: ", train_transform)
print("val_transform: ", val_transform)
train_source_dataset, train_target_dataset, val_dataset, test_dataset, num_classes, args.class_names = \
utils.get_dataset(args.data, args.root, args.source, args.target, train_transform, val_transform)
train_source_loader = DataLoader(train_source_dataset, batch_size=args.batch_size,
shuffle=True, num_workers=args.workers, drop_last=True)
train_target_loader = DataLoader(train_target_dataset, batch_size=args.batch_size,
shuffle=True, num_workers=args.workers, drop_last=True)
val_loader = DataLoader(val_dataset, batch_size=args.batch_size, shuffle=False, num_workers=args.workers)
test_loader = DataLoader(test_dataset, batch_size=args.batch_size, shuffle=False, num_workers=args.workers)
train_source_iter = ForeverDataIterator(train_source_loader)
train_target_iter = ForeverDataIterator(train_target_loader)
# create model
print("=> using model '{}'".format(args.arch))
backbone = utils.get_model(args.arch, pretrain=not args.scratch)
# pool_layer = nn.Identity() if args.no_pool else None
pool_layer = nn.Sequential(
nn.AdaptiveAvgPool2d(output_size=(1, 1)),
nn.Flatten(),
)
pool_layer = TransformerDecorator(pool_layer, add_bt=args.add_bt, dim=backbone.out_features)
classifier = ImageClassifier(backbone, num_classes, bottleneck_dim=args.bottleneck_dim,
width=args.bottleneck_dim, pool_layer=pool_layer).to(device)
mdd = MarginDisparityDiscrepancy(args.margin).to(device)
# define optimizer and lr_scheduler
# The learning rate of the classifiers are set 10 times to that of the feature extractor by default.
optimizer = SGD(classifier.get_parameters(), args.lr, momentum=args.momentum, weight_decay=args.wd, nesterov=True)
lr_scheduler = LambdaLR(optimizer, lambda x: args.lr * (1. + args.lr_gamma * float(x)) ** (-args.lr_decay))
# resume from the best checkpoint
if args.phase != 'train':
checkpoint = torch.load(logger.get_checkpoint_path('best'), map_location='cpu')
classifier.load_state_dict(checkpoint)
# analysis the model
if args.phase == 'analysis':
# extract features from both domains
feature_extractor = nn.Sequential(classifier.backbone, classifier.bottleneck).to(device)
source_feature = collect_feature(train_source_loader, feature_extractor, device)
target_feature = collect_feature(train_target_loader, feature_extractor, device)
# plot t-SNE
tSNE_filename = osp.join(logger.visualize_directory, 'TSNE.pdf')
tsne.visualize(source_feature, target_feature, tSNE_filename)
print("Saving t-SNE to", tSNE_filename)
# calculate A-distance, which is a measure for distribution discrepancy
A_distance = a_distance.calculate(source_feature, target_feature, device)
print("A-distance =", A_distance)
return
if args.phase == 'test':
acc1 = utils.validate(test_loader, classifier, args, device)
print(acc1)
return
# start training
best_acc1 = 0.
for epoch in range(args.epochs):
print(lr_scheduler.get_lr())
# train for one epoch
train(train_source_iter, train_target_iter, classifier, mdd, optimizer,
lr_scheduler, epoch, args)
# evaluate on validation set
acc1 = utils.validate(val_loader, classifier, args, device)
# remember best acc@1 and save checkpoint
torch.save(classifier.state_dict(), logger.get_checkpoint_path('latest'))
if acc1 > best_acc1:
shutil.copy(logger.get_checkpoint_path('latest'), logger.get_checkpoint_path('best'))
best_acc1 = max(acc1, best_acc1)
print("best_acc1 = {:3.1f}".format(best_acc1))
# evaluate on test set
classifier.load_state_dict(torch.load(logger.get_checkpoint_path('best')))
acc1 = utils.validate(test_loader, classifier, args, device)
print("test_acc1 = {:3.1f}".format(acc1))
logger.close()
def train(train_source_iter: ForeverDataIterator, train_target_iter: ForeverDataIterator,
classifier: ImageClassifier, mdd: MarginDisparityDiscrepancy, optimizer: SGD,
lr_scheduler: LambdaLR, epoch: int, args: argparse.Namespace):
batch_time = AverageMeter('Time', ':3.1f')
data_time = AverageMeter('Data', ':3.1f')
losses = AverageMeter('Loss', ':3.2f')
trans_losses = AverageMeter('Trans Loss', ':3.2f')
cls_accs = AverageMeter('Cls Acc', ':3.1f')
tgt_accs = AverageMeter('Tgt Acc', ':3.1f')
progress = ProgressMeter(
args.iters_per_epoch,
[batch_time, data_time, losses, trans_losses, cls_accs, tgt_accs],
prefix="Epoch: [{}]".format(epoch))
# switch to train mode
classifier.train()
mdd.train()
end = time.time()
for i in range(args.iters_per_epoch):
optimizer.zero_grad()
x_s, labels_s = next(train_source_iter)
x_t, labels_t = next(train_target_iter)
x_s = x_s.to(device)
x_t = x_t.to(device)
labels_s = labels_s.to(device)
labels_t = labels_t.to(device)
# measure data loading time
data_time.update(time.time() - end)
# compute output
x = torch.cat((x_s, x_t), dim=0)
outputs, outputs_adv = classifier(x)
if len(outputs) > len(x):
y_s, y_t, y_s1, y_t1 = outputs.chunk(4, dim=0)
y_s = torch.cat([y_s, y_s1], dim=0)
y_t = torch.cat([y_t, y_t1], dim=0)
y_s_adv, y_t_adv, y_s_adv1, y_t_adv1 = outputs_adv.chunk(4, dim=0)
y_s_adv = torch.cat([y_s_adv, y_s_adv1], dim=0)
y_t_adv = torch.cat([y_t_adv, y_t_adv1], dim=0)
else:
y_s, y_t = outputs.chunk(2, dim=0)
y_s_adv, y_t_adv = outputs_adv.chunk(2, dim=0)
if len(y_s) > len(labels_s):
labels_s = torch.cat([labels_s, labels_s], dim=0)
if len(y_t) > len(labels_t):
labels_t = torch.cat([labels_t, labels_t], dim=0)
# compute cross entropy loss on source domain
cls_loss = F.cross_entropy(y_s, labels_s)
# compute margin disparity discrepancy between domains
# for adversarial classifier, minimize negative mdd is equal to maximize mdd
transfer_loss = -mdd(y_s, y_s_adv, y_t, y_t_adv)
loss = cls_loss + transfer_loss * args.trade_off
classifier.step()
cls_acc = accuracy(y_s, labels_s)[0]
tgt_acc = accuracy(y_t, labels_t)[0]
losses.update(loss.item(), x_s.size(0))
cls_accs.update(cls_acc.item(), x_s.size(0))
tgt_accs.update(tgt_acc.item(), x_t.size(0))
trans_losses.update(transfer_loss.item(), x_s.size(0))
# compute gradient and do SGD step
loss.backward()
optimizer.step()
lr_scheduler.step()
# measure elapsed time
batch_time.update(time.time() - end)
end = time.time()
if i % args.print_freq == 0:
progress.display(i)
if __name__ == '__main__':
parser = argparse.ArgumentParser(description='MDD for Unsupervised Domain Adaptation')
# dataset parameters
parser.add_argument('root', metavar='DIR',
help='root path of dataset')
parser.add_argument('-d', '--data', metavar='DATA', default='Office31', choices=utils.get_dataset_names(),
help='dataset: ' + ' | '.join(utils.get_dataset_names()) +
' (default: Office31)')
parser.add_argument('-s', '--source', help='source domain(s)', nargs='+')
parser.add_argument('-t', '--target', help='target domain(s)', nargs='+')
parser.add_argument('--train-resizing', type=str, default='default')
parser.add_argument('--val-resizing', type=str, default='default')
parser.add_argument('--resize-size', type=int, default=224,
help='the image size after resizing')
parser.add_argument('--no-hflip', action='store_true', help='no random horizontal flipping during training')
parser.add_argument('--norm-mean', type=float, nargs='+', default=(0.485, 0.456, 0.406), help='normalization mean')
parser.add_argument('--norm-std', type=float, nargs='+', default=(0.229, 0.224, 0.225), help='normalization std')
# model parameters
parser.add_argument('-a', '--arch', metavar='ARCH', default='resnet18',
choices=utils.get_model_names(),
help='backbone architecture: ' +
' | '.join(utils.get_model_names()) +
' (default: resnet18)')
parser.add_argument('--bottleneck-dim', default=1024, type=int)
parser.add_argument('--no-pool', action='store_true',
help='no pool layer after the feature extractor.')
parser.add_argument('--scratch', action='store_true', help='whether train from scratch.')
parser.add_argument('--margin', type=float, default=4., help="margin gamma")
parser.add_argument('--trade-off', default=1., type=float,
help='the trade-off hyper-parameter for transfer loss')
# training parameters
parser.add_argument('-b', '--batch-size', default=32, type=int,
metavar='N',
help='mini-batch size (default: 32)')
parser.add_argument('--lr', '--learning-rate', default=0.004, type=float,
metavar='LR', help='initial learning rate', dest='lr')
parser.add_argument('--lr-gamma', default=0.0002, type=float)
parser.add_argument('--lr-decay', default=0.75, type=float, help='parameter for lr scheduler')
parser.add_argument('--momentum', default=0.9, type=float, metavar='M', help='momentum')
parser.add_argument('--wd', '--weight-decay', default=0.0005, type=float,
metavar='W', help='weight decay (default: 5e-4)')
parser.add_argument('-j', '--workers', default=2, type=int, metavar='N',
help='number of data loading workers (default: 4)')
parser.add_argument('--epochs', default=20, type=int, metavar='N',
help='number of total epochs to run')
parser.add_argument('-i', '--iters-per-epoch', default=1000, type=int,
help='Number of iterations per epoch')
parser.add_argument('-p', '--print-freq', default=100, type=int,
metavar='N', help='print frequency (default: 100)')
parser.add_argument('--seed', default=None, type=int,
help='seed for initializing training. ')
parser.add_argument('--per-class-eval', action='store_true',
help='whether output per-class accuracy during evaluation')
parser.add_argument("--log", type=str, default='mdd',
help="Where to save logs, checkpoints and debugging images.")
parser.add_argument("--phase", type=str, default='train', choices=['train', 'test', 'analysis'],
help="When phase is 'test', only test the model."
"When phase is 'analysis', only analysis the model.")
parser.add_argument("--add_bt", type=int, default=0,
help="add_bt")
args = parser.parse_args()
main(args)
================================================
FILE: domain_generalization/Transfer-Learning-Library/examples/domain_adaptation/image_classification/mdd.sh
================================================
#!/usr/bin/env bash
# ResNet50, Office31, Single Source
CUDA_VISIBLE_DEVICES=0 python mdd.py data/office31 -d Office31 -s A -t W -a resnet50 --epochs 20 --bottleneck-dim 1024 --seed 1 --log logs/mdd/Office31_A2W
CUDA_VISIBLE_DEVICES=0 python mdd.py data/office31 -d Office31 -s D -t W -a resnet50 --epochs 20 --bottleneck-dim 1024 --seed 1 --log logs/mdd/Office31_D2W
CUDA_VISIBLE_DEVICES=0 python mdd.py data/office31 -d Office31 -s W -t D -a resnet50 --epochs 20 --bottleneck-dim 1024 --seed 1 --log logs/mdd/Office31_W2D
CUDA_VISIBLE_DEVICES=0 python mdd.py data/office31 -d Office31 -s A -t D -a resnet50 --epochs 20 --bottleneck-dim 1024 --seed 1 --log logs/mdd/Office31_A2D
CUDA_VISIBLE_DEVICES=0 python mdd.py data/office31 -d Office31 -s D -t A -a resnet50 --epochs 20 --bottleneck-dim 1024 --seed 1 --log logs/mdd/Office31_D2A
CUDA_VISIBLE_DEVICES=0 python mdd.py data/office31 -d Office31 -s W -t A -a resnet50 --epochs 20 --bottleneck-dim 1024 --seed 1 --log logs/mdd/Office31_W2A
# ResNet50, Office-Home, Single Source
CUDA_VISIBLE_DEVICES=0 python mdd.py data/office-home -d OfficeHome -s Ar -t Cl -a resnet50 --epochs 30 --bottleneck-dim 2048 --seed 0 --log logs/mdd/OfficeHome_Ar2Cl
CUDA_VISIBLE_DEVICES=0 python mdd.py data/office-home -d OfficeHome -s Ar -t Pr -a resnet50 --epochs 30 --bottleneck-dim 2048 --seed 0 --log logs/mdd/OfficeHome_Ar2Pr
CUDA_VISIBLE_DEVICES=0 python mdd.py data/office-home -d OfficeHome -s Ar -t Rw -a resnet50 --epochs 30 --bottleneck-dim 2048 --seed 0 --log logs/mdd/OfficeHome_Ar2Rw
CUDA_VISIBLE_DEVICES=0 python mdd.py data/office-home -d OfficeHome -s Cl -t Ar -a resnet50 --epochs 30 --bottleneck-dim 2048 --seed 0 --log logs/mdd/OfficeHome_Cl2Ar
CUDA_VISIBLE_DEVICES=0 python mdd.py data/office-home -d OfficeHome -s Cl -t Pr -a resnet50 --epochs 30 --bottleneck-dim 2048 --seed 0 --log logs/mdd/OfficeHome_Cl2Pr
CUDA_VISIBLE_DEVICES=0 python mdd.py data/office-home -d OfficeHome -s Cl -t Rw -a resnet50 --epochs 30 --bottleneck-dim 2048 --seed 0 --log logs/mdd/OfficeHome_Cl2Rw
CUDA_VISIBLE_DEVICES=0 python mdd.py data/office-home -d OfficeHome -s Pr -t Ar -a resnet50 --epochs 30 --bottleneck-dim 2048 --seed 0 --log logs/mdd/OfficeHome_Pr2Ar
CUDA_VISIBLE_DEVICES=0 python mdd.py data/office-home -d OfficeHome -s Pr -t Cl -a resnet50 --epochs 30 --bottleneck-dim 2048 --seed 0 --log logs/mdd/OfficeHome_Pr2Cl
CUDA_VISIBLE_DEVICES=0 python mdd.py data/office-home -d OfficeHome -s Pr -t Rw -a resnet50 --epochs 30 --bottleneck-dim 2048 --seed 0 --log logs/mdd/OfficeHome_Pr2Rw
CUDA_VISIBLE_DEVICES=0 python mdd.py data/office-home -d OfficeHome -s Rw -t Ar -a resnet50 --epochs 30 --bottleneck-dim 2048 --seed 0 --log logs/mdd/OfficeHome_Rw2Ar
CUDA_VISIBLE_DEVICES=0 python mdd.py data/office-home -d OfficeHome -s Rw -t Cl -a resnet50 --epochs 30 --bottleneck-dim 2048 --seed 0 --log logs/mdd/OfficeHome_Rw2Cl
CUDA_VISIBLE_DEVICES=0 python mdd.py data/office-home -d OfficeHome -s Rw -t Pr -a resnet50 --epochs 30 --bottleneck-dim 2048 --seed 0 --log logs/mdd/OfficeHome_Rw2Pr
# ResNet101, VisDA-2017, Single Source
CUDA_VISIBLE_DEVICES=0 python mdd.py data/visda-2017 -d VisDA2017 -s Synthetic -t Real -a resnet101 --epochs 30 \
--bottleneck-dim 1024 --seed 0 --train-resizing cen.crop --per-class-eval -b 36 --log logs/mdd/VisDA2017
# ResNet101, DomainNet, Single Source
CUDA_VISIBLE_DEVICES=0 python mdd.py data/domainnet -d DomainNet -s c -t p -a resnet101 --epochs 40 -i 5000 -p 500 --bottleneck-dim 2048 --seed 0 --lr 0.004 --log logs/mdd/DomainNet_c2p
CUDA_VISIBLE_DEVICES=0 python mdd.py data/domainnet -d DomainNet -s c -t r -a resnet101 --epochs 40 -i 5000 -p 500 --bottleneck-dim 2048 --seed 0 --lr 0.004 --log logs/mdd/DomainNet_c2r
CUDA_VISIBLE_DEVICES=0 python mdd.py data/domainnet -d DomainNet -s c -t s -a resnet101 --epochs 40 -i 5000 -p 500 --bottleneck-dim 2048 --seed 0 --lr 0.004 --log logs/mdd/DomainNet_c2s
CUDA_VISIBLE_DEVICES=0 python mdd.py data/domainnet -d DomainNet -s p -t c -a resnet101 --epochs 40 -i 5000 -p 500 --bottleneck-dim 2048 --seed 0 --lr 0.004 --log logs/mdd/DomainNet_p2c
CUDA_VISIBLE_DEVICES=0 python mdd.py data/domainnet -d DomainNet -s p -t r -a resnet101 --epochs 40 -i 5000 -p 500 --bottleneck-dim 2048 --seed 0 --lr 0.004 --log logs/mdd/DomainNet_p2r
CUDA_VISIBLE_DEVICES=0 python mdd.py data/domainnet -d DomainNet -s p -t s -a resnet101 --epochs 40 -i 5000 -p 500 --bottleneck-dim 2048 --seed 0 --lr 0.004 --log logs/mdd/DomainNet_p2s
CUDA_VISIBLE_DEVICES=0 python mdd.py data/domainnet -d DomainNet -s r -t c -a resnet101 --epochs 40 -i 5000 -p 500 --bottleneck-dim 2048 --seed 0 --lr 0.004 --log logs/mdd/DomainNet_r2c
CUDA_VISIBLE_DEVICES=0 python mdd.py data/domainnet -d DomainNet -s r -t p -a resnet101 --epochs 40 -i 5000 -p 500 --bottleneck-dim 2048 --seed 0 --lr 0.004 --log logs/mdd/DomainNet_r2p
CUDA_VISIBLE_DEVICES=0 python mdd.py data/domainnet -d DomainNet -s r -t s -a resnet101 --epochs 40 -i 5000 -p 500 --bottleneck-dim 2048 --seed 0 --lr 0.004 --log logs/mdd/DomainNet_r2s
CUDA_VISIBLE_DEVICES=0 python mdd.py data/domainnet -d DomainNet -s s -t c -a resnet101 --epochs 40 -i 5000 -p 500 --bottleneck-dim 2048 --seed 0 --lr 0.004 --log logs/mdd/DomainNet_s2c
CUDA_VISIBLE_DEVICES=0 python mdd.py data/domainnet -d DomainNet -s s -t p -a resnet101 --epochs 40 -i 5000 -p 500 --bottleneck-dim 2048 --seed 0 --lr 0.004 --log logs/mdd/DomainNet_s2p
CUDA_VISIBLE_DEVICES=0 python mdd.py data/domainnet -d DomainNet -s s -t r -a resnet101 --epochs 40 -i 5000 -p 500 --bottleneck-dim 2048 --seed 0 --lr 0.004 --log logs/mdd/DomainNet_s2r
# ResNet50, ImageNet200 -> ImageNetR
CUDA_VISIBLE_DEVICES=0 python mdd.py data/ImageNetR -d ImageNetR -s IN -t INR -a resnet50 --epochs 40 -i 2500 -p 500 \
--bottleneck-dim 2048 --seed 0 --lr 0.004 --train-resizing cen.crop --log logs/mdd/ImageNet_IN2INR
# ig_resnext101_32x8d, ImageNet -> ImageNetSketch
CUDA_VISIBLE_DEVICES=0 python mdd.py data/imagenet-sketch -d ImageNetSketch -s IN -t sketch -a ig_resnext101_32x8d \
--epochs 40 -i 2500 -p 500 --bottleneck-dim 2048 --margin 2. --seed 0 --lr 0.004 --train-resizing cen.crop \
--log logs/mdd_ig_resnext101_32x8d/ImageNet_IN2sketch
# Vision Transformer, Office-Home, Single Source
CUDA_VISIBLE_DEVICES=0 python mdd.py data/office-home -d OfficeHome -s Ar -t Cl -a vit_base_patch16_224 --epochs 40 --bottleneck-dim 2048 --seed 0 -b 24 --no-pool --log logs/mdd_vit/OfficeHome_Ar2Cl
CUDA_VISIBLE_DEVICES=0 python mdd.py data/office-home -d OfficeHome -s Ar -t Pr -a vit_base_patch16_224 --epochs 40 --bottleneck-dim 2048 --seed 0 -b 24 --no-pool --log logs/mdd_vit/OfficeHome_Ar2Pr
CUDA_VISIBLE_DEVICES=0 python mdd.py data/office-home -d OfficeHome -s Ar -t Rw -a vit_base_patch16_224 --epochs 40 --bottleneck-dim 2048 --seed 0 -b 24 --no-pool --log logs/mdd_vit/OfficeHome_Ar2Rw
CUDA_VISIBLE_DEVICES=0 python mdd.py data/office-home -d OfficeHome -s Cl -t Ar -a vit_base_patch16_224 --epochs 40 --bottleneck-dim 2048 --seed 0 -b 24 --no-pool --log logs/mdd_vit/OfficeHome_Cl2Ar
CUDA_VISIBLE_DEVICES=0 python mdd.py data/office-home -d OfficeHome -s Cl -t Pr -a vit_base_patch16_224 --epochs 40 --bottleneck-dim 2048 --seed 0 -b 24 --no-pool --log logs/mdd_vit/OfficeHome_Cl2Pr
CUDA_VISIBLE_DEVICES=0 python mdd.py data/office-home -d OfficeHome -s Cl -t Rw -a vit_base_patch16_224 --epochs 40 --bottleneck-dim 2048 --seed 0 -b 24 --no-pool --log logs/mdd_vit/OfficeHome_Cl2Rw
CUDA_VISIBLE_DEVICES=0 python mdd.py data/office-home -d OfficeHome -s Pr -t Ar -a vit_base_patch16_224 --epochs 40 --bottleneck-dim 2048 --seed 0 -b 24 --no-pool --log logs/mdd_vit/OfficeHome_Pr2Ar
CUDA_VISIBLE_DEVICES=0 python mdd.py data/office-home -d OfficeHome -s Pr -t Cl -a vit_base_patch16_224 --epochs 40 --bottleneck-dim 2048 --seed 0 -b 24 --no-pool --log logs/mdd_vit/OfficeHome_Pr2Cl
CUDA_VISIBLE_DEVICES=0 python mdd.py data/office-home -d OfficeHome -s Pr -t Rw -a vit_base_patch16_224 --epochs 40 --bottleneck-dim 2048 --seed 0 -b 24 --no-pool --log logs/mdd_vit/OfficeHome_Pr2Rw
CUDA_VISIBLE_DEVICES=0 python mdd.py data/office-home -d OfficeHome -s Rw -t Ar -a vit_base_patch16_224 --epochs 40 --bottleneck-dim 2048 --seed 0 -b 24 --no-pool --log logs/mdd_vit/OfficeHome_Rw2Ar
CUDA_VISIBLE_DEVICES=0 python mdd.py data/office-home -d OfficeHome -s Rw -t Cl -a vit_base_patch16_224 --epochs 40 --bottleneck-dim 2048 --seed 0 -b 24 --no-pool --log logs/mdd_vit/OfficeHome_Rw2Cl
CUDA_VISIBLE_DEVICES=0 python mdd.py data/office-home -d OfficeHome -s Rw -t Pr -a vit_base_patch16_224 --epochs 40 --bottleneck-dim 2048 --seed 0 -b 24 --no-pool --log logs/mdd_vit/OfficeHome_Rw2Pr
# ResNet50, Office-Home, Multi Source
CUDA_VISIBLE_DEVICES=0 python mdd.py data/office-home -d OfficeHome -s Cl Pr Rw -t Ar -a resnet50 --epochs 30 --bottleneck-dim 2048 --seed 0 --log logs/mdd/OfficeHome_:2Ar
CUDA_VISIBLE_DEVICES=0 python mdd.py data/office-home -d OfficeHome -s Ar Pr Rw -t Cl -a resnet50 --epochs 30 --bottleneck-dim 2048 --seed 0 --log logs/mdd/OfficeHome_:2Cl
CUDA_VISIBLE_DEVICES=0 python mdd.py data/office-home -d OfficeHome -s Ar Cl Rw -t Pr -a resnet50 --epochs 30 --bottleneck-dim 2048 --seed 0 --log logs/mdd/OfficeHome_:2Pr
CUDA_VISIBLE_DEVICES=0 python mdd.py data/office-home -d OfficeHome -s Ar Cl Pr -t Rw -a resnet50 --epochs 30 --bottleneck-dim 2048 --seed 0 --log logs/mdd/OfficeHome_:2Rw
# ResNet101, DomainNet, Multi Source
CUDA_VISIBLE_DEVICES=0 python mdd.py data/domainnet -d DomainNet -s i p q r s -t c -a resnet101 --epochs 40 -i 5000 -p 500 --bottleneck-dim 2048 --seed 0 --lr 0.004 --log logs/mdd/DomainNet_:2c
CUDA_VISIBLE_DEVICES=0 python mdd.py data/domainnet -d DomainNet -s c p q r s -t i -a resnet101 --epochs 40 -i 5000 -p 500 --bottleneck-dim 2048 --seed 0 --lr 0.004 --log logs/mdd/DomainNet_:2i
CUDA_VISIBLE_DEVICES=0 python mdd.py data/domainnet -d DomainNet -s c i q r s -t p -a resnet101 --epochs 40 -i 5000 -p 500 --bottleneck-dim 2048 --seed 0 --lr 0.004 --log logs/mdd/DomainNet_:2p
CUDA_VISIBLE_DEVICES=0 python mdd.py data/domainnet -d DomainNet -s c i p r s -t q -a resnet101 --epochs 40 -i 5000 -p 500 --bottleneck-dim 2048 --seed 0 --lr 0.004 --log logs/mdd/DomainNet_:2q
CUDA_VISIBLE_DEVICES=0 python mdd.py data/domainnet -d DomainNet -s c i p q s -t r -a resnet101 --epochs 40 -i 5000 -p 500 --bottleneck-dim 2048 --seed 0 --lr 0.004 --log logs/mdd/DomainNet_:2r
CUDA_VISIBLE_DEVICES=0 python mdd.py data/domainnet -d DomainNet -s c i p q r -t s -a resnet101 --epochs 40 -i 5000 -p 500 --bottleneck-dim 2048 --seed 0 --lr 0.004 --log logs/mdd/DomainNet_:2s
# ResNet50, Wilds Dataset
CUDA_VISIBLE_DEVICES=0 python mdd.py data/wilds -d iwildcam --train-resizing 'res.' --val-resizing 'res.' \
-a resnet50 --bottleneck-dim 2048 --epochs 30 -i 1000 --seed 0 --log logs/mdd/iwildcam
# DenseNet121, Wilds Dataset
CUDA_VISIBLE_DEVICES=0 python mdd.py data/wilds -d camelyon17 --train-resizing 'res.' --val-resizing 'res.' --resize-size 96 \
-a densenet121 --scratch --epochs 30 -i 1000 --lr 0.01 --seed 0 --log logs/mdd/camelyon17
CUDA_VISIBLE_DEVICES=0 python mdd.py data/wilds -d fmow --train-resizing 'res.' --val-resizing 'res.' \
-a densenet121 --bottleneck-dim 2048 --epochs 30 -i 1000 --lr 0.01 --seed 0 --log logs/mdd/fmow
# Digits
CUDA_VISIBLE_DEVICES=0 python mdd.py data/digits -d Digits -s MNIST -t USPS --train-resizing 'res.' --val-resizing 'res.' \
--resize-size 28 --no-hflip --norm-mean 0.5 --norm-std 0.5 -a lenet --no-pool --lr 0.01 -b 128 -i 2500 --scratch --seed 0 --log logs/mdd/MNIST2USPS
CUDA_VISIBLE_DEVICES=0 python mdd.py data/digits -d Digits -s USPS -t MNIST --train-resizing 'res.' --val-resizing 'res.' \
--resize-size 28 --no-hflip --norm-mean 0.5 --norm-std 0.5 -a lenet --no-pool --lr 0.01 -b 128 -i 2500 --scratch --seed 0 --log logs/mdd/USPS2MNIST
CUDA_VISIBLE_DEVICES=0 python mdd.py data/digits -d Digits -s SVHNRGB -t MNISTRGB --train-resizing 'res.' --val-resizing 'res.' \
--resize-size 32 --no-hflip --norm-mean 0.5 0.5 0.5 --norm-std 0.5 0.5 0.5 -a dtn --no-pool --lr 0.01 -b 128 -i 2500 --scratch --seed 0 --log logs/mdd/SVHN2MNIST
================================================
FILE: domain_generalization/Transfer-Learning-Library/examples/domain_adaptation/image_classification/requirements.txt
================================================
wilds
timm
================================================
FILE: domain_generalization/Transfer-Learning-Library/examples/domain_adaptation/image_classification/self_ensemble.py
================================================
"""
@author: Baixu Chen
@contact: cbx_99_hasta@outlook.com
"""
import random
import time
import warnings
import sys
import argparse
import shutil
import os.path as osp
import torch
import torch.nn as nn
import torch.backends.cudnn as cudnn
from torch.optim import Adam
from torch.optim.lr_scheduler import LambdaLR
from torch.utils.data import DataLoader
import torchvision.transforms as T
import torch.nn.functional as F
sys.path.append('../../..')
from dalib.adaptation.self_ensemble import EmaTeacher, L2ConsistencyLoss, ClassBalanceLoss, ImageClassifier
from common.vision.transforms import ResizeImage, MultipleApply
from common.utils.data import ForeverDataIterator
from common.utils.metric import accuracy
from common.utils.meter import AverageMeter, ProgressMeter
from common.utils.logger import CompleteLogger
from common.utils.analysis import collect_feature, tsne, a_distance
sys.path.append('.')
import utils
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
def main(args: argparse.Namespace):
logger = CompleteLogger(args.log, args.phase)
print(args)
if args.seed is not None:
random.seed(args.seed)
torch.manual_seed(args.seed)
cudnn.deterministic = True
warnings.warn('You have chosen to seed training. '
'This will turn on the CUDNN deterministic setting, '
'which can slow down your training considerably! '
'You may see unexpected behavior when restarting '
'from checkpoints.')
cudnn.benchmark = True
# Data loading code
normalize = T.Normalize(mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225])
train_transform = T.Compose([
ResizeImage(256),
T.RandomCrop(224),
T.RandomHorizontalFlip(),
T.ColorJitter(brightness=0.7, contrast=0.7, saturation=0.7, hue=0.5),
T.RandomGrayscale(),
T.ToTensor(),
normalize
])
val_transform = T.Compose([
ResizeImage(256),
T.CenterCrop(224),
T.ToTensor(),
normalize
])
train_source_dataset, train_target_dataset, val_dataset, test_dataset, num_classes, args.class_names = \
utils.get_dataset(args.data, args.root, args.source, args.target,
train_transform, val_transform, MultipleApply([train_transform, val_transform]))
train_source_loader = DataLoader(train_source_dataset, batch_size=args.batch_size,
shuffle=True, num_workers=args.workers, drop_last=True)
train_target_loader = DataLoader(train_target_dataset, batch_size=args.batch_size,
shuffle=True, num_workers=args.workers, drop_last=True)
val_loader = DataLoader(val_dataset, batch_size=args.batch_size, shuffle=False, num_workers=args.workers)
test_loader = DataLoader(test_dataset, batch_size=args.batch_size, shuffle=False, num_workers=args.workers)
train_source_iter = ForeverDataIterator(train_source_loader)
train_target_iter = ForeverDataIterator(train_target_loader)
# create model
print("=> using model '{}'".format(args.arch))
backbone = utils.get_model(args.arch, pretrain=not args.scratch)
pool_layer = nn.Identity() if args.no_pool else None
classifier = ImageClassifier(backbone, num_classes, bottleneck_dim=args.bottleneck_dim,
pool_layer=pool_layer, finetune=not args.scratch).to(device)
# define optimizer and lr scheduler
optimizer = Adam(classifier.get_parameters(), args.lr)
lr_scheduler = LambdaLR(optimizer, lambda x: args.lr * (1. + args.lr_gamma * float(x)) ** (-args.lr_decay))
# resume from the best checkpoint
if args.phase != 'train':
checkpoint = torch.load(logger.get_checkpoint_path('best'), map_location='cpu')
classifier.load_state_dict(checkpoint)
# analysis the model
if args.phase == 'analysis':
# extract features from both domains
feature_extractor = nn.Sequential(classifier.backbone, classifier.pool_layer, classifier.bottleneck).to(device)
source_feature = collect_feature(train_source_loader, feature_extractor, device)
target_feature = collect_feature(train_target_loader, feature_extractor, device)
# plot t-SNE
tSNE_filename = osp.join(logger.visualize_directory, 'TSNE.pdf')
tsne.visualize(source_feature, target_feature, tSNE_filename)
print("Saving t-SNE to", tSNE_filename)
# calculate A-distance, which is a measure for distribution discrepancy
A_distance = a_distance.calculate(source_feature, target_feature, device)
print("A-distance =", A_distance)
return
if args.phase == 'test':
acc1 = utils.validate(test_loader, classifier, args, device)
print(acc1)
return
if args.pretrain is None:
# first pretrain the classifier wish source data
print("Pretraining the model on source domain.")
args.pretrain = logger.get_checkpoint_path('pretrain')
pretrain_model = ImageClassifier(backbone, num_classes, bottleneck_dim=args.bottleneck_dim,
pool_layer=pool_layer, finetune=not args.scratch).to(device)
pretrain_optimizer = Adam(pretrain_model.get_parameters(), args.pretrain_lr)
pretrain_lr_scheduler = LambdaLR(pretrain_optimizer,
lambda x: args.pretrain_lr * (1. + args.lr_gamma * float(x)) ** (
-args.lr_decay))
# start pretraining
for epoch in range(args.pretrain_epochs):
# pretrain for one epoch
utils.pretrain(train_source_iter, pretrain_model, pretrain_optimizer, pretrain_lr_scheduler, epoch, args,
device)
# validate to show pretrain process
utils.validate(val_loader, pretrain_model, args, device)
torch.save(pretrain_model.state_dict(), args.pretrain)
print("Pretraining process is done.")
checkpoint = torch.load(args.pretrain, map_location='cpu')
classifier.load_state_dict(checkpoint)
teacher = EmaTeacher(classifier, alpha=args.alpha)
consistent_loss = L2ConsistencyLoss().to(device)
class_balance_loss = ClassBalanceLoss(num_classes).to(device)
# start training
best_acc1 = 0.
for epoch in range(args.epochs):
print(lr_scheduler.get_lr())
# train for one epoch
train(train_source_iter, train_target_iter, classifier, teacher, consistent_loss, class_balance_loss, optimizer,
lr_scheduler, epoch, args)
# evaluate on validation set
acc1 = utils.validate(val_loader, classifier, args, device)
# remember best acc@1 and save checkpoint
torch.save(classifier.state_dict(), logger.get_checkpoint_path('latest'))
if acc1 > best_acc1:
shutil.copy(logger.get_checkpoint_path('latest'), logger.get_checkpoint_path('best'))
best_acc1 = max(acc1, best_acc1)
print("best_acc1 = {:3.1f}".format(best_acc1))
# evaluate on test set
classifier.load_state_dict(torch.load(logger.get_checkpoint_path('best')))
acc1 = utils.validate(test_loader, classifier, args, device)
print("test_acc1 = {:3.1f}".format(acc1))
logger.close()
def train(train_source_iter: ForeverDataIterator, train_target_iter: ForeverDataIterator, model: ImageClassifier,
teacher: EmaTeacher, consistent_loss, class_balance_loss,
optimizer: Adam, lr_scheduler: LambdaLR, epoch: int, args: argparse.Namespace):
batch_time = AverageMeter('Time', ':3.1f')
data_time = AverageMeter('Data', ':3.1f')
cls_losses = AverageMeter('Cls Loss', ':3.2f')
cons_losses = AverageMeter('Cons Loss', ':3.2f')
cls_accs = AverageMeter('Cls Acc', ':3.1f')
tgt_accs = AverageMeter('Tgt Acc', ':3.1f')
progress = ProgressMeter(
args.iters_per_epoch,
[batch_time, data_time, cls_losses, cons_losses, cls_accs, tgt_accs],
prefix="Epoch: [{}]".format(epoch))
# switch to train mode
model.train()
teacher.train()
end = time.time()
for i in range(args.iters_per_epoch):
x_s, labels_s = next(train_source_iter)
(x_t1, x_t2), labels_t = next(train_target_iter)
x_s = x_s.to(device)
x_t1 = x_t1.to(device)
x_t2 = x_t2.to(device)
labels_s = labels_s.to(device)
labels_t = labels_t.to(device)
# measure data loading time
data_time.update(time.time() - end)
# compute output
y_s, _ = model(x_s)
y_t, _ = model(x_t1)
y_t_teacher, _ = teacher(x_t2)
# classification loss
cls_loss = F.cross_entropy(y_s, labels_s)
# compute output and mask
y_t = F.softmax(y_t, dim=1)
y_t_teacher = F.softmax(y_t_teacher, dim=1)
max_prob, _ = y_t_teacher.max(dim=1)
mask = (max_prob > args.threshold).float()
# consistent loss
cons_loss = consistent_loss(y_t, y_t_teacher, mask)
# balance loss
balance_loss = class_balance_loss(y_t) * mask.mean()
loss = cls_loss + args.trade_off_cons * cons_loss + args.trade_off_balance * balance_loss
# compute gradient and do SGD step
optimizer.zero_grad()
loss.backward()
optimizer.step()
lr_scheduler.step()
# update teacher
teacher.update()
# update statistics
cls_acc = accuracy(y_s, labels_s)[0]
tgt_acc = accuracy(y_t, labels_t)[0]
cls_losses.update(cls_loss.item(), x_s.size(0))
cons_losses.update(cons_loss.item(), x_s.size(0))
cls_accs.update(cls_acc.item(), x_s.size(0))
tgt_accs.update(tgt_acc.item(), x_s.size(0))
# measure elapsed time
batch_time.update(time.time() - end)
end = time.time()
if i % args.print_freq == 0:
progress.display(i)
if __name__ == '__main__':
parser = argparse.ArgumentParser(description='Self Ensemble for Unsupervised Domain Adaptation')
# dataset parameters
parser.add_argument('root', metavar='DIR',
help='root path of dataset')
parser.add_argument('-d', '--data', metavar='DATA', default='Office31', choices=utils.get_dataset_names(),
help='dataset: ' + ' | '.join(utils.get_dataset_names()) +
' (default: Office31)')
parser.add_argument('-s', '--source', help='source domain(s)', nargs='+')
parser.add_argument('-t', '--target', help='target domain(s)', nargs='+')
parser.add_argument('--train-resizing', type=str, default='default')
parser.add_argument('--val-resizing', type=str, default='default')
parser.add_argument('--resize-size', type=int, default=224,
help='the image size after resizing')
parser.add_argument('--no-hflip', action='store_true',
help='no random horizontal flipping during training')
parser.add_argument('--norm-mean', type=float, nargs='+',
default=(0.485, 0.456, 0.406), help='normalization mean')
parser.add_argument('--norm-std', type=float, nargs='+',
default=(0.229, 0.224, 0.225), help='normalization std')
# model parameters
parser.add_argument('-a', '--arch', metavar='ARCH', default='resnet18',
choices=utils.get_model_names(),
help='backbone architecture: ' +
' | '.join(utils.get_model_names()) +
' (default: resnet18)')
parser.add_argument('--pretrain', type=str, default=None,
help='pretrain checkpoint for classification model')
parser.add_argument('--bottleneck-dim', default=256, type=int,
help='Dimension of bottleneck')
parser.add_argument('--no-pool', action='store_true',
help='no pool layer after the feature extractor.')
parser.add_argument('--scratch', action='store_true', help='whether train from scratch.')
# training parameters
parser.add_argument('-b', '--batch-size', default=36, type=int,
metavar='N',
help='mini-batch size (default: 36)')
parser.add_argument('--lr', '--learning-rate', default=1e-4, type=float,
metavar='LR', help='initial learning rate', dest='lr')
parser.add_argument('--pretrain-lr', '--pretrain-learning-rate', default=3e-5, type=float,
help='initial pretrain learning rate', dest='pretrain_lr')
parser.add_argument('--lr-gamma', default=0.001, type=float, help='parameter for lr scheduler')
parser.add_argument('--lr-decay', default=0.75, type=float, help='parameter for lr scheduler')
parser.add_argument('--alpha', default=0.99, type=float, help='ema decay rate (default: 0.99)')
parser.add_argument('--threshold', default=0.8, type=float, help='confidence threshold')
parser.add_argument('--trade-off-cons', default=3, type=float, help='trade off parameter for consistent loss')
parser.add_argument('--trade-off-balance', default=0.01, type=float,
help='trade off parameter for class balance loss')
parser.add_argument('-j', '--workers', default=2, type=int, metavar='N',
help='number of data loading workers (default: 2)')
parser.add_argument('--pretrain-epochs', default=0, type=int, metavar='N',
help='number of total epochs(pretrain) to run')
parser.add_argument('--epochs', default=10, type=int, metavar='N',
help='number of total epochs to run')
parser.add_argument('-i', '--iters-per-epoch', default=1000, type=int,
help='Number of iterations per epoch')
parser.add_argument('-p', '--print-freq', default=100, type=int,
metavar='N', help='print frequency (default: 100)')
parser.add_argument('--seed', default=None, type=int,
help='seed for initializing training. ')
parser.add_argument('--per-class-eval', action='store_true',
help='whether output per-class accuracy during evaluation')
parser.add_argument("--log", type=str, default='self_ensemble',
help="Where to save logs, checkpoints and debugging images.")
parser.add_argument("--phase", type=str, default='train', choices=['train', 'test', 'analysis'],
help="When phase is 'test', only test the model."
"When phase is 'analysis', only analysis the model.")
args = parser.parse_args()
main(args)
================================================
FILE: domain_generalization/Transfer-Learning-Library/examples/domain_adaptation/image_classification/self_ensemble.sh
================================================
#!/usr/bin/env bash
# ResNet50, Office31, Single Source
CUDA_VISIBLE_DEVICES=0 python self_ensemble.py data/office31 -d Office31 -s A -t W -a resnet50 --seed 1 --log logs/self_ensemble/Office31_A2W
CUDA_VISIBLE_DEVICES=0 python self_ensemble.py data/office31 -d Office31 -s D -t W -a resnet50 --seed 1 --log logs/self_ensemble/Office31_D2W
CUDA_VISIBLE_DEVICES=0 python self_ensemble.py data/office31 -d Office31 -s W -t D -a resnet50 --seed 1 --log logs/self_ensemble/Office31_W2D
CUDA_VISIBLE_DEVICES=0 python self_ensemble.py data/office31 -d Office31 -s A -t D -a resnet50 --seed 1 --log logs/self_ensemble/Office31_A2D
CUDA_VISIBLE_DEVICES=0 python self_ensemble.py data/office31 -d Office31 -s D -t A -a resnet50 --seed 1 --log logs/self_ensemble/Office31_D2A
CUDA_VISIBLE_DEVICES=0 python self_ensemble.py data/office31 -d Office31 -s W -t A -a resnet50 --seed 1 --log logs/self_ensemble/Office31_W2A
# ResNet50, Office-Home, Single Source
CUDA_VISIBLE_DEVICES=0 python self_ensemble.py data/office-home -d OfficeHome -s Ar -t Cl -a resnet50 --seed 0 --log logs/self_ensemble/OfficeHome_Ar2Cl
CUDA_VISIBLE_DEVICES=0 python self_ensemble.py data/office-home -d OfficeHome -s Ar -t Pr -a resnet50 --seed 0 --log logs/self_ensemble/OfficeHome_Ar2Pr
CUDA_VISIBLE_DEVICES=0 python self_ensemble.py data/office-home -d OfficeHome -s Ar -t Rw -a resnet50 --seed 0 --log logs/self_ensemble/OfficeHome_Ar2Rw
CUDA_VISIBLE_DEVICES=0 python self_ensemble.py data/office-home -d OfficeHome -s Cl -t Ar -a resnet50 --seed 0 --log logs/self_ensemble/OfficeHome_Cl2Ar
CUDA_VISIBLE_DEVICES=0 python self_ensemble.py data/office-home -d OfficeHome -s Cl -t Pr -a resnet50 --seed 0 --log logs/self_ensemble/OfficeHome_Cl2Pr
CUDA_VISIBLE_DEVICES=0 python self_ensemble.py data/office-home -d OfficeHome -s Cl -t Rw -a resnet50 --seed 0 --log logs/self_ensemble/OfficeHome_Cl2Rw
CUDA_VISIBLE_DEVICES=0 python self_ensemble.py data/office-home -d OfficeHome -s Pr -t Ar -a resnet50 --seed 0 --log logs/self_ensemble/OfficeHome_Pr2Ar
CUDA_VISIBLE_DEVICES=0 python self_ensemble.py data/office-home -d OfficeHome -s Pr -t Cl -a resnet50 --seed 0 --log logs/self_ensemble/OfficeHome_Pr2Cl
CUDA_VISIBLE_DEVICES=0 python self_ensemble.py data/office-home -d OfficeHome -s Pr -t Rw -a resnet50 --seed 0 --log logs/self_ensemble/OfficeHome_Pr2Rw
CUDA_VISIBLE_DEVICES=0 python self_ensemble.py data/office-home -d OfficeHome -s Rw -t Ar -a resnet50 --seed 0 --log logs/self_ensemble/OfficeHome_Rw2Ar
CUDA_VISIBLE_DEVICES=0 python self_ensemble.py data/office-home -d OfficeHome -s Rw -t Cl -a resnet50 --seed 0 --log logs/self_ensemble/OfficeHome_Rw2Cl
CUDA_VISIBLE_DEVICES=0 python self_ensemble.py data/office-home -d OfficeHome -s Rw -t Pr -a resnet50 --seed 0 --log logs/self_ensemble/OfficeHome_Rw2Pr
# ResNet101, VisDA-2017, Single Source
CUDA_VISIBLE_DEVICES=0 python self_ensemble.py data/visda-2017 -d VisDA2017 -s Synthetic -t Real -a resnet101 \
--epochs 20 --seed 0 --per-class-eval --log logs/self_ensemble/VisDA2017 --lr-gamma 0.0002 -b 32
# Office-Home on Vision Transformer
CUDA_VISIBLE_DEVICES=0 python self_ensemble.py data/office-home -d OfficeHome -s Ar -t Cl -a vit_base_patch16_224 --no-pool --epochs 30 --seed 0 -b 24 --log logs/self_ensemble_vit/OfficeHome_Ar2Cl
CUDA_VISIBLE_DEVICES=0 python self_ensemble.py data/office-home -d OfficeHome -s Ar -t Pr -a vit_base_patch16_224 --no-pool --epochs 30 --seed 0 -b 24 --log logs/self_ensemble_vit/OfficeHome_Ar2Pr
CUDA_VISIBLE_DEVICES=0 python self_ensemble.py data/office-home -d OfficeHome -s Ar -t Rw -a vit_base_patch16_224 --no-pool --epochs 30 --seed 0 -b 24 --log logs/self_ensemble_vit/OfficeHome_Ar2Rw
CUDA_VISIBLE_DEVICES=0 python self_ensemble.py data/office-home -d OfficeHome -s Cl -t Ar -a vit_base_patch16_224 --no-pool --epochs 30 --seed 0 -b 24 --log logs/self_ensemble_vit/OfficeHome_Cl2Ar
CUDA_VISIBLE_DEVICES=0 python self_ensemble.py data/office-home -d OfficeHome -s Cl -t Pr -a vit_base_patch16_224 --no-pool --epochs 30 --seed 0 -b 24 --log logs/self_ensemble_vit/OfficeHome_Cl2Pr
CUDA_VISIBLE_DEVICES=0 python self_ensemble.py data/office-home -d OfficeHome -s Cl -t Rw -a vit_base_patch16_224 --no-pool --epochs 30 --seed 0 -b 24 --log logs/self_ensemble_vit/OfficeHome_Cl2Rw
CUDA_VISIBLE_DEVICES=0 python self_ensemble.py data/office-home -d OfficeHome -s Pr -t Ar -a vit_base_patch16_224 --no-pool --epochs 30 --seed 0 -b 24 --log logs/self_ensemble_vit/OfficeHome_Pr2Ar
CUDA_VISIBLE_DEVICES=0 python self_ensemble.py data/office-home -d OfficeHome -s Pr -t Cl -a vit_base_patch16_224 --no-pool --epochs 30 --seed 0 -b 24 --log logs/self_ensemble_vit/OfficeHome_Pr2Cl
CUDA_VISIBLE_DEVICES=0 python self_ensemble.py data/office-home -d OfficeHome -s Pr -t Rw -a vit_base_patch16_224 --no-pool --epochs 30 --seed 0 -b 24 --log logs/self_ensemble_vit/OfficeHome_Pr2Rw
CUDA_VISIBLE_DEVICES=0 python self_ensemble.py data/office-home -d OfficeHome -s Rw -t Ar -a vit_base_patch16_224 --no-pool --epochs 30 --seed 0 -b 24 --log logs/self_ensemble_vit/OfficeHome_Rw2Ar
CUDA_VISIBLE_DEVICES=0 python self_ensemble.py data/office-home -d OfficeHome -s Rw -t Cl -a vit_base_patch16_224 --no-pool --epochs 30 --seed 0 -b 24 --log logs/self_ensemble_vit/OfficeHome_Rw2Cl
CUDA_VISIBLE_DEVICES=0 python self_ensemble.py data/office-home -d OfficeHome -s Rw -t Pr -a vit_base_patch16_224 --no-pool --epochs 30 --seed 0 -b 24 --log logs/self_ensemble_vit/OfficeHome_Rw2Pr
================================================
FILE: domain_generalization/Transfer-Learning-Library/examples/domain_adaptation/image_classification/source_only.py
================================================
"""
@author: Junguang Jiang
@contact: JiangJunguang1123@outlook.com
"""
import random
import time
import warnings
import sys
import argparse
import shutil
import os.path as osp
import torch
import torch.nn as nn
import torch.backends.cudnn as cudnn
from torch.optim import SGD
from torch.optim.lr_scheduler import LambdaLR
from torch.utils.data import DataLoader
import torch.nn.functional as F
sys.path.append('../../..')
from common.modules.classifier import Classifier
from common.utils.data import ForeverDataIterator
from common.utils.metric import accuracy
from common.utils.meter import AverageMeter, ProgressMeter
from common.utils.logger import CompleteLogger
from common.utils.analysis import collect_feature, tsne, a_distance
sys.path.append('.')
import utils
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
class TransformerDecorator(torch.nn.Module):
def __init__(self, backbone=None, add_bt=3, dim=2048, eval_global=0):
super(TransformerDecorator, self).__init__()
self.encoder_layers = torch.nn.TransformerEncoderLayer(dim, 4, dim, 0.5)
self.eval_global = eval_global
self.add_bt = add_bt
self.backbone = backbone
def forward_feats(self, feature):
feature = self.backbone(feature)
if self.training and self.add_bt > 0:
pre_feature = feature
feature = feature.unsqueeze(1)
feature = self.encoder_layers(feature)
# feature = self.encoder_layers(feature, feature, feature)
feature = feature.squeeze(1)
if self.add_bt >= 1:
return torch.cat([pre_feature, feature], dim=0)
return feature
def forward(self, feature):
feature = self.forward_feats(feature)
return feature
def main(args: argparse.Namespace):
logger = CompleteLogger(args.log, args.phase)
print(args)
if args.seed is not None:
random.seed(args.seed)
torch.manual_seed(args.seed)
cudnn.deterministic = True
warnings.warn('You have chosen to seed training. '
'This will turn on the CUDNN deterministic setting, '
'which can slow down your training considerably! '
'You may see unexpected behavior when restarting '
'from checkpoints.')
cudnn.benchmark = True
# Data loading code
train_transform = utils.get_train_transform(args.train_resizing, random_horizontal_flip=not args.no_hflip,
random_color_jitter=False, resize_size=args.resize_size,
norm_mean=args.norm_mean, norm_std=args.norm_std)
val_transform = utils.get_val_transform(args.val_resizing, resize_size=args.resize_size,
norm_mean=args.norm_mean, norm_std=args.norm_std)
print("train_transform: ", train_transform)
print("val_transform: ", val_transform)
train_source_dataset, train_target_dataset, val_dataset, test_dataset, num_classes, args.class_names = \
utils.get_dataset(args.data, args.root, args.source, args.target, train_transform, val_transform)
train_source_loader = DataLoader(train_source_dataset, batch_size=args.batch_size,
shuffle=True, num_workers=args.workers, drop_last=True)
train_target_loader = DataLoader(train_target_dataset, batch_size=args.batch_size,
shuffle=True, num_workers=args.workers, drop_last=True)
val_loader = DataLoader(val_dataset, batch_size=args.batch_size, shuffle=False, num_workers=args.workers)
test_loader = DataLoader(test_dataset, batch_size=args.batch_size, shuffle=False, num_workers=args.workers)
train_source_iter = ForeverDataIterator(train_source_loader)
# create model
print("=> using model '{}'".format(args.arch))
backbone = utils.get_model(args.arch, pretrain=not args.scratch)
pool_layer = nn.Identity() if args.no_pool else None
pool_layer = nn.Sequential(
nn.AdaptiveAvgPool2d(output_size=(1, 1)),
nn.Flatten(),
)
pool_layer = TransformerDecorator(pool_layer, add_bt=args.add_bt, dim=backbone.out_features)
classifier = Classifier(backbone, num_classes, pool_layer=pool_layer, finetune=not args.scratch).to(device)
# define optimizer and lr scheduler
optimizer = SGD(classifier.get_parameters(), args.lr, momentum=args.momentum, weight_decay=args.wd, nesterov=True)
lr_scheduler = LambdaLR(optimizer, lambda x: args.lr * (1. + args.lr_gamma * float(x)) ** (-args.lr_decay))
# resume from the best checkpoint
if args.phase != 'train':
checkpoint = torch.load(logger.get_checkpoint_path('best'), map_location='cpu')
classifier.load_state_dict(checkpoint)
# analysis the model
if args.phase == 'analysis':
# extract features from both domains
feature_extractor = nn.Sequential(classifier.backbone, classifier.pool_layer, classifier.bottleneck).to(device)
source_feature = collect_feature(train_source_loader, feature_extractor, device)
target_feature = collect_feature(train_target_loader, feature_extractor, device)
# plot t-SNE
tSNE_filename = osp.join(logger.visualize_directory, 'TSNE.pdf')
tsne.visualize(source_feature, target_feature, tSNE_filename)
print("Saving t-SNE to", tSNE_filename)
# calculate A-distance, which is a measure for distribution discrepancy
A_distance = a_distance.calculate(source_feature, target_feature, device)
print("A-distance =", A_distance)
return
if args.phase == 'test':
acc1 = utils.validate(test_loader, classifier, args, device)
print(acc1)
return
# start training
best_acc1 = 0.
for epoch in range(args.epochs):
print(lr_scheduler.get_lr())
# train for one epoch
utils.pretrain(train_source_iter, classifier, optimizer, lr_scheduler, epoch, args, device)
# evaluate on validation set
acc1 = utils.validate(val_loader, classifier, args, device)
# remember best acc@1 and save checkpoint
torch.save(classifier.state_dict(), logger.get_checkpoint_path('latest'))
if acc1 > best_acc1:
shutil.copy(logger.get_checkpoint_path('latest'), logger.get_checkpoint_path('best'))
best_acc1 = max(acc1, best_acc1)
print("best_acc1 = {:3.1f}".format(best_acc1))
# evaluate on test set
classifier.load_state_dict(torch.load(logger.get_checkpoint_path('best')))
acc1 = utils.validate(test_loader, classifier, args, device)
print("test_acc1 = {:3.1f}".format(acc1))
logger.close()
if __name__ == '__main__':
parser = argparse.ArgumentParser(description='Source Only for Unsupervised Domain Adaptation')
# dataset parameters
parser.add_argument('root', metavar='DIR',
help='root path of dataset')
parser.add_argument('-d', '--data', metavar='DATA', default='Office31', choices=utils.get_dataset_names(),
help='dataset: ' + ' | '.join(utils.get_dataset_names()) +
' (default: Office31)')
parser.add_argument('-s', '--source', help='source domain(s)', nargs='+')
parser.add_argument('-t', '--target', help='target domain(s)', nargs='+')
parser.add_argument('--train-resizing', type=str, default='default')
parser.add_argument('--val-resizing', type=str, default='default')
parser.add_argument('--resize-size', type=int, default=224,
help='the image size after resizing')
parser.add_argument('--no-hflip', action='store_true',
help='no random horizontal flipping during training')
parser.add_argument('--norm-mean', type=float, nargs='+',
default=(0.485, 0.456, 0.406), help='normalization mean')
parser.add_argument('--norm-std', type=float, nargs='+',
default=(0.229, 0.224, 0.225), help='normalization std')
# model parameters
parser.add_argument('-a', '--arch', metavar='ARCH', default='resnet18',
choices=utils.get_model_names(),
help='backbone architecture: ' +
' | '.join(utils.get_model_names()) +
' (default: resnet18)')
parser.add_argument('--no-pool', action='store_true',
help='no pool layer after the feature extractor.')
parser.add_argument('--scratch', action='store_true', help='whether train from scratch.')
# training parameters
parser.add_argument('-b', '--batch-size', default=32, type=int,
metavar='N',
help='mini-batch size (default: 32)')
parser.add_argument('--lr', '--learning-rate', default=0.001, type=float,
metavar='LR', help='initial learning rate', dest='lr')
parser.add_argument('--lr-gamma', default=0.0003, type=float, help='parameter for lr scheduler')
parser.add_argument('--lr-decay', default=0.75, type=float, help='parameter for lr scheduler')
parser.add_argument('--momentum', default=0.9, type=float, metavar='M',
help='momentum')
parser.add_argument('--wd', '--weight-decay', default=0.0005, type=float,
metavar='W', help='weight decay (default: 5e-4)')
parser.add_argument('-j', '--workers', default=2, type=int, metavar='N',
help='number of data loading workers (default: 2)')
parser.add_argument('--epochs', default=20, type=int, metavar='N',
help='number of total epochs to run')
parser.add_argument('-i', '--iters-per-epoch', default=500, type=int,
help='Number of iterations per epoch')
parser.add_argument('-p', '--print-freq', default=100, type=int,
metavar='N', help='print frequency (default: 100)')
parser.add_argument('--seed', default=None, type=int,
help='seed for initializing training. ')
parser.add_argument('--per-class-eval', action='store_true',
help='whether output per-class accuracy during evaluation')
parser.add_argument("--log", type=str, default='src_only',
help="Where to save logs, checkpoints and debugging images.")
parser.add_argument("--phase", type=str, default='train', choices=['train', 'test', 'analysis'],
help="When phase is 'test', only test the model."
"When phase is 'analysis', only analysis the model.")
parser.add_argument("--add_bt", type=int, default=0,
help="add_bt")
args = parser.parse_args()
main(args)
================================================
FILE: domain_generalization/Transfer-Learning-Library/examples/domain_adaptation/image_classification/source_only.sh
================================================
#!/usr/bin/env bash
# ResNet50, Office31, Single Source
CUDA_VISIBLE_DEVICES=0 python source_only.py data/office31 -d Office31 -s A -t W -a resnet50 --epochs 5 --seed 0 --log logs/src_only/Office31_A2W
CUDA_VISIBLE_DEVICES=0 python source_only.py data/office31 -d Office31 -s D -t W -a resnet50 --epochs 5 --seed 0 --log logs/src_only/Office31_D2W
CUDA_VISIBLE_DEVICES=0 python source_only.py data/office31 -d Office31 -s W -t D -a resnet50 --epochs 5 --seed 0 --log logs/src_only/Office31_W2D
CUDA_VISIBLE_DEVICES=0 python source_only.py data/office31 -d Office31 -s A -t D -a resnet50 --epochs 5 --seed 0 --log logs/src_only/Office31_A2D
CUDA_VISIBLE_DEVICES=0 python source_only.py data/office31 -d Office31 -s D -t A -a resnet50 --epochs 5 --seed 0 --log logs/src_only/Office31_D2A
CUDA_VISIBLE_DEVICES=0 python source_only.py data/office31 -d Office31 -s W -t A -a resnet50 --epochs 5 --seed 0 --log logs/src_only/Office31_W2A
# ResNet50, Office-Home, Single Source
CUDA_VISIBLE_DEVICES=0 python source_only.py data/office-home -d OfficeHome -s Ar -t Cl -a resnet50 --epochs 5 -i 500 --seed 0 --log logs/src_only/OfficeHome_Ar2Cl
CUDA_VISIBLE_DEVICES=0 python source_only.py data/office-home -d OfficeHome -s Ar -t Pr -a resnet50 --epochs 5 -i 500 --seed 0 --log logs/src_only/OfficeHome_Ar2Pr
CUDA_VISIBLE_DEVICES=0 python source_only.py data/office-home -d OfficeHome -s Ar -t Rw -a resnet50 --epochs 5 -i 500 --seed 0 --log logs/src_only/OfficeHome_Ar2Rw
CUDA_VISIBLE_DEVICES=0 python source_only.py data/office-home -d OfficeHome -s Cl -t Ar -a resnet50 --epochs 5 -i 500 --seed 0 --log logs/src_only/OfficeHome_Cl2Ar
CUDA_VISIBLE_DEVICES=0 python source_only.py data/office-home -d OfficeHome -s Cl -t Pr -a resnet50 --epochs 5 -i 500 --seed 0 --log logs/src_only/OfficeHome_Cl2Pr
CUDA_VISIBLE_DEVICES=0 python source_only.py data/office-home -d OfficeHome -s Cl -t Rw -a resnet50 --epochs 5 -i 500 --seed 0 --log logs/src_only/OfficeHome_Cl2Rw
CUDA_VISIBLE_DEVICES=0 python source_only.py data/office-home -d OfficeHome -s Pr -t Ar -a resnet50 --epochs 5 -i 500 --seed 0 --log logs/src_only/OfficeHome_Pr2Ar
CUDA_VISIBLE_DEVICES=0 python source_only.py data/office-home -d OfficeHome -s Pr -t Cl -a resnet50 --epochs 5 -i 500 --seed 0 --log logs/src_only/OfficeHome_Pr2Cl
CUDA_VISIBLE_DEVICES=0 python source_only.py data/office-home -d OfficeHome -s Pr -t Rw -a resnet50 --epochs 5 -i 500 --seed 0 --log logs/src_only/OfficeHome_Pr2Rw
CUDA_VISIBLE_DEVICES=0 python source_only.py data/office-home -d OfficeHome -s Rw -t Ar -a resnet50 --epochs 5 -i 500 --seed 0 --log logs/src_only/OfficeHome_Rw2Ar
CUDA_VISIBLE_DEVICES=0 python source_only.py data/office-home -d OfficeHome -s Rw -t Cl -a resnet50 --epochs 5 -i 500 --seed 0 --log logs/src_only/OfficeHome_Rw2Cl
CUDA_VISIBLE_DEVICES=0 python source_only.py data/office-home -d OfficeHome -s Rw -t Pr -a resnet50 --epochs 5 -i 500 --seed 0 --log logs/src_only/OfficeHome_Rw2Pr
# ResNet101, VisDA-2017, Single Source
CUDA_VISIBLE_DEVICES=0 python source_only.py data/visda-2017 -d VisDA2017 -s Synthetic -t Real -a resnet101 \
--epochs 20 -i 1000 --seed 0 --per-class-eval --train-resizing cen.crop --log logs/src_only/VisDA2017
# ResNet101, DomainNet, Oracle
CUDA_VISIBLE_DEVICES=0 python source_only.py data/domainnet -d DomainNet -s c -t c -a resnet101 --epochs 20 -i 2500 --seed 0 --lr 0.01 --log logs/oracle/DomainNet_c
CUDA_VISIBLE_DEVICES=0 python source_only.py data/domainnet -d DomainNet -s i -t i -a resnet101 --epochs 20 -i 2500 --seed 0 --lr 0.01 --log logs/oracle/DomainNet_i
CUDA_VISIBLE_DEVICES=0 python source_only.py data/domainnet -d DomainNet -s p -t p -a resnet101 --epochs 20 -i 2500 --seed 0 --lr 0.01 --log logs/oracle/DomainNet_p
CUDA_VISIBLE_DEVICES=0 python source_only.py data/domainnet -d DomainNet -s q -t q -a resnet101 --epochs 20 -i 2500 --seed 0 --lr 0.01 --log logs/oracle/DomainNet_q
CUDA_VISIBLE_DEVICES=0 python source_only.py data/domainnet -d DomainNet -s r -t r -a resnet101 --epochs 20 -i 2500 --seed 0 --lr 0.01 --log logs/oracle/DomainNet_r
CUDA_VISIBLE_DEVICES=0 python source_only.py data/domainnet -d DomainNet -s s -t s -a resnet101 --epochs 20 -i 2500 --seed 0 --lr 0.01 --log logs/oracle/DomainNet_s
# ResNet101, DomainNet, Single Source
CUDA_VISIBLE_DEVICES=0 python source_only.py data/domainnet -d DomainNet -s c -t p -a resnet101 --epochs 20 -i 2500 --seed 0 --lr 0.01 --log logs/src_only/DomainNet_c2p
CUDA_VISIBLE_DEVICES=0 python source_only.py data/domainnet -d DomainNet -s c -t r -a resnet101 --epochs 20 -i 2500 --seed 0 --lr 0.01 --log logs/src_only/DomainNet_c2r
CUDA_VISIBLE_DEVICES=0 python source_only.py data/domainnet -d DomainNet -s c -t s -a resnet101 --epochs 20 -i 2500 --seed 0 --lr 0.01 --log logs/src_only/DomainNet_c2s
CUDA_VISIBLE_DEVICES=0 python source_only.py data/domainnet -d DomainNet -s p -t c -a resnet101 --epochs 20 -i 2500 --seed 0 --lr 0.01 --log logs/src_only/DomainNet_p2c
CUDA_VISIBLE_DEVICES=0 python source_only.py data/domainnet -d DomainNet -s p -t r -a resnet101 --epochs 20 -i 2500 --seed 0 --lr 0.01 --log logs/src_only/DomainNet_p2r
CUDA_VISIBLE_DEVICES=0 python source_only.py data/domainnet -d DomainNet -s p -t s -a resnet101 --epochs 20 -i 2500 --seed 0 --lr 0.01 --log logs/src_only/DomainNet_p2s
CUDA_VISIBLE_DEVICES=0 python source_only.py data/domainnet -d DomainNet -s r -t c -a resnet101 --epochs 20 -i 2500 --seed 0 --lr 0.01 --log logs/src_only/DomainNet_r2c
CUDA_VISIBLE_DEVICES=0 python source_only.py data/domainnet -d DomainNet -s r -t p -a resnet101 --epochs 20 -i 2500 --seed 0 --lr 0.01 --log logs/src_only/DomainNet_r2p
CUDA_VISIBLE_DEVICES=0 python source_only.py data/domainnet -d DomainNet -s r -t s -a resnet101 --epochs 20 -i 2500 --seed 0 --lr 0.01 --log logs/src_only/DomainNet_r2s
CUDA_VISIBLE_DEVICES=0 python source_only.py data/domainnet -d DomainNet -s s -t c -a resnet101 --epochs 20 -i 2500 --seed 0 --lr 0.01 --log logs/src_only/DomainNet_s2c
CUDA_VISIBLE_DEVICES=0 python source_only.py data/domainnet -d DomainNet -s s -t p -a resnet101 --epochs 20 -i 2500 --seed 0 --lr 0.01 --log logs/src_only/DomainNet_s2p
CUDA_VISIBLE_DEVICES=0 python source_only.py data/domainnet -d DomainNet -s s -t r -a resnet101 --epochs 20 -i 2500 --seed 0 --lr 0.01 --log logs/src_only/DomainNet_s2r
# ResNet50, ImageNet200 -> ImageNetR
CUDA_VISIBLE_DEVICES=0 python source_only.py data/ImageNetR -d ImageNetR -s IN -t INR -a resnet50 --epochs 20 -i 2500 --seed 0 --log logs/src_only/ImageNet_IN2INR
# ig_resnext101_32x8d, ImageNet -> ImageNetSketch
CUDA_VISIBLE_DEVICES=0 python source_only.py data/imagenet-sketch -d ImageNetSketch -s IN -t sketch -a ig_resnext101_32x8d --epochs 20 -i 2500 --seed 0 --log logs/src_only_ig_resnext101_32x8d/ImageNet_IN2sketch
# Vision Transformer, Office-Home, Single Source
CUDA_VISIBLE_DEVICES=0 python source_only.py data/office-home -d OfficeHome -s Ar -t Cl -a vit_base_patch16_224 --no-pool --epochs 20 -i 1000 -b 24 --seed 0 --log logs/src_only_vit/OfficeHome_Ar2Cl
CUDA_VISIBLE_DEVICES=0 python source_only.py data/office-home -d OfficeHome -s Ar -t Pr -a vit_base_patch16_224 --no-pool --epochs 20 -i 1000 -b 24 --seed 0 --log logs/src_only_vit/OfficeHome_Ar2Pr
CUDA_VISIBLE_DEVICES=0 python source_only.py data/office-home -d OfficeHome -s Ar -t Rw -a vit_base_patch16_224 --no-pool --epochs 20 -i 1000 -b 24 --seed 0 --log logs/src_only_vit/OfficeHome_Ar2Rw
CUDA_VISIBLE_DEVICES=0 python source_only.py data/office-home -d OfficeHome -s Cl -t Ar -a vit_base_patch16_224 --no-pool --epochs 20 -i 1000 -b 24 --seed 0 --log logs/src_only_vit/OfficeHome_Cl2Ar
CUDA_VISIBLE_DEVICES=0 python source_only.py data/office-home -d OfficeHome -s Cl -t Pr -a vit_base_patch16_224 --no-pool --epochs 20 -i 1000 -b 24 --seed 0 --log logs/src_only_vit/OfficeHome_Cl2Pr
CUDA_VISIBLE_DEVICES=0 python source_only.py data/office-home -d OfficeHome -s Cl -t Rw -a vit_base_patch16_224 --no-pool --epochs 20 -i 1000 -b 24 --seed 0 --log logs/src_only_vit/OfficeHome_Cl2Rw
CUDA_VISIBLE_DEVICES=0 python source_only.py data/office-home -d OfficeHome -s Pr -t Ar -a vit_base_patch16_224 --no-pool --epochs 20 -i 1000 -b 24 --seed 0 --log logs/src_only_vit/OfficeHome_Pr2Ar
CUDA_VISIBLE_DEVICES=0 python source_only.py data/office-home -d OfficeHome -s Pr -t Cl -a vit_base_patch16_224 --no-pool --epochs 20 -i 1000 -b 24 --seed 0 --log logs/src_only_vit/OfficeHome_Pr2Cl
CUDA_VISIBLE_DEVICES=0 python source_only.py data/office-home -d OfficeHome -s Pr -t Rw -a vit_base_patch16_224 --no-pool --epochs 20 -i 1000 -b 24 --seed 0 --log logs/src_only_vit/OfficeHome_Pr2Rw
CUDA_VISIBLE_DEVICES=0 python source_only.py data/office-home -d OfficeHome -s Rw -t Ar -a vit_base_patch16_224 --no-pool --epochs 20 -i 1000 -b 24 --seed 0 --log logs/src_only_vit/OfficeHome_Rw2Ar
CUDA_VISIBLE_DEVICES=0 python source_only.py data/office-home -d OfficeHome -s Rw -t Cl -a vit_base_patch16_224 --no-pool --epochs 20 -i 1000 -b 24 --seed 0 --log logs/src_only_vit/OfficeHome_Rw2Cl
CUDA_VISIBLE_DEVICES=0 python source_only.py data/office-home -d OfficeHome -s Rw -t Pr -a vit_base_patch16_224 --no-pool --epochs 20 -i 1000 -b 24 --seed 0 --log logs/src_only_vit/OfficeHome_Rw2Pr
# ResNet50, Office-Home, Multi Source
CUDA_VISIBLE_DEVICES=0 python source_only.py data/office-home -d OfficeHome -s Cl Pr Rw -t Ar -a resnet50 --epochs 10 -i 1000 --seed 0 --log logs/src_only/OfficeHome_:2Ar
CUDA_VISIBLE_DEVICES=0 python source_only.py data/office-home -d OfficeHome -s Ar Pr Rw -t Cl -a resnet50 --epochs 10 -i 1000 --seed 0 --log logs/src_only/OfficeHome_:2Cl
CUDA_VISIBLE_DEVICES=0 python source_only.py data/office-home -d OfficeHome -s Ar Cl Rw -t Pr -a resnet50 --epochs 10 -i 1000 --seed 0 --log logs/src_only/OfficeHome_:2Pr
CUDA_VISIBLE_DEVICES=0 python source_only.py data/office-home -d OfficeHome -s Ar Cl Pr -t Rw -a resnet50 --epochs 10 -i 1000 --seed 0 --log logs/src_only/OfficeHome_:2Rw
# ResNet101, DomainNet, Multi Source
CUDA_VISIBLE_DEVICES=0 python source_only.py data/domainnet -d DomainNet -s i p q r s -t c -a resnet101 --epochs 20 -i 2500 --seed 0 --lr 0.01 --log logs/src_only/DomainNet_:2c
CUDA_VISIBLE_DEVICES=0 python source_only.py data/domainnet -d DomainNet -s c p q r s -t i -a resnet101 --epochs 20 -i 2500 --seed 0 --lr 0.01 --log logs/src_only/DomainNet_:2i
CUDA_VISIBLE_DEVICES=0 python source_only.py data/domainnet -d DomainNet -s c i q r s -t p -a resnet101 --epochs 20 -i 2500 --seed 0 --lr 0.01 --log logs/src_only/DomainNet_:2p
CUDA_VISIBLE_DEVICES=0 python source_only.py data/domainnet -d DomainNet -s c i p r s -t q -a resnet101 --epochs 20 -i 2500 --seed 0 --lr 0.01 --log logs/src_only/DomainNet_:2q
CUDA_VISIBLE_DEVICES=0 python source_only.py data/domainnet -d DomainNet -s c i p q s -t r -a resnet101 --epochs 20 -i 2500 --seed 0 --lr 0.01 --log logs/src_only/DomainNet_:2r
CUDA_VISIBLE_DEVICES=0 python source_only.py data/domainnet -d DomainNet -s c i p q r -t s -a resnet101 --epochs 20 -i 2500 --seed 0 --lr 0.01 --log logs/src_only/DomainNet_:2s
# ResNet50, Wilds Dataset
CUDA_VISIBLE_DEVICES=0 python source_only.py data/wilds -d iwildcam --train-resizing 'res.' --val-resizing 'res.' \
-a resnet50 --epochs 10 -i 1000 --seed 0 --log logs/src_only/iwildcam
# DenseNet121, Wilds Dataset
CUDA_VISIBLE_DEVICES=0 python source_only.py data/wilds -d camelyon17 --train-resizing 'res.' --val-resizing 'res.' --resize-size 96 \
-a densenet121 --scratch --epochs 10 -i 1000 --lr 0.01 --seed 0 --log logs/src_only/camelyon17
CUDA_VISIBLE_DEVICES=0 python source_only.py data/wilds -d fmow --train-resizing 'res.' --val-resizing 'res.' \
-a densenet121 --epochs 10 -i 1000 --lr 0.01 --seed 0 --log logs/src_only/fmow
# Digits
CUDA_VISIBLE_DEVICES=0 python source_only.py data/digits -d Digits -s MNIST -t USPS --train-resizing 'res.' --val-resizing 'res.' \
--resize-size 28 --no-hflip --norm-mean 0.5 --norm-std 0.5 -a lenet --no-pool --lr 0.01 -b 128 -i 2500 --scratch --seed 0 --log logs/src_only/MNIST2USPS
CUDA_VISIBLE_DEVICES=0 python source_only.py data/digits -d Digits -s USPS -t MNIST --train-resizing 'res.' --val-resizing 'res.' \
--resize-size 28 --no-hflip --norm-mean 0.5 --norm-std 0.5 -a lenet --no-pool --lr 0.1 -b 128 -i 2500 --scratch --seed 0 --log logs/src_only/USPS2MNIST
CUDA_VISIBLE_DEVICES=0 python source_only.py data/digits -d Digits -s SVHNRGB -t MNISTRGB --train-resizing 'res.' --val-resizing 'res.' \
--resize-size 32 --no-hflip --norm-mean 0.5 0.5 0.5 --norm-std 0.5 0.5 0.5 -a dtn --no-pool --lr 0.1 -b 128 -i 2500 --scratch --seed 0 --log logs/src_only/SVHN2MNIST
================================================
FILE: domain_generalization/Transfer-Learning-Library/examples/domain_adaptation/image_classification/utils.py
================================================
"""
@author: Junguang Jiang, Baixu Chen
@contact: JiangJunguang1123@outlook.com, cbx_99_hasta@outlook.com
"""
import sys
import os.path as osp
import time
import timm
import torch
import torch.nn as nn
import torch.nn.functional as F
import torchvision.transforms as T
from torch.utils.data import ConcatDataset
import wilds
sys.path.append('../../..')
import common.vision.datasets as datasets
import common.vision.models as models
from common.vision.transforms import ResizeImage
from common.utils.metric import accuracy, ConfusionMatrix
from common.utils.meter import AverageMeter, ProgressMeter
def get_model_names():
return sorted(
name for name in models.__dict__
if name.islower() and not name.startswith("__")
and callable(models.__dict__[name])
) + timm.list_models()
def get_model(model_name, pretrain=True):
if model_name in models.__dict__:
# load models from common.vision.models
backbone = models.__dict__[model_name](pretrained=pretrain)
else:
# load models from pytorch-image-models
backbone = timm.create_model(model_name, pretrained=pretrain)
try:
backbone.out_features = backbone.get_classifier().in_features
backbone.reset_classifier(0, '')
except:
backbone.out_features = backbone.head.in_features
backbone.head = nn.Identity()
return backbone
def convert_from_wilds_dataset(wild_dataset):
class Dataset:
def __init__(self):
self.dataset = wild_dataset
def __getitem__(self, idx):
x, y, metadata = self.dataset[idx]
return x, y
def __len__(self):
return len(self.dataset)
return Dataset()
def get_dataset_names():
return sorted(
name for name in datasets.__dict__
if not name.startswith("__") and callable(datasets.__dict__[name])
) + wilds.supported_datasets + ['Digits']
def get_dataset(dataset_name, root, source, target, train_source_transform, val_transform, train_target_transform=None):
if train_target_transform is None:
train_target_transform = train_source_transform
if dataset_name == "Digits":
train_source_dataset = datasets.__dict__[source[0]](osp.join(root, source[0]), download=True,
transform=train_source_transform)
train_target_dataset = datasets.__dict__[target[0]](osp.join(root, target[0]), download=True,
transform=train_target_transform)
val_dataset = test_dataset = datasets.__dict__[target[0]](osp.join(root, target[0]), split='test',
download=True, transform=val_transform)
class_names = datasets.MNIST.get_classes()
num_classes = len(class_names)
elif dataset_name in datasets.__dict__:
# load datasets from common.vision.datasets
dataset = datasets.__dict__[dataset_name]
def concat_dataset(tasks, **kwargs):
return ConcatDataset([dataset(task=task, **kwargs) for task in tasks])
train_source_dataset = concat_dataset(root=root, tasks=source, download=True, transform=train_source_transform)
train_target_dataset = concat_dataset(root=root, tasks=target, download=True, transform=train_target_transform)
val_dataset = concat_dataset(root=root, tasks=target, download=True, transform=val_transform)
if dataset_name == 'DomainNet':
test_dataset = concat_dataset(root=root, tasks=target, split='test', download=True, transform=val_transform)
else:
test_dataset = val_dataset
class_names = train_source_dataset.datasets[0].classes
num_classes = len(class_names)
else:
# load datasets from wilds
dataset = wilds.get_dataset(dataset_name, root_dir=root, download=True)
num_classes = dataset.n_classes
class_names = None
train_source_dataset = convert_from_wilds_dataset(dataset.get_subset('train', transform=train_source_transform))
train_target_dataset = convert_from_wilds_dataset(dataset.get_subset('test', transform=train_target_transform))
val_dataset = test_dataset = convert_from_wilds_dataset(dataset.get_subset('test', transform=val_transform))
return train_source_dataset, train_target_dataset, val_dataset, test_dataset, num_classes, class_names
def validate(val_loader, model, args, device) -> float:
batch_time = AverageMeter('Time', ':6.3f')
losses = AverageMeter('Loss', ':.4e')
top1 = AverageMeter('Acc@1', ':6.2f')
progress = ProgressMeter(
len(val_loader),
[batch_time, losses, top1],
prefix='Test: ')
# switch to evaluate mode
model.eval()
if args.per_class_eval:
confmat = ConfusionMatrix(len(args.class_names))
else:
confmat = None
with torch.no_grad():
end = time.time()
for i, (images, target) in enumerate(val_loader):
images = images.to(device)
target = target.to(device)
# compute output
output = model(images)
loss = F.cross_entropy(output, target)
# measure accuracy and record loss
acc1, = accuracy(output, target, topk=(1,))
if confmat:
confmat.update(target, output.argmax(1))
losses.update(loss.item(), images.size(0))
top1.update(acc1.item(), images.size(0))
# measure elapsed time
batch_time.update(time.time() - end)
end = time.time()
if i % args.print_freq == 0:
progress.display(i)
print(' * Acc@1 {top1.avg:.3f}'.format(top1=top1))
if confmat:
print(confmat.format(args.class_names))
return top1.avg
def get_train_transform(resizing='default', random_horizontal_flip=True, random_color_jitter=False,
resize_size=224, norm_mean=(0.485, 0.456, 0.406), norm_std=(0.229, 0.224, 0.225)):
"""
resizing mode:
- default: resize the image to 256 and take a random resized crop of size 224;
- cen.crop: resize the image to 256 and take the center crop of size 224;
- res: resize the image to 224;
"""
if resizing == 'default':
transform = T.Compose([
ResizeImage(256),
T.RandomResizedCrop(224)
])
elif resizing == 'cen.crop':
transform = T.Compose([
ResizeImage(256),
T.CenterCrop(224)
])
elif resizing == 'ran.crop':
transform = T.Compose([
ResizeImage(256),
T.RandomCrop(224)
])
elif resizing == 'res.':
transform = ResizeImage(resize_size)
else:
raise NotImplementedError(resizing)
transforms = [transform]
if random_horizontal_flip:
transforms.append(T.RandomHorizontalFlip())
if random_color_jitter:
transforms.append(T.ColorJitter(brightness=0.5, contrast=0.5, saturation=0.5, hue=0.5))
transforms.extend([
T.ToTensor(),
T.Normalize(mean=norm_mean, std=norm_std)
])
return T.Compose(transforms)
def get_val_transform(resizing='default', resize_size=224,
norm_mean=(0.485, 0.456, 0.406), norm_std=(0.229, 0.224, 0.225)):
"""
resizing mode:
- default: resize the image to 256 and take the center crop of size 224;
– res.: resize the image to 224
"""
if resizing == 'default':
transform = T.Compose([
ResizeImage(256),
T.CenterCrop(224),
])
elif resizing == 'res.':
transform = ResizeImage(resize_size)
else:
raise NotImplementedError(resizing)
return T.Compose([
transform,
T.ToTensor(),
T.Normalize(mean=norm_mean, std=norm_std)
])
def pretrain(train_source_iter, model, optimizer, lr_scheduler, epoch, args, device):
batch_time = AverageMeter('Time', ':3.1f')
data_time = AverageMeter('Data', ':3.1f')
losses = AverageMeter('Loss', ':3.2f')
cls_accs = AverageMeter('Cls Acc', ':3.1f')
progress = ProgressMeter(
args.iters_per_epoch,
[batch_time, data_time, losses, cls_accs],
prefix="Epoch: [{}]".format(epoch))
# switch to train mode
model.train()
end = time.time()
for i in range(args.iters_per_epoch):
x_s, labels_s = next(train_source_iter)
x_s = x_s.to(device)
labels_s = labels_s.to(device)
# measure data loading time
data_time.update(time.time() - end)
# compute output
y_s, f_s = model(x_s)
if len(y_s) > len(labels_s):
labels_s = torch.cat([labels_s, labels_s], dim=0)
cls_loss = F.cross_entropy(y_s, labels_s)
loss = cls_loss
cls_acc = accuracy(y_s, labels_s)[0]
losses.update(loss.item(), x_s.size(0))
cls_accs.update(cls_acc.item(), x_s.size(0))
# compute gradient and do SGD step
optimizer.zero_grad()
loss.backward()
optimizer.step()
lr_scheduler.step()
# measure elapsed time
batch_time.update(time.time() - end)
end = time.time()
if i % args.print_freq == 0:
progress.display(i)
================================================
FILE: domain_generalization/Transfer-Learning-Library/examples/domain_adaptation/image_regression/README.md
================================================
# Unsupervised Domain Adaptation for Image Regression Tasks
## Dataset
Following datasets can be downloaded automatically:
- [DSprites](https://github.com/deepmind/dsprites-dataset)
- [MPI3D](https://github.com/rr-learning/disentanglement_dataset)
## Supported Methods
Supported methods include:
- [Disparity Discrepancy (DD)](https://arxiv.org/abs/1904.05801)
- [Representation Subspace Distance (RSD)](http://ise.thss.tsinghua.edu.cn/~mlong/doc/Representation-Subspace-Distance-for-Domain-Adaptation-Regression-icml21.pdf)
## Experiment and Results
The shell files give the script to reproduce the [benchmarks](/docs/dalib/benchmarks/image_regression.rst) with specified hyper-parameters.
For example, if you want to train DD on Office31, use the following script
```shell script
# Train a DD on DSprites C->N task using ResNet 18.
# Assume you have put the datasets under the path `data/dSprites`,
# or you are glad to download the datasets automatically from the Internet to this path
CUDA_VISIBLE_DEVICES=0 python dd.py data/dSprites -d DSprites -s C -t N -a resnet18 --epochs 40 --seed 0 -b 128 --log logs/mdd/dSprites_C2N --wd 0.0005
```
For more information please refer to [Get Started](/docs/get_started/quickstart.rst) for help.
## Citation
If you use these methods in your research, please consider citing.
```
@inproceedings{MDD,
title={Bridging theory and algorithm for domain adaptation},
author={Zhang, Yuchen and Liu, Tianle and Long, Mingsheng and Jordan, Michael},
booktitle={ICML},
year={2019},
}
@inproceedings{RSD,
title={Representation Subspace Distance for Domain Adaptation Regression},
author={Chen, Xinyang and Wang, Sinan and Wang, Jianmin and Long, Mingsheng},
booktitle={ICML},
year={2021}
}
```
================================================
FILE: domain_generalization/Transfer-Learning-Library/examples/domain_adaptation/image_regression/dann.py
================================================
"""
@author: Junguang Jiang
@contact: JiangJunguang1123@outlook.com
"""
import random
import time
import warnings
import sys
import argparse
import shutil
import os.path as osp
import torch
import torch.nn as nn
import torch.backends.cudnn as cudnn
from torch.optim import SGD
from torch.optim.lr_scheduler import LambdaLR
from torch.utils.data import DataLoader
import torchvision.transforms as T
import torch.nn.functional as F
sys.path.append('../../..')
from common.modules.regressor import Regressor
from dalib.adaptation.dann import DomainAdversarialLoss
from dalib.modules.domain_discriminator import DomainDiscriminator
import common.vision.datasets.regression as datasets
import common.vision.models as models
from common.utils.data import ForeverDataIterator
from common.utils.meter import AverageMeter, ProgressMeter
from common.utils.logger import CompleteLogger
from common.utils.analysis import collect_feature, tsne, a_distance
sys.path.append('.')
from utils import convert_model, validate
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
def main(args: argparse.Namespace):
logger = CompleteLogger(args.log, args.phase)
print(args)
if args.seed is not None:
random.seed(args.seed)
torch.manual_seed(args.seed)
cudnn.deterministic = True
warnings.warn('You have chosen to seed training. '
'This will turn on the CUDNN deterministic setting, '
'which can slow down your training considerably! '
'You may see unexpected behavior when restarting '
'from checkpoints.')
cudnn.benchmark = True
# Data loading code
normalize = T.Normalize(mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225])
train_transform = T.Compose([
T.Resize(args.resize_size),
T.ToTensor(),
normalize
])
val_transform = T.Compose([
T.Resize(args.resize_size),
T.ToTensor(),
normalize
])
dataset = datasets.__dict__[args.data]
train_source_dataset = dataset(root=args.root, task=args.source, split='train', download=True, transform=train_transform)
train_source_loader = DataLoader(train_source_dataset, batch_size=args.batch_size,
shuffle=True, num_workers=args.workers, drop_last=True)
train_target_dataset = dataset(root=args.root, task=args.target, split='train', download=True, transform=train_transform)
train_target_loader = DataLoader(train_target_dataset, batch_size=args.batch_size,
shuffle=True, num_workers=args.workers, drop_last=True)
val_dataset = dataset(root=args.root, task=args.target, split='test', download=True, transform=val_transform)
val_loader = DataLoader(val_dataset, batch_size=args.batch_size, shuffle=False, num_workers=args.workers)
train_source_iter = ForeverDataIterator(train_source_loader)
train_target_iter = ForeverDataIterator(train_target_loader)
# create model
print("=> using pre-trained model '{}'".format(args.arch))
backbone = models.__dict__[args.arch](pretrained=True)
if args.normalization == 'IN':
backbone = convert_model(backbone)
num_factors = train_source_dataset.num_factors
bottleneck = nn.Sequential(
nn.AdaptiveAvgPool2d(output_size=(1, 1)),
nn.Flatten(),
nn.Linear(backbone.out_features, 256),
nn.ReLU()
)
regressor = Regressor(backbone=backbone, num_factors=num_factors, bottleneck=bottleneck, bottleneck_dim=256).to(device)
print(regressor)
domain_discri = DomainDiscriminator(in_feature=regressor.features_dim, hidden_size=1024).to(device)
# define optimizer and lr scheduler
optimizer = SGD(regressor.get_parameters() + domain_discri.get_parameters(), args.lr, momentum=args.momentum, weight_decay=args.wd, nesterov=True)
lr_scheduler = LambdaLR(optimizer, lambda x: args.lr * (1. + args.lr_gamma * float(x)) ** (-args.lr_decay))
# define loss function
dann = DomainAdversarialLoss(domain_discri).to(device)
# resume from the best checkpoint
if args.phase != 'train':
checkpoint = torch.load(logger.get_checkpoint_path('best'), map_location='cpu')
regressor.load_state_dict(checkpoint)
# analysis the model
if args.phase == 'analysis':
train_source_loader = DataLoader(train_source_dataset, batch_size=args.batch_size,
shuffle=True, num_workers=args.workers, drop_last=True)
train_target_loader = DataLoader(train_target_dataset, batch_size=args.batch_size,
shuffle=True, num_workers=args.workers, drop_last=True)
# extract features from both domains
feature_extractor = nn.Sequential(regressor.backbone, regressor.bottleneck).to(device)
source_feature = collect_feature(train_source_loader, feature_extractor, device)
target_feature = collect_feature(train_target_loader, feature_extractor, device)
# plot t-SNE
tSNE_filename = osp.join(logger.visualize_directory, 'TSNE.pdf')
tsne.visualize(source_feature, target_feature, tSNE_filename)
print("Saving t-SNE to", tSNE_filename)
# calculate A-distance, which is a measure for distribution discrepancy
A_distance = a_distance.calculate(source_feature, target_feature, device)
print("A-distance =", A_distance)
return
if args.phase == 'test':
mae = validate(val_loader, regressor, args, train_source_dataset.factors, device)
print(mae)
return
# start training
best_mae = 100000.
for epoch in range(args.epochs):
# train for one epoch
print("lr", lr_scheduler.get_lr())
train(train_source_iter, train_target_iter, regressor, dann, optimizer,
lr_scheduler, epoch, args)
# evaluate on validation set
mae = validate(val_loader, regressor, args, train_source_dataset.factors, device)
# remember best mae and save checkpoint
torch.save(regressor.state_dict(), logger.get_checkpoint_path('latest'))
if mae < best_mae:
shutil.copy(logger.get_checkpoint_path('latest'), logger.get_checkpoint_path('best'))
best_mae = min(mae, best_mae)
print("mean MAE {:6.3f} best MAE {:6.3f}".format(mae, best_mae))
print("best_mae = {:6.3f}".format(best_mae))
logger.close()
def train(train_source_iter: ForeverDataIterator, train_target_iter: ForeverDataIterator,
model: Regressor, domain_adv: DomainAdversarialLoss, optimizer: SGD,
lr_scheduler: LambdaLR, epoch: int, args: argparse.Namespace):
batch_time = AverageMeter('Time', ':4.2f')
data_time = AverageMeter('Data', ':3.1f')
mse_losses = AverageMeter('MSE Loss', ':6.3f')
dann_losses = AverageMeter('DANN Loss', ':6.3f')
domain_accs = AverageMeter('Domain Acc', ':3.1f')
mae_losses_s = AverageMeter('MAE Loss (s)', ':6.3f')
mae_losses_t = AverageMeter('MAE Loss (t)', ':6.3f')
progress = ProgressMeter(
args.iters_per_epoch,
[batch_time, data_time, mse_losses, dann_losses, mae_losses_s, mae_losses_t, domain_accs],
prefix="Epoch: [{}]".format(epoch))
# switch to train mode
model.train()
domain_adv.train()
end = time.time()
for i in range(args.iters_per_epoch):
optimizer.zero_grad()
x_s, labels_s = next(train_source_iter)
x_s = x_s.to(device)
labels_s = labels_s.to(device).float()
x_t, labels_t = next(train_target_iter)
x_t = x_t.to(device)
labels_t = labels_t.to(device).float()
# measure data loading time
data_time.update(time.time() - end)
# compute output
y_s, f_s = model(x_s)
y_t, f_t = model(x_t)
mse_loss = F.mse_loss(y_s, labels_s)
mae_loss_s = F.l1_loss(y_s, labels_s)
mae_loss_t = F.l1_loss(y_t, labels_t)
transfer_loss = domain_adv(f_s, f_t)
loss = mse_loss + transfer_loss * args.trade_off
domain_acc = domain_adv.domain_discriminator_accuracy
mse_losses.update(mse_loss.item(), x_s.size(0))
dann_losses.update(transfer_loss.item(), x_s.size(0))
mae_losses_s.update(mae_loss_s.item(), x_s.size(0))
mae_losses_t.update(mae_loss_t.item(), x_s.size(0))
domain_accs.update(domain_acc.item(), x_s.size(0))
# compute gradient and do SGD step
loss.backward()
optimizer.step()
lr_scheduler.step()
# measure elapsed time
batch_time.update(time.time() - end)
end = time.time()
if i % args.print_freq == 0:
progress.display(i)
if __name__ == '__main__':
architecture_names = sorted(
name for name in models.__dict__
if name.islower() and not name.startswith("__")
and callable(models.__dict__[name])
)
dataset_names = sorted(
name for name in datasets.__dict__
if not name.startswith("__") and callable(datasets.__dict__[name])
)
parser = argparse.ArgumentParser(description='DANN for Regression Domain Adaptation')
# dataset parameters
parser.add_argument('root', metavar='DIR',
help='root path of dataset')
parser.add_argument('-d', '--data', metavar='DATA', default='DSprites',
help='dataset: ' + ' | '.join(dataset_names) +
' (default: Office31)')
parser.add_argument('-s', '--source', help='source domain(s)')
parser.add_argument('-t', '--target', help='target domain(s)')
parser.add_argument('--resize-size', type=int, default=128)
# model parameters
parser.add_argument('-a', '--arch', metavar='ARCH', default='resnet18',
choices=architecture_names,
help='backbone architecture: ' +
' | '.join(architecture_names) +
' (default: resnet18)')
parser.add_argument('--normalization', default='BN', type=str, choices=["BN", "IN"])
parser.add_argument('--trade-off', default=1., type=float,
help='the trade-off hyper-parameter for transfer loss')
# training parameters
parser.add_argument('-b', '--batch-size', default=36, type=int,
metavar='N',
help='mini-batch size (default: 32)')
parser.add_argument('--lr', '--learning-rate', default=0.1, type=float,
metavar='LR', help='initial learning rate', dest='lr')
parser.add_argument('--lr-gamma', default=0.0001, type=float, help='parameter for lr scheduler')
parser.add_argument('--lr-decay', default=0.75, type=float, help='parameter for lr scheduler')
parser.add_argument('--momentum', default=0.9, type=float, metavar='M',
help='momentum')
parser.add_argument('--wd', '--weight-decay', default=0.001, type=float,
metavar='W', help='weight decay (default: 5e-4)')
parser.add_argument('-j', '--workers', default=2, type=int, metavar='N',
help='number of data loading workers (default: 4)')
parser.add_argument('--epochs', default=20, type=int, metavar='N',
help='number of total epochs to run')
parser.add_argument('-i', '--iters-per-epoch', default=500, type=int,
help='Number of iterations per epoch')
parser.add_argument('-p', '--print-freq', default=100, type=int,
metavar='N', help='print frequency (default: 100)')
parser.add_argument('--seed', default=None, type=int,
help='seed for initializing training. ')
parser.add_argument("--log", type=str, default='dann',
help="Where to save logs, checkpoints and debugging images.")
parser.add_argument("--phase", type=str, default='train', choices=['train', 'test', 'analysis'],
help="When phase is 'test', only test the model."
"When phase is 'analysis', only analysis the model.")
args = parser.parse_args()
main(args)
================================================
FILE: domain_generalization/Transfer-Learning-Library/examples/domain_adaptation/image_regression/dann.sh
================================================
# DSprites
CUDA_VISIBLE_DEVICES=0 python dann.py data/dSprites -d DSprites -s C -t N -a resnet18 --epochs 40 --seed 0 --log logs/dann/DSprites_C2N
CUDA_VISIBLE_DEVICES=0 python dann.py data/dSprites -d DSprites -s C -t S -a resnet18 --epochs 40 --seed 0 --log logs/dann/DSprites_C2S
CUDA_VISIBLE_DEVICES=0 python dann.py data/dSprites -d DSprites -s N -t C -a resnet18 --epochs 40 --seed 0 --log logs/dann/DSprites_N2C
CUDA_VISIBLE_DEVICES=0 python dann.py data/dSprites -d DSprites -s N -t S -a resnet18 --epochs 40 --seed 0 --log logs/dann/DSprites_N2S
CUDA_VISIBLE_DEVICES=0 python dann.py data/dSprites -d DSprites -s S -t C -a resnet18 --epochs 40 --seed 0 --log logs/dann/DSprites_S2C
CUDA_VISIBLE_DEVICES=0 python dann.py data/dSprites -d DSprites -s S -t N -a resnet18 --epochs 40 --seed 0 --log logs/dann/DSprites_S2N
# MPI3D
CUDA_VISIBLE_DEVICES=0 python dann.py data/mpi3d -d MPI3D -s RL -t RC -a resnet18 --epochs 40 --seed 0 --log logs/dann/MPI3D_RL2RC --resize-size 224
CUDA_VISIBLE_DEVICES=0 python dann.py data/mpi3d -d MPI3D -s RL -t T -a resnet18 --epochs 40 --seed 0 --log logs/dann/MPI3D_RL2T --resize-size 224
CUDA_VISIBLE_DEVICES=0 python dann.py data/mpi3d -d MPI3D -s RC -t RL -a resnet18 --epochs 40 --seed 0 --log logs/dann/MPI3D_RC2RL --resize-size 224
CUDA_VISIBLE_DEVICES=0 python dann.py data/mpi3d -d MPI3D -s RC -t T -a resnet18 --epochs 40 --seed 0 --log logs/dann/MPI3D_RC2T --resize-size 224
CUDA_VISIBLE_DEVICES=0 python dann.py data/mpi3d -d MPI3D -s T -t RL -a resnet18 --epochs 40 --seed 0 --log logs/dann/MPI3D_T2RL --resize-size 224
CUDA_VISIBLE_DEVICES=0 python dann.py data/mpi3d -d MPI3D -s T -t RC -a resnet18 --epochs 40 --seed 0 --log logs/dann/MPI3D_T2RC --resize-size 224
================================================
FILE: domain_generalization/Transfer-Learning-Library/examples/domain_adaptation/image_regression/dd.py
================================================
"""
@author: Junguang Jiang
@contact: JiangJunguang1123@outlook.com
"""
import random
import time
import warnings
import sys
import argparse
import shutil
import os.path as osp
import torch
import torch.backends.cudnn as cudnn
from torch.optim import SGD
from torch.optim.lr_scheduler import LambdaLR
from torch.utils.data import DataLoader
import torchvision.transforms as T
import torch.nn.functional as F
import torch.nn as nn
sys.path.append('../../..')
from dalib.adaptation.mdd import RegressionMarginDisparityDiscrepancy as MarginDisparityDiscrepancy, ImageRegressor
import common.vision.datasets.regression as datasets
import common.vision.models as models
from common.utils.data import ForeverDataIterator
from common.utils.meter import AverageMeter, ProgressMeter
from common.utils.logger import CompleteLogger
from common.utils.analysis import collect_feature, tsne, a_distance
sys.path.append('.')
from utils import convert_model, validate
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
def main(args: argparse.Namespace):
logger = CompleteLogger(args.log, args.phase)
print(args)
if args.seed is not None:
random.seed(args.seed)
torch.manual_seed(args.seed)
cudnn.deterministic = True
warnings.warn('You have chosen to seed training. '
'This will turn on the CUDNN deterministic setting, '
'which can slow down your training considerably! '
'You may see unexpected behavior when restarting '
'from checkpoints.')
cudnn.benchmark = True
# Data loading code
normalize = T.Normalize(mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225])
train_transform = T.Compose([
T.Resize(args.resize_size),
T.ToTensor(),
normalize
])
val_transform = T.Compose([
T.Resize(args.resize_size),
T.ToTensor(),
normalize
])
dataset = datasets.__dict__[args.data]
train_source_dataset = dataset(root=args.root, task=args.source, split='train', download=True, transform=train_transform)
train_source_loader = DataLoader(train_source_dataset, batch_size=args.batch_size,
shuffle=True, num_workers=args.workers, drop_last=True)
train_target_dataset = dataset(root=args.root, task=args.target, split='train', download=True, transform=train_transform)
train_target_loader = DataLoader(train_target_dataset, batch_size=args.batch_size,
shuffle=True, num_workers=args.workers, drop_last=True)
val_dataset = dataset(root=args.root, task=args.target, split='test', download=True, transform=val_transform)
val_loader = DataLoader(val_dataset, batch_size=args.batch_size, shuffle=False, num_workers=args.workers)
train_source_iter = ForeverDataIterator(train_source_loader)
train_target_iter = ForeverDataIterator(train_target_loader)
# create model
print("=> using pre-trained model '{}'".format(args.arch))
num_factors = train_source_dataset.num_factors
backbone = models.__dict__[args.arch](pretrained=True)
bottleneck_dim = args.bottleneck_dim
if args.normalization == 'IN':
backbone = convert_model(backbone)
bottleneck = nn.Sequential(
nn.Conv2d(backbone.out_features, bottleneck_dim, kernel_size=3, stride=1, padding=1),
nn.InstanceNorm2d(bottleneck_dim),
nn.ReLU(),
)
head = nn.Sequential(
nn.Conv2d(bottleneck_dim, bottleneck_dim, kernel_size=3, stride=1, padding=1),
nn.InstanceNorm2d(bottleneck_dim),
nn.ReLU(),
nn.Conv2d(bottleneck_dim, bottleneck_dim, kernel_size=3, stride=1, padding=1),
nn.InstanceNorm2d(bottleneck_dim),
nn.ReLU(),
nn.AdaptiveAvgPool2d(output_size=(1, 1)),
nn.Flatten(),
nn.Linear(bottleneck_dim, num_factors),
nn.Sigmoid()
)
for layer in head:
if isinstance(layer, nn.Conv2d) or isinstance(layer, nn.Linear):
nn.init.normal_(layer.weight, 0, 0.01)
nn.init.constant_(layer.bias, 0)
adv_head = nn.Sequential(
nn.Conv2d(bottleneck_dim, bottleneck_dim, kernel_size=3, stride=1, padding=1),
nn.InstanceNorm2d(bottleneck_dim),
nn.ReLU(),
nn.Conv2d(bottleneck_dim, bottleneck_dim, kernel_size=3, stride=1, padding=1),
nn.InstanceNorm2d(bottleneck_dim),
nn.ReLU(),
nn.AdaptiveAvgPool2d(output_size=(1, 1)),
nn.Flatten(),
nn.Linear(bottleneck_dim, num_factors),
nn.Sigmoid()
)
for layer in adv_head:
if isinstance(layer, nn.Conv2d) or isinstance(layer, nn.Linear):
nn.init.normal_(layer.weight, 0, 0.01)
nn.init.constant_(layer.bias, 0)
regressor = ImageRegressor(backbone, num_factors, bottleneck=bottleneck, head=head, adv_head=adv_head,
bottleneck_dim=bottleneck_dim, width=bottleneck_dim)
else:
regressor = ImageRegressor(backbone, num_factors,
bottleneck_dim=bottleneck_dim, width=bottleneck_dim)
regressor = regressor.to(device)
print(regressor)
mdd = MarginDisparityDiscrepancy(args.margin).to(device)
# define optimizer and lr scheduler
optimizer = SGD(regressor.get_parameters(), args.lr, momentum=args.momentum, weight_decay=args.wd, nesterov=True)
lr_scheduler = LambdaLR(optimizer, lambda x: args.lr * (1. + args.lr_gamma * float(x)) ** (-args.lr_decay))
# resume from the best checkpoint
if args.phase != 'train':
checkpoint = torch.load(logger.get_checkpoint_path('best'), map_location='cpu')
regressor.load_state_dict(checkpoint)
# analysis the model
if args.phase == 'analysis':
train_source_loader = DataLoader(train_source_dataset, batch_size=args.batch_size,
shuffle=True, num_workers=args.workers, drop_last=True)
train_target_loader = DataLoader(train_target_dataset, batch_size=args.batch_size,
shuffle=True, num_workers=args.workers, drop_last=True)
# extract features from both domains
feature_extractor = nn.Sequential(regressor.backbone, regressor.bottleneck, regressor.head[:-2]).to(device)
source_feature = collect_feature(train_source_loader, feature_extractor, device)
target_feature = collect_feature(train_target_loader, feature_extractor, device)
# plot t-SNE
tSNE_filename = osp.join(logger.visualize_directory, 'TSNE.pdf')
tsne.visualize(source_feature, target_feature, tSNE_filename)
print("Saving t-SNE to", tSNE_filename)
# calculate A-distance, which is a measure for distribution discrepancy
A_distance = a_distance.calculate(source_feature, target_feature, device)
print("A-distance =", A_distance)
return
if args.phase == 'test':
mae = validate(val_loader, regressor, args, train_source_dataset.factors, device)
print(mae)
return
# start training
best_mae = 100000.
for epoch in range(args.epochs):
# train for one epoch
print("lr", lr_scheduler.get_lr())
train(train_source_iter, train_target_iter, regressor, mdd, optimizer,
lr_scheduler, epoch, args)
# evaluate on validation set
mae = validate(val_loader, regressor, args, train_source_dataset.factors, device)
# remember best mae and save checkpoint
torch.save(regressor.state_dict(), logger.get_checkpoint_path('latest'))
if mae < best_mae:
shutil.copy(logger.get_checkpoint_path('latest'), logger.get_checkpoint_path('best'))
best_mae = min(mae, best_mae)
print("mean MAE {:6.3f} best MAE {:6.3f}".format(mae, best_mae))
print("best_mae = {:6.3f}".format(best_mae))
logger.close()
def train(train_source_iter: ForeverDataIterator, train_target_iter: ForeverDataIterator,
model, mdd: MarginDisparityDiscrepancy, optimizer: SGD,
lr_scheduler: LambdaLR, epoch: int, args: argparse.Namespace):
batch_time = AverageMeter('Time', ':4.2f')
data_time = AverageMeter('Data', ':3.1f')
source_losses = AverageMeter('Source Loss', ':6.3f')
trans_losses = AverageMeter('Trans Loss', ':6.3f')
mae_losses_s = AverageMeter('MAE Loss (s)', ':6.3f')
mae_losses_t = AverageMeter('MAE Loss (t)', ':6.3f')
progress = ProgressMeter(
args.iters_per_epoch,
[batch_time, data_time, source_losses, trans_losses, mae_losses_s, mae_losses_t],
prefix="Epoch: [{}]".format(epoch))
# switch to train mode
model.train()
mdd.train()
end = time.time()
for i in range(args.iters_per_epoch):
optimizer.zero_grad()
x_s, labels_s = next(train_source_iter)
x_s = x_s.to(device)
labels_s = labels_s.to(device).float()
x_t, labels_t = next(train_target_iter)
x_t = x_t.to(device)
labels_t = labels_t.to(device).float()
# measure data loading time
data_time.update(time.time() - end)
# compute output
x = torch.cat([x_s, x_t], dim=0)
outputs, outputs_adv = model(x)
y_s, y_t = outputs.chunk(2, dim=0)
y_s_adv, y_t_adv = outputs_adv.chunk(2, dim=0)
# compute mean square loss on source domain
mse_loss = F.mse_loss(y_s, labels_s)
# compute margin disparity discrepancy between domains
transfer_loss = mdd(y_s, y_s_adv, y_t, y_t_adv)
# for adversarial classifier, minimize negative mdd is equal to maximize mdd
loss = mse_loss - transfer_loss * args.trade_off
model.step()
mae_loss_s = F.l1_loss(y_s, labels_s)
mae_loss_t = F.l1_loss(y_t, labels_t)
source_losses.update(mse_loss.item(), x_s.size(0))
trans_losses.update(transfer_loss.item(), x_s.size(0))
mae_losses_s.update(mae_loss_s.item(), x_s.size(0))
mae_losses_t.update(mae_loss_t.item(), x_s.size(0))
# compute gradient and do SGD step
loss.backward()
optimizer.step()
lr_scheduler.step()
# measure elapsed time
batch_time.update(time.time() - end)
end = time.time()
if i % args.print_freq == 0:
progress.display(i)
if __name__ == '__main__':
architecture_names = sorted(
name for name in models.__dict__
if name.islower() and not name.startswith("__")
and callable(models.__dict__[name])
)
dataset_names = sorted(
name for name in datasets.__dict__
if not name.startswith("__") and callable(datasets.__dict__[name])
)
parser = argparse.ArgumentParser(description='DD for Regression Domain Adaptation')
# dataset parameters
parser.add_argument('root', metavar='DIR',
help='root path of dataset')
parser.add_argument('-d', '--data', metavar='DATA', default='DSprites',
help='dataset: ' + ' | '.join(dataset_names) +
' (default: Office31)')
parser.add_argument('-s', '--source', help='source domain(s)')
parser.add_argument('-t', '--target', help='target domain(s)')
parser.add_argument('--resize-size', type=int, default=128)
# model parameters
parser.add_argument('-a', '--arch', metavar='ARCH', default='resnet18',
choices=architecture_names,
help='backbone architecture: ' +
' | '.join(architecture_names) +
' (default: resnet18)')
parser.add_argument('--bottleneck-dim', default=512, type=int)
parser.add_argument('--normalization', default='BN', type=str, choices=['BN', 'IN'])
parser.add_argument('--margin', type=float, default=1., help="margin gamma")
parser.add_argument('--trade-off', default=1., type=float,
help='the trade-off hyper-parameter for transfer loss')
# training parameters
parser.add_argument('-b', '--batch-size', default=36, type=int,
metavar='N',
help='mini-batch size (default: 32)')
parser.add_argument('--lr', '--learning-rate', default=0.1, type=float,
metavar='LR', help='initial learning rate', dest='lr')
parser.add_argument('--lr-gamma', default=0.0001, type=float, help='parameter for lr scheduler')
parser.add_argument('--lr-decay', default=0.75, type=float, help='parameter for lr scheduler')
parser.add_argument('--momentum', default=0.9, type=float, metavar='M',
help='momentum')
parser.add_argument('--wd', '--weight-decay', default=0.0005, type=float,
metavar='W', help='weight decay (default: 5e-4)')
parser.add_argument('-j', '--workers', default=2, type=int, metavar='N',
help='number of data loading workers (default: 4)')
parser.add_argument('--epochs', default=20, type=int, metavar='N',
help='number of total epochs to run')
parser.add_argument('-i', '--iters-per-epoch', default=500, type=int,
help='Number of iterations per epoch')
parser.add_argument('-p', '--print-freq', default=100, type=int,
metavar='N', help='print frequency (default: 100)')
parser.add_argument('--seed', default=None, type=int,
help='seed for initializing training. ')
parser.add_argument("--log", type=str, default='dd',
help="Where to save logs, checkpoints and debugging images.")
parser.add_argument("--phase", type=str, default='train', choices=['train', 'test', 'analysis'],
help="When phase is 'test', only test the model."
"When phase is 'analysis', only analysis the model.")
args = parser.parse_args()
main(args)
================================================
FILE: domain_generalization/Transfer-Learning-Library/examples/domain_adaptation/image_regression/dd.sh
================================================
# DSprites
CUDA_VISIBLE_DEVICES=0 python dd.py data/dSprites -d DSprites -s C -t N -a resnet18 --epochs 40 --seed 0 -b 128 --log logs/dd/dSprites_C2N --wd 0.0005
CUDA_VISIBLE_DEVICES=0 python dd.py data/dSprites -d DSprites -s C -t S -a resnet18 --epochs 40 --seed 0 -b 128 --log logs/dd/dSprites_C2S --wd 0.0005
CUDA_VISIBLE_DEVICES=0 python dd.py data/dSprites -d DSprites -s N -t C -a resnet18 --epochs 40 --seed 0 -b 128 --log logs/dd/dSprites_N2C --wd 0.0005
CUDA_VISIBLE_DEVICES=0 python dd.py data/dSprites -d DSprites -s N -t S -a resnet18 --epochs 40 --seed 0 -b 128 --log logs/dd/dSprites_N2S --wd 0.0005
CUDA_VISIBLE_DEVICES=0 python dd.py data/dSprites -d DSprites -s S -t C -a resnet18 --epochs 40 --seed 0 -b 128 --log logs/dd/dSprites_S2C --wd 0.0005
CUDA_VISIBLE_DEVICES=0 python dd.py data/dSprites -d DSprites -s S -t N -a resnet18 --epochs 40 --seed 0 -b 128 --log logs/dd/dSprites_S2N --wd 0.0005
# MPI3D
CUDA_VISIBLE_DEVICES=0 python dd.py data/mpi3d -d MPI3D -s RL -t RC -a resnet18 --epochs 60 --seed 0 -b 36 --log logs/dd/MPI3D_RL2RC --normalization IN --resize-size 224 --weight-decay 0.001
CUDA_VISIBLE_DEVICES=0 python dd.py data/mpi3d -d MPI3D -s RL -t T -a resnet18 --epochs 60 --seed 0 -b 36 --log logs/dd/MPI3D_RL2T --normalization IN --resize-size 224 --weight-decay 0.001
CUDA_VISIBLE_DEVICES=0 python dd.py data/mpi3d -d MPI3D -s RC -t RL -a resnet18 --epochs 60 --seed 0 -b 36 --log logs/dd/MPI3D_RC2RL --normalization IN --resize-size 224 --weight-decay 0.001
CUDA_VISIBLE_DEVICES=0 python dd.py data/mpi3d -d MPI3D -s RC -t T -a resnet18 --epochs 60 --seed 0 -b 36 --log logs/dd/MPI3D_RC2T --normalization IN --resize-size 224 --weight-decay 0.001
CUDA_VISIBLE_DEVICES=0 python dd.py data/mpi3d -d MPI3D -s T -t RL -a resnet18 --epochs 60 --seed 0 -b 36 --log logs/dd/MPI3D_T2RL --normalization IN --resize-size 224 --weight-decay 0.001
CUDA_VISIBLE_DEVICES=0 python dd.py data/mpi3d -d MPI3D -s T -t RC -a resnet18 --epochs 60 --seed 0 -b 36 --log logs/dd/MPI3D_T2RC --normalization IN --resize-size 224 --weight-decay 0.001
================================================
FILE: domain_generalization/Transfer-Learning-Library/examples/domain_adaptation/image_regression/rsd.py
================================================
"""
@author: Junguang Jiang
@contact: JiangJunguang1123@outlook.com
"""
import random
import time
import warnings
import sys
import argparse
import shutil
import os.path as osp
import torch
import torch.nn as nn
import torch.backends.cudnn as cudnn
from torch.optim import SGD
from torch.optim.lr_scheduler import LambdaLR
from torch.utils.data import DataLoader
import torchvision.transforms as T
import torch.nn.functional as F
sys.path.append('../../..')
from common.modules.regressor import Regressor
from dalib.adaptation.rsd import RepresentationSubspaceDistance
import common.vision.datasets.regression as datasets
import common.vision.models as models
from common.utils.data import ForeverDataIterator
from common.utils.meter import AverageMeter, ProgressMeter
from common.utils.logger import CompleteLogger
from common.utils.analysis import collect_feature, tsne, a_distance
sys.path.append('.')
from utils import convert_model, validate
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
def main(args: argparse.Namespace):
logger = CompleteLogger(args.log, args.phase)
print(args)
if args.seed is not None:
random.seed(args.seed)
torch.manual_seed(args.seed)
cudnn.deterministic = True
warnings.warn('You have chosen to seed training. '
'This will turn on the CUDNN deterministic setting, '
'which can slow down your training considerably! '
'You may see unexpected behavior when restarting '
'from checkpoints.')
cudnn.benchmark = True
# Data loading code
normalize = T.Normalize(mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225])
train_transform = T.Compose([
T.Resize(args.resize_size),
T.ToTensor(),
normalize
])
val_transform = T.Compose([
T.Resize(args.resize_size),
T.ToTensor(),
normalize
])
dataset = datasets.__dict__[args.data]
train_source_dataset = dataset(root=args.root, task=args.source, split='train', download=True, transform=train_transform)
train_source_loader = DataLoader(train_source_dataset, batch_size=args.batch_size,
shuffle=True, num_workers=args.workers, drop_last=True)
train_target_dataset = dataset(root=args.root, task=args.target, split='train', download=True, transform=train_transform)
train_target_loader = DataLoader(train_target_dataset, batch_size=args.batch_size,
shuffle=True, num_workers=args.workers, drop_last=True)
val_dataset = dataset(root=args.root, task=args.target, split='test', download=True, transform=val_transform)
val_loader = DataLoader(val_dataset, batch_size=args.batch_size, shuffle=False, num_workers=args.workers)
train_source_iter = ForeverDataIterator(train_source_loader)
train_target_iter = ForeverDataIterator(train_target_loader)
# create model
print("=> using pre-trained model '{}'".format(args.arch))
backbone = models.__dict__[args.arch](pretrained=True)
if args.normalization == 'IN':
backbone = convert_model(backbone)
num_factors = train_source_dataset.num_factors
bottleneck = nn.Sequential(
nn.AdaptiveAvgPool2d(output_size=(1, 1)),
nn.Flatten(),
nn.Linear(backbone.out_features, 256),
nn.ReLU()
)
regressor = Regressor(backbone=backbone, num_factors=num_factors, bottleneck=bottleneck, bottleneck_dim=backbone.out_features).to(device)
print(regressor)
# define optimizer and lr scheduler
optimizer = SGD(regressor.get_parameters(), args.lr, momentum=args.momentum, weight_decay=args.wd, nesterov=True)
lr_scheduler = LambdaLR(optimizer, lambda x: args.lr * (1. + args.lr_gamma * float(x)) ** (-args.lr_decay))
# define loss function
rsd = RepresentationSubspaceDistance(args.trade_off_bmp)
# resume from the best checkpoint
if args.phase != 'train':
checkpoint = torch.load(logger.get_checkpoint_path('best'), map_location='cpu')
regressor.load_state_dict(checkpoint)
# analysis the model
if args.phase == 'analysis':
train_source_loader = DataLoader(train_source_dataset, batch_size=args.batch_size,
shuffle=True, num_workers=args.workers, drop_last=True)
train_target_loader = DataLoader(train_target_dataset, batch_size=args.batch_size,
shuffle=True, num_workers=args.workers, drop_last=True)
# extract features from both domains
feature_extractor = nn.Sequential(regressor.backbone, regressor.bottleneck).to(device)
source_feature = collect_feature(train_source_loader, feature_extractor, device)
target_feature = collect_feature(train_target_loader, feature_extractor, device)
# plot t-SNE
tSNE_filename = osp.join(logger.visualize_directory, 'TSNE.pdf')
tsne.visualize(source_feature, target_feature, tSNE_filename)
print("Saving t-SNE to", tSNE_filename)
# calculate A-distance, which is a measure for distribution discrepancy
A_distance = a_distance.calculate(source_feature, target_feature, device)
print("A-distance =", A_distance)
return
if args.phase == 'test':
mae = validate(val_loader, regressor, args, train_source_dataset.factors, device)
print(mae)
return
# start training
best_mae = 100000.
for epoch in range(args.epochs):
# train for one epoch
print("lr", lr_scheduler.get_lr())
train(train_source_iter, train_target_iter, regressor, rsd, optimizer,
lr_scheduler, epoch, args)
# evaluate on validation set
mae = validate(val_loader, regressor, args, train_source_dataset.factors, device)
# remember best mae and save checkpoint
torch.save(regressor.state_dict(), logger.get_checkpoint_path('latest'))
if mae < best_mae:
shutil.copy(logger.get_checkpoint_path('latest'), logger.get_checkpoint_path('best'))
best_mae = min(mae, best_mae)
print("mean MAE {:6.3f} best MAE {:6.3f}".format(mae, best_mae))
print("best_mae = {:6.3f}".format(best_mae))
logger.close()
def train(train_source_iter: ForeverDataIterator, train_target_iter: ForeverDataIterator,
model: Regressor, rsd, optimizer: SGD,
lr_scheduler: LambdaLR, epoch: int, args: argparse.Namespace):
batch_time = AverageMeter('Time', ':4.2f')
data_time = AverageMeter('Data', ':3.1f')
mse_losses = AverageMeter('MSE Loss', ':6.3f')
rsd_losses = AverageMeter('RSD Loss', ':6.3f')
mae_losses_s = AverageMeter('MAE Loss (s)', ':6.3f')
mae_losses_t = AverageMeter('MAE Loss (t)', ':6.3f')
progress = ProgressMeter(
args.iters_per_epoch,
[batch_time, data_time, mse_losses, rsd_losses, mae_losses_s, mae_losses_t],
prefix="Epoch: [{}]".format(epoch))
# switch to train mode
model.train()
end = time.time()
for i in range(args.iters_per_epoch):
optimizer.zero_grad()
x_s, labels_s = next(train_source_iter)
x_s = x_s.to(device)
labels_s = labels_s.to(device).float()
x_t, labels_t = next(train_target_iter)
x_t = x_t.to(device)
labels_t = labels_t.to(device).float()
# measure data loading time
data_time.update(time.time() - end)
# compute output
y_s, f_s = model(x_s)
y_t, f_t = model(x_t)
mse_loss = F.mse_loss(y_s, labels_s)
mae_loss_s = F.l1_loss(y_s, labels_s)
mae_loss_t = F.l1_loss(y_t, labels_t)
rsd_loss = rsd(f_s, f_t)
loss = mse_loss + rsd_loss * args.trade_off
mse_losses.update(mse_loss.item(), x_s.size(0))
rsd_losses.update(rsd_loss.item(), x_s.size(0))
mae_losses_s.update(mae_loss_s.item(), x_s.size(0))
mae_losses_t.update(mae_loss_t.item(), x_s.size(0))
# compute gradient and do SGD step
loss.backward()
optimizer.step()
lr_scheduler.step()
# measure elapsed time
batch_time.update(time.time() - end)
end = time.time()
if i % args.print_freq == 0:
progress.display(i)
if __name__ == '__main__':
architecture_names = sorted(
name for name in models.__dict__
if name.islower() and not name.startswith("__")
and callable(models.__dict__[name])
)
dataset_names = sorted(
name for name in datasets.__dict__
if not name.startswith("__") and callable(datasets.__dict__[name])
)
parser = argparse.ArgumentParser(description='RSD for Regression Domain Adaptation')
# dataset parameters
parser.add_argument('root', metavar='DIR',
help='root path of dataset')
parser.add_argument('-d', '--data', metavar='DATA', default='DSprites',
help='dataset: ' + ' | '.join(dataset_names) +
' (default: Office31)')
parser.add_argument('-s', '--source', help='source domain(s)')
parser.add_argument('-t', '--target', help='target domain(s)')
parser.add_argument('--resize-size', type=int, default=128)
# model parameters
parser.add_argument('-a', '--arch', metavar='ARCH', default='resnet18',
choices=architecture_names,
help='backbone architecture: ' +
' | '.join(architecture_names) +
' (default: resnet18)')
parser.add_argument('--normalization', default='BN', type=str, choices=["BN", "IN"])
parser.add_argument('--trade-off', default=0.001, type=float)
parser.add_argument('--trade-off-bmp', default=0.1, type=float)
# training parameters
parser.add_argument('-b', '--batch-size', default=36, type=int,
metavar='N',
help='mini-batch size (default: 32)')
parser.add_argument('--lr', '--learning-rate', default=0.1, type=float,
metavar='LR', help='initial learning rate', dest='lr')
parser.add_argument('--lr-gamma', default=0.0001, type=float, help='parameter for lr scheduler')
parser.add_argument('--lr-decay', default=0.75, type=float, help='parameter for lr scheduler')
parser.add_argument('--momentum', default=0.9, type=float, metavar='M',
help='momentum')
parser.add_argument('--wd', '--weight-decay', default=0.001, type=float,
metavar='W', help='weight decay (default: 5e-4)')
parser.add_argument('-j', '--workers', default=2, type=int, metavar='N',
help='number of data loading workers (default: 4)')
parser.add_argument('--epochs', default=20, type=int, metavar='N',
help='number of total epochs to run')
parser.add_argument('-i', '--iters-per-epoch', default=500, type=int,
help='Number of iterations per epoch')
parser.add_argument('-p', '--print-freq', default=100, type=int,
metavar='N', help='print frequency (default: 100)')
parser.add_argument('--seed', default=None, type=int,
help='seed for initializing training. ')
parser.add_argument("--log", type=str, default='rsd',
help="Where to save logs, checkpoints and debugging images.")
parser.add_argument("--phase", type=str, default='train', choices=['train', 'test', 'analysis'],
help="When phase is 'test', only test the model."
"When phase is 'analysis', only analysis the model.")
args = parser.parse_args()
main(args)
================================================
FILE: domain_generalization/Transfer-Learning-Library/examples/domain_adaptation/image_regression/rsd.sh
================================================
# DSprites
CUDA_VISIBLE_DEVICES=0 python rsd.py data/dSprites -d DSprites -s C -t N -a resnet18 --epochs 40 --seed 0 --log logs/rsd/DSprites_C2N
CUDA_VISIBLE_DEVICES=0 python rsd.py data/dSprites -d DSprites -s C -t S -a resnet18 --epochs 40 --seed 0 --log logs/rsd/DSprites_C2S
CUDA_VISIBLE_DEVICES=0 python rsd.py data/dSprites -d DSprites -s N -t C -a resnet18 --epochs 40 --seed 0 --log logs/rsd/DSprites_N2C
CUDA_VISIBLE_DEVICES=0 python rsd.py data/dSprites -d DSprites -s N -t S -a resnet18 --epochs 40 --seed 0 --log logs/rsd/DSprites_N2S
CUDA_VISIBLE_DEVICES=0 python rsd.py data/dSprites -d DSprites -s S -t C -a resnet18 --epochs 40 --seed 0 --log logs/rsd/DSprites_S2C
CUDA_VISIBLE_DEVICES=0 python rsd.py data/dSprites -d DSprites -s S -t N -a resnet18 --epochs 40 --seed 0 --log logs/rsd/DSprites_S2N
# MPI3D
CUDA_VISIBLE_DEVICES=0 python rsd.py data/mpi3d -d MPI3D -s RL -t RC -a resnet18 --epochs 40 --seed 0 --log logs/rsd/MPI3D_RL2RC --resize-size 224
CUDA_VISIBLE_DEVICES=0 python rsd.py data/mpi3d -d MPI3D -s RL -t T -a resnet18 --epochs 40 --seed 0 --log logs/rsd/MPI3D_RL2T --resize-size 224
CUDA_VISIBLE_DEVICES=0 python rsd.py data/mpi3d -d MPI3D -s RC -t RL -a resnet18 --epochs 40 --seed 0 --log logs/rsd/MPI3D_RC2RL --resize-size 224
CUDA_VISIBLE_DEVICES=0 python rsd.py data/mpi3d -d MPI3D -s RC -t T -a resnet18 --epochs 40 --seed 0 --log logs/rsd/MPI3D_RC2T --resize-size 224
CUDA_VISIBLE_DEVICES=0 python rsd.py data/mpi3d -d MPI3D -s T -t RL -a resnet18 --epochs 40 --seed 0 --log logs/rsd/MPI3D_T2RL --resize-size 224
CUDA_VISIBLE_DEVICES=0 python rsd.py data/mpi3d -d MPI3D -s T -t RC -a resnet18 --epochs 40 --seed 0 --log logs/rsd/MPI3D_T2RC --resize-size 224
================================================
FILE: domain_generalization/Transfer-Learning-Library/examples/domain_adaptation/image_regression/source_only.py
================================================
"""
@author: Junguang Jiang
@contact: JiangJunguang1123@outlook.com
"""
import random
import time
import warnings
import sys
import argparse
import shutil
import os.path as osp
import torch
import torch.nn as nn
import torch.backends.cudnn as cudnn
from torch.optim import SGD
from torch.optim.lr_scheduler import LambdaLR
from torch.utils.data import DataLoader
import torchvision.transforms as T
import torch.nn.functional as F
sys.path.append('../../..')
from common.modules.regressor import Regressor
import common.vision.datasets.regression as datasets
import common.vision.models as models
from common.utils.data import ForeverDataIterator
from common.utils.meter import AverageMeter, ProgressMeter
from common.utils.logger import CompleteLogger
from common.utils.analysis import collect_feature, tsne, a_distance
sys.path.append('.')
from utils import convert_model, validate
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
def main(args: argparse.Namespace):
logger = CompleteLogger(args.log, args.phase)
print(args)
if args.seed is not None:
random.seed(args.seed)
torch.manual_seed(args.seed)
cudnn.deterministic = True
warnings.warn('You have chosen to seed training. '
'This will turn on the CUDNN deterministic setting, '
'which can slow down your training considerably! '
'You may see unexpected behavior when restarting '
'from checkpoints.')
cudnn.benchmark = True
# Data loading code
normalize = T.Normalize(mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225])
train_transform = T.Compose([
T.Resize(args.resize_size),
T.ToTensor(),
normalize
])
val_transform = T.Compose([
T.Resize(args.resize_size),
T.ToTensor(),
normalize
])
dataset = datasets.__dict__[args.data]
train_source_dataset = dataset(root=args.root, task=args.source, split='train', download=True, transform=train_transform)
train_source_loader = DataLoader(train_source_dataset, batch_size=args.batch_size,
shuffle=True, num_workers=args.workers, drop_last=True)
train_target_dataset = dataset(root=args.root, task=args.target, split='train', download=True, transform=train_transform)
train_target_loader = DataLoader(train_target_dataset, batch_size=args.batch_size,
shuffle=True, num_workers=args.workers, drop_last=True)
val_dataset = dataset(root=args.root, task=args.target, split='test', download=True, transform=val_transform)
val_loader = DataLoader(val_dataset, batch_size=args.batch_size, shuffle=False, num_workers=args.workers)
train_source_iter = ForeverDataIterator(train_source_loader)
train_target_iter = ForeverDataIterator(train_target_loader)
# create model
print("=> using pre-trained model '{}'".format(args.arch))
backbone = models.__dict__[args.arch](pretrained=True)
if args.normalization == 'IN':
backbone = convert_model(backbone)
num_factors = train_source_dataset.num_factors
regressor = Regressor(backbone=backbone, num_factors=num_factors).to(device)
print(regressor)
# define optimizer and lr scheduler
optimizer = SGD(regressor.get_parameters(), args.lr, momentum=args.momentum, weight_decay=args.wd, nesterov=True)
lr_scheduler = LambdaLR(optimizer, lambda x: args.lr * (1. + args.lr_gamma * float(x)) ** (-args.lr_decay))
# resume from the best checkpoint
if args.phase != 'train':
checkpoint = torch.load(logger.get_checkpoint_path('best'), map_location='cpu')
regressor.load_state_dict(checkpoint)
# analysis the model
if args.phase == 'analysis':
train_source_loader = DataLoader(train_source_dataset, batch_size=args.batch_size,
shuffle=True, num_workers=args.workers, drop_last=True)
train_target_loader = DataLoader(train_target_dataset, batch_size=args.batch_size,
shuffle=True, num_workers=args.workers, drop_last=True)
# extract features from both domains
feature_extractor = nn.Sequential(regressor.backbone, regressor.bottleneck).to(device)
source_feature = collect_feature(train_source_loader, feature_extractor, device)
target_feature = collect_feature(train_target_loader, feature_extractor, device)
# plot t-SNE
tSNE_filename = osp.join(logger.visualize_directory, 'TSNE.pdf')
tsne.visualize(source_feature, target_feature, tSNE_filename)
print("Saving t-SNE to", tSNE_filename)
# calculate A-distance, which is a measure for distribution discrepancy
A_distance = a_distance.calculate(source_feature, target_feature, device)
print("A-distance =", A_distance)
return
if args.phase == 'test':
mae = validate(val_loader, regressor, args, train_source_dataset.factors, device)
print(mae)
return
# start training
best_mae = 100000.
for epoch in range(args.epochs):
# train for one epoch
print("lr", lr_scheduler.get_lr())
train(train_source_iter, train_target_iter, regressor, optimizer,
lr_scheduler, epoch, args)
# evaluate on validation set
mae = validate(val_loader, regressor, args, train_source_dataset.factors, device)
# remember best mae and save checkpoint
torch.save(regressor.state_dict(), logger.get_checkpoint_path('latest'))
if mae < best_mae:
shutil.copy(logger.get_checkpoint_path('latest'), logger.get_checkpoint_path('best'))
best_mae = min(mae, best_mae)
print("mean MAE {:6.3f} best MAE {:6.3f}".format(mae, best_mae))
print("best_mae = {:6.3f}".format(best_mae))
logger.close()
def train(train_source_iter: ForeverDataIterator, train_target_iter: ForeverDataIterator,
model: Regressor, optimizer: SGD,
lr_scheduler: LambdaLR, epoch: int, args: argparse.Namespace):
batch_time = AverageMeter('Time', ':4.2f')
data_time = AverageMeter('Data', ':3.1f')
mse_losses = AverageMeter('MSE Loss', ':6.3f')
mae_losses_s = AverageMeter('MAE Loss (s)', ':6.3f')
mae_losses_t = AverageMeter('MAE Loss (t)', ':6.3f')
progress = ProgressMeter(
args.iters_per_epoch,
[batch_time, data_time, mse_losses, mae_losses_s, mae_losses_t],
prefix="Epoch: [{}]".format(epoch))
# switch to train mode
model.train()
end = time.time()
for i in range(args.iters_per_epoch):
optimizer.zero_grad()
x_s, labels_s = next(train_source_iter)
x_s = x_s.to(device)
labels_s = labels_s.to(device).float()
x_t, labels_t = next(train_target_iter)
x_t = x_t.to(device)
labels_t = labels_t.to(device).float()
# measure data loading time
data_time.update(time.time() - end)
# compute output
y_s, _ = model(x_s)
y_t, _ = model(x_t)
mse_loss = F.mse_loss(y_s, labels_s)
mae_loss_s = F.l1_loss(y_s, labels_s)
mae_loss_t = F.l1_loss(y_t, labels_t)
loss = mse_loss
mse_losses.update(mse_loss.item(), x_s.size(0))
mae_losses_s.update(mae_loss_s.item(), x_s.size(0))
mae_losses_t.update(mae_loss_t.item(), x_s.size(0))
# compute gradient and do SGD step
loss.backward()
optimizer.step()
lr_scheduler.step()
# measure elapsed time
batch_time.update(time.time() - end)
end = time.time()
if i % args.print_freq == 0:
progress.display(i)
if __name__ == '__main__':
architecture_names = sorted(
name for name in models.__dict__
if name.islower() and not name.startswith("__")
and callable(models.__dict__[name])
)
dataset_names = sorted(
name for name in datasets.__dict__
if not name.startswith("__") and callable(datasets.__dict__[name])
)
parser = argparse.ArgumentParser(description='Source Only for Regression Domain Adaptation')
# dataset parameters
parser.add_argument('root', metavar='DIR',
help='root path of dataset')
parser.add_argument('-d', '--data', metavar='DATA', default='DSprites',
help='dataset: ' + ' | '.join(dataset_names) +
' (default: Office31)')
parser.add_argument('-s', '--source', help='source domain(s)')
parser.add_argument('-t', '--target', help='target domain(s)')
parser.add_argument('--resize-size', type=int, default=128)
# model parameters
parser.add_argument('-a', '--arch', metavar='ARCH', default='resnet18',
choices=architecture_names,
help='backbone architecture: ' +
' | '.join(architecture_names) +
' (default: resnet18)')
parser.add_argument('--normalization', default='BN', type=str, choices=["IN", "BN"])
# training parameters
parser.add_argument('-b', '--batch-size', default=36, type=int,
metavar='N',
help='mini-batch size (default: 32)')
parser.add_argument('--lr', '--learning-rate', default=0.1, type=float,
metavar='LR', help='initial learning rate', dest='lr')
parser.add_argument('--lr-gamma', default=0.0001, type=float, help='parameter for lr scheduler')
parser.add_argument('--lr-decay', default=0.75, type=float, help='parameter for lr scheduler')
parser.add_argument('--momentum', default=0.9, type=float, metavar='M',
help='momentum')
parser.add_argument('--wd', '--weight-decay', default=0.0005, type=float,
metavar='W', help='weight decay (default: 5e-4)')
parser.add_argument('-j', '--workers', default=2, type=int, metavar='N',
help='number of data loading workers (default: 4)')
parser.add_argument('--epochs', default=20, type=int, metavar='N',
help='number of total epochs to run')
parser.add_argument('-i', '--iters-per-epoch', default=500, type=int,
help='Number of iterations per epoch')
parser.add_argument('-p', '--print-freq', default=100, type=int,
metavar='N', help='print frequency (default: 100)')
parser.add_argument('--seed', default=None, type=int,
help='seed for initializing training. ')
parser.add_argument("--log", type=str, default='src_only',
help="Where to save logs, checkpoints and debugging images.")
parser.add_argument("--phase", type=str, default='train', choices=['train', 'test', 'analysis'],
help="When phase is 'test', only test the model."
"When phase is 'analysis', only analysis the model.")
args = parser.parse_args()
main(args)
================================================
FILE: domain_generalization/Transfer-Learning-Library/examples/domain_adaptation/image_regression/source_only.sh
================================================
# DSprites
CUDA_VISIBLE_DEVICES=0 python source_only.py data/dSprites -d DSprites -s C -t N -a resnet18 --epochs 20 --seed 0 -b 128 --log logs/src_only/DSprites_C2N
CUDA_VISIBLE_DEVICES=0 python source_only.py data/dSprites -d DSprites -s C -t S -a resnet18 --epochs 20 --seed 0 -b 128 --log logs/src_only/DSprites_C2S
CUDA_VISIBLE_DEVICES=0 python source_only.py data/dSprites -d DSprites -s N -t C -a resnet18 --epochs 20 --seed 0 -b 128 --log logs/src_only/DSprites_N2C
CUDA_VISIBLE_DEVICES=0 python source_only.py data/dSprites -d DSprites -s N -t S -a resnet18 --epochs 20 --seed 0 -b 128 --log logs/src_only/DSprites_N2S
CUDA_VISIBLE_DEVICES=0 python source_only.py data/dSprites -d DSprites -s S -t C -a resnet18 --epochs 20 --seed 0 -b 128 --log logs/src_only/DSprites_S2C
CUDA_VISIBLE_DEVICES=0 python source_only.py data/dSprites -d DSprites -s S -t N -a resnet18 --epochs 20 --seed 0 -b 128 --log logs/src_only/DSprites_S2N
# MPI3D
CUDA_VISIBLE_DEVICES=0 python source_only.py data/mpi3d -d MPI3D -s RL -t RC -a resnet18 --epochs 40 --seed 0 -b 36 --log logs/src_only/MPI3D_RL2RC
CUDA_VISIBLE_DEVICES=0 python source_only.py data/mpi3d -d MPI3D -s RL -t T -a resnet18 --epochs 40 --seed 0 -b 36 --log logs/src_only/MPI3D_RL2T
CUDA_VISIBLE_DEVICES=0 python source_only.py data/mpi3d -d MPI3D -s RC -t RL -a resnet18 --epochs 40 --seed 0 -b 36 --log logs/src_only/MPI3D_RC2RL
CUDA_VISIBLE_DEVICES=0 python source_only.py data/mpi3d -d MPI3D -s RC -t T -a resnet18 --epochs 40 --seed 0 -b 36 --log logs/src_only/MPI3D_RC2T
CUDA_VISIBLE_DEVICES=0 python source_only.py data/mpi3d -d MPI3D -s T -t RL -a resnet18 --epochs 40 --seed 0 -b 36 --log logs/src_only/MPI3D_T2RL
CUDA_VISIBLE_DEVICES=0 python source_only.py data/mpi3d -d MPI3D -s T -t RC -a resnet18 --epochs 40 --seed 0 -b 36 --log logs/src_only/MPI3D_T2RC
================================================
FILE: domain_generalization/Transfer-Learning-Library/examples/domain_adaptation/image_regression/utils.py
================================================
"""
@author: Junguang Jiang
@contact: JiangJunguang1123@outlook.com
"""
import sys
import time
import torch
import torch.nn.functional as F
sys.path.append('../../..')
from common.utils.meter import AverageMeter, ProgressMeter
from torch.nn.modules.batchnorm import BatchNorm1d, BatchNorm2d, BatchNorm3d
from torch.nn.modules.instancenorm import InstanceNorm1d, InstanceNorm2d, InstanceNorm3d
def convert_model(module):
"""convert BatchNorms in the `module` into InstanceNorms"""
source_modules = (BatchNorm1d, BatchNorm2d, BatchNorm3d)
target_modules = (InstanceNorm1d, InstanceNorm2d, InstanceNorm3d)
for src_module, tgt_module in zip(source_modules, target_modules):
if isinstance(module, src_module):
mod = tgt_module(module.num_features, module.eps, module.momentum, module.affine)
module = mod
for name, child in module.named_children():
module.add_module(name, convert_model(child))
return module
def validate(val_loader, model, args, factors, device):
batch_time = AverageMeter('Time', ':6.3f')
mae_losses = [AverageMeter('mae {}'.format(factor), ':6.3f') for factor in factors]
progress = ProgressMeter(
len(val_loader),
[batch_time] + mae_losses,
prefix='Test: ')
# switch to evaluate mode
model.eval()
with torch.no_grad():
end = time.time()
for i, (images, target) in enumerate(val_loader):
images = images.to(device)
target = target.to(device)
# compute output
output = model(images)
for j in range(len(factors)):
mae_loss = F.l1_loss(output[:, j], target[:, j])
mae_losses[j].update(mae_loss.item(), images.size(0))
# measure elapsed time
batch_time.update(time.time() - end)
end = time.time()
if i % args.print_freq == 0:
progress.display(i)
for i, factor in enumerate(factors):
print("{} MAE {mae.avg:6.3f}".format(factor, mae=mae_losses[i]))
mean_mae = sum(l.avg for l in mae_losses) / len(factors)
return mean_mae
================================================
FILE: domain_generalization/Transfer-Learning-Library/examples/domain_adaptation/keypoint_detection/README.md
================================================
# Unsupervised Domain Adaptation for Keypoint Detection
## Dataset
Following datasets can be downloaded automatically:
- [Rendered Handpose Dataset](https://lmb.informatik.uni-freiburg.de/resources/datasets/RenderedHandposeDataset.en.html)
- [Hand-3d-Studio Dataset](https://www.yangangwang.com/papers/ZHAO-H3S-2020-02.html)
- [FreiHAND Dataset](https://lmb.informatik.uni-freiburg.de/projects/freihand/)
- [Surreal Dataset](https://www.di.ens.fr/willow/research/surreal/data/)
- [LSP Dataset](http://sam.johnson.io/research/lsp.html)
You need to prepare following datasets manually if you want to use them:
- [Human3.6M Dataset](http://vision.imar.ro/human3.6m/description.php)
and prepare them following [Documentations for Human3.6M Dataset](/common/vision/datasets/keypoint_detection/human36m.py).
## Supported Methods
Supported methods include:
- [Regressive Domain Adaptation for Unsupervised Keypoint Detection (RegDA, CVPR 2021)](http://ise.thss.tsinghua.edu.cn/~mlong/doc/regressive-domain-adaptation-cvpr21.pdf)
## Experiment and Results
The shell files give the script to reproduce the [benchmarks](/docs/dalib/benchmarks/keypoint_detection.rst) with specified hyper-parameters.
For example, if you want to train DANN on Office31, use the following script
```shell script
# Train a RegDA on RHD -> H3D task using PoseResNet.
# Assume you have put the datasets under the path `data/RHD` and `data/H3D_crop`,
# or you are glad to download the datasets automatically from the Internet to this path
CUDA_VISIBLE_DEVICES=0 python regda.py data/RHD data/H3D_crop \
-s RenderedHandPose -t Hand3DStudio --finetune --seed 0 --debug --log logs/regda/rhd2h3d
```
For more information please refer to [Get Started](/docs/get_started/quickstart.rst) for help.
## TODO
Support methods: CycleGAN
## Citation
If you use these methods in your research, please consider citing.
```
@InProceedings{RegDA,
author = {Junguang Jiang and
Yifei Ji and
Ximei Wang and
Yufeng Liu and
Jianmin Wang and
Mingsheng Long},
title = {Regressive Domain Adaptation for Unsupervised Keypoint Detection},
booktitle = {CVPR},
year = {2021}
}
```
================================================
FILE: domain_generalization/Transfer-Learning-Library/examples/domain_adaptation/keypoint_detection/regda.py
================================================
"""
@author: Junguang Jiang
@contact: JiangJunguang1123@outlook.com
"""
import random
import time
import warnings
import sys
import argparse
import shutil
import torch
import torch.backends.cudnn as cudnn
from torch.optim import SGD
from torch.optim.lr_scheduler import LambdaLR, MultiStepLR
from torch.utils.data import DataLoader
from torchvision.transforms import Compose, ToPILImage
sys.path.append('../../..')
from dalib.adaptation.regda import PoseResNet2d as RegDAPoseResNet, \
PseudoLabelGenerator2d, RegressionDisparity
import common.vision.models as models
from common.vision.models.keypoint_detection.pose_resnet import Upsampling, PoseResNet
from common.vision.models.keypoint_detection.loss import JointsKLLoss
import common.vision.datasets.keypoint_detection as datasets
import common.vision.transforms.keypoint_detection as T
from common.vision.transforms import Denormalize
from common.utils.data import ForeverDataIterator
from common.utils.meter import AverageMeter, ProgressMeter, AverageMeterDict
from common.utils.metric.keypoint_detection import accuracy
from common.utils.logger import CompleteLogger
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
def main(args: argparse.Namespace):
logger = CompleteLogger(args.log, args.phase)
print(args)
if args.seed is not None:
random.seed(args.seed)
torch.manual_seed(args.seed)
cudnn.deterministic = True
warnings.warn('You have chosen to seed training. '
'This will turn on the CUDNN deterministic setting, '
'which can slow down your training considerably! '
'You may see unexpected behavior when restarting '
'from checkpoints.')
cudnn.benchmark = True
# Data loading code
normalize = T.Normalize([0.485, 0.456, 0.406], [0.229, 0.224, 0.225])
train_transform = T.Compose([
T.RandomRotation(args.rotation),
T.RandomResizedCrop(size=args.image_size, scale=args.resize_scale),
T.ColorJitter(brightness=0.25, contrast=0.25, saturation=0.25),
T.GaussianBlur(),
T.ToTensor(),
normalize
])
val_transform = T.Compose([
T.Resize(args.image_size),
T.ToTensor(),
normalize
])
image_size = (args.image_size, args.image_size)
heatmap_size = (args.heatmap_size, args.heatmap_size)
source_dataset = datasets.__dict__[args.source]
train_source_dataset = source_dataset(root=args.source_root, transforms=train_transform,
image_size=image_size, heatmap_size=heatmap_size)
train_source_loader = DataLoader(train_source_dataset, batch_size=args.batch_size,
shuffle=True, num_workers=args.workers, pin_memory=True, drop_last=True)
val_source_dataset = source_dataset(root=args.source_root, split='test', transforms=val_transform,
image_size=image_size, heatmap_size=heatmap_size)
val_source_loader = DataLoader(val_source_dataset, batch_size=args.batch_size, shuffle=False, pin_memory=True)
target_dataset = datasets.__dict__[args.target]
train_target_dataset = target_dataset(root=args.target_root, transforms=train_transform,
image_size=image_size, heatmap_size=heatmap_size)
train_target_loader = DataLoader(train_target_dataset, batch_size=args.batch_size,
shuffle=True, num_workers=args.workers, pin_memory=True, drop_last=True)
val_target_dataset = target_dataset(root=args.target_root, split='test', transforms=val_transform,
image_size=image_size, heatmap_size=heatmap_size)
val_target_loader = DataLoader(val_target_dataset, batch_size=args.batch_size, shuffle=False, pin_memory=True)
print("Source train:", len(train_source_loader))
print("Target train:", len(train_target_loader))
print("Source test:", len(val_source_loader))
print("Target test:", len(val_target_loader))
train_source_iter = ForeverDataIterator(train_source_loader)
train_target_iter = ForeverDataIterator(train_target_loader)
# create model
backbone = models.__dict__[args.arch](pretrained=True)
upsampling = Upsampling(backbone.out_features)
num_keypoints = train_source_dataset.num_keypoints
model = RegDAPoseResNet(backbone, upsampling, 256, num_keypoints, num_head_layers=args.num_head_layers, finetune=True).to(device)
# define loss function
criterion = JointsKLLoss()
pseudo_label_generator = PseudoLabelGenerator2d(num_keypoints, args.heatmap_size, args.heatmap_size)
regression_disparity = RegressionDisparity(pseudo_label_generator, JointsKLLoss(epsilon=1e-7))
# define optimizer and lr scheduler
optimizer_f = SGD([
{'params': backbone.parameters(), 'lr': 0.1},
{'params': upsampling.parameters(), 'lr': 0.1},
], lr=0.1, momentum=args.momentum, weight_decay=args.wd, nesterov=True)
optimizer_h = SGD(model.head.parameters(), lr=1., momentum=args.momentum, weight_decay=args.wd, nesterov=True)
optimizer_h_adv = SGD(model.head_adv.parameters(), lr=1., momentum=args.momentum, weight_decay=args.wd, nesterov=True)
lr_decay_function = lambda x: args.lr * (1. + args.lr_gamma * float(x)) ** (-args.lr_decay)
lr_scheduler_f = LambdaLR(optimizer_f, lr_decay_function)
lr_scheduler_h = LambdaLR(optimizer_h, lr_decay_function)
lr_scheduler_h_adv = LambdaLR(optimizer_h_adv, lr_decay_function)
start_epoch = 0
if args.resume is None:
if args.pretrain is None:
# first pretrain the backbone and upsampling
print("Pretraining the model on source domain.")
args.pretrain = logger.get_checkpoint_path('pretrain')
pretrained_model = PoseResNet(backbone, upsampling, 256, num_keypoints, True).to(device)
optimizer = SGD(pretrained_model.get_parameters(lr=args.lr), momentum=args.momentum, weight_decay=args.wd, nesterov=True)
lr_scheduler = MultiStepLR(optimizer, args.lr_step, args.lr_factor)
best_acc = 0
for epoch in range(args.pretrain_epochs):
lr_scheduler.step()
print(lr_scheduler.get_lr())
pretrain(train_source_iter, pretrained_model, criterion, optimizer, epoch, args)
source_val_acc = validate(val_source_loader, pretrained_model, criterion, None, args)
# remember best acc and save checkpoint
if source_val_acc['all'] > best_acc:
best_acc = source_val_acc['all']
torch.save(
{
'model': pretrained_model.state_dict()
}, args.pretrain
)
print("Source: {} best: {}".format(source_val_acc['all'], best_acc))
# load from the pretrained checkpoint
pretrained_dict = torch.load(args.pretrain, map_location='cpu')['model']
model_dict = model.state_dict()
# remove keys from pretrained dict that doesn't appear in model dict
pretrained_dict = {k: v for k, v in pretrained_dict.items() if k in model_dict}
model.load_state_dict(pretrained_dict, strict=False)
else:
# optionally resume from a checkpoint
checkpoint = torch.load(args.resume, map_location='cpu')
model.load_state_dict(checkpoint['model'])
optimizer_f.load_state_dict(checkpoint['optimizer_f'])
optimizer_h.load_state_dict(checkpoint['optimizer_h'])
optimizer_h_adv.load_state_dict(checkpoint['optimizer_h_adv'])
lr_scheduler_f.load_state_dict(checkpoint['lr_scheduler_f'])
lr_scheduler_h.load_state_dict(checkpoint['lr_scheduler_h'])
lr_scheduler_h_adv.load_state_dict(checkpoint['lr_scheduler_h_adv'])
start_epoch = checkpoint['epoch'] + 1
# define visualization function
tensor_to_image = Compose([
Denormalize([0.485, 0.456, 0.406], [0.229, 0.224, 0.225]),
ToPILImage()
])
def visualize(image, keypoint2d, name, heatmaps=None):
"""
Args:
image (tensor): image in shape 3 x H x W
keypoint2d (tensor): keypoints in shape K x 2
name: name of the saving image
"""
train_source_dataset.visualize(tensor_to_image(image),
keypoint2d, logger.get_image_path("{}.jpg".format(name)))
if args.phase == 'test':
# evaluate on validation set
source_val_acc = validate(val_source_loader, model, criterion, None, args)
target_val_acc = validate(val_target_loader, model, criterion, visualize, args)
print("Source: {:4.3f} Target: {:4.3f}".format(source_val_acc['all'], target_val_acc['all']))
for name, acc in target_val_acc.items():
print("{}: {:4.3f}".format(name, acc))
return
# start training
best_acc = 0
print("Start regression domain adaptation.")
for epoch in range(start_epoch, args.epochs):
logger.set_epoch(epoch)
print(lr_scheduler_f.get_lr(), lr_scheduler_h.get_lr(), lr_scheduler_h_adv.get_lr())
# train for one epoch
train(train_source_iter, train_target_iter, model, criterion, regression_disparity,
optimizer_f, optimizer_h, optimizer_h_adv, lr_scheduler_f, lr_scheduler_h, lr_scheduler_h_adv,
epoch, visualize if args.debug else None, args)
# evaluate on validation set
source_val_acc = validate(val_source_loader, model, criterion, None, args)
target_val_acc = validate(val_target_loader, model, criterion, visualize if args.debug else None, args)
# remember best acc and save checkpoint
torch.save(
{
'model': model.state_dict(),
'optimizer_f': optimizer_f.state_dict(),
'optimizer_h': optimizer_h.state_dict(),
'optimizer_h_adv': optimizer_h_adv.state_dict(),
'lr_scheduler_f': lr_scheduler_f.state_dict(),
'lr_scheduler_h': lr_scheduler_h.state_dict(),
'lr_scheduler_h_adv': lr_scheduler_h_adv.state_dict(),
'epoch': epoch,
'args': args
}, logger.get_checkpoint_path(epoch)
)
if target_val_acc['all'] > best_acc:
shutil.copy(logger.get_checkpoint_path(epoch), logger.get_checkpoint_path('best'))
best_acc = target_val_acc['all']
print("Source: {:4.3f} Target: {:4.3f} Target(best): {:4.3f}".format(source_val_acc['all'], target_val_acc['all'], best_acc))
for name, acc in target_val_acc.items():
print("{}: {:4.3f}".format(name, acc))
logger.close()
def pretrain(train_source_iter, model, criterion, optimizer,
epoch: int, args: argparse.Namespace):
batch_time = AverageMeter('Time', ':4.2f')
data_time = AverageMeter('Data', ':3.1f')
losses_s = AverageMeter('Loss (s)', ":.2e")
acc_s = AverageMeter("Acc (s)", ":3.2f")
progress = ProgressMeter(
args.iters_per_epoch,
[batch_time, data_time, losses_s, acc_s],
prefix="Epoch: [{}]".format(epoch))
# switch to train mode
model.train()
end = time.time()
for i in range(args.iters_per_epoch):
optimizer.zero_grad()
x_s, label_s, weight_s, meta_s = next(train_source_iter)
x_s = x_s.to(device)
label_s = label_s.to(device)
weight_s = weight_s.to(device)
# measure data loading time
data_time.update(time.time() - end)
# compute output
y_s = model(x_s)
loss_s = criterion(y_s, label_s, weight_s)
# compute gradient and do SGD step
loss_s.backward()
optimizer.step()
# measure accuracy and record loss
_, avg_acc_s, cnt_s, pred_s = accuracy(y_s.detach().cpu().numpy(),
label_s.detach().cpu().numpy())
acc_s.update(avg_acc_s, cnt_s)
losses_s.update(loss_s, cnt_s)
# measure elapsed time
batch_time.update(time.time() - end)
end = time.time()
if i % args.print_freq == 0:
progress.display(i)
def train(train_source_iter, train_target_iter, model, criterion,regression_disparity,
optimizer_f, optimizer_h, optimizer_h_adv, lr_scheduler_f, lr_scheduler_h, lr_scheduler_h_adv,
epoch: int, visualize, args: argparse.Namespace):
batch_time = AverageMeter('Time', ':4.2f')
data_time = AverageMeter('Data', ':3.1f')
losses_s = AverageMeter('Loss (s)', ":.2e")
losses_gf = AverageMeter('Loss (t, false)', ":.2e")
losses_gt = AverageMeter('Loss (t, truth)', ":.2e")
acc_s = AverageMeter("Acc (s)", ":3.2f")
acc_t = AverageMeter("Acc (t)", ":3.2f")
acc_s_adv = AverageMeter("Acc (s, adv)", ":3.2f")
acc_t_adv = AverageMeter("Acc (t, adv)", ":3.2f")
progress = ProgressMeter(
args.iters_per_epoch,
[batch_time, data_time, losses_s, losses_gf, losses_gt, acc_s, acc_t, acc_s_adv, acc_t_adv],
prefix="Epoch: [{}]".format(epoch))
# switch to train mode
model.train()
end = time.time()
for i in range(args.iters_per_epoch):
x_s, label_s, weight_s, meta_s = next(train_source_iter)
x_t, label_t, weight_t, meta_t = next(train_target_iter)
x_s = x_s.to(device)
label_s = label_s.to(device)
weight_s = weight_s.to(device)
x_t = x_t.to(device)
label_t = label_t.to(device)
weight_t = weight_t.to(device)
# measure data loading time
data_time.update(time.time() - end)
# Step A train all networks to minimize loss on source domain
optimizer_f.zero_grad()
optimizer_h.zero_grad()
optimizer_h_adv.zero_grad()
y_s, y_s_adv = model(x_s)
loss_s = criterion(y_s, label_s, weight_s) + \
args.margin * args.trade_off * regression_disparity(y_s, y_s_adv, weight_s, mode='min')
loss_s.backward()
optimizer_f.step()
optimizer_h.step()
optimizer_h_adv.step()
# Step B train adv regressor to maximize regression disparity
optimizer_h_adv.zero_grad()
y_t, y_t_adv = model(x_t)
loss_ground_false = args.trade_off * regression_disparity(y_t, y_t_adv, weight_t, mode='max')
loss_ground_false.backward()
optimizer_h_adv.step()
# Step C train feature extractor to minimize regression disparity
optimizer_f.zero_grad()
y_t, y_t_adv = model(x_t)
loss_ground_truth = args.trade_off * regression_disparity(y_t, y_t_adv, weight_t, mode='min')
loss_ground_truth.backward()
optimizer_f.step()
# do update step
model.step()
lr_scheduler_f.step()
lr_scheduler_h.step()
lr_scheduler_h_adv.step()
# measure accuracy and record loss
_, avg_acc_s, cnt_s, pred_s = accuracy(y_s.detach().cpu().numpy(),
label_s.detach().cpu().numpy())
acc_s.update(avg_acc_s, cnt_s)
_, avg_acc_t, cnt_t, pred_t = accuracy(y_t.detach().cpu().numpy(),
label_t.detach().cpu().numpy())
acc_t.update(avg_acc_t, cnt_t)
_, avg_acc_s_adv, cnt_s_adv, pred_s_adv = accuracy(y_s_adv.detach().cpu().numpy(),
label_s.detach().cpu().numpy())
acc_s_adv.update(avg_acc_s_adv, cnt_s)
_, avg_acc_t_adv, cnt_t_adv, pred_t_adv = accuracy(y_t_adv.detach().cpu().numpy(),
label_t.detach().cpu().numpy())
acc_t_adv.update(avg_acc_t_adv, cnt_t)
losses_s.update(loss_s, cnt_s)
losses_gf.update(loss_ground_false, cnt_s)
losses_gt.update(loss_ground_truth, cnt_s)
# measure elapsed time
batch_time.update(time.time() - end)
end = time.time()
if i % args.print_freq == 0:
progress.display(i)
if visualize is not None:
visualize(x_s[0], pred_s[0] * args.image_size / args.heatmap_size, "source_{}_pred".format(i))
visualize(x_s[0], meta_s['keypoint2d'][0], "source_{}_label".format(i))
visualize(x_t[0], pred_t[0] * args.image_size / args.heatmap_size, "target_{}_pred".format(i))
visualize(x_t[0], meta_t['keypoint2d'][0], "target_{}_label".format(i))
visualize(x_s[0], pred_s_adv[0] * args.image_size / args.heatmap_size, "source_adv_{}_pred".format(i))
visualize(x_t[0], pred_t_adv[0] * args.image_size / args.heatmap_size, "target_adv_{}_pred".format(i))
def validate(val_loader, model, criterion, visualize, args: argparse.Namespace):
batch_time = AverageMeter('Time', ':6.3f')
losses = AverageMeter('Loss', ':.2e')
acc = AverageMeterDict(val_loader.dataset.keypoints_group.keys(), ":3.2f")
progress = ProgressMeter(
len(val_loader),
[batch_time, losses, acc['all']],
prefix='Test: ')
# switch to evaluate mode
model.eval()
with torch.no_grad():
end = time.time()
for i, (x, label, weight, meta) in enumerate(val_loader):
x = x.to(device)
label = label.to(device)
weight = weight.to(device)
# compute output
y = model(x)
loss = criterion(y, label, weight)
# measure accuracy and record loss
losses.update(loss.item(), x.size(0))
acc_per_points, avg_acc, cnt, pred = accuracy(y.cpu().numpy(),
label.cpu().numpy())
group_acc = val_loader.dataset.group_accuracy(acc_per_points)
acc.update(group_acc, x.size(0))
# measure elapsed time
batch_time.update(time.time() - end)
end = time.time()
if i % args.print_freq == 0:
progress.display(i)
if visualize is not None:
visualize(x[0], pred[0] * args.image_size / args.heatmap_size, "val_{}_pred.jpg".format(i))
visualize(x[0], meta['keypoint2d'][0], "val_{}_label.jpg".format(i))
return acc.average()
if __name__ == '__main__':
architecture_names = sorted(
name for name in models.__dict__
if name.islower() and not name.startswith("__")
and callable(models.__dict__[name])
)
dataset_names = sorted(
name for name in datasets.__dict__
if not name.startswith("__") and callable(datasets.__dict__[name])
)
parser = argparse.ArgumentParser(description='RegDA for Keypoint Detection Domain Adaptation')
# dataset parameters
parser.add_argument('source_root', help='root path of the source dataset')
parser.add_argument('target_root', help='root path of the target dataset')
parser.add_argument('-s', '--source', help='source domain(s)')
parser.add_argument('-t', '--target', help='target domain(s)')
parser.add_argument('--resize-scale', nargs='+', type=float, default=(0.6, 1.3),
help='scale range for the RandomResizeCrop augmentation')
parser.add_argument('--rotation', type=int, default=180,
help='rotation range of the RandomRotation augmentation')
parser.add_argument('--image-size', type=int, default=256,
help='input image size')
parser.add_argument('--heatmap-size', type=int, default=64,
help='output heatmap size')
# model parameters
parser.add_argument('-a', '--arch', metavar='ARCH', default='resnet101',
choices=architecture_names,
help='backbone architecture: ' +
' | '.join(architecture_names) +
' (default: resnet101)')
parser.add_argument("--pretrain", type=str, default=None,
help="Where restore pretrained model parameters from.")
parser.add_argument("--resume", type=str, default=None,
help="where restore model parameters from.")
parser.add_argument('--num-head-layers', type=int, default=2)
parser.add_argument('--margin', type=float, default=4., help="margin gamma")
parser.add_argument('--trade-off', default=1., type=float,
help='the trade-off hyper-parameter for transfer loss')
# training parameters
parser.add_argument('-b', '--batch-size', default=32, type=int,
metavar='N',
help='mini-batch size (default: 32)')
parser.add_argument('--lr', '--learning-rate', default=0.1, type=float,
metavar='LR', help='initial learning rate', dest='lr')
parser.add_argument('--momentum', default=0.9, type=float, metavar='M',
help='momentum')
parser.add_argument('--wd', '--weight-decay', default=0.0001, type=float,
metavar='W', help='weight decay (default: 1e-4)')
parser.add_argument('--lr-gamma', default=0.0001, type=float)
parser.add_argument('--lr-decay', default=0.75, type=float, help='parameter for lr scheduler')
parser.add_argument('--lr-step', default=[45, 60], type=tuple, help='parameter for lr scheduler')
parser.add_argument('--lr-factor', default=0.1, type=float, help='parameter for lr scheduler')
parser.add_argument('-j', '--workers', default=4, type=int, metavar='N',
help='number of data loading workers (default: 4)')
parser.add_argument('--pretrain_epochs', default=70, type=int, metavar='N',
help='number of total epochs to run')
parser.add_argument('--epochs', default=30, type=int, metavar='N',
help='number of total epochs to run')
parser.add_argument('-i', '--iters-per-epoch', default=500, type=int,
help='Number of iterations per epoch')
parser.add_argument('-p', '--print-freq', default=100, type=int,
metavar='N', help='print frequency (default: 100)')
parser.add_argument('--seed', default=None, type=int,
help='seed for initializing training. ')
parser.add_argument("--log", type=str, default='regda',
help="Where to save logs, checkpoints and debugging images.")
parser.add_argument("--phase", type=str, default='train', choices=['train', 'test'],
help="When phase is 'test', only test the model.")
parser.add_argument('--debug', action="store_true",
help='In the debug mode, save images and predictions')
args = parser.parse_args()
main(args)
================================================
FILE: domain_generalization/Transfer-Learning-Library/examples/domain_adaptation/keypoint_detection/regda.sh
================================================
# Hands Dataset
CUDA_VISIBLE_DEVICES=0 python regda.py data/RHD data/H3D_crop \
-s RenderedHandPose -t Hand3DStudio --seed 0 --debug --log logs/regda/rhd2h3d
CUDA_VISIBLE_DEVICES=0 python regda.py data/FreiHand data/RHD \
-s FreiHand -t RenderedHandPose --seed 0 --debug --log logs/regda/freihand2rhd
# Body Dataset
CUDA_VISIBLE_DEVICES=0 python regda.py data/surreal_processed data/Human36M \
-s SURREAL -t Human36M --finetune --seed 0 --debug --rotation 30 --epochs 10 --log logs/regda/surreal2human36m
CUDA_VISIBLE_DEVICES=0 python regda.py data/surreal_processed data/lsp \
-s SURREAL -t LSP --finetune --seed 0 --debug --rotation 30 --log logs/regda/surreal2lsp
================================================
FILE: domain_generalization/Transfer-Learning-Library/examples/domain_adaptation/keypoint_detection/regda_fast.py
================================================
"""
@author: Junguang Jiang
@contact: JiangJunguang1123@outlook.com
"""
import random
import time
import warnings
import sys
import argparse
import shutil
import torch
import torch.backends.cudnn as cudnn
from torch.optim import SGD
from torch.optim.lr_scheduler import LambdaLR, MultiStepLR
from torch.utils.data import DataLoader
from torchvision.transforms import Compose, ToPILImage
sys.path.append('../../..')
from dalib.adaptation.regda import PoseResNet2d as RegDAPoseResNet, \
FastPseudoLabelGenerator2d, RegressionDisparity
import common.vision.models as models
from common.vision.models.keypoint_detection.pose_resnet import Upsampling, PoseResNet
from common.vision.models.keypoint_detection.loss import JointsKLLoss
import common.vision.datasets.keypoint_detection as datasets
import common.vision.transforms.keypoint_detection as T
from common.vision.transforms import Denormalize
from common.utils.data import ForeverDataIterator
from common.utils.meter import AverageMeter, ProgressMeter, AverageMeterDict
from common.utils.metric.keypoint_detection import accuracy
from common.utils.logger import CompleteLogger
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
def main(args: argparse.Namespace):
logger = CompleteLogger(args.log, args.phase)
print(args)
if args.seed is not None:
random.seed(args.seed)
torch.manual_seed(args.seed)
cudnn.deterministic = True
warnings.warn('You have chosen to seed training. '
'This will turn on the CUDNN deterministic setting, '
'which can slow down your training considerably! '
'You may see unexpected behavior when restarting '
'from checkpoints.')
cudnn.benchmark = True
# Data loading code
normalize = T.Normalize([0.485, 0.456, 0.406], [0.229, 0.224, 0.225])
train_transform = T.Compose([
T.RandomRotation(args.rotation),
T.RandomResizedCrop(size=args.image_size, scale=args.resize_scale),
T.ColorJitter(brightness=0.25, contrast=0.25, saturation=0.25),
T.GaussianBlur(),
T.ToTensor(),
normalize
])
val_transform = T.Compose([
T.Resize(args.image_size),
T.ToTensor(),
normalize
])
image_size = (args.image_size, args.image_size)
heatmap_size = (args.heatmap_size, args.heatmap_size)
source_dataset = datasets.__dict__[args.source]
train_source_dataset = source_dataset(root=args.source_root, transforms=train_transform,
image_size=image_size, heatmap_size=heatmap_size)
train_source_loader = DataLoader(train_source_dataset, batch_size=args.batch_size,
shuffle=True, num_workers=args.workers, pin_memory=True, drop_last=True)
val_source_dataset = source_dataset(root=args.source_root, split='test', transforms=val_transform,
image_size=image_size, heatmap_size=heatmap_size)
val_source_loader = DataLoader(val_source_dataset, batch_size=args.batch_size, shuffle=False, pin_memory=True)
target_dataset = datasets.__dict__[args.target]
train_target_dataset = target_dataset(root=args.target_root, transforms=train_transform,
image_size=image_size, heatmap_size=heatmap_size)
train_target_loader = DataLoader(train_target_dataset, batch_size=args.batch_size,
shuffle=True, num_workers=args.workers, pin_memory=True, drop_last=True)
val_target_dataset = target_dataset(root=args.target_root, split='test', transforms=val_transform,
image_size=image_size, heatmap_size=heatmap_size)
val_target_loader = DataLoader(val_target_dataset, batch_size=args.batch_size, shuffle=False, pin_memory=True)
print("Source train:", len(train_source_loader))
print("Target train:", len(train_target_loader))
print("Source test:", len(val_source_loader))
print("Target test:", len(val_target_loader))
train_source_iter = ForeverDataIterator(train_source_loader)
train_target_iter = ForeverDataIterator(train_target_loader)
# create model
backbone = models.__dict__[args.arch](pretrained=True)
upsampling = Upsampling(backbone.out_features)
num_keypoints = train_source_dataset.num_keypoints
model = RegDAPoseResNet(backbone, upsampling, 256, num_keypoints, num_head_layers=args.num_head_layers, finetune=True).to(device)
# define loss function
criterion = JointsKLLoss()
pseudo_label_generator = FastPseudoLabelGenerator2d()
regression_disparity = RegressionDisparity(pseudo_label_generator, JointsKLLoss(epsilon=1e-7))
# define optimizer and lr scheduler
optimizer_f = SGD([
{'params': backbone.parameters(), 'lr': 0.1},
{'params': upsampling.parameters(), 'lr': 0.1},
], lr=0.1, momentum=args.momentum, weight_decay=args.wd, nesterov=True)
optimizer_h = SGD(model.head.parameters(), lr=1., momentum=args.momentum, weight_decay=args.wd, nesterov=True)
optimizer_h_adv = SGD(model.head_adv.parameters(), lr=1., momentum=args.momentum, weight_decay=args.wd, nesterov=True)
lr_decay_function = lambda x: args.lr * (1. + args.lr_gamma * float(x)) ** (-args.lr_decay)
lr_scheduler_f = LambdaLR(optimizer_f, lr_decay_function)
lr_scheduler_h = LambdaLR(optimizer_h, lr_decay_function)
lr_scheduler_h_adv = LambdaLR(optimizer_h_adv, lr_decay_function)
start_epoch = 0
if args.resume is None:
if args.pretrain is None:
# first pretrain the backbone and upsampling
print("Pretraining the model on source domain.")
args.pretrain = logger.get_checkpoint_path('pretrain')
pretrained_model = PoseResNet(backbone, upsampling, 256, num_keypoints, True).to(device)
optimizer = SGD(pretrained_model.get_parameters(lr=args.lr), momentum=args.momentum, weight_decay=args.wd, nesterov=True)
lr_scheduler = MultiStepLR(optimizer, args.lr_step, args.lr_factor)
best_acc = 0
for epoch in range(args.pretrain_epochs):
lr_scheduler.step()
print(lr_scheduler.get_lr())
pretrain(train_source_iter, pretrained_model, criterion, optimizer, epoch, args)
source_val_acc = validate(val_source_loader, pretrained_model, criterion, None, args)
# remember best acc and save checkpoint
if source_val_acc['all'] > best_acc:
best_acc = source_val_acc['all']
torch.save(
{
'model': pretrained_model.state_dict()
}, args.pretrain
)
print("Source: {} best: {}".format(source_val_acc['all'], best_acc))
# load from the pretrained checkpoint
pretrained_dict = torch.load(args.pretrain, map_location='cpu')['model']
model_dict = model.state_dict()
# remove keys from pretrained dict that doesn't appear in model dict
pretrained_dict = {k: v for k, v in pretrained_dict.items() if k in model_dict}
model.load_state_dict(pretrained_dict, strict=False)
else:
# optionally resume from a checkpoint
checkpoint = torch.load(args.resume, map_location='cpu')
model.load_state_dict(checkpoint['model'])
optimizer_f.load_state_dict(checkpoint['optimizer_f'])
optimizer_h.load_state_dict(checkpoint['optimizer_h'])
optimizer_h_adv.load_state_dict(checkpoint['optimizer_h_adv'])
lr_scheduler_f.load_state_dict(checkpoint['lr_scheduler_f'])
lr_scheduler_h.load_state_dict(checkpoint['lr_scheduler_h'])
lr_scheduler_h_adv.load_state_dict(checkpoint['lr_scheduler_h_adv'])
start_epoch = checkpoint['epoch'] + 1
# define visualization function
tensor_to_image = Compose([
Denormalize([0.485, 0.456, 0.406], [0.229, 0.224, 0.225]),
ToPILImage()
])
def visualize(image, keypoint2d, name, heatmaps=None):
"""
Args:
image (tensor): image in shape 3 x H x W
keypoint2d (tensor): keypoints in shape K x 2
name: name of the saving image
"""
train_source_dataset.visualize(tensor_to_image(image),
keypoint2d, logger.get_image_path("{}.jpg".format(name)))
if args.phase == 'test':
# evaluate on validation set
source_val_acc = validate(val_source_loader, model, criterion, None, args)
target_val_acc = validate(val_target_loader, model, criterion, visualize, args)
print("Source: {:4.3f} Target: {:4.3f}".format(source_val_acc['all'], target_val_acc['all']))
for name, acc in target_val_acc.items():
print("{}: {:4.3f}".format(name, acc))
return
# start training
best_acc = 0
print("Start regression domain adaptation.")
for epoch in range(start_epoch, args.epochs):
logger.set_epoch(epoch)
print(lr_scheduler_f.get_lr(), lr_scheduler_h.get_lr(), lr_scheduler_h_adv.get_lr())
# train for one epoch
train(train_source_iter, train_target_iter, model, criterion, regression_disparity,
optimizer_f, optimizer_h, optimizer_h_adv, lr_scheduler_f, lr_scheduler_h, lr_scheduler_h_adv,
epoch, visualize if args.debug else None, args)
# evaluate on validation set
source_val_acc = validate(val_source_loader, model, criterion, None, args)
target_val_acc = validate(val_target_loader, model, criterion, visualize if args.debug else None, args)
# remember best acc and save checkpoint
torch.save(
{
'model': model.state_dict(),
'optimizer_f': optimizer_f.state_dict(),
'optimizer_h': optimizer_h.state_dict(),
'optimizer_h_adv': optimizer_h_adv.state_dict(),
'lr_scheduler_f': lr_scheduler_f.state_dict(),
'lr_scheduler_h': lr_scheduler_h.state_dict(),
'lr_scheduler_h_adv': lr_scheduler_h_adv.state_dict(),
'epoch': epoch,
'args': args
}, logger.get_checkpoint_path(epoch)
)
if target_val_acc['all'] > best_acc:
shutil.copy(logger.get_checkpoint_path(epoch), logger.get_checkpoint_path('best'))
best_acc = target_val_acc['all']
print("Source: {:4.3f} Target: {:4.3f} Target(best): {:4.3f}".format(source_val_acc['all'], target_val_acc['all'], best_acc))
for name, acc in target_val_acc.items():
print("{}: {:4.3f}".format(name, acc))
logger.close()
def pretrain(train_source_iter, model, criterion, optimizer,
epoch: int, args: argparse.Namespace):
batch_time = AverageMeter('Time', ':4.2f')
data_time = AverageMeter('Data', ':3.1f')
losses_s = AverageMeter('Loss (s)', ":.2e")
acc_s = AverageMeter("Acc (s)", ":3.2f")
progress = ProgressMeter(
args.iters_per_epoch,
[batch_time, data_time, losses_s, acc_s],
prefix="Epoch: [{}]".format(epoch))
# switch to train mode
model.train()
end = time.time()
for i in range(args.iters_per_epoch):
optimizer.zero_grad()
x_s, label_s, weight_s, meta_s = next(train_source_iter)
x_s = x_s.to(device)
label_s = label_s.to(device)
weight_s = weight_s.to(device)
# measure data loading time
data_time.update(time.time() - end)
# compute output
y_s = model(x_s)
loss_s = criterion(y_s, label_s, weight_s)
# compute gradient and do SGD step
loss_s.backward()
optimizer.step()
# measure accuracy and record loss
_, avg_acc_s, cnt_s, pred_s = accuracy(y_s.detach().cpu().numpy(),
label_s.detach().cpu().numpy())
acc_s.update(avg_acc_s, cnt_s)
losses_s.update(loss_s, cnt_s)
# measure elapsed time
batch_time.update(time.time() - end)
end = time.time()
if i % args.print_freq == 0:
progress.display(i)
def train(train_source_iter, train_target_iter, model, criterion,regression_disparity,
optimizer_f, optimizer_h, optimizer_h_adv, lr_scheduler_f, lr_scheduler_h, lr_scheduler_h_adv,
epoch: int, visualize, args: argparse.Namespace):
batch_time = AverageMeter('Time', ':4.2f')
data_time = AverageMeter('Data', ':3.1f')
losses_s = AverageMeter('Loss (s)', ":.2e")
losses_gf = AverageMeter('Loss (t, false)', ":.2e")
losses_gt = AverageMeter('Loss (t, truth)', ":.2e")
acc_s = AverageMeter("Acc (s)", ":3.2f")
acc_t = AverageMeter("Acc (t)", ":3.2f")
acc_s_adv = AverageMeter("Acc (s, adv)", ":3.2f")
acc_t_adv = AverageMeter("Acc (t, adv)", ":3.2f")
progress = ProgressMeter(
args.iters_per_epoch,
[batch_time, data_time, losses_s, losses_gf, losses_gt, acc_s, acc_t, acc_s_adv, acc_t_adv],
prefix="Epoch: [{}]".format(epoch))
# switch to train mode
model.train()
end = time.time()
for i in range(args.iters_per_epoch):
x_s, label_s, weight_s, meta_s = next(train_source_iter)
x_t, label_t, weight_t, meta_t = next(train_target_iter)
x_s = x_s.to(device)
label_s = label_s.to(device)
weight_s = weight_s.to(device)
x_t = x_t.to(device)
label_t = label_t.to(device)
weight_t = weight_t.to(device)
# measure data loading time
data_time.update(time.time() - end)
# Step A train all networks to minimize loss on source domain
optimizer_f.zero_grad()
optimizer_h.zero_grad()
optimizer_h_adv.zero_grad()
y_s, y_s_adv = model(x_s)
loss_s = criterion(y_s, label_s, weight_s) + \
args.margin * args.trade_off * regression_disparity(y_s, y_s_adv, weight_s, mode='min')
loss_s.backward()
optimizer_f.step()
optimizer_h.step()
optimizer_h_adv.step()
# Step B train adv regressor to maximize regression disparity
optimizer_h_adv.zero_grad()
y_t, y_t_adv = model(x_t)
loss_ground_false = args.trade_off * regression_disparity(y_t, y_t_adv, weight_t, mode='max')
loss_ground_false.backward()
optimizer_h_adv.step()
# Step C train feature extractor to minimize regression disparity
optimizer_f.zero_grad()
y_t, y_t_adv = model(x_t)
loss_ground_truth = args.trade_off * regression_disparity(y_t, y_t_adv, weight_t, mode='min')
loss_ground_truth.backward()
optimizer_f.step()
# do update step
model.step()
lr_scheduler_f.step()
lr_scheduler_h.step()
lr_scheduler_h_adv.step()
# measure accuracy and record loss
_, avg_acc_s, cnt_s, pred_s = accuracy(y_s.detach().cpu().numpy(),
label_s.detach().cpu().numpy())
acc_s.update(avg_acc_s, cnt_s)
_, avg_acc_t, cnt_t, pred_t = accuracy(y_t.detach().cpu().numpy(),
label_t.detach().cpu().numpy())
acc_t.update(avg_acc_t, cnt_t)
_, avg_acc_s_adv, cnt_s_adv, pred_s_adv = accuracy(y_s_adv.detach().cpu().numpy(),
label_s.detach().cpu().numpy())
acc_s_adv.update(avg_acc_s_adv, cnt_s)
_, avg_acc_t_adv, cnt_t_adv, pred_t_adv = accuracy(y_t_adv.detach().cpu().numpy(),
label_t.detach().cpu().numpy())
acc_t_adv.update(avg_acc_t_adv, cnt_t)
losses_s.update(loss_s, cnt_s)
losses_gf.update(loss_ground_false, cnt_s)
losses_gt.update(loss_ground_truth, cnt_s)
# measure elapsed time
batch_time.update(time.time() - end)
end = time.time()
if i % args.print_freq == 0:
progress.display(i)
if visualize is not None:
visualize(x_s[0], pred_s[0] * args.image_size / args.heatmap_size, "source_{}_pred".format(i))
visualize(x_s[0], meta_s['keypoint2d'][0], "source_{}_label".format(i))
visualize(x_t[0], pred_t[0] * args.image_size / args.heatmap_size, "target_{}_pred".format(i))
visualize(x_t[0], meta_t['keypoint2d'][0], "target_{}_label".format(i))
visualize(x_s[0], pred_s_adv[0] * args.image_size / args.heatmap_size, "source_adv_{}_pred".format(i))
visualize(x_t[0], pred_t_adv[0] * args.image_size / args.heatmap_size, "target_adv_{}_pred".format(i))
def validate(val_loader, model, criterion, visualize, args: argparse.Namespace):
batch_time = AverageMeter('Time', ':6.3f')
losses = AverageMeter('Loss', ':.2e')
acc = AverageMeterDict(val_loader.dataset.keypoints_group.keys(), ":3.2f")
progress = ProgressMeter(
len(val_loader),
[batch_time, losses, acc['all']],
prefix='Test: ')
# switch to evaluate mode
model.eval()
with torch.no_grad():
end = time.time()
for i, (x, label, weight, meta) in enumerate(val_loader):
x = x.to(device)
label = label.to(device)
weight = weight.to(device)
# compute output
y = model(x)
loss = criterion(y, label, weight)
# measure accuracy and record loss
losses.update(loss.item(), x.size(0))
acc_per_points, avg_acc, cnt, pred = accuracy(y.cpu().numpy(),
label.cpu().numpy())
group_acc = val_loader.dataset.group_accuracy(acc_per_points)
acc.update(group_acc, x.size(0))
# measure elapsed time
batch_time.update(time.time() - end)
end = time.time()
if i % args.print_freq == 0:
progress.display(i)
if visualize is not None:
visualize(x[0], pred[0] * args.image_size / args.heatmap_size, "val_{}_pred.jpg".format(i))
visualize(x[0], meta['keypoint2d'][0], "val_{}_label.jpg".format(i))
return acc.average()
if __name__ == '__main__':
architecture_names = sorted(
name for name in models.__dict__
if name.islower() and not name.startswith("__")
and callable(models.__dict__[name])
)
dataset_names = sorted(
name for name in datasets.__dict__
if not name.startswith("__") and callable(datasets.__dict__[name])
)
parser = argparse.ArgumentParser(description='RegDA (fast) for Keypoint Detection Domain Adaptation')
# dataset parameters
parser.add_argument('source_root', help='root path of the source dataset')
parser.add_argument('target_root', help='root path of the target dataset')
parser.add_argument('-s', '--source', help='source domain(s)')
parser.add_argument('-t', '--target', help='target domain(s)')
parser.add_argument('--resize-scale', nargs='+', type=float, default=(0.6, 1.3),
help='scale range for the RandomResizeCrop augmentation')
parser.add_argument('--rotation', type=int, default=180,
help='rotation range of the RandomRotation augmentation')
parser.add_argument('--image-size', type=int, default=256,
help='input image size')
parser.add_argument('--heatmap-size', type=int, default=64,
help='output heatmap size')
# model parameters
parser.add_argument('-a', '--arch', metavar='ARCH', default='resnet101',
choices=architecture_names,
help='backbone architecture: ' +
' | '.join(architecture_names) +
' (default: resnet101)')
parser.add_argument("--pretrain", type=str, default=None,
help="Where restore pretrained model parameters from.")
parser.add_argument("--resume", type=str, default=None,
help="where restore model parameters from.")
parser.add_argument('--num-head-layers', type=int, default=2)
parser.add_argument('--margin', type=float, default=4., help="margin gamma")
parser.add_argument('--trade-off', default=1., type=float,
help='the trade-off hyper-parameter for transfer loss')
# training parameters
parser.add_argument('-b', '--batch-size', default=32, type=int,
metavar='N',
help='mini-batch size (default: 32)')
parser.add_argument('--lr', '--learning-rate', default=0.1, type=float,
metavar='LR', help='initial learning rate', dest='lr')
parser.add_argument('--momentum', default=0.9, type=float, metavar='M',
help='momentum')
parser.add_argument('--wd', '--weight-decay', default=0.0001, type=float,
metavar='W', help='weight decay (default: 1e-4)')
parser.add_argument('--lr-gamma', default=0.0001, type=float)
parser.add_argument('--lr-decay', default=0.75, type=float, help='parameter for lr scheduler')
parser.add_argument('--lr-step', default=[45, 60], type=tuple, help='parameter for lr scheduler')
parser.add_argument('--lr-factor', default=0.1, type=float, help='parameter for lr scheduler')
parser.add_argument('-j', '--workers', default=4, type=int, metavar='N',
help='number of data loading workers (default: 4)')
parser.add_argument('--pretrain_epochs', default=70, type=int, metavar='N',
help='number of total epochs to run')
parser.add_argument('--epochs', default=30, type=int, metavar='N',
help='number of total epochs to run')
parser.add_argument('-i', '--iters-per-epoch', default=500, type=int,
help='Number of iterations per epoch')
parser.add_argument('-p', '--print-freq', default=100, type=int,
metavar='N', help='print frequency (default: 100)')
parser.add_argument('--seed', default=None, type=int,
help='seed for initializing training. ')
parser.add_argument("--log", type=str, default='regda_fast',
help="Where to save logs, checkpoints and debugging images.")
parser.add_argument("--phase", type=str, default='train', choices=['train', 'test'],
help="When phase is 'test', only test the model.")
parser.add_argument('--debug', action="store_true",
help='In the debug mode, save images and predictions')
args = parser.parse_args()
main(args)
================================================
FILE: domain_generalization/Transfer-Learning-Library/examples/domain_adaptation/keypoint_detection/regda_fast.sh
================================================
# regda_fast is provided by https://github.com/YouJiacheng?tab=repositories
# On single V100(16G), overall adversarial training time is reduced by about 40%.
# yet the PCK might drop 1% for each dataset.
# Hands Dataset
CUDA_VISIBLE_DEVICES=0 python regda_fast.py data/RHD data/H3D_crop \
-s RenderedHandPose -t Hand3DStudio --seed 0 --debug --log logs/regda_fast/rhd2h3d
CUDA_VISIBLE_DEVICES=0 python regda_fast.py data/FreiHand data/RHD \
-s FreiHand -t RenderedHandPose --seed 0 --debug --log logs/regda_fast/freihand2rhd
# Body Dataset
CUDA_VISIBLE_DEVICES=0 python regda_fast.py data/surreal_processed data/Human36M \
-s SURREAL -t Human36M --seed 0 --debug --rotation 30 --epochs 10 --log logs/regda_fast/surreal2human36m
CUDA_VISIBLE_DEVICES=0 python regda_fast.py data/surreal_processed data/lsp \
-s SURREAL -t LSP --seed 0 --debug --rotation 30 --log logs/regda_fast/surreal2lsp
================================================
FILE: domain_generalization/Transfer-Learning-Library/examples/domain_adaptation/keypoint_detection/source_only.py
================================================
"""
@author: Junguang Jiang
@contact: JiangJunguang1123@outlook.com
"""
import random
import time
import warnings
import sys
import argparse
import shutil
import torch
import torch.backends.cudnn as cudnn
from torch.optim import Adam
from torch.optim.lr_scheduler import MultiStepLR
from torch.utils.data import DataLoader
from torchvision.transforms import Compose, ToPILImage
sys.path.append('../../..')
import common.vision.models.keypoint_detection as models
from common.vision.models.keypoint_detection.loss import JointsMSELoss
import common.vision.datasets.keypoint_detection as datasets
import common.vision.transforms.keypoint_detection as T
from common.vision.transforms import Denormalize
from common.utils.data import ForeverDataIterator
from common.utils.meter import AverageMeter, ProgressMeter, AverageMeterDict
from common.utils.metric.keypoint_detection import accuracy
from common.utils.logger import CompleteLogger
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
def main(args: argparse.Namespace):
logger = CompleteLogger(args.log, args.phase)
print(args)
if args.seed is not None:
random.seed(args.seed)
torch.manual_seed(args.seed)
cudnn.deterministic = True
warnings.warn('You have chosen to seed training. '
'This will turn on the CUDNN deterministic setting, '
'which can slow down your training considerably! '
'You may see unexpected behavior when restarting '
'from checkpoints.')
cudnn.benchmark = True
# Data loading code
normalize = T.Normalize([0.485, 0.456, 0.406], [0.229, 0.224, 0.225])
train_transform = T.Compose([
T.RandomRotation(args.rotation),
T.RandomResizedCrop(size=args.image_size, scale=args.resize_scale),
T.ColorJitter(brightness=0.25, contrast=0.25, saturation=0.25),
T.GaussianBlur(),
T.ToTensor(),
normalize
])
val_transform = T.Compose([
T.Resize(args.image_size),
T.ToTensor(),
normalize
])
image_size = (args.image_size, args.image_size)
heatmap_size = (args.heatmap_size, args.heatmap_size)
source_dataset = datasets.__dict__[args.source]
train_source_dataset = source_dataset(root=args.source_root, transforms=train_transform,
image_size=image_size, heatmap_size=heatmap_size)
train_source_loader = DataLoader(train_source_dataset, batch_size=args.batch_size,
shuffle=True, num_workers=args.workers, pin_memory=True, drop_last=True)
val_source_dataset = source_dataset(root=args.source_root, split='test', transforms=val_transform,
image_size=image_size, heatmap_size=heatmap_size)
val_source_loader = DataLoader(val_source_dataset, batch_size=args.batch_size, shuffle=False, pin_memory=True)
target_dataset = datasets.__dict__[args.target]
train_target_dataset = target_dataset(root=args.target_root, transforms=train_transform,
image_size=image_size, heatmap_size=heatmap_size)
train_target_loader = DataLoader(train_target_dataset, batch_size=args.batch_size,
shuffle=True, num_workers=args.workers, pin_memory=True, drop_last=True)
val_target_dataset = target_dataset(root=args.target_root, split='test', transforms=val_transform,
image_size=image_size, heatmap_size=heatmap_size)
val_target_loader = DataLoader(val_target_dataset, batch_size=args.batch_size, shuffle=False, pin_memory=True)
print("Source train:", len(train_source_loader))
print("Target train:", len(train_target_loader))
print("Source test:", len(val_source_loader))
print("Target test:", len(val_target_loader))
train_source_iter = ForeverDataIterator(train_source_loader)
train_target_iter = ForeverDataIterator(train_target_loader)
# create model
model = models.__dict__[args.arch](num_keypoints=train_source_dataset.num_keypoints).to(device)
criterion = JointsMSELoss()
# define optimizer and lr scheduler
optimizer = Adam(model.get_parameters(lr=args.lr))
lr_scheduler = MultiStepLR(optimizer, args.lr_step, args.lr_factor)
# optionally resume from a checkpoint
start_epoch = 0
if args.resume:
checkpoint = torch.load(args.resume, map_location='cpu')
model.load_state_dict(checkpoint['model'])
optimizer.load_state_dict(checkpoint['optimizer'])
lr_scheduler.load_state_dict(checkpoint['lr_scheduler'])
start_epoch = checkpoint['epoch'] + 1
# define visualization function
tensor_to_image = Compose([
Denormalize([0.485, 0.456, 0.406], [0.229, 0.224, 0.225]),
ToPILImage()
])
def visualize(image, keypoint2d, name):
"""
Args:
image (tensor): image in shape 3 x H x W
keypoint2d (tensor): keypoints in shape K x 2
name: name of the saving image
"""
train_source_dataset.visualize(tensor_to_image(image),
keypoint2d, logger.get_image_path("{}.jpg".format(name)))
if args.phase == 'test':
# evaluate on validation set
source_val_acc = validate(val_source_loader, model, criterion, None, args)
target_val_acc = validate(val_target_loader, model, criterion, visualize, args)
print("Source: {:4.3f} Target: {:4.3f}".format(source_val_acc['all'], target_val_acc['all']))
for name, acc in target_val_acc.items():
print("{}: {:4.3f}".format(name, acc))
return
# start training
best_acc = 0
for epoch in range(start_epoch, args.epochs):
logger.set_epoch(epoch)
lr_scheduler.step()
# train for one epoch
train(train_source_iter, train_target_iter, model, criterion, optimizer, epoch,
visualize if args.debug else None, args)
# evaluate on validation set
source_val_acc = validate(val_source_loader, model, criterion, None, args)
target_val_acc = validate(val_target_loader, model, criterion, visualize if args.debug else None, args)
# remember best acc and save checkpoint
torch.save(
{
'model': model.state_dict(),
'optimizer': optimizer.state_dict(),
'lr_scheduler': lr_scheduler.state_dict(),
'epoch': epoch,
'args': args
}, logger.get_checkpoint_path(epoch)
)
if target_val_acc['all'] > best_acc:
shutil.copy(logger.get_checkpoint_path(epoch), logger.get_checkpoint_path('best'))
best_acc = target_val_acc['all']
print("Source: {:4.3f} Target: {:4.3f} Target(best): {:4.3f}".format(source_val_acc['all'], target_val_acc['all'], best_acc))
for name, acc in target_val_acc.items():
print("{}: {:4.3f}".format(name, acc))
logger.close()
def train(train_source_iter, train_target_iter, model, criterion,
optimizer, epoch: int, visualize, args: argparse.Namespace):
batch_time = AverageMeter('Time', ':4.2f')
data_time = AverageMeter('Data', ':3.1f')
losses_s = AverageMeter('Loss (s)', ":.2e")
acc_s = AverageMeter("Acc (s)", ":3.2f")
progress = ProgressMeter(
args.iters_per_epoch,
[batch_time, data_time, losses_s, acc_s],
prefix="Epoch: [{}]".format(epoch))
# switch to train mode
model.train()
end = time.time()
for i in range(args.iters_per_epoch):
optimizer.zero_grad()
x_s, label_s, weight_s, meta_s = next(train_source_iter)
x_s = x_s.to(device)
label_s = label_s.to(device)
weight_s = weight_s.to(device)
# measure data loading time
data_time.update(time.time() - end)
# compute output
y_s = model(x_s)
loss_s = criterion(y_s, label_s, weight_s)
# compute gradient and do SGD step
loss_s.backward()
optimizer.step()
# measure accuracy and record loss
_, avg_acc_s, cnt_s, pred_s = accuracy(y_s.detach().cpu().numpy(),
label_s.detach().cpu().numpy())
acc_s.update(avg_acc_s, cnt_s)
losses_s.update(loss_s, cnt_s)
# measure elapsed time
batch_time.update(time.time() - end)
end = time.time()
if i % args.print_freq == 0:
progress.display(i)
if visualize is not None:
visualize(x_s[0], pred_s[0] * args.image_size / args.heatmap_size, "source_{}_pred.jpg".format(i))
visualize(x_s[0], meta_s['keypoint2d'][0], "source_{}_label.jpg".format(i))
def validate(val_loader, model, criterion, visualize, args: argparse.Namespace):
batch_time = AverageMeter('Time', ':6.3f')
losses = AverageMeter('Loss', ':.2e')
acc = AverageMeterDict(val_loader.dataset.keypoints_group.keys(), ":3.2f")
progress = ProgressMeter(
len(val_loader),
[batch_time, losses, acc['all']],
prefix='Test: ')
# switch to evaluate mode
model.eval()
with torch.no_grad():
end = time.time()
for i, (x, label, weight, meta) in enumerate(val_loader):
x = x.to(device)
label = label.to(device)
weight = weight.to(device)
# compute output
y = model(x)
loss = criterion(y, label, weight)
# measure accuracy and record loss
losses.update(loss.item(), x.size(0))
acc_per_points, avg_acc, cnt, pred = accuracy(y.cpu().numpy(),
label.cpu().numpy())
group_acc = val_loader.dataset.group_accuracy(acc_per_points)
acc.update(group_acc, x.size(0))
# measure elapsed time
batch_time.update(time.time() - end)
end = time.time()
if i % args.print_freq == 0:
progress.display(i)
if visualize is not None:
visualize(x[0], pred[0] * args.image_size / args.heatmap_size, "val_{}_pred.jpg".format(i))
visualize(x[0], meta['keypoint2d'][0], "val_{}_label.jpg".format(i))
return acc.average()
if __name__ == '__main__':
architecture_names = sorted(
name for name in models.__dict__
if name.islower() and not name.startswith("__")
and callable(models.__dict__[name])
)
dataset_names = sorted(
name for name in datasets.__dict__
if not name.startswith("__") and callable(datasets.__dict__[name])
)
parser = argparse.ArgumentParser(description='Source Only for Keypoint Detection Domain Adaptation')
# dataset parameters
parser.add_argument('source_root', help='root path of the source dataset')
parser.add_argument('target_root', help='root path of the target dataset')
parser.add_argument('-s', '--source', help='source domain(s)')
parser.add_argument('-t', '--target', help='target domain(s)')
parser.add_argument('--resize-scale', nargs='+', type=float, default=(0.6, 1.3),
help='scale range for the RandomResizeCrop augmentation')
parser.add_argument('--rotation', type=int, default=180,
help='rotation range of the RandomRotation augmentation')
parser.add_argument('--image-size', type=int, default=256,
help='input image size')
parser.add_argument('--heatmap-size', type=int, default=64,
help='output heatmap size')
# model parameters
parser.add_argument('-a', '--arch', metavar='ARCH', default='pose_resnet101',
choices=architecture_names,
help='backbone architecture: ' +
' | '.join(architecture_names) +
' (default: pose_resnet101)')
parser.add_argument("--resume", type=str, default=None,
help="where restore model parameters from.")
# training parameters
parser.add_argument('-b', '--batch-size', default=32, type=int,
metavar='N',
help='mini-batch size (default: 32)')
parser.add_argument('--lr', '--learning-rate', default=0.001, type=float,
metavar='LR', help='initial learning rate', dest='lr')
parser.add_argument('--lr-step', default=[45, 60], type=tuple, help='parameter for lr scheduler')
parser.add_argument('--lr-factor', default=0.1, type=float, help='parameter for lr scheduler')
parser.add_argument('-j', '--workers', default=4, type=int, metavar='N',
help='number of data loading workers (default: 4)')
parser.add_argument('--epochs', default=70, type=int, metavar='N',
help='number of total epochs to run')
parser.add_argument('-i', '--iters-per-epoch', default=500, type=int,
help='Number of iterations per epoch')
parser.add_argument('-p', '--print-freq', default=100, type=int,
metavar='N', help='print frequency (default: 100)')
parser.add_argument('--seed', default=None, type=int,
help='seed for initializing training. ')
parser.add_argument("--log", type=str, default='src_only',
help="Where to save logs, checkpoints and debugging images.")
parser.add_argument("--phase", type=str, default='train', choices=['train', 'test'],
help="When phase is 'test', only test the model.")
parser.add_argument('--debug', action="store_true",
help='In the debug mode, save images and predictions')
args = parser.parse_args()
main(args)
================================================
FILE: domain_generalization/Transfer-Learning-Library/examples/domain_adaptation/keypoint_detection/source_only.sh
================================================
# Source Only
# Hands Dataset
CUDA_VISIBLE_DEVICES=0 python source_only.py data/RHD data/H3D_crop \
-s RenderedHandPose -t Hand3DStudio --log logs/baseline/rhd2h3d --debug --seed 0
CUDA_VISIBLE_DEVICES=0 python source_only.py data/FreiHand data/RHD \
-s FreiHand -t RenderedHandPose --log logs/baseline/freihand2rhd --debug --seed 0
# Body Dataset
CUDA_VISIBLE_DEVICES=0 python source_only.py data/surreal_processed data/Human36M \
-s SURREAL -t Human36M --log logs/baseline/surreal2human36m --debug --seed 0 --rotation 30
CUDA_VISIBLE_DEVICES=0 python source_only.py data/surreal_processed data/lsp \
-s SURREAL -t LSP --log logs/baseline/surreal2lsp --debug --seed 0 --rotation 30
# Oracle Results
CUDA_VISIBLE_DEVICES=0 python source_only.py data/H3D_crop data/H3D_crop \
-s Hand3DStudio -t Hand3DStudio --log logs/oracle/h3d --debug --seed 0
CUDA_VISIBLE_DEVICES=0 python source_only.py data/Human36M data/Human36M \
-s Human36M -t Human36M --log logs/oracle/human36m --debug --seed 0 --rotation 30
================================================
FILE: domain_generalization/Transfer-Learning-Library/examples/domain_adaptation/openset_domain_adaptation/README.md
================================================
# Open-set Domain Adaptation for Image Classification
## Installation
Example scripts also support all models in [PyTorch-Image-Models](https://github.com/rwightman/pytorch-image-models).
You also need to install timm to use PyTorch-Image-Models.
```
pip install timm
```
## Dataset
Following datasets can be downloaded automatically:
- [Office31](https://www.cc.gatech.edu/~judy/domainadapt/)
- [OfficeHome](https://www.hemanthdv.org/officeHomeDataset.html)
- [VisDA2017](http://ai.bu.edu/visda-2017/)
## Supported Methods
Supported methods include:
- [Open Set Domain Adaptation (OSBP)](https://arxiv.org/abs/1804.10427)
## Experiment and Results
The shell files give the script to reproduce the [benchmarks](/docs/dalib/benchmarks/openset_domain_adaptation.rst) with specified hyper-parameters.
For example, if you want to train DANN on Office31, use the following script
```shell script
# Train a DANN on Office-31 Amazon -> Webcam task using ResNet 50.
# Assume you have put the datasets under the path `data/office-31`,
# or you are glad to download the datasets automatically from the Internet to this path
CUDA_VISIBLE_DEVICES=0 python dann.py data/office31 -d Office31 -s A -t W -a resnet50 --epochs 20 --seed 1 --log logs/dann/Office31_A2W
```
For more information please refer to [Get Started](/docs/get_started/quickstart.rst) for help.
## Citation
If you use these methods in your research, please consider citing.
```
@InProceedings{OSBP,
author = {Saito, Kuniaki and Yamamoto, Shohei and Ushiku, Yoshitaka and Harada, Tatsuya},
title = {Open Set Domain Adaptation by Backpropagation},
booktitle = {ECCV},
year = {2018}
}
```
================================================
FILE: domain_generalization/Transfer-Learning-Library/examples/domain_adaptation/openset_domain_adaptation/dann.py
================================================
"""
@author: Junguang Jiang
@contact: JiangJunguang1123@outlook.com
"""
import random
import time
import warnings
import sys
import argparse
import shutil
import os.path as osp
import torch
import torch.nn as nn
import torch.backends.cudnn as cudnn
from torch.optim import SGD
from torch.optim.lr_scheduler import LambdaLR
from torch.utils.data import DataLoader
import torch.nn.functional as F
sys.path.append('../../..')
from dalib.modules.domain_discriminator import DomainDiscriminator
from common.modules.classifier import Classifier
from dalib.adaptation.dann import DomainAdversarialLoss, ImageClassifier
from common.utils.data import ForeverDataIterator
from common.utils.metric import accuracy, ConfusionMatrix
from common.utils.meter import AverageMeter, ProgressMeter
from common.utils.logger import CompleteLogger
from common.utils.analysis import collect_feature, tsne, a_distance
sys.path.append('.')
import utils
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
def main(args: argparse.Namespace):
logger = CompleteLogger(args.log, args.phase)
print(args)
if args.seed is not None:
random.seed(args.seed)
torch.manual_seed(args.seed)
cudnn.deterministic = True
warnings.warn('You have chosen to seed training. '
'This will turn on the CUDNN deterministic setting, '
'which can slow down your training considerably! '
'You may see unexpected behavior when restarting '
'from checkpoints.')
cudnn.benchmark = True
# Data loading code
train_transform = utils.get_train_transform(args.train_resizing, random_horizontal_flip=True,
random_color_jitter=False)
val_transform = utils.get_val_transform(args.val_resizing)
print("train_transform: ", train_transform)
print("val_transform: ", val_transform)
train_source_dataset, train_target_dataset, val_dataset, test_dataset, num_classes, args.class_names = \
utils.get_dataset(args.data, args.root, args.source, args.target, train_transform, val_transform)
train_source_loader = DataLoader(train_source_dataset, batch_size=args.batch_size,
shuffle=True, num_workers=args.workers, drop_last=True)
train_target_loader = DataLoader(train_target_dataset, batch_size=args.batch_size,
shuffle=True, num_workers=args.workers, drop_last=True)
val_loader = DataLoader(val_dataset, batch_size=args.batch_size, shuffle=False, num_workers=args.workers)
test_loader = DataLoader(test_dataset, batch_size=args.batch_size, shuffle=False, num_workers=args.workers)
train_source_iter = ForeverDataIterator(train_source_loader)
train_target_iter = ForeverDataIterator(train_target_loader)
# create model
print("=> using pre-trained model '{}'".format(args.arch))
backbone = utils.get_model(args.arch)
pool_layer = nn.Identity() if args.no_pool else None
classifier = ImageClassifier(backbone, num_classes, bottleneck_dim=args.bottleneck_dim, pool_layer=pool_layer).to(device)
domain_discri = DomainDiscriminator(in_feature=classifier.features_dim, hidden_size=1024).to(device)
# define optimizer and lr scheduler
optimizer = SGD(classifier.get_parameters() + domain_discri.get_parameters(),
args.lr, momentum=args.momentum, weight_decay=args.weight_decay, nesterov=True)
lr_scheduler = LambdaLR(optimizer, lambda x: args.lr * (1. + args.lr_gamma * float(x)) ** (-args.lr_decay))
# define loss function
domain_adv = DomainAdversarialLoss(domain_discri).to(device)
# analysis the model
if args.phase == 'analysis':
# extract features from both domains
feature_extractor = nn.Sequential(classifier.backbone, classifier.pool_layer, classifier.bottleneck).to(device)
source_feature = collect_feature(train_source_loader, feature_extractor, device)
target_feature = collect_feature(train_target_loader, feature_extractor, device)
# plot t-SNE
tSNE_filename = osp.join(logger.visualize_directory, 'TSNE.png')
tsne.visualize(source_feature, target_feature, tSNE_filename)
print("Saving t-SNE to", tSNE_filename)
# calculate A-distance, which is a measure for distribution discrepancy
A_distance = a_distance.calculate(source_feature, target_feature, device)
print("A-distance =", A_distance)
return
if args.phase == 'test':
acc1 = validate(test_loader, classifier, args)
print(acc1)
return
# start training
best_h_score = 0.
for epoch in range(args.epochs):
# train for one epoch
train(train_source_iter, train_target_iter, classifier, domain_adv, optimizer,
lr_scheduler, epoch, args)
# evaluate on validation set
h_score = validate(val_loader, classifier, args)
# remember best acc@1 and save checkpoint
torch.save(classifier.state_dict(), logger.get_checkpoint_path('latest'))
if h_score > best_h_score:
shutil.copy(logger.get_checkpoint_path('latest'), logger.get_checkpoint_path('best'))
best_h_score = max(h_score, best_h_score)
print("best_h_score = {:3.1f}".format(best_h_score))
# evaluate on test set
classifier.load_state_dict(torch.load(logger.get_checkpoint_path('best')))
h_score = validate(test_loader, classifier, args)
print("test_h_score = {:3.1f}".format(h_score))
logger.close()
def train(train_source_iter: ForeverDataIterator, train_target_iter: ForeverDataIterator,
model: ImageClassifier, domain_adv: DomainAdversarialLoss, optimizer: SGD,
lr_scheduler: LambdaLR, epoch: int, args: argparse.Namespace):
batch_time = AverageMeter('Time', ':5.2f')
data_time = AverageMeter('Data', ':5.2f')
losses = AverageMeter('Loss', ':6.2f')
cls_accs = AverageMeter('Cls Acc', ':3.1f')
tgt_accs = AverageMeter('Tgt Acc', ':3.1f')
domain_accs = AverageMeter('Domain Acc', ':3.1f')
progress = ProgressMeter(
args.iters_per_epoch,
[batch_time, data_time, losses, cls_accs, tgt_accs, domain_accs],
prefix="Epoch: [{}]".format(epoch))
# switch to train mode
model.train()
domain_adv.train()
end = time.time()
for i in range(args.iters_per_epoch):
x_s, labels_s = next(train_source_iter)
x_t, labels_t = next(train_target_iter)
x_s = x_s.to(device)
x_t = x_t.to(device)
labels_s = labels_s.to(device)
labels_t = labels_t.to(device)
# measure data loading time
data_time.update(time.time() - end)
# compute output
x = torch.cat((x_s, x_t), dim=0)
y, f = model(x)
y_s, y_t = y.chunk(2, dim=0)
f_s, f_t = f.chunk(2, dim=0)
cls_loss = F.cross_entropy(y_s, labels_s)
transfer_loss = domain_adv(f_s, f_t)
domain_acc = domain_adv.domain_discriminator_accuracy
loss = cls_loss + transfer_loss * args.trade_off
cls_acc = accuracy(y_s, labels_s)[0]
tgt_acc = accuracy(y_t, labels_t)[0]
losses.update(loss.item(), x_s.size(0))
cls_accs.update(cls_acc.item(), x_s.size(0))
tgt_accs.update(tgt_acc.item(), x_s.size(0))
domain_accs.update(domain_acc.item(), x_s.size(0))
# compute gradient and do SGD step
optimizer.zero_grad()
loss.backward()
optimizer.step()
lr_scheduler.step()
# measure elapsed time
batch_time.update(time.time() - end)
end = time.time()
if i % args.print_freq == 0:
progress.display(i)
def validate(val_loader: DataLoader, model: Classifier, args: argparse.Namespace) -> float:
batch_time = AverageMeter('Time', ':6.3f')
classes = val_loader.dataset.classes
confmat = ConfusionMatrix(len(classes))
progress = ProgressMeter(
len(val_loader),
[batch_time],
prefix='Test: ')
# switch to evaluate mode
model.eval()
with torch.no_grad():
end = time.time()
for i, (images, target) in enumerate(val_loader):
images = images.to(device)
target = target.to(device)
# compute output
output = model(images)
softmax_output = F.softmax(output, dim=1)
softmax_output[:, -1] = args.threshold
# measure accuracy and record loss
confmat.update(target, softmax_output.argmax(1))
# measure elapsed time
batch_time.update(time.time() - end)
end = time.time()
if i % args.print_freq == 0:
progress.display(i)
acc_global, accs, iu = confmat.compute()
all_acc = torch.mean(accs).item() * 100
known = torch.mean(accs[:-1]).item() * 100
unknown = accs[-1].item() * 100
h_score = 2 * known * unknown / (known + unknown)
if args.per_class_eval:
print(confmat.format(classes))
print(' * All {all:.3f} Known {known:.3f} Unknown {unknown:.3f} H-score {h_score:.3f}'
.format(all=all_acc, known=known, unknown=unknown, h_score=h_score))
return h_score
if __name__ == '__main__':
parser = argparse.ArgumentParser(description='DANN for Openset Domain Adaptation')
# dataset parameters
parser.add_argument('root', metavar='DIR',
help='root path of dataset')
parser.add_argument('-d', '--data', metavar='DATA', default='Office31', choices=utils.get_dataset_names(),
help='dataset: ' + ' | '.join(utils.get_dataset_names()) +
' (default: Office31)')
parser.add_argument('-s', '--source', help='source domain')
parser.add_argument('-t', '--target', help='target domain')
parser.add_argument('--train-resizing', type=str, default='default')
parser.add_argument('--val-resizing', type=str, default='default')
# model parameters
parser.add_argument('-a', '--arch', metavar='ARCH', default='resnet18',
choices=utils.get_model_names(),
help='backbone architecture: ' +
' | '.join(utils.get_model_names()) +
' (default: resnet18)')
parser.add_argument('--no-pool', action='store_true',
help='no pool layer after the feature extractor.')
parser.add_argument('--bottleneck-dim', default=256, type=int,
help='Dimension of bottleneck')
parser.add_argument('--threshold', default=0.8, type=float,
help='When class confidence is less than the given threshold, '
'model will output "unknown" (default: 0.5)')
parser.add_argument('--trade-off', default=1., type=float,
help='the trade-off hyper-parameter for transfer loss')
# training parameters
parser.add_argument('-b', '--batch-size', default=32, type=int,
metavar='N',
help='mini-batch size (default: 32)')
parser.add_argument('--lr', '--learning-rate', default=0.002, type=float,
metavar='LR', help='initial learning rate', dest='lr')
parser.add_argument('--lr-gamma', default=0.001, type=float, help='parameter for lr scheduler')
parser.add_argument('--lr-decay', default=0.75, type=float, help='parameter for lr scheduler')
parser.add_argument('--momentum', default=0.9, type=float, metavar='M',
help='momentum')
parser.add_argument('--wd', '--weight-decay',default=1e-3, type=float,
metavar='W', help='weight decay (default: 1e-3)',
dest='weight_decay')
parser.add_argument('-j', '--workers', default=2, type=int, metavar='N',
help='number of data loading workers (default: 4)')
parser.add_argument('--epochs', default=20, type=int, metavar='N',
help='number of total epochs to run')
parser.add_argument('-i', '--iters-per-epoch', default=500, type=int,
help='Number of iterations per epoch')
parser.add_argument('-p', '--print-freq', default=100, type=int,
metavar='N', help='print frequency (default: 100)')
parser.add_argument('--seed', default=None, type=int,
help='seed for initializing training. ')
parser.add_argument('--per-class-eval', action='store_true',
help='whether output per-class accuracy during evaluation')
parser.add_argument("--log", type=str, default='dann',
help="Where to save logs, checkpoints and debugging images.")
parser.add_argument("--phase", type=str, default='train', choices=['train', 'test', 'analysis'],
help="When phase is 'test', only test the model."
"When phase is 'analysis', only analysis the model.")
args = parser.parse_args()
main(args)
================================================
FILE: domain_generalization/Transfer-Learning-Library/examples/domain_adaptation/openset_domain_adaptation/dann.sh
================================================
#!/usr/bin/env bash
# Office31
CUDA_VISIBLE_DEVICES=0 python dann.py data/office31 -d Office31 -s A -t W -a resnet50 --epochs 20 --seed 0 --threshold 0.9 --log logs/dann/Office31_A2W
CUDA_VISIBLE_DEVICES=0 python dann.py data/office31 -d Office31 -s D -t W -a resnet50 --epochs 20 --seed 0 --threshold 0.9 --log logs/dann/Office31_D2W
CUDA_VISIBLE_DEVICES=0 python dann.py data/office31 -d Office31 -s W -t D -a resnet50 --epochs 20 --seed 0 --threshold 0.9 --log logs/dann/Office31_W2D
CUDA_VISIBLE_DEVICES=0 python dann.py data/office31 -d Office31 -s A -t D -a resnet50 --epochs 20 --seed 0 --threshold 0.9 --log logs/dann/Office31_A2D
CUDA_VISIBLE_DEVICES=0 python dann.py data/office31 -d Office31 -s D -t A -a resnet50 --epochs 20 --seed 0 --threshold 0.9 --log logs/dann/Office31_D2A
CUDA_VISIBLE_DEVICES=0 python dann.py data/office31 -d Office31 -s W -t A -a resnet50 --epochs 20 --seed 0 --threshold 0.9 --log logs/dann/Office31_W2A
# Office-Home
CUDA_VISIBLE_DEVICES=0 python dann.py data/office-home -d OfficeHome -s Ar -t Cl -a resnet50 --epochs 30 --seed 0 --log logs/dann/OfficeHome_Ar2Cl
CUDA_VISIBLE_DEVICES=0 python dann.py data/office-home -d OfficeHome -s Ar -t Pr -a resnet50 --epochs 30 --seed 0 --log logs/dann/OfficeHome_Ar2Pr
CUDA_VISIBLE_DEVICES=0 python dann.py data/office-home -d OfficeHome -s Ar -t Rw -a resnet50 --epochs 30 --seed 0 --log logs/dann/OfficeHome_Ar2Rw
CUDA_VISIBLE_DEVICES=0 python dann.py data/office-home -d OfficeHome -s Cl -t Ar -a resnet50 --epochs 30 --seed 0 --log logs/dann/OfficeHome_Cl2Ar
CUDA_VISIBLE_DEVICES=0 python dann.py data/office-home -d OfficeHome -s Cl -t Pr -a resnet50 --epochs 30 --seed 0 --log logs/dann/OfficeHome_Cl2Pr
CUDA_VISIBLE_DEVICES=0 python dann.py data/office-home -d OfficeHome -s Cl -t Rw -a resnet50 --epochs 30 --seed 0 --log logs/dann/OfficeHome_Cl2Rw
CUDA_VISIBLE_DEVICES=0 python dann.py data/office-home -d OfficeHome -s Pr -t Ar -a resnet50 --epochs 30 --seed 0 --log logs/dann/OfficeHome_Pr2Ar
CUDA_VISIBLE_DEVICES=0 python dann.py data/office-home -d OfficeHome -s Pr -t Cl -a resnet50 --epochs 30 --seed 0 --log logs/dann/OfficeHome_Pr2Cl
CUDA_VISIBLE_DEVICES=0 python dann.py data/office-home -d OfficeHome -s Pr -t Rw -a resnet50 --epochs 30 --seed 0 --log logs/dann/OfficeHome_Pr2Rw
CUDA_VISIBLE_DEVICES=0 python dann.py data/office-home -d OfficeHome -s Rw -t Ar -a resnet50 --epochs 30 --seed 0 --log logs/dann/OfficeHome_Rw2Ar
CUDA_VISIBLE_DEVICES=0 python dann.py data/office-home -d OfficeHome -s Rw -t Cl -a resnet50 --epochs 30 --seed 0 --log logs/dann/OfficeHome_Rw2Cl
CUDA_VISIBLE_DEVICES=0 python dann.py data/office-home -d OfficeHome -s Rw -t Pr -a resnet50 --epochs 30 --seed 0 --log logs/dann/OfficeHome_Rw2Pr
# VisDA-2017
CUDA_VISIBLE_DEVICES=0 python dann.py data/visda-2017 -d VisDA2017 -s Synthetic -t Real -a resnet50 \
--epochs 30 --seed 0 --train-resizing cen.crop --per-class-eval --log logs/dann/VisDA2017_S2R
================================================
FILE: domain_generalization/Transfer-Learning-Library/examples/domain_adaptation/openset_domain_adaptation/osbp.py
================================================
"""
@author: Junguang Jiang
@contact: JiangJunguang1123@outlook.com
"""
import random
import time
import warnings
import sys
import argparse
import shutil
import os.path as osp
import torch
import torch.nn as nn
import torch.backends.cudnn as cudnn
from torch.optim import SGD
from torch.optim.lr_scheduler import LambdaLR
from torch.utils.data import DataLoader
import torch.nn.functional as F
sys.path.append('../../..')
from dalib.adaptation.osbp import ImageClassifier as Classifier, UnknownClassBinaryCrossEntropy
from common.utils.data import ForeverDataIterator
from common.utils.metric import accuracy, ConfusionMatrix
from common.utils.meter import AverageMeter, ProgressMeter
from common.utils.logger import CompleteLogger
from common.utils.analysis import collect_feature, tsne, a_distance
sys.path.append('.')
import utils
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
def main(args: argparse.Namespace):
logger = CompleteLogger(args.log, args.phase)
print(args)
if args.seed is not None:
random.seed(args.seed)
torch.manual_seed(args.seed)
cudnn.deterministic = True
warnings.warn('You have chosen to seed training. '
'This will turn on the CUDNN deterministic setting, '
'which can slow down your training considerably! '
'You may see unexpected behavior when restarting '
'from checkpoints.')
cudnn.benchmark = True
# Data loading code
train_transform = utils.get_train_transform(args.train_resizing, random_horizontal_flip=True,
random_color_jitter=False)
val_transform = utils.get_val_transform(args.val_resizing)
print("train_transform: ", train_transform)
print("val_transform: ", val_transform)
train_source_dataset, train_target_dataset, val_dataset, test_dataset, num_classes, args.class_names = \
utils.get_dataset(args.data, args.root, args.source, args.target, train_transform, val_transform)
train_source_loader = DataLoader(train_source_dataset, batch_size=args.batch_size,
shuffle=True, num_workers=args.workers, drop_last=True)
train_target_loader = DataLoader(train_target_dataset, batch_size=args.batch_size,
shuffle=True, num_workers=args.workers, drop_last=True)
val_loader = DataLoader(val_dataset, batch_size=args.batch_size, shuffle=False, num_workers=args.workers)
test_loader = DataLoader(test_dataset, batch_size=args.batch_size, shuffle=False, num_workers=args.workers)
train_source_iter = ForeverDataIterator(train_source_loader)
train_target_iter = ForeverDataIterator(train_target_loader)
# create model
print("=> using pre-trained model '{}'".format(args.arch))
backbone = utils.get_model(args.arch)
pool_layer = nn.Identity() if args.no_pool else None
classifier = Classifier(backbone, num_classes, bottleneck_dim=args.bottleneck_dim, pool_layer=pool_layer).to(device)
print(classifier)
unknown_bce = UnknownClassBinaryCrossEntropy(t=0.5)
# define optimizer and lr scheduler
optimizer = SGD(classifier.get_parameters(), args.lr, momentum=args.momentum, weight_decay=args.wd, nesterov=True)
lr_scheduler = LambdaLR(optimizer, lambda x: args.lr * (1. + args.lr_gamma * float(x)) ** (-args.lr_decay))
# analysis the model
if args.phase == 'analysis':
# extract features from both domains
feature_extractor = nn.Sequential(classifier.backbone, classifier.pool_layer, classifier.bottleneck).to(device)
source_feature = collect_feature(train_source_loader, feature_extractor, device)
target_feature = collect_feature(train_target_loader, feature_extractor, device)
# plot t-SNE
tSNE_filename = osp.join(logger.visualize_directory, 'TSNE.png')
tsne.visualize(source_feature, target_feature, tSNE_filename)
print("Saving t-SNE to", tSNE_filename)
# calculate A-distance, which is a measure for distribution discrepancy
A_distance = a_distance.calculate(source_feature, target_feature, device)
print("A-distance =", A_distance)
return
if args.phase == 'test':
acc1 = validate(test_loader, classifier, args)
print(acc1)
return
# start training
best_h_score = 0.
for epoch in range(args.epochs):
# train for one epoch
train(train_source_iter, train_target_iter, classifier, unknown_bce, optimizer,
lr_scheduler, epoch, args)
# evaluate on validation set
h_score = validate(val_loader, classifier, args)
# remember best acc@1 and save checkpoint
torch.save(classifier.state_dict(), logger.get_checkpoint_path('latest'))
if h_score > best_h_score:
shutil.copy(logger.get_checkpoint_path('latest'), logger.get_checkpoint_path('best'))
best_h_score = max(h_score, best_h_score)
print("best_h_score = {:3.1f}".format(best_h_score))
# evaluate on test set
classifier.load_state_dict(torch.load(logger.get_checkpoint_path('best')))
h_score = validate(test_loader, classifier, args)
print("test_h_score = {:3.1f}".format(h_score))
logger.close()
def train(train_source_iter: ForeverDataIterator, train_target_iter: ForeverDataIterator,model: Classifier, unknown_bce: UnknownClassBinaryCrossEntropy, optimizer: SGD,
lr_scheduler: LambdaLR, epoch: int, args: argparse.Namespace):
batch_time = AverageMeter('Time', ':4.2f')
data_time = AverageMeter('Data', ':3.1f')
losses = AverageMeter('Loss', ':3.2f')
cls_accs = AverageMeter('Cls Acc', ':3.1f')
tgt_accs = AverageMeter('Tgt Acc', ':3.1f')
trans_losses = AverageMeter('Trans Loss', ':3.2f')
progress = ProgressMeter(
args.iters_per_epoch,
[batch_time, data_time, losses, trans_losses, cls_accs, tgt_accs],
prefix="Epoch: [{}]".format(epoch))
# switch to train mode
model.train()
end = time.time()
for i in range(args.iters_per_epoch):
x_s, labels_s = next(train_source_iter)
x_t, labels_t = next(train_target_iter)
x_s = x_s.to(device)
x_t = x_t.to(device)
labels_s = labels_s.to(device)
labels_t = labels_t.to(device)
# measure data loading time
data_time.update(time.time() - end)
# compute output
y_s, _ = model(x_s, grad_reverse=False)
y_t, _ = model(x_t, grad_reverse=True)
cls_loss = F.cross_entropy(y_s, labels_s)
trans_loss = unknown_bce(y_t)
loss = cls_loss + trans_loss
cls_acc = accuracy(y_s, labels_s)[0]
tgt_acc = accuracy(y_t, labels_t)[0]
losses.update(loss.item(), x_s.size(0))
trans_losses.update(trans_loss.item(), x_s.size(0))
cls_accs.update(cls_acc.item(), x_s.size(0))
tgt_accs.update(tgt_acc.item(), x_t.size(0))
# compute gradient and do SGD step
optimizer.zero_grad()
loss.backward()
optimizer.step()
lr_scheduler.step()
# measure elapsed time
batch_time.update(time.time() - end)
end = time.time()
if i % args.print_freq == 0:
progress.display(i)
def validate(val_loader: DataLoader, model: Classifier, args: argparse.Namespace) -> float:
batch_time = AverageMeter('Time', ':6.3f')
classes = val_loader.dataset.classes
confmat = ConfusionMatrix(len(classes))
progress = ProgressMeter(
len(val_loader),
[batch_time],
prefix='Test: ')
# switch to evaluate mode
model.eval()
with torch.no_grad():
end = time.time()
for i, (images, target) in enumerate(val_loader):
images = images.to(device)
target = target.to(device)
# compute output
output = model(images)
# measure accuracy and record loss
confmat.update(target, output.argmax(1))
# measure elapsed time
batch_time.update(time.time() - end)
end = time.time()
if i % args.print_freq == 0:
progress.display(i)
acc_global, accs, iu = confmat.compute()
all_acc = torch.mean(accs).item() * 100
known = torch.mean(accs[:-1]).item() * 100
unknown = accs[-1].item() * 100
h_score = 2 * known * unknown / (known + unknown)
if args.per_class_eval:
print(confmat.format(classes))
print(' * All {all:.3f} Known {known:.3f} Unknown {unknown:.3f} H-score {h_score:.3f}'
.format(all=all_acc, known=known, unknown=unknown, h_score=h_score))
return h_score
if __name__ == '__main__':
parser = argparse.ArgumentParser(description='OSBP for Openset Domain Adaptation')
# dataset parameters
parser.add_argument('root', metavar='DIR',
help='root path of dataset')
parser.add_argument('-d', '--data', metavar='DATA', default='Office31', choices=utils.get_dataset_names(),
help='dataset: ' + ' | '.join(utils.get_dataset_names()) +
' (default: Office31)')
parser.add_argument('-s', '--source', help='source domain')
parser.add_argument('-t', '--target', help='target domain')
parser.add_argument('--train-resizing', type=str, default='default')
parser.add_argument('--val-resizing', type=str, default='default')
# model parameters
parser.add_argument('-a', '--arch', metavar='ARCH', default='resnet18',
choices=utils.get_model_names(),
help='backbone architecture: ' +
' | '.join(utils.get_model_names()) +
' (default: resnet18)')
parser.add_argument('--no-pool', action='store_true',
help='no pool layer after the feature extractor.')
parser.add_argument('--bottleneck-dim', default=256, type=int,
help='Dimension of bottleneck')
# training parameters
parser.add_argument('-b', '--batch-size', default=32, type=int,
metavar='N',
help='mini-batch size (default: 32)')
parser.add_argument('--lr', '--learning-rate', default=0.001, type=float,
metavar='LR', help='initial learning rate', dest='lr')
parser.add_argument('--lr-gamma', default=0.0003, type=float, help='parameter for lr scheduler')
parser.add_argument('--lr-decay', default=0.75, type=float, help='parameter for lr scheduler')
parser.add_argument('--momentum', default=0.9, type=float, metavar='M',
help='momentum')
parser.add_argument('--wd', '--weight-decay', default=0.0005, type=float,
metavar='W', help='weight decay (default: 5e-4)')
parser.add_argument('-j', '--workers', default=2, type=int, metavar='N',
help='number of data loading workers (default: 4)')
parser.add_argument('--epochs', default=20, type=int, metavar='N',
help='number of total epochs to run')
parser.add_argument('-i', '--iters-per-epoch', default=500, type=int,
help='Number of iterations per epoch')
parser.add_argument('-p', '--print-freq', default=100, type=int,
metavar='N', help='print frequency (default: 100)')
parser.add_argument('--seed', default=None, type=int,
help='seed for initializing training. ')
parser.add_argument('--per-class-eval', action='store_true',
help='whether output per-class accuracy during evaluation')
parser.add_argument("--log", type=str, default='osbp',
help="Where to save logs, checkpoints and debugging images.")
parser.add_argument("--phase", type=str, default='train', choices=['train', 'test', 'analysis'],
help="When phase is 'test', only test the model."
"When phase is 'analysis', only analysis the model.")
args = parser.parse_args()
main(args)
================================================
FILE: domain_generalization/Transfer-Learning-Library/examples/domain_adaptation/openset_domain_adaptation/osbp.sh
================================================
#!/usr/bin/env bash
# Office31
CUDA_VISIBLE_DEVICES=0 python osbp.py data/office31 -d Office31 -s A -t W -a resnet50 --epochs 20 --seed 0 --log logs/osbp/Office31_A2W
CUDA_VISIBLE_DEVICES=0 python osbp.py data/office31 -d Office31 -s D -t W -a resnet50 --epochs 20 --seed 0 --log logs/osbp/Office31_D2W
CUDA_VISIBLE_DEVICES=0 python osbp.py data/office31 -d Office31 -s W -t D -a resnet50 --epochs 20 --seed 0 --log logs/osbp/Office31_W2D
CUDA_VISIBLE_DEVICES=0 python osbp.py data/office31 -d Office31 -s A -t D -a resnet50 --epochs 20 --seed 0 --log logs/osbp/Office31_A2D
CUDA_VISIBLE_DEVICES=0 python osbp.py data/office31 -d Office31 -s D -t A -a resnet50 --epochs 20 --seed 0 --log logs/osbp/Office31_D2A
CUDA_VISIBLE_DEVICES=0 python osbp.py data/office31 -d Office31 -s W -t A -a resnet50 --epochs 20 --seed 0 --log logs/osbp/Office31_W2A
# Office-Home
CUDA_VISIBLE_DEVICES=0 python osbp.py data/office-home -d OfficeHome -s Ar -t Cl -a resnet50 --epochs 30 --seed 0 --log logs/osbp/OfficeHome_Ar2Cl
CUDA_VISIBLE_DEVICES=0 python osbp.py data/office-home -d OfficeHome -s Ar -t Pr -a resnet50 --epochs 30 --seed 0 --log logs/osbp/OfficeHome_Ar2Pr
CUDA_VISIBLE_DEVICES=0 python osbp.py data/office-home -d OfficeHome -s Ar -t Rw -a resnet50 --epochs 30 --seed 0 --log logs/osbp/OfficeHome_Ar2Rw
CUDA_VISIBLE_DEVICES=0 python osbp.py data/office-home -d OfficeHome -s Cl -t Ar -a resnet50 --epochs 30 --seed 0 --log logs/osbp/OfficeHome_Cl2Ar
CUDA_VISIBLE_DEVICES=0 python osbp.py data/office-home -d OfficeHome -s Cl -t Pr -a resnet50 --epochs 30 --seed 0 --log logs/osbp/OfficeHome_Cl2Pr
CUDA_VISIBLE_DEVICES=0 python osbp.py data/office-home -d OfficeHome -s Cl -t Rw -a resnet50 --epochs 30 --seed 0 --log logs/osbp/OfficeHome_Cl2Rw
CUDA_VISIBLE_DEVICES=0 python osbp.py data/office-home -d OfficeHome -s Pr -t Ar -a resnet50 --epochs 30 --seed 0 --log logs/osbp/OfficeHome_Pr2Ar
CUDA_VISIBLE_DEVICES=0 python osbp.py data/office-home -d OfficeHome -s Pr -t Cl -a resnet50 --epochs 30 --seed 0 --log logs/osbp/OfficeHome_Pr2Cl
CUDA_VISIBLE_DEVICES=0 python osbp.py data/office-home -d OfficeHome -s Pr -t Rw -a resnet50 --epochs 30 --seed 0 --log logs/osbp/OfficeHome_Pr2Rw
CUDA_VISIBLE_DEVICES=0 python osbp.py data/office-home -d OfficeHome -s Rw -t Ar -a resnet50 --epochs 30 --seed 0 --log logs/osbp/OfficeHome_Rw2Ar
CUDA_VISIBLE_DEVICES=0 python osbp.py data/office-home -d OfficeHome -s Rw -t Cl -a resnet50 --epochs 30 --seed 0 --log logs/osbp/OfficeHome_Rw2Cl
CUDA_VISIBLE_DEVICES=0 python osbp.py data/office-home -d OfficeHome -s Rw -t Pr -a resnet50 --epochs 30 --seed 0 --log logs/osbp/OfficeHome_Rw2Pr
# VisDA-2017
CUDA_VISIBLE_DEVICES=0 python osbp.py data/visda-2017 -d VisDA2017 -s Synthetic -t Real -a resnet50 \
--epochs 30 -i 1000 --seed 0 --train-resizing cen.crop --per-class-eval --log logs/osbp/VisDA2017_S2R
================================================
FILE: domain_generalization/Transfer-Learning-Library/examples/domain_adaptation/openset_domain_adaptation/source_only.py
================================================
"""
@author: Junguang Jiang
@contact: JiangJunguang1123@outlook.com
"""
import random
import time
import warnings
import sys
import argparse
import shutil
import os.path as osp
import torch
import torch.nn as nn
import torch.backends.cudnn as cudnn
from torch.optim import SGD
from torch.optim.lr_scheduler import LambdaLR
from torch.utils.data import DataLoader
import torch.nn.functional as F
sys.path.append('../../..')
from common.modules.classifier import Classifier
from common.utils.data import ForeverDataIterator
from common.utils.metric import accuracy, ConfusionMatrix
from common.utils.meter import AverageMeter, ProgressMeter
from common.utils.logger import CompleteLogger
from common.utils.analysis import collect_feature, tsne, a_distance
sys.path.append('.')
import utils
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
def main(args: argparse.Namespace):
logger = CompleteLogger(args.log, args.phase)
print(args)
if args.seed is not None:
random.seed(args.seed)
torch.manual_seed(args.seed)
cudnn.deterministic = True
warnings.warn('You have chosen to seed training. '
'This will turn on the CUDNN deterministic setting, '
'which can slow down your training considerably! '
'You may see unexpected behavior when restarting '
'from checkpoints.')
cudnn.benchmark = True
# Data loading code
# Data loading code
train_transform = utils.get_train_transform(args.train_resizing, random_horizontal_flip=True, random_color_jitter=False)
val_transform = utils.get_val_transform(args.val_resizing)
print("train_transform: ", train_transform)
print("val_transform: ", val_transform)
train_source_dataset, _, val_dataset, test_dataset, num_classes, args.class_names = \
utils.get_dataset(args.data, args.root, args.source, args.target, train_transform, val_transform)
train_source_loader = DataLoader(train_source_dataset, batch_size=args.batch_size,
shuffle=True, num_workers=args.workers, drop_last=True)
val_loader = DataLoader(val_dataset, batch_size=args.batch_size, shuffle=False, num_workers=args.workers)
test_loader = DataLoader(test_dataset, batch_size=args.batch_size, shuffle=False, num_workers=args.workers)
train_source_iter = ForeverDataIterator(train_source_loader)
# create model
print("=> using pre-trained model '{}'".format(args.arch))
backbone = utils.get_model(args.arch)
pool_layer = nn.Identity() if args.no_pool else None
classifier = Classifier(backbone, num_classes, pool_layer=pool_layer).to(device)
# define optimizer and lr scheduler
optimizer = SGD(classifier.get_parameters(), args.lr, momentum=args.momentum, weight_decay=args.wd, nesterov=True)
lr_scheduler = LambdaLR(optimizer, lambda x: args.lr * (1. + args.lr_gamma * float(x)) ** (-args.lr_decay))
# analysis the model
if args.phase == 'analysis':
# using shuffled val loader
val_loader = DataLoader(val_dataset, batch_size=args.batch_size, shuffle=True, num_workers=args.workers)
# extract features from both domains
feature_extractor = nn.Sequential(classifier.backbone, classifier.pool_layer, classifier.bottleneck).to(device)
source_feature = collect_feature(train_source_loader, feature_extractor, device)
target_feature = collect_feature(val_loader, feature_extractor, device)
# plot t-SNE
tSNE_filename = osp.join(logger.visualize_directory, 'TSNE.png')
tsne.visualize(source_feature, target_feature, tSNE_filename)
print("Saving t-SNE to", tSNE_filename)
# calculate A-distance, which is a measure for distribution discrepancy
A_distance = a_distance.calculate(source_feature, target_feature, device)
print("A-distance =", A_distance)
return
if args.phase == 'test':
acc1 = validate(test_loader, classifier, args)
print(acc1)
return
# start training
best_h_score = 0.
for epoch in range(args.epochs):
# train for one epoch
train(train_source_iter, classifier, optimizer,
lr_scheduler, epoch, args)
# evaluate on validation set
h_score = validate(val_loader, classifier, args)
# remember best acc@1 and save checkpoint
torch.save(classifier.state_dict(), logger.get_checkpoint_path('latest'))
if h_score > best_h_score:
shutil.copy(logger.get_checkpoint_path('latest'), logger.get_checkpoint_path('best'))
best_h_score = max(h_score, best_h_score)
print("best_h_score = {:3.1f}".format(best_h_score))
# evaluate on test set
classifier.load_state_dict(torch.load(logger.get_checkpoint_path('best')))
h_score = validate(test_loader, classifier, args)
print("test_h_score = {:3.1f}".format(h_score))
logger.close()
def train(train_source_iter: ForeverDataIterator, model: Classifier, optimizer: SGD,
lr_scheduler: LambdaLR, epoch: int, args: argparse.Namespace):
batch_time = AverageMeter('Time', ':4.2f')
data_time = AverageMeter('Data', ':3.1f')
losses = AverageMeter('Loss', ':3.2f')
cls_accs = AverageMeter('Cls Acc', ':3.1f')
progress = ProgressMeter(
args.iters_per_epoch,
[batch_time, data_time, losses, cls_accs],
prefix="Epoch: [{}]".format(epoch))
# switch to train mode
model.train()
end = time.time()
for i in range(args.iters_per_epoch):
x_s, labels_s = next(train_source_iter)
x_s = x_s.to(device)
labels_s = labels_s.to(device)
# measure data loading time
data_time.update(time.time() - end)
# compute output
y_s, f_s = model(x_s)
cls_loss = F.cross_entropy(y_s, labels_s)
loss = cls_loss
cls_acc = accuracy(y_s, labels_s)[0]
losses.update(loss.item(), x_s.size(0))
cls_accs.update(cls_acc.item(), x_s.size(0))
# compute gradient and do SGD step
optimizer.zero_grad()
loss.backward()
optimizer.step()
lr_scheduler.step()
# measure elapsed time
batch_time.update(time.time() - end)
end = time.time()
if i % args.print_freq == 0:
progress.display(i)
def validate(val_loader: DataLoader, model: Classifier, args: argparse.Namespace) -> float:
batch_time = AverageMeter('Time', ':6.3f')
classes = val_loader.dataset.classes
confmat = ConfusionMatrix(len(classes))
progress = ProgressMeter(
len(val_loader),
[batch_time],
prefix='Test: ')
# switch to evaluate mode
model.eval()
with torch.no_grad():
end = time.time()
for i, (images, target) in enumerate(val_loader):
images = images.to(device)
target = target.to(device)
# compute output
output = model(images)
softmax_output = F.softmax(output, dim=1)
softmax_output[:, -1] = args.threshold
# measure accuracy and record loss
confmat.update(target, softmax_output.argmax(1))
# measure elapsed time
batch_time.update(time.time() - end)
end = time.time()
if i % args.print_freq == 0:
progress.display(i)
acc_global, accs, iu = confmat.compute()
all_acc = torch.mean(accs).item() * 100
known = torch.mean(accs[:-1]).item() * 100
unknown = accs[-1].item() * 100
h_score = 2 * known * unknown / (known + unknown)
if args.per_class_eval:
print(confmat.format(classes))
print(' * All {all:.3f} Known {known:.3f} Unknown {unknown:.3f} H-score {h_score:.3f}'
.format(all=all_acc, known=known, unknown=unknown, h_score=h_score))
return h_score
if __name__ == '__main__':
parser = argparse.ArgumentParser(description='Source Only for Openset Domain Adaptation')
# dataset parameters
parser.add_argument('root', metavar='DIR',
help='root path of dataset')
parser.add_argument('-d', '--data', metavar='DATA', default='Office31', choices=utils.get_dataset_names(),
help='dataset: ' + ' | '.join(utils.get_dataset_names()) +
' (default: Office31)')
parser.add_argument('-s', '--source', help='source domain')
parser.add_argument('-t', '--target', help='target domain')
parser.add_argument('--train-resizing', type=str, default='default')
parser.add_argument('--val-resizing', type=str, default='default')
# model parameters
parser.add_argument('-a', '--arch', metavar='ARCH', default='resnet18',
choices=utils.get_model_names(),
help='backbone architecture: ' +
' | '.join(utils.get_model_names()) +
' (default: resnet18)')
parser.add_argument('--no-pool', action='store_true',
help='no pool layer after the feature extractor.')
parser.add_argument('--threshold', default=0.8, type=float,
help='When class confidence is less than the given threshold, '
'model will output "unknown" (default: 0.5)')
# training parameters
parser.add_argument('-b', '--batch-size', default=32, type=int,
metavar='N',
help='mini-batch size (default: 32)')
parser.add_argument('--lr', '--learning-rate', default=0.001, type=float,
metavar='LR', help='initial learning rate', dest='lr')
parser.add_argument('--lr-gamma', default=0.0003, type=float, help='parameter for lr scheduler')
parser.add_argument('--lr-decay', default=0.75, type=float, help='parameter for lr scheduler')
parser.add_argument('--momentum', default=0.9, type=float, metavar='M',
help='momentum')
parser.add_argument('--wd', '--weight-decay', default=0.0005, type=float,
metavar='W', help='weight decay (default: 5e-4)')
parser.add_argument('-j', '--workers', default=2, type=int, metavar='N',
help='number of data loading workers (default: 4)')
parser.add_argument('--epochs', default=20, type=int, metavar='N',
help='number of total epochs to run')
parser.add_argument('-i', '--iters-per-epoch', default=500, type=int,
help='Number of iterations per epoch')
parser.add_argument('-p', '--print-freq', default=100, type=int,
metavar='N', help='print frequency (default: 100)')
parser.add_argument('--seed', default=None, type=int,
help='seed for initializing training. ')
parser.add_argument('--per-class-eval', action='store_true',
help='whether output per-class accuracy during evaluation')
parser.add_argument("--log", type=str, default='src_only',
help="Where to save logs, checkpoints and debugging images.")
parser.add_argument("--phase", type=str, default='train', choices=['train', 'test', 'analysis'],
help="When phase is 'test', only test the model."
"When phase is 'analysis', only analysis the model.")
args = parser.parse_args()
main(args)
================================================
FILE: domain_generalization/Transfer-Learning-Library/examples/domain_adaptation/openset_domain_adaptation/source_only.sh
================================================
#!/usr/bin/env bash
# Office31
CUDA_VISIBLE_DEVICES=0 python source_only.py data/office31 -d Office31 -s A -t W -a resnet50 --epochs 20 --seed 0 --log logs/source_only/Office31_A2W
CUDA_VISIBLE_DEVICES=0 python source_only.py data/office31 -d Office31 -s D -t W -a resnet50 --epochs 20 --seed 0 --log logs/source_only/Office31_D2W
CUDA_VISIBLE_DEVICES=0 python source_only.py data/office31 -d Office31 -s W -t D -a resnet50 --epochs 20 --seed 0 --log logs/source_only/Office31_W2D
CUDA_VISIBLE_DEVICES=0 python source_only.py data/office31 -d Office31 -s A -t D -a resnet50 --epochs 20 --seed 0 --log logs/source_only/Office31_A2D
CUDA_VISIBLE_DEVICES=0 python source_only.py data/office31 -d Office31 -s D -t A -a resnet50 --epochs 20 --seed 0 --log logs/source_only/Office31_D2A
CUDA_VISIBLE_DEVICES=0 python source_only.py data/office31 -d Office31 -s W -t A -a resnet50 --epochs 20 --seed 0 --log logs/source_only/Office31_W2A
# Office-Home
CUDA_VISIBLE_DEVICES=0 python source_only.py data/office-home -d OfficeHome -s Ar -t Cl -a resnet50 --epochs 30 --seed 0 --log logs/source_only/OfficeHome_Ar2Cl
CUDA_VISIBLE_DEVICES=0 python source_only.py data/office-home -d OfficeHome -s Ar -t Pr -a resnet50 --epochs 30 --seed 0 --log logs/source_only/OfficeHome_Ar2Pr
CUDA_VISIBLE_DEVICES=0 python source_only.py data/office-home -d OfficeHome -s Ar -t Rw -a resnet50 --epochs 30 --seed 0 --log logs/source_only/OfficeHome_Ar2Rw
CUDA_VISIBLE_DEVICES=0 python source_only.py data/office-home -d OfficeHome -s Cl -t Ar -a resnet50 --epochs 30 --seed 0 --log logs/source_only/OfficeHome_Cl2Ar
CUDA_VISIBLE_DEVICES=0 python source_only.py data/office-home -d OfficeHome -s Cl -t Pr -a resnet50 --epochs 30 --seed 0 --log logs/source_only/OfficeHome_Cl2Pr
CUDA_VISIBLE_DEVICES=0 python source_only.py data/office-home -d OfficeHome -s Cl -t Rw -a resnet50 --epochs 30 --seed 0 --log logs/source_only/OfficeHome_Cl2Rw
CUDA_VISIBLE_DEVICES=0 python source_only.py data/office-home -d OfficeHome -s Pr -t Ar -a resnet50 --epochs 30 --seed 0 --log logs/source_only/OfficeHome_Pr2Ar
CUDA_VISIBLE_DEVICES=0 python source_only.py data/office-home -d OfficeHome -s Pr -t Cl -a resnet50 --epochs 30 --seed 0 --log logs/source_only/OfficeHome_Pr2Cl
CUDA_VISIBLE_DEVICES=0 python source_only.py data/office-home -d OfficeHome -s Pr -t Rw -a resnet50 --epochs 30 --seed 0 --log logs/source_only/OfficeHome_Pr2Rw
CUDA_VISIBLE_DEVICES=0 python source_only.py data/office-home -d OfficeHome -s Rw -t Ar -a resnet50 --epochs 30 --seed 0 --log logs/source_only/OfficeHome_Rw2Ar
CUDA_VISIBLE_DEVICES=0 python source_only.py data/office-home -d OfficeHome -s Rw -t Cl -a resnet50 --epochs 30 --seed 0 --log logs/source_only/OfficeHome_Rw2Cl
CUDA_VISIBLE_DEVICES=0 python source_only.py data/office-home -d OfficeHome -s Rw -t Pr -a resnet50 --epochs 30 --seed 0 --log logs/source_only/OfficeHome_Rw2Pr
# VisDA-2017
CUDA_VISIBLE_DEVICES=0 python source_only.py data/visda-2017 -d VisDA2017 -s Synthetic -t Real -a resnet50 \
--epochs 30 -i 500 --seed 0 --train-resizing cen.crop --per-class-eval --log logs/source_only/VisDA2017_S2R
================================================
FILE: domain_generalization/Transfer-Learning-Library/examples/domain_adaptation/openset_domain_adaptation/utils.py
================================================
import sys
import time
import timm
import torch
import torch.nn as nn
import torch.nn.functional as F
import torchvision.transforms as T
sys.path.append('../../..')
import common.vision.datasets.openset as datasets
from common.vision.datasets.openset import default_open_set as open_set
import common.vision.models as models
from common.vision.transforms import ResizeImage
from common.utils.metric import accuracy, ConfusionMatrix
from common.utils.meter import AverageMeter, ProgressMeter
def get_model_names():
return sorted(
name for name in models.__dict__
if name.islower() and not name.startswith("__")
and callable(models.__dict__[name])
) + timm.list_models()
def get_model(model_name):
if model_name in models.__dict__:
# load models from common.vision.models
backbone = models.__dict__[model_name](pretrained=True)
else:
# load models from pytorch-image-models
backbone = timm.create_model(model_name, pretrained=True)
try:
backbone.out_features = backbone.get_classifier().in_features
backbone.reset_classifier(0, '')
backbone.copy_head = backbone.get_classifier
except:
backbone.out_features = backbone.head.in_features
backbone.head = nn.Identity()
backbone.copy_head = lambda x: x.head
return backbone
def get_dataset_names():
return sorted(
name for name in datasets.__dict__
if not name.startswith("__") and callable(datasets.__dict__[name])
)
def get_dataset(dataset_name, root, source, target, train_source_transform, val_transform, train_target_transform=None):
if train_target_transform is None:
train_target_transform = train_source_transform
# load datasets from common.vision.datasets
dataset = datasets.__dict__[dataset_name]
source_dataset = open_set(dataset, source=True)
target_dataset = open_set(dataset, source=False)
train_source_dataset = source_dataset(root=root, task=source, download=True, transform=train_source_transform)
train_target_dataset = target_dataset(root=root, task=target, download=True, transform=train_target_transform)
val_dataset = target_dataset(root=root, task=target, download=True, transform=val_transform)
if dataset_name == 'DomainNet':
test_dataset = target_dataset(root=root, task=target, split='test', download=True, transform=val_transform)
else:
test_dataset = val_dataset
class_names = train_source_dataset.classes
num_classes = len(class_names)
return train_source_dataset, train_target_dataset, val_dataset, test_dataset, num_classes, class_names
def get_train_transform(resizing='default', random_horizontal_flip=True, random_color_jitter=False):
"""
resizing mode:
- default: resize the image to 256 and take a random resized crop of size 224;
- cen.crop: resize the image to 256 and take the center crop of size 224;
- res: resize the image to 224;
- res.|crop: resize the image to 256 and take a random crop of size 224;
- res.sma|crop: resize the image keeping its aspect ratio such that the
smaller side is 256, then take a random crop of size 224;
– inc.crop: “inception crop” from (Szegedy et al., 2015);
– cif.crop: resize the image to 224, zero-pad it by 28 on each side, then take a random crop of size 224.
"""
if resizing == 'default':
transform = T.Compose([
ResizeImage(256),
T.RandomResizedCrop(224)
])
elif resizing == 'cen.crop':
transform = T.Compose([
ResizeImage(256),
T.CenterCrop(224)
])
elif resizing == 'ran.crop':
transform = T.Compose([
ResizeImage(256),
T.RandomCrop(224)
])
elif resizing == 'res.':
transform = T.Resize(224)
elif resizing == 'res.|crop':
transform = T.Compose([
T.Resize((256, 256)),
T.RandomCrop(224)
])
elif resizing == "res.sma|crop":
transform = T.Compose([
T.Resize(256),
T.RandomCrop(224)
])
elif resizing == 'inc.crop':
transform = T.RandomResizedCrop(224)
elif resizing == 'cif.crop':
transform = T.Compose([
T.Resize((224, 224)),
T.Pad(28),
T.RandomCrop(224),
])
else:
raise NotImplementedError(resizing)
transforms = [transform]
if random_horizontal_flip:
transforms.append(T.RandomHorizontalFlip())
if random_color_jitter:
transforms.append(T.ColorJitter(brightness=0.5, contrast=0.5, saturation=0.5, hue=0.5))
transforms.extend([
T.ToTensor(),
T.Normalize(mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225])
])
return T.Compose(transforms)
def get_val_transform(resizing='default'):
"""
resizing mode:
- default: resize the image to 256 and take the center crop of size 224;
– res.: resize the image to 224
– res.|crop: resize the image such that the smaller side is of size 256 and
then take a central crop of size 224.
"""
if resizing == 'default':
transform = T.Compose([
ResizeImage(256),
T.CenterCrop(224),
])
elif resizing == 'res.':
transform = T.Resize((224, 224))
elif resizing == 'res.|crop':
transform = T.Compose([
T.Resize(256),
T.CenterCrop(224),
])
else:
raise NotImplementedError(resizing)
return T.Compose([
transform,
T.ToTensor(),
T.Normalize(mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225])
])
================================================
FILE: domain_generalization/Transfer-Learning-Library/examples/domain_adaptation/partial_domain_adaptation/README.md
================================================
# Partial Domain Adaptation for Image Classification
## Installation
Example scripts also support all models in [PyTorch-Image-Models](https://github.com/rwightman/pytorch-image-models).
You also need to install timm to use PyTorch-Image-Models.
```
pip install timm
```
## Dataset
Following datasets can be downloaded automatically:
- [Office31](https://www.cc.gatech.edu/~judy/domainadapt/)
- [OfficeHome](https://www.hemanthdv.org/officeHomeDataset.html)
- [VisDA2017](http://ai.bu.edu/visda-2017/)
You need to prepare following datasets manually if you want to use them:
- [ImageNet](https://www.image-net.org/)
and prepare them following [Documentation for ImageNetCaltech](/common/vision/datasets/partial/imagenet_caltech.py) and [CaltechImageNet](/common/vision/datasets/partial/caltech_imagenet.py).
## Supported Methods
Supported methods include:
- [Domain Adversarial Neural Network (DANN)](https://arxiv.org/abs/1505.07818)
- [Partial Adversarial Domain Adaptation (PADA)](https://arxiv.org/abs/1808.04205)
- [Importance Weighted Adversarial Nets (IWAN)](https://arxiv.org/abs/1803.09210)
- [Adaptive Feature Norm (AFN)](https://arxiv.org/pdf/1811.07456v2.pdf)
## Experiment and Results
The shell files give the script to reproduce the [benchmarks](/docs/dalib/benchmarks/partial_domain_adaptation.rst) with specified hyper-parameters.
For example, if you want to train DANN on Office31, use the following script
```shell script
# Train a DANN on Office-31 Amazon -> Webcam task using ResNet 50.
# Assume you have put the datasets under the path `data/office-31`,
# or you are glad to download the datasets automatically from the Internet to this path
CUDA_VISIBLE_DEVICES=0 python dann.py data/office31 -d Office31 -s A -t W -a resnet50 --epochs 20 --seed 1 --log logs/dann/Office31_A2W
```
For more information please refer to [Get Started](/docs/get_started/quickstart.rst) for help.
## Citation
If you use these methods in your research, please consider citing.
```
@inproceedings{DANN,
author = {Ganin, Yaroslav and Lempitsky, Victor},
Booktitle = {ICML},
Title = {Unsupervised domain adaptation by backpropagation},
Year = {2015}
}
@InProceedings{PADA,
author = {Zhangjie Cao and
Lijia Ma and
Mingsheng Long and
Jianmin Wang},
title = {Partial Adversarial Domain Adaptation},
booktitle = {ECCV},
year = {2018}
}
@InProceedings{IWAN,
author = {Jing Zhang and
Zewei Ding and
Wanqing Li and
Philip Ogunbona},
title = {Importance Weighted Adversarial Nets for Partial Domain Adaptation},
booktitle = {CVPR},
year = {2018}
}
@InProceedings{AFN,
author = {Xu, Ruijia and Li, Guanbin and Yang, Jihan and Lin, Liang},
title = {Larger Norm More Transferable: An Adaptive Feature Norm Approach for Unsupervised Domain Adaptation},
booktitle = {ICCV},
year = {2019}
}
```
================================================
FILE: domain_generalization/Transfer-Learning-Library/examples/domain_adaptation/partial_domain_adaptation/afn.py
================================================
"""
@author: Baixu Chen
@contact: cbx_99_hasta@outlook.com
"""
import random
import time
import warnings
import sys
import argparse
import shutil
import os.path as osp
import torch
import torch.nn as nn
import torch.backends.cudnn as cudnn
from torch.optim import SGD
from torch.utils.data import DataLoader
import torch.nn.functional as F
sys.path.append('../../..')
from dalib.adaptation.afn import AdaptiveFeatureNorm, ImageClassifier
from dalib.modules.entropy import entropy
import common.vision.models as models
from common.utils.data import ForeverDataIterator
from common.utils.metric import accuracy
from common.utils.meter import AverageMeter, ProgressMeter
from common.utils.logger import CompleteLogger
from common.utils.analysis import collect_feature, tsne, a_distance
sys.path.append('.')
import utils
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
def main(args: argparse.Namespace):
logger = CompleteLogger(args.log, args.phase)
print(args)
if args.seed is not None:
random.seed(args.seed)
torch.manual_seed(args.seed)
cudnn.deterministic = True
warnings.warn('You have chosen to seed training. '
'This will turn on the CUDNN deterministic setting, '
'which can slow down your training considerably! '
'You may see unexpected behavior when restarting '
'from checkpoints.')
cudnn.benchmark = True
# Data loading code
train_transform = utils.get_train_transform(args.train_resizing, random_horizontal_flip=True,
random_color_jitter=False)
val_transform = utils.get_val_transform(args.val_resizing)
print("train_transform: ", train_transform)
print("val_transform: ", val_transform)
train_source_dataset, train_target_dataset, val_dataset, test_dataset, num_classes, args.class_names = \
utils.get_dataset(args.data, args.root, args.source, args.target, train_transform, val_transform)
train_source_loader = DataLoader(train_source_dataset, batch_size=args.batch_size,
shuffle=True, num_workers=args.workers, drop_last=True)
train_target_loader = DataLoader(train_target_dataset, batch_size=args.batch_size,
shuffle=True, num_workers=args.workers, drop_last=True)
val_loader = DataLoader(val_dataset, batch_size=args.batch_size, shuffle=False, num_workers=args.workers)
test_loader = DataLoader(test_dataset, batch_size=args.batch_size, shuffle=False, num_workers=args.workers)
train_source_iter = ForeverDataIterator(train_source_loader)
train_target_iter = ForeverDataIterator(train_target_loader)
# create model
print("=> using pre-trained model '{}'".format(args.arch))
backbone = utils.get_model(args.arch)
pool_layer = nn.Identity() if args.no_pool else None
backbone = models.__dict__[args.arch](pretrained=True)
classifier = ImageClassifier(backbone, train_source_dataset.num_classes, args.num_blocks,
bottleneck_dim=args.bottleneck_dim, dropout_p=args.dropout_p, pool_layer=pool_layer).to(device)
adaptive_feature_norm = AdaptiveFeatureNorm(args.delta).to(device)
# define optimizer
# the learning rate is fixed according to origin paper
optimizer = SGD(classifier.get_parameters(), args.lr, weight_decay=args.weight_decay)
# resume from the best checkpoint
if args.phase != 'train':
checkpoint = torch.load(logger.get_checkpoint_path('best'), map_location='cpu')
classifier.load_state_dict(checkpoint)
# analysis the model
if args.phase == 'analysis':
# extract features from both domains
feature_extractor = nn.Sequential(classifier.backbone, classifier.pool_layer, classifier.bottleneck).to(device)
source_feature = collect_feature(train_source_loader, feature_extractor, device)
target_feature = collect_feature(train_target_loader, feature_extractor, device)
# plot t-SNE
tSNE_filename = osp.join(logger.visualize_directory, 'TSNE.png')
tsne.visualize(source_feature, target_feature, tSNE_filename)
print("Saving t-SNE to", tSNE_filename)
# calculate A-distance, which is a measure for distribution discrepancy
A_distance = a_distance.calculate(source_feature, target_feature, device)
print("A-distance =", A_distance)
return
if args.phase == 'test':
acc1 = utils.validate(test_loader, classifier, args, device)
print(acc1)
return
# start training
best_acc1 = 0.
for epoch in range(args.epochs):
# train for one epoch
train(train_source_iter, train_target_iter, classifier, adaptive_feature_norm, optimizer, epoch, args)
# evaluate on validation set
acc1 = utils.validate(val_loader, classifier, args, device)
# remember best acc@1 and save checkpoint
torch.save(classifier.state_dict(), logger.get_checkpoint_path('latest'))
if acc1 > best_acc1:
shutil.copy(logger.get_checkpoint_path('latest'), logger.get_checkpoint_path('best'))
best_acc1 = max(acc1, best_acc1)
print("best_acc1 = {:3.1f}".format(best_acc1))
# evaluate on test set
classifier.load_state_dict(torch.load(logger.get_checkpoint_path('best')))
acc1 = utils.validate(test_loader, classifier, args, device)
print("test_acc1 = {:3.1f}".format(acc1))
logger.close()
def train(train_source_iter: ForeverDataIterator, train_target_iter: ForeverDataIterator, model: ImageClassifier,
adaptive_feature_norm: AdaptiveFeatureNorm, optimizer: SGD, epoch: int, args: argparse.Namespace):
batch_time = AverageMeter('Time', ':3.1f')
data_time = AverageMeter('Data', ':3.1f')
cls_losses = AverageMeter('Cls Loss', ':3.2f')
norm_losses = AverageMeter('Norm Loss', ':3.2f')
src_feature_norm = AverageMeter('Source Feature Norm', ':3.2f')
tgt_feature_norm = AverageMeter('Target Feature Norm', ':3.2f')
cls_accs = AverageMeter('Cls Acc', ':3.1f')
tgt_accs = AverageMeter('Tgt Acc', ':3.1f')
progress = ProgressMeter(
args.iters_per_epoch,
[batch_time, data_time, cls_losses, norm_losses, src_feature_norm, tgt_feature_norm, cls_accs, tgt_accs],
prefix="Epoch: [{}]".format(epoch))
# switch to train mode
model.train()
end = time.time()
for i in range(args.iters_per_epoch):
x_s, labels_s = next(train_source_iter)
x_t, labels_t = next(train_target_iter)
x_s = x_s.to(device)
x_t = x_t.to(device)
labels_s = labels_s.to(device)
labels_t = labels_t.to(device)
# measure data loading time
data_time.update(time.time() - end)
# compute output
y_s, f_s = model(x_s)
y_t, f_t = model(x_t)
# classification loss
cls_loss = F.cross_entropy(y_s, labels_s)
# norm loss
norm_loss = adaptive_feature_norm(f_s) + adaptive_feature_norm(f_t)
loss = cls_loss + norm_loss * args.trade_off_norm
# using entropy minimization
if args.trade_off_entropy:
y_t = F.softmax(y_t, dim=1)
entropy_loss = entropy(y_t, reduction='mean')
loss += entropy_loss * args.trade_off_entropy
# compute gradient and do SGD step
optimizer.zero_grad()
loss.backward()
optimizer.step()
# update statistics
cls_acc = accuracy(y_s, labels_s)[0]
tgt_acc = accuracy(y_t, labels_t)[0]
cls_losses.update(cls_loss.item(), x_s.size(0))
norm_losses.update(norm_loss.item(), x_s.size(0))
src_feature_norm.update(f_s.norm(p=2, dim=1).mean().item(), x_s.size(0))
tgt_feature_norm.update(f_t.norm(p=2, dim=1).mean().item(), x_s.size(0))
cls_accs.update(cls_acc.item(), x_s.size(0))
tgt_accs.update(tgt_acc.item(), x_s.size(0))
# measure elapsed time
batch_time.update(time.time() - end)
end = time.time()
if i % args.print_freq == 0:
progress.display(i)
if __name__ == '__main__':
parser = argparse.ArgumentParser(description='AFN for Partial Domain Adaptation')
# dataset parameters
parser.add_argument('root', metavar='DIR',
help='root path of dataset')
parser.add_argument('-d', '--data', metavar='DATA', default='Office31', choices=utils.get_dataset_names(),
help='dataset: ' + ' | '.join(utils.get_dataset_names()) +
' (default: Office31)')
parser.add_argument('-s', '--source', help='source domain')
parser.add_argument('-t', '--target', help='target domain')
parser.add_argument('--train-resizing', type=str, default='ran.crop')
parser.add_argument('--val-resizing', type=str, default='default')
# model parameters
parser.add_argument('-a', '--arch', metavar='ARCH', default='resnet18',
choices=utils.get_model_names(),
help='backbone architecture: ' +
' | '.join(utils.get_model_names()) +
' (default: resnet18)')
parser.add_argument('--no-pool', action='store_true',
help='no pool layer after the feature extractor.')
parser.add_argument('-n', '--num-blocks', default=1, type=int, help='Number of basic blocks for classifier')
parser.add_argument('--bottleneck-dim', default=1000, type=int, help='Dimension of bottleneck')
parser.add_argument('--dropout-p', default=0.5, type=float,
help='Dropout probability')
# training parameters
parser.add_argument('-b', '--batch-size', default=32, type=int,
metavar='N',
help='mini-batch size (default: 32)')
parser.add_argument('--lr', '--learning-rate', default=0.001, type=float,
metavar='LR', help='initial learning rate', dest='lr')
parser.add_argument('--momentum', default=0.9, type=float, metavar='M',
help='momentum')
parser.add_argument('--wd', '--weight-decay', default=5e-4, type=float,
metavar='W', help='weight decay (default: 5e-4)',
dest='weight_decay')
parser.add_argument('--trade-off-norm', default=0.05, type=float,
help='the trade-off hyper-parameter for norm loss')
parser.add_argument('--trade-off-entropy', default=None, type=float,
help='the trade-off hyper-parameter for entropy loss')
parser.add_argument('-r', '--delta', default=1, type=float, help='Increment for L2 norm')
parser.add_argument('-j', '--workers', default=4, type=int, metavar='N',
help='number of data loading workers (default: 4)')
parser.add_argument('--epochs', default=20, type=int, metavar='N',
help='number of total epochs to run')
parser.add_argument('-i', '--iters-per-epoch', default=500, type=int,
help='Number of iterations per epoch')
parser.add_argument('-p', '--print-freq', default=100, type=int,
metavar='N', help='print frequency (default: 100)')
parser.add_argument('--seed', default=None, type=int,
help='seed for initializing training. ')
parser.add_argument('--per-class-eval', action='store_true',
help='whether output per-class accuracy during evaluation')
parser.add_argument("--log", type=str, default='afn',
help="Where to save logs, checkpoints and debugging images.")
parser.add_argument("--phase", type=str, default='train', choices=['train', 'test', 'analysis'],
help="When phase is 'test', only test the model."
"When phase is 'analysis', only analysis the model.")
args = parser.parse_args()
main(args)
================================================
FILE: domain_generalization/Transfer-Learning-Library/examples/domain_adaptation/partial_domain_adaptation/afn.sh
================================================
#!/usr/bin/env bash
# Office31
CUDA_VISIBLE_DEVICES=0 python afn.py data/office31 -d Office31 -s A -t W -a resnet50 --trade-off-entropy 0.1 --epochs 20 --seed 1 --log logs/afn/Office31_A2W
CUDA_VISIBLE_DEVICES=0 python afn.py data/office31 -d Office31 -s D -t W -a resnet50 --trade-off-entropy 0.1 --epochs 20 --seed 1 --log logs/afn/Office31_D2W
CUDA_VISIBLE_DEVICES=0 python afn.py data/office31 -d Office31 -s W -t D -a resnet50 --trade-off-entropy 0.1 --epochs 20 --seed 1 --log logs/afn/Office31_W2D
CUDA_VISIBLE_DEVICES=0 python afn.py data/office31 -d Office31 -s A -t D -a resnet50 --trade-off-entropy 0.1 --epochs 20 --seed 1 --log logs/afn/Office31_A2D
CUDA_VISIBLE_DEVICES=0 python afn.py data/office31 -d Office31 -s D -t A -a resnet50 --trade-off-entropy 0.1 --epochs 20 --seed 1 --log logs/afn/Office31_D2A
CUDA_VISIBLE_DEVICES=0 python afn.py data/office31 -d Office31 -s W -t A -a resnet50 --trade-off-entropy 0.1 --epochs 20 --seed 1 --log logs/afn/Office31_W2A
# Office-Home
CUDA_VISIBLE_DEVICES=0 python afn.py data/office-home -d OfficeHome -s Ar -t Cl -a resnet50 --epochs 30 --seed 0 --log logs/afn/OfficeHome_Ar2Cl
CUDA_VISIBLE_DEVICES=0 python afn.py data/office-home -d OfficeHome -s Ar -t Pr -a resnet50 --epochs 30 --seed 0 --log logs/afn/OfficeHome_Ar2Pr
CUDA_VISIBLE_DEVICES=0 python afn.py data/office-home -d OfficeHome -s Ar -t Rw -a resnet50 --epochs 30 --seed 0 --log logs/afn/OfficeHome_Ar2Rw
CUDA_VISIBLE_DEVICES=0 python afn.py data/office-home -d OfficeHome -s Cl -t Ar -a resnet50 --epochs 30 --seed 0 --log logs/afn/OfficeHome_Cl2Ar
CUDA_VISIBLE_DEVICES=0 python afn.py data/office-home -d OfficeHome -s Cl -t Pr -a resnet50 --epochs 30 --seed 0 --log logs/afn/OfficeHome_Cl2Pr
CUDA_VISIBLE_DEVICES=0 python afn.py data/office-home -d OfficeHome -s Cl -t Rw -a resnet50 --epochs 30 --seed 0 --log logs/afn/OfficeHome_Cl2Rw
CUDA_VISIBLE_DEVICES=0 python afn.py data/office-home -d OfficeHome -s Pr -t Ar -a resnet50 --epochs 30 --seed 0 --log logs/afn/OfficeHome_Pr2Ar
CUDA_VISIBLE_DEVICES=0 python afn.py data/office-home -d OfficeHome -s Pr -t Cl -a resnet50 --epochs 30 --seed 0 --log logs/afn/OfficeHome_Pr2Cl
CUDA_VISIBLE_DEVICES=0 python afn.py data/office-home -d OfficeHome -s Pr -t Rw -a resnet50 --epochs 30 --seed 0 --log logs/afn/OfficeHome_Pr2Rw
CUDA_VISIBLE_DEVICES=0 python afn.py data/office-home -d OfficeHome -s Rw -t Ar -a resnet50 --epochs 30 --seed 0 --log logs/afn/OfficeHome_Rw2Ar
CUDA_VISIBLE_DEVICES=0 python afn.py data/office-home -d OfficeHome -s Rw -t Cl -a resnet50 --epochs 30 --seed 0 --log logs/afn/OfficeHome_Rw2Cl
CUDA_VISIBLE_DEVICES=0 python afn.py data/office-home -d OfficeHome -s Rw -t Pr -a resnet50 --epochs 30 --seed 0 --log logs/afn/OfficeHome_Rw2Pr
# VisDA-2017
CUDA_VISIBLE_DEVICES=0 python afn.py data/visda-2017 -d VisDA2017 -s Synthetic -t Real -a resnet50 -r 0.3 -b 36 \
--epochs 30 -i 1000 --seed 0 --per-class-eval --train-resizing cen.crop --log logs/afn/VisDA2017
================================================
FILE: domain_generalization/Transfer-Learning-Library/examples/domain_adaptation/partial_domain_adaptation/dann.py
================================================
"""
@author: Junguang Jiang
@contact: JiangJunguang1123@outlook.com
"""
import random
import time
import warnings
import sys
import argparse
import shutil
import os.path as osp
import torch
import torch.nn as nn
import torch.backends.cudnn as cudnn
from torch.optim import SGD
from torch.optim.lr_scheduler import LambdaLR
from torch.utils.data import DataLoader
import torch.nn.functional as F
sys.path.append('../../..')
from dalib.modules.domain_discriminator import DomainDiscriminator
from common.modules.classifier import Classifier
from dalib.adaptation.dann import DomainAdversarialLoss, ImageClassifier
from common.utils.data import ForeverDataIterator
from common.utils.metric import accuracy
from common.utils.meter import AverageMeter, ProgressMeter
from common.utils.logger import CompleteLogger
from common.utils.analysis import collect_feature, tsne, a_distance
sys.path.append('.')
import utils
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
def main(args: argparse.Namespace):
logger = CompleteLogger(args.log, args.phase)
print(args)
if args.seed is not None:
random.seed(args.seed)
torch.manual_seed(args.seed)
cudnn.deterministic = True
warnings.warn('You have chosen to seed training. '
'This will turn on the CUDNN deterministic setting, '
'which can slow down your training considerably! '
'You may see unexpected behavior when restarting '
'from checkpoints.')
cudnn.benchmark = True
# Data loading code
train_transform = utils.get_train_transform(args.train_resizing, random_horizontal_flip=True,
random_color_jitter=False)
val_transform = utils.get_val_transform(args.val_resizing)
print("train_transform: ", train_transform)
print("val_transform: ", val_transform)
train_source_dataset, train_target_dataset, val_dataset, test_dataset, num_classes, args.class_names = \
utils.get_dataset(args.data, args.root, args.source, args.target, train_transform, val_transform)
train_source_loader = DataLoader(train_source_dataset, batch_size=args.batch_size,
shuffle=True, num_workers=args.workers, drop_last=True)
train_target_loader = DataLoader(train_target_dataset, batch_size=args.batch_size,
shuffle=True, num_workers=args.workers, drop_last=True)
val_loader = DataLoader(val_dataset, batch_size=args.batch_size, shuffle=False, num_workers=args.workers)
test_loader = DataLoader(test_dataset, batch_size=args.batch_size, shuffle=False, num_workers=args.workers)
train_source_iter = ForeverDataIterator(train_source_loader)
train_target_iter = ForeverDataIterator(train_target_loader)
# create model
print("=> using pre-trained model '{}'".format(args.arch))
backbone = utils.get_model(args.arch)
pool_layer = nn.Identity() if args.no_pool else None
if args.data == 'ImageNetCaltech':
classifier = Classifier(backbone, num_classes, head=backbone.copy_head(), pool_layer=pool_layer).to(device)
else:
classifier = ImageClassifier(backbone, num_classes, args.bottleneck_dim, pool_layer=pool_layer).to(device)
domain_discri = DomainDiscriminator(in_feature=classifier.features_dim, hidden_size=1024).to(device)
# define optimizer and lr scheduler
optimizer = SGD(classifier.get_parameters() + domain_discri.get_parameters(),
args.lr, momentum=args.momentum, weight_decay=args.weight_decay, nesterov=True)
lr_scheduler = LambdaLR(optimizer, lambda x: args.lr * (1. + args.lr_gamma * float(x)) ** (-args.lr_decay))
# define loss function
domain_adv = DomainAdversarialLoss(domain_discri).to(device)
# resume from the best checkpoint
if args.phase != 'train':
checkpoint = torch.load(logger.get_checkpoint_path('best'), map_location='cpu')
classifier.load_state_dict(checkpoint)
# analysis the model
if args.phase == 'analysis':
# extract features from both domains
feature_extractor = nn.Sequential(classifier.backbone, classifier.pool_layer, classifier.bottleneck).to(device)
source_feature = collect_feature(train_source_loader, feature_extractor, device)
target_feature = collect_feature(train_target_loader, feature_extractor, device)
# plot t-SNE
tSNE_filename = osp.join(logger.visualize_directory, 'TSNE.png')
tsne.visualize(source_feature, target_feature, tSNE_filename)
print("Saving t-SNE to", tSNE_filename)
# calculate A-distance, which is a measure for distribution discrepancy
A_distance = a_distance.calculate(source_feature, target_feature, device)
print("A-distance =", A_distance)
return
if args.phase == 'test':
acc1 = utils.validate(test_loader, classifier, args, device)
print(acc1)
return
# start training
best_acc1 = 0.
for epoch in range(args.epochs):
# train for one epoch
train(train_source_iter, train_target_iter, classifier, domain_adv, optimizer,
lr_scheduler, epoch, args)
# evaluate on validation set
acc1 = utils.validate(val_loader, classifier, args, device)
# remember best acc@1 and save checkpoint
torch.save(classifier.state_dict(), logger.get_checkpoint_path('latest'))
if acc1 > best_acc1:
shutil.copy(logger.get_checkpoint_path('latest'), logger.get_checkpoint_path('best'))
best_acc1 = max(acc1, best_acc1)
print("best_acc1 = {:3.1f}".format(best_acc1))
# evaluate on test set
classifier.load_state_dict(torch.load(logger.get_checkpoint_path('best')))
acc1 = utils.validate(test_loader, classifier, args, device)
print("test_acc1 = {:3.1f}".format(acc1))
logger.close()
def train(train_source_iter: ForeverDataIterator, train_target_iter: ForeverDataIterator,
model: ImageClassifier, domain_adv: DomainAdversarialLoss, optimizer: SGD,
lr_scheduler: LambdaLR, epoch: int, args: argparse.Namespace):
batch_time = AverageMeter('Time', ':5.2f')
data_time = AverageMeter('Data', ':5.2f')
losses = AverageMeter('Loss', ':6.2f')
cls_accs = AverageMeter('Cls Acc', ':3.1f')
tgt_accs = AverageMeter('Tgt Acc', ':3.1f')
domain_accs = AverageMeter('Domain Acc', ':3.1f')
progress = ProgressMeter(
args.iters_per_epoch,
[batch_time, data_time, losses, cls_accs, tgt_accs, domain_accs],
prefix="Epoch: [{}]".format(epoch))
# switch to train mode
model.train()
domain_adv.train()
end = time.time()
for i in range(args.iters_per_epoch):
x_s, labels_s = next(train_source_iter)
x_t, labels_t = next(train_target_iter)
x_s = x_s.to(device)
x_t = x_t.to(device)
labels_s = labels_s.to(device)
labels_t = labels_t.to(device)
# measure data loading time
data_time.update(time.time() - end)
# compute output
x = torch.cat((x_s, x_t), dim=0)
y, f = model(x)
y_s, y_t = y.chunk(2, dim=0)
f_s, f_t = f.chunk(2, dim=0)
cls_loss = F.cross_entropy(y_s, labels_s)
transfer_loss = domain_adv(f_s, f_t)
domain_acc = domain_adv.domain_discriminator_accuracy
loss = cls_loss + transfer_loss * args.trade_off
cls_acc = accuracy(y_s, labels_s)[0]
tgt_acc = accuracy(y_t, labels_t)[0]
losses.update(loss.item(), x_s.size(0))
cls_accs.update(cls_acc.item(), x_s.size(0))
tgt_accs.update(tgt_acc.item(), x_s.size(0))
domain_accs.update(domain_acc.item(), x_s.size(0))
# compute gradient and do SGD step
optimizer.zero_grad()
loss.backward()
optimizer.step()
lr_scheduler.step()
# measure elapsed time
batch_time.update(time.time() - end)
end = time.time()
if i % args.print_freq == 0:
progress.display(i)
if __name__ == '__main__':
parser = argparse.ArgumentParser(description='DANN for Partial Domain Adaptation')
# dataset parameters
parser.add_argument('root', metavar='DIR',
help='root path of dataset')
parser.add_argument('-d', '--data', metavar='DATA', default='Office31', choices=utils.get_dataset_names(),
help='dataset: ' + ' | '.join(utils.get_dataset_names()) +
' (default: Office31)')
parser.add_argument('-s', '--source', help='source domain')
parser.add_argument('-t', '--target', help='target domain')
parser.add_argument('--train-resizing', type=str, default='default')
parser.add_argument('--val-resizing', type=str, default='default')
# model parameters
parser.add_argument('-a', '--arch', metavar='ARCH', default='resnet18',
choices=utils.get_model_names(),
help='backbone architecture: ' +
' | '.join(utils.get_model_names()) +
' (default: resnet18)')
parser.add_argument('--no-pool', action='store_true',
help='no pool layer after the feature extractor.')
parser.add_argument('--bottleneck-dim', default=256, type=int,
help='Dimension of bottleneck')
parser.add_argument('--trade-off', default=1., type=float,
help='the trade-off hyper-parameter for transfer loss')
# training parameters
parser.add_argument('-b', '--batch-size', default=36, type=int,
metavar='N',
help='mini-batch size (default: 36)')
parser.add_argument('--lr', '--learning-rate', default=0.002, type=float,
metavar='LR', help='initial learning rate', dest='lr')
parser.add_argument('--lr-gamma', default=0.001, type=float, help='parameter for lr scheduler')
parser.add_argument('--lr-decay', default=0.75, type=float, help='parameter for lr scheduler')
parser.add_argument('--momentum', default=0.9, type=float, metavar='M',
help='momentum')
parser.add_argument('--wd', '--weight-decay',default=1e-3, type=float,
metavar='W', help='weight decay (default: 1e-3)',
dest='weight_decay')
parser.add_argument('-j', '--workers', default=2, type=int, metavar='N',
help='number of data loading workers (default: 4)')
parser.add_argument('--epochs', default=20, type=int, metavar='N',
help='number of total epochs to run')
parser.add_argument('-i', '--iters-per-epoch', default=1000, type=int,
help='Number of iterations per epoch')
parser.add_argument('-p', '--print-freq', default=100, type=int,
metavar='N', help='print frequency (default: 100)')
parser.add_argument('--seed', default=None, type=int,
help='seed for initializing training. ')
parser.add_argument('--per-class-eval', action='store_true',
help='whether output per-class accuracy during evaluation')
parser.add_argument("--log", type=str, default='dann',
help="Where to save logs, checkpoints and debugging images.")
parser.add_argument("--phase", type=str, default='train', choices=['train', 'test', 'analysis'],
help="When phase is 'test', only test the model."
"When phase is 'analysis', only analysis the model.")
args = parser.parse_args()
main(args)
================================================
FILE: domain_generalization/Transfer-Learning-Library/examples/domain_adaptation/partial_domain_adaptation/dann.sh
================================================
#!/usr/bin/env bash
# Office31
CUDA_VISIBLE_DEVICES=0 python dann.py data/office31 -d Office31 -s A -t W -a resnet50 --epochs 5 --seed 0 --log logs/dann/Office31_A2W
CUDA_VISIBLE_DEVICES=0 python dann.py data/office31 -d Office31 -s D -t W -a resnet50 --epochs 5 --seed 0 --log logs/dann/Office31_D2W
CUDA_VISIBLE_DEVICES=0 python dann.py data/office31 -d Office31 -s W -t D -a resnet50 --epochs 5 --seed 0 --log logs/dann/Office31_W2D
CUDA_VISIBLE_DEVICES=0 python dann.py data/office31 -d Office31 -s A -t D -a resnet50 --epochs 5 --seed 0 --log logs/dann/Office31_A2D
CUDA_VISIBLE_DEVICES=0 python dann.py data/office31 -d Office31 -s D -t A -a resnet50 --epochs 5 --seed 0 --log logs/dann/Office31_D2A
CUDA_VISIBLE_DEVICES=0 python dann.py data/office31 -d Office31 -s W -t A -a resnet50 --epochs 5 --seed 0 --log logs/dann/Office31_W2A
# Office-Home
CUDA_VISIBLE_DEVICES=0 python dann.py data/office-home -d OfficeHome -s Ar -t Cl -a resnet50 --epochs 5 --seed 0 --log logs/dann/OfficeHome_Ar2Cl
CUDA_VISIBLE_DEVICES=0 python dann.py data/office-home -d OfficeHome -s Ar -t Pr -a resnet50 --epochs 5 --seed 0 --log logs/dann/OfficeHome_Ar2Pr
CUDA_VISIBLE_DEVICES=0 python dann.py data/office-home -d OfficeHome -s Ar -t Rw -a resnet50 --epochs 5 --seed 0 --log logs/dann/OfficeHome_Ar2Rw
CUDA_VISIBLE_DEVICES=0 python dann.py data/office-home -d OfficeHome -s Cl -t Ar -a resnet50 --epochs 5 --seed 0 --log logs/dann/OfficeHome_Cl2Ar
CUDA_VISIBLE_DEVICES=0 python dann.py data/office-home -d OfficeHome -s Cl -t Pr -a resnet50 --epochs 5 --seed 0 --log logs/dann/OfficeHome_Cl2Pr
CUDA_VISIBLE_DEVICES=0 python dann.py data/office-home -d OfficeHome -s Cl -t Rw -a resnet50 --epochs 5 --seed 0 --log logs/dann/OfficeHome_Cl2Rw
CUDA_VISIBLE_DEVICES=0 python dann.py data/office-home -d OfficeHome -s Pr -t Ar -a resnet50 --epochs 5 --seed 0 --log logs/dann/OfficeHome_Pr2Ar
CUDA_VISIBLE_DEVICES=0 python dann.py data/office-home -d OfficeHome -s Pr -t Cl -a resnet50 --epochs 5 --seed 0 --log logs/dann/OfficeHome_Pr2Cl
CUDA_VISIBLE_DEVICES=0 python dann.py data/office-home -d OfficeHome -s Pr -t Rw -a resnet50 --epochs 5 --seed 0 --log logs/dann/OfficeHome_Pr2Rw
CUDA_VISIBLE_DEVICES=0 python dann.py data/office-home -d OfficeHome -s Rw -t Ar -a resnet50 --epochs 5 --seed 0 --log logs/dann/OfficeHome_Rw2Ar
CUDA_VISIBLE_DEVICES=0 python dann.py data/office-home -d OfficeHome -s Rw -t Cl -a resnet50 --epochs 5 --seed 0 --log logs/dann/OfficeHome_Rw2Cl
CUDA_VISIBLE_DEVICES=0 python dann.py data/office-home -d OfficeHome -s Rw -t Pr -a resnet50 --epochs 5 --seed 0 --log logs/dann/OfficeHome_Rw2Pr
# VisDA-2017
CUDA_VISIBLE_DEVICES=0 python dann.py data/visda-2017 -d VisDA2017 -s Synthetic -t Real -a resnet50 \
--epochs 5 --seed 0 --train-resizing cen.crop --per-class-eval --log logs/dann/VisDA2017_S2R
# ImageNet-Caltech
CUDA_VISIBLE_DEVICES=0 python dann.py data/ImageNetCaltech -d ImageNetCaltech -s I -t C -a resnet50 \
--epochs 5 --seed 0 --log logs/dann/I2C
CUDA_VISIBLE_DEVICES=0 python dann.py data/ImageNetCaltech -d CaltechImageNet -s C -t I -a resnet50 \
--epochs 5 --seed 0 --bottleneck-dim 2048 --log logs/dann/C2I
================================================
FILE: domain_generalization/Transfer-Learning-Library/examples/domain_adaptation/partial_domain_adaptation/iwan.py
================================================
"""
@author: Baixu Chen
@contact: cbx_99_hasta@outlook.com
"""
import random
import time
import warnings
import sys
import argparse
import shutil
import os.path as osp
import torch
import torch.nn as nn
import torch.backends.cudnn as cudnn
from torch.optim import SGD
from torch.optim.lr_scheduler import LambdaLR
from torch.utils.data import DataLoader
import torch.nn.functional as F
sys.path.append('../../..')
from common.modules.classifier import Classifier
from dalib.modules.entropy import entropy
from dalib.modules.domain_discriminator import DomainDiscriminator
from dalib.adaptation.iwan import ImportanceWeightModule, ImageClassifier
from dalib.adaptation.dann import DomainAdversarialLoss
from common.utils.data import ForeverDataIterator
from common.utils.metric import accuracy
from common.utils.meter import AverageMeter, ProgressMeter
from common.utils.logger import CompleteLogger
from common.utils.analysis import collect_feature, tsne, a_distance
sys.path.append('.')
import utils
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
def main(args: argparse.Namespace):
logger = CompleteLogger(args.log, args.phase)
print(args)
if args.seed is not None:
random.seed(args.seed)
torch.manual_seed(args.seed)
cudnn.deterministic = True
warnings.warn('You have chosen to seed training. '
'This will turn on the CUDNN deterministic setting, '
'which can slow down your training considerably! '
'You may see unexpected behavior when restarting '
'from checkpoints.')
cudnn.benchmark = True
# Data loading code
train_transform = utils.get_train_transform(args.train_resizing, random_horizontal_flip=True,
random_color_jitter=False)
val_transform = utils.get_val_transform(args.val_resizing)
print("train_transform: ", train_transform)
print("val_transform: ", val_transform)
train_source_dataset, train_target_dataset, val_dataset, test_dataset, num_classes, args.class_names = \
utils.get_dataset(args.data, args.root, args.source, args.target, train_transform, val_transform)
train_source_loader = DataLoader(train_source_dataset, batch_size=args.batch_size,
shuffle=True, num_workers=args.workers, drop_last=True)
train_target_loader = DataLoader(train_target_dataset, batch_size=args.batch_size,
shuffle=True, num_workers=args.workers, drop_last=True)
val_loader = DataLoader(val_dataset, batch_size=args.batch_size, shuffle=False, num_workers=args.workers)
test_loader = DataLoader(test_dataset, batch_size=args.batch_size, shuffle=False, num_workers=args.workers)
train_source_iter = ForeverDataIterator(train_source_loader)
train_target_iter = ForeverDataIterator(train_target_loader)
# create model
print("=> using pre-trained model '{}'".format(args.arch))
backbone = utils.get_model(args.arch)
pool_layer = nn.Identity() if args.no_pool else None
if args.data == 'ImageNetCaltech':
classifier = Classifier(backbone, num_classes, head=backbone.copy_head(), pool_layer=pool_layer).to(device)
else:
classifier = ImageClassifier(backbone, num_classes, args.bottleneck_dim, pool_layer=pool_layer).to(device)
# define domain classifier D, D_0
D = DomainDiscriminator(in_feature=classifier.features_dim, hidden_size=1024, batch_norm=False).to(device)
D_0 = DomainDiscriminator(in_feature=classifier.features_dim, hidden_size=1024, batch_norm=False).to(device)
# define optimizer and lr scheduler
optimizer = SGD(classifier.get_parameters() + D.get_parameters() + D_0.get_parameters(),
args.lr, momentum=args.momentum, weight_decay=args.weight_decay, nesterov=True)
lr_scheduler = LambdaLR(optimizer, lambda x: args.lr * (1. + args.lr_gamma * float(x)) ** (-args.lr_decay))
# define loss function
domain_adv_D = DomainAdversarialLoss(D).to(device)
domain_adv_D_0 = DomainAdversarialLoss(D_0).to(device)
# define importance weight module
importance_weight_module = ImportanceWeightModule(D, train_target_dataset.partial_classes_idx)
# resume from the best checkpoint
if args.phase != 'train':
checkpoint = torch.load(logger.get_checkpoint_path('best'), map_location='cpu')
classifier.load_state_dict(checkpoint)
# analysis the model
if args.phase == 'analysis':
# extract features from both domains
feature_extractor = nn.Sequential(classifier.backbone, classifier.pool_layer, classifier.bottleneck).to(device)
source_feature = collect_feature(train_source_loader, feature_extractor, device)
target_feature = collect_feature(train_target_loader, feature_extractor, device)
# plot t-SNE
tSNE_filename = osp.join(logger.visualize_directory, 'TSNE.png')
tsne.visualize(source_feature, target_feature, tSNE_filename)
print("Saving t-SNE to", tSNE_filename)
# calculate A-distance, which is a measure for distribution discrepancy
A_distance = a_distance.calculate(source_feature, target_feature, device)
print("A-distance =", A_distance)
return
if args.phase == 'test':
acc1 = utils.validate(test_loader, classifier, args, device)
print(acc1)
return
# start training
best_acc1 = 0.
for epoch in range(args.epochs):
# train for one epoch
train(train_source_iter, train_target_iter, classifier, domain_adv_D, domain_adv_D_0,
importance_weight_module, optimizer, lr_scheduler, epoch, args)
# evaluate on validation set
acc1 = utils.validate(val_loader, classifier, args, device)
# remember best acc@1 and save checkpoint
torch.save(classifier.state_dict(), logger.get_checkpoint_path('latest'))
if acc1 > best_acc1:
shutil.copy(logger.get_checkpoint_path('latest'), logger.get_checkpoint_path('best'))
best_acc1 = max(acc1, best_acc1)
print("best_acc1 = {:3.1f}".format(best_acc1))
# evaluate on test set
classifier.load_state_dict(torch.load(logger.get_checkpoint_path('best')))
acc1 = utils.validate(test_loader, classifier, args, device)
print("test_acc1 = {:3.1f}".format(acc1))
logger.close()
def train(train_source_iter: ForeverDataIterator, train_target_iter: ForeverDataIterator, model: ImageClassifier,
domain_adv_D: DomainAdversarialLoss, domain_adv_D_0: DomainAdversarialLoss,
importance_weight_module, optimizer: SGD, lr_scheduler: LambdaLR, epoch: int, args: argparse.Namespace):
batch_time = AverageMeter('Time', ':5.2f')
data_time = AverageMeter('Data', ':5.2f')
losses = AverageMeter('Loss', ':6.2f')
cls_accs = AverageMeter('Cls Acc', ':3.1f')
tgt_accs = AverageMeter('Tgt Acc', ':3.1f')
domain_accs_D = AverageMeter('Domain Acc for D', ':3.1f')
domain_accs_D_0 = AverageMeter('Domain Acc for D_0', ':3.1f')
partial_classes_weights = AverageMeter('Partial Weight', ':3.2f')
non_partial_classes_weights = AverageMeter('Non-Partial Weight', ':3.2f')
progress = ProgressMeter(
args.iters_per_epoch,
[batch_time, data_time, losses, cls_accs, tgt_accs,
domain_accs_D, domain_accs_D_0, partial_classes_weights, non_partial_classes_weights],
prefix="Epoch: [{}]".format(epoch))
# switch to train mode
model.train()
domain_adv_D.train()
domain_adv_D_0.train()
end = time.time()
for i in range(args.iters_per_epoch):
x_s, labels_s = next(train_source_iter)
x_t, labels_t = next(train_target_iter)
x_s = x_s.to(device)
x_t = x_t.to(device)
labels_s = labels_s.to(device)
labels_t = labels_t.to(device)
# measure data loading time
data_time.update(time.time() - end)
# compute output
x = torch.cat((x_s, x_t), dim=0)
y, f = model(x)
y_s, y_t = y.chunk(2, dim=0)
f_s, f_t = f.chunk(2, dim=0)
# classification loss
cls_loss = F.cross_entropy(y_s, labels_s)
# domain adversarial loss for D
adv_loss_D = domain_adv_D(f_s.detach(), f_t.detach())
# get importance weights
w_s = importance_weight_module.get_importance_weight(f_s)
# domain adversarial loss for D_0
adv_loss_D_0 = domain_adv_D_0(f_s, f_t, w_s=w_s)
# entropy loss
y_t = F.softmax(y_t, dim=1)
entropy_loss = entropy(y_t, reduction='mean')
loss = cls_loss + 1.5 * args.trade_off * adv_loss_D + \
args.trade_off * adv_loss_D_0 + args.gamma * entropy_loss
# compute gradient and do SGD step
optimizer.zero_grad()
loss.backward()
optimizer.step()
lr_scheduler.step()
cls_acc = accuracy(y_s, labels_s)[0]
tgt_acc = accuracy(y_t, labels_t)[0]
losses.update(loss.item(), x_s.size(0))
cls_accs.update(cls_acc.item(), x_s.size(0))
tgt_accs.update(tgt_acc.item(), x_s.size(0))
domain_accs_D.update(domain_adv_D.domain_discriminator_accuracy, x_s.size(0))
domain_accs_D_0.update(domain_adv_D_0.domain_discriminator_accuracy, x_s.size(0))
# debug: output class weight averaged on the partial classes and non-partial classes respectively
partial_class_weight, non_partial_classes_weight = \
importance_weight_module.get_partial_classes_weight(w_s, labels_s)
partial_classes_weights.update(partial_class_weight.item(), x_s.size(0))
non_partial_classes_weights.update(non_partial_classes_weight.item(), x_s.size(0))
batch_time.update(time.time() - end)
end = time.time()
if i % args.print_freq == 0:
progress.display(i)
if __name__ == '__main__':
parser = argparse.ArgumentParser(description='IWAN for Partial Domain Adaptation')
# dataset parameters
parser.add_argument('root', metavar='DIR',
help='root path of source (and target) dataset')
parser.add_argument('-d', '--data', metavar='DATA', default='Office31', choices=utils.get_dataset_names(),
help='dataset: ' + ' | '.join(utils.get_dataset_names()) +
' (default: Office31)')
parser.add_argument('-s', '--source', help='source domain')
parser.add_argument('-t', '--target', help='target domain')
parser.add_argument('--train-resizing', type=str, default='default')
parser.add_argument('--val-resizing', type=str, default='default')
# model parameters
parser.add_argument('-a', '--arch', metavar='ARCH', default='resnet18',
choices=utils.get_model_names(),
help='backbone architecture: ' +
' | '.join(utils.get_model_names()) +
' (default: resnet18)')
parser.add_argument('--no-pool', action='store_true',
help='no pool layer after the feature extractor.')
parser.add_argument('--bottleneck-dim', default=256, type=int,
help='Dimension of bottleneck')
parser.add_argument('--gamma', default=0.1, type=float,
help='the trade-off hyper-parameter for entropy loss(default: 0.1)')
parser.add_argument('--trade-off', default=3, type=float,
help='the trade-off hyper-parameter for transfer loss(default: 3))')
# training parameters
parser.add_argument('-b', '--batch-size', default=36, type=int,
metavar='N',
help='mini-batch size (default: 36)')
parser.add_argument('--lr', '--learning-rate', default=0.001, type=float,
metavar='LR', help='initial learning rate', dest='lr')
parser.add_argument('--lr-gamma', default=0.001, type=float, help='parameter for lr scheduler')
parser.add_argument('--lr-decay', default=0.75, type=float, help='parameter for lr scheduler')
parser.add_argument('--momentum', default=0.9, type=float, metavar='M',
help='momentum')
parser.add_argument('--wd', '--weight-decay', default=1e-3, type=float,
metavar='W', help='weight decay (default: 1e-3)',
dest='weight_decay')
parser.add_argument('-j', '--workers', default=2, type=int, metavar='N',
help='number of data loading workers (default: 2)')
parser.add_argument('--epochs', default=10, type=int, metavar='N',
help='number of total epochs to run')
parser.add_argument('-i', '--iters-per-epoch', default=1000, type=int,
help='Number of iterations per epoch')
parser.add_argument('-p', '--print-freq', default=100, type=int,
metavar='N', help='print frequency (default: 100)')
parser.add_argument('--seed', default=None, type=int,
help='seed for initializing training. ')
parser.add_argument('--per-class-eval', action='store_true',
help='whether output per-class accuracy during evaluation')
parser.add_argument("--log", type=str, default='iwan',
help="Where to save logs, checkpoints and debugging images.")
parser.add_argument("--phase", type=str, default='train', choices=['train', 'test', 'analysis'],
help="When phase is 'test', only test the model."
"When phase is 'analysis', only analysis the model.")
args = parser.parse_args()
main(args)
================================================
FILE: domain_generalization/Transfer-Learning-Library/examples/domain_adaptation/partial_domain_adaptation/iwan.sh
================================================
#!/usr/bin/env bash
# Office31
CUDA_VISIBLE_DEVICES=0 python iwan.py data/office31 -d Office31 -s A -t W -a resnet50 --lr 0.0003 --seed 0 --log logs/iwan/Office31_A2W
CUDA_VISIBLE_DEVICES=0 python iwan.py data/office31 -d Office31 -s D -t W -a resnet50 --lr 0.0003 --seed 0 --log logs/iwan/Office31_D2W
CUDA_VISIBLE_DEVICES=0 python iwan.py data/office31 -d Office31 -s W -t D -a resnet50 --lr 0.0003 --seed 0 --log logs/iwan/Office31_W2D
CUDA_VISIBLE_DEVICES=0 python iwan.py data/office31 -d Office31 -s A -t D -a resnet50 --lr 0.0003 --seed 0 --log logs/iwan/Office31_A2D
CUDA_VISIBLE_DEVICES=0 python iwan.py data/office31 -d Office31 -s D -t A -a resnet50 --lr 0.0003 --seed 0 --log logs/iwan/Office31_D2A
CUDA_VISIBLE_DEVICES=0 python iwan.py data/office31 -d Office31 -s W -t A -a resnet50 --lr 0.0003 --seed 0 --log logs/iwan/Office31_W2A
# Office-Home
CUDA_VISIBLE_DEVICES=0 python iwan.py data/office-home -d OfficeHome -s Ar -t Cl -a resnet50 --epochs 30 --seed 0 --log logs/iwan/OfficeHome_Ar2Cl
CUDA_VISIBLE_DEVICES=0 python iwan.py data/office-home -d OfficeHome -s Ar -t Pr -a resnet50 --epochs 30 --seed 0 --log logs/iwan/OfficeHome_Ar2Pr
CUDA_VISIBLE_DEVICES=0 python iwan.py data/office-home -d OfficeHome -s Ar -t Rw -a resnet50 --epochs 30 --seed 0 --log logs/iwan/OfficeHome_Ar2Rw
CUDA_VISIBLE_DEVICES=0 python iwan.py data/office-home -d OfficeHome -s Cl -t Ar -a resnet50 --epochs 30 --seed 0 --log logs/iwan/OfficeHome_Cl2Ar
CUDA_VISIBLE_DEVICES=0 python iwan.py data/office-home -d OfficeHome -s Cl -t Pr -a resnet50 --epochs 30 --seed 0 --log logs/iwan/OfficeHome_Cl2Pr
CUDA_VISIBLE_DEVICES=0 python iwan.py data/office-home -d OfficeHome -s Cl -t Rw -a resnet50 --epochs 30 --seed 0 --log logs/iwan/OfficeHome_Cl2Rw
CUDA_VISIBLE_DEVICES=0 python iwan.py data/office-home -d OfficeHome -s Pr -t Ar -a resnet50 --epochs 30 --seed 0 --log logs/iwan/OfficeHome_Pr2Ar
CUDA_VISIBLE_DEVICES=0 python iwan.py data/office-home -d OfficeHome -s Pr -t Cl -a resnet50 --epochs 30 --seed 0 --log logs/iwan/OfficeHome_Pr2Cl
CUDA_VISIBLE_DEVICES=0 python iwan.py data/office-home -d OfficeHome -s Pr -t Rw -a resnet50 --epochs 30 --seed 0 --log logs/iwan/OfficeHome_Pr2Rw
CUDA_VISIBLE_DEVICES=0 python iwan.py data/office-home -d OfficeHome -s Rw -t Ar -a resnet50 --epochs 30 --seed 0 --log logs/iwan/OfficeHome_Rw2Ar
CUDA_VISIBLE_DEVICES=0 python iwan.py data/office-home -d OfficeHome -s Rw -t Cl -a resnet50 --epochs 30 --seed 0 --log logs/iwan/OfficeHome_Rw2Cl
CUDA_VISIBLE_DEVICES=0 python iwan.py data/office-home -d OfficeHome -s Rw -t Pr -a resnet50 --epochs 30 --seed 0 --log logs/iwan/OfficeHome_Rw2Pr
# VisDA-2017
CUDA_VISIBLE_DEVICES=0 python iwan.py data/visda-2017 -d VisDA2017 -s Synthetic -t Real -a resnet50 \
--lr 0.0003 --seed 0 --train-resizing cen.crop --per-class-eval --log logs/iwan/VisDA2017_S2R
# ImageNet-Caltech
CUDA_VISIBLE_DEVICES=0 python iwan.py data/ImageNetCaltech -d ImageNetCaltech -s I -t C -a resnet50 \
--seed 0 --log logs/iwan/I2C
CUDA_VISIBLE_DEVICES=0 python iwan.py data/ImageNetCaltech -d CaltechImageNet -s C -t I -a resnet50 \
--seed 0 --bottleneck-dim 2048 --log logs/iwan/C2I
================================================
FILE: domain_generalization/Transfer-Learning-Library/examples/domain_adaptation/partial_domain_adaptation/pada.py
================================================
"""
@author: Junguang Jiang
@contact: JiangJunguang1123@outlook.com
"""
import random
import time
import warnings
import sys
import argparse
import shutil
import os.path as osp
import torch
import torch.nn as nn
import torch.backends.cudnn as cudnn
from torch.optim import SGD
from torch.optim.lr_scheduler import LambdaLR
from torch.utils.data import DataLoader
import torch.nn.functional as F
sys.path.append('../../..')
from dalib.modules.domain_discriminator import DomainDiscriminator
from common.modules.classifier import Classifier
from dalib.adaptation.dann import DomainAdversarialLoss, ImageClassifier
from dalib.adaptation.pada import AutomaticUpdateClassWeightModule
from common.utils.data import ForeverDataIterator
from common.utils.metric import accuracy
from common.utils.meter import AverageMeter, ProgressMeter
from common.utils.logger import CompleteLogger
from common.utils.analysis import collect_feature, tsne, a_distance
sys.path.append('.')
import utils
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
def main(args: argparse.Namespace):
logger = CompleteLogger(args.log, args.phase)
print(args)
if args.seed is not None:
random.seed(args.seed)
torch.manual_seed(args.seed)
cudnn.deterministic = True
warnings.warn('You have chosen to seed training. '
'This will turn on the CUDNN deterministic setting, '
'which can slow down your training considerably! '
'You may see unexpected behavior when restarting '
'from checkpoints.')
cudnn.benchmark = True
# Data loading code
train_transform = utils.get_train_transform(args.train_resizing, random_horizontal_flip=True,
random_color_jitter=False)
val_transform = utils.get_val_transform(args.val_resizing)
print("train_transform: ", train_transform)
print("val_transform: ", val_transform)
train_source_dataset, train_target_dataset, val_dataset, test_dataset, num_classes, args.class_names = \
utils.get_dataset(args.data, args.root, args.source, args.target, train_transform, val_transform)
train_source_loader = DataLoader(train_source_dataset, batch_size=args.batch_size,
shuffle=True, num_workers=args.workers, drop_last=True)
train_target_loader = DataLoader(train_target_dataset, batch_size=args.batch_size,
shuffle=True, num_workers=args.workers, drop_last=True)
val_loader = DataLoader(val_dataset, batch_size=args.batch_size, shuffle=False, num_workers=args.workers)
test_loader = DataLoader(test_dataset, batch_size=args.batch_size, shuffle=False, num_workers=args.workers)
train_source_iter = ForeverDataIterator(train_source_loader)
train_target_iter = ForeverDataIterator(train_target_loader)
# create model
print("=> using pre-trained model '{}'".format(args.arch))
backbone = utils.get_model(args.arch)
pool_layer = nn.Identity() if args.no_pool else None
if args.data == 'ImageNetCaltech':
classifier = Classifier(backbone, num_classes, pool_layer=pool_layer, head=backbone.copy_head()).to(device)
else:
classifier = ImageClassifier(backbone, num_classes, args.bottleneck_dim, pool_layer=pool_layer).to(device)
domain_discri = DomainDiscriminator(in_feature=classifier.features_dim, hidden_size=1024).to(device)
class_weight_module = AutomaticUpdateClassWeightModule(args.class_weight_update_steps, train_target_loader,
classifier, num_classes, device, args.temperature,
train_target_dataset.partial_classes_idx)
# define optimizer and lr scheduler
optimizer = SGD(classifier.get_parameters() + domain_discri.get_parameters(),
args.lr, momentum=args.momentum, weight_decay=args.weight_decay, nesterov=True)
lr_scheduler = LambdaLR(optimizer, lambda x: args.lr * (1. + args.lr_gamma * float(x)) ** (-args.lr_decay))
# define loss function
domain_adv = DomainAdversarialLoss(domain_discri).to(device)
# resume from the best checkpoint
if args.phase != 'train':
checkpoint = torch.load(logger.get_checkpoint_path('best'), map_location='cpu')
classifier.load_state_dict(checkpoint)
# analysis the model
if args.phase == 'analysis':
# extract features from both domains
feature_extractor = nn.Sequential(classifier.backbone, classifier.pool_layer, classifier.bottleneck).to(device)
source_feature = collect_feature(train_source_loader, feature_extractor, device)
target_feature = collect_feature(train_target_loader, feature_extractor, device)
# plot t-SNE
tSNE_filename = osp.join(logger.visualize_directory, 'TSNE.png')
tsne.visualize(source_feature, target_feature, tSNE_filename)
print("Saving t-SNE to", tSNE_filename)
# calculate A-distance, which is a measure for distribution discrepancy
A_distance = a_distance.calculate(source_feature, target_feature, device)
print("A-distance =", A_distance)
return
if args.phase == 'test':
acc1 = utils.validate(test_loader, classifier, args, device)
print(acc1)
return
# start training
best_acc1 = 0.
for epoch in range(args.epochs):
# train for one epoch
train(train_source_iter, train_target_iter, classifier, domain_adv, class_weight_module,
optimizer, lr_scheduler, epoch, args)
# evaluate on validation set
acc1 = utils.validate(val_loader, classifier, args, device)
# remember best acc@1 and save checkpoint
torch.save(classifier.state_dict(), logger.get_checkpoint_path('latest'))
if acc1 > best_acc1:
shutil.copy(logger.get_checkpoint_path('latest'), logger.get_checkpoint_path('best'))
best_acc1 = max(acc1, best_acc1)
print("best_acc1 = {:3.1f}".format(best_acc1))
# evaluate on test set
classifier.load_state_dict(torch.load(logger.get_checkpoint_path('best')))
acc1 = utils.validate(test_loader, classifier, args, device)
print("test_acc1 = {:3.1f}".format(acc1))
logger.close()
def train(train_source_iter: ForeverDataIterator, train_target_iter: ForeverDataIterator, model: ImageClassifier,
domain_adv: DomainAdversarialLoss, class_weight_module: AutomaticUpdateClassWeightModule,
optimizer: SGD, lr_scheduler: LambdaLR, epoch: int, args: argparse.Namespace):
batch_time = AverageMeter('Time', ':5.2f')
data_time = AverageMeter('Data', ':5.2f')
losses = AverageMeter('Loss', ':6.2f')
cls_accs = AverageMeter('Cls Acc', ':3.1f')
domain_accs = AverageMeter('Domain Acc', ':3.1f')
tgt_accs = AverageMeter('Tgt Acc', ':3.1f')
partial_classes_weights = AverageMeter('Partial Weight', ':3.1f')
non_partial_classes_weights = AverageMeter('Non-partial Weight', ':3.1f')
progress = ProgressMeter(
args.iters_per_epoch,
[batch_time, data_time, losses, cls_accs, domain_accs, tgt_accs, partial_classes_weights, non_partial_classes_weights],
prefix="Epoch: [{}]".format(epoch))
# switch to train mode
model.train()
domain_adv.train()
end = time.time()
for i in range(args.iters_per_epoch):
x_s, labels_s = next(train_source_iter)
x_t, labels_t = next(train_target_iter)
x_s = x_s.to(device)
x_t = x_t.to(device)
labels_s = labels_s.to(device)
labels_t = labels_t.to(device)
# measure data loading time
data_time.update(time.time() - end)
# compute output
x = torch.cat((x_s, x_t), dim=0)
y, f = model(x)
y_s, y_t = y.chunk(2, dim=0)
f_s, f_t = f.chunk(2, dim=0)
cls_loss = F.cross_entropy(y_s, labels_s, class_weight_module.get_class_weight_for_cross_entropy_loss())
w_s, w_t = class_weight_module.get_class_weight_for_adversarial_loss(labels_s)
transfer_loss = domain_adv(f_s, f_t, w_s, w_t)
class_weight_module.step()
partial_classes_weight, non_partial_classes_weight = class_weight_module.get_partial_classes_weight()
domain_acc = domain_adv.domain_discriminator_accuracy
loss = cls_loss + transfer_loss * args.trade_off
cls_acc = accuracy(y_s, labels_s)[0]
tgt_acc = accuracy(y_t, labels_t)[0]
losses.update(loss.item(), x_s.size(0))
cls_accs.update(cls_acc.item(), x_s.size(0))
domain_accs.update(domain_acc.item(), x_s.size(0))
tgt_accs.update(tgt_acc.item(), x_s.size(0))
partial_classes_weights.update(partial_classes_weight.item(), x_s.size(0))
non_partial_classes_weights.update(non_partial_classes_weight.item(), x_s.size(0))
# compute gradient and do SGD step
optimizer.zero_grad()
loss.backward()
optimizer.step()
lr_scheduler.step()
# measure elapsed time
batch_time.update(time.time() - end)
end = time.time()
if i % args.print_freq == 0:
progress.display(i)
if __name__ == '__main__':
parser = argparse.ArgumentParser(description='PADA for Partial Domain Adaptation')
# dataset parameters
parser.add_argument('root', metavar='DIR',
help='root path of source (and target) dataset')
parser.add_argument('-d', '--data', metavar='DATA', default='Office31', choices=utils.get_dataset_names(),
help='dataset: ' + ' | '.join(utils.get_dataset_names()) +
' (default: Office31)')
parser.add_argument('-s', '--source', help='source domain')
parser.add_argument('-t', '--target', help='target domain')
parser.add_argument('--train-resizing', type=str, default='default')
parser.add_argument('--val-resizing', type=str, default='default')
# model parameters
parser.add_argument('-a', '--arch', metavar='ARCH', default='resnet18',
choices=utils.get_model_names(),
help='backbone architecture: ' +
' | '.join(utils.get_model_names()) +
' (default: resnet18)')
parser.add_argument('--no-pool', action='store_true',
help='no pool layer after the feature extractor.')
parser.add_argument('--bottleneck-dim', default=256, type=int,
help='Dimension of bottleneck')
parser.add_argument('-u', '--class-weight-update-steps', default=500, type=int,
help='Number of steps to update class weight once')
parser.add_argument('--temperature', default=0.1, type=float,
help='temperature for softmax when calculating class weight')
parser.add_argument('--trade-off', default=1., type=float,
help='the trade-off hyper-parameter for transfer loss')
# training parameters
parser.add_argument('-b', '--batch-size', default=36, type=int,
metavar='N',
help='mini-batch size (default: 36)')
parser.add_argument('--lr', '--learning-rate', default=0.002, type=float,
metavar='LR', help='initial learning rate', dest='lr')
parser.add_argument('--lr-gamma', default=0.001, type=float, help='parameter for lr scheduler')
parser.add_argument('--lr-decay', default=0.75, type=float, help='parameter for lr scheduler')
parser.add_argument('--momentum', default=0.9, type=float, metavar='M',
help='momentum')
parser.add_argument('--wd', '--weight-decay',default=1e-3, type=float,
metavar='W', help='weight decay (default: 1e-3)',
dest='weight_decay')
parser.add_argument('-j', '--workers', default=2, type=int, metavar='N',
help='number of data loading workers (default: 4)')
parser.add_argument('--epochs', default=20, type=int, metavar='N',
help='number of total epochs to run')
parser.add_argument('-i', '--iters-per-epoch', default=1000, type=int,
help='Number of iterations per epoch')
parser.add_argument('-p', '--print-freq', default=100, type=int,
metavar='N', help='print frequency (default: 100)')
parser.add_argument('--seed', default=None, type=int,
help='seed for initializing training. ')
parser.add_argument('--per-class-eval', action='store_true',
help='whether output per-class accuracy during evaluation')
parser.add_argument("--log", type=str, default='pada',
help="Where to save logs, checkpoints and debugging images.")
parser.add_argument("--phase", type=str, default='train', choices=['train', 'test', 'analysis'],
help="When phase is 'test', only test the model."
"When phase is 'analysis', only analysis the model.")
args = parser.parse_args()
main(args)
================================================
FILE: domain_generalization/Transfer-Learning-Library/examples/domain_adaptation/partial_domain_adaptation/pada.sh
================================================
#!/usr/bin/env bash
# Office31
CUDA_VISIBLE_DEVICES=0 python pada.py data/office31 -d Office31 -s A -t W -a resnet50 --epochs 20 --seed 0 --log logs/pada/Office31_A2W
CUDA_VISIBLE_DEVICES=0 python pada.py data/office31 -d Office31 -s D -t W -a resnet50 --epochs 20 --seed 0 --log logs/pada/Office31_D2W
CUDA_VISIBLE_DEVICES=0 python pada.py data/office31 -d Office31 -s W -t D -a resnet50 --epochs 20 --seed 0 --log logs/pada/Office31_W2D
CUDA_VISIBLE_DEVICES=0 python pada.py data/office31 -d Office31 -s A -t D -a resnet50 --epochs 20 --seed 0 --log logs/pada/Office31_A2D
CUDA_VISIBLE_DEVICES=0 python pada.py data/office31 -d Office31 -s D -t A -a resnet50 --epochs 20 --seed 0 --log logs/pada/Office31_D2A
CUDA_VISIBLE_DEVICES=0 python pada.py data/office31 -d Office31 -s W -t A -a resnet50 --epochs 20 --seed 0 --log logs/pada/Office31_W2A
# Office-Home
CUDA_VISIBLE_DEVICES=0 python pada.py data/office-home -d OfficeHome -s Ar -t Cl -a resnet50 --epochs 30 --temperature 0.01 --seed 0 --log logs/pada/OfficeHome_Ar2Cl
CUDA_VISIBLE_DEVICES=0 python pada.py data/office-home -d OfficeHome -s Ar -t Pr -a resnet50 --epochs 30 --temperature 0.01 --seed 0 --log logs/pada/OfficeHome_Ar2Pr
CUDA_VISIBLE_DEVICES=0 python pada.py data/office-home -d OfficeHome -s Ar -t Rw -a resnet50 --epochs 30 --temperature 0.01 --seed 0 --log logs/pada/OfficeHome_Ar2Rw
CUDA_VISIBLE_DEVICES=0 python pada.py data/office-home -d OfficeHome -s Cl -t Ar -a resnet50 --epochs 30 --temperature 0.01 --seed 0 --log logs/pada/OfficeHome_Cl2Ar
CUDA_VISIBLE_DEVICES=0 python pada.py data/office-home -d OfficeHome -s Cl -t Pr -a resnet50 --epochs 30 --temperature 0.01 --seed 0 --log logs/pada/OfficeHome_Cl2Pr
CUDA_VISIBLE_DEVICES=0 python pada.py data/office-home -d OfficeHome -s Cl -t Rw -a resnet50 --epochs 30 --temperature 0.01 --seed 0 --log logs/pada/OfficeHome_Cl2Rw
CUDA_VISIBLE_DEVICES=0 python pada.py data/office-home -d OfficeHome -s Pr -t Ar -a resnet50 --epochs 30 --temperature 0.01 --seed 0 --log logs/pada/OfficeHome_Pr2Ar
CUDA_VISIBLE_DEVICES=0 python pada.py data/office-home -d OfficeHome -s Pr -t Cl -a resnet50 --epochs 30 --temperature 0.01 --seed 0 --log logs/pada/OfficeHome_Pr2Cl
CUDA_VISIBLE_DEVICES=0 python pada.py data/office-home -d OfficeHome -s Pr -t Rw -a resnet50 --epochs 30 --temperature 0.01 --seed 0 --log logs/pada/OfficeHome_Pr2Rw
CUDA_VISIBLE_DEVICES=0 python pada.py data/office-home -d OfficeHome -s Rw -t Ar -a resnet50 --epochs 30 --temperature 0.01 --seed 0 --log logs/pada/OfficeHome_Rw2Ar
CUDA_VISIBLE_DEVICES=0 python pada.py data/office-home -d OfficeHome -s Rw -t Cl -a resnet50 --epochs 30 --temperature 0.01 --seed 0 --log logs/pada/OfficeHome_Rw2Cl
CUDA_VISIBLE_DEVICES=0 python pada.py data/office-home -d OfficeHome -s Rw -t Pr -a resnet50 --epochs 30 --temperature 0.01 --seed 0 --log logs/pada/OfficeHome_Rw2Pr
# VisDA-2017
CUDA_VISIBLE_DEVICES=0 python pada.py data/visda-2017 -d VisDA2017 -s Synthetic -t Real -a resnet50 \
--epochs 20 --seed 0 -u 500 -i 500 --train-resizing cen.crop --trade-off 0.4 --per-class-eval --log logs/pada/VisDA2017_S2R
# ImageNet-Caltech
CUDA_VISIBLE_DEVICES=0 python pada.py data/ImageNetCaltech -d ImageNetCaltech -s I -t C -a resnet50 \
--epochs 20 --seed 0 --lr 0.003 --temperature 0.01 -u 2000 -i 2000 --log logs/pada/I2C
CUDA_VISIBLE_DEVICES=0 python pada.py data/ImageNetCaltech -d CaltechImageNet -s C -t I -a resnet50 \
--epochs 20 --seed 0 --lr 0.003 --temperature 0.01 -u 2000 -i 2000 --bottleneck-dim 2048 --log logs/pada/C2I
================================================
FILE: domain_generalization/Transfer-Learning-Library/examples/domain_adaptation/partial_domain_adaptation/requirements.txt
================================================
timm
================================================
FILE: domain_generalization/Transfer-Learning-Library/examples/domain_adaptation/partial_domain_adaptation/source_only.py
================================================
"""
@author: Junguang Jiang
@contact: JiangJunguang1123@outlook.com
"""
import random
import time
import warnings
import sys
import argparse
import shutil
import os.path as osp
import torch
import torch.nn as nn
import torch.backends.cudnn as cudnn
from torch.optim import SGD
from torch.optim.lr_scheduler import LambdaLR
from torch.utils.data import DataLoader
import torch.nn.functional as F
sys.path.append('../../..')
from common.modules.classifier import Classifier
from common.utils.data import ForeverDataIterator
from common.utils.metric import accuracy
from common.utils.meter import AverageMeter, ProgressMeter
from common.utils.logger import CompleteLogger
from common.utils.analysis import collect_feature, tsne, a_distance
sys.path.append('.')
import utils
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
def main(args: argparse.Namespace):
logger = CompleteLogger(args.log, args.phase)
print(args)
if args.seed is not None:
random.seed(args.seed)
torch.manual_seed(args.seed)
cudnn.deterministic = True
warnings.warn('You have chosen to seed training. '
'This will turn on the CUDNN deterministic setting, '
'which can slow down your training considerably! '
'You may see unexpected behavior when restarting '
'from checkpoints.')
cudnn.benchmark = True
# Data loading code
train_transform = utils.get_train_transform(args.train_resizing, random_horizontal_flip=True, random_color_jitter=False)
val_transform = utils.get_val_transform(args.val_resizing)
print("train_transform: ", train_transform)
print("val_transform: ", val_transform)
train_source_dataset, _, val_dataset, test_dataset, num_classes, args.class_names = \
utils.get_dataset(args.data, args.root, args.source, args.target, train_transform, val_transform)
train_source_loader = DataLoader(train_source_dataset, batch_size=args.batch_size,
shuffle=True, num_workers=args.workers, drop_last=True)
val_loader = DataLoader(val_dataset, batch_size=args.batch_size, shuffle=False, num_workers=args.workers)
test_loader = DataLoader(test_dataset, batch_size=args.batch_size, shuffle=False, num_workers=args.workers)
train_source_iter = ForeverDataIterator(train_source_loader)
# create model
print("=> using pre-trained model '{}'".format(args.arch))
backbone = utils.get_model(args.arch)
pool_layer = nn.Identity() if args.no_pool else None
head = backbone.copy_head() if args.data == 'ImageNetCaltech' else None
classifier = Classifier(backbone, num_classes, pool_layer=pool_layer, head=head).to(device)
# define optimizer and lr scheduler
optimizer = SGD(classifier.get_parameters(), args.lr, momentum=args.momentum, weight_decay=args.wd, nesterov=True)
lr_scheduler = LambdaLR(optimizer, lambda x: args.lr * (1. + args.lr_gamma * float(x)) ** (-args.lr_decay))
# resume from the best checkpoint
if args.phase != 'train':
checkpoint = torch.load(logger.get_checkpoint_path('best'), map_location='cpu')
classifier.load_state_dict(checkpoint)
# analysis the model
if args.phase == 'analysis':
# using shuffled val loader
val_loader = DataLoader(val_dataset, batch_size=args.batch_size, shuffle=True, num_workers=args.workers)
# extract features from both domains
feature_extractor = nn.Sequential(classifier.backbone, classifier.pool_layer, classifier.bottleneck).to(device)
source_feature = collect_feature(train_source_loader, feature_extractor, device)
target_feature = collect_feature(val_loader, feature_extractor, device)
# plot t-SNE
tSNE_filename = osp.join(logger.visualize_directory, 'TSNE.png')
tsne.visualize(source_feature, target_feature, tSNE_filename)
print("Saving t-SNE to", tSNE_filename)
# calculate A-distance, which is a measure for distribution discrepancy
A_distance = a_distance.calculate(source_feature, target_feature, device)
print("A-distance =", A_distance)
return
if args.phase == 'test':
acc1 = utils.validate(test_loader, classifier, args, device)
print(acc1)
return
# start training
best_acc1 = 0.
for epoch in range(args.epochs):
# train for one epoch
train(train_source_iter, classifier, optimizer,
lr_scheduler, epoch, args)
# evaluate on validation set
acc1 = utils.validate(val_loader, classifier, args, device)
# remember best acc@1 and save checkpoint
torch.save(classifier.state_dict(), logger.get_checkpoint_path('latest'))
if acc1 > best_acc1:
shutil.copy(logger.get_checkpoint_path('latest'), logger.get_checkpoint_path('best'))
best_acc1 = max(acc1, best_acc1)
print("best_acc1 = {:3.1f}".format(best_acc1))
# evaluate on test set
classifier.load_state_dict(torch.load(logger.get_checkpoint_path('best')))
acc1 = utils.validate(test_loader, classifier, args, device)
print("test_acc1 = {:3.1f}".format(acc1))
logger.close()
def train(train_source_iter: ForeverDataIterator, model: Classifier, optimizer: SGD,
lr_scheduler: LambdaLR, epoch: int, args: argparse.Namespace):
batch_time = AverageMeter('Time', ':4.2f')
data_time = AverageMeter('Data', ':3.1f')
losses = AverageMeter('Loss', ':3.2f')
cls_accs = AverageMeter('Cls Acc', ':3.1f')
progress = ProgressMeter(
args.iters_per_epoch,
[batch_time, data_time, losses, cls_accs],
prefix="Epoch: [{}]".format(epoch))
# switch to train mode
model.train()
end = time.time()
for i in range(args.iters_per_epoch):
x_s, labels_s = next(train_source_iter)
x_s = x_s.to(device)
labels_s = labels_s.to(device)
# measure data loading time
data_time.update(time.time() - end)
# compute output
y_s, f_s = model(x_s)
cls_loss = F.cross_entropy(y_s, labels_s)
loss = cls_loss
cls_acc = accuracy(y_s, labels_s)[0]
losses.update(loss.item(), x_s.size(0))
cls_accs.update(cls_acc.item(), x_s.size(0))
# compute gradient and do SGD step
optimizer.zero_grad()
loss.backward()
optimizer.step()
lr_scheduler.step()
# measure elapsed time
batch_time.update(time.time() - end)
end = time.time()
if i % args.print_freq == 0:
progress.display(i)
if __name__ == '__main__':
parser = argparse.ArgumentParser(description='Source Only for Partial Domain Adaptation')
# dataset parameters
parser.add_argument('root', metavar='DIR',
help='root path of dataset')
parser.add_argument('-d', '--data', metavar='DATA', default='Office31', choices=utils.get_dataset_names(),
help='dataset: ' + ' | '.join(utils.get_dataset_names()) +
' (default: Office31)')
parser.add_argument('-s', '--source', help='source domain')
parser.add_argument('-t', '--target', help='target domain')
parser.add_argument('--train-resizing', type=str, default='default')
parser.add_argument('--val-resizing', type=str, default='default')
# model parameters
parser.add_argument('-a', '--arch', metavar='ARCH', default='resnet18',
choices=utils.get_model_names(),
help='backbone architecture: ' +
' | '.join(utils.get_model_names()) +
' (default: resnet18)')
parser.add_argument('--no-pool', action='store_true',
help='no pool layer after the feature extractor.')
# training parameters
parser.add_argument('-b', '--batch-size', default=36, type=int,
metavar='N',
help='mini-batch size (default: 36)')
parser.add_argument('--lr', '--learning-rate', default=0.001, type=float,
metavar='LR', help='initial learning rate', dest='lr')
parser.add_argument('--lr-gamma', default=0.0003, type=float, help='parameter for lr scheduler')
parser.add_argument('--lr-decay', default=0.75, type=float, help='parameter for lr scheduler')
parser.add_argument('--momentum', default=0.9, type=float, metavar='M',
help='momentum')
parser.add_argument('--wd', '--weight-decay', default=0.0005, type=float,
metavar='W', help='weight decay (default: 5e-4)')
parser.add_argument('-j', '--workers', default=2, type=int, metavar='N',
help='number of data loading workers (default: 4)')
parser.add_argument('--epochs', default=20, type=int, metavar='N',
help='number of total epochs to run')
parser.add_argument('-i', '--iters-per-epoch', default=500, type=int,
help='Number of iterations per epoch')
parser.add_argument('-p', '--print-freq', default=100, type=int,
metavar='N', help='print frequency (default: 100)')
parser.add_argument('--seed', default=None, type=int,
help='seed for initializing training. ')
parser.add_argument('--per-class-eval', action='store_true',
help='whether output per-class accuracy during evaluation')
parser.add_argument("--log", type=str, default='src_only',
help="Where to save logs, checkpoints and debugging images.")
parser.add_argument("--phase", type=str, default='train', choices=['train', 'test', 'analysis'],
help="When phase is 'test', only test the model."
"When phase is 'analysis', only analysis the model.")
args = parser.parse_args()
main(args)
================================================
FILE: domain_generalization/Transfer-Learning-Library/examples/domain_adaptation/partial_domain_adaptation/source_only.sh
================================================
#!/usr/bin/env bash
# Office31
CUDA_VISIBLE_DEVICES=0 python source_only.py data/office31 -d Office31 -s A -t W -a resnet50 --epochs 5 --seed 0 --log logs/src_only/Office31_A2W
CUDA_VISIBLE_DEVICES=0 python source_only.py data/office31 -d Office31 -s D -t W -a resnet50 --epochs 5 --seed 0 --log logs/src_only/Office31_D2W
CUDA_VISIBLE_DEVICES=0 python source_only.py data/office31 -d Office31 -s W -t D -a resnet50 --epochs 5 --seed 0 --log logs/src_only/Office31_W2D
CUDA_VISIBLE_DEVICES=0 python source_only.py data/office31 -d Office31 -s A -t D -a resnet50 --epochs 5 --seed 0 --log logs/src_only/Office31_A2D
CUDA_VISIBLE_DEVICES=0 python source_only.py data/office31 -d Office31 -s D -t A -a resnet50 --epochs 5 --seed 0 --log logs/src_only/Office31_D2A
CUDA_VISIBLE_DEVICES=0 python source_only.py data/office31 -d Office31 -s W -t A -a resnet50 --epochs 5 --seed 0 --log logs/src_only/Office31_W2A
# Office-Home
CUDA_VISIBLE_DEVICES=0 python source_only.py data/office-home -d OfficeHome -s Ar -t Cl -a resnet50 --epochs 5 -i 500 --seed 0 --log logs/src_only/OfficeHome_Ar2Cl
CUDA_VISIBLE_DEVICES=0 python source_only.py data/office-home -d OfficeHome -s Ar -t Pr -a resnet50 --epochs 5 -i 500 --seed 0 --log logs/src_only/OfficeHome_Ar2Pr
CUDA_VISIBLE_DEVICES=0 python source_only.py data/office-home -d OfficeHome -s Ar -t Rw -a resnet50 --epochs 5 -i 500 --seed 0 --log logs/src_only/OfficeHome_Ar2Rw
CUDA_VISIBLE_DEVICES=0 python source_only.py data/office-home -d OfficeHome -s Cl -t Ar -a resnet50 --epochs 5 -i 500 --seed 0 --log logs/src_only/OfficeHome_Cl2Ar
CUDA_VISIBLE_DEVICES=0 python source_only.py data/office-home -d OfficeHome -s Cl -t Pr -a resnet50 --epochs 5 -i 500 --seed 0 --log logs/src_only/OfficeHome_Cl2Pr
CUDA_VISIBLE_DEVICES=0 python source_only.py data/office-home -d OfficeHome -s Cl -t Rw -a resnet50 --epochs 5 -i 500 --seed 0 --log logs/src_only/OfficeHome_Cl2Rw
CUDA_VISIBLE_DEVICES=0 python source_only.py data/office-home -d OfficeHome -s Pr -t Ar -a resnet50 --epochs 5 -i 500 --seed 0 --log logs/src_only/OfficeHome_Pr2Ar
CUDA_VISIBLE_DEVICES=0 python source_only.py data/office-home -d OfficeHome -s Pr -t Cl -a resnet50 --epochs 5 -i 500 --seed 0 --log logs/src_only/OfficeHome_Pr2Cl
CUDA_VISIBLE_DEVICES=0 python source_only.py data/office-home -d OfficeHome -s Pr -t Rw -a resnet50 --epochs 5 -i 500 --seed 0 --log logs/src_only/OfficeHome_Pr2Rw
CUDA_VISIBLE_DEVICES=0 python source_only.py data/office-home -d OfficeHome -s Rw -t Ar -a resnet50 --epochs 5 -i 500 --seed 0 --log logs/src_only/OfficeHome_Rw2Ar
CUDA_VISIBLE_DEVICES=0 python source_only.py data/office-home -d OfficeHome -s Rw -t Cl -a resnet50 --epochs 5 -i 500 --seed 0 --log logs/src_only/OfficeHome_Rw2Cl
CUDA_VISIBLE_DEVICES=0 python source_only.py data/office-home -d OfficeHome -s Rw -t Pr -a resnet50 --epochs 5 -i 500 --seed 0 --log logs/src_only/OfficeHome_Rw2Pr
# VisDA-2017
CUDA_VISIBLE_DEVICES=0 python source_only.py data/visda-2017 -d VisDA2017 -s Synthetic -t Real -a resnet50 \
--epochs 10 -i 500 --seed 0 --per-class-eval --log logs/src_only/VisDA2017_S2R
# ImageNet-Caltech
CUDA_VISIBLE_DEVICES=0 python source_only.py data/ImageNetCaltech -d ImageNetCaltech -s I -t C -a resnet50 \
--epochs 20 --seed 0 -i 2000 --log logs/src_only/I2C
CUDA_VISIBLE_DEVICES=0 python source_only.py data/ImageNetCaltech -d CaltechImageNet -s C -t I -a resnet50 \
--epochs 20 --seed 0 -i 2000 --log logs/src_only/C2I
================================================
FILE: domain_generalization/Transfer-Learning-Library/examples/domain_adaptation/partial_domain_adaptation/utils.py
================================================
import sys
import time
import timm
import torch
import torch.nn as nn
import torch.nn.functional as F
import torchvision.transforms as T
sys.path.append('../../..')
import common.vision.datasets.partial as datasets
from common.vision.datasets.partial import default_partial as partial
import common.vision.models as models
from common.vision.transforms import ResizeImage
from common.utils.metric import accuracy, ConfusionMatrix
from common.utils.meter import AverageMeter, ProgressMeter
def get_model_names():
return sorted(
name for name in models.__dict__
if name.islower() and not name.startswith("__")
and callable(models.__dict__[name])
) + timm.list_models()
def get_model(model_name):
if model_name in models.__dict__:
# load models from common.vision.models
backbone = models.__dict__[model_name](pretrained=True)
else:
# load models from pytorch-image-models
backbone = timm.create_model(model_name, pretrained=True)
try:
backbone.out_features = backbone.get_classifier().in_features
backbone.reset_classifier(0, '')
backbone.copy_head = backbone.get_classifier
except:
backbone.out_features = backbone.head.in_features
backbone.head = nn.Identity()
backbone.copy_head = lambda x: x.head
return backbone
def get_dataset_names():
return sorted(
name for name in datasets.__dict__
if not name.startswith("__") and callable(datasets.__dict__[name])
)
def get_dataset(dataset_name, root, source, target, train_source_transform, val_transform, train_target_transform=None):
if train_target_transform is None:
train_target_transform = train_source_transform
# load datasets from common.vision.datasets
dataset = datasets.__dict__[dataset_name]
partial_dataset = partial(dataset)
train_source_dataset = dataset(root=root, task=source, download=True, transform=train_source_transform)
train_target_dataset = partial_dataset(root=root, task=target, download=True, transform=train_target_transform)
val_dataset = partial_dataset(root=root, task=target, download=True, transform=val_transform)
if dataset_name == 'DomainNet':
test_dataset = partial_dataset(root=root, task=target, split='test', download=True, transform=val_transform)
else:
test_dataset = val_dataset
class_names = train_source_dataset.classes
num_classes = len(class_names)
return train_source_dataset, train_target_dataset, val_dataset, test_dataset, num_classes, class_names
def validate(val_loader, model, args, device) -> float:
batch_time = AverageMeter('Time', ':6.3f')
losses = AverageMeter('Loss', ':.4e')
top1 = AverageMeter('Acc@1', ':6.2f')
progress = ProgressMeter(
len(val_loader),
[batch_time, losses, top1],
prefix='Test: ')
# switch to evaluate mode
model.eval()
if args.per_class_eval:
confmat = ConfusionMatrix(len(args.class_names))
else:
confmat = None
with torch.no_grad():
end = time.time()
for i, (images, target) in enumerate(val_loader):
images = images.to(device)
target = target.to(device)
# compute output
output = model(images)
loss = F.cross_entropy(output, target)
# measure accuracy and record loss
acc1, = accuracy(output, target, topk=(1,))
if confmat:
confmat.update(target, output.argmax(1))
losses.update(loss.item(), images.size(0))
top1.update(acc1.item(), images.size(0))
# measure elapsed time
batch_time.update(time.time() - end)
end = time.time()
if i % args.print_freq == 0:
progress.display(i)
print(' * Acc@1 {top1.avg:.3f}'.format(top1=top1))
if confmat:
print(confmat.format(args.class_names))
return top1.avg
def get_train_transform(resizing='default', random_horizontal_flip=True, random_color_jitter=False):
"""
resizing mode:
- default: resize the image to 256 and take a random resized crop of size 224;
- cen.crop: resize the image to 256 and take the center crop of size 224;
- res: resize the image to 224;
- res.|crop: resize the image to 256 and take a random crop of size 224;
- res.sma|crop: resize the image keeping its aspect ratio such that the
smaller side is 256, then take a random crop of size 224;
– inc.crop: “inception crop” from (Szegedy et al., 2015);
– cif.crop: resize the image to 224, zero-pad it by 28 on each side, then take a random crop of size 224.
"""
if resizing == 'default':
transform = T.Compose([
ResizeImage(256),
T.RandomResizedCrop(224)
])
elif resizing == 'cen.crop':
transform = T.Compose([
ResizeImage(256),
T.CenterCrop(224)
])
elif resizing == 'res.':
transform = T.Resize(224)
elif resizing == 'res.|crop':
transform = T.Compose([
T.Resize((256, 256)),
T.RandomCrop(224)
])
elif resizing == "res.sma|crop":
transform = T.Compose([
T.Resize(256),
T.RandomCrop(224)
])
elif resizing == 'inc.crop':
transform = T.RandomResizedCrop(224)
elif resizing == 'cif.crop':
transform = T.Compose([
T.Resize((224, 224)),
T.Pad(28),
T.RandomCrop(224),
])
else:
raise NotImplementedError(resizing)
transforms = [transform]
if random_horizontal_flip:
transforms.append(T.RandomHorizontalFlip())
if random_color_jitter:
transforms.append(T.ColorJitter(brightness=0.5, contrast=0.5, saturation=0.5, hue=0.5))
transforms.extend([
T.ToTensor(),
T.Normalize(mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225])
])
return T.Compose(transforms)
def get_val_transform(resizing='default'):
"""
resizing mode:
- default: resize the image to 256 and take the center crop of size 224;
– res.: resize the image to 224
– res.|crop: resize the image such that the smaller side is of size 256 and
then take a central crop of size 224.
"""
if resizing == 'default':
transform = T.Compose([
ResizeImage(256),
T.CenterCrop(224),
])
elif resizing == 'res.':
transform = T.Resize((224, 224))
elif resizing == 'res.|crop':
transform = T.Compose([
T.Resize(256),
T.CenterCrop(224),
])
else:
raise NotImplementedError(resizing)
return T.Compose([
transform,
T.ToTensor(),
T.Normalize(mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225])
])
================================================
FILE: domain_generalization/Transfer-Learning-Library/examples/domain_adaptation/re_identification/README.md
================================================
# Domain Adaptation for Person Re-Identification
## Installation
Example scripts support all models in [PyTorch-Image-Models](https://github.com/rwightman/pytorch-image-models).
You also need to install timm to use PyTorch-Image-Models.
```
pip install timm
```
## Dataset
Following datasets can be downloaded automatically:
- [Market1501](http://zheng-lab.cecs.anu.edu.au/Project/project_reid.html)
- [DukeMTMC](https://exposing.ai/duke_mtmc/)
- [MSMT17](https://arxiv.org/pdf/1711.08565.pdf)
## Supported Methods
Supported methods include:
- [Two at Once: Enhancing Learning and Generalization Capacities via IBN-Net (IBN-Net, 2018 ECCV)](https://openaccess.thecvf.com/content_ECCV_2018/papers/Xingang_Pan_Two_at_Once_ECCV_2018_paper.pdf)
- [Mutual Mean-Teaching: Pseudo Label Refinery for Unsupervised Domain Adaptation on Person Re-identification (MMT, 2020 ICLR)](https://arxiv.org/abs/2001.01526)
- [Similarity Preserving Generative Adversarial Network (SPGAN, 2018 CVPR)](https://arxiv.org/pdf/1811.10551.pdf)
## Experiment and Results
The shell files give the script to reproduce the [benchmarks](/docs/dalib/benchmarks/re_identification.rst) with specified hyper-parameters.
For example, if you want to reproduce MMT on Market1501 -> DukeMTMC task, use the following script
```shell script
# Train MMT on Market1501 -> DukeMTMC task using ResNet 50.
# Assume you have put the datasets under the path `data/market1501` and `data/dukemtmc`,
# or you are glad to download the datasets automatically from the Internet to this path
# MMT involves two training steps:
# step1: pretrain
CUDA_VISIBLE_DEVICES=0 python baseline.py data -s Market1501 -t DukeMTMC -a reid_resnet50 \
--iters-per-epoch 800 --print-freq 80 --finetune --seed 0 --log logs/baseline/Market2DukeSeed0
CUDA_VISIBLE_DEVICES=0 python baseline.py data -s Market1501 -t DukeMTMC -a reid_resnet50 \
--iters-per-epoch 800 --print-freq 80 --finetune --seed 1 --log logs/baseline/Market2DukeSeed1
# step2: train mmt
CUDA_VISIBLE_DEVICES=0,1,2,3 python mmt.py data -t DukeMTMC -a reid_resnet50 \
--pretrained-model-1-path logs/baseline/Market2DukeSeed0/checkpoints/best.pth \
--pretrained-model-2-path logs/baseline/Market2DukeSeed1/checkpoints/best.pth \
--finetune --seed 0 --log logs/mmt/Market2Duke
```
For more information please refer to [Get Started](/docs/get_started/quickstart.rst) for help.
## Citation
If you use these methods in your research, please consider citing.
```
@inproceedings{IBN-Net,
author = {Xingang Pan, Ping Luo, Jianping Shi, and Xiaoou Tang},
title = {Two at Once: Enhancing Learning and Generalization Capacities via IBN-Net},
booktitle = {ECCV},
year = {2018}
}
@inproceedings{SPGAN,
title={Image-image domain adaptation with preserved self-similarity and domain-dissimilarity for person re-identification},
author={Deng, Weijian and Zheng, Liang and Ye, Qixiang and Kang, Guoliang and Yang, Yi and Jiao, Jianbin},
booktitle={CVPR},
year={2018}
}
@inproceedings{
MMT,
title={Mutual Mean-Teaching: Pseudo Label Refinery for Unsupervised Domain Adaptation on Person Re-identification},
author={Yixiao Ge and Dapeng Chen and Hongsheng Li},
booktitle={ICLR},
year={2020},
}
```
================================================
FILE: domain_generalization/Transfer-Learning-Library/examples/domain_adaptation/re_identification/baseline.py
================================================
"""
@author: Baixu Chen
@contact: cbx_99_hasta@outlook.com
"""
import random
import time
import warnings
import sys
import argparse
import shutil
import os.path as osp
import numpy as np
import torch
import torch.nn as nn
from torch.nn import DataParallel
import torch.backends.cudnn as cudnn
from torch.optim import Adam
from torch.utils.data import DataLoader
sys.path.append('../../..')
from common.vision.models.reid.loss import CrossEntropyLossWithLabelSmooth, SoftTripletLoss
from common.vision.models.reid.identifier import ReIdentifier
import common.vision.datasets.reid as datasets
from common.vision.datasets.reid.convert import convert_to_pytorch_dataset
from common.utils.scheduler import WarmupMultiStepLR
from common.utils.metric.reid import validate, visualize_ranked_results
from common.utils.data import ForeverDataIterator, RandomMultipleGallerySampler
from common.utils.metric import accuracy
from common.utils.meter import AverageMeter, ProgressMeter
from common.utils.logger import CompleteLogger
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
sys.path.append('.')
import utils
def main(args: argparse.Namespace):
logger = CompleteLogger(args.log, args.phase)
print(args)
if args.seed is not None:
random.seed(args.seed)
np.random.seed(args.seed)
torch.manual_seed(args.seed)
cudnn.deterministic = True
warnings.warn('You have chosen to seed training. '
'This will turn on the CUDNN deterministic setting, '
'which can slow down your training considerably! '
'You may see unexpected behavior when restarting '
'from checkpoints.')
cudnn.benchmark = True
# Data loading code
train_transform = utils.get_train_transform(args.height, args.width, args.train_resizing,
random_horizontal_flip=True, random_color_jitter=False,
random_gray_scale=False, random_erasing=False)
val_transform = utils.get_val_transform(args.height, args.width)
print("train_transform: ", train_transform)
print("val_transform: ", val_transform)
working_dir = osp.dirname(osp.abspath(__file__))
source_root = osp.join(working_dir, args.source_root)
target_root = osp.join(working_dir, args.target_root)
# source dataset
source_dataset = datasets.__dict__[args.source](root=osp.join(source_root, args.source.lower()))
sampler = RandomMultipleGallerySampler(source_dataset.train, args.num_instances)
train_source_loader = DataLoader(
convert_to_pytorch_dataset(source_dataset.train, root=source_dataset.images_dir, transform=train_transform),
batch_size=args.batch_size, num_workers=args.workers, sampler=sampler, pin_memory=True, drop_last=True)
train_source_iter = ForeverDataIterator(train_source_loader)
val_loader = DataLoader(
convert_to_pytorch_dataset(list(set(source_dataset.query) | set(source_dataset.gallery)),
root=source_dataset.images_dir,
transform=val_transform),
batch_size=args.batch_size, num_workers=args.workers, shuffle=False, pin_memory=True)
# target dataset
target_dataset = datasets.__dict__[args.target](root=osp.join(target_root, args.target.lower()))
train_target_loader = DataLoader(
convert_to_pytorch_dataset(target_dataset.train, root=target_dataset.images_dir, transform=train_transform),
batch_size=args.batch_size, num_workers=args.workers, shuffle=True, pin_memory=True, drop_last=True)
train_target_iter = ForeverDataIterator(train_target_loader)
test_loader = DataLoader(
convert_to_pytorch_dataset(list(set(target_dataset.query) | set(target_dataset.gallery)),
root=target_dataset.images_dir,
transform=val_transform),
batch_size=args.batch_size, num_workers=args.workers, shuffle=False, pin_memory=True)
# create model
num_classes = source_dataset.num_train_pids
backbone = utils.get_model(args.arch)
pool_layer = nn.Identity() if args.no_pool else None
model = ReIdentifier(backbone, num_classes, finetune=args.finetune, pool_layer=pool_layer).to(device)
model = DataParallel(model)
# define optimizer and lr scheduler
optimizer = Adam(model.module.get_parameters(base_lr=args.lr, rate=args.rate), args.lr,
weight_decay=args.weight_decay)
lr_scheduler = WarmupMultiStepLR(optimizer, args.milestones, gamma=0.1, warmup_factor=0.1,
warmup_steps=args.warmup_steps)
# resume from the best checkpoint
if args.phase != 'train':
checkpoint = torch.load(logger.get_checkpoint_path('best'), map_location='cpu')
model.load_state_dict(checkpoint)
# analysis the model
if args.phase == 'analysis':
# plot t-SNE
utils.visualize_tsne(source_loader=val_loader, target_loader=test_loader, model=model,
filename=osp.join(logger.visualize_directory, 'analysis', 'TSNE.pdf'), device=device)
# visualize ranked results
visualize_ranked_results(test_loader, model, target_dataset.query, target_dataset.gallery, device,
visualize_dir=logger.visualize_directory, width=args.width, height=args.height,
rerank=args.rerank)
return
if args.phase == 'test':
print("Test on source domain:")
validate(val_loader, model, source_dataset.query, source_dataset.gallery, device, cmc_flag=True,
rerank=args.rerank)
print("Test on target domain:")
validate(test_loader, model, target_dataset.query, target_dataset.gallery, device, cmc_flag=True,
rerank=args.rerank)
return
# define loss function
criterion_ce = CrossEntropyLossWithLabelSmooth(num_classes).to(device)
criterion_triplet = SoftTripletLoss(margin=args.margin).to(device)
# start training
best_val_mAP = 0.
best_test_mAP = 0.
for epoch in range(args.epochs):
# print learning rate
print(lr_scheduler.get_lr())
# train for one epoch
train(train_source_iter, train_target_iter, model, criterion_ce, criterion_triplet, optimizer, epoch, args)
# update learning rate
lr_scheduler.step()
if (epoch + 1) % args.eval_step == 0 or (epoch == args.epochs - 1):
# evaluate on validation set
print("Validation on source domain...")
_, val_mAP = validate(val_loader, model, source_dataset.query, source_dataset.gallery, device,
cmc_flag=True)
# remember best mAP and save checkpoint
torch.save(model.state_dict(), logger.get_checkpoint_path('latest'))
if val_mAP > best_val_mAP:
shutil.copy(logger.get_checkpoint_path('latest'), logger.get_checkpoint_path('best'))
best_val_mAP = max(val_mAP, best_val_mAP)
# evaluate on test set
print("Test on target domain...")
_, test_mAP = validate(test_loader, model, target_dataset.query, target_dataset.gallery, device,
cmc_flag=True, rerank=args.rerank)
best_test_mAP = max(test_mAP, best_test_mAP)
# evaluate on test set
model.load_state_dict(torch.load(logger.get_checkpoint_path('best')))
print("Test on target domain:")
_, test_mAP = validate(test_loader, model, target_dataset.query, target_dataset.gallery, device,
cmc_flag=True, rerank=args.rerank)
print("test mAP on target = {}".format(test_mAP))
print("oracle mAP on target = {}".format(best_test_mAP))
logger.close()
def train(train_source_iter: ForeverDataIterator, train_target_iter: ForeverDataIterator, model,
criterion_ce: CrossEntropyLossWithLabelSmooth, criterion_triplet: SoftTripletLoss, optimizer: Adam,
epoch: int, args: argparse.Namespace):
batch_time = AverageMeter('Time', ':4.2f')
data_time = AverageMeter('Data', ':3.1f')
losses_ce = AverageMeter('CeLoss', ':3.2f')
losses_triplet = AverageMeter('TripletLoss', ':3.2f')
losses = AverageMeter('Loss', ':3.2f')
cls_accs = AverageMeter('Cls Acc', ':3.1f')
progress = ProgressMeter(
args.iters_per_epoch,
[batch_time, data_time, losses_ce, losses_triplet, losses, cls_accs],
prefix="Epoch: [{}]".format(epoch))
# switch to train mode
model.train()
end = time.time()
for i in range(args.iters_per_epoch):
x_s, _, labels_s, _ = next(train_source_iter)
x_t, _, _, _ = next(train_target_iter)
x_s = x_s.to(device)
x_t = x_t.to(device)
labels_s = labels_s.to(device)
# measure data loading time
data_time.update(time.time() - end)
# compute output
y_s, f_s = model(x_s)
y_t, f_t = model(x_t)
# cross entropy loss
loss_ce = criterion_ce(y_s, labels_s)
# triplet loss
loss_triplet = criterion_triplet(f_s, f_s, labels_s)
loss = loss_ce + loss_triplet * args.trade_off
cls_acc = accuracy(y_s, labels_s)[0]
losses_ce.update(loss_ce.item(), x_s.size(0))
losses_triplet.update(loss_triplet.item(), x_s.size(0))
losses.update(loss.item(), x_s.size(0))
cls_accs.update(cls_acc.item(), x_s.size(0))
# compute gradient and do SGD step
optimizer.zero_grad()
loss.backward()
optimizer.step()
# measure elapsed time
batch_time.update(time.time() - end)
end = time.time()
if i % args.print_freq == 0:
progress.display(i)
if __name__ == '__main__':
dataset_names = sorted(
name for name in datasets.__dict__
if not name.startswith("__") and callable(datasets.__dict__[name])
)
parser = argparse.ArgumentParser(description="Baseline for Domain Adaptative ReID")
# dataset parameters
parser.add_argument('source_root', help='root path of the source dataset')
parser.add_argument('target_root', help='root path of the target dataset')
parser.add_argument('-s', '--source', type=str, help='source domain')
parser.add_argument('-t', '--target', type=str, help='target domain')
parser.add_argument('--train-resizing', type=str, default='default')
# model parameters
parser.add_argument('-a', '--arch', metavar='ARCH', default='reid_resnet50',
choices=utils.get_model_names(),
help='backbone architecture: ' +
' | '.join(utils.get_model_names()) +
' (default: reid_resnet50)')
parser.add_argument('--no-pool', action='store_true', help='no pool layer after the feature extractor.')
parser.add_argument('--finetune', action='store_true', help='whether use 10x smaller lr for backbone')
parser.add_argument('--rate', type=float, default=0.2)
# training parameters
parser.add_argument('--trade-off', type=float, default=1,
help='trade-off hyper parameter between cross entropy loss and triplet loss')
parser.add_argument('--margin', type=float, default=0.0, help='margin for the triplet loss with batch hard')
parser.add_argument('-j', '--workers', type=int, default=4)
parser.add_argument('-b', '--batch-size', type=int, default=16)
parser.add_argument('--height', type=int, default=256, help="input height")
parser.add_argument('--width', type=int, default=128, help="input width")
parser.add_argument('--num-instances', type=int, default=4,
help="each minibatch consist of "
"(batch_size // num_instances) identities, and "
"each identity has num_instances instances, "
"default: 4")
parser.add_argument('--lr', type=float, default=0.00035,
help="initial learning rate")
parser.add_argument('--weight-decay', type=float, default=5e-4)
parser.add_argument('--epochs', type=int, default=80)
parser.add_argument('--warmup-steps', type=int, default=10, help='number of warm-up steps')
parser.add_argument('--milestones', nargs='+', type=int, default=[40, 70],
help='milestones for the learning rate decay')
parser.add_argument('--eval-step', type=int, default=40)
parser.add_argument('--iters-per-epoch', type=int, default=400)
parser.add_argument('--print-freq', type=int, default=40)
parser.add_argument('--seed', default=None, type=int, help='seed for initializing training.')
parser.add_argument('--rerank', action='store_true', help="evaluation only")
parser.add_argument("--log", type=str, default='baseline',
help="Where to save logs, checkpoints and debugging images.")
parser.add_argument("--phase", type=str, default='train', choices=['train', 'test', 'analysis'],
help="When phase is 'test', only test the model."
"When phase is 'analysis', only analysis the model.")
args = parser.parse_args()
main(args)
================================================
FILE: domain_generalization/Transfer-Learning-Library/examples/domain_adaptation/re_identification/baseline.sh
================================================
#!/usr/bin/env bash
# Market1501 -> Duke
CUDA_VISIBLE_DEVICES=0 python baseline.py data data -s Market1501 -t DukeMTMC -a reid_resnet50 \
--iters-per-epoch 800 --print-freq 80 --finetune --seed 0 --log logs/baseline/Market2Duke
# Duke -> Market1501
CUDA_VISIBLE_DEVICES=0 python baseline.py data data -s DukeMTMC -t Market1501 -a reid_resnet50 \
--iters-per-epoch 800 --print-freq 80 --finetune --seed 0 --log logs/baseline/Duke2Market
# Market1501 -> MSMT
CUDA_VISIBLE_DEVICES=0 python baseline.py data data -s Market1501 -t MSMT17 -a reid_resnet50 \
--iters-per-epoch 800 --print-freq 80 --finetune --seed 0 --log logs/baseline/Market2MSMT
# MSMT -> Market1501
CUDA_VISIBLE_DEVICES=0 python baseline.py data data -s MSMT17 -t Market1501 -a reid_resnet50 \
--iters-per-epoch 800 --print-freq 80 --finetune --seed 0 --log logs/baseline/MSMT2Market
# Duke -> MSMT
CUDA_VISIBLE_DEVICES=0 python baseline.py data data -s DukeMTMC -t MSMT17 -a reid_resnet50 \
--iters-per-epoch 800 --print-freq 80 --finetune --seed 0 --log logs/baseline/Duke2MSMT
# MSMT -> Duke
CUDA_VISIBLE_DEVICES=0 python baseline.py data data -s MSMT17 -t DukeMTMC -a reid_resnet50 \
--iters-per-epoch 800 --print-freq 80 --finetune --seed 0 --log logs/baseline/MSMT2Duke
================================================
FILE: domain_generalization/Transfer-Learning-Library/examples/domain_adaptation/re_identification/baseline_cluster.py
================================================
"""
@author: Baixu Chen
@contact: cbx_99_hasta@outlook.com
"""
import random
import time
import warnings
import sys
import argparse
import shutil
import os.path as osp
import numpy as np
import torch
import torch.nn as nn
import torch.nn.functional as F
from torch.nn import DataParallel
import torch.backends.cudnn as cudnn
from torch.optim import Adam
from torch.utils.data import DataLoader
from sklearn.cluster import KMeans, DBSCAN
sys.path.append('../../..')
import common.vision.datasets.reid as datasets
from common.vision.datasets.reid.convert import convert_to_pytorch_dataset
from common.vision.models.reid.identifier import ReIdentifier
from common.vision.models.reid.loss import CrossEntropyLossWithLabelSmooth, SoftTripletLoss
from common.utils.metric.reid import extract_reid_feature, validate, visualize_ranked_results
from common.utils.data import ForeverDataIterator, RandomMultipleGallerySampler
from common.utils.metric import accuracy
from common.utils.meter import AverageMeter, ProgressMeter
from common.utils.logger import CompleteLogger
sys.path.append('.')
import utils
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
def main(args: argparse.Namespace):
logger = CompleteLogger(args.log, args.phase)
print(args)
if args.seed is not None:
random.seed(args.seed)
np.random.seed(args.seed)
torch.manual_seed(args.seed)
cudnn.deterministic = True
warnings.warn('You have chosen to seed training. '
'This will turn on the CUDNN deterministic setting, '
'which can slow down your training considerably! '
'You may see unexpected behavior when restarting '
'from checkpoints.')
cudnn.benchmark = True
# Data loading code
train_transform = utils.get_train_transform(args.height, args.width, args.train_resizing,
random_horizontal_flip=True, random_color_jitter=False,
random_gray_scale=False, random_erasing=True)
val_transform = utils.get_val_transform(args.height, args.width)
print("train_transform: ", train_transform)
print("val_transform: ", val_transform)
working_dir = osp.dirname(osp.abspath(__file__))
source_root = osp.join(working_dir, args.source_root)
target_root = osp.join(working_dir, args.target_root)
# source dataset
source_dataset = datasets.__dict__[args.source](root=osp.join(source_root, args.source.lower()))
val_loader = DataLoader(
convert_to_pytorch_dataset(list(set(source_dataset.query) | set(source_dataset.gallery)),
root=source_dataset.images_dir,
transform=val_transform),
batch_size=args.batch_size, num_workers=args.workers, shuffle=False, pin_memory=True)
# target dataset
target_dataset = datasets.__dict__[args.target](root=osp.join(target_root, args.target.lower()))
cluster_loader = DataLoader(
convert_to_pytorch_dataset(target_dataset.train, root=target_dataset.images_dir, transform=val_transform),
batch_size=args.batch_size, num_workers=args.workers, shuffle=False, pin_memory=True)
test_loader = DataLoader(
convert_to_pytorch_dataset(list(set(target_dataset.query) | set(target_dataset.gallery)),
root=target_dataset.images_dir,
transform=val_transform),
batch_size=args.batch_size, num_workers=args.workers, shuffle=False, pin_memory=True)
# create model
num_classes = args.num_clusters
backbone = utils.get_model(args.arch)
pool_layer = nn.Identity() if args.no_pool else None
model = ReIdentifier(backbone, num_classes, finetune=args.finetune, pool_layer=pool_layer).to(device)
model = DataParallel(model)
# load pretrained weights
pretrained_model = torch.load(args.pretrained_model_path)
utils.copy_state_dict(model, pretrained_model)
# resume from the best checkpoint
if args.phase != 'train':
checkpoint = torch.load(logger.get_checkpoint_path('best'), map_location='cpu')
utils.copy_state_dict(model, checkpoint['model'])
# analysis the model
if args.phase == 'analysis':
# plot t-SNE
utils.visualize_tsne(source_loader=val_loader, target_loader=test_loader, model=model,
filename=osp.join(logger.visualize_directory, 'analysis', 'TSNE.pdf'), device=device)
# visualize ranked results
visualize_ranked_results(test_loader, model, target_dataset.query, target_dataset.gallery, device,
visualize_dir=logger.visualize_directory, width=args.width, height=args.height,
rerank=args.rerank)
return
if args.phase == 'test':
print("Test on Source domain:")
validate(val_loader, model, source_dataset.query, source_dataset.gallery, device, cmc_flag=True,
rerank=args.rerank)
print("Test on target domain:")
validate(test_loader, model, target_dataset.query, target_dataset.gallery, device, cmc_flag=True,
rerank=args.rerank)
return
# define loss function
criterion_ce = CrossEntropyLossWithLabelSmooth(num_classes).to(device)
criterion_triplet = SoftTripletLoss(margin=args.margin).to(device)
# optionally resume from a checkpoint
if args.resume:
checkpoint = torch.load(args.resume, map_location='cpu')
utils.copy_state_dict(model, checkpoint['model'])
args.start_epoch = checkpoint['epoch'] + 1
# start training
best_test_mAP = 0.
for epoch in range(args.start_epoch, args.epochs):
# run clustering algorithm and generate pseudo labels
if args.clustering_algorithm == 'kmeans':
train_target_iter = run_kmeans(cluster_loader, model, target_dataset, train_transform, args)
elif args.clustering_algorithm == 'dbscan':
train_target_iter, num_classes = run_dbscan(cluster_loader, model, target_dataset, train_transform, args)
# define cross entropy loss with current number of classes
criterion_ce = CrossEntropyLossWithLabelSmooth(num_classes).to(device)
# define optimizer
optimizer = Adam(model.module.get_parameters(base_lr=args.lr, rate=args.rate), args.lr,
weight_decay=args.weight_decay)
# train for one epoch
train(train_target_iter, model, optimizer, criterion_ce, criterion_triplet, epoch, args)
if (epoch + 1) % args.eval_step == 0 or (epoch == args.epochs - 1):
# remember best mAP and save checkpoint
torch.save(
{
'model': model.state_dict(),
'epoch': epoch
}, logger.get_checkpoint_path(epoch)
)
print("Test on target domain...")
_, test_mAP = validate(test_loader, model, target_dataset.query, target_dataset.gallery, device,
cmc_flag=True, rerank=args.rerank)
if test_mAP > best_test_mAP:
shutil.copy(logger.get_checkpoint_path(epoch), logger.get_checkpoint_path('best'))
best_test_mAP = max(test_mAP, best_test_mAP)
print("best mAP on target = {}".format(best_test_mAP))
logger.close()
def run_kmeans(cluster_loader: DataLoader, model: DataParallel, target_dataset, train_transform,
args: argparse.Namespace):
# run kmeans clustering algorithm
print('Clustering into {} classes'.format(args.num_clusters))
feature_dict = extract_reid_feature(cluster_loader, model, device, normalize=True)
feature = torch.stack(list(feature_dict.values())).cpu().numpy()
km = KMeans(n_clusters=args.num_clusters, random_state=args.seed).fit(feature)
cluster_labels = km.labels_
cluster_centers = km.cluster_centers_
print('Clustering finished')
# normalize cluster centers and convert to pytorch tensor
cluster_centers = torch.from_numpy(cluster_centers).float().to(device)
cluster_centers = F.normalize(cluster_centers, dim=1)
# reinitialize classifier head
model.module.head.weight.data.copy_(cluster_centers)
# generate training set with pseudo labels
target_train_set = []
for (fname, _, cid), label in zip(target_dataset.train, cluster_labels):
target_train_set.append((fname, int(label), cid))
sampler = RandomMultipleGallerySampler(target_train_set, args.num_instances)
train_target_loader = DataLoader(
convert_to_pytorch_dataset(target_train_set, root=target_dataset.images_dir, transform=train_transform),
batch_size=args.batch_size, num_workers=args.workers, sampler=sampler, pin_memory=True, drop_last=True)
train_target_iter = ForeverDataIterator(train_target_loader)
return train_target_iter
def run_dbscan(cluster_loader: DataLoader, model: DataParallel, target_dataset, train_transform,
args: argparse.Namespace):
# run dbscan clustering algorithm
feature_dict = extract_reid_feature(cluster_loader, model, device, normalize=True)
feature = torch.stack(list(feature_dict.values())).cpu()
rerank_dist = utils.compute_rerank_dist(feature).numpy()
print('Clustering with dbscan algorithm')
dbscan = DBSCAN(eps=0.6, min_samples=4, metric='precomputed', n_jobs=-1)
cluster_labels = dbscan.fit_predict(rerank_dist)
print('Clustering finished')
# generate training set with pseudo labels and calculate cluster centers
target_train_set = []
cluster_centers = {}
for i, ((fname, _, cid), label) in enumerate(zip(target_dataset.train, cluster_labels)):
if label == -1:
continue
target_train_set.append((fname, label, cid))
if label not in cluster_centers:
cluster_centers[label] = []
cluster_centers[label].append(feature[i])
cluster_centers = [torch.stack(cluster_centers[idx]).mean(0) for idx in sorted(cluster_centers.keys())]
cluster_centers = torch.stack(cluster_centers)
# normalize cluster centers
cluster_centers = F.normalize(cluster_centers, dim=1).float().to(device)
# reinitialize classifier head
features_dim = model.module.features_dim
num_clusters = len(set(cluster_labels)) - (1 if -1 in cluster_labels else 0)
model.module.head = nn.Linear(features_dim, num_clusters, bias=False).to(device)
model.module.head.weight.data.copy_(cluster_centers)
sampler = RandomMultipleGallerySampler(target_train_set, args.num_instances)
train_target_loader = DataLoader(
convert_to_pytorch_dataset(target_train_set, root=target_dataset.images_dir, transform=train_transform),
batch_size=args.batch_size, num_workers=args.workers, sampler=sampler, pin_memory=True, drop_last=True)
train_target_iter = ForeverDataIterator(train_target_loader)
return train_target_iter, num_clusters
def train(train_target_iter: ForeverDataIterator, model, optimizer, criterion_ce: CrossEntropyLossWithLabelSmooth,
criterion_triplet: SoftTripletLoss, epoch: int, args: argparse.Namespace):
# train with pseudo labels
batch_time = AverageMeter('Time', ':4.2f')
data_time = AverageMeter('Data', ':3.1f')
losses_ce = AverageMeter('CeLoss', ':3.2f')
losses_triplet = AverageMeter('TripletLoss', ':3.2f')
losses = AverageMeter('Loss', ':3.2f')
cls_accs = AverageMeter('Cls Acc', ':3.1f')
progress = ProgressMeter(
args.iters_per_epoch,
[batch_time, data_time, losses_ce, losses_triplet, losses, cls_accs],
prefix="Epoch: [{}]".format(epoch))
# switch to train mode
model.train()
end = time.time()
for i in range(args.iters_per_epoch):
x_t, _, labels_t, _ = next(train_target_iter)
x_t = x_t.to(device)
labels_t = labels_t.to(device)
# measure data loading time
data_time.update(time.time() - end)
# compute output
y_t, f_t = model(x_t)
# cross entropy loss
loss_ce = criterion_ce(y_t, labels_t)
# triplet loss
loss_triplet = criterion_triplet(f_t, f_t, labels_t)
loss = loss_ce + loss_triplet * args.trade_off
cls_acc = accuracy(y_t, labels_t)[0]
losses_ce.update(loss_ce.item(), x_t.size(0))
losses_triplet.update(loss_triplet.item(), x_t.size(0))
losses.update(loss.item(), x_t.size(0))
cls_accs.update(cls_acc.item(), x_t.size(0))
# compute gradient and do SGD step
optimizer.zero_grad()
loss.backward()
optimizer.step()
# measure elapsed time
batch_time.update(time.time() - end)
end = time.time()
if i % args.print_freq == 0:
progress.display(i)
if __name__ == '__main__':
dataset_names = sorted(
name for name in datasets.__dict__
if not name.startswith("__") and callable(datasets.__dict__[name])
)
parser = argparse.ArgumentParser(description="Cluster Baseline for Domain Adaptative ReID")
# dataset parameters
parser.add_argument('source_root', help='root path of the source dataset')
parser.add_argument('target_root', help='root path of the target dataset')
parser.add_argument('-s', '--source', type=str, help='source domain')
parser.add_argument('-t', '--target', type=str, help='target domain')
parser.add_argument('--train-resizing', type=str, default='default')
# model parameters
parser.add_argument('-a', '--arch', metavar='ARCH', default='reid_resnet50',
choices=utils.get_model_names(),
help='backbone architecture: ' +
' | '.join(utils.get_model_names()) +
' (default: reid_resnet50)')
parser.add_argument('--num-clusters', type=int, default=500)
parser.add_argument('--no-pool', action='store_true', help='no pool layer after the feature extractor.')
parser.add_argument('--finetune', action='store_true', help='whether use 10x smaller lr for backbone')
parser.add_argument('--rate', type=float, default=0.2)
# training parameters
parser.add_argument('--clustering-algorithm', type=str, default='dbscan', choices=['kmeans', 'dbscan'],
help='clustering algorithm to run, currently supported method: ["kmeans", "dbscan"]')
parser.add_argument('--resume', type=str, default=None,
help="Where restore model parameters from.")
parser.add_argument('--pretrained-model-path', type=str, help='path to pretrained (source-only) model')
parser.add_argument('--trade-off', type=float, default=1,
help='trade-off hyper parameter between cross entropy loss and triplet loss')
parser.add_argument('--margin', type=float, default=0.0, help='margin for the triplet loss with batch hard')
parser.add_argument('-j', '--workers', type=int, default=4)
parser.add_argument('-b', '--batch-size', type=int, default=64)
parser.add_argument('--height', type=int, default=256, help="input height")
parser.add_argument('--width', type=int, default=128, help="input width")
parser.add_argument('--num-instances', type=int, default=4,
help="each minibatch consist of "
"(batch_size // num_instances) identities, and "
"each identity has num_instances instances, "
"default: 4")
parser.add_argument('--lr', type=float, default=0.00035,
help="learning rate")
parser.add_argument('--weight-decay', type=float, default=5e-4)
parser.add_argument('--epochs', type=int, default=40)
parser.add_argument('--start-epoch', default=0, type=int, help='start epoch')
parser.add_argument('--eval-step', type=int, default=1)
parser.add_argument('--iters-per-epoch', type=int, default=400)
parser.add_argument('--print-freq', type=int, default=40)
parser.add_argument('--seed', default=None, type=int, help='seed for initializing training.')
parser.add_argument('--rerank', action='store_true', help="evaluation only")
parser.add_argument("--log", type=str, default='baseline_cluster',
help="Where to save logs, checkpoints and debugging images.")
parser.add_argument("--phase", type=str, default='train', choices=['train', 'test', 'analysis'],
help="When phase is 'test', only test the model."
"When phase is 'analysis', only analysis the model.")
args = parser.parse_args()
main(args)
================================================
FILE: domain_generalization/Transfer-Learning-Library/examples/domain_adaptation/re_identification/baseline_cluster.sh
================================================
#!/usr/bin/env bash
# Market1501 -> Duke
# step1: pretrain
CUDA_VISIBLE_DEVICES=0 python baseline.py data data -s Market1501 -t DukeMTMC -a reid_resnet50 \
--iters-per-epoch 800 --print-freq 80 --finetune --seed 0 --log logs/baseline/Market2Duke
# step2: train with pseudo labels assigned by cluster algorithm
CUDA_VISIBLE_DEVICES=0,1,2,3 python baseline_cluster.py data data -s Market1501 -t DukeMTMC -a reid_resnet50 \
--pretrained-model-path logs/baseline/Market2Duke/checkpoints/best.pth \
--finetune --seed 0 --log logs/baseline_cluster/Market2Duke
# Duke -> Market1501
# step1: pretrain
CUDA_VISIBLE_DEVICES=0 python baseline.py data data -s DukeMTMC -t Market1501 -a reid_resnet50 \
--iters-per-epoch 800 --print-freq 80 --finetune --seed 0 --log logs/baseline/Duke2Market
# step2: train with pseudo labels assigned by cluster algorithm
CUDA_VISIBLE_DEVICES=0,1,2,3 python baseline_cluster.py data data -s DukeMTMC -t Market1501 -a reid_resnet50 \
--pretrained-model-path logs/baseline/Duke2Market/checkpoints/best.pth \
--finetune --seed 0 --log logs/baseline_cluster/Duke2Market
# Market1501 -> MSMT
# step1: pretrain
CUDA_VISIBLE_DEVICES=0 python baseline.py data data -s Market1501 -t MSMT17 -a reid_resnet50 \
--iters-per-epoch 800 --print-freq 80 --finetune --seed 0 --log logs/baseline/Market2MSMT
# step2: train with pseudo labels assigned by cluster algorithm
CUDA_VISIBLE_DEVICES=0,1,2,3 python baseline_cluster.py data data -s Market1501 -t MSMT17 -a reid_resnet50 \
--pretrained-model-path logs/baseline/Market2MSMT/checkpoints/best.pth \
--num-clusters 1000 --finetune --seed 0 --log logs/baseline_cluster/Market2MSMT
# MSMT -> Market1501
# step1: pretrain
CUDA_VISIBLE_DEVICES=0 python baseline.py data data -s MSMT17 -t Market1501 -a reid_resnet50 \
--iters-per-epoch 800 --print-freq 80 --finetune --seed 0 --log logs/baseline/MSMT2Market
# step2: train with pseudo labels assigned by cluster algorithm
CUDA_VISIBLE_DEVICES=0,1,2,3 python baseline_cluster.py data data -s MSMT17 -t Market1501 -a reid_resnet50 \
--pretrained-model-path logs/baseline/MSMT2Market/checkpoints/best.pth \
--finetune --seed 0 --log logs/baseline_cluster/MSMT2Market
# Duke -> MSMT
# step1: pretrain
CUDA_VISIBLE_DEVICES=0 python baseline.py data data -s DukeMTMC -t MSMT17 -a reid_resnet50 \
--iters-per-epoch 800 --print-freq 80 --finetune --seed 0 --log logs/baseline/Duke2MSMT
# step2: train with pseudo labels assigned by cluster algorithm
CUDA_VISIBLE_DEVICES=0,1,2,3 python baseline_cluster.py data data -s DukeMTMC -t MSMT17 -a reid_resnet50 \
--pretrained-model-path logs/baseline/Duke2MSMT/checkpoints/best.pth \
--num-clusters 1000 --finetune --seed 0 --log logs/baseline_cluster/Duke2MSMT
# MSMT -> Duke
# step1: pretrain
CUDA_VISIBLE_DEVICES=0 python baseline.py data data -s MSMT17 -t DukeMTMC -a reid_resnet50 \
--iters-per-epoch 800 --print-freq 80 --finetune --seed 0 --log logs/baseline/MSMT2Duke
# step2: train with pseudo labels assigned by cluster algorithm
CUDA_VISIBLE_DEVICES=0,1,2,3 python baseline_cluster.py data data -s MSMT17 -t DukeMTMC -a reid_resnet50 \
--pretrained-model-path logs/baseline/MSMT2Duke/checkpoints/best.pth \
--finetune --seed 0 --log logs/baseline_cluster/MSMT2Duke
================================================
FILE: domain_generalization/Transfer-Learning-Library/examples/domain_adaptation/re_identification/ibn.sh
================================================
#!/usr/bin/env bash
# Market1501 -> Duke
CUDA_VISIBLE_DEVICES=0 python baseline.py data data -s Market1501 -t DukeMTMC -a resnet50_ibn_a \
--iters-per-epoch 800 --print-freq 80 --finetune --seed 0 --log logs/ibn/Market2Duke
CUDA_VISIBLE_DEVICES=0 python baseline.py data data -s Market1501 -t DukeMTMC -a resnet50_ibn_b \
--iters-per-epoch 800 --print-freq 80 --finetune --seed 0 --log logs/ibn/Market2Duke
# Duke -> Market1501
CUDA_VISIBLE_DEVICES=0 python baseline.py data data -s DukeMTMC -t Market1501 -a resnet50_ibn_a \
--iters-per-epoch 800 --print-freq 80 --finetune --seed 0 --log logs/ibn/Duke2Market
CUDA_VISIBLE_DEVICES=0 python baseline.py data data -s DukeMTMC -t Market1501 -a resnet50_ibn_b \
--iters-per-epoch 800 --print-freq 80 --finetune --seed 0 --log logs/ibn/Duke2Market
# Market1501 -> MSMT
CUDA_VISIBLE_DEVICES=0 python baseline.py data data -s Market1501 -t MSMT17 -a resnet50_ibn_a \
--iters-per-epoch 800 --print-freq 80 --finetune --seed 0 --log logs/ibn/Market2MSMT
CUDA_VISIBLE_DEVICES=0 python baseline.py data data -s Market1501 -t MSMT17 -a resnet50_ibn_b \
--iters-per-epoch 800 --print-freq 80 --finetune --seed 0 --log logs/ibn/Market2MSMT
# MSMT -> Market1501
CUDA_VISIBLE_DEVICES=0 python baseline.py data data -s MSMT17 -t Market1501 -a resnet50_ibn_a \
--iters-per-epoch 800 --print-freq 80 --finetune --seed 0 --log logs/ibn/MSMT2Market
CUDA_VISIBLE_DEVICES=0 python baseline.py data data -s MSMT17 -t Market1501 -a resnet50_ibn_b \
--iters-per-epoch 800 --print-freq 80 --finetune --seed 0 --log logs/ibn/MSMT2Market
# Duke -> MSMT
CUDA_VISIBLE_DEVICES=0 python baseline.py data data -s DukeMTMC -t MSMT17 -a resnet50_ibn_a \
--iters-per-epoch 800 --print-freq 80 --finetune --seed 0 --log logs/ibn/Duke2MSMT
CUDA_VISIBLE_DEVICES=0 python baseline.py data data -s DukeMTMC -t MSMT17 -a resnet50_ibn_b \
--iters-per-epoch 800 --print-freq 80 --finetune --seed 0 --log logs/ibn/Duke2MSMT
# MSMT -> Duke
CUDA_VISIBLE_DEVICES=0 python baseline.py data data -s MSMT17 -t DukeMTMC -a resnet50_ibn_a \
--iters-per-epoch 800 --print-freq 80 --finetune --seed 0 --log logs/ibn/MSMT2Duke
CUDA_VISIBLE_DEVICES=0 python baseline.py data data -s MSMT17 -t DukeMTMC -a resnet50_ibn_b \
--iters-per-epoch 800 --print-freq 80 --finetune --seed 0 --log logs/ibn/MSMT2Duke
================================================
FILE: domain_generalization/Transfer-Learning-Library/examples/domain_adaptation/re_identification/idm.py
================================================
import random
import time
import warnings
import sys
import argparse
import shutil
import os.path as osp
import numpy as np
import torch
import torch.nn as nn
import torch.nn.functional as F
from torch.nn import DataParallel
import torch.backends.cudnn as cudnn
from torch.optim import Adam
from torch.optim.lr_scheduler import StepLR
from torch.utils.data import DataLoader
from sklearn.cluster import DBSCAN
sys.path.append('../../..')
import dalib.adaptation.idm.models as models
from dalib.adaptation.idm.models.identifier import ReIdentifier
from dalib.adaptation.idm.xbm import XBM
from dalib.adaptation.idm.loss import BridgeFeatLoss, BridgeProbLoss, DivLoss
from dalib.adaptation.idm.utils import filter_layers, convert_dsbn_idm
import common.vision.datasets.reid as datasets
from common.vision.datasets.reid.convert import convert_to_pytorch_dataset
from common.utils.metric.reid import extract_reid_feature, validate, visualize_ranked_results
from common.vision.models.reid.loss import TripletLoss, TripletLossXBM
from common.utils.data import ForeverDataIterator, RandomMultipleGallerySampler
from common.utils.metric import accuracy
from common.utils.meter import AverageMeter, ProgressMeter
from common.utils.logger import CompleteLogger
sys.path.append('.')
import utils
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
def main(args: argparse.Namespace):
logger = CompleteLogger(args.log, args.phase)
print(args)
if args.seed is not None:
random.seed(args.seed)
np.random.seed(args.seed)
torch.manual_seed(args.seed)
cudnn.deterministic = True
warnings.warn('You have chosen to seed training. '
'This will turn on the CUDNN deterministic setting, '
'which can slow down your training considerably! '
'You may see unexpected behavior when restarting '
'from checkpoints.')
cudnn.benchmark = True
# Data loading code
train_transform = utils.get_train_transform(args.height, args.width, args.train_resizing,
random_horizontal_flip=True, random_color_jitter=False,
random_gray_scale=False, random_erasing=True)
val_transform = utils.get_val_transform(args.height, args.width)
print("train_transform: ", train_transform)
print("val_transform: ", val_transform)
working_dir = osp.dirname(osp.abspath(__file__))
source_root = osp.join(working_dir, args.source_root)
target_root = osp.join(working_dir, args.target_root)
# source dataset
source_dataset = datasets.__dict__[args.source](root=osp.join(source_root, args.source.lower()))
sampler = RandomMultipleGallerySampler(source_dataset.train, args.num_instances)
train_source_loader = DataLoader(
convert_to_pytorch_dataset(source_dataset.train, root=source_dataset.images_dir, transform=train_transform),
batch_size=args.batch_size, num_workers=args.workers, sampler=sampler, pin_memory=True, drop_last=True)
train_source_iter = ForeverDataIterator(train_source_loader)
cluster_source_loader = DataLoader(
convert_to_pytorch_dataset(source_dataset.train, root=source_dataset.images_dir, transform=val_transform),
batch_size=args.batch_size, num_workers=args.workers, shuffle=False, pin_memory=True)
val_loader = DataLoader(
convert_to_pytorch_dataset(list(set(source_dataset.query) | set(source_dataset.gallery)),
root=source_dataset.images_dir, transform=val_transform),
batch_size=args.batch_size, num_workers=args.workers, shuffle=False, pin_memory=True)
# target dataset
target_dataset = datasets.__dict__[args.target](root=osp.join(target_root, args.target.lower()))
cluster_target_loader = DataLoader(
convert_to_pytorch_dataset(target_dataset.train, root=target_dataset.images_dir, transform=val_transform),
batch_size=args.batch_size, num_workers=args.workers, shuffle=False, pin_memory=True)
test_loader = DataLoader(
convert_to_pytorch_dataset(list(set(target_dataset.query) | set(target_dataset.gallery)),
root=target_dataset.images_dir, transform=val_transform),
batch_size=args.batch_size, num_workers=args.workers, shuffle=False, pin_memory=True)
n_s_classes = source_dataset.num_train_pids
args.n_classes = n_s_classes + len(target_dataset.train)
args.n_s_classes = n_s_classes
args.n_t_classes = len(target_dataset.train)
# create model
backbone = models.__dict__[args.arch](pretrained=True)
pool_layer = nn.Identity() if args.no_pool else None
model = ReIdentifier(backbone, args.n_classes, finetune=args.finetune, pool_layer=pool_layer)
features_dim = model.features_dim
idm_bn_names = filter_layers(args.stage)
convert_dsbn_idm(model, idm_bn_names, idm=False)
model = model.to(device)
model = DataParallel(model)
# resume from the best checkpoint
if args.phase != 'train':
checkpoint = torch.load(logger.get_checkpoint_path('best'), map_location='cpu')
utils.copy_state_dict(model, checkpoint['model'])
# analysis the model
if args.phase == 'analysis':
# plot t-SNE
utils.visualize_tsne(source_loader=val_loader, target_loader=test_loader, model=model,
filename=osp.join(logger.visualize_directory, 'analysis', 'TSNE.pdf'), device=device)
# visualize ranked results
visualize_ranked_results(test_loader, model, target_dataset.query, target_dataset.gallery, device,
visualize_dir=logger.visualize_directory, width=args.width, height=args.height,
rerank=args.rerank)
return
if args.phase == 'test':
print("Test on target domain:")
validate(test_loader, model, target_dataset.query, target_dataset.gallery, device, cmc_flag=True,
rerank=args.rerank)
return
# create XBM
dataset_size = len(source_dataset.train) + len(target_dataset.train)
memory_size = int(args.ratio * dataset_size)
xbm = XBM(memory_size, features_dim)
# initialize source-domain class centroids
source_feature_dict = extract_reid_feature(cluster_source_loader, model, device, normalize=True)
source_features_per_id = {}
for f, pid, _ in source_dataset.train:
if pid not in source_features_per_id:
source_features_per_id[pid] = []
source_features_per_id[pid].append(source_feature_dict[f].unsqueeze(0))
source_centers = [torch.cat(source_features_per_id[pid], 0).mean(0) for pid in
sorted(source_features_per_id.keys())]
source_centers = torch.stack(source_centers, 0)
source_centers = F.normalize(source_centers, dim=1)
model.module.head.weight.data[0: n_s_classes].copy_(source_centers.to(device))
# save memory
del source_centers, cluster_source_loader, source_features_per_id
# define optimizer and lr scheduler
optimizer = Adam(model.module.get_parameters(base_lr=args.lr, rate=args.rate), args.lr,
weight_decay=args.weight_decay)
lr_scheduler = StepLR(optimizer, step_size=args.step_size, gamma=0.1)
if args.resume:
checkpoint = torch.load(args.resume, map_location='cpu')
utils.copy_state_dict(model, checkpoint['model'])
optimizer.load_state_dict(checkpoint['optimizer'])
lr_scheduler.load_state_dict(checkpoint['lr_scheduler'])
args.start_epoch = checkpoint['epoch'] + 1
# start training
best_test_mAP = 0.
for epoch in range(args.start_epoch, args.epochs):
# run clustering algorithm and generate pseudo labels
train_target_iter = run_dbscan(cluster_target_loader, model, target_dataset, train_transform, args)
# train for one epoch
print(lr_scheduler.get_lr())
train(train_source_iter, train_target_iter, model, optimizer, xbm, epoch, args)
if (epoch + 1) % args.eval_step == 0 or (epoch == args.epochs - 1):
# remember best mAP and save checkpoint
torch.save(
{
'model': model.state_dict(),
'optimizer': optimizer.state_dict(),
'lr_scheduler': lr_scheduler.state_dict(),
'epoch': epoch
}, logger.get_checkpoint_path(epoch)
)
print("Test on target domain...")
_, test_mAP = validate(test_loader, model, target_dataset.query, target_dataset.gallery, device,
cmc_flag=True, rerank=args.rerank)
if test_mAP > best_test_mAP:
shutil.copy(logger.get_checkpoint_path(epoch), logger.get_checkpoint_path('best'))
best_test_mAP = max(test_mAP, best_test_mAP)
# update lr
lr_scheduler.step()
print("best mAP on target = {}".format(best_test_mAP))
logger.close()
def run_dbscan(cluster_loader: DataLoader, model: DataParallel, target_dataset, train_transform,
args: argparse.Namespace):
# run dbscan clustering algorithm
feature_dict = extract_reid_feature(cluster_loader, model, device, normalize=True)
feature = torch.stack(list(feature_dict.values())).cpu()
rerank_dist = utils.compute_rerank_dist(feature).numpy()
print('Clustering with dbscan algorithm')
dbscan = DBSCAN(eps=args.eps, min_samples=4, metric='precomputed', n_jobs=-1)
cluster_labels = dbscan.fit_predict(rerank_dist)
print('Clustering finished')
# generate training set with pseudo labels and calculate cluster centers
target_train_set = []
cluster_centers = {}
for i, ((fname, _, cid), label) in enumerate(zip(target_dataset.train, cluster_labels)):
if label == -1:
continue
target_train_set.append((fname, args.n_s_classes + label, cid))
if label not in cluster_centers:
cluster_centers[label] = []
cluster_centers[label].append(feature[i])
cluster_centers = [torch.stack(cluster_centers[idx]).mean(0) for idx in sorted(cluster_centers.keys())]
cluster_centers = torch.stack(cluster_centers)
# normalize cluster centers
cluster_centers = F.normalize(cluster_centers, dim=1).float().to(device)
# reinitialize classifier head
num_clusters = len(set(cluster_labels)) - (1 if -1 in cluster_labels else 0)
args.n_t_classes = num_clusters
model.module.head.weight.data[args.n_s_classes: args.n_s_classes + args.n_t_classes].copy_(cluster_centers)
sampler = RandomMultipleGallerySampler(target_train_set, args.num_instances)
train_target_loader = DataLoader(
convert_to_pytorch_dataset(target_train_set, root=target_dataset.images_dir, transform=train_transform),
batch_size=args.batch_size, num_workers=args.workers, sampler=sampler, pin_memory=True, drop_last=True)
train_target_iter = ForeverDataIterator(train_target_loader)
return train_target_iter
def train(train_source_iter: ForeverDataIterator, train_target_iter: ForeverDataIterator, model, optimizer: Adam,
xbm: XBM, epoch: int, args: argparse.Namespace):
# define loss function
criterion_ce = BridgeProbLoss(args.n_s_classes + args.n_t_classes).to(device)
criterion_triplet = TripletLoss(margin=args.margin).to(device)
criterion_triplet_xbm = TripletLossXBM(margin=args.margin).to(device)
criterion_bridge_feat = BridgeFeatLoss().to(device)
criterion_diverse = DivLoss().to(device)
batch_time = AverageMeter('Time', ':4.2f')
data_time = AverageMeter('Data', ':3.1f')
losses_ce = AverageMeter('CeLoss', ':3.2f')
losses_triplet = AverageMeter('TripletLoss', ':3.2f')
losses_triplet_xbm = AverageMeter('XBMTripletLoss', ':3.2f')
losses_bridge_prob = AverageMeter('BridgeProbLoss', ':3.2f')
losses_bridge_feat = AverageMeter('BridgeFeatLoss', ':3.2f')
losses_diverse = AverageMeter('DiverseLoss', ':3.2f')
losses = AverageMeter('Loss', ':3.2f')
cls_accs_s = AverageMeter('Src Cls Acc', ':3.1f')
cls_accs_t = AverageMeter('Tgt Cls Acc', ':3.1f')
progress = ProgressMeter(
args.iters_per_epoch,
[batch_time, data_time, losses_ce, losses_triplet, losses_triplet_xbm, losses_bridge_prob, losses_bridge_feat,
losses_diverse, losses, cls_accs_s, cls_accs_t],
prefix="Epoch: [{}]".format(epoch))
# switch to train mode
model.train()
end = time.time()
for i in range(args.iters_per_epoch):
x_s, _, labels_s, _ = next(train_source_iter)
x_t, _, labels_t, _ = next(train_target_iter)
x_s = x_s.to(device)
x_t = x_t.to(device)
labels_s = labels_s.to(device)
labels_t = labels_t.to(device)
# measure data loading time
data_time.update(time.time() - end)
# arrange batch for domain-specific BN
device_num = torch.cuda.device_count()
B, C, H, W = x_s.size()
def reshape(tensor):
return tensor.view(device_num, -1, C, H, W)
x_s, x_t = reshape(x_s), reshape(x_t)
x = torch.cat((x_s, x_t), 1).view(-1, C, H, W)
labels = torch.cat((labels_s.view(device_num, -1), labels_t.view(device_num, -1)), 1)
labels = labels.view(-1)
# compute output
y, f, attention_lam = model(x, stage=args.stage)
y = y[:, 0:args.n_s_classes + args.n_t_classes] # only (n_s_classes + n_t_classes) classes are meaningful
# split feats
ori_f = f.view(device_num, -1, f.size(-1))
f_s, f_t, f_mixed = ori_f.split(ori_f.size(1) // 3, dim=1)
ori_f = torch.cat((f_s, f_t), 1).view(-1, ori_f.size(-1))
# cross entropy loss
loss_ce, loss_bridge_prob = criterion_ce(y, labels, attention_lam[:, 0].detach(), device_num)
# triplet loss
loss_triplet = criterion_triplet(ori_f, labels)
# diverse loss
loss_diverse = criterion_diverse(attention_lam)
# bridge feature loss
f_s = f_s.contiguous().view(-1, f.size(-1))
f_t = f_t.contiguous().view(-1, f.size(-1))
f_mixed = f_mixed.contiguous().view(-1, f.size(-1))
loss_bridge_feat = criterion_bridge_feat(f_s, f_t, f_mixed, attention_lam)
# xbm triplet loss
xbm.enqueue_dequeue(ori_f.detach(), labels.detach())
xbm_f, xbm_labels = xbm.get()
loss_triplet_xbm = criterion_triplet_xbm(ori_f, labels, xbm_f, xbm_labels)
loss = (1. - args.mu1) * loss_ce + loss_triplet + loss_triplet_xbm + \
args.mu1 * loss_bridge_prob + args.mu2 * loss_bridge_feat + args.mu3 * loss_diverse
# compute gradient and do SGD step
optimizer.zero_grad()
loss.backward()
optimizer.step()
ori_y = y.view(device_num, -1, y.size(-1))
y_s, y_t, _ = ori_y.split(ori_y.size(1) // 3, dim=1)
cls_acc_s = accuracy(y_s.reshape(-1, y_s.size(-1)), labels_s)[0]
cls_acc_t = accuracy(y_t.reshape(-1, y_t.size(-1)), labels_t)[0]
# update statistics
losses_ce.update(loss_ce.item(), x_s.size(0))
losses_triplet.update(loss_triplet.item(), x_s.size(0))
losses_triplet_xbm.update(loss_triplet_xbm.item(), x_s.size(0))
losses_bridge_prob.update(loss_bridge_prob.item(), x_s.size(0))
losses_bridge_feat.update(loss_bridge_feat.item(), x_s.size(0))
losses_diverse.update(loss_diverse.item(), x_s.size(0))
losses.update(loss.item(), x_s.size(0))
cls_accs_s.update(cls_acc_s.item(), x_s.size(0))
cls_accs_t.update(cls_acc_t.item(), x_s.size(0))
# measure elapsed time
batch_time.update(time.time() - end)
end = time.time()
if i % args.print_freq == 0:
progress.display(i)
if __name__ == '__main__':
dataset_names = sorted(
name for name in datasets.__dict__
if not name.startswith("__") and callable(datasets.__dict__[name])
)
architecture_names = sorted(
name for name in models.__dict__
if name.islower() and not name.startswith("__")
and callable(models.__dict__[name])
)
parser = argparse.ArgumentParser(description="IDM for Domain Adaptative ReID")
# dataset parameters
parser.add_argument('source_root', help='root path of the source dataset')
parser.add_argument('target_root', help='root path of the target dataset')
parser.add_argument('-s', '--source', type=str, help='source domain')
parser.add_argument('-t', '--target', type=str, help='target domain')
parser.add_argument('--train-resizing', type=str, default='default')
# model parameters
parser.add_argument('-a', '--arch', metavar='ARCH', default='reid_resnet50',
choices=architecture_names,
help='backbone architecture: ' +
' | '.join(architecture_names) +
' (default: reid_resnet50)')
parser.add_argument('--no-pool', action='store_true', help='no pool layer after the feature extractor.')
parser.add_argument('--finetune', action='store_true', help='whether use 10x smaller lr for backbone')
parser.add_argument('--rate', type=float, default=0.2)
parser.add_argument('--n-classes', type=int, default=1000, help='number of classes')
parser.add_argument('--n-s-classes', type=int, default=1000, help="number of source classes")
parser.add_argument('--n-t-classes', type=int, default=1000, help="number of target classes")
# training parameters
parser.add_argument('--resume', type=str, default=None, help="Where to restore model parameters from.")
parser.add_argument('--eps', type=float, default=0.6, help="max neighbor distance for DBSCAN")
parser.add_argument('--margin', type=float, default=0.3, help='margin for triplet loss')
parser.add_argument('--mu1', type=float, default=0.7, help="weight for prediction bridge loss")
parser.add_argument('--mu2', type=float, default=0.1, help="weight for feature bridge loss")
parser.add_argument('--mu3', type=float, default=1, help="weight for diverse loss")
parser.add_argument('--ratio', type=float, default=1, help='ratio of dataset to store in memory bank')
parser.add_argument('--stage', type=int, default=0, choices=range(5))
parser.add_argument('-j', '--workers', type=int, default=4)
parser.add_argument('-b', '--batch-size', type=int, default=64)
parser.add_argument('--height', type=int, default=256, help="input height")
parser.add_argument('--width', type=int, default=128, help="input width")
parser.add_argument('--num-instances', type=int, default=4,
help="each minibatch consist of "
"(batch_size // num_instances) identities, and "
"each identity has num_instances instances, "
"default: 4")
parser.add_argument('--lr', type=float, default=0.00035, help="learning rate")
parser.add_argument('--weight-decay', type=float, default=5e-4)
parser.add_argument('--epochs', type=int, default=50)
parser.add_argument('--step-size', type=int, default=20)
parser.add_argument('--start-epoch', default=0, type=int, help='start epoch')
parser.add_argument('--eval-step', type=int, default=1)
parser.add_argument('--iters-per-epoch', type=int, default=400)
parser.add_argument('--print-freq', type=int, default=40)
parser.add_argument('--seed', default=None, type=int, help='seed for initializing training.')
parser.add_argument('--rerank', action='store_true', help="evaluation only")
parser.add_argument("--log", type=str, default='idm',
help="Where to save logs, checkpoints and debugging images.")
parser.add_argument("--phase", type=str, default='train', choices=['train', 'test', 'analysis'],
help="When phase is 'test', only test the model."
"When phase is 'analysis', only analysis the model.")
args = parser.parse_args()
main(args)
================================================
FILE: domain_generalization/Transfer-Learning-Library/examples/domain_adaptation/re_identification/idm.sh
================================================
#!/usr/bin/env bash
# Market1501 -> Duke
CUDA_VISIBLE_DEVICES=0,1,2,3 python idm.py data data -s Market1501 -t DukeMTMC -a reid_resnet50 \
--seed 0 --log logs/idm/Market2Duke
# Duke -> Market1501
CUDA_VISIBLE_DEVICES=0,1,2,3 python idm.py data data -s DukeMTMC -t Market1501 -a reid_resnet50 \
--seed 0 --log logs/idm/Duke2Market
# Market1501 -> MSMT
CUDA_VISIBLE_DEVICES=0,1,2,3 python idm.py data data -s Market1501 -t MSMT17 -a reid_resnet50 \
--seed 0 --log logs/idm/Market2MSMT
# MSMT -> Market1501
CUDA_VISIBLE_DEVICES=0,1,2,3 python idm.py data data -s MSMT17 -t Market1501 -a reid_resnet50 \
--seed 0 --log logs/idm/MSMT2Market
# Duke -> MSMT
CUDA_VISIBLE_DEVICES=0,1,2,3 python idm.py data data -s DukeMTMC -t MSMT17 -a reid_resnet50 \
--seed 1 --log logs/idm/Duke2MSMT
# MSMT -> Duke
CUDA_VISIBLE_DEVICES=0,1,2,3 python idm.py data data -s MSMT17 -t DukeMTMC -a reid_resnet50 \
--seed 0 --log logs/idm/MSMT2Duke
================================================
FILE: domain_generalization/Transfer-Learning-Library/examples/domain_adaptation/re_identification/mmt.py
================================================
"""
@author: Baixu Chen
@contact: cbx_99_hasta@outlook.com
"""
import random
import time
import warnings
import sys
import argparse
import os.path as osp
import numpy as np
import torch
import torch.nn as nn
import torch.nn.functional as F
from torch.nn import DataParallel
import torch.backends.cudnn as cudnn
from torch.optim import Adam
from torch.utils.data import DataLoader
from sklearn.cluster import KMeans, DBSCAN
sys.path.append('../../..')
import common.vision.datasets.reid as datasets
from common.vision.datasets.reid.convert import convert_to_pytorch_dataset
from common.vision.models.reid.identifier import ReIdentifier
from common.vision.models.reid.loss import CrossEntropyLossWithLabelSmooth, SoftTripletLoss, CrossEntropyLoss
from dalib.adaptation.self_ensemble import EmaTeacher
from common.vision.transforms import MultipleApply
from common.utils.metric.reid import extract_reid_feature, validate, visualize_ranked_results
from common.utils.data import ForeverDataIterator, RandomMultipleGallerySampler
from common.utils.metric import accuracy
from common.utils.meter import AverageMeter, ProgressMeter
from common.utils.logger import CompleteLogger
sys.path.append('.')
import utils
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
def main(args: argparse.Namespace):
logger = CompleteLogger(args.log, args.phase)
print(args)
if args.seed is not None:
random.seed(args.seed)
np.random.seed(args.seed)
torch.manual_seed(args.seed)
cudnn.deterministic = True
warnings.warn('You have chosen to seed training. '
'This will turn on the CUDNN deterministic setting, '
'which can slow down your training considerably! '
'You may see unexpected behavior when restarting '
'from checkpoints.')
cudnn.benchmark = True
# Data loading code
train_transform = utils.get_train_transform(args.height, args.width, args.train_resizing,
random_horizontal_flip=True, random_color_jitter=False,
random_gray_scale=False, random_erasing=True)
val_transform = utils.get_val_transform(args.height, args.width)
print("train_transform: ", train_transform)
print("val_transform: ", val_transform)
working_dir = osp.dirname(osp.abspath(__file__))
source_root = osp.join(working_dir, args.source_root)
target_root = osp.join(working_dir, args.target_root)
# source dataset
source_dataset = datasets.__dict__[args.source](root=osp.join(source_root, args.source.lower()))
val_loader = DataLoader(
convert_to_pytorch_dataset(list(set(source_dataset.query) | set(source_dataset.gallery)),
root=source_dataset.images_dir,
transform=val_transform),
batch_size=args.batch_size, num_workers=args.workers, shuffle=False, pin_memory=True)
# target dataset
target_dataset = datasets.__dict__[args.target](root=osp.join(target_root, args.target.lower()))
cluster_loader = DataLoader(
convert_to_pytorch_dataset(target_dataset.train, root=target_dataset.images_dir, transform=val_transform),
batch_size=args.batch_size, num_workers=args.workers, shuffle=False, pin_memory=True)
test_loader = DataLoader(
convert_to_pytorch_dataset(list(set(target_dataset.query) | set(target_dataset.gallery)),
root=target_dataset.images_dir, transform=val_transform),
batch_size=args.batch_size, num_workers=args.workers, shuffle=False, pin_memory=True)
# create model
model_1, model_1_ema = create_model(args, args.pretrained_model_1_path)
model_2, model_2_ema = create_model(args, args.pretrained_model_2_path)
# resume from the best checkpoint
if args.phase != 'train':
checkpoint = torch.load(logger.get_checkpoint_path('best'), map_location='cpu')
utils.copy_state_dict(model_1_ema, checkpoint)
# analysis the model
if args.phase == 'analysis':
# plot t-SNE
utils.visualize_tsne(source_loader=val_loader, target_loader=test_loader, model=model_1_ema,
filename=osp.join(logger.visualize_directory, 'analysis', 'TSNE.pdf'), device=device)
# visualize ranked results
visualize_ranked_results(test_loader, model_1_ema, target_dataset.query, target_dataset.gallery, device,
visualize_dir=logger.visualize_directory, width=args.width, height=args.height,
rerank=args.rerank)
return
if args.phase == 'test':
print("Test on Source domain:")
validate(val_loader, model_1_ema, source_dataset.query, source_dataset.gallery, device, cmc_flag=True,
rerank=args.rerank)
print("Test on target domain:")
validate(test_loader, model_1_ema, target_dataset.query, target_dataset.gallery, device, cmc_flag=True,
rerank=args.rerank)
return
# define loss function
num_classes = args.num_clusters
criterion_ce = CrossEntropyLossWithLabelSmooth(num_classes).to(device)
criterion_ce_soft = CrossEntropyLoss().to(device)
criterion_triplet = SoftTripletLoss(margin=0.0).to(device)
criterion_triplet_soft = SoftTripletLoss(margin=None).to(device)
# optionally resume from a checkpoint
if args.resume:
checkpoint = torch.load(args.resume, map_location='cpu')
utils.copy_state_dict(model_1, checkpoint['model_1'])
utils.copy_state_dict(model_1_ema, checkpoint['model_1_ema'])
utils.copy_state_dict(model_2, checkpoint['model_2'])
utils.copy_state_dict(model_2_ema, checkpoint['model_2_ema'])
args.start_epoch = checkpoint['epoch'] + 1
# start training
best_test_mAP = 0.
for epoch in range(args.start_epoch, args.epochs):
# run clustering algorithm and generate pseudo labels
if args.clustering_algorithm == 'kmeans':
train_target_iter = run_kmeans(cluster_loader, model_1, model_2, model_1_ema, model_2_ema, target_dataset,
train_transform, args)
elif args.clustering_algorithm == 'dbscan':
train_target_iter, num_classes = run_dbscan(cluster_loader, model_1, model_2, model_1_ema, model_2_ema,
target_dataset, train_transform, args)
# define cross entropy loss with current number of classes
criterion_ce = CrossEntropyLossWithLabelSmooth(num_classes).to(device)
# define optimizer
optimizer = Adam(model_1.module.get_parameters(base_lr=args.lr, rate=args.rate) + model_2.module.get_parameters(
base_lr=args.lr, rate=args.rate), args.lr, weight_decay=args.weight_decay)
# train for one epoch
train(train_target_iter, model_1, model_1_ema, model_2, model_2_ema, optimizer, criterion_ce, criterion_ce_soft,
criterion_triplet, criterion_triplet_soft, epoch, args)
if (epoch + 1) % args.eval_step == 0 or (epoch == args.epochs - 1):
# save checkpoint and remember best mAP
torch.save(
{
'model_1': model_1.state_dict(),
'model_1_ema': model_1_ema.state_dict(),
'model_2': model_2.state_dict(),
'model_2_ema': model_2_ema.state_dict(),
'epoch': epoch
}, logger.get_checkpoint_path(epoch)
)
print("Test model_1 on target domain...")
_, test_mAP_1 = validate(test_loader, model_1_ema, target_dataset.query, target_dataset.gallery,
device, cmc_flag=True, rerank=args.rerank)
print("Test model_2 on target domain...")
_, test_mAP_2 = validate(test_loader, model_2_ema, target_dataset.query, target_dataset.gallery,
device, cmc_flag=True, rerank=args.rerank)
if test_mAP_1 > test_mAP_2 and test_mAP_1 > best_test_mAP:
torch.save(model_1_ema.state_dict(), logger.get_checkpoint_path('best'))
best_test_mAP = test_mAP_1
if test_mAP_2 > test_mAP_1 and test_mAP_2 > best_test_mAP:
torch.save(model_2_ema.state_dict(), logger.get_checkpoint_path('best'))
best_test_mAP = test_mAP_2
print("best mAP on target = {}".format(best_test_mAP))
logger.close()
def create_model(args: argparse.Namespace, pretrained_model_path: str):
num_classes = args.num_clusters
backbone = utils.get_model(args.arch)
pool_layer = nn.Identity() if args.no_pool else None
model = ReIdentifier(backbone, num_classes, finetune=args.finetune, pool_layer=pool_layer).to(device)
model = DataParallel(model)
# load pretrained weights
pretrained_model = torch.load(pretrained_model_path)
utils.copy_state_dict(model, pretrained_model)
# EMA model
model_ema = EmaTeacher(model, args.alpha)
return model, model_ema
def run_kmeans(cluster_loader: DataLoader, model_1: DataParallel, model_2: DataParallel, model_1_ema: EmaTeacher,
model_2_ema: EmaTeacher, target_dataset, train_transform, args: argparse.Namespace):
# run kmeans clustering algorithm
print('Clustering into {} classes'.format(args.num_clusters))
# collect feature with different ema teachers
feature_dict_1 = extract_reid_feature(cluster_loader, model_1_ema, device, normalize=True)
feature_1 = torch.stack(list(feature_dict_1.values())).cpu().numpy()
feature_dict_2 = extract_reid_feature(cluster_loader, model_2_ema, device, normalize=True)
feature_2 = torch.stack(list(feature_dict_2.values())).cpu().numpy()
# average feature_1, feature_2 to create final feature
feature = (feature_1 + feature_2) / 2
km = KMeans(n_clusters=args.num_clusters, random_state=args.seed).fit(feature)
cluster_labels = km.labels_
cluster_centers = km.cluster_centers_
print('Clustering finished')
# normalize cluster centers and convert to pytorch tensor
cluster_centers = torch.from_numpy(cluster_centers).float().to(device)
cluster_centers = F.normalize(cluster_centers, dim=1)
# reinitialize classifier head
model_1.module.head.weight.data.copy_(cluster_centers)
model_2.module.head.weight.data.copy_(cluster_centers)
model_1_ema.module.head.weight.data.copy_(cluster_centers)
model_2_ema.module.head.weight.data.copy_(cluster_centers)
# generate training set with pseudo labels
target_train_set = []
for (fname, _, cid), label in zip(target_dataset.train, cluster_labels):
target_train_set.append((fname, int(label), cid))
sampler = RandomMultipleGallerySampler(target_train_set, args.num_instances)
train_target_loader = DataLoader(
convert_to_pytorch_dataset(target_train_set, root=target_dataset.images_dir,
transform=MultipleApply([train_transform, train_transform])),
batch_size=args.batch_size, num_workers=args.workers, sampler=sampler, pin_memory=True, drop_last=True)
train_target_iter = ForeverDataIterator(train_target_loader)
return train_target_iter
def run_dbscan(cluster_loader: DataLoader, model_1: DataParallel, model_2: DataParallel, model_1_ema: EmaTeacher,
model_2_ema: EmaTeacher, target_dataset, train_transform, args: argparse.Namespace):
# run dbscan clustering algorithm
# collect feature with different ema teachers
feature_dict_1 = extract_reid_feature(cluster_loader, model_1_ema, device, normalize=True)
feature_1 = torch.stack(list(feature_dict_1.values())).cpu()
feature_dict_2 = extract_reid_feature(cluster_loader, model_2_ema, device, normalize=True)
feature_2 = torch.stack(list(feature_dict_2.values())).cpu()
# average feature_1, feature_2 to create final feature
feature = (feature_1 + feature_2) / 2
feature = F.normalize(feature, dim=1)
rerank_dist = utils.compute_rerank_dist(feature).numpy()
print('Clustering with dbscan algorithm')
dbscan = DBSCAN(eps=0.7, min_samples=4, metric='precomputed', n_jobs=-1)
cluster_labels = dbscan.fit_predict(rerank_dist)
print('Clustering finished')
# generate training set with pseudo labels and calculate cluster centers
target_train_set = []
cluster_centers = {}
for i, ((fname, _, cid), label) in enumerate(zip(target_dataset.train, cluster_labels)):
if label == -1:
continue
target_train_set.append((fname, label, cid))
if label not in cluster_centers:
cluster_centers[label] = []
cluster_centers[label].append(feature[i])
cluster_centers = [torch.stack(cluster_centers[idx]).mean(0) for idx in sorted(cluster_centers.keys())]
cluster_centers = torch.stack(cluster_centers)
# normalize cluster centers
cluster_centers = F.normalize(cluster_centers, dim=1).float().to(device)
# reinitialize classifier head
features_dim = model_1.module.features_dim
num_clusters = len(set(cluster_labels)) - (1 if -1 in cluster_labels else 0)
model_1.module.head = nn.Linear(features_dim, num_clusters, bias=False).to(device)
model_2.module.head = nn.Linear(features_dim, num_clusters, bias=False).to(device)
model_1_ema.module.head = nn.Linear(features_dim, num_clusters, bias=False).to(device)
model_2_ema.module.head = nn.Linear(features_dim, num_clusters, bias=False).to(device)
model_1.module.head.weight.data.copy_(cluster_centers)
model_2.module.head.weight.data.copy_(cluster_centers)
model_1_ema.module.head.weight.data.copy_(cluster_centers)
model_2_ema.module.head.weight.data.copy_(cluster_centers)
sampler = RandomMultipleGallerySampler(target_train_set, args.num_instances)
train_target_loader = DataLoader(
convert_to_pytorch_dataset(target_train_set, root=target_dataset.images_dir,
transform=MultipleApply([train_transform, train_transform])),
batch_size=args.batch_size, num_workers=args.workers, sampler=sampler, pin_memory=True, drop_last=True)
train_target_iter = ForeverDataIterator(train_target_loader)
return train_target_iter, num_clusters
def train(train_target_iter: ForeverDataIterator, model_1: DataParallel, model_1_ema: EmaTeacher, model_2: DataParallel,
model_2_ema: EmaTeacher, optimizer: Adam, criterion_ce: CrossEntropyLossWithLabelSmooth,
criterion_ce_soft: CrossEntropyLoss, criterion_triplet: SoftTripletLoss,
criterion_triplet_soft: SoftTripletLoss, epoch: int, args: argparse.Namespace):
# train with pseudo labels
batch_time = AverageMeter('Time', ':4.2f')
data_time = AverageMeter('Data', ':3.1f')
# statistics for model_1
losses_ce_1 = AverageMeter('Model_1 CELoss', ':3.2f')
losses_triplet_1 = AverageMeter('Model_1 TripletLoss', ':3.2f')
cls_accs_1 = AverageMeter('Model_1 Cls Acc', ':3.1f')
# statistics for model_2
losses_ce_2 = AverageMeter('Model_2 CELoss', ':3.2f')
losses_triplet_2 = AverageMeter('Model_2 TripletLoss', ':3.2f')
cls_accs_2 = AverageMeter('Model_2 Cls Acc', ':3.1f')
losses_ce_soft = AverageMeter('Soft CELoss', ':3.2f')
losses_triplet_soft = AverageMeter('Soft TripletLoss', ':3.2f')
losses = AverageMeter('Loss', ':3.2f')
progress = ProgressMeter(
args.iters_per_epoch,
[batch_time, data_time, losses_ce_1, losses_triplet_1, cls_accs_1, losses_ce_2, losses_triplet_2, cls_accs_2,
losses_ce_soft, losses_triplet_soft, losses],
prefix="Epoch: [{}]".format(epoch))
# switch to train mode
model_1.train()
model_2.train()
model_1_ema.train()
model_2_ema.train()
end = time.time()
for i in range(args.iters_per_epoch):
# below we ignore subscript `t` and use `x_1`, `x_2` to denote different augmented versions of origin samples
# `x_t` from target domain
(x_1, x_2), _, labels, _ = next(train_target_iter)
x_1 = x_1.to(device)
x_2 = x_2.to(device)
labels = labels.to(device)
# measure data loading time
data_time.update(time.time() - end)
# compute output
y_1, f_1 = model_1(x_1)
y_2, f_2 = model_2(x_2)
# compute output by ema-teacher
y_1_teacher, f_1_teacher = model_1_ema(x_1)
y_2_teacher, f_2_teacher = model_2_ema(x_2)
# cross entropy loss
loss_ce_1 = criterion_ce(y_1, labels)
loss_ce_2 = criterion_ce(y_2, labels)
# triplet loss
loss_triplet_1 = criterion_triplet(f_1, f_1, labels)
loss_triplet_2 = criterion_triplet(f_2, f_2, labels)
# soft cross entropy loss
loss_ce_soft = criterion_ce_soft(y_1, y_2_teacher) + \
criterion_ce_soft(y_2, y_1_teacher)
# soft triplet loss
loss_triplet_soft = criterion_triplet_soft(f_1, f_2_teacher, labels) + \
criterion_triplet_soft(f_2, f_1_teacher, labels)
# final objective
loss = (loss_ce_1 + loss_ce_2) * (1 - args.trade_off_ce_soft) + \
(loss_triplet_1 + loss_triplet_2) * (1 - args.trade_off_triplet_soft) + \
loss_ce_soft * args.trade_off_ce_soft + \
loss_triplet_soft * args.trade_off_triplet_soft
# update statistics
batch_size = args.batch_size
cls_acc_1 = accuracy(y_1, labels)[0]
cls_acc_2 = accuracy(y_2, labels)[0]
# model 1
losses_ce_1.update(loss_ce_1.item(), batch_size)
losses_triplet_1.update(loss_triplet_1.item(), batch_size)
cls_accs_1.update(cls_acc_1.item(), batch_size)
# model 2
losses_ce_2.update(loss_ce_2.item(), batch_size)
losses_triplet_2.update(loss_triplet_2.item(), batch_size)
cls_accs_2.update(cls_acc_2.item(), batch_size)
losses_ce_soft.update(loss_ce_soft.item(), batch_size)
losses_triplet_soft.update(loss_triplet_soft.item(), batch_size)
losses.update(loss.item(), batch_size)
# compute gradient and do SGD step
optimizer.zero_grad()
loss.backward()
optimizer.step()
# update teacher
global_step = epoch * args.iters_per_epoch + i + 1
model_1_ema.set_alpha(min(args.alpha, 1 - 1 / global_step))
model_2_ema.set_alpha(min(args.alpha, 1 - 1 / global_step))
model_1_ema.update()
model_2_ema.update()
# measure elapsed time
batch_time.update(time.time() - end)
end = time.time()
if i % args.print_freq == 0:
progress.display(i)
if __name__ == '__main__':
dataset_names = sorted(
name for name in datasets.__dict__
if not name.startswith("__") and callable(datasets.__dict__[name])
)
parser = argparse.ArgumentParser(description="MMT for Domain Adaptative ReID")
# dataset parameters
parser.add_argument('source_root', help='root path of the source dataset')
parser.add_argument('target_root', help='root path of the target dataset')
parser.add_argument('-s', '--source', type=str, help='source domain')
parser.add_argument('-t', '--target', type=str, help='target domain')
parser.add_argument('--train-resizing', type=str, default='default')
# model parameters
parser.add_argument('-a', '--arch', metavar='ARCH', default='reid_resnet50',
choices=utils.get_model_names(),
help='backbone architecture: ' +
' | '.join(utils.get_model_names()) +
' (default: reid_resnet50)')
parser.add_argument('--num-clusters', type=int, default=500)
parser.add_argument('--no-pool', action='store_true', help='no pool layer after the feature extractor.')
parser.add_argument('--alpha', type=float, default=0.999, help='ema alpha')
parser.add_argument('--finetune', action='store_true', help='whether use 10x smaller lr for backbone')
parser.add_argument('--rate', type=float, default=0.2)
# training parameters
parser.add_argument('--clustering-algorithm', type=str, default='dbscan', choices=['kmeans', 'dbscan'],
help='clustering algorithm to run, currently supported method: ["kmeans", "dbscan"]')
parser.add_argument('--resume', type=str, default=None,
help="Where restore model parameters from.")
parser.add_argument('--pretrained-model-1-path', type=str, help='path to pretrained (source-only) model_1')
parser.add_argument('--pretrained-model-2-path', type=str, help='path to pretrained (source-only) model_2')
parser.add_argument('--trade-off-ce-soft', type=float, default=0.5,
help='the trade off hyper parameter between cross entropy loss and soft cross entropy loss')
parser.add_argument('--trade-off-triplet-soft', type=float, default=0.8,
help='the trade off hyper parameter between triplet loss and soft triplet loss')
parser.add_argument('-j', '--workers', type=int, default=4)
parser.add_argument('-b', '--batch-size', type=int, default=64)
parser.add_argument('--height', type=int, default=256, help="input height")
parser.add_argument('--width', type=int, default=128, help="input width")
parser.add_argument('--num-instances', type=int, default=4,
help="each minibatch consist of "
"(batch_size // num_instances) identities, and "
"each identity has num_instances instances, "
"default: 4")
parser.add_argument('--lr', type=float, default=0.00035,
help="learning rate")
parser.add_argument('--weight-decay', type=float, default=5e-4)
parser.add_argument('--epochs', type=int, default=40)
parser.add_argument('--start-epoch', default=0, type=int, help='start epoch')
parser.add_argument('--eval-step', type=int, default=1)
parser.add_argument('--iters-per-epoch', type=int, default=400)
parser.add_argument('--print-freq', type=int, default=40)
parser.add_argument('--seed', default=None, type=int, help='seed for initializing training.')
parser.add_argument('--rerank', action='store_true', help="evaluation only")
parser.add_argument("--log", type=str, default='mmt',
help="Where to save logs, checkpoints and debugging images.")
parser.add_argument("--phase", type=str, default='train', choices=['train', 'test', 'analysis'],
help="When phase is 'test', only test the model."
"When phase is 'analysis', only analysis the model.")
args = parser.parse_args()
main(args)
================================================
FILE: domain_generalization/Transfer-Learning-Library/examples/domain_adaptation/re_identification/mmt.sh
================================================
#!/usr/bin/env bash
# Market1501 -> Duke
# step1: pretrain
CUDA_VISIBLE_DEVICES=0 python baseline.py data data -s Market1501 -t DukeMTMC -a reid_resnet50 \
--iters-per-epoch 800 --print-freq 80 --finetune --seed 0 --log logs/baseline/Market2DukeSeed0
CUDA_VISIBLE_DEVICES=0 python baseline.py data data -s Market1501 -t DukeMTMC -a reid_resnet50 \
--iters-per-epoch 800 --print-freq 80 --finetune --seed 1 --log logs/baseline/Market2DukeSeed1
# step2: train mmt
CUDA_VISIBLE_DEVICES=0,1,2,3 python mmt.py data data -s Market1501 -t DukeMTMC -a reid_resnet50 \
--pretrained-model-1-path logs/baseline/Market2DukeSeed0/checkpoints/best.pth \
--pretrained-model-2-path logs/baseline/Market2DukeSeed1/checkpoints/best.pth \
--finetune --seed 0 --log logs/mmt/Market2Duke
# Duke -> Market1501
# step1: pretrain
CUDA_VISIBLE_DEVICES=0 python baseline.py data data -s DukeMTMC -t Market1501 -a reid_resnet50 \
--iters-per-epoch 800 --print-freq 80 --finetune --seed 0 --log logs/baseline/Duke2MarketSeed0
CUDA_VISIBLE_DEVICES=0 python baseline.py data data -s DukeMTMC -t Market1501 -a reid_resnet50 \
--iters-per-epoch 800 --print-freq 80 --finetune --seed 1 --log logs/baseline/Duke2MarketSeed1
# step2: train mmt
CUDA_VISIBLE_DEVICES=0,1,2,3 python mmt.py data data -s DukeMTMC -t Market1501 -a reid_resnet50 \
--pretrained-model-1-path logs/baseline/Duke2MarketSeed0/checkpoints/best.pth \
--pretrained-model-2-path logs/baseline/Duke2MarketSeed1/checkpoints/best.pth \
--finetune --seed 0 --log logs/mmt/Duke2Market
# Market1501 -> MSMT
# step1: pretrain
CUDA_VISIBLE_DEVICES=0 python baseline.py data data -s Market1501 -t MSMT17 -a reid_resnet50 \
--iters-per-epoch 800 --print-freq 80 --finetune --seed 0 --log logs/baseline/Market2MSMTSeed0
CUDA_VISIBLE_DEVICES=0 python baseline.py data data -s Market1501 -t MSMT17 -a reid_resnet50 \
--iters-per-epoch 800 --print-freq 80 --finetune --seed 1 --log logs/baseline/Market2MSMTSeed1
# step2: train mmt
CUDA_VISIBLE_DEVICES=0,1,2,3 python mmt.py data data -s Market1501 -t MSMT17 -a reid_resnet50 \
--pretrained-model-1-path logs/baseline/Market2MSMTSeed0/checkpoints/best.pth \
--pretrained-model-2-path logs/baseline/Market2MSMTSeed1/checkpoints/best.pth \
--num-clusters 1000 --finetune --seed 0 --log logs/mmt/Market2MSMT
# MSMT -> Market1501
# step1: pretrain
CUDA_VISIBLE_DEVICES=0 python baseline.py data data -s MSMT17 -t Market1501 -a reid_resnet50 \
--iters-per-epoch 800 --print-freq 80 --finetune --seed 0 --log logs/baseline/MSMT2MarketSeed0
CUDA_VISIBLE_DEVICES=0 python baseline.py data data -s MSMT17 -t Market1501 -a reid_resnet50 \
--iters-per-epoch 800 --print-freq 80 --finetune --seed 1 --log logs/baseline/MSMT2MarketSeed1
# step2: train mmt
CUDA_VISIBLE_DEVICES=0,1,2,3 python mmt.py data data -s MSMT17 -t Market1501 -a reid_resnet50 \
--pretrained-model-1-path logs/baseline/MSMT2MarketSeed0/checkpoints/best.pth \
--pretrained-model-2-path logs/baseline/MSMT2MarketSeed1/checkpoints/best.pth \
--finetune --seed 0 --log logs/mmt/MSMT2Market
# Duke -> MSMT
# step1: pretrain
CUDA_VISIBLE_DEVICES=0 python baseline.py data data -s DukeMTMC -t MSMT17 -a reid_resnet50 \
--iters-per-epoch 800 --print-freq 80 --finetune --seed 0 --log logs/baseline/Duke2MSMTSeed0
CUDA_VISIBLE_DEVICES=0 python baseline.py data data -s DukeMTMC -t MSMT17 -a reid_resnet50 \
--iters-per-epoch 800 --print-freq 80 --finetune --seed 1 --log logs/baseline/Duke2MSMTSeed1
# step2: train mmt
CUDA_VISIBLE_DEVICES=0,1,2,3 python mmt.py data data -s DukeMTMC -t MSMT17 -a reid_resnet50 \
--pretrained-model-1-path logs/baseline/Duke2MSMTSeed0/checkpoints/best.pth \
--pretrained-model-2-path logs/baseline/Duke2MSMTSeed1/checkpoints/best.pth \
--num-clusters 1000 --finetune --seed 0 --log logs/mmt/Duke2MSMT
# MSMT -> Duke
# step1: pretrain
CUDA_VISIBLE_DEVICES=0 python baseline.py data data -s MSMT17 -t DukeMTMC -a reid_resnet50 \
--iters-per-epoch 800 --print-freq 80 --finetune --seed 0 --log logs/baseline/MSMT2DukeSeed0
CUDA_VISIBLE_DEVICES=0 python baseline.py data data -s MSMT17 -t DukeMTMC -a reid_resnet50 \
--iters-per-epoch 800 --print-freq 80 --finetune --seed 1 --log logs/baseline/MSMT2DukeSeed1
# step2: train mmt
CUDA_VISIBLE_DEVICES=0,1,2,3 python mmt.py data data -s MSMT17 -t DukeMTMC -a reid_resnet50 \
--pretrained-model-1-path logs/baseline/MSMT2DukeSeed0/checkpoints/best.pth \
--pretrained-model-2-path logs/baseline/MSMT2DukeSeed1/checkpoints/best.pth \
--finetune --seed 0 --log logs/mmt/MSMT2Duke
================================================
FILE: domain_generalization/Transfer-Learning-Library/examples/domain_adaptation/re_identification/requirements.txt
================================================
timm
opencv-python
================================================
FILE: domain_generalization/Transfer-Learning-Library/examples/domain_adaptation/re_identification/spgan.py
================================================
"""
@author: Baixu Chen
@contact: cbx_99_hasta@outlook.com
"""
import random
import time
import warnings
import sys
import argparse
import itertools
import os.path as osp
from PIL import Image
import torch
import torch.nn as nn
import torch.backends.cudnn as cudnn
from torch.optim import Adam
from torch.optim.lr_scheduler import LambdaLR
from torch.utils.data import DataLoader
import torchvision.transforms as T
sys.path.append('../../..')
import dalib.translation.cyclegan as cyclegan
import dalib.translation.spgan as spgan
from dalib.translation.cyclegan.util import ImagePool, set_requires_grad
import common.vision.datasets.reid as datasets
from common.vision.datasets.reid.convert import convert_to_pytorch_dataset
from common.vision.transforms import Denormalize
from common.utils.data import ForeverDataIterator
from common.utils.meter import AverageMeter, ProgressMeter
from common.utils.logger import CompleteLogger
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
def main(args):
logger = CompleteLogger(args.log, args.phase)
print(args)
if args.seed is not None:
random.seed(args.seed)
torch.manual_seed(args.seed)
cudnn.deterministic = True
warnings.warn('You have chosen to seed training. '
'This will turn on the CUDNN deterministic setting, '
'which can slow down your training considerably! '
'You may see unexpected behavior when restarting '
'from checkpoints.')
cudnn.benchmark = True
# Data loading code
train_transform = T.Compose([
T.Resize(args.load_size, Image.BICUBIC),
T.RandomCrop(args.input_size),
T.RandomHorizontalFlip(),
T.ToTensor(),
T.Normalize((0.5, 0.5, 0.5), (0.5, 0.5, 0.5))
])
working_dir = osp.dirname(osp.abspath(__file__))
root = osp.join(working_dir, args.root)
source_dataset = datasets.__dict__[args.source](root=osp.join(root, args.source.lower()))
train_source_loader = DataLoader(
convert_to_pytorch_dataset(source_dataset.train, root=source_dataset.images_dir, transform=train_transform),
batch_size=args.batch_size, num_workers=args.workers, shuffle=True, pin_memory=True, drop_last=True)
target_dataset = datasets.__dict__[args.target](root=osp.join(root, args.target.lower()))
train_target_loader = DataLoader(
convert_to_pytorch_dataset(target_dataset.train, root=target_dataset.images_dir, transform=train_transform),
batch_size=args.batch_size, num_workers=args.workers, shuffle=True, pin_memory=True, drop_last=True)
train_source_iter = ForeverDataIterator(train_source_loader)
train_target_iter = ForeverDataIterator(train_target_loader)
# define networks (generators, discriminators and siamese network)
netG_S2T = cyclegan.generator.__dict__[args.netG](ngf=args.ngf, norm=args.norm, use_dropout=False).to(device)
netG_T2S = cyclegan.generator.__dict__[args.netG](ngf=args.ngf, norm=args.norm, use_dropout=False).to(device)
netD_S = cyclegan.discriminator.__dict__[args.netD](ndf=args.ndf, norm=args.norm).to(device)
netD_T = cyclegan.discriminator.__dict__[args.netD](ndf=args.ndf, norm=args.norm).to(device)
siamese_net = spgan.SiameseNetwork(nsf=args.nsf).to(device)
# create image buffer to store previously generated images
fake_S_pool = ImagePool(args.pool_size)
fake_T_pool = ImagePool(args.pool_size)
# define optimizer and lr scheduler
optimizer_G = Adam(itertools.chain(netG_S2T.parameters(), netG_T2S.parameters()), lr=args.lr,
betas=(args.beta1, 0.999))
optimizer_D = Adam(itertools.chain(netD_S.parameters(), netD_T.parameters()), lr=args.lr, betas=(args.beta1, 0.999))
optimizer_siamese = Adam(siamese_net.parameters(), lr=args.lr, betas=(args.beta1, 0.999))
lr_decay_function = lambda epoch: 1.0 - max(0, epoch - args.epochs) / float(args.epochs_decay)
lr_scheduler_G = LambdaLR(optimizer_G, lr_lambda=lr_decay_function)
lr_scheduler_D = LambdaLR(optimizer_D, lr_lambda=lr_decay_function)
lr_scheduler_siamese = LambdaLR(optimizer_siamese, lr_lambda=lr_decay_function)
# optionally resume from a checkpoint
if args.resume:
print("Resume from", args.resume)
checkpoint = torch.load(args.resume, map_location='cpu')
netG_S2T.load_state_dict(checkpoint['netG_S2T'])
netG_T2S.load_state_dict(checkpoint['netG_T2S'])
netD_S.load_state_dict(checkpoint['netD_S'])
netD_T.load_state_dict(checkpoint['netD_T'])
siamese_net.load_state_dict(checkpoint['siamese_net'])
optimizer_G.load_state_dict(checkpoint['optimizer_G'])
optimizer_D.load_state_dict(checkpoint['optimizer_D'])
optimizer_siamese.load_state_dict(checkpoint['optimizer_siamese'])
lr_scheduler_G.load_state_dict(checkpoint['lr_scheduler_G'])
lr_scheduler_D.load_state_dict(checkpoint['lr_scheduler_D'])
lr_scheduler_siamese.load_state_dict(checkpoint['lr_scheduler_siamese'])
args.start_epoch = checkpoint['epoch'] + 1
if args.phase == 'test':
transform = T.Compose([
T.Resize(args.test_input_size, Image.BICUBIC),
cyclegan.transform.Translation(netG_S2T, device)
])
source_dataset.translate(transform, osp.join(args.translated_root, args.source.lower()))
return
# define loss function
criterion_gan = cyclegan.LeastSquaresGenerativeAdversarialLoss()
criterion_cycle = nn.L1Loss()
criterion_identity = nn.L1Loss()
criterion_contrastive = spgan.ContrastiveLoss(margin=args.margin)
# define visualization function
tensor_to_image = T.Compose([
Denormalize((0.5, 0.5, 0.5), (0.5, 0.5, 0.5)),
T.ToPILImage()
])
def visualize(image, name):
"""
Args:
image (tensor): image in shape 3 x H x W
name: name of the saving image
"""
tensor_to_image(image).save(logger.get_image_path("{}.png".format(name)))
# start training
for epoch in range(args.start_epoch, args.epochs + args.epochs_decay):
logger.set_epoch(epoch)
print(lr_scheduler_G.get_lr())
# train for one epoch
train(train_source_iter, train_target_iter, netG_S2T, netG_T2S, netD_S, netD_T, siamese_net,
criterion_gan, criterion_cycle, criterion_identity, criterion_contrastive,
optimizer_G, optimizer_D, optimizer_siamese,
fake_S_pool, fake_T_pool, epoch, visualize, args)
# update learning rates
lr_scheduler_G.step()
lr_scheduler_D.step()
lr_scheduler_siamese.step()
# save checkpoint
torch.save(
{
'netG_S2T': netG_S2T.state_dict(),
'netG_T2S': netG_T2S.state_dict(),
'netD_S': netD_S.state_dict(),
'netD_T': netD_T.state_dict(),
'siamese_net': siamese_net.state_dict(),
'optimizer_G': optimizer_G.state_dict(),
'optimizer_D': optimizer_D.state_dict(),
'optimizer_siamese': optimizer_siamese.state_dict(),
'lr_scheduler_G': lr_scheduler_G.state_dict(),
'lr_scheduler_D': lr_scheduler_D.state_dict(),
'lr_scheduler_siamese': lr_scheduler_siamese.state_dict(),
'epoch': epoch,
'args': args
}, logger.get_checkpoint_path(epoch)
)
if args.translated_root is not None:
transform = T.Compose([
T.Resize(args.test_input_size, Image.BICUBIC),
cyclegan.transform.Translation(netG_S2T, device)
])
source_dataset.translate(transform, osp.join(args.translated_root, args.source.lower()))
logger.close()
def train(train_source_iter: ForeverDataIterator, train_target_iter: ForeverDataIterator,
netG_S2T, netG_T2S, netD_S, netD_T, siamese_net: spgan.SiameseNetwork,
criterion_gan: cyclegan.LeastSquaresGenerativeAdversarialLoss,
criterion_cycle: nn.L1Loss, criterion_identity: nn.L1Loss,
criterion_contrastive: spgan.ContrastiveLoss,
optimizer_G: Adam, optimizer_D: Adam, optimizer_siamese: Adam,
fake_S_pool: ImagePool, fake_T_pool: ImagePool, epoch: int, visualize, args: argparse.Namespace):
batch_time = AverageMeter('Time', ':4.2f')
data_time = AverageMeter('Data', ':3.1f')
losses_G_S2T = AverageMeter('G_S2T', ':3.2f')
losses_G_T2S = AverageMeter('G_T2S', ':3.2f')
losses_D_S = AverageMeter('D_S', ':3.2f')
losses_D_T = AverageMeter('D_T', ':3.2f')
losses_cycle_S = AverageMeter('cycle_S', ':3.2f')
losses_cycle_T = AverageMeter('cycle_T', ':3.2f')
losses_identity_S = AverageMeter('idt_S', ':3.2f')
losses_identity_T = AverageMeter('idt_T', ':3.2f')
losses_contrastive_G = AverageMeter('contrastive_G', ':3.2f')
losses_contrastive_siamese = AverageMeter('contrastive_siamese', ':3.2f')
progress = ProgressMeter(
args.iters_per_epoch,
[batch_time, data_time, losses_G_S2T, losses_G_T2S, losses_D_S, losses_D_T,
losses_cycle_S, losses_cycle_T, losses_identity_S, losses_identity_T,
losses_contrastive_G, losses_contrastive_siamese],
prefix="Epoch: [{}]".format(epoch))
end = time.time()
for i in range(args.iters_per_epoch):
real_S, _, _, _ = next(train_source_iter)
real_T, _, _, _ = next(train_target_iter)
real_S = real_S.to(device)
real_T = real_T.to(device)
# measure data loading time
data_time.update(time.time() - end)
# Compute fake images and reconstruction images.
fake_T = netG_S2T(real_S)
rec_S = netG_T2S(fake_T)
fake_S = netG_T2S(real_T)
rec_T = netG_S2T(fake_S)
# ===============================================
# train the generators (every two iterations)
# ===============================================
if i % 2 == 0:
# save memory
set_requires_grad(netD_S, False)
set_requires_grad(netD_T, False)
set_requires_grad(siamese_net, False)
# GAN loss D_T(G_S2T(S))
loss_G_S2T = criterion_gan(netD_T(fake_T), real=True)
# GAN loss D_S(G_T2S(B))
loss_G_T2S = criterion_gan(netD_S(fake_S), real=True)
# Cycle loss || G_T2S(G_S2T(S)) - S||
loss_cycle_S = criterion_cycle(rec_S, real_S) * args.trade_off_cycle
# Cycle loss || G_S2T(G_T2S(T)) - T||
loss_cycle_T = criterion_cycle(rec_T, real_T) * args.trade_off_cycle
# Identity loss
# G_S2T should be identity if real_T is fed: ||G_S2T(real_T) - real_T||
identity_T = netG_S2T(real_T)
loss_identity_T = criterion_identity(identity_T, real_T) * args.trade_off_identity
# G_T2S should be identity if real_S is fed: ||G_T2S(real_S) - real_S||
identity_S = netG_T2S(real_S)
loss_identity_S = criterion_identity(identity_S, real_S) * args.trade_off_identity
# siamese network output
f_real_S = siamese_net(real_S)
f_fake_T = siamese_net(fake_T)
f_real_T = siamese_net(real_T)
f_fake_S = siamese_net(fake_S)
# positive pair
loss_contrastive_p_G = criterion_contrastive(f_real_S, f_fake_T, 0) + \
criterion_contrastive(f_real_T, f_fake_S, 0)
# negative pair
loss_contrastive_n_G = criterion_contrastive(f_fake_T, f_real_T, 1) + \
criterion_contrastive(f_fake_S, f_real_S, 1) + \
criterion_contrastive(f_real_S, f_real_T, 1)
# contrastive loss
loss_contrastive_G = (loss_contrastive_p_G + 0.5 * loss_contrastive_n_G) / 4 * args.trade_off_contrastive
# combined loss and calculate gradients
loss_G = loss_G_S2T + loss_G_T2S + loss_cycle_S + loss_cycle_T + loss_identity_S + loss_identity_T
if epoch > 1:
loss_G += loss_contrastive_G
netG_S2T.zero_grad()
netG_T2S.zero_grad()
loss_G.backward()
optimizer_G.step()
# update corresponding statistics
losses_G_S2T.update(loss_G_S2T.item(), real_S.size(0))
losses_G_T2S.update(loss_G_T2S.item(), real_S.size(0))
losses_cycle_S.update(loss_cycle_S.item(), real_S.size(0))
losses_cycle_T.update(loss_cycle_T.item(), real_S.size(0))
losses_identity_S.update(loss_identity_S.item(), real_S.size(0))
losses_identity_T.update(loss_identity_T.item(), real_S.size(0))
if epoch > 1:
losses_contrastive_G.update(loss_contrastive_G, real_S.size(0))
# ===============================================
# train the siamese network (when epoch > 0)
# ===============================================
if epoch > 0:
set_requires_grad(siamese_net, True)
# siamese network output
f_real_S = siamese_net(real_S)
f_fake_T = siamese_net(fake_T.detach())
f_real_T = siamese_net(real_T)
f_fake_S = siamese_net(fake_S.detach())
# positive pair
loss_contrastive_p_siamese = criterion_contrastive(f_real_S, f_fake_T, 0) + \
criterion_contrastive(f_real_T, f_fake_S, 0)
# negative pair
loss_contrastive_n_siamese = criterion_contrastive(f_real_S, f_real_T, 1)
# contrastive loss
loss_contrastive_siamese = (loss_contrastive_p_siamese + 2 * loss_contrastive_n_siamese) / 3
# update siamese network
siamese_net.zero_grad()
loss_contrastive_siamese.backward()
optimizer_siamese.step()
# update corresponding statistics
losses_contrastive_siamese.update(loss_contrastive_siamese, real_S.size(0))
# ===============================================
# train the discriminators
# ===============================================
set_requires_grad(netD_S, True)
set_requires_grad(netD_T, True)
# Calculate GAN loss for discriminator D_S
fake_S_ = fake_S_pool.query(fake_S.detach())
loss_D_S = 0.5 * (criterion_gan(netD_S(real_S), True) + criterion_gan(netD_S(fake_S_), False))
# Calculate GAN loss for discriminator D_T
fake_T_ = fake_T_pool.query(fake_T.detach())
loss_D_T = 0.5 * (criterion_gan(netD_T(real_T), True) + criterion_gan(netD_T(fake_T_), False))
# update discriminators
netD_S.zero_grad()
netD_T.zero_grad()
loss_D_S.backward()
loss_D_T.backward()
optimizer_D.step()
# update corresponding statistics
losses_D_S.update(loss_D_S.item(), real_S.size(0))
losses_D_T.update(loss_D_T.item(), real_S.size(0))
# measure elapsed time
batch_time.update(time.time() - end)
end = time.time()
if i % args.print_freq == 0:
progress.display(i)
for tensor, name in zip([real_S, real_T, fake_S, fake_T, rec_S, rec_T, identity_S, identity_T],
["real_S", "real_T", "fake_S", "fake_T", "rec_S",
"rec_T", "identity_S", "identity_T"]):
visualize(tensor[0], "{}_{}".format(i, name))
if __name__ == '__main__':
dataset_names = sorted(
name for name in datasets.__dict__
if not name.startswith("__") and callable(datasets.__dict__[name])
)
parser = argparse.ArgumentParser(description='SPGAN for Domain Adaptative ReID')
# dataset parameters
parser.add_argument('root', metavar='DIR',
help='root path of dataset')
parser.add_argument('-s', '--source', type=str, help='source domain')
parser.add_argument('-t', '--target', type=str, help='target domain')
parser.add_argument('--load-size', nargs='+', type=int, default=(286, 144), help='loading image size')
parser.add_argument('--input-size', nargs='+', type=int, default=(256, 128),
help='the input and output image size during training')
# model parameters
parser.add_argument('--ngf', type=int, default=64, help='# of gen filters in the last conv layer')
parser.add_argument('--ndf', type=int, default=64, help='# of discrim filters in the first conv layer')
parser.add_argument('--nsf', type=int, default=64, help='# of sianet filters int the first conv layer')
parser.add_argument('--netD', type=str, default='patch',
help='specify discriminator architecture [patch | pixel]. The basic model is a 70x70 PatchGAN.')
parser.add_argument('--netG', type=str, default='resnet_9',
help='specify generator architecture [resnet_9 | resnet_6 | unet_256 | unet_128]')
parser.add_argument('--norm', type=str, default='instance',
help='instance normalization or batch normalization [instance | batch | none]')
# training parameters
parser.add_argument("--resume", type=str, default=None,
help="Where restore model parameters from.")
parser.add_argument('--trade-off-cycle', type=float, default=10.0, help='trade off for cycle loss')
parser.add_argument('--trade-off-identity', type=float, default=5.0, help='trade off for identity loss')
parser.add_argument('--trade-off-contrastive', type=float, default=2.0, help='trade off for contrastive loss')
parser.add_argument('--margin', type=float, default=2,
help='margin for contrastive loss')
parser.add_argument('-b', '--batch-size', default=8, type=int,
metavar='N', help='mini-batch size (default: 8)')
parser.add_argument('--lr', type=float, default=0.0002, help='initial learning rate for adam')
parser.add_argument('--beta1', type=float, default=0.5, help='momentum term of adam')
parser.add_argument('-j', '--workers', default=4, type=int, metavar='N',
help='number of data loading workers (default: 4)')
parser.add_argument('--epochs', default=15, type=int, metavar='N',
help='number of total epochs to run')
parser.add_argument('--epochs-decay', type=int, default=15,
help='number of epochs to linearly decay learning rate to zero')
parser.add_argument('--start-epoch', default=0, type=int, metavar='N',
help='start epoch')
parser.add_argument('-i', '--iters-per-epoch', default=2000, type=int,
help='Number of iterations per epoch')
parser.add_argument('--pool-size', type=int, default=50,
help='the size of image buffer that stores previously generated images')
parser.add_argument('-p', '--print-freq', default=500, type=int,
metavar='N', help='print frequency (default: 500)')
parser.add_argument('--seed', default=None, type=int,
help='seed for initializing training. ')
parser.add_argument("--log", type=str, default='spgan',
help="Where to save logs, checkpoints and debugging images.")
# test parameters
parser.add_argument("--phase", type=str, default='train', choices=['train', 'test'],
help="When phase is 'test', only test the model.")
parser.add_argument('--translated-root', type=str, default=None,
help="The root to put the translated dataset")
parser.add_argument('--test-input-size', nargs='+', type=int, default=(256, 128),
help='the input image size during testing')
args = parser.parse_args()
main(args)
================================================
FILE: domain_generalization/Transfer-Learning-Library/examples/domain_adaptation/re_identification/spgan.sh
================================================
# Market1501 -> Duke
# step1: train SPGAN
CUDA_VISIBLE_DEVICES=0 python spgan.py data -s Market1501 -t DukeMTMC \
--log logs/spgan/Market2Duke --translated-root data/spganM2D --seed 0
# step2: train baseline on translated source dataset
CUDA_VISIBLE_DEVICES=0 python baseline.py data/spganM2D data -s Market1501 -t DukeMTMC -a reid_resnet50 \
--iters-per-epoch 800 --print-freq 80 --finetune --seed 0 --log logs/spgan/Market2Duke
# Duke -> Market1501
# step1: train SPGAN
CUDA_VISIBLE_DEVICES=0 python spgan.py data -s DukeMTMC -t Market1501 \
--log logs/spgan/Duke2Market --translated-root data/spganD2M --seed 0
# step2: train baseline on translated source dataset
CUDA_VISIBLE_DEVICES=0 python baseline.py data/spganD2M data -s DukeMTMC -t Market1501 -a reid_resnet50 \
--iters-per-epoch 800 --print-freq 80 --finetune --seed 0 --log logs/spgan/Duke2Market
# Market1501 -> MSMT17
# step1: train SPGAN
CUDA_VISIBLE_DEVICES=0 python spgan.py data -s Market1501 -t MSMT17 \
--log logs/spgan/Market2MSMT --translated-root data/spganM2S --seed 0
# step2: train baseline on translated source dataset
CUDA_VISIBLE_DEVICES=0 python baseline.py data/spganM2S data -s Market1501 -t MSMT17 -a reid_resnet50 \
--iters-per-epoch 800 --print-freq 80 --finetune --seed 0 --log logs/spgan/Market2MSMT
# MSMT -> Market1501
# step1: train SPGAN
CUDA_VISIBLE_DEVICES=0 python spgan.py data -s MSMT17 -t Market1501 \
--log logs/spgan/MSMT2Market --translated-root data/spganS2M --seed 0
# step2: train baseline on translated source dataset
CUDA_VISIBLE_DEVICES=0 python baseline.py data/spganS2M data -s MSMT17 -t Market1501 -a reid_resnet50 \
--iters-per-epoch 800 --print-freq 80 --finetune --seed 0 --log logs/spgan/MSMT2Market
# Duke -> MSMT
# step1: train SPGAN
CUDA_VISIBLE_DEVICES=0 python spgan.py data -s DukeMTMC -t MSMT17 \
--log logs/spgan/Duke2MSMT --translated-root data/spganD2S --seed 0
# step2: train baseline on translated source dataset
CUDA_VISIBLE_DEVICES=0 python baseline.py data/spganD2S data -s DukeMTMC -t MSMT17 -a reid_resnet50 \
--iters-per-epoch 800 --print-freq 80 --finetune --seed 0 --log logs/spgan/Duke2MSMT
# MSMT -> Duke
# step1: train SPGAN
CUDA_VISIBLE_DEVICES=0 python spgan.py data -s MSMT17 -t DukeMTMC \
--log logs/spgan/MSMT2Duke --translated-root data/spganS2D --seed 0
# step2: train baseline on translated source dataset
CUDA_VISIBLE_DEVICES=0 python baseline.py data/spganS2D data -s MSMT17 -t DukeMTMC -a reid_resnet50 \
--iters-per-epoch 800 --print-freq 80 --finetune --seed 0 --log logs/spgan/MSMT2Duke
================================================
FILE: domain_generalization/Transfer-Learning-Library/examples/domain_adaptation/re_identification/utils.py
================================================
"""
@author: Baixu Chen
@contact: cbx_99_hasta@outlook.com
"""
import sys
import timm
import numpy as np
import torch
import torch.nn as nn
from torch.nn import Parameter
import torchvision.transforms as T
sys.path.append('../../..')
from common.utils.metric.reid import extract_reid_feature
from common.utils.analysis import tsne
from common.vision.transforms import RandomErasing
import common.vision.models.reid as models
def copy_state_dict(model, state_dict, strip=None):
"""Copy state dict into the passed in ReID model. As we are using classification loss, which means we need to output
different number of classes(identities) for different datasets, we will not copy the parameters of last `fc` layer.
"""
tgt_state = model.state_dict()
copied_names = set()
for name, param in state_dict.items():
if strip is not None and name.startswith(strip):
name = name[len(strip):]
if name not in tgt_state:
continue
if isinstance(param, Parameter):
param = param.data
if param.size() != tgt_state[name].size():
print('mismatch:', name, param.size(), tgt_state[name].size())
continue
tgt_state[name].copy_(param)
copied_names.add(name)
missing = set(tgt_state.keys()) - copied_names
if len(missing) > 0:
print("missing keys in state_dict:", missing)
return model
def get_model_names():
return sorted(
name for name in models.__dict__
if name.islower() and not name.startswith("__")
and callable(models.__dict__[name])
) + timm.list_models()
def get_model(model_name):
if model_name in models.__dict__:
# load models from common.vision.models
backbone = models.__dict__[model_name](pretrained=True)
else:
# load models from pytorch-image-models
backbone = timm.create_model(model_name, pretrained=True)
try:
backbone.out_features = backbone.get_classifier().in_features
backbone.reset_classifier(0, '')
except:
backbone.out_features = backbone.head.in_features
backbone.head = nn.Identity()
return backbone
def get_train_transform(height, width, resizing='default', random_horizontal_flip=True, random_color_jitter=False,
random_gray_scale=False, random_erasing=False):
"""
resizing mode:
- default: resize the image to (height, width), zero-pad it by 10 on each size, the take a random crop of
(height, width)
- res: resize the image to(height, width)
"""
if resizing == 'default':
transform = T.Compose([
T.Resize((height, width), interpolation=3),
T.Pad(10),
T.RandomCrop((height, width))
])
elif resizing == 'res':
transform = T.Resize((height, width), interpolation=3)
else:
raise NotImplementedError(resizing)
transforms = [transform]
if random_horizontal_flip:
transforms.append(T.RandomHorizontalFlip())
if random_color_jitter:
transforms.append(T.ColorJitter(brightness=0.3, contrast=0.3, saturation=0.3, hue=0.3))
if random_gray_scale:
transforms.append(T.RandomGrayscale())
transforms.extend([
T.ToTensor(),
T.Normalize(mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225])
])
if random_erasing:
transforms.append(RandomErasing(probability=0.5, mean=[0.485, 0.456, 0.406]))
return T.Compose(transforms)
def get_val_transform(height, width):
return T.Compose([
T.Resize((height, width), interpolation=3),
T.ToTensor(),
T.Normalize(mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225])
])
def visualize_tsne(source_loader, target_loader, model, filename, device, n_data_points_per_domain=3000):
"""Visualize features from different domains using t-SNE. As we can have very large number of samples in each
domain, only `n_data_points_per_domain` number of samples are randomly selected in each domain.
"""
source_feature_dict = extract_reid_feature(source_loader, model, device, normalize=True)
source_feature = torch.stack(list(source_feature_dict.values())).cpu()
source_feature = source_feature[torch.randperm(len(source_feature))]
source_feature = source_feature[:n_data_points_per_domain]
target_feature_dict = extract_reid_feature(target_loader, model, device, normalize=True)
target_feature = torch.stack(list(target_feature_dict.values())).cpu()
target_feature = target_feature[torch.randperm(len(target_feature))]
target_feature = target_feature[:n_data_points_per_domain]
tsne.visualize(source_feature, target_feature, filename, source_color='cornflowerblue', target_color='darkorange')
print('T-SNE process is done, figure is saved to {}'.format(filename))
def k_reciprocal_neigh(initial_rank, i, k1):
"""Compute k-reciprocal neighbors of i-th sample. Two samples f_i, f_j are k reciprocal-neighbors if and only if
each one of them is among the k-nearest samples of another sample.
"""
forward_k_neigh_index = initial_rank[i, :k1 + 1]
backward_k_neigh_index = initial_rank[forward_k_neigh_index, :k1 + 1]
fi = torch.nonzero(backward_k_neigh_index == i)[:, 0]
return forward_k_neigh_index[fi]
def compute_rerank_dist(target_features, k1=30, k2=6):
"""Compute distance according to `Re-ranking Person Re-identification with k-reciprocal Encoding
(CVPR 2017) `_.
"""
n = target_features.size(0)
original_dist = torch.pow(target_features, 2).sum(dim=1, keepdim=True) * 2
original_dist = original_dist.expand(n, n) - 2 * torch.mm(target_features, target_features.t())
original_dist /= original_dist.max(0)[0]
original_dist = original_dist.t()
initial_rank = torch.argsort(original_dist, dim=-1)
all_num = gallery_num = original_dist.size(0)
del target_features
nn_k1 = []
nn_k1_half = []
for i in range(all_num):
nn_k1.append(k_reciprocal_neigh(initial_rank, i, k1))
nn_k1_half.append(k_reciprocal_neigh(initial_rank, i, int(np.around(k1 / 2))))
V = torch.zeros(all_num, all_num)
for i in range(all_num):
k_reciprocal_index = nn_k1[i]
k_reciprocal_expansion_index = k_reciprocal_index
for candidate in k_reciprocal_index:
candidate_k_reciprocal_index = nn_k1_half[candidate]
if (len(np.intersect1d(candidate_k_reciprocal_index, k_reciprocal_index)) > 2 / 3 * len(
candidate_k_reciprocal_index)):
k_reciprocal_expansion_index = torch.cat((k_reciprocal_expansion_index, candidate_k_reciprocal_index))
k_reciprocal_expansion_index = torch.unique(k_reciprocal_expansion_index)
weight = torch.exp(-original_dist[i, k_reciprocal_expansion_index])
V[i, k_reciprocal_expansion_index] = weight / torch.sum(weight)
if k2 != 1:
k2_rank = initial_rank[:, :k2].clone().view(-1)
V_qe = V[k2_rank]
V_qe = V_qe.view(initial_rank.size(0), k2, -1).sum(1)
V_qe /= k2
V = V_qe
del V_qe
del initial_rank
invIndex = []
for i in range(gallery_num):
invIndex.append(torch.nonzero(V[:, i])[:, 0])
jaccard_dist = torch.zeros_like(original_dist)
for i in range(all_num):
temp_min = torch.zeros(1, gallery_num)
indNonZero = torch.nonzero(V[i, :])[:, 0]
indImages = [invIndex[ind] for ind in indNonZero]
for j in range(len(indNonZero)):
temp_min[0, indImages[j]] = temp_min[0, indImages[j]] + \
torch.min(V[i, indNonZero[j]], V[indImages[j], indNonZero[j]])
jaccard_dist[i] = 1 - temp_min / (2 - temp_min)
del invIndex
del V
pos_bool = (jaccard_dist < 0)
jaccard_dist[pos_bool] = 0.0
return jaccard_dist
================================================
FILE: domain_generalization/Transfer-Learning-Library/examples/domain_adaptation/semantic_segmentation/README.md
================================================
# Unsupervised Domain Adaptation for Semantic Segmentation
## Dataset
You need to prepare following datasets manually if you want to use them:
- [Cityscapes](https://www.cityscapes-dataset.com/)
- [GTA5](https://download.visinf.tu-darmstadt.de/data/from_games/)
- [Synthia](https://synthia-dataset.net/)
and prepare them following [Documentations for Cityscapes](/common/vision/datasets/segmentation/cityscapes.py), [GTA5](/common/vision/datasets/segmentation/gta5.py) and [Synthia](/common/vision/datasets/segmentation/synthia.py),
## Supported Methods
Supported methods include:
- [Cycle-Consistent Adversarial Networks (CycleGAN)](https://arxiv.org/pdf/1703.10593.pdf)
- [CyCADA: Cycle-Consistent Adversarial Domain Adaptation](https://arxiv.org/abs/1711.03213)
- [Adversarial Entropy Minimization (ADVENT)](https://arxiv.org/abs/1811.12833)
- [Fourier Domain Adaptation (FDA)](https://arxiv.org/abs/2004.05498)
## Experiment and Results
The shell files give the script to reproduce the [benchmarks](/docs/dalib/benchmarks/semantic_segmentation.rst) with specified hyper-parameters.
For example, if you want to train ADVENT on Office31, use the following script
```shell script
# Train a ADVENT on GTA5 to Cityscapes.
# Assume you have put the datasets under the path `data/GTA5` and `data/Cityscapes`,
CUDA_VISIBLE_DEVICES=0 python advent.py data/GTA5 data/Cityscapes -s GTA5 -t Cityscapes \
--log logs/advent/gtav2cityscapes
```
For more information please refer to [Get Started](/docs/get_started/quickstart.rst) for help.
## TODO
Support methods: AdaptSeg
## Citation
If you use these methods in your research, please consider citing.
```
@inproceedings{CycleGAN,
title={Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks},
author={Zhu, Jun-Yan and Park, Taesung and Isola, Phillip and Efros, Alexei A},
booktitle={ICCV},
year={2017}
}
@inproceedings{cycada,
title={Cycada: Cycle-consistent adversarial domain adaptation},
author={Hoffman, Judy and Tzeng, Eric and Park, Taesung and Zhu, Jun-Yan and Isola, Phillip and Saenko, Kate and Efros, Alexei and Darrell, Trevor},
booktitle={ICML},
year={2018},
}
@inproceedings{Advent,
author = {Vu, Tuan-Hung and Jain, Himalaya and Bucher, Maxime and Cord, Matthieu and Perez, Patrick},
title = {ADVENT: Adversarial Entropy Minimization for Domain Adaptation in Semantic Segmentation},
booktitle = {CVPR},
year = {2019}
}
@inproceedings{FDA,
author = {Yanchao Yang and
Stefano Soatto},
title = {{FDA:} Fourier Domain Adaptation for Semantic Segmentation},
booktitle = {CVPR},
year = {2020}
}
```
================================================
FILE: domain_generalization/Transfer-Learning-Library/examples/domain_adaptation/semantic_segmentation/advent.py
================================================
"""
@author: Junguang Jiang
@contact: JiangJunguang1123@outlook.com
"""
import random
import time
import warnings
import sys
import argparse
from PIL import Image
import numpy as np
import shutil
import torch
import torch.nn as nn
import torch.backends.cudnn as cudnn
from torch.optim import SGD, Adam
from torch.optim.lr_scheduler import LambdaLR
from torch.utils.data import DataLoader
sys.path.append('../../..')
from dalib.adaptation.advent import Discriminator, DomainAdversarialEntropyLoss
import common.vision.models.segmentation as models
import common.vision.datasets.segmentation as datasets
import common.vision.transforms.segmentation as T
from common.vision.transforms import DeNormalizeAndTranspose
from common.utils.data import ForeverDataIterator
from common.utils.metric import ConfusionMatrix
from common.utils.meter import AverageMeter, ProgressMeter, Meter
from common.utils.logger import CompleteLogger
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
def main(args: argparse.Namespace):
logger = CompleteLogger(args.log, args.phase)
print(args)
if args.seed is not None:
random.seed(args.seed)
torch.manual_seed(args.seed)
cudnn.deterministic = True
warnings.warn('You have chosen to seed training. '
'This will turn on the CUDNN deterministic setting, '
'which can slow down your training considerably! '
'You may see unexpected behavior when restarting '
'from checkpoints.')
cudnn.benchmark = True
# Data loading code
source_dataset = datasets.__dict__[args.source]
train_source_dataset = source_dataset(
root=args.source_root,
transforms=T.Compose([
T.RandomResizedCrop(size=args.train_size, ratio=args.resize_ratio, scale=(0.5, 1.)),
T.ColorJitter(brightness=0.3, contrast=0.3),
T.RandomHorizontalFlip(),
T.NormalizeAndTranspose(),
]),
)
train_source_loader = DataLoader(train_source_dataset, batch_size=args.batch_size,
shuffle=True, num_workers=args.workers, pin_memory=True, drop_last=True)
target_dataset = datasets.__dict__[args.target]
train_target_dataset = target_dataset(
root=args.target_root,
transforms=T.Compose([
T.RandomResizedCrop(size=args.train_size, ratio=(2., 2.), scale=(0.5, 1.)),
T.RandomHorizontalFlip(),
T.NormalizeAndTranspose(),
]),
)
train_target_loader = DataLoader(train_target_dataset, batch_size=args.batch_size,
shuffle=True, num_workers=args.workers, pin_memory=True, drop_last=True)
val_target_dataset = target_dataset(
root=args.target_root, split='val',
transforms=T.Compose([
T.Resize(image_size=args.test_input_size, label_size=args.test_output_size),
T.NormalizeAndTranspose(),
]),
)
val_target_loader = DataLoader(val_target_dataset, batch_size=1, shuffle=False, pin_memory=True)
train_source_iter = ForeverDataIterator(train_source_loader)
train_target_iter = ForeverDataIterator(train_target_loader)
# create model
num_classes = train_source_dataset.num_classes
model = models.__dict__[args.arch](num_classes=num_classes).to(device)
discriminator = Discriminator(num_classes=num_classes).to(device)
# define optimizer and lr scheduler
optimizer = SGD(model.get_parameters(), lr=args.lr, momentum=args.momentum, weight_decay=args.weight_decay)
optimizer_d = Adam(discriminator.parameters(), lr=args.lr_d, betas=(0.9, 0.99))
lr_scheduler = LambdaLR(optimizer, lambda x: args.lr * (1. - float(x) / args.epochs / args.iters_per_epoch) ** (args.lr_power))
lr_scheduler_d = LambdaLR(optimizer_d, lambda x: (1. - float(x) / args.epochs / args.iters_per_epoch) ** (args.lr_power))
# optionally resume from a checkpoint
if args.resume:
checkpoint = torch.load(args.resume, map_location='cpu')
model.load_state_dict(checkpoint['model'])
discriminator.load_state_dict(checkpoint['discriminator'])
optimizer.load_state_dict(checkpoint['optimizer'])
lr_scheduler.load_state_dict(checkpoint['lr_scheduler'])
optimizer_d.load_state_dict(checkpoint['optimizer_d'])
lr_scheduler_d.load_state_dict(checkpoint['lr_scheduler_d'])
args.start_epoch = checkpoint['epoch'] + 1
# define loss function (criterion)
criterion = torch.nn.CrossEntropyLoss(ignore_index=args.ignore_label).to(device)
dann = DomainAdversarialEntropyLoss(discriminator)
interp_train = nn.Upsample(size=args.train_size[::-1], mode='bilinear', align_corners=True)
interp_val = nn.Upsample(size=args.test_output_size[::-1], mode='bilinear', align_corners=True)
# define visualization function
decode = train_source_dataset.decode_target
def visualize(image, pred, label, prefix):
"""
Args:
image (tensor): 3 x H x W
pred (tensor): C x H x W
label (tensor): H x W
prefix: prefix of the saving image
"""
image = image.detach().cpu().numpy()
pred = pred.detach().max(dim=0)[1].cpu().numpy()
label = label.cpu().numpy()
for tensor, name in [
(Image.fromarray(np.uint8(DeNormalizeAndTranspose()(image))), "image"),
(decode(label), "label"),
(decode(pred), "pred")
]:
tensor.save(logger.get_image_path("{}_{}.png".format(prefix, name)))
if args.phase == 'test':
confmat = validate(val_target_loader, model, interp_val, criterion, visualize, args)
print(confmat)
return
# start training
best_iou = 0.
for epoch in range(args.start_epoch, args.epochs):
logger.set_epoch(epoch)
print(lr_scheduler.get_lr(), lr_scheduler_d.get_lr())
# train for one epoch
train(train_source_iter, train_target_iter, model, interp_train, criterion, dann, optimizer,
lr_scheduler, optimizer_d, lr_scheduler_d, epoch, visualize if args.debug else None, args)
# evaluate on validation set
confmat = validate(val_target_loader, model, interp_val, criterion, None, args)
print(confmat.format(train_source_dataset.classes))
acc_global, acc, iu = confmat.compute()
# calculate the mean iou over partial classes
indexes = [train_source_dataset.classes.index(name) for name
in train_source_dataset.evaluate_classes]
iu = iu[indexes]
mean_iou = iu.mean()
# remember best acc@1 and save checkpoint
torch.save(
{
'model': model.state_dict(),
'discriminator': discriminator.state_dict(),
'optimizer': optimizer.state_dict(),
'optimizer_d': optimizer_d.state_dict(),
'lr_scheduler': lr_scheduler.state_dict(),
'lr_scheduler_d': lr_scheduler_d.state_dict(),
'epoch': epoch,
'args': args
}, logger.get_checkpoint_path(epoch)
)
if mean_iou > best_iou:
shutil.copy(logger.get_checkpoint_path(epoch), logger.get_checkpoint_path('best'))
best_iou = max(best_iou, mean_iou)
print("Target: {} Best: {}".format(mean_iou, best_iou))
logger.close()
def train(train_source_iter: ForeverDataIterator, train_target_iter: ForeverDataIterator,
model, interp, criterion, dann,
optimizer: SGD, lr_scheduler: LambdaLR, optimizer_d: SGD, lr_scheduler_d: LambdaLR,
epoch: int, visualize, args: argparse.Namespace):
batch_time = AverageMeter('Time', ':4.2f')
data_time = AverageMeter('Data', ':3.1f')
losses_s = AverageMeter('Loss (s)', ':3.2f')
losses_transfer = AverageMeter('Loss (transfer)', ':3.2f')
losses_discriminator = AverageMeter('Loss (discriminator)', ':3.2f')
accuracies_s = Meter('Acc (s)', ':3.2f')
accuracies_t = Meter('Acc (t)', ':3.2f')
iou_s = Meter('IoU (s)', ':3.2f')
iou_t = Meter('IoU (t)', ':3.2f')
confmat_s = ConfusionMatrix(model.num_classes)
confmat_t = ConfusionMatrix(model.num_classes)
progress = ProgressMeter(
args.iters_per_epoch,
[batch_time, data_time, losses_s, losses_transfer, losses_discriminator,
accuracies_s, accuracies_t, iou_s, iou_t],
prefix="Epoch: [{}]".format(epoch))
# switch to train mode
model.train()
end = time.time()
for i in range(args.iters_per_epoch):
x_s, label_s = next(train_source_iter)
x_t, label_t = next(train_target_iter)
x_s = x_s.to(device)
label_s = label_s.long().to(device)
x_t = x_t.to(device)
label_t = label_t.long().to(device)
# measure data loading time
data_time.update(time.time() - end)
optimizer.zero_grad()
optimizer_d.zero_grad()
# Step 1: Train the segmentation network, freeze the discriminator
dann.eval()
y_s = model(x_s)
pred_s = interp(y_s)
loss_cls_s = criterion(pred_s, label_s)
loss_cls_s.backward()
# adversarial training to fool the discriminator
y_t = model(x_t)
pred_t = interp(y_t)
loss_transfer = dann(pred_t, 'source')
(loss_transfer * args.trade_off).backward()
# Step 2: Train the discriminator
dann.train()
loss_discriminator = 0.5 * (dann(pred_s.detach(), 'source') + dann(pred_t.detach(), 'target'))
loss_discriminator.backward()
# compute gradient and do SGD step
optimizer.step()
optimizer_d.step()
lr_scheduler.step()
lr_scheduler_d.step()
# measure accuracy and record loss
losses_s.update(loss_cls_s.item(), x_s.size(0))
losses_transfer.update(loss_transfer.item(), x_s.size(0))
losses_discriminator.update(loss_discriminator.item(), x_s.size(0))
confmat_s.update(label_s.flatten(), pred_s.argmax(1).flatten())
confmat_t.update(label_t.flatten(), pred_t.argmax(1).flatten())
acc_global_s, acc_s, iu_s = confmat_s.compute()
acc_global_t, acc_t, iu_t = confmat_t.compute()
accuracies_s.update(acc_s.mean().item())
accuracies_t.update(acc_t.mean().item())
iou_s.update(iu_s.mean().item())
iou_t.update(iu_t.mean().item())
# measure elapsed time
batch_time.update(time.time() - end)
end = time.time()
if i % args.print_freq == 0:
progress.display(i)
if visualize is not None:
visualize(x_s[0], pred_s[0], label_s[0], "source_{}".format(i))
visualize(x_t[0], pred_t[0], label_t[0], "target_{}".format(i))
def validate(val_loader: DataLoader, model, interp, criterion, visualize, args: argparse.Namespace):
batch_time = AverageMeter('Time', ':6.3f')
losses = AverageMeter('Loss', ':.4e')
acc = Meter('Acc', ':3.2f')
iou = Meter('IoU', ':3.2f')
progress = ProgressMeter(
len(val_loader),
[batch_time, losses, acc, iou],
prefix='Test: ')
# switch to evaluate mode
model.eval()
confmat = ConfusionMatrix(model.num_classes)
with torch.no_grad():
end = time.time()
for i, (x, label) in enumerate(val_loader):
x = x.to(device)
label = label.long().to(device)
# compute output
output = interp(model(x))
loss = criterion(output, label)
# measure accuracy and record loss
losses.update(loss.item(), x.size(0))
confmat.update(label.flatten(), output.argmax(1).flatten())
acc_global, accs, iu = confmat.compute()
acc.update(accs.mean().item())
iou.update(iu.mean().item())
# measure elapsed time
batch_time.update(time.time() - end)
end = time.time()
if i % args.print_freq == 0:
progress.display(i)
if visualize is not None:
visualize(x[0], output[0], label[0], "val_{}".format(i))
return confmat
if __name__ == '__main__':
architecture_names = sorted(
name for name in models.__dict__
if name.islower() and not name.startswith("__")
and callable(models.__dict__[name])
)
dataset_names = sorted(
name for name in datasets.__dict__
if not name.startswith("__") and callable(datasets.__dict__[name])
)
parser = argparse.ArgumentParser(description='ADVENT for Segmentation Domain Adaptation')
# dataset parameters
parser.add_argument('source_root', help='root path of the source dataset')
parser.add_argument('target_root', help='root path of the target dataset')
parser.add_argument('-s', '--source', help='source domain(s)')
parser.add_argument('-t', '--target', help='target domain(s)')
parser.add_argument('--resize-ratio', nargs='+', type=float, default=(1.5, 8 / 3.),
help='the resize ratio for the random resize crop')
parser.add_argument('--train-size', nargs='+', type=int, default=(1024, 512),
help='the input and output image size during training')
parser.add_argument('--test-input-size', nargs='+', type=int, default=(1024, 512),
help='the input image size during test')
parser.add_argument('--test-output-size', nargs='+', type=int, default=(2048, 1024),
help='the output image size during test')
# model parameters
parser.add_argument('-a', '--arch', metavar='ARCH', default='deeplabv2_resnet101',
choices=architecture_names,
help='backbone architecture: ' +
' | '.join(architecture_names) +
' (default: deeplabv2_resnet101)')
parser.add_argument("--resume", type=str, default=None,
help="Where restore model parameters from.")
parser.add_argument('--trade-off', type=float, default=0.001,
help='trade-off parameter for the advent loss')
# training parameters
parser.add_argument('-b', '--batch-size', default=2, type=int,
metavar='N',
help='mini-batch size (default: 2)')
parser.add_argument('--lr', '--learning-rate', default=2.5e-3, type=float,
metavar='LR', help='initial learning rate', dest='lr')
parser.add_argument("--momentum", type=float, default=0.9, help="Momentum component of the optimiser.")
parser.add_argument("--weight-decay", type=float, default=0.0005, help="Regularisation parameter for L2-loss.")
parser.add_argument("--lr-power", type=float, default=0.9,
help="Decay parameter to compute the learning rate (only for deeplab).")
parser.add_argument("--lr-d", default=1e-4, type=float,
metavar='LR', help='initial learning rate for discriminator')
parser.add_argument('-j', '--workers', default=4, type=int, metavar='N',
help='number of data loading workers (default: 4)')
parser.add_argument('--epochs', default=60, type=int, metavar='N',
help='number of total epochs to run')
parser.add_argument('--start-epoch', default=0, type=int, metavar='N',
help='start epoch')
parser.add_argument('-i', '--iters-per-epoch', default=2500, type=int,
help='Number of iterations per epoch')
parser.add_argument('-p', '--print-freq', default=100, type=int,
metavar='N', help='print frequency (default: 100)')
parser.add_argument('--seed', default=None, type=int,
help='seed for initializing training. ')
parser.add_argument("--ignore-label", type=int, default=255,
help="The index of the label to ignore during the training.")
parser.add_argument("--log", type=str, default='advent',
help="Where to save logs, checkpoints and debugging images.")
parser.add_argument("--phase", type=str, default='train', choices=['train', 'test'],
help="When phase is 'test', only test the model.")
parser.add_argument('--debug', action="store_true",
help='In the debug mode, save images and predictions during training')
args = parser.parse_args()
main(args)
================================================
FILE: domain_generalization/Transfer-Learning-Library/examples/domain_adaptation/semantic_segmentation/advent.sh
================================================
# GTA5 to Cityscapes
CUDA_VISIBLE_DEVICES=0 python advent.py data/GTA5 data/Cityscapes -s GTA5 -t Cityscapes \
--log logs/advent/gtav2cityscapes
# Synthia to Cityscapes
CUDA_VISIBLE_DEVICES=0 python advent.py data/synthia data/Cityscapes -s Synthia -t Cityscapes \
--log logs/advent/synthia2cityscapes
# Cityscapes to Foggy
CUDA_VISIBLE_DEVICES=0 python advent.py data/Cityscapes data/Cityscapes -s Cityscapes -t FoggyCityscapes \
--log logs/advent/cityscapes2foggy
================================================
FILE: domain_generalization/Transfer-Learning-Library/examples/domain_adaptation/semantic_segmentation/cycada.py
================================================
"""
@author: Junguang Jiang
@contact: JiangJunguang1123@outlook.com
"""
import random
import time
import warnings
import sys
import argparse
import itertools
import torch
import torch.nn as nn
import torch.backends.cudnn as cudnn
from torch.optim import Adam
from torch.optim.lr_scheduler import LambdaLR
from torch.utils.data import DataLoader
from torchvision.transforms import ToPILImage, Compose, Lambda
sys.path.append('../../..')
import dalib.translation.cyclegan as cyclegan
from dalib.translation.cyclegan.util import ImagePool, set_requires_grad
from dalib.translation.cycada import SemanticConsistency
import common.vision.models.segmentation as models
import common.vision.datasets.segmentation as datasets
from common.vision.transforms import Denormalize, NormalizeAndTranspose
import common.vision.transforms.segmentation as T
from common.utils.data import ForeverDataIterator
from common.utils.meter import AverageMeter, ProgressMeter
from common.utils.logger import CompleteLogger
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
def main(args):
logger = CompleteLogger(args.log, args.phase)
print(args)
if args.seed is not None:
random.seed(args.seed)
torch.manual_seed(args.seed)
cudnn.deterministic = True
warnings.warn('You have chosen to seed training. '
'This will turn on the CUDNN deterministic setting, '
'which can slow down your training considerably! '
'You may see unexpected behavior when restarting '
'from checkpoints.')
cudnn.benchmark = True
# Data loading code
train_transform = T.Compose([
T.RandomResizedCrop(size=args.train_size, ratio=args.resize_ratio, scale=(0.5, 1.)),
T.RandomHorizontalFlip(),
T.ToTensor(),
T.Normalize((0.5, 0.5, 0.5), (0.5, 0.5, 0.5))
])
source_dataset = datasets.__dict__[args.source]
train_source_dataset = source_dataset(root=args.source_root, transforms=train_transform)
train_source_loader = DataLoader(train_source_dataset, batch_size=args.batch_size,
shuffle=True, num_workers=args.workers, pin_memory=True, drop_last=True)
target_dataset = datasets.__dict__[args.target]
train_target_dataset = target_dataset(root=args.target_root, transforms=train_transform)
train_target_loader = DataLoader(train_target_dataset, batch_size=args.batch_size,
shuffle=True, num_workers=args.workers, pin_memory=True, drop_last=True)
train_source_iter = ForeverDataIterator(train_source_loader)
train_target_iter = ForeverDataIterator(train_target_loader)
# define networks (both generators and discriminators)
netG_S2T = cyclegan.generator.__dict__[args.netG](ngf=args.ngf, norm=args.norm, use_dropout=False).to(device)
netG_T2S = cyclegan.generator.__dict__[args.netG](ngf=args.ngf, norm=args.norm, use_dropout=False).to(device)
netD_S = cyclegan.discriminator.__dict__[args.netD](ndf=args.ndf, norm=args.norm).to(device)
netD_T = cyclegan.discriminator.__dict__[args.netD](ndf=args.ndf, norm=args.norm).to(device)
# create image buffer to store previously generated images
fake_S_pool = ImagePool(args.pool_size)
fake_T_pool = ImagePool(args.pool_size)
# define optimizer and lr scheduler
optimizer_G = Adam(itertools.chain(netG_S2T.parameters(), netG_T2S.parameters()), lr=args.lr, betas=(args.beta1, 0.999))
optimizer_D = Adam(itertools.chain(netD_S.parameters(), netD_T.parameters()), lr=args.lr, betas=(args.beta1, 0.999))
lr_decay_function = lambda epoch: 1.0 - max(0, epoch - args.epochs) / float(args.epochs_decay)
lr_scheduler_G = LambdaLR(optimizer_G, lr_lambda=lr_decay_function)
lr_scheduler_D = LambdaLR(optimizer_D, lr_lambda=lr_decay_function)
# optionally resume from a checkpoint
if args.resume:
print("Resume from", args.resume)
checkpoint = torch.load(args.resume, map_location='cpu')
netG_S2T.load_state_dict(checkpoint['netG_S2T'])
netG_T2S.load_state_dict(checkpoint['netG_T2S'])
netD_S.load_state_dict(checkpoint['netD_S'])
netD_T.load_state_dict(checkpoint['netD_T'])
optimizer_G.load_state_dict(checkpoint['optimizer_G'])
optimizer_D.load_state_dict(checkpoint['optimizer_D'])
lr_scheduler_G.load_state_dict(checkpoint['lr_scheduler_G'])
lr_scheduler_D.load_state_dict(checkpoint['lr_scheduler_D'])
args.start_epoch = checkpoint['epoch'] + 1
if args.phase == 'test':
transform = T.Compose([
T.Resize(image_size=args.test_input_size),
T.wrapper(cyclegan.transform.Translation)(netG_S2T, device),
])
train_source_dataset.translate(transform, args.translated_root)
return
# define loss function
criterion_gan = cyclegan.LeastSquaresGenerativeAdversarialLoss()
criterion_cycle = nn.L1Loss()
criterion_identity = nn.L1Loss()
criterion_semantic = SemanticConsistency(ignore_index=[args.ignore_label]+train_source_dataset.ignore_classes).to(device)
interp_train = nn.Upsample(size=args.train_size[::-1], mode='bilinear', align_corners=True)
# define segmentation model and predict function
model = models.__dict__[args.arch](num_classes=train_source_dataset.num_classes).to(device)
if args.pretrain:
print("Loading pretrain segmentation model from", args.pretrain)
checkpoint = torch.load(args.pretrain, map_location='cpu')
model.load_state_dict(checkpoint['model'])
model.eval()
cycle_gan_tensor_to_segmentation_tensor = Compose([
Denormalize((0.5, 0.5, 0.5), (0.5, 0.5, 0.5)),
Lambda(lambda image: image.mul(255).permute((1, 2, 0))),
NormalizeAndTranspose(),
])
def predict(image):
image = cycle_gan_tensor_to_segmentation_tensor(image.squeeze())
image = image.unsqueeze(dim=0).to(device)
prediction = model(image)
return interp_train(prediction)
# define visualization function
tensor_to_image = Compose([
Denormalize((0.5, 0.5, 0.5), (0.5, 0.5, 0.5)),
ToPILImage()
])
decode = train_source_dataset.decode_target
def visualize(image, name, pred=None):
"""
Args:
image (tensor): image in shape 3 x H x W
name: name of the saving image
pred (tensor): predictions in shape C x H x W
"""
tensor_to_image(image).save(logger.get_image_path("{}.png".format(name)))
if pred is not None:
pred = pred.detach().max(dim=0).indices.cpu().numpy()
pred = decode(pred)
pred.save(logger.get_image_path("pred_{}.png".format(name)))
# start training
for epoch in range(args.start_epoch, args.epochs+args.epochs_decay):
logger.set_epoch(epoch)
print(lr_scheduler_G.get_lr())
# train for one epoch
train(train_source_iter, train_target_iter, netG_S2T, netG_T2S, netD_S, netD_T, predict,
criterion_gan, criterion_cycle, criterion_identity, criterion_semantic, optimizer_G, optimizer_D,
fake_S_pool, fake_T_pool, epoch, visualize, args)
# update learning rates
lr_scheduler_G.step()
lr_scheduler_D.step()
# save checkpoint
torch.save(
{
'netG_S2T': netG_S2T.state_dict(),
'netG_T2S': netG_T2S.state_dict(),
'netD_S': netD_S.state_dict(),
'netD_T': netD_T.state_dict(),
'optimizer_G': optimizer_G.state_dict(),
'optimizer_D': optimizer_D.state_dict(),
'lr_scheduler_G': lr_scheduler_G.state_dict(),
'lr_scheduler_D': lr_scheduler_D.state_dict(),
'epoch': epoch,
'args': args
}, logger.get_checkpoint_path(epoch)
)
if args.translated_root is not None:
transform = T.Compose([
T.Resize(image_size=args.test_input_size),
T.wrapper(cyclegan.transform.Translation)(netG_S2T, device),
])
train_source_dataset.translate(transform, args.translated_root)
logger.close()
def train(train_source_iter, train_target_iter, netG_S2T, netG_T2S, netD_S, netD_T, predict,
criterion_gan, criterion_cycle, criterion_identity, criterion_semantic,
optimizer_G, optimizer_D, fake_S_pool, fake_T_pool,
epoch: int, visualize, args: argparse.Namespace):
batch_time = AverageMeter('Time', ':4.2f')
data_time = AverageMeter('Data', ':3.1f')
losses_G_S2T = AverageMeter('G_S2T', ':3.2f')
losses_G_T2S = AverageMeter('G_T2S', ':3.2f')
losses_D_S = AverageMeter('D_S', ':3.2f')
losses_D_T = AverageMeter('D_T', ':3.2f')
losses_cycle_S = AverageMeter('cycle_S', ':3.2f')
losses_cycle_T = AverageMeter('cycle_T', ':3.2f')
losses_identity_S = AverageMeter('idt_S', ':3.2f')
losses_identity_T = AverageMeter('idt_T', ':3.2f')
losses_semantic_S2T = AverageMeter('sem_S2T', ':3.2f')
losses_semantic_T2S = AverageMeter('sem_T2S', ':3.2f')
progress = ProgressMeter(
args.iters_per_epoch,
[batch_time, data_time, losses_G_S2T, losses_G_T2S, losses_D_S, losses_D_T,
losses_cycle_S, losses_cycle_T, losses_identity_S, losses_identity_T,
losses_semantic_S2T, losses_semantic_T2S],
prefix="Epoch: [{}]".format(epoch))
end = time.time()
for i in range(args.iters_per_epoch):
real_S, label_s = next(train_source_iter)
real_T, _ = next(train_target_iter)
real_S = real_S.to(device)
real_T = real_T.to(device)
label_s = label_s.to(device)
# measure data loading time
data_time.update(time.time() - end)
# Compute fake images and reconstruction images.
fake_T = netG_S2T(real_S)
rec_S = netG_T2S(fake_T)
fake_S = netG_T2S(real_T)
rec_T = netG_S2T(fake_S)
# Optimizing generators
# discriminators require no gradients
set_requires_grad(netD_S, False)
set_requires_grad(netD_T, False)
optimizer_G.zero_grad()
# GAN loss D_T(G_S2T(S))
loss_G_S2T = criterion_gan(netD_T(fake_T), real=True)
# GAN loss D_S(G_T2S(B))
loss_G_T2S = criterion_gan(netD_S(fake_S), real=True)
# Cycle loss || G_T2S(G_S2T(S)) - S||
loss_cycle_S = criterion_cycle(rec_S, real_S) * args.trade_off_cycle
# Cycle loss || G_S2T(G_T2S(T)) - T||
loss_cycle_T = criterion_cycle(rec_T, real_T) * args.trade_off_cycle
# Identity loss
# G_S2T should be identity if real_T is fed: ||G_S2T(real_T) - real_T||
identity_T = netG_S2T(real_T)
loss_identity_T = criterion_identity(identity_T, real_T) * args.trade_off_identity
# G_T2S should be identity if real_S is fed: ||G_T2S(real_S) - real_S||
identity_S = netG_T2S(real_S)
loss_identity_S = criterion_identity(identity_S, real_S) * args.trade_off_identity
# Semantic loss
pred_fake_T = predict(fake_T)
pred_real_S = predict(real_S)
loss_semantic_S2T = criterion_semantic(pred_fake_T, label_s) * args.trade_off_semantic
pred_fake_S = predict(fake_S)
pred_real_T = predict(real_T)
loss_semantic_T2S = criterion_semantic(pred_fake_S, pred_real_T.max(1).indices) * args.trade_off_semantic
# combined loss and calculate gradients
loss_G = loss_G_S2T + loss_G_T2S + loss_cycle_S + loss_cycle_T + \
loss_identity_S + loss_identity_T + loss_semantic_S2T + loss_semantic_T2S
loss_G.backward()
optimizer_G.step()
# Optimize discriminator
set_requires_grad(netD_S, True)
set_requires_grad(netD_T, True)
optimizer_D.zero_grad()
# Calculate GAN loss for discriminator D_S
fake_S_ = fake_S_pool.query(fake_S.detach())
loss_D_S = 0.5 * (criterion_gan(netD_S(real_S), True) + criterion_gan(netD_S(fake_S_), False))
loss_D_S.backward()
# Calculate GAN loss for discriminator D_T
fake_T_ = fake_T_pool.query(fake_T.detach())
loss_D_T = 0.5 * (criterion_gan(netD_T(real_T), True) + criterion_gan(netD_T(fake_T_), False))
loss_D_T.backward()
optimizer_D.step()
# measure elapsed time
losses_G_S2T.update(loss_G_S2T.item(), real_S.size(0))
losses_G_T2S.update(loss_G_T2S.item(), real_S.size(0))
losses_D_S.update(loss_D_S.item(), real_S.size(0))
losses_D_T.update(loss_D_T.item(), real_S.size(0))
losses_cycle_S.update(loss_cycle_S.item(), real_S.size(0))
losses_cycle_T.update(loss_cycle_T.item(), real_S.size(0))
losses_identity_S.update(loss_identity_S.item(), real_S.size(0))
losses_identity_T.update(loss_identity_T.item(), real_S.size(0))
losses_semantic_S2T.update(loss_semantic_S2T.item(), real_S.size(0))
losses_semantic_T2S.update(loss_semantic_T2S.item(), real_S.size(0))
batch_time.update(time.time() - end)
end = time.time()
if i % args.print_freq == 0:
progress.display(i)
for image, prediction, name in zip([real_S, real_T, fake_S, fake_T],
[pred_real_S, pred_real_T, pred_fake_S, pred_fake_T],
["real_S", "real_T", "fake_S", "fake_T"]):
visualize(image[0], "{}_{}".format(i, name), prediction[0])
for image, name in zip([rec_S, rec_T, identity_S, identity_T],
["rec_S", "rec_T", "identity_S", "identity_T"]):
visualize(image[0], "{}_{}".format(i, name))
if __name__ == '__main__':
architecture_names = sorted(
name for name in models.__dict__
if name.islower() and not name.startswith("__")
and callable(models.__dict__[name])
)
dataset_names = sorted(
name for name in datasets.__dict__
if not name.startswith("__") and callable(datasets.__dict__[name])
)
# dataset parameters
parser = argparse.ArgumentParser(description='Cycada for Segmentation Domain Adaptation')
parser.add_argument('source_root', help='root path of the source dataset')
parser.add_argument('target_root', help='root path of the target dataset')
parser.add_argument('-s', '--source', help='source domain(s)')
parser.add_argument('-t', '--target', help='target domain(s)')
parser.add_argument('--resize-ratio', nargs='+', type=float, default=(1.5, 8 / 3.),
help='the resize ratio for the random resize crop')
parser.add_argument('--train-size', nargs='+', type=int, default=(512, 256),
help='the input and output image size during training')
# model parameters
parser.add_argument('-a', '--arch', metavar='ARCH', default='deeplabv2_resnet101',
choices=architecture_names,
help='backbone architecture: ' +
' | '.join(architecture_names) +
' (default: deeplabv2_resnet101)')
parser.add_argument('--pretrain', type=str, default=None,
help='pretrain checkpoints for segementation model')
parser.add_argument('--ngf', type=int, default=64, help='# of gen filters in the last conv layer')
parser.add_argument('--ndf', type=int, default=64, help='# of discrim filters in the first conv layer')
parser.add_argument('--netD', type=str, default='patch',
help='specify discriminator architecture [patch | pixel]. The basic model is a 70x70 PatchGAN.')
parser.add_argument('--netG', type=str, default='unet_256',
help='specify generator architecture [resnet_9 | resnet_6 | unet_256 | unet_128]')
parser.add_argument('--norm', type=str, default='instance',
help='instance normalization or batch normalization [instance | batch | none]')
parser.add_argument("--resume", type=str, default=None,
help="Where restore cyclegan model parameters from.")
parser.add_argument('--trade-off-cycle', type=float, default=10.0, help='trade off for cycle loss')
parser.add_argument('--trade-off-identity', type=float, default=5.0, help='trade off for identity loss')
parser.add_argument('--trade-off-semantic', type=float, default=1.0, help='trade off for semantic loss')
# training parameters
parser.add_argument('-b', '--batch-size', default=1, type=int,
metavar='N',
help='mini-batch size (default: 1)')
parser.add_argument('--lr', type=float, default=0.0002, help='initial learning rate for adam')
parser.add_argument('--beta1', type=float, default=0.5, help='momentum term of adam')
parser.add_argument('-j', '--workers', default=4, type=int, metavar='N',
help='number of data loading workers (default: 4)')
parser.add_argument('--epochs', default=20, type=int, metavar='N',
help='number of total epochs to run')
parser.add_argument('--epochs-decay', type=int, default=20,
help='number of epochs to linearly decay learning rate to zero')
parser.add_argument('--start-epoch', default=0, type=int, metavar='N',
help='start epoch')
parser.add_argument('-i', '--iters-per-epoch', default=5000, type=int,
help='Number of iterations per epoch')
parser.add_argument('--pool-size', type=int, default=50,
help='the size of image buffer that stores previously generated images')
parser.add_argument('-p', '--print-freq', default=400, type=int,
metavar='N', help='print frequency (default: 100)')
parser.add_argument('--seed', default=None, type=int,
help='seed for initializing training. ')
parser.add_argument("--ignore-label", type=int, default=255,
help="The index of the label to ignore during the training.")
parser.add_argument("--log", type=str, default='cycada',
help="Where to save logs, checkpoints and debugging images.")
# test parameters
parser.add_argument("--phase", type=str, default='train', choices=['train', 'test'],
help="When phase is 'test', only test the model.")
parser.add_argument('--translated-root', type=str, default=None,
help="The root to put the translated dataset")
parser.add_argument('--test-input-size', nargs='+', type=int, default=(1024, 512),
help='the input image size during test')
args = parser.parse_args()
main(args)
================================================
FILE: domain_generalization/Transfer-Learning-Library/examples/domain_adaptation/semantic_segmentation/cycada.sh
================================================
# GTA5 to Cityscapes
# First, train the CycleGAN
CUDA_VISIBLE_DEVICES=0 python cycada.py data/GTA5 data/Cityscapes -s GTA5 -t Cityscapes \
--log logs/cycada/gtav2cityscapes --pretrain logs/src_only/gtav2cityscapes/checkpoints/59.pth \
--translated-root data/GTA52Cityscapes/cycada_39
# Then, train the src_only model on the translated source dataset
CUDA_VISIBLE_DEVICES=0 python source_only.py data/GTA52Cityscapes/cycada_39 data/Cityscapes \
-s GTA5 -t Cityscapes --log logs/cycada_src_only/gtav2cityscapes
## Synthia to Cityscapes
# First, train the Cycada
CUDA_VISIBLE_DEVICES=0 python cycada.py data/synthia data/Cityscapes -s Synthia -t Cityscapes \
--log logs/cycada/synthia2cityscapes --pretrain logs/src_only/synthia2cityscapes/checkpoints/59.pth \
--translated-root data/Synthia2Cityscapes/cycada_39
# Then, train the src_only model on the translated source dataset
CUDA_VISIBLE_DEVICES=0 python source_only.py data/Synthia2Cityscapes/cycada_39 data/Cityscapes \
-s Synthia -t Cityscapes --log logs/cycada_src_only/synthia2cityscapes
# Cityscapes to FoggyCityscapes
# First, train the CycleGAN
CUDA_VISIBLE_DEVICES=0 python cycada.py data/Cityscapes data/Cityscapes -s Cityscapes -t FoggyCityscapes \
--log logs/cycada/cityscapes2foggy --pretrain logs/src_only/cityscapes2foggy/checkpoints/59.pth \
--translated-root data/Cityscapes2Foggy/cycada_39
# Then, train the src_only model on the translated source dataset
CUDA_VISIBLE_DEVICES=0 python source_only.py data/Cityscapes2Foggy/cycada_39 data/Cityscapes \
-s Cityscapes -t FoggyCityscapes --log logs/cycada_src_only/cityscapes2foggy
================================================
FILE: domain_generalization/Transfer-Learning-Library/examples/domain_adaptation/semantic_segmentation/cycle_gan.py
================================================
"""
@author: Junguang Jiang
@contact: JiangJunguang1123@outlook.com
"""
import random
import time
import warnings
import sys
import argparse
import itertools
import torch
import torch.nn as nn
import torch.backends.cudnn as cudnn
from torch.optim import Adam
from torch.optim.lr_scheduler import LambdaLR
from torch.utils.data import DataLoader
from torchvision.transforms import ToPILImage, Compose
sys.path.append('../../..')
import dalib.translation.cyclegan as cyclegan
from dalib.translation.cyclegan.util import ImagePool, set_requires_grad
import common.vision.datasets.segmentation as datasets
from common.vision.transforms import Denormalize
import common.vision.transforms.segmentation as T
from common.utils.data import ForeverDataIterator
from common.utils.meter import AverageMeter, ProgressMeter
from common.utils.logger import CompleteLogger
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
def main(args):
logger = CompleteLogger(args.log, args.phase)
print(args)
if args.seed is not None:
random.seed(args.seed)
torch.manual_seed(args.seed)
cudnn.deterministic = True
warnings.warn('You have chosen to seed training. '
'This will turn on the CUDNN deterministic setting, '
'which can slow down your training considerably! '
'You may see unexpected behavior when restarting '
'from checkpoints.')
cudnn.benchmark = True
# Data loading code
train_transform = T.Compose([
T.RandomResizedCrop(size=args.train_size, ratio=args.resize_ratio, scale=(0.5, 1.)),
T.RandomHorizontalFlip(),
T.ToTensor(),
T.Normalize((0.5, 0.5, 0.5), (0.5, 0.5, 0.5))
])
source_dataset = datasets.__dict__[args.source]
train_source_dataset = source_dataset(root=args.source_root, transforms=train_transform)
train_source_loader = DataLoader(train_source_dataset, batch_size=args.batch_size,
shuffle=True, num_workers=args.workers, pin_memory=True, drop_last=True)
target_dataset = datasets.__dict__[args.target]
train_target_dataset = target_dataset(root=args.target_root, transforms=train_transform)
train_target_loader = DataLoader(train_target_dataset, batch_size=args.batch_size,
shuffle=True, num_workers=args.workers, pin_memory=True, drop_last=True)
train_source_iter = ForeverDataIterator(train_source_loader)
train_target_iter = ForeverDataIterator(train_target_loader)
# define networks (both generators and discriminators)
netG_S2T = cyclegan.generator.__dict__[args.netG](ngf=args.ngf, norm=args.norm, use_dropout=False).to(device)
netG_T2S = cyclegan.generator.__dict__[args.netG](ngf=args.ngf, norm=args.norm, use_dropout=False).to(device)
netD_S = cyclegan.discriminator.__dict__[args.netD](ndf=args.ndf, norm=args.norm).to(device)
netD_T = cyclegan.discriminator.__dict__[args.netD](ndf=args.ndf, norm=args.norm).to(device)
# create image buffer to store previously generated images
fake_S_pool = ImagePool(args.pool_size)
fake_T_pool = ImagePool(args.pool_size)
# define optimizer and lr scheduler
optimizer_G = Adam(itertools.chain(netG_S2T.parameters(), netG_T2S.parameters()), lr=args.lr, betas=(args.beta1, 0.999))
optimizer_D = Adam(itertools.chain(netD_S.parameters(), netD_T.parameters()), lr=args.lr, betas=(args.beta1, 0.999))
lr_decay_function = lambda epoch: 1.0 - max(0, epoch - args.epochs) / float(args.epochs_decay)
lr_scheduler_G = LambdaLR(optimizer_G, lr_lambda=lr_decay_function)
lr_scheduler_D = LambdaLR(optimizer_D, lr_lambda=lr_decay_function)
# optionally resume from a checkpoint
if args.resume:
print("Resume from", args.resume)
checkpoint = torch.load(args.resume, map_location='cpu')
netG_S2T.load_state_dict(checkpoint['netG_S2T'])
netG_T2S.load_state_dict(checkpoint['netG_T2S'])
netD_S.load_state_dict(checkpoint['netD_S'])
netD_T.load_state_dict(checkpoint['netD_T'])
optimizer_G.load_state_dict(checkpoint['optimizer_G'])
optimizer_D.load_state_dict(checkpoint['optimizer_D'])
lr_scheduler_G.load_state_dict(checkpoint['lr_scheduler_G'])
lr_scheduler_D.load_state_dict(checkpoint['lr_scheduler_D'])
args.start_epoch = checkpoint['epoch'] + 1
if args.phase == 'test':
transform = T.Compose([
T.Resize(image_size=args.test_input_size),
T.wrapper(cyclegan.transform.Translation)(netG_S2T, device),
])
train_source_dataset.translate(transform, args.translated_root)
return
# define loss function
criterion_gan = cyclegan.LeastSquaresGenerativeAdversarialLoss()
criterion_cycle = nn.L1Loss()
criterion_identity = nn.L1Loss()
# define visualization function
tensor_to_image = Compose([
Denormalize((0.5, 0.5, 0.5), (0.5, 0.5, 0.5)),
ToPILImage()
])
def visualize(image, name):
"""
Args:
image (tensor): image in shape 3 x H x W
name: name of the saving image
"""
tensor_to_image(image).save(logger.get_image_path("{}.png".format(name)))
# start training
for epoch in range(args.start_epoch, args.epochs+args.epochs_decay):
logger.set_epoch(epoch)
print(lr_scheduler_G.get_lr())
# train for one epoch
train(train_source_iter, train_target_iter, netG_S2T, netG_T2S, netD_S, netD_T,
criterion_gan, criterion_cycle, criterion_identity, optimizer_G, optimizer_D,
fake_S_pool, fake_T_pool, epoch, visualize, args)
# update learning rates
lr_scheduler_G.step()
lr_scheduler_D.step()
# save checkpoint
torch.save(
{
'netG_S2T': netG_S2T.state_dict(),
'netG_T2S': netG_T2S.state_dict(),
'netD_S': netD_S.state_dict(),
'netD_T': netD_T.state_dict(),
'optimizer_G': optimizer_G.state_dict(),
'optimizer_D': optimizer_D.state_dict(),
'lr_scheduler_G': lr_scheduler_G.state_dict(),
'lr_scheduler_D': lr_scheduler_D.state_dict(),
'epoch': epoch,
'args': args
}, logger.get_checkpoint_path(epoch)
)
if args.translated_root is not None:
transform = T.Compose([
T.Resize(image_size=args.test_input_size),
T.wrapper(cyclegan.transform.Translation)(netG_S2T, device),
])
train_source_dataset.translate(transform, args.translated_root)
logger.close()
def train(train_source_iter, train_target_iter, netG_S2T, netG_T2S, netD_S, netD_T,
criterion_gan, criterion_cycle, criterion_identity, optimizer_G, optimizer_D,
fake_S_pool, fake_T_pool, epoch: int, visualize, args: argparse.Namespace):
batch_time = AverageMeter('Time', ':4.2f')
data_time = AverageMeter('Data', ':3.1f')
losses_G_S2T = AverageMeter('G_S2T', ':3.2f')
losses_G_T2S = AverageMeter('G_T2S', ':3.2f')
losses_D_S = AverageMeter('D_S', ':3.2f')
losses_D_T = AverageMeter('D_T', ':3.2f')
losses_cycle_S = AverageMeter('cycle_S', ':3.2f')
losses_cycle_T = AverageMeter('cycle_T', ':3.2f')
losses_identity_S = AverageMeter('idt_S', ':3.2f')
losses_identity_T = AverageMeter('idt_T', ':3.2f')
progress = ProgressMeter(
args.iters_per_epoch,
[batch_time, data_time, losses_G_S2T, losses_G_T2S, losses_D_S, losses_D_T,
losses_cycle_S, losses_cycle_T, losses_identity_S, losses_identity_T],
prefix="Epoch: [{}]".format(epoch))
end = time.time()
for i in range(args.iters_per_epoch):
real_S, _ = next(train_source_iter)
real_T, _ = next(train_target_iter)
real_S = real_S.to(device)
real_T = real_T.to(device)
# measure data loading time
data_time.update(time.time() - end)
# Compute fake images and reconstruction images.
fake_T = netG_S2T(real_S)
rec_S = netG_T2S(fake_T)
fake_S = netG_T2S(real_T)
rec_T = netG_S2T(fake_S)
# Optimizing generators
# discriminators require no gradients
set_requires_grad(netD_S, False)
set_requires_grad(netD_T, False)
optimizer_G.zero_grad()
# GAN loss D_T(G_S2T(S))
loss_G_S2T = criterion_gan(netD_T(fake_T), real=True)
# GAN loss D_S(G_T2S(B))
loss_G_T2S = criterion_gan(netD_S(fake_S), real=True)
# Cycle loss || G_T2S(G_S2T(S)) - S||
loss_cycle_S = criterion_cycle(rec_S, real_S) * args.trade_off_cycle
# Cycle loss || G_S2T(G_T2S(T)) - T||
loss_cycle_T = criterion_cycle(rec_T, real_T) * args.trade_off_cycle
# Identity loss
# G_S2T should be identity if real_T is fed: ||G_S2T(real_T) - real_T||
identity_T = netG_S2T(real_T)
loss_identity_T = criterion_identity(identity_T, real_T) * args.trade_off_identity
# G_T2S should be identity if real_S is fed: ||G_T2S(real_S) - real_S||
identity_S = netG_T2S(real_S)
loss_identity_S = criterion_identity(identity_S, real_S) * args.trade_off_identity
# combined loss and calculate gradients
loss_G = loss_G_S2T + loss_G_T2S + loss_cycle_S + loss_cycle_T + loss_identity_S + loss_identity_T
loss_G.backward()
optimizer_G.step()
# Optimize discriminator
set_requires_grad(netD_S, True)
set_requires_grad(netD_T, True)
optimizer_D.zero_grad()
# Calculate GAN loss for discriminator D_S
fake_S_ = fake_S_pool.query(fake_S.detach())
loss_D_S = 0.5 * (criterion_gan(netD_S(real_S), True) + criterion_gan(netD_S(fake_S_), False))
loss_D_S.backward()
# Calculate GAN loss for discriminator D_T
fake_T_ = fake_T_pool.query(fake_T.detach())
loss_D_T = 0.5 * (criterion_gan(netD_T(real_T), True) + criterion_gan(netD_T(fake_T_), False))
loss_D_T.backward()
optimizer_D.step()
# measure elapsed time
losses_G_S2T.update(loss_G_S2T.item(), real_S.size(0))
losses_G_T2S.update(loss_G_T2S.item(), real_S.size(0))
losses_D_S.update(loss_D_S.item(), real_S.size(0))
losses_D_T.update(loss_D_T.item(), real_S.size(0))
losses_cycle_S.update(loss_cycle_S.item(), real_S.size(0))
losses_cycle_T.update(loss_cycle_T.item(), real_S.size(0))
losses_identity_S.update(loss_identity_S.item(), real_S.size(0))
losses_identity_T.update(loss_identity_T.item(), real_S.size(0))
batch_time.update(time.time() - end)
end = time.time()
if i % args.print_freq == 0:
progress.display(i)
for tensor, name in zip([real_S, real_T, fake_S, fake_T, rec_S, rec_T, identity_S, identity_T],
["real_S", "real_T", "fake_S", "fake_T", "rec_S",
"rec_T", "identity_S", "identity_T"]):
visualize(tensor[0], "{}_{}".format(i, name))
if __name__ == '__main__':
dataset_names = sorted(
name for name in datasets.__dict__
if not name.startswith("__") and callable(datasets.__dict__[name])
)
parser = argparse.ArgumentParser(description='CycleGAN for Segmentation Domain Adaptation')
# dataset parameters
parser.add_argument('source_root', help='root path of the source dataset')
parser.add_argument('target_root', help='root path of the target dataset')
parser.add_argument('-s', '--source', help='source domain(s)')
parser.add_argument('-t', '--target', help='target domain(s)')
parser.add_argument('--resize-ratio', nargs='+', type=float, default=(1.5, 8 / 3.),
help='the resize ratio for the random resize crop')
parser.add_argument('--train-size', nargs='+', type=int, default=(1024, 512),
help='the input and output image size during training')
# model parameters
parser.add_argument('--ngf', type=int, default=64, help='# of gen filters in the last conv layer')
parser.add_argument('--ndf', type=int, default=64, help='# of discrim filters in the first conv layer')
parser.add_argument('--netD', type=str, default='patch',
help='specify discriminator architecture [patch | pixel]. The basic model is a 70x70 PatchGAN.')
parser.add_argument('--netG', type=str, default='unet_256',
help='specify generator architecture [resnet_9 | resnet_6 | unet_256 | unet_128]')
parser.add_argument('--norm', type=str, default='instance',
help='instance normalization or batch normalization [instance | batch | none]')
parser.add_argument("--resume", type=str, default=None,
help="Where restore model parameters from.")
parser.add_argument('--trade-off-cycle', type=float, default=10.0, help='trade off for cycle loss')
parser.add_argument('--trade-off-identity', type=float, default=5.0, help='trade off for identity loss')
# training parameters
parser.add_argument('-b', '--batch-size', default=1, type=int,
metavar='N',
help='mini-batch size (default: 1)')
parser.add_argument('--lr', type=float, default=0.0002, help='initial learning rate for adam')
parser.add_argument('--beta1', type=float, default=0.5, help='momentum term of adam')
parser.add_argument('-j', '--workers', default=4, type=int, metavar='N',
help='number of data loading workers (default: 4)')
parser.add_argument('--epochs', default=20, type=int, metavar='N',
help='number of total epochs to run')
parser.add_argument('--epochs-decay', type=int, default=20,
help='number of epochs to linearly decay learning rate to zero')
parser.add_argument('--start-epoch', default=0, type=int, metavar='N',
help='start epoch')
parser.add_argument('-i', '--iters-per-epoch', default=5000, type=int,
help='Number of iterations per epoch')
parser.add_argument('--pool-size', type=int, default=50,
help='the size of image buffer that stores previously generated images')
parser.add_argument('-p', '--print-freq', default=400, type=int,
metavar='N', help='print frequency (default: 100)')
parser.add_argument('--seed', default=None, type=int,
help='seed for initializing training. ')
parser.add_argument("--log", type=str, default='cyclegan',
help="Where to save logs, checkpoints and debugging images.")
# test parameters
parser.add_argument("--phase", type=str, default='train', choices=['train', 'test'],
help="When phase is 'test', only test the model.")
parser.add_argument('--translated-root', type=str, default=None,
help="The root to put the translated dataset")
parser.add_argument('--test-input-size', nargs='+', type=int, default=(1024, 512),
help='the input image size during test')
args = parser.parse_args()
main(args)
================================================
FILE: domain_generalization/Transfer-Learning-Library/examples/domain_adaptation/semantic_segmentation/cycle_gan.sh
================================================
# GTA5 to Cityscapes
# First, train the CycleGAN
CUDA_VISIBLE_DEVICES=0 python cycle_gan.py data/GTA5 data/Cityscapes -s GTA5 -t Cityscapes \
--log logs/cyclegan/gtav2cityscapes --translated-root data/GTA52Cityscapes/CycleGAN_39
# Then, train the src_only model on the translated source dataset
CUDA_VISIBLE_DEVICES=0 python source_only.py data/GTA52Cityscapes/CycleGAN_39 data/Cityscapes \
-s GTA5 -t Cityscapes --log logs/cyclegan_src_only/gtav2cityscapes
# Cityscapes to FoggyCityscapes
# First, train the CycleGAN
CUDA_VISIBLE_DEVICES=0 python cycle_gan.py data/Cityscapes data/Cityscapes -s Cityscapes -t FoggyCityscapes \
--log logs/cyclegan/cityscapes2foggy --translated-root data/Cityscapes2Foggy/CycleGAN_39
# Then, train the src_only model on the translated source dataset
CUDA_VISIBLE_DEVICES=0 python source_only.py data/Cityscapes2Foggy/CycleGAN_39 data/Cityscapes \
-s Cityscapes -t FoggyCityscapes --log logs/cyclegan_src_only/cityscapes2foggy
================================================
FILE: domain_generalization/Transfer-Learning-Library/examples/domain_adaptation/semantic_segmentation/fda.py
================================================
"""
@author: Junguang Jiang
@contact: JiangJunguang1123@outlook.com
"""
import random
import time
import warnings
import sys
import argparse
from PIL import Image
import numpy as np
import os
import shutil
import torch
import torch.nn as nn
import torch.backends.cudnn as cudnn
from torch.optim import SGD
from torch.optim.lr_scheduler import LambdaLR
from torch.utils.data import DataLoader
sys.path.append('../../..')
from dalib.translation.fourier_transform import FourierTransform
from dalib.adaptation.fda import robust_entropy
import common.vision.models.segmentation as models
import common.vision.datasets.segmentation as datasets
import common.vision.transforms.segmentation as T
from common.vision.transforms import DeNormalizeAndTranspose
from common.utils.data import ForeverDataIterator
from common.utils.metric import ConfusionMatrix
from common.utils.meter import AverageMeter, ProgressMeter, Meter
from common.utils.logger import CompleteLogger
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
def main(args: argparse.Namespace):
logger = CompleteLogger(args.log, args.phase)
print(args)
if args.seed is not None:
random.seed(args.seed)
torch.manual_seed(args.seed)
cudnn.deterministic = True
warnings.warn('You have chosen to seed training. '
'This will turn on the CUDNN deterministic setting, '
'which can slow down your training considerably! '
'You may see unexpected behavior when restarting '
'from checkpoints.')
cudnn.benchmark = True
# Data loading code
target_dataset = datasets.__dict__[args.target]
train_target_dataset = target_dataset(
root=args.target_root,
transforms=T.Compose([
T.Resize(image_size=args.train_size),
T.NormalizeAndTranspose(),
]),
)
train_target_loader = DataLoader(train_target_dataset, batch_size=args.batch_size,
shuffle=True, num_workers=args.workers, pin_memory=True, drop_last=True)
val_target_dataset = target_dataset(
root=args.target_root, split='val',
transforms=T.Compose([
T.Resize(image_size=args.test_input_size, label_size=args.test_output_size),
T.NormalizeAndTranspose(),
])
)
val_target_loader = DataLoader(val_target_dataset, batch_size=1, shuffle=False, pin_memory=True)
# collect the absolute paths of all images in the target dataset
target_image_list = train_target_dataset.collect_image_paths()
# build a fourier transform that translate source images to the target style
fourier_transform = T.wrapper(FourierTransform)(target_image_list, os.path.join(logger.root, "amplitudes"),
rebuild=False, beta=args.beta)
source_dataset = datasets.__dict__[args.source]
train_source_dataset = source_dataset(
root=args.source_root,
transforms=T.Compose([
T.Resize((2048, 1024)), # convert source image to the size of the target image before fourier transform
fourier_transform,
T.RandomResizedCrop(size=args.train_size, ratio=args.resize_ratio, scale=(0.5, 1.)),
T.ColorJitter(brightness=0.3, contrast=0.3),
T.RandomHorizontalFlip(),
T.NormalizeAndTranspose(),
]),
)
train_source_loader = DataLoader(train_source_dataset, batch_size=args.batch_size,
shuffle=True, num_workers=args.workers, pin_memory=True, drop_last=True)
train_source_iter = ForeverDataIterator(train_source_loader)
train_target_iter = ForeverDataIterator(train_target_loader)
# create model
model = models.__dict__[args.arch](num_classes=train_source_dataset.num_classes).to(device)
# define optimizer and lr scheduler
optimizer = SGD(model.get_parameters(), lr=args.lr, momentum=args.momentum, weight_decay=args.weight_decay)
lr_scheduler = LambdaLR(optimizer,
lambda x: args.lr * (1. - float(x) / args.epochs / args.iters_per_epoch) ** (args.lr_power))
# optionally resume from a checkpoint
if args.resume:
checkpoint = torch.load(args.resume, map_location='cpu')
model.load_state_dict(checkpoint['model'])
optimizer.load_state_dict(checkpoint['optimizer'])
lr_scheduler.load_state_dict(checkpoint['lr_scheduler'])
args.start_epoch = checkpoint['epoch'] + 1
# define loss function (criterion)
criterion = torch.nn.CrossEntropyLoss(ignore_index=args.ignore_label).to(device)
interp_train = nn.Upsample(size=args.train_size[::-1], mode='bilinear', align_corners=True)
interp_val = nn.Upsample(size=args.test_output_size[::-1], mode='bilinear', align_corners=True)
# define visualization function
decode = train_source_dataset.decode_target
def visualize(image, pred, label, prefix):
"""
Args:
image (tensor): 3 x H x W
pred (tensor): C x H x W
label (tensor): H x W
prefix: prefix of the saving image
"""
image = image.detach().cpu().numpy()
pred = pred.detach().max(dim=0)[1].cpu().numpy()
label = label.cpu().numpy()
for tensor, name in [
(Image.fromarray(np.uint8(DeNormalizeAndTranspose()(image))), "image"),
(decode(label), "label"),
(decode(pred), "pred")
]:
tensor.save(logger.get_image_path("{}_{}.png".format(prefix, name)))
if args.phase == 'test':
confmat = validate(val_target_loader, model, interp_val, criterion, visualize, args)
print(confmat)
return
# start training
best_iou = 0.
for epoch in range(args.start_epoch, args.epochs):
logger.set_epoch(epoch)
print(lr_scheduler.get_lr())
# train for one epoch
train(train_source_iter, train_target_iter, model, interp_train, criterion, optimizer,
lr_scheduler, epoch, visualize if args.debug else None, args)
# evaluate on validation set
confmat = validate(val_target_loader, model, interp_val, criterion, None, args)
print(confmat.format(train_source_dataset.classes))
acc_global, acc, iu = confmat.compute()
# calculate the mean iou over partial classes
indexes = [train_source_dataset.classes.index(name) for name
in train_source_dataset.evaluate_classes]
iu = iu[indexes]
mean_iou = iu.mean()
# remember best acc@1 and save checkpoint
torch.save(
{
'model': model.state_dict(),
'optimizer': optimizer.state_dict(),
'lr_scheduler': lr_scheduler.state_dict(),
'epoch': epoch,
'args': args
}, logger.get_checkpoint_path(epoch)
)
if mean_iou > best_iou:
shutil.copy(logger.get_checkpoint_path(epoch), logger.get_checkpoint_path('best'))
best_iou = max(best_iou, mean_iou)
print("Target: {} Best: {}".format(mean_iou, best_iou))
logger.close()
def train(train_source_iter: ForeverDataIterator, train_target_iter: ForeverDataIterator,
model, interp, criterion, optimizer: SGD,
lr_scheduler: LambdaLR, epoch: int, visualize, args: argparse.Namespace):
batch_time = AverageMeter('Time', ':4.2f')
data_time = AverageMeter('Data', ':3.1f')
losses_s = AverageMeter('Loss (s)', ':3.2f')
losses_t = AverageMeter('Loss (t)', ':3.2f')
losses_entropy_t = AverageMeter('Entropy (t)', ':3.2f')
accuracies_s = Meter('Acc (s)', ':3.2f')
accuracies_t = Meter('Acc (t)', ':3.2f')
iou_s = Meter('IoU (s)', ':3.2f')
iou_t = Meter('IoU (t)', ':3.2f')
confmat_s = ConfusionMatrix(model.num_classes)
confmat_t = ConfusionMatrix(model.num_classes)
progress = ProgressMeter(
args.iters_per_epoch,
[batch_time, data_time, losses_s, losses_t, losses_entropy_t,
accuracies_s, accuracies_t, iou_s, iou_t],
prefix="Epoch: [{}]".format(epoch))
# switch to train mode
model.train()
end = time.time()
for i in range(args.iters_per_epoch):
optimizer.zero_grad()
x_s, label_s = next(train_source_iter)
x_t, label_t = next(train_target_iter)
x_s = x_s.to(device)
label_s = label_s.long().to(device)
x_t = x_t.to(device)
label_t = label_t.long().to(device)
# measure data loading time
data_time.update(time.time() - end)
# compute output
y_s = model(x_s)
pred_s = interp(y_s)
loss_cls_s = criterion(pred_s, label_s)
loss_cls_s.backward()
y_t = model(x_t)
pred_t = interp(y_t)
loss_cls_t = criterion(pred_t, label_t)
loss_entropy_t = robust_entropy(y_t, args.ita)
(args.entropy_weight * loss_entropy_t).backward()
# compute gradient and do SGD step
optimizer.step()
lr_scheduler.step()
# measure accuracy and record loss
losses_s.update(loss_cls_s.item(), x_s.size(0))
losses_t.update(loss_cls_t.item(), x_s.size(0))
losses_entropy_t.update(loss_entropy_t.item(), x_s.size(0))
confmat_s.update(label_s.flatten(), pred_s.argmax(1).flatten())
confmat_t.update(label_t.flatten(), pred_t.argmax(1).flatten())
acc_global_s, acc_s, iu_s = confmat_s.compute()
acc_global_t, acc_t, iu_t = confmat_t.compute()
accuracies_s.update(acc_s.mean().item())
accuracies_t.update(acc_t.mean().item())
iou_s.update(iu_s.mean().item())
iou_t.update(iu_t.mean().item())
# measure elapsed time
batch_time.update(time.time() - end)
end = time.time()
if i % args.print_freq == 0:
progress.display(i)
if visualize is not None:
visualize(x_s[0], pred_s[0], label_s[0], "source_{}".format(i))
visualize(x_t[0], pred_t[0], label_t[0], "target_{}".format(i))
def validate(val_loader: DataLoader, model, interp, criterion, visualize, args: argparse.Namespace):
batch_time = AverageMeter('Time', ':6.3f')
losses = AverageMeter('Loss', ':.4e')
acc = Meter('Acc', ':3.2f')
iou = Meter('IoU', ':3.2f')
progress = ProgressMeter(
len(val_loader),
[batch_time, losses, acc, iou],
prefix='Test: ')
# switch to evaluate mode
model.eval()
confmat = ConfusionMatrix(model.num_classes)
with torch.no_grad():
end = time.time()
for i, (x, label) in enumerate(val_loader):
x = x.to(device)
label = label.long().to(device)
# compute output
output = interp(model(x))
loss = criterion(output, label)
# measure accuracy and record loss
losses.update(loss.item(), x.size(0))
confmat.update(label.flatten(), output.argmax(1).flatten())
acc_global, accs, iu = confmat.compute()
acc.update(accs.mean().item())
iou.update(iu.mean().item())
# measure elapsed time
batch_time.update(time.time() - end)
end = time.time()
if i % args.print_freq == 0:
progress.display(i)
if visualize is not None:
visualize(x[0], output[0], label[0], "val_{}".format(i))
return confmat
if __name__ == '__main__':
architecture_names = sorted(
name for name in models.__dict__
if name.islower() and not name.startswith("__")
and callable(models.__dict__[name])
)
dataset_names = sorted(
name for name in datasets.__dict__
if not name.startswith("__") and callable(datasets.__dict__[name])
)
parser = argparse.ArgumentParser(description='FDA for Segmentation Domain Adaptation')
# dataset parameters
parser.add_argument('source_root', help='root path of the source dataset')
parser.add_argument('target_root', help='root path of the target dataset')
parser.add_argument('-s', '--source', help='source domain(s)')
parser.add_argument('-t', '--target', help='target domain(s)')
parser.add_argument('--resize-ratio', nargs='+', type=float, default=(1.5, 8 / 3.),
help='the resize ratio for the random resize crop')
parser.add_argument('--train-size', nargs='+', type=int, default=(1024, 512),
help='the input and output image size during training')
parser.add_argument('--test-input-size', nargs='+', type=int, default=(1024, 512),
help='the input image size during test')
parser.add_argument('--test-output-size', nargs='+', type=int, default=(2048, 1024),
help='the output image size during test')
# model parameters
parser.add_argument('-a', '--arch', metavar='ARCH', default='deeplabv2_resnet101',
choices=architecture_names,
help='backbone architecture: ' +
' | '.join(architecture_names) +
' (default: deeplabv2_resnet101)')
parser.add_argument("--entropy-weight", type=float, default=0., help="weight for entropy")
parser.add_argument("--ita", type=float, default=2.0, help="ita for robust entropy")
parser.add_argument("--beta", type=int, default=1, help="beta for FDA")
parser.add_argument("--resume", type=str, default=None,
help="Where restore model parameters from.")
# training parameters
parser.add_argument('-b', '--batch-size', default=2, type=int,
metavar='N',
help='mini-batch size (default: 2)')
parser.add_argument('--lr', '--learning-rate', default=2.5e-3, type=float,
metavar='LR', help='initial learning rate', dest='lr')
parser.add_argument("--momentum", type=float, default=0.9, help="Momentum component of the optimiser.")
parser.add_argument("--weight-decay", type=float, default=0.0005, help="Regularisation parameter for L2-loss.")
parser.add_argument("--lr-power", type=float, default=0.9,
help="Decay parameter to compute the learning rate (only for deeplab).")
parser.add_argument('-j', '--workers', default=4, type=int, metavar='N',
help='number of data loading workers (default: 4)')
parser.add_argument('--epochs', default=60, type=int, metavar='N',
help='number of total epochs to run')
parser.add_argument('--start-epoch', default=0, type=int, metavar='N',
help='start epoch')
parser.add_argument('-i', '--iters-per-epoch', default=2500, type=int,
help='Number of iterations per epoch')
parser.add_argument('-p', '--print-freq', default=100, type=int,
metavar='N', help='print frequency (default: 100)')
parser.add_argument('--seed', default=None, type=int,
help='seed for initializing training. ')
parser.add_argument("--ignore-label", type=int, default=255,
help="The index of the label to ignore during the training.")
parser.add_argument("--log", type=str, default='fda',
help="Where to save logs, checkpoints and debugging images.")
parser.add_argument("--phase", type=str, default='train', choices=['train', 'test'],
help="When phase is 'test', only test the model.")
parser.add_argument('--debug', action="store_true",
help='In the debug mode, save images and predictions during training')
args = parser.parse_args()
main(args)
================================================
FILE: domain_generalization/Transfer-Learning-Library/examples/domain_adaptation/semantic_segmentation/fda.sh
================================================
# GTA5 to Cityscapes
CUDA_VISIBLE_DEVICES=0 python fda.py data/GTA5 data/Cityscapes -s GTA5 -t Cityscapes \
--log logs/fda/gtav2cityscapes --debug
# Synthia to Cityscapes
CUDA_VISIBLE_DEVICES=0 python fda.py data/synthia data/Cityscapes -s Synthia -t Cityscapes \
--log logs/fda/synthia2cityscapes --debug
# Cityscapes to FoggyCityscapes
CUDA_VISIBLE_DEVICES=0 python fda.py data/Cityscapes data/Cityscapes -s Cityscapes -t FoggyCityscapes \
--log logs/fda/cityscapes2foggy --debug
================================================
FILE: domain_generalization/Transfer-Learning-Library/examples/domain_adaptation/semantic_segmentation/source_only.py
================================================
"""
@author: Junguang Jiang
@contact: JiangJunguang1123@outlook.com
"""
import random
import time
import warnings
import sys
import argparse
from PIL import Image
import numpy as np
import shutil
import torch
import torch.nn as nn
import torch.backends.cudnn as cudnn
from torch.optim import SGD
from torch.optim.lr_scheduler import LambdaLR
from torch.utils.data import DataLoader
sys.path.append('../../..')
import common.vision.models.segmentation as models
import common.vision.datasets.segmentation as datasets
import common.vision.transforms.segmentation as T
from common.vision.transforms import DeNormalizeAndTranspose
from common.utils.data import ForeverDataIterator
from common.utils.metric import ConfusionMatrix
from common.utils.meter import AverageMeter, ProgressMeter, Meter
from common.utils.logger import CompleteLogger
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
def main(args: argparse.Namespace):
logger = CompleteLogger(args.log, args.phase)
print(args)
if args.seed is not None:
random.seed(args.seed)
torch.manual_seed(args.seed)
cudnn.deterministic = True
warnings.warn('You have chosen to seed training. '
'This will turn on the CUDNN deterministic setting, '
'which can slow down your training considerably! '
'You may see unexpected behavior when restarting '
'from checkpoints.')
cudnn.benchmark = True
# Data loading code
source_dataset = datasets.__dict__[args.source]
train_source_dataset = source_dataset(
root=args.source_root,
transforms=T.Compose([
T.RandomResizedCrop(size=args.train_size, ratio=args.resize_ratio, scale=args.resize_scale),
T.ColorJitter(brightness=0.3, contrast=0.3),
T.RandomHorizontalFlip(),
T.NormalizeAndTranspose(),
]),
)
train_source_loader = DataLoader(train_source_dataset, batch_size=args.batch_size,
shuffle=True, num_workers=args.workers, pin_memory=True, drop_last=True)
target_dataset = datasets.__dict__[args.target]
val_target_dataset = target_dataset(
root=args.target_root, split='val',
transforms=T.Compose([
T.Resize(image_size=args.test_input_size, label_size=args.test_output_size),
T.NormalizeAndTranspose(),
]),
)
val_target_loader = DataLoader(val_target_dataset, batch_size=1, shuffle=False, pin_memory=True)
train_source_iter = ForeverDataIterator(train_source_loader)
# create model
model = models.__dict__[args.arch](num_classes=train_source_dataset.num_classes).to(device)
# define optimizer and lr scheduler
optimizer = SGD(model.get_parameters(), lr=args.lr, momentum=args.momentum, weight_decay=args.weight_decay)
lr_scheduler = LambdaLR(optimizer, lambda x: args.lr * (1. - float(x) / args.epochs / args.iters_per_epoch)
** (args.lr_power))
# optionally resume from a checkpoint
if args.resume:
checkpoint = torch.load(args.resume, map_location='cpu')
model.load_state_dict(checkpoint['model'])
optimizer.load_state_dict(checkpoint['optimizer'])
lr_scheduler.load_state_dict(checkpoint['lr_scheduler'])
args.start_epoch = checkpoint['epoch'] + 1
# define loss function (criterion)
criterion = torch.nn.CrossEntropyLoss(ignore_index=args.ignore_label).to(device)
interp_train = nn.Upsample(size=args.train_size[::-1], mode='bilinear', align_corners=True)
interp_val = nn.Upsample(size=args.test_output_size[::-1], mode='bilinear', align_corners=True)
# define visualization function
decode = train_source_dataset.decode_target
def visualize(image, pred, label, prefix):
"""
Args:
image (tensor): 3 x H x W
pred (tensor): C x H x W
label (tensor): H x W
prefix: prefix of the saving image
"""
image = image.detach().cpu().numpy()
pred = pred.detach().max(dim=0)[1].cpu().numpy()
label = label.cpu().numpy()
for tensor, name in [
(Image.fromarray(np.uint8(DeNormalizeAndTranspose()(image))), "image"),
(decode(label), "label"),
(decode(pred), "pred")
]:
tensor.save(logger.get_image_path("{}_{}.png".format(prefix, name)))
if args.phase == 'test':
confmat = validate(val_target_loader, model, interp_val, criterion, visualize, args)
print(confmat)
return
# start training
best_iou = 0.
for epoch in range(args.start_epoch, args.epochs):
logger.set_epoch(epoch)
print(lr_scheduler.get_lr())
# train for one epoch
train(train_source_iter, model, interp_train, criterion, optimizer,
lr_scheduler, epoch, visualize if args.debug else None, args)
# evaluate on validation set
confmat = validate(val_target_loader, model, interp_val, criterion, visualize if args.debug else None, args)
print(confmat.format(train_source_dataset.classes))
acc_global, acc, iu = confmat.compute()
# calculate the mean iou over partial classes
indexes = [train_source_dataset.classes.index(name) for name
in train_source_dataset.evaluate_classes]
iu = iu[indexes]
mean_iou = iu.mean()
# remember best iou and save checkpoint
torch.save(
{
'model': model.state_dict(),
'optimizer': optimizer.state_dict(),
'lr_scheduler': lr_scheduler.state_dict(),
'epoch': epoch,
'args': args
}, logger.get_checkpoint_path(epoch)
)
if mean_iou > best_iou:
shutil.copy(logger.get_checkpoint_path(epoch), logger.get_checkpoint_path('best'))
best_iou = max(best_iou, mean_iou)
print("Target: {} Best: {}".format(mean_iou, best_iou))
logger.close()
def train(train_source_iter: ForeverDataIterator, model, interp, criterion, optimizer: SGD,
lr_scheduler: LambdaLR, epoch: int, visualize, args: argparse.Namespace):
batch_time = AverageMeter('Time', ':4.2f')
data_time = AverageMeter('Data', ':3.1f')
losses_s = AverageMeter('Loss (s)', ':3.2f')
accuracies_s = Meter('Acc (s)', ':3.2f')
iou_s = Meter('IoU (s)', ':3.2f')
confmat_s = ConfusionMatrix(model.num_classes)
progress = ProgressMeter(
args.iters_per_epoch,
[batch_time, data_time, losses_s,
accuracies_s, iou_s],
prefix="Epoch: [{}]".format(epoch))
# switch to train mode
model.train()
end = time.time()
for i in range(args.iters_per_epoch):
optimizer.zero_grad()
x_s, label_s = next(train_source_iter)
x_s = x_s.to(device)
label_s = label_s.long().to(device)
# measure data loading time
data_time.update(time.time() - end)
# compute output
y_s = model(x_s)
pred_s = interp(y_s)
loss_cls_s = criterion(pred_s, label_s)
loss_cls_s.backward()
# compute gradient and do SGD step
optimizer.step()
lr_scheduler.step()
# measure accuracy and record loss
losses_s.update(loss_cls_s.item(), x_s.size(0))
confmat_s.update(label_s.flatten(), pred_s.argmax(1).flatten())
acc_global_s, acc_s, iu_s = confmat_s.compute()
accuracies_s.update(acc_s.mean().item())
iou_s.update(iu_s.mean().item())
# measure elapsed time
batch_time.update(time.time() - end)
end = time.time()
if i % args.print_freq == 0:
progress.display(i)
if visualize is not None:
visualize(x_s[0], pred_s[0], label_s[0], "source_{}".format(i))
def validate(val_loader: DataLoader, model, interp, criterion, visualize, args: argparse.Namespace):
batch_time = AverageMeter('Time', ':6.3f')
losses = AverageMeter('Loss', ':.4e')
acc = Meter('Acc', ':3.2f')
iou = Meter('IoU', ':3.2f')
progress = ProgressMeter(
len(val_loader),
[batch_time, losses, acc, iou],
prefix='Test: ')
# switch to evaluate mode
model.eval()
confmat = ConfusionMatrix(model.num_classes)
with torch.no_grad():
end = time.time()
for i, (x, label) in enumerate(val_loader):
x = x.to(device)
label = label.long().to(device)
# compute output
output = interp(model(x))
loss = criterion(output, label)
# measure accuracy and record loss
losses.update(loss.item(), x.size(0))
confmat.update(label.flatten(), output.argmax(1).flatten())
acc_global, accs, iu = confmat.compute()
acc.update(accs.mean().item())
iou.update(iu.mean().item())
# measure elapsed time
batch_time.update(time.time() - end)
end = time.time()
if i % args.print_freq == 0:
progress.display(i)
if visualize is not None:
visualize(x[0], output[0], label[0], "val_{}".format(i))
return confmat
if __name__ == '__main__':
architecture_names = sorted(
name for name in models.__dict__
if name.islower() and not name.startswith("__")
and callable(models.__dict__[name])
)
dataset_names = sorted(
name for name in datasets.__dict__
if not name.startswith("__") and callable(datasets.__dict__[name])
)
parser = argparse.ArgumentParser(description='Source Only for Segmentation Domain Adaptation')
# dataset parameters
parser.add_argument('source_root', help='root path of the source dataset')
parser.add_argument('target_root', help='root path of the target dataset')
parser.add_argument('-s', '--source', help='source domain(s)')
parser.add_argument('-t', '--target', help='target domain(s)')
parser.add_argument('--resize-ratio', nargs='+', type=float, default=(1.5, 8 / 3.),
help='the resize ratio for the random resize crop')
parser.add_argument('--resize-scale', nargs='+', type=float, default=(0.5, 1.),
help='the resize scale for the random resize crop')
parser.add_argument('--train-size', nargs='+', type=int, default=(1024, 512),
help='the input and output image size during training')
parser.add_argument('--test-input-size', nargs='+', type=int, default=(1024, 512),
help='the input image size during test')
parser.add_argument('--test-output-size', nargs='+', type=int, default=(2048, 1024),
help='the output image size during test')
# model parameters
parser.add_argument('-a', '--arch', metavar='ARCH', default='deeplabv2_resnet101',
choices=architecture_names,
help='backbone architecture: ' +
' | '.join(architecture_names) +
' (default: deeplabv2_resnet101)')
parser.add_argument("--resume", type=str, default=None,
help="Where restore model parameters from.")
# training parameters
parser.add_argument('-b', '--batch-size', default=2, type=int,
metavar='N',
help='mini-batch size (default: 2)')
parser.add_argument('--lr', '--learning-rate', default=2.5e-3, type=float,
metavar='LR', help='initial learning rate', dest='lr')
parser.add_argument("--momentum", type=float, default=0.9, help="Momentum component of the optimiser.")
parser.add_argument("--weight-decay", type=float, default=0.0005,
help="Regularisation parameter for L2-loss.")
parser.add_argument("--lr-power", type=float, default=0.9,
help="Decay parameter to compute the learning rate (only for deeplab).")
parser.add_argument('-j', '--workers', default=4, type=int, metavar='N',
help='number of data loading workers (default: 4)')
parser.add_argument('--epochs', default=60, type=int, metavar='N',
help='number of total epochs to run')
parser.add_argument('--start-epoch', default=0, type=int, metavar='N',
help='start epoch')
parser.add_argument('-i', '--iters-per-epoch', default=2500, type=int,
help='Number of iterations per epoch')
parser.add_argument('-p', '--print-freq', default=100, type=int,
metavar='N', help='print frequency (default: 100)')
parser.add_argument('--seed', default=None, type=int,
help='seed for initializing training. ')
parser.add_argument("--ignore-label", type=int, default=255,
help="The index of the label to ignore during the training.")
parser.add_argument("--log", type=str, default='src_only',
help="Where to save logs, checkpoints and debugging images.")
parser.add_argument("--phase", type=str, default='train', choices=['train', 'test'],
help="When phase is 'test', only test the model.")
parser.add_argument('--debug', action="store_true",
help='In the debug mode, save images and predictions during training')
args = parser.parse_args()
main(args)
================================================
FILE: domain_generalization/Transfer-Learning-Library/examples/domain_adaptation/semantic_segmentation/source_only.sh
================================================
# Source Only
# GTA5 to Cityscapes
CUDA_VISIBLE_DEVICES=0 python source_only.py data/GTA5 data/Cityscapes \
-s GTA5 -t Cityscapes --log logs/src_only/gtav2cityscapes
# Synthia to Cityscapes
CUDA_VISIBLE_DEVICES=0 python source_only.py data/synthia data/Cityscapes \
-s Synthia -t Cityscapes --log logs/src_only/synthia2cityscapes
# Cityscapes to FoggyCityscapes
CUDA_VISIBLE_DEVICES=0 python source_only.py data/Cityscapes data/Cityscapes \
-s Cityscapes -t FoggyCityscapes --log logs/src_only/cityscapes2foggy
# Oracle
# Oracle Results on Cityscapes
CUDA_VISIBLE_DEVICES=0 python source_only.py data/Cityscapes data/Cityscapes \
-s Cityscapes -t Cityscapes --log logs/oracle/cityscapes
# Oracle Results on Foggy Cityscapes
CUDA_VISIBLE_DEVICES=0 python source_only.py data/Cityscapes data/Cityscapes \
-s FoggyCityscapes -t FoggyCityscapes --log logs/oracle/foggy_cityscapes
================================================
FILE: domain_generalization/Transfer-Learning-Library/examples/domain_generalization/image_classification/README.md
================================================
# Domain Generalization for Image Classification
## Installation
Example scripts can deal with [WILDS datasets](https://wilds.stanford.edu/).
You should first install ``wilds`` before using these scripts.
```
pip install wilds
```
Example scripts also support all models in [PyTorch-Image-Models](https://github.com/rwightman/pytorch-image-models).
You also need to install timm to use PyTorch-Image-Models.
```
pip install timm
```
## Dataset
Following datasets can be downloaded automatically:
- [Office31](https://www.cc.gatech.edu/~judy/domainadapt/)
- [OfficeHome](https://www.hemanthdv.org/officeHomeDataset.html)
- [DomainNet](http://ai.bu.edu/M3SDA/)
- [PACS](https://domaingeneralization.github.io/#data)
- [iwildcam (WILDS)](https://wilds.stanford.edu/datasets/)
- [camelyon17 (WILDS)](https://wilds.stanford.edu/datasets/)
- [fmow (WILDS)](https://wilds.stanford.edu/datasets/)
## Supported Methods
- [Two at Once: Enhancing Learning and Generalization Capacities via IBN-Net (IBN-Net, 2018 ECCV)](https://openaccess.thecvf.com/content_ECCV_2018/papers/Xingang_Pan_Two_at_Once_ECCV_2018_paper.pdf)
- [Domain Generalization with MixStyle (MixStyle, 2021 ICLR)](https://arxiv.org/abs/2104.02008)
- [Learning to Generalize: Meta-Learning for Domain Generalization (MLDG, 2018 AAAI)](https://arxiv.org/pdf/1710.03463.pdf)
- [Invariant Risk Minimization (IRM)](https://arxiv.org/abs/1907.02893)
- [Out-of-Distribution Generalization via Risk Extrapolation (VREx, 2021 ICML)](https://arxiv.org/abs/2003.00688)
- [Distributionally Robust Neural Networks for Group Shifts: On the Importance of Regularization for Worst-Case Generalization (GroupDRO)](https://arxiv.org/abs/1911.08731)
- [Deep CORAL: Correlation Alignment for Deep Domain Adaptation (Deep Coral, 2016 ECCV)](https://arxiv.org/abs/1607.01719)
## Experiment and Results
The shell files give the script to reproduce the [benchmarks](/docs/dglib/benchmarks/image_classification.rst) with specified hyper-parameters.
For example, if you want to reproduce IRM on Office-Home, use the following script
```shell script
# Train with IRM on Office-Home Ar Cl Rw -> Pr task using ResNet 50.
# Assume you have put the datasets under the path `data/office-home`,
# or you are glad to download the datasets automatically from the Internet to this path
CUDA_VISIBLE_DEVICES=0 python irm.py data/office-home -d OfficeHome -s Ar Cl Rw -t Pr -a resnet50 --seed 0 --log logs/irm/OfficeHome_Pr
```
For more information please refer to [Get Started](/docs/get_started/quickstart.rst) for help.
## Citation
If you use these methods in your research, please consider citing.
```
@inproceedings{IBN-Net,
author = {Xingang Pan, Ping Luo, Jianping Shi, and Xiaoou Tang},
title = {Two at Once: Enhancing Learning and Generalization Capacities via IBN-Net},
booktitle = {ECCV},
year = {2018}
}
@inproceedings{mixstyle,
title={Domain Generalization with MixStyle},
author={Zhou, Kaiyang and Yang, Yongxin and Qiao, Yu and Xiang, Tao},
booktitle={ICLR},
year={2021}
}
@inproceedings{MLDG,
title={Learning to Generalize: Meta-Learning for Domain Generalization},
author={Li, Da and Yang, Yongxin and Song, Yi-Zhe and Hospedales, Timothy},
booktitle={AAAI},
year={2018}
}
@misc{IRM,
title={Invariant Risk Minimization},
author={Martin Arjovsky and Léon Bottou and Ishaan Gulrajani and David Lopez-Paz},
year={2020},
eprint={1907.02893},
archivePrefix={arXiv},
primaryClass={stat.ML}
}
@inproceedings{VREx,
title={Out-of-Distribution Generalization via Risk Extrapolation (REx)},
author={David Krueger and Ethan Caballero and Joern-Henrik Jacobsen and Amy Zhang and Jonathan Binas and Dinghuai Zhang and Remi Le Priol and Aaron Courville},
year={2021},
booktitle={ICML},
}
@inproceedings{GroupDRO,
title={Distributionally Robust Neural Networks for Group Shifts: On the Importance of Regularization for Worst-Case Generalization},
author={Shiori Sagawa and Pang Wei Koh and Tatsunori B. Hashimoto and Percy Liang},
year={2020},
booktitle={ICLR}
}
@inproceedings{deep_coral,
title={Deep coral: Correlation alignment for deep domain adaptation},
author={Sun, Baochen and Saenko, Kate},
booktitle={ECCV},
year={2016},
}
```
================================================
FILE: domain_generalization/Transfer-Learning-Library/examples/domain_generalization/image_classification/baseline.py
================================================
"""
@author: Baixu Chen
@contact: cbx_99_hasta@outlook.com
"""
import random
import time
import warnings
import sys
import argparse
import shutil
import os.path as osp
import torch
import torch.nn as nn
import torch.backends.cudnn as cudnn
from torch.optim import SGD
from torch.optim.lr_scheduler import CosineAnnealingLR
from torch.utils.data import DataLoader
import torch.nn.functional as F
sys.path.append('../../..')
from dglib.modules.classifier import ImageClassifier as Classifier
from common.utils.data import ForeverDataIterator
from common.utils.metric import accuracy
from common.utils.meter import AverageMeter, ProgressMeter
from common.utils.logger import CompleteLogger
from common.utils.analysis import tsne, a_distance
sys.path.append('.')
import utils
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
class TransformerDecorator(torch.nn.Module):
def __init__(self, backbone=None, add_bt=3, dim=2048, eval_global=0):
super(TransformerDecorator, self).__init__()
self.encoder_layers = torch.nn.TransformerEncoderLayer(dim, 4, dim, 0.5)
self.eval_global = eval_global
self.add_bt = add_bt
self.backbone = backbone
def forward_feats(self, feature):
feature = self.backbone(feature)
if self.training and self.add_bt > 0:
pre_feature = feature
feature = feature.unsqueeze(1)
feature = self.encoder_layers(feature)
# feature = self.encoder_layers(feature, feature, feature)
feature = feature.squeeze(1)
if self.add_bt:
return torch.cat([pre_feature, feature], dim=0)
return feature
def forward(self, feature):
feature = self.forward_feats(feature)
return feature
def main(args: argparse.Namespace):
logger = CompleteLogger(args.log, args.phase)
print(args)
if args.seed is not None:
random.seed(args.seed)
torch.manual_seed(args.seed)
cudnn.deterministic = True
warnings.warn('You have chosen to seed training. '
'This will turn on the CUDNN deterministic setting, '
'which can slow down your training considerably! '
'You may see unexpected behavior when restarting '
'from checkpoints.')
cudnn.benchmark = True
# Data loading code
train_transform = utils.get_train_transform(args.train_resizing, random_horizontal_flip=True,
random_color_jitter=True, random_gray_scale=True)
val_transform = utils.get_val_transform(args.val_resizing)
print("train_transform: ", train_transform)
print("val_transform: ", val_transform)
train_dataset, num_classes = utils.get_dataset(dataset_name=args.data, root=args.root, task_list=args.sources,
split='train', download=True, transform=train_transform,
seed=args.seed)
train_loader = DataLoader(train_dataset, batch_size=args.batch_size,
shuffle=True, num_workers=args.workers, drop_last=True)
val_dataset, _ = utils.get_dataset(dataset_name=args.data, root=args.root, task_list=args.sources, split='val',
download=True, transform=val_transform, seed=args.seed)
val_loader = DataLoader(val_dataset, batch_size=args.batch_size, shuffle=False, num_workers=args.workers)
test_dataset, _ = utils.get_dataset(dataset_name=args.data, root=args.root, task_list=args.targets, split='test',
download=True, transform=val_transform, seed=args.seed)
test_loader = DataLoader(test_dataset, batch_size=args.batch_size, shuffle=False, num_workers=args.workers)
print("train_dataset_size: ", len(train_dataset))
print('val_dataset_size: ', len(val_dataset))
print("test_dataset_size: ", len(test_dataset))
train_iter = ForeverDataIterator(train_loader)
# create model
print("=> using pre-trained model '{}'".format(args.arch))
backbone = utils.get_model(args.arch)
pool_layer = nn.Identity() if args.no_pool else None
pool_layer = nn.Sequential(
nn.AdaptiveAvgPool2d(output_size=(1, 1)),
nn.Flatten(),
)
pool_layer = TransformerDecorator(pool_layer, add_bt=args.add_bt, dim=backbone.out_features)
classifier = Classifier(backbone, num_classes, freeze_bn=args.freeze_bn, dropout_p=args.dropout_p,
finetune=args.finetune, pool_layer=pool_layer).to(device)
# define optimizer and lr scheduler
optimizer = SGD(classifier.get_parameters(base_lr=args.lr), args.lr, momentum=args.momentum, weight_decay=args.wd,
nesterov=True)
lr_scheduler = CosineAnnealingLR(optimizer, args.epochs * args.iters_per_epoch)
# resume from the best checkpoint
if args.phase != 'train':
checkpoint = torch.load(logger.get_checkpoint_path('best'), map_location='cpu')
classifier.load_state_dict(checkpoint)
# analysis the model
if args.phase == 'analysis':
# extract features from both domains
feature_extractor = nn.Sequential(classifier.backbone, classifier.pool_layer, classifier.bottleneck).to(device)
source_feature = utils.collect_feature(val_loader, feature_extractor, device, max_num_features=100)
target_feature = utils.collect_feature(test_loader, feature_extractor, device, max_num_features=100)
print(len(source_feature), len(target_feature))
# plot t-SNE
tSNE_filename = osp.join(logger.visualize_directory, 'TSNE.png')
tsne.visualize(source_feature, target_feature, tSNE_filename)
print("Saving t-SNE to", tSNE_filename)
# calculate A-distance, which is a measure for distribution discrepancy
A_distance = a_distance.calculate(source_feature, target_feature, device)
print("A-distance =", A_distance)
return
if args.phase == 'test':
acc1 = utils.validate(test_loader, classifier, args, device)
print(acc1)
return
# start training
best_val_acc1 = 0.
best_test_acc1 = 0.
for epoch in range(args.epochs):
print(lr_scheduler.get_lr())
# train for one epoch
train(train_iter, classifier, optimizer, lr_scheduler, epoch, args)
# evaluate on validation set
print("Evaluate on validation set...")
acc1 = utils.validate(val_loader, classifier, args, device)
# remember best acc@1 and save checkpoint
torch.save(classifier.state_dict(), logger.get_checkpoint_path('latest'))
if acc1 > best_val_acc1:
shutil.copy(logger.get_checkpoint_path('latest'), logger.get_checkpoint_path('best'))
best_val_acc1 = max(acc1, best_val_acc1)
# evaluate on test set
print("Evaluate on test set...")
best_test_acc1 = max(best_test_acc1, utils.validate(test_loader, classifier, args, device))
# evaluate on test set
classifier.load_state_dict(torch.load(logger.get_checkpoint_path('best')))
acc1 = utils.validate(test_loader, classifier, args, device)
print("test acc on test set = {}".format(acc1))
print("oracle acc on test set = {}".format(best_test_acc1))
logger.close()
def train(train_iter: ForeverDataIterator, model: Classifier, optimizer,
lr_scheduler: CosineAnnealingLR, epoch: int, args: argparse.Namespace):
batch_time = AverageMeter('Time', ':4.2f')
data_time = AverageMeter('Data', ':3.1f')
losses = AverageMeter('Loss', ':3.2f')
cls_accs = AverageMeter('Cls Acc', ':3.1f')
progress = ProgressMeter(
args.iters_per_epoch,
[batch_time, data_time, losses, cls_accs],
prefix="Epoch: [{}]".format(epoch))
# switch to train mode
model.train()
end = time.time()
for i in range(args.iters_per_epoch):
x, labels, _ = next(train_iter)
x = x.to(device)
labels = labels.to(device)
# measure data loading time
data_time.update(time.time() - end)
# compute output
y, _ = model(x)
if len(y) > len(labels):
labels = torch.cat([labels, labels], dim=0)
loss = F.cross_entropy(y, labels)
cls_acc = accuracy(y, labels)[0]
losses.update(loss.item(), x.size(0))
cls_accs.update(cls_acc.item(), x.size(0))
# compute gradient and do SGD step
optimizer.zero_grad()
loss.backward()
optimizer.step()
lr_scheduler.step()
# measure elapsed time
batch_time.update(time.time() - end)
end = time.time()
if i % args.print_freq == 0:
progress.display(i)
if __name__ == '__main__':
parser = argparse.ArgumentParser(description='Baseline for Domain Generalization')
# dataset parameters
parser.add_argument('root', metavar='DIR',
help='root path of dataset')
parser.add_argument('-d', '--data', metavar='DATA', default='PACS',
help='dataset: ' + ' | '.join(utils.get_dataset_names()) +
' (default: PACS)')
parser.add_argument('-s', '--sources', nargs='+', default=None,
help='source domain(s)')
parser.add_argument('-t', '--targets', nargs='+', default=None,
help='target domain(s)')
parser.add_argument('--train-resizing', type=str, default='default')
parser.add_argument('--val-resizing', type=str, default='default')
# model parameters
parser.add_argument('-a', '--arch', metavar='ARCH', default='resnet50',
choices=utils.get_model_names(),
help='backbone architecture: ' +
' | '.join(utils.get_model_names()) +
' (default: resnet50)')
parser.add_argument('--no-pool', action='store_true', help='no pool layer after the feature extractor.')
parser.add_argument('--finetune', action='store_true', help='whether use 10x smaller lr for backbone')
parser.add_argument('--freeze-bn', action='store_true', help='whether freeze all bn layers')
parser.add_argument('--dropout-p', type=float, default=0.1, help='only activated when freeze-bn is True')
# training parameters
parser.add_argument('-b', '--batch-size', default=36, type=int,
metavar='N',
help='mini-batch size (default: 36)')
parser.add_argument('--lr', '--learning-rate', default=1e-3, type=float,
metavar='LR', help='initial learning rate', dest='lr')
parser.add_argument('--momentum', default=0.9, type=float, metavar='M',
help='momentum')
parser.add_argument('--wd', '--weight-decay', default=0.0005, type=float,
metavar='W', help='weight decay (default: 5e-4)')
parser.add_argument('-j', '--workers', default=4, type=int, metavar='N',
help='number of data loading workers (default: 4)')
parser.add_argument('--epochs', default=20, type=int, metavar='N',
help='number of total epochs to run')
parser.add_argument('-i', '--iters-per-epoch', default=500, type=int,
help='Number of iterations per epoch')
parser.add_argument('-p', '--print-freq', default=100, type=int,
metavar='N', help='print frequency (default: 100)')
parser.add_argument('--seed', default=0, type=int,
help='seed for initializing training. ')
parser.add_argument("--log", type=str, default='baseline',
help="Where to save logs, checkpoints and debugging images.")
parser.add_argument("--phase", type=str, default='train', choices=['train', 'test', 'analysis'],
help="When phase is 'test', only test the model."
"When phase is 'analysis', only analysis the model.")
parser.add_argument("--add_bt", type=int, default=0,
help="add_bt")
args = parser.parse_args()
main(args)
================================================
FILE: domain_generalization/Transfer-Learning-Library/examples/domain_generalization/image_classification/baseline.sh
================================================
#!/usr/bin/env bash
# ResNet50, PACS
CUDA_VISIBLE_DEVICES=0 python baseline.py data/PACS -d PACS -s A C S -t P -a resnet50 --freeze-bn --seed 0 --log logs/baseline/PACS_P
CUDA_VISIBLE_DEVICES=0 python baseline.py data/PACS -d PACS -s P C S -t A -a resnet50 --freeze-bn --seed 0 --log logs/baseline/PACS_A
CUDA_VISIBLE_DEVICES=0 python baseline.py data/PACS -d PACS -s P A S -t C -a resnet50 --freeze-bn --seed 0 --log logs/baseline/PACS_C
CUDA_VISIBLE_DEVICES=0 python baseline.py data/PACS -d PACS -s P A C -t S -a resnet50 --freeze-bn --seed 0 --log logs/baseline/PACS_S
# ResNet50, Office-Home
CUDA_VISIBLE_DEVICES=0 python baseline.py data/office-home -d OfficeHome -s Ar Cl Rw -t Pr -a resnet50 --seed 0 --log logs/baseline/OfficeHome_Pr
CUDA_VISIBLE_DEVICES=0 python baseline.py data/office-home -d OfficeHome -s Ar Cl Pr -t Rw -a resnet50 --seed 0 --log logs/baseline/OfficeHome_Rw
CUDA_VISIBLE_DEVICES=0 python baseline.py data/office-home -d OfficeHome -s Ar Rw Pr -t Cl -a resnet50 --seed 0 --log logs/baseline/OfficeHome_Cl
CUDA_VISIBLE_DEVICES=0 python baseline.py data/office-home -d OfficeHome -s Cl Rw Pr -t Ar -a resnet50 --seed 0 --log logs/baseline/OfficeHome_Ar
# ResNet50, DomainNet
CUDA_VISIBLE_DEVICES=0 python baseline.py data/domainnet -d DomainNet -s i p q r s -t c -a resnet50 -i 2500 --lr 0.01 --seed 0 --log logs/baseline/DomainNet_c
CUDA_VISIBLE_DEVICES=0 python baseline.py data/domainnet -d DomainNet -s c p q r s -t i -a resnet50 -i 2500 --lr 0.01 --seed 0 --log logs/baseline/DomainNet_i
CUDA_VISIBLE_DEVICES=0 python baseline.py data/domainnet -d DomainNet -s c i q r s -t p -a resnet50 -i 2500 --lr 0.01 --seed 0 --log logs/baseline/DomainNet_p
CUDA_VISIBLE_DEVICES=0 python baseline.py data/domainnet -d DomainNet -s c i p r s -t q -a resnet50 -i 2500 --lr 0.01 --seed 0 --log logs/baseline/DomainNet_q
CUDA_VISIBLE_DEVICES=0 python baseline.py data/domainnet -d DomainNet -s c i p q s -t r -a resnet50 -i 2500 --lr 0.01 --seed 0 --log logs/baseline/DomainNet_r
CUDA_VISIBLE_DEVICES=0 python baseline.py data/domainnet -d DomainNet -s c i p q r -t s -a resnet50 -i 2500 --lr 0.01 --seed 0 --log logs/baseline/DomainNet_s
# ResNet50, Wilds Dataset
CUDA_VISIBLE_DEVICES=0 python baseline.py data/wilds -d iwildcam --train-resizing 'res2x' --val-resizing 'res2x' \
-a resnet50 -b 16 --epochs 60 -i 1000 --lr 0.001 --finetune --seed 0 --log logs/baseline/iwildcam
CUDA_VISIBLE_DEVICES=0 python baseline.py data/wilds -d camelyon17 -a resnet50 -b 36 --epochs 20 -i 1000 --lr 0.01 \
--seed 0 --log logs/baseline/camelyon17
# DenseNet121, Wilds Dataset
CUDA_VISIBLE_DEVICES=0 python baseline.py data/wilds -d fmow --train-resizing 'cen.crop' -a densenet121 \
-b 36 --epochs 60 -i 1000 --lr 0.003 --seed 0 --log logs/baseline/fmow
================================================
FILE: domain_generalization/Transfer-Learning-Library/examples/domain_generalization/image_classification/coral.py
================================================
"""
@author: Baixu Chen
@contact: cbx_99_hasta@outlook.com
"""
import random
import time
import warnings
import sys
import argparse
import shutil
import os.path as osp
import torch
import torch.nn as nn
import torch.backends.cudnn as cudnn
from torch.optim import SGD
from torch.optim.lr_scheduler import CosineAnnealingLR
from torch.utils.data import DataLoader
import torch.nn.functional as F
sys.path.append('../../..')
from dglib.generalization.coral import CorrelationAlignmentLoss
from dglib.modules.sampler import RandomDomainSampler
from dglib.modules.classifier import ImageClassifier as Classifier
from common.utils.data import ForeverDataIterator
from common.utils.metric import accuracy
from common.utils.meter import AverageMeter, ProgressMeter
from common.utils.logger import CompleteLogger
from common.utils.analysis import tsne, a_distance
sys.path.append('.')
import utils
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
class TransformerDecorator(torch.nn.Module):
def __init__(self, backbone=None, add_bt=3, dim=2048, eval_global=0):
super(TransformerDecorator, self).__init__()
self.encoder_layers = torch.nn.TransformerEncoderLayer(dim, 4, dim, 0.5)
self.eval_global = eval_global
self.add_bt = add_bt
self.backbone = backbone
def forward_feats(self, feature):
feature = self.backbone(feature)
if self.training and self.add_bt > 0:
pre_feature = feature
feature = feature.unsqueeze(1)
feature = self.encoder_layers(feature)
# feature = self.encoder_layers(feature, feature, feature)
feature = feature.squeeze(1)
if self.add_bt:
return torch.cat([pre_feature, feature], dim=0)
return feature
def forward(self, feature):
feature = self.forward_feats(feature)
return feature
def main(args: argparse.Namespace):
logger = CompleteLogger(args.log, args.phase)
print(args)
if args.seed is not None:
random.seed(args.seed)
torch.manual_seed(args.seed)
cudnn.deterministic = True
warnings.warn('You have chosen to seed training. '
'This will turn on the CUDNN deterministic setting, '
'which can slow down your training considerably! '
'You may see unexpected behavior when restarting '
'from checkpoints.')
cudnn.benchmark = True
# Data loading code
train_transform = utils.get_train_transform(args.train_resizing, random_horizontal_flip=True,
random_color_jitter=True, random_gray_scale=True)
val_transform = utils.get_val_transform(args.val_resizing)
print("train_transform: ", train_transform)
print("val_transform: ", val_transform)
train_dataset, num_classes = utils.get_dataset(dataset_name=args.data, root=args.root, task_list=args.sources,
split='train', download=True, transform=train_transform,
seed=args.seed)
sampler = RandomDomainSampler(train_dataset, args.batch_size, n_domains_per_batch=args.n_domains_per_batch)
train_loader = DataLoader(train_dataset, batch_size=args.batch_size, num_workers=args.workers,
sampler=sampler, drop_last=True)
val_dataset, _ = utils.get_dataset(dataset_name=args.data, root=args.root, task_list=args.sources, split='val',
download=True, transform=val_transform, seed=args.seed)
val_loader = DataLoader(val_dataset, batch_size=args.batch_size, shuffle=False, num_workers=args.workers)
test_dataset, _ = utils.get_dataset(dataset_name=args.data, root=args.root, task_list=args.targets, split='test',
download=True, transform=val_transform, seed=args.seed)
test_loader = DataLoader(test_dataset, batch_size=args.batch_size, shuffle=False, num_workers=args.workers)
print("train_dataset_size: ", len(train_dataset))
print('val_dataset_size: ', len(val_dataset))
print("test_dataset_size: ", len(test_dataset))
train_iter = ForeverDataIterator(train_loader)
# create model
print("=> using pre-trained model '{}'".format(args.arch))
backbone = utils.get_model(args.arch)
pool_layer = nn.Identity() if args.no_pool else None
pool_layer = nn.Sequential(
nn.AdaptiveAvgPool2d(output_size=(1, 1)),
nn.Flatten(),
)
pool_layer = TransformerDecorator(pool_layer, add_bt=args.add_bt, dim=backbone.out_features)
classifier = Classifier(backbone, num_classes, freeze_bn=args.freeze_bn, dropout_p=args.dropout_p,
finetune=args.finetune, pool_layer=pool_layer).to(device)
# define optimizer and lr scheduler
optimizer = SGD(classifier.get_parameters(base_lr=args.lr), args.lr, momentum=args.momentum, weight_decay=args.wd,
nesterov=True)
lr_scheduler = CosineAnnealingLR(optimizer, args.epochs * args.iters_per_epoch)
# define loss function
correlation_alignment_loss = CorrelationAlignmentLoss().to(device)
# resume from the best checkpoint
if args.phase != 'train':
checkpoint = torch.load(logger.get_checkpoint_path('best'), map_location='cpu')
classifier.load_state_dict(checkpoint)
# analysis the model
if args.phase == 'analysis':
# extract features from both domains
feature_extractor = nn.Sequential(classifier.backbone, classifier.pool_layer, classifier.bottleneck).to(device)
source_feature = utils.collect_feature(val_loader, feature_extractor, device, max_num_features=100)
target_feature = utils.collect_feature(test_loader, feature_extractor, device, max_num_features=100)
print(len(source_feature), len(target_feature))
# plot t-SNE
tSNE_filename = osp.join(logger.visualize_directory, 'TSNE.png')
tsne.visualize(source_feature, target_feature, tSNE_filename)
print("Saving t-SNE to", tSNE_filename)
# calculate A-distance, which is a measure for distribution discrepancy
A_distance = a_distance.calculate(source_feature, target_feature, device)
print("A-distance =", A_distance)
return
if args.phase == 'test':
acc1 = utils.validate(test_loader, classifier, args, device)
print(acc1)
return
# start training
best_val_acc1 = 0.
best_test_acc1 = 0.
for epoch in range(args.epochs):
print(lr_scheduler.get_lr())
# train for one epoch
train(train_iter, classifier, optimizer, lr_scheduler, correlation_alignment_loss, args.n_domains_per_batch,
epoch, args)
# evaluate on validation set
print("Evaluate on validation set...")
acc1 = utils.validate(val_loader, classifier, args, device)
# remember best acc@1 and save checkpoint
torch.save(classifier.state_dict(), logger.get_checkpoint_path('latest'))
if acc1 > best_val_acc1:
shutil.copy(logger.get_checkpoint_path('latest'), logger.get_checkpoint_path('best'))
best_val_acc1 = max(acc1, best_val_acc1)
# evaluate on test set
print("Evaluate on test set...")
best_test_acc1 = max(best_test_acc1, utils.validate(test_loader, classifier, args, device))
# evaluate on test set
classifier.load_state_dict(torch.load(logger.get_checkpoint_path('best')))
acc1 = utils.validate(test_loader, classifier, args, device)
print("test acc on test set = {}".format(acc1))
print("oracle acc on test set = {}".format(best_test_acc1))
logger.close()
def train(train_iter: ForeverDataIterator, model: Classifier, optimizer, lr_scheduler: CosineAnnealingLR,
correlation_alignment_loss: CorrelationAlignmentLoss, n_domains_per_batch: int, epoch: int,
args: argparse.Namespace):
batch_time = AverageMeter('Time', ':4.2f')
data_time = AverageMeter('Data', ':3.1f')
losses = AverageMeter('Loss', ':3.2f')
losses_ce = AverageMeter('CELoss', ':3.2f')
losses_penalty = AverageMeter('Penalty Loss', ':3.2f')
cls_accs = AverageMeter('Cls Acc', ':3.1f')
progress = ProgressMeter(
args.iters_per_epoch,
[batch_time, data_time, losses, losses_ce, losses_penalty, cls_accs],
prefix="Epoch: [{}]".format(epoch))
# switch to train mode
model.train()
end = time.time()
for i in range(args.iters_per_epoch):
x_all, labels_all, _ = next(train_iter)
x_all = x_all.to(device)
labels_all = labels_all.to(device)
# compute output
y_all, f_all = model(x_all)
# measure data loading time
data_time.update(time.time() - end)
if len(y_all) > len(labels_all):
labels_all = torch.cat([labels_all, labels_all], dim=0)
y_all_ = y_all.chunk(n_domains_per_batch*2, dim=0)
f_all_ = f_all.chunk(n_domains_per_batch*2, dim=0)
labels_all_ = labels_all.chunk(n_domains_per_batch*2, dim=0)
y_all = []
f_all = []
labels_all = []
for i in range(n_domains_per_batch):
y_all.append(torch.cat([y_all_[i], y_all_[i+n_domains_per_batch]], dim=0))
f_all.append(torch.cat([f_all_[i], f_all_[i+n_domains_per_batch]], dim=0))
labels_all.append(torch.cat([labels_all_[i], labels_all_[i+n_domains_per_batch]], dim=0))
else:
# separate into different domains
y_all = y_all.chunk(n_domains_per_batch, dim=0)
f_all = f_all.chunk(n_domains_per_batch, dim=0)
labels_all = labels_all.chunk(n_domains_per_batch, dim=0)
loss_ce = 0
loss_penalty = 0
cls_acc = 0
for domain_i in range(n_domains_per_batch):
# cls loss
y_i, labels_i = y_all[domain_i], labels_all[domain_i]
loss_ce += F.cross_entropy(y_i, labels_i)
# update acc
cls_acc += accuracy(y_i, labels_i)[0] / n_domains_per_batch
# correlation alignment loss
for domain_j in range(domain_i + 1, n_domains_per_batch):
f_i = f_all[domain_i]
f_j = f_all[domain_j]
loss_penalty += correlation_alignment_loss(f_i, f_j)
# normalize loss
loss_ce /= n_domains_per_batch
loss_penalty /= n_domains_per_batch * (n_domains_per_batch - 1) / 2
loss = loss_ce + loss_penalty * args.trade_off
losses.update(loss.item(), x_all.size(0))
losses_ce.update(loss_ce.item(), x_all.size(0))
losses_penalty.update(loss_penalty.item(), x_all.size(0))
cls_accs.update(cls_acc.item(), x_all.size(0))
# compute gradient and do SGD step
optimizer.zero_grad()
loss.backward()
optimizer.step()
lr_scheduler.step()
# measure elapsed time
batch_time.update(time.time() - end)
end = time.time()
if i % args.print_freq == 0:
progress.display(i)
if __name__ == '__main__':
parser = argparse.ArgumentParser(description='CORAL for Domain Generalization')
# dataset parameters
parser.add_argument('root', metavar='DIR',
help='root path of dataset')
parser.add_argument('-d', '--data', metavar='DATA', default='PACS',
help='dataset: ' + ' | '.join(utils.get_dataset_names()) +
' (default: PACS)')
parser.add_argument('-s', '--sources', nargs='+', default=None,
help='source domain(s)')
parser.add_argument('-t', '--targets', nargs='+', default=None,
help='target domain(s)')
parser.add_argument('--train-resizing', type=str, default='default')
parser.add_argument('--val-resizing', type=str, default='default')
# model parameters
parser.add_argument('-a', '--arch', metavar='ARCH', default='resnet50',
choices=utils.get_model_names(),
help='backbone architecture: ' +
' | '.join(utils.get_model_names()) +
' (default: resnet50)')
parser.add_argument('--no-pool', action='store_true', help='no pool layer after the feature extractor.')
parser.add_argument('--finetune', action='store_true', help='whether use 10x smaller lr for backbone')
parser.add_argument('--freeze-bn', action='store_true', help='whether freeze all bn layers')
parser.add_argument('--dropout-p', type=float, default=0.1, help='only activated when freeze-bn is True')
# training parameters
parser.add_argument('--trade-off', default=1, type=float,
help='the trade off hyper parameter for correlation alignment loss')
parser.add_argument('-b', '--batch-size', default=36, type=int,
metavar='N',
help='mini-batch size (default: 36)')
parser.add_argument('--n-domains-per-batch', default=3, type=int,
help='number of domains in each mini-batch')
parser.add_argument('--lr', '--learning-rate', default=5e-4, type=float,
metavar='LR', help='initial learning rate', dest='lr')
parser.add_argument('--momentum', default=0.9, type=float, metavar='M',
help='momentum')
parser.add_argument('--wd', '--weight-decay', default=0.0005, type=float,
metavar='W', help='weight decay (default: 5e-4)')
parser.add_argument('-j', '--workers', default=4, type=int, metavar='N',
help='number of data loading workers (default: 4)')
parser.add_argument('--epochs', default=20, type=int, metavar='N',
help='number of total epochs to run')
parser.add_argument('-i', '--iters-per-epoch', default=500, type=int,
help='Number of iterations per epoch')
parser.add_argument('-p', '--print-freq', default=100, type=int,
metavar='N', help='print frequency (default: 100)')
parser.add_argument('--seed', default=None, type=int,
help='seed for initializing training. ')
parser.add_argument("--log", type=str, default='coral',
help="Where to save logs, checkpoints and debugging images.")
parser.add_argument("--phase", type=str, default='train', choices=['train', 'test', 'analysis'],
help="When phase is 'test', only test the model."
"When phase is 'analysis', only analysis the model.")
parser.add_argument("--add_bt", type=int, default=0,
help="add_bt")
args = parser.parse_args()
main(args)
================================================
FILE: domain_generalization/Transfer-Learning-Library/examples/domain_generalization/image_classification/coral.sh
================================================
#!/usr/bin/env bash
# ResNet50, PACS
CUDA_VISIBLE_DEVICES=0 python coral.py data/PACS -d PACS -s A C S -t P -a resnet50 --freeze-bn --seed 0 --log logs/coral/PACS_P
CUDA_VISIBLE_DEVICES=0 python coral.py data/PACS -d PACS -s P C S -t A -a resnet50 --freeze-bn --seed 0 --log logs/coral/PACS_A
CUDA_VISIBLE_DEVICES=0 python coral.py data/PACS -d PACS -s P A S -t C -a resnet50 --freeze-bn --seed 0 --log logs/coral/PACS_C
CUDA_VISIBLE_DEVICES=0 python coral.py data/PACS -d PACS -s P A C -t S -a resnet50 --freeze-bn --seed 0 --log logs/coral/PACS_S
# ResNet50, Office-Home
CUDA_VISIBLE_DEVICES=0 python coral.py data/office-home -d OfficeHome -s Ar Cl Rw -t Pr -a resnet50 --seed 0 --log logs/coral/OfficeHome_Pr
CUDA_VISIBLE_DEVICES=0 python coral.py data/office-home -d OfficeHome -s Ar Cl Pr -t Rw -a resnet50 --seed 0 --log logs/coral/OfficeHome_Rw
CUDA_VISIBLE_DEVICES=0 python coral.py data/office-home -d OfficeHome -s Ar Rw Pr -t Cl -a resnet50 --seed 0 --log logs/coral/OfficeHome_Cl
CUDA_VISIBLE_DEVICES=0 python coral.py data/office-home -d OfficeHome -s Cl Rw Pr -t Ar -a resnet50 --seed 0 --log logs/coral/OfficeHome_Ar
# ResNet50, DomainNet
CUDA_VISIBLE_DEVICES=0 python coral.py data/domainnet -d DomainNet -s i p q r s -t c -a resnet50 -i 2500 --lr 0.01 --seed 0 --log logs/coral/DomainNet_c
CUDA_VISIBLE_DEVICES=0 python coral.py data/domainnet -d DomainNet -s c p q r s -t i -a resnet50 -i 2500 --lr 0.01 --seed 0 --log logs/coral/DomainNet_i
CUDA_VISIBLE_DEVICES=0 python coral.py data/domainnet -d DomainNet -s c i q r s -t p -a resnet50 -i 2500 --lr 0.01 --seed 0 --log logs/coral/DomainNet_p
CUDA_VISIBLE_DEVICES=0 python coral.py data/domainnet -d DomainNet -s c i p r s -t q -a resnet50 -i 2500 --lr 0.01 --seed 0 --log logs/coral/DomainNet_q
CUDA_VISIBLE_DEVICES=0 python coral.py data/domainnet -d DomainNet -s c i p q s -t r -a resnet50 -i 2500 --lr 0.01 --seed 0 --log logs/coral/DomainNet_r
CUDA_VISIBLE_DEVICES=0 python coral.py data/domainnet -d DomainNet -s c i p q r -t s -a resnet50 -i 2500 --lr 0.01 --seed 0 --log logs/coral/DomainNet_s
# ResNet50, Wilds Dataset
CUDA_VISIBLE_DEVICES=0 python coral.py data/wilds -d iwildcam --train-resizing 'res2x' --val-resizing 'res2x' \
-a resnet50 -b 16 --n-domains-per-batch 2 --epochs 60 -i 1000 --lr 0.001 --finetune --trade-off 10 \
--seed 0 --log logs/coral/iwildcam
CUDA_VISIBLE_DEVICES=0 python coral.py data/wilds -d camelyon17 -a resnet50 -b 36 --n-domains-per-batch 3 \
--epochs 20 -i 1000 --lr 0.01 --trade-off 0.1 --seed 0 --log logs/coral/camelyon17
# DenseNet121, Wilds Dataset
CUDA_VISIBLE_DEVICES=0 python coral.py data/wilds -d fmow --train-resizing 'cen.crop' -a densenet121 \
-b 36 --n-domains-per-batch 4 --epochs 60 -i 1000 --lr 0.003 --trade-off 0.1 --seed 0 --log logs/coral/fmow
================================================
FILE: domain_generalization/Transfer-Learning-Library/examples/domain_generalization/image_classification/gpu_base.pbs
================================================
#!/bin/bash
#PBS -P ZHIHOU
#PBS -l select=1:ngpus=1:ncpus=6:mem=28GB
#PBS -l walltime=24:00:00
#PBS -q alloc-xxx
source activate pyt
#env
cd "$PBS_O_WORKDIR"
# ResNet50, PACS
#CUDA_VISIBLE_DEVICES=0 python baseline.py data/PACS -d PACS -s A C S -t P -a resnet50 --freeze-bn --seed 0 --log logs/baseline/PACS_P
#CUDA_VISIBLE_DEVICES=0 python baseline.py data/PACS -d PACS -s P C S -t A -a resnet50 --freeze-bn --seed 0 --log logs/baseline/PACS_A
#CUDA_VISIBLE_DEVICES=0 python baseline.py data/PACS -d PACS -s P A S -t C -a resnet50 --freeze-bn --seed 0 --log logs/baseline/PACS_C
#CUDA_VISIBLE_DEVICES=0 python baseline.py data/PACS -d PACS -s P A C -t S -a resnet50 --freeze-bn --seed 0 --log logs/baseline/PACS_S
#CUDA_VISIBLE_DEVICES=0 python baseline.py data/PACS -d PACS -s A C S -t P -a resnet50 --freeze-bn --seed 1 --log logs/baseline/PACS_P
#CUDA_VISIBLE_DEVICES=0 python baseline.py data/PACS -d PACS -s P C S -t A -a resnet50 --freeze-bn --seed 1 --log logs/baseline/PACS_A
#CUDA_VISIBLE_DEVICES=0 python baseline.py data/PACS -d PACS -s P A S -t C -a resnet50 --freeze-bn --seed 1 --log logs/baseline/PACS_C
#CUDA_VISIBLE_DEVICES=0 python baseline.py data/PACS -d PACS -s P A C -t S -a resnet50 --freeze-bn --seed 1 --log logs/baseline/PACS_S
#
#CUDA_VISIBLE_DEVICES=0 python baseline.py data/PACS -d PACS -s A C S -t P -a resnet50 --freeze-bn --seed 2 --log logs/baseline/PACS_P
#CUDA_VISIBLE_DEVICES=0 python baseline.py data/PACS -d PACS -s P C S -t A -a resnet50 --freeze-bn --seed 2 --log logs/baseline/PACS_A
#CUDA_VISIBLE_DEVICES=0 python baseline.py data/PACS -d PACS -s P A S -t C -a resnet50 --freeze-bn --seed 2 --log logs/baseline/PACS_C
#CUDA_VISIBLE_DEVICES=0 python baseline.py data/PACS -d PACS -s P A C -t S -a resnet50 --freeze-bn --seed 2 --log logs/baseline/PACS_S
CUDA_VISIBLE_DEVICES=0 python baseline.py data/PACS -d PACS -s P A C -t S -a resnet50 --freeze-bn --seed 0 --log logs/baseline/PACS_S_g16 --add_bt 16
CUDA_VISIBLE_DEVICES=0 python baseline.py data/PACS -d PACS -s P A C -t S -a resnet50 --freeze-bn --seed 1 --log logs/baseline/PACS_S_g16 --add_bt 16
CUDA_VISIBLE_DEVICES=0 python baseline.py data/PACS -d PACS -s P A C -t S -a resnet50 --freeze-bn --seed 2 --log logs/baseline/PACS_S_g16 --add_bt 16
#
#
#CUDA_VISIBLE_DEVICES=0 python baseline.py data/PACS -d PACS -s A C S -t P -a resnet18 --freeze-bn --seed 0 --log logs/baseline_r18/PACS_P_g16 --add_bt 16
#CUDA_VISIBLE_DEVICES=0 python baseline.py data/PACS -d PACS -s P C S -t A -a resnet18 --freeze-bn --seed 0 --log logs/baseline_r18/PACS_A_g16 --add_bt 16
#CUDA_VISIBLE_DEVICES=0 python baseline.py data/PACS -d PACS -s P A S -t C -a resnet18 --freeze-bn --seed 0 --log logs/baseline_r18/PACS_C_g16 --add_bt 16
#CUDA_VISIBLE_DEVICES=0 python baseline.py data/PACS -d PACS -s P A C -t S -a resnet18 --freeze-bn --seed 0 --log logs/baseline_r18/PACS_S_g16 --add_bt 16
#
#CUDA_VISIBLE_DEVICES=0 python baseline.py data/PACS -d PACS -s A C S -t P -a resnet18 --freeze-bn --seed 1 --log logs/baseline_r18/PACS_P_g16 --add_bt 16
#CUDA_VISIBLE_DEVICES=0 python baseline.py data/PACS -d PACS -s P C S -t A -a resnet18 --freeze-bn --seed 1 --log logs/baseline_r18/PACS_A_g16 --add_bt 16
#CUDA_VISIBLE_DEVICES=0 python baseline.py data/PACS -d PACS -s P A S -t C -a resnet18 --freeze-bn --seed 1 --log logs/baseline_r18/PACS_C_g16 --add_bt 16
#CUDA_VISIBLE_DEVICES=0 python baseline.py data/PACS -d PACS -s P A C -t S -a resnet18 --freeze-bn --seed 1 --log logs/baseline_r18/PACS_S_g16 --add_bt 16
###
#CUDA_VISIBLE_DEVICES=0 python baseline.py data/PACS -d PACS -s A C S -t P -a resnet18 --freeze-bn --seed 2 --log logs/baseline_r18/PACS_P_g16 --add_bt 16
#CUDA_VISIBLE_DEVICES=0 python baseline.py data/PACS -d PACS -s P C S -t A -a resnet18 --freeze-bn --seed 2 --log logs/baseline_r18/PACS_A_g16 --add_bt 16
#CUDA_VISIBLE_DEVICES=0 python baseline.py data/PACS -d PACS -s P A S -t C -a resnet18 --freeze-bn --seed 2 --log logs/baseline_r18/PACS_C_g16 --add_bt 16
#CUDA_VISIBLE_DEVICES=0 python baseline.py data/PACS -d PACS -s P A C -t S -a resnet18 --freeze-bn --seed 2 --log logs/baseline_r18/PACS_S_g16 --add_bt 16
#CUDA_VISIBLE_DEVICES=0 python baseline.py data/PACS -d PACS -s A C S -t P -a resnet50 --freeze-bn --seed 0 --log logs/baseline/PACS_P_g16 --add_bt 16
#CUDA_VISIBLE_DEVICES=0 python baseline.py data/PACS -d PACS -s P C S -t A -a resnet50 --freeze-bn --seed 0 --log logs/baseline/PACS_A_g16 --add_bt 16
#CUDA_VISIBLE_DEVICES=0 python baseline.py data/PACS -d PACS -s P A S -t C -a resnet50 --freeze-bn --seed 0 --log logs/baseline/PACS_C_g16 --add_bt 16
#CUDA_VISIBLE_DEVICES=0 python baseline.py data/PACS -d PACS -s P A C -t S -a resnet50 --freeze-bn --seed 0 --log logs/baseline/PACS_S_g16 --add_bt 16
#
#CUDA_VISIBLE_DEVICES=0 python baseline.py data/PACS -d PACS -s A C S -t P -a resnet50 --freeze-bn --seed 1 --log logs/baseline/PACS_P_g16 --add_bt 16
#CUDA_VISIBLE_DEVICES=0 python baseline.py data/PACS -d PACS -s P C S -t A -a resnet50 --freeze-bn --seed 1 --log logs/baseline/PACS_A_g16 --add_bt 16
#CUDA_VISIBLE_DEVICES=0 python baseline.py data/PACS -d PACS -s P A S -t C -a resnet50 --freeze-bn --seed 1 --log logs/baseline/PACS_C_g16 --add_bt 16
#CUDA_VISIBLE_DEVICES=0 python baseline.py data/PACS -d PACS -s P A C -t S -a resnet50 --freeze-bn --seed 1 --log logs/baseline/PACS_S_g16 --add_bt 16
#
#CUDA_VISIBLE_DEVICES=0 python baseline.py data/PACS -d PACS -s A C S -t P -a resnet50 --freeze-bn --seed 2 --log logs/baseline/PACS_P_g16 --add_bt 16
#CUDA_VISIBLE_DEVICES=0 python baseline.py data/PACS -d PACS -s P C S -t A -a resnet50 --freeze-bn --seed 2 --log logs/baseline/PACS_A_g16 --add_bt 16
#CUDA_VISIBLE_DEVICES=0 python baseline.py data/PACS -d PACS -s P A S -t C -a resnet50 --freeze-bn --seed 2 --log logs/baseline/PACS_C_g16 --add_bt 16
#CUDA_VISIBLE_DEVICES=0 python baseline.py data/PACS -d PACS -s P A C -t S -a resnet50 --freeze-bn --seed 2 --log logs/baseline/PACS_S_g16 --add_bt 16
#
#
## ResNet50, Office-Home
#CUDA_VISIBLE_DEVICES=0 python baseline.py data/office-home -d OfficeHome -s Ar Cl Rw -t Pr -a resnet50 --seed 0 --log logs/baseline/OfficeHome_Pr_g16 --add_bt 16
#CUDA_VISIBLE_DEVICES=0 python baseline.py data/office-home -d OfficeHome -s Ar Cl Pr -t Rw -a resnet50 --seed 0 --log logs/baseline/OfficeHome_Rw_g16 --add_bt 16
#CUDA_VISIBLE_DEVICES=0 python baseline.py data/office-home -d OfficeHome -s Ar Rw Pr -t Cl -a resnet50 --seed 0 --log logs/baseline/OfficeHome_Cl_g16 --add_bt 16
#CUDA_VISIBLE_DEVICES=0 python baseline.py data/office-home -d OfficeHome -s Cl Rw Pr -t Ar -a resnet50 --seed 0 --log logs/baseline/OfficeHome_Ar_g16 --add_bt 16
#
#CUDA_VISIBLE_DEVICES=0 python baseline.py data/office-home -d OfficeHome -s Ar Cl Rw -t Pr -a resnet50 --seed 1 --log logs/baseline/OfficeHome_Pr_g16 --add_bt 16;\
#CUDA_VISIBLE_DEVICES=0 python baseline.py data/office-home -d OfficeHome -s Ar Cl Pr -t Rw -a resnet50 --seed 1 --log logs/baseline/OfficeHome_Rw_g16 --add_bt 16;\
#CUDA_VISIBLE_DEVICES=0 python baseline.py data/office-home -d OfficeHome -s Ar Rw Pr -t Cl -a resnet50 --seed 1 --log logs/baseline/OfficeHome_Cl_g16 --add_bt 16;\
#CUDA_VISIBLE_DEVICES=0 python baseline.py data/office-home -d OfficeHome -s Cl Rw Pr -t Ar -a resnet50 --seed 1 --log logs/baseline/OfficeHome_Ar_g16 --add_bt 16
#
#
#CUDA_VISIBLE_DEVICES=0 python baseline.py data/office-home -d OfficeHome -s Ar Cl Rw -t Pr -a resnet50 --seed 2 --log logs/baseline/OfficeHome_Pr_g16 --add_bt 16;\
#CUDA_VISIBLE_DEVICES=0 python baseline.py data/office-home -d OfficeHome -s Ar Cl Pr -t Rw -a resnet50 --seed 2 --log logs/baseline/OfficeHome_Rw_g16 --add_bt 16;\
#CUDA_VISIBLE_DEVICES=0 python baseline.py data/office-home -d OfficeHome -s Ar Rw Pr -t Cl -a resnet50 --seed 2 --log logs/baseline/OfficeHome_Cl_g16 --add_bt 16;\
#CUDA_VISIBLE_DEVICES=0 python baseline.py data/office-home -d OfficeHome -s Cl Rw Pr -t Ar -a resnet50 --seed 2 --log logs/baseline/OfficeHome_Ar_g16 --add_bt 16
================================================
FILE: domain_generalization/Transfer-Learning-Library/examples/domain_generalization/image_classification/gpu_coral.pbs
================================================
#!/bin/bash
#PBS -P ZHIHOU
#PBS -l select=1:ngpus=1:ncpus=6:mem=28GB
#PBS -l walltime=24:00:00
#PBS -q alloc-xxx
source activate pyt
#env
cd "$PBS_O_WORKDIR"
# ResNet50, PACS
#CUDA_VISIBLE_DEVICES=0 python coral.py data/PACS -d PACS -s A C S -t P -a resnet50 --freeze-bn --seed 0 --log logs/coral/PACS_P_g16 --add_bt 16
#CUDA_VISIBLE_DEVICES=0 python coral.py data/PACS -d PACS -s P C S -t A -a resnet50 --freeze-bn --seed 0 --log logs/coral/PACS_A_g16 --add_bt 16
#CUDA_VISIBLE_DEVICES=0 python coral.py data/PACS -d PACS -s P A S -t C -a resnet50 --freeze-bn --seed 0 --log logs/coral/PACS_C_g16 --add_bt 16
#CUDA_VISIBLE_DEVICES=0 python coral.py data/PACS -d PACS -s P A C -t S -a resnet50 --freeze-bn --seed 0 --log logs/coral/PACS_S_g16 --add_bt 16
#
#CUDA_VISIBLE_DEVICES=0 python coral.py data/PACS -d PACS -s A C S -t P -a resnet50 --freeze-bn --seed 1 --log logs/coral/PACS_P_g16 --add_bt 16
#CUDA_VISIBLE_DEVICES=0 python coral.py data/PACS -d PACS -s P C S -t A -a resnet50 --freeze-bn --seed 1 --log logs/coral/PACS_A_g16 --add_bt 16
#CUDA_VISIBLE_DEVICES=0 python coral.py data/PACS -d PACS -s P A S -t C -a resnet50 --freeze-bn --seed 1 --log logs/coral/PACS_C_g16 --add_bt 16
#CUDA_VISIBLE_DEVICES=0 python coral.py data/PACS -d PACS -s P A C -t S -a resnet50 --freeze-bn --seed 1 --log logs/coral/PACS_S_g16 --add_bt 16
#
#CUDA_VISIBLE_DEVICES=0 python coral.py data/PACS -d PACS -s A C S -t P -a resnet50 --freeze-bn --seed 2 --log logs/coral/PACS_P_g16 --add_bt 16
#CUDA_VISIBLE_DEVICES=0 python coral.py data/PACS -d PACS -s P C S -t A -a resnet50 --freeze-bn --seed 2 --log logs/coral/PACS_A_g16 --add_bt 16
#CUDA_VISIBLE_DEVICES=0 python coral.py data/PACS -d PACS -s P A S -t C -a resnet50 --freeze-bn --seed 2 --log logs/coral/PACS_C_g16 --add_bt 16
#CUDA_VISIBLE_DEVICES=0 python coral.py data/PACS -d PACS -s P A C -t S -a resnet50 --freeze-bn --seed 2 --log logs/coral/PACS_S_g16 --add_bt 16
# ResNet50, Office-Home
#CUDA_VISIBLE_DEVICES=0 python coral.py data/office-home -d OfficeHome -s Ar Cl Rw -t Pr -a resnet50 --seed 0 --log logs/coral/OfficeHome_Pr_g16 --add_bt 16
#CUDA_VISIBLE_DEVICES=0 python coral.py data/office-home -d OfficeHome -s Ar Cl Pr -t Rw -a resnet50 --seed 0 --log logs/coral/OfficeHome_Rw_g16 --add_bt 16
#CUDA_VISIBLE_DEVICES=0 python coral.py data/office-home -d OfficeHome -s Ar Rw Pr -t Cl -a resnet50 --seed 0 --log logs/coral/OfficeHome_Cl_g16 --add_bt 16
#CUDA_VISIBLE_DEVICES=0 python coral.py data/office-home -d OfficeHome -s Cl Rw Pr -t Ar -a resnet50 --seed 0 --log logs/coral/OfficeHome_Ar_g16 --add_bt 16
#
#CUDA_VISIBLE_DEVICES=0 python coral.py data/office-home -d OfficeHome -s Ar Cl Rw -t Pr -a resnet50 --seed 1 --log logs/coral/OfficeHome_Pr
#CUDA_VISIBLE_DEVICES=0 python coral.py data/office-home -d OfficeHome -s Ar Cl Pr -t Rw -a resnet50 --seed 1 --log logs/coral/OfficeHome_Rw
#CUDA_VISIBLE_DEVICES=0 python coral.py data/office-home -d OfficeHome -s Ar Rw Pr -t Cl -a resnet50 --seed 1 --log logs/coral/OfficeHome_Cl
#CUDA_VISIBLE_DEVICES=0 python coral.py data/office-home -d OfficeHome -s Cl Rw Pr -t Ar -a resnet50 --seed 1 --log logs/coral/OfficeHome_Ar
##
#CUDA_VISIBLE_DEVICES=0 python coral.py data/office-home -d OfficeHome -s Ar Cl Rw -t Pr -a resnet50 --seed 2 --log logs/coral/OfficeHome_Pr
#CUDA_VISIBLE_DEVICES=0 python coral.py data/office-home -d OfficeHome -s Ar Cl Pr -t Rw -a resnet50 --seed 2 --log logs/coral/OfficeHome_Rw
#CUDA_VISIBLE_DEVICES=0 python coral.py data/office-home -d OfficeHome -s Ar Rw Pr -t Cl -a resnet50 --seed 2 --log logs/coral/OfficeHome_Cl
#CUDA_VISIBLE_DEVICES=0 python coral.py data/office-home -d OfficeHome -s Cl Rw Pr -t Ar -a resnet50 --seed 2 --log logs/coral/OfficeHome_Ar
================================================
FILE: domain_generalization/Transfer-Learning-Library/examples/domain_generalization/image_classification/gpu_irm.pbs
================================================
#!/bin/bash
#PBS -P ZHIHOU
#PBS -l select=1:ngpus=1:ncpus=6:mem=28GB
#PBS -l walltime=24:00:00
#PBS -q alloc-xxx
source activate pyt
#env
cd "$PBS_O_WORKDIR"
#
#CUDA_VISIBLE_DEVICES=0 python irm.py data/PACS -d PACS -s A C S -t P -a resnet50 --freeze-bn --seed 0 --log logs/irm/PACS_P_g16 --add_bt 16
#CUDA_VISIBLE_DEVICES=0 python irm.py data/PACS -d PACS -s P C S -t A -a resnet50 --freeze-bn --seed 0 --log logs/irm/PACS_A_g16 --add_bt 16
#CUDA_VISIBLE_DEVICES=0 python irm.py data/PACS -d PACS -s P A S -t C -a resnet50 --freeze-bn --seed 0 --log logs/irm/PACS_C_g16 --add_bt 16
#CUDA_VISIBLE_DEVICES=0 python irm.py data/PACS -d PACS -s P A C -t S -a resnet50 --freeze-bn --seed 0 --log logs/irm/PACS_S_g16 --add_bt 16
#
#CUDA_VISIBLE_DEVICES=0 python irm.py data/PACS -d PACS -s A C S -t P -a resnet50 --freeze-bn --seed 1 --log logs/irm/PACS_P_g16 --add_bt 16
#CUDA_VISIBLE_DEVICES=0 python irm.py data/PACS -d PACS -s P C S -t A -a resnet50 --freeze-bn --seed 1 --log logs/irm/PACS_A_g16 --add_bt 16
#CUDA_VISIBLE_DEVICES=0 python irm.py data/PACS -d PACS -s P A S -t C -a resnet50 --freeze-bn --seed 1 --log logs/irm/PACS_C_g16 --add_bt 16
#CUDA_VISIBLE_DEVICES=0 python irm.py data/PACS -d PACS -s P A C -t S -a resnet50 --freeze-bn --seed 1 --log logs/irm/PACS_S_g16 --add_bt 16
#
#CUDA_VISIBLE_DEVICES=0 python irm.py data/PACS -d PACS -s A C S -t P -a resnet50 --freeze-bn --seed 2 --log logs/irm/PACS_P_g16 --add_bt 16
#CUDA_VISIBLE_DEVICES=0 python irm.py data/PACS -d PACS -s P C S -t A -a resnet50 --freeze-bn --seed 2 --log logs/irm/PACS_A_g16 --add_bt 16
#CUDA_VISIBLE_DEVICES=0 python irm.py data/PACS -d PACS -s P A S -t C -a resnet50 --freeze-bn --seed 2 --log logs/irm/PACS_C_g16 --add_bt 16
#CUDA_VISIBLE_DEVICES=0 python irm.py data/PACS -d PACS -s P A C -t S -a resnet50 --freeze-bn --seed 2 --log logs/irm/PACS_S_g16 --add_bt 16
#
python irm.py data/PACS -d PACS -s P A C -t S -a resnet50 --freeze-bn --seed 0 --log logs/irm/PACS_S
python irm.py data/PACS -d PACS -s P A C -t S -a resnet50 --freeze-bn --seed 1 --log logs/irm/PACS_S
python irm.py data/PACS -d PACS -s P A C -t S -a resnet50 --freeze-bn --seed 2 --log logs/irm/PACS_S
### ResNet50, Office-Home
#CUDA_VISIBLE_DEVICES=0 python irm.py data/office-home -d OfficeHome -s Ar Cl Rw -t Pr -a resnet50 --seed 0 --log logs/irm/OfficeHome_Pr_g16 --add_bt 16
#CUDA_VISIBLE_DEVICES=0 python irm.py data/office-home -d OfficeHome -s Ar Cl Pr -t Rw -a resnet50 --seed 0 --log logs/irm/OfficeHome_Rw_g16 --add_bt 16
#CUDA_VISIBLE_DEVICES=0 python irm.py data/office-home -d OfficeHome -s Ar Rw Pr -t Cl -a resnet50 --seed 0 --log logs/irm/OfficeHome_Cl_g16 --add_bt 16
#CUDA_VISIBLE_DEVICES=0 python irm.py data/office-home -d OfficeHome -s Cl Rw Pr -t Ar -a resnet50 --seed 0 --log logs/irm/OfficeHome_Ar_g16 --add_bt 16
#
#CUDA_VISIBLE_DEVICES=0 python irm.py data/office-home -d OfficeHome -s Ar Cl Rw -t Pr -a resnet50 --seed 1 --log logs/irm/OfficeHome_Pr_g16 --add_bt 16
#CUDA_VISIBLE_DEVICES=0 python irm.py data/office-home -d OfficeHome -s Ar Cl Pr -t Rw -a resnet50 --seed 1 --log logs/irm/OfficeHome_Rw_g16 --add_bt 16
#CUDA_VISIBLE_DEVICES=0 python irm.py data/office-home -d OfficeHome -s Ar Rw Pr -t Cl -a resnet50 --seed 1 --log logs/irm/OfficeHome_Cl_g16 --add_bt 16
#CUDA_VISIBLE_DEVICES=0 python irm.py data/office-home -d OfficeHome -s Cl Rw Pr -t Ar -a resnet50 --seed 1 --log logs/irm/OfficeHome_Ar_g16 --add_bt 16
#
#CUDA_VISIBLE_DEVICES=0 python irm.py data/office-home -d OfficeHome -s Ar Cl Rw -t Pr -a resnet50 --seed 2 --log logs/irm/OfficeHome_Pr
#CUDA_VISIBLE_DEVICES=0 python irm.py data/office-home -d OfficeHome -s Cl Rw Pr -t Ar -a resnet50 --seed 2 --log logs/irm/OfficeHome_Ar
#CUDA_VISIBLE_DEVICES=0 python irm.py data/office-home -d OfficeHome -s Ar Cl Pr -t Rw -a resnet50 --seed 2 --log logs/irm/OfficeHome_Rw
#CUDA_VISIBLE_DEVICES=0 python irm.py data/office-home -d OfficeHome -s Ar Rw Pr -t Cl -a resnet50 --seed 2 --log logs/irm/OfficeHome_Cl
================================================
FILE: domain_generalization/Transfer-Learning-Library/examples/domain_generalization/image_classification/groupdro.py
================================================
"""
@author: Baixu Chen
@contact: cbx_99_hasta@outlook.com
"""
import random
import time
import warnings
import sys
import argparse
import shutil
import os.path as osp
import torch
import torch.nn as nn
import torch.backends.cudnn as cudnn
from torch.optim import SGD
from torch.optim.lr_scheduler import CosineAnnealingLR
from torch.utils.data import DataLoader
import torch.nn.functional as F
sys.path.append('../../..')
from dglib.generalization.groupdro import AutomaticUpdateDomainWeightModule
from dglib.modules.sampler import RandomDomainSampler
from dglib.modules.classifier import ImageClassifier as Classifier
from common.utils.data import ForeverDataIterator
from common.utils.metric import accuracy
from common.utils.meter import AverageMeter, ProgressMeter
from common.utils.logger import CompleteLogger
from common.utils.analysis import tsne, a_distance
sys.path.append('.')
import utils
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
def main(args: argparse.Namespace):
logger = CompleteLogger(args.log, args.phase)
print(args)
if args.seed is not None:
random.seed(args.seed)
torch.manual_seed(args.seed)
cudnn.deterministic = True
warnings.warn('You have chosen to seed training. '
'This will turn on the CUDNN deterministic setting, '
'which can slow down your training considerably! '
'You may see unexpected behavior when restarting '
'from checkpoints.')
cudnn.benchmark = True
# Data loading code
train_transform = utils.get_train_transform(args.train_resizing, random_horizontal_flip=True,
random_color_jitter=True, random_gray_scale=True)
val_transform = utils.get_val_transform(args.val_resizing)
print("train_transform: ", train_transform)
print("val_transform: ", val_transform)
train_dataset, num_classes = utils.get_dataset(dataset_name=args.data, root=args.root, task_list=args.sources,
split='train', download=True, transform=train_transform,
seed=args.seed)
sampler = RandomDomainSampler(train_dataset, args.batch_size, args.n_domains_per_batch)
train_loader = DataLoader(train_dataset, batch_size=args.batch_size, num_workers=args.workers,
sampler=sampler, drop_last=True)
val_dataset, _ = utils.get_dataset(dataset_name=args.data, root=args.root, task_list=args.sources, split='val',
download=True, transform=val_transform, seed=args.seed)
val_loader = DataLoader(val_dataset, batch_size=args.batch_size, shuffle=False, num_workers=args.workers)
test_dataset, _ = utils.get_dataset(dataset_name=args.data, root=args.root, task_list=args.targets, split='test',
download=True, transform=val_transform, seed=args.seed)
test_loader = DataLoader(test_dataset, batch_size=args.batch_size, shuffle=False, num_workers=args.workers)
print("train_dataset_size: ", len(train_dataset))
print('val_dataset_size: ', len(val_dataset))
print("test_dataset_size: ", len(test_dataset))
train_iter = ForeverDataIterator(train_loader)
# create model
print("=> using pre-trained model '{}'".format(args.arch))
backbone = utils.get_model(args.arch)
pool_layer = nn.Identity() if args.no_pool else None
classifier = Classifier(backbone, num_classes, freeze_bn=args.freeze_bn, dropout_p=args.dropout_p,
finetune=args.finetune, pool_layer=pool_layer).to(device)
num_all_domains = len(train_dataset.datasets)
# define optimizer and lr scheduler
optimizer = SGD(classifier.get_parameters(base_lr=args.lr), args.lr, momentum=args.momentum, weight_decay=args.wd,
nesterov=True)
lr_scheduler = CosineAnnealingLR(optimizer, args.epochs * args.iters_per_epoch)
domain_weight_module = AutomaticUpdateDomainWeightModule(num_all_domains, args.eta, device)
# resume from the best checkpoint
if args.phase != 'train':
checkpoint = torch.load(logger.get_checkpoint_path('best'), map_location='cpu')
classifier.load_state_dict(checkpoint)
# analysis the model
if args.phase == 'analysis':
# extract features from both domains
feature_extractor = nn.Sequential(classifier.backbone, classifier.pool_layer, classifier.bottleneck).to(device)
source_feature = utils.collect_feature(val_loader, feature_extractor, device, max_num_features=100)
target_feature = utils.collect_feature(test_loader, feature_extractor, device, max_num_features=100)
print(len(source_feature), len(target_feature))
# plot t-SNE
tSNE_filename = osp.join(logger.visualize_directory, 'TSNE.png')
tsne.visualize(source_feature, target_feature, tSNE_filename)
print("Saving t-SNE to", tSNE_filename)
# calculate A-distance, which is a measure for distribution discrepancy
A_distance = a_distance.calculate(source_feature, target_feature, device)
print("A-distance =", A_distance)
return
if args.phase == 'test':
acc1 = utils.validate(test_loader, classifier, args, device)
print(acc1)
return
# start training
best_val_acc1 = 0.
best_test_acc1 = 0.
for epoch in range(args.epochs):
print(lr_scheduler.get_lr())
# train for one epoch
train(train_iter, classifier, optimizer, lr_scheduler, domain_weight_module, args.n_domains_per_batch, epoch,
args)
# evaluate on validation set
print("Evaluate on validation set...")
acc1 = utils.validate(val_loader, classifier, args, device)
# remember best acc@1 and save checkpoint
torch.save(classifier.state_dict(), logger.get_checkpoint_path('latest'))
if acc1 > best_val_acc1:
shutil.copy(logger.get_checkpoint_path('latest'), logger.get_checkpoint_path('best'))
best_val_acc1 = max(acc1, best_val_acc1)
# evaluate on test set
print("Evaluate on test set...")
best_test_acc1 = max(best_test_acc1, utils.validate(test_loader, classifier, args, device))
# evaluate on test set
classifier.load_state_dict(torch.load(logger.get_checkpoint_path('best')))
acc1 = utils.validate(test_loader, classifier, args, device)
print("test acc on test set = {}".format(acc1))
print("oracle acc on test set = {}".format(best_test_acc1))
logger.close()
def train(train_iter: ForeverDataIterator, model: Classifier, optimizer, lr_scheduler: CosineAnnealingLR,
domain_weight_module: AutomaticUpdateDomainWeightModule, n_domains_per_batch: int, epoch: int,
args: argparse.Namespace):
batch_time = AverageMeter('Time', ':4.2f')
data_time = AverageMeter('Data', ':3.1f')
losses = AverageMeter('Loss', ':3.2f')
cls_accs = AverageMeter('Cls Acc', ':3.1f')
progress = ProgressMeter(
args.iters_per_epoch,
[batch_time, data_time, losses, cls_accs],
prefix="Epoch: [{}]".format(epoch))
# switch to train mode
model.train()
end = time.time()
for i in range(args.iters_per_epoch):
x_all, labels_all, domain_labels = next(train_iter)
x_all = x_all.to(device)
labels_all = labels_all.to(device)
domain_labels = domain_labels.to(device)
# get selected domain idxes
domain_labels = domain_labels.chunk(n_domains_per_batch, dim=0)
sampled_domain_idxes = [domain_labels[i][0].item() for i in range(n_domains_per_batch)]
# measure data loading time
data_time.update(time.time() - end)
loss_per_domain = torch.zeros(n_domains_per_batch).to(device)
cls_acc = 0
for domain_id, (x_per_domain, labels_per_domain) in enumerate(
zip(x_all.chunk(n_domains_per_batch, dim=0), labels_all.chunk(n_domains_per_batch, dim=0))):
y_per_domain, _ = model(x_per_domain)
loss_per_domain[domain_id] = F.cross_entropy(y_per_domain, labels_per_domain)
cls_acc += accuracy(y_per_domain, labels_per_domain)[0] / n_domains_per_batch
# update domain weight
domain_weight_module.update(loss_per_domain, sampled_domain_idxes)
domain_weight = domain_weight_module.get_domain_weight(sampled_domain_idxes)
# weighted cls loss
loss = (loss_per_domain * domain_weight).sum()
losses.update(loss.item(), x_all.size(0))
cls_accs.update(cls_acc.item(), x_all.size(0))
# compute gradient and do SGD step
optimizer.zero_grad()
loss.backward()
optimizer.step()
lr_scheduler.step()
# measure elapsed time
batch_time.update(time.time() - end)
end = time.time()
if i % args.print_freq == 0:
progress.display(i)
if __name__ == '__main__':
parser = argparse.ArgumentParser(description='GroupDRO for Domain Generalization')
# dataset parameters
parser.add_argument('root', metavar='DIR',
help='root path of dataset')
parser.add_argument('-d', '--data', metavar='DATA', default='PACS',
help='dataset: ' + ' | '.join(utils.get_dataset_names()) +
' (default: PACS)')
parser.add_argument('-s', '--sources', nargs='+', default=None,
help='source domain(s)')
parser.add_argument('-t', '--targets', nargs='+', default=None,
help='target domain(s)')
parser.add_argument('--train-resizing', type=str, default='default')
parser.add_argument('--val-resizing', type=str, default='default')
# model parameters
parser.add_argument('-a', '--arch', metavar='ARCH', default='resnet50',
choices=utils.get_model_names(),
help='backbone architecture: ' +
' | '.join(utils.get_model_names()) +
' (default: resnet50)')
parser.add_argument('--no-pool', action='store_true', help='no pool layer after the feature extractor.')
parser.add_argument('--finetune', action='store_true', help='whether use 10x smaller lr for backbone')
parser.add_argument('--freeze-bn', action='store_true', help='whether freeze all bn layers')
parser.add_argument('--dropout-p', type=float, default=0.1, help='only activated when freeze-bn is True')
# training parameters
parser.add_argument('--eta', default=1e-2, type=float,
help='the eta hyper parameter')
parser.add_argument('-b', '--batch-size', default=36, type=int,
metavar='N',
help='mini-batch size (default: 36)')
parser.add_argument('--n-domains-per-batch', default=3, type=int,
help='number of domains in each mini-batch')
parser.add_argument('--lr', '--learning-rate', default=5e-4, type=float,
metavar='LR', help='initial learning rate', dest='lr')
parser.add_argument('--momentum', default=0.9, type=float, metavar='M',
help='momentum')
parser.add_argument('--wd', '--weight-decay', default=0.0005, type=float,
metavar='W', help='weight decay (default: 5e-4)')
parser.add_argument('-j', '--workers', default=4, type=int, metavar='N',
help='number of data loading workers (default: 4)')
parser.add_argument('--epochs', default=20, type=int, metavar='N',
help='number of total epochs to run')
parser.add_argument('-i', '--iters-per-epoch', default=500, type=int,
help='Number of iterations per epoch')
parser.add_argument('-p', '--print-freq', default=100, type=int,
metavar='N', help='print frequency (default: 100)')
parser.add_argument('--seed', default=None, type=int,
help='seed for initializing training. ')
parser.add_argument("--log", type=str, default='groupdro',
help="Where to save logs, checkpoints and debugging images.")
parser.add_argument("--phase", type=str, default='train', choices=['train', 'test', 'analysis'],
help="When phase is 'test', only test the model."
"When phase is 'analysis', only analysis the model.")
args = parser.parse_args()
main(args)
================================================
FILE: domain_generalization/Transfer-Learning-Library/examples/domain_generalization/image_classification/groupdro.sh
================================================
#!/usr/bin/env bash
# ResNet50, PACS
CUDA_VISIBLE_DEVICES=0 python groupdro.py data/PACS -d PACS -s A C S -t P -a resnet50 --freeze-bn --seed 0 --log logs/groupdro/PACS_P
CUDA_VISIBLE_DEVICES=0 python groupdro.py data/PACS -d PACS -s P C S -t A -a resnet50 --freeze-bn --seed 0 --log logs/groupdro/PACS_A
CUDA_VISIBLE_DEVICES=0 python groupdro.py data/PACS -d PACS -s P A S -t C -a resnet50 --freeze-bn --seed 0 --log logs/groupdro/PACS_C
CUDA_VISIBLE_DEVICES=0 python groupdro.py data/PACS -d PACS -s P A C -t S -a resnet50 --freeze-bn --seed 0 --log logs/groupdro/PACS_S
# ResNet50, Office-Home
CUDA_VISIBLE_DEVICES=0 python groupdro.py data/office-home -d OfficeHome -s Ar Cl Rw -t Pr -a resnet50 --seed 0 --log logs/groupdro/OfficeHome_Pr
CUDA_VISIBLE_DEVICES=0 python groupdro.py data/office-home -d OfficeHome -s Ar Cl Pr -t Rw -a resnet50 --seed 0 --log logs/groupdro/OfficeHome_Rw
CUDA_VISIBLE_DEVICES=0 python groupdro.py data/office-home -d OfficeHome -s Ar Rw Pr -t Cl -a resnet50 --seed 0 --log logs/groupdro/OfficeHome_Cl
CUDA_VISIBLE_DEVICES=0 python groupdro.py data/office-home -d OfficeHome -s Cl Rw Pr -t Ar -a resnet50 --seed 0 --log logs/groupdro/OfficeHome_Ar
# ResNet50, DomainNet
CUDA_VISIBLE_DEVICES=0 python groupdro.py data/domainnet -d DomainNet -s i p q r s -t c -a resnet50 -i 4000 --lr 0.005 --seed 0 --log logs/groupdro/DomainNet_c
CUDA_VISIBLE_DEVICES=0 python groupdro.py data/domainnet -d DomainNet -s c p q r s -t i -a resnet50 -i 4000 --lr 0.005 --seed 0 --log logs/groupdro/DomainNet_i
CUDA_VISIBLE_DEVICES=0 python groupdro.py data/domainnet -d DomainNet -s c i q r s -t p -a resnet50 -i 4000 --lr 0.005 --seed 0 --log logs/groupdro/DomainNet_p
CUDA_VISIBLE_DEVICES=0 python groupdro.py data/domainnet -d DomainNet -s c i p r s -t q -a resnet50 -i 4000 --lr 0.005 --seed 0 --log logs/groupdro/DomainNet_q
CUDA_VISIBLE_DEVICES=0 python groupdro.py data/domainnet -d DomainNet -s c i p q s -t r -a resnet50 -i 4000 --lr 0.005 --seed 0 --log logs/groupdro/DomainNet_r
CUDA_VISIBLE_DEVICES=0 python groupdro.py data/domainnet -d DomainNet -s c i p q r -t s -a resnet50 -i 4000 --lr 0.005 --seed 0 --log logs/groupdro/DomainNet_s
# ResNet50, Wilds Dataset
CUDA_VISIBLE_DEVICES=0 python groupdro.py data/wilds -d iwildcam --train-resizing 'res2x' --val-resizing 'res2x' \
-a resnet50 -b 16 --n-domains-per-batch 2 --epochs 60 -i 1000 --lr 0.001 --finetune --seed 0 --log logs/groupdro/iwildcam
CUDA_VISIBLE_DEVICES=0 python groupdro.py data/wilds -d camelyon17 -a resnet50 -b 36 --n-domains-per-batch 3 \
--epochs 20 -i 1000 --lr 0.01 --seed 0 --log logs/groupdro/camelyon17
# DenseNet121, Wilds Dataset
CUDA_VISIBLE_DEVICES=0 python groupdro.py data/wilds -d fmow --train-resizing 'cen.crop' -a densenet121 \
-b 36 --n-domains-per-batch 4 --epochs 60 -i 1000 --lr 0.003 --seed 0 --log logs/groupdro/fmow
================================================
FILE: domain_generalization/Transfer-Learning-Library/examples/domain_generalization/image_classification/ibn.sh
================================================
#!/usr/bin/env bash
# IBN_ResNet50_b, PACS
CUDA_VISIBLE_DEVICES=0 python baseline.py data/PACS -d PACS -s A C S -t P -a resnet50_ibn_b --freeze-bn --seed 0 --log logs/baseline/PACS_P
CUDA_VISIBLE_DEVICES=0 python baseline.py data/PACS -d PACS -s P C S -t A -a resnet50_ibn_b --freeze-bn --seed 0 --log logs/baseline/PACS_A
CUDA_VISIBLE_DEVICES=2 python baseline.py data/PACS -d PACS -s P A S -t C -a resnet50_ibn_b --freeze-bn --seed 0 --log logs/baseline/PACS_C
CUDA_VISIBLE_DEVICES=0 python baseline.py data/PACS -d PACS -s P A C -t S -a resnet50_ibn_b --freeze-bn --seed 0 --log logs/baseline/PACS_S
# IBN_ResNet50_b, Office-Home
CUDA_VISIBLE_DEVICES=0 python baseline.py data/office-home -d OfficeHome -s Ar Cl Rw -t Pr -a resnet50_ibn_b --seed 0 --log logs/baseline/OfficeHome_Pr
CUDA_VISIBLE_DEVICES=0 python baseline.py data/office-home -d OfficeHome -s Ar Cl Pr -t Rw -a resnet50_ibn_b --seed 0 --log logs/baseline/OfficeHome_Rw
CUDA_VISIBLE_DEVICES=0 python baseline.py data/office-home -d OfficeHome -s Ar Rw Pr -t Cl -a resnet50_ibn_b --seed 0 --log logs/baseline/OfficeHome_Cl
CUDA_VISIBLE_DEVICES=0 python baseline.py data/office-home -d OfficeHome -s Cl Rw Pr -t Ar -a resnet50_ibn_b --seed 0 --log logs/baseline/OfficeHome_Ar
# IBN_ResNet50_b, DomainNet
CUDA_VISIBLE_DEVICES=0 python baseline.py data/domainnet -d DomainNet -s i p q r s -t c -a resnet50_ibn_b -i 2500 --lr 0.01 --seed 0 --log logs/baseline/DomainNet_c
CUDA_VISIBLE_DEVICES=0 python baseline.py data/domainnet -d DomainNet -s c p q r s -t i -a resnet50_ibn_b -i 2500 --lr 0.01 --seed 0 --log logs/baseline/DomainNet_i
CUDA_VISIBLE_DEVICES=0 python baseline.py data/domainnet -d DomainNet -s c i q r s -t p -a resnet50_ibn_b -i 2500 --lr 0.01 --seed 0 --log logs/baseline/DomainNet_p
CUDA_VISIBLE_DEVICES=0 python baseline.py data/domainnet -d DomainNet -s c i p r s -t q -a resnet50_ibn_b -i 2500 --lr 0.01 --seed 0 --log logs/baseline/DomainNet_q
CUDA_VISIBLE_DEVICES=0 python baseline.py data/domainnet -d DomainNet -s c i p q s -t r -a resnet50_ibn_b -i 2500 --lr 0.01 --seed 0 --log logs/baseline/DomainNet_r
CUDA_VISIBLE_DEVICES=0 python baseline.py data/domainnet -d DomainNet -s c i p q r -t s -a resnet50_ibn_b -i 2500 --lr 0.01 --seed 0 --log logs/baseline/DomainNet_s
# IBN_ResNet50_b, Wilds Dataset
CUDA_VISIBLE_DEVICES=0 python baseline.py data/wilds -d iwildcam --train-resizing 'res2x' --val-resizing 'res2x' \
-a resnet50_ibn_b -b 16 --epochs 60 -i 1000 --lr 0.001 --finetune --seed 0 --log logs/baseline/iwildcam
CUDA_VISIBLE_DEVICES=0 python baseline.py data/wilds -d camelyon17 -a resnet50_ibn_b -b 36 --epochs 20 -i 1000 --lr 0.01 \
--seed 0 --log logs/baseline/camelyon17
================================================
FILE: domain_generalization/Transfer-Learning-Library/examples/domain_generalization/image_classification/irm.py
================================================
"""
@author: Baixu Chen
@contact: cbx_99_hasta@outlook.com
"""
import random
import time
import warnings
import sys
import argparse
import shutil
import os.path as osp
import torch
import torch.nn as nn
import torch.backends.cudnn as cudnn
from torch.optim import SGD
from torch.optim.lr_scheduler import CosineAnnealingLR
from torch.utils.data import DataLoader
import torch.nn.functional as F
sys.path.append('../../..')
from dglib.generalization.irm import InvariancePenaltyLoss
from dglib.modules.sampler import RandomDomainSampler
from dglib.modules.classifier import ImageClassifier as Classifier
from common.utils.data import ForeverDataIterator
from common.utils.metric import accuracy
from common.utils.meter import AverageMeter, ProgressMeter
from common.utils.logger import CompleteLogger
from common.utils.analysis import tsne, a_distance
sys.path.append('.')
import utils
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
class TransformerDecorator(torch.nn.Module):
def __init__(self, backbone=None, add_bt=3, dim=2048, eval_global=0):
super(TransformerDecorator, self).__init__()
self.eval_global = eval_global
self.add_bt = add_bt
self.backbone = backbone
if self.add_bt > 0:
self.encoder_layers = torch.nn.TransformerEncoderLayer(dim, 4, dim, 0.5)
def forward(self, feature):
feature = self.backbone(feature)
if self.training and self.add_bt > 0:
pre_feature = feature
feature = feature.unsqueeze(1)
feature = self.encoder_layers(feature)
# feature = self.encoder_layers(feature, feature, feature)
feature = feature.squeeze(1)
if self.add_bt:
return torch.cat([pre_feature, feature], dim=0)
return feature
def main(args: argparse.Namespace):
logger = CompleteLogger(args.log, args.phase)
print(args)
if args.seed is not None:
random.seed(args.seed)
torch.manual_seed(args.seed)
cudnn.deterministic = True
warnings.warn('You have chosen to seed training. '
'This will turn on the CUDNN deterministic setting, '
'which can slow down your training considerably! '
'You may see unexpected behavior when restarting '
'from checkpoints.')
cudnn.benchmark = True
# Data loading code
train_transform = utils.get_train_transform(args.train_resizing, random_horizontal_flip=True,
random_color_jitter=True, random_gray_scale=True)
val_transform = utils.get_val_transform(args.val_resizing)
print("train_transform: ", train_transform)
print("val_transform: ", val_transform)
train_dataset, num_classes = utils.get_dataset(dataset_name=args.data, root=args.root, task_list=args.sources,
split='train', download=True, transform=train_transform,
seed=args.seed)
sampler = RandomDomainSampler(train_dataset, args.batch_size, n_domains_per_batch=args.n_domains_per_batch)
train_loader = DataLoader(train_dataset, batch_size=args.batch_size, num_workers=args.workers,
sampler=sampler, drop_last=True)
val_dataset, _ = utils.get_dataset(dataset_name=args.data, root=args.root, task_list=args.sources, split='val',
download=True, transform=val_transform, seed=args.seed)
val_loader = DataLoader(val_dataset, batch_size=args.batch_size, shuffle=False, num_workers=args.workers)
test_dataset, _ = utils.get_dataset(dataset_name=args.data, root=args.root, task_list=args.targets, split='test',
download=True, transform=val_transform, seed=args.seed)
test_loader = DataLoader(test_dataset, batch_size=args.batch_size, shuffle=False, num_workers=args.workers)
print("train_dataset_size: ", len(train_dataset))
print('val_dataset_size: ', len(val_dataset))
print("test_dataset_size: ", len(test_dataset))
train_iter = ForeverDataIterator(train_loader)
# create model
print("=> using pre-trained model '{}'".format(args.arch))
backbone = utils.get_model(args.arch)
pool_layer = nn.Identity() if args.no_pool else None
pool_layer = nn.Sequential(
nn.AdaptiveAvgPool2d(output_size=(1, 1)),
nn.Flatten(),
)
pool_layer = TransformerDecorator(pool_layer, add_bt=args.add_bt, dim=backbone.out_features)
classifier = Classifier(backbone, num_classes, freeze_bn=args.freeze_bn, dropout_p=args.dropout_p,
finetune=args.finetune, pool_layer=pool_layer).to(device)
# define optimizer and lr scheduler
optimizer = SGD(classifier.get_parameters(base_lr=args.lr), args.lr, momentum=args.momentum, weight_decay=args.wd,
nesterov=True)
lr_scheduler = CosineAnnealingLR(optimizer, args.epochs * args.iters_per_epoch)
# define loss function
invariance_penalty_loss = InvariancePenaltyLoss().to(device)
# for simplicity
assert args.anneal_iters % args.iters_per_epoch == 0
# resume from the best checkpoint
if args.phase != 'train':
checkpoint = torch.load(logger.get_checkpoint_path('best'), map_location='cpu')
classifier.load_state_dict(checkpoint)
# analysis the model
if args.phase == 'analysis':
# extract features from both domains
feature_extractor = nn.Sequential(classifier.backbone, classifier.pool_layer, classifier.bottleneck).to(device)
source_feature = utils.collect_feature(val_loader, feature_extractor, device, max_num_features=100)
target_feature = utils.collect_feature(test_loader, feature_extractor, device, max_num_features=100)
print(len(source_feature), len(target_feature))
# plot t-SNE
tSNE_filename = osp.join(logger.visualize_directory, 'TSNE.png')
tsne.visualize(source_feature, target_feature, tSNE_filename)
print("Saving t-SNE to", tSNE_filename)
# calculate A-distance, which is a measure for distribution discrepancy
A_distance = a_distance.calculate(source_feature, target_feature, device)
print("A-distance =", A_distance)
return
if args.phase == 'test':
acc1 = utils.validate(test_loader, classifier, args, device)
print(acc1)
return
# start training
best_val_acc1 = 0.
best_test_acc1 = 0.
for epoch in range(args.epochs):
if epoch * args.iters_per_epoch == args.anneal_iters:
# reset optimizer to avoid sharp jump in gradient magnitudes
optimizer = SGD(classifier.get_parameters(base_lr=args.lr), args.lr, momentum=args.momentum,
weight_decay=args.wd, nesterov=True)
lr_scheduler = CosineAnnealingLR(optimizer, args.epochs * args.iters_per_epoch - args.anneal_iters)
print(lr_scheduler.get_lr())
# train for one epoch
train(train_iter, classifier, optimizer, lr_scheduler, invariance_penalty_loss, args.n_domains_per_batch, epoch,
args)
# evaluate on validation set
print("Evaluate on validation set...")
acc1 = utils.validate(val_loader, classifier, args, device)
# remember best acc@1 and save checkpoint
torch.save(classifier.state_dict(), logger.get_checkpoint_path('latest'))
if acc1 > best_val_acc1:
shutil.copy(logger.get_checkpoint_path('latest'), logger.get_checkpoint_path('best'))
best_val_acc1 = max(acc1, best_val_acc1)
# evaluate on test set
print("Evaluate on test set...")
best_test_acc1 = max(best_test_acc1, utils.validate(test_loader, classifier, args, device))
# evaluate on test set
classifier.load_state_dict(torch.load(logger.get_checkpoint_path('best')))
acc1 = utils.validate(test_loader, classifier, args, device)
print("test acc on test set = {}".format(acc1))
print("oracle acc on test set = {}".format(best_test_acc1))
logger.close()
def train(train_iter: ForeverDataIterator, model: Classifier, optimizer, lr_scheduler: CosineAnnealingLR,
invariance_penalty_loss: InvariancePenaltyLoss, n_domains_per_batch: int, epoch: int,
args: argparse.Namespace):
batch_time = AverageMeter('Time', ':4.2f')
data_time = AverageMeter('Data', ':3.1f')
losses = AverageMeter('Loss', ':3.2f')
losses_ce = AverageMeter('CELoss', ':3.2f')
losses_penalty = AverageMeter('Penalty Loss', ':3.2f')
cls_accs = AverageMeter('Cls Acc', ':3.1f')
progress = ProgressMeter(
args.iters_per_epoch,
[batch_time, data_time, losses, losses_ce, losses_penalty, cls_accs],
prefix="Epoch: [{}]".format(epoch))
# switch to train mode
model.train()
end = time.time()
for i in range(args.iters_per_epoch):
x_all, labels_all, _ = next(train_iter)
x_all = x_all.to(device)
labels_all = labels_all.to(device)
# measure data loading time
data_time.update(time.time() - end)
# compute output
y_all, _ = model(x_all)
if len(y_all) > len(labels_all):
labels_all = torch.cat([labels_all, labels_all], dim=0)
# cls loss
loss_ce = F.cross_entropy(y_all, labels_all)
# penalty loss
loss_penalty = 0
for y_per_domain, labels_per_domain in zip(y_all.chunk(n_domains_per_batch, dim=0),
labels_all.chunk(n_domains_per_batch, dim=0)):
# normalize loss by domain num
loss_penalty += invariance_penalty_loss(y_per_domain, labels_per_domain) / n_domains_per_batch
global_iter = epoch * args.iters_per_epoch + i
if global_iter >= args.anneal_iters:
trade_off = args.trade_off
else:
trade_off = 1
loss = loss_ce + loss_penalty * trade_off
cls_acc = accuracy(y_all, labels_all)[0]
losses.update(loss.item(), x_all.size(0))
losses_ce.update(loss_ce.item(), x_all.size(0))
losses_penalty.update(loss_penalty.item(), x_all.size(0))
cls_accs.update(cls_acc.item(), x_all.size(0))
# compute gradient and do SGD step
optimizer.zero_grad()
loss.backward()
optimizer.step()
lr_scheduler.step()
# measure elapsed time
batch_time.update(time.time() - end)
end = time.time()
if i % args.print_freq == 0:
progress.display(i)
if __name__ == '__main__':
parser = argparse.ArgumentParser(description='IRM for Domain Generalization')
# dataset parameters
parser.add_argument('root', metavar='DIR',
help='root path of dataset')
parser.add_argument('-d', '--data', metavar='DATA', default='PACS',
help='dataset: ' + ' | '.join(utils.get_dataset_names()) +
' (default: PACS)')
parser.add_argument('-s', '--sources', nargs='+', default=None,
help='source domain(s)')
parser.add_argument('-t', '--targets', nargs='+', default=None,
help='target domain(s)')
parser.add_argument('--train-resizing', type=str, default='default')
parser.add_argument('--val-resizing', type=str, default='default')
# model parameters
parser.add_argument('-a', '--arch', metavar='ARCH', default='resnet50',
choices=utils.get_model_names(),
help='backbone architecture: ' +
' | '.join(utils.get_model_names()) +
' (default: resnet50)')
parser.add_argument('--no-pool', action='store_true', help='no pool layer after the feature extractor.')
parser.add_argument('--finetune', action='store_true', help='whether use 10x smaller lr for backbone')
parser.add_argument('--freeze-bn', action='store_true', help='whether freeze all bn layers')
parser.add_argument('--dropout-p', type=float, default=0.1, help='only activated when freeze-bn is True')
# training parameters
parser.add_argument('--trade-off', default=1, type=float,
help='the trade off hyper parameter for irm penalty')
parser.add_argument('--anneal-iters', default=500, type=int,
help='anneal iterations (trade off is set to 1 during these iterations)')
parser.add_argument('-b', '--batch-size', default=36, type=int,
metavar='N',
help='mini-batch size (default: 36)')
parser.add_argument('--n-domains-per-batch', default=3, type=int,
help='number of domains in each mini-batch')
parser.add_argument('--lr', '--learning-rate', default=5e-4, type=float,
metavar='LR', help='initial learning rate', dest='lr')
parser.add_argument('--momentum', default=0.9, type=float, metavar='M',
help='momentum')
parser.add_argument('--wd', '--weight-decay', default=0.0005, type=float,
metavar='W', help='weight decay (default: 5e-4)')
parser.add_argument('-j', '--workers', default=4, type=int, metavar='N',
help='number of data loading workers (default: 4)')
parser.add_argument('--epochs', default=20, type=int, metavar='N',
help='number of total epochs to run')
parser.add_argument('-i', '--iters-per-epoch', default=500, type=int,
help='Number of iterations per epoch')
parser.add_argument('-p', '--print-freq', default=100, type=int,
metavar='N', help='print frequency (default: 100)')
parser.add_argument('--seed', default=None, type=int,
help='seed for initializing training. ')
parser.add_argument("--log", type=str, default='irm',
help="Where to save logs, checkpoints and debugging images.")
parser.add_argument("--phase", type=str, default='train', choices=['train', 'test', 'analysis'],
help="When phase is 'test', only test the model."
"When phase is 'analysis', only analysis the model.")
parser.add_argument("--add_bt", type=int, default=0,
help="add_bt")
args = parser.parse_args()
main(args)
================================================
FILE: domain_generalization/Transfer-Learning-Library/examples/domain_generalization/image_classification/irm.sh
================================================
#!/usr/bin/env bash
# ResNet50, PACS
CUDA_VISIBLE_DEVICES=0 python irm.py data/PACS -d PACS -s A C S -t P -a resnet50 --freeze-bn --seed 0 --log logs/irm/PACS_P
CUDA_VISIBLE_DEVICES=0 python irm.py data/PACS -d PACS -s P C S -t A -a resnet50 --freeze-bn --seed 0 --log logs/irm/PACS_A
CUDA_VISIBLE_DEVICES=0 python irm.py data/PACS -d PACS -s P A S -t C -a resnet50 --freeze-bn --seed 0 --log logs/irm/PACS_C
CUDA_VISIBLE_DEVICES=0 python irm.py data/PACS -d PACS -s P A C -t S -a resnet50 --freeze-bn --seed 0 --log logs/irm/PACS_S
# ResNet50, Office-Home
CUDA_VISIBLE_DEVICES=0 python irm.py data/office-home -d OfficeHome -s Ar Cl Rw -t Pr -a resnet50 --seed 0 --log logs/irm/OfficeHome_Pr
CUDA_VISIBLE_DEVICES=0 python irm.py data/office-home -d OfficeHome -s Ar Cl Pr -t Rw -a resnet50 --seed 0 --log logs/irm/OfficeHome_Rw
CUDA_VISIBLE_DEVICES=0 python irm.py data/office-home -d OfficeHome -s Ar Rw Pr -t Cl -a resnet50 --seed 0 --log logs/irm/OfficeHome_Cl
CUDA_VISIBLE_DEVICES=0 python irm.py data/office-home -d OfficeHome -s Cl Rw Pr -t Ar -a resnet50 --seed 0 --log logs/irm/OfficeHome_Ar
# ResNet50, DomainNet
CUDA_VISIBLE_DEVICES=0 python irm.py data/domainnet -d DomainNet -s i p q r s -t c -a resnet50 -i 4000 --anneal-iters 4000 --lr 0.005 --seed 0 --log logs/irm/DomainNet_c
CUDA_VISIBLE_DEVICES=0 python irm.py data/domainnet -d DomainNet -s c p q r s -t i -a resnet50 -i 4000 --anneal-iters 4000 --lr 0.005 --seed 0 --log logs/irm/DomainNet_i
CUDA_VISIBLE_DEVICES=0 python irm.py data/domainnet -d DomainNet -s c i q r s -t p -a resnet50 -i 4000 --anneal-iters 4000 --lr 0.005 --seed 0 --log logs/irm/DomainNet_p
CUDA_VISIBLE_DEVICES=0 python irm.py data/domainnet -d DomainNet -s c i p r s -t q -a resnet50 -i 4000 --anneal-iters 4000 --lr 0.005 --seed 0 --log logs/irm/DomainNet_q
CUDA_VISIBLE_DEVICES=0 python irm.py data/domainnet -d DomainNet -s c i p q s -t r -a resnet50 -i 4000 --anneal-iters 4000 --lr 0.005 --seed 0 --log logs/irm/DomainNet_r
CUDA_VISIBLE_DEVICES=0 python irm.py data/domainnet -d DomainNet -s c i p q r -t s -a resnet50 -i 4000 --anneal-iters 4000 --lr 0.005 --seed 0 --log logs/irm/DomainNet_s
# ResNet50, Wilds Dataset
CUDA_VISIBLE_DEVICES=0 python irm.py data/wilds -d iwildcam --train-resizing 'res2x' --val-resizing 'res2x' \
-a resnet50 -b 16 --n-domains-per-batch 2 --epochs 60 -i 1000 --anneal-iters 1000 --lr 0.0003 --finetune --trade-off 0.1 \
--seed 0 --log logs/irm/iwildcam
CUDA_VISIBLE_DEVICES=0 python irm.py data/wilds -d camelyon17 -a resnet50 -b 36 --n-domains-per-batch 3 \
--epochs 20 -i 1000 --anneal-iters 1000 --lr 0.001 --trade-off 0.1 --seed 0 --log logs/irm/camelyon17
# DenseNet121, Wilds Dataset
CUDA_VISIBLE_DEVICES=0 python irm.py data/wilds -d fmow --train-resizing 'cen.crop' -a densenet121 \
-b 36 --n-domains-per-batch 4 --epochs 60 -i 1000 --anneal-iters 1000 --lr 0.003 --trade-off 0.1 --seed 0 --log logs/irm/fmow
================================================
FILE: domain_generalization/Transfer-Learning-Library/examples/domain_generalization/image_classification/mixstyle.py
================================================
"""
@author: Baixu Chen
@contact: cbx_99_hasta@outlook.com
"""
import random
import time
import warnings
import sys
import argparse
import shutil
import os.path as osp
import torch
import torch.nn as nn
import torch.backends.cudnn as cudnn
from torch.optim import SGD
from torch.optim.lr_scheduler import CosineAnnealingLR
from torch.utils.data import DataLoader
import torch.nn.functional as F
sys.path.append('../../..')
import dglib.generalization.mixstyle.models as models
from dglib.modules.sampler import RandomDomainSampler
from dglib.modules.classifier import ImageClassifier as Classifier
from common.utils.data import ForeverDataIterator
from common.utils.metric import accuracy
from common.utils.meter import AverageMeter, ProgressMeter
from common.utils.logger import CompleteLogger
from common.utils.analysis import tsne, a_distance
sys.path.append('.')
import utils
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
class TransformerDecorator(torch.nn.Module):
def __init__(self, backbone=None, add_bt=3, dim=2048, eval_global=0):
super(TransformerDecorator, self).__init__()
self.eval_global = eval_global
self.add_bt = add_bt
self.backbone = backbone
self.encoder_layers = torch.nn.TransformerEncoderLayer(dim, 4, dim, 0.5)
def forward(self, feature):
feature = self.backbone(feature)
if self.training and self.add_bt > 0:
pre_feature = feature
feature = feature.unsqueeze(1)
feature = self.encoder_layers(feature)
# feature = self.encoder_layers(feature, feature, feature)
feature = feature.squeeze(1)
if self.add_bt:
return torch.cat([pre_feature, feature], dim=0)
return feature
def main(args: argparse.Namespace):
logger = CompleteLogger(args.log, args.phase)
print(args)
if args.seed is not None:
random.seed(args.seed)
torch.manual_seed(args.seed)
cudnn.deterministic = True
warnings.warn('You have chosen to seed training. '
'This will turn on the CUDNN deterministic setting, '
'which can slow down your training considerably! '
'You may see unexpected behavior when restarting '
'from checkpoints.')
cudnn.benchmark = True
# Data loading code
train_transform = utils.get_train_transform(args.train_resizing, random_horizontal_flip=True,
random_color_jitter=True, random_gray_scale=True)
val_transform = utils.get_val_transform(args.val_resizing)
print("train_transform: ", train_transform)
print("val_transform: ", val_transform)
train_dataset, num_classes = utils.get_dataset(dataset_name=args.data, root=args.root, task_list=args.sources,
split='train', download=True, transform=train_transform,
seed=args.seed)
sampler = RandomDomainSampler(train_dataset, args.batch_size, n_domains_per_batch=2)
train_loader = DataLoader(train_dataset, batch_size=args.batch_size, num_workers=args.workers,
sampler=sampler, drop_last=True)
val_dataset, _ = utils.get_dataset(dataset_name=args.data, root=args.root, task_list=args.sources, split='val',
download=True, transform=val_transform, seed=args.seed)
val_loader = DataLoader(val_dataset, batch_size=args.batch_size, shuffle=False, num_workers=args.workers)
test_dataset, _ = utils.get_dataset(dataset_name=args.data, root=args.root, task_list=args.targets, split='test',
download=True, transform=val_transform, seed=args.seed)
test_loader = DataLoader(test_dataset, batch_size=args.batch_size, shuffle=False, num_workers=args.workers)
print("train_dataset_size: ", len(train_dataset))
print('val_dataset_size: ', len(val_dataset))
print("test_dataset_size: ", len(test_dataset))
train_iter = ForeverDataIterator(train_loader)
# create model
print("=> using pre-trained model '{}'".format(args.arch))
backbone = models.__dict__[args.arch](mix_layers=args.mix_layers, mix_p=args.mix_p, mix_alpha=args.mix_alpha,
pretrained=True)
pool_layer = nn.Identity() if args.no_pool else None
pool_layer = nn.Sequential(
nn.AdaptiveAvgPool2d(output_size=(1, 1)),
nn.Flatten(),
)
pool_layer = TransformerDecorator(pool_layer, add_bt=args.add_bt, dim=backbone.out_features)
classifier = Classifier(backbone, num_classes, freeze_bn=args.freeze_bn, dropout_p=args.dropout_p,
finetune=args.finetune, pool_layer=pool_layer).to(device)
# define optimizer and lr scheduler
optimizer = SGD(classifier.get_parameters(base_lr=args.lr), args.lr, momentum=args.momentum, weight_decay=args.wd,
nesterov=True)
lr_scheduler = CosineAnnealingLR(optimizer, args.epochs * args.iters_per_epoch)
# resume from the best checkpoint
if args.phase != 'train':
checkpoint = torch.load(logger.get_checkpoint_path('best'), map_location='cpu')
classifier.load_state_dict(checkpoint)
# analysis the model
if args.phase == 'analysis':
# extract features from both domains
feature_extractor = nn.Sequential(classifier.backbone, classifier.pool_layer, classifier.bottleneck).to(device)
source_feature = utils.collect_feature(val_loader, feature_extractor, device, max_num_features=100)
target_feature = utils.collect_feature(test_loader, feature_extractor, device, max_num_features=100)
print(len(source_feature), len(target_feature))
# plot t-SNE
tSNE_filename = osp.join(logger.visualize_directory, 'TSNE.png')
tsne.visualize(source_feature, target_feature, tSNE_filename)
print("Saving t-SNE to", tSNE_filename)
# calculate A-distance, which is a measure for distribution discrepancy
A_distance = a_distance.calculate(source_feature, target_feature, device)
print("A-distance =", A_distance)
return
if args.phase == 'test':
acc1 = utils.validate(test_loader, classifier, args, device)
print(acc1)
return
# start training
best_val_acc1 = 0.
best_test_acc1 = 0.
for epoch in range(args.epochs):
print(lr_scheduler.get_lr())
# train for one epoch
train(train_iter, classifier, optimizer, lr_scheduler, epoch, args)
# evaluate on validation set
print("Evaluate on validation set...")
acc1 = utils.validate(val_loader, classifier, args, device)
# remember best acc@1 and save checkpoint
torch.save(classifier.state_dict(), logger.get_checkpoint_path('latest'))
if acc1 > best_val_acc1:
shutil.copy(logger.get_checkpoint_path('latest'), logger.get_checkpoint_path('best'))
best_val_acc1 = max(acc1, best_val_acc1)
# evaluate on test set
print("Evaluate on test set...")
best_test_acc1 = max(best_test_acc1, utils.validate(test_loader, classifier, args, device))
# evaluate on test set
classifier.load_state_dict(torch.load(logger.get_checkpoint_path('best')))
acc1 = utils.validate(test_loader, classifier, args, device)
print("test acc on test set = {}".format(acc1))
print("oracle acc on test set = {}".format(best_test_acc1))
logger.close()
def train(train_iter: ForeverDataIterator, model: Classifier, optimizer,
lr_scheduler: CosineAnnealingLR, epoch: int, args: argparse.Namespace):
batch_time = AverageMeter('Time', ':4.2f')
data_time = AverageMeter('Data', ':3.1f')
losses = AverageMeter('Loss', ':3.2f')
cls_accs = AverageMeter('Cls Acc', ':3.1f')
progress = ProgressMeter(
args.iters_per_epoch,
[batch_time, data_time, losses, cls_accs],
prefix="Epoch: [{}]".format(epoch))
# switch to train mode
model.train()
end = time.time()
for i in range(args.iters_per_epoch):
x, labels, _ = next(train_iter)
x = x.to(device)
labels = labels.to(device)
# measure data loading time
data_time.update(time.time() - end)
# compute output
y, _ = model(x)
if len(y) > len(labels):
labels = torch.cat([labels, labels], dim=0)
cls_loss = F.cross_entropy(y, labels)
loss = cls_loss
cls_acc = accuracy(y, labels)[0]
losses.update(loss.item(), x.size(0))
cls_accs.update(cls_acc.item(), x.size(0))
# compute gradient and do SGD step
optimizer.zero_grad()
loss.backward()
optimizer.step()
lr_scheduler.step()
# measure elapsed time
batch_time.update(time.time() - end)
end = time.time()
if i % args.print_freq == 0:
progress.display(i)
if __name__ == '__main__':
architecture_names = sorted(
name for name in models.__dict__
if name.islower() and not name.startswith("__")
and callable(models.__dict__[name])
)
parser = argparse.ArgumentParser(description='MixStyle for Domain Generalization')
# dataset parameters
parser.add_argument('root', metavar='DIR',
help='root path of dataset')
parser.add_argument('-d', '--data', metavar='DATA', default='PACS',
help='dataset: ' + ' | '.join(utils.get_dataset_names()) +
' (default: PACS)')
parser.add_argument('-s', '--sources', nargs='+', default=None,
help='source domain(s)')
parser.add_argument('-t', '--targets', nargs='+', default=None,
help='target domain(s)')
parser.add_argument('--train-resizing', type=str, default='default')
parser.add_argument('--val-resizing', type=str, default='default')
# model parameters
parser.add_argument('-a', '--arch', metavar='ARCH', default='resnet50',
choices=architecture_names,
help='backbone architecture: ' +
' | '.join(architecture_names) +
' (default: resnet50)')
parser.add_argument('--no-pool', action='store_true', help='no pool layer after the feature extractor.')
parser.add_argument('--mix-layers', nargs='+', help='layers to apply MixStyle')
parser.add_argument('--mix-p', default=0.5, type=float, help='probability to apply MixStyle')
parser.add_argument('--mix-alpha', default=0.1, type=float, help='parameter alpha for beta distribution')
parser.add_argument('--finetune', action='store_true', help='whether use 10x smaller lr for backbone')
parser.add_argument('--freeze-bn', action='store_true', help='whether freeze all bn layers')
parser.add_argument('--dropout-p', type=float, default=0.1, help='only activated when freeze-bn is True')
# training parameters
parser.add_argument('-b', '--batch-size', default=36, type=int,
metavar='N',
help='mini-batch size (default: 36)')
parser.add_argument('--lr', '--learning-rate', default=1e-3, type=float,
metavar='LR', help='initial learning rate', dest='lr')
parser.add_argument('--momentum', default=0.9, type=float, metavar='M',
help='momentum')
parser.add_argument('--wd', '--weight-decay', default=0.0005, type=float,
metavar='W', help='weight decay (default: 5e-4)')
parser.add_argument('-j', '--workers', default=4, type=int, metavar='N',
help='number of data loading workers (default: 4)')
parser.add_argument('--epochs', default=20, type=int, metavar='N',
help='number of total epochs to run')
parser.add_argument('-i', '--iters-per-epoch', default=500, type=int,
help='Number of iterations per epoch')
parser.add_argument('-p', '--print-freq', default=100, type=int,
metavar='N', help='print frequency (default: 100)')
parser.add_argument('--seed', default=None, type=int,
help='seed for initializing training. ')
parser.add_argument("--log", type=str, default='mixstyle',
help="Where to save logs, checkpoints and debugging images.")
parser.add_argument("--phase", type=str, default='train', choices=['train', 'test', 'analysis'],
help="When phase is 'test', only test the model."
"When phase is 'analysis', only analysis the model.")
parser.add_argument("--add_bt", type=int, default=0,
help="add_bt")
args = parser.parse_args()
main(args)
================================================
FILE: domain_generalization/Transfer-Learning-Library/examples/domain_generalization/image_classification/mixstyle.sh
================================================
#!/usr/bin/env bash
# ResNet50, PACS
CUDA_VISIBLE_DEVICES=0 python mixstyle.py data/PACS -d PACS -s A C S -t P -a resnet50 --mix-layers layer1 layer2 layer3 --freeze-bn --seed 0 --log logs/mixstyle/PACS_P
CUDA_VISIBLE_DEVICES=0 python mixstyle.py data/PACS -d PACS -s P C S -t A -a resnet50 --mix-layers layer1 layer2 layer3 --freeze-bn --seed 0 --log logs/mixstyle/PACS_A
CUDA_VISIBLE_DEVICES=0 python mixstyle.py data/PACS -d PACS -s P A S -t C -a resnet50 --mix-layers layer1 layer2 layer3 --freeze-bn --seed 0 --log logs/mixstyle/PACS_C
CUDA_VISIBLE_DEVICES=0 python mixstyle.py data/PACS -d PACS -s P A C -t S -a resnet50 --mix-layers layer1 layer2 layer3 --freeze-bn --seed 0 --log logs/mixstyle/PACS_S
# ResNet50, Office-Home
CUDA_VISIBLE_DEVICES=0 python mixstyle.py data/office-home -d OfficeHome -s Ar Cl Rw -t Pr -a resnet50 --mix-layers layer1 layer2 --seed 0 --log logs/mixstyle/OfficeHome_Pr
CUDA_VISIBLE_DEVICES=0 python mixstyle.py data/office-home -d OfficeHome -s Ar Cl Pr -t Rw -a resnet50 --mix-layers layer1 layer2 --seed 0 --log logs/mixstyle/OfficeHome_Rw
CUDA_VISIBLE_DEVICES=0 python mixstyle.py data/office-home -d OfficeHome -s Ar Rw Pr -t Cl -a resnet50 --mix-layers layer1 layer2 --seed 0 --log logs/mixstyle/OfficeHome_Cl
CUDA_VISIBLE_DEVICES=0 python mixstyle.py data/office-home -d OfficeHome -s Cl Rw Pr -t Ar -a resnet50 --mix-layers layer1 layer2 --seed 0 --log logs/mixstyle/OfficeHome_Ar
# ResNet50, DomainNet
CUDA_VISIBLE_DEVICES=0 python mixstyle.py data/domainnet -d DomainNet -s i p q r s -t c -a resnet50 -i 2500 --lr 0.01 --seed 0 --log logs/mixstyle/DomainNet_c
CUDA_VISIBLE_DEVICES=0 python mixstyle.py data/domainnet -d DomainNet -s c p q r s -t i -a resnet50 -i 2500 --lr 0.01 --seed 0 --log logs/mixstyle/DomainNet_i
CUDA_VISIBLE_DEVICES=0 python mixstyle.py data/domainnet -d DomainNet -s c i q r s -t p -a resnet50 -i 2500 --lr 0.01 --seed 0 --log logs/mixstyle/DomainNet_p
CUDA_VISIBLE_DEVICES=0 python mixstyle.py data/domainnet -d DomainNet -s c i p r s -t q -a resnet50 -i 2500 --lr 0.01 --seed 0 --log logs/mixstyle/DomainNet_q
CUDA_VISIBLE_DEVICES=0 python mixstyle.py data/domainnet -d DomainNet -s c i p q s -t r -a resnet50 -i 2500 --lr 0.01 --seed 0 --log logs/mixstyle/DomainNet_r
CUDA_VISIBLE_DEVICES=0 python mixstyle.py data/domainnet -d DomainNet -s c i p q r -t s -a resnet50 -i 2500 --lr 0.01 --seed 0 --log logs/mixstyle/DomainNet_s
# ResNet50, Wilds Dataset
CUDA_VISIBLE_DEVICES=3 python mixstyle.py data/wilds -d iwildcam --train-resizing 'res2x' --val-resizing 'res2x' \
-a resnet50 --mix-layers layer1 layer2 -b 16 --epochs 60 -i 1000 --lr 0.001 --finetune --seed 0 --log logs/mixstyle/iwildcam
CUDA_VISIBLE_DEVICES=0 python mixstyle.py data/wilds -d camelyon17 -a resnet50 --mix-layers layer1 layer2 \
-b 36 --epochs 20 -i 1000 --lr 0.01 --seed 0 --log logs/mixstyle/camelyon17
================================================
FILE: domain_generalization/Transfer-Learning-Library/examples/domain_generalization/image_classification/mldg.py
================================================
"""
@author: Baixu Chen
@contact: cbx_99_hasta@outlook.com
"""
import random
import time
import warnings
import sys
import argparse
import shutil
import os.path as osp
import torch
import torch.nn as nn
import torch.backends.cudnn as cudnn
from torch.optim import SGD
from torch.optim.lr_scheduler import CosineAnnealingLR
from torch.utils.data import DataLoader
import torch.nn.functional as F
import higher
sys.path.append('../../..')
from dglib.modules.sampler import RandomDomainSampler
from dglib.modules.classifier import ImageClassifier as Classifier
from common.utils.data import ForeverDataIterator
from common.utils.metric import accuracy
from common.utils.meter import AverageMeter, ProgressMeter
from common.utils.logger import CompleteLogger
from common.utils.analysis import tsne, a_distance
sys.path.append('.')
import utils
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
def main(args: argparse.Namespace):
logger = CompleteLogger(args.log, args.phase)
print(args)
if args.seed is not None:
random.seed(args.seed)
torch.manual_seed(args.seed)
cudnn.deterministic = True
warnings.warn('You have chosen to seed training. '
'This will turn on the CUDNN deterministic setting, '
'which can slow down your training considerably! '
'You may see unexpected behavior when restarting '
'from checkpoints.')
cudnn.benchmark = True
# Data loading code
train_transform = utils.get_train_transform(args.train_resizing, random_horizontal_flip=True,
random_color_jitter=True, random_gray_scale=True)
val_transform = utils.get_val_transform(args.val_resizing)
print("train_transform: ", train_transform)
print("val_transform: ", val_transform)
train_dataset, num_classes = utils.get_dataset(dataset_name=args.data, root=args.root, task_list=args.sources,
split='train', download=True, transform=train_transform,
seed=args.seed)
n_domains_per_batch = args.n_support_domains + args.n_query_domains
sampler = RandomDomainSampler(train_dataset, args.batch_size, n_domains_per_batch=n_domains_per_batch)
train_loader = DataLoader(train_dataset, batch_size=args.batch_size, num_workers=args.workers,
sampler=sampler, drop_last=True)
val_dataset, _ = utils.get_dataset(dataset_name=args.data, root=args.root, task_list=args.sources, split='val',
download=True, transform=val_transform, seed=args.seed)
val_loader = DataLoader(val_dataset, batch_size=args.batch_size, shuffle=False, num_workers=args.workers)
test_dataset, _ = utils.get_dataset(dataset_name=args.data, root=args.root, task_list=args.targets, split='test',
download=True, transform=val_transform, seed=args.seed)
test_loader = DataLoader(test_dataset, batch_size=args.batch_size, shuffle=False, num_workers=args.workers)
print("train_dataset_size: ", len(train_dataset))
print('val_dataset_size: ', len(val_dataset))
print("test_dataset_size: ", len(test_dataset))
train_iter = ForeverDataIterator(train_loader)
# create model
print("=> using pre-trained model '{}'".format(args.arch))
backbone = utils.get_model(args.arch)
pool_layer = nn.Identity() if args.no_pool else None
classifier = Classifier(backbone, num_classes, freeze_bn=args.freeze_bn, dropout_p=args.dropout_p,
finetune=args.finetune, pool_layer=pool_layer).to(device)
# define optimizer and lr scheduler
optimizer = SGD(classifier.get_parameters(base_lr=args.lr), args.lr, momentum=args.momentum, weight_decay=args.wd,
nesterov=True)
lr_scheduler = CosineAnnealingLR(optimizer, args.epochs * args.iters_per_epoch)
# resume from the best checkpoint
if args.phase != 'train':
checkpoint = torch.load(logger.get_checkpoint_path('best'), map_location='cpu')
classifier.load_state_dict(checkpoint)
# analysis the model
if args.phase == 'analysis':
# extract features from both domains
feature_extractor = nn.Sequential(classifier.backbone, classifier.pool_layer, classifier.bottleneck).to(device)
source_feature = utils.collect_feature(val_loader, feature_extractor, device, max_num_features=100)
target_feature = utils.collect_feature(test_loader, feature_extractor, device, max_num_features=100)
print(len(source_feature), len(target_feature))
# plot t-SNE
tSNE_filename = osp.join(logger.visualize_directory, 'TSNE.png')
tsne.visualize(source_feature, target_feature, tSNE_filename)
print("Saving t-SNE to", tSNE_filename)
# calculate A-distance, which is a measure for distribution discrepancy
A_distance = a_distance.calculate(source_feature, target_feature, device)
print("A-distance =", A_distance)
return
if args.phase == 'test':
acc1 = utils.validate(test_loader, classifier, args, device)
print(acc1)
return
# start training
best_val_acc1 = 0.
best_test_acc1 = 0.
for epoch in range(args.epochs):
print(lr_scheduler.get_lr())
# train for one epoch
train(train_iter, classifier, optimizer, lr_scheduler, epoch, n_domains_per_batch, args)
# evaluate on validation set
print("Evaluate on validation set...")
acc1 = utils.validate(val_loader, classifier, args, device)
# remember best acc@1 and save checkpoint
torch.save(classifier.state_dict(), logger.get_checkpoint_path('latest'))
if acc1 > best_val_acc1:
shutil.copy(logger.get_checkpoint_path('latest'), logger.get_checkpoint_path('best'))
best_val_acc1 = max(acc1, best_val_acc1)
# evaluate on test set
print("Evaluate on test set...")
best_test_acc1 = max(best_test_acc1, utils.validate(test_loader, classifier, args, device))
# evaluate on test set
classifier.load_state_dict(torch.load(logger.get_checkpoint_path('best')))
acc1 = utils.validate(test_loader, classifier, args, device)
print("test acc on test set = {}".format(acc1))
print("oracle acc on test set = {}".format(best_test_acc1))
logger.close()
def random_split(x_list, labels_list, n_domains_per_batch, n_support_domains):
assert n_support_domains < n_domains_per_batch
support_domain_idxes = random.sample(range(n_domains_per_batch), n_support_domains)
support_domain_list = [(x_list[idx], labels_list[idx]) for idx in range(n_domains_per_batch) if
idx in support_domain_idxes]
query_domain_list = [(x_list[idx], labels_list[idx]) for idx in range(n_domains_per_batch) if
idx not in support_domain_idxes]
return support_domain_list, query_domain_list
def train(train_iter: ForeverDataIterator, model: Classifier, optimizer, lr_scheduler: CosineAnnealingLR, epoch: int,
n_domains_per_batch: int, args: argparse.Namespace):
batch_time = AverageMeter('Time', ':4.2f')
data_time = AverageMeter('Data', ':3.1f')
losses = AverageMeter('Loss', ':3.2f')
cls_accs = AverageMeter('Cls Acc', ':3.1f')
progress = ProgressMeter(
args.iters_per_epoch,
[batch_time, data_time, losses, cls_accs],
prefix="Epoch: [{}]".format(epoch))
# switch to train mode
model.train()
end = time.time()
for i in range(args.iters_per_epoch):
x, labels, _ = next(train_iter)
x = x.to(device)
labels = labels.to(device)
# measure data loading time
data_time.update(time.time() - end)
# split into support domain and query domain
x_list = x.chunk(n_domains_per_batch, dim=0)
labels_list = labels.chunk(n_domains_per_batch, dim=0)
support_domain_list, query_domain_list = random_split(x_list, labels_list, n_domains_per_batch,
args.n_support_domains)
# clear grad
optimizer.zero_grad()
# compute output
with higher.innerloop_ctx(model, optimizer, copy_initial_weights=False) as (inner_model, inner_optimizer):
# perform inner optimization
for _ in range(args.inner_iters):
loss_inner = 0
for (x_s, labels_s) in support_domain_list:
y_s, _ = inner_model(x_s)
# normalize loss by support domain num
loss_inner += F.cross_entropy(y_s, labels_s) / args.n_support_domains
inner_optimizer.step(loss_inner)
# calculate outer loss
loss_outer = 0
cls_acc = 0
# loss on support domains
for (x_s, labels_s) in support_domain_list:
y_s, _ = model(x_s)
# normalize loss by support domain num
loss_outer += F.cross_entropy(y_s, labels_s) / args.n_support_domains
# loss on query domains
for (x_q, labels_q) in query_domain_list:
y_q, _ = inner_model(x_q)
# normalize loss by query domain num
loss_outer += F.cross_entropy(y_q, labels_q) * args.trade_off / args.n_query_domains
cls_acc += accuracy(y_q, labels_q)[0] / args.n_query_domains
# update statistics
losses.update(loss_outer.item(), args.batch_size)
cls_accs.update(cls_acc.item(), args.batch_size)
# compute gradient and do SGD step
loss_outer.backward()
optimizer.step()
lr_scheduler.step()
# measure elapsed time
batch_time.update(time.time() - end)
end = time.time()
if i % args.print_freq == 0:
progress.display(i)
if __name__ == '__main__':
parser = argparse.ArgumentParser(description='Meta Learning for Domain Generalization')
# dataset parameters
parser.add_argument('root', metavar='DIR',
help='root path of dataset')
parser.add_argument('-d', '--data', metavar='DATA', default='PACS',
help='dataset: ' + ' | '.join(utils.get_dataset_names()) +
' (default: PACS)')
parser.add_argument('-s', '--sources', nargs='+', default=None,
help='source domain(s)')
parser.add_argument('-t', '--targets', nargs='+', default=None,
help='target domain(s)')
parser.add_argument('--train-resizing', type=str, default='default')
parser.add_argument('--val-resizing', type=str, default='default')
# model parameters
parser.add_argument('-a', '--arch', metavar='ARCH', default='resnet50',
choices=utils.get_model_names(),
help='backbone architecture: ' +
' | '.join(utils.get_model_names()) +
' (default: resnet50)')
parser.add_argument('--no-pool', action='store_true', help='no pool layer after the feature extractor.')
parser.add_argument('--finetune', action='store_true', help='whether use 10x smaller lr for backbone')
parser.add_argument('--freeze-bn', action='store_true', help='whether freeze all bn layers')
parser.add_argument('--dropout-p', type=float, default=0.1, help='only activated when freeze-bn is True')
# training parameters
parser.add_argument('--n-support-domains', type=int, default=1,
help='Number of support domains sampled in each iteration')
parser.add_argument('--n-query-domains', type=int, default=2,
help='Number of query domains in each iteration')
parser.add_argument('--trade-off', type=float, default=1,
help='hyper parameter beta')
parser.add_argument('-b', '--batch-size', default=36, type=int,
metavar='N',
help='mini-batch size (default: 36)')
parser.add_argument('--lr', '--learning-rate', default=5e-4, type=float,
metavar='LR', help='initial learning rate', dest='lr')
parser.add_argument('--momentum', default=0.9, type=float, metavar='M',
help='momentum')
parser.add_argument('--wd', '--weight-decay', default=0.0005, type=float,
metavar='W', help='weight decay (default: 5e-4)')
parser.add_argument('-j', '--workers', default=4, type=int, metavar='N',
help='number of data loading workers (default: 4)')
parser.add_argument('--epochs', default=20, type=int, metavar='N',
help='number of total epochs to run')
parser.add_argument('-i', '--iters-per-epoch', default=500, type=int,
help='Number of iterations per epoch')
parser.add_argument('--inner-iters', default=1, type=int,
help='Number of iterations in inner loop')
parser.add_argument('-p', '--print-freq', default=100, type=int,
metavar='N', help='print frequency (default: 100)')
parser.add_argument('--seed', default=None, type=int,
help='seed for initializing training. ')
parser.add_argument("--log", type=str, default='mldg',
help="Where to save logs, checkpoints and debugging images.")
parser.add_argument("--phase", type=str, default='train', choices=['train', 'test', 'analysis'],
help="When phase is 'test', only test the model."
"When phase is 'analysis', only analysis the model.")
args = parser.parse_args()
main(args)
================================================
FILE: domain_generalization/Transfer-Learning-Library/examples/domain_generalization/image_classification/mldg.sh
================================================
#!/usr/bin/env bash
# ResNet50, PACS
CUDA_VISIBLE_DEVICES=0 python mldg.py data/PACS -d PACS -s A C S -t P -a resnet50 --freeze-bn --seed 0 --log logs/mldg/PACS_P
CUDA_VISIBLE_DEVICES=0 python mldg.py data/PACS -d PACS -s P C S -t A -a resnet50 --freeze-bn --seed 0 --log logs/mldg/PACS_A
CUDA_VISIBLE_DEVICES=0 python mldg.py data/PACS -d PACS -s P A S -t C -a resnet50 --freeze-bn --seed 0 --log logs/mldg/PACS_C
CUDA_VISIBLE_DEVICES=0 python mldg.py data/PACS -d PACS -s P A C -t S -a resnet50 --freeze-bn --seed 0 --log logs/mldg/PACS_S
# ResNet50, Office-Home
CUDA_VISIBLE_DEVICES=0 python mldg.py data/office-home -d OfficeHome -s Ar Cl Rw -t Pr -a resnet50 --seed 0 --log logs/mldg/OfficeHome_Pr
CUDA_VISIBLE_DEVICES=0 python mldg.py data/office-home -d OfficeHome -s Ar Cl Pr -t Rw -a resnet50 --seed 0 --log logs/mldg/OfficeHome_Rw
CUDA_VISIBLE_DEVICES=0 python mldg.py data/office-home -d OfficeHome -s Ar Rw Pr -t Cl -a resnet50 --seed 0 --log logs/mldg/OfficeHome_Cl
CUDA_VISIBLE_DEVICES=0 python mldg.py data/office-home -d OfficeHome -s Cl Rw Pr -t Ar -a resnet50 --seed 0 --log logs/mldg/OfficeHome_Ar
# ResNet50, DomainNet
CUDA_VISIBLE_DEVICES=0 python mldg.py data/domainnet -d DomainNet -s i p q r s -t c -a resnet50 -i 5000 --lr 0.005 --seed 0 --log logs/mldg/DomainNet_c
CUDA_VISIBLE_DEVICES=0 python mldg.py data/domainnet -d DomainNet -s c p q r s -t i -a resnet50 -i 5000 --lr 0.005 --seed 0 --log logs/mldg/DomainNet_i
CUDA_VISIBLE_DEVICES=0 python mldg.py data/domainnet -d DomainNet -s c i q r s -t p -a resnet50 -i 5000 --lr 0.005 --seed 0 --log logs/mldg/DomainNet_p
CUDA_VISIBLE_DEVICES=0 python mldg.py data/domainnet -d DomainNet -s c i p r s -t q -a resnet50 -i 5000 --lr 0.005 --seed 0 --log logs/mldg/DomainNet_q
CUDA_VISIBLE_DEVICES=0 python mldg.py data/domainnet -d DomainNet -s c i p q s -t r -a resnet50 -i 5000 --lr 0.005 --seed 0 --log logs/mldg/DomainNet_r
CUDA_VISIBLE_DEVICES=0 python mldg.py data/domainnet -d DomainNet -s c i p q r -t s -a resnet50 -i 5000 --lr 0.005 --seed 0 --log logs/mldg/DomainNet_s
# ResNet50, Wilds Dataset
CUDA_VISIBLE_DEVICES=0 python mldg.py data/wilds -d camelyon17 -a resnet50 -b 36 \
--n-support-domains 1 --n-query-domains 2 --epochs 20 -i 1000 --lr 0.003 --seed 0 --log logs/mldg/camelyon17
# DenseNet121, Wilds Dataset
CUDA_VISIBLE_DEVICES=5 python mldg.py data/wilds -d fmow --train-resizing 'cen.crop' -a densenet121 \
-b 36 --n-support-domains 2 --n-query-domains 2 --epochs 60 -i 1000 --lr 0.001 --seed 0 --log logs/mldg/fmow
================================================
FILE: domain_generalization/Transfer-Learning-Library/examples/domain_generalization/image_classification/requirements.txt
================================================
timm
wilds
higher
================================================
FILE: domain_generalization/Transfer-Learning-Library/examples/domain_generalization/image_classification/utils.py
================================================
"""
@author: Baixu Chen
@contact: cbx_99_hasta@outlook.com
"""
import sys
import time
import timm
import tqdm
import torch
import torch.nn as nn
import torchvision.transforms as T
import torch.nn.functional as F
import numpy as np
from torch.utils.data.dataset import Subset, ConcatDataset
import wilds
sys.path.append('../../..')
import common.vision.datasets as datasets
import common.vision.models as models
from common.vision.transforms import ResizeImage
from common.utils.metric import accuracy
from common.utils.meter import AverageMeter, ProgressMeter
def get_model_names():
return sorted(
name for name in models.__dict__
if name.islower() and not name.startswith("__")
and callable(models.__dict__[name])
) + timm.list_models()
def get_model(model_name):
if model_name in models.__dict__:
# load models from common.vision.models
backbone = models.__dict__[model_name](pretrained=True)
else:
# load models from pytorch-image-models
backbone = timm.create_model(model_name, pretrained=True)
try:
backbone.out_features = backbone.get_classifier().in_features
backbone.reset_classifier(0, '')
except:
backbone.out_features = backbone.head.in_features
backbone.head = nn.Identity()
return backbone
def get_dataset_names():
return sorted(
name for name in datasets.__dict__
if not name.startswith("__") and callable(datasets.__dict__[name])
) + wilds.supported_datasets
class ConcatDatasetWithDomainLabel(ConcatDataset):
"""ConcatDataset with domain label"""
def __init__(self, *args, **kwargs):
super(ConcatDatasetWithDomainLabel, self).__init__(*args, **kwargs)
self.index_to_domain_id = {}
domain_id = 0
start = 0
for end in self.cumulative_sizes:
for idx in range(start, end):
self.index_to_domain_id[idx] = domain_id
start = end
domain_id += 1
def __getitem__(self, index):
img, target = super(ConcatDatasetWithDomainLabel, self).__getitem__(index)
domain_id = self.index_to_domain_id[index]
return img, target, domain_id
def convert_from_wilds_dataset(dataset_name, wild_dataset):
metadata_array = wild_dataset.metadata_array
sample_idxes_per_domain = {}
for idx, metadata in enumerate(metadata_array):
if dataset_name == 'iwildcam':
# In iwildcam dataset, domain id is specified by location
domain = metadata[0].item()
elif dataset_name == 'camelyon17':
# In camelyon17 dataset, domain id is specified by hospital
domain = metadata[0].item()
elif dataset_name == 'fmow':
# In fmow dataset, domain id is specified by (region, year) tuple
domain = (metadata[0].item(), metadata[1].item())
if domain not in sample_idxes_per_domain:
sample_idxes_per_domain[domain] = []
sample_idxes_per_domain[domain].append(idx)
class Dataset:
def __init__(self):
self.dataset = wild_dataset
def __getitem__(self, idx):
x, y, metadata = self.dataset[idx]
return x, y
def __len__(self):
return len(self.dataset)
dataset = Dataset()
concat_dataset = ConcatDatasetWithDomainLabel(
[Subset(dataset, sample_idxes_per_domain[domain]) for domain in sample_idxes_per_domain])
return concat_dataset
def get_dataset(dataset_name, root, task_list, split='train', download=True, transform=None, seed=0):
assert split in ['train', 'val', 'test']
if dataset_name in datasets.__dict__:
# load datasets from common.vision.datasets
# currently only PACS, OfficeHome and DomainNet are supported
supported_dataset = ['PACS', 'OfficeHome', 'DomainNet']
assert dataset_name in supported_dataset
dataset = datasets.__dict__[dataset_name]
train_split_list = []
val_split_list = []
test_split_list = []
# we follow DomainBed and split each dataset randomly into two parts, with 80% samples and 20% samples
# respectively, the former (larger) will be used as training set, and the latter will be used as validation set.
split_ratio = 0.8
num_classes = 0
# under domain generalization setting, we use all samples in target domain as test set
for task in task_list:
if dataset_name == 'PACS':
all_split = dataset(root=root, task=task, split='all', download=download, transform=transform)
num_classes = all_split.num_classes
elif dataset_name == 'OfficeHome':
all_split = dataset(root=root, task=task, download=download, transform=transform)
num_classes = all_split.num_classes
elif dataset_name == 'DomainNet':
train_split = dataset(root=root, task=task, split='train', download=download, transform=transform)
test_split = dataset(root=root, task=task, split='test', download=download, transform=transform)
num_classes = train_split.num_classes
all_split = ConcatDataset([train_split, test_split])
train_split, val_split = split_dataset(all_split, int(len(all_split) * split_ratio), seed)
train_split_list.append(train_split)
val_split_list.append(val_split)
test_split_list.append(all_split)
train_dataset = ConcatDatasetWithDomainLabel(train_split_list)
val_dataset = ConcatDatasetWithDomainLabel(val_split_list)
test_dataset = ConcatDatasetWithDomainLabel(test_split_list)
dataset_dict = {
'train': train_dataset,
'val': val_dataset,
'test': test_dataset
}
return dataset_dict[split], num_classes
else:
# load datasets from wilds
# currently only iwildcam, camelyon17 and fmow are supported
supported_dataset = ['iwildcam', 'camelyon17', 'fmow']
assert dataset_name in supported_dataset
dataset = wilds.get_dataset(dataset_name, root_dir=root, download=True)
num_classes = dataset.n_classes
return convert_from_wilds_dataset(dataset_name,
dataset.get_subset(split=split, transform=transform)), num_classes
def split_dataset(dataset, n, seed=0):
"""
Return a pair of datasets corresponding to a random split of the given
dataset, with n data points in the first dataset and the rest in the last,
using the given random seed
"""
assert (n <= len(dataset))
idxes = list(range(len(dataset)))
np.random.RandomState(seed).shuffle(idxes)
subset_1 = idxes[:n]
subset_2 = idxes[n:]
return Subset(dataset, subset_1), Subset(dataset, subset_2)
def validate(val_loader, model, args, device) -> float:
batch_time = AverageMeter('Time', ':6.3f')
losses = AverageMeter('Loss', ':.4e')
top1 = AverageMeter('Acc@1', ':6.2f')
progress = ProgressMeter(
len(val_loader),
[batch_time, losses, top1],
prefix='Test: ')
# switch to evaluate mode
model.eval()
with torch.no_grad():
end = time.time()
for i, (images, target, _) in enumerate(val_loader):
images = images.to(device)
target = target.to(device)
# compute output
output = model(images)
loss = F.cross_entropy(output, target)
# measure accuracy and record loss
acc1 = accuracy(output, target)[0]
losses.update(loss.item(), images.size(0))
top1.update(acc1.item(), images.size(0))
# measure elapsed time
batch_time.update(time.time() - end)
end = time.time()
if i % args.print_freq == 0:
progress.display(i)
print(' * Acc@1 {top1.avg:.3f} '.format(top1=top1))
return top1.avg
def get_train_transform(resizing='default', random_horizontal_flip=True, random_color_jitter=True,
random_gray_scale=True):
"""
resizing mode:
- default: random resized crop with scale factor(0.7, 1.0) and size 224;
- cen.crop: take the center crop of 224;
- res.|cen.crop: resize the image to 256 and take the center crop of size 224;
- res: resize the image to 224;
- res2x: resize the image to 448;
- res.|crop: resize the image to 256 and take a random crop of size 224;
- res.sma|crop: resize the image keeping its aspect ratio such that the
smaller side is 256, then take a random crop of size 224;
– inc.crop: “inception crop” from (Szegedy et al., 2015);
– cif.crop: resize the image to 224, zero-pad it by 28 on each side, then take a random crop of size 224.
"""
if resizing == 'default':
transform = T.RandomResizedCrop(224, scale=(0.7, 1.0))
elif resizing == 'cen.crop':
transform = T.CenterCrop(224)
elif resizing == 'res.|cen.crop':
transform = T.Compose([
ResizeImage(256),
T.CenterCrop(224)
])
elif resizing == 'res':
transform = ResizeImage(224)
elif resizing == 'res2x':
transform = ResizeImage(448)
elif resizing == 'res.|crop':
transform = T.Compose([
T.Resize((256, 256)),
T.RandomCrop(224)
])
elif resizing == "res.sma|crop":
transform = T.Compose([
T.Resize(256),
T.RandomCrop(224)
])
elif resizing == 'inc.crop':
transform = T.RandomResizedCrop(224)
elif resizing == 'cif.crop':
transform = T.Compose([
T.Resize((224, 224)),
T.Pad(28),
T.RandomCrop(224),
])
else:
raise NotImplementedError(resizing)
transforms = [transform]
if random_horizontal_flip:
transforms.append(T.RandomHorizontalFlip())
if random_color_jitter:
transforms.append(T.ColorJitter(brightness=0.3, contrast=0.3, saturation=0.3, hue=0.3))
if random_gray_scale:
transforms.append(T.RandomGrayscale())
transforms.extend([
T.ToTensor(),
T.Normalize(mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225])
])
return T.Compose(transforms)
def get_val_transform(resizing='default'):
"""
resizing mode:
- default: resize the image to 224;
- res2x: resize the image to 448;
- res.|cen.crop: resize the image to 256 and take the center crop of size 224;
"""
if resizing == 'default':
transform = ResizeImage(224)
elif resizing == 'res2x':
transform = ResizeImage(448)
elif resizing == 'res.|cen.crop':
transform = T.Compose([
ResizeImage(256),
T.CenterCrop(224),
])
else:
raise NotImplementedError(resizing)
return T.Compose([
transform,
T.ToTensor(),
T.Normalize(mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225])
])
def collect_feature(data_loader, feature_extractor: nn.Module, device: torch.device,
max_num_features=None) -> torch.Tensor:
"""
Fetch data from `data_loader`, and then use `feature_extractor` to collect features. This function is
specific for domain generalization because each element in data_loader is a tuple
(images, labels, domain_labels).
Args:
data_loader (torch.utils.data.DataLoader): Data loader.
feature_extractor (torch.nn.Module): A feature extractor.
device (torch.device)
max_num_features (int): The max number of features to return
Returns:
Features in shape (min(len(data_loader), max_num_features * mini-batch size), :math:`|\mathcal{F}|`).
"""
feature_extractor.eval()
all_features = []
with torch.no_grad():
for i, (images, target, domain_labels) in enumerate(tqdm.tqdm(data_loader)):
if max_num_features is not None and i >= max_num_features:
break
images = images.to(device)
feature = feature_extractor(images).cpu()
all_features.append(feature)
return torch.cat(all_features, dim=0)
================================================
FILE: domain_generalization/Transfer-Learning-Library/examples/domain_generalization/image_classification/vrex.py
================================================
"""
@author: Baixu Chen
@contact: cbx_99_hasta@outlook.com
"""
import random
import time
import warnings
import sys
import argparse
import shutil
import os.path as osp
import torch
import torch.nn as nn
import torch.backends.cudnn as cudnn
from torch.optim import SGD
from torch.optim.lr_scheduler import CosineAnnealingLR
from torch.utils.data import DataLoader
import torch.nn.functional as F
sys.path.append('../../..')
from dglib.modules.sampler import RandomDomainSampler
from dglib.modules.classifier import ImageClassifier as Classifier
from common.utils.data import ForeverDataIterator
from common.utils.metric import accuracy
from common.utils.meter import AverageMeter, ProgressMeter
from common.utils.logger import CompleteLogger
from common.utils.analysis import tsne, a_distance
sys.path.append('.')
import utils
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
class TransformerDecorator(torch.nn.Module):
def __init__(self, backbone=None, add_bt=3, dim=2048, eval_global=0):
super(TransformerDecorator, self).__init__()
self.eval_global = eval_global
self.add_bt = add_bt
self.backbone = backbone
if self.add_bt > 0:
self.encoder_layers = torch.nn.TransformerEncoderLayer(dim, 4, dim, 0.5)
def forward(self, feature):
feature = self.backbone(feature)
if self.training and self.add_bt > 0:
pre_feature = feature
feature = feature.unsqueeze(1)
feature = self.encoder_layers(feature)
# feature = self.encoder_layers(feature, feature, feature)
feature = feature.squeeze(1)
if self.add_bt >= 1:
return torch.cat([pre_feature, feature], dim=0)
return feature
def main(args: argparse.Namespace):
logger = CompleteLogger(args.log, args.phase)
print(args)
if args.seed is not None:
random.seed(args.seed)
torch.manual_seed(args.seed)
cudnn.deterministic = True
warnings.warn('You have chosen to seed training. '
'This will turn on the CUDNN deterministic setting, '
'which can slow down your training considerably! '
'You may see unexpected behavior when restarting '
'from checkpoints.')
cudnn.benchmark = True
# Data loading code
train_transform = utils.get_train_transform(args.train_resizing, random_horizontal_flip=True,
random_color_jitter=True, random_gray_scale=True)
val_transform = utils.get_val_transform(args.val_resizing)
print("train_transform: ", train_transform)
print("val_transform: ", val_transform)
train_dataset, num_classes = utils.get_dataset(dataset_name=args.data, root=args.root, task_list=args.sources,
split='train', download=True, transform=train_transform,
seed=args.seed)
sampler = RandomDomainSampler(train_dataset, args.batch_size, args.n_domains_per_batch)
train_loader = DataLoader(train_dataset, batch_size=args.batch_size, num_workers=args.workers,
sampler=sampler, drop_last=True)
val_dataset, _ = utils.get_dataset(dataset_name=args.data, root=args.root, task_list=args.sources, split='val',
download=True, transform=val_transform, seed=args.seed)
val_loader = DataLoader(val_dataset, batch_size=args.batch_size, shuffle=False, num_workers=args.workers)
test_dataset, _ = utils.get_dataset(dataset_name=args.data, root=args.root, task_list=args.targets, split='test',
download=True, transform=val_transform, seed=args.seed)
test_loader = DataLoader(test_dataset, batch_size=args.batch_size, shuffle=False, num_workers=args.workers)
print("train_dataset_size: ", len(train_dataset))
print('val_dataset_size: ', len(val_dataset))
print("test_dataset_size: ", len(test_dataset))
train_iter = ForeverDataIterator(train_loader)
# create model
print("=> using pre-trained model '{}'".format(args.arch))
backbone = utils.get_model(args.arch)
pool_layer = nn.Identity() if args.no_pool else None
pool_layer = nn.Sequential(
nn.AdaptiveAvgPool2d(output_size=(1, 1)),
nn.Flatten(),
)
pool_layer = TransformerDecorator(pool_layer, add_bt=args.add_bt, dim=backbone.out_features)
classifier = Classifier(backbone, num_classes, freeze_bn=args.freeze_bn, dropout_p=args.dropout_p,
finetune=args.finetune, pool_layer=pool_layer).to(device)
# define optimizer and lr scheduler
optimizer = SGD(classifier.get_parameters(base_lr=args.lr), args.lr, momentum=args.momentum, weight_decay=args.wd,
nesterov=True)
lr_scheduler = CosineAnnealingLR(optimizer, args.epochs * args.iters_per_epoch)
# for simplicity
assert args.anneal_iters % args.iters_per_epoch == 0
# resume from the best checkpoint
if args.phase != 'train':
checkpoint = torch.load(logger.get_checkpoint_path('best'), map_location='cpu')
classifier.load_state_dict(checkpoint)
# analysis the model
if args.phase == 'analysis':
# extract features from both domains
feature_extractor = nn.Sequential(classifier.backbone, classifier.pool_layer, classifier.bottleneck).to(device)
source_feature = utils.collect_feature(val_loader, feature_extractor, device, max_num_features=100)
target_feature = utils.collect_feature(test_loader, feature_extractor, device, max_num_features=100)
print(len(source_feature), len(target_feature))
# plot t-SNE
tSNE_filename = osp.join(logger.visualize_directory, 'TSNE.png')
tsne.visualize(source_feature, target_feature, tSNE_filename)
print("Saving t-SNE to", tSNE_filename)
# calculate A-distance, which is a measure for distribution discrepancy
A_distance = a_distance.calculate(source_feature, target_feature, device)
print("A-distance =", A_distance)
return
if args.phase == 'test':
acc1 = utils.validate(test_loader, classifier, args, device)
print(acc1)
return
# start training
best_val_acc1 = 0.
best_test_acc1 = 0.
for epoch in range(args.epochs):
if epoch * args.iters_per_epoch == args.anneal_iters:
# reset optimizer to avoid sharp jump in gradient magnitudes
optimizer = SGD(classifier.get_parameters(base_lr=args.lr), args.lr, momentum=args.momentum,
weight_decay=args.wd, nesterov=True)
lr_scheduler = CosineAnnealingLR(optimizer, args.epochs * args.iters_per_epoch - args.anneal_iters)
print(lr_scheduler.get_lr())
# train for one epoch
train(train_iter, classifier, optimizer, lr_scheduler, args.n_domains_per_batch, epoch, args)
# evaluate on validation set
print("Evaluate on validation set...")
acc1 = utils.validate(val_loader, classifier, args, device)
# remember best acc@1 and save checkpoint
torch.save(classifier.state_dict(), logger.get_checkpoint_path('latest'))
if acc1 > best_val_acc1:
shutil.copy(logger.get_checkpoint_path('latest'), logger.get_checkpoint_path('best'))
best_val_acc1 = max(acc1, best_val_acc1)
# evaluate on test set
print("Evaluate on test set...")
best_test_acc1 = max(best_test_acc1, utils.validate(test_loader, classifier, args, device))
# evaluate on test set
classifier.load_state_dict(torch.load(logger.get_checkpoint_path('best')))
acc1 = utils.validate(test_loader, classifier, args, device)
print("test acc on test set = {}".format(acc1))
print("oracle acc on test set = {}".format(best_test_acc1))
logger.close()
def train(train_iter: ForeverDataIterator, model: Classifier, optimizer, lr_scheduler: CosineAnnealingLR,
n_domains_per_batch: int, epoch: int, args: argparse.Namespace):
batch_time = AverageMeter('Time', ':4.2f')
data_time = AverageMeter('Data', ':3.1f')
losses = AverageMeter('Loss', ':3.2f')
losses_ce = AverageMeter('CELoss', ':3.2f')
losses_penalty = AverageMeter('Penalty Loss', ':3.2f')
cls_accs = AverageMeter('Cls Acc', ':3.1f')
progress = ProgressMeter(
args.iters_per_epoch,
[batch_time, data_time, losses, losses_ce, losses_penalty, cls_accs],
prefix="Epoch: [{}]".format(epoch))
# switch to train mode
model.train()
end = time.time()
for i in range(args.iters_per_epoch):
x_all, labels_all, _ = next(train_iter)
x_all = x_all.to(device)
labels_all = labels_all.to(device)
# measure data loading time
data_time.update(time.time() - end)
# compute output
y_all, _ = model(x_all)
if len(y_all) > len(labels_all):
all_shape = y_all.shape
y_all = torch.unsqueeze(y_all, 1)
y_all = torch.cat([y_all[:len(y_all)//2], y_all[len(y_all)//2:]], dim=1)
y_all = torch.reshape(y_all, all_shape)
# old_label_shape = labels_all.shape
labels_all = torch.unsqueeze(labels_all, 1)
labels_all = torch.cat([labels_all, labels_all], dim=1)
labels_all = torch.reshape(labels_all, [-1])
loss_ce_per_domain = torch.zeros(n_domains_per_batch).to(device)
for domain_id, (y_per_domain, labels_per_domain) in enumerate(
zip(y_all.chunk(n_domains_per_batch, dim=0), labels_all.chunk(n_domains_per_batch, dim=0))):
loss_ce_per_domain[domain_id] = F.cross_entropy(y_per_domain, labels_per_domain)
# cls loss
loss_ce = loss_ce_per_domain.mean()
# penalty loss
loss_penalty = ((loss_ce_per_domain - loss_ce) ** 2).mean()
global_iter = epoch * args.iters_per_epoch + i
if global_iter >= args.anneal_iters:
trade_off = args.trade_off
else:
trade_off = 1
loss = loss_ce + loss_penalty * trade_off
cls_acc = accuracy(y_all, labels_all)[0]
losses.update(loss.item(), x_all.size(0))
losses_ce.update(loss_ce.item(), x_all.size(0))
losses_penalty.update(loss_penalty.item(), x_all.size(0))
cls_accs.update(cls_acc.item(), x_all.size(0))
# compute gradient and do SGD step
optimizer.zero_grad()
loss.backward()
optimizer.step()
lr_scheduler.step()
# measure elapsed time
batch_time.update(time.time() - end)
end = time.time()
if i % args.print_freq == 0:
progress.display(i)
if __name__ == '__main__':
parser = argparse.ArgumentParser(description='VREx for Domain Generalization')
# dataset parameters
parser.add_argument('root', metavar='DIR',
help='root path of dataset')
parser.add_argument('-d', '--data', metavar='DATA', default='PACS',
help='dataset: ' + ' | '.join(utils.get_dataset_names()) +
' (default: PACS)')
parser.add_argument('-s', '--sources', nargs='+', default=None,
help='source domain(s)')
parser.add_argument('-t', '--targets', nargs='+', default=None,
help='target domain(s)')
parser.add_argument('--train-resizing', type=str, default='default')
parser.add_argument('--val-resizing', type=str, default='default')
# model parameters
parser.add_argument('-a', '--arch', metavar='ARCH', default='resnet50',
choices=utils.get_model_names(),
help='backbone architecture: ' +
' | '.join(utils.get_model_names()) +
' (default: resnet50)')
parser.add_argument('--no-pool', action='store_true', help='no pool layer after the feature extractor.')
parser.add_argument('--finetune', action='store_true', help='whether use 10x smaller lr for backbone')
parser.add_argument('--freeze-bn', action='store_true', help='whether freeze all bn layers')
parser.add_argument('--dropout-p', type=float, default=0.1, help='only activated when freeze-bn is True')
# training parameters
parser.add_argument('--trade-off', default=3, type=float,
help='the trade off hyper parameter for vrex penalty')
parser.add_argument('--anneal-iters', default=500, type=int,
help='anneal iterations (trade off is set to 1 during these iterations)')
parser.add_argument('-b', '--batch-size', default=36, type=int,
metavar='N',
help='mini-batch size (default: 36)')
parser.add_argument('--n-domains-per-batch', default=3, type=int,
help='number of domains in each mini-batch')
parser.add_argument('--lr', '--learning-rate', default=5e-4, type=float,
metavar='LR', help='initial learning rate', dest='lr')
parser.add_argument('--momentum', default=0.9, type=float, metavar='M',
help='momentum')
parser.add_argument('--wd', '--weight-decay', default=0.0005, type=float,
metavar='W', help='weight decay (default: 5e-4)')
parser.add_argument('-j', '--workers', default=4, type=int, metavar='N',
help='number of data loading workers (default: 4)')
parser.add_argument('--epochs', default=20, type=int, metavar='N',
help='number of total epochs to run')
parser.add_argument('-i', '--iters-per-epoch', default=500, type=int,
help='Number of iterations per epoch')
parser.add_argument('-p', '--print-freq', default=100, type=int,
metavar='N', help='print frequency (default: 100)')
parser.add_argument('--seed', default=None, type=int,
help='seed for initializing training. ')
parser.add_argument("--log", type=str, default='vrex',
help="Where to save logs, checkpoints and debugging images.")
parser.add_argument("--phase", type=str, default='train', choices=['train', 'test', 'analysis'],
help="When phase is 'test', only test the model."
"When phase is 'analysis', only analysis the model.")
parser.add_argument("--add_bt", type=int, default=0,
help="add_bt")
args = parser.parse_args()
main(args)
================================================
FILE: domain_generalization/Transfer-Learning-Library/examples/domain_generalization/image_classification/vrex.sh
================================================
#!/usr/bin/env bash
# ResNet50, PACS
CUDA_VISIBLE_DEVICES=0 python vrex.py data/PACS -d PACS -s A C S -t P -a resnet50 --freeze-bn --seed 0 --log logs/vrex/PACS_P
CUDA_VISIBLE_DEVICES=0 python vrex.py data/PACS -d PACS -s P C S -t A -a resnet50 --freeze-bn --seed 0 --log logs/vrex/PACS_A
CUDA_VISIBLE_DEVICES=0 python vrex.py data/PACS -d PACS -s P A S -t C -a resnet50 --freeze-bn --seed 0 --log logs/vrex/PACS_C
CUDA_VISIBLE_DEVICES=0 python vrex.py data/PACS -d PACS -s P A C -t S -a resnet50 --freeze-bn --seed 0 --log logs/vrex/PACS_S
# ResNet50, Office-Home
CUDA_VISIBLE_DEVICES=0 python vrex.py data/office-home -d OfficeHome -s Ar Cl Rw -t Pr -a resnet50 --seed 0 --log logs/vrex/OfficeHome_Pr
CUDA_VISIBLE_DEVICES=0 python vrex.py data/office-home -d OfficeHome -s Ar Cl Pr -t Rw -a resnet50 --seed 0 --log logs/vrex/OfficeHome_Rw
CUDA_VISIBLE_DEVICES=0 python vrex.py data/office-home -d OfficeHome -s Ar Rw Pr -t Cl -a resnet50 --seed 0 --log logs/vrex/OfficeHome_Cl
CUDA_VISIBLE_DEVICES=0 python vrex.py data/office-home -d OfficeHome -s Cl Rw Pr -t Ar -a resnet50 --seed 0 --log logs/vrex/OfficeHome_Ar
# ResNet50, DomainNet
CUDA_VISIBLE_DEVICES=0 python vrex.py data/domainnet -d DomainNet -s i p q r s -t c -a resnet50 -i 4000 --anneal-iters 4000 --lr 0.005 --trade-off 1 --seed 0 --log logs/vrex/DomainNet_c
CUDA_VISIBLE_DEVICES=0 python vrex.py data/domainnet -d DomainNet -s c p q r s -t i -a resnet50 -i 4000 --anneal-iters 4000 --lr 0.005 --trade-off 1 --seed 0 --log logs/vrex/DomainNet_i
CUDA_VISIBLE_DEVICES=0 python vrex.py data/domainnet -d DomainNet -s c i q r s -t p -a resnet50 -i 4000 --anneal-iters 4000 --lr 0.005 --trade-off 1 --seed 0 --log logs/vrex/DomainNet_p
CUDA_VISIBLE_DEVICES=0 python vrex.py data/domainnet -d DomainNet -s c i p r s -t q -a resnet50 -i 4000 --anneal-iters 4000 --lr 0.005 --trade-off 1 --seed 0 --log logs/vrex/DomainNet_q
CUDA_VISIBLE_DEVICES=0 python vrex.py data/domainnet -d DomainNet -s c i p q s -t r -a resnet50 -i 4000 --anneal-iters 4000 --lr 0.005 --trade-off 1 --seed 0 --log logs/vrex/DomainNet_r
CUDA_VISIBLE_DEVICES=0 python vrex.py data/domainnet -d DomainNet -s c i p q r -t s -a resnet50 -i 4000 --anneal-iters 4000 --lr 0.005 --trade-off 1 --seed 0 --log logs/vrex/DomainNet_s
# ResNet50, Wilds Dataset
CUDA_VISIBLE_DEVICES=0 python vrex.py data/wilds -d iwildcam --train-resizing 'res2x' --val-resizing 'res2x' \
-a resnet50 -b 16 --n-domains-per-batch 2 --epochs 60 -i 1000 --anneal-iters 1000 --lr 0.001 --finetune --trade-off 0.1 \
--seed 0 --log logs/vrex/iwildcam
CUDA_VISIBLE_DEVICES=0 python vrex.py data/wilds -d camelyon17 -a resnet50 -b 36 --n-domains-per-batch 3 \
--epochs 20 -i 1000 --anneal-iters 1000 --lr 0.01 --trade-off 0.1 --seed 0 --log logs/vrex/camelyon17
# DenseNet121, Wilds Dataset
CUDA_VISIBLE_DEVICES=0 python vrex.py data/wilds -d fmow --train-resizing 'cen.crop' -a densenet121 \
-b 36 --n-domains-per-batch 4 --epochs 60 -i 1000 --anneal-iters 1000 --lr 0.003 --trade-off 0.1 --seed 0 --log logs/vrex/fmow
================================================
FILE: domain_generalization/Transfer-Learning-Library/examples/domain_generalization/re_identification/README.md
================================================
# Domain Generalization for Person Re-Identification
## Installation
Example scripts support all models in [PyTorch-Image-Models](https://github.com/rwightman/pytorch-image-models).
You also need to install timm to use PyTorch-Image-Models.
```
pip install timm
```
## Dataset
Following datasets can be downloaded automatically:
- [Market1501](http://zheng-lab.cecs.anu.edu.au/Project/project_reid.html)
- [DukeMTMC](https://exposing.ai/duke_mtmc/)
- [MSMT17](https://arxiv.org/pdf/1711.08565.pdf)
## Supported Methods
Supported methods include:
- [Two at Once: Enhancing Learning and Generalization Capacities via IBN-Net (IBN-Net, 2018 ECCV)](https://openaccess.thecvf.com/content_ECCV_2018/papers/Xingang_Pan_Two_at_Once_ECCV_2018_paper.pdf)
- [Domain Generalization with MixStyle (MixStyle, 2021 ICLR)](https://arxiv.org/abs/2104.02008)
## Experiment and Results
The shell files give the script to reproduce the [benchmarks](/docs/dglib/benchmarks/re_identification.rst) with specified hyper-parameters.
For example, if you want to reproduce MixStyle on Market1501 -> DukeMTMC task, use the following script
```shell script
# Train MixStyle on Market1501 -> DukeMTMC task using ResNet 50.
# Assume you have put the datasets under the path `data/market1501` and `data/dukemtmc`,
# or you are glad to download the datasets automatically from the Internet to this path
CUDA_VISIBLE_DEVICES=0 python mixstyle.py data -s Market1501 -t DukeMTMC -a resnet50 \
--mix-layers layer1 layer2 --finetune --seed 0 --log logs/mixstyle/Market2Duke
```
For more information please refer to [Get Started](/docs/get_started/quickstart.rst) for help.
## Citation
If you use these methods in your research, please consider citing.
```
@inproceedings{IBN-Net,
author = {Xingang Pan, Ping Luo, Jianping Shi, and Xiaoou Tang},
title = {Two at Once: Enhancing Learning and Generalization Capacities via IBN-Net},
booktitle = {ECCV},
year = {2018}
}
@inproceedings{mixstyle,
title={Domain Generalization with MixStyle},
author={Zhou, Kaiyang and Yang, Yongxin and Qiao, Yu and Xiang, Tao},
booktitle={ICLR},
year={2021}
}
```
================================================
FILE: domain_generalization/Transfer-Learning-Library/examples/domain_generalization/re_identification/baseline.py
================================================
"""
@author: Baixu Chen
@contact: cbx_99_hasta@outlook.com
"""
import random
import time
import warnings
import sys
import argparse
import shutil
import os.path as osp
import numpy as np
import torch
import torch.nn as nn
from torch.nn import DataParallel
import torch.backends.cudnn as cudnn
from torch.optim import Adam
from torch.utils.data import DataLoader
sys.path.append('../../..')
from common.vision.models.reid.loss import CrossEntropyLossWithLabelSmooth, SoftTripletLoss
from common.vision.models.reid.identifier import ReIdentifier
import common.vision.datasets.reid as datasets
from common.vision.datasets.reid.convert import convert_to_pytorch_dataset
from common.utils.scheduler import WarmupMultiStepLR
from common.utils.metric.reid import validate, visualize_ranked_results
from common.utils.data import ForeverDataIterator, RandomMultipleGallerySampler
from common.utils.metric import accuracy
from common.utils.meter import AverageMeter, ProgressMeter
from common.utils.logger import CompleteLogger
sys.path.append('.')
import utils
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
def main(args: argparse.Namespace):
logger = CompleteLogger(args.log, args.phase)
print(args)
if args.seed is not None:
random.seed(args.seed)
np.random.seed(args.seed)
torch.manual_seed(args.seed)
cudnn.deterministic = True
warnings.warn('You have chosen to seed training. '
'This will turn on the CUDNN deterministic setting, '
'which can slow down your training considerably! '
'You may see unexpected behavior when restarting '
'from checkpoints.')
cudnn.benchmark = True
# Data loading code
train_transform = utils.get_train_transform(args.height, args.width, args.train_resizing,
random_horizontal_flip=True,
random_color_jitter=False,
random_gray_scale=False)
val_transform = utils.get_val_transform(args.height, args.width)
print("train_transform: ", train_transform)
print("val_transform: ", val_transform)
working_dir = osp.dirname(osp.abspath(__file__))
root = osp.join(working_dir, args.root)
# source dataset
source_dataset = datasets.__dict__[args.source](root=osp.join(root, args.source.lower()))
sampler = RandomMultipleGallerySampler(source_dataset.train, args.num_instances)
train_loader = DataLoader(
convert_to_pytorch_dataset(source_dataset.train, root=source_dataset.images_dir, transform=train_transform),
batch_size=args.batch_size, num_workers=args.workers, sampler=sampler, pin_memory=True, drop_last=True)
train_iter = ForeverDataIterator(train_loader)
val_loader = DataLoader(
convert_to_pytorch_dataset(list(set(source_dataset.query) | set(source_dataset.gallery)),
root=source_dataset.images_dir,
transform=val_transform),
batch_size=args.batch_size, num_workers=args.workers, shuffle=False, pin_memory=True)
# target dataset
target_dataset = datasets.__dict__[args.target](root=osp.join(root, args.target.lower()))
test_loader = DataLoader(
convert_to_pytorch_dataset(list(set(target_dataset.query) | set(target_dataset.gallery)),
root=target_dataset.images_dir,
transform=val_transform),
batch_size=args.batch_size, num_workers=args.workers, shuffle=False, pin_memory=True)
# create model
num_classes = source_dataset.num_train_pids
backbone = utils.get_model(args.arch)
pool_layer = nn.Identity() if args.no_pool else None
model = ReIdentifier(backbone, num_classes, finetune=args.finetune, pool_layer=pool_layer).to(device)
model = DataParallel(model)
# define optimizer and learning rate scheduler
optimizer = Adam(model.module.get_parameters(base_lr=args.lr, rate=args.rate), args.lr,
weight_decay=args.weight_decay)
lr_scheduler = WarmupMultiStepLR(optimizer, args.milestones, gamma=0.1, warmup_factor=0.1,
warmup_steps=args.warmup_steps)
# resume from the best checkpoint
if args.phase != 'train':
checkpoint = torch.load(logger.get_checkpoint_path('best'), map_location='cpu')
model.load_state_dict(checkpoint)
# analysis the model
if args.phase == 'analysis':
# plot t-SNE
utils.visualize_tsne(source_loader=val_loader, target_loader=test_loader, model=model,
filename=osp.join(logger.visualize_directory, 'analysis', 'TSNE.pdf'), device=device)
# visualize ranked results
visualize_ranked_results(test_loader, model, target_dataset.query, target_dataset.gallery, device,
visualize_dir=logger.visualize_directory, width=args.width, height=args.height,
rerank=args.rerank)
return
if args.phase == 'test':
print("Test on source domain:")
validate(val_loader, model, source_dataset.query, source_dataset.gallery, device, cmc_flag=True,
rerank=args.rerank)
print("Test on target domain:")
validate(test_loader, model, target_dataset.query, target_dataset.gallery, device, cmc_flag=True,
rerank=args.rerank)
return
# define loss function
criterion_ce = CrossEntropyLossWithLabelSmooth(num_classes).to(device)
criterion_triplet = SoftTripletLoss(margin=args.margin).to(device)
# start training
best_val_mAP = 0.
best_test_mAP = 0.
for epoch in range(args.epochs):
# print learning rate
print(lr_scheduler.get_lr())
# train for one epoch
train(train_iter, model, criterion_ce, criterion_triplet, optimizer, epoch, args)
# update learning rate
lr_scheduler.step()
if (epoch + 1) % args.eval_step == 0 or (epoch == args.epochs - 1):
# evaluate on validation set
print("Validation on source domain...")
_, val_mAP = validate(val_loader, model, source_dataset.query, source_dataset.gallery, device,
cmc_flag=True)
# remember best mAP and save checkpoint
torch.save(model.state_dict(), logger.get_checkpoint_path('latest'))
if val_mAP > best_val_mAP:
shutil.copy(logger.get_checkpoint_path('latest'), logger.get_checkpoint_path('best'))
best_val_mAP = max(val_mAP, best_val_mAP)
# evaluate on test set
print("Test on target domain...")
_, test_mAP = validate(test_loader, model, target_dataset.query, target_dataset.gallery, device,
cmc_flag=True, rerank=args.rerank)
best_test_mAP = max(test_mAP, best_test_mAP)
# evaluate on test set
model.load_state_dict(torch.load(logger.get_checkpoint_path('best')))
print("Test on target domain:")
_, test_mAP = validate(test_loader, model, target_dataset.query, target_dataset.gallery, device,
cmc_flag=True, rerank=args.rerank)
print("test mAP on target = {}".format(test_mAP))
print("oracle mAP on target = {}".format(best_test_mAP))
logger.close()
def train(train_iter: ForeverDataIterator, model, criterion_ce: CrossEntropyLossWithLabelSmooth,
criterion_triplet: SoftTripletLoss, optimizer: Adam, epoch: int, args: argparse.Namespace):
batch_time = AverageMeter('Time', ':4.2f')
data_time = AverageMeter('Data', ':3.1f')
losses_ce = AverageMeter('CeLoss', ':3.2f')
losses_triplet = AverageMeter('TripletLoss', ':3.2f')
losses = AverageMeter('Loss', ':3.2f')
cls_accs = AverageMeter('Cls Acc', ':3.1f')
progress = ProgressMeter(
args.iters_per_epoch,
[batch_time, data_time, losses_ce, losses_triplet, losses, cls_accs],
prefix="Epoch: [{}]".format(epoch))
# switch to train mode
model.train()
end = time.time()
for i in range(args.iters_per_epoch):
x, _, labels, _ = next(train_iter)
x = x.to(device)
labels = labels.to(device)
# measure data loading time
data_time.update(time.time() - end)
# compute output
y, f = model(x)
# cross entropy loss
loss_ce = criterion_ce(y, labels)
# triplet loss
loss_triplet = criterion_triplet(f, f, labels)
loss = loss_ce + loss_triplet * args.trade_off
cls_acc = accuracy(y, labels)[0]
losses_ce.update(loss_ce.item(), x.size(0))
losses_triplet.update(loss_triplet.item(), x.size(0))
losses.update(loss.item(), x.size(0))
cls_accs.update(cls_acc.item(), x.size(0))
# compute gradient and do SGD step
optimizer.zero_grad()
loss.backward()
optimizer.step()
# measure elapsed time
batch_time.update(time.time() - end)
end = time.time()
if i % args.print_freq == 0:
progress.display(i)
if __name__ == '__main__':
dataset_names = sorted(
name for name in datasets.__dict__
if not name.startswith("__") and callable(datasets.__dict__[name])
)
parser = argparse.ArgumentParser(description="Baseline for Domain Generalizable ReID")
# dataset parameters
parser.add_argument('root', metavar='DIR',
help='root path of dataset')
parser.add_argument('-s', '--source', type=str, help='source domain')
parser.add_argument('-t', '--target', type=str, help='target domain')
parser.add_argument('--train-resizing', type=str, default='default')
# model parameters
parser.add_argument('-a', '--arch', metavar='ARCH', default='reid_resnet50',
choices=utils.get_model_names(),
help='backbone architecture: ' +
' | '.join(utils.get_model_names()) +
' (default: reid_resnet50)')
parser.add_argument('--no-pool', action='store_true', help='no pool layer after the feature extractor.')
parser.add_argument('--finetune', action='store_true', help='whether use 10x smaller lr for backbone')
parser.add_argument('--rate', type=float, default=0.2)
# training parameters
parser.add_argument('--trade-off', type=float, default=1,
help='trade-off hyper parameter between cross entropy loss and triplet loss')
parser.add_argument('--margin', type=float, default=0.0, help='margin for the triplet loss with batch hard')
parser.add_argument('-j', '--workers', type=int, default=4)
parser.add_argument('-b', '--batch-size', type=int, default=16)
parser.add_argument('--height', type=int, default=256, help="input height")
parser.add_argument('--width', type=int, default=128, help="input width")
parser.add_argument('--num-instances', type=int, default=4,
help="each minibatch consist of "
"(batch_size // num_instances) identities, and "
"each identity has num_instances instances, "
"default: 4")
parser.add_argument('--lr', type=float, default=0.00035,
help="initial learning rate")
parser.add_argument('--weight-decay', type=float, default=5e-4)
parser.add_argument('--epochs', type=int, default=80)
parser.add_argument('--warmup-steps', type=int, default=10, help='number of warp-up steps')
parser.add_argument('--milestones', nargs='+', type=int, default=[40, 70],
help='milestones for the learning rate decay')
parser.add_argument('--eval-step', type=int, default=40)
parser.add_argument('--iters-per-epoch', type=int, default=400)
parser.add_argument('--print-freq', type=int, default=40)
parser.add_argument('--seed', default=None, type=int, help='seed for initializing training.')
parser.add_argument('--rerank', action='store_true', help="evaluation only")
parser.add_argument("--log", type=str, default='baseline',
help="Where to save logs, checkpoints and debugging images.")
parser.add_argument("--phase", type=str, default='train', choices=['train', 'test', 'analysis'],
help="When phase is 'test', only test the model."
"When phase is 'analysis', only analysis the model.")
args = parser.parse_args()
main(args)
================================================
FILE: domain_generalization/Transfer-Learning-Library/examples/domain_generalization/re_identification/baseline.sh
================================================
#!/usr/bin/env bash
# Market1501 -> Duke
CUDA_VISIBLE_DEVICES=0 python baseline.py data -s Market1501 -t DukeMTMC -a reid_resnet50 \
--finetune --seed 0 --log logs/baseline/Market2Duke
# Duke -> Market1501
CUDA_VISIBLE_DEVICES=0 python baseline.py data -s DukeMTMC -t Market1501 -a reid_resnet50 \
--finetune --seed 0 --log logs/baseline/Duke2Market
# Market1501 -> MSMT
CUDA_VISIBLE_DEVICES=0 python baseline.py data -s Market1501 -t MSMT17 -a reid_resnet50 \
--finetune --seed 0 --log logs/baseline/Market2MSMT
# MSMT -> Market1501
CUDA_VISIBLE_DEVICES=0 python baseline.py data -s MSMT17 -t Market1501 -a reid_resnet50 \
--finetune --seed 0 --log logs/baseline/MSMT2Market
# Duke -> MSMT
CUDA_VISIBLE_DEVICES=0 python baseline.py data -s DukeMTMC -t MSMT17 -a reid_resnet50 \
--finetune --seed 0 --log logs/baseline/Duke2MSMT
# MSMT -> Duke
CUDA_VISIBLE_DEVICES=0 python baseline.py data -s MSMT17 -t DukeMTMC -a reid_resnet50 \
--finetune --seed 0 --log logs/baseline/MSMT2Duke
================================================
FILE: domain_generalization/Transfer-Learning-Library/examples/domain_generalization/re_identification/ibn.sh
================================================
#!/usr/bin/env bash
# Market1501 -> Duke
CUDA_VISIBLE_DEVICES=0 python baseline.py data -s Market1501 -t DukeMTMC -a resnet50_ibn_a \
--finetune --seed 0 --log logs/ibn/Market2Duke
CUDA_VISIBLE_DEVICES=0 python baseline.py data -s Market1501 -t DukeMTMC -a resnet50_ibn_b \
--finetune --seed 0 --log logs/ibn/Market2Duke
# Duke -> Market1501
CUDA_VISIBLE_DEVICES=0 python baseline.py data -s DukeMTMC -t Market1501 -a resnet50_ibn_a \
--finetune --seed 0 --log logs/ibn/Duke2Market
CUDA_VISIBLE_DEVICES=0 python baseline.py data -s DukeMTMC -t Market1501 -a resnet50_ibn_b \
--finetune --seed 0 --log logs/ibn/Duke2Market
# Market1501 -> MSMT
CUDA_VISIBLE_DEVICES=0 python baseline.py data -s Market1501 -t MSMT17 -a resnet50_ibn_a \
--finetune --seed 0 --log logs/ibn/Market2MSMT
CUDA_VISIBLE_DEVICES=0 python baseline.py data -s Market1501 -t MSMT17 -a resnet50_ibn_b \
--finetune --seed 0 --log logs/ibn/Market2MSMT
# MSMT -> Market1501
CUDA_VISIBLE_DEVICES=0 python baseline.py data -s MSMT17 -t Market1501 -a resnet50_ibn_a \
--finetune --seed 0 --log logs/ibn/MSMT2Market
CUDA_VISIBLE_DEVICES=0 python baseline.py data -s MSMT17 -t Market1501 -a resnet50_ibn_b \
--finetune --seed 0 --log logs/ibn/MSMT2Market
# Duke -> MSMT
CUDA_VISIBLE_DEVICES=0 python baseline.py data -s DukeMTMC -t MSMT17 -a resnet50_ibn_a \
--finetune --seed 0 --log logs/ibn/Duke2MSMT
CUDA_VISIBLE_DEVICES=0 python baseline.py data -s DukeMTMC -t MSMT17 -a resnet50_ibn_b \
--finetune --seed 0 --log logs/ibn/Duke2MSMT
# MSMT -> Duke
CUDA_VISIBLE_DEVICES=0 python baseline.py data -s MSMT17 -t DukeMTMC -a resnet50_ibn_a \
--finetune --seed 0 --log logs/ibn/MSMT2Duke
CUDA_VISIBLE_DEVICES=0 python baseline.py data -s MSMT17 -t DukeMTMC -a resnet50_ibn_b \
--finetune --seed 0 --log logs/ibn/MSMT2Duke
================================================
FILE: domain_generalization/Transfer-Learning-Library/examples/domain_generalization/re_identification/mixstyle.py
================================================
"""
@author: Baixu Chen
@contact: cbx_99_hasta@outlook.com
"""
import random
import time
import warnings
import sys
import argparse
import shutil
import os.path as osp
import numpy as np
import torch
from torch.nn import DataParallel
import torch.backends.cudnn as cudnn
from torch.optim import Adam
from torch.utils.data import DataLoader
sys.path.append('../../..')
from dglib.generalization.mixstyle.sampler import RandomDomainMultiInstanceSampler
import dglib.generalization.mixstyle.models as models
from common.vision.models.reid.identifier import ReIdentifier
from common.vision.models.reid.loss import CrossEntropyLossWithLabelSmooth, SoftTripletLoss
import common.vision.datasets.reid as datasets
from common.vision.datasets.reid.convert import convert_to_pytorch_dataset
from common.vision.models.reid.resnet import ReidResNet
from common.utils.scheduler import WarmupMultiStepLR
from common.utils.metric.reid import validate, visualize_ranked_results
from common.utils.data import ForeverDataIterator
from common.utils.metric import accuracy
from common.utils.meter import AverageMeter, ProgressMeter
from common.utils.logger import CompleteLogger
sys.path.append('.')
import utils
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
def main(args: argparse.Namespace):
logger = CompleteLogger(args.log, args.phase)
print(args)
if args.seed is not None:
random.seed(args.seed)
np.random.seed(args.seed)
torch.manual_seed(args.seed)
cudnn.deterministic = True
warnings.warn('You have chosen to seed training. '
'This will turn on the CUDNN deterministic setting, '
'which can slow down your training considerably! '
'You may see unexpected behavior when restarting '
'from checkpoints.')
cudnn.benchmark = True
# Data loading code
train_transform = utils.get_train_transform(args.height, args.width, args.train_resizing,
random_horizontal_flip=True,
random_color_jitter=False,
random_gray_scale=False)
val_transform = utils.get_val_transform(args.height, args.width)
print("train_transform: ", train_transform)
print("val_transform: ", val_transform)
working_dir = osp.dirname(osp.abspath(__file__))
root = osp.join(working_dir, args.root)
# source dataset
source_dataset = datasets.__dict__[args.source](root=osp.join(root, args.source.lower()))
sampler = RandomDomainMultiInstanceSampler(source_dataset.train, batch_size=args.batch_size, n_domains_per_batch=2,
num_instances=args.num_instances)
train_loader = DataLoader(
convert_to_pytorch_dataset(source_dataset.train, root=source_dataset.images_dir, transform=train_transform),
batch_size=args.batch_size, num_workers=args.workers, sampler=sampler, pin_memory=True, drop_last=True)
train_iter = ForeverDataIterator(train_loader)
val_loader = DataLoader(
convert_to_pytorch_dataset(list(set(source_dataset.query) | set(source_dataset.gallery)),
root=source_dataset.images_dir,
transform=val_transform),
batch_size=args.batch_size, num_workers=args.workers, shuffle=False, pin_memory=True)
# target dataset
target_dataset = datasets.__dict__[args.target](root=osp.join(root, args.target.lower()))
test_loader = DataLoader(
convert_to_pytorch_dataset(list(set(target_dataset.query) | set(target_dataset.gallery)),
root=target_dataset.images_dir,
transform=val_transform),
batch_size=args.batch_size, num_workers=args.workers, shuffle=False, pin_memory=True)
# create model
num_classes = source_dataset.num_train_pids
backbone = models.__dict__[args.arch](mix_layers=args.mix_layers, mix_p=args.mix_p, mix_alpha=args.mix_alpha,
resnet_class=ReidResNet, pretrained=True)
model = ReIdentifier(backbone, num_classes, finetune=args.finetune).to(device)
model = DataParallel(model)
# define optimizer and learning rate scheduler
optimizer = Adam(model.module.get_parameters(base_lr=args.lr, rate=args.rate), args.lr,
weight_decay=args.weight_decay)
lr_scheduler = WarmupMultiStepLR(optimizer, args.milestones, gamma=0.1, warmup_factor=0.1,
warmup_steps=args.warmup_steps)
# resume from the best checkpoint
if args.phase != 'train':
checkpoint = torch.load(logger.get_checkpoint_path('best'), map_location='cpu')
model.load_state_dict(checkpoint)
# analysis the model
if args.phase == 'analysis':
# plot t-SNE
utils.visualize_tsne(source_loader=val_loader, target_loader=test_loader, model=model,
filename=osp.join(logger.visualize_directory, 'analysis', 'TSNE.pdf'), device=device)
# visualize ranked results
visualize_ranked_results(test_loader, model, target_dataset.query, target_dataset.gallery, device,
visualize_dir=logger.visualize_directory, width=args.width, height=args.height,
rerank=args.rerank)
return
if args.phase == 'test':
print("Test on source domain:")
validate(val_loader, model, source_dataset.query, source_dataset.gallery, device, cmc_flag=True,
rerank=args.rerank)
print("Test on target domain:")
validate(test_loader, model, target_dataset.query, target_dataset.gallery, device, cmc_flag=True,
rerank=args.rerank)
return
# define loss function
criterion_ce = CrossEntropyLossWithLabelSmooth(num_classes).to(device)
criterion_triplet = SoftTripletLoss(margin=args.margin).to(device)
# start training
best_val_mAP = 0.
best_test_mAP = 0.
for epoch in range(args.epochs):
# print learning rate
print(lr_scheduler.get_lr())
# train for one epoch
train(train_iter, model, criterion_ce, criterion_triplet, optimizer, epoch, args)
# update learning rate
lr_scheduler.step()
if (epoch + 1) % args.eval_step == 0 or (epoch == args.epochs - 1):
# evaluate on validation set
print("Validation on source domain...")
_, val_mAP = validate(val_loader, model, source_dataset.query, source_dataset.gallery, device,
cmc_flag=True)
# remember best mAP and save checkpoint
torch.save(model.state_dict(), logger.get_checkpoint_path('latest'))
if val_mAP > best_val_mAP:
shutil.copy(logger.get_checkpoint_path('latest'), logger.get_checkpoint_path('best'))
best_val_mAP = max(val_mAP, best_val_mAP)
# evaluate on test set
print("Test on target domain...")
_, test_mAP = validate(test_loader, model, target_dataset.query, target_dataset.gallery, device,
cmc_flag=True, rerank=args.rerank)
best_test_mAP = max(test_mAP, best_test_mAP)
# evaluate on test set
model.load_state_dict(torch.load(logger.get_checkpoint_path('best')))
print("Test on target domain:")
_, test_mAP = validate(test_loader, model, target_dataset.query, target_dataset.gallery, device,
cmc_flag=True, rerank=args.rerank)
print("test mAP on target = {}".format(test_mAP))
print("oracle mAP on target = {}".format(best_test_mAP))
logger.close()
def train(train_iter: ForeverDataIterator, model, criterion_ce: CrossEntropyLossWithLabelSmooth,
criterion_triplet: SoftTripletLoss, optimizer: Adam, epoch: int, args: argparse.Namespace):
batch_time = AverageMeter('Time', ':4.2f')
data_time = AverageMeter('Data', ':3.1f')
losses_ce = AverageMeter('CeLoss', ':3.2f')
losses_triplet = AverageMeter('TripletLoss', ':3.2f')
losses = AverageMeter('Loss', ':3.2f')
cls_accs = AverageMeter('Cls Acc', ':3.1f')
progress = ProgressMeter(
args.iters_per_epoch,
[batch_time, data_time, losses_ce, losses_triplet, losses, cls_accs],
prefix="Epoch: [{}]".format(epoch))
# switch to train mode
model.train()
end = time.time()
for i in range(args.iters_per_epoch):
x, _, labels, _ = next(train_iter)
x = x.to(device)
labels = labels.to(device)
# measure data loading time
data_time.update(time.time() - end)
# compute output
y, f = model(x)
# cross entropy loss
loss_ce = criterion_ce(y, labels)
# triplet loss
loss_triplet = criterion_triplet(f, f, labels)
loss = loss_ce + loss_triplet * args.trade_off
cls_acc = accuracy(y, labels)[0]
losses_ce.update(loss_ce.item(), x.size(0))
losses_triplet.update(loss_triplet.item(), x.size(0))
losses.update(loss.item(), x.size(0))
cls_accs.update(cls_acc.item(), x.size(0))
# compute gradient and do SGD step
optimizer.zero_grad()
loss.backward()
optimizer.step()
# measure elapsed time
batch_time.update(time.time() - end)
end = time.time()
if i % args.print_freq == 0:
progress.display(i)
if __name__ == '__main__':
architecture_names = sorted(
name for name in models.__dict__
if name.islower() and not name.startswith("__")
and callable(models.__dict__[name])
)
dataset_names = sorted(
name for name in datasets.__dict__
if not name.startswith("__") and callable(datasets.__dict__[name])
)
parser = argparse.ArgumentParser(description="MixStyle for Domain Generalizable ReID")
# dataset parameters
parser.add_argument('root', metavar='DIR',
help='root path of dataset')
parser.add_argument('-s', '--source', type=str, help='source domain')
parser.add_argument('-t', '--target', type=str, help='target domain')
parser.add_argument('--train-resizing', type=str, default='default')
# model parameters
parser.add_argument('-a', '--arch', metavar='ARCH', default='resnet50',
choices=architecture_names,
help='backbone architecture: ' +
' | '.join(architecture_names) +
' (default: resnet50)')
parser.add_argument('--finetune', action='store_true', help='whether use 10x smaller lr for backbone')
parser.add_argument('--rate', type=float, default=0.2)
parser.add_argument('--mix-layers', nargs='+', help='layers to apply MixStyle')
parser.add_argument('--mix-p', default=0.5, type=float, help='probability to apply MixStyle')
parser.add_argument('--mix-alpha', default=0.1, type=float, help='parameter alpha for beta distribution')
# training parameters
parser.add_argument('--trade-off', type=float, default=1,
help='trade-off hyper parameter between cross entropy loss and triplet loss')
parser.add_argument('--margin', type=float, default=0.0, help='margin for the triplet loss with batch hard')
parser.add_argument('-j', '--workers', type=int, default=4)
parser.add_argument('-b', '--batch-size', type=int, default=16)
parser.add_argument('--height', type=int, default=256, help="input height")
parser.add_argument('--width', type=int, default=128, help="input width")
parser.add_argument('--num-instances', type=int, default=4,
help="each minibatch consist of "
"(batch_size // num_instances) identities, and "
"each identity has num_instances instances, "
"default: 4")
parser.add_argument('--lr', type=float, default=0.00035,
help="learning rate of new parameters, for pretrained ")
parser.add_argument('--weight-decay', type=float, default=5e-4)
parser.add_argument('--epochs', type=int, default=80)
parser.add_argument('--warmup-steps', type=int, default=10, help='number of warm-up steps')
parser.add_argument('--milestones', nargs='+', type=int, default=[40, 70],
help='milestones for the learning rate decay')
parser.add_argument('--eval-step', type=int, default=40)
parser.add_argument('--iters-per-epoch', type=int, default=400)
parser.add_argument('--print-freq', type=int, default=40)
parser.add_argument('--seed', default=None, type=int, help='seed for initializing training.')
parser.add_argument('--rerank', action='store_true', help="evaluation only")
parser.add_argument("--log", type=str, default='mixstyle',
help="Where to save logs, checkpoints and debugging images.")
parser.add_argument("--phase", type=str, default='train', choices=['train', 'test', 'analysis'],
help="When phase is 'test', only test the model."
"When phase is 'analysis', only analysis the model.")
args = parser.parse_args()
main(args)
================================================
FILE: domain_generalization/Transfer-Learning-Library/examples/domain_generalization/re_identification/mixstyle.sh
================================================
#!/usr/bin/env bash
# Market1501 -> Duke
CUDA_VISIBLE_DEVICES=0 python mixstyle.py data -s Market1501 -t DukeMTMC -a resnet50 \
--mix-layers layer1 layer2 --finetune --seed 0 --log logs/mixstyle/Market2Duke
# Duke -> Market1501
CUDA_VISIBLE_DEVICES=0 python mixstyle.py data -s DukeMTMC -t Market1501 -a resnet50 \
--mix-layers layer1 layer2 --finetune --seed 0 --log logs/mixstyle/Duke2Market
# Market1501 -> MSMT
CUDA_VISIBLE_DEVICES=0 python mixstyle.py data -s Market1501 -t MSMT17 -a resnet50 \
--mix-layers layer1 layer2 --finetune --seed 0 --log logs/mixstyle/Market2MSMT
# MSMT -> Market1501
CUDA_VISIBLE_DEVICES=0 python mixstyle.py data -s MSMT17 -t Market1501 -a resnet50 \
--mix-layers layer1 layer2 --finetune --seed 0 --log logs/mixstyle/MSMT2Market
# Duke -> MSMT
CUDA_VISIBLE_DEVICES=0 python mixstyle.py data -s DukeMTMC -t MSMT17 -a resnet50 \
--mix-layers layer1 layer2 --finetune --seed 0 --log logs/mixstyle/Duke2MSMT
# MSMT -> Duke
CUDA_VISIBLE_DEVICES=0 python mixstyle.py data -s MSMT17 -t DukeMTMC -a resnet50 \
--mix-layers layer1 layer2 --finetune --seed 0 --log logs/mixstyle/MSMT2Duke
================================================
FILE: domain_generalization/Transfer-Learning-Library/examples/domain_generalization/re_identification/requirements.txt
================================================
timm
opencv-python
================================================
FILE: domain_generalization/Transfer-Learning-Library/examples/domain_generalization/re_identification/utils.py
================================================
"""
@author: Baixu Chen
@contact: cbx_99_hasta@outlook.com
"""
import sys
import timm
import torch
import torch.nn as nn
import torchvision.transforms as T
sys.path.append('../../..')
from common.utils.metric.reid import extract_reid_feature
from common.utils.analysis import tsne
import common.vision.models.reid as models
def get_model_names():
return sorted(
name for name in models.__dict__
if name.islower() and not name.startswith("__")
and callable(models.__dict__[name])
) + timm.list_models()
def get_model(model_name):
if model_name in models.__dict__:
# load models from common.vision.models
backbone = models.__dict__[model_name](pretrained=True)
else:
# load models from pytorch-image-models
backbone = timm.create_model(model_name, pretrained=True)
try:
backbone.out_features = backbone.get_classifier().in_features
backbone.reset_classifier(0, '')
except:
backbone.out_features = backbone.head.in_features
backbone.head = nn.Identity()
return backbone
def get_train_transform(height, width, resizing='default', random_horizontal_flip=True, random_color_jitter=False,
random_gray_scale=False):
"""
resizing mode:
- default: resize the image to (height, width), zero-pad it by 10 on each size, the take a random crop of
(height, width)
- res: resize the image to(height, width)
"""
if resizing == 'default':
transform = T.Compose([
T.Resize((height, width), interpolation=3),
T.Pad(10),
T.RandomCrop((height, width))
])
elif resizing == 'res':
transform = T.Resize((height, width), interpolation=3)
else:
raise NotImplementedError(resizing)
transforms = [transform]
if random_horizontal_flip:
transforms.append(T.RandomHorizontalFlip())
if random_color_jitter:
transforms.append(T.ColorJitter(brightness=0.3, contrast=0.3, saturation=0.3, hue=0.3))
if random_gray_scale:
transforms.append(T.RandomGrayscale())
transforms.extend([
T.ToTensor(),
T.Normalize(mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225])
])
return T.Compose(transforms)
def get_val_transform(height, width):
return T.Compose([
T.Resize((height, width), interpolation=3),
T.ToTensor(),
T.Normalize(mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225])
])
def visualize_tsne(source_loader, target_loader, model, filename, device, n_data_points_per_domain=3000):
"""Visualize features from different domains using t-SNE. As we can have very large number of samples in each
domain, only `n_data_points_per_domain` number of samples are randomly selected in each domain.
"""
source_feature_dict = extract_reid_feature(source_loader, model, device, normalize=True)
source_feature = torch.stack(list(source_feature_dict.values())).cpu()
source_feature = source_feature[torch.randperm(len(source_feature))]
source_feature = source_feature[:n_data_points_per_domain]
target_feature_dict = extract_reid_feature(target_loader, model, device, normalize=True)
target_feature = torch.stack(list(target_feature_dict.values())).cpu()
target_feature = target_feature[torch.randperm(len(target_feature))]
target_feature = target_feature[:n_data_points_per_domain]
tsne.visualize(source_feature, target_feature, filename, source_color='cornflowerblue', target_color='darkorange')
print('T-SNE process is done, figure is saved to {}'.format(filename))
================================================
FILE: domain_generalization/Transfer-Learning-Library/examples/task_adaptation/image_classification/README.md
================================================
# Task Adaptation for Image Classification
## Installation
Example scripts support all models in [PyTorch-Image-Models](https://github.com/rwightman/pytorch-image-models).
You need to install timm to use PyTorch-Image-Models.
```
pip install timm
```
## Dataset
Following datasets can be downloaded automatically:
- [CUB200](http://www.vision.caltech.edu/visipedia/CUB-200-2011.html)
- [StanfordCars](https://ai.stanford.edu/~jkrause/cars/car_dataset.html)
- [Aircraft](https://www.robots.ox.ac.uk/~vgg/data/fgvc-aircraft/)
- [StanfordDogs](http://vision.stanford.edu/aditya86/ImageNetDogs/)
- [OxfordIIITPet](https://www.robots.ox.ac.uk/~vgg/data/pets/)
- [OxfordFlowers102](https://www.robots.ox.ac.uk/~vgg/data/flowers/102/)
- [DTD](https://www.robots.ox.ac.uk/~vgg/data/dtd/index.html)
- [PatchCamelyon](https://patchcamelyon.grand-challenge.org/)
- [EuroSAT](https://github.com/phelber/eurosat)
You need to prepare following datasets manually if you want to use them:
- [Retinopathy](https://www.kaggle.com/c/diabetic-retinopathy-detection/data)
- [Resisc45](http://www.escience.cn/people/JunweiHan/NWPU-RESISC45.html)
and prepare them following [Documentation for Retinopathy](/common/vision/datasets/retinopathy.py) and [Resisc45](/common/vision/datasets/resisc45.py).
## Supported Methods
Supported methods include:
- [Explicit inductive bias for transfer learning with convolutional networks
(L2-SP, ICML 2018)](https://arxiv.org/abs/1802.01483)
- [Catastrophic Forgetting Meets Negative Transfer: Batch Spectral Shrinkage for Safe Transfer Learning (BSS, NIPS 2019)](https://proceedings.neurips.cc/paper/2019/file/c6bff625bdb0393992c9d4db0c6bbe45-Paper.pdf)
- [DEep Learning Transfer using Fea- ture Map with Attention for convolutional networks (DELTA, ICLR 2019)](https://openreview.net/pdf?id=rkgbwsAcYm)
- [Co-Tuning for Transfer Learning (Co-Tuning, NIPS 2020)](http://ise.thss.tsinghua.edu.cn/~mlong/doc/co-tuning-for-transfer-learning-nips20.pdf)
- [Stochastic Normalization (StochNorm, NIPS 2020)](https://papers.nips.cc/paper/2020/file/bc573864331a9e42e4511de6f678aa83-Paper.pdf)
- [Learning Without Forgetting (LWF, ECCV 2016)](https://arxiv.org/abs/1606.09282)
- [Bi-tuning of Pre-trained Representations (Bi-Tuning)](https://arxiv.org/abs/2011.06182?utm_source=feedburner&utm_medium=feed&utm_campaign=Feed%3A+arxiv%2FQSXk+%28ExcitingAds%21+cs+updates+on+arXiv.org%29)
## Experiment and Results
### Fine-tune the supervised pre-trained model
The shell files give the script to reproduce the [supervised pretrained benchmarks](/docs/talib/benchmarks/image_classification.rst) with specified hyper-parameters.
For example, if you want to use vanilla fine-tune on CUB200, use the following script
```shell script
# Fine-tune ResNet50 on CUB200.
# Assume you have put the datasets under the path `data/cub200`,
# or you are glad to download the datasets automatically from the Internet to this path
CUDA_VISIBLE_DEVICES=0 python baseline.py data/cub200 -d CUB200 -sr 100 --seed 0 --finetune --log logs/baseline/cub200_100
```
### Fine-tune the unsupervised pre-trained model
Take MoCo as an example.
1. Download MoCo pretrained checkpoints from https://github.com/facebookresearch/moco
2. Convert the format of the MoCo checkpoints to the standard format of pytorch
```shell
mkdir checkpoints
python convert_moco_to_pretrained.py checkpoints/moco_v1_200ep_pretrain.pth.tar checkpoints/moco_v1_200ep_backbone.pth checkpoints/moco_v1_200ep_fc.pth
```
3. Start training
```shell
CUDA_VISIBLE_DEVICES=0 python bi_tuning.py data/cub200 -d CUB200 -sr 100 --seed 0 --lr 0.1 -i 2000 --lr-decay-epochs 3 6 9 --epochs 12 \
--log logs/moco_pretrain_bi_tuning/cub200_100 --pretrained checkpoints/moco_v1_200ep_backbone.pth
```
The shell files als give the script to reproduce the [unsupervised pretrained benchmarks](/docs/talib/benchmarks/image_classification.rst#) with specified hyper-parameters.
## Citation
If you use these methods in your research, please consider citing.
```
@inproceedings{LWF,
author = {Zhizhong Li and
Derek Hoiem},
title = {Learning without Forgetting},
booktitle={ECCV},
year = {2016},
}
@inproceedings{L2SP,
title={Explicit inductive bias for transfer learning with convolutional networks},
author={Xuhong, LI and Grandvalet, Yves and Davoine, Franck},
booktitle={ICML},
year={2018},
}
@inproceedings{BSS,
title={Catastrophic forgetting meets negative transfer: Batch spectral shrinkage for safe transfer learning},
author={Chen, Xinyang and Wang, Sinan and Fu, Bo and Long, Mingsheng and Wang, Jianmin},
booktitle={NeurIPS},
year={2019}
}
@inproceedings{DELTA,
title={Delta: Deep learning transfer using feature map with attention for convolutional networks},
author={Li, Xingjian and Xiong, Haoyi and Wang, Hanchao and Rao, Yuxuan and Liu, Liping and Chen, Zeyu and Huan, Jun},
booktitle={ICLR},
year={2019}
}
@inproceedings{StocNorm,
title={Stochastic Normalization},
author={Kou, Zhi and You, Kaichao and Long, Mingsheng and Wang, Jianmin},
booktitle={NeurIPS},
year={2020}
}
@inproceedings{CoTuning,
title={Co-Tuning for Transfer Learning},
author={You, Kaichao and Kou, Zhi and Long, Mingsheng and Wang, Jianmin},
booktitle={NeurIPS},
year={2020}
}
@article{BiTuning,
title={Bi-tuning of Pre-trained Representations},
author={Zhong, Jincheng and Wang, Ximei and Kou, Zhi and Wang, Jianmin and Long, Mingsheng},
journal={arXiv preprint arXiv:2011.06182},
year={2020}
}
```
================================================
FILE: domain_generalization/Transfer-Learning-Library/examples/task_adaptation/image_classification/baseline.py
================================================
"""
@author: Yifei Ji, Junguang Jiang
@contact: jiyf990330@163.com, JiangJunguang1123@outlook.com
"""
import random
import time
import warnings
import sys
import argparse
import shutil
import torch
import torch.nn as nn
import torch.backends.cudnn as cudnn
from torch.optim import SGD
from torch.utils.data import DataLoader
import torch.nn.functional as F
sys.path.append('../../..')
from common.modules.classifier import Classifier
from common.utils.metric import accuracy
from common.utils.meter import AverageMeter, ProgressMeter
from common.utils.data import ForeverDataIterator
from common.utils.logger import CompleteLogger
sys.path.append('.')
import utils
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
def main(args: argparse.Namespace):
logger = CompleteLogger(args.log, args.phase)
print(args)
if args.seed is not None:
random.seed(args.seed)
torch.manual_seed(args.seed)
cudnn.deterministic = True
warnings.warn('You have chosen to seed training. '
'This will turn on the CUDNN deterministic setting, '
'which can slow down your training considerably! '
'You may see unexpected behavior when restarting '
'from checkpoints.')
cudnn.benchmark = True
# Data loading code
train_transform = utils.get_train_transform(args.train_resizing, not args.no_hflip, args.color_jitter)
val_transform = utils.get_val_transform(args.val_resizing)
print("train_transform: ", train_transform)
print("val_transform: ", val_transform)
train_dataset, val_dataset, num_classes = utils.get_dataset(args.data, args.root, train_transform,
val_transform, args.sample_rate, args.num_samples_per_classes)
train_loader = DataLoader(train_dataset, batch_size=args.batch_size, shuffle=True,
num_workers=args.workers, drop_last=True)
train_iter = ForeverDataIterator(train_loader)
val_loader = DataLoader(val_dataset, batch_size=args.batch_size, shuffle=False, num_workers=args.workers)
print("training dataset size: {} test dataset size: {}".format(len(train_dataset), len(val_dataset)))
# create model
print("=> using pre-trained model '{}'".format(args.arch))
backbone = utils.get_model(args.arch, args.pretrained)
pool_layer = nn.Identity() if args.no_pool else None
classifier = Classifier(backbone, num_classes, pool_layer=pool_layer, finetune=args.finetune).to(device)
# define optimizer and lr scheduler
optimizer = SGD(classifier.get_parameters(args.lr), lr=args.lr, momentum=args.momentum, weight_decay=args.wd, nesterov=True)
lr_scheduler = torch.optim.lr_scheduler.MultiStepLR(optimizer, args.lr_decay_epochs, gamma=args.lr_gamma)
# resume from the best checkpoint
if args.phase == 'test':
checkpoint = torch.load(logger.get_checkpoint_path('best'), map_location='cpu')
classifier.load_state_dict(checkpoint)
acc1 = utils.validate(val_loader, classifier, args, device)
print(acc1)
return
# start training
best_acc1 = 0.0
for epoch in range(args.epochs):
logger.set_epoch(epoch)
print(lr_scheduler.get_lr())
# train for one epoch
train(train_iter, classifier, optimizer, epoch, args)
lr_scheduler.step()
# evaluate on validation set
acc1 = utils.validate(val_loader, classifier, args, device)
# remember best acc@1 and save checkpoint
torch.save(classifier.state_dict(), logger.get_checkpoint_path('latest'))
if acc1 > best_acc1:
shutil.copy(logger.get_checkpoint_path('latest'), logger.get_checkpoint_path('best'))
best_acc1 = max(acc1, best_acc1)
print("best_acc1 = {:3.1f}".format(best_acc1))
logger.close()
def train(train_iter: ForeverDataIterator, model: Classifier, optimizer: SGD,
epoch: int, args: argparse.Namespace):
batch_time = AverageMeter('Time', ':4.2f')
data_time = AverageMeter('Data', ':3.1f')
losses = AverageMeter('Loss', ':3.2f')
cls_accs = AverageMeter('Cls Acc', ':3.1f')
progress = ProgressMeter(
args.iters_per_epoch,
[batch_time, data_time, losses, cls_accs],
prefix="Epoch: [{}]".format(epoch))
# switch to train mode
model.train()
end = time.time()
for i in range(args.iters_per_epoch):
x, labels = next(train_iter)
x = x.to(device)
label = labels.to(device)
# measure data loading time
data_time.update(time.time() - end)
# compute output
y, f = model(x)
cls_loss = F.cross_entropy(y, label)
loss = cls_loss
# measure accuracy and record loss
losses.update(loss.item(), x.size(0))
cls_acc = accuracy(y, label)[0]
cls_accs.update(cls_acc.item(), x.size(0))
# compute gradient and do SGD step
optimizer.zero_grad()
loss.backward()
optimizer.step()
# measure elapsed time
batch_time.update(time.time() - end)
end = time.time()
if i % args.print_freq == 0:
progress.display(i)
if __name__ == '__main__':
parser = argparse.ArgumentParser(description='Baseline for Finetuning')
# dataset parameters
parser.add_argument('root', metavar='DIR',
help='root path of dataset')
parser.add_argument('-d', '--data', metavar='DATA')
parser.add_argument('-sr', '--sample-rate', default=100, type=float,
metavar='N',
help='sample rate of training dataset (default: 100)')
parser.add_argument('-sc', '--num-samples-per-classes', default=None, type=int,
help='number of samples per classes.')
parser.add_argument('--train-resizing', type=str, default='default')
parser.add_argument('--val-resizing', type=str, default='default')
parser.add_argument('--no-hflip', action='store_true', help='no random horizontal flipping during training')
parser.add_argument('--color-jitter', action='store_true')
# model parameters
parser.add_argument('-a', '--arch', metavar='ARCH', default='resnet50',
choices=utils.get_model_names(),
help='backbone architecture: ' +
' | '.join(utils.get_model_names()) +
' (default: resnet50)')
parser.add_argument('--no-pool', action='store_true',
help='no pool layer after the feature extractor.')
parser.add_argument('--finetune', action='store_true', help='whether use 10x smaller lr for backbone')
parser.add_argument('--pretrained', default=None,
help="pretrained checkpoint of the backbone. "
"(default: None, use the ImageNet supervised pretrained backbone)")
# training parameters
parser.add_argument('-b', '--batch-size', default=48, type=int,
metavar='N',
help='mini-batch size (default: 48)')
parser.add_argument('--optimizer', type=str, default='SGD', choices=['SGD', 'Adam'])
parser.add_argument('--lr', '--learning-rate', default=0.01, type=float,
metavar='LR', help='initial learning rate', dest='lr')
parser.add_argument('--lr-gamma', default=0.1, type=float, help='parameter for lr scheduler')
parser.add_argument('--lr-decay-epochs', type=int, default=(12, ), nargs='+', help='epochs to decay lr')
parser.add_argument('--momentum', default=0.9, type=float, metavar='M',
help='momentum')
parser.add_argument('--wd', '--weight-decay', default=0.0005, type=float,
metavar='W', help='weight decay (default: 5e-4)')
parser.add_argument('-j', '--workers', default=2, type=int, metavar='N',
help='number of data loading workers (default: 4)')
parser.add_argument('--epochs', default=20, type=int, metavar='N',
help='number of total epochs to run')
parser.add_argument('-i', '--iters-per-epoch', default=500, type=int,
help='Number of iterations per epoch')
parser.add_argument('-p', '--print-freq', default=100, type=int,
metavar='N', help='print frequency (default: 100)')
parser.add_argument('--seed', default=None, type=int,
help='seed for initializing training. ')
parser.add_argument("--log", type=str, default='baseline',
help="Where to save logs, checkpoints and debugging images.")
parser.add_argument("--phase", type=str, default='train', choices=['train', 'test'],
help="When phase is 'test', only test the model.")
args = parser.parse_args()
main(args)
================================================
FILE: domain_generalization/Transfer-Learning-Library/examples/task_adaptation/image_classification/baseline.sh
================================================
#!/usr/bin/env bash
# Supervised Pretraining
# CUB-200-2011
CUDA_VISIBLE_DEVICES=0 python baseline.py data/cub200 -d CUB200 -sr 100 --seed 0 --finetune --log logs/baseline/cub200_100
CUDA_VISIBLE_DEVICES=0 python baseline.py data/cub200 -d CUB200 -sr 50 --seed 0 --finetune --log logs/baseline/cub200_50
CUDA_VISIBLE_DEVICES=0 python baseline.py data/cub200 -d CUB200 -sr 30 --seed 0 --finetune --log logs/baseline/cub200_30
CUDA_VISIBLE_DEVICES=0 python baseline.py data/cub200 -d CUB200 -sr 15 --seed 0 --finetune --log logs/baseline/cub200_15
# Standford Cars
CUDA_VISIBLE_DEVICES=0 python baseline.py data/stanford_cars -d StanfordCars -sr 100 --seed 0 --finetune --log logs/baseline/car_100
CUDA_VISIBLE_DEVICES=0 python baseline.py data/stanford_cars -d StanfordCars -sr 50 --seed 0 --finetune --log logs/baseline/car_50
CUDA_VISIBLE_DEVICES=0 python baseline.py data/stanford_cars -d StanfordCars -sr 30 --seed 0 --finetune --log logs/baseline/car_30
CUDA_VISIBLE_DEVICES=0 python baseline.py data/stanford_cars -d StanfordCars -sr 15 --seed 0 --finetune --log logs/baseline/car_15
# Aircrafts
CUDA_VISIBLE_DEVICES=0 python baseline.py data/aircraft -d Aircraft -sr 100 --seed 0 --finetune --log logs/baseline/aircraft_100
CUDA_VISIBLE_DEVICES=0 python baseline.py data/aircraft -d Aircraft -sr 50 --seed 0 --finetune --log logs/baseline/aircraft_50
CUDA_VISIBLE_DEVICES=0 python baseline.py data/aircraft -d Aircraft -sr 30 --seed 0 --finetune --log logs/baseline/aircraft_30
CUDA_VISIBLE_DEVICES=0 python baseline.py data/aircraft -d Aircraft -sr 15 --seed 0 --finetune --log logs/baseline/aircraft_15
# Resisc45
CUDA_VISIBLE_DEVICES=0 python baseline.py data/vtab/resisc45 -d Resisc45 -sc 10 --finetune --seed 0 \
--log logs/baseline/resisc45_10
CUDA_VISIBLE_DEVICES=0 python baseline.py data/vtab/resisc45 -d Resisc45 -sc 20 --finetune --seed 0 \
--log logs/baseline/resisc45_20
CUDA_VISIBLE_DEVICES=0 python baseline.py data/vtab/resisc45 -d Resisc45 -sc 40 --finetune --seed 0 \
--log logs/baseline/resisc45_40
CUDA_VISIBLE_DEVICES=0 python baseline.py data/vtab/resisc45 -d Resisc45 -sc 80 --finetune --seed 0 \
--log logs/baseline/resisc45_80
# Patch Camelyon
CUDA_VISIBLE_DEVICES=0 python baseline.py data/vtab/patch_camelyon -d PatchCamelyon -sc 40 --finetune \
--seed 0 --log logs/baseline/patch_camelyon_40
CUDA_VISIBLE_DEVICES=0 python baseline.py data/vtab/patch_camelyon -d PatchCamelyon -sc 80 --finetune \
--seed 0 --log logs/baseline/patch_camelyon_80
CUDA_VISIBLE_DEVICES=0 python baseline.py data/vtab/patch_camelyon -d PatchCamelyon -sc 160 --finetune \
--seed 0 --log logs/baseline/patch_camelyon_160
CUDA_VISIBLE_DEVICES=0 python baseline.py data/vtab/patch_camelyon -d PatchCamelyon -sc 320 --finetune \
--seed 0 --log logs/baseline/patch_camelyon_320
# MoCo (Unsupervised Pretraining)
#CUB-200-2011
CUDA_VISIBLE_DEVICES=0 python baseline.py data/cub200 -d CUB200 -sr 100 --seed 0 --lr 0.1 --finetune -i 2000 --lr-decay-epochs 3 6 9 --epochs 12 \
--log logs/moco_pretrain_baseline/cub200_100 --pretrained checkpoints/moco_v1_200ep_backbone.pth
CUDA_VISIBLE_DEVICES=0 python baseline.py data/cub200 -d CUB200 -sr 50 --seed 0 --lr 0.1 --finetune -i 2000 --lr-decay-epochs 3 6 9 --epochs 12 \
--log logs/moco_pretrain_baseline/cub200_50 --pretrained checkpoints/moco_v1_200ep_backbone.pth
CUDA_VISIBLE_DEVICES=0 python baseline.py data/cub200 -d CUB200 -sr 30 --seed 0 --lr 0.1 --finetune -i 2000 --lr-decay-epochs 3 6 9 --epochs 12 \
--log logs/moco_pretrain_baseline/cub200_30 --pretrained checkpoints/moco_v1_200ep_backbone.pth
CUDA_VISIBLE_DEVICES=0 python baseline.py data/cub200 -d CUB200 -sr 15 --seed 0 --lr 0.1 --finetune -i 2000 --lr-decay-epochs 3 6 9 --epochs 12 \
--log logs/moco_pretrain_baseline/cub200_15 --pretrained checkpoints/moco_v1_200ep_backbone.pth
# Standford Cars
CUDA_VISIBLE_DEVICES=0 python baseline.py data/stanford_cars -d StanfordCars -sr 100 --seed 0 --lr 0.1 --finetune -i 2000 --lr-decay-epochs 3 6 9 --epochs 12 \
--log logs/moco_pretrain_baseline/cars_100 --pretrained checkpoints/moco_v1_200ep_backbone.pth
CUDA_VISIBLE_DEVICES=0 python baseline.py data/stanford_cars -d StanfordCars -sr 50 --seed 0 --lr 0.1 --finetune -i 2000 --lr-decay-epochs 3 6 9 --epochs 12 \
--log logs/moco_pretrain_baseline/cars_50 --pretrained checkpoints/moco_v1_200ep_backbone.pth
CUDA_VISIBLE_DEVICES=0 python baseline.py data/stanford_cars -d StanfordCars -sr 30 --seed 0 --lr 0.1 --finetune -i 2000 --lr-decay-epochs 3 6 9 --epochs 12 \
--log logs/moco_pretrain_baseline/cars_30 --pretrained checkpoints/moco_v1_200ep_backbone.pth
CUDA_VISIBLE_DEVICES=0 python baseline.py data/stanford_cars -d StanfordCars -sr 15 --seed 0 --lr 0.1 --finetune -i 2000 --lr-decay-epochs 3 6 9 --epochs 12 \
--log logs/moco_pretrain_baseline/cars_15 --pretrained checkpoints/moco_v1_200ep_backbone.pth
# Aircrafts
CUDA_VISIBLE_DEVICES=0 python baseline.py data/aircraft -d Aircraft -sr 100 --seed 0 --lr 0.1 --finetune -i 2000 --lr-decay-epochs 3 6 9 --epochs 12 \
--log logs/moco_pretrain_baseline/aircraft_100 --pretrained checkpoints/moco_v1_200ep_backbone.pth
CUDA_VISIBLE_DEVICES=0 python baseline.py data/aircraft -d Aircraft -sr 50 --seed 0 --lr 0.1 --finetune -i 2000 --lr-decay-epochs 3 6 9 --epochs 12 \
--log logs/moco_pretrain_baseline/aircraft_50 --pretrained checkpoints/moco_v1_200ep_backbone.pth
CUDA_VISIBLE_DEVICES=0 python baseline.py data/aircraft -d Aircraft -sr 30 --seed 0 --lr 0.1 --finetune -i 2000 --lr-decay-epochs 3 6 9 --epochs 12 \
--log logs/moco_pretrain_baseline/aircraft_30 --pretrained checkpoints/moco_v1_200ep_backbone.pth
CUDA_VISIBLE_DEVICES=0 python baseline.py data/aircraft -d Aircraft -sr 15 --seed 0 --lr 0.1 --finetune -i 2000 --lr-decay-epochs 3 6 9 --epochs 12 \
--log logs/moco_pretrain_baseline/aircraft_15 --pretrained checkpoints/moco_v1_200ep_backbone.pth
================================================
FILE: domain_generalization/Transfer-Learning-Library/examples/task_adaptation/image_classification/bi_tuning.py
================================================
"""
@author: Junguang Jiang
@contact: JiangJunguang1123@outlook.com
"""
import random
import time
import warnings
import sys
import argparse
import shutil
import torch
import torch.nn as nn
import torch.backends.cudnn as cudnn
from torch.optim import SGD
from torch.utils.data import DataLoader
sys.path.append('../../..')
from common.vision.transforms import MultipleApply
from common.utils.metric import accuracy
from common.utils.meter import AverageMeter, ProgressMeter
from common.utils.data import ForeverDataIterator
from common.utils.logger import CompleteLogger
from talib.finetune.bi_tuning import Classifier, BiTuning
sys.path.append('.')
import utils
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
def main(args: argparse.Namespace):
logger = CompleteLogger(args.log, args.phase)
print(args)
if args.seed is not None:
random.seed(args.seed)
torch.manual_seed(args.seed)
cudnn.deterministic = True
warnings.warn('You have chosen to seed training. '
'This will turn on the CUDNN deterministic setting, '
'which can slow down your training considerably! '
'You may see unexpected behavior when restarting '
'from checkpoints.')
cudnn.benchmark = True
# Data loading code
train_augmentation = utils.get_train_transform(args.train_resizing, not args.no_hflip, args.color_jitter)
val_transform = utils.get_val_transform(args.val_resizing)
train_transform = MultipleApply([train_augmentation, train_augmentation])
print("train_transform: ", train_transform)
print("val_transform: ", val_transform)
train_dataset, val_dataset, num_classes = utils.get_dataset(args.data, args.root, train_transform,
val_transform, args.sample_rate,
args.num_samples_per_classes)
train_loader = DataLoader(train_dataset, batch_size=args.batch_size, shuffle=True,
num_workers=args.workers, drop_last=True)
train_iter = ForeverDataIterator(train_loader)
val_loader = DataLoader(val_dataset, batch_size=args.batch_size, shuffle=False, num_workers=args.workers)
print("training dataset size: {} test dataset size: {}".format(len(train_dataset), len(val_dataset)))
# create model
print("=> using pre-trained model '{}'".format(args.arch))
backbone_q = utils.get_model(args.arch, args.pretrained)
pool_layer = nn.Identity() if args.no_pool else None
classifier_q = Classifier(backbone_q, num_classes, pool_layer=pool_layer, projection_dim=args.projection_dim,
finetune=args.finetune)
if args.pretrained_fc:
print("=> loading pre-trained fc from '{}'".format(args.pretrained_fc))
pretrained_fc_dict = torch.load(args.pretrained_fc)
classifier_q.projector.load_state_dict(pretrained_fc_dict, strict=False)
classifier_q = classifier_q.to(device)
backbone_k = utils.get_model(args.arch)
classifier_k = Classifier(backbone_k, num_classes, pool_layer=pool_layer).to(device)
bituning = BiTuning(classifier_q, classifier_k, num_classes, K=args.K, m=args.m, T=args.T)
# define optimizer and lr scheduler
optimizer = SGD(classifier_q.get_parameters(args.lr), lr=args.lr, momentum=args.momentum, weight_decay=args.wd,
nesterov=True)
lr_scheduler = torch.optim.lr_scheduler.MultiStepLR(optimizer, args.lr_decay_epochs, gamma=args.lr_gamma)
# resume from the best checkpoint
if args.phase == 'test':
checkpoint = torch.load(logger.get_checkpoint_path('best'), map_location='cpu')
classifier_q.load_state_dict(checkpoint)
acc1 = utils.validate(val_loader, classifier_q, args, device)
print(acc1)
return
# start training
best_acc1 = 0.0
for epoch in range(args.epochs):
print(lr_scheduler.get_lr())
# train for one epoch
train(train_iter, bituning, optimizer, epoch, args)
lr_scheduler.step()
# evaluate on validation set
acc1 = utils.validate(val_loader, classifier_q, args, device)
# remember best acc@1 and save checkpoint
torch.save(classifier_q.state_dict(), logger.get_checkpoint_path('latest'))
if acc1 > best_acc1:
shutil.copy(logger.get_checkpoint_path('latest'), logger.get_checkpoint_path('best'))
best_acc1 = max(acc1, best_acc1)
print("best_acc1 = {:3.1f}".format(best_acc1))
logger.close()
def train(train_iter: ForeverDataIterator, bituning: BiTuning, optimizer: SGD, epoch: int, args: argparse.Namespace):
batch_time = AverageMeter('Time', ':4.2f')
data_time = AverageMeter('Data', ':3.1f')
cls_losses = AverageMeter('Cls Loss', ':3.2f')
contrastive_losses = AverageMeter('Contrastive Loss', ':3.2f')
losses = AverageMeter('Loss', ':3.2f')
cls_accs = AverageMeter('Cls Acc', ':3.1f')
progress = ProgressMeter(
args.iters_per_epoch,
[batch_time, data_time, losses, cls_losses, contrastive_losses, cls_accs],
prefix="Epoch: [{}]".format(epoch))
classifier_criterion = torch.nn.CrossEntropyLoss().to(device)
contrastive_criterion = torch.nn.KLDivLoss(reduction='batchmean').to(device)
# switch to train mode
bituning.train()
end = time.time()
for i in range(args.iters_per_epoch):
x, labels = next(train_iter)
img_q, img_k = x[0], x[1]
img_q = img_q.to(device)
img_k = img_k.to(device)
labels = labels.to(device)
# measure data loading time
data_time.update(time.time() - end)
# compute output
y, logits_z, logits_y, bituning_labels = bituning(img_q, img_k, labels)
cls_loss = classifier_criterion(y, labels)
contrastive_loss_z = contrastive_criterion(logits_z, bituning_labels)
contrastive_loss_y = contrastive_criterion(logits_y, bituning_labels)
contrastive_loss = (contrastive_loss_z + contrastive_loss_y)
loss = cls_loss + contrastive_loss * args.trade_off
# measure accuracy and record loss
losses.update(loss.item(), x[0].size(0))
cls_losses.update(cls_loss.item(), x[0].size(0))
contrastive_losses.update(contrastive_loss.item(), x[0].size(0))
cls_acc = accuracy(y, labels)[0]
cls_accs.update(cls_acc.item(), x[0].size(0))
# compute gradient and do SGD step
optimizer.zero_grad()
loss.backward()
optimizer.step()
# measure elapsed time
batch_time.update(time.time() - end)
end = time.time()
if i % args.print_freq == 0:
progress.display(i)
if __name__ == '__main__':
parser = argparse.ArgumentParser(description='Bi-tuning for Finetuning')
# dataset parameters
parser.add_argument('root', metavar='DIR',
help='root path of dataset')
parser.add_argument('-d', '--data', metavar='DATA')
parser.add_argument('-sr', '--sample-rate', default=100, type=int,
metavar='N',
help='sample rate of training dataset (default: 100)')
parser.add_argument('-sc', '--num-samples-per-classes', default=None, type=int,
help='number of samples per classes.')
parser.add_argument('--train-resizing', type=str, default='default', help='resize mode during training')
parser.add_argument('--val-resizing', type=str, default='default', help='resize mode during validation')
parser.add_argument('--no-hflip', action='store_true', help='no random horizontal flipping during training')
parser.add_argument('--color-jitter', action='store_true', help='no color jitter during training')
# model parameters
parser.add_argument('-a', '--arch', metavar='ARCH', default='resnet50',
choices=utils.get_model_names(),
help='backbone architecture: ' +
' | '.join(utils.get_model_names()) +
' (default: resnet50)')
parser.add_argument('--no-pool', action='store_true',
help='no pool layer after the feature extractor. Used in models such as ViT.')
parser.add_argument('--finetune', action='store_true', help='whether use 10x smaller lr for backbone')
parser.add_argument('--pretrained', default=None,
help="pretrained checkpoint of the backbone. "
"(default: None, use the ImageNet supervised pretrained backbone)")
parser.add_argument('--pretrained-fc', default=None,
help="pretrained checkpoint of the fc. "
"(default: None)")
parser.add_argument('--T', default=0.07, type=float, help="temperature. (default: 0.07)")
parser.add_argument('--K', type=int, default=40, help="queue size. (default: 40)")
parser.add_argument('--m', type=float, default=0.999, help="momentum coefficient. (default: 0.999)")
parser.add_argument('--projection-dim', type=int, default=128,
help="dimension of the projection head. (default: 128)")
parser.add_argument('--trade-off', type=float, default=1.0, help="trade-off parameters. (default: 1.0)")
# training parameters
parser.add_argument('-b', '--batch-size', default=48, type=int,
metavar='N',
help='mini-batch size (default: 48)')
parser.add_argument('--optimizer', type=str, default='SGD', choices=['SGD', 'Adam'])
parser.add_argument('--lr', '--learning-rate', default=0.01, type=float,
metavar='LR', help='initial learning rate', dest='lr')
parser.add_argument('--lr-gamma', default=0.1, type=float, help='parameter for lr scheduler')
parser.add_argument('--lr-decay-epochs', type=int, default=(12,), nargs='+', help='epochs to decay lr')
parser.add_argument('--momentum', default=0.9, type=float, metavar='M',
help='momentum')
parser.add_argument('--wd', '--weight-decay', default=0.0005, type=float,
metavar='W', help='weight decay (default: 5e-4)')
parser.add_argument('-j', '--workers', default=2, type=int, metavar='N',
help='number of data loading workers (default: 4)')
parser.add_argument('--epochs', default=20, type=int, metavar='N',
help='number of total epochs to run')
parser.add_argument('-i', '--iters-per-epoch', default=500, type=int,
help='Number of iterations per epoch')
parser.add_argument('-p', '--print-freq', default=100, type=int,
metavar='N', help='print frequency (default: 100)')
parser.add_argument('--seed', default=None, type=int,
help='seed for initializing training. ')
parser.add_argument("--log", type=str, default='bi_tuning',
help="Where to save logs, checkpoints and debugging images.")
parser.add_argument("--phase", type=str, default='train', choices=['train', 'test'],
help="When phase is 'test', only test the model.")
args = parser.parse_args()
main(args)
================================================
FILE: domain_generalization/Transfer-Learning-Library/examples/task_adaptation/image_classification/bi_tuning.sh
================================================
#!/usr/bin/env bash
# Supervised Pretraining
# CUB-200-2011
CUDA_VISIBLE_DEVICES=0 python bi_tuning.py data/cub200 -d CUB200 -sr 100 --seed 0 --log logs/bi_tuning/cub200_100
CUDA_VISIBLE_DEVICES=0 python bi_tuning.py data/cub200 -d CUB200 -sr 50 --seed 0 --log logs/bi_tuning/cub200_50
CUDA_VISIBLE_DEVICES=0 python bi_tuning.py data/cub200 -d CUB200 -sr 30 --seed 0 --log logs/bi_tuning/cub200_30
CUDA_VISIBLE_DEVICES=0 python bi_tuning.py data/cub200 -d CUB200 -sr 15 --seed 0 --log logs/bi_tuning/cub200_15
# Standford Cars
CUDA_VISIBLE_DEVICES=0 python bi_tuning.py data/stanford_cars -d StanfordCars -sr 100 --seed 0 --log logs/bi_tuning/car_100
CUDA_VISIBLE_DEVICES=0 python bi_tuning.py data/stanford_cars -d StanfordCars -sr 50 --seed 0 --log logs/bi_tuning/car_50
CUDA_VISIBLE_DEVICES=0 python bi_tuning.py data/stanford_cars -d StanfordCars -sr 30 --seed 0 --log logs/bi_tuning/car_30
CUDA_VISIBLE_DEVICES=0 python bi_tuning.py data/stanford_cars -d StanfordCars -sr 15 --seed 0 --log logs/bi_tuning/car_15
# Aircrafts
CUDA_VISIBLE_DEVICES=0 python bi_tuning.py data/aircraft -d Aircraft -sr 100 --seed 0 --log logs/bi_tuning/aircraft_100
CUDA_VISIBLE_DEVICES=0 python bi_tuning.py data/aircraft -d Aircraft -sr 50 --seed 0 --log logs/bi_tuning/aircraft_50
CUDA_VISIBLE_DEVICES=0 python bi_tuning.py data/aircraft -d Aircraft -sr 30 --seed 0 --log logs/bi_tuning/aircraft_30
CUDA_VISIBLE_DEVICES=0 python bi_tuning.py data/aircraft -d Aircraft -sr 15 --seed 0 --log logs/bi_tuning/aircraft_15
# Resisc45
CUDA_VISIBLE_DEVICES=0 python bi_tuning.py data/vtab/resisc45 -d Resisc45 -sc 10 --finetune --seed 0 \
--log logs/bi_tuning/resisc45_10
CUDA_VISIBLE_DEVICES=0 python bi_tuning.py data/vtab/resisc45 -d Resisc45 -sc 20 --finetune --seed 0 \
--log logs/bi_tuning/resisc45_20
CUDA_VISIBLE_DEVICES=0 python bi_tuning.py data/vtab/resisc45 -d Resisc45 -sc 40 --finetune --seed 0 \
--log logs/bi_tuning/resisc45_40
CUDA_VISIBLE_DEVICES=0 python bi_tuning.py data/vtab/resisc45 -d Resisc45 -sc 80 --finetune --seed 0 \
--log logs/bi_tuning/resisc45_80
# Patch Camelyon
CUDA_VISIBLE_DEVICES=0 python bi_tuning.py data/vtab/patch_camelyon -d PatchCamelyon -sc 40 --finetune \
--seed 0 --log logs/bi_tuning/patch_camelyon_40
CUDA_VISIBLE_DEVICES=0 python bi_tuning.py data/vtab/patch_camelyon -d PatchCamelyon -sc 80 --finetune \
--seed 0 --log logs/bi_tuning/patch_camelyon_80
CUDA_VISIBLE_DEVICES=0 python bi_tuning.py data/vtab/patch_camelyon -d PatchCamelyon -sc 160 --finetune \
--seed 0 --log logs/bi_tuning/patch_camelyon_160
CUDA_VISIBLE_DEVICES=0 python bi_tuning.py data/vtab/patch_camelyon -d PatchCamelyon -sc 320 --finetune \
--seed 0 --log logs/bi_tuning/patch_camelyon_320
# MoCo (Unsupervised Pretraining)
# CUB-200-2011
CUDA_VISIBLE_DEVICES=0 python bi_tuning.py data/cub200 -d CUB200 -sr 100 --seed 0 --lr 0.1 -i 2000 --lr-decay-epochs 3 6 9 --epochs 12 \
--log logs/moco_pretrain_bi_tuning/cub200_100 --pretrained checkpoints/moco_v1_200ep_backbone.pth
CUDA_VISIBLE_DEVICES=0 python bi_tuning.py data/cub200 -d CUB200 -sr 50 --seed 0 --lr 0.1 -i 2000 --lr-decay-epochs 3 6 9 --epochs 12 \
--log logs/moco_pretrain_bi_tuning/cub200_50 --pretrained checkpoints/moco_v1_200ep_backbone.pth
CUDA_VISIBLE_DEVICES=0 python bi_tuning.py data/cub200 -d CUB200 -sr 30 --seed 0 --lr 0.1 -i 2000 --lr-decay-epochs 3 6 9 --epochs 12 \
--log logs/moco_pretrain_bi_tuning/cub200_30 --pretrained checkpoints/moco_v1_200ep_backbone.pth
CUDA_VISIBLE_DEVICES=0 python bi_tuning.py data/cub200 -d CUB200 -sr 15 --seed 0 --lr 0.1 -i 2000 --lr-decay-epochs 3 6 9 --epochs 12 \
--log logs/moco_pretrain_bi_tuning/cub200_15 --pretrained checkpoints/moco_v1_200ep_backbone.pth
# Standford Cars
CUDA_VISIBLE_DEVICES=0 python bi_tuning.py data/stanford_cars -d StanfordCars -sr 100 --seed 0 --lr 0.1 -i 2000 --lr-decay-epochs 3 6 9 --epochs 12 \
--log logs/moco_pretrain_bi_tuning/cars_100 --pretrained checkpoints/moco_v1_200ep_backbone.pth
CUDA_VISIBLE_DEVICES=0 python bi_tuning.py data/stanford_cars -d StanfordCars -sr 50 --seed 0 --lr 0.1 -i 2000 --lr-decay-epochs 3 6 9 --epochs 12 \
--log logs/moco_pretrain_bi_tuning/cars_50 --pretrained checkpoints/moco_v1_200ep_backbone.pth
CUDA_VISIBLE_DEVICES=0 python bi_tuning.py data/stanford_cars -d StanfordCars -sr 30 --seed 0 --lr 0.1 -i 2000 --lr-decay-epochs 3 6 9 --epochs 12 \
--log logs/moco_pretrain_bi_tuning/cars_30 --pretrained checkpoints/moco_v1_200ep_backbone.pth
CUDA_VISIBLE_DEVICES=0 python bi_tuning.py data/stanford_cars -d StanfordCars -sr 15 --seed 0 --lr 0.1 -i 2000 --lr-decay-epochs 3 6 9 --epochs 12 \
--log logs/moco_pretrain_bi_tuning/cars_15 --pretrained checkpoints/moco_v1_200ep_backbone.pth
# Aircrafts
CUDA_VISIBLE_DEVICES=0 python bi_tuning.py data/aircraft -d Aircraft -sr 100 --seed 0 --lr 0.1 -i 2000 --lr-decay-epochs 3 6 9 --epochs 12 \
--log logs/moco_pretrain_bi_tuning/aircraft_100 --pretrained checkpoints/moco_v1_200ep_backbone.pth
CUDA_VISIBLE_DEVICES=0 python bi_tuning.py data/aircraft -d Aircraft -sr 50 --seed 0 --lr 0.1 -i 2000 --lr-decay-epochs 3 6 9 --epochs 12 \
--log logs/moco_pretrain_bi_tuning/aircraft_50 --pretrained checkpoints/moco_v1_200ep_backbone.pth
CUDA_VISIBLE_DEVICES=0 python bi_tuning.py data/aircraft -d Aircraft -sr 30 --seed 0 --lr 0.1 -i 2000 --lr-decay-epochs 3 6 9 --epochs 12 \
--log logs/moco_pretrain_bi_tuning/aircraft_30 --pretrained checkpoints/moco_v1_200ep_backbone.pth
CUDA_VISIBLE_DEVICES=0 python bi_tuning.py data/aircraft -d Aircraft -sr 15 --seed 0 --lr 0.1 -i 2000 --lr-decay-epochs 3 6 9 --epochs 12 \
--log logs/moco_pretrain_bi_tuning/aircraft_15 --pretrained checkpoints/moco_v1_200ep_backbone.pth
================================================
FILE: domain_generalization/Transfer-Learning-Library/examples/task_adaptation/image_classification/bss.py
================================================
"""
@author: Yifei Ji, Junguang Jiang
@contact: jiyf990330@163.com, JiangJunguang1123@outlook.com
"""
import random
import time
import warnings
import sys
import argparse
import shutil
import torch
import torch.nn as nn
import torch.backends.cudnn as cudnn
from torch.optim import SGD
from torch.utils.data import DataLoader
import torch.nn.functional as F
sys.path.append('../../..')
from talib.finetune.bss import BatchSpectralShrinkage
from common.modules.classifier import Classifier
from common.utils.metric import accuracy
from common.utils.meter import AverageMeter, ProgressMeter
from common.utils.data import ForeverDataIterator
from common.utils.logger import CompleteLogger
sys.path.append('.')
import utils
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
def main(args: argparse.Namespace):
logger = CompleteLogger(args.log, args.phase)
print(args)
if args.seed is not None:
random.seed(args.seed)
torch.manual_seed(args.seed)
cudnn.deterministic = True
warnings.warn('You have chosen to seed training. '
'This will turn on the CUDNN deterministic setting, '
'which can slow down your training considerably! '
'You may see unexpected behavior when restarting '
'from checkpoints.')
cudnn.benchmark = True
# Data loading code
train_transform = utils.get_train_transform(args.train_resizing, not args.no_hflip, args.color_jitter)
val_transform = utils.get_val_transform(args.val_resizing)
print("train_transform: ", train_transform)
print("val_transform: ", val_transform)
train_dataset, val_dataset, num_classes = utils.get_dataset(args.data, args.root, train_transform,
val_transform, args.sample_rate, args.num_samples_per_classes)
train_loader = DataLoader(train_dataset, batch_size=args.batch_size, shuffle=True,
num_workers=args.workers, drop_last=True)
train_iter = ForeverDataIterator(train_loader)
val_loader = DataLoader(val_dataset, batch_size=args.batch_size, shuffle=False, num_workers=args.workers)
print("training dataset size: {} test dataset size: {}".format(len(train_dataset), len(val_dataset)))
# create model
print("=> using pre-trained model '{}'".format(args.arch))
backbone = utils.get_model(args.arch, args.pretrained)
pool_layer = nn.Identity() if args.no_pool else None
classifier = Classifier(backbone, num_classes, pool_layer=pool_layer, finetune=args.finetune).to(device)
bss_module = BatchSpectralShrinkage(k=args.k)
# define optimizer and lr scheduler
optimizer = SGD(classifier.get_parameters(args.lr), momentum=args.momentum, weight_decay=args.wd, nesterov=True)
lr_scheduler = torch.optim.lr_scheduler.MultiStepLR(optimizer, args.lr_decay_epochs, gamma=args.lr_gamma)
# resume from the best checkpoint
if args.phase == 'test':
checkpoint = torch.load(logger.get_checkpoint_path('best'), map_location='cpu')
classifier.load_state_dict(checkpoint)
acc1 = utils.validate(val_loader, classifier, args, device)
print(acc1)
return
# start training
best_acc1 = 0.0
for epoch in range(args.epochs):
print(lr_scheduler.get_lr())
# train for one epoch
train(train_iter, classifier, bss_module, optimizer, epoch, args)
lr_scheduler.step()
# evaluate on validation set
acc1 = utils.validate(val_loader, classifier, args, device)
# remember best acc@1 and save checkpoint
torch.save(classifier.state_dict(), logger.get_checkpoint_path('latest'))
if acc1 > best_acc1:
shutil.copy(logger.get_checkpoint_path('latest'), logger.get_checkpoint_path('best'))
best_acc1 = max(acc1, best_acc1)
print("best_acc1 = {:3.1f}".format(best_acc1))
logger.close()
def train(train_iter: ForeverDataIterator, model: Classifier, bss_module, optimizer: SGD,
epoch: int, args: argparse.Namespace):
batch_time = AverageMeter('Time', ':4.2f')
data_time = AverageMeter('Data', ':3.1f')
losses = AverageMeter('Loss', ':3.2f')
cls_accs = AverageMeter('Cls Acc', ':3.1f')
progress = ProgressMeter(
args.iters_per_epoch,
[batch_time, data_time, losses, cls_accs],
prefix="Epoch: [{}]".format(epoch))
# switch to train mode
model.train()
end = time.time()
for i in range(args.iters_per_epoch):
x, labels = next(train_iter)
x = x.to(device)
label = labels.to(device)
# measure data loading time
data_time.update(time.time() - end)
# compute output
y, f = model(x)
cls_loss = F.cross_entropy(y, label)
bss_loss = bss_module(f)
loss = cls_loss + args.trade_off * bss_loss
cls_acc = accuracy(y, label)[0]
losses.update(loss.item(), x.size(0))
cls_accs.update(cls_acc.item(), x.size(0))
# compute gradient and do SGD step
optimizer.zero_grad()
loss.backward()
optimizer.step()
# measure elapsed time
batch_time.update(time.time() - end)
end = time.time()
if i % args.print_freq == 0:
progress.display(i)
if __name__ == '__main__':
parser = argparse.ArgumentParser(description='BSS for Finetuning')
# dataset parameters
parser.add_argument('root', metavar='DIR',
help='root path of dataset')
parser.add_argument('-d', '--data', metavar='DATA')
parser.add_argument('-sr', '--sample-rate', default=100, type=int,
metavar='N',
help='sample rate of training dataset (default: 100)')
parser.add_argument('-sc', '--num-samples-per-classes', default=None, type=int,
help='number of samples per classes.')
parser.add_argument('--train-resizing', type=str, default='default')
parser.add_argument('--val-resizing', type=str, default='default')
parser.add_argument('--no-hflip', action='store_true', help='no random horizontal flipping during training')
parser.add_argument('--color-jitter', action='store_true')
# model parameters
parser.add_argument('-a', '--arch', metavar='ARCH', default='resnet50',
choices=utils.get_model_names(),
help='backbone architecture: ' +
' | '.join(utils.get_model_names()) +
' (default: resnet50)')
parser.add_argument('--no-pool', action='store_true',
help='no pool layer after the feature extractor.')
parser.add_argument('--finetune', action='store_true', help='whether use 10x smaller lr for backbone')
parser.add_argument('--pretrained', default=None,
help="pretrained checkpoint of the backbone. "
"(default: None, use the ImageNet supervised pretrained backbone)")
parser.add_argument('-k', '--k', default=1, type=int,
metavar='N',
help='hyper-parameter for BSS loss')
parser.add_argument('--trade-off', default=0.001, type=float,
metavar='P', help='trade-off weight of BSS loss')
# training parameters
parser.add_argument('-b', '--batch-size', default=48, type=int,
metavar='N',
help='mini-batch size (default: 48)')
parser.add_argument('--lr', '--learning-rate', default=0.01, type=float,
metavar='LR', help='initial learning rate', dest='lr')
parser.add_argument('--lr-gamma', default=0.1, type=float, help='parameter for lr scheduler')
parser.add_argument('--lr-decay-epochs', type=int, default=(12, ), nargs='+', help='epochs to decay lr')
parser.add_argument('--momentum', default=0.9, type=float, metavar='M',
help='momentum')
parser.add_argument('--wd', '--weight-decay', default=0.0005, type=float,
metavar='W', help='weight decay (default: 5e-4)')
parser.add_argument('-j', '--workers', default=2, type=int, metavar='N',
help='number of data loading workers (default: 4)')
parser.add_argument('--epochs', default=20, type=int, metavar='N',
help='number of total epochs to run')
parser.add_argument('-i', '--iters-per-epoch', default=500, type=int,
help='Number of iterations per epoch')
parser.add_argument('-p', '--print-freq', default=100, type=int,
metavar='N', help='print frequency (default: 100)')
parser.add_argument('--seed', default=None, type=int,
help='seed for initializing training. ')
parser.add_argument("--log", type=str, default='bss',
help="Where to save logs, checkpoints and debugging images.")
parser.add_argument("--phase", type=str, default='train', choices=['train', 'test'],
help="When phase is 'test', only test the model.")
args = parser.parse_args()
main(args)
================================================
FILE: domain_generalization/Transfer-Learning-Library/examples/task_adaptation/image_classification/bss.sh
================================================
#!/usr/bin/env bash
# Supervised Pretraining
#CUB-200-2011
CUDA_VISIBLE_DEVICES=0 python bss.py data/cub200 -d CUB200 -sr 100 --seed 0 --finetune --log logs/bss/cub200_100
CUDA_VISIBLE_DEVICES=0 python bss.py data/cub200 -d CUB200 -sr 50 --seed 0 --finetune --log logs/bss/cub200_50
CUDA_VISIBLE_DEVICES=0 python bss.py data/cub200 -d CUB200 -sr 30 --seed 0 --finetune --log logs/bss/cub200_30
CUDA_VISIBLE_DEVICES=0 python bss.py data/cub200 -d CUB200 -sr 15 --seed 0 --finetune --log logs/bss/cub200_15
# Standford Cars
CUDA_VISIBLE_DEVICES=0 python bss.py data/stanford_cars -d StanfordCars -sr 100 --seed 0 --finetune --log logs/bss/car_100
CUDA_VISIBLE_DEVICES=0 python bss.py data/stanford_cars -d StanfordCars -sr 50 --seed 0 --finetune --log logs/bss/car_50
CUDA_VISIBLE_DEVICES=0 python bss.py data/stanford_cars -d StanfordCars -sr 30 --seed 0 --finetune --log logs/bss/car_30
CUDA_VISIBLE_DEVICES=0 python bss.py data/stanford_cars -d StanfordCars -sr 15 --seed 0 --finetune --log logs/bss/car_15
# Aircrafts
CUDA_VISIBLE_DEVICES=0 python bss.py data/aircraft -d Aircraft -sr 100 --seed 0 --finetune --log logs/bss/aircraft_100
CUDA_VISIBLE_DEVICES=0 python bss.py data/aircraft -d Aircraft -sr 50 --seed 0 --finetune --log logs/bss/aircraft_50
CUDA_VISIBLE_DEVICES=0 python bss.py data/aircraft -d Aircraft -sr 30 --seed 0 --finetune --log logs/bss/aircraft_30
CUDA_VISIBLE_DEVICES=0 python bss.py data/aircraft -d Aircraft -sr 15 --seed 0 --finetune --log logs/bss/aircraft_15
# Resisc45
CUDA_VISIBLE_DEVICES=0 python bss.py data/vtab/resisc45 -d Resisc45 -sc 10 --finetune --seed 0 \
--log logs/bss/resisc45_10
CUDA_VISIBLE_DEVICES=0 python bss.py data/vtab/resisc45 -d Resisc45 -sc 20 --finetune --seed 0 \
--log logs/bss/resisc45_20
CUDA_VISIBLE_DEVICES=0 python bss.py data/vtab/resisc45 -d Resisc45 -sc 40 --finetune --seed 0 \
--log logs/bss/resisc45_40
CUDA_VISIBLE_DEVICES=0 python bss.py data/vtab/resisc45 -d Resisc45 -sc 80 --finetune --seed 0 \
--log logs/bss/resisc45_80
# Patch Camelyon
CUDA_VISIBLE_DEVICES=0 python bss.py data/vtab/patch_camelyon -d PatchCamelyon -sc 40 --finetune \
--seed 0 --log logs/bss/patch_camelyon_40
CUDA_VISIBLE_DEVICES=0 python bss.py data/vtab/patch_camelyon -d PatchCamelyon -sc 80 --finetune \
--seed 0 --log logs/bss/patch_camelyon_80
CUDA_VISIBLE_DEVICES=0 python bss.py data/vtab/patch_camelyon -d PatchCamelyon -sc 160 --finetune \
--seed 0 --log logs/bss/patch_camelyon_160
CUDA_VISIBLE_DEVICES=0 python bss.py data/vtab/patch_camelyon -d PatchCamelyon -sc 320 --finetune \
--seed 0 --log logs/bss/patch_camelyon_320
# MoCo (Unsupervised Pretraining)
#CUB-200-2011
CUDA_VISIBLE_DEVICES=0 python bss.py data/cub200 -d CUB200 -sr 100 --seed 0 --lr 0.1 -i 2000 --lr-decay-epochs 3 6 9 --epochs 12 \
--log logs/moco_pretrain_bss/cub200_100 --pretrained checkpoints/moco_v1_200ep_backbone.pth
CUDA_VISIBLE_DEVICES=0 python bss.py data/cub200 -d CUB200 -sr 50 --seed 0 --lr 0.1 -i 2000 --lr-decay-epochs 3 6 9 --epochs 12 \
--log logs/moco_pretrain_bss/cub200_50 --pretrained checkpoints/moco_v1_200ep_backbone.pth
CUDA_VISIBLE_DEVICES=0 python bss.py data/cub200 -d CUB200 -sr 30 --seed 0 --lr 0.1 -i 2000 --lr-decay-epochs 3 6 9 --epochs 12 \
--log logs/moco_pretrain_bss/cub200_30 --pretrained checkpoints/moco_v1_200ep_backbone.pth
CUDA_VISIBLE_DEVICES=0 python bss.py data/cub200 -d CUB200 -sr 15 --seed 0 --lr 0.1 -i 2000 --lr-decay-epochs 3 6 9 --epochs 12 \
--log logs/moco_pretrain_bss/cub200_15 --pretrained checkpoints/moco_v1_200ep_backbone.pth
# Standford Cars
CUDA_VISIBLE_DEVICES=0 python bss.py data/stanford_cars -d StanfordCars -sr 100 --seed 0 --lr 0.1 -i 2000 --lr-decay-epochs 3 6 9 --epochs 12 \
--log logs/moco_pretrain_bss/cars_100 --pretrained checkpoints/moco_v1_200ep_backbone.pth
CUDA_VISIBLE_DEVICES=0 python bss.py data/stanford_cars -d StanfordCars -sr 50 --seed 0 --lr 0.1 -i 2000 --lr-decay-epochs 3 6 9 --epochs 12 \
--log logs/moco_pretrain_bss/cars_50 --pretrained checkpoints/moco_v1_200ep_backbone.pth
CUDA_VISIBLE_DEVICES=0 python bss.py data/stanford_cars -d StanfordCars -sr 30 --seed 0 --lr 0.1 -i 2000 --lr-decay-epochs 3 6 9 --epochs 12 \
--log logs/moco_pretrain_bss/cars_30 --pretrained checkpoints/moco_v1_200ep_backbone.pth
CUDA_VISIBLE_DEVICES=0 python bss.py data/stanford_cars -d StanfordCars -sr 15 --seed 0 --lr 0.1 -i 2000 --lr-decay-epochs 3 6 9 --epochs 12 \
--log logs/moco_pretrain_bss/cars_15 --pretrained checkpoints/moco_v1_200ep_backbone.pth
# Aircrafts
CUDA_VISIBLE_DEVICES=0 python bss.py data/aircraft -d Aircraft -sr 100 --seed 0 --lr 0.1 -i 2000 --lr-decay-epochs 3 6 9 --epochs 12 \
--log logs/moco_pretrain_bss/aircraft_100 --pretrained checkpoints/moco_v1_200ep_backbone.pth
CUDA_VISIBLE_DEVICES=0 python bss.py data/aircraft -d Aircraft -sr 50 --seed 0 --lr 0.1 -i 2000 --lr-decay-epochs 3 6 9 --epochs 12 \
--log logs/moco_pretrain_bss/aircraft_50 --pretrained checkpoints/moco_v1_200ep_backbone.pth
CUDA_VISIBLE_DEVICES=0 python bss.py data/aircraft -d Aircraft -sr 30 --seed 0 --lr 0.1 -i 2000 --lr-decay-epochs 3 6 9 --epochs 12 \
--log logs/moco_pretrain_bss/aircraft_30 --pretrained checkpoints/moco_v1_200ep_backbone.pth
CUDA_VISIBLE_DEVICES=0 python bss.py data/aircraft -d Aircraft -sr 15 --seed 0 --lr 0.1 -i 2000 --lr-decay-epochs 3 6 9 --epochs 12 \
--log logs/moco_pretrain_bss/aircraft_15 --pretrained checkpoints/moco_v1_200ep_backbone.pth
================================================
FILE: domain_generalization/Transfer-Learning-Library/examples/task_adaptation/image_classification/co_tuning.py
================================================
"""
@author: Yifei Ji, Junguang Jiang
@contact: jiyf990330@163.com, JiangJunguang1123@outlook.com
"""
import random
import time
import warnings
import sys
import argparse
import shutil
import os
import torch
import torch.nn as nn
import torch.backends.cudnn as cudnn
from torch.optim import SGD
from torch.utils.data import DataLoader
import torch.nn.functional as F
from torch.utils.data import Subset
sys.path.append('../../..')
from talib.finetune.co_tuning import CoTuningLoss, Relationship, Classifier
from common.utils.metric import accuracy
from common.utils.meter import AverageMeter, ProgressMeter
from common.utils.logger import CompleteLogger
from common.utils.data import ForeverDataIterator
import common.vision.datasets as datasets
sys.path.append('.')
import utils
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
def get_dataset(dataset_name, root, train_transform, val_transform, sample_rate=100, num_samples_per_classes=None):
dataset = datasets.__dict__[dataset_name]
if sample_rate < 100:
train_dataset = dataset(root=root, split='train', sample_rate=sample_rate, download=True, transform=train_transform)
determin_train_dataset = dataset(root=root, split='train', sample_rate=sample_rate, download=True, transform=val_transform)
test_dataset = dataset(root=root, split='test', sample_rate=100, download=True, transform=val_transform)
num_classes = train_dataset.num_classes
else:
train_dataset = dataset(root=root, split='train', transform=train_transform)
determin_train_dataset = dataset(root=root, split='train', transform=val_transform)
test_dataset = dataset(root=root, split='test', transform=val_transform)
num_classes = train_dataset.num_classes
if num_samples_per_classes is not None:
samples = list(range(len(train_dataset)))
random.shuffle(samples)
samples_len = min(num_samples_per_classes * num_classes, len(train_dataset))
train_dataset = Subset(train_dataset, samples[:samples_len])
determin_train_dataset = Subset(determin_train_dataset, samples[:samples_len])
return train_dataset, determin_train_dataset, test_dataset, num_classes
def main(args: argparse.Namespace):
logger = CompleteLogger(args.log, args.phase)
print(args)
if args.seed is not None:
random.seed(args.seed)
torch.manual_seed(args.seed)
cudnn.deterministic = True
warnings.warn('You have chosen to seed training. '
'This will turn on the CUDNN deterministic setting, '
'which can slow down your training considerably! '
'You may see unexpected behavior when restarting '
'from checkpoints.')
cudnn.benchmark = True
# Data loading code
train_transform = utils.get_train_transform(args.train_resizing, not args.no_hflip, args.color_jitter)
val_transform = utils.get_val_transform(args.val_resizing)
print("train_transform: ", train_transform)
print("val_transform: ", val_transform)
train_dataset, determin_train_dataset, val_dataset, num_classes = get_dataset(args.data, args.root, train_transform,
val_transform, args.sample_rate, args.num_samples_per_classes)
print("training dataset size: {} test dataset size: {}".format(len(train_dataset), len(val_dataset)))
train_loader = DataLoader(train_dataset, batch_size=args.batch_size, shuffle=True,
num_workers=args.workers, drop_last=True)
determin_train_loader = DataLoader(determin_train_dataset, batch_size=args.batch_size,
shuffle=False, num_workers=args.workers, drop_last=False)
train_iter = ForeverDataIterator(train_loader)
val_loader = DataLoader(val_dataset, batch_size=args.batch_size, shuffle=False, num_workers=args.workers)
# create model
print("=> using pre-trained model '{}'".format(args.arch))
backbone = utils.get_model(args.arch, args.pretrained)
pool_layer = nn.Identity() if args.no_pool else None
classifier = Classifier(backbone, num_classes, head_source=backbone.copy_head(), pool_layer=pool_layer, finetune=args.finetune).to(device)
# define optimizer and lr scheduler
optimizer = SGD(classifier.get_parameters(args.lr), momentum=args.momentum, weight_decay=args.wd, nesterov=True)
lr_scheduler = torch.optim.lr_scheduler.MultiStepLR(optimizer, args.lr_decay_epochs, gamma=args.lr_gamma)
# resume from the best checkpoint
if args.phase == 'test':
checkpoint = torch.load(logger.get_checkpoint_path('best'), map_location='cpu')
classifier.load_state_dict(checkpoint)
acc1 = utils.validate(val_loader, classifier, args, device)
print(acc1)
return
# build relationship between source classes and target classes
source_classifier = nn.Sequential(classifier.backbone, classifier.pool_layer, classifier.head_source)
relationship = Relationship(determin_train_loader, source_classifier, device, os.path.join(logger.root, args.relationship))
co_tuning_loss = CoTuningLoss()
# start training
best_acc1 = 0.0
for epoch in range(args.epochs):
# train for one epoch
train(train_iter, classifier, optimizer, epoch, relationship, co_tuning_loss, args)
lr_scheduler.step()
# evaluate on validation set
acc1 = utils.validate(val_loader, classifier, args, device)
# remember best acc@1 and save checkpoint
torch.save(classifier.state_dict(), logger.get_checkpoint_path('latest'))
if acc1 > best_acc1:
shutil.copy(logger.get_checkpoint_path('latest'), logger.get_checkpoint_path('best'))
best_acc1 = max(acc1, best_acc1)
print("best_acc1 = {:3.1f}".format(best_acc1))
logger.close()
def train(train_iter: ForeverDataIterator, model: Classifier, optimizer: SGD,
epoch: int, relationship, co_tuning_loss, args: argparse.Namespace):
batch_time = AverageMeter('Time', ':4.2f')
data_time = AverageMeter('Data', ':3.1f')
losses = AverageMeter('Loss', ':3.2f')
cls_accs = AverageMeter('Cls Acc', ':3.1f')
progress = ProgressMeter(
args.iters_per_epoch,
[batch_time, data_time, losses, cls_accs],
prefix="Epoch: [{}]".format(epoch))
# switch to train mode
model.train()
end = time.time()
for i in range(args.iters_per_epoch):
x, label_t = next(train_iter)
x = x.to(device)
label_s = torch.from_numpy(relationship[label_t]).cuda().float()
label_t = label_t.to(device)
# measure data loading time
data_time.update(time.time() - end)
# compute output
y_s, y_t = model(x)
tgt_loss = F.cross_entropy(y_t, label_t)
src_loss = co_tuning_loss(y_s, label_s)
loss = tgt_loss + args.trade_off * src_loss
# measure accuracy and record loss
losses.update(loss.item(), x.size(0))
cls_acc = accuracy(y_t, label_t)[0]
cls_accs.update(cls_acc.item(), x.size(0))
# compute gradient and do SGD step
optimizer.zero_grad()
loss.backward()
optimizer.step()
# measure elapsed time
batch_time.update(time.time() - end)
end = time.time()
if i % args.print_freq == 0:
progress.display(i)
if __name__ == '__main__':
parser = argparse.ArgumentParser(description='Co-Tuning for Finetuning')
# dataset parameters
parser.add_argument('root', metavar='DIR',
help='root path of dataset')
parser.add_argument('-d', '--data', metavar='DATA')
parser.add_argument('-sr', '--sample-rate', default=100, type=int,
metavar='N',
help='sample rate of training dataset (default: 100)')
parser.add_argument('-sc', '--num-samples-per-classes', default=None, type=int,
help='number of samples per classes.')
parser.add_argument('--train-resizing', type=str, default='default')
parser.add_argument('--val-resizing', type=str, default='default')
parser.add_argument('--no-hflip', action='store_true', help='no random horizontal flipping during training')
parser.add_argument('--color-jitter', action='store_true')
# model parameters
parser.add_argument('-a', '--arch', metavar='ARCH', default='resnet50',
choices=utils.get_model_names(),
help='backbone architecture: ' +
' | '.join(utils.get_model_names()) +
' (default: resnet50)')
parser.add_argument('--no-pool', action='store_true',
help='no pool layer after the feature extractor.')
parser.add_argument('--finetune', action='store_true', help='whether use 10x smaller lr for backbone')
parser.add_argument('--trade-off', default=2.3, type=float,
metavar='P', help='the trade-off hyper-parameter for co-tuning loss')
parser.add_argument("--relationship", type=str, default='relationship.npy',
help="Where to save relationship file.")
parser.add_argument('--pretrained', default=None,
help="pretrained checkpoint of the backbone. "
"(default: None, use the ImageNet supervised pretrained backbone)")
# training parameters
parser.add_argument('-b', '--batch-size', default=48, type=int,
metavar='N',
help='mini-batch size (default: 48)')
parser.add_argument('--lr', '--learning-rate', default=0.01, type=float,
metavar='LR', help='initial learning rate', dest='lr')
parser.add_argument('--lr-gamma', default=0.1, type=float, help='parameter for lr scheduler')
parser.add_argument('--lr-decay-epochs', type=int, default=(12, ), nargs='+', help='epochs to decay lr')
parser.add_argument('--momentum', default=0.9, type=float, metavar='M',
help='momentum')
parser.add_argument('--wd', '--weight-decay', default=0.0005, type=float,
metavar='W', help='weight decay (default: 5e-4)')
parser.add_argument('-j', '--workers', default=2, type=int, metavar='N',
help='number of data loading workers (default: 4)')
parser.add_argument('--epochs', default=20, type=int, metavar='N',
help='number of total epochs to run')
parser.add_argument('-i', '--iters-per-epoch', default=500, type=int,
help='Number of iterations per epoch')
parser.add_argument('-p', '--print-freq', default=100, type=int,
metavar='N', help='print frequency (default: 100)')
parser.add_argument('--seed', default=None, type=int,
help='seed for initializing training. ')
parser.add_argument("--log", type=str, default='cotuning',
help="Where to save logs, checkpoints and debugging images.")
parser.add_argument("--phase", type=str, default='train', choices=['train', 'test'],
help="When phase is 'test', only test the model.")
args = parser.parse_args()
main(args)
================================================
FILE: domain_generalization/Transfer-Learning-Library/examples/task_adaptation/image_classification/co_tuning.sh
================================================
#!/usr/bin/env bash
# Supervised Pretraining
# CUB-200-2011
CUDA_VISIBLE_DEVICES=0 python co_tuning.py data/cub200 -d CUB200 -sr 100 --seed 0 --finetune --log logs/co_tuning/cub200_100
CUDA_VISIBLE_DEVICES=0 python co_tuning.py data/cub200 -d CUB200 -sr 50 --seed 0 --finetune --log logs/co_tuning/cub200_50
CUDA_VISIBLE_DEVICES=0 python co_tuning.py data/cub200 -d CUB200 -sr 30 --seed 0 --finetune --log logs/co_tuning/cub200_30
CUDA_VISIBLE_DEVICES=0 python co_tuning.py data/cub200 -d CUB200 -sr 15 --seed 0 --finetune --log logs/co_tuning/cub200_15
# Standford Cars
CUDA_VISIBLE_DEVICES=0 python co_tuning.py data/stanford_cars -d StanfordCars -sr 100 --seed 0 --finetune --log logs/co_tuning/car_100
CUDA_VISIBLE_DEVICES=0 python co_tuning.py data/stanford_cars -d StanfordCars -sr 50 --seed 0 --finetune --log logs/co_tuning/car_50
CUDA_VISIBLE_DEVICES=0 python co_tuning.py data/stanford_cars -d StanfordCars -sr 30 --seed 0 --finetune --log logs/co_tuning/car_30
CUDA_VISIBLE_DEVICES=0 python co_tuning.py data/stanford_cars -d StanfordCars -sr 15 --seed 0 --finetune --log logs/co_tuning/car_15
# Aircrafts
CUDA_VISIBLE_DEVICES=0 python co_tuning.py data/aircraft -d Aircraft -sr 100 --seed 0 --finetune --log logs/co_tuning/aircraft_100
CUDA_VISIBLE_DEVICES=0 python co_tuning.py data/aircraft -d Aircraft -sr 50 --seed 0 --finetune --log logs/co_tuning/aircraft_50
CUDA_VISIBLE_DEVICES=0 python co_tuning.py data/aircraft -d Aircraft -sr 30 --seed 0 --finetune --log logs/co_tuning/aircraft_30
CUDA_VISIBLE_DEVICES=0 python co_tuning.py data/aircraft -d Aircraft -sr 15 --seed 0 --finetune --log logs/co_tuning/aircraft_15
# Resisc45
CUDA_VISIBLE_DEVICES=0 python co_tuning.py data/vtab/resisc45 -d Resisc45 -sc 10 --finetune --seed 0 \
--log logs/co_tuning/resisc45_10
CUDA_VISIBLE_DEVICES=0 python co_tuning.py data/vtab/resisc45 -d Resisc45 -sc 20 --finetune --seed 0 \
--log logs/co_tuning/resisc45_20
CUDA_VISIBLE_DEVICES=0 python co_tuning.py data/vtab/resisc45 -d Resisc45 -sc 40 --finetune --seed 0 \
--log logs/co_tuning/resisc45_40
CUDA_VISIBLE_DEVICES=0 python co_tuning.py data/vtab/resisc45 -d Resisc45 -sc 80 --finetune --seed 0 \
--log logs/co_tuning/resisc45_80
# Patch Camelyon
CUDA_VISIBLE_DEVICES=0 python co_tuning.py data/vtab/patch_camelyon -d PatchCamelyon -sc 40 --finetune \
--seed 0 --log logs/co_tuning/patch_camelyon_40
CUDA_VISIBLE_DEVICES=0 python co_tuning.py data/vtab/patch_camelyon -d PatchCamelyon -sc 80 --finetune \
--seed 0 --log logs/co_tuning/patch_camelyon_80
CUDA_VISIBLE_DEVICES=0 python co_tuning.py data/vtab/patch_camelyon -d PatchCamelyon -sc 160 --finetune \
--seed 0 --log logs/co_tuning/patch_camelyon_160
CUDA_VISIBLE_DEVICES=0 python co_tuning.py data/vtab/patch_camelyon -d PatchCamelyon -sc 320 --finetune \
--seed 0 --log logs/co_tuning/patch_camelyon_320
# MoCo (Unsupervised Pretraining)
#CUB-200-2011
CUDA_VISIBLE_DEVICES=0 python co_tuning.py data/cub200 -d CUB200 -sr 100 --seed 0 --lr 0.1 -i 2000 --lr-decay-epochs 3 6 9 --epochs 12 \
--log logs/moco_pretrain_co_tuning/cub200_100 --pretrained checkpoints/moco_v1_200ep_backbone.pth
CUDA_VISIBLE_DEVICES=0 python co_tuning.py data/cub200 -d CUB200 -sr 50 --seed 0 --lr 0.1 -i 2000 --lr-decay-epochs 3 6 9 --epochs 12 \
--log logs/moco_pretrain_co_tuning/cub200_50 --pretrained checkpoints/moco_v1_200ep_backbone.pth
CUDA_VISIBLE_DEVICES=0 python co_tuning.py data/cub200 -d CUB200 -sr 30 --seed 0 --lr 0.1 -i 2000 --lr-decay-epochs 3 6 9 --epochs 12 \
--log logs/moco_pretrain_co_tuning/cub200_30 --pretrained checkpoints/moco_v1_200ep_backbone.pth
CUDA_VISIBLE_DEVICES=0 python co_tuning.py data/cub200 -d CUB200 -sr 15 --seed 0 --lr 0.1 -i 2000 --lr-decay-epochs 3 6 9 --epochs 12 \
--log logs/moco_pretrain_co_tuning/cub200_15 --pretrained checkpoints/moco_v1_200ep_backbone.pth
# Standford Cars
CUDA_VISIBLE_DEVICES=0 python co_tuning.py data/stanford_cars -d StanfordCars -sr 100 --seed 0 --lr 0.1 -i 2000 --lr-decay-epochs 3 6 9 --epochs 12 \
--log logs/moco_pretrain_co_tuning/cars_100 --pretrained checkpoints/moco_v1_200ep_backbone.pth
CUDA_VISIBLE_DEVICES=0 python co_tuning.py data/stanford_cars -d StanfordCars -sr 50 --seed 0 --lr 0.1 -i 2000 --lr-decay-epochs 3 6 9 --epochs 12 \
--log logs/moco_pretrain_co_tuning/cars_50 --pretrained checkpoints/moco_v1_200ep_backbone.pth
CUDA_VISIBLE_DEVICES=0 python co_tuning.py data/stanford_cars -d StanfordCars -sr 30 --seed 0 --lr 0.1 -i 2000 --lr-decay-epochs 3 6 9 --epochs 12 \
--log logs/moco_pretrain_co_tuning/cars_30 --pretrained checkpoints/moco_v1_200ep_backbone.pth
CUDA_VISIBLE_DEVICES=0 python co_tuning.py data/stanford_cars -d StanfordCars -sr 15 --seed 0 --lr 0.1 -i 2000 --lr-decay-epochs 3 6 9 --epochs 12 \
--log logs/moco_pretrain_co_tuning/cars_15 --pretrained checkpoints/moco_v1_200ep_backbone.pth
# Aircrafts
CUDA_VISIBLE_DEVICES=0 python co_tuning.py data/aircraft -d Aircraft -sr 100 --seed 0 --lr 0.1 -i 2000 --lr-decay-epochs 3 6 9 --epochs 12 \
--log logs/moco_pretrain_co_tuning/aircraft_100 --pretrained checkpoints/moco_v1_200ep_backbone.pth
CUDA_VISIBLE_DEVICES=0 python co_tuning.py data/aircraft -d Aircraft -sr 50 --seed 0 --lr 0.1 -i 2000 --lr-decay-epochs 3 6 9 --epochs 12 \
--log logs/moco_pretrain_co_tuning/aircraft_50 --pretrained checkpoints/moco_v1_200ep_backbone.pth
CUDA_VISIBLE_DEVICES=0 python co_tuning.py data/aircraft -d Aircraft -sr 30 --seed 0 --lr 0.1 -i 2000 --lr-decay-epochs 3 6 9 --epochs 12 \
--log logs/moco_pretrain_co_tuning/aircraft_30 --pretrained checkpoints/moco_v1_200ep_backbone.pth
CUDA_VISIBLE_DEVICES=0 python co_tuning.py data/aircraft -d Aircraft -sr 15 --seed 0 --lr 0.1 -i 2000 --lr-decay-epochs 3 6 9 --epochs 12 \
--log logs/moco_pretrain_co_tuning/aircraft_15 --pretrained checkpoints/moco_v1_200ep_backbone.pth
================================================
FILE: domain_generalization/Transfer-Learning-Library/examples/task_adaptation/image_classification/convert_moco_to_pretrained.py
================================================
"""
@author: Junguang Jiang
@contact: JiangJunguang1123@outlook.com
"""
import sys
import torch
if __name__ == "__main__":
input = sys.argv[1]
obj = torch.load(input, map_location="cpu")
obj = obj["state_dict"]
newmodel = {}
fc = {}
for k, v in obj.items():
if not k.startswith("module.encoder_q."):
continue
old_k = k
k = k.replace("module.encoder_q.", "")
if k.startswith("fc"):
print(k)
fc[k] = v
else:
newmodel[k] = v
with open(sys.argv[2], "wb") as f:
torch.save(newmodel, f)
with open(sys.argv[3], "wb") as f:
torch.save(fc, f)
================================================
FILE: domain_generalization/Transfer-Learning-Library/examples/task_adaptation/image_classification/delta.py
================================================
"""
@author: Yifei Ji, Junguang Jiang
@contact: jiyf990330@163.com, JiangJunguang1123@outlook.com
"""
import math
import os
import random
import time
import warnings
import sys
import argparse
import shutil
import numpy as np
from tqdm import tqdm
import torch.backends.cudnn as cudnn
from torch.optim import SGD
from torch.utils.data import DataLoader
import torch.nn.functional as F
sys.path.append('../../..')
from talib.finetune.delta import *
from common.modules.classifier import Classifier
from common.utils.data import ForeverDataIterator
from common.utils.metric import accuracy
from common.utils.meter import AverageMeter, ProgressMeter
from common.utils.logger import CompleteLogger
sys.path.append('.')
import utils
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
def main(args: argparse.Namespace):
logger = CompleteLogger(args.log, args.phase)
print(args)
if args.seed is not None:
random.seed(args.seed)
torch.manual_seed(args.seed)
cudnn.deterministic = True
warnings.warn('You have chosen to seed training. '
'This will turn on the CUDNN deterministic setting, '
'which can slow down your training considerably! '
'You may see unexpected behavior when restarting '
'from checkpoints.')
cudnn.benchmark = True
# Data loading code
train_transform = utils.get_train_transform(args.train_resizing, not args.no_hflip, args.color_jitter)
val_transform = utils.get_val_transform(args.val_resizing)
print("train_transform: ", train_transform)
print("val_transform: ", val_transform)
train_dataset, val_dataset, num_classes = utils.get_dataset(args.data, args.root, train_transform,
val_transform, args.sample_rate, args.num_samples_per_classes)
train_loader = DataLoader(train_dataset, batch_size=args.batch_size, shuffle=True,
num_workers=args.workers, drop_last=True)
train_iter = ForeverDataIterator(train_loader)
val_loader = DataLoader(val_dataset, batch_size=args.batch_size, shuffle=False, num_workers=args.workers)
print("training dataset size: {} test dataset size: {}".format(len(train_dataset), len(val_dataset)))
# create model
print("=> using pre-trained model '{}'".format(args.arch))
backbone = utils.get_model(args.arch, args.pretrained)
backbone_source = utils.get_model(args.arch, args.pretrained)
pool_layer = nn.Identity() if args.no_pool else None
classifier = Classifier(backbone, num_classes, pool_layer=pool_layer, finetune=args.finetune).to(device)
source_classifier = Classifier(backbone_source, num_classes=backbone_source.fc.out_features,
head=backbone_source.copy_head(), pool_layer=pool_layer).to(device)
for param in source_classifier.parameters():
param.requires_grad = False
source_classifier.eval()
# define optimizer and lr scheduler
optimizer = SGD(classifier.get_parameters(args.lr), momentum=args.momentum, weight_decay=args.wd, nesterov=True)
lr_scheduler = torch.optim.lr_scheduler.MultiStepLR(optimizer, args.lr_decay_epochs, gamma=args.lr_gamma)
# resume from the best checkpoint
if args.phase == 'test':
checkpoint = torch.load(logger.get_checkpoint_path('best'), map_location='cpu')
classifier.load_state_dict(checkpoint)
acc1 = utils.validate(val_loader, classifier, args, device)
print(acc1)
return
# create intermediate layer getter
if args.arch == 'resnet50':
return_layers = ['backbone.layer1.2.conv3', 'backbone.layer2.3.conv3', 'backbone.layer3.5.conv3', 'backbone.layer4.2.conv3']
elif args.arch == 'resnet101':
return_layers = ['backbone.layer1.2.conv3', 'backbone.layer2.3.conv3', 'backbone.layer3.5.conv3', 'backbone.layer4.2.conv3']
else:
raise NotImplementedError(args.arch)
source_getter = IntermediateLayerGetter(source_classifier, return_layers=return_layers)
target_getter = IntermediateLayerGetter(classifier, return_layers=return_layers)
# get regularization
if args.regularization_type == 'l2_sp':
backbone_regularization = SPRegularization(source_classifier.backbone, classifier.backbone)
elif args.regularization_type == 'feature_map':
backbone_regularization = BehavioralRegularization()
elif args.regularization_type == 'attention_feature_map':
attention_file = os.path.join(logger.root, args.attention_file)
if not os.path.exists(attention_file):
attention = calculate_channel_attention(train_dataset, return_layers, num_classes, args)
torch.save(attention, attention_file)
else:
print("Loading channel attention from", attention_file)
attention = torch.load(attention_file)
attention = [a.to(device) for a in attention]
backbone_regularization = AttentionBehavioralRegularization(attention)
else:
raise NotImplementedError(args.regularization_type)
head_regularization = L2Regularization(nn.ModuleList([classifier.head, classifier.bottleneck]))
# start training
best_acc1 = 0.0
for epoch in range(args.epochs):
print(lr_scheduler.get_lr())
# train for one epoch
train(train_iter, classifier, backbone_regularization, head_regularization, target_getter, source_getter, optimizer, epoch, args)
lr_scheduler.step()
# evaluate on validation set
acc1 = utils.validate(val_loader, classifier, args, device)
# remember best acc@1 and save checkpoint
torch.save(classifier.state_dict(), logger.get_checkpoint_path('latest'))
if acc1 > best_acc1:
shutil.copy(logger.get_checkpoint_path('latest'), logger.get_checkpoint_path('best'))
best_acc1 = max(acc1, best_acc1)
print("best_acc1 = {:3.1f}".format(best_acc1))
logger.close()
def calculate_channel_attention(dataset, return_layers, num_classes, args):
backbone = utils.get_model(args.arch)
classifier = Classifier(backbone, num_classes).to(device)
optimizer = SGD(classifier.get_parameters(args.lr), momentum=args.momentum, weight_decay=args.wd, nesterov=True)
data_loader = DataLoader(dataset, batch_size=args.attention_batch_size, shuffle=True,
num_workers=args.workers, drop_last=False)
lr_scheduler = torch.optim.lr_scheduler.ExponentialLR(optimizer, gamma=math.exp(math.log(0.1) / args.attention_lr_decay_epochs))
criterion = nn.CrossEntropyLoss()
channel_weights = []
for layer_id, name in enumerate(return_layers):
layer = get_attribute(classifier, name)
layer_channel_weight = [0] * layer.out_channels
channel_weights.append(layer_channel_weight)
# train the classifier
classifier.train()
classifier.backbone.requires_grad = False
print("Pretrain a classifier to calculate channel attention.")
for epoch in range(args.attention_epochs):
losses = AverageMeter('Loss', ':3.2f')
cls_accs = AverageMeter('Cls Acc', ':3.1f')
progress = ProgressMeter(
len(data_loader),
[losses, cls_accs],
prefix="Epoch: [{}]".format(epoch))
for i, data in enumerate(data_loader):
inputs, labels = data
inputs = inputs.to(device)
labels = labels.to(device)
outputs, _ = classifier(inputs)
loss = criterion(outputs, labels)
optimizer.zero_grad()
loss.backward()
optimizer.step()
cls_acc = accuracy(outputs, labels)[0]
losses.update(loss.item(), inputs.size(0))
cls_accs.update(cls_acc.item(), inputs.size(0))
if i % args.print_freq == 0:
progress.display(i)
lr_scheduler.step()
# calculate the channel attention
print('Calculating channel attention.')
classifier.eval()
if args.attention_iteration_limit > 0:
total_iteration = min(len(data_loader), args.attention_iteration_limit)
else:
total_iteration = len(args.data_loader)
progress = ProgressMeter(
total_iteration,
[],
prefix="Iteration: ")
for i, data in enumerate(data_loader):
if i >= total_iteration:
break
inputs, labels = data
inputs = inputs.to(device)
labels = labels.to(device)
outputs = classifier(inputs)
loss_0 = criterion(outputs, labels)
progress.display(i)
for layer_id, name in enumerate(tqdm(return_layers)):
layer = get_attribute(classifier, name)
for j in range(layer.out_channels):
tmp = classifier.state_dict()[name + '.weight'][j,].clone()
classifier.state_dict()[name + '.weight'][j,] = 0.0
outputs = classifier(inputs)
loss_1 = criterion(outputs, labels)
difference = loss_1 - loss_0
difference = difference.detach().cpu().numpy().item()
history_value = channel_weights[layer_id][j]
channel_weights[layer_id][j] = 1.0 * (i * history_value + difference) / (i + 1)
classifier.state_dict()[name + '.weight'][j, ] = tmp
channel_attention = []
for weight in channel_weights:
weight = np.array(weight)
weight = (weight - np.mean(weight)) / np.std(weight)
weight = torch.from_numpy(weight).float().to(device)
channel_attention.append(F.softmax(weight / 5).detach())
return channel_attention
def train(train_iter: ForeverDataIterator, model: Classifier, backbone_regularization:nn.Module, head_regularization:nn.Module,
target_getter: IntermediateLayerGetter,
source_getter: IntermediateLayerGetter,
optimizer: SGD, epoch: int, args: argparse.Namespace):
batch_time = AverageMeter('Time', ':4.2f')
data_time = AverageMeter('Data', ':3.1f')
losses = AverageMeter('Loss', ':3.2f')
losses_reg_head = AverageMeter('Loss (reg, head)', ':3.2f')
losses_reg_backbone = AverageMeter('Loss (reg, backbone)', ':3.2f')
cls_accs = AverageMeter('Cls Acc', ':3.1f')
progress = ProgressMeter(
args.iters_per_epoch,
[batch_time, data_time, losses, losses_reg_head, losses_reg_backbone, cls_accs],
prefix="Epoch: [{}]".format(epoch))
# switch to train mode
model.train()
end = time.time()
for i in range(args.iters_per_epoch):
x, labels = next(train_iter)
x = x.to(device)
label = labels.to(device)
# measure data loading time
data_time.update(time.time() - end)
# compute output
intermediate_output_s, output_s = source_getter(x)
intermediate_output_t, output_t = target_getter(x)
y, f = output_t
# measure accuracy and record loss
cls_acc = accuracy(y, label)[0]
cls_loss = F.cross_entropy(y, label)
if args.regularization_type == 'feature_map':
loss_reg_backbone = backbone_regularization(intermediate_output_s, intermediate_output_t)
elif args.regularization_type == 'attention_feature_map':
loss_reg_backbone = backbone_regularization(intermediate_output_s, intermediate_output_t)
else:
loss_reg_backbone = backbone_regularization()
loss_reg_head = head_regularization()
loss = cls_loss + args.trade_off_backbone * loss_reg_backbone + args.trade_off_head * loss_reg_head
losses_reg_backbone.update(loss_reg_backbone.item() * args.trade_off_backbone, x.size(0))
losses_reg_head.update(loss_reg_head.item() * args.trade_off_head, x.size(0))
losses.update(loss.item(), x.size(0))
cls_accs.update(cls_acc.item(), x.size(0))
# compute gradient and do SGD step
optimizer.zero_grad()
loss.backward()
optimizer.step()
# measure elapsed time
batch_time.update(time.time() - end)
end = time.time()
if i % args.print_freq == 0:
progress.display(i)
if __name__ == '__main__':
parser = argparse.ArgumentParser(description='Delta for Finetuning')
# dataset parameters
parser.add_argument('root', metavar='DIR',
help='root path of dataset')
parser.add_argument('-d', '--data', metavar='DATA')
parser.add_argument('-sr', '--sample-rate', default=100, type=int,
metavar='N',
help='sample rate of training dataset (default: 100)')
parser.add_argument('-sc', '--num-samples-per-classes', default=None, type=int,
help='number of samples per classes.')
parser.add_argument('--train-resizing', type=str, default='default')
parser.add_argument('--val-resizing', type=str, default='default')
parser.add_argument('--no-hflip', action='store_true', help='no random horizontal flipping during training')
parser.add_argument('--color-jitter', action='store_true')
# model parameters
parser.add_argument('-a', '--arch', metavar='ARCH', default='resnet50',
choices=utils.get_model_names(),
help='backbone architecture: ' +
' | '.join(utils.get_model_names()) +
' (default: resnet50)')
parser.add_argument('--no-pool', action='store_true',
help='no pool layer after the feature extractor.')
parser.add_argument('--finetune', action='store_true', help='whether use 10x smaller lr for backbone')
parser.add_argument('--pretrained', default=None,
help="pretrained checkpoint of the backbone. "
"(default: None, use the ImageNet supervised pretrained backbone)")
parser.add_argument('--regularization-type', choices=['l2_sp', 'feature_map', 'attention_feature_map'],
default='attention_feature_map')
parser.add_argument('--trade-off-backbone', default=0.01, type=float,
help='trade-off for backbone regularization')
parser.add_argument('--trade-off-head', default=0.01, type=float,
help='trade-off for head regularization')
# training parameters
parser.add_argument('-b', '--batch-size', default=48, type=int,
metavar='N',
help='mini-batch size (default: 48)')
parser.add_argument('--lr', '--learning-rate', default=0.01, type=float,
metavar='LR', help='initial learning rate', dest='lr')
parser.add_argument('--lr-gamma', default=0.1, type=float, help='parameter for lr scheduler')
parser.add_argument('--lr-decay-epochs', type=int, default=(12, ), nargs='+', help='epochs to decay lr')
parser.add_argument('--momentum', default=0.9, type=float, metavar='M',
help='momentum')
parser.add_argument('--wd', '--weight-decay', default=0.0, type=float,
metavar='W', help='weight decay (default: 0.0d)')
parser.add_argument('-j', '--workers', default=2, type=int, metavar='N',
help='number of data loading workers (default: 4)')
parser.add_argument('--epochs', default=20, type=int, metavar='N',
help='number of total epochs to run')
parser.add_argument('-i', '--iters-per-epoch', default=500, type=int,
help='Number of iterations per epoch')
parser.add_argument('-p', '--print-freq', default=100, type=int,
metavar='N', help='print frequency (default: 100)')
parser.add_argument('--seed', default=None, type=int,
help='seed for initializing training. ')
parser.add_argument("--log", type=str, default='delta',
help="Where to save logs, checkpoints and debugging images.")
parser.add_argument("--phase", type=str, default='train', choices=['train', 'test'],
help="When phase is 'test', only test the model."
"When phase is 'analysis', only analysis the model.")
# parameters for calculating channel attention
parser.add_argument("--attention-file", type=str, default='channel_attention.pt',
help="Where to save and load channel attention file.")
parser.add_argument('--attention-batch-size', default=32, type=int,
metavar='N',
help='mini-batch size for calculating channel attention (default: 32)')
parser.add_argument('--attention-epochs', default=10, type=int, metavar='N',
help='number of epochs to train for training before calculating channel weight')
parser.add_argument('--attention-lr-decay-epochs', default=6, type=int, metavar='N',
help='epochs to decay lr for training before calculating channel weight')
parser.add_argument('--attention-iteration-limit', default=10, type=int, metavar='N',
help='iteration limits for calculating channel attention, -1 means no limits')
args = parser.parse_args()
main(args)
================================================
FILE: domain_generalization/Transfer-Learning-Library/examples/task_adaptation/image_classification/delta.sh
================================================
#!/usr/bin/env bash
# CUB-200-2011
CUDA_VISIBLE_DEVICES=0 python delta.py data/cub200 -d CUB200 -sr 100 --seed 0 --finetune --log logs/delta/cub200_100
CUDA_VISIBLE_DEVICES=0 python delta.py data/cub200 -d CUB200 -sr 50 --seed 0 --finetune --log logs/delta/cub200_50
CUDA_VISIBLE_DEVICES=0 python delta.py data/cub200 -d CUB200 -sr 30 --seed 0 --finetune --log logs/delta/cub200_30
CUDA_VISIBLE_DEVICES=0 python delta.py data/cub200 -d CUB200 -sr 15 --seed 0 --finetune --log logs/delta/cub200_15
# Stanford Cars
CUDA_VISIBLE_DEVICES=0 python delta.py data/stanford_cars -d StanfordCars -sr 100 --seed 0 --finetune --log logs/delta/car_100
CUDA_VISIBLE_DEVICES=0 python delta.py data/stanford_cars -d StanfordCars -sr 50 --seed 0 --finetune --log logs/delta/car_50
CUDA_VISIBLE_DEVICES=0 python delta.py data/stanford_cars -d StanfordCars -sr 30 --seed 0 --finetune --log logs/delta/car_30
CUDA_VISIBLE_DEVICES=0 python delta.py data/stanford_cars -d StanfordCars -sr 15 --seed 0 --finetune --log logs/delta/car_15
# Aircrafts
CUDA_VISIBLE_DEVICES=0 python delta.py data/aircraft -d Aircraft -sr 100 --seed 0 --finetune --log logs/delta/aircraft_100
CUDA_VISIBLE_DEVICES=0 python delta.py data/aircraft -d Aircraft -sr 50 --seed 0 --finetune --log logs/delta/aircraft_50
CUDA_VISIBLE_DEVICES=0 python delta.py data/aircraft -d Aircraft -sr 30 --seed 0 --finetune --log logs/delta/aircraft_30
CUDA_VISIBLE_DEVICES=0 python delta.py data/aircraft -d Aircraft -sr 15 --seed 0 --finetune --log logs/delta/aircraft_15
# Resisc45
CUDA_VISIBLE_DEVICES=0 python delta.py data/vtab/resisc45 -d Resisc45 -sc 10 --finetune --seed 0 \
--log logs/delta/resisc45_10
CUDA_VISIBLE_DEVICES=0 python delta.py data/vtab/resisc45 -d Resisc45 -sc 20 --finetune --seed 0 \
--log logs/delta/resisc45_20
CUDA_VISIBLE_DEVICES=0 python delta.py data/vtab/resisc45 -d Resisc45 -sc 40 --finetune --seed 0 \
--log logs/delta/resisc45_40
CUDA_VISIBLE_DEVICES=0 python delta.py data/vtab/resisc45 -d Resisc45 -sc 80 --finetune --seed 0 \
--log logs/delta/resisc45_80
# Patch Camelyon
CUDA_VISIBLE_DEVICES=0 python delta.py data/vtab/patch_camelyon -d PatchCamelyon -sc 40 --finetune \
--seed 0 --log logs/delta/patch_camelyon_40
CUDA_VISIBLE_DEVICES=0 python delta.py data/vtab/patch_camelyon -d PatchCamelyon -sc 80 --finetune \
--seed 0 --log logs/delta/patch_camelyon_80
CUDA_VISIBLE_DEVICES=0 python delta.py data/vtab/patch_camelyon -d PatchCamelyon -sc 160 --finetune \
--seed 0 --log logs/delta/patch_camelyon_160
CUDA_VISIBLE_DEVICES=0 python delta.py data/vtab/patch_camelyon -d PatchCamelyon -sc 320 --finetune \
--seed 0 --log logs/delta/patch_camelyon_320
# MoCo (Unsupervised Pretraining)
#CUB-200-2011
CUDA_VISIBLE_DEVICES=0 python delta.py data/cub200 -d CUB200 -sr 100 --seed 0 --lr 0.1 -i 2000 --lr-decay-epochs 3 6 9 --epochs 12 \
--log logs/moco_pretrain_delta/cub200_100 --pretrained checkpoints/moco_v1_200ep_pretrain.pth
CUDA_VISIBLE_DEVICES=0 python delta.py data/cub200 -d CUB200 -sr 50 --seed 0 --lr 0.1 -i 2000 --lr-decay-epochs 3 6 9 --epochs 12 \
--log logs/moco_pretrain_delta/cub200_50 --pretrained checkpoints/moco_v1_200ep_pretrain.pth
CUDA_VISIBLE_DEVICES=0 python delta.py data/cub200 -d CUB200 -sr 30 --seed 0 --lr 0.1 -i 2000 --lr-decay-epochs 3 6 9 --epochs 12 \
--log logs/moco_pretrain_delta/cub200_30 --pretrained checkpoints/moco_v1_200ep_pretrain.pth
CUDA_VISIBLE_DEVICES=0 python delta.py data/cub200 -d CUB200 -sr 15 --seed 0 --lr 0.1 -i 2000 --lr-decay-epochs 3 6 9 --epochs 12 \
--log logs/moco_pretrain_delta/cub200_15 --pretrained checkpoints/moco_v1_200ep_pretrain.pth
# Standford Cars
CUDA_VISIBLE_DEVICES=0 python delta.py data/stanford_cars -d StanfordCars -sr 100 --seed 0 --lr 0.1 -i 2000 --lr-decay-epochs 3 6 9 --epochs 12 \
--log logs/moco_pretrain_delta/cars_100 --pretrained checkpoints/moco_v1_200ep_pretrain.pth
CUDA_VISIBLE_DEVICES=0 python delta.py data/stanford_cars -d StanfordCars -sr 50 --seed 0 --lr 0.1 -i 2000 --lr-decay-epochs 3 6 9 --epochs 12 \
--log logs/moco_pretrain_delta/cars_50 --pretrained checkpoints/moco_v1_200ep_pretrain.pth
CUDA_VISIBLE_DEVICES=0 python delta.py data/stanford_cars -d StanfordCars -sr 30 --seed 0 --lr 0.1 -i 2000 --lr-decay-epochs 3 6 9 --epochs 12 \
--log logs/moco_pretrain_delta/cars_30 --pretrained checkpoints/moco_v1_200ep_pretrain.pth
CUDA_VISIBLE_DEVICES=0 python delta.py data/stanford_cars -d StanfordCars -sr 15 --seed 0 --lr 0.1 -i 2000 --lr-decay-epochs 3 6 9 --epochs 12 \
--log logs/moco_pretrain_delta/cars_15 --pretrained checkpoints/moco_v1_200ep_pretrain.pth
# Aircrafts
CUDA_VISIBLE_DEVICES=0 python delta.py data/aircraft -d Aircraft -sr 100 --seed 0 --lr 0.1 -i 2000 --lr-decay-epochs 3 6 9 --epochs 12 \
--log logs/moco_pretrain_delta/aircraft_100 --pretrained checkpoints/moco_v1_200ep_pretrain.pth
CUDA_VISIBLE_DEVICES=0 python delta.py data/aircraft -d Aircraft -sr 50 --seed 0 --lr 0.1 -i 2000 --lr-decay-epochs 3 6 9 --epochs 12 \
--log logs/moco_pretrain_delta/aircraft_50 --pretrained checkpoints/moco_v1_200ep_pretrain.pth
CUDA_VISIBLE_DEVICES=0 python delta.py data/aircraft -d Aircraft -sr 30 --seed 0 --lr 0.1 -i 2000 --lr-decay-epochs 3 6 9 --epochs 12 \
--log logs/moco_pretrain_delta/aircraft_30 --pretrained checkpoints/moco_v1_200ep_pretrain.pth
CUDA_VISIBLE_DEVICES=0 python delta.py data/aircraft -d Aircraft -sr 15 --seed 0 --lr 0.1 -i 2000 --lr-decay-epochs 3 6 9 --epochs 12 \
--log logs/moco_pretrain_delta/aircraft_15 --pretrained checkpoints/moco_v1_200ep_pretrain.pth
================================================
FILE: domain_generalization/Transfer-Learning-Library/examples/task_adaptation/image_classification/lwf.py
================================================
"""
Learning Without Forgetting in Finetune.
@author: Junguang Jiang
@contact: JiangJunguang1123@outlook.com
"""
import random
import time
import warnings
import sys
import argparse
import shutil
import torch
import torch.nn as nn
import torch.backends.cudnn as cudnn
from torch.optim import SGD
from torch.utils.data import DataLoader, TensorDataset
import torch.nn.functional as F
sys.path.append('../../..')
from talib.finetune.lwf import collect_pretrain_labels, Classifier
from common.loss import KnowledgeDistillationLoss
from common.utils.metric import accuracy
from common.utils.meter import AverageMeter, ProgressMeter
from common.utils.logger import CompleteLogger
from common.utils.data import ForeverDataIterator, CombineDataset
sys.path.append('.')
import utils
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
def main(args: argparse.Namespace):
logger = CompleteLogger(args.log, args.phase)
print(args)
if args.seed is not None:
random.seed(args.seed)
torch.manual_seed(args.seed)
cudnn.deterministic = True
warnings.warn('You have chosen to seed training. '
'This will turn on the CUDNN deterministic setting, '
'which can slow down your training considerably! '
'You may see unexpected behavior when restarting '
'from checkpoints.')
cudnn.benchmark = True
# Data loading code
train_transform = utils.get_train_transform(args.train_resizing, not args.no_hflip, args.color_jitter)
val_transform = utils.get_val_transform(args.val_resizing)
print("train_transform: ", train_transform)
print("val_transform: ", val_transform)
train_dataset, val_dataset, num_classes = utils.get_dataset(args.data, args.root, train_transform,
val_transform, args.sample_rate, args.num_samples_per_classes)
train_loader = DataLoader(train_dataset, batch_size=args.batch_size, shuffle=False,
num_workers=args.workers, drop_last=False)
val_loader = DataLoader(val_dataset, batch_size=args.batch_size, shuffle=False, num_workers=args.workers)
print("training dataset size: {} test dataset size: {}".format(len(train_dataset), len(val_dataset)))
# create model
print("=> using pre-trained model '{}'".format(args.arch))
backbone = utils.get_model(args.arch, args.pretrained)
pool_layer = nn.Identity() if args.no_pool else None
classifier = Classifier(backbone, num_classes, head_source=backbone.copy_head(), pool_layer=pool_layer,
finetune=args.finetune).to(device)
kd = KnowledgeDistillationLoss(args.T)
source_classifier = nn.Sequential(classifier.backbone, classifier.pool_layer, classifier.head_source)
pretrain_labels = collect_pretrain_labels(train_loader, source_classifier, device)
train_dataset = CombineDataset([train_dataset, TensorDataset(pretrain_labels)])
train_loader = DataLoader(train_dataset, batch_size=args.batch_size,
shuffle=True, num_workers=args.workers, drop_last=True)
train_iter = ForeverDataIterator(train_loader)
# define optimizer and lr scheduler
optimizer = SGD(classifier.get_parameters(args.lr), momentum=args.momentum, weight_decay=args.wd, nesterov=True)
lr_scheduler = torch.optim.lr_scheduler.MultiStepLR(optimizer, args.lr_decay_epochs, gamma=args.lr_gamma)
# resume from the best checkpoint
if args.phase == 'test':
checkpoint = torch.load(logger.get_checkpoint_path('best'), map_location='cpu')
classifier.load_state_dict(checkpoint)
acc1 = utils.validate(val_loader, classifier, args, device)
print(acc1)
return
# start training
best_acc1 = 0.0
for epoch in range(args.epochs):
# train for one epoch
train(train_iter, classifier, kd, optimizer, epoch, args)
lr_scheduler.step()
# evaluate on validation set
acc1 = utils.validate(val_loader, classifier, args, device)
# remember best acc@1 and save checkpoint
torch.save(classifier.state_dict(), logger.get_checkpoint_path('latest'))
if acc1 > best_acc1:
shutil.copy(logger.get_checkpoint_path('latest'), logger.get_checkpoint_path('best'))
best_acc1 = max(acc1, best_acc1)
print("best_acc1 = {:3.1f}".format(best_acc1))
logger.close()
def train(train_iter: ForeverDataIterator, model: Classifier, kd, optimizer: SGD,
epoch: int, args: argparse.Namespace):
batch_time = AverageMeter('Time', ':4.2f')
data_time = AverageMeter('Data', ':3.1f')
losses = AverageMeter('Loss', ':3.2f')
losses_kd = AverageMeter('Loss (KD)', ':5.4f')
cls_accs = AverageMeter('Cls Acc', ':3.1f')
progress = ProgressMeter(
args.iters_per_epoch,
[batch_time, data_time, losses, losses_kd, cls_accs],
prefix="Epoch: [{}]".format(epoch))
# switch to train mode
model.train()
end = time.time()
for i in range(args.iters_per_epoch):
x, label_t, label_s = next(train_iter)
x = x.to(device)
label_s = label_s.to(device)
label_t = label_t.to(device)
# measure data loading time
data_time.update(time.time() - end)
# compute output
y_s, y_t = model(x)
tgt_loss = F.cross_entropy(y_t, label_t)
src_loss = kd(y_s, label_s)
loss = tgt_loss + args.trade_off * src_loss
# measure accuracy and record loss
losses.update(tgt_loss.item(), x.size(0))
losses_kd.update(src_loss.item(), x.size(0))
cls_acc = accuracy(y_t, label_t)[0]
cls_accs.update(cls_acc.item(), x.size(0))
# compute gradient and do SGD step
optimizer.zero_grad()
loss.backward()
optimizer.step()
# measure elapsed time
batch_time.update(time.time() - end)
end = time.time()
if i % args.print_freq == 0:
progress.display(i)
if __name__ == '__main__':
parser = argparse.ArgumentParser(description='LWF (Learning without Forgetting) for Finetuning')
# dataset parameters
parser.add_argument('root', metavar='DIR',
help='root path of dataset')
parser.add_argument('-d', '--data', metavar='DATA')
parser.add_argument('-sr', '--sample-rate', default=100, type=int,
metavar='N',
help='sample rate of training dataset (default: 100)')
parser.add_argument('-sc', '--num-samples-per-classes', default=None, type=int,
help='number of samples per classes.')
parser.add_argument('--train-resizing', type=str, default='default')
parser.add_argument('--val-resizing', type=str, default='default')
parser.add_argument('--no-hflip', action='store_true', help='no random horizontal flipping during training')
parser.add_argument('--color-jitter', action='store_true')
# model parameters
parser.add_argument('-a', '--arch', metavar='ARCH', default='resnet50',
choices=utils.get_model_names(),
help='backbone architecture: ' +
' | '.join(utils.get_model_names()) +
' (default: resnet50)')
parser.add_argument('--no-pool', action='store_true',
help='no pool layer after the feature extractor.')
parser.add_argument('--finetune', action='store_true', help='whether use 10x smaller lr for backbone')
parser.add_argument('--trade-off', default=4, type=float,
metavar='P', help='weight of pretrained loss')
parser.add_argument("-T", type=float, default=3,
help="temperature for knowledge distillation")
parser.add_argument('--pretrained', default=None,
help="pretrained checkpoint of the backbone. "
"(default: None, use the ImageNet supervised pretrained backbone)")
# training parameters
parser.add_argument('-b', '--batch-size', default=48, type=int,
metavar='N',
help='mini-batch size (default: 48)')
parser.add_argument('--lr', '--learning-rate', default=0.01, type=float,
metavar='LR', help='initial learning rate', dest='lr')
parser.add_argument('--lr-gamma', default=0.1, type=float, help='parameter for lr scheduler')
parser.add_argument('--lr-decay-epochs', type=int, default=(12, ), nargs='+', help='epochs to decay lr')
parser.add_argument('--momentum', default=0.9, type=float, metavar='M',
help='momentum')
parser.add_argument('--wd', '--weight-decay', default=0.0005, type=float,
metavar='W', help='weight decay (default: 5e-4)')
parser.add_argument('-j', '--workers', default=2, type=int, metavar='N',
help='number of data loading workers (default: 4)')
parser.add_argument('--epochs', default=20, type=int, metavar='N',
help='number of total epochs to run')
parser.add_argument('-i', '--iters-per-epoch', default=500, type=int,
help='Number of iterations per epoch')
parser.add_argument('-p', '--print-freq', default=100, type=int,
metavar='N', help='print frequency (default: 100)')
parser.add_argument('--seed', default=None, type=int,
help='seed for initializing training. ')
parser.add_argument("--log", type=str, default='lwf',
help="Where to save logs, checkpoints and debugging images.")
parser.add_argument("--phase", type=str, default='train', choices=['train', 'test'],
help="When phase is 'test', only test the model.")
args = parser.parse_args()
main(args)
================================================
FILE: domain_generalization/Transfer-Learning-Library/examples/task_adaptation/image_classification/lwf.sh
================================================
#!/usr/bin/env bash
# CUB-200-2011
CUDA_VISIBLE_DEVICES=0 python lwf.py data/cub200 -d CUB200 -sr 100 --seed 0 --finetune --log logs/lwf/cub200_100 --lr 0.01
CUDA_VISIBLE_DEVICES=0 python lwf.py data/cub200 -d CUB200 -sr 50 --seed 0 --finetune --log logs/lwf/cub200_50 --lr 0.001
CUDA_VISIBLE_DEVICES=0 python lwf.py data/cub200 -d CUB200 -sr 30 --seed 0 --finetune --log logs/lwf/cub200_30 --lr 0.001
CUDA_VISIBLE_DEVICES=0 python lwf.py data/cub200 -d CUB200 -sr 15 --seed 0 --finetune --log logs/lwf/cub200_15 --lr 0.001
# Standford Cars
CUDA_VISIBLE_DEVICES=0 python lwf.py data/stanford_cars -d StanfordCars -sr 100 --seed 0 --finetune --log logs/lwf/car_100 --lr 0.01
CUDA_VISIBLE_DEVICES=0 python lwf.py data/stanford_cars -d StanfordCars -sr 50 --seed 0 --finetune --log logs/lwf/car_50 --lr 0.01
CUDA_VISIBLE_DEVICES=0 python lwf.py data/stanford_cars -d StanfordCars -sr 30 --seed 0 --finetune --log logs/lwf/car_30 --lr 0.01
CUDA_VISIBLE_DEVICES=0 python lwf.py data/stanford_cars -d StanfordCars -sr 15 --seed 0 --finetune --log logs/lwf/car_15 --lr 0.01
# Aircrafts
CUDA_VISIBLE_DEVICES=0 python lwf.py data/aircraft -d Aircraft -sr 100 --seed 0 --finetune --log logs/lwf/aircraft_100 --lr 0.001
CUDA_VISIBLE_DEVICES=0 python lwf.py data/aircraft -d Aircraft -sr 50 --seed 0 --finetune --log logs/lwf/aircraft_50 --lr 0.001
CUDA_VISIBLE_DEVICES=0 python lwf.py data/aircraft -d Aircraft -sr 30 --seed 0 --finetune --log logs/lwf/aircraft_30 --lr 0.001
CUDA_VISIBLE_DEVICES=0 python lwf.py data/aircraft -d Aircraft -sr 15 --seed 0 --finetune --log logs/lwf/aircraft_15 --lr 0.001
# Resisc45
CUDA_VISIBLE_DEVICES=0 python lwf.py data/vtab/resisc45 -d Resisc45 -sc 10 --finetune --seed 0 \
--log logs/lwf/resisc45_10
CUDA_VISIBLE_DEVICES=0 python lwf.py data/vtab/resisc45 -d Resisc45 -sc 20 --finetune --seed 0 \
--log logs/lwf/resisc45_20
CUDA_VISIBLE_DEVICES=0 python lwf.py data/vtab/resisc45 -d Resisc45 -sc 40 --finetune --seed 0 \
--log logs/lwf/resisc45_40
CUDA_VISIBLE_DEVICES=0 python lwf.py data/vtab/resisc45 -d Resisc45 -sc 80 --finetune --seed 0 \
--log logs/lwf/resisc45_80
# Patch Camelyon
CUDA_VISIBLE_DEVICES=0 python lwf.py data/vtab/patch_camelyon -d PatchCamelyon -sc 40 --finetune \
--seed 0 --log logs/lwf/patch_camelyon_40
CUDA_VISIBLE_DEVICES=0 python lwf.py data/vtab/patch_camelyon -d PatchCamelyon -sc 80 --finetune \
--seed 0 --log logs/lwf/patch_camelyon_80
CUDA_VISIBLE_DEVICES=0 python lwf.py data/vtab/patch_camelyon -d PatchCamelyon -sc 160 --finetune \
--seed 0 --log logs/lwf/patch_camelyon_160
CUDA_VISIBLE_DEVICES=0 python lwf.py data/vtab/patch_camelyon -d PatchCamelyon -sc 320 --finetune \
--seed 0 --log logs/lwf/patch_camelyon_320
================================================
FILE: domain_generalization/Transfer-Learning-Library/examples/task_adaptation/image_classification/requirements.txt
================================================
timm
================================================
FILE: domain_generalization/Transfer-Learning-Library/examples/task_adaptation/image_classification/stochnorm.py
================================================
"""
@author: Yifei Ji, Junguang Jiang
@contact: jiyf990330@163.com, JiangJunguang1123@outlook.com
"""
import random
import time
import warnings
import sys
import argparse
import shutil
import torch
import torch.nn as nn
import torch.backends.cudnn as cudnn
from torch.optim import SGD
from torch.utils.data import DataLoader
import torch.nn.functional as F
sys.path.append('../../..')
from talib.finetune.stochnorm import convert_model
from common.modules.classifier import Classifier
from common.utils.metric import accuracy
from common.utils.meter import AverageMeter, ProgressMeter
from common.utils.data import ForeverDataIterator
from common.utils.logger import CompleteLogger
sys.path.append('.')
import utils
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
def main(args: argparse.Namespace):
logger = CompleteLogger(args.log, args.phase)
print(args)
if args.seed is not None:
random.seed(args.seed)
torch.manual_seed(args.seed)
cudnn.deterministic = True
warnings.warn('You have chosen to seed training. '
'This will turn on the CUDNN deterministic setting, '
'which can slow down your training considerably! '
'You may see unexpected behavior when restarting '
'from checkpoints.')
cudnn.benchmark = True
# Data loading code
train_transform = utils.get_train_transform(args.train_resizing, not args.no_hflip, args.color_jitter)
val_transform = utils.get_val_transform(args.val_resizing)
print("train_transform: ", train_transform)
print("val_transform: ", val_transform)
train_dataset, val_dataset, num_classes = utils.get_dataset(args.data, args.root, train_transform,
val_transform, args.sample_rate, args.num_samples_per_classes)
train_loader = DataLoader(train_dataset, batch_size=args.batch_size, shuffle=True,
num_workers=args.workers, drop_last=True)
train_iter = ForeverDataIterator(train_loader)
val_loader = DataLoader(val_dataset, batch_size=args.batch_size, shuffle=False, num_workers=args.workers)
print("training dataset size: {} test dataset size: {}".format(len(train_dataset), len(val_dataset)))
# create model
print("=> using pre-trained model '{}'".format(args.arch))
backbone = utils.get_model(args.arch, args.pretrained)
pool_layer = nn.Identity() if args.no_pool else None
classifier = Classifier(backbone, num_classes, pool_layer=pool_layer, finetune=args.finetune).to(device)
classifier = convert_model(classifier, p=args.prob)
# define optimizer and lr scheduler
optimizer = SGD(classifier.get_parameters(args.lr), lr=args.lr, momentum=args.momentum, weight_decay=args.wd, nesterov=True)
lr_scheduler = torch.optim.lr_scheduler.MultiStepLR(optimizer, args.lr_decay_epochs, gamma=args.lr_gamma)
# resume from the best checkpoint
if args.phase == 'test':
checkpoint = torch.load(logger.get_checkpoint_path('best'), map_location='cpu')
classifier.load_state_dict(checkpoint)
acc1 = utils.validate(val_loader, classifier, args, device)
print(acc1)
return
# start training
best_acc1 = 0.0
for epoch in range(args.epochs):
print(lr_scheduler.get_lr())
# train for one epoch
train(train_iter, classifier, optimizer, epoch, args)
lr_scheduler.step()
# evaluate on validation set
acc1 = utils.validate(val_loader, classifier, args, device)
# remember best acc@1 and save checkpoint
torch.save(classifier.state_dict(), logger.get_checkpoint_path('latest'))
if acc1 > best_acc1:
shutil.copy(logger.get_checkpoint_path('latest'), logger.get_checkpoint_path('best'))
best_acc1 = max(acc1, best_acc1)
print("best_acc1 = {:3.1f}".format(best_acc1))
logger.close()
def train(train_iter: ForeverDataIterator, model: Classifier, optimizer: SGD,
epoch: int, args: argparse.Namespace):
batch_time = AverageMeter('Time', ':4.2f')
data_time = AverageMeter('Data', ':3.1f')
losses = AverageMeter('Loss', ':3.2f')
cls_accs = AverageMeter('Cls Acc', ':3.1f')
progress = ProgressMeter(
args.iters_per_epoch,
[batch_time, data_time, losses, cls_accs],
prefix="Epoch: [{}]".format(epoch))
# switch to train mode
model.train()
end = time.time()
for i in range(args.iters_per_epoch):
x, labels = next(train_iter)
x = x.to(device)
label = labels.to(device)
# measure data loading time
data_time.update(time.time() - end)
# compute output
y, f = model(x)
cls_loss = F.cross_entropy(y, label)
loss = cls_loss
cls_acc = accuracy(y, label)[0]
losses.update(loss.item(), x.size(0))
cls_accs.update(cls_acc.item(), x.size(0))
# compute gradient and do SGD step
optimizer.zero_grad()
loss.backward()
optimizer.step()
# measure elapsed time
batch_time.update(time.time() - end)
end = time.time()
if i % args.print_freq == 0:
progress.display(i)
if __name__ == '__main__':
parser = argparse.ArgumentParser(description='StochNorm for Finetuning')
# dataset parameters
parser.add_argument('root', metavar='DIR',
help='root path of dataset')
parser.add_argument('-d', '--data', metavar='DATA')
parser.add_argument('-sr', '--sample-rate', default=100, type=int,
metavar='N',
help='sample rate of training dataset (default: 100)')
parser.add_argument('-sc', '--num-samples-per-classes', default=None, type=int,
help='number of samples per classes.')
parser.add_argument('--train-resizing', type=str, default='default')
parser.add_argument('--val-resizing', type=str, default='default')
parser.add_argument('--no-hflip', action='store_true', help='no random horizontal flipping during training')
parser.add_argument('--color-jitter', action='store_true')
# model parameters
parser.add_argument('-a', '--arch', metavar='ARCH', default='resnet50',
choices=utils.get_model_names(),
help='backbone architecture: ' +
' | '.join(utils.get_model_names()) +
' (default: resnet50)')
parser.add_argument('--no-pool', action='store_true',
help='no pool layer after the feature extractor.')
parser.add_argument('--finetune', action='store_true', help='whether use 10x smaller lr for backbone')
parser.add_argument('--prob', '--probability', default=0.5, type=float,
metavar='P', help='Probability for StochNorm layers')
parser.add_argument('--pretrained', default=None,
help="pretrained checkpoint of the backbone. "
"(default: None, use the ImageNet supervised pretrained backbone)")
# training parameters
parser.add_argument('-b', '--batch-size', default=48, type=int,
metavar='N',
help='mini-batch size (default: 48)')
parser.add_argument('--lr', '--learning-rate', default=0.01, type=float,
metavar='LR', help='initial learning rate', dest='lr')
parser.add_argument('--lr-gamma', default=0.1, type=float, help='parameter for lr scheduler')
parser.add_argument('--lr-decay-epochs', type=int, default=(12, ), nargs='+', help='epochs to decay lr')
parser.add_argument('--momentum', default=0.9, type=float, metavar='M',
help='momentum')
parser.add_argument('--wd', '--weight-decay', default=0.0005, type=float,
metavar='W', help='weight decay (default: 5e-4)')
parser.add_argument('-j', '--workers', default=2, type=int, metavar='N',
help='number of data loading workers (default: 4)')
parser.add_argument('--epochs', default=20, type=int, metavar='N',
help='number of total epochs to run')
parser.add_argument('-i', '--iters-per-epoch', default=500, type=int,
help='Number of iterations per epoch')
parser.add_argument('-p', '--print-freq', default=100, type=int,
metavar='N', help='print frequency (default: 100)')
parser.add_argument('--seed', default=None, type=int,
help='seed for initializing training. ')
parser.add_argument("--log", type=str, default='stochnorm',
help="Where to save logs, checkpoints and debugging images.")
parser.add_argument("--phase", type=str, default='train', choices=['train', 'test'],
help="When phase is 'test', only test the model.")
args = parser.parse_args()
main(args)
================================================
FILE: domain_generalization/Transfer-Learning-Library/examples/task_adaptation/image_classification/stochnorm.sh
================================================
#!/usr/bin/env bash
#CUB-200-2011
CUDA_VISIBLE_DEVICES=0 python stochnorm.py data/cub200 -d CUB200 -sr 100 --seed 0 --finetune --log logs/stochnorm/cub200_100
CUDA_VISIBLE_DEVICES=0 python stochnorm.py data/cub200 -d CUB200 -sr 50 --seed 0 --finetune --log logs/stochnorm/cub200_50
CUDA_VISIBLE_DEVICES=0 python stochnorm.py data/cub200 -d CUB200 -sr 30 --seed 0 --finetune --log logs/stochnorm/cub200_30
CUDA_VISIBLE_DEVICES=0 python stochnorm.py data/cub200 -d CUB200 -sr 15 --seed 0 --finetune --log logs/stochnorm/cub200_15
# Standford Cars
CUDA_VISIBLE_DEVICES=0 python stochnorm.py data/stanford_cars -d StanfordCars -sr 100 --seed 0 --finetune --log logs/stochnorm/car_100
CUDA_VISIBLE_DEVICES=0 python stochnorm.py data/stanford_cars -d StanfordCars -sr 50 --seed 0 --finetune --log logs/stochnorm/car_50
CUDA_VISIBLE_DEVICES=0 python stochnorm.py data/stanford_cars -d StanfordCars -sr 30 --seed 0 --finetune --log logs/stochnorm/car_30
CUDA_VISIBLE_DEVICES=0 python stochnorm.py data/stanford_cars -d StanfordCars -sr 15 --seed 0 --finetune --log logs/stochnorm/car_15
# Aircrafts
CUDA_VISIBLE_DEVICES=0 python stochnorm.py data/aircraft -d Aircraft -sr 100 --seed 0 --finetune --log logs/stochnorm/aircraft_100
CUDA_VISIBLE_DEVICES=0 python stochnorm.py data/aircraft -d Aircraft -sr 50 --seed 0 --finetune --log logs/stochnorm/aircraft_50
CUDA_VISIBLE_DEVICES=0 python stochnorm.py data/aircraft -d Aircraft -sr 30 --seed 0 --finetune --log logs/stochnorm/aircraft_30
CUDA_VISIBLE_DEVICES=0 python stochnorm.py data/aircraft -d Aircraft -sr 15 --seed 0 --finetune --log logs/stochnorm/aircraft_15
# Resisc45
CUDA_VISIBLE_DEVICES=0 python stochnorm.py data/vtab/resisc45 -d Resisc45 -sc 10 --finetune --seed 0 \
--log logs/stochnorm/resisc45_10
CUDA_VISIBLE_DEVICES=0 python stochnorm.py data/vtab/resisc45 -d Resisc45 -sc 20 --finetune --seed 0 \
--log logs/stochnorm/resisc45_20
CUDA_VISIBLE_DEVICES=0 python stochnorm.py data/vtab/resisc45 -d Resisc45 -sc 40 --finetune --seed 0 \
--log logs/stochnorm/resisc45_40
CUDA_VISIBLE_DEVICES=0 python stochnorm.py data/vtab/resisc45 -d Resisc45 -sc 80 --finetune --seed 0 \
--log logs/stochnorm/resisc45_80
# Patch Camelyon
CUDA_VISIBLE_DEVICES=0 python stochnorm.py data/vtab/patch_camelyon -d PatchCamelyon -sc 40 --finetune \
--seed 0 --log logs/stochnorm/patch_camelyon_40
CUDA_VISIBLE_DEVICES=0 python stochnorm.py data/vtab/patch_camelyon -d PatchCamelyon -sc 80 --finetune \
--seed 0 --log logs/stochnorm/patch_camelyon_80
CUDA_VISIBLE_DEVICES=0 python stochnorm.py data/vtab/patch_camelyon -d PatchCamelyon -sc 160 --finetune \
--seed 0 --log logs/stochnorm/patch_camelyon_160
CUDA_VISIBLE_DEVICES=0 python stochnorm.py data/vtab/patch_camelyon -d PatchCamelyon -sc 320 --finetune \
--seed 0 --log logs/stochnorm/patch_camelyon_320
# MoCo (Unsupervised Pretraining)
# CUB-200-2011
CUDA_VISIBLE_DEVICES=0 python stochnorm.py data/cub200 -d CUB200 -sr 100 --seed 0 --lr 0.1 -i 2000 --lr-decay-epochs 3 6 9 --epochs 12 \
--log logs/moco_pretrain_stochnorm/cub200_100 --pretrained checkpoints/moco_v1_200ep_backbone.pth
CUDA_VISIBLE_DEVICES=0 python stochnorm.py data/cub200 -d CUB200 -sr 50 --seed 0 --lr 0.1 -i 2000 --lr-decay-epochs 3 6 9 --epochs 12 \
--log logs/moco_pretrain_stochnorm/cub200_50 --pretrained checkpoints/moco_v1_200ep_backbone.pth
CUDA_VISIBLE_DEVICES=0 python stochnorm.py data/cub200 -d CUB200 -sr 30 --seed 0 --lr 0.1 -i 2000 --lr-decay-epochs 3 6 9 --epochs 12 \
--log logs/moco_pretrain_stochnorm/cub200_30 --pretrained checkpoints/moco_v1_200ep_backbone.pth
CUDA_VISIBLE_DEVICES=0 python stochnorm.py data/cub200 -d CUB200 -sr 15 --seed 0 --lr 0.1 -i 2000 --lr-decay-epochs 3 6 9 --epochs 12 \
--log logs/moco_pretrain_stochnorm/cub200_15 --pretrained checkpoints/moco_v1_200ep_backbone.pth
# Standford Cars
CUDA_VISIBLE_DEVICES=0 python stochnorm.py data/stanford_cars -d StanfordCars -sr 100 --seed 0 --lr 0.1 -i 2000 --lr-decay-epochs 3 6 9 --epochs 12 \
--log logs/moco_pretrain_stochnorm/cars_100 --pretrained checkpoints/moco_v1_200ep_backbone.pth
CUDA_VISIBLE_DEVICES=0 python stochnorm.py data/stanford_cars -d StanfordCars -sr 50 --seed 0 --lr 0.1 -i 2000 --lr-decay-epochs 3 6 9 --epochs 12 \
--log logs/moco_pretrain_stochnorm/cars_50 --pretrained checkpoints/moco_v1_200ep_backbone.pth
CUDA_VISIBLE_DEVICES=0 python stochnorm.py data/stanford_cars -d StanfordCars -sr 30 --seed 0 --lr 0.1 -i 2000 --lr-decay-epochs 3 6 9 --epochs 12 \
--log logs/moco_pretrain_stochnorm/cars_30 --pretrained checkpoints/moco_v1_200ep_backbone.pth
CUDA_VISIBLE_DEVICES=0 python stochnorm.py data/stanford_cars -d StanfordCars -sr 15 --seed 0 --lr 0.1 -i 2000 --lr-decay-epochs 3 6 9 --epochs 12 \
--log logs/moco_pretrain_stochnorm/cars_15 --pretrained checkpoints/moco_v1_200ep_backbone.pth
# Aircrafts
CUDA_VISIBLE_DEVICES=0 python stochnorm.py data/aircraft -d Aircraft -sr 100 --seed 0 --lr 0.1 -i 2000 --lr-decay-epochs 3 6 9 --epochs 12 \
--log logs/moco_pretrain_stochnorm/aircraft_100 --pretrained checkpoints/moco_v1_200ep_backbone.pth
CUDA_VISIBLE_DEVICES=0 python stochnorm.py data/aircraft -d Aircraft -sr 50 --seed 0 --lr 0.1 -i 2000 --lr-decay-epochs 3 6 9 --epochs 12 \
--log logs/moco_pretrain_stochnorm/aircraft_50 --pretrained checkpoints/moco_v1_200ep_backbone.pth
CUDA_VISIBLE_DEVICES=0 python stochnorm.py data/aircraft -d Aircraft -sr 30 --seed 0 --lr 0.1 -i 2000 --lr-decay-epochs 3 6 9 --epochs 12 \
--log logs/moco_pretrain_stochnorm/aircraft_30 --pretrained checkpoints/moco_v1_200ep_backbone.pth
CUDA_VISIBLE_DEVICES=0 python stochnorm.py data/aircraft -d Aircraft -sr 15 --seed 0 --lr 0.1 -i 2000 --lr-decay-epochs 3 6 9 --epochs 12 \
--log logs/moco_pretrain_stochnorm/aircraft_15 --pretrained checkpoints/moco_v1_200ep_backbone.pth
================================================
FILE: domain_generalization/Transfer-Learning-Library/examples/task_adaptation/image_classification/utils.py
================================================
"""
@author: Junguang Jiang
@contact: JiangJunguang1123@outlook.com
"""
import time
from PIL import Image
import timm
import numpy as np
import random
import torch
import torch.nn as nn
import torch.nn.functional as F
from torch.utils.data import Subset
import torchvision.transforms as T
from torch.optim import SGD, Adam
import common.vision.datasets as datasets
import common.vision.models as models
from common.utils.metric import accuracy
from common.utils.meter import AverageMeter, ProgressMeter
from common.vision.transforms import Denormalize
def get_model_names():
return sorted(
name for name in models.__dict__
if name.islower() and not name.startswith("__")
and callable(models.__dict__[name])
) + timm.list_models()
def get_model(model_name, pretrained_checkpoint=None):
if model_name in models.__dict__:
# load models from common.vision.models
backbone = models.__dict__[model_name](pretrained=True)
else:
# load models from pytorch-image-models
backbone = timm.create_model(model_name, pretrained=True)
try:
backbone.out_features = backbone.get_classifier().in_features
backbone.reset_classifier(0, '')
backbone.copy_head = backbone.get_classifier
except:
backbone.out_features = backbone.head.in_features
backbone.head = nn.Identity()
backbone.copy_head = lambda x: x.head
if pretrained_checkpoint:
print("=> loading pre-trained model from '{}'".format(pretrained_checkpoint))
pretrained_dict = torch.load(pretrained_checkpoint)
backbone.load_state_dict(pretrained_dict, strict=False)
return backbone
def get_dataset(dataset_name, root, train_transform, val_transform, sample_rate=100, num_samples_per_classes=None):
"""
When sample_rate < 100, e.g. sample_rate = 50, use 50% data to train the model.
Otherwise,
if num_samples_per_classes is not None, e.g. 5, then sample 5 images for each class, and use them to train the model;
otherwise, keep all the data.
"""
dataset = datasets.__dict__[dataset_name]
if sample_rate < 100:
train_dataset = dataset(root=root, split='train', sample_rate=sample_rate, download=True, transform=train_transform)
test_dataset = dataset(root=root, split='test', sample_rate=100, download=True, transform=val_transform)
num_classes = train_dataset.num_classes
else:
train_dataset = dataset(root=root, split='train', download=True, transform=train_transform)
test_dataset = dataset(root=root, split='test', download=True, transform=val_transform)
num_classes = train_dataset.num_classes
if num_samples_per_classes is not None:
samples = list(range(len(train_dataset)))
random.shuffle(samples)
samples_len = min(num_samples_per_classes * num_classes, len(train_dataset))
print("Origin dataset:", len(train_dataset), "Sampled dataset:", samples_len, "Ratio:", float(samples_len) / len(train_dataset))
train_dataset = Subset(train_dataset, samples[:samples_len])
return train_dataset, test_dataset, num_classes
def validate(val_loader, model, args, device, visualize=None) -> float:
batch_time = AverageMeter('Time', ':6.3f')
losses = AverageMeter('Loss', ':.4e')
top1 = AverageMeter('Acc@1', ':6.2f')
progress = ProgressMeter(
len(val_loader),
[batch_time, losses, top1],
prefix='Test: ')
# switch to evaluate mode
model.eval()
with torch.no_grad():
end = time.time()
for i, (images, target) in enumerate(val_loader):
images = images.to(device)
target = target.to(device)
# compute output
output = model(images)
loss = F.cross_entropy(output, target)
# measure accuracy and record loss
acc1, = accuracy(output, target, topk=(1, ))
losses.update(loss.item(), images.size(0))
top1.update(acc1.item(), images.size(0))
# measure elapsed time
batch_time.update(time.time() - end)
end = time.time()
if i % args.print_freq == 0:
progress.display(i)
if visualize is not None:
visualize(images[0], "val_{}".format(i))
print(' * Acc@1 {top1.avg:.3f}'.format(top1=top1))
return top1.avg
def get_train_transform(resizing='default', random_horizontal_flip=True, random_color_jitter=False):
"""
resizing mode:
- default: take a random resized crop of size 224 with scale in [0.2, 1.];
- res: resize the image to 224;
- res.|crop: resize the image to 256 and take a random crop of size 224;
- res.sma|crop: resize the image keeping its aspect ratio such that the
smaller side is 256, then take a random crop of size 224;
– inc.crop: “inception crop” from (Szegedy et al., 2015);
– cif.crop: resize the image to 224, zero-pad it by 28 on each side, then take a random crop of size 224.
"""
if resizing == 'default':
transform = T.RandomResizedCrop(224, scale=(0.2, 1.))
elif resizing == 'res.':
transform = T.Resize((224, 224))
elif resizing == 'res.|crop':
transform = T.Compose([
T.Resize((256, 256)),
T.RandomCrop(224)
])
elif resizing == "res.sma|crop":
transform = T.Compose([
T.Resize(256),
T.RandomCrop(224)
])
elif resizing == 'inc.crop':
transform = T.RandomResizedCrop(224)
elif resizing == 'cif.crop':
transform = T.Compose([
T.Resize((224, 224)),
T.Pad(28),
T.RandomCrop(224),
])
else:
raise NotImplementedError(resizing)
transforms = [transform]
if random_horizontal_flip:
transforms.append(T.RandomHorizontalFlip())
if random_color_jitter:
transforms.append(T.ColorJitter(brightness=0.5, contrast=0.5, saturation=0.5, hue=0.5))
transforms.extend([
T.ToTensor(),
T.Normalize(mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225])
])
return T.Compose(transforms)
def get_val_transform(resizing='default'):
"""
resizing mode:
- default: resize the image to 256 and take the center crop of size 224;
– res.: resize the image to 224
– res.|crop: resize the image such that the smaller side is of size 256 and
then take a central crop of size 224.
"""
if resizing == 'default':
transform = T.Compose([
T.Resize(256),
T.CenterCrop(224),
])
elif resizing == 'res.':
transform = T.Resize((224, 224))
elif resizing == 'res.|crop':
transform = T.Compose([
T.Resize((256, 256)),
T.CenterCrop(224),
])
else:
raise NotImplementedError(resizing)
return T.Compose([
transform,
T.ToTensor(),
T.Normalize(mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225])
])
def get_optimizer(optimizer_name, params, lr, wd, momentum):
'''
Args:
optimizer_name:
- SGD
- Adam
params: iterable of parameters to optimize or dicts defining parameter groups
lr: learning rate
weight_decay: weight decay
momentum: momentum factor for SGD
'''
if optimizer_name == 'SGD':
optimizer = SGD(params=params, lr=lr, momentum=momentum, weight_decay=wd, nesterov=True)
elif optimizer_name == 'Adam':
optimizer = Adam(params=params, lr=lr, weight_decay=wd)
else:
raise NotImplementedError(optimizer_name)
return optimizer
def visualize(image, filename):
"""
Args:
image (tensor): 3 x H x W
filename: filename of the saving image
"""
image = image.detach().cpu()
image = Denormalize(mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225])(image)
image = image.numpy().transpose((1, 2, 0)) * 255
Image.fromarray(np.uint8(image)).save(filename)
================================================
FILE: domain_generalization/Transfer-Learning-Library/requirements.txt
================================================
torch>=1.7.0
torchvision
numpy
prettytable
tqdm
scikit-learn
webcolors
matplotlib
================================================
FILE: domain_generalization/Transfer-Learning-Library/setup.py
================================================
from setuptools import setup, find_packages
import re
from os import path
here = path.abspath(path.dirname(__file__))
# Get the version string
with open(path.join(here, 'dalib', '__init__.py')) as f:
version = re.search(r'__version__ = \'(.*?)\'', f.read()).group(1)
if __name__ == '__main__':
setup(
name="dalib", # Replace with your own username
version=version,
author="THUML",
author_email="JiangJunguang1123@outlook.com",
keywords="domain adaptation, task adaptation, domain generalization, "
"transfer learning, deep learning, pytorch",
description="A Transfer Learning Library for Domain Adaptation, Task Adaptation, and Domain Generalization",
long_description=open('README.md', encoding='utf8').read(),
long_description_content_type="text/markdown",
url="https://github.com/thuml/Transfer-Learning-Library",
packages=find_packages(exclude=['docs', 'examples']),
classifiers=[
# How mature is this project? Common values are
# 3 - Alpha
# 4 - Beta
# 5 - Production/Stable
'Development Status :: 3 - Alpha',
# Indicate who your project is intended for
'Intended Audience :: Science/Research',
'Topic :: Scientific/Engineering :: Artificial Intelligence',
'Topic :: Software Development :: Libraries :: Python Modules',
# Pick your license as you wish (should match "license" above)
'License :: OSI Approved :: MIT License',
# Specify the Python versions you support here. In particular, ensure
# that you indicate whether you support Python 2, Python 3 or both.
'Programming Language :: Python :: 3.6',
'Programming Language :: Python :: 3.7',
'Programming Language :: Python :: 3.8',
],
python_requires='>=3.6',
install_requires=[
'torch>=1.7.0',
'torchvision>=0.5.0',
'numpy',
'prettytable',
'tqdm',
'scikit-learn',
'webcolors',
'matplotlib'
],
extras_require={
'dev': [
'Sphinx',
'sphinx_rtd_theme',
]
},
)
================================================
FILE: domain_generalization/Transfer-Learning-Library/talib/__init__.py
================================================
__all__ = ['finetune']
__version__ = '0.3'
================================================
FILE: domain_generalization/Transfer-Learning-Library/talib/finetune/__init__.py
================================================
from .stochnorm import *
from .bss import *
from .co_tuning import *
from .delta import *
from .bi_tuning import *
__all__ = ['stochnorm', 'bss', 'co_tuning', 'delta', 'bi_tuning']
================================================
FILE: domain_generalization/Transfer-Learning-Library/talib/finetune/bi_tuning.py
================================================
"""
@author: Junguang Jiang
@contact: JiangJunguang1123@outlook.com
"""
import torch
import torch.nn as nn
from torch.nn.functional import normalize
from common.modules.classifier import Classifier as ClassifierBase
class Classifier(ClassifierBase):
"""Classifier class for Bi-Tuning.
Args:
backbone (torch.nn.Module): Any backbone to extract 2-d features from data
num_classes (int): Number of classes
projection_dim (int, optional): Dimension of the projector head. Default: 128
finetune (bool): Whether finetune the classifier or train from scratch. Default: True
.. note::
The learning rate of this classifier is set 10 times to that of the feature extractor for better accuracy
by default. If you have other optimization strategies, please over-ride :meth:`~Classifier.get_parameters`.
Inputs:
- x (tensor): input data fed to `backbone`
Outputs:
In the training mode,
- y: classifier's predictions
- z: projector's predictions
- hn: normalized features after `bottleneck` layer and before `head` layer
In the eval mode,
- y: classifier's predictions
Shape:
- Inputs: (minibatch, *) where * means, any number of additional dimensions
- y: (minibatch, `num_classes`)
- z: (minibatch, `projection_dim`)
- hn: (minibatch, `features_dim`)
"""
def __init__(self, backbone: nn.Module, num_classes: int, projection_dim=128, finetune=True, pool_layer=None):
head = nn.Linear(backbone.out_features, num_classes)
head.weight.data.normal_(0, 0.01)
head.bias.data.fill_(0.0)
super(Classifier, self).__init__(backbone, num_classes=num_classes, head=head, finetune=finetune,
pool_layer=pool_layer)
self.projector = nn.Linear(backbone.out_features, projection_dim)
self.projection_dim = projection_dim
def forward(self, x: torch.Tensor):
batch_size = x.shape[0]
h = self.backbone(x)
h = self.pool_layer(h)
h = self.bottleneck(h)
y = self.head(h)
z = normalize(self.projector(h), dim=1)
hn = torch.cat([h, torch.ones(batch_size, 1, dtype=torch.float).to(h.device)], dim=1)
hn = normalize(hn, dim=1)
if self.training:
return y, z, hn
else:
return y
def get_parameters(self, base_lr=1.0):
"""A parameter list which decides optimization hyper-parameters,
such as the relative learning rate of each layer
"""
params = [
{"params": self.backbone.parameters(), "lr": 0.1 * base_lr if self.finetune else 1.0 * base_lr},
{"params": self.bottleneck.parameters(), "lr": 1.0 * base_lr},
{"params": self.head.parameters(), "lr": 1.0 * base_lr},
{"params": self.projector.parameters(), "lr": 0.1 * base_lr if self.finetune else 1.0 * base_lr},
]
return params
class BiTuning(nn.Module):
"""
Bi-Tuning Module in `Bi-tuning of Pre-trained Representations `_.
Args:
encoder_q (Classifier): Query encoder.
encoder_k (Classifier): Key encoder.
num_classes (int): Number of classes
K (int): Queue size. Default: 40
m (float): Momentum coefficient. Default: 0.999
T (float): Temperature. Default: 0.07
Inputs:
- im_q (tensor): input data fed to `encoder_q`
- im_k (tensor): input data fed to `encoder_k`
- labels (tensor): classification labels of input data
Outputs: y_q, logits_z, logits_y, labels_c
- y_q: query classifier's predictions
- logits_z: projector's predictions on both positive and negative samples
- logits_y: classifier's predictions on both positive and negative samples
- labels_c: contrastive labels
Shape:
- im_q, im_k: (minibatch, *) where * means, any number of additional dimensions
- labels: (minibatch, )
- y_q: (minibatch, `num_classes`)
- logits_z: (minibatch, 1 + `num_classes` x `K`, `projection_dim`)
- logits_y: (minibatch, 1 + `num_classes` x `K`, `num_classes`)
- labels_c: (minibatch, 1 + `num_classes` x `K`)
"""
def __init__(self, encoder_q: Classifier, encoder_k: Classifier, num_classes, K=40, m=0.999, T=0.07):
super(BiTuning, self).__init__()
self.K = K
self.m = m
self.T = T
self.num_classes = num_classes
# create the encoders
# num_classes is the output fc dimension
self.encoder_q = encoder_q
self.encoder_k = encoder_k
for param_q, param_k in zip(self.encoder_q.parameters(), self.encoder_k.parameters()):
param_k.data.copy_(param_q.data) # initialize
param_k.requires_grad = False # not update by gradient
# create the queue
self.register_buffer("queue_h", torch.randn(encoder_q.features_dim + 1, num_classes, K))
self.register_buffer("queue_z", torch.randn(encoder_q.projection_dim, num_classes, K))
self.queue_h = normalize(self.queue_h, dim=0)
self.queue_z = normalize(self.queue_z, dim=0)
self.register_buffer("queue_ptr", torch.zeros(num_classes, dtype=torch.long))
@torch.no_grad()
def _momentum_update_key_encoder(self):
"""
Momentum update of the key encoder
"""
for param_q, param_k in zip(self.encoder_q.parameters(), self.encoder_k.parameters()):
param_k.data = param_k.data * self.m + param_q.data * (1. - self.m)
@torch.no_grad()
def _dequeue_and_enqueue(self, h, z, label):
batch_size = h.shape[0]
assert self.K % batch_size == 0 # for simplicity
ptr = int(self.queue_ptr[label])
# replace the keys at ptr (dequeue and enqueue)
self.queue_h[:, label, ptr: ptr + batch_size] = h.T
self.queue_z[:, label, ptr: ptr + batch_size] = z.T
# move pointer
self.queue_ptr[label] = (ptr + batch_size) % self.K
def forward(self, im_q, im_k, labels):
batch_size = im_q.size(0)
device = im_q.device
# compute query features
y_q, z_q, h_q = self.encoder_q(im_q)
# compute key features
with torch.no_grad(): # no gradient to keys
self._momentum_update_key_encoder() # update the key encoder
y_k, z_k, h_k = self.encoder_k(im_k)
# compute logits for projection z
# current positive logits: Nx1
logits_z_cur = torch.einsum('nc,nc->n', [z_q, z_k]).unsqueeze(-1)
queue_z = self.queue_z.clone().detach().to(device)
# positive logits: N x K
logits_z_pos = torch.Tensor([]).to(device)
# negative logits: N x ((C-1) x K)
logits_z_neg = torch.Tensor([]).to(device)
for i in range(batch_size):
c = labels[i]
pos_samples = queue_z[:, c, :] # D x K
neg_samples = torch.cat([queue_z[:, 0: c, :], queue_z[:, c + 1:, :]], dim=1).flatten(
start_dim=1) # D x ((C-1)xK)
ith_pos = torch.einsum('nc,ck->nk', [z_q[i: i + 1], pos_samples]) # 1 x D
ith_neg = torch.einsum('nc,ck->nk', [z_q[i: i + 1], neg_samples]) # 1 x ((C-1)xK)
logits_z_pos = torch.cat((logits_z_pos, ith_pos), dim=0)
logits_z_neg = torch.cat((logits_z_neg, ith_neg), dim=0)
self._dequeue_and_enqueue(h_k[i:i + 1], z_k[i:i + 1], labels[i])
logits_z = torch.cat([logits_z_cur, logits_z_pos, logits_z_neg], dim=1) # Nx(1+C*K)
# apply temperature
logits_z /= self.T
logits_z = nn.LogSoftmax(dim=1)(logits_z)
# compute logits for classification y
w = torch.cat([self.encoder_q.head.weight.data, self.encoder_q.head.bias.data.unsqueeze(-1)], dim=1)
w = normalize(w, dim=1) # C x F
# current positive logits: Nx1
logits_y_cur = torch.einsum('nk,kc->nc', [h_q, w.T]) # N x C
queue_y = self.queue_h.clone().detach().to(device).flatten(start_dim=1).T # (C * K) x F
logits_y_queue = torch.einsum('nk,kc->nc', [queue_y, w.T]).reshape(self.num_classes, -1,
self.num_classes) # C x K x C
logits_y = torch.Tensor([]).to(device)
for i in range(batch_size):
c = labels[i]
# calculate the ith sample in the batch
cur_sample = logits_y_cur[i:i + 1, c] # 1
pos_samples = logits_y_queue[c, :, c] # K
neg_samples = torch.cat([logits_y_queue[0: c, :, c], logits_y_queue[c + 1:, :, c]], dim=0).view(
-1) # (C-1)*K
ith = torch.cat([cur_sample, pos_samples, neg_samples]) # 1+C*K
logits_y = torch.cat([logits_y, ith.unsqueeze(dim=0)], dim=0)
logits_y /= self.T
logits_y = nn.LogSoftmax(dim=1)(logits_y)
# contrastive labels
labels_c = torch.zeros([batch_size, self.K * self.num_classes + 1]).to(device)
labels_c[:, 0:self.K + 1].fill_(1.0 / (self.K + 1))
return y_q, logits_z, logits_y, labels_c
================================================
FILE: domain_generalization/Transfer-Learning-Library/talib/finetune/bss.py
================================================
"""
@author: Yifei Ji
@contact: jiyf990330@163.com
"""
import torch
import torch.nn as nn
__all__ = ['BatchSpectralShrinkage']
class BatchSpectralShrinkage(nn.Module):
r"""
The regularization term in `Catastrophic Forgetting Meets Negative Transfer:
Batch Spectral Shrinkage for Safe Transfer Learning (NIPS 2019) `_.
The BSS regularization of feature matrix :math:`F` can be described as:
.. math::
L_{bss}(F) = \sum_{i=1}^{k} \sigma_{-i}^2 ,
where :math:`k` is the number of singular values to be penalized, :math:`\sigma_{-i}` is the :math:`i`-th smallest singular value of feature matrix :math:`F`.
All the singular values of feature matrix :math:`F` are computed by `SVD`:
.. math::
F = U\Sigma V^T,
where the main diagonal elements of the singular value matrix :math:`\Sigma` is :math:`[\sigma_1, \sigma_2, ..., \sigma_b]`.
Args:
k (int): The number of singular values to be penalized. Default: 1
Shape:
- Input: :math:`(b, |\mathcal{f}|)` where :math:`b` is the batch size and :math:`|\mathcal{f}|` is feature dimension.
- Output: scalar.
"""
def __init__(self, k=1):
super(BatchSpectralShrinkage, self).__init__()
self.k = k
def forward(self, feature):
result = 0
u, s, v = torch.svd(feature.t())
num = s.size(0)
for i in range(self.k):
result += torch.pow(s[num-1-i], 2)
return result
================================================
FILE: domain_generalization/Transfer-Learning-Library/talib/finetune/co_tuning.py
================================================
"""
@author: Yifei Ji
@contact: jiyf990330@163.com
"""
from typing import Tuple, Optional, List, Dict
import os
import torch
import torch.nn as nn
import numpy as np
import torch.nn.functional as F
import tqdm
from .lwf import Classifier as ClassifierBase
__all__ = ['Classifier', 'CoTuningLoss', 'Relationship']
class CoTuningLoss(nn.Module):
"""
The Co-Tuning loss in `Co-Tuning for Transfer Learning (NIPS 2020)
`_.
Inputs:
- input: p(y_s) predicted by source classifier.
- target: p(y_s|y_t), where y_t is the ground truth class label in target dataset.
Shape:
- input: (b, N_p), where b is the batch size and N_p is the number of classes in source dataset
- target: (b, N_p), where b is the batch size and N_p is the number of classes in source dataset
- Outputs: scalar.
"""
def __init__(self):
super(CoTuningLoss, self).__init__()
def forward(self, input: torch.Tensor, target: torch.Tensor) -> torch.Tensor:
y = - target * F.log_softmax(input, dim=-1)
y = torch.mean(torch.sum(y, dim=-1))
return y
class Relationship(object):
"""Learns the category relationship p(y_s|y_t) between source dataset and target dataset.
Args:
data_loader (torch.utils.data.DataLoader): A data loader of target dataset.
classifier (torch.nn.Module): A classifier for Co-Tuning.
device (torch.nn.Module): The device to run classifier.
cache (str, optional): Path to find and save the relationship file.
"""
def __init__(self, data_loader, classifier, device, cache=None):
super(Relationship, self).__init__()
self.data_loader = data_loader
self.classifier = classifier
self.device = device
if cache is None or not os.path.exists(cache):
source_predictions, target_labels = self.collect_labels()
self.relationship = self.get_category_relationship(source_predictions, target_labels)
if cache is not None:
np.save(cache, self.relationship)
else:
self.relationship = np.load(cache)
def __getitem__(self, category):
return self.relationship[category]
def collect_labels(self):
"""
Collects predictions of target dataset by source model and corresponding ground truth class labels.
Returns:
- source_probabilities, [N, N_p], where N_p is the number of classes in source dataset
- target_labels, [N], where 0 <= each number < N_t, and N_t is the number of classes in target dataset
"""
print("Collecting labels to calculate relationship")
source_predictions = []
target_labels = []
self.classifier.eval()
with torch.no_grad():
for i, (x, label) in enumerate(tqdm.tqdm(self.data_loader)):
x = x.to(self.device)
y_s = self.classifier(x)
source_predictions.append(F.softmax(y_s, dim=1).detach().cpu().numpy())
target_labels.append(label)
return np.concatenate(source_predictions, 0), np.concatenate(target_labels, 0)
def get_category_relationship(self, source_probabilities, target_labels):
"""
The direct approach of learning category relationship p(y_s | y_t).
Args:
source_probabilities (numpy.array): [N, N_p], where N_p is the number of classes in source dataset
target_labels (numpy.array): [N], where 0 <= each number < N_t, and N_t is the number of classes in target dataset
Returns:
Conditional probability, [N_c, N_p] matrix representing the conditional probability p(pre-trained class | target_class)
"""
N_t = np.max(target_labels) + 1 # the number of target classes
conditional = []
for i in range(N_t):
this_class = source_probabilities[target_labels == i]
average = np.mean(this_class, axis=0, keepdims=True)
conditional.append(average)
return np.concatenate(conditional)
class Classifier(ClassifierBase):
"""A Classifier used in `Co-Tuning for Transfer Learning (NIPS 2020)
`_..
Args:
backbone (torch.nn.Module): Any backbone to extract 2-d features from data.
num_classes (int): Number of classes.
head_source (torch.nn.Module): Classifier head of source model.
head_target (torch.nn.Module, optional): Any classifier head. Use :class:`torch.nn.Linear` by default
finetune (bool): Whether finetune the classifier or train from scratch. Default: True
Inputs:
- x (tensor): input data fed to backbone
Outputs:
- y_s: predictions of source classifier head
- y_t: predictions of target classifier head
Shape:
- Inputs: (b, *) where b is the batch size and * means any number of additional dimensions
- y_s: (b, N), where b is the batch size and N is the number of classes
- y_t: (b, N), where b is the batch size and N is the number of classes
"""
def __init__(self, backbone: nn.Module, num_classes: int, head_source, **kwargs):
super(Classifier, self).__init__(backbone, num_classes, head_source, **kwargs)
def get_parameters(self, base_lr=1.0) -> List[Dict]:
"""A parameter list which decides optimization hyper-parameters,
such as the relative learning rate of each layer
"""
params = [
{"params": self.backbone.parameters(), "lr": 0.1 * base_lr if self.finetune else 1.0 * base_lr},
{"params": self.head_source.parameters(), "lr": 0.1 * base_lr if self.finetune else 1.0 * base_lr},
{"params": self.bottleneck.parameters(), "lr": 1.0 * base_lr},
{"params": self.head_target.parameters(), "lr": 1.0 * base_lr},
]
return params
================================================
FILE: domain_generalization/Transfer-Learning-Library/talib/finetune/delta.py
================================================
"""
@author: Yifei Ji
@contact: jiyf990330@163.com
"""
import torch
import torch.nn as nn
import functools
from collections import OrderedDict
class L2Regularization(nn.Module):
r"""The L2 regularization of parameters :math:`w` can be described as:
.. math::
{\Omega} (w) = \dfrac{1}{2} \Vert w\Vert_2^2 ,
Args:
model (torch.nn.Module): The model to apply L2 penalty.
Shape:
- Output: scalar.
"""
def __init__(self, model: nn.Module):
super(L2Regularization, self).__init__()
self.model = model
def forward(self):
output = 0.0
for param in self.model.parameters():
output += 0.5 * torch.norm(param) ** 2
return output
class SPRegularization(nn.Module):
r"""
The SP (Starting Point) regularization from `Explicit inductive bias for transfer learning with convolutional networks
(ICML 2018) `_
The SP regularization of parameters :math:`w` can be described as:
.. math::
{\Omega} (w) = \dfrac{1}{2} \Vert w-w_0\Vert_2^2 ,
where :math:`w_0` is the parameter vector of the model pretrained on the source problem, acting as the starting point (SP) in fine-tuning.
Args:
source_model (torch.nn.Module): The source (starting point) model.
target_model (torch.nn.Module): The target (fine-tuning) model.
Shape:
- Output: scalar.
"""
def __init__(self, source_model: nn.Module, target_model: nn.Module):
super(SPRegularization, self).__init__()
self.target_model = target_model
self.source_weight = {}
for name, param in source_model.named_parameters():
self.source_weight[name] = param.detach()
def forward(self):
output = 0.0
for name, param in self.target_model.named_parameters():
output += 0.5 * torch.norm(param - self.source_weight[name]) ** 2
return output
class BehavioralRegularization(nn.Module):
r"""
The behavioral regularization from `DELTA:DEep Learning Transfer using Feature Map with Attention
for convolutional networks (ICLR 2019) `_
It can be described as:
.. math::
{\Omega} (w) = \sum_{j=1}^{N} \Vert FM_j(w, \boldsymbol x)-FM_j(w^0, \boldsymbol x)\Vert_2^2 ,
where :math:`w^0` is the parameter vector of the model pretrained on the source problem, acting as the starting point (SP) in fine-tuning,
:math:`FM_j(w, \boldsymbol x)` is feature maps generated from the :math:`j`-th layer of the model parameterized with :math:`w`, given the input :math:`\boldsymbol x`.
Inputs:
layer_outputs_source (OrderedDict): The dictionary for source model, where the keys are layer names and the values are feature maps correspondingly.
layer_outputs_target (OrderedDict): The dictionary for target model, where the keys are layer names and the values are feature maps correspondingly.
Shape:
- Output: scalar.
"""
def __init__(self):
super(BehavioralRegularization, self).__init__()
def forward(self, layer_outputs_source, layer_outputs_target):
output = 0.0
for fm_src, fm_tgt in zip(layer_outputs_source.values(), layer_outputs_target.values()):
output += 0.5 * (torch.norm(fm_tgt - fm_src.detach()) ** 2)
return output
class AttentionBehavioralRegularization(nn.Module):
r"""
The behavioral regularization with attention from `DELTA:DEep Learning Transfer using Feature Map with Attention
for convolutional networks (ICLR 2019) `_
It can be described as:
.. math::
{\Omega} (w) = \sum_{j=1}^{N} W_j(w) \Vert FM_j(w, \boldsymbol x)-FM_j(w^0, \boldsymbol x)\Vert_2^2 ,
where
:math:`w^0` is the parameter vector of the model pretrained on the source problem, acting as the starting point (SP) in fine-tuning.
:math:`FM_j(w, \boldsymbol x)` is feature maps generated from the :math:`j`-th layer of the model parameterized with :math:`w`, given the input :math:`\boldsymbol x`.
:math:`W_j(w)` is the channel attention of the :math:`j`-th layer of the model parameterized with :math:`w`.
Args:
channel_attention (list): The channel attentions of feature maps generated by each selected layer. For the layer with C channels, the channel attention is a tensor of shape [C].
Inputs:
layer_outputs_source (OrderedDict): The dictionary for source model, where the keys are layer names and the values are feature maps correspondingly.
layer_outputs_target (OrderedDict): The dictionary for target model, where the keys are layer names and the values are feature maps correspondingly.
Shape:
- Output: scalar.
"""
def __init__(self, channel_attention):
super(AttentionBehavioralRegularization, self).__init__()
self.channel_attention = channel_attention
def forward(self, layer_outputs_source, layer_outputs_target):
output = 0.0
for i, (fm_src, fm_tgt) in enumerate(zip(layer_outputs_source.values(), layer_outputs_target.values())):
b, c, h, w = fm_src.shape
fm_src = fm_src.reshape(b, c, h * w)
fm_tgt = fm_tgt.reshape(b, c, h * w)
distance = torch.norm(fm_tgt - fm_src.detach(), 2, 2)
distance = c * torch.mul(self.channel_attention[i], distance ** 2) / (h * w)
output += 0.5 * torch.sum(distance)
return output
def get_attribute(obj, attr, *args):
def _getattr(obj, attr):
return getattr(obj, attr, *args)
return functools.reduce(_getattr, [obj] + attr.split('.'))
class IntermediateLayerGetter:
r"""
Wraps a model to get intermediate output values of selected layers.
Args:
model (torch.nn.Module): The model to collect intermediate layer feature maps.
return_layers (list): The names of selected modules to return the output.
keep_output (bool): If True, `model_output` contains the final model's output, else return None. Default: True
Returns:
- An OrderedDict of intermediate outputs. The keys are selected layer names in `return_layers` and the values are the feature map outputs. The order is the same as `return_layers`.
- The model's final output. If `keep_output` is False, return None.
"""
def __init__(self, model, return_layers, keep_output=True):
self._model = model
self.return_layers = return_layers
self.keep_output = keep_output
def __call__(self, *args, **kwargs):
ret = OrderedDict()
handles = []
for name in self.return_layers:
layer = get_attribute(self._model, name)
def hook(module, input, output, name=name):
ret[name] = output
try:
h = layer.register_forward_hook(hook)
except AttributeError as e:
raise AttributeError(f'Module {name} not found')
handles.append(h)
if self.keep_output:
output = self._model(*args, **kwargs)
else:
self._model(*args, **kwargs)
output = None
for h in handles:
h.remove()
return ret, output
================================================
FILE: domain_generalization/Transfer-Learning-Library/talib/finetune/lwf.py
================================================
"""
@author: Junguang Jiang
@contact: JiangJunguang1123@outlook.com
"""
from typing import Optional, List, Dict
import torch
import torch.nn as nn
import tqdm
def collect_pretrain_labels(data_loader, classifier, device):
source_predictions = []
classifier.eval()
with torch.no_grad():
for i, (x, label) in enumerate(tqdm.tqdm(data_loader)):
x = x.to(device)
y_s = classifier(x)
source_predictions.append(y_s.detach().cpu())
return torch.cat(source_predictions, dim=0)
class Classifier(nn.Module):
"""A Classifier used in `Learning Without Forgetting (ECCV 2016)
`_..
Args:
backbone (torch.nn.Module): Any backbone to extract 2-d features from data.
num_classes (int): Number of classes.
head_source (torch.nn.Module): Classifier head of source model.
head_target (torch.nn.Module, optional): Any classifier head. Use :class:`torch.nn.Linear` by default
finetune (bool): Whether finetune the classifier or train from scratch. Default: True
Inputs:
- x (tensor): input data fed to backbone
Outputs:
- y_s: predictions of source classifier head
- y_t: predictions of target classifier head
Shape:
- Inputs: (b, *) where b is the batch size and * means any number of additional dimensions
- y_s: (b, N), where b is the batch size and N is the number of classes
- y_t: (b, N), where b is the batch size and N is the number of classes
"""
def __init__(self, backbone: nn.Module, num_classes: int, head_source,
head_target: Optional[nn.Module] = None, bottleneck: Optional[nn.Module] = None,
bottleneck_dim: Optional[int] = -1, finetune=True, pool_layer=None):
super(Classifier, self).__init__()
self.backbone = backbone
self.num_classes = num_classes
if pool_layer is None:
self.pool_layer = nn.Sequential(
nn.AdaptiveAvgPool2d(output_size=(1, 1)),
nn.Flatten()
)
else:
self.pool_layer = pool_layer
if bottleneck is None:
self.bottleneck = nn.Identity()
self._features_dim = backbone.out_features
else:
self.bottleneck = bottleneck
assert bottleneck_dim > 0
self._features_dim = bottleneck_dim
self.head_source = head_source
if head_target is None:
self.head_target = nn.Linear(self._features_dim, num_classes)
else:
self.head_target = head_target
self.finetune = finetune
@property
def features_dim(self) -> int:
"""The dimension of features before the final `head` layer"""
return self._features_dim
def forward(self, x: torch.Tensor):
""""""
f = self.backbone(x)
f = self.pool_layer(f)
y_s = self.head_source(f)
y_t = self.head_target(self.bottleneck(f))
if self.training:
return y_s, y_t
else:
return y_t
def get_parameters(self, base_lr=1.0) -> List[Dict]:
"""A parameter list which decides optimization hyper-parameters,
such as the relative learning rate of each layer
"""
params = [
{"params": self.backbone.parameters(), "lr": 0.1 * base_lr if self.finetune else 1.0 * base_lr},
# {"params": self.head_source.parameters(), "lr": 0.1 * base_lr if self.finetune else 1.0 * base_lr},
{"params": self.bottleneck.parameters(), "lr": 1.0 * base_lr},
{"params": self.head_target.parameters(), "lr": 1.0 * base_lr},
]
return params
================================================
FILE: domain_generalization/Transfer-Learning-Library/talib/finetune/stochnorm.py
================================================
"""
@author: Yifei Ji
@contact: jiyf990330@163.com
"""
import numpy as np
import torch
import torch.nn as nn
import torch.nn.functional as F
from torch.nn.parameter import Parameter
__all__ = ['StochNorm1d', 'StochNorm2d', 'convert_model']
class _StochNorm(nn.Module):
def __init__(self, num_features, eps=1e-5, momentum=0.1, affine=True, track_running_stats=True, p=0.5):
super(_StochNorm, self).__init__()
self.num_features = num_features
self.eps = eps
self.momentum = momentum
self.affine = affine
self.track_running_stats = track_running_stats
self.p = p
if self.affine:
self.weight = Parameter(torch.Tensor(num_features))
self.bias = Parameter(torch.Tensor(num_features))
else:
self.register_parameter('weight', None)
self.register_parameter('bias', None)
if self.track_running_stats:
self.register_buffer('running_mean', torch.zeros(num_features))
self.register_buffer('running_var', torch.ones(num_features))
else:
self.register_parameter('running_mean', None)
self.register_parameter('running_var', None)
self.reset_parameters()
def reset_parameters(self):
if self.track_running_stats:
self.running_mean.zero_()
self.running_var.fill_(1)
if self.affine:
self.weight.data.uniform_()
self.bias.data.zero_()
def _check_input_dim(self, input):
return NotImplemented
def forward(self, input):
self._check_input_dim(input)
if self.training:
z_0 = F.batch_norm(
input, self.running_mean, self.running_var, self.weight, self.bias,
False, self.momentum, self.eps)
z_1 = F.batch_norm(
input, self.running_mean, self.running_var, self.weight, self.bias,
True, self.momentum, self.eps)
if input.dim() == 2:
s = torch.from_numpy(
np.random.binomial(n=1, p=self.p, size=self.num_features).reshape(1,
self.num_features)).float().cuda()
elif input.dim() == 3:
s = torch.from_numpy(
np.random.binomial(n=1, p=self.p, size=self.num_features).reshape(1, self.num_features,
1)).float().cuda()
elif input.dim() == 4:
s = torch.from_numpy(
np.random.binomial(n=1, p=self.p, size=self.num_features).reshape(1, self.num_features, 1,
1)).float().cuda()
else:
raise BaseException()
z = (1 - s) * z_0 + s * z_1
else:
z = F.batch_norm(
input, self.running_mean, self.running_var, self.weight, self.bias,
False, self.momentum, self.eps)
return z
class StochNorm1d(_StochNorm):
r"""Applies Stochastic Normalization over a 2D or 3D input (a mini-batch of 1D inputs with optional additional channel dimension)
Stochastic Normalization is proposed in `Stochastic Normalization (NIPS 2020) `_
.. math::
\hat{x}_{i,0} = \frac{x_i - \tilde{\mu}}{ \sqrt{\tilde{\sigma} + \epsilon}}
\hat{x}_{i,1} = \frac{x_i - \mu}{ \sqrt{\sigma + \epsilon}}
\hat{x}_i = (1-s)\cdot \hat{x}_{i,0} + s\cdot \hat{x}_{i,1}
y_i = \gamma \hat{x}_i + \beta
where :math:`\mu` and :math:`\sigma` are mean and variance of current mini-batch data.
:math:`\tilde{\mu}` and :math:`\tilde{\sigma}` are current moving statistics of training data.
:math:`s` is a branch-selection variable generated from a Bernoulli distribution, where :math:`P(s=1)=p`.
During training, there are two normalization branches. One uses mean and
variance of current mini-batch data, while the other uses current moving
statistics of the training data as usual batch normalization.
During evaluation, the moving statistics is used for normalization.
Args:
num_features (int): :math:`c` from an expected input of size :math:`(b, c, l)` or :math:`l` from an expected input of size :math:`(b, l)`.
eps (float): A value added to the denominator for numerical stability.
Default: 1e-5
momentum (float): The value used for the running_mean and running_var
computation. Default: 0.1
affine (bool): A boolean value that when set to ``True``, gives the layer learnable
affine parameters. Default: ``True``
track_running_stats (bool): A boolean value that when set to True, this module tracks
the running mean and variance, and when set to False, this module does not
track such statistics, and initializes statistics buffers running_mean and
running_var as None. When these buffers are None, this module always uses
batch statistics in both training and eval modes. Default: True
p (float): The probability to choose the second branch (usual BN). Default: 0.5
Shape:
- Input: :math:`(b, l)` or :math:`(b, c, l)`
- Output: :math:`(b, l)` or :math:`(b, c, l)` (same shape as input)
"""
def _check_input_dim(self, input):
if input.dim() != 2 and input.dim() != 3:
raise ValueError('expected 2D or 3D input (got {}D input)'
.format(input.dim()))
class StochNorm2d(_StochNorm):
r"""
Applies Stochastic Normalization over a 4D input (a mini-batch of 2D inputs with additional channel dimension)
Stochastic Normalization is proposed in `Stochastic Normalization (NIPS 2020) `_
.. math::
\hat{x}_{i,0} = \frac{x_i - \tilde{\mu}}{ \sqrt{\tilde{\sigma} + \epsilon}}
\hat{x}_{i,1} = \frac{x_i - \mu}{ \sqrt{\sigma + \epsilon}}
\hat{x}_i = (1-s)\cdot \hat{x}_{i,0} + s\cdot \hat{x}_{i,1}
y_i = \gamma \hat{x}_i + \beta
where :math:`\mu` and :math:`\sigma` are mean and variance of current mini-batch data.
:math:`\tilde{\mu}` and :math:`\tilde{\sigma}` are current moving statistics of training data.
:math:`s` is a branch-selection variable generated from a Bernoulli distribution, where :math:`P(s=1)=p`.
During training, there are two normalization branches. One uses mean and
variance of current mini-batch data, while the other uses current moving
statistics of the training data as usual batch normalization.
During evaluation, the moving statistics is used for normalization.
Args:
num_features (int): :math:`c` from an expected input of size :math:`(b, c, h, w)`.
eps (float): A value added to the denominator for numerical stability.
Default: 1e-5
momentum (float): The value used for the running_mean and running_var
computation. Default: 0.1
affine (bool): A boolean value that when set to ``True``, gives the layer learnable
affine parameters. Default: ``True``
track_running_stats (bool): A boolean value that when set to True, this module tracks
the running mean and variance, and when set to False, this module does not
track such statistics, and initializes statistics buffers running_mean and
running_var as None. When these buffers are None, this module always uses
batch statistics in both training and eval modes. Default: True
p (float): The probability to choose the second branch (usual BN). Default: 0.5
Shape:
- Input: :math:`(b, c, h, w)`
- Output: :math:`(b, c, h, w)` (same shape as input)
"""
def _check_input_dim(self, input):
if input.dim() != 4:
raise ValueError('expected 4D input (got {}D input)'
.format(input.dim()))
class StochNorm3d(_StochNorm):
r"""
Applies Stochastic Normalization over a 5D input (a mini-batch of 3D inputs with additional channel dimension)
Stochastic Normalization is proposed in `Stochastic Normalization (NIPS 2020) `_
.. math::
\hat{x}_{i,0} = \frac{x_i - \tilde{\mu}}{ \sqrt{\tilde{\sigma} + \epsilon}}
\hat{x}_{i,1} = \frac{x_i - \mu}{ \sqrt{\sigma + \epsilon}}
\hat{x}_i = (1-s)\cdot \hat{x}_{i,0} + s\cdot \hat{x}_{i,1}
y_i = \gamma \hat{x}_i + \beta
where :math:`\mu` and :math:`\sigma` are mean and variance of current mini-batch data.
:math:`\tilde{\mu}` and :math:`\tilde{\sigma}` are current moving statistics of training data.
:math:`s` is a branch-selection variable generated from a Bernoulli distribution, where :math:`P(s=1)=p`.
During training, there are two normalization branches. One uses mean and
variance of current mini-batch data, while the other uses current moving
statistics of the training data as usual batch normalization.
During evaluation, the moving statistics is used for normalization.
Args:
num_features (int): :math:`c` from an expected input of size :math:`(b, c, d, h, w)`
eps (float): A value added to the denominator for numerical stability.
Default: 1e-5
momentum (float): The value used for the running_mean and running_var
computation. Default: 0.1
affine (bool): A boolean value that when set to ``True``, gives the layer learnable
affine parameters. Default: ``True``
track_running_stats (bool): A boolean value that when set to True, this module tracks
the running mean and variance, and when set to False, this module does not
track such statistics, and initializes statistics buffers running_mean and
running_var as None. When these buffers are None, this module always uses
batch statistics in both training and eval modes. Default: True
p (float): The probability to choose the second branch (usual BN). Default: 0.5
Shape:
- Input: :math:`(b, c, d, h, w)`
- Output: :math:`(b, c, d, h, w)` (same shape as input)
"""
def _check_input_dim(self, input):
if input.dim() != 5:
raise ValueError('expected 4D input (got {}D input)'
.format(input.dim()))
def convert_model(module, p):
"""
Traverses the input module and its child recursively and replaces all
instance of BatchNorm to StochNorm.
Args:
module (torch.nn.Module): The input module needs to be convert to StochNorm model.
p (float): The hyper-parameter for StochNorm layer.
Returns:
The module converted to StochNorm version.
"""
mod = module
for pth_module, stoch_module in zip([torch.nn.modules.batchnorm.BatchNorm1d,
torch.nn.modules.batchnorm.BatchNorm2d,
torch.nn.modules.batchnorm.BatchNorm3d],
[StochNorm1d,
StochNorm2d,
StochNorm3d]):
if isinstance(module, pth_module):
mod = stoch_module(module.num_features, module.eps, module.momentum, module.affine, p)
mod.running_mean = module.running_mean
mod.running_var = module.running_var
if module.affine:
mod.weight.data = module.weight.data.clone().detach()
mod.bias.data = module.bias.data.clone().detach()
for name, child in module.named_children():
mod.add_module(name, convert_model(child, p))
return mod
================================================
FILE: domain_generalization/baseline.py
================================================
"""
@author: Baixu Chen
@contact: cbx_99_hasta@outlook.com
"""
import random
import time
import warnings
import sys
import argparse
import shutil
import os.path as osp
import torch
import torch.nn as nn
import torch.backends.cudnn as cudnn
from torch.optim import SGD
from torch.optim.lr_scheduler import CosineAnnealingLR
from torch.utils.data import DataLoader
import torch.nn.functional as F
sys.path.append('../../..')
from dglib.modules.classifier import ImageClassifier as Classifier
from common.utils.data import ForeverDataIterator
from common.utils.metric import accuracy
from common.utils.meter import AverageMeter, ProgressMeter
from common.utils.logger import CompleteLogger
from common.utils.analysis import tsne, a_distance
sys.path.append('.')
import utils
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
class TransformerDecorator(torch.nn.Module):
def __init__(self, backbone=None, add_bt=3, dim=2048, eval_global=0):
super(TransformerDecorator, self).__init__()
self.encoder_layers = torch.nn.TransformerEncoderLayer(dim, 4, dim, 0.5)
self.eval_global = eval_global
self.add_bt = add_bt
self.backbone = backbone
def forward_feats(self, feature):
feature = self.backbone(feature)
if self.training and self.add_bt > 0:
pre_feature = feature
feature = feature.unsqueeze(1)
feature = self.encoder_layers(feature)
# feature = self.encoder_layers(feature, feature, feature)
feature = feature.squeeze(1)
if self.add_bt:
return torch.cat([pre_feature, feature], dim=0)
return feature
def forward(self, feature):
feature = self.forward_feats(feature)
return feature
def main(args: argparse.Namespace):
logger = CompleteLogger(args.log, args.phase)
print(args)
if args.seed is not None:
random.seed(args.seed)
torch.manual_seed(args.seed)
cudnn.deterministic = True
warnings.warn('You have chosen to seed training. '
'This will turn on the CUDNN deterministic setting, '
'which can slow down your training considerably! '
'You may see unexpected behavior when restarting '
'from checkpoints.')
cudnn.benchmark = True
# Data loading code
train_transform = utils.get_train_transform(args.train_resizing, random_horizontal_flip=True,
random_color_jitter=True, random_gray_scale=True)
val_transform = utils.get_val_transform(args.val_resizing)
print("train_transform: ", train_transform)
print("val_transform: ", val_transform)
train_dataset, num_classes = utils.get_dataset(dataset_name=args.data, root=args.root, task_list=args.sources,
split='train', download=True, transform=train_transform,
seed=args.seed)
train_loader = DataLoader(train_dataset, batch_size=args.batch_size,
shuffle=True, num_workers=args.workers, drop_last=True)
val_dataset, _ = utils.get_dataset(dataset_name=args.data, root=args.root, task_list=args.sources, split='val',
download=True, transform=val_transform, seed=args.seed)
val_loader = DataLoader(val_dataset, batch_size=args.batch_size, shuffle=False, num_workers=args.workers)
test_dataset, _ = utils.get_dataset(dataset_name=args.data, root=args.root, task_list=args.targets, split='test',
download=True, transform=val_transform, seed=args.seed)
test_loader = DataLoader(test_dataset, batch_size=args.batch_size, shuffle=False, num_workers=args.workers)
print("train_dataset_size: ", len(train_dataset))
print('val_dataset_size: ', len(val_dataset))
print("test_dataset_size: ", len(test_dataset))
train_iter = ForeverDataIterator(train_loader)
# create model
print("=> using pre-trained model '{}'".format(args.arch))
backbone = utils.get_model(args.arch)
pool_layer = nn.Identity() if args.no_pool else None
pool_layer = nn.Sequential(
nn.AdaptiveAvgPool2d(output_size=(1, 1)),
nn.Flatten(),
)
pool_layer = TransformerDecorator(pool_layer, add_bt=args.add_bt, dim=backbone.out_features)
classifier = Classifier(backbone, num_classes, freeze_bn=args.freeze_bn, dropout_p=args.dropout_p,
finetune=args.finetune, pool_layer=pool_layer).to(device)
# define optimizer and lr scheduler
optimizer = SGD(classifier.get_parameters(base_lr=args.lr), args.lr, momentum=args.momentum, weight_decay=args.wd,
nesterov=True)
lr_scheduler = CosineAnnealingLR(optimizer, args.epochs * args.iters_per_epoch)
# resume from the best checkpoint
if args.phase != 'train':
checkpoint = torch.load(logger.get_checkpoint_path('best'), map_location='cpu')
classifier.load_state_dict(checkpoint)
# analysis the model
if args.phase == 'analysis':
# extract features from both domains
feature_extractor = nn.Sequential(classifier.backbone, classifier.pool_layer, classifier.bottleneck).to(device)
source_feature = utils.collect_feature(val_loader, feature_extractor, device, max_num_features=100)
target_feature = utils.collect_feature(test_loader, feature_extractor, device, max_num_features=100)
print(len(source_feature), len(target_feature))
# plot t-SNE
tSNE_filename = osp.join(logger.visualize_directory, 'TSNE.png')
tsne.visualize(source_feature, target_feature, tSNE_filename)
print("Saving t-SNE to", tSNE_filename)
# calculate A-distance, which is a measure for distribution discrepancy
A_distance = a_distance.calculate(source_feature, target_feature, device)
print("A-distance =", A_distance)
return
if args.phase == 'test':
acc1 = utils.validate(test_loader, classifier, args, device)
print(acc1)
return
# start training
best_val_acc1 = 0.
best_test_acc1 = 0.
for epoch in range(args.epochs):
print(lr_scheduler.get_lr())
# train for one epoch
train(train_iter, classifier, optimizer, lr_scheduler, epoch, args)
# evaluate on validation set
print("Evaluate on validation set...")
acc1 = utils.validate(val_loader, classifier, args, device)
# remember best acc@1 and save checkpoint
torch.save(classifier.state_dict(), logger.get_checkpoint_path('latest'))
if acc1 > best_val_acc1:
shutil.copy(logger.get_checkpoint_path('latest'), logger.get_checkpoint_path('best'))
best_val_acc1 = max(acc1, best_val_acc1)
# evaluate on test set
print("Evaluate on test set...")
best_test_acc1 = max(best_test_acc1, utils.validate(test_loader, classifier, args, device))
# evaluate on test set
classifier.load_state_dict(torch.load(logger.get_checkpoint_path('best')))
acc1 = utils.validate(test_loader, classifier, args, device)
print("test acc on test set = {}".format(acc1))
print("oracle acc on test set = {}".format(best_test_acc1))
logger.close()
def train(train_iter: ForeverDataIterator, model: Classifier, optimizer,
lr_scheduler: CosineAnnealingLR, epoch: int, args: argparse.Namespace):
batch_time = AverageMeter('Time', ':4.2f')
data_time = AverageMeter('Data', ':3.1f')
losses = AverageMeter('Loss', ':3.2f')
cls_accs = AverageMeter('Cls Acc', ':3.1f')
progress = ProgressMeter(
args.iters_per_epoch,
[batch_time, data_time, losses, cls_accs],
prefix="Epoch: [{}]".format(epoch))
# switch to train mode
model.train()
end = time.time()
for i in range(args.iters_per_epoch):
x, labels, _ = next(train_iter)
x = x.to(device)
labels = labels.to(device)
# measure data loading time
data_time.update(time.time() - end)
# compute output
y, _ = model(x)
if len(y) > len(labels):
labels = torch.cat([labels, labels], dim=0)
loss = F.cross_entropy(y, labels)
cls_acc = accuracy(y, labels)[0]
losses.update(loss.item(), x.size(0))
cls_accs.update(cls_acc.item(), x.size(0))
# compute gradient and do SGD step
optimizer.zero_grad()
loss.backward()
optimizer.step()
lr_scheduler.step()
# measure elapsed time
batch_time.update(time.time() - end)
end = time.time()
if i % args.print_freq == 0:
progress.display(i)
if __name__ == '__main__':
parser = argparse.ArgumentParser(description='Baseline for Domain Generalization')
# dataset parameters
parser.add_argument('root', metavar='DIR',
help='root path of dataset')
parser.add_argument('-d', '--data', metavar='DATA', default='PACS',
help='dataset: ' + ' | '.join(utils.get_dataset_names()) +
' (default: PACS)')
parser.add_argument('-s', '--sources', nargs='+', default=None,
help='source domain(s)')
parser.add_argument('-t', '--targets', nargs='+', default=None,
help='target domain(s)')
parser.add_argument('--train-resizing', type=str, default='default')
parser.add_argument('--val-resizing', type=str, default='default')
# model parameters
parser.add_argument('-a', '--arch', metavar='ARCH', default='resnet50',
choices=utils.get_model_names(),
help='backbone architecture: ' +
' | '.join(utils.get_model_names()) +
' (default: resnet50)')
parser.add_argument('--no-pool', action='store_true', help='no pool layer after the feature extractor.')
parser.add_argument('--finetune', action='store_true', help='whether use 10x smaller lr for backbone')
parser.add_argument('--freeze-bn', action='store_true', help='whether freeze all bn layers')
parser.add_argument('--dropout-p', type=float, default=0.1, help='only activated when freeze-bn is True')
# training parameters
parser.add_argument('-b', '--batch-size', default=36, type=int,
metavar='N',
help='mini-batch size (default: 36)')
parser.add_argument('--lr', '--learning-rate', default=1e-3, type=float,
metavar='LR', help='initial learning rate', dest='lr')
parser.add_argument('--momentum', default=0.9, type=float, metavar='M',
help='momentum')
parser.add_argument('--wd', '--weight-decay', default=0.0005, type=float,
metavar='W', help='weight decay (default: 5e-4)')
parser.add_argument('-j', '--workers', default=4, type=int, metavar='N',
help='number of data loading workers (default: 4)')
parser.add_argument('--epochs', default=20, type=int, metavar='N',
help='number of total epochs to run')
parser.add_argument('-i', '--iters-per-epoch', default=500, type=int,
help='Number of iterations per epoch')
parser.add_argument('-p', '--print-freq', default=100, type=int,
metavar='N', help='print frequency (default: 100)')
parser.add_argument('--seed', default=0, type=int,
help='seed for initializing training. ')
parser.add_argument("--log", type=str, default='baseline',
help="Where to save logs, checkpoints and debugging images.")
parser.add_argument("--phase", type=str, default='train', choices=['train', 'test', 'analysis'],
help="When phase is 'test', only test the model."
"When phase is 'analysis', only analysis the model.")
parser.add_argument("--add_bt", type=int, default=0,
help="add_bt")
args = parser.parse_args()
main(args)
================================================
FILE: domain_generalization/coral.py
================================================
"""
@author: Baixu Chen
@contact: cbx_99_hasta@outlook.com
"""
import random
import time
import warnings
import sys
import argparse
import shutil
import os.path as osp
import torch
import torch.nn as nn
import torch.backends.cudnn as cudnn
from torch.optim import SGD
from torch.optim.lr_scheduler import CosineAnnealingLR
from torch.utils.data import DataLoader
import torch.nn.functional as F
sys.path.append('../../..')
from dglib.generalization.coral import CorrelationAlignmentLoss
from dglib.modules.sampler import RandomDomainSampler
from dglib.modules.classifier import ImageClassifier as Classifier
from common.utils.data import ForeverDataIterator
from common.utils.metric import accuracy
from common.utils.meter import AverageMeter, ProgressMeter
from common.utils.logger import CompleteLogger
from common.utils.analysis import tsne, a_distance
sys.path.append('.')
import utils
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
class TransformerDecorator(torch.nn.Module):
def __init__(self, backbone=None, add_bt=3, dim=2048, eval_global=0):
super(TransformerDecorator, self).__init__()
self.encoder_layers = torch.nn.TransformerEncoderLayer(dim, 4, dim, 0.5)
self.eval_global = eval_global
self.add_bt = add_bt
self.backbone = backbone
def forward_feats(self, feature):
feature = self.backbone(feature)
if self.training and self.add_bt > 0:
pre_feature = feature
feature = feature.unsqueeze(1)
feature = self.encoder_layers(feature)
# feature = self.encoder_layers(feature, feature, feature)
feature = feature.squeeze(1)
if self.add_bt:
return torch.cat([pre_feature, feature], dim=0)
return feature
def forward(self, feature):
feature = self.forward_feats(feature)
return feature
def main(args: argparse.Namespace):
logger = CompleteLogger(args.log, args.phase)
print(args)
if args.seed is not None:
random.seed(args.seed)
torch.manual_seed(args.seed)
cudnn.deterministic = True
warnings.warn('You have chosen to seed training. '
'This will turn on the CUDNN deterministic setting, '
'which can slow down your training considerably! '
'You may see unexpected behavior when restarting '
'from checkpoints.')
cudnn.benchmark = True
# Data loading code
train_transform = utils.get_train_transform(args.train_resizing, random_horizontal_flip=True,
random_color_jitter=True, random_gray_scale=True)
val_transform = utils.get_val_transform(args.val_resizing)
print("train_transform: ", train_transform)
print("val_transform: ", val_transform)
train_dataset, num_classes = utils.get_dataset(dataset_name=args.data, root=args.root, task_list=args.sources,
split='train', download=True, transform=train_transform,
seed=args.seed)
sampler = RandomDomainSampler(train_dataset, args.batch_size, n_domains_per_batch=args.n_domains_per_batch)
train_loader = DataLoader(train_dataset, batch_size=args.batch_size, num_workers=args.workers,
sampler=sampler, drop_last=True)
val_dataset, _ = utils.get_dataset(dataset_name=args.data, root=args.root, task_list=args.sources, split='val',
download=True, transform=val_transform, seed=args.seed)
val_loader = DataLoader(val_dataset, batch_size=args.batch_size, shuffle=False, num_workers=args.workers)
test_dataset, _ = utils.get_dataset(dataset_name=args.data, root=args.root, task_list=args.targets, split='test',
download=True, transform=val_transform, seed=args.seed)
test_loader = DataLoader(test_dataset, batch_size=args.batch_size, shuffle=False, num_workers=args.workers)
print("train_dataset_size: ", len(train_dataset))
print('val_dataset_size: ', len(val_dataset))
print("test_dataset_size: ", len(test_dataset))
train_iter = ForeverDataIterator(train_loader)
# create model
print("=> using pre-trained model '{}'".format(args.arch))
backbone = utils.get_model(args.arch)
pool_layer = nn.Identity() if args.no_pool else None
pool_layer = nn.Sequential(
nn.AdaptiveAvgPool2d(output_size=(1, 1)),
nn.Flatten(),
)
pool_layer = TransformerDecorator(pool_layer, add_bt=args.add_bt, dim=backbone.out_features)
classifier = Classifier(backbone, num_classes, freeze_bn=args.freeze_bn, dropout_p=args.dropout_p,
finetune=args.finetune, pool_layer=pool_layer).to(device)
# define optimizer and lr scheduler
optimizer = SGD(classifier.get_parameters(base_lr=args.lr), args.lr, momentum=args.momentum, weight_decay=args.wd,
nesterov=True)
lr_scheduler = CosineAnnealingLR(optimizer, args.epochs * args.iters_per_epoch)
# define loss function
correlation_alignment_loss = CorrelationAlignmentLoss().to(device)
# resume from the best checkpoint
if args.phase != 'train':
checkpoint = torch.load(logger.get_checkpoint_path('best'), map_location='cpu')
classifier.load_state_dict(checkpoint)
# analysis the model
if args.phase == 'analysis':
# extract features from both domains
feature_extractor = nn.Sequential(classifier.backbone, classifier.pool_layer, classifier.bottleneck).to(device)
source_feature = utils.collect_feature(val_loader, feature_extractor, device, max_num_features=100)
target_feature = utils.collect_feature(test_loader, feature_extractor, device, max_num_features=100)
print(len(source_feature), len(target_feature))
# plot t-SNE
tSNE_filename = osp.join(logger.visualize_directory, 'TSNE.png')
tsne.visualize(source_feature, target_feature, tSNE_filename)
print("Saving t-SNE to", tSNE_filename)
# calculate A-distance, which is a measure for distribution discrepancy
A_distance = a_distance.calculate(source_feature, target_feature, device)
print("A-distance =", A_distance)
return
if args.phase == 'test':
acc1 = utils.validate(test_loader, classifier, args, device)
print(acc1)
return
# start training
best_val_acc1 = 0.
best_test_acc1 = 0.
for epoch in range(args.epochs):
print(lr_scheduler.get_lr())
# train for one epoch
train(train_iter, classifier, optimizer, lr_scheduler, correlation_alignment_loss, args.n_domains_per_batch,
epoch, args)
# evaluate on validation set
print("Evaluate on validation set...")
acc1 = utils.validate(val_loader, classifier, args, device)
# remember best acc@1 and save checkpoint
torch.save(classifier.state_dict(), logger.get_checkpoint_path('latest'))
if acc1 > best_val_acc1:
shutil.copy(logger.get_checkpoint_path('latest'), logger.get_checkpoint_path('best'))
best_val_acc1 = max(acc1, best_val_acc1)
# evaluate on test set
print("Evaluate on test set...")
best_test_acc1 = max(best_test_acc1, utils.validate(test_loader, classifier, args, device))
# evaluate on test set
classifier.load_state_dict(torch.load(logger.get_checkpoint_path('best')))
acc1 = utils.validate(test_loader, classifier, args, device)
print("test acc on test set = {}".format(acc1))
print("oracle acc on test set = {}".format(best_test_acc1))
logger.close()
def train(train_iter: ForeverDataIterator, model: Classifier, optimizer, lr_scheduler: CosineAnnealingLR,
correlation_alignment_loss: CorrelationAlignmentLoss, n_domains_per_batch: int, epoch: int,
args: argparse.Namespace):
batch_time = AverageMeter('Time', ':4.2f')
data_time = AverageMeter('Data', ':3.1f')
losses = AverageMeter('Loss', ':3.2f')
losses_ce = AverageMeter('CELoss', ':3.2f')
losses_penalty = AverageMeter('Penalty Loss', ':3.2f')
cls_accs = AverageMeter('Cls Acc', ':3.1f')
progress = ProgressMeter(
args.iters_per_epoch,
[batch_time, data_time, losses, losses_ce, losses_penalty, cls_accs],
prefix="Epoch: [{}]".format(epoch))
# switch to train mode
model.train()
end = time.time()
for i in range(args.iters_per_epoch):
x_all, labels_all, _ = next(train_iter)
x_all = x_all.to(device)
labels_all = labels_all.to(device)
# compute output
y_all, f_all = model(x_all)
# measure data loading time
data_time.update(time.time() - end)
if len(y_all) > len(labels_all):
labels_all = torch.cat([labels_all, labels_all], dim=0)
y_all_ = y_all.chunk(n_domains_per_batch*2, dim=0)
f_all_ = f_all.chunk(n_domains_per_batch*2, dim=0)
labels_all_ = labels_all.chunk(n_domains_per_batch*2, dim=0)
y_all = []
f_all = []
labels_all = []
for i in range(n_domains_per_batch):
y_all.append(torch.cat([y_all_[i], y_all_[i+n_domains_per_batch]], dim=0))
f_all.append(torch.cat([f_all_[i], f_all_[i+n_domains_per_batch]], dim=0))
labels_all.append(torch.cat([labels_all_[i], labels_all_[i+n_domains_per_batch]], dim=0))
else:
# separate into different domains
y_all = y_all.chunk(n_domains_per_batch, dim=0)
f_all = f_all.chunk(n_domains_per_batch, dim=0)
labels_all = labels_all.chunk(n_domains_per_batch, dim=0)
loss_ce = 0
loss_penalty = 0
cls_acc = 0
for domain_i in range(n_domains_per_batch):
# cls loss
y_i, labels_i = y_all[domain_i], labels_all[domain_i]
loss_ce += F.cross_entropy(y_i, labels_i)
# update acc
cls_acc += accuracy(y_i, labels_i)[0] / n_domains_per_batch
# correlation alignment loss
for domain_j in range(domain_i + 1, n_domains_per_batch):
f_i = f_all[domain_i]
f_j = f_all[domain_j]
loss_penalty += correlation_alignment_loss(f_i, f_j)
# normalize loss
loss_ce /= n_domains_per_batch
loss_penalty /= n_domains_per_batch * (n_domains_per_batch - 1) / 2
loss = loss_ce + loss_penalty * args.trade_off
losses.update(loss.item(), x_all.size(0))
losses_ce.update(loss_ce.item(), x_all.size(0))
losses_penalty.update(loss_penalty.item(), x_all.size(0))
cls_accs.update(cls_acc.item(), x_all.size(0))
# compute gradient and do SGD step
optimizer.zero_grad()
loss.backward()
optimizer.step()
lr_scheduler.step()
# measure elapsed time
batch_time.update(time.time() - end)
end = time.time()
if i % args.print_freq == 0:
progress.display(i)
if __name__ == '__main__':
parser = argparse.ArgumentParser(description='CORAL for Domain Generalization')
# dataset parameters
parser.add_argument('root', metavar='DIR',
help='root path of dataset')
parser.add_argument('-d', '--data', metavar='DATA', default='PACS',
help='dataset: ' + ' | '.join(utils.get_dataset_names()) +
' (default: PACS)')
parser.add_argument('-s', '--sources', nargs='+', default=None,
help='source domain(s)')
parser.add_argument('-t', '--targets', nargs='+', default=None,
help='target domain(s)')
parser.add_argument('--train-resizing', type=str, default='default')
parser.add_argument('--val-resizing', type=str, default='default')
# model parameters
parser.add_argument('-a', '--arch', metavar='ARCH', default='resnet50',
choices=utils.get_model_names(),
help='backbone architecture: ' +
' | '.join(utils.get_model_names()) +
' (default: resnet50)')
parser.add_argument('--no-pool', action='store_true', help='no pool layer after the feature extractor.')
parser.add_argument('--finetune', action='store_true', help='whether use 10x smaller lr for backbone')
parser.add_argument('--freeze-bn', action='store_true', help='whether freeze all bn layers')
parser.add_argument('--dropout-p', type=float, default=0.1, help='only activated when freeze-bn is True')
# training parameters
parser.add_argument('--trade-off', default=1, type=float,
help='the trade off hyper parameter for correlation alignment loss')
parser.add_argument('-b', '--batch-size', default=36, type=int,
metavar='N',
help='mini-batch size (default: 36)')
parser.add_argument('--n-domains-per-batch', default=3, type=int,
help='number of domains in each mini-batch')
parser.add_argument('--lr', '--learning-rate', default=5e-4, type=float,
metavar='LR', help='initial learning rate', dest='lr')
parser.add_argument('--momentum', default=0.9, type=float, metavar='M',
help='momentum')
parser.add_argument('--wd', '--weight-decay', default=0.0005, type=float,
metavar='W', help='weight decay (default: 5e-4)')
parser.add_argument('-j', '--workers', default=4, type=int, metavar='N',
help='number of data loading workers (default: 4)')
parser.add_argument('--epochs', default=20, type=int, metavar='N',
help='number of total epochs to run')
parser.add_argument('-i', '--iters-per-epoch', default=500, type=int,
help='Number of iterations per epoch')
parser.add_argument('-p', '--print-freq', default=100, type=int,
metavar='N', help='print frequency (default: 100)')
parser.add_argument('--seed', default=None, type=int,
help='seed for initializing training. ')
parser.add_argument("--log", type=str, default='coral',
help="Where to save logs, checkpoints and debugging images.")
parser.add_argument("--phase", type=str, default='train', choices=['train', 'test', 'analysis'],
help="When phase is 'test', only test the model."
"When phase is 'analysis', only analysis the model.")
parser.add_argument("--add_bt", type=int, default=0,
help="add_bt")
args = parser.parse_args()
main(args)
================================================
FILE: domain_generalization/irm.py
================================================
"""
@author: Baixu Chen
@contact: cbx_99_hasta@outlook.com
"""
import random
import time
import warnings
import sys
import argparse
import shutil
import os.path as osp
import torch
import torch.nn as nn
import torch.backends.cudnn as cudnn
from torch.optim import SGD
from torch.optim.lr_scheduler import CosineAnnealingLR
from torch.utils.data import DataLoader
import torch.nn.functional as F
sys.path.append('../../..')
from dglib.generalization.irm import InvariancePenaltyLoss
from dglib.modules.sampler import RandomDomainSampler
from dglib.modules.classifier import ImageClassifier as Classifier
from common.utils.data import ForeverDataIterator
from common.utils.metric import accuracy
from common.utils.meter import AverageMeter, ProgressMeter
from common.utils.logger import CompleteLogger
from common.utils.analysis import tsne, a_distance
sys.path.append('.')
import utils
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
class TransformerDecorator(torch.nn.Module):
def __init__(self, backbone=None, add_bt=3, dim=2048, eval_global=0):
super(TransformerDecorator, self).__init__()
self.eval_global = eval_global
self.add_bt = add_bt
self.backbone = backbone
if self.add_bt > 0:
self.encoder_layers = torch.nn.TransformerEncoderLayer(dim, 4, dim, 0.5)
def forward(self, feature):
feature = self.backbone(feature)
if self.training and self.add_bt > 0:
pre_feature = feature
feature = feature.unsqueeze(1)
feature = self.encoder_layers(feature)
# feature = self.encoder_layers(feature, feature, feature)
feature = feature.squeeze(1)
if self.add_bt:
return torch.cat([pre_feature, feature], dim=0)
return feature
def main(args: argparse.Namespace):
logger = CompleteLogger(args.log, args.phase)
print(args)
if args.seed is not None:
random.seed(args.seed)
torch.manual_seed(args.seed)
cudnn.deterministic = True
warnings.warn('You have chosen to seed training. '
'This will turn on the CUDNN deterministic setting, '
'which can slow down your training considerably! '
'You may see unexpected behavior when restarting '
'from checkpoints.')
cudnn.benchmark = True
# Data loading code
train_transform = utils.get_train_transform(args.train_resizing, random_horizontal_flip=True,
random_color_jitter=True, random_gray_scale=True)
val_transform = utils.get_val_transform(args.val_resizing)
print("train_transform: ", train_transform)
print("val_transform: ", val_transform)
train_dataset, num_classes = utils.get_dataset(dataset_name=args.data, root=args.root, task_list=args.sources,
split='train', download=True, transform=train_transform,
seed=args.seed)
sampler = RandomDomainSampler(train_dataset, args.batch_size, n_domains_per_batch=args.n_domains_per_batch)
train_loader = DataLoader(train_dataset, batch_size=args.batch_size, num_workers=args.workers,
sampler=sampler, drop_last=True)
val_dataset, _ = utils.get_dataset(dataset_name=args.data, root=args.root, task_list=args.sources, split='val',
download=True, transform=val_transform, seed=args.seed)
val_loader = DataLoader(val_dataset, batch_size=args.batch_size, shuffle=False, num_workers=args.workers)
test_dataset, _ = utils.get_dataset(dataset_name=args.data, root=args.root, task_list=args.targets, split='test',
download=True, transform=val_transform, seed=args.seed)
test_loader = DataLoader(test_dataset, batch_size=args.batch_size, shuffle=False, num_workers=args.workers)
print("train_dataset_size: ", len(train_dataset))
print('val_dataset_size: ', len(val_dataset))
print("test_dataset_size: ", len(test_dataset))
train_iter = ForeverDataIterator(train_loader)
# create model
print("=> using pre-trained model '{}'".format(args.arch))
backbone = utils.get_model(args.arch)
pool_layer = nn.Identity() if args.no_pool else None
pool_layer = nn.Sequential(
nn.AdaptiveAvgPool2d(output_size=(1, 1)),
nn.Flatten(),
)
pool_layer = TransformerDecorator(pool_layer, add_bt=args.add_bt, dim=backbone.out_features)
classifier = Classifier(backbone, num_classes, freeze_bn=args.freeze_bn, dropout_p=args.dropout_p,
finetune=args.finetune, pool_layer=pool_layer).to(device)
# define optimizer and lr scheduler
optimizer = SGD(classifier.get_parameters(base_lr=args.lr), args.lr, momentum=args.momentum, weight_decay=args.wd,
nesterov=True)
lr_scheduler = CosineAnnealingLR(optimizer, args.epochs * args.iters_per_epoch)
# define loss function
invariance_penalty_loss = InvariancePenaltyLoss().to(device)
# for simplicity
assert args.anneal_iters % args.iters_per_epoch == 0
# resume from the best checkpoint
if args.phase != 'train':
checkpoint = torch.load(logger.get_checkpoint_path('best'), map_location='cpu')
classifier.load_state_dict(checkpoint)
# analysis the model
if args.phase == 'analysis':
# extract features from both domains
feature_extractor = nn.Sequential(classifier.backbone, classifier.pool_layer, classifier.bottleneck).to(device)
source_feature = utils.collect_feature(val_loader, feature_extractor, device, max_num_features=100)
target_feature = utils.collect_feature(test_loader, feature_extractor, device, max_num_features=100)
print(len(source_feature), len(target_feature))
# plot t-SNE
tSNE_filename = osp.join(logger.visualize_directory, 'TSNE.png')
tsne.visualize(source_feature, target_feature, tSNE_filename)
print("Saving t-SNE to", tSNE_filename)
# calculate A-distance, which is a measure for distribution discrepancy
A_distance = a_distance.calculate(source_feature, target_feature, device)
print("A-distance =", A_distance)
return
if args.phase == 'test':
acc1 = utils.validate(test_loader, classifier, args, device)
print(acc1)
return
# start training
best_val_acc1 = 0.
best_test_acc1 = 0.
for epoch in range(args.epochs):
if epoch * args.iters_per_epoch == args.anneal_iters:
# reset optimizer to avoid sharp jump in gradient magnitudes
optimizer = SGD(classifier.get_parameters(base_lr=args.lr), args.lr, momentum=args.momentum,
weight_decay=args.wd, nesterov=True)
lr_scheduler = CosineAnnealingLR(optimizer, args.epochs * args.iters_per_epoch - args.anneal_iters)
print(lr_scheduler.get_lr())
# train for one epoch
train(train_iter, classifier, optimizer, lr_scheduler, invariance_penalty_loss, args.n_domains_per_batch, epoch,
args)
# evaluate on validation set
print("Evaluate on validation set...")
acc1 = utils.validate(val_loader, classifier, args, device)
# remember best acc@1 and save checkpoint
torch.save(classifier.state_dict(), logger.get_checkpoint_path('latest'))
if acc1 > best_val_acc1:
shutil.copy(logger.get_checkpoint_path('latest'), logger.get_checkpoint_path('best'))
best_val_acc1 = max(acc1, best_val_acc1)
# evaluate on test set
print("Evaluate on test set...")
best_test_acc1 = max(best_test_acc1, utils.validate(test_loader, classifier, args, device))
# evaluate on test set
classifier.load_state_dict(torch.load(logger.get_checkpoint_path('best')))
acc1 = utils.validate(test_loader, classifier, args, device)
print("test acc on test set = {}".format(acc1))
print("oracle acc on test set = {}".format(best_test_acc1))
logger.close()
def train(train_iter: ForeverDataIterator, model: Classifier, optimizer, lr_scheduler: CosineAnnealingLR,
invariance_penalty_loss: InvariancePenaltyLoss, n_domains_per_batch: int, epoch: int,
args: argparse.Namespace):
batch_time = AverageMeter('Time', ':4.2f')
data_time = AverageMeter('Data', ':3.1f')
losses = AverageMeter('Loss', ':3.2f')
losses_ce = AverageMeter('CELoss', ':3.2f')
losses_penalty = AverageMeter('Penalty Loss', ':3.2f')
cls_accs = AverageMeter('Cls Acc', ':3.1f')
progress = ProgressMeter(
args.iters_per_epoch,
[batch_time, data_time, losses, losses_ce, losses_penalty, cls_accs],
prefix="Epoch: [{}]".format(epoch))
# switch to train mode
model.train()
end = time.time()
for i in range(args.iters_per_epoch):
x_all, labels_all, _ = next(train_iter)
x_all = x_all.to(device)
labels_all = labels_all.to(device)
# measure data loading time
data_time.update(time.time() - end)
# compute output
y_all, _ = model(x_all)
if len(y_all) > len(labels_all):
labels_all = torch.cat([labels_all, labels_all], dim=0)
# cls loss
loss_ce = F.cross_entropy(y_all, labels_all)
# penalty loss
loss_penalty = 0
for y_per_domain, labels_per_domain in zip(y_all.chunk(n_domains_per_batch, dim=0),
labels_all.chunk(n_domains_per_batch, dim=0)):
# normalize loss by domain num
loss_penalty += invariance_penalty_loss(y_per_domain, labels_per_domain) / n_domains_per_batch
global_iter = epoch * args.iters_per_epoch + i
if global_iter >= args.anneal_iters:
trade_off = args.trade_off
else:
trade_off = 1
loss = loss_ce + loss_penalty * trade_off
cls_acc = accuracy(y_all, labels_all)[0]
losses.update(loss.item(), x_all.size(0))
losses_ce.update(loss_ce.item(), x_all.size(0))
losses_penalty.update(loss_penalty.item(), x_all.size(0))
cls_accs.update(cls_acc.item(), x_all.size(0))
# compute gradient and do SGD step
optimizer.zero_grad()
loss.backward()
optimizer.step()
lr_scheduler.step()
# measure elapsed time
batch_time.update(time.time() - end)
end = time.time()
if i % args.print_freq == 0:
progress.display(i)
if __name__ == '__main__':
parser = argparse.ArgumentParser(description='IRM for Domain Generalization')
# dataset parameters
parser.add_argument('root', metavar='DIR',
help='root path of dataset')
parser.add_argument('-d', '--data', metavar='DATA', default='PACS',
help='dataset: ' + ' | '.join(utils.get_dataset_names()) +
' (default: PACS)')
parser.add_argument('-s', '--sources', nargs='+', default=None,
help='source domain(s)')
parser.add_argument('-t', '--targets', nargs='+', default=None,
help='target domain(s)')
parser.add_argument('--train-resizing', type=str, default='default')
parser.add_argument('--val-resizing', type=str, default='default')
# model parameters
parser.add_argument('-a', '--arch', metavar='ARCH', default='resnet50',
choices=utils.get_model_names(),
help='backbone architecture: ' +
' | '.join(utils.get_model_names()) +
' (default: resnet50)')
parser.add_argument('--no-pool', action='store_true', help='no pool layer after the feature extractor.')
parser.add_argument('--finetune', action='store_true', help='whether use 10x smaller lr for backbone')
parser.add_argument('--freeze-bn', action='store_true', help='whether freeze all bn layers')
parser.add_argument('--dropout-p', type=float, default=0.1, help='only activated when freeze-bn is True')
# training parameters
parser.add_argument('--trade-off', default=1, type=float,
help='the trade off hyper parameter for irm penalty')
parser.add_argument('--anneal-iters', default=500, type=int,
help='anneal iterations (trade off is set to 1 during these iterations)')
parser.add_argument('-b', '--batch-size', default=36, type=int,
metavar='N',
help='mini-batch size (default: 36)')
parser.add_argument('--n-domains-per-batch', default=3, type=int,
help='number of domains in each mini-batch')
parser.add_argument('--lr', '--learning-rate', default=5e-4, type=float,
metavar='LR', help='initial learning rate', dest='lr')
parser.add_argument('--momentum', default=0.9, type=float, metavar='M',
help='momentum')
parser.add_argument('--wd', '--weight-decay', default=0.0005, type=float,
metavar='W', help='weight decay (default: 5e-4)')
parser.add_argument('-j', '--workers', default=4, type=int, metavar='N',
help='number of data loading workers (default: 4)')
parser.add_argument('--epochs', default=20, type=int, metavar='N',
help='number of total epochs to run')
parser.add_argument('-i', '--iters-per-epoch', default=500, type=int,
help='Number of iterations per epoch')
parser.add_argument('-p', '--print-freq', default=100, type=int,
metavar='N', help='print frequency (default: 100)')
parser.add_argument('--seed', default=None, type=int,
help='seed for initializing training. ')
parser.add_argument("--log", type=str, default='irm',
help="Where to save logs, checkpoints and debugging images.")
parser.add_argument("--phase", type=str, default='train', choices=['train', 'test', 'analysis'],
help="When phase is 'test', only test the model."
"When phase is 'analysis', only analysis the model.")
parser.add_argument("--add_global", type=int, default=0,
help="add_global")
args = parser.parse_args()
main(args)
================================================
FILE: domain_generalization/mixstyle.py
================================================
"""
@author: Baixu Chen
@contact: cbx_99_hasta@outlook.com
"""
import random
import time
import warnings
import sys
import argparse
import shutil
import os.path as osp
import torch
import torch.nn as nn
import torch.backends.cudnn as cudnn
from torch.optim import SGD
from torch.optim.lr_scheduler import CosineAnnealingLR
from torch.utils.data import DataLoader
import torch.nn.functional as F
sys.path.append('../../..')
import dglib.generalization.mixstyle.models as models
from dglib.modules.sampler import RandomDomainSampler
from dglib.modules.classifier import ImageClassifier as Classifier
from common.utils.data import ForeverDataIterator
from common.utils.metric import accuracy
from common.utils.meter import AverageMeter, ProgressMeter
from common.utils.logger import CompleteLogger
from common.utils.analysis import tsne, a_distance
sys.path.append('.')
import utils
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
class TransformerDecorator(torch.nn.Module):
def __init__(self, backbone=None, add_bt=3, dim=2048, eval_global=0):
super(TransformerDecorator, self).__init__()
self.eval_global = eval_global
self.add_bt = add_bt
self.backbone = backbone
self.encoder_layers = torch.nn.TransformerEncoderLayer(dim, 4, dim, 0.5)
def forward(self, feature):
feature = self.backbone(feature)
if self.training and self.add_bt > 0:
pre_feature = feature
feature = feature.unsqueeze(1)
feature = self.encoder_layers(feature)
# feature = self.encoder_layers(feature, feature, feature)
feature = feature.squeeze(1)
if self.add_bt:
return torch.cat([pre_feature, feature], dim=0)
return feature
def main(args: argparse.Namespace):
logger = CompleteLogger(args.log, args.phase)
print(args)
if args.seed is not None:
random.seed(args.seed)
torch.manual_seed(args.seed)
cudnn.deterministic = True
warnings.warn('You have chosen to seed training. '
'This will turn on the CUDNN deterministic setting, '
'which can slow down your training considerably! '
'You may see unexpected behavior when restarting '
'from checkpoints.')
cudnn.benchmark = True
# Data loading code
train_transform = utils.get_train_transform(args.train_resizing, random_horizontal_flip=True,
random_color_jitter=True, random_gray_scale=True)
val_transform = utils.get_val_transform(args.val_resizing)
print("train_transform: ", train_transform)
print("val_transform: ", val_transform)
train_dataset, num_classes = utils.get_dataset(dataset_name=args.data, root=args.root, task_list=args.sources,
split='train', download=True, transform=train_transform,
seed=args.seed)
sampler = RandomDomainSampler(train_dataset, args.batch_size, n_domains_per_batch=2)
train_loader = DataLoader(train_dataset, batch_size=args.batch_size, num_workers=args.workers,
sampler=sampler, drop_last=True)
val_dataset, _ = utils.get_dataset(dataset_name=args.data, root=args.root, task_list=args.sources, split='val',
download=True, transform=val_transform, seed=args.seed)
val_loader = DataLoader(val_dataset, batch_size=args.batch_size, shuffle=False, num_workers=args.workers)
test_dataset, _ = utils.get_dataset(dataset_name=args.data, root=args.root, task_list=args.targets, split='test',
download=True, transform=val_transform, seed=args.seed)
test_loader = DataLoader(test_dataset, batch_size=args.batch_size, shuffle=False, num_workers=args.workers)
print("train_dataset_size: ", len(train_dataset))
print('val_dataset_size: ', len(val_dataset))
print("test_dataset_size: ", len(test_dataset))
train_iter = ForeverDataIterator(train_loader)
# create model
print("=> using pre-trained model '{}'".format(args.arch))
backbone = models.__dict__[args.arch](mix_layers=args.mix_layers, mix_p=args.mix_p, mix_alpha=args.mix_alpha,
pretrained=True)
pool_layer = nn.Identity() if args.no_pool else None
pool_layer = nn.Sequential(
nn.AdaptiveAvgPool2d(output_size=(1, 1)),
nn.Flatten(),
)
pool_layer = TransformerDecorator(pool_layer, add_bt=args.add_bt, dim=backbone.out_features)
classifier = Classifier(backbone, num_classes, freeze_bn=args.freeze_bn, dropout_p=args.dropout_p,
finetune=args.finetune, pool_layer=pool_layer).to(device)
# define optimizer and lr scheduler
optimizer = SGD(classifier.get_parameters(base_lr=args.lr), args.lr, momentum=args.momentum, weight_decay=args.wd,
nesterov=True)
lr_scheduler = CosineAnnealingLR(optimizer, args.epochs * args.iters_per_epoch)
# resume from the best checkpoint
if args.phase != 'train':
checkpoint = torch.load(logger.get_checkpoint_path('best'), map_location='cpu')
classifier.load_state_dict(checkpoint)
# analysis the model
if args.phase == 'analysis':
# extract features from both domains
feature_extractor = nn.Sequential(classifier.backbone, classifier.pool_layer, classifier.bottleneck).to(device)
source_feature = utils.collect_feature(val_loader, feature_extractor, device, max_num_features=100)
target_feature = utils.collect_feature(test_loader, feature_extractor, device, max_num_features=100)
print(len(source_feature), len(target_feature))
# plot t-SNE
tSNE_filename = osp.join(logger.visualize_directory, 'TSNE.png')
tsne.visualize(source_feature, target_feature, tSNE_filename)
print("Saving t-SNE to", tSNE_filename)
# calculate A-distance, which is a measure for distribution discrepancy
A_distance = a_distance.calculate(source_feature, target_feature, device)
print("A-distance =", A_distance)
return
if args.phase == 'test':
acc1 = utils.validate(test_loader, classifier, args, device)
print(acc1)
return
# start training
best_val_acc1 = 0.
best_test_acc1 = 0.
for epoch in range(args.epochs):
print(lr_scheduler.get_lr())
# train for one epoch
train(train_iter, classifier, optimizer, lr_scheduler, epoch, args)
# evaluate on validation set
print("Evaluate on validation set...")
acc1 = utils.validate(val_loader, classifier, args, device)
# remember best acc@1 and save checkpoint
torch.save(classifier.state_dict(), logger.get_checkpoint_path('latest'))
if acc1 > best_val_acc1:
shutil.copy(logger.get_checkpoint_path('latest'), logger.get_checkpoint_path('best'))
best_val_acc1 = max(acc1, best_val_acc1)
# evaluate on test set
print("Evaluate on test set...")
best_test_acc1 = max(best_test_acc1, utils.validate(test_loader, classifier, args, device))
# evaluate on test set
classifier.load_state_dict(torch.load(logger.get_checkpoint_path('best')))
acc1 = utils.validate(test_loader, classifier, args, device)
print("test acc on test set = {}".format(acc1))
print("oracle acc on test set = {}".format(best_test_acc1))
logger.close()
def train(train_iter: ForeverDataIterator, model: Classifier, optimizer,
lr_scheduler: CosineAnnealingLR, epoch: int, args: argparse.Namespace):
batch_time = AverageMeter('Time', ':4.2f')
data_time = AverageMeter('Data', ':3.1f')
losses = AverageMeter('Loss', ':3.2f')
cls_accs = AverageMeter('Cls Acc', ':3.1f')
progress = ProgressMeter(
args.iters_per_epoch,
[batch_time, data_time, losses, cls_accs],
prefix="Epoch: [{}]".format(epoch))
# switch to train mode
model.train()
end = time.time()
for i in range(args.iters_per_epoch):
x, labels, _ = next(train_iter)
x = x.to(device)
labels = labels.to(device)
# measure data loading time
data_time.update(time.time() - end)
# compute output
y, _ = model(x)
if len(y) > len(labels):
labels = torch.cat([labels, labels], dim=0)
cls_loss = F.cross_entropy(y, labels)
loss = cls_loss
cls_acc = accuracy(y, labels)[0]
losses.update(loss.item(), x.size(0))
cls_accs.update(cls_acc.item(), x.size(0))
# compute gradient and do SGD step
optimizer.zero_grad()
loss.backward()
optimizer.step()
lr_scheduler.step()
# measure elapsed time
batch_time.update(time.time() - end)
end = time.time()
if i % args.print_freq == 0:
progress.display(i)
if __name__ == '__main__':
architecture_names = sorted(
name for name in models.__dict__
if name.islower() and not name.startswith("__")
and callable(models.__dict__[name])
)
parser = argparse.ArgumentParser(description='MixStyle for Domain Generalization')
# dataset parameters
parser.add_argument('root', metavar='DIR',
help='root path of dataset')
parser.add_argument('-d', '--data', metavar='DATA', default='PACS',
help='dataset: ' + ' | '.join(utils.get_dataset_names()) +
' (default: PACS)')
parser.add_argument('-s', '--sources', nargs='+', default=None,
help='source domain(s)')
parser.add_argument('-t', '--targets', nargs='+', default=None,
help='target domain(s)')
parser.add_argument('--train-resizing', type=str, default='default')
parser.add_argument('--val-resizing', type=str, default='default')
# model parameters
parser.add_argument('-a', '--arch', metavar='ARCH', default='resnet50',
choices=architecture_names,
help='backbone architecture: ' +
' | '.join(architecture_names) +
' (default: resnet50)')
parser.add_argument('--no-pool', action='store_true', help='no pool layer after the feature extractor.')
parser.add_argument('--mix-layers', nargs='+', help='layers to apply MixStyle')
parser.add_argument('--mix-p', default=0.5, type=float, help='probability to apply MixStyle')
parser.add_argument('--mix-alpha', default=0.1, type=float, help='parameter alpha for beta distribution')
parser.add_argument('--finetune', action='store_true', help='whether use 10x smaller lr for backbone')
parser.add_argument('--freeze-bn', action='store_true', help='whether freeze all bn layers')
parser.add_argument('--dropout-p', type=float, default=0.1, help='only activated when freeze-bn is True')
# training parameters
parser.add_argument('-b', '--batch-size', default=36, type=int,
metavar='N',
help='mini-batch size (default: 36)')
parser.add_argument('--lr', '--learning-rate', default=1e-3, type=float,
metavar='LR', help='initial learning rate', dest='lr')
parser.add_argument('--momentum', default=0.9, type=float, metavar='M',
help='momentum')
parser.add_argument('--wd', '--weight-decay', default=0.0005, type=float,
metavar='W', help='weight decay (default: 5e-4)')
parser.add_argument('-j', '--workers', default=4, type=int, metavar='N',
help='number of data loading workers (default: 4)')
parser.add_argument('--epochs', default=20, type=int, metavar='N',
help='number of total epochs to run')
parser.add_argument('-i', '--iters-per-epoch', default=500, type=int,
help='Number of iterations per epoch')
parser.add_argument('-p', '--print-freq', default=100, type=int,
metavar='N', help='print frequency (default: 100)')
parser.add_argument('--seed', default=None, type=int,
help='seed for initializing training. ')
parser.add_argument("--log", type=str, default='mixstyle',
help="Where to save logs, checkpoints and debugging images.")
parser.add_argument("--phase", type=str, default='train', choices=['train', 'test', 'analysis'],
help="When phase is 'test', only test the model."
"When phase is 'analysis', only analysis the model.")
parser.add_argument("--add_bt", type=int, default=0,
help="add_bt")
args = parser.parse_args()
main(args)
================================================
FILE: domain_generalization/swad/.gitignore
================================================
__pycache__
tags
train_output/
.vim/
.ipynb_checkpoints/
*.log
*.json
swa_plane_results/
features/
perturb_results/
*.pdf
*.sh
*.ipynb
================================================
FILE: domain_generalization/swad/LICENSE
================================================
MIT License
Copyright (c) 2021 Junbum Cha
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
================================================
FILE: domain_generalization/swad/README.md
================================================
# BatchFormer for SWAD
## Preparation
### Dependencies
```sh
pip install -r requirements.txt
```
### Datasets
```sh
python -m domainbed.scripts.download --data_dir=/my/datasets/path
```
### Environments
Environment details used for our study.
```
Python: 3.8.6
PyTorch: 1.7.0+cu92
Torchvision: 0.8.1+cu92
CUDA: 9.2
CUDNN: 7603
NumPy: 1.19.4
PIL: 8.0.1
```
### How to run
We provide the instructions to reproduce the main results of the paper, Table 1 and 2.
Note that the difference in a detailed environment or uncontrolled randomness may bring a little different result from the paper.
- PACS
```
python train_all.py PACS0 --dataset PACS --deterministic --trial_seed 0 --checkpoint_freq 100 --data_dir /my/datasets/path --resnet18 --add_bt 16
python train_all.py PACS1 --dataset PACS --deterministic --trial_seed 1 --checkpoint_freq 100 --data_dir /my/datasets/path --resnet18 --add_bt 16
python train_all.py PACS2 --dataset PACS --deterministic --trial_seed 2 --checkpoint_freq 100 --data_dir /my/datasets/path --resnet18 --add_bt 16
```
**You can also run the code with "--add_bt 1" which do not include shared classifier. Meanwhile, you can also set a smaller learning rate with "--lr_scale_bt 0.1".
That is also beneficial for some datasets, especially when the backbone is ResNet50.**
## License
This code is based on SWAD.
This source code is released under the MIT license, included [here](./LICENSE).
This project includes some code from [DomainBed](https://github.com/facebookresearch/DomainBed/tree/3fe9d7bb4bc14777a42b3a9be8dd887e709ec414), also MIT licensed.
================================================
FILE: domain_generalization/swad/config.yaml
================================================
# Default update config
# Config order: hparams_registry -> config.yaml -> CLI
scheduler: const
swad: LossValley
swad_kwargs:
n_converge: 3
n_tolerance: 6
tolerance_ratio: 0.3
test_batchsize: 128
indomain_test: False # use indomain test split or not
================================================
FILE: domain_generalization/swad/domainbed/algorithms/__init__.py
================================================
from .algorithms import *
def get_algorithm_class(algorithm_name):
"""Return the algorithm class with the given name."""
if algorithm_name not in globals():
raise NotImplementedError("Algorithm not found: {}".format(algorithm_name))
return globals()[algorithm_name]
================================================
FILE: domain_generalization/swad/domainbed/algorithms/algorithms.py
================================================
# Copyright (c) Facebook, Inc. and its affiliates. All Rights Reserved
import copy
from typing import List
import torch
import torch.nn as nn
import torch.nn.functional as F
import torch.autograd as autograd
import numpy as np
# import higher
from domainbed import networks
from domainbed.lib.misc import random_pairs_of_minibatches
from domainbed.optimizers import get_optimizer
from domainbed.models.resnet_mixstyle import (
resnet18_mixstyle_L234_p0d5_a0d1,
resnet50_mixstyle_L234_p0d5_a0d1,
)
from domainbed.models.resnet_mixstyle2 import (
resnet18_mixstyle2_L234_p0d5_a0d1,
resnet50_mixstyle2_L234_p0d5_a0d1,
)
def to_minibatch(x, y):
minibatches = list(zip(x, y))
return minibatches
class Algorithm(torch.nn.Module):
"""
A subclass of Algorithm implements a domain generalization algorithm.
Subclasses should implement the following:
- update()
- predict()
"""
transforms = {}
def __init__(self, input_shape, num_classes, num_domains, hparams):
super(Algorithm, self).__init__()
self.input_shape = input_shape
self.num_classes = num_classes
self.num_domains = num_domains
self.hparams = hparams
def update(self, x, y, **kwargs):
"""
Perform one update step, given a list of (x, y) tuples for all
environments.
"""
raise NotImplementedError
def predict(self, x):
raise NotImplementedError
def forward(self, x):
return self.predict(x)
def new_optimizer(self, parameters):
optimizer = get_optimizer(
self.hparams["optimizer"],
parameters,
lr=self.hparams["lr"],
weight_decay=self.hparams["weight_decay"],
)
return optimizer
def clone(self):
clone = copy.deepcopy(self)
clone.optimizer = self.new_optimizer(clone.network.parameters())
clone.optimizer.load_state_dict(self.optimizer.state_dict())
return clone
class ERM(Algorithm):
"""
Empirical Risk Minimization (ERM)
"""
def __init__(self, input_shape, num_classes, num_domains, hparams):
super(ERM, self).__init__(input_shape, num_classes, num_domains, hparams)
self.featurizer = networks.Featurizer(input_shape, self.hparams)
self.classifier = nn.Linear(self.featurizer.n_outputs, num_classes)
self.network = nn.Sequential(self.featurizer, self.classifier)
self.optimizer = get_optimizer(
hparams["optimizer"],
self.network.parameters(),
lr=self.hparams["lr"],
weight_decay=self.hparams["weight_decay"],
)
def update(self, x, y, **kwargs):
all_x = torch.cat(x)
all_y = torch.cat(y)
all_preds = self.predict(all_x)
while len(all_preds) > len(all_y):
all_y = torch.cat([all_y, all_y], dim=0)
loss = F.cross_entropy(all_preds, all_y)
self.optimizer.zero_grad()
loss.backward()
self.optimizer.step()
return {"loss": loss.item()}
def predict(self, x):
return self.network(x)
class Mixstyle(Algorithm):
"""MixStyle w/o domain label (random shuffle)"""
def __init__(self, input_shape, num_classes, num_domains, hparams):
assert input_shape[1:3] == (224, 224), "Mixstyle support R18 and R50 only"
super().__init__(input_shape, num_classes, num_domains, hparams)
if hparams["resnet18"]:
network = resnet18_mixstyle_L234_p0d5_a0d1()
else:
network = resnet50_mixstyle_L234_p0d5_a0d1()
self.featurizer = networks.ResNet(input_shape, self.hparams, network)
self.classifier = nn.Linear(self.featurizer.n_outputs, num_classes)
self.network = nn.Sequential(self.featurizer, self.classifier)
self.optimizer = self.new_optimizer(self.network.parameters())
def update(self, x, y, **kwargs):
all_x = torch.cat(x)
all_y = torch.cat(y)
all_preds = self.predict(all_x)
while len(all_preds) > len(all_y):
all_y = torch.cat([all_y, all_y], dim=0)
loss = F.cross_entropy(all_preds, all_y)
self.optimizer.zero_grad()
loss.backward()
self.optimizer.step()
return {"loss": loss.item()}
def predict(self, x):
return self.network(x)
class Mixstyle2(Algorithm):
"""MixStyle w/ domain label"""
def __init__(self, input_shape, num_classes, num_domains, hparams):
assert input_shape[1:3] == (224, 224), "Mixstyle support R18 and R50 only"
super().__init__(input_shape, num_classes, num_domains, hparams)
if hparams["resnet18"]:
network = resnet18_mixstyle2_L234_p0d5_a0d1()
else:
network = resnet50_mixstyle2_L234_p0d5_a0d1()
self.featurizer = networks.ResNet(input_shape, self.hparams, network)
self.classifier = nn.Linear(self.featurizer.n_outputs, num_classes)
self.network = nn.Sequential(self.featurizer, self.classifier)
self.optimizer = self.new_optimizer(self.network.parameters())
def pair_batches(self, xs, ys):
xs = [x.chunk(2) for x in xs]
ys = [y.chunk(2) for y in ys]
N = len(xs)
pairs = []
for i in range(N):
j = i + 1 if i < (N - 1) else 0
xi, yi = xs[i][0], ys[i][0]
xj, yj = xs[j][1], ys[j][1]
pairs.append(((xi, yi), (xj, yj)))
return pairs
def update(self, x, y, **kwargs):
pairs = self.pair_batches(x, y)
loss = 0.0
for (xi, yi), (xj, yj) in pairs:
# Mixstyle2:
# For the input x, the first half comes from one domain,
# while the second half comes from the other domain.
x2 = torch.cat([xi, xj])
y2 = torch.cat([yi, yj])
preds = self.predict(x2)
while len(preds)> len(y2):
y2 = torch.cat([y2, y2], dim=0)
loss += F.cross_entropy(preds, y2)
loss /= len(pairs)
self.optimizer.zero_grad()
loss.backward()
self.optimizer.step()
return {"loss": loss.item()}
def predict(self, x):
return self.network(x)
class ARM(ERM):
"""Adaptive Risk Minimization (ARM)"""
def __init__(self, input_shape, num_classes, num_domains, hparams):
original_input_shape = input_shape
input_shape = (1 + original_input_shape[0],) + original_input_shape[1:]
super(ARM, self).__init__(input_shape, num_classes, num_domains, hparams)
self.context_net = networks.ContextNet(original_input_shape)
self.support_size = hparams["batch_size"]
def predict(self, x):
batch_size, c, h, w = x.shape
if batch_size % self.support_size == 0:
meta_batch_size = batch_size // self.support_size
support_size = self.support_size
else:
meta_batch_size, support_size = 1, batch_size
context = self.context_net(x)
context = context.reshape((meta_batch_size, support_size, 1, h, w))
context = context.mean(dim=1)
context = torch.repeat_interleave(context, repeats=support_size, dim=0)
x = torch.cat([x, context], dim=1)
return self.network(x)
class SAM(ERM):
"""Sharpness-Aware Minimization
"""
@staticmethod
def norm(tensor_list: List[torch.tensor], p=2):
"""Compute p-norm for tensor list"""
return torch.cat([x.flatten() for x in tensor_list]).norm(p)
def update(self, x, y, **kwargs):
all_x = torch.cat([xi for xi in x])
all_y = torch.cat([yi for yi in y])
loss = F.cross_entropy(self.predict(all_x), all_y)
# 1. eps(w) = rho * g(w) / g(w).norm(2)
# = (rho / g(w).norm(2)) * g(w)
grad_w = autograd.grad(loss, self.network.parameters())
scale = self.hparams["rho"] / self.norm(grad_w)
eps = [g * scale for g in grad_w]
# 2. w' = w + eps(w)
with torch.no_grad():
for p, v in zip(self.network.parameters(), eps):
p.add_(v)
# 3. w = w - lr * g(w')
loss = F.cross_entropy(self.predict(all_x), all_y)
self.optimizer.zero_grad()
loss.backward()
# restore original network params
with torch.no_grad():
for p, v in zip(self.network.parameters(), eps):
p.sub_(v)
self.optimizer.step()
return {"loss": loss.item()}
class AbstractDANN(Algorithm):
"""Domain-Adversarial Neural Networks (abstract class)"""
def __init__(self, input_shape, num_classes, num_domains, hparams, conditional, class_balance):
super(AbstractDANN, self).__init__(input_shape, num_classes, num_domains, hparams)
self.register_buffer("update_count", torch.tensor([0]))
self.conditional = conditional
self.class_balance = class_balance
# Algorithms
self.featurizer = networks.Featurizer(input_shape, self.hparams)
self.classifier = nn.Linear(self.featurizer.n_outputs, num_classes)
self.discriminator = networks.MLP(self.featurizer.n_outputs, num_domains, self.hparams)
self.class_embeddings = nn.Embedding(num_classes, self.featurizer.n_outputs)
# Optimizers
self.disc_opt = get_optimizer(
hparams["optimizer"],
(list(self.discriminator.parameters()) + list(self.class_embeddings.parameters())),
lr=self.hparams["lr_d"],
weight_decay=self.hparams["weight_decay_d"],
betas=(self.hparams["beta1"], 0.9),
)
self.gen_opt = get_optimizer(
hparams["optimizer"],
(list(self.featurizer.parameters()) + list(self.classifier.parameters())),
lr=self.hparams["lr_g"],
weight_decay=self.hparams["weight_decay_g"],
betas=(self.hparams["beta1"], 0.9),
)
def update(self, x, y, **kwargs):
self.update_count += 1
all_x = torch.cat([xi for xi in x])
all_y = torch.cat([yi for yi in y])
minibatches = to_minibatch(x, y)
all_z = self.featurizer(all_x)
if self.conditional:
disc_input = all_z + self.class_embeddings(all_y)
else:
disc_input = all_z
disc_out = self.discriminator(disc_input)
disc_labels = torch.cat(
[
torch.full((x.shape[0],), i, dtype=torch.int64, device="cuda")
for i, (x, y) in enumerate(minibatches)
]
)
if len(disc_out) > len(disc_labels):
disc_labels = torch.cat([disc_labels, disc_labels], dim=0)
if self.class_balance:
y_counts = F.one_hot(all_y).sum(dim=0)
weights = 1.0 / (y_counts[all_y] * y_counts.shape[0]).float()
disc_loss = F.cross_entropy(disc_out, disc_labels, reduction="none")
disc_loss = (weights * disc_loss).sum()
else:
disc_loss = F.cross_entropy(disc_out, disc_labels)
disc_softmax = F.softmax(disc_out, dim=1)
input_grad = autograd.grad(
disc_softmax[:, disc_labels].sum(), [disc_input], create_graph=True
)[0]
grad_penalty = (input_grad ** 2).sum(dim=1).mean(dim=0)
disc_loss += self.hparams["grad_penalty"] * grad_penalty
d_steps_per_g = self.hparams["d_steps_per_g_step"]
if self.update_count.item() % (1 + d_steps_per_g) < d_steps_per_g:
self.disc_opt.zero_grad()
disc_loss.backward()
self.disc_opt.step()
return {"disc_loss": disc_loss.item()}
else:
all_preds = self.classifier(all_z)
if len(all_preds) > len(all_y):
all_y = torch.cat([all_y, all_y], dim=0)
classifier_loss = F.cross_entropy(all_preds, all_y)
gen_loss = classifier_loss + (self.hparams["lambda"] * -disc_loss)
self.disc_opt.zero_grad()
self.gen_opt.zero_grad()
gen_loss.backward()
self.gen_opt.step()
return {"gen_loss": gen_loss.item()}
def predict(self, x):
return self.classifier(self.featurizer(x))
class DANN(AbstractDANN):
"""Unconditional DANN"""
def __init__(self, input_shape, num_classes, num_domains, hparams):
super(DANN, self).__init__(
input_shape,
num_classes,
num_domains,
hparams,
conditional=False,
class_balance=False,
)
class CDANN(AbstractDANN):
"""Conditional DANN"""
def __init__(self, input_shape, num_classes, num_domains, hparams):
super(CDANN, self).__init__(
input_shape,
num_classes,
num_domains,
hparams,
conditional=True,
class_balance=True,
)
class IRM(ERM):
"""Invariant Risk Minimization"""
def __init__(self, input_shape, num_classes, num_domains, hparams):
super(IRM, self).__init__(input_shape, num_classes, num_domains, hparams)
self.register_buffer("update_count", torch.tensor([0]))
@staticmethod
def _irm_penalty(logits, y):
scale = torch.tensor(1.0).cuda().requires_grad_()
loss_1 = F.cross_entropy(logits[::2] * scale, y[::2])
loss_2 = F.cross_entropy(logits[1::2] * scale, y[1::2])
grad_1 = autograd.grad(loss_1, [scale], create_graph=True)[0]
grad_2 = autograd.grad(loss_2, [scale], create_graph=True)[0]
result = torch.sum(grad_1 * grad_2)
return result
def update(self, x, y, **kwargs):
minibatches = to_minibatch(x, y)
penalty_weight = (
self.hparams["irm_lambda"]
if self.update_count >= self.hparams["irm_penalty_anneal_iters"]
else 1.0
)
nll = 0.0
penalty = 0.0
all_x = torch.cat([x for x, y in minibatches])
all_logits = self.network(all_x)
all_logits_idx = 0
for i, (x, y) in enumerate(minibatches):
logits = all_logits[all_logits_idx : all_logits_idx + x.shape[0]]
all_logits_idx += x.shape[0]
nll += F.cross_entropy(logits, y)
penalty += self._irm_penalty(logits, y)
nll /= len(minibatches)
penalty /= len(minibatches)
loss = nll + (penalty_weight * penalty)
if self.update_count == self.hparams["irm_penalty_anneal_iters"]:
# Reset Adam, because it doesn't like the sharp jump in gradient
# magnitudes that happens at this step.
self.optimizer = get_optimizer(
self.hparams["optimizer"],
self.network.parameters(),
lr=self.hparams["lr"],
weight_decay=self.hparams["weight_decay"],
)
self.optimizer.zero_grad()
loss.backward()
self.optimizer.step()
self.update_count += 1
return {"loss": loss.item(), "nll": nll.item(), "penalty": penalty.item()}
class VREx(ERM):
"""V-REx algorithm from http://arxiv.org/abs/2003.00688"""
def __init__(self, input_shape, num_classes, num_domains, hparams):
super(VREx, self).__init__(input_shape, num_classes, num_domains, hparams)
self.register_buffer("update_count", torch.tensor([0]))
def update(self, x, y, **kwargs):
minibatches = to_minibatch(x, y)
if self.update_count >= self.hparams["vrex_penalty_anneal_iters"]:
penalty_weight = self.hparams["vrex_lambda"]
else:
penalty_weight = 1.0
nll = 0.0
all_x = torch.cat([x for x, y in minibatches])
all_logits = self.network(all_x)
all_logits_idx = 0
losses = torch.zeros(len(minibatches))
for i, (x, y) in enumerate(minibatches):
logits = all_logits[all_logits_idx : all_logits_idx + x.shape[0]]
all_logits_idx += x.shape[0]
nll = F.cross_entropy(logits, y)
losses[i] = nll
mean = losses.mean()
penalty = ((losses - mean) ** 2).mean()
loss = mean + penalty_weight * penalty
if self.update_count == self.hparams["vrex_penalty_anneal_iters"]:
# Reset Adam (like IRM), because it doesn't like the sharp jump in
# gradient magnitudes that happens at this step.
self.optimizer = get_optimizer(
self.hparams["optimizer"],
self.network.parameters(),
lr=self.hparams["lr"],
weight_decay=self.hparams["weight_decay"],
)
self.optimizer.zero_grad()
loss.backward()
self.optimizer.step()
self.update_count += 1
return {"loss": loss.item(), "nll": nll.item(), "penalty": penalty.item()}
class Mixup(ERM):
"""
Mixup of minibatches from different domains
https://arxiv.org/pdf/2001.00677.pdf
https://arxiv.org/pdf/1912.01805.pdf
"""
def __init__(self, input_shape, num_classes, num_domains, hparams):
super(Mixup, self).__init__(input_shape, num_classes, num_domains, hparams)
def update(self, x, y, **kwargs):
minibatches = to_minibatch(x, y)
objective = 0
for (xi, yi), (xj, yj) in random_pairs_of_minibatches(minibatches):
lam = np.random.beta(self.hparams["mixup_alpha"], self.hparams["mixup_alpha"])
x = lam * xi + (1 - lam) * xj
predictions = self.predict(x)
objective += lam * F.cross_entropy(predictions, yi)
objective += (1 - lam) * F.cross_entropy(predictions, yj)
objective /= len(minibatches)
self.optimizer.zero_grad()
objective.backward()
self.optimizer.step()
return {"loss": objective.item()}
class OrgMixup(ERM):
"""
Original Mixup independent with domains
"""
def update(self, x, y, **kwargs):
x = torch.cat(x)
y = torch.cat(y)
indices = torch.randperm(x.size(0))
x2 = x[indices]
y2 = y[indices]
lam = np.random.beta(self.hparams["mixup_alpha"], self.hparams["mixup_alpha"])
x = lam * x + (1 - lam) * x2
predictions = self.predict(x)
objective = lam * F.cross_entropy(predictions, y)
objective += (1 - lam) * F.cross_entropy(predictions, y2)
self.optimizer.zero_grad()
objective.backward()
self.optimizer.step()
return {"loss": objective.item()}
class CutMix(ERM):
@staticmethod
def rand_bbox(size, lam):
W = size[2]
H = size[3]
cut_rat = np.sqrt(1.0 - lam)
cut_w = np.int(W * cut_rat)
cut_h = np.int(H * cut_rat)
# uniform
cx = np.random.randint(W)
cy = np.random.randint(H)
bbx1 = np.clip(cx - cut_w // 2, 0, W)
bby1 = np.clip(cy - cut_h // 2, 0, H)
bbx2 = np.clip(cx + cut_w // 2, 0, W)
bby2 = np.clip(cy + cut_h // 2, 0, H)
return bbx1, bby1, bbx2, bby2
def update(self, x, y, **kwargs):
# cutmix_prob is set to 1.0 for ImageNet and 0.5 for CIFAR100 in the original paper.
x = torch.cat(x)
y = torch.cat(y)
r = np.random.rand(1)
if self.hparams["beta"] > 0 and r < self.hparams["cutmix_prob"]:
# generate mixed sample
beta = self.hparams["beta"]
lam = np.random.beta(beta, beta)
rand_index = torch.randperm(x.size()[0]).cuda()
target_a = y
target_b = y[rand_index]
bbx1, bby1, bbx2, bby2 = self.rand_bbox(x.size(), lam)
x[:, :, bbx1:bbx2, bby1:bby2] = x[rand_index, :, bbx1:bbx2, bby1:bby2]
# adjust lambda to exactly match pixel ratio
lam = 1 - ((bbx2 - bbx1) * (bby2 - bby1) / (x.size()[-1] * x.size()[-2]))
# compute output
output = self.predict(x)
objective = F.cross_entropy(output, target_a) * lam + F.cross_entropy(
output, target_b
) * (1.0 - lam)
else:
output = self.predict(x)
objective = F.cross_entropy(output, y)
self.optimizer.zero_grad()
objective.backward()
self.optimizer.step()
return {"loss": objective.item()}
class GroupDRO(ERM):
"""
Robust ERM minimizes the error at the worst minibatch
Algorithm 1 from [https://arxiv.org/pdf/1911.08731.pdf]
"""
def __init__(self, input_shape, num_classes, num_domains, hparams):
super(GroupDRO, self).__init__(input_shape, num_classes, num_domains, hparams)
self.register_buffer("q", torch.Tensor())
def update(self, x, y, **kwargs):
minibatches = to_minibatch(x, y)
device = "cuda" if minibatches[0][0].is_cuda else "cpu"
if not len(self.q):
self.q = torch.ones(len(minibatches)).to(device)
losses = torch.zeros(len(minibatches)).to(device)
for m in range(len(minibatches)):
x, y = minibatches[m]
losses[m] = F.cross_entropy(self.predict(x), y)
self.q[m] *= (self.hparams["groupdro_eta"] * losses[m].data).exp()
self.q /= self.q.sum()
loss = torch.dot(losses, self.q) / len(minibatches)
self.optimizer.zero_grad()
loss.backward()
self.optimizer.step()
return {"loss": loss.item()}
class MLDG(ERM):
"""
Model-Agnostic Meta-Learning
Algorithm 1 / Equation (3) from: https://arxiv.org/pdf/1710.03463.pdf
Related: https://arxiv.org/pdf/1703.03400.pdf
Related: https://arxiv.org/pdf/1910.13580.pdf
"""
def __init__(self, input_shape, num_classes, num_domains, hparams):
super(MLDG, self).__init__(input_shape, num_classes, num_domains, hparams)
def update(self, x, y, **kwargs):
"""
Terms being computed:
* Li = Loss(xi, yi, params)
* Gi = Grad(Li, params)
* Lj = Loss(xj, yj, Optimizer(params, grad(Li, params)))
* Gj = Grad(Lj, params)
* params = Optimizer(params, Grad(Li + beta * Lj, params))
* = Optimizer(params, Gi + beta * Gj)
That is, when calling .step(), we want grads to be Gi + beta * Gj
For computational efficiency, we do not compute second derivatives.
"""
minibatches = to_minibatch(x, y)
num_mb = len(minibatches)
objective = 0
self.optimizer.zero_grad()
for p in self.network.parameters():
if p.grad is None:
p.grad = torch.zeros_like(p)
for (xi, yi), (xj, yj) in random_pairs_of_minibatches(minibatches):
# fine tune clone-network on task "i"
inner_net = copy.deepcopy(self.network)
inner_opt = get_optimizer(
self.hparams["optimizer"],
# "SGD",
inner_net.parameters(),
lr=self.hparams["lr"],
weight_decay=self.hparams["weight_decay"],
)
inner_obj = F.cross_entropy(inner_net(xi), yi)
inner_opt.zero_grad()
inner_obj.backward()
inner_opt.step()
# 1. Compute supervised loss for meta-train set
# The network has now accumulated gradients Gi
# The clone-network has now parameters P - lr * Gi
for p_tgt, p_src in zip(self.network.parameters(), inner_net.parameters()):
if p_src.grad is not None:
p_tgt.grad.data.add_(p_src.grad.data / num_mb)
# `objective` is populated for reporting purposes
objective += inner_obj.item()
# 2. Compute meta loss for meta-val set
# this computes Gj on the clone-network
loss_inner_j = F.cross_entropy(inner_net(xj), yj)
grad_inner_j = autograd.grad(loss_inner_j, inner_net.parameters(), allow_unused=True)
# `objective` is populated for reporting purposes
objective += (self.hparams["mldg_beta"] * loss_inner_j).item()
for p, g_j in zip(self.network.parameters(), grad_inner_j):
if g_j is not None:
p.grad.data.add_(self.hparams["mldg_beta"] * g_j.data / num_mb)
# The network has now accumulated gradients Gi + beta * Gj
# Repeat for all train-test splits, do .step()
objective /= len(minibatches)
self.optimizer.step()
return {"loss": objective}
# class SOMLDG(MLDG):
# """Second-order MLDG"""
# # This commented "update" method back-propagates through the gradients of
# # the inner update, as suggested in the original MAML paper. However, this
# # is twice as expensive as the uncommented "update" method, which does not
# # compute second-order derivatives, implementing the First-Order MAML
# # method (FOMAML) described in the original MAML paper.
# def update(self, x, y, **kwargs):
# minibatches = to_minibatch(x, y)
# objective = 0
# beta = self.hparams["mldg_beta"]
# inner_iterations = self.hparams.get("inner_iterations", 1)
# self.optimizer.zero_grad()
# with higher.innerloop_ctx(
# self.network, self.optimizer, copy_initial_weights=False
# ) as (inner_network, inner_optimizer):
# for (xi, yi), (xj, yj) in random_pairs_of_minibatches(minibatches):
# for inner_iteration in range(inner_iterations):
# li = F.cross_entropy(inner_network(xi), yi)
# inner_optimizer.step(li)
# objective += F.cross_entropy(self.network(xi), yi)
# objective += beta * F.cross_entropy(inner_network(xj), yj)
# objective /= len(minibatches)
# objective.backward()
# self.optimizer.step()
# return {"loss": objective.item()}
class AbstractMMD(ERM):
"""
Perform ERM while matching the pair-wise domain feature distributions
using MMD (abstract class)
"""
def __init__(self, input_shape, num_classes, num_domains, hparams, gaussian):
super(AbstractMMD, self).__init__(input_shape, num_classes, num_domains, hparams)
if gaussian:
self.kernel_type = "gaussian"
else:
self.kernel_type = "mean_cov"
def my_cdist(self, x1, x2):
x1_norm = x1.pow(2).sum(dim=-1, keepdim=True)
x2_norm = x2.pow(2).sum(dim=-1, keepdim=True)
res = torch.addmm(x2_norm.transpose(-2, -1), x1, x2.transpose(-2, -1), alpha=-2).add_(
x1_norm
)
return res.clamp_min_(1e-30)
def gaussian_kernel(self, x, y, gamma=(0.001, 0.01, 0.1, 1, 10, 100, 1000)):
D = self.my_cdist(x, y)
K = torch.zeros_like(D)
for g in gamma:
K.add_(torch.exp(D.mul(-g)))
return K
def mmd(self, x, y):
if self.kernel_type == "gaussian":
Kxx = self.gaussian_kernel(x, x).mean()
Kyy = self.gaussian_kernel(y, y).mean()
Kxy = self.gaussian_kernel(x, y).mean()
return Kxx + Kyy - 2 * Kxy
else:
mean_x = x.mean(0, keepdim=True)
mean_y = y.mean(0, keepdim=True)
cent_x = x - mean_x
cent_y = y - mean_y
cova_x = (cent_x.t() @ cent_x) / (len(x) - 1)
cova_y = (cent_y.t() @ cent_y) / (len(y) - 1)
mean_diff = (mean_x - mean_y).pow(2).mean()
cova_diff = (cova_x - cova_y).pow(2).mean()
return mean_diff + cova_diff
def update(self, x, y, **kwargs):
minibatches = to_minibatch(x, y)
objective = 0
penalty = 0
nmb = len(minibatches)
features = [self.featurizer(xi) for xi, _ in minibatches]
classifs = [self.classifier(fi) for fi in features]
targets = [yi for _, yi in minibatches]
for i in range(nmb):
objective += F.cross_entropy(classifs[i], targets[i])
for j in range(i + 1, nmb):
penalty += self.mmd(features[i], features[j])
objective /= nmb
if nmb > 1:
penalty /= nmb * (nmb - 1) / 2
self.optimizer.zero_grad()
(objective + (self.hparams["mmd_gamma"] * penalty)).backward()
self.optimizer.step()
if torch.is_tensor(penalty):
penalty = penalty.item()
return {"loss": objective.item(), "penalty": penalty}
class MMD(AbstractMMD):
"""
MMD using Gaussian kernel
"""
def __init__(self, input_shape, num_classes, num_domains, hparams):
super(MMD, self).__init__(input_shape, num_classes, num_domains, hparams, gaussian=True)
class CORAL(AbstractMMD):
"""
MMD using mean and covariance difference
"""
def __init__(self, input_shape, num_classes, num_domains, hparams):
super(CORAL, self).__init__(input_shape, num_classes, num_domains, hparams, gaussian=False)
class MTL(Algorithm):
"""
A neural network version of
Domain Generalization by Marginal Transfer Learning
(https://arxiv.org/abs/1711.07910)
"""
def __init__(self, input_shape, num_classes, num_domains, hparams):
super(MTL, self).__init__(input_shape, num_classes, num_domains, hparams)
self.featurizer = networks.Featurizer(input_shape, self.hparams)
self.classifier = nn.Linear(self.featurizer.n_outputs * 2, num_classes)
self.optimizer = get_optimizer(
hparams["optimizer"],
list(self.featurizer.parameters()) + list(self.classifier.parameters()),
lr=self.hparams["lr"],
weight_decay=self.hparams["weight_decay"],
)
self.register_buffer("embeddings", torch.zeros(num_domains, self.featurizer.n_outputs))
self.ema = self.hparams["mtl_ema"]
def update(self, x, y, **kwargs):
minibatches = to_minibatch(x, y)
loss = 0
for env, (x, y) in enumerate(minibatches):
loss += F.cross_entropy(self.predict(x, env), y)
self.optimizer.zero_grad()
loss.backward()
self.optimizer.step()
return {"loss": loss.item()}
def update_embeddings_(self, features, env=None):
return_embedding = features.mean(0)
if env is not None:
return_embedding = self.ema * return_embedding + (1 - self.ema) * self.embeddings[env]
self.embeddings[env] = return_embedding.clone().detach()
return return_embedding.view(1, -1).repeat(len(features), 1)
def predict(self, x, env=None):
features = self.featurizer(x)
embedding = self.update_embeddings_(features, env).normal_()
return self.classifier(torch.cat((features, embedding), 1))
class SagNet(Algorithm):
"""
Style Agnostic Network
Algorithm 1 from: https://arxiv.org/abs/1910.11645
"""
def __init__(self, input_shape, num_classes, num_domains, hparams):
super(SagNet, self).__init__(input_shape, num_classes, num_domains, hparams)
# featurizer network
self.network_f = networks.Featurizer(input_shape, self.hparams)
# content network
self.network_c = nn.Linear(self.network_f.n_outputs, num_classes)
# style network
self.network_s = nn.Linear(self.network_f.n_outputs, num_classes)
# # This commented block of code implements something closer to the
# # original paper, but is specific to ResNet and puts in disadvantage
# # the other algorithms.
# resnet_c = networks.Featurizer(input_shape, self.hparams)
# resnet_s = networks.Featurizer(input_shape, self.hparams)
# # featurizer network
# self.network_f = torch.nn.Sequential(
# resnet_c.network.conv1,
# resnet_c.network.bn1,
# resnet_c.network.relu,
# resnet_c.network.maxpool,
# resnet_c.network.layer1,
# resnet_c.network.layer2,
# resnet_c.network.layer3)
# # content network
# self.network_c = torch.nn.Sequential(
# resnet_c.network.layer4,
# resnet_c.network.avgpool,
# networks.Flatten(),
# resnet_c.network.fc)
# # style network
# self.network_s = torch.nn.Sequential(
# resnet_s.network.layer4,
# resnet_s.network.avgpool,
# networks.Flatten(),
# resnet_s.network.fc)
def opt(p):
return get_optimizer(
hparams["optimizer"], p, lr=hparams["lr"], weight_decay=hparams["weight_decay"]
)
self.optimizer_f = opt(self.network_f.parameters())
self.optimizer_c = opt(self.network_c.parameters())
self.optimizer_s = opt(self.network_s.parameters())
self.weight_adv = hparams["sag_w_adv"]
def forward_c(self, x):
# learning content network on randomized style
return self.network_c(self.randomize(self.network_f(x), "style"))
def forward_s(self, x):
# learning style network on randomized content
return self.network_s(self.randomize(self.network_f(x), "content"))
def randomize(self, x, what="style", eps=1e-5):
sizes = x.size()
alpha = torch.rand(sizes[0], 1).cuda()
if len(sizes) == 4:
x = x.view(sizes[0], sizes[1], -1)
alpha = alpha.unsqueeze(-1)
mean = x.mean(-1, keepdim=True)
var = x.var(-1, keepdim=True)
x = (x - mean) / (var + eps).sqrt()
idx_swap = torch.randperm(sizes[0])
if what == "style":
mean = alpha * mean + (1 - alpha) * mean[idx_swap]
var = alpha * var + (1 - alpha) * var[idx_swap]
else:
x = x[idx_swap].detach()
x = x * (var + eps).sqrt() + mean
return x.view(*sizes)
def update(self, x, y, **kwargs):
all_x = torch.cat([xi for xi in x])
all_y = torch.cat([yi for yi in y])
# learn content
self.optimizer_f.zero_grad()
self.optimizer_c.zero_grad()
loss_c = F.cross_entropy(self.forward_c(all_x), all_y)
loss_c.backward()
self.optimizer_f.step()
self.optimizer_c.step()
# learn style
self.optimizer_s.zero_grad()
loss_s = F.cross_entropy(self.forward_s(all_x), all_y)
loss_s.backward()
self.optimizer_s.step()
# learn adversary
self.optimizer_f.zero_grad()
loss_adv = -F.log_softmax(self.forward_s(all_x), dim=1).mean(1).mean()
loss_adv = loss_adv * self.weight_adv
loss_adv.backward()
self.optimizer_f.step()
return {
"loss_c": loss_c.item(),
"loss_s": loss_s.item(),
"loss_adv": loss_adv.item(),
}
def predict(self, x):
return self.network_c(self.network_f(x))
class RSC(ERM):
def __init__(self, input_shape, num_classes, num_domains, hparams):
super(RSC, self).__init__(input_shape, num_classes, num_domains, hparams)
self.drop_f = (1 - hparams["rsc_f_drop_factor"]) * 100
self.drop_b = (1 - hparams["rsc_b_drop_factor"]) * 100
self.num_classes = num_classes
def update(self, x, y, **kwargs):
# inputs
all_x = torch.cat([xi for xi in x])
# labels
all_y = torch.cat([yi for yi in y])
# one-hot labels
all_o = torch.nn.functional.one_hot(all_y, self.num_classes)
# features
all_f = self.featurizer(all_x)
# predictions
all_p = self.classifier(all_f)
# Equation (1): compute gradients with respect to representation
all_g = autograd.grad((all_p * all_o).sum(), all_f)[0]
# Equation (2): compute top-gradient-percentile mask
percentiles = np.percentile(all_g.cpu(), self.drop_f, axis=1)
percentiles = torch.Tensor(percentiles)
percentiles = percentiles.unsqueeze(1).repeat(1, all_g.size(1))
mask_f = all_g.lt(percentiles.cuda()).float()
# Equation (3): mute top-gradient-percentile activations
all_f_muted = all_f * mask_f
# Equation (4): compute muted predictions
all_p_muted = self.classifier(all_f_muted)
# Section 3.3: Batch Percentage
all_s = F.softmax(all_p, dim=1)
all_s_muted = F.softmax(all_p_muted, dim=1)
changes = (all_s * all_o).sum(1) - (all_s_muted * all_o).sum(1)
percentile = np.percentile(changes.detach().cpu(), self.drop_b)
mask_b = changes.lt(percentile).float().view(-1, 1)
mask = torch.logical_or(mask_f, mask_b).float()
# Equations (3) and (4) again, this time mutting over examples
all_p_muted_again = self.classifier(all_f * mask)
# Equation (5): update
loss = F.cross_entropy(all_p_muted_again, all_y)
self.optimizer.zero_grad()
loss.backward()
self.optimizer.step()
return {"loss": loss.item()}
================================================
FILE: domain_generalization/swad/domainbed/datasets/__init__.py
================================================
import torch
import numpy as np
from domainbed.datasets import datasets
from domainbed.lib import misc
from domainbed.datasets import transforms as DBT
def set_transfroms(dset, data_type, hparams, algorithm_class=None):
"""
Args:
data_type: ['train', 'valid', 'test', 'mnist']
"""
assert hparams["data_augmentation"]
additional_data = False
if data_type == "train":
dset.transforms = {"x": DBT.aug}
additional_data = True
elif data_type == "valid":
if hparams["val_augment"] is False:
dset.transforms = {"x": DBT.basic}
else:
# Originally, DomainBed use same training augmentation policy to validation.
# We turn off the augmentation for validation as default,
# but left the option to reproducibility.
dset.transforms = {"x": DBT.aug}
elif data_type == "test":
dset.transforms = {"x": DBT.basic}
elif data_type == "mnist":
# No augmentation for mnist
dset.transforms = {"x": lambda x: x}
else:
raise ValueError(data_type)
if additional_data and algorithm_class is not None:
for key, transform in algorithm_class.transforms.items():
dset.transforms[key] = transform
def get_dataset(test_envs, args, hparams, algorithm_class=None):
"""Get dataset and split."""
is_mnist = "MNIST" in args.dataset
dataset = vars(datasets)[args.dataset](args.data_dir)
# if not isinstance(dataset, MultipleEnvironmentImageFolder):
# raise ValueError("SMALL image datasets are not implemented (corrupted), for transform.")
in_splits = []
out_splits = []
for env_i, env in enumerate(dataset):
# The split only depends on seed_hash (= trial_seed).
# It means that the split is always identical only if use same trial_seed,
# independent to run the code where, when, or how many times.
out, in_ = split_dataset(
env,
int(len(env) * args.holdout_fraction),
misc.seed_hash(args.trial_seed, env_i),
)
if env_i in test_envs:
in_type = "test"
out_type = "test"
else:
in_type = "train"
out_type = "valid"
if is_mnist:
in_type = "mnist"
out_type = "mnist"
set_transfroms(in_, in_type, hparams, algorithm_class)
set_transfroms(out, out_type, hparams, algorithm_class)
if hparams["class_balanced"]:
in_weights = misc.make_weights_for_balanced_classes(in_)
out_weights = misc.make_weights_for_balanced_classes(out)
else:
in_weights, out_weights = None, None
in_splits.append((in_, in_weights))
out_splits.append((out, out_weights))
return dataset, in_splits, out_splits
class _SplitDataset(torch.utils.data.Dataset):
"""Used by split_dataset"""
def __init__(self, underlying_dataset, keys):
super(_SplitDataset, self).__init__()
self.underlying_dataset = underlying_dataset
self.keys = keys
self.transforms = {}
self.direct_return = isinstance(underlying_dataset, _SplitDataset)
def __getitem__(self, key):
if self.direct_return:
return self.underlying_dataset[self.keys[key]]
x, y = self.underlying_dataset[self.keys[key]]
ret = {"y": y}
for key, transform in self.transforms.items():
ret[key] = transform(x)
return ret
def __len__(self):
return len(self.keys)
def split_dataset(dataset, n, seed=0):
"""
Return a pair of datasets corresponding to a random split of the given
dataset, with n datapoints in the first dataset and the rest in the last,
using the given random seed
"""
assert n <= len(dataset)
keys = list(range(len(dataset)))
np.random.RandomState(seed).shuffle(keys)
keys_1 = keys[:n]
keys_2 = keys[n:]
return _SplitDataset(dataset, keys_1), _SplitDataset(dataset, keys_2)
================================================
FILE: domain_generalization/swad/domainbed/datasets/datasets.py
================================================
# Copyright (c) Facebook, Inc. and its affiliates. All Rights Reserved
import os
import torch
from PIL import Image, ImageFile
from torchvision import transforms as T
from torch.utils.data import TensorDataset
from torchvision.datasets import MNIST, ImageFolder
from torchvision.transforms.functional import rotate
ImageFile.LOAD_TRUNCATED_IMAGES = True
DATASETS = [
# Debug
"Debug28",
"Debug224",
# Small images
"ColoredMNIST",
"RotatedMNIST",
# Big images
"VLCS",
"PACS",
"OfficeHome",
"TerraIncognita",
"DomainNet",
]
def get_dataset_class(dataset_name):
"""Return the dataset class with the given name."""
if dataset_name not in globals():
raise NotImplementedError("Dataset not found: {}".format(dataset_name))
return globals()[dataset_name]
def num_environments(dataset_name):
return len(get_dataset_class(dataset_name).ENVIRONMENTS)
class MultipleDomainDataset:
N_STEPS = 5001 # Default, subclasses may override
CHECKPOINT_FREQ = 100 # Default, subclasses may override
N_WORKERS = 4 # Default, subclasses may override
ENVIRONMENTS = None # Subclasses should override
INPUT_SHAPE = None # Subclasses should override
def __getitem__(self, index):
"""
Return: sub-dataset for specific domain
"""
return self.datasets[index]
def __len__(self):
"""
Return: # of sub-datasets
"""
return len(self.datasets)
class Debug(MultipleDomainDataset):
def __init__(self, root):
super().__init__()
self.input_shape = self.INPUT_SHAPE
self.num_classes = 2
self.datasets = []
for _ in [0, 1, 2]:
self.datasets.append(
TensorDataset(
torch.randn(16, *self.INPUT_SHAPE),
torch.randint(0, self.num_classes, (16,)),
)
)
class Debug28(Debug):
INPUT_SHAPE = (3, 28, 28)
ENVIRONMENTS = ["0", "1", "2"]
class Debug224(Debug):
INPUT_SHAPE = (3, 224, 224)
ENVIRONMENTS = ["0", "1", "2"]
class MultipleEnvironmentMNIST(MultipleDomainDataset):
def __init__(self, root, environments, dataset_transform, input_shape, num_classes):
"""
Args:
root: root dir for saving MNIST dataset
environments: env properties for each dataset
dataset_transform: dataset generator function
"""
super().__init__()
if root is None:
raise ValueError("Data directory not specified!")
original_dataset_tr = MNIST(root, train=True, download=True)
original_dataset_te = MNIST(root, train=False, download=True)
original_images = torch.cat((original_dataset_tr.data, original_dataset_te.data))
original_labels = torch.cat((original_dataset_tr.targets, original_dataset_te.targets))
shuffle = torch.randperm(len(original_images))
original_images = original_images[shuffle]
original_labels = original_labels[shuffle]
self.datasets = []
self.environments = environments
for i in range(len(environments)):
images = original_images[i :: len(environments)]
labels = original_labels[i :: len(environments)]
self.datasets.append(dataset_transform(images, labels, environments[i]))
self.input_shape = input_shape
self.num_classes = num_classes
class ColoredMNIST(MultipleEnvironmentMNIST):
ENVIRONMENTS = ["+90%", "+80%", "-90%"]
def __init__(self, root):
super(ColoredMNIST, self).__init__(
root,
[0.1, 0.2, 0.9],
self.color_dataset,
(2, 28, 28),
2,
)
def color_dataset(self, images, labels, environment):
# # Subsample 2x for computational convenience
# images = images.reshape((-1, 28, 28))[:, ::2, ::2]
# Assign a binary label based on the digit
labels = (labels < 5).float()
# Flip label with probability 0.25
labels = self.torch_xor_(labels, self.torch_bernoulli_(0.25, len(labels)))
# Assign a color based on the label; flip the color with probability e
colors = self.torch_xor_(labels, self.torch_bernoulli_(environment, len(labels)))
images = torch.stack([images, images], dim=1)
# Apply the color to the image by zeroing out the other color channel
images[torch.tensor(range(len(images))), (1 - colors).long(), :, :] *= 0
x = images.float().div_(255.0)
y = labels.view(-1).long()
return TensorDataset(x, y)
def torch_bernoulli_(self, p, size):
return (torch.rand(size) < p).float()
def torch_xor_(self, a, b):
return (a - b).abs()
class RotatedMNIST(MultipleEnvironmentMNIST):
ENVIRONMENTS = ["0", "15", "30", "45", "60", "75"]
def __init__(self, root):
super(RotatedMNIST, self).__init__(
root,
[0, 15, 30, 45, 60, 75],
self.rotate_dataset,
(1, 28, 28),
10,
)
def rotate_dataset(self, images, labels, angle):
rotation = T.Compose(
[
T.ToPILImage(),
T.Lambda(lambda x: rotate(x, angle, fill=(0,), resample=Image.BICUBIC)),
T.ToTensor(),
]
)
x = torch.zeros(len(images), 1, 28, 28)
for i in range(len(images)):
x[i] = rotation(images[i])
y = labels.view(-1)
return TensorDataset(x, y)
class MultipleEnvironmentImageFolder(MultipleDomainDataset):
def __init__(self, root):
super().__init__()
environments = [f.name for f in os.scandir(root) if f.is_dir()]
environments = sorted(environments)
self.environments = environments
self.datasets = []
for environment in environments:
path = os.path.join(root, environment)
env_dataset = ImageFolder(path)
self.datasets.append(env_dataset)
self.input_shape = (3, 224, 224)
self.num_classes = len(self.datasets[-1].classes)
class VLCS(MultipleEnvironmentImageFolder):
CHECKPOINT_FREQ = 200
ENVIRONMENTS = ["C", "L", "S", "V"]
def __init__(self, root):
self.dir = os.path.join(root, "VLCS/")
super().__init__(self.dir)
class PACS(MultipleEnvironmentImageFolder):
CHECKPOINT_FREQ = 200
ENVIRONMENTS = ["A", "C", "P", "S"]
def __init__(self, root):
self.dir = os.path.join(root, "PACS/")
super().__init__(self.dir)
class DomainNet(MultipleEnvironmentImageFolder):
CHECKPOINT_FREQ = 1000
N_STEPS = 15001
ENVIRONMENTS = ["clip", "info", "paint", "quick", "real", "sketch"]
def __init__(self, root):
self.dir = os.path.join(root, "domain_net/")
super().__init__(self.dir)
class OfficeHome(MultipleEnvironmentImageFolder):
CHECKPOINT_FREQ = 200
ENVIRONMENTS = ["A", "C", "P", "R"]
def __init__(self, root):
self.dir = os.path.join(root, "office_home/")
super().__init__(self.dir)
class TerraIncognita(MultipleEnvironmentImageFolder):
CHECKPOINT_FREQ = 200
ENVIRONMENTS = ["L100", "L38", "L43", "L46"]
def __init__(self, root):
self.dir = os.path.join(root, "terra_incognita/")
super().__init__(self.dir)
================================================
FILE: domain_generalization/swad/domainbed/datasets/transforms.py
================================================
from torchvision import transforms as T
basic = T.Compose(
[
T.Resize((224, 224)),
T.ToTensor(),
T.Normalize(mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225]),
]
)
aug = T.Compose(
[
T.RandomResizedCrop(224, scale=(0.7, 1.0)),
T.RandomHorizontalFlip(),
T.ColorJitter(0.3, 0.3, 0.3, 0.3),
T.RandomGrayscale(p=0.1),
T.ToTensor(),
T.Normalize(mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225]),
]
)
================================================
FILE: domain_generalization/swad/domainbed/evaluator.py
================================================
import collections
import torch
import torch.nn as nn
import torch.nn.functional as F
from domainbed.lib.fast_data_loader import FastDataLoader
if torch.cuda.is_available():
device = "cuda"
else:
device = "cpu"
def accuracy_from_loader(algorithm, loader, weights, debug=False):
correct = 0
total = 0
losssum = 0.0
weights_offset = 0
algorithm.eval()
for i, batch in enumerate(loader):
x = batch["x"].to(device)
y = batch["y"].to(device)
with torch.no_grad():
logits = algorithm.predict(x)
loss = F.cross_entropy(logits, y).item()
B = len(x)
losssum += loss * B
if weights is None:
batch_weights = torch.ones(len(x))
else:
batch_weights = weights[weights_offset : weights_offset + len(x)]
weights_offset += len(x)
batch_weights = batch_weights.to(device)
if logits.size(1) == 1:
correct += (logits.gt(0).eq(y).float() * batch_weights).sum().item()
else:
correct += (logits.argmax(1).eq(y).float() * batch_weights).sum().item()
total += batch_weights.sum().item()
if debug:
break
algorithm.train()
acc = correct / total
loss = losssum / total
return acc, loss
def accuracy(algorithm, loader_kwargs, weights, **kwargs):
if isinstance(loader_kwargs, dict):
loader = FastDataLoader(**loader_kwargs)
elif isinstance(loader_kwargs, FastDataLoader):
loader = loader_kwargs
else:
raise ValueError(loader_kwargs)
return accuracy_from_loader(algorithm, loader, weights, **kwargs)
class Evaluator:
def __init__(
self, test_envs, eval_meta, n_envs, logger, evalmode="fast", debug=False, target_env=None
):
all_envs = list(range(n_envs))
train_envs = sorted(set(all_envs) - set(test_envs))
self.test_envs = test_envs
self.train_envs = train_envs
self.eval_meta = eval_meta
self.n_envs = n_envs
self.logger = logger
self.evalmode = evalmode
self.debug = debug
if target_env is not None:
self.set_target_env(target_env)
def set_target_env(self, target_env):
"""When len(test_envs) == 2, you can specify target env for computing exact test acc."""
self.test_envs = [target_env]
def evaluate(self, algorithm, ret_losses=False):
n_train_envs = len(self.train_envs)
n_test_envs = len(self.test_envs)
assert n_test_envs == 1
summaries = collections.defaultdict(float)
# for key order
summaries["test_in"] = 0.0
summaries["test_out"] = 0.0
summaries["train_in"] = 0.0
summaries["train_out"] = 0.0
accuracies = {}
losses = {}
# order: in_splits + out_splits.
for name, loader_kwargs, weights in self.eval_meta:
# env\d_[in|out]
env_name, inout = name.split("_")
env_num = int(env_name[3:])
skip_eval = self.evalmode == "fast" and inout == "in" and env_num not in self.test_envs
if skip_eval:
continue
is_test = env_num in self.test_envs
acc, loss = accuracy(algorithm, loader_kwargs, weights, debug=self.debug)
accuracies[name] = acc
losses[name] = loss
if env_num in self.train_envs:
summaries["train_" + inout] += acc / n_train_envs
if inout == "out":
summaries["tr_" + inout + "loss"] += loss / n_train_envs
elif is_test:
summaries["test_" + inout] += acc / n_test_envs
if ret_losses:
return accuracies, summaries, losses
else:
return accuracies, summaries
================================================
FILE: domain_generalization/swad/domainbed/hparams_registry.py
================================================
# Copyright (c) Facebook, Inc. and its affiliates. All Rights Reserved
import numpy as np
def _hparams(algorithm, dataset, random_state):
"""
Global registry of hyperparams. Each entry is a (default, random) tuple.
New algorithms / networks / etc. should add entries here.
"""
SMALL_IMAGES = ["Debug28", "RotatedMNIST", "ColoredMNIST"]
hparams = {}
hparams["data_augmentation"] = (True, True)
hparams["val_augment"] = (False, False) # augmentation for in-domain validation set
hparams["resnet18"] = (False, False)
hparams["resnet_dropout"] = (0.0, random_state.choice([0.0, 0.1, 0.5]))
hparams["class_balanced"] = (False, False)
hparams["scheduler"] = ("const", "const")
hparams["optimizer"] = ("adam", "adam")
hparams["freeze_bn"] = (True, True)
hparams["pretrained"] = (True, True) # only for ResNet
if dataset not in SMALL_IMAGES:
hparams["lr"] = (5e-5, 10 ** random_state.uniform(-5, -3.5))
if dataset == "DomainNet":
hparams["batch_size"] = (32, int(2 ** random_state.uniform(3, 5)))
else:
hparams["batch_size"] = (32, int(2 ** random_state.uniform(3, 5.5)))
if algorithm == "ARM":
hparams["batch_size"] = (8, 8)
else:
hparams["lr"] = (1e-3, 10 ** random_state.uniform(-4.5, -2.5))
hparams["batch_size"] = (64, int(2 ** random_state.uniform(3, 9)))
if dataset in SMALL_IMAGES:
hparams["weight_decay"] = (0.0, 0.0)
else:
hparams["weight_decay"] = (0.0, 10 ** random_state.uniform(-6, -2))
if algorithm in ["DANN", "CDANN"]:
if dataset not in SMALL_IMAGES:
hparams["lr_g"] = (5e-5, 10 ** random_state.uniform(-5, -3.5))
hparams["lr_d"] = (5e-5, 10 ** random_state.uniform(-5, -3.5))
else:
hparams["lr_g"] = (1e-3, 10 ** random_state.uniform(-4.5, -2.5))
hparams["lr_d"] = (1e-3, 10 ** random_state.uniform(-4.5, -2.5))
if dataset in SMALL_IMAGES:
hparams["weight_decay_g"] = (0.0, 0.0)
else:
hparams["weight_decay_g"] = (0.0, 10 ** random_state.uniform(-6, -2))
hparams["lambda"] = (1.0, 10 ** random_state.uniform(-2, 2))
hparams["weight_decay_d"] = (0.0, 10 ** random_state.uniform(-6, -2))
hparams["d_steps_per_g_step"] = (1, int(2 ** random_state.uniform(0, 3)))
hparams["grad_penalty"] = (0.0, 10 ** random_state.uniform(-2, 1))
hparams["beta1"] = (0.5, random_state.choice([0.0, 0.5]))
hparams["mlp_width"] = (256, int(2 ** random_state.uniform(6, 10)))
hparams["mlp_depth"] = (3, int(random_state.choice([3, 4, 5])))
hparams["mlp_dropout"] = (0.0, random_state.choice([0.0, 0.1, 0.5]))
elif algorithm == "RSC":
hparams["rsc_f_drop_factor"] = (1 / 3, random_state.uniform(0, 0.5))
hparams["rsc_b_drop_factor"] = (1 / 3, random_state.uniform(0, 0.5))
elif algorithm == "SagNet":
hparams["sag_w_adv"] = (0.1, 10 ** random_state.uniform(-2, 1))
elif algorithm == "IRM":
hparams["irm_lambda"] = (1e2, 10 ** random_state.uniform(-1, 5))
hparams["irm_penalty_anneal_iters"] = (
500,
int(10 ** random_state.uniform(0, 4)),
)
elif algorithm in ["Mixup", "OrgMixup"]:
hparams["mixup_alpha"] = (0.2, 10 ** random_state.uniform(-1, -1))
elif algorithm == "GroupDRO":
hparams["groupdro_eta"] = (1e-2, 10 ** random_state.uniform(-3, -1))
elif algorithm in ("MMD", "CORAL"):
hparams["mmd_gamma"] = (1.0, 10 ** random_state.uniform(-1, 1))
elif algorithm in ("MLDG", "SOMLDG"):
hparams["mldg_beta"] = (1.0, 10 ** random_state.uniform(-1, 1))
elif algorithm == "MTL":
hparams["mtl_ema"] = (0.99, random_state.choice([0.5, 0.9, 0.99, 1.0]))
elif algorithm == "VREx":
hparams["vrex_lambda"] = (1e1, 10 ** random_state.uniform(-1, 5))
hparams["vrex_penalty_anneal_iters"] = (
500,
int(10 ** random_state.uniform(0, 4)),
)
elif algorithm == "SAM":
hparams["rho"] = (0.05, random_state.choice([0.01, 0.02, 0.05, 0.1]))
elif algorithm == "CutMix":
hparams["beta"] = (1.0, 1.0)
# cutmix_prob is set to 1.0 for ImageNet and 0.5 for CIFAR100 in the original paper.
hparams["cutmix_prob"] = (1.0, 1.0)
return hparams
def default_hparams(algorithm, dataset):
dummy_random_state = np.random.RandomState(0)
return {a: b for a, (b, c) in _hparams(algorithm, dataset, dummy_random_state).items()}
def random_hparams(algorithm, dataset, seed):
random_state = np.random.RandomState(seed)
return {a: c for a, (b, c) in _hparams(algorithm, dataset, random_state).items()}
================================================
FILE: domain_generalization/swad/domainbed/lib/fast_data_loader.py
================================================
# Copyright (c) Facebook, Inc. and its affiliates. All Rights Reserved
import torch
class _InfiniteSampler(torch.utils.data.Sampler):
"""Wraps another Sampler to yield an infinite stream."""
def __init__(self, sampler):
self.sampler = sampler
def __iter__(self):
while True:
for batch in self.sampler:
yield batch
class InfiniteDataLoader:
def __init__(self, dataset, weights, batch_size, num_workers):
super().__init__()
if weights:
sampler = torch.utils.data.WeightedRandomSampler(
weights, replacement=True, num_samples=batch_size
)
else:
sampler = torch.utils.data.RandomSampler(dataset, replacement=True)
batch_sampler = torch.utils.data.BatchSampler(
sampler, batch_size=batch_size, drop_last=True
)
self._infinite_iterator = iter(
torch.utils.data.DataLoader(
dataset,
num_workers=num_workers,
batch_sampler=_InfiniteSampler(batch_sampler),
)
)
def __iter__(self):
while True:
yield next(self._infinite_iterator)
def __len__(self):
raise ValueError
class FastDataLoader:
"""
DataLoader wrapper with slightly improved speed by not respawning worker
processes at every epoch.
"""
def __init__(self, dataset, batch_size, num_workers, shuffle=False):
super().__init__()
if shuffle:
sampler = torch.utils.data.RandomSampler(dataset, replacement=False)
else:
sampler = torch.utils.data.SequentialSampler(dataset)
batch_sampler = torch.utils.data.BatchSampler(
sampler,
batch_size=batch_size,
drop_last=False,
)
self._infinite_iterator = iter(
torch.utils.data.DataLoader(
dataset,
num_workers=num_workers,
batch_sampler=_InfiniteSampler(batch_sampler),
)
)
self._length = len(batch_sampler)
def __iter__(self):
for _ in range(len(self)):
yield next(self._infinite_iterator)
def __len__(self):
return self._length
================================================
FILE: domain_generalization/swad/domainbed/lib/logger.py
================================================
""" Singleton Logger """
import sys
import logging
def levelize(levelname):
"""Convert levelname to level only if it is levelname"""
if isinstance(levelname, str):
return logging.getLevelName(levelname)
else:
return levelname # already level
class ColorFormatter(logging.Formatter):
color_dic = {
"DEBUG": 37, # white
"INFO": 36, # cyan
"WARNING": 33, # yellow
"ERROR": 31, # red
"CRITICAL": 41, # white on red bg
}
def format(self, record):
color = self.color_dic.get(record.levelname, 37) # default white
record.levelname = "\033[{}m{}\033[0m".format(color, record.levelname)
return logging.Formatter.format(self, record)
class Logger(logging.Logger):
NAME = "SingletonLogger"
@classmethod
def get(cls, file_path=None, level="INFO", colorize=True, track_code=False):
logging.setLoggerClass(cls)
logger = logging.getLogger(cls.NAME)
logging.setLoggerClass(logging.Logger) # restore
logger.setLevel(level)
if logger.hasHandlers():
# If logger already got all handlers (# handlers == 2), use the logger.
# else, re-set handlers.
if len(logger.handlers) == 2:
return logger
logger.handlers.clear()
log_format = "%(levelname)s %(asctime)s | %(message)s"
# log_format = '%(asctime)s | %(message)s'
if track_code:
log_format = (
"%(levelname)s::%(asctime)s | [%(filename)s] [%(funcName)s:%(lineno)d] "
"%(message)s"
)
date_format = "%m/%d %H:%M:%S"
if colorize:
formatter = ColorFormatter(log_format, date_format)
else:
formatter = logging.Formatter(log_format, date_format)
# standard output handler
# NOTE as default, StreamHandler use stderr stream instead of stdout stream.
# Use StreamHandler(sys.stdout) for stdout stream.
stream_handler = logging.StreamHandler(sys.stdout)
stream_handler.setFormatter(formatter)
logger.addHandler(stream_handler)
if file_path:
# file output handler
file_handler = logging.FileHandler(file_path)
file_handler.setFormatter(formatter)
logger.addHandler(file_handler)
logger.propagate = False
return logger
def nofmt(self, msg, *args, level="INFO", **kwargs):
level = levelize(level)
formatters = self.remove_formats()
super().log(level, msg, *args, **kwargs)
self.set_formats(formatters)
def remove_formats(self):
"""Remove all formats from logger"""
formatters = []
for handler in self.handlers:
formatters.append(handler.formatter)
handler.setFormatter(logging.Formatter("%(message)s"))
return formatters
def set_formats(self, formatters):
"""Set formats to every handler of logger"""
for handler, formatter in zip(self.handlers, formatters):
handler.setFormatter(formatter)
def set_file_handler(self, file_path):
file_handler = logging.FileHandler(file_path)
formatter = self.handlers[0].formatter
file_handler.setFormatter(formatter)
self.addHandler(file_handler)
================================================
FILE: domain_generalization/swad/domainbed/lib/misc.py
================================================
# Copyright (c) Facebook, Inc. and its affiliates. All Rights Reserved
"""
Things that don't belong anywhere else
"""
import hashlib
import os
import shutil
import errno
from datetime import datetime
from collections import Counter
import numpy as np
import torch
import torch.nn as nn
def make_weights_for_balanced_classes(dataset):
counts = Counter()
classes = []
for _, y in dataset:
y = int(y)
counts[y] += 1
classes.append(y)
n_classes = len(counts)
weight_per_class = {}
for y in counts:
weight_per_class[y] = 1 / (counts[y] * n_classes)
weights = torch.zeros(len(dataset))
for i, y in enumerate(classes):
weights[i] = weight_per_class[int(y)]
return weights
def seed_hash(*args):
"""
Derive an integer hash from all args, for use as a random seed.
"""
args_str = str(args)
return int(hashlib.md5(args_str.encode("utf-8")).hexdigest(), 16) % (2 ** 31)
def to_row(row, colwidth=10, latex=False):
"""Convert value list to row string"""
if latex:
sep = " & "
end_ = "\\\\"
else:
sep = " "
end_ = ""
def format_val(x):
if np.issubdtype(type(x), np.floating):
x = "{:.6f}".format(x)
return str(x).ljust(colwidth)[:colwidth]
return sep.join([format_val(x) for x in row]) + " " + end_
def random_pairs_of_minibatches(minibatches):
# n_tr_envs = len(minibatches)
perm = torch.randperm(len(minibatches)).tolist()
pairs = []
for i in range(len(minibatches)):
# j = cyclic(i + 1)
j = i + 1 if i < (len(minibatches) - 1) else 0
xi, yi = minibatches[perm[i]][0], minibatches[perm[i]][1]
xj, yj = minibatches[perm[j]][0], minibatches[perm[j]][1]
min_n = min(len(xi), len(xj))
pairs.append(((xi[:min_n], yi[:min_n]), (xj[:min_n], yj[:min_n])))
return pairs
###########################################################
# Custom utils
###########################################################
def index_conditional_iterate(skip_condition, iterable, index):
for i, x in enumerate(iterable):
if skip_condition(i):
continue
if index:
yield i, x
else:
yield x
class SplitIterator:
def __init__(self, test_envs):
self.test_envs = test_envs
def train(self, iterable, index=False):
return index_conditional_iterate(lambda idx: idx in self.test_envs, iterable, index)
def test(self, iterable, index=False):
return index_conditional_iterate(lambda idx: idx not in self.test_envs, iterable, index)
class AverageMeter:
"""Computes and stores the average and current value"""
def __init__(self):
self.reset()
def reset(self):
"""Reset all statistics"""
self.val = 0
self.avg = 0
self.sum = 0
self.count = 0
def update(self, val, n=1):
"""Update statistics"""
self.val = val
self.sum += val * n
self.count += n
self.avg = self.sum / self.count
def __repr__(self):
return "{:.3f} (val={:.3f}, count={})".format(self.avg, self.val, self.count)
class AverageMeters:
def __init__(self, *keys):
self.keys = keys
for k in keys:
setattr(self, k, AverageMeter())
def resets(self):
for k in self.keys:
getattr(self, k).reset()
def updates(self, dic, n=1):
for k, v in dic.items():
getattr(self, k).update(v, n)
def __repr__(self):
return " ".join(["{}: {}".format(k, str(getattr(self, k))) for k in self.keys])
def get_averages(self):
dic = {k: getattr(self, k).avg for k in self.keys}
return dic
def timestamp(fmt="%y%m%d_%H-%M-%S"):
return datetime.now().strftime(fmt)
def makedirs(path):
if not os.path.exists(path):
try:
os.makedirs(path)
except OSError as exc:
if exc.errno != errno.EEXIST:
raise
def rm(path):
"""remove dir recursively"""
if os.path.isdir(path):
shutil.rmtree(path, ignore_errors=True)
elif os.path.exists(path):
os.remove(path)
def cp(src, dst):
shutil.copy2(src, dst)
def get_lr(optimizer):
"""Assume that the optimizer has single lr"""
lr = optimizer.param_groups[0]["lr"]
return lr
@torch.no_grad()
def hash_bn(module):
summary = []
for m in module.modules():
if isinstance(m, (nn.BatchNorm1d, nn.BatchNorm2d, nn.BatchNorm3d)):
w = m.weight.detach().mean().item()
b = m.bias.detach().mean().item()
rm = m.running_mean.detach().mean().item()
rv = m.running_var.detach().mean().item()
summary.append((w, b, rm, rv))
w, b, rm, rv = [np.mean(col) for col in zip(*summary)]
return w, b, rm, rv
def merge_dictlist(dictlist):
"""Merge list of dicts into dict of lists, by grouping same key."""
ret = {k: [] for k in dictlist[0].keys()}
for dic in dictlist:
for data_key, v in dic.items():
ret[data_key].append(v)
return ret
================================================
FILE: domain_generalization/swad/domainbed/lib/query.py
================================================
# Copyright (c) Facebook, Inc. and its affiliates. All Rights Reserved
"""Small query library."""
import inspect
import json
import types
import warnings
import numpy as np
def make_selector_fn(selector):
"""
If selector is a function, return selector.
Otherwise, return a function corresponding to the selector string. Examples
of valid selector strings and the corresponding functions:
x lambda obj: obj['x']
x.y lambda obj: obj['x']['y']
x,y lambda obj: (obj['x'], obj['y'])
"""
if isinstance(selector, str):
if "," in selector:
parts = selector.split(",")
part_selectors = [make_selector_fn(part) for part in parts]
return lambda obj: tuple(sel(obj) for sel in part_selectors)
elif "." in selector:
parts = selector.split(".")
part_selectors = [make_selector_fn(part) for part in parts]
def f(obj):
for sel in part_selectors:
obj = sel(obj)
return obj
return f
else:
key = selector.strip()
return lambda obj: obj[key]
elif isinstance(selector, types.FunctionType):
return selector
else:
raise TypeError
def hashable(obj):
try:
hash(obj)
return obj
except TypeError:
return json.dumps({"_": obj}, sort_keys=True)
class Q(object):
def __init__(self, list_):
super(Q, self).__init__()
self._list = list_
def __len__(self):
return len(self._list)
def __getitem__(self, key):
return self._list[key]
def __eq__(self, other):
if isinstance(other, self.__class__):
return self._list == other._list
else:
return self._list == other
def __str__(self):
return str(self._list)
def __repr__(self):
return repr(self._list)
def _append(self, item):
"""Unsafe, be careful you know what you're doing."""
self._list.append(item)
def group(self, selector):
"""
Group elements by selector and return a list of (group, group_records)
tuples.
"""
selector = make_selector_fn(selector)
groups = {}
for x in self._list:
group = selector(x)
group_key = hashable(group)
if group_key not in groups:
groups[group_key] = (group, Q([]))
groups[group_key][1]._append(x)
results = [groups[key] for key in sorted(groups.keys())]
return Q(results)
def group_map(self, selector, fn):
"""
Group elements by selector, apply fn to each group, and return a list
of the results.
"""
return self.group(selector).map(fn)
def map(self, fn):
"""
map self onto fn. If fn takes multiple args, tuple-unpacking
is applied.
"""
if len(inspect.signature(fn).parameters) > 1:
return Q([fn(*x) for x in self._list])
else:
return Q([fn(x) for x in self._list])
def select(self, selector):
selector = make_selector_fn(selector)
return Q([selector(x) for x in self._list])
def min(self):
return min(self._list)
def max(self):
return max(self._list)
def sum(self):
return sum(self._list)
def len(self):
return len(self._list)
def mean(self):
with warnings.catch_warnings():
warnings.simplefilter("ignore")
return float(np.mean(self._list))
def std(self):
with warnings.catch_warnings():
warnings.simplefilter("ignore")
return float(np.std(self._list))
def mean_std(self):
return (self.mean(), self.std())
def argmax(self, selector):
selector = make_selector_fn(selector)
return max(self._list, key=selector)
def filter(self, fn):
return Q([x for x in self._list if fn(x)])
def filter_equals(self, selector, value):
"""like [x for x in y if x.selector == value]"""
selector = make_selector_fn(selector)
return self.filter(lambda r: selector(r) == value)
def filter_not_none(self):
return self.filter(lambda r: r is not None)
def filter_not_nan(self):
return self.filter(lambda r: not np.isnan(r))
def flatten(self):
return Q([y for x in self._list for y in x])
def unique(self):
result = []
result_set = set()
for x in self._list:
hashable_x = hashable(x)
if hashable_x not in result_set:
result_set.add(hashable_x)
result.append(x)
return Q(result)
def sorted(self, key=None, reverse=False):
if key is None:
key = lambda x: x
def key2(x):
x = key(x)
if isinstance(x, (np.floating, float)) and np.isnan(x):
return float("-inf")
else:
return x
return Q(sorted(self._list, key=key2, reverse=reverse))
================================================
FILE: domain_generalization/swad/domainbed/lib/swa_utils.py
================================================
# Burrowed from https://github.com/pytorch/pytorch/blob/master/torch/optim/swa_utils.py
# modified for the DomainBed.
import copy
import torch
from torch.nn import Module
from copy import deepcopy
class AveragedModel(Module):
def __init__(self, model, device=None, avg_fn=None, rm_optimizer=False):
super(AveragedModel, self).__init__()
self.start_step = -1
self.end_step = -1
if isinstance(model, AveragedModel):
# prevent nested averagedmodel
model = model.module
self.module = deepcopy(model)
if rm_optimizer:
for k, v in vars(self.module).items():
if isinstance(v, torch.optim.Optimizer):
setattr(self.module, k, None)
if device is not None:
self.module = self.module.to(device)
self.register_buffer("n_averaged", torch.tensor(0, dtype=torch.long, device=device))
if avg_fn is None:
def avg_fn(averaged_model_parameter, model_parameter, num_averaged):
return averaged_model_parameter + (model_parameter - averaged_model_parameter) / (
num_averaged + 1
)
self.avg_fn = avg_fn
def forward(self, *args, **kwargs):
# return self.predict(*args, **kwargs)
return self.module(*args, **kwargs)
def predict(self, *args, **kwargs):
return self.module(*args, **kwargs)
@property
def network(self):
return self.module.network
def update_parameters(self, model, step=None, start_step=None, end_step=None):
"""Update averaged model parameters
Args:
model: current model to update params
step: current step. step is saved for log the averaged range
start_step: set start_step only for first update
end_step: set end_step
"""
if isinstance(model, AveragedModel):
model = model.module
for p_swa, p_model in zip(self.parameters(), model.parameters()):
device = p_swa.device
p_model_ = p_model.detach().to(device)
if self.n_averaged == 0:
p_swa.detach().copy_(p_model_)
else:
p_swa.detach().copy_(
self.avg_fn(p_swa.detach(), p_model_, self.n_averaged.to(device))
)
self.n_averaged += 1
if step is not None:
if start_step is None:
start_step = step
if end_step is None:
end_step = step
if start_step is not None:
if self.n_averaged == 1:
self.start_step = start_step
if end_step is not None:
self.end_step = end_step
def clone(self):
clone = copy.deepcopy(self.module)
clone.optimizer = clone.new_optimizer(clone.network.parameters())
return clone
def cvt_dbiterator_to_loader(dbiterator, n_iter):
"""Convert DB iterator to the loader"""
for _ in range(n_iter):
minibatches = [(x, y) for x, y in next(dbiterator)]
all_x = torch.cat([x for x, y in minibatches])
all_y = torch.cat([y for x, y in minibatches])
yield all_x, all_y
@torch.no_grad()
def update_bn(iterator, model, n_steps, device="cuda"):
momenta = {}
for module in model.modules():
if isinstance(module, torch.nn.modules.batchnorm._BatchNorm):
module.running_mean = torch.zeros_like(module.running_mean)
module.running_var = torch.ones_like(module.running_var)
momenta[module] = module.momentum
if not momenta:
return
was_training = model.training
model.train()
for module in momenta.keys():
module.momentum = None
module.num_batches_tracked *= 0
for i in range(n_steps):
# batches_dictlist: [{env0_data_key: tensor, env0_...}, env1_..., ...]
batches_dictlist = next(iterator)
x = torch.cat([dic["x"] for dic in batches_dictlist])
x = x.to(device)
model(x)
for bn_module in momenta.keys():
bn_module.momentum = momenta[bn_module]
model.train(was_training)
================================================
FILE: domain_generalization/swad/domainbed/lib/wide_resnet.py
================================================
# Copyright (c) Facebook, Inc. and its affiliates. All Rights Reserved
"""
From https://github.com/meliketoy/wide-resnet.pytorch
"""
import sys
import numpy as np
import torch
import torch.nn as nn
import torch.nn.functional as F
import torch.nn.init as init
from torch.autograd import Variable
def conv3x3(in_planes, out_planes, stride=1):
return nn.Conv2d(in_planes, out_planes, kernel_size=3, stride=stride, padding=1, bias=True)
def conv_init(m):
classname = m.__class__.__name__
if classname.find("Conv") != -1:
init.xavier_uniform_(m.weight, gain=np.sqrt(2))
init.constant_(m.bias, 0)
elif classname.find("BatchNorm") != -1:
init.constant_(m.weight, 1)
init.constant_(m.bias, 0)
class wide_basic(nn.Module):
def __init__(self, in_planes, planes, dropout_rate, stride=1):
super(wide_basic, self).__init__()
self.bn1 = nn.BatchNorm2d(in_planes)
self.conv1 = nn.Conv2d(in_planes, planes, kernel_size=3, padding=1, bias=True)
self.dropout = nn.Dropout(p=dropout_rate)
self.bn2 = nn.BatchNorm2d(planes)
self.conv2 = nn.Conv2d(planes, planes, kernel_size=3, stride=stride, padding=1, bias=True)
self.shortcut = nn.Sequential()
if stride != 1 or in_planes != planes:
self.shortcut = nn.Sequential(
nn.Conv2d(in_planes, planes, kernel_size=1, stride=stride, bias=True),
)
def forward(self, x):
out = self.dropout(self.conv1(F.relu(self.bn1(x))))
out = self.conv2(F.relu(self.bn2(out)))
out += self.shortcut(x)
return out
class Wide_ResNet(nn.Module):
"""Wide Resnet with the softmax layer chopped off"""
def __init__(self, input_shape, depth, widen_factor, dropout_rate):
super(Wide_ResNet, self).__init__()
self.in_planes = 16
assert (depth - 4) % 6 == 0, "Wide-resnet depth should be 6n+4"
n = (depth - 4) / 6
k = widen_factor
# print('| Wide-Resnet %dx%d' % (depth, k))
nStages = [16, 16 * k, 32 * k, 64 * k]
self.conv1 = conv3x3(input_shape[0], nStages[0])
self.layer1 = self._wide_layer(wide_basic, nStages[1], n, dropout_rate, stride=1)
self.layer2 = self._wide_layer(wide_basic, nStages[2], n, dropout_rate, stride=2)
self.layer3 = self._wide_layer(wide_basic, nStages[3], n, dropout_rate, stride=2)
self.bn1 = nn.BatchNorm2d(nStages[3], momentum=0.9)
self.n_outputs = nStages[3]
def _wide_layer(self, block, planes, num_blocks, dropout_rate, stride):
strides = [stride] + [1] * (int(num_blocks) - 1)
layers = []
for stride in strides:
layers.append(block(self.in_planes, planes, dropout_rate, stride))
self.in_planes = planes
return nn.Sequential(*layers)
def forward(self, x):
out = self.conv1(x)
out = self.layer1(out)
out = self.layer2(out)
out = self.layer3(out)
out = F.relu(self.bn1(out))
out = F.avg_pool2d(out, 8)
return out[:, :, 0, 0]
================================================
FILE: domain_generalization/swad/domainbed/lib/writers.py
================================================
class Writer:
def add_scalars(self, tag_scalar_dic, global_step):
raise NotImplementedError()
def add_scalars_with_prefix(self, tag_scalar_dic, global_step, prefix):
tag_scalar_dic = {prefix + k: v for k, v in tag_scalar_dic.items()}
self.add_scalars(tag_scalar_dic, global_step)
class TBWriter(Writer):
def __init__(self, dir_path):
from tensorboardX import SummaryWriter
self.writer = SummaryWriter(dir_path, flush_secs=30)
def add_scalars(self, tag_scalar_dic, global_step):
for tag, scalar in tag_scalar_dic.items():
self.writer.add_scalar(tag, scalar, global_step)
def get_writer(dir_path):
"""
Args:
dir_path: tb dir
"""
writer = TBWriter(dir_path)
return writer
================================================
FILE: domain_generalization/swad/domainbed/lr_scheduler/__init__.py
================================================
from .lr_scheduler import get_scheduler
================================================
FILE: domain_generalization/swad/domainbed/lr_scheduler/lr_scheduler.py
================================================
from torch.optim import lr_scheduler
def get_scheduler(name, optimizer, lr, total_steps, final_div_factor=1e4):
name = name.lower()
if name == "onecycle":
scheduler = lr_scheduler.OneCycleLR(
optimizer, lr, total_steps=total_steps, pct_start=0.1, final_div_factor=final_div_factor
)
elif name in ["cos", "cosine"]:
eta_min = lr / final_div_factor
scheduler = lr_scheduler.CosineAnnealingLR(optimizer, T_max=total_steps, eta_min=eta_min)
elif name == "step":
decay_pct = 0.6
decay_step = int(total_steps * decay_pct)
scheduler = lr_scheduler.StepLR(optimizer, step_size=decay_step, gamma=0.1)
elif name in ["const", "swa"]:
scheduler = lr_scheduler.LambdaLR(optimizer, lr_lambda=lambda step: 1.0)
else:
raise ValueError(name)
return scheduler
================================================
FILE: domain_generalization/swad/domainbed/misc/domain_net_duplicates.txt
================================================
clipart/animal_migration/clipart_006_000227.jpg
clipart/baseball_bat/clipart_018_000011.jpg
clipart/baseball_bat/clipart_018_000021.jpg
clipart/baseball_bat/clipart_018_000051.jpg
clipart/baseball_bat/clipart_018_000072.jpg
clipart/binoculars/clipart_031_000213.jpg
clipart/bird/clipart_032_000255.jpg
clipart/book/clipart_036_000137.jpg
clipart/bowtie/clipart_039_000059.jpg
clipart/bowtie/clipart_039_000068.jpg
clipart/bowtie/clipart_039_000091.jpg
clipart/bowtie/clipart_039_000115.jpg
clipart/bowtie/clipart_039_000118.jpg
clipart/bowtie/clipart_039_000121.jpg
clipart/bowtie/clipart_039_000132.jpg
clipart/bowtie/clipart_039_000134.jpg
clipart/bowtie/clipart_039_000136.jpg
clipart/bowtie/clipart_039_000138.jpg
clipart/bracelet/clipart_040_000139.jpg
clipart/bracelet/clipart_040_000248.jpg
clipart/bread/clipart_042_000006.jpg
clipart/bread/clipart_042_000081.jpg
clipart/broom/clipart_045_000026.jpg
clipart/cake/clipart_052_000004.jpg
clipart/cake/clipart_052_000005.jpg
clipart/cake/clipart_052_000007.jpg
clipart/cake/clipart_052_000012.jpg
clipart/cake/clipart_052_000013.jpg
clipart/cake/clipart_052_000014.jpg
clipart/cake/clipart_052_000015.jpg
clipart/cake/clipart_052_000017.jpg
clipart/cake/clipart_052_000029.jpg
clipart/cake/clipart_052_000042.jpg
clipart/cake/clipart_052_000043.jpg
clipart/cake/clipart_052_000046.jpg
clipart/cake/clipart_052_000048.jpg
clipart/cake/clipart_052_000051.jpg
clipart/cake/clipart_052_000055.jpg
clipart/cake/clipart_052_000058.jpg
clipart/cake/clipart_052_000061.jpg
clipart/cake/clipart_052_000069.jpg
clipart/cake/clipart_052_000070.jpg
clipart/cake/clipart_052_000073.jpg
clipart/cake/clipart_052_000074.jpg
clipart/cake/clipart_052_000075.jpg
clipart/cake/clipart_052_000078.jpg
clipart/cake/clipart_052_000087.jpg
clipart/cake/clipart_052_000088.jpg
clipart/cake/clipart_052_000098.jpg
clipart/candle/clipart_059_000004.jpg
clipart/candle/clipart_059_000017.jpg
clipart/candle/clipart_059_000043.jpg
clipart/candle/clipart_059_000046.jpg
clipart/candle/clipart_059_000083.jpg
clipart/candle/clipart_059_000091.jpg
clipart/candle/clipart_059_000098.jpg
clipart/candle/clipart_059_000104.jpg
clipart/candle/clipart_059_000105.jpg
clipart/circle/clipart_072_000176.jpg
clipart/circle/clipart_072_000180.jpg
clipart/circle/clipart_072_000199.jpg
clipart/clock/clipart_074_000056.jpg
clipart/clock/clipart_074_000059.jpg
clipart/coffee_cup/clipart_076_000178.jpg
clipart/compass/clipart_077_000026.jpg
clipart/compass/clipart_077_000079.jpg
clipart/couch/clipart_081_000001.jpg
clipart/couch/clipart_081_000002.jpg
clipart/couch/clipart_081_000003.jpg
clipart/couch/clipart_081_000004.jpg
clipart/couch/clipart_081_000005.jpg
clipart/couch/clipart_081_000006.jpg
clipart/couch/clipart_081_000007.jpg
clipart/couch/clipart_081_000008.jpg
clipart/couch/clipart_081_000009.jpg
clipart/couch/clipart_081_000010.jpg
clipart/couch/clipart_081_000011.jpg
clipart/couch/clipart_081_000012.jpg
clipart/couch/clipart_081_000013.jpg
clipart/couch/clipart_081_000014.jpg
clipart/couch/clipart_081_000015.jpg
clipart/couch/clipart_081_000016.jpg
clipart/couch/clipart_081_000017.jpg
clipart/couch/clipart_081_000018.jpg
clipart/couch/clipart_081_000019.jpg
clipart/couch/clipart_081_000020.jpg
clipart/couch/clipart_081_000021.jpg
clipart/couch/clipart_081_000022.jpg
clipart/couch/clipart_081_000023.jpg
clipart/couch/clipart_081_000024.jpg
clipart/couch/clipart_081_000025.jpg
clipart/couch/clipart_081_000026.jpg
clipart/couch/clipart_081_000027.jpg
clipart/couch/clipart_081_000028.jpg
clipart/couch/clipart_081_000029.jpg
clipart/couch/clipart_081_000030.jpg
clipart/couch/clipart_081_000031.jpg
clipart/couch/clipart_081_000032.jpg
clipart/couch/clipart_081_000033.jpg
clipart/couch/clipart_081_000034.jpg
clipart/couch/clipart_081_000035.jpg
clipart/couch/clipart_081_000036.jpg
clipart/couch/clipart_081_000037.jpg
clipart/couch/clipart_081_000038.jpg
clipart/couch/clipart_081_000039.jpg
clipart/couch/clipart_081_000040.jpg
clipart/couch/clipart_081_000041.jpg
clipart/couch/clipart_081_000042.jpg
clipart/couch/clipart_081_000043.jpg
clipart/couch/clipart_081_000044.jpg
clipart/couch/clipart_081_000045.jpg
clipart/couch/clipart_081_000046.jpg
clipart/couch/clipart_081_000047.jpg
clipart/couch/clipart_081_000048.jpg
clipart/couch/clipart_081_000049.jpg
clipart/couch/clipart_081_000050.jpg
clipart/couch/clipart_081_000051.jpg
clipart/couch/clipart_081_000052.jpg
clipart/couch/clipart_081_000053.jpg
clipart/couch/clipart_081_000054.jpg
clipart/couch/clipart_081_000055.jpg
clipart/couch/clipart_081_000056.jpg
clipart/couch/clipart_081_000057.jpg
clipart/couch/clipart_081_000058.jpg
clipart/couch/clipart_081_000059.jpg
clipart/couch/clipart_081_000060.jpg
clipart/couch/clipart_081_000061.jpg
clipart/couch/clipart_081_000062.jpg
clipart/couch/clipart_081_000063.jpg
clipart/couch/clipart_081_000064.jpg
clipart/couch/clipart_081_000065.jpg
clipart/couch/clipart_081_000066.jpg
clipart/couch/clipart_081_000067.jpg
clipart/couch/clipart_081_000068.jpg
clipart/couch/clipart_081_000069.jpg
clipart/couch/clipart_081_000070.jpg
clipart/couch/clipart_081_000071.jpg
clipart/couch/clipart_081_000072.jpg
clipart/couch/clipart_081_000073.jpg
clipart/couch/clipart_081_000074.jpg
clipart/couch/clipart_081_000075.jpg
clipart/couch/clipart_081_000076.jpg
clipart/couch/clipart_081_000077.jpg
clipart/couch/clipart_081_000078.jpg
clipart/couch/clipart_081_000079.jpg
clipart/couch/clipart_081_000080.jpg
clipart/couch/clipart_081_000081.jpg
clipart/couch/clipart_081_000082.jpg
clipart/couch/clipart_081_000083.jpg
clipart/couch/clipart_081_000084.jpg
clipart/couch/clipart_081_000085.jpg
clipart/couch/clipart_081_000086.jpg
clipart/couch/clipart_081_000087.jpg
clipart/couch/clipart_081_000088.jpg
clipart/couch/clipart_081_000089.jpg
clipart/couch/clipart_081_000090.jpg
clipart/couch/clipart_081_000091.jpg
clipart/couch/clipart_081_000092.jpg
clipart/couch/clipart_081_000093.jpg
clipart/couch/clipart_081_000094.jpg
clipart/couch/clipart_081_000095.jpg
clipart/couch/clipart_081_000096.jpg
clipart/couch/clipart_081_000097.jpg
clipart/couch/clipart_081_000098.jpg
clipart/couch/clipart_081_000099.jpg
clipart/couch/clipart_081_000100.jpg
clipart/couch/clipart_081_000101.jpg
clipart/couch/clipart_081_000103.jpg
clipart/couch/clipart_081_000104.jpg
clipart/couch/clipart_081_000105.jpg
clipart/couch/clipart_081_000106.jpg
clipart/couch/clipart_081_000108.jpg
clipart/couch/clipart_081_000109.jpg
clipart/couch/clipart_081_000110.jpg
clipart/couch/clipart_081_000112.jpg
clipart/couch/clipart_081_000113.jpg
clipart/couch/clipart_081_000114.jpg
clipart/couch/clipart_081_000115.jpg
clipart/couch/clipart_081_000116.jpg
clipart/couch/clipart_081_000118.jpg
clipart/couch/clipart_081_000120.jpg
clipart/couch/clipart_081_000122.jpg
clipart/couch/clipart_081_000124.jpg
clipart/couch/clipart_081_000125.jpg
clipart/couch/clipart_081_000126.jpg
clipart/couch/clipart_081_000127.jpg
clipart/couch/clipart_081_000128.jpg
clipart/couch/clipart_081_000129.jpg
clipart/couch/clipart_081_000130.jpg
clipart/couch/clipart_081_000131.jpg
clipart/couch/clipart_081_000132.jpg
clipart/couch/clipart_081_000133.jpg
clipart/couch/clipart_081_000134.jpg
clipart/couch/clipart_081_000135.jpg
clipart/couch/clipart_081_000136.jpg
clipart/couch/clipart_081_000137.jpg
clipart/couch/clipart_081_000138.jpg
clipart/couch/clipart_081_000139.jpg
clipart/couch/clipart_081_000140.jpg
clipart/couch/clipart_081_000141.jpg
clipart/couch/clipart_081_000143.jpg
clipart/couch/clipart_081_000144.jpg
clipart/couch/clipart_081_000145.jpg
clipart/couch/clipart_081_000146.jpg
clipart/couch/clipart_081_000147.jpg
clipart/couch/clipart_081_000148.jpg
clipart/couch/clipart_081_000149.jpg
clipart/couch/clipart_081_000150.jpg
clipart/couch/clipart_081_000151.jpg
clipart/couch/clipart_081_000152.jpg
clipart/couch/clipart_081_000153.jpg
clipart/couch/clipart_081_000154.jpg
clipart/couch/clipart_081_000155.jpg
clipart/couch/clipart_081_000156.jpg
clipart/couch/clipart_081_000157.jpg
clipart/couch/clipart_081_000158.jpg
clipart/couch/clipart_081_000159.jpg
clipart/couch/clipart_081_000160.jpg
clipart/couch/clipart_081_000161.jpg
clipart/couch/clipart_081_000162.jpg
clipart/couch/clipart_081_000163.jpg
clipart/couch/clipart_081_000164.jpg
clipart/couch/clipart_081_000165.jpg
clipart/couch/clipart_081_000166.jpg
clipart/couch/clipart_081_000167.jpg
clipart/couch/clipart_081_000168.jpg
clipart/couch/clipart_081_000169.jpg
clipart/couch/clipart_081_000171.jpg
clipart/couch/clipart_081_000173.jpg
clipart/couch/clipart_081_000174.jpg
clipart/couch/clipart_081_000175.jpg
clipart/couch/clipart_081_000176.jpg
clipart/couch/clipart_081_000178.jpg
clipart/couch/clipart_081_000180.jpg
clipart/couch/clipart_081_000181.jpg
clipart/couch/clipart_081_000182.jpg
clipart/couch/clipart_081_000183.jpg
clipart/couch/clipart_081_000184.jpg
clipart/couch/clipart_081_000185.jpg
clipart/couch/clipart_081_000186.jpg
clipart/couch/clipart_081_000187.jpg
clipart/couch/clipart_081_000189.jpg
clipart/couch/clipart_081_000190.jpg
clipart/couch/clipart_081_000192.jpg
clipart/couch/clipart_081_000193.jpg
clipart/couch/clipart_081_000194.jpg
clipart/couch/clipart_081_000195.jpg
clipart/couch/clipart_081_000196.jpg
clipart/couch/clipart_081_000197.jpg
clipart/couch/clipart_081_000198.jpg
clipart/couch/clipart_081_000199.jpg
clipart/couch/clipart_081_000200.jpg
clipart/couch/clipart_081_000201.jpg
clipart/couch/clipart_081_000202.jpg
clipart/couch/clipart_081_000203.jpg
clipart/couch/clipart_081_000204.jpg
clipart/couch/clipart_081_000205.jpg
clipart/couch/clipart_081_000206.jpg
clipart/couch/clipart_081_000207.jpg
clipart/couch/clipart_081_000208.jpg
clipart/couch/clipart_081_000209.jpg
clipart/couch/clipart_081_000210.jpg
clipart/couch/clipart_081_000212.jpg
clipart/couch/clipart_081_000213.jpg
clipart/couch/clipart_081_000214.jpg
clipart/couch/clipart_081_000215.jpg
clipart/couch/clipart_081_000216.jpg
clipart/couch/clipart_081_000217.jpg
clipart/couch/clipart_081_000218.jpg
clipart/couch/clipart_081_000219.jpg
clipart/couch/clipart_081_000220.jpg
clipart/couch/clipart_081_000221.jpg
clipart/couch/clipart_081_000222.jpg
clipart/couch/clipart_081_000224.jpg
clipart/couch/clipart_081_000225.jpg
clipart/couch/clipart_081_000228.jpg
clipart/couch/clipart_081_000229.jpg
clipart/couch/clipart_081_000230.jpg
clipart/couch/clipart_081_000231.jpg
clipart/couch/clipart_081_000232.jpg
clipart/crown/clipart_086_000085.jpg
clipart/cruise_ship/clipart_087_000086.jpg
clipart/cruise_ship/clipart_087_000174.jpg
clipart/cup/clipart_088_000001.jpg
clipart/cup/clipart_088_000004.jpg
clipart/cup/clipart_088_000008.jpg
clipart/cup/clipart_088_000009.jpg
clipart/cup/clipart_088_000011.jpg
clipart/cup/clipart_088_000012.jpg
clipart/cup/clipart_088_000030.jpg
clipart/cup/clipart_088_000032.jpg
clipart/cup/clipart_088_000039.jpg
clipart/cup/clipart_088_000042.jpg
clipart/cup/clipart_088_000069.jpg
clipart/cup/clipart_088_000072.jpg
clipart/cup/clipart_088_000077.jpg
clipart/cup/clipart_088_000079.jpg
clipart/cup/clipart_088_000081.jpg
clipart/cup/clipart_088_000082.jpg
clipart/cup/clipart_088_000087.jpg
clipart/cup/clipart_088_000091.jpg
clipart/cup/clipart_088_000093.jpg
clipart/cup/clipart_088_000094.jpg
clipart/cup/clipart_088_000096.jpg
clipart/cup/clipart_088_000108.jpg
clipart/cup/clipart_088_000111.jpg
clipart/cup/clipart_088_000113.jpg
clipart/cup/clipart_088_000117.jpg
clipart/drill/clipart_098_000095.jpg
clipart/dumbbell/clipart_101_000079.jpg
clipart/eraser/clipart_106_000069.jpg
clipart/eyeglasses/clipart_108_000102.jpg
clipart/eyeglasses/clipart_108_000127.jpg
clipart/face/clipart_109_000035.jpg
clipart/fan/clipart_110_000055.jpg
clipart/fan/clipart_110_000057.jpg
clipart/fan/clipart_110_000069.jpg
clipart/fan/clipart_110_000076.jpg
clipart/fan/clipart_110_000085.jpg
clipart/fan/clipart_110_000115.jpg
clipart/fan/clipart_110_000120.jpg
clipart/fence/clipart_112_000163.jpg
clipart/firetruck/clipart_116_000028.jpg
clipart/flashlight/clipart_119_000152.jpg
clipart/flower/clipart_122_000069.jpg
clipart/foot/clipart_124_000023.jpg
clipart/fork/clipart_125_000106.jpg
clipart/fork/clipart_125_000184.jpg
clipart/garden/clipart_128_000049.jpg
clipart/garden/clipart_128_000056.jpg
clipart/garden/clipart_128_000058.jpg
clipart/garden/clipart_128_000059.jpg
clipart/garden/clipart_128_000061.jpg
clipart/garden/clipart_128_000063.jpg
clipart/giraffe/clipart_130_000004.jpg
clipart/goatee/clipart_131_000002.jpg
clipart/goatee/clipart_131_000005.jpg
clipart/goatee/clipart_131_000015.jpg
clipart/goatee/clipart_131_000017.jpg
clipart/goatee/clipart_131_000037.jpg
clipart/goatee/clipart_131_000043.jpg
clipart/goatee/clipart_131_000057.jpg
clipart/goatee/clipart_131_000067.jpg
clipart/goatee/clipart_131_000068.jpg
clipart/goatee/clipart_131_000092.jpg
clipart/goatee/clipart_131_000101.jpg
clipart/goatee/clipart_131_000104.jpg
clipart/goatee/clipart_131_000118.jpg
clipart/goatee/clipart_131_000123.jpg
clipart/goatee/clipart_131_000124.jpg
clipart/goatee/clipart_131_000130.jpg
clipart/goatee/clipart_131_000132.jpg
clipart/goatee/clipart_131_000152.jpg
clipart/goatee/clipart_131_000155.jpg
clipart/goatee/clipart_131_000166.jpg
clipart/goatee/clipart_131_000179.jpg
clipart/goatee/clipart_131_000182.jpg
clipart/goatee/clipart_131_000184.jpg
clipart/goatee/clipart_131_000195.jpg
clipart/goatee/clipart_131_000196.jpg
clipart/goatee/clipart_131_000202.jpg
clipart/goatee/clipart_131_000203.jpg
clipart/goatee/clipart_131_000214.jpg
clipart/goatee/clipart_131_000227.jpg
clipart/goatee/clipart_131_000237.jpg
clipart/goatee/clipart_131_000244.jpg
clipart/goatee/clipart_131_000254.jpg
clipart/golf_club/clipart_132_000088.jpg
clipart/golf_club/clipart_132_000147.jpg
clipart/grass/clipart_134_000051.jpg
clipart/grass/clipart_134_000110.jpg
clipart/grass/clipart_134_000123.jpg
clipart/guitar/clipart_135_000023.jpg
clipart/guitar/clipart_135_000044.jpg
clipart/guitar/clipart_135_000060.jpg
clipart/hamburger/clipart_136_000079.jpg
clipart/hamburger/clipart_136_000080.jpg
clipart/hamburger/clipart_136_000085.jpg
clipart/hamburger/clipart_136_000154.jpg
clipart/hamburger/clipart_136_000159.jpg
clipart/hamburger/clipart_136_000160.jpg
clipart/hamburger/clipart_136_000162.jpg
clipart/hamburger/clipart_136_000167.jpg
clipart/hamburger/clipart_136_000169.jpg
clipart/hamburger/clipart_136_000170.jpg
clipart/hamburger/clipart_136_000171.jpg
clipart/hamburger/clipart_136_000173.jpg
clipart/hamburger/clipart_136_000175.jpg
clipart/hamburger/clipart_136_000177.jpg
clipart/hamburger/clipart_136_000179.jpg
clipart/hamburger/clipart_136_000181.jpg
clipart/hamburger/clipart_136_000187.jpg
clipart/hand/clipart_138_000038.jpg
clipart/hand/clipart_138_000044.jpg
clipart/hand/clipart_138_000060.jpg
clipart/hand/clipart_138_000087.jpg
clipart/hand/clipart_138_000095.jpg
clipart/harp/clipart_139_000056.jpg
clipart/harp/clipart_139_000200.jpg
clipart/headphones/clipart_141_000001.jpg
clipart/headphones/clipart_141_000282.jpg
clipart/helicopter/clipart_143_000115.jpg
clipart/helicopter/clipart_143_000127.jpg
clipart/hexagon/clipart_145_000112.jpg
clipart/hexagon/clipart_145_000192.jpg
clipart/hockey_stick/clipart_147_000036.jpg
clipart/hockey_stick/clipart_147_000135.jpg
clipart/horse/clipart_148_000062.jpg
clipart/hot_air_balloon/clipart_150_000181.jpg
clipart/hot_dog/clipart_151_000009.jpg
clipart/hot_tub/clipart_152_000106.jpg
clipart/house/clipart_154_000019.jpg
clipart/hurricane/clipart_156_000004.jpg
clipart/jail/clipart_159_000014.jpg
clipart/kangaroo/clipart_160_000033.jpg
clipart/key/clipart_161_000036.jpg
clipart/keyboard/clipart_162_000010.jpg
clipart/laptop/clipart_167_000013.jpg
clipart/laptop/clipart_167_000014.jpg
clipart/leg/clipart_169_000052.jpg
clipart/lightning/clipart_173_000055.jpg
clipart/lightning/clipart_173_000070.jpg
clipart/lightning/clipart_173_000121.jpg
clipart/lightning/clipart_173_000137.jpg
clipart/matches/clipart_182_000024.jpg
clipart/microphone/clipart_185_000070.jpg
clipart/microphone/clipart_185_000110.jpg
clipart/microphone/clipart_185_000132.jpg
clipart/monkey/clipart_187_000025.jpg
clipart/monkey/clipart_187_000076.jpg
clipart/monkey/clipart_187_000098.jpg
clipart/monkey/clipart_187_000120.jpg
clipart/moustache/clipart_193_000021.jpg
clipart/moustache/clipart_193_000040.jpg
clipart/moustache/clipart_193_000064.jpg
clipart/moustache/clipart_193_000081.jpg
clipart/moustache/clipart_193_000102.jpg
clipart/moustache/clipart_193_000122.jpg
clipart/moustache/clipart_193_000140.jpg
clipart/mug/clipart_195_000010.jpg
clipart/mug/clipart_195_000028.jpg
clipart/mug/clipart_195_000061.jpg
clipart/mug/clipart_195_000076.jpg
clipart/mug/clipart_195_000083.jpg
clipart/mug/clipart_195_000094.jpg
clipart/mug/clipart_195_000100.jpg
clipart/mug/clipart_195_000103.jpg
clipart/mug/clipart_195_000123.jpg
clipart/mug/clipart_195_000126.jpg
clipart/mug/clipart_195_000128.jpg
clipart/mug/clipart_195_000131.jpg
clipart/mug/clipart_195_000134.jpg
clipart/mug/clipart_195_000141.jpg
clipart/mug/clipart_195_000153.jpg
clipart/mug/clipart_195_000161.jpg
clipart/mushroom/clipart_196_000132.jpg
clipart/nail/clipart_197_000034.jpg
clipart/necklace/clipart_198_000008.jpg
clipart/necklace/clipart_198_000038.jpg
clipart/necklace/clipart_198_000043.jpg
clipart/necklace/clipart_198_000057.jpg
clipart/ocean/clipart_200_000052.jpg
clipart/octagon/clipart_201_000016.jpg
clipart/octagon/clipart_201_000024.jpg
clipart/octagon/clipart_201_000026.jpg
clipart/owl/clipart_205_000089.jpg
clipart/owl/clipart_205_000121.jpg
clipart/paint_can/clipart_207_000006.jpg
clipart/paper_clip/clipart_211_000015.jpg
clipart/paper_clip/clipart_211_000045.jpg
clipart/paper_clip/clipart_211_000051.jpg
clipart/paper_clip/clipart_211_000058.jpg
clipart/paper_clip/clipart_211_000092.jpg
clipart/paper_clip/clipart_211_000097.jpg
clipart/paper_clip/clipart_211_000112.jpg
clipart/parrot/clipart_213_000016.jpg
clipart/parrot/clipart_213_000027.jpg
clipart/parrot/clipart_213_000068.jpg
clipart/peanut/clipart_215_000035.jpg
clipart/picture_frame/clipart_222_000074.jpg
clipart/picture_frame/clipart_222_000087.jpg
clipart/pig/clipart_223_000029.jpg
clipart/pillow/clipart_224_000026.jpg
clipart/police_car/clipart_228_000028.jpg
clipart/pond/clipart_229_000051.jpg
clipart/pond/clipart_229_000092.jpg
clipart/pool/clipart_230_000001.jpg
clipart/popsicle/clipart_231_000005.jpg
clipart/popsicle/clipart_231_000072.jpg
clipart/popsicle/clipart_231_000093.jpg
clipart/popsicle/clipart_231_000179.jpg
clipart/purse/clipart_235_000010.jpg
clipart/purse/clipart_235_000041.jpg
clipart/rabbit/clipart_236_000057.jpg
clipart/rabbit/clipart_236_000063.jpg
clipart/raccoon/clipart_237_000046.jpg
clipart/rain/clipart_239_000009.jpg
clipart/rain/clipart_239_000013.jpg
clipart/rain/clipart_239_000018.jpg
clipart/rain/clipart_239_000022.jpg
clipart/rain/clipart_239_000033.jpg
clipart/rain/clipart_239_000035.jpg
clipart/rain/clipart_239_000042.jpg
clipart/rain/clipart_239_000047.jpg
clipart/rainbow/clipart_240_000034.jpg
clipart/remote_control/clipart_242_000013.jpg
clipart/river/clipart_245_000076.jpg
clipart/river/clipart_245_000082.jpg
clipart/rollerskates/clipart_247_000022.jpg
clipart/rollerskates/clipart_247_000068.jpg
clipart/rollerskates/clipart_247_000069.jpg
clipart/sandwich/clipart_249_000045.jpg
clipart/sandwich/clipart_249_000064.jpg
clipart/sandwich/clipart_249_000088.jpg
clipart/sandwich/clipart_249_000109.jpg
clipart/saxophone/clipart_251_000091.jpg
clipart/school_bus/clipart_252_000001.jpg
clipart/school_bus/clipart_252_000023.jpg
clipart/school_bus/clipart_252_000026.jpg
clipart/school_bus/clipart_252_000029.jpg
clipart/school_bus/clipart_252_000043.jpg
clipart/school_bus/clipart_252_000044.jpg
clipart/school_bus/clipart_252_000048.jpg
clipart/school_bus/clipart_252_000063.jpg
clipart/school_bus/clipart_252_000069.jpg
clipart/school_bus/clipart_252_000084.jpg
clipart/school_bus/clipart_252_000088.jpg
clipart/school_bus/clipart_252_000089.jpg
clipart/school_bus/clipart_252_000094.jpg
clipart/school_bus/clipart_252_000096.jpg
clipart/school_bus/clipart_252_000135.jpg
clipart/school_bus/clipart_252_000142.jpg
clipart/school_bus/clipart_252_000147.jpg
clipart/school_bus/clipart_252_000159.jpg
clipart/school_bus/clipart_252_000167.jpg
clipart/school_bus/clipart_252_000169.jpg
clipart/school_bus/clipart_252_000177.jpg
clipart/school_bus/clipart_252_000178.jpg
clipart/school_bus/clipart_252_000183.jpg
clipart/school_bus/clipart_252_000186.jpg
clipart/school_bus/clipart_252_000188.jpg
clipart/school_bus/clipart_252_000189.jpg
clipart/school_bus/clipart_252_000198.jpg
clipart/school_bus/clipart_252_000221.jpg
clipart/school_bus/clipart_252_000229.jpg
clipart/scissors/clipart_253_000088.jpg
clipart/sea_turtle/clipart_256_000177.jpg
clipart/sheep/clipart_259_000030.jpg
clipart/shoe/clipart_260_000020.jpg
clipart/shorts/clipart_261_000097.jpg
clipart/shorts/clipart_261_000108.jpg
clipart/shorts/clipart_261_000113.jpg
clipart/shovel/clipart_262_000153.jpg
clipart/shovel/clipart_262_000188.jpg
clipart/skateboard/clipart_264_000191.jpg
clipart/skateboard/clipart_264_000220.jpg
clipart/smiley_face/clipart_268_000006.jpg
clipart/snail/clipart_269_000070.jpg
clipart/soccer_ball/clipart_274_000063.jpg
clipart/sock/clipart_275_000034.jpg
clipart/speedboat/clipart_276_000011.jpg
clipart/speedboat/clipart_276_000109.jpg
clipart/speedboat/clipart_276_000111.jpg
clipart/spoon/clipart_278_000162.jpg
clipart/spreadsheet/clipart_279_000150.jpg
clipart/square/clipart_280_000034.jpg
clipart/square/clipart_280_000066.jpg
clipart/square/clipart_280_000123.jpg
clipart/square/clipart_280_000153.jpg
clipart/squiggle/clipart_281_000021.jpg
clipart/star/clipart_284_000105.jpg
clipart/star/clipart_284_000108.jpg
clipart/stereo/clipart_286_000168.jpg
clipart/stitches/clipart_288_000163.jpg
clipart/stop_sign/clipart_289_000125.jpg
clipart/stove/clipart_290_000161.jpg
clipart/stove/clipart_290_000196.jpg
clipart/strawberry/clipart_291_000003.jpg
clipart/suitcase/clipart_295_000052.jpg
clipart/suitcase/clipart_295_000250.jpg
clipart/sun/clipart_296_000112.jpg
clipart/sun/clipart_296_000139.jpg
clipart/sun/clipart_296_000178.jpg
clipart/sun/clipart_296_000210.jpg
clipart/swan/clipart_297_000104.jpg
clipart/swan/clipart_297_000313.jpg
clipart/swing_set/clipart_299_000105.jpg
clipart/swing_set/clipart_299_000123.jpg
clipart/swing_set/clipart_299_000128.jpg
clipart/sword/clipart_300_000002.jpg
clipart/sword/clipart_300_000044.jpg
clipart/sword/clipart_300_000131.jpg
clipart/syringe/clipart_301_000010.jpg
clipart/syringe/clipart_301_000024.jpg
clipart/syringe/clipart_301_000084.jpg
clipart/table/clipart_302_000115.jpg
clipart/teapot/clipart_303_000181.jpg
clipart/teddy-bear/clipart_304_000003.jpg
clipart/teddy-bear/clipart_304_000004.jpg
clipart/teddy-bear/clipart_304_000013.jpg
clipart/teddy-bear/clipart_304_000014.jpg
clipart/teddy-bear/clipart_304_000022.jpg
clipart/teddy-bear/clipart_304_000024.jpg
clipart/teddy-bear/clipart_304_000073.jpg
clipart/teddy-bear/clipart_304_000077.jpg
clipart/teddy-bear/clipart_304_000078.jpg
clipart/teddy-bear/clipart_304_000087.jpg
clipart/teddy-bear/clipart_304_000089.jpg
clipart/teddy-bear/clipart_304_000092.jpg
clipart/teddy-bear/clipart_304_000098.jpg
clipart/teddy-bear/clipart_304_000102.jpg
clipart/telephone/clipart_305_000130.jpg
clipart/television/clipart_306_000079.jpg
clipart/tennis_racquet/clipart_307_000052.jpg
clipart/The_Eiffel_Tower/clipart_309_000077.jpg
clipart/The_Great_Wall_of_China/clipart_310_000014.jpg
clipart/tiger/clipart_312_000191.jpg
clipart/toe/clipart_314_000029.jpg
clipart/toe/clipart_314_000050.jpg
clipart/toe/clipart_314_000072.jpg
clipart/toe/clipart_314_000083.jpg
clipart/toilet/clipart_315_000040.jpg
clipart/toilet/clipart_315_000076.jpg
clipart/toilet/clipart_315_000101.jpg
clipart/tooth/clipart_316_000052.jpg
clipart/tooth/clipart_316_000081.jpg
clipart/toothbrush/clipart_317_000144.jpg
clipart/toothpaste/clipart_318_000004.jpg
clipart/toothpaste/clipart_318_000011.jpg
clipart/toothpaste/clipart_318_000014.jpg
clipart/toothpaste/clipart_318_000031.jpg
clipart/toothpaste/clipart_318_000051.jpg
clipart/toothpaste/clipart_318_000053.jpg
clipart/toothpaste/clipart_318_000089.jpg
clipart/toothpaste/clipart_318_000090.jpg
clipart/tornado/clipart_319_000004.jpg
clipart/tornado/clipart_319_000041.jpg
clipart/tornado/clipart_319_000111.jpg
clipart/tornado/clipart_319_000129.jpg
clipart/tornado/clipart_319_000132.jpg
clipart/tractor/clipart_320_000016.jpg
clipart/traffic_light/clipart_321_000038.jpg
clipart/traffic_light/clipart_321_000065.jpg
clipart/traffic_light/clipart_321_000198.jpg
clipart/train/clipart_322_000090.jpg
clipart/tree/clipart_323_000023.jpg
clipart/tree/clipart_323_000112.jpg
clipart/triangle/clipart_324_000013.jpg
clipart/triangle/clipart_324_000040.jpg
clipart/triangle/clipart_324_000041.jpg
clipart/triangle/clipart_324_000066.jpg
clipart/triangle/clipart_324_000075.jpg
clipart/triangle/clipart_324_000138.jpg
clipart/triangle/clipart_324_000166.jpg
clipart/trombone/clipart_325_000038.jpg
clipart/trombone/clipart_325_000178.jpg
clipart/truck/clipart_326_000005.jpg
clipart/truck/clipart_326_000009.jpg
clipart/truck/clipart_326_000019.jpg
clipart/truck/clipart_326_000028.jpg
clipart/truck/clipart_326_000032.jpg
clipart/truck/clipart_326_000041.jpg
clipart/truck/clipart_326_000042.jpg
clipart/truck/clipart_326_000079.jpg
clipart/truck/clipart_326_000099.jpg
clipart/truck/clipart_326_000110.jpg
clipart/trumpet/clipart_327_000082.jpg
clipart/trumpet/clipart_327_000107.jpg
clipart/t-shirt/clipart_328_000019.jpg
clipart/umbrella/clipart_329_000064.jpg
clipart/umbrella/clipart_329_000072.jpg
clipart/umbrella/clipart_329_000090.jpg
clipart/umbrella/clipart_329_000091.jpg
clipart/umbrella/clipart_329_000131.jpg
clipart/underwear/clipart_330_000081.jpg
clipart/underwear/clipart_330_000084.jpg
clipart/van/clipart_331_000048.jpg
clipart/van/clipart_331_000049.jpg
clipart/van/clipart_331_000120.jpg
clipart/van/clipart_331_000152.jpg
clipart/van/clipart_331_000172.jpg
clipart/vase/clipart_332_000018.jpg
clipart/violin/clipart_333_000028.jpg
clipart/violin/clipart_333_000048.jpg
clipart/violin/clipart_333_000095.jpg
clipart/washing_machine/clipart_334_000151.jpg
clipart/watermelon/clipart_335_000011.jpg
clipart/watermelon/clipart_335_000134.jpg
clipart/whale/clipart_337_000282.jpg
clipart/wheel/clipart_338_000042.jpg
clipart/wine_bottle/clipart_340_000182.jpg
clipart/wine_bottle/clipart_340_000212.jpg
clipart/wine_glass/clipart_341_000087.jpg
clipart/wine_glass/clipart_341_000112.jpg
clipart/wine_glass/clipart_341_000172.jpg
clipart/wine_glass/clipart_341_000206.jpg
clipart/yoga/clipart_343_000049.jpg
clipart/zebra/clipart_344_000105.jpg
clipart/zigzag/clipart_345_000011.jpg
clipart/zigzag/clipart_345_000071.jpg
clipart/zigzag/clipart_345_000097.jpg
clipart/zigzag/clipart_345_000137.jpg
clipart/zigzag/clipart_345_000188.jpg
clipart/zigzag/clipart_345_000271.jpg
infograph/axe/infograph_012_000052.jpg
infograph/axe/infograph_012_000070.jpg
infograph/axe/infograph_012_000071.jpg
infograph/axe/infograph_012_000078.jpg
infograph/backpack/infograph_013_000100.jpg
infograph/banana/infograph_014_000063.jpg
infograph/banana/infograph_014_000272.jpg
infograph/bandage/infograph_015_000116.jpg
infograph/bandage/infograph_015_000182.jpg
infograph/barn/infograph_016_000105.jpg
infograph/barn/infograph_016_000110.jpg
infograph/barn/infograph_016_000129.jpg
infograph/baseball/infograph_017_000102.jpg
infograph/baseball_bat/infograph_018_000028.jpg
infograph/baseball_bat/infograph_018_000029.jpg
infograph/baseball_bat/infograph_018_000058.jpg
infograph/baseball_bat/infograph_018_000061.jpg
infograph/baseball_bat/infograph_018_000075.jpg
infograph/baseball_bat/infograph_018_000092.jpg
infograph/baseball_bat/infograph_018_000113.jpg
infograph/baseball_bat/infograph_018_000126.jpg
infograph/baseball_bat/infograph_018_000130.jpg
infograph/baseball_bat/infograph_018_000149.jpg
infograph/baseball_bat/infograph_018_000159.jpg
infograph/baseball_bat/infograph_018_000177.jpg
infograph/baseball_bat/infograph_018_000181.jpg
infograph/baseball_bat/infograph_018_000233.jpg
infograph/baseball_bat/infograph_018_000236.jpg
infograph/baseball_bat/infograph_018_000237.jpg
infograph/baseball_bat/infograph_018_000266.jpg
infograph/baseball_bat/infograph_018_000347.jpg
infograph/basketball/infograph_020_000014.jpg
infograph/basketball/infograph_020_000064.jpg
infograph/basketball/infograph_020_000110.jpg
infograph/basketball/infograph_020_000140.jpg
infograph/basketball/infograph_020_000143.jpg
infograph/basketball/infograph_020_000157.jpg
infograph/bat/infograph_021_000057.jpg
infograph/bat/infograph_021_000083.jpg
infograph/bear/infograph_024_000080.jpg
infograph/bee/infograph_027_000016.jpg
infograph/bench/infograph_029_000037.jpg
infograph/bird/infograph_032_000001.jpg
infograph/bird/infograph_032_000068.jpg
infograph/bird/infograph_032_000074.jpg
infograph/bird/infograph_032_000127.jpg
infograph/bird/infograph_032_000158.jpg
infograph/bird/infograph_032_000199.jpg
infograph/bread/infograph_042_000060.jpg
infograph/bread/infograph_042_000087.jpg
infograph/bread/infograph_042_000214.jpg
infograph/broccoli/infograph_044_000033.jpg
infograph/broccoli/infograph_044_000085.jpg
infograph/broccoli/infograph_044_000129.jpg
infograph/broccoli/infograph_044_000159.jpg
infograph/broccoli/infograph_044_000194.jpg
infograph/broccoli/infograph_044_000228.jpg
infograph/broccoli/infograph_044_000229.jpg
infograph/butterfly/infograph_050_000071.jpg
infograph/butterfly/infograph_050_000135.jpg
infograph/cake/infograph_052_000003.jpg
infograph/cake/infograph_052_000005.jpg
infograph/cake/infograph_052_000013.jpg
infograph/cake/infograph_052_000034.jpg
infograph/cake/infograph_052_000061.jpg
infograph/cake/infograph_052_000067.jpg
infograph/cake/infograph_052_000080.jpg
infograph/cake/infograph_052_000087.jpg
infograph/cake/infograph_052_000098.jpg
infograph/cake/infograph_052_000116.jpg
infograph/cake/infograph_052_000118.jpg
infograph/cake/infograph_052_000134.jpg
infograph/cake/infograph_052_000140.jpg
infograph/camera/infograph_056_000066.jpg
infograph/candle/infograph_059_000018.jpg
infograph/candle/infograph_059_000021.jpg
infograph/candle/infograph_059_000033.jpg
infograph/candle/infograph_059_000048.jpg
infograph/canoe/infograph_061_000010.jpg
infograph/carrot/infograph_063_000036.jpg
infograph/carrot/infograph_063_000038.jpg
infograph/carrot/infograph_063_000048.jpg
infograph/carrot/infograph_063_000056.jpg
infograph/carrot/infograph_063_000091.jpg
infograph/carrot/infograph_063_000105.jpg
infograph/carrot/infograph_063_000108.jpg
infograph/carrot/infograph_063_000173.jpg
infograph/carrot/infograph_063_000196.jpg
infograph/carrot/infograph_063_000197.jpg
infograph/carrot/infograph_063_000220.jpg
infograph/carrot/infograph_063_000229.jpg
infograph/carrot/infograph_063_000238.jpg
infograph/cell_phone/infograph_068_000009.jpg
infograph/cell_phone/infograph_068_000015.jpg
infograph/cell_phone/infograph_068_000139.jpg
infograph/chair/infograph_069_000017.jpg
infograph/chair/infograph_069_000092.jpg
infograph/clock/infograph_074_000017.jpg
infograph/computer/infograph_078_000044.jpg
infograph/cooler/infograph_080_000004.jpg
infograph/couch/infograph_081_000004.jpg
infograph/couch/infograph_081_000033.jpg
infograph/couch/infograph_081_000055.jpg
infograph/cow/infograph_082_000009.jpg
infograph/crocodile/infograph_085_000049.jpg
infograph/cup/infograph_088_000002.jpg
infograph/cup/infograph_088_000004.jpg
infograph/cup/infograph_088_000005.jpg
infograph/cup/infograph_088_000007.jpg
infograph/cup/infograph_088_000021.jpg
infograph/cup/infograph_088_000029.jpg
infograph/cup/infograph_088_000033.jpg
infograph/cup/infograph_088_000036.jpg
infograph/cup/infograph_088_000041.jpg
infograph/cup/infograph_088_000051.jpg
infograph/cup/infograph_088_000052.jpg
infograph/diving_board/infograph_091_000011.jpg
infograph/dog/infograph_092_000005.jpg
infograph/dog/infograph_092_000007.jpg
infograph/dog/infograph_092_000090.jpg
infograph/dog/infograph_092_000197.jpg
infograph/dog/infograph_092_000215.jpg
infograph/dolphin/infograph_093_000028.jpg
infograph/dolphin/infograph_093_000040.jpg
infograph/dolphin/infograph_093_000049.jpg
infograph/dolphin/infograph_093_000052.jpg
infograph/dolphin/infograph_093_000080.jpg
infograph/dolphin/infograph_093_000100.jpg
infograph/donut/infograph_094_000020.jpg
infograph/drill/infograph_098_000021.jpg
infograph/duck/infograph_100_000043.jpg
infograph/dumbbell/infograph_101_000021.jpg
infograph/dumbbell/infograph_101_000022.jpg
infograph/dumbbell/infograph_101_000047.jpg
infograph/elbow/infograph_103_000003.jpg
infograph/elbow/infograph_103_000010.jpg
infograph/elbow/infograph_103_000033.jpg
infograph/elbow/infograph_103_000052.jpg
infograph/elephant/infograph_104_000101.jpg
infograph/eye/infograph_107_000097.jpg
infograph/eyeglasses/infograph_108_000097.jpg
infograph/eyeglasses/infograph_108_000105.jpg
infograph/face/infograph_109_000005.jpg
infograph/face/infograph_109_000010.jpg
infograph/face/infograph_109_000033.jpg
infograph/face/infograph_109_000081.jpg
infograph/fan/infograph_110_000003.jpg
infograph/fan/infograph_110_000009.jpg
infograph/fan/infograph_110_000013.jpg
infograph/fan/infograph_110_000014.jpg
infograph/fan/infograph_110_000017.jpg
infograph/fan/infograph_110_000022.jpg
infograph/fan/infograph_110_000028.jpg
infograph/fan/infograph_110_000032.jpg
infograph/fan/infograph_110_000033.jpg
infograph/fan/infograph_110_000035.jpg
infograph/fan/infograph_110_000042.jpg
infograph/fan/infograph_110_000043.jpg
infograph/fan/infograph_110_000046.jpg
infograph/feather/infograph_111_000033.jpg
infograph/feather/infograph_111_000044.jpg
infograph/feather/infograph_111_000063.jpg
infograph/feather/infograph_111_000084.jpg
infograph/feather/infograph_111_000162.jpg
infograph/feather/infograph_111_000184.jpg
infograph/feather/infograph_111_000189.jpg
infograph/feather/infograph_111_000201.jpg
infograph/fireplace/infograph_115_000046.jpg
infograph/firetruck/infograph_116_000031.jpg
infograph/fish/infograph_117_000135.jpg
infograph/fish/infograph_117_000176.jpg
infograph/flashlight/infograph_119_000011.jpg
infograph/flashlight/infograph_119_000026.jpg
infograph/flashlight/infograph_119_000039.jpg
infograph/floor_lamp/infograph_121_000086.jpg
infograph/foot/infograph_124_000089.jpg
infograph/foot/infograph_124_000104.jpg
infograph/fork/infograph_125_000007.jpg
infograph/frog/infograph_126_000010.jpg
infograph/frying_pan/infograph_127_000004.jpg
infograph/frying_pan/infograph_127_000055.jpg
infograph/garden/infograph_128_000034.jpg
infograph/garden/infograph_128_000039.jpg
infograph/garden/infograph_128_000060.jpg
infograph/garden/infograph_128_000076.jpg
infograph/garden/infograph_128_000113.jpg
infograph/garden/infograph_128_000118.jpg
infograph/garden/infograph_128_000151.jpg
infograph/garden/infograph_128_000172.jpg
infograph/garden/infograph_128_000186.jpg
infograph/garden/infograph_128_000242.jpg
infograph/garden/infograph_128_000283.jpg
infograph/garden_hose/infograph_129_000002.jpg
infograph/garden_hose/infograph_129_000015.jpg
infograph/garden_hose/infograph_129_000031.jpg
infograph/giraffe/infograph_130_000003.jpg
infograph/giraffe/infograph_130_000087.jpg
infograph/giraffe/infograph_130_000106.jpg
infograph/giraffe/infograph_130_000149.jpg
infograph/goatee/infograph_131_000001.jpg
infograph/goatee/infograph_131_000005.jpg
infograph/goatee/infograph_131_000009.jpg
infograph/goatee/infograph_131_000016.jpg
infograph/goatee/infograph_131_000024.jpg
infograph/goatee/infograph_131_000054.jpg
infograph/goatee/infograph_131_000056.jpg
infograph/goatee/infograph_131_000059.jpg
infograph/goatee/infograph_131_000060.jpg
infograph/goatee/infograph_131_000067.jpg
infograph/goatee/infograph_131_000068.jpg
infograph/goatee/infograph_131_000069.jpg
infograph/goatee/infograph_131_000076.jpg
infograph/goatee/infograph_131_000082.jpg
infograph/goatee/infograph_131_000094.jpg
infograph/goatee/infograph_131_000109.jpg
infograph/goatee/infograph_131_000113.jpg
infograph/goatee/infograph_131_000118.jpg
infograph/goatee/infograph_131_000135.jpg
infograph/goatee/infograph_131_000147.jpg
infograph/goatee/infograph_131_000156.jpg
infograph/goatee/infograph_131_000166.jpg
infograph/goatee/infograph_131_000170.jpg
infograph/goatee/infograph_131_000198.jpg
infograph/goatee/infograph_131_000206.jpg
infograph/goatee/infograph_131_000208.jpg
infograph/goatee/infograph_131_000217.jpg
infograph/goatee/infograph_131_000227.jpg
infograph/goatee/infograph_131_000232.jpg
infograph/goatee/infograph_131_000236.jpg
infograph/grapes/infograph_133_000019.jpg
infograph/grapes/infograph_133_000067.jpg
infograph/grapes/infograph_133_000117.jpg
infograph/grass/infograph_134_000036.jpg
infograph/grass/infograph_134_000042.jpg
infograph/grass/infograph_134_000054.jpg
infograph/grass/infograph_134_000065.jpg
infograph/grass/infograph_134_000069.jpg
infograph/grass/infograph_134_000075.jpg
infograph/grass/infograph_134_000128.jpg
infograph/grass/infograph_134_000130.jpg
infograph/grass/infograph_134_000132.jpg
infograph/grass/infograph_134_000148.jpg
infograph/grass/infograph_134_000172.jpg
infograph/grass/infograph_134_000180.jpg
infograph/grass/infograph_134_000218.jpg
infograph/grass/infograph_134_000226.jpg
infograph/grass/infograph_134_000244.jpg
infograph/grass/infograph_134_000247.jpg
infograph/grass/infograph_134_000253.jpg
infograph/guitar/infograph_135_000089.jpg
infograph/guitar/infograph_135_000103.jpg
infograph/guitar/infograph_135_000159.jpg
infograph/hamburger/infograph_136_000040.jpg
infograph/hamburger/infograph_136_000114.jpg
infograph/hamburger/infograph_136_000117.jpg
infograph/hamburger/infograph_136_000175.jpg
infograph/hammer/infograph_137_000009.jpg
infograph/hammer/infograph_137_000041.jpg
infograph/hammer/infograph_137_000047.jpg
infograph/hand/infograph_138_000048.jpg
infograph/hand/infograph_138_000049.jpg
infograph/hand/infograph_138_000069.jpg
infograph/hand/infograph_138_000101.jpg
infograph/hand/infograph_138_000122.jpg
infograph/hand/infograph_138_000154.jpg
infograph/hand/infograph_138_000167.jpg
infograph/hand/infograph_138_000172.jpg
infograph/hand/infograph_138_000173.jpg
infograph/hand/infograph_138_000185.jpg
infograph/hand/infograph_138_000187.jpg
infograph/hand/infograph_138_000190.jpg
infograph/hand/infograph_138_000206.jpg
infograph/hand/infograph_138_000252.jpg
infograph/harp/infograph_139_000037.jpg
infograph/hat/infograph_140_000040.jpg
infograph/hat/infograph_140_000083.jpg
infograph/hat/infograph_140_000183.jpg
infograph/headphones/infograph_141_000158.jpg
infograph/helicopter/infograph_143_000015.jpg
infograph/helicopter/infograph_143_000208.jpg
infograph/helmet/infograph_144_000026.jpg
infograph/helmet/infograph_144_000041.jpg
infograph/helmet/infograph_144_000060.jpg
infograph/helmet/infograph_144_000078.jpg
infograph/helmet/infograph_144_000098.jpg
infograph/helmet/infograph_144_000099.jpg
infograph/helmet/infograph_144_000102.jpg
infograph/helmet/infograph_144_000107.jpg
infograph/helmet/infograph_144_000137.jpg
infograph/helmet/infograph_144_000140.jpg
infograph/helmet/infograph_144_000177.jpg
infograph/helmet/infograph_144_000180.jpg
infograph/helmet/infograph_144_000201.jpg
infograph/helmet/infograph_144_000227.jpg
infograph/hockey_stick/infograph_147_000009.jpg
infograph/hockey_stick/infograph_147_000011.jpg
infograph/hockey_stick/infograph_147_000017.jpg
infograph/hockey_stick/infograph_147_000018.jpg
infograph/hockey_stick/infograph_147_000027.jpg
infograph/hockey_stick/infograph_147_000030.jpg
infograph/hockey_stick/infograph_147_000052.jpg
infograph/hockey_stick/infograph_147_000058.jpg
infograph/hockey_stick/infograph_147_000061.jpg
infograph/hockey_stick/infograph_147_000064.jpg
infograph/hockey_stick/infograph_147_000069.jpg
infograph/hockey_stick/infograph_147_000077.jpg
infograph/hockey_stick/infograph_147_000100.jpg
infograph/hockey_stick/infograph_147_000125.jpg
infograph/hockey_stick/infograph_147_000130.jpg
infograph/hockey_stick/infograph_147_000178.jpg
infograph/hockey_stick/infograph_147_000197.jpg
infograph/horse/infograph_148_000002.jpg
infograph/horse/infograph_148_000040.jpg
infograph/horse/infograph_148_000061.jpg
infograph/horse/infograph_148_000081.jpg
infograph/horse/infograph_148_000103.jpg
infograph/horse/infograph_148_000146.jpg
infograph/hot_dog/infograph_151_000028.jpg
infograph/hot_dog/infograph_151_000034.jpg
infograph/hot_dog/infograph_151_000074.jpg
infograph/hot_dog/infograph_151_000075.jpg
infograph/hot_dog/infograph_151_000082.jpg
infograph/hot_dog/infograph_151_000105.jpg
infograph/hot_dog/infograph_151_000112.jpg
infograph/hot_dog/infograph_151_000113.jpg
infograph/hot_dog/infograph_151_000117.jpg
infograph/hot_dog/infograph_151_000133.jpg
infograph/hot_tub/infograph_152_000029.jpg
infograph/hot_tub/infograph_152_000050.jpg
infograph/hot_tub/infograph_152_000078.jpg
infograph/hot_tub/infograph_152_000085.jpg
infograph/house/infograph_154_000306.jpg
infograph/house_plant/infograph_155_000095.jpg
infograph/house_plant/infograph_155_000101.jpg
infograph/house_plant/infograph_155_000110.jpg
infograph/house_plant/infograph_155_000113.jpg
infograph/house_plant/infograph_155_000138.jpg
infograph/house_plant/infograph_155_000193.jpg
infograph/house_plant/infograph_155_000268.jpg
infograph/ice_cream/infograph_157_000038.jpg
infograph/ice_cream/infograph_157_000151.jpg
infograph/ice_cream/infograph_157_000178.jpg
infograph/ice_cream/infograph_157_000180.jpg
infograph/jacket/infograph_158_000023.jpg
infograph/jacket/infograph_158_000029.jpg
infograph/jacket/infograph_158_000078.jpg
infograph/keyboard/infograph_162_000053.jpg
infograph/knee/infograph_163_000018.jpg
infograph/lantern/infograph_166_000014.jpg
infograph/laptop/infograph_167_000008.jpg
infograph/laptop/infograph_167_000082.jpg
infograph/laptop/infograph_167_000112.jpg
infograph/leg/infograph_169_000018.jpg
infograph/leg/infograph_169_000032.jpg
infograph/leg/infograph_169_000063.jpg
infograph/leg/infograph_169_000082.jpg
infograph/leg/infograph_169_000085.jpg
infograph/leg/infograph_169_000087.jpg
infograph/leg/infograph_169_000116.jpg
infograph/leg/infograph_169_000118.jpg
infograph/leg/infograph_169_000158.jpg
infograph/light_bulb/infograph_170_000014.jpg
infograph/light_bulb/infograph_170_000086.jpg
infograph/light_bulb/infograph_170_000098.jpg
infograph/light_bulb/infograph_170_000174.jpg
infograph/lighter/infograph_171_000006.jpg
infograph/lighter/infograph_171_000008.jpg
infograph/line/infograph_174_000027.jpg
infograph/line/infograph_174_000098.jpg
infograph/line/infograph_174_000117.jpg
infograph/lobster/infograph_177_000009.jpg
infograph/lobster/infograph_177_000014.jpg
infograph/lobster/infograph_177_000017.jpg
infograph/lobster/infograph_177_000029.jpg
infograph/lobster/infograph_177_000033.jpg
infograph/lobster/infograph_177_000038.jpg
infograph/lobster/infograph_177_000043.jpg
infograph/map/infograph_180_000043.jpg
infograph/map/infograph_180_000044.jpg
infograph/matches/infograph_182_000005.jpg
infograph/microwave/infograph_186_000027.jpg
infograph/microwave/infograph_186_000083.jpg
infograph/microwave/infograph_186_000102.jpg
infograph/mosquito/infograph_189_000195.jpg
infograph/motorbike/infograph_190_000010.jpg
infograph/motorbike/infograph_190_000024.jpg
infograph/motorbike/infograph_190_000131.jpg
infograph/mouse/infograph_192_000019.jpg
infograph/mouse/infograph_192_000036.jpg
infograph/mouse/infograph_192_000043.jpg
infograph/moustache/infograph_193_000017.jpg
infograph/moustache/infograph_193_000023.jpg
infograph/moustache/infograph_193_000027.jpg
infograph/mug/infograph_195_000008.jpg
infograph/mug/infograph_195_000014.jpg
infograph/mug/infograph_195_000016.jpg
infograph/mug/infograph_195_000041.jpg
infograph/mushroom/infograph_196_000016.jpg
infograph/mushroom/infograph_196_000053.jpg
infograph/mushroom/infograph_196_000069.jpg
infograph/mushroom/infograph_196_000071.jpg
infograph/mushroom/infograph_196_000081.jpg
infograph/mushroom/infograph_196_000095.jpg
infograph/mushroom/infograph_196_000101.jpg
infograph/mushroom/infograph_196_000125.jpg
infograph/mushroom/infograph_196_000131.jpg
infograph/mushroom/infograph_196_000145.jpg
infograph/mushroom/infograph_196_000157.jpg
infograph/mushroom/infograph_196_000193.jpg
infograph/mushroom/infograph_196_000218.jpg
infograph/mushroom/infograph_196_000228.jpg
infograph/mushroom/infograph_196_000232.jpg
infograph/mushroom/infograph_196_000252.jpg
infograph/mushroom/infograph_196_000260.jpg
infograph/mushroom/infograph_196_000289.jpg
infograph/nail/infograph_197_000024.jpg
infograph/nail/infograph_197_000044.jpg
infograph/nail/infograph_197_000045.jpg
infograph/nail/infograph_197_000050.jpg
infograph/nail/infograph_197_000078.jpg
infograph/nail/infograph_197_000096.jpg
infograph/nail/infograph_197_000107.jpg
infograph/nail/infograph_197_000146.jpg
infograph/nail/infograph_197_000176.jpg
infograph/necklace/infograph_198_000013.jpg
infograph/necklace/infograph_198_000023.jpg
infograph/necklace/infograph_198_000060.jpg
infograph/necklace/infograph_198_000077.jpg
infograph/nose/infograph_199_000013.jpg
infograph/nose/infograph_199_000037.jpg
infograph/nose/infograph_199_000095.jpg
infograph/nose/infograph_199_000097.jpg
infograph/nose/infograph_199_000105.jpg
infograph/nose/infograph_199_000114.jpg
infograph/nose/infograph_199_000165.jpg
infograph/nose/infograph_199_000179.jpg
infograph/nose/infograph_199_000186.jpg
infograph/nose/infograph_199_000204.jpg
infograph/nose/infograph_199_000214.jpg
infograph/nose/infograph_199_000219.jpg
infograph/octopus/infograph_202_000001.jpg
infograph/octopus/infograph_202_000022.jpg
infograph/octopus/infograph_202_000028.jpg
infograph/onion/infograph_203_000028.jpg
infograph/onion/infograph_203_000039.jpg
infograph/oven/infograph_204_000006.jpg
infograph/oven/infograph_204_000012.jpg
infograph/oven/infograph_204_000013.jpg
infograph/oven/infograph_204_000023.jpg
infograph/oven/infograph_204_000027.jpg
infograph/oven/infograph_204_000031.jpg
infograph/oven/infograph_204_000032.jpg
infograph/oven/infograph_204_000037.jpg
infograph/oven/infograph_204_000040.jpg
infograph/oven/infograph_204_000051.jpg
infograph/owl/infograph_205_000004.jpg
infograph/owl/infograph_205_000039.jpg
infograph/owl/infograph_205_000048.jpg
infograph/owl/infograph_205_000086.jpg
infograph/owl/infograph_205_000092.jpg
infograph/owl/infograph_205_000100.jpg
infograph/owl/infograph_205_000111.jpg
infograph/paintbrush/infograph_206_000003.jpg
infograph/paint_can/infograph_207_000026.jpg
infograph/palm_tree/infograph_208_000001.jpg
infograph/palm_tree/infograph_208_000004.jpg
infograph/palm_tree/infograph_208_000024.jpg
infograph/palm_tree/infograph_208_000041.jpg
infograph/palm_tree/infograph_208_000079.jpg
infograph/palm_tree/infograph_208_000086.jpg
infograph/palm_tree/infograph_208_000109.jpg
infograph/palm_tree/infograph_208_000117.jpg
infograph/palm_tree/infograph_208_000146.jpg
infograph/palm_tree/infograph_208_000253.jpg
infograph/palm_tree/infograph_208_000268.jpg
infograph/palm_tree/infograph_208_000276.jpg
infograph/pants/infograph_210_000009.jpg
infograph/pants/infograph_210_000037.jpg
infograph/pants/infograph_210_000049.jpg
infograph/pants/infograph_210_000050.jpg
infograph/pants/infograph_210_000053.jpg
infograph/pants/infograph_210_000059.jpg
infograph/pants/infograph_210_000120.jpg
infograph/parrot/infograph_213_000052.jpg
infograph/peanut/infograph_215_000014.jpg
infograph/pear/infograph_216_000005.jpg
infograph/pear/infograph_216_000017.jpg
infograph/pear/infograph_216_000040.jpg
infograph/pear/infograph_216_000043.jpg
infograph/pear/infograph_216_000059.jpg
infograph/pear/infograph_216_000065.jpg
infograph/pear/infograph_216_000076.jpg
infograph/peas/infograph_217_000008.jpg
infograph/peas/infograph_217_000011.jpg
infograph/peas/infograph_217_000026.jpg
infograph/peas/infograph_217_000044.jpg
infograph/peas/infograph_217_000045.jpg
infograph/peas/infograph_217_000050.jpg
infograph/peas/infograph_217_000052.jpg
infograph/peas/infograph_217_000072.jpg
infograph/peas/infograph_217_000076.jpg
infograph/peas/infograph_217_000077.jpg
infograph/peas/infograph_217_000080.jpg
infograph/peas/infograph_217_000098.jpg
infograph/peas/infograph_217_000119.jpg
infograph/pencil/infograph_218_000033.jpg
infograph/pencil/infograph_218_000310.jpg
infograph/pencil/infograph_218_000361.jpg
infograph/penguin/infograph_219_000034.jpg
infograph/penguin/infograph_219_000092.jpg
infograph/penguin/infograph_219_000122.jpg
infograph/penguin/infograph_219_000149.jpg
infograph/piano/infograph_220_000006.jpg
infograph/piano/infograph_220_000013.jpg
infograph/piano/infograph_220_000019.jpg
infograph/pig/infograph_223_000121.jpg
infograph/pig/infograph_223_000187.jpg
infograph/pillow/infograph_224_000001.jpg
infograph/pillow/infograph_224_000003.jpg
infograph/pillow/infograph_224_000011.jpg
infograph/pillow/infograph_224_000047.jpg
infograph/pillow/infograph_224_000050.jpg
infograph/pillow/infograph_224_000087.jpg
infograph/pillow/infograph_224_000093.jpg
infograph/pillow/infograph_224_000097.jpg
infograph/pillow/infograph_224_000142.jpg
infograph/pillow/infograph_224_000154.jpg
infograph/pineapple/infograph_225_000005.jpg
infograph/pineapple/infograph_225_000045.jpg
infograph/pineapple/infograph_225_000048.jpg
infograph/pineapple/infograph_225_000065.jpg
infograph/pineapple/infograph_225_000067.jpg
infograph/pineapple/infograph_225_000086.jpg
infograph/pizza/infograph_226_000044.jpg
infograph/pizza/infograph_226_000090.jpg
infograph/pizza/infograph_226_000140.jpg
infograph/pizza/infograph_226_000146.jpg
infograph/pliers/infograph_227_000063.jpg
infograph/police_car/infograph_228_000035.jpg
infograph/pond/infograph_229_000012.jpg
infograph/pond/infograph_229_000015.jpg
infograph/pond/infograph_229_000047.jpg
infograph/pond/infograph_229_000062.jpg
infograph/pool/infograph_230_000011.jpg
infograph/pool/infograph_230_000076.jpg
infograph/pool/infograph_230_000091.jpg
infograph/pool/infograph_230_000106.jpg
infograph/popsicle/infograph_231_000047.jpg
infograph/popsicle/infograph_231_000057.jpg
infograph/potato/infograph_233_000010.jpg
infograph/potato/infograph_233_000026.jpg
infograph/potato/infograph_233_000036.jpg
infograph/potato/infograph_233_000055.jpg
infograph/power_outlet/infograph_234_000013.jpg
infograph/power_outlet/infograph_234_000022.jpg
infograph/power_outlet/infograph_234_000040.jpg
infograph/purse/infograph_235_000009.jpg
infograph/purse/infograph_235_000029.jpg
infograph/purse/infograph_235_000060.jpg
infograph/purse/infograph_235_000076.jpg
infograph/rabbit/infograph_236_000006.jpg
infograph/rabbit/infograph_236_000044.jpg
infograph/raccoon/infograph_237_000016.jpg
infograph/radio/infograph_238_000057.jpg
infograph/radio/infograph_238_000085.jpg
infograph/rain/infograph_239_000046.jpg
infograph/rain/infograph_239_000077.jpg
infograph/rake/infograph_241_000023.jpg
infograph/rake/infograph_241_000029.jpg
infograph/rake/infograph_241_000063.jpg
infograph/remote_control/infograph_242_000066.jpg
infograph/rhinoceros/infograph_243_000015.jpg
infograph/rhinoceros/infograph_243_000039.jpg
infograph/rhinoceros/infograph_243_000065.jpg
infograph/rifle/infograph_244_000064.jpg
infograph/river/infograph_245_000026.jpg
infograph/river/infograph_245_000038.jpg
infograph/river/infograph_245_000047.jpg
infograph/river/infograph_245_000050.jpg
infograph/river/infograph_245_000118.jpg
infograph/river/infograph_245_000144.jpg
infograph/sailboat/infograph_248_000054.jpg
infograph/sailboat/infograph_248_000067.jpg
infograph/sandwich/infograph_249_000006.jpg
infograph/sandwich/infograph_249_000009.jpg
infograph/sandwich/infograph_249_000018.jpg
infograph/sandwich/infograph_249_000023.jpg
infograph/sandwich/infograph_249_000028.jpg
infograph/sandwich/infograph_249_000039.jpg
infograph/sandwich/infograph_249_000048.jpg
infograph/sandwich/infograph_249_000066.jpg
infograph/sandwich/infograph_249_000071.jpg
infograph/sandwich/infograph_249_000074.jpg
infograph/sandwich/infograph_249_000082.jpg
infograph/sandwich/infograph_249_000085.jpg
infograph/sandwich/infograph_249_000092.jpg
infograph/sandwich/infograph_249_000094.jpg
infograph/sandwich/infograph_249_000107.jpg
infograph/saw/infograph_250_000006.jpg
infograph/saw/infograph_250_000016.jpg
infograph/saw/infograph_250_000020.jpg
infograph/saxophone/infograph_251_000007.jpg
infograph/saxophone/infograph_251_000009.jpg
infograph/saxophone/infograph_251_000067.jpg
infograph/saxophone/infograph_251_000069.jpg
infograph/school_bus/infograph_252_000003.jpg
infograph/school_bus/infograph_252_000006.jpg
infograph/school_bus/infograph_252_000010.jpg
infograph/school_bus/infograph_252_000012.jpg
infograph/school_bus/infograph_252_000014.jpg
infograph/school_bus/infograph_252_000018.jpg
infograph/school_bus/infograph_252_000019.jpg
infograph/school_bus/infograph_252_000028.jpg
infograph/school_bus/infograph_252_000035.jpg
infograph/school_bus/infograph_252_000036.jpg
infograph/school_bus/infograph_252_000044.jpg
infograph/school_bus/infograph_252_000048.jpg
infograph/school_bus/infograph_252_000049.jpg
infograph/school_bus/infograph_252_000051.jpg
infograph/school_bus/infograph_252_000054.jpg
infograph/school_bus/infograph_252_000063.jpg
infograph/school_bus/infograph_252_000066.jpg
infograph/school_bus/infograph_252_000071.jpg
infograph/school_bus/infograph_252_000074.jpg
infograph/school_bus/infograph_252_000075.jpg
infograph/school_bus/infograph_252_000085.jpg
infograph/school_bus/infograph_252_000086.jpg
infograph/school_bus/infograph_252_000091.jpg
infograph/school_bus/infograph_252_000093.jpg
infograph/school_bus/infograph_252_000100.jpg
infograph/school_bus/infograph_252_000105.jpg
infograph/school_bus/infograph_252_000112.jpg
infograph/school_bus/infograph_252_000119.jpg
infograph/school_bus/infograph_252_000121.jpg
infograph/school_bus/infograph_252_000125.jpg
infograph/school_bus/infograph_252_000126.jpg
infograph/school_bus/infograph_252_000127.jpg
infograph/school_bus/infograph_252_000129.jpg
infograph/school_bus/infograph_252_000131.jpg
infograph/school_bus/infograph_252_000132.jpg
infograph/school_bus/infograph_252_000133.jpg
infograph/school_bus/infograph_252_000134.jpg
infograph/school_bus/infograph_252_000136.jpg
infograph/school_bus/infograph_252_000138.jpg
infograph/school_bus/infograph_252_000139.jpg
infograph/scissors/infograph_253_000008.jpg
infograph/scissors/infograph_253_000015.jpg
infograph/scissors/infograph_253_000024.jpg
infograph/scissors/infograph_253_000060.jpg
infograph/scissors/infograph_253_000075.jpg
infograph/scorpion/infograph_254_000003.jpg
infograph/scorpion/infograph_254_000007.jpg
infograph/scorpion/infograph_254_000042.jpg
infograph/scorpion/infograph_254_000046.jpg
infograph/scorpion/infograph_254_000049.jpg
infograph/screwdriver/infograph_255_000004.jpg
infograph/screwdriver/infograph_255_000008.jpg
infograph/screwdriver/infograph_255_000020.jpg
infograph/screwdriver/infograph_255_000027.jpg
infograph/screwdriver/infograph_255_000030.jpg
infograph/sea_turtle/infograph_256_000020.jpg
infograph/sea_turtle/infograph_256_000033.jpg
infograph/sea_turtle/infograph_256_000055.jpg
infograph/sea_turtle/infograph_256_000058.jpg
infograph/sea_turtle/infograph_256_000069.jpg
infograph/sea_turtle/infograph_256_000086.jpg
infograph/sea_turtle/infograph_256_000087.jpg
infograph/sea_turtle/infograph_256_000112.jpg
infograph/shark/infograph_258_000011.jpg
infograph/shark/infograph_258_000073.jpg
infograph/shark/infograph_258_000109.jpg
infograph/shark/infograph_258_000231.jpg
infograph/shark/infograph_258_000278.jpg
infograph/sheep/infograph_259_000043.jpg
infograph/shoe/infograph_260_000037.jpg
infograph/shoe/infograph_260_000056.jpg
infograph/shoe/infograph_260_000142.jpg
infograph/shoe/infograph_260_000175.jpg
infograph/shoe/infograph_260_000183.jpg
infograph/shorts/infograph_261_000022.jpg
infograph/shorts/infograph_261_000029.jpg
infograph/skateboard/infograph_264_000046.jpg
infograph/skyscraper/infograph_266_000003.jpg
infograph/skyscraper/infograph_266_000125.jpg
infograph/snail/infograph_269_000014.jpg
infograph/snake/infograph_270_000010.jpg
infograph/snake/infograph_270_000012.jpg
infograph/snake/infograph_270_000021.jpg
infograph/snake/infograph_270_000056.jpg
infograph/snowman/infograph_273_000100.jpg
infograph/sock/infograph_275_000050.jpg
infograph/speedboat/infograph_276_000038.jpg
infograph/speedboat/infograph_276_000062.jpg
infograph/spider/infograph_277_000036.jpg
infograph/spreadsheet/infograph_279_000001.jpg
infograph/spreadsheet/infograph_279_000067.jpg
infograph/spreadsheet/infograph_279_000122.jpg
infograph/square/infograph_280_000074.jpg
infograph/square/infograph_280_000085.jpg
infograph/squiggle/infograph_281_000042.jpg
infograph/squirrel/infograph_282_000158.jpg
infograph/stairs/infograph_283_000013.jpg
infograph/steak/infograph_285_000016.jpg
infograph/steak/infograph_285_000102.jpg
infograph/steak/infograph_285_000162.jpg
infograph/steak/infograph_285_000168.jpg
infograph/steak/infograph_285_000198.jpg
infograph/steak/infograph_285_000206.jpg
infograph/steak/infograph_285_000211.jpg
infograph/steak/infograph_285_000349.jpg
infograph/stereo/infograph_286_000011.jpg
infograph/stereo/infograph_286_000283.jpg
infograph/stove/infograph_290_000001.jpg
infograph/stove/infograph_290_000047.jpg
infograph/stove/infograph_290_000084.jpg
infograph/stove/infograph_290_000124.jpg
infograph/strawberry/infograph_291_000274.jpg
infograph/strawberry/infograph_291_000279.jpg
infograph/string_bean/infograph_293_000009.jpg
infograph/string_bean/infograph_293_000020.jpg
infograph/string_bean/infograph_293_000086.jpg
infograph/submarine/infograph_294_000033.jpg
infograph/submarine/infograph_294_000086.jpg
infograph/suitcase/infograph_295_000060.jpg
infograph/suitcase/infograph_295_000143.jpg
infograph/suitcase/infograph_295_000159.jpg
infograph/suitcase/infograph_295_000183.jpg
infograph/sun/infograph_296_000006.jpg
infograph/sun/infograph_296_000147.jpg
infograph/sweater/infograph_298_000066.jpg
infograph/sweater/infograph_298_000074.jpg
infograph/sword/infograph_300_000019.jpg
infograph/syringe/infograph_301_000126.jpg
infograph/table/infograph_302_000011.jpg
infograph/table/infograph_302_000030.jpg
infograph/table/infograph_302_000069.jpg
infograph/table/infograph_302_000184.jpg
infograph/table/infograph_302_000240.jpg
infograph/table/infograph_302_000241.jpg
infograph/table/infograph_302_000245.jpg
infograph/table/infograph_302_000306.jpg
infograph/table/infograph_302_000316.jpg
infograph/table/infograph_302_000351.jpg
infograph/table/infograph_302_000387.jpg
infograph/table/infograph_302_000491.jpg
infograph/table/infograph_302_000566.jpg
infograph/table/infograph_302_000606.jpg
infograph/table/infograph_302_000675.jpg
infograph/table/infograph_302_000678.jpg
infograph/teddy-bear/infograph_304_000102.jpg
infograph/telephone/infograph_305_000227.jpg
infograph/telephone/infograph_305_000263.jpg
infograph/television/infograph_306_000011.jpg
infograph/television/infograph_306_000094.jpg
infograph/television/infograph_306_000138.jpg
infograph/television/infograph_306_000217.jpg
infograph/television/infograph_306_000366.jpg
infograph/television/infograph_306_000469.jpg
infograph/tennis_racquet/infograph_307_000046.jpg
infograph/tent/infograph_308_000150.jpg
infograph/tent/infograph_308_000207.jpg
infograph/The_Eiffel_Tower/infograph_309_000030.jpg
infograph/tiger/infograph_312_000008.jpg
infograph/tiger/infograph_312_000039.jpg
infograph/tiger/infograph_312_000049.jpg
infograph/tiger/infograph_312_000056.jpg
infograph/tiger/infograph_312_000062.jpg
infograph/tiger/infograph_312_000082.jpg
infograph/tiger/infograph_312_000091.jpg
infograph/tiger/infograph_312_000100.jpg
infograph/tiger/infograph_312_000136.jpg
infograph/tiger/infograph_312_000166.jpg
infograph/tiger/infograph_312_000197.jpg
infograph/tiger/infograph_312_000212.jpg
infograph/tiger/infograph_312_000224.jpg
infograph/tiger/infograph_312_000240.jpg
infograph/tiger/infograph_312_000253.jpg
infograph/tiger/infograph_312_000284.jpg
infograph/toaster/infograph_313_000007.jpg
infograph/toaster/infograph_313_000011.jpg
infograph/toaster/infograph_313_000030.jpg
infograph/toaster/infograph_313_000040.jpg
infograph/toaster/infograph_313_000060.jpg
infograph/toaster/infograph_313_000088.jpg
infograph/toaster/infograph_313_000093.jpg
infograph/toaster/infograph_313_000105.jpg
infograph/toaster/infograph_313_000145.jpg
infograph/toaster/infograph_313_000147.jpg
infograph/toaster/infograph_313_000180.jpg
infograph/toaster/infograph_313_000201.jpg
infograph/toaster/infograph_313_000252.jpg
infograph/toaster/infograph_313_000260.jpg
infograph/toaster/infograph_313_000265.jpg
infograph/toaster/infograph_313_000277.jpg
infograph/toaster/infograph_313_000286.jpg
infograph/toaster/infograph_313_000300.jpg
infograph/toaster/infograph_313_000315.jpg
infograph/toaster/infograph_313_000333.jpg
infograph/toe/infograph_314_000001.jpg
infograph/toe/infograph_314_000021.jpg
infograph/toe/infograph_314_000030.jpg
infograph/toe/infograph_314_000035.jpg
infograph/toe/infograph_314_000043.jpg
infograph/toe/infograph_314_000044.jpg
infograph/toe/infograph_314_000047.jpg
infograph/toe/infograph_314_000053.jpg
infograph/toe/infograph_314_000068.jpg
infograph/toe/infograph_314_000078.jpg
infograph/toe/infograph_314_000081.jpg
infograph/toe/infograph_314_000095.jpg
infograph/toe/infograph_314_000097.jpg
infograph/toe/infograph_314_000098.jpg
infograph/toe/infograph_314_000100.jpg
infograph/toe/infograph_314_000101.jpg
infograph/toe/infograph_314_000107.jpg
infograph/toe/infograph_314_000114.jpg
infograph/toe/infograph_314_000122.jpg
infograph/toe/infograph_314_000125.jpg
infograph/toe/infograph_314_000127.jpg
infograph/toe/infograph_314_000148.jpg
infograph/toe/infograph_314_000149.jpg
infograph/toe/infograph_314_000151.jpg
infograph/toe/infograph_314_000152.jpg
infograph/toe/infograph_314_000162.jpg
infograph/toe/infograph_314_000163.jpg
infograph/toe/infograph_314_000168.jpg
infograph/toe/infograph_314_000169.jpg
infograph/toe/infograph_314_000173.jpg
infograph/toe/infograph_314_000174.jpg
infograph/toe/infograph_314_000179.jpg
infograph/toe/infograph_314_000186.jpg
infograph/toe/infograph_314_000221.jpg
infograph/toe/infograph_314_000223.jpg
infograph/toe/infograph_314_000230.jpg
infograph/toe/infograph_314_000232.jpg
infograph/toe/infograph_314_000238.jpg
infograph/toe/infograph_314_000249.jpg
infograph/toe/infograph_314_000251.jpg
infograph/toe/infograph_314_000253.jpg
infograph/toe/infograph_314_000254.jpg
infograph/toe/infograph_314_000280.jpg
infograph/toe/infograph_314_000282.jpg
infograph/toe/infograph_314_000293.jpg
infograph/toe/infograph_314_000297.jpg
infograph/toe/infograph_314_000301.jpg
infograph/toe/infograph_314_000304.jpg
infograph/toe/infograph_314_000307.jpg
infograph/toe/infograph_314_000311.jpg
infograph/toe/infograph_314_000320.jpg
infograph/toe/infograph_314_000325.jpg
infograph/toe/infograph_314_000326.jpg
infograph/toe/infograph_314_000333.jpg
infograph/toe/infograph_314_000336.jpg
infograph/toe/infograph_314_000337.jpg
infograph/toe/infograph_314_000339.jpg
infograph/toe/infograph_314_000349.jpg
infograph/toe/infograph_314_000352.jpg
infograph/toe/infograph_314_000359.jpg
infograph/toe/infograph_314_000361.jpg
infograph/toe/infograph_314_000362.jpg
infograph/toe/infograph_314_000366.jpg
infograph/toe/infograph_314_000382.jpg
infograph/toe/infograph_314_000384.jpg
infograph/toe/infograph_314_000385.jpg
infograph/toe/infograph_314_000386.jpg
infograph/toe/infograph_314_000388.jpg
infograph/toe/infograph_314_000401.jpg
infograph/toe/infograph_314_000403.jpg
infograph/toe/infograph_314_000405.jpg
infograph/toilet/infograph_315_000008.jpg
infograph/toilet/infograph_315_000054.jpg
infograph/toilet/infograph_315_000060.jpg
infograph/toilet/infograph_315_000071.jpg
infograph/toilet/infograph_315_000126.jpg
infograph/toilet/infograph_315_000127.jpg
infograph/toilet/infograph_315_000154.jpg
infograph/toilet/infograph_315_000155.jpg
infograph/toilet/infograph_315_000172.jpg
infograph/toilet/infograph_315_000224.jpg
infograph/toilet/infograph_315_000280.jpg
infograph/toilet/infograph_315_000391.jpg
infograph/toilet/infograph_315_000418.jpg
infograph/toilet/infograph_315_000433.jpg
infograph/toilet/infograph_315_000483.jpg
infograph/toilet/infograph_315_000489.jpg
infograph/tooth/infograph_316_000027.jpg
infograph/tooth/infograph_316_000063.jpg
infograph/tooth/infograph_316_000106.jpg
infograph/tooth/infograph_316_000131.jpg
infograph/tooth/infograph_316_000140.jpg
infograph/tooth/infograph_316_000158.jpg
infograph/tooth/infograph_316_000218.jpg
infograph/tooth/infograph_316_000232.jpg
infograph/tooth/infograph_316_000320.jpg
infograph/tooth/infograph_316_000353.jpg
infograph/tooth/infograph_316_000406.jpg
infograph/toothbrush/infograph_317_000004.jpg
infograph/toothbrush/infograph_317_000014.jpg
infograph/toothbrush/infograph_317_000022.jpg
infograph/toothbrush/infograph_317_000046.jpg
infograph/toothbrush/infograph_317_000070.jpg
infograph/toothbrush/infograph_317_000101.jpg
infograph/toothbrush/infograph_317_000115.jpg
infograph/toothbrush/infograph_317_000116.jpg
infograph/toothbrush/infograph_317_000124.jpg
infograph/toothbrush/infograph_317_000147.jpg
infograph/toothbrush/infograph_317_000148.jpg
infograph/toothbrush/infograph_317_000151.jpg
infograph/toothbrush/infograph_317_000156.jpg
infograph/toothbrush/infograph_317_000161.jpg
infograph/toothbrush/infograph_317_000163.jpg
infograph/toothbrush/infograph_317_000182.jpg
infograph/toothbrush/infograph_317_000186.jpg
infograph/toothbrush/infograph_317_000209.jpg
infograph/toothbrush/infograph_317_000212.jpg
infograph/toothbrush/infograph_317_000231.jpg
infograph/toothbrush/infograph_317_000240.jpg
infograph/toothbrush/infograph_317_000250.jpg
infograph/toothbrush/infograph_317_000265.jpg
infograph/toothbrush/infograph_317_000273.jpg
infograph/toothbrush/infograph_317_000279.jpg
infograph/toothbrush/infograph_317_000284.jpg
infograph/toothbrush/infograph_317_000290.jpg
infograph/toothbrush/infograph_317_000291.jpg
infograph/toothbrush/infograph_317_000295.jpg
infograph/toothbrush/infograph_317_000300.jpg
infograph/toothbrush/infograph_317_000303.jpg
infograph/toothbrush/infograph_317_000314.jpg
infograph/toothbrush/infograph_317_000328.jpg
infograph/toothbrush/infograph_317_000331.jpg
infograph/toothbrush/infograph_317_000340.jpg
infograph/toothbrush/infograph_317_000356.jpg
infograph/toothbrush/infograph_317_000368.jpg
infograph/toothbrush/infograph_317_000391.jpg
infograph/toothbrush/infograph_317_000392.jpg
infograph/toothbrush/infograph_317_000400.jpg
infograph/toothbrush/infograph_317_000415.jpg
infograph/toothbrush/infograph_317_000419.jpg
infograph/toothbrush/infograph_317_000420.jpg
infograph/toothbrush/infograph_317_000421.jpg
infograph/toothbrush/infograph_317_000430.jpg
infograph/toothbrush/infograph_317_000437.jpg
infograph/toothbrush/infograph_317_000455.jpg
infograph/toothbrush/infograph_317_000456.jpg
infograph/toothbrush/infograph_317_000467.jpg
infograph/toothbrush/infograph_317_000478.jpg
infograph/toothbrush/infograph_317_000482.jpg
infograph/toothbrush/infograph_317_000494.jpg
infograph/toothbrush/infograph_317_000502.jpg
infograph/toothbrush/infograph_317_000515.jpg
infograph/toothbrush/infograph_317_000523.jpg
infograph/toothbrush/infograph_317_000525.jpg
infograph/toothbrush/infograph_317_000526.jpg
infograph/toothbrush/infograph_317_000531.jpg
infograph/toothbrush/infograph_317_000536.jpg
infograph/toothbrush/infograph_317_000556.jpg
infograph/toothpaste/infograph_318_000003.jpg
infograph/toothpaste/infograph_318_000009.jpg
infograph/toothpaste/infograph_318_000012.jpg
infograph/toothpaste/infograph_318_000016.jpg
infograph/toothpaste/infograph_318_000019.jpg
infograph/toothpaste/infograph_318_000020.jpg
infograph/toothpaste/infograph_318_000023.jpg
infograph/toothpaste/infograph_318_000024.jpg
infograph/toothpaste/infograph_318_000028.jpg
infograph/toothpaste/infograph_318_000029.jpg
infograph/toothpaste/infograph_318_000032.jpg
infograph/toothpaste/infograph_318_000033.jpg
infograph/toothpaste/infograph_318_000034.jpg
infograph/toothpaste/infograph_318_000044.jpg
infograph/toothpaste/infograph_318_000046.jpg
infograph/toothpaste/infograph_318_000047.jpg
infograph/toothpaste/infograph_318_000049.jpg
infograph/toothpaste/infograph_318_000053.jpg
infograph/toothpaste/infograph_318_000054.jpg
infograph/toothpaste/infograph_318_000062.jpg
infograph/toothpaste/infograph_318_000080.jpg
infograph/toothpaste/infograph_318_000081.jpg
infograph/toothpaste/infograph_318_000085.jpg
infograph/toothpaste/infograph_318_000086.jpg
infograph/toothpaste/infograph_318_000091.jpg
infograph/toothpaste/infograph_318_000096.jpg
infograph/toothpaste/infograph_318_000104.jpg
infograph/toothpaste/infograph_318_000106.jpg
infograph/toothpaste/infograph_318_000111.jpg
infograph/toothpaste/infograph_318_000114.jpg
infograph/toothpaste/infograph_318_000118.jpg
infograph/toothpaste/infograph_318_000120.jpg
infograph/toothpaste/infograph_318_000122.jpg
infograph/toothpaste/infograph_318_000132.jpg
infograph/toothpaste/infograph_318_000135.jpg
infograph/toothpaste/infograph_318_000136.jpg
infograph/toothpaste/infograph_318_000148.jpg
infograph/toothpaste/infograph_318_000153.jpg
infograph/toothpaste/infograph_318_000157.jpg
infograph/toothpaste/infograph_318_000159.jpg
infograph/toothpaste/infograph_318_000174.jpg
infograph/toothpaste/infograph_318_000176.jpg
infograph/toothpaste/infograph_318_000177.jpg
infograph/toothpaste/infograph_318_000178.jpg
infograph/toothpaste/infograph_318_000186.jpg
infograph/toothpaste/infograph_318_000188.jpg
infograph/toothpaste/infograph_318_000193.jpg
infograph/toothpaste/infograph_318_000194.jpg
infograph/toothpaste/infograph_318_000196.jpg
infograph/toothpaste/infograph_318_000200.jpg
infograph/toothpaste/infograph_318_000208.jpg
infograph/toothpaste/infograph_318_000214.jpg
infograph/toothpaste/infograph_318_000215.jpg
infograph/toothpaste/infograph_318_000219.jpg
infograph/toothpaste/infograph_318_000222.jpg
infograph/toothpaste/infograph_318_000224.jpg
infograph/toothpaste/infograph_318_000231.jpg
infograph/toothpaste/infograph_318_000235.jpg
infograph/toothpaste/infograph_318_000243.jpg
infograph/toothpaste/infograph_318_000245.jpg
infograph/toothpaste/infograph_318_000246.jpg
infograph/toothpaste/infograph_318_000247.jpg
infograph/toothpaste/infograph_318_000251.jpg
infograph/toothpaste/infograph_318_000264.jpg
infograph/toothpaste/infograph_318_000266.jpg
infograph/toothpaste/infograph_318_000269.jpg
infograph/toothpaste/infograph_318_000270.jpg
infograph/toothpaste/infograph_318_000273.jpg
infograph/toothpaste/infograph_318_000276.jpg
infograph/toothpaste/infograph_318_000287.jpg
infograph/toothpaste/infograph_318_000288.jpg
infograph/toothpaste/infograph_318_000297.jpg
infograph/toothpaste/infograph_318_000298.jpg
infograph/toothpaste/infograph_318_000305.jpg
infograph/toothpaste/infograph_318_000311.jpg
infograph/toothpaste/infograph_318_000317.jpg
infograph/toothpaste/infograph_318_000318.jpg
infograph/toothpaste/infograph_318_000324.jpg
infograph/toothpaste/infograph_318_000325.jpg
infograph/toothpaste/infograph_318_000326.jpg
infograph/toothpaste/infograph_318_000328.jpg
infograph/toothpaste/infograph_318_000329.jpg
infograph/toothpaste/infograph_318_000338.jpg
infograph/toothpaste/infograph_318_000341.jpg
infograph/toothpaste/infograph_318_000343.jpg
infograph/toothpaste/infograph_318_000346.jpg
infograph/toothpaste/infograph_318_000355.jpg
infograph/toothpaste/infograph_318_000357.jpg
infograph/toothpaste/infograph_318_000358.jpg
infograph/toothpaste/infograph_318_000360.jpg
infograph/toothpaste/infograph_318_000375.jpg
infograph/toothpaste/infograph_318_000381.jpg
infograph/toothpaste/infograph_318_000382.jpg
infograph/toothpaste/infograph_318_000384.jpg
infograph/toothpaste/infograph_318_000386.jpg
infograph/toothpaste/infograph_318_000387.jpg
infograph/toothpaste/infograph_318_000391.jpg
infograph/toothpaste/infograph_318_000395.jpg
infograph/toothpaste/infograph_318_000397.jpg
infograph/toothpaste/infograph_318_000411.jpg
infograph/toothpaste/infograph_318_000421.jpg
infograph/toothpaste/infograph_318_000423.jpg
infograph/toothpaste/infograph_318_000425.jpg
infograph/toothpaste/infograph_318_000431.jpg
infograph/toothpaste/infograph_318_000432.jpg
infograph/toothpaste/infograph_318_000435.jpg
infograph/toothpaste/infograph_318_000437.jpg
infograph/toothpaste/infograph_318_000452.jpg
infograph/toothpaste/infograph_318_000454.jpg
infograph/toothpaste/infograph_318_000455.jpg
infograph/toothpaste/infograph_318_000457.jpg
infograph/toothpaste/infograph_318_000460.jpg
infograph/toothpaste/infograph_318_000466.jpg
infograph/toothpaste/infograph_318_000467.jpg
infograph/toothpaste/infograph_318_000468.jpg
infograph/tornado/infograph_319_000048.jpg
infograph/tornado/infograph_319_000083.jpg
infograph/tornado/infograph_319_000089.jpg
infograph/tornado/infograph_319_000170.jpg
infograph/tornado/infograph_319_000260.jpg
infograph/tornado/infograph_319_000311.jpg
infograph/tractor/infograph_320_000006.jpg
infograph/tractor/infograph_320_000046.jpg
infograph/tractor/infograph_320_000060.jpg
infograph/tractor/infograph_320_000071.jpg
infograph/tractor/infograph_320_000085.jpg
infograph/tractor/infograph_320_000094.jpg
infograph/tractor/infograph_320_000107.jpg
infograph/tractor/infograph_320_000160.jpg
infograph/tractor/infograph_320_000186.jpg
infograph/tractor/infograph_320_000227.jpg
infograph/tractor/infograph_320_000266.jpg
infograph/tractor/infograph_320_000267.jpg
infograph/tractor/infograph_320_000274.jpg
infograph/tractor/infograph_320_000291.jpg
infograph/traffic_light/infograph_321_000040.jpg
infograph/traffic_light/infograph_321_000047.jpg
infograph/traffic_light/infograph_321_000088.jpg
infograph/traffic_light/infograph_321_000134.jpg
infograph/traffic_light/infograph_321_000180.jpg
infograph/traffic_light/infograph_321_000247.jpg
infograph/train/infograph_322_000016.jpg
infograph/train/infograph_322_000033.jpg
infograph/train/infograph_322_000098.jpg
infograph/train/infograph_322_000104.jpg
infograph/train/infograph_322_000123.jpg
infograph/train/infograph_322_000147.jpg
infograph/tree/infograph_323_000092.jpg
infograph/tree/infograph_323_000143.jpg
infograph/tree/infograph_323_000148.jpg
infograph/tree/infograph_323_000181.jpg
infograph/tree/infograph_323_000236.jpg
infograph/tree/infograph_323_000262.jpg
infograph/tree/infograph_323_000365.jpg
infograph/tree/infograph_323_000376.jpg
infograph/tree/infograph_323_000415.jpg
infograph/tree/infograph_323_000488.jpg
infograph/tree/infograph_323_000505.jpg
infograph/triangle/infograph_324_000232.jpg
infograph/trombone/infograph_325_000011.jpg
infograph/trombone/infograph_325_000033.jpg
infograph/trombone/infograph_325_000035.jpg
infograph/trombone/infograph_325_000113.jpg
infograph/trombone/infograph_325_000114.jpg
infograph/trombone/infograph_325_000129.jpg
infograph/trombone/infograph_325_000136.jpg
infograph/trombone/infograph_325_000143.jpg
infograph/trombone/infograph_325_000156.jpg
infograph/trombone/infograph_325_000157.jpg
infograph/trombone/infograph_325_000162.jpg
infograph/truck/infograph_326_000008.jpg
infograph/truck/infograph_326_000057.jpg
infograph/truck/infograph_326_000063.jpg
infograph/truck/infograph_326_000077.jpg
infograph/truck/infograph_326_000101.jpg
infograph/truck/infograph_326_000103.jpg
infograph/truck/infograph_326_000149.jpg
infograph/truck/infograph_326_000165.jpg
infograph/truck/infograph_326_000174.jpg
infograph/truck/infograph_326_000177.jpg
infograph/truck/infograph_326_000210.jpg
infograph/truck/infograph_326_000226.jpg
infograph/truck/infograph_326_000243.jpg
infograph/truck/infograph_326_000260.jpg
infograph/truck/infograph_326_000264.jpg
infograph/truck/infograph_326_000283.jpg
infograph/truck/infograph_326_000299.jpg
infograph/truck/infograph_326_000316.jpg
infograph/truck/infograph_326_000324.jpg
infograph/truck/infograph_326_000326.jpg
infograph/truck/infograph_326_000344.jpg
infograph/truck/infograph_326_000391.jpg
infograph/truck/infograph_326_000393.jpg
infograph/truck/infograph_326_000404.jpg
infograph/truck/infograph_326_000489.jpg
infograph/truck/infograph_326_000513.jpg
infograph/truck/infograph_326_000540.jpg
infograph/truck/infograph_326_000547.jpg
infograph/truck/infograph_326_000553.jpg
infograph/truck/infograph_326_000568.jpg
infograph/truck/infograph_326_000571.jpg
infograph/truck/infograph_326_000581.jpg
infograph/truck/infograph_326_000614.jpg
infograph/truck/infograph_326_000648.jpg
infograph/truck/infograph_326_000649.jpg
infograph/truck/infograph_326_000658.jpg
infograph/truck/infograph_326_000671.jpg
infograph/trumpet/infograph_327_000006.jpg
infograph/trumpet/infograph_327_000009.jpg
infograph/trumpet/infograph_327_000017.jpg
infograph/trumpet/infograph_327_000026.jpg
infograph/trumpet/infograph_327_000030.jpg
infograph/trumpet/infograph_327_000035.jpg
infograph/trumpet/infograph_327_000036.jpg
infograph/trumpet/infograph_327_000048.jpg
infograph/trumpet/infograph_327_000070.jpg
infograph/trumpet/infograph_327_000073.jpg
infograph/trumpet/infograph_327_000074.jpg
infograph/trumpet/infograph_327_000078.jpg
infograph/trumpet/infograph_327_000100.jpg
infograph/trumpet/infograph_327_000107.jpg
infograph/trumpet/infograph_327_000113.jpg
infograph/trumpet/infograph_327_000120.jpg
infograph/trumpet/infograph_327_000134.jpg
infograph/trumpet/infograph_327_000146.jpg
infograph/trumpet/infograph_327_000148.jpg
infograph/trumpet/infograph_327_000157.jpg
infograph/trumpet/infograph_327_000177.jpg
infograph/trumpet/infograph_327_000178.jpg
infograph/trumpet/infograph_327_000198.jpg
infograph/trumpet/infograph_327_000201.jpg
infograph/trumpet/infograph_327_000214.jpg
infograph/trumpet/infograph_327_000220.jpg
infograph/trumpet/infograph_327_000245.jpg
infograph/t-shirt/infograph_328_000054.jpg
infograph/t-shirt/infograph_328_000071.jpg
infograph/t-shirt/infograph_328_000074.jpg
infograph/t-shirt/infograph_328_000112.jpg
infograph/t-shirt/infograph_328_000128.jpg
infograph/t-shirt/infograph_328_000138.jpg
infograph/t-shirt/infograph_328_000154.jpg
infograph/t-shirt/infograph_328_000176.jpg
infograph/t-shirt/infograph_328_000246.jpg
infograph/umbrella/infograph_329_000041.jpg
infograph/umbrella/infograph_329_000064.jpg
infograph/umbrella/infograph_329_000114.jpg
infograph/umbrella/infograph_329_000123.jpg
infograph/umbrella/infograph_329_000187.jpg
infograph/umbrella/infograph_329_000191.jpg
infograph/umbrella/infograph_329_000225.jpg
infograph/umbrella/infograph_329_000229.jpg
infograph/umbrella/infograph_329_000235.jpg
infograph/umbrella/infograph_329_000305.jpg
infograph/umbrella/infograph_329_000316.jpg
infograph/umbrella/infograph_329_000320.jpg
infograph/umbrella/infograph_329_000334.jpg
infograph/umbrella/infograph_329_000341.jpg
infograph/umbrella/infograph_329_000409.jpg
infograph/umbrella/infograph_329_000442.jpg
infograph/umbrella/infograph_329_000449.jpg
infograph/umbrella/infograph_329_000478.jpg
infograph/underwear/infograph_330_000003.jpg
infograph/underwear/infograph_330_000063.jpg
infograph/underwear/infograph_330_000075.jpg
infograph/underwear/infograph_330_000089.jpg
infograph/underwear/infograph_330_000156.jpg
infograph/underwear/infograph_330_000158.jpg
infograph/underwear/infograph_330_000164.jpg
infograph/underwear/infograph_330_000172.jpg
infograph/underwear/infograph_330_000182.jpg
infograph/underwear/infograph_330_000195.jpg
infograph/underwear/infograph_330_000247.jpg
infograph/underwear/infograph_330_000292.jpg
infograph/underwear/infograph_330_000303.jpg
infograph/underwear/infograph_330_000329.jpg
infograph/underwear/infograph_330_000345.jpg
infograph/van/infograph_331_000046.jpg
infograph/van/infograph_331_000097.jpg
infograph/van/infograph_331_000102.jpg
infograph/van/infograph_331_000127.jpg
infograph/van/infograph_331_000144.jpg
infograph/van/infograph_331_000176.jpg
infograph/van/infograph_331_000179.jpg
infograph/van/infograph_331_000200.jpg
infograph/van/infograph_331_000202.jpg
infograph/van/infograph_331_000208.jpg
infograph/van/infograph_331_000234.jpg
infograph/van/infograph_331_000262.jpg
infograph/van/infograph_331_000297.jpg
infograph/van/infograph_331_000325.jpg
infograph/van/infograph_331_000335.jpg
infograph/van/infograph_331_000341.jpg
infograph/van/infograph_331_000349.jpg
infograph/van/infograph_331_000398.jpg
infograph/van/infograph_331_000428.jpg
infograph/van/infograph_331_000436.jpg
infograph/van/infograph_331_000464.jpg
infograph/van/infograph_331_000549.jpg
infograph/van/infograph_331_000550.jpg
infograph/van/infograph_331_000572.jpg
infograph/van/infograph_331_000575.jpg
infograph/van/infograph_331_000666.jpg
infograph/van/infograph_331_000702.jpg
infograph/van/infograph_331_000720.jpg
infograph/van/infograph_331_000749.jpg
infograph/van/infograph_331_000759.jpg
infograph/van/infograph_331_000762.jpg
infograph/van/infograph_331_000763.jpg
infograph/van/infograph_331_000773.jpg
infograph/van/infograph_331_000779.jpg
infograph/van/infograph_331_000790.jpg
infograph/van/infograph_331_000799.jpg
infograph/vase/infograph_332_000002.jpg
infograph/vase/infograph_332_000015.jpg
infograph/vase/infograph_332_000029.jpg
infograph/vase/infograph_332_000065.jpg
infograph/vase/infograph_332_000175.jpg
infograph/vase/infograph_332_000176.jpg
infograph/vase/infograph_332_000177.jpg
infograph/vase/infograph_332_000182.jpg
infograph/vase/infograph_332_000236.jpg
infograph/vase/infograph_332_000253.jpg
infograph/vase/infograph_332_000279.jpg
infograph/vase/infograph_332_000296.jpg
infograph/vase/infograph_332_000309.jpg
infograph/vase/infograph_332_000311.jpg
infograph/violin/infograph_333_000001.jpg
infograph/violin/infograph_333_000025.jpg
infograph/violin/infograph_333_000031.jpg
infograph/violin/infograph_333_000060.jpg
infograph/violin/infograph_333_000069.jpg
infograph/violin/infograph_333_000071.jpg
infograph/violin/infograph_333_000081.jpg
infograph/violin/infograph_333_000090.jpg
infograph/violin/infograph_333_000098.jpg
infograph/violin/infograph_333_000101.jpg
infograph/violin/infograph_333_000114.jpg
infograph/violin/infograph_333_000124.jpg
infograph/violin/infograph_333_000130.jpg
infograph/violin/infograph_333_000131.jpg
infograph/violin/infograph_333_000138.jpg
infograph/violin/infograph_333_000156.jpg
infograph/violin/infograph_333_000158.jpg
infograph/violin/infograph_333_000171.jpg
infograph/violin/infograph_333_000177.jpg
infograph/violin/infograph_333_000184.jpg
infograph/violin/infograph_333_000186.jpg
infograph/violin/infograph_333_000192.jpg
infograph/violin/infograph_333_000193.jpg
infograph/violin/infograph_333_000204.jpg
infograph/violin/infograph_333_000227.jpg
infograph/violin/infograph_333_000248.jpg
infograph/washing_machine/infograph_334_000018.jpg
infograph/washing_machine/infograph_334_000043.jpg
infograph/washing_machine/infograph_334_000054.jpg
infograph/washing_machine/infograph_334_000102.jpg
infograph/washing_machine/infograph_334_000176.jpg
infograph/washing_machine/infograph_334_000183.jpg
infograph/washing_machine/infograph_334_000185.jpg
infograph/washing_machine/infograph_334_000222.jpg
infograph/washing_machine/infograph_334_000226.jpg
infograph/washing_machine/infograph_334_000273.jpg
infograph/washing_machine/infograph_334_000305.jpg
infograph/washing_machine/infograph_334_000308.jpg
infograph/washing_machine/infograph_334_000345.jpg
infograph/washing_machine/infograph_334_000347.jpg
infograph/washing_machine/infograph_334_000376.jpg
infograph/washing_machine/infograph_334_000381.jpg
infograph/washing_machine/infograph_334_000393.jpg
infograph/washing_machine/infograph_334_000460.jpg
infograph/washing_machine/infograph_334_000466.jpg
infograph/watermelon/infograph_335_000006.jpg
infograph/watermelon/infograph_335_000010.jpg
infograph/watermelon/infograph_335_000020.jpg
infograph/watermelon/infograph_335_000033.jpg
infograph/watermelon/infograph_335_000034.jpg
infograph/watermelon/infograph_335_000045.jpg
infograph/watermelon/infograph_335_000046.jpg
infograph/watermelon/infograph_335_000072.jpg
infograph/watermelon/infograph_335_000080.jpg
infograph/watermelon/infograph_335_000082.jpg
infograph/watermelon/infograph_335_000087.jpg
infograph/watermelon/infograph_335_000094.jpg
infograph/watermelon/infograph_335_000105.jpg
infograph/watermelon/infograph_335_000107.jpg
infograph/watermelon/infograph_335_000117.jpg
infograph/watermelon/infograph_335_000144.jpg
infograph/watermelon/infograph_335_000157.jpg
infograph/watermelon/infograph_335_000170.jpg
infograph/watermelon/infograph_335_000175.jpg
infograph/watermelon/infograph_335_000178.jpg
infograph/watermelon/infograph_335_000181.jpg
infograph/watermelon/infograph_335_000189.jpg
infograph/watermelon/infograph_335_000196.jpg
infograph/watermelon/infograph_335_000237.jpg
infograph/watermelon/infograph_335_000275.jpg
infograph/watermelon/infograph_335_000277.jpg
infograph/watermelon/infograph_335_000285.jpg
infograph/watermelon/infograph_335_000295.jpg
infograph/watermelon/infograph_335_000317.jpg
infograph/watermelon/infograph_335_000358.jpg
infograph/watermelon/infograph_335_000359.jpg
infograph/watermelon/infograph_335_000376.jpg
infograph/watermelon/infograph_335_000384.jpg
infograph/watermelon/infograph_335_000393.jpg
infograph/waterslide/infograph_336_000004.jpg
infograph/waterslide/infograph_336_000018.jpg
infograph/waterslide/infograph_336_000052.jpg
infograph/waterslide/infograph_336_000080.jpg
infograph/waterslide/infograph_336_000094.jpg
infograph/waterslide/infograph_336_000139.jpg
infograph/waterslide/infograph_336_000168.jpg
infograph/waterslide/infograph_336_000207.jpg
infograph/waterslide/infograph_336_000272.jpg
infograph/waterslide/infograph_336_000279.jpg
infograph/waterslide/infograph_336_000308.jpg
infograph/whale/infograph_337_000003.jpg
infograph/whale/infograph_337_000006.jpg
infograph/whale/infograph_337_000010.jpg
infograph/whale/infograph_337_000011.jpg
infograph/whale/infograph_337_000017.jpg
infograph/whale/infograph_337_000025.jpg
infograph/whale/infograph_337_000038.jpg
infograph/whale/infograph_337_000070.jpg
infograph/whale/infograph_337_000073.jpg
infograph/whale/infograph_337_000080.jpg
infograph/whale/infograph_337_000089.jpg
infograph/whale/infograph_337_000090.jpg
infograph/whale/infograph_337_000096.jpg
infograph/whale/infograph_337_000103.jpg
infograph/whale/infograph_337_000104.jpg
infograph/whale/infograph_337_000107.jpg
infograph/whale/infograph_337_000114.jpg
infograph/whale/infograph_337_000124.jpg
infograph/whale/infograph_337_000128.jpg
infograph/whale/infograph_337_000138.jpg
infograph/whale/infograph_337_000139.jpg
infograph/whale/infograph_337_000146.jpg
infograph/whale/infograph_337_000162.jpg
infograph/whale/infograph_337_000169.jpg
infograph/whale/infograph_337_000183.jpg
infograph/whale/infograph_337_000186.jpg
infograph/whale/infograph_337_000196.jpg
infograph/whale/infograph_337_000198.jpg
infograph/whale/infograph_337_000215.jpg
infograph/whale/infograph_337_000226.jpg
infograph/whale/infograph_337_000230.jpg
infograph/whale/infograph_337_000238.jpg
infograph/whale/infograph_337_000240.jpg
infograph/whale/infograph_337_000247.jpg
infograph/whale/infograph_337_000287.jpg
infograph/whale/infograph_337_000289.jpg
infograph/whale/infograph_337_000294.jpg
infograph/whale/infograph_337_000300.jpg
infograph/whale/infograph_337_000302.jpg
infograph/whale/infograph_337_000304.jpg
infograph/whale/infograph_337_000308.jpg
infograph/whale/infograph_337_000315.jpg
infograph/whale/infograph_337_000317.jpg
infograph/whale/infograph_337_000318.jpg
infograph/whale/infograph_337_000321.jpg
infograph/whale/infograph_337_000325.jpg
infograph/whale/infograph_337_000327.jpg
infograph/whale/infograph_337_000346.jpg
infograph/whale/infograph_337_000347.jpg
infograph/whale/infograph_337_000350.jpg
infograph/whale/infograph_337_000360.jpg
infograph/whale/infograph_337_000376.jpg
infograph/whale/infograph_337_000381.jpg
infograph/whale/infograph_337_000384.jpg
infograph/whale/infograph_337_000388.jpg
infograph/whale/infograph_337_000400.jpg
infograph/whale/infograph_337_000409.jpg
infograph/whale/infograph_337_000410.jpg
infograph/whale/infograph_337_000418.jpg
infograph/whale/infograph_337_000419.jpg
infograph/whale/infograph_337_000426.jpg
infograph/wheel/infograph_338_000085.jpg
infograph/wheel/infograph_338_000147.jpg
infograph/wheel/infograph_338_000153.jpg
infograph/wheel/infograph_338_000158.jpg
infograph/wheel/infograph_338_000172.jpg
infograph/wheel/infograph_338_000239.jpg
infograph/wheel/infograph_338_000292.jpg
infograph/wheel/infograph_338_000338.jpg
infograph/wheel/infograph_338_000363.jpg
infograph/wheel/infograph_338_000367.jpg
infograph/wheel/infograph_338_000385.jpg
infograph/windmill/infograph_339_000001.jpg
infograph/windmill/infograph_339_000011.jpg
infograph/windmill/infograph_339_000026.jpg
infograph/windmill/infograph_339_000092.jpg
infograph/windmill/infograph_339_000105.jpg
infograph/windmill/infograph_339_000118.jpg
infograph/windmill/infograph_339_000123.jpg
infograph/windmill/infograph_339_000127.jpg
infograph/windmill/infograph_339_000128.jpg
infograph/windmill/infograph_339_000154.jpg
infograph/windmill/infograph_339_000159.jpg
infograph/windmill/infograph_339_000201.jpg
infograph/windmill/infograph_339_000217.jpg
infograph/windmill/infograph_339_000238.jpg
infograph/windmill/infograph_339_000244.jpg
infograph/windmill/infograph_339_000246.jpg
infograph/windmill/infograph_339_000264.jpg
infograph/windmill/infograph_339_000265.jpg
infograph/windmill/infograph_339_000283.jpg
infograph/windmill/infograph_339_000291.jpg
infograph/windmill/infograph_339_000334.jpg
infograph/windmill/infograph_339_000350.jpg
infograph/wine_bottle/infograph_340_000030.jpg
infograph/wine_bottle/infograph_340_000084.jpg
infograph/wine_bottle/infograph_340_000090.jpg
infograph/wine_bottle/infograph_340_000097.jpg
infograph/wine_bottle/infograph_340_000322.jpg
infograph/wine_bottle/infograph_340_000415.jpg
infograph/wine_bottle/infograph_340_000429.jpg
infograph/wine_glass/infograph_341_000009.jpg
infograph/wine_glass/infograph_341_000019.jpg
infograph/wine_glass/infograph_341_000022.jpg
infograph/wine_glass/infograph_341_000044.jpg
infograph/wine_glass/infograph_341_000064.jpg
infograph/wine_glass/infograph_341_000065.jpg
infograph/wine_glass/infograph_341_000075.jpg
infograph/wine_glass/infograph_341_000076.jpg
infograph/wine_glass/infograph_341_000080.jpg
infograph/wine_glass/infograph_341_000085.jpg
infograph/wine_glass/infograph_341_000087.jpg
infograph/wine_glass/infograph_341_000090.jpg
infograph/wine_glass/infograph_341_000101.jpg
infograph/wine_glass/infograph_341_000102.jpg
infograph/wine_glass/infograph_341_000104.jpg
infograph/wine_glass/infograph_341_000113.jpg
infograph/wine_glass/infograph_341_000115.jpg
infograph/wine_glass/infograph_341_000116.jpg
infograph/wine_glass/infograph_341_000123.jpg
infograph/wine_glass/infograph_341_000124.jpg
infograph/wine_glass/infograph_341_000127.jpg
infograph/wine_glass/infograph_341_000132.jpg
infograph/wine_glass/infograph_341_000133.jpg
infograph/wine_glass/infograph_341_000135.jpg
infograph/wine_glass/infograph_341_000141.jpg
infograph/wine_glass/infograph_341_000150.jpg
infograph/wine_glass/infograph_341_000157.jpg
infograph/wine_glass/infograph_341_000175.jpg
infograph/wine_glass/infograph_341_000180.jpg
infograph/wine_glass/infograph_341_000198.jpg
infograph/wine_glass/infograph_341_000207.jpg
infograph/wine_glass/infograph_341_000216.jpg
infograph/wine_glass/infograph_341_000239.jpg
infograph/wine_glass/infograph_341_000245.jpg
infograph/wine_glass/infograph_341_000247.jpg
infograph/wine_glass/infograph_341_000249.jpg
infograph/wine_glass/infograph_341_000257.jpg
infograph/wine_glass/infograph_341_000263.jpg
infograph/wine_glass/infograph_341_000270.jpg
infograph/wine_glass/infograph_341_000272.jpg
infograph/wine_glass/infograph_341_000278.jpg
infograph/wine_glass/infograph_341_000283.jpg
infograph/wine_glass/infograph_341_000284.jpg
infograph/wine_glass/infograph_341_000288.jpg
infograph/wine_glass/infograph_341_000303.jpg
infograph/wine_glass/infograph_341_000314.jpg
infograph/wine_glass/infograph_341_000317.jpg
infograph/wine_glass/infograph_341_000318.jpg
infograph/wine_glass/infograph_341_000321.jpg
infograph/wine_glass/infograph_341_000343.jpg
infograph/wine_glass/infograph_341_000345.jpg
infograph/wine_glass/infograph_341_000368.jpg
infograph/wine_glass/infograph_341_000378.jpg
infograph/wine_glass/infograph_341_000380.jpg
infograph/wine_glass/infograph_341_000394.jpg
infograph/wine_glass/infograph_341_000395.jpg
infograph/wine_glass/infograph_341_000399.jpg
infograph/wine_glass/infograph_341_000402.jpg
infograph/wine_glass/infograph_341_000408.jpg
infograph/wine_glass/infograph_341_000412.jpg
infograph/wine_glass/infograph_341_000423.jpg
infograph/wine_glass/infograph_341_000428.jpg
infograph/wine_glass/infograph_341_000435.jpg
infograph/wine_glass/infograph_341_000461.jpg
infograph/wine_glass/infograph_341_000464.jpg
infograph/wine_glass/infograph_341_000475.jpg
infograph/wine_glass/infograph_341_000490.jpg
infograph/wine_glass/infograph_341_000495.jpg
infograph/wine_glass/infograph_341_000499.jpg
infograph/wine_glass/infograph_341_000507.jpg
infograph/wine_glass/infograph_341_000511.jpg
infograph/wine_glass/infograph_341_000516.jpg
infograph/wine_glass/infograph_341_000518.jpg
infograph/wine_glass/infograph_341_000524.jpg
infograph/wine_glass/infograph_341_000530.jpg
infograph/wine_glass/infograph_341_000531.jpg
infograph/wine_glass/infograph_341_000550.jpg
infograph/wine_glass/infograph_341_000558.jpg
infograph/wine_glass/infograph_341_000579.jpg
infograph/wine_glass/infograph_341_000586.jpg
infograph/wine_glass/infograph_341_000597.jpg
infograph/wine_glass/infograph_341_000606.jpg
infograph/wine_glass/infograph_341_000607.jpg
infograph/wine_glass/infograph_341_000608.jpg
infograph/wine_glass/infograph_341_000613.jpg
infograph/wristwatch/infograph_342_000135.jpg
infograph/wristwatch/infograph_342_000233.jpg
infograph/wristwatch/infograph_342_000300.jpg
infograph/wristwatch/infograph_342_000334.jpg
infograph/yoga/infograph_343_000035.jpg
infograph/yoga/infograph_343_000124.jpg
infograph/yoga/infograph_343_000134.jpg
infograph/yoga/infograph_343_000163.jpg
infograph/yoga/infograph_343_000164.jpg
infograph/yoga/infograph_343_000234.jpg
infograph/yoga/infograph_343_000263.jpg
infograph/yoga/infograph_343_000287.jpg
infograph/yoga/infograph_343_000328.jpg
infograph/yoga/infograph_343_000422.jpg
infograph/zebra/infograph_344_000084.jpg
infograph/zebra/infograph_344_000103.jpg
infograph/zebra/infograph_344_000127.jpg
infograph/zebra/infograph_344_000136.jpg
infograph/zebra/infograph_344_000184.jpg
infograph/zebra/infograph_344_000188.jpg
infograph/zebra/infograph_344_000267.jpg
infograph/zebra/infograph_344_000284.jpg
infograph/zebra/infograph_344_000292.jpg
infograph/zebra/infograph_344_000297.jpg
infograph/zigzag/infograph_345_000027.jpg
infograph/zigzag/infograph_345_000050.jpg
infograph/zigzag/infograph_345_000102.jpg
infograph/zigzag/infograph_345_000143.jpg
infograph/zigzag/infograph_345_000163.jpg
infograph/zigzag/infograph_345_000173.jpg
infograph/zigzag/infograph_345_000209.jpg
infograph/zigzag/infograph_345_000220.jpg
infograph/zigzag/infograph_345_000221.jpg
infograph/zigzag/infograph_345_000230.jpg
infograph/zigzag/infograph_345_000276.jpg
infograph/zigzag/infograph_345_000287.jpg
infograph/zigzag/infograph_345_000333.jpg
infograph/zigzag/infograph_345_000347.jpg
painting/airplane/painting_002_000134.jpg
painting/airplane/painting_002_000169.jpg
painting/airplane/painting_002_000171.jpg
painting/airplane/painting_002_000197.jpg
painting/airplane/painting_002_000203.jpg
painting/airplane/painting_002_000208.jpg
painting/animal_migration/painting_006_000103.jpg
painting/animal_migration/painting_006_000209.jpg
painting/animal_migration/painting_006_000292.jpg
painting/animal_migration/painting_006_000340.jpg
painting/animal_migration/painting_006_000424.jpg
painting/animal_migration/painting_006_000576.jpg
painting/animal_migration/painting_006_000579.jpg
painting/anvil/painting_008_000051.jpg
painting/asparagus/painting_011_000310.jpg
painting/asparagus/painting_011_000317.jpg
painting/asparagus/painting_011_000343.jpg
painting/banana/painting_014_000064.jpg
painting/banana/painting_014_000269.jpg
painting/bandage/painting_015_000024.jpg
painting/bandage/painting_015_000142.jpg
painting/bandage/painting_015_000163.jpg
painting/baseball/painting_017_000069.jpg
painting/baseball_bat/painting_018_000001.jpg
painting/baseball_bat/painting_018_000008.jpg
painting/baseball_bat/painting_018_000064.jpg
painting/baseball_bat/painting_018_000109.jpg
painting/baseball_bat/painting_018_000115.jpg
painting/baseball_bat/painting_018_000134.jpg
painting/baseball_bat/painting_018_000135.jpg
painting/baseball_bat/painting_018_000136.jpg
painting/baseball_bat/painting_018_000143.jpg
painting/baseball_bat/painting_018_000144.jpg
painting/basket/painting_019_000112.jpg
painting/basket/painting_019_000186.jpg
painting/basket/painting_019_000206.jpg
painting/basketball/painting_020_000118.jpg
painting/basketball/painting_020_000140.jpg
painting/bat/painting_021_000087.jpg
painting/bat/painting_021_000271.jpg
painting/bat/painting_021_000277.jpg
painting/bathtub/painting_022_000036.jpg
painting/beard/painting_025_000029.jpg
painting/beard/painting_025_000336.jpg
painting/bicycle/painting_030_000123.jpg
painting/bicycle/painting_030_000177.jpg
painting/bird/painting_032_000033.jpg
painting/bird/painting_032_000102.jpg
painting/blueberry/painting_035_000055.jpg
painting/blueberry/painting_035_000160.jpg
painting/book/painting_036_000019.jpg
painting/bottlecap/painting_038_000294.jpg
painting/bottlecap/painting_038_000309.jpg
painting/bottlecap/painting_038_000323.jpg
painting/bottlecap/painting_038_000324.jpg
painting/bowtie/painting_039_000003.jpg
painting/bowtie/painting_039_000122.jpg
painting/bowtie/painting_039_000171.jpg
painting/bowtie/painting_039_000172.jpg
painting/bowtie/painting_039_000175.jpg
painting/bowtie/painting_039_000211.jpg
painting/bowtie/painting_039_000219.jpg
painting/bowtie/painting_039_000229.jpg
painting/bowtie/painting_039_000259.jpg
painting/bowtie/painting_039_000260.jpg
painting/bowtie/painting_039_000284.jpg
painting/bowtie/painting_039_000285.jpg
painting/bread/painting_042_000288.jpg
painting/cake/painting_052_000009.jpg
painting/cake/painting_052_000022.jpg
painting/cake/painting_052_000039.jpg
painting/cake/painting_052_000041.jpg
painting/cake/painting_052_000047.jpg
painting/cake/painting_052_000051.jpg
painting/cake/painting_052_000052.jpg
painting/cake/painting_052_000069.jpg
painting/cake/painting_052_000072.jpg
painting/cake/painting_052_000074.jpg
painting/cake/painting_052_000084.jpg
painting/cake/painting_052_000095.jpg
painting/cake/painting_052_000102.jpg
painting/cake/painting_052_000130.jpg
painting/cake/painting_052_000132.jpg
painting/cake/painting_052_000141.jpg
painting/cake/painting_052_000144.jpg
painting/cake/painting_052_000148.jpg
painting/cake/painting_052_000156.jpg
painting/cake/painting_052_000172.jpg
painting/candle/painting_059_000031.jpg
painting/candle/painting_059_000075.jpg
painting/canoe/painting_061_000060.jpg
painting/canoe/painting_061_000061.jpg
painting/canoe/painting_061_000062.jpg
painting/canoe/painting_061_000066.jpg
painting/canoe/painting_061_000070.jpg
painting/canoe/painting_061_000071.jpg
painting/canoe/painting_061_000075.jpg
painting/canoe/painting_061_000078.jpg
painting/canoe/painting_061_000080.jpg
painting/canoe/painting_061_000082.jpg
painting/canoe/painting_061_000104.jpg
painting/canoe/painting_061_000190.jpg
painting/canoe/painting_061_000239.jpg
painting/canoe/painting_061_000242.jpg
painting/canoe/painting_061_000245.jpg
painting/canoe/painting_061_000247.jpg
painting/canoe/painting_061_000249.jpg
painting/canoe/painting_061_000303.jpg
painting/canoe/painting_061_000304.jpg
painting/canoe/painting_061_000305.jpg
painting/canoe/painting_061_000375.jpg
painting/canoe/painting_061_000378.jpg
painting/carrot/painting_063_000134.jpg
painting/cell_phone/painting_068_000053.jpg
painting/cell_phone/painting_068_000079.jpg
painting/cell_phone/painting_068_000106.jpg
painting/chair/painting_069_000048.jpg
painting/clock/painting_074_000002.jpg
painting/clock/painting_074_000085.jpg
painting/clock/painting_074_000150.jpg
painting/clock/painting_074_000174.jpg
painting/clock/painting_074_000248.jpg
painting/clock/painting_074_000261.jpg
painting/cloud/painting_075_000148.jpg
painting/cloud/painting_075_000190.jpg
painting/coffee_cup/painting_076_000003.jpg
painting/computer/painting_078_000008.jpg
painting/computer/painting_078_000017.jpg
painting/crayon/painting_084_000010.jpg
painting/crown/painting_086_000066.jpg
painting/cruise_ship/painting_087_000015.jpg
painting/cruise_ship/painting_087_000142.jpg
painting/cruise_ship/painting_087_000213.jpg
painting/cup/painting_088_000001.jpg
painting/cup/painting_088_000002.jpg
painting/cup/painting_088_000006.jpg
painting/cup/painting_088_000013.jpg
painting/cup/painting_088_000016.jpg
painting/cup/painting_088_000035.jpg
painting/cup/painting_088_000036.jpg
painting/cup/painting_088_000043.jpg
painting/cup/painting_088_000048.jpg
painting/cup/painting_088_000049.jpg
painting/cup/painting_088_000054.jpg
painting/cup/painting_088_000064.jpg
painting/cup/painting_088_000077.jpg
painting/cup/painting_088_000078.jpg
painting/cup/painting_088_000088.jpg
painting/cup/painting_088_000092.jpg
painting/cup/painting_088_000094.jpg
painting/cup/painting_088_000123.jpg
painting/cup/painting_088_000128.jpg
painting/cup/painting_088_000129.jpg
painting/cup/painting_088_000135.jpg
painting/cup/painting_088_000137.jpg
painting/cup/painting_088_000149.jpg
painting/cup/painting_088_000154.jpg
painting/cup/painting_088_000156.jpg
painting/cup/painting_088_000162.jpg
painting/cup/painting_088_000170.jpg
painting/cup/painting_088_000190.jpg
painting/cup/painting_088_000192.jpg
painting/cup/painting_088_000209.jpg
painting/cup/painting_088_000210.jpg
painting/cup/painting_088_000232.jpg
painting/cup/painting_088_000233.jpg
painting/cup/painting_088_000258.jpg
painting/cup/painting_088_000278.jpg
painting/cup/painting_088_000283.jpg
painting/cup/painting_088_000293.jpg
painting/cup/painting_088_000295.jpg
painting/cup/painting_088_000297.jpg
painting/cup/painting_088_000307.jpg
painting/cup/painting_088_000308.jpg
painting/cup/painting_088_000318.jpg
painting/cup/painting_088_000333.jpg
painting/cup/painting_088_000342.jpg
painting/cup/painting_088_000346.jpg
painting/cup/painting_088_000358.jpg
painting/cup/painting_088_000363.jpg
painting/cup/painting_088_000401.jpg
painting/cup/painting_088_000413.jpg
painting/cup/painting_088_000466.jpg
painting/cup/painting_088_000467.jpg
painting/cup/painting_088_000475.jpg
painting/cup/painting_088_000481.jpg
painting/cup/painting_088_000492.jpg
painting/cup/painting_088_000500.jpg
painting/cup/painting_088_000528.jpg
painting/cup/painting_088_000532.jpg
painting/cup/painting_088_000549.jpg
painting/cup/painting_088_000559.jpg
painting/cup/painting_088_000560.jpg
painting/cup/painting_088_000561.jpg
painting/cup/painting_088_000562.jpg
painting/cup/painting_088_000563.jpg
painting/cup/painting_088_000576.jpg
painting/cup/painting_088_000577.jpg
painting/cup/painting_088_000580.jpg
painting/diving_board/painting_091_000033.jpg
painting/diving_board/painting_091_000064.jpg
painting/diving_board/painting_091_000102.jpg
painting/diving_board/painting_091_000107.jpg
painting/dog/painting_092_000179.jpg
painting/dog/painting_092_000407.jpg
painting/dog/painting_092_000522.jpg
painting/drums/painting_099_000030.jpg
painting/elbow/painting_103_000046.jpg
painting/elbow/painting_103_000047.jpg
painting/elbow/painting_103_000053.jpg
painting/elbow/painting_103_000056.jpg
painting/elbow/painting_103_000057.jpg
painting/elbow/painting_103_000060.jpg
painting/elbow/painting_103_000064.jpg
painting/elbow/painting_103_000067.jpg
painting/elbow/painting_103_000072.jpg
painting/elbow/painting_103_000073.jpg
painting/elbow/painting_103_000094.jpg
painting/elbow/painting_103_000096.jpg
painting/elephant/painting_104_000204.jpg
painting/eye/painting_107_000049.jpg
painting/eye/painting_107_000080.jpg
painting/eye/painting_107_000098.jpg
painting/eye/painting_107_000101.jpg
painting/feather/painting_111_000097.jpg
painting/firetruck/painting_116_000002.jpg
painting/firetruck/painting_116_000004.jpg
painting/firetruck/painting_116_000006.jpg
painting/firetruck/painting_116_000008.jpg
painting/firetruck/painting_116_000013.jpg
painting/firetruck/painting_116_000014.jpg
painting/firetruck/painting_116_000015.jpg
painting/firetruck/painting_116_000019.jpg
painting/firetruck/painting_116_000024.jpg
painting/firetruck/painting_116_000026.jpg
painting/firetruck/painting_116_000036.jpg
painting/firetruck/painting_116_000039.jpg
painting/firetruck/painting_116_000053.jpg
painting/firetruck/painting_116_000156.jpg
painting/firetruck/painting_116_000193.jpg
painting/firetruck/painting_116_000304.jpg
painting/fish/painting_117_000059.jpg
painting/fish/painting_117_000060.jpg
painting/fish/painting_117_000062.jpg
painting/fish/painting_117_000067.jpg
painting/fish/painting_117_000077.jpg
painting/fish/painting_117_000078.jpg
painting/fish/painting_117_000080.jpg
painting/fish/painting_117_000082.jpg
painting/fish/painting_117_000084.jpg
painting/fish/painting_117_000085.jpg
painting/fish/painting_117_000140.jpg
painting/fish/painting_117_000143.jpg
painting/fish/painting_117_000144.jpg
painting/fish/painting_117_000145.jpg
painting/fish/painting_117_000155.jpg
painting/fish/painting_117_000157.jpg
painting/fish/painting_117_000159.jpg
painting/fish/painting_117_000170.jpg
painting/fish/painting_117_000171.jpg
painting/fish/painting_117_000176.jpg
painting/fish/painting_117_000181.jpg
painting/fish/painting_117_000189.jpg
painting/fish/painting_117_000222.jpg
painting/fish/painting_117_000223.jpg
painting/fish/painting_117_000230.jpg
painting/fish/painting_117_000235.jpg
painting/fish/painting_117_000245.jpg
painting/fish/painting_117_000249.jpg
painting/fish/painting_117_000261.jpg
painting/fish/painting_117_000263.jpg
painting/fish/painting_117_000271.jpg
painting/fish/painting_117_000283.jpg
painting/fish/painting_117_000296.jpg
painting/fish/painting_117_000308.jpg
painting/fish/painting_117_000310.jpg
painting/fish/painting_117_000321.jpg
painting/fish/painting_117_000344.jpg
painting/fish/painting_117_000345.jpg
painting/fish/painting_117_000348.jpg
painting/fish/painting_117_000352.jpg
painting/fish/painting_117_000359.jpg
painting/fish/painting_117_000360.jpg
painting/fish/painting_117_000361.jpg
painting/fish/painting_117_000369.jpg
painting/fish/painting_117_000370.jpg
painting/fish/painting_117_000377.jpg
painting/fish/painting_117_000379.jpg
painting/fish/painting_117_000380.jpg
painting/fish/painting_117_000424.jpg
painting/flamingo/painting_118_000057.jpg
painting/flamingo/painting_118_000060.jpg
painting/foot/painting_124_000045.jpg
painting/garden/painting_128_000054.jpg
painting/garden/painting_128_000174.jpg
painting/goatee/painting_131_000004.jpg
painting/goatee/painting_131_000019.jpg
painting/goatee/painting_131_000024.jpg
painting/goatee/painting_131_000027.jpg
painting/goatee/painting_131_000030.jpg
painting/goatee/painting_131_000036.jpg
painting/goatee/painting_131_000046.jpg
painting/goatee/painting_131_000052.jpg
painting/goatee/painting_131_000053.jpg
painting/goatee/painting_131_000061.jpg
painting/goatee/painting_131_000074.jpg
painting/goatee/painting_131_000076.jpg
painting/goatee/painting_131_000080.jpg
painting/goatee/painting_131_000086.jpg
painting/goatee/painting_131_000088.jpg
painting/goatee/painting_131_000091.jpg
painting/goatee/painting_131_000096.jpg
painting/goatee/painting_131_000111.jpg
painting/goatee/painting_131_000114.jpg
painting/goatee/painting_131_000122.jpg
painting/golf_club/painting_132_000115.jpg
painting/golf_club/painting_132_000123.jpg
painting/golf_club/painting_132_000164.jpg
painting/golf_club/painting_132_000210.jpg
painting/golf_club/painting_132_000259.jpg
painting/golf_club/painting_132_000317.jpg
painting/golf_club/painting_132_000363.jpg
painting/golf_club/painting_132_000417.jpg
painting/golf_club/painting_132_000426.jpg
painting/golf_club/painting_132_000434.jpg
painting/golf_club/painting_132_000435.jpg
painting/golf_club/painting_132_000437.jpg
painting/golf_club/painting_132_000462.jpg
painting/golf_club/painting_132_000556.jpg
painting/grapes/painting_133_000048.jpg
painting/grapes/painting_133_000107.jpg
painting/grapes/painting_133_000109.jpg
painting/grapes/painting_133_000114.jpg
painting/grapes/painting_133_000279.jpg
painting/grapes/painting_133_000312.jpg
painting/grass/painting_134_000030.jpg
painting/grass/painting_134_000086.jpg
painting/grass/painting_134_000208.jpg
painting/grass/painting_134_000256.jpg
painting/grass/painting_134_000279.jpg
painting/guitar/painting_135_000007.jpg
painting/hamburger/painting_136_000110.jpg
painting/hamburger/painting_136_000124.jpg
painting/hammer/painting_137_000028.jpg
painting/hand/painting_138_000007.jpg
painting/hand/painting_138_000008.jpg
painting/hand/painting_138_000034.jpg
painting/hand/painting_138_000057.jpg
painting/hand/painting_138_000102.jpg
painting/hand/painting_138_000195.jpg
painting/hat/painting_140_000036.jpg
painting/hat/painting_140_000053.jpg
painting/hat/painting_140_000070.jpg
painting/hat/painting_140_000151.jpg
painting/hat/painting_140_000198.jpg
painting/hat/painting_140_000365.jpg
painting/hedgehog/painting_142_000069.jpg
painting/hockey_puck/painting_146_000002.jpg
painting/hockey_puck/painting_146_000024.jpg
painting/hockey_puck/painting_146_000068.jpg
painting/hockey_puck/painting_146_000069.jpg
painting/hockey_puck/painting_146_000092.jpg
painting/hockey_puck/painting_146_000097.jpg
painting/hockey_puck/painting_146_000098.jpg
painting/hockey_puck/painting_146_000101.jpg
painting/hockey_puck/painting_146_000117.jpg
painting/hockey_puck/painting_146_000183.jpg
painting/hockey_puck/painting_146_000193.jpg
painting/hockey_puck/painting_146_000208.jpg
painting/hockey_puck/painting_146_000220.jpg
painting/hockey_stick/painting_147_000001.jpg
painting/hockey_stick/painting_147_000005.jpg
painting/hockey_stick/painting_147_000031.jpg
painting/hockey_stick/painting_147_000057.jpg
painting/hockey_stick/painting_147_000075.jpg
painting/hockey_stick/painting_147_000146.jpg
painting/hot_dog/painting_151_000128.jpg
painting/hot_tub/painting_152_000051.jpg
painting/hot_tub/painting_152_000054.jpg
painting/hourglass/painting_153_000021.jpg
painting/hourglass/painting_153_000079.jpg
painting/hourglass/painting_153_000107.jpg
painting/hourglass/painting_153_000110.jpg
painting/hourglass/painting_153_000139.jpg
painting/hourglass/painting_153_000169.jpg
painting/house_plant/painting_155_000057.jpg
painting/house_plant/painting_155_000168.jpg
painting/house_plant/painting_155_000170.jpg
painting/house_plant/painting_155_000171.jpg
painting/house_plant/painting_155_000318.jpg
painting/kangaroo/painting_160_000057.jpg
painting/kangaroo/painting_160_000110.jpg
painting/kangaroo/painting_160_000142.jpg
painting/key/painting_161_000083.jpg
painting/knee/painting_163_000059.jpg
painting/knee/painting_163_000066.jpg
painting/knee/painting_163_000121.jpg
painting/ladder/painting_165_000102.jpg
painting/ladder/painting_165_000306.jpg
painting/laptop/painting_167_000005.jpg
painting/laptop/painting_167_000019.jpg
painting/laptop/painting_167_000047.jpg
painting/laptop/painting_167_000071.jpg
painting/laptop/painting_167_000080.jpg
painting/laptop/painting_167_000149.jpg
painting/leaf/painting_168_000029.jpg
painting/leaf/painting_168_000198.jpg
painting/light_bulb/painting_170_000335.jpg
painting/lighthouse/painting_172_000289.jpg
painting/lighthouse/painting_172_000336.jpg
painting/lighthouse/painting_172_000341.jpg
painting/lighthouse/painting_172_000358.jpg
painting/lighthouse/painting_172_000360.jpg
painting/lighthouse/painting_172_000361.jpg
painting/lighthouse/painting_172_000392.jpg
painting/lightning/painting_173_000066.jpg
painting/lightning/painting_173_000144.jpg
painting/line/painting_174_000266.jpg
painting/line/painting_174_000329.jpg
painting/lion/painting_175_000431.jpg
painting/lobster/painting_177_000092.jpg
painting/lobster/painting_177_000179.jpg
painting/lollipop/painting_178_000026.jpg
painting/lollipop/painting_178_000095.jpg
painting/lollipop/painting_178_000143.jpg
painting/lollipop/painting_178_000220.jpg
painting/megaphone/painting_183_000015.jpg
painting/megaphone/painting_183_000017.jpg
painting/megaphone/painting_183_000078.jpg
painting/megaphone/painting_183_000097.jpg
painting/megaphone/painting_183_000154.jpg
painting/megaphone/painting_183_000157.jpg
painting/microphone/painting_185_000045.jpg
painting/microphone/painting_185_000052.jpg
painting/microphone/painting_185_000088.jpg
painting/microphone/painting_185_000111.jpg
painting/microphone/painting_185_000143.jpg
painting/monkey/painting_187_000036.jpg
painting/monkey/painting_187_000099.jpg
painting/monkey/painting_187_000152.jpg
painting/monkey/painting_187_000181.jpg
painting/monkey/painting_187_000400.jpg
painting/moon/painting_188_000037.jpg
painting/moon/painting_188_000212.jpg
painting/mouse/painting_192_000007.jpg
painting/mouse/painting_192_000428.jpg
painting/mouse/painting_192_000434.jpg
painting/mouse/painting_192_000442.jpg
painting/mouse/painting_192_000444.jpg
painting/mouse/painting_192_000445.jpg
painting/moustache/painting_193_000025.jpg
painting/moustache/painting_193_000035.jpg
painting/moustache/painting_193_000074.jpg
painting/moustache/painting_193_000084.jpg
painting/moustache/painting_193_000085.jpg
painting/moustache/painting_193_000098.jpg
painting/moustache/painting_193_000101.jpg
painting/moustache/painting_193_000143.jpg
painting/moustache/painting_193_000149.jpg
painting/moustache/painting_193_000177.jpg
painting/moustache/painting_193_000186.jpg
painting/moustache/painting_193_000217.jpg
painting/moustache/painting_193_000230.jpg
painting/moustache/painting_193_000279.jpg
painting/moustache/painting_193_000332.jpg
painting/moustache/painting_193_000345.jpg
painting/moustache/painting_193_000371.jpg
painting/moustache/painting_193_000424.jpg
painting/mouth/painting_194_000041.jpg
painting/mug/painting_195_000003.jpg
painting/mug/painting_195_000006.jpg
painting/mug/painting_195_000009.jpg
painting/mug/painting_195_000016.jpg
painting/mug/painting_195_000018.jpg
painting/mug/painting_195_000019.jpg
painting/mug/painting_195_000022.jpg
painting/mug/painting_195_000023.jpg
painting/mug/painting_195_000025.jpg
painting/mug/painting_195_000033.jpg
painting/mug/painting_195_000034.jpg
painting/mug/painting_195_000037.jpg
painting/mug/painting_195_000038.jpg
painting/mug/painting_195_000041.jpg
painting/mug/painting_195_000043.jpg
painting/mug/painting_195_000047.jpg
painting/mug/painting_195_000049.jpg
painting/mug/painting_195_000051.jpg
painting/mug/painting_195_000060.jpg
painting/mug/painting_195_000062.jpg
painting/mug/painting_195_000063.jpg
painting/mug/painting_195_000064.jpg
painting/mug/painting_195_000070.jpg
painting/mug/painting_195_000073.jpg
painting/mug/painting_195_000075.jpg
painting/mug/painting_195_000078.jpg
painting/mug/painting_195_000080.jpg
painting/mug/painting_195_000082.jpg
painting/mug/painting_195_000083.jpg
painting/mug/painting_195_000086.jpg
painting/mug/painting_195_000091.jpg
painting/mug/painting_195_000093.jpg
painting/mug/painting_195_000094.jpg
painting/mug/painting_195_000096.jpg
painting/mug/painting_195_000097.jpg
painting/mug/painting_195_000098.jpg
painting/mug/painting_195_000101.jpg
painting/mug/painting_195_000102.jpg
painting/mug/painting_195_000105.jpg
painting/mug/painting_195_000106.jpg
painting/mug/painting_195_000108.jpg
painting/mug/painting_195_000112.jpg
painting/mug/painting_195_000113.jpg
painting/mug/painting_195_000114.jpg
painting/mug/painting_195_000116.jpg
painting/mug/painting_195_000121.jpg
painting/mug/painting_195_000125.jpg
painting/mug/painting_195_000127.jpg
painting/mug/painting_195_000137.jpg
painting/mug/painting_195_000140.jpg
painting/mug/painting_195_000144.jpg
painting/mug/painting_195_000146.jpg
painting/mug/painting_195_000148.jpg
painting/mug/painting_195_000149.jpg
painting/mug/painting_195_000151.jpg
painting/mug/painting_195_000153.jpg
painting/mug/painting_195_000154.jpg
painting/mug/painting_195_000155.jpg
painting/mug/painting_195_000157.jpg
painting/mug/painting_195_000162.jpg
painting/mug/painting_195_000170.jpg
painting/mug/painting_195_000171.jpg
painting/mug/painting_195_000173.jpg
painting/mug/painting_195_000174.jpg
painting/mug/painting_195_000175.jpg
painting/mug/painting_195_000176.jpg
painting/mug/painting_195_000183.jpg
painting/mug/painting_195_000186.jpg
painting/mug/painting_195_000188.jpg
painting/mug/painting_195_000189.jpg
painting/mug/painting_195_000200.jpg
painting/mug/painting_195_000202.jpg
painting/mug/painting_195_000203.jpg
painting/mug/painting_195_000207.jpg
painting/mug/painting_195_000212.jpg
painting/mug/painting_195_000218.jpg
painting/mug/painting_195_000220.jpg
painting/mug/painting_195_000233.jpg
painting/mug/painting_195_000236.jpg
painting/mug/painting_195_000238.jpg
painting/mug/painting_195_000241.jpg
painting/mug/painting_195_000256.jpg
painting/mug/painting_195_000258.jpg
painting/mug/painting_195_000260.jpg
painting/mug/painting_195_000264.jpg
painting/mug/painting_195_000269.jpg
painting/mug/painting_195_000281.jpg
painting/mug/painting_195_000305.jpg
painting/mug/painting_195_000308.jpg
painting/mug/painting_195_000309.jpg
painting/mug/painting_195_000310.jpg
painting/mug/painting_195_000317.jpg
painting/mug/painting_195_000322.jpg
painting/mug/painting_195_000339.jpg
painting/mug/painting_195_000345.jpg
painting/mug/painting_195_000348.jpg
painting/mug/painting_195_000351.jpg
painting/mug/painting_195_000358.jpg
painting/mug/painting_195_000361.jpg
painting/mug/painting_195_000365.jpg
painting/mug/painting_195_000371.jpg
painting/mug/painting_195_000377.jpg
painting/mug/painting_195_000387.jpg
painting/mug/painting_195_000392.jpg
painting/mug/painting_195_000398.jpg
painting/mug/painting_195_000401.jpg
painting/mug/painting_195_000403.jpg
painting/mug/painting_195_000408.jpg
painting/mug/painting_195_000413.jpg
painting/mug/painting_195_000438.jpg
painting/mug/painting_195_000449.jpg
painting/mug/painting_195_000453.jpg
painting/mug/painting_195_000464.jpg
painting/mug/painting_195_000467.jpg
painting/mug/painting_195_000476.jpg
painting/mug/painting_195_000477.jpg
painting/mug/painting_195_000481.jpg
painting/mug/painting_195_000483.jpg
painting/mug/painting_195_000484.jpg
painting/mug/painting_195_000485.jpg
painting/mug/painting_195_000486.jpg
painting/mug/painting_195_000488.jpg
painting/mug/painting_195_000489.jpg
painting/mug/painting_195_000491.jpg
painting/mug/painting_195_000495.jpg
painting/mug/painting_195_000497.jpg
painting/mushroom/painting_196_000099.jpg
painting/necklace/painting_198_000060.jpg
painting/necklace/painting_198_000092.jpg
painting/nose/painting_199_000314.jpg
painting/nose/painting_199_000376.jpg
painting/nose/painting_199_000439.jpg
painting/nose/painting_199_000444.jpg
painting/ocean/painting_200_000012.jpg
painting/ocean/painting_200_000090.jpg
painting/ocean/painting_200_000151.jpg
painting/ocean/painting_200_000282.jpg
painting/ocean/painting_200_000293.jpg
painting/ocean/painting_200_000314.jpg
painting/octopus/painting_202_000011.jpg
painting/octopus/painting_202_000148.jpg
painting/octopus/painting_202_000227.jpg
painting/owl/painting_205_000285.jpg
painting/paintbrush/painting_206_000101.jpg
painting/paintbrush/painting_206_000135.jpg
painting/paintbrush/painting_206_000170.jpg
painting/paintbrush/painting_206_000172.jpg
painting/paintbrush/painting_206_000200.jpg
painting/paintbrush/painting_206_000284.jpg
painting/paintbrush/painting_206_000295.jpg
painting/paintbrush/painting_206_000318.jpg
painting/paint_can/painting_207_000026.jpg
painting/paint_can/painting_207_000037.jpg
painting/paint_can/painting_207_000055.jpg
painting/paint_can/painting_207_000111.jpg
painting/paint_can/painting_207_000158.jpg
painting/paint_can/painting_207_000170.jpg
painting/paint_can/painting_207_000171.jpg
painting/palm_tree/painting_208_000002.jpg
painting/palm_tree/painting_208_000020.jpg
painting/palm_tree/painting_208_000041.jpg
painting/palm_tree/painting_208_000058.jpg
painting/palm_tree/painting_208_000119.jpg
painting/palm_tree/painting_208_000160.jpg
painting/palm_tree/painting_208_000164.jpg
painting/palm_tree/painting_208_000186.jpg
painting/palm_tree/painting_208_000190.jpg
painting/palm_tree/painting_208_000213.jpg
painting/palm_tree/painting_208_000323.jpg
painting/palm_tree/painting_208_000385.jpg
painting/palm_tree/painting_208_000404.jpg
painting/palm_tree/painting_208_000447.jpg
painting/palm_tree/painting_208_000454.jpg
painting/palm_tree/painting_208_000490.jpg
painting/palm_tree/painting_208_000501.jpg
painting/palm_tree/painting_208_000526.jpg
painting/palm_tree/painting_208_000543.jpg
painting/palm_tree/painting_208_000559.jpg
painting/palm_tree/painting_208_000560.jpg
painting/palm_tree/painting_208_000572.jpg
painting/panda/painting_209_000253.jpg
painting/pants/painting_210_000196.jpg
painting/pants/painting_210_000373.jpg
painting/parachute/painting_212_000012.jpg
painting/parachute/painting_212_000025.jpg
painting/parachute/painting_212_000038.jpg
painting/parachute/painting_212_000074.jpg
painting/parachute/painting_212_000089.jpg
painting/parachute/painting_212_000116.jpg
painting/parachute/painting_212_000119.jpg
painting/parrot/painting_213_000111.jpg
painting/parrot/painting_213_000254.jpg
painting/passport/painting_214_000019.jpg
painting/pear/painting_216_000076.jpg
painting/pear/painting_216_000113.jpg
painting/peas/painting_217_000090.jpg
painting/pencil/painting_218_000018.jpg
painting/pencil/painting_218_000023.jpg
painting/pencil/painting_218_000030.jpg
painting/pencil/painting_218_000055.jpg
painting/penguin/painting_219_000151.jpg
painting/penguin/painting_219_000238.jpg
painting/penguin/painting_219_000425.jpg
painting/penguin/painting_219_000440.jpg
painting/piano/painting_220_000039.jpg
painting/piano/painting_220_000075.jpg
painting/piano/painting_220_000085.jpg
painting/piano/painting_220_000091.jpg
painting/piano/painting_220_000096.jpg
painting/piano/painting_220_000150.jpg
painting/piano/painting_220_000167.jpg
painting/piano/painting_220_000192.jpg
painting/piano/painting_220_000216.jpg
painting/piano/painting_220_000249.jpg
painting/piano/painting_220_000267.jpg
painting/piano/painting_220_000282.jpg
painting/pickup_truck/painting_221_000135.jpg
painting/picture_frame/painting_222_000011.jpg
painting/picture_frame/painting_222_000019.jpg
painting/picture_frame/painting_222_000156.jpg
painting/pig/painting_223_000163.jpg
painting/pillow/painting_224_000035.jpg
painting/pillow/painting_224_000053.jpg
painting/pillow/painting_224_000071.jpg
painting/pillow/painting_224_000090.jpg
painting/pillow/painting_224_000101.jpg
painting/pillow/painting_224_000105.jpg
painting/pillow/painting_224_000120.jpg
painting/pliers/painting_227_000003.jpg
painting/pliers/painting_227_000027.jpg
painting/pliers/painting_227_000179.jpg
painting/pliers/painting_227_000277.jpg
painting/pond/painting_229_000028.jpg
painting/pond/painting_229_000036.jpg
painting/pond/painting_229_000271.jpg
painting/pond/painting_229_000274.jpg
painting/pond/painting_229_000377.jpg
painting/pond/painting_229_000456.jpg
painting/pond/painting_229_000458.jpg
painting/pond/painting_229_000468.jpg
painting/pond/painting_229_000543.jpg
painting/pond/painting_229_000597.jpg
painting/popsicle/painting_231_000024.jpg
painting/postcard/painting_232_000050.jpg
painting/power_outlet/painting_234_000017.jpg
painting/rabbit/painting_236_000121.jpg
painting/raccoon/painting_237_000154.jpg
painting/rain/painting_239_000081.jpg
painting/remote_control/painting_242_000103.jpg
painting/rhinoceros/painting_243_000031.jpg
painting/river/painting_245_000089.jpg
painting/river/painting_245_000187.jpg
painting/river/painting_245_000259.jpg
painting/river/painting_245_000460.jpg
painting/rollerskates/painting_247_000037.jpg
painting/rollerskates/painting_247_000052.jpg
painting/rollerskates/painting_247_000059.jpg
painting/rollerskates/painting_247_000064.jpg
painting/rollerskates/painting_247_000096.jpg
painting/rollerskates/painting_247_000126.jpg
painting/rollerskates/painting_247_000127.jpg
painting/rollerskates/painting_247_000128.jpg
painting/rollerskates/painting_247_000132.jpg
painting/rollerskates/painting_247_000158.jpg
painting/rollerskates/painting_247_000160.jpg
painting/rollerskates/painting_247_000216.jpg
painting/rollerskates/painting_247_000218.jpg
painting/rollerskates/painting_247_000257.jpg
painting/rollerskates/painting_247_000268.jpg
painting/rollerskates/painting_247_000273.jpg
painting/rollerskates/painting_247_000275.jpg
painting/rollerskates/painting_247_000281.jpg
painting/rollerskates/painting_247_000284.jpg
painting/sailboat/painting_248_000144.jpg
painting/sandwich/painting_249_000019.jpg
painting/sandwich/painting_249_000027.jpg
painting/sandwich/painting_249_000041.jpg
painting/sandwich/painting_249_000062.jpg
painting/sandwich/painting_249_000063.jpg
painting/sandwich/painting_249_000093.jpg
painting/sandwich/painting_249_000096.jpg
painting/sandwich/painting_249_000105.jpg
painting/sandwich/painting_249_000125.jpg
painting/sandwich/painting_249_000136.jpg
painting/saw/painting_250_000116.jpg
painting/saxophone/painting_251_000148.jpg
painting/saxophone/painting_251_000290.jpg
painting/saxophone/painting_251_000331.jpg
painting/school_bus/painting_252_000001.jpg
painting/school_bus/painting_252_000009.jpg
painting/school_bus/painting_252_000016.jpg
painting/school_bus/painting_252_000024.jpg
painting/school_bus/painting_252_000048.jpg
painting/school_bus/painting_252_000059.jpg
painting/scissors/painting_253_000011.jpg
painting/scissors/painting_253_000014.jpg
painting/scissors/painting_253_000024.jpg
painting/scorpion/painting_254_000023.jpg
painting/scorpion/painting_254_000033.jpg
painting/scorpion/painting_254_000059.jpg
painting/scorpion/painting_254_000090.jpg
painting/scorpion/painting_254_000112.jpg
painting/screwdriver/painting_255_000009.jpg
painting/screwdriver/painting_255_000030.jpg
painting/screwdriver/painting_255_000044.jpg
painting/screwdriver/painting_255_000046.jpg
painting/screwdriver/painting_255_000058.jpg
painting/screwdriver/painting_255_000067.jpg
painting/sea_turtle/painting_256_000043.jpg
painting/sea_turtle/painting_256_000138.jpg
painting/sea_turtle/painting_256_000257.jpg
painting/sea_turtle/painting_256_000285.jpg
painting/sea_turtle/painting_256_000289.jpg
painting/sea_turtle/painting_256_000369.jpg
painting/sea_turtle/painting_256_000373.jpg
painting/sea_turtle/painting_256_000400.jpg
painting/see_saw/painting_257_000039.jpg
painting/see_saw/painting_257_000041.jpg
painting/see_saw/painting_257_000065.jpg
painting/see_saw/painting_257_000164.jpg
painting/sheep/painting_259_000092.jpg
painting/shorts/painting_261_000077.jpg
painting/shorts/painting_261_000078.jpg
painting/shovel/painting_262_000043.jpg
painting/shovel/painting_262_000101.jpg
painting/skateboard/painting_264_000014.jpg
painting/skateboard/painting_264_000148.jpg
painting/skull/painting_265_000143.jpg
painting/skull/painting_265_000182.jpg
painting/skull/painting_265_000183.jpg
painting/skyscraper/painting_266_000045.jpg
painting/skyscraper/painting_266_000152.jpg
painting/snail/painting_269_000010.jpg
painting/snail/painting_269_000181.jpg
painting/snake/painting_270_000139.jpg
painting/snake/painting_270_000199.jpg
painting/snorkel/painting_271_000030.jpg
painting/snorkel/painting_271_000031.jpg
painting/snorkel/painting_271_000048.jpg
painting/snorkel/painting_271_000049.jpg
painting/snorkel/painting_271_000057.jpg
painting/snorkel/painting_271_000063.jpg
painting/snorkel/painting_271_000065.jpg
painting/snorkel/painting_271_000072.jpg
painting/snorkel/painting_271_000074.jpg
painting/snorkel/painting_271_000075.jpg
painting/snorkel/painting_271_000076.jpg
painting/snorkel/painting_271_000084.jpg
painting/snorkel/painting_271_000085.jpg
painting/snorkel/painting_271_000088.jpg
painting/snorkel/painting_271_000089.jpg
painting/snorkel/painting_271_000090.jpg
painting/snorkel/painting_271_000091.jpg
painting/snorkel/painting_271_000092.jpg
painting/snorkel/painting_271_000093.jpg
painting/snorkel/painting_271_000097.jpg
painting/snorkel/painting_271_000098.jpg
painting/snorkel/painting_271_000101.jpg
painting/snorkel/painting_271_000104.jpg
painting/snorkel/painting_271_000105.jpg
painting/snorkel/painting_271_000108.jpg
painting/snorkel/painting_271_000109.jpg
painting/snorkel/painting_271_000111.jpg
painting/snorkel/painting_271_000116.jpg
painting/snorkel/painting_271_000117.jpg
painting/snorkel/painting_271_000118.jpg
painting/snorkel/painting_271_000120.jpg
painting/snorkel/painting_271_000128.jpg
painting/snorkel/painting_271_000130.jpg
painting/snorkel/painting_271_000132.jpg
painting/snorkel/painting_271_000133.jpg
painting/snorkel/painting_271_000136.jpg
painting/snorkel/painting_271_000137.jpg
painting/snorkel/painting_271_000142.jpg
painting/snorkel/painting_271_000144.jpg
painting/snorkel/painting_271_000145.jpg
painting/snorkel/painting_271_000147.jpg
painting/snorkel/painting_271_000148.jpg
painting/snorkel/painting_271_000150.jpg
painting/snorkel/painting_271_000151.jpg
painting/snorkel/painting_271_000152.jpg
painting/snorkel/painting_271_000153.jpg
painting/snorkel/painting_271_000154.jpg
painting/snorkel/painting_271_000156.jpg
painting/snorkel/painting_271_000157.jpg
painting/snorkel/painting_271_000158.jpg
painting/snorkel/painting_271_000161.jpg
painting/snorkel/painting_271_000162.jpg
painting/snorkel/painting_271_000163.jpg
painting/snorkel/painting_271_000164.jpg
painting/snorkel/painting_271_000165.jpg
painting/snorkel/painting_271_000166.jpg
painting/snorkel/painting_271_000168.jpg
painting/snorkel/painting_271_000170.jpg
painting/snorkel/painting_271_000171.jpg
painting/snorkel/painting_271_000175.jpg
painting/snorkel/painting_271_000176.jpg
painting/snorkel/painting_271_000177.jpg
painting/snorkel/painting_271_000178.jpg
painting/snorkel/painting_271_000179.jpg
painting/snowflake/painting_272_000033.jpg
painting/snowflake/painting_272_000046.jpg
painting/snowflake/painting_272_000047.jpg
painting/snowflake/painting_272_000054.jpg
painting/snowflake/painting_272_000057.jpg
painting/snowflake/painting_272_000059.jpg
painting/snowflake/painting_272_000067.jpg
painting/snowflake/painting_272_000084.jpg
painting/snowflake/painting_272_000090.jpg
painting/snowflake/painting_272_000100.jpg
painting/snowflake/painting_272_000104.jpg
painting/snowflake/painting_272_000109.jpg
painting/snowflake/painting_272_000114.jpg
painting/snowflake/painting_272_000125.jpg
painting/snowflake/painting_272_000131.jpg
painting/snowflake/painting_272_000144.jpg
painting/snowflake/painting_272_000145.jpg
painting/snowflake/painting_272_000147.jpg
painting/snowflake/painting_272_000152.jpg
painting/snowflake/painting_272_000154.jpg
painting/snowflake/painting_272_000155.jpg
painting/snowflake/painting_272_000161.jpg
painting/snowflake/painting_272_000162.jpg
painting/snowflake/painting_272_000169.jpg
painting/snowflake/painting_272_000172.jpg
painting/snowflake/painting_272_000173.jpg
painting/snowflake/painting_272_000179.jpg
painting/snowflake/painting_272_000180.jpg
painting/snowflake/painting_272_000181.jpg
painting/snowflake/painting_272_000182.jpg
painting/snowflake/painting_272_000184.jpg
painting/snowflake/painting_272_000190.jpg
painting/snowflake/painting_272_000197.jpg
painting/snowflake/painting_272_000199.jpg
painting/snowflake/painting_272_000200.jpg
painting/snowflake/painting_272_000204.jpg
painting/snowflake/painting_272_000206.jpg
painting/snowflake/painting_272_000208.jpg
painting/snowflake/painting_272_000212.jpg
painting/snowflake/painting_272_000222.jpg
painting/snowflake/painting_272_000224.jpg
painting/snowflake/painting_272_000227.jpg
painting/snowflake/painting_272_000231.jpg
painting/snowflake/painting_272_000236.jpg
painting/snowflake/painting_272_000239.jpg
painting/snowflake/painting_272_000241.jpg
painting/snowflake/painting_272_000242.jpg
painting/snowflake/painting_272_000244.jpg
painting/snowflake/painting_272_000249.jpg
painting/snowflake/painting_272_000253.jpg
painting/snowflake/painting_272_000256.jpg
painting/snowflake/painting_272_000257.jpg
painting/snowflake/painting_272_000258.jpg
painting/snowflake/painting_272_000264.jpg
painting/snowflake/painting_272_000265.jpg
painting/snowflake/painting_272_000267.jpg
painting/snowflake/painting_272_000268.jpg
painting/snowflake/painting_272_000271.jpg
painting/snowflake/painting_272_000272.jpg
painting/snowflake/painting_272_000273.jpg
painting/snowflake/painting_272_000281.jpg
painting/snowflake/painting_272_000285.jpg
painting/snowflake/painting_272_000287.jpg
painting/snowflake/painting_272_000290.jpg
painting/snowflake/painting_272_000292.jpg
painting/snowflake/painting_272_000294.jpg
painting/snowflake/painting_272_000295.jpg
painting/snowflake/painting_272_000296.jpg
painting/snowflake/painting_272_000297.jpg
painting/snowflake/painting_272_000298.jpg
painting/snowflake/painting_272_000299.jpg
painting/snowflake/painting_272_000301.jpg
painting/snowflake/painting_272_000304.jpg
painting/snowflake/painting_272_000307.jpg
painting/snowflake/painting_272_000308.jpg
painting/snowflake/painting_272_000310.jpg
painting/snowflake/painting_272_000311.jpg
painting/snowflake/painting_272_000312.jpg
painting/snowflake/painting_272_000315.jpg
painting/snowflake/painting_272_000320.jpg
painting/snowflake/painting_272_000324.jpg
painting/snowflake/painting_272_000325.jpg
painting/snowflake/painting_272_000327.jpg
painting/snowflake/painting_272_000330.jpg
painting/snowflake/painting_272_000332.jpg
painting/snowflake/painting_272_000341.jpg
painting/snowflake/painting_272_000342.jpg
painting/snowflake/painting_272_000343.jpg
painting/snowflake/painting_272_000344.jpg
painting/snowflake/painting_272_000345.jpg
painting/snowflake/painting_272_000346.jpg
painting/snowflake/painting_272_000348.jpg
painting/snowflake/painting_272_000349.jpg
painting/snowflake/painting_272_000350.jpg
painting/snowflake/painting_272_000351.jpg
painting/snowflake/painting_272_000352.jpg
painting/snowflake/painting_272_000354.jpg
painting/snowflake/painting_272_000356.jpg
painting/snowflake/painting_272_000357.jpg
painting/snowflake/painting_272_000359.jpg
painting/snowflake/painting_272_000360.jpg
painting/snowflake/painting_272_000362.jpg
painting/snowflake/painting_272_000363.jpg
painting/snowflake/painting_272_000366.jpg
painting/snowflake/painting_272_000369.jpg
painting/snowflake/painting_272_000371.jpg
painting/snowflake/painting_272_000372.jpg
painting/snowflake/painting_272_000373.jpg
painting/snowflake/painting_272_000374.jpg
painting/snowflake/painting_272_000375.jpg
painting/snowflake/painting_272_000377.jpg
painting/snowflake/painting_272_000378.jpg
painting/snowflake/painting_272_000379.jpg
painting/snowflake/painting_272_000380.jpg
painting/snowflake/painting_272_000381.jpg
painting/snowflake/painting_272_000382.jpg
painting/snowflake/painting_272_000383.jpg
painting/snowflake/painting_272_000385.jpg
painting/snowflake/painting_272_000387.jpg
painting/snowflake/painting_272_000388.jpg
painting/snowflake/painting_272_000389.jpg
painting/snowflake/painting_272_000390.jpg
painting/snowflake/painting_272_000391.jpg
painting/snowflake/painting_272_000392.jpg
painting/snowflake/painting_272_000393.jpg
painting/snowflake/painting_272_000396.jpg
painting/snowflake/painting_272_000397.jpg
painting/snowflake/painting_272_000398.jpg
painting/snowflake/painting_272_000399.jpg
painting/snowflake/painting_272_000400.jpg
painting/snowflake/painting_272_000401.jpg
painting/snowflake/painting_272_000402.jpg
painting/snowflake/painting_272_000403.jpg
painting/snowflake/painting_272_000404.jpg
painting/snowflake/painting_272_000405.jpg
painting/snowman/painting_273_000023.jpg
painting/snowman/painting_273_000073.jpg
painting/snowman/painting_273_000090.jpg
painting/snowman/painting_273_000112.jpg
painting/snowman/painting_273_000117.jpg
painting/snowman/painting_273_000131.jpg
painting/snowman/painting_273_000135.jpg
painting/snowman/painting_273_000146.jpg
painting/snowman/painting_273_000147.jpg
painting/snowman/painting_273_000153.jpg
painting/snowman/painting_273_000172.jpg
painting/snowman/painting_273_000178.jpg
painting/snowman/painting_273_000193.jpg
painting/snowman/painting_273_000194.jpg
painting/snowman/painting_273_000195.jpg
painting/snowman/painting_273_000205.jpg
painting/snowman/painting_273_000215.jpg
painting/snowman/painting_273_000223.jpg
painting/snowman/painting_273_000238.jpg
painting/snowman/painting_273_000239.jpg
painting/snowman/painting_273_000241.jpg
painting/snowman/painting_273_000243.jpg
painting/snowman/painting_273_000244.jpg
painting/snowman/painting_273_000256.jpg
painting/snowman/painting_273_000261.jpg
painting/snowman/painting_273_000268.jpg
painting/snowman/painting_273_000275.jpg
painting/snowman/painting_273_000276.jpg
painting/snowman/painting_273_000282.jpg
painting/snowman/painting_273_000285.jpg
painting/snowman/painting_273_000289.jpg
painting/snowman/painting_273_000293.jpg
painting/snowman/painting_273_000294.jpg
painting/snowman/painting_273_000304.jpg
painting/snowman/painting_273_000306.jpg
painting/snowman/painting_273_000313.jpg
painting/snowman/painting_273_000315.jpg
painting/snowman/painting_273_000321.jpg
painting/snowman/painting_273_000335.jpg
painting/snowman/painting_273_000342.jpg
painting/snowman/painting_273_000345.jpg
painting/snowman/painting_273_000346.jpg
painting/snowman/painting_273_000359.jpg
painting/snowman/painting_273_000362.jpg
painting/snowman/painting_273_000368.jpg
painting/snowman/painting_273_000370.jpg
painting/snowman/painting_273_000378.jpg
painting/snowman/painting_273_000382.jpg
painting/snowman/painting_273_000383.jpg
painting/snowman/painting_273_000384.jpg
painting/snowman/painting_273_000388.jpg
painting/snowman/painting_273_000389.jpg
painting/snowman/painting_273_000390.jpg
painting/snowman/painting_273_000391.jpg
painting/snowman/painting_273_000396.jpg
painting/snowman/painting_273_000399.jpg
painting/snowman/painting_273_000400.jpg
painting/snowman/painting_273_000403.jpg
painting/snowman/painting_273_000414.jpg
painting/snowman/painting_273_000420.jpg
painting/snowman/painting_273_000421.jpg
painting/snowman/painting_273_000429.jpg
painting/snowman/painting_273_000430.jpg
painting/snowman/painting_273_000432.jpg
painting/snowman/painting_273_000434.jpg
painting/snowman/painting_273_000439.jpg
painting/snowman/painting_273_000444.jpg
painting/snowman/painting_273_000445.jpg
painting/snowman/painting_273_000447.jpg
painting/snowman/painting_273_000448.jpg
painting/snowman/painting_273_000450.jpg
painting/snowman/painting_273_000452.jpg
painting/snowman/painting_273_000453.jpg
painting/snowman/painting_273_000455.jpg
painting/snowman/painting_273_000456.jpg
painting/snowman/painting_273_000467.jpg
painting/snowman/painting_273_000469.jpg
painting/snowman/painting_273_000476.jpg
painting/snowman/painting_273_000478.jpg
painting/snowman/painting_273_000479.jpg
painting/snowman/painting_273_000480.jpg
painting/snowman/painting_273_000483.jpg
painting/snowman/painting_273_000484.jpg
painting/snowman/painting_273_000485.jpg
painting/snowman/painting_273_000486.jpg
painting/snowman/painting_273_000494.jpg
painting/snowman/painting_273_000495.jpg
painting/snowman/painting_273_000496.jpg
painting/snowman/painting_273_000499.jpg
painting/snowman/painting_273_000501.jpg
painting/snowman/painting_273_000502.jpg
painting/snowman/painting_273_000506.jpg
painting/snowman/painting_273_000510.jpg
painting/snowman/painting_273_000513.jpg
painting/snowman/painting_273_000519.jpg
painting/snowman/painting_273_000520.jpg
painting/snowman/painting_273_000521.jpg
painting/snowman/painting_273_000523.jpg
painting/snowman/painting_273_000527.jpg
painting/snowman/painting_273_000529.jpg
painting/snowman/painting_273_000531.jpg
painting/snowman/painting_273_000533.jpg
painting/snowman/painting_273_000536.jpg
painting/snowman/painting_273_000537.jpg
painting/snowman/painting_273_000541.jpg
painting/snowman/painting_273_000546.jpg
painting/snowman/painting_273_000550.jpg
painting/snowman/painting_273_000557.jpg
painting/snowman/painting_273_000559.jpg
painting/snowman/painting_273_000563.jpg
painting/snowman/painting_273_000564.jpg
painting/snowman/painting_273_000570.jpg
painting/snowman/painting_273_000571.jpg
painting/snowman/painting_273_000572.jpg
painting/snowman/painting_273_000574.jpg
painting/snowman/painting_273_000578.jpg
painting/snowman/painting_273_000579.jpg
painting/snowman/painting_273_000580.jpg
painting/snowman/painting_273_000581.jpg
painting/snowman/painting_273_000585.jpg
painting/snowman/painting_273_000586.jpg
painting/snowman/painting_273_000587.jpg
painting/snowman/painting_273_000598.jpg
painting/snowman/painting_273_000603.jpg
painting/snowman/painting_273_000605.jpg
painting/snowman/painting_273_000606.jpg
painting/snowman/painting_273_000608.jpg
painting/snowman/painting_273_000609.jpg
painting/snowman/painting_273_000610.jpg
painting/snowman/painting_273_000611.jpg
painting/snowman/painting_273_000612.jpg
painting/snowman/painting_273_000615.jpg
painting/snowman/painting_273_000618.jpg
painting/snowman/painting_273_000620.jpg
painting/snowman/painting_273_000624.jpg
painting/snowman/painting_273_000626.jpg
painting/snowman/painting_273_000628.jpg
painting/snowman/painting_273_000629.jpg
painting/snowman/painting_273_000631.jpg
painting/snowman/painting_273_000633.jpg
painting/snowman/painting_273_000634.jpg
painting/snowman/painting_273_000636.jpg
painting/snowman/painting_273_000639.jpg
painting/snowman/painting_273_000643.jpg
painting/snowman/painting_273_000646.jpg
painting/snowman/painting_273_000647.jpg
painting/snowman/painting_273_000650.jpg
painting/snowman/painting_273_000651.jpg
painting/snowman/painting_273_000654.jpg
painting/snowman/painting_273_000656.jpg
painting/snowman/painting_273_000658.jpg
painting/snowman/painting_273_000659.jpg
painting/snowman/painting_273_000660.jpg
painting/snowman/painting_273_000662.jpg
painting/snowman/painting_273_000663.jpg
painting/snowman/painting_273_000664.jpg
painting/snowman/painting_273_000665.jpg
painting/snowman/painting_273_000667.jpg
painting/snowman/painting_273_000668.jpg
painting/snowman/painting_273_000669.jpg
painting/snowman/painting_273_000670.jpg
painting/snowman/painting_273_000671.jpg
painting/snowman/painting_273_000674.jpg
painting/snowman/painting_273_000677.jpg
painting/snowman/painting_273_000679.jpg
painting/snowman/painting_273_000680.jpg
painting/snowman/painting_273_000682.jpg
painting/snowman/painting_273_000686.jpg
painting/snowman/painting_273_000687.jpg
painting/snowman/painting_273_000689.jpg
painting/snowman/painting_273_000690.jpg
painting/snowman/painting_273_000691.jpg
painting/snowman/painting_273_000693.jpg
painting/snowman/painting_273_000695.jpg
painting/snowman/painting_273_000698.jpg
painting/snowman/painting_273_000699.jpg
painting/snowman/painting_273_000700.jpg
painting/snowman/painting_273_000705.jpg
painting/snowman/painting_273_000706.jpg
painting/snowman/painting_273_000707.jpg
painting/snowman/painting_273_000708.jpg
painting/snowman/painting_273_000710.jpg
painting/snowman/painting_273_000711.jpg
painting/snowman/painting_273_000713.jpg
painting/snowman/painting_273_000715.jpg
painting/snowman/painting_273_000716.jpg
painting/snowman/painting_273_000717.jpg
painting/snowman/painting_273_000718.jpg
painting/snowman/painting_273_000720.jpg
painting/snowman/painting_273_000723.jpg
painting/snowman/painting_273_000724.jpg
painting/snowman/painting_273_000725.jpg
painting/snowman/painting_273_000726.jpg
painting/snowman/painting_273_000728.jpg
painting/snowman/painting_273_000733.jpg
painting/snowman/painting_273_000734.jpg
painting/snowman/painting_273_000737.jpg
painting/snowman/painting_273_000739.jpg
painting/snowman/painting_273_000740.jpg
painting/snowman/painting_273_000741.jpg
painting/snowman/painting_273_000742.jpg
painting/snowman/painting_273_000745.jpg
painting/snowman/painting_273_000749.jpg
painting/snowman/painting_273_000753.jpg
painting/snowman/painting_273_000756.jpg
painting/snowman/painting_273_000757.jpg
painting/snowman/painting_273_000762.jpg
painting/snowman/painting_273_000767.jpg
painting/snowman/painting_273_000768.jpg
painting/snowman/painting_273_000770.jpg
painting/snowman/painting_273_000771.jpg
painting/snowman/painting_273_000773.jpg
painting/snowman/painting_273_000775.jpg
painting/snowman/painting_273_000776.jpg
painting/snowman/painting_273_000780.jpg
painting/snowman/painting_273_000782.jpg
painting/snowman/painting_273_000783.jpg
painting/snowman/painting_273_000785.jpg
painting/snowman/painting_273_000787.jpg
painting/snowman/painting_273_000789.jpg
painting/snowman/painting_273_000790.jpg
painting/snowman/painting_273_000791.jpg
painting/snowman/painting_273_000792.jpg
painting/snowman/painting_273_000793.jpg
painting/snowman/painting_273_000795.jpg
painting/snowman/painting_273_000796.jpg
painting/snowman/painting_273_000797.jpg
painting/snowman/painting_273_000799.jpg
painting/snowman/painting_273_000801.jpg
painting/snowman/painting_273_000802.jpg
painting/snowman/painting_273_000807.jpg
painting/snowman/painting_273_000809.jpg
painting/snowman/painting_273_000810.jpg
painting/snowman/painting_273_000813.jpg
painting/snowman/painting_273_000814.jpg
painting/snowman/painting_273_000815.jpg
painting/snowman/painting_273_000818.jpg
painting/snowman/painting_273_000819.jpg
painting/snowman/painting_273_000821.jpg
painting/snowman/painting_273_000822.jpg
painting/snowman/painting_273_000823.jpg
painting/snowman/painting_273_000824.jpg
painting/snowman/painting_273_000826.jpg
painting/snowman/painting_273_000827.jpg
painting/snowman/painting_273_000828.jpg
painting/snowman/painting_273_000830.jpg
painting/snowman/painting_273_000833.jpg
painting/snowman/painting_273_000834.jpg
painting/snowman/painting_273_000835.jpg
painting/snowman/painting_273_000836.jpg
painting/snowman/painting_273_000837.jpg
painting/snowman/painting_273_000838.jpg
painting/snowman/painting_273_000840.jpg
painting/snowman/painting_273_000841.jpg
painting/snowman/painting_273_000842.jpg
painting/snowman/painting_273_000843.jpg
painting/snowman/painting_273_000844.jpg
painting/snowman/painting_273_000845.jpg
painting/snowman/painting_273_000846.jpg
painting/snowman/painting_273_000847.jpg
painting/snowman/painting_273_000848.jpg
painting/snowman/painting_273_000850.jpg
painting/snowman/painting_273_000851.jpg
painting/snowman/painting_273_000852.jpg
painting/snowman/painting_273_000855.jpg
painting/snowman/painting_273_000856.jpg
painting/snowman/painting_273_000860.jpg
painting/snowman/painting_273_000863.jpg
painting/snowman/painting_273_000864.jpg
painting/snowman/painting_273_000865.jpg
painting/snowman/painting_273_000868.jpg
painting/snowman/painting_273_000869.jpg
painting/snowman/painting_273_000871.jpg
painting/snowman/painting_273_000873.jpg
painting/snowman/painting_273_000874.jpg
painting/snowman/painting_273_000875.jpg
painting/snowman/painting_273_000876.jpg
painting/snowman/painting_273_000878.jpg
painting/snowman/painting_273_000879.jpg
painting/snowman/painting_273_000880.jpg
painting/snowman/painting_273_000881.jpg
painting/snowman/painting_273_000882.jpg
painting/snowman/painting_273_000883.jpg
painting/snowman/painting_273_000885.jpg
painting/snowman/painting_273_000886.jpg
painting/snowman/painting_273_000888.jpg
painting/snowman/painting_273_000889.jpg
painting/snowman/painting_273_000890.jpg
painting/snowman/painting_273_000896.jpg
painting/snowman/painting_273_000899.jpg
painting/snowman/painting_273_000900.jpg
painting/snowman/painting_273_000901.jpg
painting/soccer_ball/painting_274_000016.jpg
painting/soccer_ball/painting_274_000031.jpg
painting/soccer_ball/painting_274_000045.jpg
painting/soccer_ball/painting_274_000047.jpg
painting/soccer_ball/painting_274_000061.jpg
painting/soccer_ball/painting_274_000074.jpg
painting/soccer_ball/painting_274_000079.jpg
painting/soccer_ball/painting_274_000087.jpg
painting/soccer_ball/painting_274_000091.jpg
painting/soccer_ball/painting_274_000095.jpg
painting/soccer_ball/painting_274_000100.jpg
painting/soccer_ball/painting_274_000104.jpg
painting/soccer_ball/painting_274_000109.jpg
painting/soccer_ball/painting_274_000123.jpg
painting/soccer_ball/painting_274_000127.jpg
painting/soccer_ball/painting_274_000129.jpg
painting/soccer_ball/painting_274_000132.jpg
painting/soccer_ball/painting_274_000133.jpg
painting/soccer_ball/painting_274_000144.jpg
painting/soccer_ball/painting_274_000155.jpg
painting/soccer_ball/painting_274_000161.jpg
painting/soccer_ball/painting_274_000163.jpg
painting/soccer_ball/painting_274_000172.jpg
painting/soccer_ball/painting_274_000175.jpg
painting/soccer_ball/painting_274_000176.jpg
painting/soccer_ball/painting_274_000186.jpg
painting/soccer_ball/painting_274_000190.jpg
painting/soccer_ball/painting_274_000193.jpg
painting/soccer_ball/painting_274_000195.jpg
painting/soccer_ball/painting_274_000197.jpg
painting/soccer_ball/painting_274_000202.jpg
painting/soccer_ball/painting_274_000204.jpg
painting/soccer_ball/painting_274_000205.jpg
painting/soccer_ball/painting_274_000209.jpg
painting/soccer_ball/painting_274_000215.jpg
painting/soccer_ball/painting_274_000216.jpg
painting/soccer_ball/painting_274_000217.jpg
painting/soccer_ball/painting_274_000218.jpg
painting/soccer_ball/painting_274_000219.jpg
painting/soccer_ball/painting_274_000221.jpg
painting/soccer_ball/painting_274_000223.jpg
painting/soccer_ball/painting_274_000224.jpg
painting/soccer_ball/painting_274_000225.jpg
painting/soccer_ball/painting_274_000227.jpg
painting/soccer_ball/painting_274_000234.jpg
painting/soccer_ball/painting_274_000235.jpg
painting/soccer_ball/painting_274_000242.jpg
painting/soccer_ball/painting_274_000245.jpg
painting/soccer_ball/painting_274_000249.jpg
painting/soccer_ball/painting_274_000250.jpg
painting/soccer_ball/painting_274_000257.jpg
painting/soccer_ball/painting_274_000258.jpg
painting/soccer_ball/painting_274_000259.jpg
painting/soccer_ball/painting_274_000262.jpg
painting/soccer_ball/painting_274_000265.jpg
painting/soccer_ball/painting_274_000267.jpg
painting/soccer_ball/painting_274_000268.jpg
painting/sock/painting_275_000028.jpg
painting/sock/painting_275_000029.jpg
painting/speedboat/painting_276_000025.jpg
painting/speedboat/painting_276_000026.jpg
painting/speedboat/painting_276_000038.jpg
painting/speedboat/painting_276_000039.jpg
painting/speedboat/painting_276_000044.jpg
painting/speedboat/painting_276_000045.jpg
painting/speedboat/painting_276_000048.jpg
painting/speedboat/painting_276_000052.jpg
painting/speedboat/painting_276_000055.jpg
painting/speedboat/painting_276_000056.jpg
painting/speedboat/painting_276_000060.jpg
painting/speedboat/painting_276_000069.jpg
painting/speedboat/painting_276_000076.jpg
painting/speedboat/painting_276_000078.jpg
painting/speedboat/painting_276_000084.jpg
painting/speedboat/painting_276_000088.jpg
painting/speedboat/painting_276_000091.jpg
painting/speedboat/painting_276_000094.jpg
painting/speedboat/painting_276_000099.jpg
painting/speedboat/painting_276_000102.jpg
painting/speedboat/painting_276_000103.jpg
painting/speedboat/painting_276_000105.jpg
painting/speedboat/painting_276_000106.jpg
painting/speedboat/painting_276_000109.jpg
painting/speedboat/painting_276_000110.jpg
painting/speedboat/painting_276_000111.jpg
painting/speedboat/painting_276_000114.jpg
painting/speedboat/painting_276_000115.jpg
painting/speedboat/painting_276_000116.jpg
painting/speedboat/painting_276_000118.jpg
painting/speedboat/painting_276_000120.jpg
painting/speedboat/painting_276_000125.jpg
painting/speedboat/painting_276_000127.jpg
painting/speedboat/painting_276_000130.jpg
painting/speedboat/painting_276_000132.jpg
painting/speedboat/painting_276_000133.jpg
painting/speedboat/painting_276_000134.jpg
painting/speedboat/painting_276_000135.jpg
painting/speedboat/painting_276_000136.jpg
painting/speedboat/painting_276_000137.jpg
painting/speedboat/painting_276_000139.jpg
painting/speedboat/painting_276_000140.jpg
painting/spider/painting_277_000020.jpg
painting/spider/painting_277_000031.jpg
painting/spider/painting_277_000045.jpg
painting/spider/painting_277_000049.jpg
painting/spider/painting_277_000065.jpg
painting/spider/painting_277_000069.jpg
painting/spider/painting_277_000075.jpg
painting/spider/painting_277_000091.jpg
painting/spider/painting_277_000102.jpg
painting/spider/painting_277_000105.jpg
painting/spider/painting_277_000107.jpg
painting/spider/painting_277_000113.jpg
painting/spider/painting_277_000115.jpg
painting/spider/painting_277_000116.jpg
painting/spider/painting_277_000117.jpg
painting/spider/painting_277_000119.jpg
painting/spider/painting_277_000120.jpg
painting/spider/painting_277_000128.jpg
painting/spider/painting_277_000130.jpg
painting/spider/painting_277_000134.jpg
painting/spider/painting_277_000135.jpg
painting/spider/painting_277_000137.jpg
painting/spider/painting_277_000140.jpg
painting/spider/painting_277_000143.jpg
painting/spider/painting_277_000146.jpg
painting/spider/painting_277_000155.jpg
painting/spider/painting_277_000156.jpg
painting/spider/painting_277_000163.jpg
painting/spider/painting_277_000164.jpg
painting/spider/painting_277_000166.jpg
painting/spider/painting_277_000172.jpg
painting/spider/painting_277_000174.jpg
painting/spider/painting_277_000177.jpg
painting/spider/painting_277_000179.jpg
painting/spider/painting_277_000180.jpg
painting/spider/painting_277_000185.jpg
painting/spider/painting_277_000186.jpg
painting/spider/painting_277_000190.jpg
painting/spider/painting_277_000191.jpg
painting/spider/painting_277_000192.jpg
painting/spider/painting_277_000193.jpg
painting/spider/painting_277_000194.jpg
painting/spider/painting_277_000197.jpg
painting/spider/painting_277_000198.jpg
painting/spider/painting_277_000199.jpg
painting/spider/painting_277_000202.jpg
painting/spider/painting_277_000203.jpg
painting/spider/painting_277_000205.jpg
painting/spider/painting_277_000206.jpg
painting/spider/painting_277_000207.jpg
painting/spider/painting_277_000208.jpg
painting/spider/painting_277_000210.jpg
painting/spider/painting_277_000211.jpg
painting/spider/painting_277_000212.jpg
painting/spider/painting_277_000213.jpg
painting/spider/painting_277_000215.jpg
painting/spider/painting_277_000218.jpg
painting/spider/painting_277_000219.jpg
painting/spider/painting_277_000221.jpg
painting/spider/painting_277_000223.jpg
painting/spider/painting_277_000230.jpg
painting/spider/painting_277_000232.jpg
painting/spider/painting_277_000234.jpg
painting/spider/painting_277_000235.jpg
painting/spider/painting_277_000236.jpg
painting/spider/painting_277_000237.jpg
painting/spider/painting_277_000239.jpg
painting/spider/painting_277_000240.jpg
painting/spider/painting_277_000241.jpg
painting/spider/painting_277_000242.jpg
painting/spider/painting_277_000245.jpg
painting/spider/painting_277_000249.jpg
painting/spider/painting_277_000250.jpg
painting/spider/painting_277_000251.jpg
painting/spider/painting_277_000261.jpg
painting/spider/painting_277_000263.jpg
painting/spider/painting_277_000265.jpg
painting/spider/painting_277_000267.jpg
painting/spider/painting_277_000272.jpg
painting/spider/painting_277_000273.jpg
painting/spider/painting_277_000275.jpg
painting/spider/painting_277_000277.jpg
painting/spider/painting_277_000278.jpg
painting/spider/painting_277_000279.jpg
painting/spider/painting_277_000282.jpg
painting/spider/painting_277_000283.jpg
painting/spider/painting_277_000284.jpg
painting/spider/painting_277_000286.jpg
painting/spider/painting_277_000287.jpg
painting/spider/painting_277_000288.jpg
painting/spider/painting_277_000290.jpg
painting/spider/painting_277_000292.jpg
painting/spider/painting_277_000293.jpg
painting/spider/painting_277_000294.jpg
painting/spider/painting_277_000295.jpg
painting/spider/painting_277_000296.jpg
painting/spider/painting_277_000297.jpg
painting/spider/painting_277_000298.jpg
painting/spider/painting_277_000299.jpg
painting/spider/painting_277_000300.jpg
painting/spider/painting_277_000302.jpg
painting/spider/painting_277_000303.jpg
painting/spider/painting_277_000304.jpg
painting/spider/painting_277_000305.jpg
painting/spider/painting_277_000306.jpg
painting/spider/painting_277_000307.jpg
painting/spoon/painting_278_000001.jpg
painting/spoon/painting_278_000016.jpg
painting/spoon/painting_278_000024.jpg
painting/spoon/painting_278_000026.jpg
painting/spoon/painting_278_000040.jpg
painting/spoon/painting_278_000045.jpg
painting/spoon/painting_278_000046.jpg
painting/spoon/painting_278_000050.jpg
painting/spoon/painting_278_000052.jpg
painting/spoon/painting_278_000055.jpg
painting/spoon/painting_278_000057.jpg
painting/spoon/painting_278_000064.jpg
painting/spoon/painting_278_000074.jpg
painting/spoon/painting_278_000076.jpg
painting/spoon/painting_278_000081.jpg
painting/spoon/painting_278_000083.jpg
painting/spoon/painting_278_000085.jpg
painting/spoon/painting_278_000089.jpg
painting/spoon/painting_278_000091.jpg
painting/spoon/painting_278_000093.jpg
painting/spoon/painting_278_000095.jpg
painting/spoon/painting_278_000096.jpg
painting/spoon/painting_278_000099.jpg
painting/spoon/painting_278_000101.jpg
painting/spoon/painting_278_000106.jpg
painting/spoon/painting_278_000107.jpg
painting/spoon/painting_278_000109.jpg
painting/spoon/painting_278_000110.jpg
painting/spoon/painting_278_000114.jpg
painting/spoon/painting_278_000115.jpg
painting/spoon/painting_278_000120.jpg
painting/spoon/painting_278_000121.jpg
painting/spoon/painting_278_000123.jpg
painting/spoon/painting_278_000125.jpg
painting/spoon/painting_278_000126.jpg
painting/spoon/painting_278_000127.jpg
painting/spoon/painting_278_000128.jpg
painting/spoon/painting_278_000130.jpg
painting/spoon/painting_278_000138.jpg
painting/spoon/painting_278_000139.jpg
painting/spoon/painting_278_000142.jpg
painting/spoon/painting_278_000143.jpg
painting/spoon/painting_278_000144.jpg
painting/spoon/painting_278_000145.jpg
painting/spoon/painting_278_000146.jpg
painting/spoon/painting_278_000147.jpg
painting/spoon/painting_278_000149.jpg
painting/spoon/painting_278_000151.jpg
painting/spoon/painting_278_000152.jpg
painting/spoon/painting_278_000153.jpg
painting/spoon/painting_278_000154.jpg
painting/spoon/painting_278_000157.jpg
painting/spreadsheet/painting_279_000003.jpg
painting/square/painting_280_000026.jpg
painting/square/painting_280_000034.jpg
painting/square/painting_280_000046.jpg
painting/square/painting_280_000053.jpg
painting/square/painting_280_000055.jpg
painting/square/painting_280_000056.jpg
painting/square/painting_280_000070.jpg
painting/square/painting_280_000071.jpg
painting/square/painting_280_000072.jpg
painting/square/painting_280_000073.jpg
painting/square/painting_280_000074.jpg
painting/square/painting_280_000076.jpg
painting/square/painting_280_000077.jpg
painting/square/painting_280_000078.jpg
painting/square/painting_280_000080.jpg
painting/square/painting_280_000082.jpg
painting/square/painting_280_000086.jpg
painting/square/painting_280_000091.jpg
painting/square/painting_280_000096.jpg
painting/square/painting_280_000101.jpg
painting/square/painting_280_000104.jpg
painting/square/painting_280_000108.jpg
painting/square/painting_280_000112.jpg
painting/square/painting_280_000115.jpg
painting/square/painting_280_000116.jpg
painting/square/painting_280_000120.jpg
painting/square/painting_280_000121.jpg
painting/square/painting_280_000124.jpg
painting/square/painting_280_000128.jpg
painting/square/painting_280_000135.jpg
painting/square/painting_280_000138.jpg
painting/square/painting_280_000139.jpg
painting/square/painting_280_000142.jpg
painting/square/painting_280_000143.jpg
painting/square/painting_280_000144.jpg
painting/squiggle/painting_281_000021.jpg
painting/squiggle/painting_281_000024.jpg
painting/squiggle/painting_281_000027.jpg
painting/squiggle/painting_281_000049.jpg
painting/squiggle/painting_281_000079.jpg
painting/squiggle/painting_281_000084.jpg
painting/squiggle/painting_281_000088.jpg
painting/squiggle/painting_281_000097.jpg
painting/squiggle/painting_281_000104.jpg
painting/squiggle/painting_281_000106.jpg
painting/squiggle/painting_281_000108.jpg
painting/squiggle/painting_281_000116.jpg
painting/squiggle/painting_281_000121.jpg
painting/squiggle/painting_281_000123.jpg
painting/squiggle/painting_281_000131.jpg
painting/squiggle/painting_281_000144.jpg
painting/squiggle/painting_281_000150.jpg
painting/squiggle/painting_281_000152.jpg
painting/squiggle/painting_281_000153.jpg
painting/squiggle/painting_281_000166.jpg
painting/squiggle/painting_281_000168.jpg
painting/squiggle/painting_281_000173.jpg
painting/squiggle/painting_281_000175.jpg
painting/squiggle/painting_281_000178.jpg
painting/squiggle/painting_281_000181.jpg
painting/squiggle/painting_281_000182.jpg
painting/squiggle/painting_281_000184.jpg
painting/squiggle/painting_281_000191.jpg
painting/squiggle/painting_281_000200.jpg
painting/squiggle/painting_281_000202.jpg
painting/squiggle/painting_281_000206.jpg
painting/squiggle/painting_281_000219.jpg
painting/squiggle/painting_281_000221.jpg
painting/squiggle/painting_281_000223.jpg
painting/squiggle/painting_281_000227.jpg
painting/squiggle/painting_281_000237.jpg
painting/squiggle/painting_281_000246.jpg
painting/squiggle/painting_281_000249.jpg
painting/squiggle/painting_281_000250.jpg
painting/squiggle/painting_281_000253.jpg
painting/squiggle/painting_281_000260.jpg
painting/squiggle/painting_281_000262.jpg
painting/squiggle/painting_281_000263.jpg
painting/squiggle/painting_281_000265.jpg
painting/squiggle/painting_281_000268.jpg
painting/squiggle/painting_281_000269.jpg
painting/squiggle/painting_281_000270.jpg
painting/squiggle/painting_281_000271.jpg
painting/squiggle/painting_281_000273.jpg
painting/squiggle/painting_281_000275.jpg
painting/squiggle/painting_281_000277.jpg
painting/squiggle/painting_281_000278.jpg
painting/squiggle/painting_281_000284.jpg
painting/squiggle/painting_281_000287.jpg
painting/squiggle/painting_281_000295.jpg
painting/squiggle/painting_281_000300.jpg
painting/squiggle/painting_281_000305.jpg
painting/squiggle/painting_281_000306.jpg
painting/squiggle/painting_281_000308.jpg
painting/squiggle/painting_281_000309.jpg
painting/squiggle/painting_281_000311.jpg
painting/squiggle/painting_281_000313.jpg
painting/squiggle/painting_281_000315.jpg
painting/squiggle/painting_281_000316.jpg
painting/squiggle/painting_281_000317.jpg
painting/squiggle/painting_281_000324.jpg
painting/squiggle/painting_281_000328.jpg
painting/squiggle/painting_281_000329.jpg
painting/squiggle/painting_281_000338.jpg
painting/squiggle/painting_281_000341.jpg
painting/squiggle/painting_281_000345.jpg
painting/squiggle/painting_281_000346.jpg
painting/squiggle/painting_281_000348.jpg
painting/squiggle/painting_281_000351.jpg
painting/squiggle/painting_281_000360.jpg
painting/squiggle/painting_281_000361.jpg
painting/squiggle/painting_281_000362.jpg
painting/squiggle/painting_281_000368.jpg
painting/squiggle/painting_281_000369.jpg
painting/squiggle/painting_281_000370.jpg
painting/squiggle/painting_281_000371.jpg
painting/squiggle/painting_281_000372.jpg
painting/squiggle/painting_281_000375.jpg
painting/squiggle/painting_281_000379.jpg
painting/squiggle/painting_281_000382.jpg
painting/squiggle/painting_281_000384.jpg
painting/squiggle/painting_281_000386.jpg
painting/squiggle/painting_281_000388.jpg
painting/squiggle/painting_281_000390.jpg
painting/squiggle/painting_281_000391.jpg
painting/squiggle/painting_281_000392.jpg
painting/squiggle/painting_281_000393.jpg
painting/squiggle/painting_281_000395.jpg
painting/squiggle/painting_281_000403.jpg
painting/squiggle/painting_281_000414.jpg
painting/squiggle/painting_281_000416.jpg
painting/squiggle/painting_281_000417.jpg
painting/squiggle/painting_281_000419.jpg
painting/squiggle/painting_281_000422.jpg
painting/squiggle/painting_281_000426.jpg
painting/squiggle/painting_281_000430.jpg
painting/squiggle/painting_281_000431.jpg
painting/squiggle/painting_281_000434.jpg
painting/squiggle/painting_281_000435.jpg
painting/squiggle/painting_281_000439.jpg
painting/squiggle/painting_281_000443.jpg
painting/squiggle/painting_281_000444.jpg
painting/squiggle/painting_281_000447.jpg
painting/squiggle/painting_281_000448.jpg
painting/squiggle/painting_281_000460.jpg
painting/squiggle/painting_281_000462.jpg
painting/squiggle/painting_281_000463.jpg
painting/squiggle/painting_281_000468.jpg
painting/squiggle/painting_281_000469.jpg
painting/squiggle/painting_281_000470.jpg
painting/squiggle/painting_281_000472.jpg
painting/squiggle/painting_281_000474.jpg
painting/squiggle/painting_281_000480.jpg
painting/squiggle/painting_281_000481.jpg
painting/squiggle/painting_281_000484.jpg
painting/squiggle/painting_281_000485.jpg
painting/squiggle/painting_281_000488.jpg
painting/squiggle/painting_281_000491.jpg
painting/squiggle/painting_281_000494.jpg
painting/squiggle/painting_281_000495.jpg
painting/squiggle/painting_281_000497.jpg
painting/squiggle/painting_281_000498.jpg
painting/squiggle/painting_281_000500.jpg
painting/squiggle/painting_281_000501.jpg
painting/squiggle/painting_281_000502.jpg
painting/squiggle/painting_281_000505.jpg
painting/squiggle/painting_281_000509.jpg
painting/squiggle/painting_281_000510.jpg
painting/squiggle/painting_281_000511.jpg
painting/squiggle/painting_281_000512.jpg
painting/squiggle/painting_281_000514.jpg
painting/squiggle/painting_281_000516.jpg
painting/squiggle/painting_281_000518.jpg
painting/squiggle/painting_281_000522.jpg
painting/squiggle/painting_281_000525.jpg
painting/squiggle/painting_281_000526.jpg
painting/squiggle/painting_281_000527.jpg
painting/squiggle/painting_281_000529.jpg
painting/squiggle/painting_281_000530.jpg
painting/squiggle/painting_281_000531.jpg
painting/squiggle/painting_281_000533.jpg
painting/squiggle/painting_281_000534.jpg
painting/squiggle/painting_281_000536.jpg
painting/squiggle/painting_281_000538.jpg
painting/squiggle/painting_281_000541.jpg
painting/squiggle/painting_281_000542.jpg
painting/squiggle/painting_281_000543.jpg
painting/squiggle/painting_281_000549.jpg
painting/squiggle/painting_281_000551.jpg
painting/squiggle/painting_281_000552.jpg
painting/squiggle/painting_281_000555.jpg
painting/squiggle/painting_281_000558.jpg
painting/squiggle/painting_281_000559.jpg
painting/squiggle/painting_281_000561.jpg
painting/squiggle/painting_281_000562.jpg
painting/squiggle/painting_281_000563.jpg
painting/squiggle/painting_281_000566.jpg
painting/squiggle/painting_281_000569.jpg
painting/squiggle/painting_281_000571.jpg
painting/squiggle/painting_281_000572.jpg
painting/squiggle/painting_281_000573.jpg
painting/squiggle/painting_281_000576.jpg
painting/squiggle/painting_281_000578.jpg
painting/squiggle/painting_281_000581.jpg
painting/squiggle/painting_281_000582.jpg
painting/squiggle/painting_281_000583.jpg
painting/squiggle/painting_281_000584.jpg
painting/squiggle/painting_281_000585.jpg
painting/squiggle/painting_281_000586.jpg
painting/squiggle/painting_281_000590.jpg
painting/squiggle/painting_281_000591.jpg
painting/squiggle/painting_281_000594.jpg
painting/squiggle/painting_281_000596.jpg
painting/squiggle/painting_281_000598.jpg
painting/squiggle/painting_281_000601.jpg
painting/squiggle/painting_281_000603.jpg
painting/squiggle/painting_281_000604.jpg
painting/squiggle/painting_281_000606.jpg
painting/squiggle/painting_281_000608.jpg
painting/squiggle/painting_281_000611.jpg
painting/squiggle/painting_281_000612.jpg
painting/squiggle/painting_281_000613.jpg
painting/squiggle/painting_281_000614.jpg
painting/squiggle/painting_281_000616.jpg
painting/squiggle/painting_281_000617.jpg
painting/squiggle/painting_281_000618.jpg
painting/squiggle/painting_281_000619.jpg
painting/squiggle/painting_281_000625.jpg
painting/squiggle/painting_281_000630.jpg
painting/squiggle/painting_281_000631.jpg
painting/squiggle/painting_281_000632.jpg
painting/squiggle/painting_281_000633.jpg
painting/squiggle/painting_281_000635.jpg
painting/squiggle/painting_281_000636.jpg
painting/squiggle/painting_281_000637.jpg
painting/squiggle/painting_281_000638.jpg
painting/squiggle/painting_281_000640.jpg
painting/squiggle/painting_281_000644.jpg
painting/squiggle/painting_281_000645.jpg
painting/squiggle/painting_281_000648.jpg
painting/squiggle/painting_281_000649.jpg
painting/squiggle/painting_281_000650.jpg
painting/squiggle/painting_281_000651.jpg
painting/squiggle/painting_281_000654.jpg
painting/squiggle/painting_281_000655.jpg
painting/squiggle/painting_281_000657.jpg
painting/squiggle/painting_281_000658.jpg
painting/squiggle/painting_281_000659.jpg
painting/squiggle/painting_281_000660.jpg
painting/squiggle/painting_281_000661.jpg
painting/squiggle/painting_281_000663.jpg
painting/squiggle/painting_281_000665.jpg
painting/squiggle/painting_281_000666.jpg
painting/squiggle/painting_281_000668.jpg
painting/squiggle/painting_281_000670.jpg
painting/squiggle/painting_281_000671.jpg
painting/squiggle/painting_281_000673.jpg
painting/squiggle/painting_281_000674.jpg
painting/squirrel/painting_282_000026.jpg
painting/squirrel/painting_282_000059.jpg
painting/squirrel/painting_282_000079.jpg
painting/squirrel/painting_282_000082.jpg
painting/squirrel/painting_282_000090.jpg
painting/squirrel/painting_282_000097.jpg
painting/squirrel/painting_282_000108.jpg
painting/squirrel/painting_282_000122.jpg
painting/squirrel/painting_282_000123.jpg
painting/squirrel/painting_282_000127.jpg
painting/squirrel/painting_282_000131.jpg
painting/squirrel/painting_282_000136.jpg
painting/squirrel/painting_282_000143.jpg
painting/squirrel/painting_282_000146.jpg
painting/squirrel/painting_282_000163.jpg
painting/squirrel/painting_282_000176.jpg
painting/squirrel/painting_282_000180.jpg
painting/squirrel/painting_282_000183.jpg
painting/squirrel/painting_282_000185.jpg
painting/squirrel/painting_282_000188.jpg
painting/squirrel/painting_282_000214.jpg
painting/squirrel/painting_282_000218.jpg
painting/squirrel/painting_282_000223.jpg
painting/squirrel/painting_282_000231.jpg
painting/squirrel/painting_282_000232.jpg
painting/squirrel/painting_282_000241.jpg
painting/squirrel/painting_282_000248.jpg
painting/squirrel/painting_282_000259.jpg
painting/squirrel/painting_282_000260.jpg
painting/squirrel/painting_282_000262.jpg
painting/squirrel/painting_282_000264.jpg
painting/squirrel/painting_282_000266.jpg
painting/squirrel/painting_282_000270.jpg
painting/squirrel/painting_282_000280.jpg
painting/squirrel/painting_282_000281.jpg
painting/squirrel/painting_282_000294.jpg
painting/squirrel/painting_282_000295.jpg
painting/squirrel/painting_282_000296.jpg
painting/squirrel/painting_282_000298.jpg
painting/squirrel/painting_282_000300.jpg
painting/squirrel/painting_282_000304.jpg
painting/squirrel/painting_282_000306.jpg
painting/squirrel/painting_282_000308.jpg
painting/squirrel/painting_282_000310.jpg
painting/squirrel/painting_282_000316.jpg
painting/squirrel/painting_282_000317.jpg
painting/squirrel/painting_282_000318.jpg
painting/squirrel/painting_282_000323.jpg
painting/squirrel/painting_282_000328.jpg
painting/squirrel/painting_282_000335.jpg
painting/squirrel/painting_282_000337.jpg
painting/squirrel/painting_282_000339.jpg
painting/squirrel/painting_282_000340.jpg
painting/squirrel/painting_282_000345.jpg
painting/squirrel/painting_282_000351.jpg
painting/squirrel/painting_282_000352.jpg
painting/squirrel/painting_282_000353.jpg
painting/squirrel/painting_282_000354.jpg
painting/squirrel/painting_282_000355.jpg
painting/squirrel/painting_282_000358.jpg
painting/squirrel/painting_282_000363.jpg
painting/squirrel/painting_282_000374.jpg
painting/squirrel/painting_282_000377.jpg
painting/squirrel/painting_282_000383.jpg
painting/squirrel/painting_282_000388.jpg
painting/squirrel/painting_282_000389.jpg
painting/squirrel/painting_282_000395.jpg
painting/squirrel/painting_282_000396.jpg
painting/squirrel/painting_282_000397.jpg
painting/squirrel/painting_282_000399.jpg
painting/squirrel/painting_282_000404.jpg
painting/squirrel/painting_282_000410.jpg
painting/squirrel/painting_282_000411.jpg
painting/squirrel/painting_282_000414.jpg
painting/squirrel/painting_282_000421.jpg
painting/squirrel/painting_282_000422.jpg
painting/squirrel/painting_282_000423.jpg
painting/squirrel/painting_282_000426.jpg
painting/squirrel/painting_282_000427.jpg
painting/squirrel/painting_282_000429.jpg
painting/squirrel/painting_282_000431.jpg
painting/squirrel/painting_282_000432.jpg
painting/squirrel/painting_282_000434.jpg
painting/squirrel/painting_282_000435.jpg
painting/squirrel/painting_282_000436.jpg
painting/squirrel/painting_282_000443.jpg
painting/squirrel/painting_282_000444.jpg
painting/squirrel/painting_282_000446.jpg
painting/squirrel/painting_282_000447.jpg
painting/squirrel/painting_282_000456.jpg
painting/squirrel/painting_282_000458.jpg
painting/squirrel/painting_282_000462.jpg
painting/squirrel/painting_282_000463.jpg
painting/squirrel/painting_282_000466.jpg
painting/squirrel/painting_282_000478.jpg
painting/squirrel/painting_282_000480.jpg
painting/squirrel/painting_282_000482.jpg
painting/squirrel/painting_282_000483.jpg
painting/squirrel/painting_282_000486.jpg
painting/squirrel/painting_282_000488.jpg
painting/squirrel/painting_282_000494.jpg
painting/squirrel/painting_282_000495.jpg
painting/squirrel/painting_282_000496.jpg
painting/squirrel/painting_282_000497.jpg
painting/squirrel/painting_282_000502.jpg
painting/squirrel/painting_282_000503.jpg
painting/squirrel/painting_282_000506.jpg
painting/squirrel/painting_282_000507.jpg
painting/squirrel/painting_282_000508.jpg
painting/squirrel/painting_282_000510.jpg
painting/squirrel/painting_282_000511.jpg
painting/squirrel/painting_282_000513.jpg
painting/squirrel/painting_282_000514.jpg
painting/squirrel/painting_282_000517.jpg
painting/squirrel/painting_282_000519.jpg
painting/squirrel/painting_282_000524.jpg
painting/squirrel/painting_282_000525.jpg
painting/squirrel/painting_282_000526.jpg
painting/squirrel/painting_282_000529.jpg
painting/squirrel/painting_282_000531.jpg
painting/squirrel/painting_282_000534.jpg
painting/squirrel/painting_282_000536.jpg
painting/squirrel/painting_282_000538.jpg
painting/squirrel/painting_282_000539.jpg
painting/squirrel/painting_282_000542.jpg
painting/squirrel/painting_282_000543.jpg
painting/squirrel/painting_282_000544.jpg
painting/squirrel/painting_282_000548.jpg
painting/squirrel/painting_282_000550.jpg
painting/squirrel/painting_282_000552.jpg
painting/squirrel/painting_282_000555.jpg
painting/squirrel/painting_282_000557.jpg
painting/squirrel/painting_282_000558.jpg
painting/squirrel/painting_282_000561.jpg
painting/squirrel/painting_282_000564.jpg
painting/squirrel/painting_282_000566.jpg
painting/squirrel/painting_282_000567.jpg
painting/squirrel/painting_282_000568.jpg
painting/squirrel/painting_282_000569.jpg
painting/squirrel/painting_282_000572.jpg
painting/squirrel/painting_282_000573.jpg
painting/squirrel/painting_282_000577.jpg
painting/squirrel/painting_282_000581.jpg
painting/squirrel/painting_282_000582.jpg
painting/squirrel/painting_282_000583.jpg
painting/squirrel/painting_282_000584.jpg
painting/squirrel/painting_282_000586.jpg
painting/squirrel/painting_282_000587.jpg
painting/squirrel/painting_282_000588.jpg
painting/squirrel/painting_282_000589.jpg
painting/squirrel/painting_282_000590.jpg
painting/squirrel/painting_282_000591.jpg
painting/squirrel/painting_282_000592.jpg
painting/squirrel/painting_282_000597.jpg
painting/squirrel/painting_282_000598.jpg
painting/squirrel/painting_282_000599.jpg
painting/squirrel/painting_282_000607.jpg
painting/squirrel/painting_282_000609.jpg
painting/squirrel/painting_282_000615.jpg
painting/squirrel/painting_282_000618.jpg
painting/squirrel/painting_282_000622.jpg
painting/squirrel/painting_282_000625.jpg
painting/squirrel/painting_282_000626.jpg
painting/squirrel/painting_282_000627.jpg
painting/squirrel/painting_282_000629.jpg
painting/squirrel/painting_282_000631.jpg
painting/squirrel/painting_282_000632.jpg
painting/squirrel/painting_282_000634.jpg
painting/squirrel/painting_282_000636.jpg
painting/squirrel/painting_282_000639.jpg
painting/squirrel/painting_282_000640.jpg
painting/squirrel/painting_282_000643.jpg
painting/squirrel/painting_282_000644.jpg
painting/squirrel/painting_282_000646.jpg
painting/squirrel/painting_282_000647.jpg
painting/squirrel/painting_282_000648.jpg
painting/squirrel/painting_282_000649.jpg
painting/squirrel/painting_282_000650.jpg
painting/squirrel/painting_282_000651.jpg
painting/squirrel/painting_282_000653.jpg
painting/squirrel/painting_282_000656.jpg
painting/squirrel/painting_282_000657.jpg
painting/squirrel/painting_282_000658.jpg
painting/squirrel/painting_282_000659.jpg
painting/squirrel/painting_282_000660.jpg
painting/squirrel/painting_282_000664.jpg
painting/squirrel/painting_282_000665.jpg
painting/squirrel/painting_282_000666.jpg
painting/squirrel/painting_282_000668.jpg
painting/squirrel/painting_282_000674.jpg
painting/squirrel/painting_282_000676.jpg
painting/squirrel/painting_282_000677.jpg
painting/squirrel/painting_282_000679.jpg
painting/squirrel/painting_282_000681.jpg
painting/squirrel/painting_282_000682.jpg
painting/squirrel/painting_282_000684.jpg
painting/squirrel/painting_282_000685.jpg
painting/squirrel/painting_282_000686.jpg
painting/squirrel/painting_282_000691.jpg
painting/squirrel/painting_282_000693.jpg
painting/squirrel/painting_282_000694.jpg
painting/squirrel/painting_282_000695.jpg
painting/squirrel/painting_282_000696.jpg
painting/squirrel/painting_282_000698.jpg
painting/squirrel/painting_282_000701.jpg
painting/squirrel/painting_282_000703.jpg
painting/squirrel/painting_282_000704.jpg
painting/squirrel/painting_282_000705.jpg
painting/squirrel/painting_282_000706.jpg
painting/squirrel/painting_282_000707.jpg
painting/squirrel/painting_282_000708.jpg
painting/squirrel/painting_282_000709.jpg
painting/squirrel/painting_282_000715.jpg
painting/squirrel/painting_282_000716.jpg
painting/squirrel/painting_282_000718.jpg
painting/squirrel/painting_282_000720.jpg
painting/squirrel/painting_282_000724.jpg
painting/squirrel/painting_282_000726.jpg
painting/squirrel/painting_282_000727.jpg
painting/squirrel/painting_282_000729.jpg
painting/squirrel/painting_282_000730.jpg
painting/squirrel/painting_282_000731.jpg
painting/squirrel/painting_282_000733.jpg
painting/squirrel/painting_282_000734.jpg
painting/squirrel/painting_282_000745.jpg
painting/squirrel/painting_282_000746.jpg
painting/squirrel/painting_282_000753.jpg
painting/squirrel/painting_282_000755.jpg
painting/squirrel/painting_282_000756.jpg
painting/squirrel/painting_282_000757.jpg
painting/squirrel/painting_282_000761.jpg
painting/squirrel/painting_282_000762.jpg
painting/squirrel/painting_282_000763.jpg
painting/squirrel/painting_282_000764.jpg
painting/squirrel/painting_282_000765.jpg
painting/squirrel/painting_282_000766.jpg
painting/squirrel/painting_282_000767.jpg
painting/squirrel/painting_282_000770.jpg
painting/squirrel/painting_282_000771.jpg
painting/squirrel/painting_282_000773.jpg
painting/squirrel/painting_282_000774.jpg
painting/squirrel/painting_282_000775.jpg
painting/squirrel/painting_282_000777.jpg
painting/squirrel/painting_282_000778.jpg
painting/squirrel/painting_282_000779.jpg
painting/stairs/painting_283_000003.jpg
painting/stairs/painting_283_000012.jpg
painting/stairs/painting_283_000017.jpg
painting/stairs/painting_283_000019.jpg
painting/stairs/painting_283_000022.jpg
painting/stairs/painting_283_000023.jpg
painting/stairs/painting_283_000024.jpg
painting/stairs/painting_283_000026.jpg
painting/stairs/painting_283_000027.jpg
painting/star/painting_284_000013.jpg
painting/star/painting_284_000014.jpg
painting/star/painting_284_000017.jpg
painting/star/painting_284_000030.jpg
painting/star/painting_284_000037.jpg
painting/star/painting_284_000041.jpg
painting/star/painting_284_000042.jpg
painting/star/painting_284_000043.jpg
painting/star/painting_284_000046.jpg
painting/star/painting_284_000048.jpg
painting/star/painting_284_000052.jpg
painting/star/painting_284_000054.jpg
painting/star/painting_284_000057.jpg
painting/star/painting_284_000059.jpg
painting/star/painting_284_000060.jpg
painting/star/painting_284_000066.jpg
painting/star/painting_284_000067.jpg
painting/star/painting_284_000072.jpg
painting/star/painting_284_000074.jpg
painting/star/painting_284_000078.jpg
painting/star/painting_284_000085.jpg
painting/star/painting_284_000088.jpg
painting/star/painting_284_000089.jpg
painting/star/painting_284_000092.jpg
painting/star/painting_284_000095.jpg
painting/star/painting_284_000096.jpg
painting/steak/painting_285_000010.jpg
painting/steak/painting_285_000013.jpg
painting/steak/painting_285_000017.jpg
painting/steak/painting_285_000023.jpg
painting/steak/painting_285_000024.jpg
painting/steak/painting_285_000029.jpg
painting/steak/painting_285_000039.jpg
painting/steak/painting_285_000042.jpg
painting/steak/painting_285_000043.jpg
painting/steak/painting_285_000044.jpg
painting/steak/painting_285_000045.jpg
painting/steak/painting_285_000047.jpg
painting/stereo/painting_286_000004.jpg
painting/stereo/painting_286_000005.jpg
painting/stereo/painting_286_000008.jpg
painting/stereo/painting_286_000012.jpg
painting/stethoscope/painting_287_000049.jpg
painting/stethoscope/painting_287_000078.jpg
painting/stethoscope/painting_287_000086.jpg
painting/stethoscope/painting_287_000091.jpg
painting/stethoscope/painting_287_000113.jpg
painting/stethoscope/painting_287_000116.jpg
painting/stethoscope/painting_287_000118.jpg
painting/stethoscope/painting_287_000121.jpg
painting/stethoscope/painting_287_000124.jpg
painting/stethoscope/painting_287_000125.jpg
painting/stethoscope/painting_287_000130.jpg
painting/stethoscope/painting_287_000131.jpg
painting/stethoscope/painting_287_000133.jpg
painting/stethoscope/painting_287_000140.jpg
painting/stethoscope/painting_287_000144.jpg
painting/stethoscope/painting_287_000150.jpg
painting/stethoscope/painting_287_000152.jpg
painting/stethoscope/painting_287_000156.jpg
painting/stethoscope/painting_287_000158.jpg
painting/stethoscope/painting_287_000160.jpg
painting/stethoscope/painting_287_000161.jpg
painting/stethoscope/painting_287_000167.jpg
painting/stethoscope/painting_287_000168.jpg
painting/stethoscope/painting_287_000169.jpg
painting/stethoscope/painting_287_000185.jpg
painting/stethoscope/painting_287_000188.jpg
painting/stethoscope/painting_287_000200.jpg
painting/stethoscope/painting_287_000207.jpg
painting/stethoscope/painting_287_000208.jpg
painting/stethoscope/painting_287_000211.jpg
painting/stethoscope/painting_287_000215.jpg
painting/stethoscope/painting_287_000218.jpg
painting/stethoscope/painting_287_000220.jpg
painting/stethoscope/painting_287_000222.jpg
painting/stethoscope/painting_287_000231.jpg
painting/stethoscope/painting_287_000239.jpg
painting/stethoscope/painting_287_000246.jpg
painting/stethoscope/painting_287_000248.jpg
painting/stethoscope/painting_287_000250.jpg
painting/stethoscope/painting_287_000254.jpg
painting/stethoscope/painting_287_000256.jpg
painting/stethoscope/painting_287_000264.jpg
painting/stethoscope/painting_287_000273.jpg
painting/stethoscope/painting_287_000274.jpg
painting/stethoscope/painting_287_000284.jpg
painting/stethoscope/painting_287_000286.jpg
painting/stethoscope/painting_287_000287.jpg
painting/stethoscope/painting_287_000289.jpg
painting/stethoscope/painting_287_000290.jpg
painting/stethoscope/painting_287_000293.jpg
painting/stethoscope/painting_287_000294.jpg
painting/stethoscope/painting_287_000298.jpg
painting/stethoscope/painting_287_000301.jpg
painting/stethoscope/painting_287_000302.jpg
painting/stethoscope/painting_287_000304.jpg
painting/stethoscope/painting_287_000306.jpg
painting/stethoscope/painting_287_000310.jpg
painting/stethoscope/painting_287_000315.jpg
painting/stethoscope/painting_287_000316.jpg
painting/stethoscope/painting_287_000319.jpg
painting/stethoscope/painting_287_000323.jpg
painting/stethoscope/painting_287_000328.jpg
painting/stethoscope/painting_287_000330.jpg
painting/stethoscope/painting_287_000334.jpg
painting/stethoscope/painting_287_000335.jpg
painting/stethoscope/painting_287_000338.jpg
painting/stethoscope/painting_287_000341.jpg
painting/stethoscope/painting_287_000342.jpg
painting/stitches/painting_288_000007.jpg
painting/stitches/painting_288_000012.jpg
painting/stitches/painting_288_000013.jpg
painting/stop_sign/painting_289_000015.jpg
painting/stop_sign/painting_289_000031.jpg
painting/stop_sign/painting_289_000043.jpg
painting/stop_sign/painting_289_000046.jpg
painting/stop_sign/painting_289_000050.jpg
painting/stop_sign/painting_289_000058.jpg
painting/stop_sign/painting_289_000061.jpg
painting/stop_sign/painting_289_000062.jpg
painting/stop_sign/painting_289_000065.jpg
painting/stop_sign/painting_289_000081.jpg
painting/stove/painting_290_000009.jpg
painting/stove/painting_290_000016.jpg
painting/strawberry/painting_291_000073.jpg
painting/strawberry/painting_291_000109.jpg
painting/strawberry/painting_291_000110.jpg
painting/strawberry/painting_291_000124.jpg
painting/strawberry/painting_291_000132.jpg
painting/strawberry/painting_291_000133.jpg
painting/strawberry/painting_291_000134.jpg
painting/strawberry/painting_291_000137.jpg
painting/strawberry/painting_291_000141.jpg
painting/strawberry/painting_291_000144.jpg
painting/strawberry/painting_291_000153.jpg
painting/strawberry/painting_291_000157.jpg
painting/strawberry/painting_291_000175.jpg
painting/strawberry/painting_291_000177.jpg
painting/strawberry/painting_291_000178.jpg
painting/strawberry/painting_291_000186.jpg
painting/strawberry/painting_291_000192.jpg
painting/strawberry/painting_291_000193.jpg
painting/strawberry/painting_291_000198.jpg
painting/strawberry/painting_291_000207.jpg
painting/strawberry/painting_291_000209.jpg
painting/strawberry/painting_291_000210.jpg
painting/strawberry/painting_291_000212.jpg
painting/strawberry/painting_291_000216.jpg
painting/strawberry/painting_291_000222.jpg
painting/strawberry/painting_291_000223.jpg
painting/strawberry/painting_291_000225.jpg
painting/strawberry/painting_291_000226.jpg
painting/strawberry/painting_291_000227.jpg
painting/strawberry/painting_291_000234.jpg
painting/strawberry/painting_291_000237.jpg
painting/strawberry/painting_291_000240.jpg
painting/strawberry/painting_291_000243.jpg
painting/strawberry/painting_291_000246.jpg
painting/strawberry/painting_291_000254.jpg
painting/strawberry/painting_291_000255.jpg
painting/strawberry/painting_291_000256.jpg
painting/strawberry/painting_291_000264.jpg
painting/strawberry/painting_291_000265.jpg
painting/strawberry/painting_291_000273.jpg
painting/strawberry/painting_291_000275.jpg
painting/strawberry/painting_291_000278.jpg
painting/strawberry/painting_291_000279.jpg
painting/strawberry/painting_291_000283.jpg
painting/strawberry/painting_291_000284.jpg
painting/strawberry/painting_291_000293.jpg
painting/strawberry/painting_291_000297.jpg
painting/strawberry/painting_291_000298.jpg
painting/strawberry/painting_291_000300.jpg
painting/strawberry/painting_291_000304.jpg
painting/strawberry/painting_291_000310.jpg
painting/strawberry/painting_291_000311.jpg
painting/strawberry/painting_291_000312.jpg
painting/strawberry/painting_291_000314.jpg
painting/strawberry/painting_291_000315.jpg
painting/strawberry/painting_291_000316.jpg
painting/strawberry/painting_291_000317.jpg
painting/strawberry/painting_291_000320.jpg
painting/strawberry/painting_291_000322.jpg
painting/strawberry/painting_291_000323.jpg
painting/strawberry/painting_291_000325.jpg
painting/strawberry/painting_291_000330.jpg
painting/strawberry/painting_291_000334.jpg
painting/strawberry/painting_291_000337.jpg
painting/strawberry/painting_291_000338.jpg
painting/strawberry/painting_291_000340.jpg
painting/strawberry/painting_291_000341.jpg
painting/strawberry/painting_291_000342.jpg
painting/strawberry/painting_291_000345.jpg
painting/strawberry/painting_291_000348.jpg
painting/strawberry/painting_291_000354.jpg
painting/strawberry/painting_291_000355.jpg
painting/strawberry/painting_291_000356.jpg
painting/strawberry/painting_291_000361.jpg
painting/strawberry/painting_291_000362.jpg
painting/strawberry/painting_291_000363.jpg
painting/strawberry/painting_291_000364.jpg
painting/strawberry/painting_291_000368.jpg
painting/strawberry/painting_291_000371.jpg
painting/strawberry/painting_291_000373.jpg
painting/strawberry/painting_291_000376.jpg
painting/strawberry/painting_291_000377.jpg
painting/strawberry/painting_291_000378.jpg
painting/strawberry/painting_291_000379.jpg
painting/strawberry/painting_291_000380.jpg
painting/strawberry/painting_291_000383.jpg
painting/strawberry/painting_291_000384.jpg
painting/strawberry/painting_291_000385.jpg
painting/strawberry/painting_291_000387.jpg
painting/strawberry/painting_291_000388.jpg
painting/strawberry/painting_291_000392.jpg
painting/strawberry/painting_291_000394.jpg
painting/strawberry/painting_291_000395.jpg
painting/strawberry/painting_291_000397.jpg
painting/strawberry/painting_291_000401.jpg
painting/strawberry/painting_291_000402.jpg
painting/strawberry/painting_291_000405.jpg
painting/strawberry/painting_291_000406.jpg
painting/strawberry/painting_291_000407.jpg
painting/strawberry/painting_291_000409.jpg
painting/strawberry/painting_291_000410.jpg
painting/strawberry/painting_291_000411.jpg
painting/strawberry/painting_291_000412.jpg
painting/strawberry/painting_291_000413.jpg
painting/strawberry/painting_291_000418.jpg
painting/strawberry/painting_291_000419.jpg
painting/strawberry/painting_291_000423.jpg
painting/strawberry/painting_291_000427.jpg
painting/strawberry/painting_291_000433.jpg
painting/strawberry/painting_291_000435.jpg
painting/strawberry/painting_291_000436.jpg
painting/strawberry/painting_291_000439.jpg
painting/strawberry/painting_291_000444.jpg
painting/strawberry/painting_291_000445.jpg
painting/strawberry/painting_291_000448.jpg
painting/strawberry/painting_291_000449.jpg
painting/strawberry/painting_291_000450.jpg
painting/strawberry/painting_291_000453.jpg
painting/strawberry/painting_291_000454.jpg
painting/strawberry/painting_291_000458.jpg
painting/strawberry/painting_291_000459.jpg
painting/strawberry/painting_291_000460.jpg
painting/strawberry/painting_291_000461.jpg
painting/strawberry/painting_291_000462.jpg
painting/strawberry/painting_291_000465.jpg
painting/strawberry/painting_291_000466.jpg
painting/strawberry/painting_291_000469.jpg
painting/strawberry/painting_291_000470.jpg
painting/strawberry/painting_291_000471.jpg
painting/strawberry/painting_291_000472.jpg
painting/strawberry/painting_291_000474.jpg
painting/strawberry/painting_291_000476.jpg
painting/strawberry/painting_291_000477.jpg
painting/strawberry/painting_291_000478.jpg
painting/strawberry/painting_291_000482.jpg
painting/strawberry/painting_291_000485.jpg
painting/strawberry/painting_291_000487.jpg
painting/strawberry/painting_291_000495.jpg
painting/strawberry/painting_291_000497.jpg
painting/strawberry/painting_291_000498.jpg
painting/strawberry/painting_291_000500.jpg
painting/strawberry/painting_291_000501.jpg
painting/strawberry/painting_291_000502.jpg
painting/strawberry/painting_291_000505.jpg
painting/strawberry/painting_291_000507.jpg
painting/strawberry/painting_291_000508.jpg
painting/strawberry/painting_291_000509.jpg
painting/strawberry/painting_291_000510.jpg
painting/strawberry/painting_291_000511.jpg
painting/strawberry/painting_291_000514.jpg
painting/strawberry/painting_291_000516.jpg
painting/strawberry/painting_291_000524.jpg
painting/strawberry/painting_291_000525.jpg
painting/streetlight/painting_292_000036.jpg
painting/streetlight/painting_292_000096.jpg
painting/streetlight/painting_292_000099.jpg
painting/streetlight/painting_292_000126.jpg
painting/streetlight/painting_292_000137.jpg
painting/streetlight/painting_292_000149.jpg
painting/streetlight/painting_292_000159.jpg
painting/streetlight/painting_292_000161.jpg
painting/streetlight/painting_292_000165.jpg
painting/streetlight/painting_292_000169.jpg
painting/streetlight/painting_292_000178.jpg
painting/streetlight/painting_292_000183.jpg
painting/streetlight/painting_292_000185.jpg
painting/streetlight/painting_292_000191.jpg
painting/streetlight/painting_292_000205.jpg
painting/streetlight/painting_292_000213.jpg
painting/streetlight/painting_292_000216.jpg
painting/streetlight/painting_292_000219.jpg
painting/streetlight/painting_292_000231.jpg
painting/streetlight/painting_292_000235.jpg
painting/streetlight/painting_292_000242.jpg
painting/streetlight/painting_292_000256.jpg
painting/streetlight/painting_292_000261.jpg
painting/streetlight/painting_292_000265.jpg
painting/streetlight/painting_292_000266.jpg
painting/streetlight/painting_292_000268.jpg
painting/streetlight/painting_292_000270.jpg
painting/streetlight/painting_292_000273.jpg
painting/streetlight/painting_292_000274.jpg
painting/streetlight/painting_292_000282.jpg
painting/streetlight/painting_292_000292.jpg
painting/streetlight/painting_292_000305.jpg
painting/streetlight/painting_292_000309.jpg
painting/streetlight/painting_292_000314.jpg
painting/streetlight/painting_292_000318.jpg
painting/streetlight/painting_292_000319.jpg
painting/streetlight/painting_292_000324.jpg
painting/streetlight/painting_292_000326.jpg
painting/streetlight/painting_292_000331.jpg
painting/streetlight/painting_292_000337.jpg
painting/streetlight/painting_292_000344.jpg
painting/streetlight/painting_292_000346.jpg
painting/streetlight/painting_292_000347.jpg
painting/streetlight/painting_292_000348.jpg
painting/streetlight/painting_292_000352.jpg
painting/streetlight/painting_292_000354.jpg
painting/streetlight/painting_292_000358.jpg
painting/streetlight/painting_292_000362.jpg
painting/streetlight/painting_292_000364.jpg
painting/streetlight/painting_292_000371.jpg
painting/streetlight/painting_292_000375.jpg
painting/streetlight/painting_292_000377.jpg
painting/streetlight/painting_292_000384.jpg
painting/streetlight/painting_292_000387.jpg
painting/streetlight/painting_292_000388.jpg
painting/streetlight/painting_292_000391.jpg
painting/streetlight/painting_292_000398.jpg
painting/streetlight/painting_292_000402.jpg
painting/streetlight/painting_292_000403.jpg
painting/streetlight/painting_292_000404.jpg
painting/streetlight/painting_292_000407.jpg
painting/streetlight/painting_292_000408.jpg
painting/streetlight/painting_292_000410.jpg
painting/streetlight/painting_292_000414.jpg
painting/streetlight/painting_292_000417.jpg
painting/streetlight/painting_292_000425.jpg
painting/streetlight/painting_292_000429.jpg
painting/streetlight/painting_292_000432.jpg
painting/streetlight/painting_292_000438.jpg
painting/streetlight/painting_292_000440.jpg
painting/streetlight/painting_292_000447.jpg
painting/streetlight/painting_292_000449.jpg
painting/streetlight/painting_292_000450.jpg
painting/streetlight/painting_292_000452.jpg
painting/streetlight/painting_292_000460.jpg
painting/streetlight/painting_292_000464.jpg
painting/streetlight/painting_292_000465.jpg
painting/streetlight/painting_292_000468.jpg
painting/streetlight/painting_292_000471.jpg
painting/streetlight/painting_292_000473.jpg
painting/streetlight/painting_292_000475.jpg
painting/streetlight/painting_292_000481.jpg
painting/streetlight/painting_292_000482.jpg
painting/streetlight/painting_292_000483.jpg
painting/streetlight/painting_292_000485.jpg
painting/streetlight/painting_292_000489.jpg
painting/streetlight/painting_292_000490.jpg
painting/streetlight/painting_292_000492.jpg
painting/streetlight/painting_292_000496.jpg
painting/streetlight/painting_292_000497.jpg
painting/streetlight/painting_292_000502.jpg
painting/streetlight/painting_292_000503.jpg
painting/streetlight/painting_292_000505.jpg
painting/streetlight/painting_292_000508.jpg
painting/streetlight/painting_292_000509.jpg
painting/streetlight/painting_292_000510.jpg
painting/streetlight/painting_292_000511.jpg
painting/streetlight/painting_292_000512.jpg
painting/streetlight/painting_292_000520.jpg
painting/streetlight/painting_292_000521.jpg
painting/streetlight/painting_292_000524.jpg
painting/streetlight/painting_292_000526.jpg
painting/streetlight/painting_292_000533.jpg
painting/streetlight/painting_292_000534.jpg
painting/streetlight/painting_292_000537.jpg
painting/string_bean/painting_293_000008.jpg
painting/string_bean/painting_293_000026.jpg
painting/string_bean/painting_293_000028.jpg
painting/string_bean/painting_293_000038.jpg
painting/string_bean/painting_293_000044.jpg
painting/string_bean/painting_293_000045.jpg
painting/string_bean/painting_293_000049.jpg
painting/string_bean/painting_293_000050.jpg
painting/string_bean/painting_293_000056.jpg
painting/string_bean/painting_293_000057.jpg
painting/string_bean/painting_293_000059.jpg
painting/string_bean/painting_293_000061.jpg
painting/submarine/painting_294_000055.jpg
painting/submarine/painting_294_000056.jpg
painting/submarine/painting_294_000067.jpg
painting/submarine/painting_294_000071.jpg
painting/submarine/painting_294_000095.jpg
painting/submarine/painting_294_000102.jpg
painting/submarine/painting_294_000133.jpg
painting/submarine/painting_294_000140.jpg
painting/submarine/painting_294_000157.jpg
painting/submarine/painting_294_000178.jpg
painting/submarine/painting_294_000181.jpg
painting/submarine/painting_294_000185.jpg
painting/submarine/painting_294_000186.jpg
painting/submarine/painting_294_000189.jpg
painting/submarine/painting_294_000190.jpg
painting/submarine/painting_294_000196.jpg
painting/submarine/painting_294_000202.jpg
painting/submarine/painting_294_000206.jpg
painting/submarine/painting_294_000209.jpg
painting/submarine/painting_294_000212.jpg
painting/submarine/painting_294_000231.jpg
painting/submarine/painting_294_000232.jpg
painting/submarine/painting_294_000239.jpg
painting/submarine/painting_294_000242.jpg
painting/submarine/painting_294_000247.jpg
painting/submarine/painting_294_000253.jpg
painting/submarine/painting_294_000258.jpg
painting/submarine/painting_294_000259.jpg
painting/submarine/painting_294_000262.jpg
painting/submarine/painting_294_000264.jpg
painting/submarine/painting_294_000271.jpg
painting/submarine/painting_294_000273.jpg
painting/submarine/painting_294_000275.jpg
painting/submarine/painting_294_000277.jpg
painting/submarine/painting_294_000287.jpg
painting/submarine/painting_294_000289.jpg
painting/submarine/painting_294_000291.jpg
painting/submarine/painting_294_000293.jpg
painting/submarine/painting_294_000306.jpg
painting/submarine/painting_294_000313.jpg
painting/submarine/painting_294_000314.jpg
painting/submarine/painting_294_000317.jpg
painting/submarine/painting_294_000324.jpg
painting/submarine/painting_294_000326.jpg
painting/submarine/painting_294_000327.jpg
painting/submarine/painting_294_000330.jpg
painting/submarine/painting_294_000331.jpg
painting/submarine/painting_294_000332.jpg
painting/submarine/painting_294_000337.jpg
painting/submarine/painting_294_000345.jpg
painting/submarine/painting_294_000346.jpg
painting/submarine/painting_294_000353.jpg
painting/submarine/painting_294_000358.jpg
painting/submarine/painting_294_000360.jpg
painting/submarine/painting_294_000362.jpg
painting/submarine/painting_294_000363.jpg
painting/submarine/painting_294_000366.jpg
painting/submarine/painting_294_000377.jpg
painting/submarine/painting_294_000378.jpg
painting/submarine/painting_294_000380.jpg
painting/submarine/painting_294_000382.jpg
painting/submarine/painting_294_000383.jpg
painting/submarine/painting_294_000384.jpg
painting/submarine/painting_294_000388.jpg
painting/submarine/painting_294_000389.jpg
painting/submarine/painting_294_000390.jpg
painting/submarine/painting_294_000392.jpg
painting/submarine/painting_294_000394.jpg
painting/submarine/painting_294_000396.jpg
painting/submarine/painting_294_000397.jpg
painting/submarine/painting_294_000399.jpg
painting/submarine/painting_294_000401.jpg
painting/submarine/painting_294_000408.jpg
painting/submarine/painting_294_000409.jpg
painting/submarine/painting_294_000411.jpg
painting/submarine/painting_294_000413.jpg
painting/submarine/painting_294_000414.jpg
painting/submarine/painting_294_000415.jpg
painting/submarine/painting_294_000416.jpg
painting/submarine/painting_294_000417.jpg
painting/submarine/painting_294_000418.jpg
painting/submarine/painting_294_000420.jpg
painting/submarine/painting_294_000421.jpg
painting/submarine/painting_294_000429.jpg
painting/submarine/painting_294_000431.jpg
painting/submarine/painting_294_000432.jpg
painting/submarine/painting_294_000434.jpg
painting/submarine/painting_294_000435.jpg
painting/submarine/painting_294_000438.jpg
painting/submarine/painting_294_000444.jpg
painting/submarine/painting_294_000445.jpg
painting/submarine/painting_294_000448.jpg
painting/submarine/painting_294_000449.jpg
painting/submarine/painting_294_000452.jpg
painting/submarine/painting_294_000453.jpg
painting/submarine/painting_294_000455.jpg
painting/submarine/painting_294_000456.jpg
painting/submarine/painting_294_000458.jpg
painting/submarine/painting_294_000460.jpg
painting/submarine/painting_294_000464.jpg
painting/submarine/painting_294_000467.jpg
painting/submarine/painting_294_000469.jpg
painting/submarine/painting_294_000472.jpg
painting/submarine/painting_294_000477.jpg
painting/submarine/painting_294_000478.jpg
painting/submarine/painting_294_000484.jpg
painting/submarine/painting_294_000485.jpg
painting/submarine/painting_294_000486.jpg
painting/submarine/painting_294_000487.jpg
painting/submarine/painting_294_000488.jpg
painting/submarine/painting_294_000490.jpg
painting/submarine/painting_294_000491.jpg
painting/submarine/painting_294_000494.jpg
painting/submarine/painting_294_000498.jpg
painting/submarine/painting_294_000500.jpg
painting/submarine/painting_294_000503.jpg
painting/submarine/painting_294_000504.jpg
painting/submarine/painting_294_000505.jpg
painting/submarine/painting_294_000507.jpg
painting/submarine/painting_294_000511.jpg
painting/submarine/painting_294_000513.jpg
painting/submarine/painting_294_000516.jpg
painting/submarine/painting_294_000517.jpg
painting/submarine/painting_294_000518.jpg
painting/submarine/painting_294_000519.jpg
painting/submarine/painting_294_000523.jpg
painting/submarine/painting_294_000524.jpg
painting/submarine/painting_294_000527.jpg
painting/submarine/painting_294_000528.jpg
painting/submarine/painting_294_000530.jpg
painting/submarine/painting_294_000531.jpg
painting/submarine/painting_294_000533.jpg
painting/submarine/painting_294_000534.jpg
painting/submarine/painting_294_000536.jpg
painting/submarine/painting_294_000537.jpg
painting/submarine/painting_294_000539.jpg
painting/submarine/painting_294_000541.jpg
painting/submarine/painting_294_000544.jpg
painting/submarine/painting_294_000545.jpg
painting/submarine/painting_294_000547.jpg
painting/submarine/painting_294_000548.jpg
painting/submarine/painting_294_000549.jpg
painting/submarine/painting_294_000550.jpg
painting/suitcase/painting_295_000026.jpg
painting/suitcase/painting_295_000032.jpg
painting/suitcase/painting_295_000039.jpg
painting/suitcase/painting_295_000061.jpg
painting/suitcase/painting_295_000065.jpg
painting/suitcase/painting_295_000072.jpg
painting/suitcase/painting_295_000074.jpg
painting/suitcase/painting_295_000079.jpg
painting/suitcase/painting_295_000080.jpg
painting/suitcase/painting_295_000086.jpg
painting/suitcase/painting_295_000097.jpg
painting/suitcase/painting_295_000100.jpg
painting/suitcase/painting_295_000109.jpg
painting/suitcase/painting_295_000111.jpg
painting/suitcase/painting_295_000114.jpg
painting/suitcase/painting_295_000116.jpg
painting/suitcase/painting_295_000117.jpg
painting/suitcase/painting_295_000118.jpg
painting/suitcase/painting_295_000124.jpg
painting/suitcase/painting_295_000128.jpg
painting/suitcase/painting_295_000130.jpg
painting/suitcase/painting_295_000137.jpg
painting/suitcase/painting_295_000144.jpg
painting/suitcase/painting_295_000146.jpg
painting/suitcase/painting_295_000148.jpg
painting/suitcase/painting_295_000151.jpg
painting/suitcase/painting_295_000156.jpg
painting/suitcase/painting_295_000159.jpg
painting/suitcase/painting_295_000161.jpg
painting/suitcase/painting_295_000162.jpg
painting/suitcase/painting_295_000163.jpg
painting/suitcase/painting_295_000165.jpg
painting/suitcase/painting_295_000166.jpg
painting/suitcase/painting_295_000167.jpg
painting/suitcase/painting_295_000169.jpg
painting/suitcase/painting_295_000170.jpg
painting/suitcase/painting_295_000174.jpg
painting/suitcase/painting_295_000175.jpg
painting/suitcase/painting_295_000178.jpg
painting/suitcase/painting_295_000182.jpg
painting/suitcase/painting_295_000183.jpg
painting/suitcase/painting_295_000185.jpg
painting/suitcase/painting_295_000187.jpg
painting/sun/painting_296_000010.jpg
painting/sun/painting_296_000017.jpg
painting/sun/painting_296_000030.jpg
painting/sun/painting_296_000050.jpg
painting/sun/painting_296_000067.jpg
painting/sun/painting_296_000072.jpg
painting/sun/painting_296_000090.jpg
painting/sun/painting_296_000091.jpg
painting/sun/painting_296_000095.jpg
painting/sun/painting_296_000102.jpg
painting/sun/painting_296_000110.jpg
painting/sun/painting_296_000113.jpg
painting/sun/painting_296_000118.jpg
painting/sun/painting_296_000122.jpg
painting/sun/painting_296_000133.jpg
painting/sun/painting_296_000139.jpg
painting/sun/painting_296_000140.jpg
painting/sun/painting_296_000150.jpg
painting/sun/painting_296_000155.jpg
painting/sun/painting_296_000158.jpg
painting/sun/painting_296_000163.jpg
painting/sun/painting_296_000168.jpg
painting/sun/painting_296_000169.jpg
painting/sun/painting_296_000171.jpg
painting/sun/painting_296_000174.jpg
painting/sun/painting_296_000175.jpg
painting/sun/painting_296_000178.jpg
painting/sun/painting_296_000186.jpg
painting/sun/painting_296_000190.jpg
painting/sun/painting_296_000191.jpg
painting/sun/painting_296_000192.jpg
painting/sun/painting_296_000195.jpg
painting/sun/painting_296_000198.jpg
painting/sun/painting_296_000200.jpg
painting/sun/painting_296_000202.jpg
painting/sun/painting_296_000206.jpg
painting/sun/painting_296_000208.jpg
painting/sun/painting_296_000209.jpg
painting/sun/painting_296_000211.jpg
painting/sun/painting_296_000212.jpg
painting/sun/painting_296_000213.jpg
painting/sun/painting_296_000217.jpg
painting/sun/painting_296_000218.jpg
painting/sun/painting_296_000221.jpg
painting/sun/painting_296_000224.jpg
painting/sun/painting_296_000226.jpg
painting/sun/painting_296_000237.jpg
painting/sun/painting_296_000243.jpg
painting/sun/painting_296_000245.jpg
painting/sun/painting_296_000247.jpg
painting/sun/painting_296_000251.jpg
painting/sun/painting_296_000252.jpg
painting/sun/painting_296_000253.jpg
painting/sun/painting_296_000255.jpg
painting/sun/painting_296_000258.jpg
painting/sun/painting_296_000267.jpg
painting/sun/painting_296_000269.jpg
painting/sun/painting_296_000271.jpg
painting/sun/painting_296_000272.jpg
painting/sun/painting_296_000275.jpg
painting/sun/painting_296_000280.jpg
painting/sun/painting_296_000281.jpg
painting/sun/painting_296_000283.jpg
painting/sun/painting_296_000285.jpg
painting/sun/painting_296_000287.jpg
painting/sun/painting_296_000288.jpg
painting/sun/painting_296_000289.jpg
painting/sun/painting_296_000290.jpg
painting/sun/painting_296_000294.jpg
painting/sun/painting_296_000295.jpg
painting/sun/painting_296_000297.jpg
painting/sun/painting_296_000299.jpg
painting/sun/painting_296_000300.jpg
painting/sun/painting_296_000301.jpg
painting/sun/painting_296_000302.jpg
painting/sun/painting_296_000304.jpg
painting/sun/painting_296_000305.jpg
painting/sun/painting_296_000310.jpg
painting/sun/painting_296_000313.jpg
painting/sun/painting_296_000316.jpg
painting/sun/painting_296_000317.jpg
painting/sun/painting_296_000319.jpg
painting/sun/painting_296_000320.jpg
painting/sun/painting_296_000323.jpg
painting/sun/painting_296_000324.jpg
painting/sun/painting_296_000326.jpg
painting/sun/painting_296_000327.jpg
painting/sun/painting_296_000328.jpg
painting/sun/painting_296_000330.jpg
painting/sun/painting_296_000331.jpg
painting/sun/painting_296_000332.jpg
painting/sun/painting_296_000336.jpg
painting/sun/painting_296_000337.jpg
painting/sun/painting_296_000340.jpg
painting/sun/painting_296_000342.jpg
painting/sun/painting_296_000343.jpg
painting/sun/painting_296_000345.jpg
painting/sun/painting_296_000347.jpg
painting/sun/painting_296_000350.jpg
painting/sun/painting_296_000353.jpg
painting/sun/painting_296_000354.jpg
painting/sun/painting_296_000356.jpg
painting/sun/painting_296_000358.jpg
painting/sun/painting_296_000359.jpg
painting/sun/painting_296_000361.jpg
painting/sun/painting_296_000362.jpg
painting/sun/painting_296_000363.jpg
painting/sun/painting_296_000365.jpg
painting/sun/painting_296_000367.jpg
painting/sun/painting_296_000369.jpg
painting/sun/painting_296_000373.jpg
painting/sun/painting_296_000374.jpg
painting/sun/painting_296_000375.jpg
painting/sun/painting_296_000376.jpg
painting/sun/painting_296_000377.jpg
painting/sun/painting_296_000378.jpg
painting/sun/painting_296_000379.jpg
painting/sun/painting_296_000380.jpg
painting/sun/painting_296_000381.jpg
painting/sun/painting_296_000382.jpg
painting/sun/painting_296_000388.jpg
painting/sun/painting_296_000390.jpg
painting/sun/painting_296_000392.jpg
painting/sun/painting_296_000393.jpg
painting/sun/painting_296_000396.jpg
painting/sun/painting_296_000397.jpg
painting/sun/painting_296_000398.jpg
painting/sun/painting_296_000399.jpg
painting/sun/painting_296_000402.jpg
painting/sun/painting_296_000403.jpg
painting/sun/painting_296_000404.jpg
painting/sun/painting_296_000406.jpg
painting/sun/painting_296_000408.jpg
painting/sun/painting_296_000409.jpg
painting/sun/painting_296_000410.jpg
painting/sun/painting_296_000413.jpg
painting/sun/painting_296_000416.jpg
painting/sun/painting_296_000419.jpg
painting/sun/painting_296_000421.jpg
painting/sun/painting_296_000422.jpg
painting/sun/painting_296_000423.jpg
painting/sun/painting_296_000424.jpg
painting/sun/painting_296_000425.jpg
painting/sun/painting_296_000427.jpg
painting/sun/painting_296_000428.jpg
painting/sun/painting_296_000430.jpg
painting/sun/painting_296_000432.jpg
painting/sun/painting_296_000434.jpg
painting/sun/painting_296_000436.jpg
painting/sun/painting_296_000439.jpg
painting/sun/painting_296_000440.jpg
painting/sun/painting_296_000442.jpg
painting/sun/painting_296_000445.jpg
painting/sun/painting_296_000447.jpg
painting/sun/painting_296_000448.jpg
painting/sun/painting_296_000449.jpg
painting/sun/painting_296_000450.jpg
painting/sun/painting_296_000451.jpg
painting/sun/painting_296_000453.jpg
painting/sun/painting_296_000456.jpg
painting/sun/painting_296_000458.jpg
painting/sun/painting_296_000460.jpg
painting/sun/painting_296_000461.jpg
painting/sun/painting_296_000462.jpg
painting/sun/painting_296_000463.jpg
painting/sun/painting_296_000464.jpg
painting/sun/painting_296_000465.jpg
painting/sun/painting_296_000468.jpg
painting/sun/painting_296_000470.jpg
painting/sun/painting_296_000471.jpg
painting/sun/painting_296_000472.jpg
painting/sun/painting_296_000473.jpg
painting/sun/painting_296_000476.jpg
painting/sun/painting_296_000477.jpg
painting/sun/painting_296_000480.jpg
painting/sun/painting_296_000482.jpg
painting/sun/painting_296_000483.jpg
painting/sun/painting_296_000484.jpg
painting/sun/painting_296_000486.jpg
painting/sun/painting_296_000487.jpg
painting/sun/painting_296_000489.jpg
painting/sun/painting_296_000490.jpg
painting/sun/painting_296_000493.jpg
painting/sun/painting_296_000497.jpg
painting/sun/painting_296_000498.jpg
painting/sun/painting_296_000499.jpg
painting/sun/painting_296_000500.jpg
painting/sun/painting_296_000502.jpg
painting/sun/painting_296_000505.jpg
painting/sun/painting_296_000509.jpg
painting/sun/painting_296_000511.jpg
painting/sun/painting_296_000512.jpg
painting/sun/painting_296_000513.jpg
painting/sun/painting_296_000515.jpg
painting/sun/painting_296_000517.jpg
painting/sun/painting_296_000519.jpg
painting/sun/painting_296_000520.jpg
painting/sun/painting_296_000523.jpg
painting/sun/painting_296_000524.jpg
painting/sun/painting_296_000525.jpg
painting/sun/painting_296_000526.jpg
painting/sun/painting_296_000527.jpg
painting/sun/painting_296_000528.jpg
painting/sun/painting_296_000529.jpg
painting/sun/painting_296_000530.jpg
painting/sun/painting_296_000531.jpg
painting/sun/painting_296_000533.jpg
painting/sun/painting_296_000534.jpg
painting/sun/painting_296_000535.jpg
painting/sun/painting_296_000537.jpg
painting/sun/painting_296_000538.jpg
painting/sun/painting_296_000539.jpg
painting/sun/painting_296_000540.jpg
painting/sun/painting_296_000541.jpg
painting/sun/painting_296_000542.jpg
painting/sun/painting_296_000543.jpg
painting/sun/painting_296_000544.jpg
painting/sun/painting_296_000545.jpg
painting/sun/painting_296_000546.jpg
painting/sun/painting_296_000548.jpg
painting/sun/painting_296_000550.jpg
painting/sun/painting_296_000551.jpg
painting/sun/painting_296_000552.jpg
painting/sun/painting_296_000555.jpg
painting/sun/painting_296_000556.jpg
painting/sun/painting_296_000557.jpg
painting/sun/painting_296_000558.jpg
painting/sun/painting_296_000559.jpg
painting/sun/painting_296_000560.jpg
painting/sun/painting_296_000561.jpg
painting/sun/painting_296_000562.jpg
painting/sun/painting_296_000563.jpg
painting/sun/painting_296_000565.jpg
painting/sun/painting_296_000566.jpg
painting/sun/painting_296_000567.jpg
painting/sun/painting_296_000568.jpg
painting/sun/painting_296_000569.jpg
painting/sun/painting_296_000570.jpg
painting/sun/painting_296_000572.jpg
painting/swan/painting_297_000009.jpg
painting/swan/painting_297_000034.jpg
painting/swan/painting_297_000049.jpg
painting/swan/painting_297_000052.jpg
painting/swan/painting_297_000075.jpg
painting/swan/painting_297_000083.jpg
painting/swan/painting_297_000087.jpg
painting/swan/painting_297_000100.jpg
painting/swan/painting_297_000101.jpg
painting/swan/painting_297_000109.jpg
painting/swan/painting_297_000111.jpg
painting/swan/painting_297_000121.jpg
painting/swan/painting_297_000123.jpg
painting/swan/painting_297_000133.jpg
painting/swan/painting_297_000136.jpg
painting/swan/painting_297_000139.jpg
painting/swan/painting_297_000141.jpg
painting/swan/painting_297_000153.jpg
painting/swan/painting_297_000155.jpg
painting/swan/painting_297_000157.jpg
painting/swan/painting_297_000158.jpg
painting/swan/painting_297_000163.jpg
painting/swan/painting_297_000166.jpg
painting/swan/painting_297_000172.jpg
painting/swan/painting_297_000177.jpg
painting/swan/painting_297_000178.jpg
painting/swan/painting_297_000180.jpg
painting/swan/painting_297_000191.jpg
painting/swan/painting_297_000196.jpg
painting/swan/painting_297_000197.jpg
painting/swan/painting_297_000202.jpg
painting/swan/painting_297_000204.jpg
painting/swan/painting_297_000215.jpg
painting/swan/painting_297_000217.jpg
painting/swan/painting_297_000219.jpg
painting/swan/painting_297_000220.jpg
painting/swan/painting_297_000224.jpg
painting/swan/painting_297_000226.jpg
painting/swan/painting_297_000230.jpg
painting/swan/painting_297_000232.jpg
painting/swan/painting_297_000234.jpg
painting/swan/painting_297_000235.jpg
painting/swan/painting_297_000236.jpg
painting/swan/painting_297_000239.jpg
painting/swan/painting_297_000250.jpg
painting/swan/painting_297_000253.jpg
painting/swan/painting_297_000257.jpg
painting/swan/painting_297_000259.jpg
painting/swan/painting_297_000265.jpg
painting/swan/painting_297_000271.jpg
painting/swan/painting_297_000272.jpg
painting/swan/painting_297_000273.jpg
painting/swan/painting_297_000278.jpg
painting/swan/painting_297_000280.jpg
painting/swan/painting_297_000282.jpg
painting/swan/painting_297_000287.jpg
painting/swan/painting_297_000288.jpg
painting/swan/painting_297_000291.jpg
painting/swan/painting_297_000294.jpg
painting/swan/painting_297_000295.jpg
painting/swan/painting_297_000297.jpg
painting/swan/painting_297_000298.jpg
painting/swan/painting_297_000299.jpg
painting/swan/painting_297_000302.jpg
painting/swan/painting_297_000308.jpg
painting/swan/painting_297_000312.jpg
painting/swan/painting_297_000316.jpg
painting/swan/painting_297_000317.jpg
painting/swan/painting_297_000320.jpg
painting/swan/painting_297_000322.jpg
painting/swan/painting_297_000325.jpg
painting/swan/painting_297_000326.jpg
painting/swan/painting_297_000329.jpg
painting/swan/painting_297_000330.jpg
painting/swan/painting_297_000331.jpg
painting/swan/painting_297_000332.jpg
painting/swan/painting_297_000333.jpg
painting/swan/painting_297_000338.jpg
painting/swan/painting_297_000339.jpg
painting/swan/painting_297_000340.jpg
painting/swan/painting_297_000342.jpg
painting/swan/painting_297_000346.jpg
painting/swan/painting_297_000347.jpg
painting/swan/painting_297_000348.jpg
painting/swan/painting_297_000349.jpg
painting/swan/painting_297_000350.jpg
painting/swan/painting_297_000351.jpg
painting/swan/painting_297_000356.jpg
painting/swan/painting_297_000357.jpg
painting/swan/painting_297_000358.jpg
painting/swan/painting_297_000359.jpg
painting/swan/painting_297_000360.jpg
painting/swan/painting_297_000363.jpg
painting/swan/painting_297_000364.jpg
painting/swan/painting_297_000365.jpg
painting/swan/painting_297_000366.jpg
painting/swan/painting_297_000369.jpg
painting/swan/painting_297_000371.jpg
painting/swan/painting_297_000372.jpg
painting/swan/painting_297_000373.jpg
painting/swan/painting_297_000374.jpg
painting/swan/painting_297_000375.jpg
painting/swan/painting_297_000377.jpg
painting/swan/painting_297_000379.jpg
painting/swan/painting_297_000381.jpg
painting/swan/painting_297_000384.jpg
painting/swan/painting_297_000385.jpg
painting/swan/painting_297_000388.jpg
painting/swan/painting_297_000389.jpg
painting/swan/painting_297_000391.jpg
painting/swan/painting_297_000392.jpg
painting/swan/painting_297_000395.jpg
painting/swan/painting_297_000396.jpg
painting/swan/painting_297_000402.jpg
painting/swan/painting_297_000403.jpg
painting/swan/painting_297_000404.jpg
painting/swan/painting_297_000406.jpg
painting/swan/painting_297_000407.jpg
painting/swan/painting_297_000409.jpg
painting/swan/painting_297_000411.jpg
painting/swan/painting_297_000413.jpg
painting/swan/painting_297_000414.jpg
painting/swan/painting_297_000415.jpg
painting/swan/painting_297_000417.jpg
painting/swan/painting_297_000420.jpg
painting/swan/painting_297_000422.jpg
painting/swan/painting_297_000423.jpg
painting/swan/painting_297_000427.jpg
painting/swan/painting_297_000429.jpg
painting/swan/painting_297_000430.jpg
painting/swan/painting_297_000431.jpg
painting/swan/painting_297_000432.jpg
painting/swan/painting_297_000433.jpg
painting/swan/painting_297_000434.jpg
painting/swan/painting_297_000438.jpg
painting/swan/painting_297_000440.jpg
painting/swan/painting_297_000441.jpg
painting/swan/painting_297_000442.jpg
painting/swan/painting_297_000445.jpg
painting/swan/painting_297_000453.jpg
painting/swan/painting_297_000454.jpg
painting/swan/painting_297_000455.jpg
painting/swan/painting_297_000457.jpg
painting/swan/painting_297_000459.jpg
painting/swan/painting_297_000460.jpg
painting/swan/painting_297_000461.jpg
painting/swan/painting_297_000463.jpg
painting/swan/painting_297_000465.jpg
painting/swan/painting_297_000466.jpg
painting/swan/painting_297_000467.jpg
painting/swan/painting_297_000470.jpg
painting/swan/painting_297_000471.jpg
painting/swan/painting_297_000473.jpg
painting/swan/painting_297_000475.jpg
painting/swan/painting_297_000476.jpg
painting/swan/painting_297_000478.jpg
painting/swan/painting_297_000479.jpg
painting/swan/painting_297_000483.jpg
painting/swan/painting_297_000484.jpg
painting/swan/painting_297_000486.jpg
painting/swan/painting_297_000487.jpg
painting/swan/painting_297_000489.jpg
painting/swan/painting_297_000490.jpg
painting/swan/painting_297_000491.jpg
painting/swan/painting_297_000493.jpg
painting/swan/painting_297_000494.jpg
painting/swan/painting_297_000496.jpg
painting/swan/painting_297_000498.jpg
painting/swan/painting_297_000499.jpg
painting/swan/painting_297_000501.jpg
painting/swan/painting_297_000502.jpg
painting/swan/painting_297_000503.jpg
painting/swan/painting_297_000504.jpg
painting/swan/painting_297_000505.jpg
painting/swan/painting_297_000507.jpg
painting/sweater/painting_298_000013.jpg
painting/sweater/painting_298_000014.jpg
painting/sweater/painting_298_000032.jpg
painting/sweater/painting_298_000034.jpg
painting/sweater/painting_298_000039.jpg
painting/sweater/painting_298_000040.jpg
painting/sweater/painting_298_000046.jpg
painting/sweater/painting_298_000054.jpg
painting/sweater/painting_298_000063.jpg
painting/sweater/painting_298_000069.jpg
painting/sweater/painting_298_000073.jpg
painting/sweater/painting_298_000077.jpg
painting/sweater/painting_298_000079.jpg
painting/sweater/painting_298_000088.jpg
painting/sweater/painting_298_000091.jpg
painting/sweater/painting_298_000095.jpg
painting/sweater/painting_298_000098.jpg
painting/sweater/painting_298_000100.jpg
painting/sweater/painting_298_000101.jpg
painting/sweater/painting_298_000102.jpg
painting/sweater/painting_298_000103.jpg
painting/sweater/painting_298_000106.jpg
painting/sweater/painting_298_000107.jpg
painting/sweater/painting_298_000111.jpg
painting/sweater/painting_298_000113.jpg
painting/sweater/painting_298_000114.jpg
painting/sweater/painting_298_000116.jpg
painting/sweater/painting_298_000118.jpg
painting/sweater/painting_298_000122.jpg
painting/sweater/painting_298_000124.jpg
painting/sweater/painting_298_000126.jpg
painting/sweater/painting_298_000128.jpg
painting/sweater/painting_298_000129.jpg
painting/sweater/painting_298_000131.jpg
painting/sweater/painting_298_000133.jpg
painting/sweater/painting_298_000134.jpg
painting/sweater/painting_298_000136.jpg
painting/sweater/painting_298_000138.jpg
painting/sweater/painting_298_000139.jpg
painting/sweater/painting_298_000140.jpg
painting/sweater/painting_298_000142.jpg
painting/sweater/painting_298_000143.jpg
painting/sweater/painting_298_000145.jpg
painting/sweater/painting_298_000147.jpg
painting/sweater/painting_298_000149.jpg
painting/sweater/painting_298_000150.jpg
painting/sweater/painting_298_000151.jpg
painting/sweater/painting_298_000153.jpg
painting/swing_set/painting_299_000020.jpg
painting/swing_set/painting_299_000036.jpg
painting/swing_set/painting_299_000052.jpg
painting/swing_set/painting_299_000057.jpg
painting/swing_set/painting_299_000058.jpg
painting/swing_set/painting_299_000059.jpg
painting/swing_set/painting_299_000064.jpg
painting/swing_set/painting_299_000065.jpg
painting/swing_set/painting_299_000069.jpg
painting/swing_set/painting_299_000070.jpg
painting/swing_set/painting_299_000073.jpg
painting/swing_set/painting_299_000076.jpg
painting/swing_set/painting_299_000081.jpg
painting/swing_set/painting_299_000082.jpg
painting/swing_set/painting_299_000085.jpg
painting/swing_set/painting_299_000087.jpg
painting/swing_set/painting_299_000091.jpg
painting/swing_set/painting_299_000099.jpg
painting/swing_set/painting_299_000100.jpg
painting/swing_set/painting_299_000101.jpg
painting/swing_set/painting_299_000110.jpg
painting/swing_set/painting_299_000111.jpg
painting/swing_set/painting_299_000113.jpg
painting/swing_set/painting_299_000116.jpg
painting/swing_set/painting_299_000117.jpg
painting/swing_set/painting_299_000122.jpg
painting/swing_set/painting_299_000127.jpg
painting/swing_set/painting_299_000129.jpg
painting/sword/painting_300_000041.jpg
painting/sword/painting_300_000044.jpg
painting/sword/painting_300_000045.jpg
painting/sword/painting_300_000050.jpg
painting/sword/painting_300_000058.jpg
painting/sword/painting_300_000067.jpg
painting/sword/painting_300_000078.jpg
painting/sword/painting_300_000088.jpg
painting/sword/painting_300_000089.jpg
painting/sword/painting_300_000107.jpg
painting/sword/painting_300_000119.jpg
painting/sword/painting_300_000131.jpg
painting/sword/painting_300_000133.jpg
painting/sword/painting_300_000137.jpg
painting/sword/painting_300_000138.jpg
painting/sword/painting_300_000146.jpg
painting/sword/painting_300_000147.jpg
painting/sword/painting_300_000150.jpg
painting/sword/painting_300_000155.jpg
painting/sword/painting_300_000156.jpg
painting/sword/painting_300_000157.jpg
painting/sword/painting_300_000161.jpg
painting/sword/painting_300_000171.jpg
painting/sword/painting_300_000174.jpg
painting/sword/painting_300_000182.jpg
painting/sword/painting_300_000183.jpg
painting/sword/painting_300_000186.jpg
painting/sword/painting_300_000187.jpg
painting/sword/painting_300_000190.jpg
painting/sword/painting_300_000195.jpg
painting/sword/painting_300_000196.jpg
painting/sword/painting_300_000197.jpg
painting/sword/painting_300_000201.jpg
painting/sword/painting_300_000207.jpg
painting/sword/painting_300_000222.jpg
painting/sword/painting_300_000226.jpg
painting/sword/painting_300_000229.jpg
painting/sword/painting_300_000232.jpg
painting/sword/painting_300_000236.jpg
painting/sword/painting_300_000240.jpg
painting/sword/painting_300_000241.jpg
painting/sword/painting_300_000244.jpg
painting/sword/painting_300_000246.jpg
painting/sword/painting_300_000250.jpg
painting/sword/painting_300_000251.jpg
painting/sword/painting_300_000254.jpg
painting/sword/painting_300_000257.jpg
painting/sword/painting_300_000262.jpg
painting/sword/painting_300_000263.jpg
painting/sword/painting_300_000266.jpg
painting/sword/painting_300_000268.jpg
painting/sword/painting_300_000269.jpg
painting/sword/painting_300_000272.jpg
painting/sword/painting_300_000273.jpg
painting/sword/painting_300_000277.jpg
painting/sword/painting_300_000280.jpg
painting/sword/painting_300_000282.jpg
painting/sword/painting_300_000286.jpg
painting/sword/painting_300_000287.jpg
painting/sword/painting_300_000293.jpg
painting/sword/painting_300_000294.jpg
painting/sword/painting_300_000295.jpg
painting/sword/painting_300_000297.jpg
painting/sword/painting_300_000298.jpg
painting/sword/painting_300_000299.jpg
painting/sword/painting_300_000305.jpg
painting/sword/painting_300_000311.jpg
painting/sword/painting_300_000313.jpg
painting/sword/painting_300_000314.jpg
painting/sword/painting_300_000315.jpg
painting/sword/painting_300_000318.jpg
painting/sword/painting_300_000320.jpg
painting/sword/painting_300_000321.jpg
painting/sword/painting_300_000325.jpg
painting/sword/painting_300_000326.jpg
painting/sword/painting_300_000327.jpg
painting/sword/painting_300_000330.jpg
painting/sword/painting_300_000332.jpg
painting/sword/painting_300_000334.jpg
painting/sword/painting_300_000335.jpg
painting/sword/painting_300_000338.jpg
painting/sword/painting_300_000339.jpg
painting/sword/painting_300_000341.jpg
painting/sword/painting_300_000343.jpg
painting/sword/painting_300_000344.jpg
painting/sword/painting_300_000346.jpg
painting/sword/painting_300_000347.jpg
painting/sword/painting_300_000349.jpg
painting/sword/painting_300_000350.jpg
painting/sword/painting_300_000352.jpg
painting/sword/painting_300_000354.jpg
painting/sword/painting_300_000356.jpg
painting/sword/painting_300_000359.jpg
painting/sword/painting_300_000361.jpg
painting/sword/painting_300_000362.jpg
painting/sword/painting_300_000363.jpg
painting/sword/painting_300_000364.jpg
painting/sword/painting_300_000365.jpg
painting/sword/painting_300_000366.jpg
painting/sword/painting_300_000367.jpg
painting/sword/painting_300_000370.jpg
painting/sword/painting_300_000371.jpg
painting/sword/painting_300_000374.jpg
painting/sword/painting_300_000375.jpg
painting/sword/painting_300_000376.jpg
painting/sword/painting_300_000378.jpg
painting/sword/painting_300_000381.jpg
painting/sword/painting_300_000383.jpg
painting/sword/painting_300_000387.jpg
painting/sword/painting_300_000388.jpg
painting/sword/painting_300_000390.jpg
painting/sword/painting_300_000394.jpg
painting/sword/painting_300_000395.jpg
painting/sword/painting_300_000397.jpg
painting/sword/painting_300_000398.jpg
painting/sword/painting_300_000400.jpg
painting/sword/painting_300_000401.jpg
painting/sword/painting_300_000402.jpg
painting/sword/painting_300_000403.jpg
painting/sword/painting_300_000404.jpg
painting/sword/painting_300_000408.jpg
painting/sword/painting_300_000410.jpg
painting/sword/painting_300_000411.jpg
painting/sword/painting_300_000412.jpg
painting/sword/painting_300_000413.jpg
painting/sword/painting_300_000415.jpg
painting/sword/painting_300_000417.jpg
painting/sword/painting_300_000418.jpg
painting/sword/painting_300_000419.jpg
painting/sword/painting_300_000421.jpg
painting/sword/painting_300_000423.jpg
painting/sword/painting_300_000424.jpg
painting/sword/painting_300_000426.jpg
painting/sword/painting_300_000430.jpg
painting/sword/painting_300_000431.jpg
painting/sword/painting_300_000433.jpg
painting/sword/painting_300_000434.jpg
painting/sword/painting_300_000435.jpg
painting/sword/painting_300_000437.jpg
painting/sword/painting_300_000438.jpg
painting/sword/painting_300_000439.jpg
painting/sword/painting_300_000440.jpg
painting/sword/painting_300_000441.jpg
painting/sword/painting_300_000442.jpg
painting/sword/painting_300_000444.jpg
painting/sword/painting_300_000445.jpg
painting/sword/painting_300_000447.jpg
painting/sword/painting_300_000449.jpg
painting/sword/painting_300_000450.jpg
painting/sword/painting_300_000451.jpg
painting/sword/painting_300_000453.jpg
painting/sword/painting_300_000455.jpg
painting/sword/painting_300_000456.jpg
painting/sword/painting_300_000457.jpg
painting/sword/painting_300_000458.jpg
painting/sword/painting_300_000459.jpg
painting/sword/painting_300_000460.jpg
painting/sword/painting_300_000462.jpg
painting/sword/painting_300_000464.jpg
painting/sword/painting_300_000465.jpg
painting/sword/painting_300_000466.jpg
painting/sword/painting_300_000467.jpg
painting/sword/painting_300_000470.jpg
painting/syringe/painting_301_000006.jpg
painting/syringe/painting_301_000007.jpg
painting/syringe/painting_301_000008.jpg
painting/syringe/painting_301_000009.jpg
painting/syringe/painting_301_000010.jpg
painting/table/painting_302_000006.jpg
painting/table/painting_302_000016.jpg
painting/table/painting_302_000054.jpg
painting/teapot/painting_303_000058.jpg
painting/teapot/painting_303_000115.jpg
painting/teapot/painting_303_000133.jpg
painting/teapot/painting_303_000224.jpg
painting/teapot/painting_303_000353.jpg
painting/teapot/painting_303_000389.jpg
painting/teddy-bear/painting_304_000001.jpg
painting/teddy-bear/painting_304_000020.jpg
painting/teddy-bear/painting_304_000021.jpg
painting/teddy-bear/painting_304_000024.jpg
painting/teddy-bear/painting_304_000025.jpg
painting/teddy-bear/painting_304_000026.jpg
painting/teddy-bear/painting_304_000029.jpg
painting/teddy-bear/painting_304_000033.jpg
painting/teddy-bear/painting_304_000036.jpg
painting/teddy-bear/painting_304_000043.jpg
painting/teddy-bear/painting_304_000048.jpg
painting/teddy-bear/painting_304_000049.jpg
painting/teddy-bear/painting_304_000056.jpg
painting/teddy-bear/painting_304_000061.jpg
painting/teddy-bear/painting_304_000065.jpg
painting/teddy-bear/painting_304_000072.jpg
painting/teddy-bear/painting_304_000080.jpg
painting/teddy-bear/painting_304_000081.jpg
painting/teddy-bear/painting_304_000087.jpg
painting/teddy-bear/painting_304_000098.jpg
painting/teddy-bear/painting_304_000109.jpg
painting/teddy-bear/painting_304_000112.jpg
painting/teddy-bear/painting_304_000118.jpg
painting/teddy-bear/painting_304_000148.jpg
painting/teddy-bear/painting_304_000151.jpg
painting/teddy-bear/painting_304_000161.jpg
painting/telephone/painting_305_000076.jpg
painting/television/painting_306_000035.jpg
painting/tent/painting_308_000095.jpg
painting/The_Eiffel_Tower/painting_309_000019.jpg
painting/The_Eiffel_Tower/painting_309_000063.jpg
painting/The_Eiffel_Tower/painting_309_000125.jpg
painting/The_Eiffel_Tower/painting_309_000127.jpg
painting/The_Great_Wall_of_China/painting_310_000017.jpg
painting/The_Great_Wall_of_China/painting_310_000042.jpg
painting/The_Great_Wall_of_China/painting_310_000049.jpg
painting/The_Great_Wall_of_China/painting_310_000054.jpg
painting/The_Mona_Lisa/painting_311_000009.jpg
painting/The_Mona_Lisa/painting_311_000028.jpg
painting/The_Mona_Lisa/painting_311_000036.jpg
painting/The_Mona_Lisa/painting_311_000039.jpg
painting/The_Mona_Lisa/painting_311_000053.jpg
painting/The_Mona_Lisa/painting_311_000081.jpg
painting/tiger/painting_312_000256.jpg
painting/tiger/painting_312_000395.jpg
painting/toaster/painting_313_000060.jpg
painting/toaster/painting_313_000061.jpg
painting/toaster/painting_313_000094.jpg
painting/toe/painting_314_000003.jpg
painting/toe/painting_314_000004.jpg
painting/toe/painting_314_000005.jpg
painting/toe/painting_314_000006.jpg
painting/toe/painting_314_000007.jpg
painting/toe/painting_314_000008.jpg
painting/toe/painting_314_000009.jpg
painting/toe/painting_314_000010.jpg
painting/toe/painting_314_000011.jpg
painting/toe/painting_314_000012.jpg
painting/toothbrush/painting_317_000001.jpg
painting/toothpaste/painting_318_000011.jpg
painting/toothpaste/painting_318_000028.jpg
painting/tornado/painting_319_000115.jpg
painting/tornado/painting_319_000124.jpg
painting/tornado/painting_319_000310.jpg
painting/tornado/painting_319_000326.jpg
painting/tractor/painting_320_000146.jpg
painting/traffic_light/painting_321_000020.jpg
painting/traffic_light/painting_321_000031.jpg
painting/traffic_light/painting_321_000050.jpg
painting/train/painting_322_000210.jpg
painting/train/painting_322_000325.jpg
painting/tree/painting_323_000056.jpg
painting/tree/painting_323_000070.jpg
painting/tree/painting_323_000092.jpg
painting/tree/painting_323_000113.jpg
painting/tree/painting_323_000125.jpg
painting/tree/painting_323_000140.jpg
painting/tree/painting_323_000171.jpg
painting/tree/painting_323_000182.jpg
painting/tree/painting_323_000247.jpg
painting/tree/painting_323_000270.jpg
painting/tree/painting_323_000284.jpg
painting/tree/painting_323_000301.jpg
painting/tree/painting_323_000316.jpg
painting/tree/painting_323_000324.jpg
painting/tree/painting_323_000417.jpg
painting/tree/painting_323_000421.jpg
painting/tree/painting_323_000464.jpg
painting/tree/painting_323_000469.jpg
painting/tree/painting_323_000482.jpg
painting/tree/painting_323_000566.jpg
painting/triangle/painting_324_000230.jpg
painting/truck/painting_326_000001.jpg
painting/truck/painting_326_000002.jpg
painting/truck/painting_326_000007.jpg
painting/truck/painting_326_000010.jpg
painting/truck/painting_326_000012.jpg
painting/truck/painting_326_000017.jpg
painting/truck/painting_326_000019.jpg
painting/truck/painting_326_000020.jpg
painting/truck/painting_326_000021.jpg
painting/truck/painting_326_000022.jpg
painting/truck/painting_326_000025.jpg
painting/truck/painting_326_000031.jpg
painting/truck/painting_326_000032.jpg
painting/truck/painting_326_000040.jpg
painting/truck/painting_326_000045.jpg
painting/truck/painting_326_000046.jpg
painting/truck/painting_326_000050.jpg
painting/truck/painting_326_000055.jpg
painting/truck/painting_326_000065.jpg
painting/truck/painting_326_000074.jpg
painting/truck/painting_326_000075.jpg
painting/truck/painting_326_000076.jpg
painting/truck/painting_326_000084.jpg
painting/truck/painting_326_000091.jpg
painting/truck/painting_326_000093.jpg
painting/truck/painting_326_000094.jpg
painting/truck/painting_326_000099.jpg
painting/truck/painting_326_000102.jpg
painting/truck/painting_326_000109.jpg
painting/truck/painting_326_000110.jpg
painting/truck/painting_326_000111.jpg
painting/truck/painting_326_000113.jpg
painting/truck/painting_326_000115.jpg
painting/truck/painting_326_000117.jpg
painting/truck/painting_326_000118.jpg
painting/trumpet/painting_327_000083.jpg
painting/t-shirt/painting_328_000002.jpg
painting/t-shirt/painting_328_000004.jpg
painting/t-shirt/painting_328_000005.jpg
painting/t-shirt/painting_328_000006.jpg
painting/t-shirt/painting_328_000007.jpg
painting/t-shirt/painting_328_000008.jpg
painting/t-shirt/painting_328_000009.jpg
painting/t-shirt/painting_328_000010.jpg
painting/t-shirt/painting_328_000011.jpg
painting/t-shirt/painting_328_000012.jpg
painting/umbrella/painting_329_000013.jpg
painting/umbrella/painting_329_000019.jpg
painting/umbrella/painting_329_000039.jpg
painting/umbrella/painting_329_000052.jpg
painting/umbrella/painting_329_000071.jpg
painting/umbrella/painting_329_000078.jpg
painting/umbrella/painting_329_000084.jpg
painting/umbrella/painting_329_000098.jpg
painting/umbrella/painting_329_000106.jpg
painting/umbrella/painting_329_000205.jpg
painting/umbrella/painting_329_000207.jpg
painting/umbrella/painting_329_000218.jpg
painting/underwear/painting_330_000010.jpg
painting/underwear/painting_330_000011.jpg
painting/underwear/painting_330_000012.jpg
painting/vase/painting_332_000093.jpg
painting/vase/painting_332_000182.jpg
painting/vase/painting_332_000230.jpg
painting/violin/painting_333_000011.jpg
painting/violin/painting_333_000058.jpg
painting/violin/painting_333_000158.jpg
painting/violin/painting_333_000197.jpg
painting/watermelon/painting_335_000208.jpg
painting/waterslide/painting_336_000009.jpg
painting/waterslide/painting_336_000010.jpg
painting/waterslide/painting_336_000011.jpg
painting/waterslide/painting_336_000012.jpg
painting/whale/painting_337_000172.jpg
painting/whale/painting_337_000283.jpg
painting/whale/painting_337_000323.jpg
painting/windmill/painting_339_000073.jpg
painting/wine_glass/painting_341_000041.jpg
painting/wine_glass/painting_341_000049.jpg
painting/wine_glass/painting_341_000102.jpg
painting/wine_glass/painting_341_000125.jpg
painting/wine_glass/painting_341_000126.jpg
painting/wine_glass/painting_341_000151.jpg
painting/wristwatch/painting_342_000018.jpg
painting/zebra/painting_344_000149.jpg
real/asparagus/real_011_000194.jpg
real/asparagus/real_011_000593.jpg
real/asparagus/real_011_000615.jpg
real/banana/real_014_000223.jpg
real/bandage/real_015_000046.jpg
real/bandage/real_015_000077.jpg
real/bandage/real_015_000153.jpg
real/bandage/real_015_000155.jpg
real/bandage/real_015_000198.jpg
real/bandage/real_015_000208.jpg
real/barn/real_016_000003.jpg
real/baseball_bat/real_018_000023.jpg
real/baseball_bat/real_018_000064.jpg
real/baseball_bat/real_018_000097.jpg
real/basket/real_019_000426.jpg
real/bat/real_021_000057.jpg
real/bee/real_027_000022.jpg
real/bee/real_027_000095.jpg
real/bee/real_027_000100.jpg
real/bee/real_027_000160.jpg
real/bee/real_027_000225.jpg
real/bee/real_027_000226.jpg
real/bee/real_027_000251.jpg
real/bee/real_027_000319.jpg
real/bee/real_027_000330.jpg
real/bee/real_027_000335.jpg
real/bee/real_027_000367.jpg
real/bee/real_027_000371.jpg
real/bee/real_027_000378.jpg
real/bee/real_027_000433.jpg
real/belt/real_028_000613.jpg
real/bench/real_029_000152.jpg
real/bench/real_029_000200.jpg
real/bicycle/real_030_000229.jpg
real/binoculars/real_031_000040.jpg
real/binoculars/real_031_000057.jpg
real/binoculars/real_031_000076.jpg
real/binoculars/real_031_000259.jpg
real/bird/real_032_000313.jpg
real/bird/real_032_000411.jpg
real/bird/real_032_000495.jpg
real/bird/real_032_000517.jpg
real/birthday_cake/real_033_000106.jpg
real/birthday_cake/real_033_000121.jpg
real/birthday_cake/real_033_000126.jpg
real/birthday_cake/real_033_000129.jpg
real/birthday_cake/real_033_000132.jpg
real/birthday_cake/real_033_000167.jpg
real/birthday_cake/real_033_000174.jpg
real/birthday_cake/real_033_000175.jpg
real/birthday_cake/real_033_000186.jpg
real/birthday_cake/real_033_000190.jpg
real/birthday_cake/real_033_000201.jpg
real/birthday_cake/real_033_000230.jpg
real/blueberry/real_035_000272.jpg
real/book/real_036_000009.jpg
real/book/real_036_000053.jpg
real/book/real_036_000082.jpg
real/book/real_036_000089.jpg
real/book/real_036_000092.jpg
real/book/real_036_000094.jpg
real/book/real_036_000135.jpg
real/book/real_036_000152.jpg
real/book/real_036_000202.jpg
real/book/real_036_000258.jpg
real/book/real_036_000285.jpg
real/book/real_036_000296.jpg
real/book/real_036_000306.jpg
real/book/real_036_000324.jpg
real/book/real_036_000360.jpg
real/book/real_036_000385.jpg
real/book/real_036_000530.jpg
real/book/real_036_000588.jpg
real/book/real_036_000670.jpg
real/book/real_036_000671.jpg
real/boomerang/real_037_000012.jpg
real/boomerang/real_037_000124.jpg
real/boomerang/real_037_000204.jpg
real/bottlecap/real_038_000030.jpg
real/bottlecap/real_038_000132.jpg
real/bottlecap/real_038_000199.jpg
real/bottlecap/real_038_000208.jpg
real/bottlecap/real_038_000277.jpg
real/bottlecap/real_038_000289.jpg
real/bottlecap/real_038_000318.jpg
real/bottlecap/real_038_000331.jpg
real/bottlecap/real_038_000363.jpg
real/bottlecap/real_038_000365.jpg
real/bottlecap/real_038_000389.jpg
real/bottlecap/real_038_000428.jpg
real/bottlecap/real_038_000446.jpg
real/bottlecap/real_038_000473.jpg
real/bottlecap/real_038_000484.jpg
real/bottlecap/real_038_000487.jpg
real/bottlecap/real_038_000523.jpg
real/bottlecap/real_038_000531.jpg
real/bottlecap/real_038_000567.jpg
real/bottlecap/real_038_000577.jpg
real/bottlecap/real_038_000590.jpg
real/bottlecap/real_038_000593.jpg
real/bowtie/real_039_000045.jpg
real/bowtie/real_039_000056.jpg
real/bowtie/real_039_000082.jpg
real/bowtie/real_039_000089.jpg
real/bowtie/real_039_000150.jpg
real/bowtie/real_039_000214.jpg
real/bowtie/real_039_000224.jpg
real/bowtie/real_039_000266.jpg
real/bowtie/real_039_000278.jpg
real/bowtie/real_039_000319.jpg
real/bowtie/real_039_000352.jpg
real/bowtie/real_039_000358.jpg
real/bowtie/real_039_000394.jpg
real/bowtie/real_039_000396.jpg
real/bowtie/real_039_000474.jpg
real/bowtie/real_039_000489.jpg
real/bowtie/real_039_000506.jpg
real/bracelet/real_040_000476.jpg
real/bracelet/real_040_000712.jpg
real/brain/real_041_000024.jpg
real/brain/real_041_000058.jpg
real/brain/real_041_000078.jpg
real/brain/real_041_000192.jpg
real/brain/real_041_000317.jpg
real/brain/real_041_000388.jpg
real/brain/real_041_000507.jpg
real/brain/real_041_000590.jpg
real/brain/real_041_000711.jpg
real/bread/real_042_000045.jpg
real/bread/real_042_000690.jpg
real/bridge/real_043_000127.jpg
real/bridge/real_043_000706.jpg
real/broccoli/real_044_000140.jpg
real/broccoli/real_044_000497.jpg
real/broom/real_045_000022.jpg
real/broom/real_045_000242.jpg
real/broom/real_045_000496.jpg
real/broom/real_045_000521.jpg
real/broom/real_045_000601.jpg
real/broom/real_045_000608.jpg
real/broom/real_045_000619.jpg
real/bucket/real_046_000039.jpg
real/bucket/real_046_000075.jpg
real/bucket/real_046_000147.jpg
real/bucket/real_046_000253.jpg
real/bucket/real_046_000304.jpg
real/bucket/real_046_000305.jpg
real/bucket/real_046_000324.jpg
real/bulldozer/real_047_000027.jpg
real/bulldozer/real_047_000055.jpg
real/bulldozer/real_047_000466.jpg
real/bulldozer/real_047_000555.jpg
real/bulldozer/real_047_000573.jpg
real/bulldozer/real_047_000580.jpg
real/bus/real_048_000417.jpg
real/butterfly/real_050_000072.jpg
real/butterfly/real_050_000114.jpg
real/butterfly/real_050_000158.jpg
real/butterfly/real_050_000198.jpg
real/butterfly/real_050_000269.jpg
real/butterfly/real_050_000312.jpg
real/butterfly/real_050_000326.jpg
real/butterfly/real_050_000495.jpg
real/butterfly/real_050_000496.jpg
real/butterfly/real_050_000498.jpg
real/butterfly/real_050_000506.jpg
real/butterfly/real_050_000540.jpg
real/butterfly/real_050_000542.jpg
real/butterfly/real_050_000548.jpg
real/butterfly/real_050_000573.jpg
real/butterfly/real_050_000593.jpg
real/butterfly/real_050_000594.jpg
real/butterfly/real_050_000595.jpg
real/butterfly/real_050_000631.jpg
real/cactus/real_051_000427.jpg
real/cake/real_052_000006.jpg
real/cake/real_052_000073.jpg
real/cake/real_052_000285.jpg
real/cake/real_052_000340.jpg
real/cake/real_052_000343.jpg
real/cake/real_052_000435.jpg
real/cake/real_052_000444.jpg
real/cake/real_052_000488.jpg
real/cake/real_052_000526.jpg
real/cake/real_052_000538.jpg
real/cake/real_052_000604.jpg
real/cake/real_052_000672.jpg
real/camel/real_055_000006.jpg
real/camel/real_055_000066.jpg
real/camel/real_055_000147.jpg
real/camel/real_055_000171.jpg
real/camel/real_055_000270.jpg
real/camel/real_055_000301.jpg
real/camera/real_056_000070.jpg
real/camera/real_056_000352.jpg
real/camouflage/real_057_000119.jpg
real/campfire/real_058_000100.jpg
real/campfire/real_058_000174.jpg
real/campfire/real_058_000193.jpg
real/campfire/real_058_000206.jpg
real/campfire/real_058_000249.jpg
real/campfire/real_058_000264.jpg
real/campfire/real_058_000273.jpg
real/campfire/real_058_000279.jpg
real/campfire/real_058_000341.jpg
real/campfire/real_058_000361.jpg
real/campfire/real_058_000408.jpg
real/campfire/real_058_000439.jpg
real/candle/real_059_000429.jpg
real/candle/real_059_000465.jpg
real/cannon/real_060_000006.jpg
real/cannon/real_060_000079.jpg
real/cannon/real_060_000126.jpg
real/cannon/real_060_000244.jpg
real/cannon/real_060_000258.jpg
real/canoe/real_061_000274.jpg
real/canoe/real_061_000555.jpg
real/car/real_062_000372.jpg
real/carrot/real_063_000126.jpg
real/carrot/real_063_000179.jpg
real/cat/real_065_000117.jpg
real/cello/real_067_000108.jpg
real/cello/real_067_000368.jpg
real/circle/real_072_000014.jpg
real/circle/real_072_000065.jpg
real/circle/real_072_000067.jpg
real/circle/real_072_000079.jpg
real/circle/real_072_000110.jpg
real/circle/real_072_000112.jpg
real/circle/real_072_000123.jpg
real/circle/real_072_000136.jpg
real/circle/real_072_000139.jpg
real/circle/real_072_000146.jpg
real/circle/real_072_000152.jpg
real/circle/real_072_000157.jpg
real/circle/real_072_000164.jpg
real/circle/real_072_000173.jpg
real/circle/real_072_000185.jpg
real/circle/real_072_000211.jpg
real/circle/real_072_000242.jpg
real/clarinet/real_073_000063.jpg
real/clarinet/real_073_000084.jpg
real/clarinet/real_073_000160.jpg
real/clarinet/real_073_000366.jpg
real/clock/real_074_000045.jpg
real/clock/real_074_000063.jpg
real/clock/real_074_000064.jpg
real/clock/real_074_000087.jpg
real/clock/real_074_000089.jpg
real/clock/real_074_000132.jpg
real/clock/real_074_000164.jpg
real/clock/real_074_000224.jpg
real/clock/real_074_000227.jpg
real/clock/real_074_000249.jpg
real/clock/real_074_000270.jpg
real/clock/real_074_000271.jpg
real/clock/real_074_000322.jpg
real/clock/real_074_000354.jpg
real/clock/real_074_000375.jpg
real/clock/real_074_000410.jpg
real/clock/real_074_000418.jpg
real/clock/real_074_000424.jpg
real/clock/real_074_000429.jpg
real/clock/real_074_000507.jpg
real/clock/real_074_000508.jpg
real/clock/real_074_000522.jpg
real/clock/real_074_000523.jpg
real/clock/real_074_000536.jpg
real/clock/real_074_000543.jpg
real/clock/real_074_000551.jpg
real/clock/real_074_000577.jpg
real/clock/real_074_000595.jpg
real/clock/real_074_000605.jpg
real/coffee_cup/real_076_000041.jpg
real/coffee_cup/real_076_000089.jpg
real/coffee_cup/real_076_000095.jpg
real/coffee_cup/real_076_000102.jpg
real/coffee_cup/real_076_000128.jpg
real/coffee_cup/real_076_000159.jpg
real/coffee_cup/real_076_000182.jpg
real/coffee_cup/real_076_000202.jpg
real/coffee_cup/real_076_000234.jpg
real/coffee_cup/real_076_000244.jpg
real/coffee_cup/real_076_000253.jpg
real/coffee_cup/real_076_000289.jpg
real/coffee_cup/real_076_000322.jpg
real/coffee_cup/real_076_000334.jpg
real/coffee_cup/real_076_000345.jpg
real/coffee_cup/real_076_000358.jpg
real/coffee_cup/real_076_000398.jpg
real/coffee_cup/real_076_000402.jpg
real/coffee_cup/real_076_000430.jpg
real/coffee_cup/real_076_000499.jpg
real/coffee_cup/real_076_000502.jpg
real/coffee_cup/real_076_000547.jpg
real/coffee_cup/real_076_000567.jpg
real/coffee_cup/real_076_000581.jpg
real/coffee_cup/real_076_000594.jpg
real/compass/real_077_000051.jpg
real/compass/real_077_000062.jpg
real/compass/real_077_000072.jpg
real/compass/real_077_000073.jpg
real/compass/real_077_000077.jpg
real/compass/real_077_000081.jpg
real/compass/real_077_000103.jpg
real/compass/real_077_000106.jpg
real/compass/real_077_000134.jpg
real/compass/real_077_000137.jpg
real/compass/real_077_000139.jpg
real/compass/real_077_000157.jpg
real/compass/real_077_000162.jpg
real/compass/real_077_000190.jpg
real/compass/real_077_000194.jpg
real/compass/real_077_000199.jpg
real/compass/real_077_000211.jpg
real/compass/real_077_000220.jpg
real/compass/real_077_000229.jpg
real/compass/real_077_000256.jpg
real/compass/real_077_000261.jpg
real/compass/real_077_000271.jpg
real/computer/real_078_000012.jpg
real/computer/real_078_000117.jpg
real/computer/real_078_000165.jpg
real/computer/real_078_000226.jpg
real/computer/real_078_000227.jpg
real/computer/real_078_000271.jpg
real/computer/real_078_000351.jpg
real/cookie/real_079_000112.jpg
real/cookie/real_079_000440.jpg
real/cooler/real_080_000095.jpg
real/cooler/real_080_000156.jpg
real/cooler/real_080_000333.jpg
real/cooler/real_080_000414.jpg
real/cooler/real_080_000484.jpg
real/couch/real_081_000554.jpg
real/cow/real_082_000145.jpg
real/cow/real_082_000335.jpg
real/cow/real_082_000427.jpg
real/cow/real_082_000443.jpg
real/crab/real_083_000062.jpg
real/crab/real_083_000219.jpg
real/crab/real_083_000475.jpg
real/crab/real_083_000528.jpg
real/crab/real_083_000572.jpg
real/crab/real_083_000579.jpg
real/crab/real_083_000607.jpg
real/crab/real_083_000698.jpg
real/crab/real_083_000705.jpg
real/crayon/real_084_000081.jpg
real/crayon/real_084_000135.jpg
real/crayon/real_084_000199.jpg
real/crayon/real_084_000207.jpg
real/crayon/real_084_000218.jpg
real/crayon/real_084_000275.jpg
real/crayon/real_084_000484.jpg
real/crayon/real_084_000486.jpg
real/crayon/real_084_000487.jpg
real/crocodile/real_085_000631.jpg
real/crown/real_086_000009.jpg
real/crown/real_086_000014.jpg
real/crown/real_086_000025.jpg
real/crown/real_086_000030.jpg
real/crown/real_086_000034.jpg
real/crown/real_086_000050.jpg
real/crown/real_086_000062.jpg
real/crown/real_086_000074.jpg
real/crown/real_086_000077.jpg
real/crown/real_086_000078.jpg
real/crown/real_086_000086.jpg
real/crown/real_086_000095.jpg
real/crown/real_086_000105.jpg
real/crown/real_086_000145.jpg
real/crown/real_086_000156.jpg
real/crown/real_086_000160.jpg
real/cruise_ship/real_087_000224.jpg
real/cruise_ship/real_087_000235.jpg
real/cruise_ship/real_087_000276.jpg
real/cruise_ship/real_087_000488.jpg
real/cruise_ship/real_087_000580.jpg
real/cup/real_088_000006.jpg
real/cup/real_088_000009.jpg
real/cup/real_088_000042.jpg
real/cup/real_088_000046.jpg
real/cup/real_088_000047.jpg
real/cup/real_088_000064.jpg
real/cup/real_088_000097.jpg
real/cup/real_088_000105.jpg
real/cup/real_088_000106.jpg
real/cup/real_088_000113.jpg
real/cup/real_088_000120.jpg
real/cup/real_088_000142.jpg
real/cup/real_088_000150.jpg
real/cup/real_088_000160.jpg
real/cup/real_088_000162.jpg
real/cup/real_088_000171.jpg
real/cup/real_088_000176.jpg
real/cup/real_088_000193.jpg
real/cup/real_088_000207.jpg
real/cup/real_088_000232.jpg
real/cup/real_088_000237.jpg
real/cup/real_088_000240.jpg
real/cup/real_088_000249.jpg
real/cup/real_088_000260.jpg
real/cup/real_088_000287.jpg
real/cup/real_088_000296.jpg
real/cup/real_088_000297.jpg
real/cup/real_088_000308.jpg
real/cup/real_088_000328.jpg
real/cup/real_088_000331.jpg
real/cup/real_088_000341.jpg
real/cup/real_088_000373.jpg
real/cup/real_088_000376.jpg
real/cup/real_088_000379.jpg
real/diamond/real_089_000071.jpg
real/diamond/real_089_000083.jpg
real/diamond/real_089_000116.jpg
real/diamond/real_089_000143.jpg
real/diamond/real_089_000151.jpg
real/diamond/real_089_000195.jpg
real/diamond/real_089_000204.jpg
real/diamond/real_089_000268.jpg
real/diamond/real_089_000285.jpg
real/diamond/real_089_000353.jpg
real/diamond/real_089_000379.jpg
real/diamond/real_089_000496.jpg
real/diamond/real_089_000498.jpg
real/diamond/real_089_000499.jpg
real/diamond/real_089_000527.jpg
real/diamond/real_089_000529.jpg
real/diamond/real_089_000533.jpg
real/diamond/real_089_000535.jpg
real/diamond/real_089_000548.jpg
real/diamond/real_089_000553.jpg
real/diamond/real_089_000572.jpg
real/diamond/real_089_000577.jpg
real/dishwasher/real_090_000320.jpg
real/dishwasher/real_090_000435.jpg
real/diving_board/real_091_000050.jpg
real/diving_board/real_091_000064.jpg
real/diving_board/real_091_000075.jpg
real/diving_board/real_091_000076.jpg
real/diving_board/real_091_000095.jpg
real/diving_board/real_091_000096.jpg
real/diving_board/real_091_000130.jpg
real/diving_board/real_091_000131.jpg
real/diving_board/real_091_000140.jpg
real/diving_board/real_091_000161.jpg
real/diving_board/real_091_000189.jpg
real/diving_board/real_091_000216.jpg
real/diving_board/real_091_000252.jpg
real/diving_board/real_091_000273.jpg
real/diving_board/real_091_000281.jpg
real/diving_board/real_091_000291.jpg
real/diving_board/real_091_000325.jpg
real/diving_board/real_091_000372.jpg
real/diving_board/real_091_000387.jpg
real/diving_board/real_091_000444.jpg
real/diving_board/real_091_000447.jpg
real/diving_board/real_091_000462.jpg
real/diving_board/real_091_000496.jpg
real/diving_board/real_091_000504.jpg
real/diving_board/real_091_000548.jpg
real/diving_board/real_091_000586.jpg
real/dog/real_092_000099.jpg
real/dog/real_092_000523.jpg
real/dog/real_092_000676.jpg
real/dolphin/real_093_000025.jpg
real/dolphin/real_093_000326.jpg
real/donut/real_094_000276.jpg
real/donut/real_094_000279.jpg
real/donut/real_094_000451.jpg
real/door/real_095_000083.jpg
real/door/real_095_000085.jpg
real/door/real_095_000114.jpg
real/door/real_095_000145.jpg
real/door/real_095_000153.jpg
real/door/real_095_000296.jpg
real/door/real_095_000324.jpg
real/door/real_095_000337.jpg
real/dragon/real_096_000059.jpg
real/dragon/real_096_000090.jpg
real/dragon/real_096_000233.jpg
real/dragon/real_096_000304.jpg
real/dragon/real_096_000337.jpg
real/dragon/real_096_000353.jpg
real/dresser/real_097_000141.jpg
real/drill/real_098_000046.jpg
real/drill/real_098_000200.jpg
real/drill/real_098_000286.jpg
real/drill/real_098_000389.jpg
real/drill/real_098_000396.jpg
real/drill/real_098_000429.jpg
real/drill/real_098_000500.jpg
real/drill/real_098_000501.jpg
real/duck/real_100_000085.jpg
real/duck/real_100_000182.jpg
real/duck/real_100_000341.jpg
real/dumbbell/real_101_000124.jpg
real/dumbbell/real_101_000127.jpg
real/dumbbell/real_101_000135.jpg
real/dumbbell/real_101_000169.jpg
real/dumbbell/real_101_000276.jpg
real/dumbbell/real_101_000380.jpg
real/dumbbell/real_101_000396.jpg
real/ear/real_102_000018.jpg
real/ear/real_102_000023.jpg
real/ear/real_102_000052.jpg
real/ear/real_102_000107.jpg
real/ear/real_102_000176.jpg
real/ear/real_102_000183.jpg
real/ear/real_102_000330.jpg
real/elbow/real_103_000015.jpg
real/elbow/real_103_000075.jpg
real/elbow/real_103_000152.jpg
real/elbow/real_103_000226.jpg
real/elbow/real_103_000368.jpg
real/elephant/real_104_000222.jpg
real/elephant/real_104_000241.jpg
real/elephant/real_104_000347.jpg
real/elephant/real_104_000370.jpg
real/elephant/real_104_000678.jpg
real/elephant/real_104_000728.jpg
real/envelope/real_105_000025.jpg
real/envelope/real_105_000093.jpg
real/envelope/real_105_000102.jpg
real/envelope/real_105_000251.jpg
real/envelope/real_105_000343.jpg
real/envelope/real_105_000411.jpg
real/envelope/real_105_000515.jpg
real/envelope/real_105_000580.jpg
real/envelope/real_105_000595.jpg
real/envelope/real_105_000635.jpg
real/eraser/real_106_000008.jpg
real/eraser/real_106_000136.jpg
real/eraser/real_106_000148.jpg
real/eraser/real_106_000194.jpg
real/eye/real_107_000014.jpg
real/eye/real_107_000047.jpg
real/eye/real_107_000061.jpg
real/eye/real_107_000098.jpg
real/eye/real_107_000151.jpg
real/eye/real_107_000182.jpg
real/eye/real_107_000253.jpg
real/eye/real_107_000255.jpg
real/eye/real_107_000281.jpg
real/eye/real_107_000301.jpg
real/eye/real_107_000328.jpg
real/eye/real_107_000372.jpg
real/eye/real_107_000381.jpg
real/eye/real_107_000463.jpg
real/eye/real_107_000468.jpg
real/eye/real_107_000687.jpg
real/eyeglasses/real_108_000076.jpg
real/eyeglasses/real_108_000079.jpg
real/eyeglasses/real_108_000194.jpg
real/eyeglasses/real_108_000220.jpg
real/eyeglasses/real_108_000278.jpg
real/eyeglasses/real_108_000314.jpg
real/eyeglasses/real_108_000382.jpg
real/eyeglasses/real_108_000408.jpg
real/eyeglasses/real_108_000445.jpg
real/eyeglasses/real_108_000462.jpg
real/eyeglasses/real_108_000521.jpg
real/eyeglasses/real_108_000571.jpg
real/face/real_109_000124.jpg
real/face/real_109_000283.jpg
real/face/real_109_000661.jpg
real/fan/real_110_000005.jpg
real/fan/real_110_000008.jpg
real/fan/real_110_000096.jpg
real/fan/real_110_000100.jpg
real/fan/real_110_000119.jpg
real/fan/real_110_000153.jpg
real/fan/real_110_000200.jpg
real/fan/real_110_000237.jpg
real/fan/real_110_000249.jpg
real/fan/real_110_000263.jpg
real/fan/real_110_000271.jpg
real/fan/real_110_000279.jpg
real/fan/real_110_000287.jpg
real/fan/real_110_000291.jpg
real/fan/real_110_000395.jpg
real/fan/real_110_000422.jpg
real/fan/real_110_000434.jpg
real/fan/real_110_000436.jpg
real/fan/real_110_000444.jpg
real/fan/real_110_000450.jpg
real/feather/real_111_000005.jpg
real/feather/real_111_000006.jpg
real/feather/real_111_000011.jpg
real/feather/real_111_000019.jpg
real/feather/real_111_000025.jpg
real/feather/real_111_000031.jpg
real/feather/real_111_000035.jpg
real/feather/real_111_000051.jpg
real/feather/real_111_000057.jpg
real/feather/real_111_000084.jpg
real/feather/real_111_000131.jpg
real/feather/real_111_000156.jpg
real/feather/real_111_000172.jpg
real/feather/real_111_000177.jpg
real/feather/real_111_000197.jpg
real/feather/real_111_000215.jpg
real/feather/real_111_000220.jpg
real/feather/real_111_000229.jpg
real/feather/real_111_000285.jpg
real/feather/real_111_000416.jpg
real/feather/real_111_000466.jpg
real/feather/real_111_000479.jpg
real/feather/real_111_000484.jpg
real/feather/real_111_000490.jpg
real/fence/real_112_000044.jpg
real/fence/real_112_000084.jpg
real/fence/real_112_000146.jpg
real/fence/real_112_000514.jpg
real/finger/real_113_000008.jpg
real/finger/real_113_000037.jpg
real/finger/real_113_000062.jpg
real/finger/real_113_000076.jpg
real/finger/real_113_000117.jpg
real/finger/real_113_000138.jpg
real/finger/real_113_000142.jpg
real/finger/real_113_000188.jpg
real/finger/real_113_000623.jpg
real/fire_hydrant/real_114_000049.jpg
real/fire_hydrant/real_114_000104.jpg
real/fire_hydrant/real_114_000132.jpg
real/fire_hydrant/real_114_000174.jpg
real/fire_hydrant/real_114_000328.jpg
real/fire_hydrant/real_114_000371.jpg
real/fire_hydrant/real_114_000457.jpg
real/fire_hydrant/real_114_000472.jpg
real/fire_hydrant/real_114_000539.jpg
real/fire_hydrant/real_114_000552.jpg
real/firetruck/real_116_000061.jpg
real/firetruck/real_116_000091.jpg
real/firetruck/real_116_000092.jpg
real/firetruck/real_116_000102.jpg
real/firetruck/real_116_000106.jpg
real/firetruck/real_116_000116.jpg
real/firetruck/real_116_000138.jpg
real/firetruck/real_116_000158.jpg
real/firetruck/real_116_000177.jpg
real/firetruck/real_116_000214.jpg
real/firetruck/real_116_000220.jpg
real/firetruck/real_116_000223.jpg
real/firetruck/real_116_000225.jpg
real/firetruck/real_116_000234.jpg
real/firetruck/real_116_000236.jpg
real/firetruck/real_116_000248.jpg
real/firetruck/real_116_000263.jpg
real/firetruck/real_116_000271.jpg
real/firetruck/real_116_000310.jpg
real/firetruck/real_116_000321.jpg
real/firetruck/real_116_000326.jpg
real/firetruck/real_116_000351.jpg
real/firetruck/real_116_000368.jpg
real/firetruck/real_116_000374.jpg
real/firetruck/real_116_000401.jpg
real/firetruck/real_116_000407.jpg
real/firetruck/real_116_000434.jpg
real/firetruck/real_116_000454.jpg
real/firetruck/real_116_000473.jpg
real/firetruck/real_116_000475.jpg
real/firetruck/real_116_000479.jpg
real/firetruck/real_116_000481.jpg
real/firetruck/real_116_000490.jpg
real/firetruck/real_116_000520.jpg
real/firetruck/real_116_000522.jpg
real/firetruck/real_116_000523.jpg
real/firetruck/real_116_000537.jpg
real/fish/real_117_000172.jpg
real/fish/real_117_000326.jpg
real/fish/real_117_000395.jpg
real/fish/real_117_000412.jpg
real/flamingo/real_118_000058.jpg
real/flamingo/real_118_000099.jpg
real/flamingo/real_118_000110.jpg
real/flamingo/real_118_000247.jpg
real/flamingo/real_118_000272.jpg
real/flamingo/real_118_000301.jpg
real/flamingo/real_118_000316.jpg
real/flamingo/real_118_000321.jpg
real/flamingo/real_118_000327.jpg
real/flamingo/real_118_000354.jpg
real/flamingo/real_118_000400.jpg
real/flamingo/real_118_000422.jpg
real/flamingo/real_118_000426.jpg
real/flamingo/real_118_000448.jpg
real/flamingo/real_118_000516.jpg
real/flamingo/real_118_000520.jpg
real/flamingo/real_118_000542.jpg
real/flashlight/real_119_000263.jpg
real/flashlight/real_119_000337.jpg
real/flip_flops/real_120_000048.jpg
real/flip_flops/real_120_000072.jpg
real/flip_flops/real_120_000105.jpg
real/flip_flops/real_120_000166.jpg
real/flip_flops/real_120_000190.jpg
real/flip_flops/real_120_000268.jpg
real/flip_flops/real_120_000288.jpg
real/flip_flops/real_120_000421.jpg
real/flip_flops/real_120_000443.jpg
real/flower/real_122_000161.jpg
real/flower/real_122_000319.jpg
real/flying_saucer/real_123_000008.jpg
real/flying_saucer/real_123_000052.jpg
real/flying_saucer/real_123_000062.jpg
real/flying_saucer/real_123_000074.jpg
real/flying_saucer/real_123_000085.jpg
real/flying_saucer/real_123_000088.jpg
real/flying_saucer/real_123_000092.jpg
real/flying_saucer/real_123_000095.jpg
real/flying_saucer/real_123_000099.jpg
real/flying_saucer/real_123_000103.jpg
real/flying_saucer/real_123_000117.jpg
real/flying_saucer/real_123_000167.jpg
real/flying_saucer/real_123_000172.jpg
real/flying_saucer/real_123_000200.jpg
real/flying_saucer/real_123_000211.jpg
real/flying_saucer/real_123_000214.jpg
real/flying_saucer/real_123_000254.jpg
real/flying_saucer/real_123_000288.jpg
real/flying_saucer/real_123_000299.jpg
real/flying_saucer/real_123_000349.jpg
real/flying_saucer/real_123_000350.jpg
real/flying_saucer/real_123_000364.jpg
real/flying_saucer/real_123_000392.jpg
real/foot/real_124_000033.jpg
real/foot/real_124_000077.jpg
real/foot/real_124_000520.jpg
real/foot/real_124_000543.jpg
real/fork/real_125_000029.jpg
real/fork/real_125_000143.jpg
real/fork/real_125_000157.jpg
real/fork/real_125_000162.jpg
real/fork/real_125_000170.jpg
real/fork/real_125_000220.jpg
real/fork/real_125_000230.jpg
real/fork/real_125_000305.jpg
real/garden_hose/real_129_000012.jpg
real/garden_hose/real_129_000089.jpg
real/garden_hose/real_129_000097.jpg
real/garden_hose/real_129_000180.jpg
real/garden_hose/real_129_000201.jpg
real/garden_hose/real_129_000247.jpg
real/garden_hose/real_129_000327.jpg
real/garden_hose/real_129_000355.jpg
real/garden_hose/real_129_000360.jpg
real/garden_hose/real_129_000380.jpg
real/giraffe/real_130_000230.jpg
real/giraffe/real_130_000408.jpg
real/giraffe/real_130_000496.jpg
real/goatee/real_131_000152.jpg
real/goatee/real_131_000224.jpg
real/goatee/real_131_000247.jpg
real/goatee/real_131_000333.jpg
real/goatee/real_131_000341.jpg
real/goatee/real_131_000367.jpg
real/goatee/real_131_000442.jpg
real/goatee/real_131_000546.jpg
real/golf_club/real_132_000097.jpg
real/golf_club/real_132_000110.jpg
real/golf_club/real_132_000178.jpg
real/golf_club/real_132_000181.jpg
real/golf_club/real_132_000187.jpg
real/golf_club/real_132_000190.jpg
real/golf_club/real_132_000191.jpg
real/golf_club/real_132_000194.jpg
real/golf_club/real_132_000195.jpg
real/golf_club/real_132_000198.jpg
real/golf_club/real_132_000224.jpg
real/golf_club/real_132_000229.jpg
real/golf_club/real_132_000268.jpg
real/golf_club/real_132_000276.jpg
real/golf_club/real_132_000280.jpg
real/golf_club/real_132_000282.jpg
real/golf_club/real_132_000285.jpg
real/golf_club/real_132_000289.jpg
real/golf_club/real_132_000291.jpg
real/golf_club/real_132_000293.jpg
real/golf_club/real_132_000296.jpg
real/golf_club/real_132_000299.jpg
real/golf_club/real_132_000303.jpg
real/golf_club/real_132_000342.jpg
real/golf_club/real_132_000381.jpg
real/golf_club/real_132_000443.jpg
real/golf_club/real_132_000446.jpg
real/golf_club/real_132_000465.jpg
real/golf_club/real_132_000482.jpg
real/golf_club/real_132_000490.jpg
real/golf_club/real_132_000510.jpg
real/golf_club/real_132_000536.jpg
real/golf_club/real_132_000541.jpg
real/grapes/real_133_000405.jpg
real/grapes/real_133_000463.jpg
real/grapes/real_133_000490.jpg
real/grapes/real_133_000519.jpg
real/grapes/real_133_000554.jpg
real/grapes/real_133_000637.jpg
real/grass/real_134_000050.jpg
real/grass/real_134_000087.jpg
real/grass/real_134_000111.jpg
real/grass/real_134_000129.jpg
real/grass/real_134_000167.jpg
real/grass/real_134_000186.jpg
real/grass/real_134_000236.jpg
real/grass/real_134_000295.jpg
real/grass/real_134_000342.jpg
real/guitar/real_135_000043.jpg
real/guitar/real_135_000279.jpg
real/guitar/real_135_000496.jpg
real/guitar/real_135_000527.jpg
real/guitar/real_135_000562.jpg
real/guitar/real_135_000582.jpg
real/hamburger/real_136_000163.jpg
real/hammer/real_137_000079.jpg
real/hammer/real_137_000151.jpg
real/hammer/real_137_000170.jpg
real/hammer/real_137_000204.jpg
real/hammer/real_137_000253.jpg
real/hammer/real_137_000271.jpg
real/hand/real_138_000006.jpg
real/hand/real_138_000011.jpg
real/hand/real_138_000034.jpg
real/hand/real_138_000038.jpg
real/hand/real_138_000041.jpg
real/hand/real_138_000078.jpg
real/hand/real_138_000094.jpg
real/hand/real_138_000113.jpg
real/hand/real_138_000116.jpg
real/hand/real_138_000157.jpg
real/hand/real_138_000205.jpg
real/hand/real_138_000210.jpg
real/hand/real_138_000227.jpg
real/hand/real_138_000228.jpg
real/hand/real_138_000246.jpg
real/hand/real_138_000247.jpg
real/hand/real_138_000251.jpg
real/hand/real_138_000252.jpg
real/hand/real_138_000282.jpg
real/hand/real_138_000286.jpg
real/hand/real_138_000292.jpg
real/hand/real_138_000297.jpg
real/hand/real_138_000299.jpg
real/hand/real_138_000301.jpg
real/hand/real_138_000309.jpg
real/hand/real_138_000310.jpg
real/hand/real_138_000357.jpg
real/hand/real_138_000371.jpg
real/hand/real_138_000394.jpg
real/hand/real_138_000421.jpg
real/hand/real_138_000422.jpg
real/hand/real_138_000459.jpg
real/hand/real_138_000474.jpg
real/hand/real_138_000485.jpg
real/hand/real_138_000490.jpg
real/hand/real_138_000520.jpg
real/hand/real_138_000535.jpg
real/hand/real_138_000538.jpg
real/hand/real_138_000540.jpg
real/hand/real_138_000543.jpg
real/hand/real_138_000557.jpg
real/hand/real_138_000560.jpg
real/harp/real_139_000038.jpg
real/harp/real_139_000051.jpg
real/harp/real_139_000062.jpg
real/harp/real_139_000087.jpg
real/harp/real_139_000290.jpg
real/harp/real_139_000320.jpg
real/harp/real_139_000479.jpg
real/harp/real_139_000623.jpg
real/hat/real_140_000178.jpg
real/hat/real_140_000242.jpg
real/hat/real_140_000301.jpg
real/hat/real_140_000311.jpg
real/hat/real_140_000405.jpg
real/headphones/real_141_000320.jpg
real/headphones/real_141_000326.jpg
real/headphones/real_141_000410.jpg
real/headphones/real_141_000430.jpg
real/hedgehog/real_142_000044.jpg
real/hedgehog/real_142_000095.jpg
real/helicopter/real_143_000015.jpg
real/helicopter/real_143_000689.jpg
real/helmet/real_144_000322.jpg
real/helmet/real_144_000421.jpg
real/helmet/real_144_000476.jpg
real/hexagon/real_145_000043.jpg
real/hexagon/real_145_000064.jpg
real/hexagon/real_145_000078.jpg
real/hexagon/real_145_000096.jpg
real/hexagon/real_145_000121.jpg
real/hexagon/real_145_000188.jpg
real/hexagon/real_145_000190.jpg
real/hexagon/real_145_000255.jpg
real/hexagon/real_145_000261.jpg
real/hexagon/real_145_000321.jpg
real/hexagon/real_145_000384.jpg
real/hexagon/real_145_000421.jpg
real/hexagon/real_145_000422.jpg
real/hexagon/real_145_000437.jpg
real/hexagon/real_145_000464.jpg
real/hexagon/real_145_000501.jpg
real/hexagon/real_145_000526.jpg
real/hexagon/real_145_000540.jpg
real/hockey_puck/real_146_000030.jpg
real/hockey_puck/real_146_000057.jpg
real/hockey_puck/real_146_000093.jpg
real/hockey_puck/real_146_000118.jpg
real/hockey_puck/real_146_000119.jpg
real/hockey_puck/real_146_000122.jpg
real/hockey_puck/real_146_000130.jpg
real/hockey_puck/real_146_000132.jpg
real/hockey_puck/real_146_000144.jpg
real/hockey_puck/real_146_000178.jpg
real/hockey_puck/real_146_000182.jpg
real/hockey_puck/real_146_000204.jpg
real/hockey_puck/real_146_000232.jpg
real/hockey_puck/real_146_000234.jpg
real/hockey_puck/real_146_000239.jpg
real/hockey_puck/real_146_000245.jpg
real/hockey_puck/real_146_000247.jpg
real/hockey_puck/real_146_000248.jpg
real/hockey_puck/real_146_000259.jpg
real/hockey_puck/real_146_000260.jpg
real/hockey_puck/real_146_000266.jpg
real/hockey_puck/real_146_000279.jpg
real/hockey_puck/real_146_000311.jpg
real/hockey_puck/real_146_000319.jpg
real/hockey_puck/real_146_000329.jpg
real/hockey_puck/real_146_000348.jpg
real/hockey_puck/real_146_000377.jpg
real/hockey_puck/real_146_000382.jpg
real/hockey_puck/real_146_000384.jpg
real/hockey_stick/real_147_000023.jpg
real/hockey_stick/real_147_000024.jpg
real/hockey_stick/real_147_000032.jpg
real/hockey_stick/real_147_000042.jpg
real/hockey_stick/real_147_000049.jpg
real/hockey_stick/real_147_000051.jpg
real/hockey_stick/real_147_000075.jpg
real/hockey_stick/real_147_000080.jpg
real/hockey_stick/real_147_000088.jpg
real/hockey_stick/real_147_000103.jpg
real/hockey_stick/real_147_000114.jpg
real/hockey_stick/real_147_000126.jpg
real/hockey_stick/real_147_000129.jpg
real/hockey_stick/real_147_000141.jpg
real/hockey_stick/real_147_000148.jpg
real/hockey_stick/real_147_000155.jpg
real/hockey_stick/real_147_000181.jpg
real/hockey_stick/real_147_000188.jpg
real/hockey_stick/real_147_000193.jpg
real/hockey_stick/real_147_000199.jpg
real/hockey_stick/real_147_000213.jpg
real/hockey_stick/real_147_000221.jpg
real/hockey_stick/real_147_000241.jpg
real/hockey_stick/real_147_000243.jpg
real/hockey_stick/real_147_000287.jpg
real/hockey_stick/real_147_000296.jpg
real/hockey_stick/real_147_000308.jpg
real/hockey_stick/real_147_000321.jpg
real/hockey_stick/real_147_000322.jpg
real/hockey_stick/real_147_000352.jpg
real/hockey_stick/real_147_000355.jpg
real/hockey_stick/real_147_000359.jpg
real/hockey_stick/real_147_000362.jpg
real/hockey_stick/real_147_000366.jpg
real/hockey_stick/real_147_000379.jpg
real/hockey_stick/real_147_000388.jpg
real/hockey_stick/real_147_000392.jpg
real/horse/real_148_000081.jpg
real/horse/real_148_000092.jpg
real/horse/real_148_000110.jpg
real/horse/real_148_000243.jpg
real/horse/real_148_000247.jpg
real/horse/real_148_000280.jpg
real/horse/real_148_000338.jpg
real/horse/real_148_000341.jpg
real/horse/real_148_000344.jpg
real/horse/real_148_000365.jpg
real/horse/real_148_000366.jpg
real/horse/real_148_000438.jpg
real/horse/real_148_000439.jpg
real/horse/real_148_000441.jpg
real/horse/real_148_000442.jpg
real/horse/real_148_000443.jpg
real/horse/real_148_000447.jpg
real/horse/real_148_000448.jpg
real/horse/real_148_000452.jpg
real/horse/real_148_000453.jpg
real/horse/real_148_000455.jpg
real/horse/real_148_000457.jpg
real/horse/real_148_000459.jpg
real/horse/real_148_000460.jpg
real/horse/real_148_000461.jpg
real/horse/real_148_000462.jpg
real/horse/real_148_000463.jpg
real/horse/real_148_000464.jpg
real/horse/real_148_000465.jpg
real/horse/real_148_000468.jpg
real/horse/real_148_000469.jpg
real/horse/real_148_000470.jpg
real/horse/real_148_000471.jpg
real/horse/real_148_000473.jpg
real/horse/real_148_000477.jpg
real/horse/real_148_000480.jpg
real/horse/real_148_000487.jpg
real/horse/real_148_000488.jpg
real/horse/real_148_000491.jpg
real/horse/real_148_000493.jpg
real/horse/real_148_000499.jpg
real/horse/real_148_000526.jpg
real/horse/real_148_000567.jpg
real/horse/real_148_000568.jpg
real/horse/real_148_000569.jpg
real/horse/real_148_000570.jpg
real/horse/real_148_000608.jpg
real/horse/real_148_000624.jpg
real/horse/real_148_000625.jpg
real/horse/real_148_000629.jpg
real/hospital/real_149_000197.jpg
real/hospital/real_149_000200.jpg
real/hospital/real_149_000224.jpg
real/hospital/real_149_000318.jpg
real/hospital/real_149_000390.jpg
real/hospital/real_149_000517.jpg
real/hospital/real_149_000668.jpg
real/hot_air_balloon/real_150_000094.jpg
real/hot_air_balloon/real_150_000130.jpg
real/hot_air_balloon/real_150_000141.jpg
real/hot_air_balloon/real_150_000237.jpg
real/hot_air_balloon/real_150_000261.jpg
real/hot_air_balloon/real_150_000357.jpg
real/hot_air_balloon/real_150_000378.jpg
real/hot_air_balloon/real_150_000485.jpg
real/hot_air_balloon/real_150_000569.jpg
real/hot_air_balloon/real_150_000649.jpg
real/hot_air_balloon/real_150_000654.jpg
real/hot_air_balloon/real_150_000720.jpg
real/hot_dog/real_151_000032.jpg
real/hot_dog/real_151_000045.jpg
real/hot_dog/real_151_000208.jpg
real/hot_dog/real_151_000248.jpg
real/hot_dog/real_151_000509.jpg
real/hot_dog/real_151_000563.jpg
real/hot_tub/real_152_000244.jpg
real/hot_tub/real_152_000318.jpg
real/hot_tub/real_152_000460.jpg
real/hot_tub/real_152_000719.jpg
real/hot_tub/real_152_000743.jpg
real/hourglass/real_153_000010.jpg
real/hourglass/real_153_000064.jpg
real/hourglass/real_153_000115.jpg
real/hourglass/real_153_000118.jpg
real/hourglass/real_153_000122.jpg
real/hourglass/real_153_000160.jpg
real/hourglass/real_153_000201.jpg
real/hourglass/real_153_000211.jpg
real/hourglass/real_153_000212.jpg
real/hourglass/real_153_000226.jpg
real/hourglass/real_153_000232.jpg
real/house_plant/real_155_000192.jpg
real/ice_cream/real_157_000075.jpg
real/ice_cream/real_157_000392.jpg
real/ice_cream/real_157_000461.jpg
real/ice_cream/real_157_000648.jpg
real/jacket/real_158_000162.jpg
real/jail/real_159_000224.jpg
real/kangaroo/real_160_000420.jpg
real/kangaroo/real_160_000426.jpg
real/kangaroo/real_160_000581.jpg
real/key/real_161_000018.jpg
real/key/real_161_000024.jpg
real/key/real_161_000026.jpg
real/key/real_161_000061.jpg
real/key/real_161_000063.jpg
real/key/real_161_000068.jpg
real/key/real_161_000080.jpg
real/key/real_161_000118.jpg
real/key/real_161_000131.jpg
real/key/real_161_000142.jpg
real/key/real_161_000148.jpg
real/key/real_161_000167.jpg
real/key/real_161_000197.jpg
real/key/real_161_000216.jpg
real/keyboard/real_162_000120.jpg
real/keyboard/real_162_000495.jpg
real/knee/real_163_000083.jpg
real/knee/real_163_000120.jpg
real/knee/real_163_000256.jpg
real/knee/real_163_000293.jpg
real/knee/real_163_000479.jpg
real/knife/real_164_000035.jpg
real/knife/real_164_000140.jpg
real/knife/real_164_000455.jpg
real/ladder/real_165_000137.jpg
real/ladder/real_165_000171.jpg
real/ladder/real_165_000221.jpg
real/ladder/real_165_000241.jpg
real/ladder/real_165_000274.jpg
real/ladder/real_165_000355.jpg
real/ladder/real_165_000415.jpg
real/lantern/real_166_000023.jpg
real/lantern/real_166_000068.jpg
real/lantern/real_166_000097.jpg
real/lantern/real_166_000210.jpg
real/lantern/real_166_000254.jpg
real/lantern/real_166_000309.jpg
real/lantern/real_166_000312.jpg
real/lantern/real_166_000417.jpg
real/laptop/real_167_000001.jpg
real/laptop/real_167_000025.jpg
real/laptop/real_167_000035.jpg
real/laptop/real_167_000099.jpg
real/laptop/real_167_000136.jpg
real/laptop/real_167_000211.jpg
real/laptop/real_167_000286.jpg
real/laptop/real_167_000341.jpg
real/laptop/real_167_000353.jpg
real/laptop/real_167_000367.jpg
real/laptop/real_167_000375.jpg
real/leaf/real_168_000072.jpg
real/leaf/real_168_000162.jpg
real/leaf/real_168_000229.jpg
real/leaf/real_168_000268.jpg
real/leaf/real_168_000275.jpg
real/leaf/real_168_000276.jpg
real/leaf/real_168_000349.jpg
real/leg/real_169_000035.jpg
real/leg/real_169_000055.jpg
real/leg/real_169_000200.jpg
real/leg/real_169_000210.jpg
real/leg/real_169_000269.jpg
real/leg/real_169_000304.jpg
real/leg/real_169_000327.jpg
real/leg/real_169_000350.jpg
real/leg/real_169_000352.jpg
real/leg/real_169_000401.jpg
real/leg/real_169_000457.jpg
real/light_bulb/real_170_000004.jpg
real/light_bulb/real_170_000010.jpg
real/light_bulb/real_170_000129.jpg
real/light_bulb/real_170_000153.jpg
real/light_bulb/real_170_000242.jpg
real/lighter/real_171_000514.jpg
real/lighthouse/real_172_000189.jpg
real/lighthouse/real_172_000250.jpg
real/lighthouse/real_172_000392.jpg
real/lighthouse/real_172_000401.jpg
real/lighthouse/real_172_000471.jpg
real/lighthouse/real_172_000590.jpg
real/lighthouse/real_172_000627.jpg
real/lighthouse/real_172_000628.jpg
real/lightning/real_173_000337.jpg
real/line/real_174_000001.jpg
real/line/real_174_000006.jpg
real/line/real_174_000043.jpg
real/line/real_174_000046.jpg
real/line/real_174_000073.jpg
real/lobster/real_177_000459.jpg
real/lollipop/real_178_000153.jpg
real/lollipop/real_178_000192.jpg
real/lollipop/real_178_000253.jpg
real/lollipop/real_178_000379.jpg
real/lollipop/real_178_000482.jpg
real/mailbox/real_179_000005.jpg
real/map/real_180_000141.jpg
real/map/real_180_000488.jpg
real/map/real_180_000490.jpg
real/marker/real_181_000025.jpg
real/marker/real_181_000058.jpg
real/marker/real_181_000084.jpg
real/marker/real_181_000113.jpg
real/marker/real_181_000127.jpg
real/marker/real_181_000156.jpg
real/marker/real_181_000184.jpg
real/marker/real_181_000243.jpg
real/marker/real_181_000260.jpg
real/marker/real_181_000330.jpg
real/matches/real_182_000045.jpg
real/megaphone/real_183_000014.jpg
real/megaphone/real_183_000046.jpg
real/megaphone/real_183_000058.jpg
real/megaphone/real_183_000069.jpg
real/megaphone/real_183_000133.jpg
real/megaphone/real_183_000156.jpg
real/megaphone/real_183_000169.jpg
real/megaphone/real_183_000173.jpg
real/megaphone/real_183_000191.jpg
real/megaphone/real_183_000195.jpg
real/megaphone/real_183_000237.jpg
real/megaphone/real_183_000249.jpg
real/megaphone/real_183_000255.jpg
real/megaphone/real_183_000262.jpg
real/megaphone/real_183_000263.jpg
real/megaphone/real_183_000272.jpg
real/megaphone/real_183_000286.jpg
real/megaphone/real_183_000317.jpg
real/megaphone/real_183_000319.jpg
real/megaphone/real_183_000325.jpg
real/megaphone/real_183_000351.jpg
real/megaphone/real_183_000358.jpg
real/megaphone/real_183_000374.jpg
real/megaphone/real_183_000406.jpg
real/megaphone/real_183_000413.jpg
real/megaphone/real_183_000420.jpg
real/megaphone/real_183_000432.jpg
real/megaphone/real_183_000442.jpg
real/megaphone/real_183_000443.jpg
real/megaphone/real_183_000454.jpg
real/megaphone/real_183_000461.jpg
real/megaphone/real_183_000464.jpg
real/megaphone/real_183_000469.jpg
real/megaphone/real_183_000479.jpg
real/megaphone/real_183_000491.jpg
real/megaphone/real_183_000527.jpg
real/megaphone/real_183_000528.jpg
real/megaphone/real_183_000539.jpg
real/megaphone/real_183_000544.jpg
real/mermaid/real_184_000342.jpg
real/mermaid/real_184_000371.jpg
real/mermaid/real_184_000378.jpg
real/microphone/real_185_000104.jpg
real/microphone/real_185_000211.jpg
real/microphone/real_185_000339.jpg
real/microphone/real_185_000359.jpg
real/microphone/real_185_000362.jpg
real/microphone/real_185_000376.jpg
real/microphone/real_185_000383.jpg
real/microphone/real_185_000451.jpg
real/microphone/real_185_000469.jpg
real/microphone/real_185_000473.jpg
real/moon/real_188_000312.jpg
real/moon/real_188_000367.jpg
real/mosquito/real_189_000008.jpg
real/mosquito/real_189_000451.jpg
real/mosquito/real_189_000514.jpg
real/mountain/real_191_000462.jpg
real/mountain/real_191_000476.jpg
real/mountain/real_191_000696.jpg
real/mouse/real_192_000007.jpg
real/mouse/real_192_000061.jpg
real/mouse/real_192_000089.jpg
real/mouse/real_192_000090.jpg
real/mouse/real_192_000091.jpg
real/mouse/real_192_000092.jpg
real/mouse/real_192_000093.jpg
real/mouse/real_192_000094.jpg
real/mouse/real_192_000097.jpg
real/mouse/real_192_000101.jpg
real/mouse/real_192_000107.jpg
real/mouse/real_192_000117.jpg
real/mouse/real_192_000139.jpg
real/moustache/real_193_000009.jpg
real/moustache/real_193_000015.jpg
real/moustache/real_193_000045.jpg
real/moustache/real_193_000074.jpg
real/moustache/real_193_000119.jpg
real/moustache/real_193_000148.jpg
real/moustache/real_193_000172.jpg
real/moustache/real_193_000185.jpg
real/moustache/real_193_000198.jpg
real/moustache/real_193_000215.jpg
real/moustache/real_193_000221.jpg
real/moustache/real_193_000252.jpg
real/moustache/real_193_000279.jpg
real/moustache/real_193_000301.jpg
real/moustache/real_193_000318.jpg
real/moustache/real_193_000321.jpg
real/moustache/real_193_000368.jpg
real/moustache/real_193_000406.jpg
real/moustache/real_193_000410.jpg
real/moustache/real_193_000415.jpg
real/mouth/real_194_000041.jpg
real/mouth/real_194_000048.jpg
real/mouth/real_194_000058.jpg
real/mouth/real_194_000076.jpg
real/mouth/real_194_000112.jpg
real/mouth/real_194_000114.jpg
real/mouth/real_194_000131.jpg
real/mouth/real_194_000139.jpg
real/mouth/real_194_000147.jpg
real/mouth/real_194_000152.jpg
real/mouth/real_194_000156.jpg
real/mouth/real_194_000170.jpg
real/mouth/real_194_000187.jpg
real/mouth/real_194_000203.jpg
real/mouth/real_194_000207.jpg
real/mouth/real_194_000237.jpg
real/mouth/real_194_000256.jpg
real/mouth/real_194_000285.jpg
real/mouth/real_194_000326.jpg
real/mouth/real_194_000368.jpg
real/mug/real_195_000014.jpg
real/mug/real_195_000080.jpg
real/mug/real_195_000082.jpg
real/mug/real_195_000098.jpg
real/mug/real_195_000108.jpg
real/mug/real_195_000109.jpg
real/mug/real_195_000113.jpg
real/mug/real_195_000137.jpg
real/mug/real_195_000138.jpg
real/mug/real_195_000146.jpg
real/mug/real_195_000154.jpg
real/mug/real_195_000183.jpg
real/mug/real_195_000186.jpg
real/mug/real_195_000203.jpg
real/mug/real_195_000205.jpg
real/mug/real_195_000213.jpg
real/mug/real_195_000242.jpg
real/mug/real_195_000251.jpg
real/mug/real_195_000272.jpg
real/mug/real_195_000283.jpg
real/mug/real_195_000300.jpg
real/mug/real_195_000306.jpg
real/mug/real_195_000320.jpg
real/mug/real_195_000345.jpg
real/mug/real_195_000373.jpg
real/mug/real_195_000376.jpg
real/mug/real_195_000379.jpg
real/mug/real_195_000384.jpg
real/mug/real_195_000420.jpg
real/mug/real_195_000444.jpg
real/mug/real_195_000462.jpg
real/mug/real_195_000463.jpg
real/mug/real_195_000474.jpg
real/mug/real_195_000496.jpg
real/mug/real_195_000501.jpg
real/mug/real_195_000503.jpg
real/mug/real_195_000564.jpg
real/mug/real_195_000592.jpg
real/mug/real_195_000595.jpg
real/mug/real_195_000596.jpg
real/mug/real_195_000598.jpg
real/mushroom/real_196_000104.jpg
real/mushroom/real_196_000591.jpg
real/mushroom/real_196_000628.jpg
real/nail/real_197_000072.jpg
real/nail/real_197_000091.jpg
real/nail/real_197_000101.jpg
real/nail/real_197_000110.jpg
real/nail/real_197_000179.jpg
real/nail/real_197_000183.jpg
real/nail/real_197_000187.jpg
real/nail/real_197_000188.jpg
real/nail/real_197_000201.jpg
real/nail/real_197_000210.jpg
real/nail/real_197_000288.jpg
real/nail/real_197_000338.jpg
real/nail/real_197_000354.jpg
real/nail/real_197_000357.jpg
real/nail/real_197_000450.jpg
real/nail/real_197_000596.jpg
real/nail/real_197_000606.jpg
real/nail/real_197_000623.jpg
real/nail/real_197_000643.jpg
real/nail/real_197_000649.jpg
real/nail/real_197_000659.jpg
real/nail/real_197_000660.jpg
real/necklace/real_198_000057.jpg
real/necklace/real_198_000063.jpg
real/necklace/real_198_000158.jpg
real/necklace/real_198_000200.jpg
real/necklace/real_198_000408.jpg
real/necklace/real_198_000644.jpg
real/nose/real_199_000018.jpg
real/nose/real_199_000066.jpg
real/nose/real_199_000096.jpg
real/nose/real_199_000133.jpg
real/nose/real_199_000188.jpg
real/nose/real_199_000237.jpg
real/ocean/real_200_000159.jpg
real/ocean/real_200_000224.jpg
real/ocean/real_200_000379.jpg
real/octagon/real_201_000001.jpg
real/octagon/real_201_000006.jpg
real/octagon/real_201_000012.jpg
real/octagon/real_201_000041.jpg
real/octagon/real_201_000045.jpg
real/octagon/real_201_000060.jpg
real/octagon/real_201_000082.jpg
real/octagon/real_201_000205.jpg
real/octagon/real_201_000217.jpg
real/octagon/real_201_000280.jpg
real/octagon/real_201_000301.jpg
real/octagon/real_201_000340.jpg
real/octagon/real_201_000349.jpg
real/octagon/real_201_000464.jpg
real/octopus/real_202_000010.jpg
real/octopus/real_202_000018.jpg
real/octopus/real_202_000086.jpg
real/octopus/real_202_000144.jpg
real/octopus/real_202_000352.jpg
real/octopus/real_202_000457.jpg
real/octopus/real_202_000533.jpg
real/octopus/real_202_000606.jpg
real/octopus/real_202_000622.jpg
real/onion/real_203_000025.jpg
real/onion/real_203_000034.jpg
real/onion/real_203_000284.jpg
real/onion/real_203_000345.jpg
real/oven/real_204_000019.jpg
real/oven/real_204_000118.jpg
real/oven/real_204_000158.jpg
real/oven/real_204_000217.jpg
real/oven/real_204_000287.jpg
real/oven/real_204_000296.jpg
real/oven/real_204_000330.jpg
real/owl/real_205_000127.jpg
real/owl/real_205_000261.jpg
real/owl/real_205_000291.jpg
real/owl/real_205_000408.jpg
real/owl/real_205_000446.jpg
real/owl/real_205_000519.jpg
real/owl/real_205_000532.jpg
real/owl/real_205_000650.jpg
real/owl/real_205_000757.jpg
real/paintbrush/real_206_000001.jpg
real/paintbrush/real_206_000012.jpg
real/paintbrush/real_206_000020.jpg
real/paintbrush/real_206_000025.jpg
real/paintbrush/real_206_000028.jpg
real/paintbrush/real_206_000041.jpg
real/paintbrush/real_206_000042.jpg
real/paintbrush/real_206_000043.jpg
real/paintbrush/real_206_000046.jpg
real/paintbrush/real_206_000048.jpg
real/paintbrush/real_206_000052.jpg
real/paintbrush/real_206_000057.jpg
real/paintbrush/real_206_000059.jpg
real/paintbrush/real_206_000064.jpg
real/paintbrush/real_206_000085.jpg
real/paintbrush/real_206_000105.jpg
real/paintbrush/real_206_000109.jpg
real/paintbrush/real_206_000112.jpg
real/paintbrush/real_206_000113.jpg
real/paintbrush/real_206_000116.jpg
real/paintbrush/real_206_000120.jpg
real/paintbrush/real_206_000123.jpg
real/paintbrush/real_206_000125.jpg
real/paintbrush/real_206_000132.jpg
real/paintbrush/real_206_000135.jpg
real/paintbrush/real_206_000153.jpg
real/paintbrush/real_206_000154.jpg
real/paintbrush/real_206_000160.jpg
real/paintbrush/real_206_000171.jpg
real/paintbrush/real_206_000180.jpg
real/paintbrush/real_206_000189.jpg
real/paintbrush/real_206_000250.jpg
real/paintbrush/real_206_000258.jpg
real/paintbrush/real_206_000260.jpg
real/paintbrush/real_206_000268.jpg
real/paintbrush/real_206_000298.jpg
real/paintbrush/real_206_000315.jpg
real/paintbrush/real_206_000320.jpg
real/paintbrush/real_206_000321.jpg
real/paintbrush/real_206_000359.jpg
real/paintbrush/real_206_000369.jpg
real/paintbrush/real_206_000384.jpg
real/paintbrush/real_206_000385.jpg
real/paintbrush/real_206_000388.jpg
real/paintbrush/real_206_000390.jpg
real/paintbrush/real_206_000401.jpg
real/paint_can/real_207_000034.jpg
real/paint_can/real_207_000036.jpg
real/paint_can/real_207_000065.jpg
real/paint_can/real_207_000108.jpg
real/paint_can/real_207_000157.jpg
real/paint_can/real_207_000175.jpg
real/paint_can/real_207_000183.jpg
real/paint_can/real_207_000193.jpg
real/paint_can/real_207_000201.jpg
real/paint_can/real_207_000208.jpg
real/paint_can/real_207_000225.jpg
real/paint_can/real_207_000257.jpg
real/paint_can/real_207_000266.jpg
real/paint_can/real_207_000278.jpg
real/paint_can/real_207_000279.jpg
real/paint_can/real_207_000313.jpg
real/paint_can/real_207_000330.jpg
real/paint_can/real_207_000347.jpg
real/paint_can/real_207_000368.jpg
real/paint_can/real_207_000384.jpg
real/paint_can/real_207_000408.jpg
real/paint_can/real_207_000437.jpg
real/paint_can/real_207_000467.jpg
real/paint_can/real_207_000480.jpg
real/paint_can/real_207_000481.jpg
real/paint_can/real_207_000482.jpg
real/paint_can/real_207_000543.jpg
real/paint_can/real_207_000551.jpg
real/palm_tree/real_208_000044.jpg
real/palm_tree/real_208_000104.jpg
real/palm_tree/real_208_000113.jpg
real/palm_tree/real_208_000119.jpg
real/palm_tree/real_208_000133.jpg
real/palm_tree/real_208_000138.jpg
real/palm_tree/real_208_000152.jpg
real/palm_tree/real_208_000189.jpg
real/palm_tree/real_208_000213.jpg
real/palm_tree/real_208_000218.jpg
real/palm_tree/real_208_000222.jpg
real/panda/real_209_000112.jpg
real/panda/real_209_000412.jpg
real/panda/real_209_000453.jpg
real/panda/real_209_000457.jpg
real/pants/real_210_000090.jpg
real/pants/real_210_000131.jpg
real/pants/real_210_000254.jpg
real/pants/real_210_000337.jpg
real/paper_clip/real_211_000017.jpg
real/paper_clip/real_211_000026.jpg
real/paper_clip/real_211_000027.jpg
real/paper_clip/real_211_000030.jpg
real/paper_clip/real_211_000038.jpg
real/paper_clip/real_211_000043.jpg
real/paper_clip/real_211_000048.jpg
real/paper_clip/real_211_000049.jpg
real/paper_clip/real_211_000051.jpg
real/paper_clip/real_211_000054.jpg
real/paper_clip/real_211_000081.jpg
real/paper_clip/real_211_000082.jpg
real/paper_clip/real_211_000109.jpg
real/paper_clip/real_211_000110.jpg
real/paper_clip/real_211_000114.jpg
real/paper_clip/real_211_000118.jpg
real/paper_clip/real_211_000119.jpg
real/paper_clip/real_211_000155.jpg
real/paper_clip/real_211_000175.jpg
real/paper_clip/real_211_000185.jpg
real/paper_clip/real_211_000213.jpg
real/paper_clip/real_211_000228.jpg
real/paper_clip/real_211_000269.jpg
real/paper_clip/real_211_000271.jpg
real/paper_clip/real_211_000273.jpg
real/paper_clip/real_211_000274.jpg
real/paper_clip/real_211_000308.jpg
real/paper_clip/real_211_000320.jpg
real/paper_clip/real_211_000358.jpg
real/paper_clip/real_211_000393.jpg
real/paper_clip/real_211_000396.jpg
real/paper_clip/real_211_000422.jpg
real/paper_clip/real_211_000423.jpg
real/paper_clip/real_211_000446.jpg
real/paper_clip/real_211_000447.jpg
real/paper_clip/real_211_000450.jpg
real/paper_clip/real_211_000453.jpg
real/paper_clip/real_211_000479.jpg
real/paper_clip/real_211_000480.jpg
real/paper_clip/real_211_000528.jpg
real/paper_clip/real_211_000533.jpg
real/paper_clip/real_211_000544.jpg
real/parachute/real_212_000108.jpg
real/parachute/real_212_000194.jpg
real/parachute/real_212_000214.jpg
real/parachute/real_212_000225.jpg
real/parachute/real_212_000348.jpg
real/parachute/real_212_000427.jpg
real/parachute/real_212_000445.jpg
real/parachute/real_212_000493.jpg
real/parachute/real_212_000496.jpg
real/parachute/real_212_000544.jpg
real/parachute/real_212_000546.jpg
real/parachute/real_212_000590.jpg
real/parachute/real_212_000615.jpg
real/parrot/real_213_000016.jpg
real/parrot/real_213_000022.jpg
real/parrot/real_213_000052.jpg
real/parrot/real_213_000148.jpg
real/parrot/real_213_000351.jpg
real/parrot/real_213_000545.jpg
real/parrot/real_213_000748.jpg
real/passport/real_214_000294.jpg
real/peanut/real_215_000050.jpg
real/peanut/real_215_000101.jpg
real/peanut/real_215_000148.jpg
real/peanut/real_215_000182.jpg
real/peanut/real_215_000203.jpg
real/peanut/real_215_000273.jpg
real/pear/real_216_000117.jpg
real/pear/real_216_000279.jpg
real/pear/real_216_000295.jpg
real/pear/real_216_000326.jpg
real/peas/real_217_000239.jpg
real/pencil/real_218_000034.jpg
real/pencil/real_218_000244.jpg
real/pencil/real_218_000260.jpg
real/pencil/real_218_000432.jpg
real/penguin/real_219_000028.jpg
real/penguin/real_219_000275.jpg
real/penguin/real_219_000339.jpg
real/penguin/real_219_000478.jpg
real/piano/real_220_000346.jpg
real/piano/real_220_000564.jpg
real/pickup_truck/real_221_000112.jpg
real/picture_frame/real_222_000010.jpg
real/picture_frame/real_222_000024.jpg
real/picture_frame/real_222_000110.jpg
real/picture_frame/real_222_000115.jpg
real/picture_frame/real_222_000182.jpg
real/pig/real_223_000366.jpg
real/pig/real_223_000491.jpg
real/pig/real_223_000497.jpg
real/pig/real_223_000511.jpg
real/pillow/real_224_000095.jpg
real/pillow/real_224_000097.jpg
real/pillow/real_224_000103.jpg
real/pillow/real_224_000318.jpg
real/pillow/real_224_000354.jpg
real/pillow/real_224_000377.jpg
real/pillow/real_224_000389.jpg
real/pillow/real_224_000456.jpg
real/pillow/real_224_000505.jpg
real/pineapple/real_225_000005.jpg
real/pineapple/real_225_000071.jpg
real/pineapple/real_225_000074.jpg
real/pineapple/real_225_000205.jpg
real/pineapple/real_225_000286.jpg
real/pineapple/real_225_000444.jpg
real/pineapple/real_225_000462.jpg
real/pineapple/real_225_000500.jpg
real/pineapple/real_225_000531.jpg
real/pineapple/real_225_000648.jpg
real/pizza/real_226_000362.jpg
real/pliers/real_227_000304.jpg
real/police_car/real_228_000067.jpg
real/police_car/real_228_000101.jpg
real/police_car/real_228_000123.jpg
real/police_car/real_228_000198.jpg
real/police_car/real_228_000262.jpg
real/police_car/real_228_000530.jpg
real/police_car/real_228_000659.jpg
real/popsicle/real_231_000078.jpg
real/popsicle/real_231_000081.jpg
real/popsicle/real_231_000110.jpg
real/popsicle/real_231_000122.jpg
real/popsicle/real_231_000128.jpg
real/popsicle/real_231_000176.jpg
real/popsicle/real_231_000190.jpg
real/popsicle/real_231_000226.jpg
real/popsicle/real_231_000267.jpg
real/popsicle/real_231_000311.jpg
real/popsicle/real_231_000317.jpg
real/popsicle/real_231_000333.jpg
real/popsicle/real_231_000432.jpg
real/popsicle/real_231_000532.jpg
real/popsicle/real_231_000558.jpg
real/popsicle/real_231_000575.jpg
real/popsicle/real_231_000587.jpg
real/postcard/real_232_000038.jpg
real/postcard/real_232_000095.jpg
real/postcard/real_232_000098.jpg
real/postcard/real_232_000219.jpg
real/postcard/real_232_000506.jpg
real/power_outlet/real_234_000005.jpg
real/power_outlet/real_234_000171.jpg
real/power_outlet/real_234_000385.jpg
real/power_outlet/real_234_000402.jpg
real/power_outlet/real_234_000455.jpg
real/rabbit/real_236_000087.jpg
real/rabbit/real_236_000115.jpg
real/rabbit/real_236_000314.jpg
real/rabbit/real_236_000574.jpg
real/rabbit/real_236_000579.jpg
real/raccoon/real_237_000456.jpg
real/radio/real_238_000173.jpg
real/rain/real_239_000022.jpg
real/rain/real_239_000035.jpg
real/rain/real_239_000068.jpg
real/rain/real_239_000097.jpg
real/rain/real_239_000098.jpg
real/rain/real_239_000101.jpg
real/rain/real_239_000102.jpg
real/rain/real_239_000103.jpg
real/rain/real_239_000104.jpg
real/rain/real_239_000105.jpg
real/rain/real_239_000106.jpg
real/rain/real_239_000110.jpg
real/rain/real_239_000111.jpg
real/rain/real_239_000112.jpg
real/rain/real_239_000203.jpg
real/rain/real_239_000262.jpg
real/rain/real_239_000264.jpg
real/rain/real_239_000266.jpg
real/rain/real_239_000267.jpg
real/rain/real_239_000268.jpg
real/rain/real_239_000270.jpg
real/rain/real_239_000271.jpg
real/rain/real_239_000273.jpg
real/rainbow/real_240_000026.jpg
real/rainbow/real_240_000032.jpg
real/rainbow/real_240_000038.jpg
real/rainbow/real_240_000094.jpg
real/rainbow/real_240_000126.jpg
real/rainbow/real_240_000143.jpg
real/rainbow/real_240_000162.jpg
real/rainbow/real_240_000164.jpg
real/rainbow/real_240_000165.jpg
real/rainbow/real_240_000190.jpg
real/rainbow/real_240_000222.jpg
real/rake/real_241_000309.jpg
real/rake/real_241_000378.jpg
real/rake/real_241_000580.jpg
real/remote_control/real_242_000049.jpg
real/remote_control/real_242_000087.jpg
real/remote_control/real_242_000128.jpg
real/remote_control/real_242_000448.jpg
real/rhinoceros/real_243_000230.jpg
real/rhinoceros/real_243_000667.jpg
real/rifle/real_244_000239.jpg
real/river/real_245_000387.jpg
real/river/real_245_000414.jpg
real/roller_coaster/real_246_000256.jpg
real/rollerskates/real_247_000186.jpg
real/rollerskates/real_247_000270.jpg
real/rollerskates/real_247_000339.jpg
real/rollerskates/real_247_000401.jpg
real/rollerskates/real_247_000474.jpg
real/rollerskates/real_247_000482.jpg
real/sailboat/real_248_000180.jpg
real/sailboat/real_248_000183.jpg
real/sailboat/real_248_000314.jpg
real/sailboat/real_248_000358.jpg
real/sailboat/real_248_000359.jpg
real/sandwich/real_249_000022.jpg
real/sandwich/real_249_000133.jpg
real/sandwich/real_249_000144.jpg
real/sandwich/real_249_000178.jpg
real/sandwich/real_249_000328.jpg
real/sandwich/real_249_000431.jpg
real/sandwich/real_249_000444.jpg
real/saxophone/real_251_000056.jpg
real/saxophone/real_251_000085.jpg
real/saxophone/real_251_000109.jpg
real/saxophone/real_251_000114.jpg
real/saxophone/real_251_000197.jpg
real/saxophone/real_251_000292.jpg
real/saxophone/real_251_000359.jpg
real/saxophone/real_251_000363.jpg
real/saxophone/real_251_000414.jpg
real/saxophone/real_251_000424.jpg
real/school_bus/real_252_000013.jpg
real/school_bus/real_252_000023.jpg
real/school_bus/real_252_000024.jpg
real/school_bus/real_252_000037.jpg
real/school_bus/real_252_000043.jpg
real/school_bus/real_252_000052.jpg
real/school_bus/real_252_000063.jpg
real/school_bus/real_252_000066.jpg
real/school_bus/real_252_000067.jpg
real/school_bus/real_252_000068.jpg
real/school_bus/real_252_000074.jpg
real/school_bus/real_252_000087.jpg
real/school_bus/real_252_000114.jpg
real/school_bus/real_252_000116.jpg
real/school_bus/real_252_000137.jpg
real/school_bus/real_252_000164.jpg
real/school_bus/real_252_000178.jpg
real/school_bus/real_252_000183.jpg
real/school_bus/real_252_000197.jpg
real/school_bus/real_252_000206.jpg
real/school_bus/real_252_000218.jpg
real/school_bus/real_252_000254.jpg
real/school_bus/real_252_000275.jpg
real/school_bus/real_252_000292.jpg
real/school_bus/real_252_000302.jpg
real/school_bus/real_252_000312.jpg
real/school_bus/real_252_000315.jpg
real/school_bus/real_252_000333.jpg
real/school_bus/real_252_000335.jpg
real/school_bus/real_252_000356.jpg
real/school_bus/real_252_000381.jpg
real/school_bus/real_252_000459.jpg
real/school_bus/real_252_000464.jpg
real/scissors/real_253_000009.jpg
real/scissors/real_253_000337.jpg
real/scissors/real_253_000371.jpg
real/scissors/real_253_000412.jpg
real/scorpion/real_254_000113.jpg
real/scorpion/real_254_000427.jpg
real/screwdriver/real_255_000295.jpg
real/sea_turtle/real_256_000032.jpg
real/sea_turtle/real_256_000037.jpg
real/sea_turtle/real_256_000226.jpg
real/sea_turtle/real_256_000377.jpg
real/sea_turtle/real_256_000476.jpg
real/sea_turtle/real_256_000531.jpg
real/sea_turtle/real_256_000619.jpg
real/see_saw/real_257_000017.jpg
real/see_saw/real_257_000049.jpg
real/see_saw/real_257_000116.jpg
real/see_saw/real_257_000190.jpg
real/see_saw/real_257_000199.jpg
real/see_saw/real_257_000209.jpg
real/see_saw/real_257_000224.jpg
real/sheep/real_259_000212.jpg
real/sheep/real_259_000393.jpg
real/shovel/real_262_000140.jpg
real/shovel/real_262_000347.jpg
real/shovel/real_262_000430.jpg
real/sink/real_263_000060.jpg
real/skateboard/real_264_000114.jpg
real/skateboard/real_264_000251.jpg
real/skull/real_265_000028.jpg
real/skull/real_265_000080.jpg
real/skull/real_265_000123.jpg
real/skull/real_265_000187.jpg
real/skull/real_265_000213.jpg
real/skull/real_265_000217.jpg
real/skull/real_265_000239.jpg
real/skull/real_265_000254.jpg
real/skull/real_265_000262.jpg
real/skull/real_265_000296.jpg
real/skull/real_265_000304.jpg
real/skull/real_265_000305.jpg
real/sleeping_bag/real_267_000153.jpg
real/smiley_face/real_268_000003.jpg
real/smiley_face/real_268_000004.jpg
real/smiley_face/real_268_000019.jpg
real/smiley_face/real_268_000029.jpg
real/smiley_face/real_268_000034.jpg
real/smiley_face/real_268_000039.jpg
real/smiley_face/real_268_000043.jpg
real/smiley_face/real_268_000054.jpg
real/smiley_face/real_268_000058.jpg
real/smiley_face/real_268_000060.jpg
real/smiley_face/real_268_000069.jpg
real/smiley_face/real_268_000084.jpg
real/smiley_face/real_268_000120.jpg
real/smiley_face/real_268_000129.jpg
real/smiley_face/real_268_000133.jpg
real/smiley_face/real_268_000142.jpg
real/smiley_face/real_268_000143.jpg
real/smiley_face/real_268_000144.jpg
real/smiley_face/real_268_000152.jpg
real/smiley_face/real_268_000155.jpg
real/smiley_face/real_268_000193.jpg
real/snake/real_270_000006.jpg
real/soccer_ball/real_274_000005.jpg
real/soccer_ball/real_274_000037.jpg
real/soccer_ball/real_274_000065.jpg
real/soccer_ball/real_274_000072.jpg
real/soccer_ball/real_274_000073.jpg
real/soccer_ball/real_274_000120.jpg
real/soccer_ball/real_274_000128.jpg
real/soccer_ball/real_274_000130.jpg
real/soccer_ball/real_274_000143.jpg
real/soccer_ball/real_274_000153.jpg
real/soccer_ball/real_274_000170.jpg
real/soccer_ball/real_274_000196.jpg
real/sock/real_275_000014.jpg
real/sock/real_275_000370.jpg
real/sock/real_275_000459.jpg
real/speedboat/real_276_000568.jpg
real/spider/real_277_000016.jpg
real/spoon/real_278_000061.jpg
real/spoon/real_278_000063.jpg
real/spoon/real_278_000179.jpg
real/spoon/real_278_000330.jpg
real/spoon/real_278_000335.jpg
real/spoon/real_278_000383.jpg
real/spoon/real_278_000392.jpg
real/spoon/real_278_000407.jpg
real/spoon/real_278_000420.jpg
real/spoon/real_278_000481.jpg
real/spoon/real_278_000493.jpg
real/spoon/real_278_000519.jpg
real/spreadsheet/real_279_000042.jpg
real/square/real_280_000011.jpg
real/square/real_280_000015.jpg
real/square/real_280_000017.jpg
real/square/real_280_000028.jpg
real/square/real_280_000034.jpg
real/square/real_280_000035.jpg
real/square/real_280_000055.jpg
real/square/real_280_000083.jpg
real/squiggle/real_281_000012.jpg
real/squiggle/real_281_000018.jpg
real/squiggle/real_281_000045.jpg
real/squiggle/real_281_000071.jpg
real/stairs/real_283_000134.jpg
real/stairs/real_283_000239.jpg
real/stethoscope/real_287_000004.jpg
real/stethoscope/real_287_000006.jpg
real/stethoscope/real_287_000016.jpg
real/stethoscope/real_287_000035.jpg
real/stethoscope/real_287_000044.jpg
real/stethoscope/real_287_000122.jpg
real/stethoscope/real_287_000155.jpg
real/stethoscope/real_287_000173.jpg
real/stethoscope/real_287_000238.jpg
real/stethoscope/real_287_000239.jpg
real/stethoscope/real_287_000322.jpg
real/stethoscope/real_287_000343.jpg
real/stethoscope/real_287_000354.jpg
real/stethoscope/real_287_000369.jpg
real/stethoscope/real_287_000387.jpg
real/stethoscope/real_287_000402.jpg
real/stethoscope/real_287_000428.jpg
real/stethoscope/real_287_000435.jpg
real/stethoscope/real_287_000443.jpg
real/stethoscope/real_287_000449.jpg
real/stethoscope/real_287_000459.jpg
real/stethoscope/real_287_000461.jpg
real/stethoscope/real_287_000471.jpg
real/stop_sign/real_289_000138.jpg
real/stop_sign/real_289_000140.jpg
real/stove/real_290_000123.jpg
real/stove/real_290_000350.jpg
real/strawberry/real_291_000381.jpg
real/strawberry/real_291_000448.jpg
real/strawberry/real_291_000453.jpg
real/streetlight/real_292_000012.jpg
real/streetlight/real_292_000114.jpg
real/streetlight/real_292_000160.jpg
real/streetlight/real_292_000300.jpg
real/streetlight/real_292_000349.jpg
real/string_bean/real_293_000182.jpg
real/string_bean/real_293_000230.jpg
real/string_bean/real_293_000484.jpg
real/submarine/real_294_000195.jpg
real/suitcase/real_295_000092.jpg
real/suitcase/real_295_000139.jpg
real/swing_set/real_299_000300.jpg
real/sword/real_300_000266.jpg
real/sword/real_300_000276.jpg
real/sword/real_300_000389.jpg
real/sword/real_300_000400.jpg
real/sword/real_300_000419.jpg
real/sword/real_300_000431.jpg
real/sword/real_300_000440.jpg
real/syringe/real_301_000124.jpg
real/syringe/real_301_000278.jpg
real/syringe/real_301_000315.jpg
real/syringe/real_301_000337.jpg
real/syringe/real_301_000387.jpg
real/syringe/real_301_000547.jpg
real/table/real_302_000151.jpg
real/table/real_302_000233.jpg
real/table/real_302_000274.jpg
real/table/real_302_000433.jpg
real/teapot/real_303_000141.jpg
real/teapot/real_303_000269.jpg
real/teapot/real_303_000298.jpg
real/teapot/real_303_000303.jpg
real/teapot/real_303_000569.jpg
real/teapot/real_303_000573.jpg
real/teapot/real_303_000582.jpg
real/teddy-bear/real_304_000005.jpg
real/teddy-bear/real_304_000097.jpg
real/teddy-bear/real_304_000142.jpg
real/teddy-bear/real_304_000452.jpg
real/telephone/real_305_000084.jpg
real/telephone/real_305_000390.jpg
real/television/real_306_000106.jpg
real/television/real_306_000168.jpg
real/television/real_306_000201.jpg
real/television/real_306_000202.jpg
real/television/real_306_000229.jpg
real/television/real_306_000256.jpg
real/television/real_306_000317.jpg
real/television/real_306_000373.jpg
real/tennis_racquet/real_307_000001.jpg
real/tennis_racquet/real_307_000003.jpg
real/tennis_racquet/real_307_000043.jpg
real/tennis_racquet/real_307_000088.jpg
real/tennis_racquet/real_307_000090.jpg
real/tennis_racquet/real_307_000101.jpg
real/tennis_racquet/real_307_000117.jpg
real/tennis_racquet/real_307_000128.jpg
real/tennis_racquet/real_307_000137.jpg
real/tennis_racquet/real_307_000164.jpg
real/tennis_racquet/real_307_000209.jpg
real/tent/real_308_000433.jpg
real/tent/real_308_000468.jpg
real/tent/real_308_000510.jpg
real/tent/real_308_000521.jpg
real/The_Eiffel_Tower/real_309_000007.jpg
real/The_Eiffel_Tower/real_309_000021.jpg
real/The_Eiffel_Tower/real_309_000062.jpg
real/The_Eiffel_Tower/real_309_000098.jpg
real/The_Eiffel_Tower/real_309_000141.jpg
real/The_Eiffel_Tower/real_309_000145.jpg
real/The_Eiffel_Tower/real_309_000183.jpg
real/The_Eiffel_Tower/real_309_000189.jpg
real/The_Eiffel_Tower/real_309_000190.jpg
real/The_Eiffel_Tower/real_309_000192.jpg
real/The_Eiffel_Tower/real_309_000196.jpg
real/The_Eiffel_Tower/real_309_000199.jpg
real/The_Eiffel_Tower/real_309_000202.jpg
real/The_Eiffel_Tower/real_309_000203.jpg
real/The_Eiffel_Tower/real_309_000205.jpg
real/The_Eiffel_Tower/real_309_000210.jpg
real/The_Eiffel_Tower/real_309_000211.jpg
real/The_Eiffel_Tower/real_309_000216.jpg
real/The_Eiffel_Tower/real_309_000217.jpg
real/The_Eiffel_Tower/real_309_000222.jpg
real/The_Eiffel_Tower/real_309_000239.jpg
real/The_Eiffel_Tower/real_309_000240.jpg
real/The_Eiffel_Tower/real_309_000242.jpg
real/The_Eiffel_Tower/real_309_000250.jpg
real/The_Eiffel_Tower/real_309_000287.jpg
real/The_Eiffel_Tower/real_309_000309.jpg
real/The_Eiffel_Tower/real_309_000345.jpg
real/The_Eiffel_Tower/real_309_000379.jpg
real/The_Eiffel_Tower/real_309_000436.jpg
real/The_Eiffel_Tower/real_309_000438.jpg
real/The_Eiffel_Tower/real_309_000451.jpg
real/The_Eiffel_Tower/real_309_000521.jpg
real/The_Eiffel_Tower/real_309_000527.jpg
real/The_Eiffel_Tower/real_309_000531.jpg
real/The_Eiffel_Tower/real_309_000532.jpg
real/The_Eiffel_Tower/real_309_000534.jpg
real/The_Eiffel_Tower/real_309_000536.jpg
real/The_Eiffel_Tower/real_309_000544.jpg
real/The_Eiffel_Tower/real_309_000549.jpg
real/The_Eiffel_Tower/real_309_000552.jpg
real/The_Great_Wall_of_China/real_310_000044.jpg
real/The_Great_Wall_of_China/real_310_000084.jpg
real/The_Great_Wall_of_China/real_310_000147.jpg
real/The_Great_Wall_of_China/real_310_000158.jpg
real/The_Great_Wall_of_China/real_310_000238.jpg
real/The_Great_Wall_of_China/real_310_000296.jpg
real/The_Mona_Lisa/real_311_000001.jpg
real/The_Mona_Lisa/real_311_000002.jpg
real/The_Mona_Lisa/real_311_000004.jpg
real/The_Mona_Lisa/real_311_000006.jpg
real/The_Mona_Lisa/real_311_000007.jpg
real/The_Mona_Lisa/real_311_000008.jpg
real/The_Mona_Lisa/real_311_000010.jpg
real/The_Mona_Lisa/real_311_000011.jpg
real/The_Mona_Lisa/real_311_000016.jpg
real/The_Mona_Lisa/real_311_000017.jpg
real/The_Mona_Lisa/real_311_000018.jpg
real/The_Mona_Lisa/real_311_000019.jpg
real/The_Mona_Lisa/real_311_000020.jpg
real/The_Mona_Lisa/real_311_000021.jpg
real/The_Mona_Lisa/real_311_000023.jpg
real/The_Mona_Lisa/real_311_000027.jpg
real/The_Mona_Lisa/real_311_000028.jpg
real/The_Mona_Lisa/real_311_000031.jpg
real/The_Mona_Lisa/real_311_000035.jpg
real/The_Mona_Lisa/real_311_000037.jpg
real/The_Mona_Lisa/real_311_000044.jpg
real/The_Mona_Lisa/real_311_000045.jpg
real/The_Mona_Lisa/real_311_000047.jpg
real/The_Mona_Lisa/real_311_000051.jpg
real/The_Mona_Lisa/real_311_000052.jpg
real/The_Mona_Lisa/real_311_000054.jpg
real/The_Mona_Lisa/real_311_000060.jpg
real/The_Mona_Lisa/real_311_000062.jpg
real/The_Mona_Lisa/real_311_000067.jpg
real/The_Mona_Lisa/real_311_000069.jpg
real/The_Mona_Lisa/real_311_000070.jpg
real/The_Mona_Lisa/real_311_000072.jpg
real/The_Mona_Lisa/real_311_000076.jpg
real/The_Mona_Lisa/real_311_000077.jpg
real/The_Mona_Lisa/real_311_000084.jpg
real/The_Mona_Lisa/real_311_000086.jpg
real/The_Mona_Lisa/real_311_000090.jpg
real/The_Mona_Lisa/real_311_000095.jpg
real/The_Mona_Lisa/real_311_000097.jpg
real/The_Mona_Lisa/real_311_000107.jpg
real/The_Mona_Lisa/real_311_000109.jpg
real/The_Mona_Lisa/real_311_000113.jpg
real/The_Mona_Lisa/real_311_000115.jpg
real/The_Mona_Lisa/real_311_000118.jpg
real/The_Mona_Lisa/real_311_000120.jpg
real/The_Mona_Lisa/real_311_000123.jpg
real/The_Mona_Lisa/real_311_000125.jpg
real/The_Mona_Lisa/real_311_000127.jpg
real/The_Mona_Lisa/real_311_000140.jpg
real/The_Mona_Lisa/real_311_000143.jpg
real/The_Mona_Lisa/real_311_000148.jpg
real/The_Mona_Lisa/real_311_000158.jpg
real/The_Mona_Lisa/real_311_000160.jpg
real/The_Mona_Lisa/real_311_000161.jpg
real/The_Mona_Lisa/real_311_000165.jpg
real/The_Mona_Lisa/real_311_000166.jpg
real/The_Mona_Lisa/real_311_000170.jpg
real/The_Mona_Lisa/real_311_000171.jpg
real/The_Mona_Lisa/real_311_000172.jpg
real/The_Mona_Lisa/real_311_000186.jpg
real/The_Mona_Lisa/real_311_000211.jpg
real/The_Mona_Lisa/real_311_000220.jpg
real/The_Mona_Lisa/real_311_000222.jpg
real/The_Mona_Lisa/real_311_000230.jpg
real/The_Mona_Lisa/real_311_000243.jpg
real/The_Mona_Lisa/real_311_000244.jpg
real/The_Mona_Lisa/real_311_000251.jpg
real/The_Mona_Lisa/real_311_000271.jpg
real/The_Mona_Lisa/real_311_000273.jpg
real/tiger/real_312_000063.jpg
real/tiger/real_312_000120.jpg
real/tiger/real_312_000129.jpg
real/tiger/real_312_000198.jpg
real/tiger/real_312_000233.jpg
real/tiger/real_312_000246.jpg
real/tiger/real_312_000259.jpg
real/tiger/real_312_000310.jpg
real/tiger/real_312_000313.jpg
real/tiger/real_312_000320.jpg
real/tiger/real_312_000324.jpg
real/tiger/real_312_000358.jpg
real/tiger/real_312_000372.jpg
real/tiger/real_312_000599.jpg
real/toaster/real_313_000069.jpg
real/toaster/real_313_000089.jpg
real/toaster/real_313_000163.jpg
real/toaster/real_313_000236.jpg
real/toaster/real_313_000344.jpg
real/toaster/real_313_000346.jpg
real/toaster/real_313_000387.jpg
real/toaster/real_313_000401.jpg
real/toaster/real_313_000402.jpg
real/toaster/real_313_000450.jpg
real/toaster/real_313_000497.jpg
real/toe/real_314_000033.jpg
real/toe/real_314_000076.jpg
real/toe/real_314_000115.jpg
real/toe/real_314_000179.jpg
real/toe/real_314_000196.jpg
real/toe/real_314_000221.jpg
real/toe/real_314_000249.jpg
real/toe/real_314_000261.jpg
real/toe/real_314_000271.jpg
real/toe/real_314_000278.jpg
real/toe/real_314_000301.jpg
real/toe/real_314_000302.jpg
real/toe/real_314_000339.jpg
real/toilet/real_315_000443.jpg
real/toilet/real_315_000468.jpg
real/tooth/real_316_000006.jpg
real/tooth/real_316_000084.jpg
real/toothbrush/real_317_000079.jpg
real/toothbrush/real_317_000122.jpg
real/toothbrush/real_317_000195.jpg
real/toothbrush/real_317_000409.jpg
real/toothpaste/real_318_000342.jpg
real/tornado/real_319_000002.jpg
real/tornado/real_319_000004.jpg
real/tornado/real_319_000025.jpg
real/tornado/real_319_000093.jpg
real/tornado/real_319_000094.jpg
real/tornado/real_319_000095.jpg
real/tornado/real_319_000096.jpg
real/tornado/real_319_000098.jpg
real/tornado/real_319_000099.jpg
real/tornado/real_319_000100.jpg
real/tornado/real_319_000101.jpg
real/tornado/real_319_000102.jpg
real/tornado/real_319_000103.jpg
real/tornado/real_319_000104.jpg
real/tornado/real_319_000106.jpg
real/tornado/real_319_000107.jpg
real/tornado/real_319_000108.jpg
real/tornado/real_319_000110.jpg
real/tornado/real_319_000111.jpg
real/tornado/real_319_000137.jpg
real/tornado/real_319_000173.jpg
real/tornado/real_319_000275.jpg
real/tornado/real_319_000279.jpg
real/tornado/real_319_000295.jpg
real/tornado/real_319_000351.jpg
real/tornado/real_319_000497.jpg
real/traffic_light/real_321_000020.jpg
real/traffic_light/real_321_000056.jpg
real/traffic_light/real_321_000083.jpg
real/traffic_light/real_321_000132.jpg
real/traffic_light/real_321_000175.jpg
real/traffic_light/real_321_000186.jpg
real/traffic_light/real_321_000219.jpg
real/traffic_light/real_321_000285.jpg
real/train/real_322_000301.jpg
real/tree/real_323_000089.jpg
real/tree/real_323_000104.jpg
real/tree/real_323_000242.jpg
real/tree/real_323_000515.jpg
real/tree/real_323_000522.jpg
real/tree/real_323_000528.jpg
real/triangle/real_324_000007.jpg
real/triangle/real_324_000010.jpg
real/triangle/real_324_000016.jpg
real/triangle/real_324_000020.jpg
real/triangle/real_324_000050.jpg
real/triangle/real_324_000075.jpg
real/triangle/real_324_000093.jpg
real/triangle/real_324_000105.jpg
real/triangle/real_324_000178.jpg
real/triangle/real_324_000243.jpg
real/triangle/real_324_000259.jpg
real/triangle/real_324_000279.jpg
real/triangle/real_324_000293.jpg
real/triangle/real_324_000311.jpg
real/triangle/real_324_000373.jpg
real/triangle/real_324_000375.jpg
real/triangle/real_324_000376.jpg
real/trombone/real_325_000041.jpg
real/trombone/real_325_000075.jpg
real/trombone/real_325_000185.jpg
real/trombone/real_325_000291.jpg
real/truck/real_326_000121.jpg
real/truck/real_326_000135.jpg
real/truck/real_326_000198.jpg
real/truck/real_326_000233.jpg
real/truck/real_326_000239.jpg
real/truck/real_326_000259.jpg
real/truck/real_326_000264.jpg
real/truck/real_326_000334.jpg
real/truck/real_326_000340.jpg
real/truck/real_326_000366.jpg
real/truck/real_326_000395.jpg
real/truck/real_326_000428.jpg
real/truck/real_326_000460.jpg
real/truck/real_326_000497.jpg
real/truck/real_326_000503.jpg
real/truck/real_326_000540.jpg
real/truck/real_326_000542.jpg
real/truck/real_326_000594.jpg
real/truck/real_326_000617.jpg
real/trumpet/real_327_000046.jpg
real/trumpet/real_327_000108.jpg
real/trumpet/real_327_000147.jpg
real/trumpet/real_327_000275.jpg
real/trumpet/real_327_000295.jpg
real/trumpet/real_327_000309.jpg
real/trumpet/real_327_000326.jpg
real/trumpet/real_327_000384.jpg
real/umbrella/real_329_000025.jpg
real/umbrella/real_329_000077.jpg
real/umbrella/real_329_000130.jpg
real/umbrella/real_329_000160.jpg
real/umbrella/real_329_000204.jpg
real/umbrella/real_329_000218.jpg
real/umbrella/real_329_000232.jpg
real/umbrella/real_329_000303.jpg
real/umbrella/real_329_000308.jpg
real/umbrella/real_329_000328.jpg
real/umbrella/real_329_000338.jpg
real/umbrella/real_329_000347.jpg
real/van/real_331_000027.jpg
real/van/real_331_000048.jpg
real/van/real_331_000370.jpg
real/van/real_331_000382.jpg
real/vase/real_332_000034.jpg
real/vase/real_332_000382.jpg
real/vase/real_332_000423.jpg
real/vase/real_332_000429.jpg
real/vase/real_332_000514.jpg
real/vase/real_332_000562.jpg
real/violin/real_333_000199.jpg
real/violin/real_333_000265.jpg
real/violin/real_333_000324.jpg
real/violin/real_333_000349.jpg
real/violin/real_333_000353.jpg
real/violin/real_333_000354.jpg
real/washing_machine/real_334_000126.jpg
real/washing_machine/real_334_000371.jpg
real/washing_machine/real_334_000426.jpg
real/watermelon/real_335_000067.jpg
real/watermelon/real_335_000097.jpg
real/watermelon/real_335_000121.jpg
real/watermelon/real_335_000196.jpg
real/watermelon/real_335_000242.jpg
real/watermelon/real_335_000244.jpg
real/watermelon/real_335_000261.jpg
real/watermelon/real_335_000293.jpg
real/watermelon/real_335_000472.jpg
real/watermelon/real_335_000534.jpg
real/waterslide/real_336_000069.jpg
real/waterslide/real_336_000584.jpg
real/whale/real_337_000086.jpg
real/whale/real_337_000299.jpg
real/whale/real_337_000344.jpg
real/whale/real_337_000406.jpg
real/whale/real_337_000486.jpg
real/whale/real_337_000494.jpg
real/whale/real_337_000583.jpg
real/whale/real_337_000588.jpg
real/wheel/real_338_000006.jpg
real/wheel/real_338_000086.jpg
real/wheel/real_338_000094.jpg
real/wheel/real_338_000101.jpg
real/wheel/real_338_000112.jpg
real/wheel/real_338_000174.jpg
real/wheel/real_338_000187.jpg
real/wheel/real_338_000302.jpg
real/wheel/real_338_000337.jpg
real/wheel/real_338_000387.jpg
real/windmill/real_339_000082.jpg
real/windmill/real_339_000128.jpg
real/windmill/real_339_000202.jpg
real/windmill/real_339_000499.jpg
real/windmill/real_339_000552.jpg
real/windmill/real_339_000589.jpg
real/wine_bottle/real_340_000030.jpg
real/wine_bottle/real_340_000034.jpg
real/wine_bottle/real_340_000088.jpg
real/wine_bottle/real_340_000138.jpg
real/wine_bottle/real_340_000152.jpg
real/wine_bottle/real_340_000183.jpg
real/wine_bottle/real_340_000215.jpg
real/wine_bottle/real_340_000281.jpg
real/wine_glass/real_341_000030.jpg
real/wine_glass/real_341_000053.jpg
real/wine_glass/real_341_000090.jpg
real/wine_glass/real_341_000094.jpg
real/wine_glass/real_341_000179.jpg
real/wine_glass/real_341_000180.jpg
real/wine_glass/real_341_000181.jpg
real/wine_glass/real_341_000236.jpg
real/wine_glass/real_341_000237.jpg
real/wine_glass/real_341_000258.jpg
real/wine_glass/real_341_000275.jpg
real/wristwatch/real_342_000229.jpg
real/wristwatch/real_342_000300.jpg
real/wristwatch/real_342_000338.jpg
real/wristwatch/real_342_000388.jpg
real/wristwatch/real_342_000544.jpg
real/wristwatch/real_342_000548.jpg
real/yoga/real_343_000143.jpg
real/yoga/real_343_000198.jpg
real/yoga/real_343_000233.jpg
real/yoga/real_343_000284.jpg
real/yoga/real_343_000292.jpg
real/yoga/real_343_000293.jpg
real/yoga/real_343_000296.jpg
real/yoga/real_343_000297.jpg
real/yoga/real_343_000313.jpg
real/zebra/real_344_000095.jpg
real/zebra/real_344_000121.jpg
real/zebra/real_344_000138.jpg
real/zebra/real_344_000145.jpg
real/zebra/real_344_000188.jpg
real/zebra/real_344_000258.jpg
real/zebra/real_344_000266.jpg
real/zebra/real_344_000350.jpg
real/zebra/real_344_000383.jpg
real/zebra/real_344_000531.jpg
real/zebra/real_344_000551.jpg
real/zebra/real_344_000559.jpg
real/zebra/real_344_000614.jpg
real/zebra/real_344_000620.jpg
real/zigzag/real_345_000004.jpg
real/zigzag/real_345_000006.jpg
real/zigzag/real_345_000009.jpg
real/zigzag/real_345_000012.jpg
real/zigzag/real_345_000016.jpg
real/zigzag/real_345_000017.jpg
real/zigzag/real_345_000026.jpg
real/zigzag/real_345_000032.jpg
real/zigzag/real_345_000096.jpg
real/zigzag/real_345_000116.jpg
real/zigzag/real_345_000125.jpg
real/zigzag/real_345_000127.jpg
real/zigzag/real_345_000139.jpg
real/zigzag/real_345_000140.jpg
real/zigzag/real_345_000160.jpg
real/zigzag/real_345_000164.jpg
real/zigzag/real_345_000166.jpg
real/zigzag/real_345_000176.jpg
real/zigzag/real_345_000178.jpg
real/zigzag/real_345_000241.jpg
real/zigzag/real_345_000247.jpg
real/zigzag/real_345_000248.jpg
real/zigzag/real_345_000252.jpg
real/zigzag/real_345_000275.jpg
real/zigzag/real_345_000340.jpg
real/zigzag/real_345_000354.jpg
real/zigzag/real_345_000384.jpg
real/zigzag/real_345_000423.jpg
real/zigzag/real_345_000443.jpg
real/zigzag/real_345_000460.jpg
real/zigzag/real_345_000470.jpg
real/zigzag/real_345_000498.jpg
real/zigzag/real_345_000510.jpg
real/zigzag/real_345_000511.jpg
real/zigzag/real_345_000513.jpg
sketch/angel/sketch_005_000012.jpg
sketch/animal_migration/sketch_006_000076.jpg
sketch/anvil/sketch_008_000050.jpg
sketch/anvil/sketch_008_000091.jpg
sketch/asparagus/sketch_011_000041.jpg
sketch/asparagus/sketch_011_000099.jpg
sketch/axe/sketch_012_000030.jpg
sketch/axe/sketch_012_000106.jpg
sketch/axe/sketch_012_000155.jpg
sketch/axe/sketch_012_000157.jpg
sketch/axe/sketch_012_000163.jpg
sketch/axe/sketch_012_000165.jpg
sketch/axe/sketch_012_000167.jpg
sketch/axe/sketch_012_000168.jpg
sketch/axe/sketch_012_000169.jpg
sketch/axe/sketch_012_000170.jpg
sketch/axe/sketch_012_000172.jpg
sketch/axe/sketch_012_000177.jpg
sketch/axe/sketch_012_000180.jpg
sketch/axe/sketch_012_000199.jpg
sketch/axe/sketch_012_000202.jpg
sketch/backpack/sketch_013_000031.jpg
sketch/banana/sketch_014_000161.jpg
sketch/baseball_bat/sketch_018_000061.jpg
sketch/baseball_bat/sketch_018_000082.jpg
sketch/baseball_bat/sketch_018_000131.jpg
sketch/baseball_bat/sketch_018_000189.jpg
sketch/basket/sketch_019_000124.jpg
sketch/basket/sketch_019_000150.jpg
sketch/basket/sketch_019_000175.jpg
sketch/basketball/sketch_020_000096.jpg
sketch/bathtub/sketch_022_000001.jpg
sketch/beard/sketch_025_000221.jpg
sketch/bench/sketch_029_000067.jpg
sketch/bench/sketch_029_000069.jpg
sketch/bench/sketch_029_000074.jpg
sketch/bench/sketch_029_000079.jpg
sketch/bench/sketch_029_000178.jpg
sketch/bench/sketch_029_000179.jpg
sketch/bench/sketch_029_000183.jpg
sketch/bench/sketch_029_000184.jpg
sketch/bench/sketch_029_000188.jpg
sketch/bench/sketch_029_000189.jpg
sketch/bench/sketch_029_000190.jpg
sketch/bench/sketch_029_000191.jpg
sketch/bench/sketch_029_000195.jpg
sketch/bench/sketch_029_000199.jpg
sketch/bench/sketch_029_000200.jpg
sketch/bench/sketch_029_000201.jpg
sketch/bench/sketch_029_000202.jpg
sketch/bench/sketch_029_000203.jpg
sketch/bench/sketch_029_000214.jpg
sketch/binoculars/sketch_031_000015.jpg
sketch/binoculars/sketch_031_000053.jpg
sketch/binoculars/sketch_031_000096.jpg
sketch/binoculars/sketch_031_000176.jpg
sketch/binoculars/sketch_031_000197.jpg
sketch/blueberry/sketch_035_000072.jpg
sketch/boomerang/sketch_037_000047.jpg
sketch/boomerang/sketch_037_000058.jpg
sketch/bowtie/sketch_039_000231.jpg
sketch/bowtie/sketch_039_000291.jpg
sketch/bowtie/sketch_039_000320.jpg
sketch/bracelet/sketch_040_000035.jpg
sketch/bread/sketch_042_000029.jpg
sketch/bridge/sketch_043_000167.jpg
sketch/broccoli/sketch_044_000063.jpg
sketch/broom/sketch_045_000219.jpg
sketch/bus/sketch_048_000160.jpg
sketch/bus/sketch_048_000164.jpg
sketch/bus/sketch_048_000168.jpg
sketch/bus/sketch_048_000169.jpg
sketch/bus/sketch_048_000171.jpg
sketch/bus/sketch_048_000172.jpg
sketch/bus/sketch_048_000173.jpg
sketch/bus/sketch_048_000175.jpg
sketch/bus/sketch_048_000177.jpg
sketch/bus/sketch_048_000180.jpg
sketch/bus/sketch_048_000183.jpg
sketch/bush/sketch_049_000029.jpg
sketch/bush/sketch_049_000039.jpg
sketch/bush/sketch_049_000047.jpg
sketch/bush/sketch_049_000102.jpg
sketch/bush/sketch_049_000171.jpg
sketch/bush/sketch_049_000500.jpg
sketch/bush/sketch_049_000599.jpg
sketch/cake/sketch_052_000001.jpg
sketch/cake/sketch_052_000002.jpg
sketch/cake/sketch_052_000003.jpg
sketch/cake/sketch_052_000007.jpg
sketch/cake/sketch_052_000010.jpg
sketch/cake/sketch_052_000022.jpg
sketch/cake/sketch_052_000049.jpg
sketch/cake/sketch_052_000050.jpg
sketch/cake/sketch_052_000059.jpg
sketch/cake/sketch_052_000076.jpg
sketch/campfire/sketch_058_000050.jpg
sketch/campfire/sketch_058_000072.jpg
sketch/clarinet/sketch_073_000009.jpg
sketch/clarinet/sketch_073_000015.jpg
sketch/clarinet/sketch_073_000018.jpg
sketch/compass/sketch_077_000013.jpg
sketch/computer/sketch_078_000011.jpg
sketch/computer/sketch_078_000025.jpg
sketch/cooler/sketch_080_000021.jpg
sketch/cooler/sketch_080_000035.jpg
sketch/cooler/sketch_080_000078.jpg
sketch/cooler/sketch_080_000086.jpg
sketch/crown/sketch_086_000084.jpg
sketch/crown/sketch_086_000107.jpg
sketch/crown/sketch_086_000142.jpg
sketch/cruise_ship/sketch_087_000130.jpg
sketch/cruise_ship/sketch_087_000145.jpg
sketch/cruise_ship/sketch_087_000147.jpg
sketch/cup/sketch_088_000008.jpg
sketch/cup/sketch_088_000035.jpg
sketch/cup/sketch_088_000137.jpg
sketch/cup/sketch_088_000229.jpg
sketch/cup/sketch_088_000231.jpg
sketch/cup/sketch_088_000260.jpg
sketch/cup/sketch_088_000357.jpg
sketch/cup/sketch_088_000366.jpg
sketch/diamond/sketch_089_000078.jpg
sketch/diamond/sketch_089_000102.jpg
sketch/dishwasher/sketch_090_000019.jpg
sketch/diving_board/sketch_091_000001.jpg
sketch/diving_board/sketch_091_000016.jpg
sketch/diving_board/sketch_091_000021.jpg
sketch/diving_board/sketch_091_000030.jpg
sketch/diving_board/sketch_091_000033.jpg
sketch/diving_board/sketch_091_000038.jpg
sketch/dog/sketch_092_000300.jpg
sketch/door/sketch_095_000095.jpg
sketch/dragon/sketch_096_000051.jpg
sketch/duck/sketch_100_000274.jpg
sketch/dumbbell/sketch_101_000054.jpg
sketch/dumbbell/sketch_101_000125.jpg
sketch/dumbbell/sketch_101_000143.jpg
sketch/dumbbell/sketch_101_000161.jpg
sketch/dumbbell/sketch_101_000173.jpg
sketch/dumbbell/sketch_101_000189.jpg
sketch/elbow/sketch_103_000029.jpg
sketch/elbow/sketch_103_000086.jpg
sketch/elbow/sketch_103_000115.jpg
sketch/elbow/sketch_103_000152.jpg
sketch/envelope/sketch_105_000004.jpg
sketch/envelope/sketch_105_000065.jpg
sketch/envelope/sketch_105_000156.jpg
sketch/eraser/sketch_106_000011.jpg
sketch/eraser/sketch_106_000045.jpg
sketch/eraser/sketch_106_000048.jpg
sketch/eyeglasses/sketch_108_000032.jpg
sketch/eyeglasses/sketch_108_000129.jpg
sketch/eyeglasses/sketch_108_000207.jpg
sketch/face/sketch_109_000120.jpg
sketch/face/sketch_109_000229.jpg
sketch/face/sketch_109_000304.jpg
sketch/face/sketch_109_000379.jpg
sketch/fan/sketch_110_000007.jpg
sketch/fan/sketch_110_000014.jpg
sketch/fan/sketch_110_000031.jpg
sketch/fan/sketch_110_000034.jpg
sketch/fan/sketch_110_000038.jpg
sketch/fan/sketch_110_000049.jpg
sketch/fan/sketch_110_000051.jpg
sketch/fan/sketch_110_000053.jpg
sketch/feather/sketch_111_000072.jpg
sketch/feather/sketch_111_000088.jpg
sketch/feather/sketch_111_000092.jpg
sketch/feather/sketch_111_000094.jpg
sketch/feather/sketch_111_000095.jpg
sketch/feather/sketch_111_000097.jpg
sketch/feather/sketch_111_000098.jpg
sketch/feather/sketch_111_000099.jpg
sketch/feather/sketch_111_000102.jpg
sketch/feather/sketch_111_000104.jpg
sketch/feather/sketch_111_000105.jpg
sketch/feather/sketch_111_000108.jpg
sketch/feather/sketch_111_000118.jpg
sketch/feather/sketch_111_000123.jpg
sketch/feather/sketch_111_000126.jpg
sketch/feather/sketch_111_000147.jpg
sketch/feather/sketch_111_000165.jpg
sketch/feather/sketch_111_000169.jpg
sketch/feather/sketch_111_000176.jpg
sketch/feather/sketch_111_000180.jpg
sketch/feather/sketch_111_000188.jpg
sketch/feather/sketch_111_000213.jpg
sketch/feather/sketch_111_000269.jpg
sketch/feather/sketch_111_000282.jpg
sketch/feather/sketch_111_000293.jpg
sketch/feather/sketch_111_000312.jpg
sketch/feather/sketch_111_000319.jpg
sketch/feather/sketch_111_000322.jpg
sketch/feather/sketch_111_000329.jpg
sketch/feather/sketch_111_000333.jpg
sketch/feather/sketch_111_000335.jpg
sketch/fire_hydrant/sketch_114_000027.jpg
sketch/fire_hydrant/sketch_114_000067.jpg
sketch/fire_hydrant/sketch_114_000103.jpg
sketch/fire_hydrant/sketch_114_000104.jpg
sketch/fire_hydrant/sketch_114_000106.jpg
sketch/fire_hydrant/sketch_114_000133.jpg
sketch/fireplace/sketch_115_000010.jpg
sketch/fireplace/sketch_115_000081.jpg
sketch/fish/sketch_117_000142.jpg
sketch/fish/sketch_117_000369.jpg
sketch/flashlight/sketch_119_000026.jpg
sketch/flashlight/sketch_119_000061.jpg
sketch/flashlight/sketch_119_000087.jpg
sketch/flip_flops/sketch_120_000083.jpg
sketch/floor_lamp/sketch_121_000026.jpg
sketch/floor_lamp/sketch_121_000072.jpg
sketch/floor_lamp/sketch_121_000124.jpg
sketch/floor_lamp/sketch_121_000267.jpg
sketch/flying_saucer/sketch_123_000020.jpg
sketch/flying_saucer/sketch_123_000030.jpg
sketch/flying_saucer/sketch_123_000043.jpg
sketch/flying_saucer/sketch_123_000050.jpg
sketch/foot/sketch_124_000006.jpg
sketch/foot/sketch_124_000248.jpg
sketch/foot/sketch_124_000252.jpg
sketch/foot/sketch_124_000256.jpg
sketch/foot/sketch_124_000258.jpg
sketch/foot/sketch_124_000260.jpg
sketch/frying_pan/sketch_127_000120.jpg
sketch/garden/sketch_128_000077.jpg
sketch/goatee/sketch_131_000020.jpg
sketch/goatee/sketch_131_000029.jpg
sketch/goatee/sketch_131_000037.jpg
sketch/goatee/sketch_131_000038.jpg
sketch/goatee/sketch_131_000042.jpg
sketch/goatee/sketch_131_000044.jpg
sketch/goatee/sketch_131_000049.jpg
sketch/goatee/sketch_131_000052.jpg
sketch/goatee/sketch_131_000057.jpg
sketch/goatee/sketch_131_000058.jpg
sketch/goatee/sketch_131_000067.jpg
sketch/goatee/sketch_131_000072.jpg
sketch/goatee/sketch_131_000074.jpg
sketch/goatee/sketch_131_000076.jpg
sketch/goatee/sketch_131_000083.jpg
sketch/goatee/sketch_131_000085.jpg
sketch/goatee/sketch_131_000088.jpg
sketch/goatee/sketch_131_000090.jpg
sketch/goatee/sketch_131_000097.jpg
sketch/goatee/sketch_131_000099.jpg
sketch/goatee/sketch_131_000105.jpg
sketch/goatee/sketch_131_000117.jpg
sketch/goatee/sketch_131_000118.jpg
sketch/goatee/sketch_131_000123.jpg
sketch/goatee/sketch_131_000128.jpg
sketch/goatee/sketch_131_000136.jpg
sketch/goatee/sketch_131_000137.jpg
sketch/goatee/sketch_131_000140.jpg
sketch/goatee/sketch_131_000148.jpg
sketch/goatee/sketch_131_000160.jpg
sketch/goatee/sketch_131_000180.jpg
sketch/goatee/sketch_131_000187.jpg
sketch/goatee/sketch_131_000189.jpg
sketch/goatee/sketch_131_000191.jpg
sketch/goatee/sketch_131_000192.jpg
sketch/goatee/sketch_131_000197.jpg
sketch/goatee/sketch_131_000200.jpg
sketch/goatee/sketch_131_000206.jpg
sketch/golf_club/sketch_132_000012.jpg
sketch/golf_club/sketch_132_000064.jpg
sketch/golf_club/sketch_132_000307.jpg
sketch/golf_club/sketch_132_000308.jpg
sketch/golf_club/sketch_132_000484.jpg
sketch/grass/sketch_134_000072.jpg
sketch/guitar/sketch_135_000100.jpg
sketch/guitar/sketch_135_000131.jpg
sketch/hamburger/sketch_136_000068.jpg
sketch/hamburger/sketch_136_000155.jpg
sketch/hand/sketch_138_000021.jpg
sketch/hand/sketch_138_000022.jpg
sketch/hand/sketch_138_000028.jpg
sketch/hand/sketch_138_000029.jpg
sketch/hand/sketch_138_000033.jpg
sketch/hand/sketch_138_000034.jpg
sketch/hand/sketch_138_000063.jpg
sketch/hand/sketch_138_000069.jpg
sketch/hand/sketch_138_000080.jpg
sketch/hand/sketch_138_000086.jpg
sketch/hand/sketch_138_000091.jpg
sketch/hand/sketch_138_000109.jpg
sketch/hand/sketch_138_000111.jpg
sketch/hand/sketch_138_000134.jpg
sketch/hand/sketch_138_000167.jpg
sketch/hand/sketch_138_000173.jpg
sketch/hand/sketch_138_000179.jpg
sketch/hand/sketch_138_000183.jpg
sketch/hand/sketch_138_000186.jpg
sketch/hand/sketch_138_000188.jpg
sketch/hand/sketch_138_000209.jpg
sketch/hand/sketch_138_000215.jpg
sketch/hand/sketch_138_000216.jpg
sketch/hand/sketch_138_000229.jpg
sketch/hand/sketch_138_000243.jpg
sketch/hand/sketch_138_000260.jpg
sketch/hand/sketch_138_000262.jpg
sketch/hand/sketch_138_000264.jpg
sketch/headphones/sketch_141_000039.jpg
sketch/helicopter/sketch_143_000025.jpg
sketch/helicopter/sketch_143_000096.jpg
sketch/helicopter/sketch_143_000176.jpg
sketch/helmet/sketch_144_000146.jpg
sketch/hexagon/sketch_145_000029.jpg
sketch/hockey_puck/sketch_146_000011.jpg
sketch/hockey_puck/sketch_146_000024.jpg
sketch/hockey_puck/sketch_146_000047.jpg
sketch/hockey_puck/sketch_146_000059.jpg
sketch/hockey_puck/sketch_146_000086.jpg
sketch/hockey_puck/sketch_146_000090.jpg
sketch/hockey_stick/sketch_147_000009.jpg
sketch/hockey_stick/sketch_147_000019.jpg
sketch/hockey_stick/sketch_147_000022.jpg
sketch/hockey_stick/sketch_147_000030.jpg
sketch/hockey_stick/sketch_147_000042.jpg
sketch/hockey_stick/sketch_147_000048.jpg
sketch/hockey_stick/sketch_147_000058.jpg
sketch/hockey_stick/sketch_147_000071.jpg
sketch/hockey_stick/sketch_147_000073.jpg
sketch/hockey_stick/sketch_147_000080.jpg
sketch/hockey_stick/sketch_147_000097.jpg
sketch/hockey_stick/sketch_147_000103.jpg
sketch/hockey_stick/sketch_147_000106.jpg
sketch/hockey_stick/sketch_147_000109.jpg
sketch/hockey_stick/sketch_147_000110.jpg
sketch/hockey_stick/sketch_147_000111.jpg
sketch/hockey_stick/sketch_147_000114.jpg
sketch/hockey_stick/sketch_147_000116.jpg
sketch/horse/sketch_148_000054.jpg
sketch/horse/sketch_148_000093.jpg
sketch/hot_dog/sketch_151_000040.jpg
sketch/hot_dog/sketch_151_000054.jpg
sketch/hot_dog/sketch_151_000081.jpg
sketch/hot_dog/sketch_151_000084.jpg
sketch/hot_tub/sketch_152_000032.jpg
sketch/hourglass/sketch_153_000115.jpg
sketch/hurricane/sketch_156_000040.jpg
sketch/hurricane/sketch_156_000079.jpg
sketch/hurricane/sketch_156_000088.jpg
sketch/ice_cream/sketch_157_000037.jpg
sketch/jacket/sketch_158_000007.jpg
sketch/jacket/sketch_158_000015.jpg
sketch/jacket/sketch_158_000044.jpg
sketch/jacket/sketch_158_000083.jpg
sketch/jacket/sketch_158_000084.jpg
sketch/kangaroo/sketch_160_000060.jpg
sketch/knee/sketch_163_000073.jpg
sketch/knife/sketch_164_000057.jpg
sketch/knife/sketch_164_000071.jpg
sketch/knife/sketch_164_000072.jpg
sketch/knife/sketch_164_000076.jpg
sketch/knife/sketch_164_000112.jpg
sketch/ladder/sketch_165_000124.jpg
sketch/ladder/sketch_165_000185.jpg
sketch/lantern/sketch_166_000037.jpg
sketch/lantern/sketch_166_000039.jpg
sketch/laptop/sketch_167_000067.jpg
sketch/laptop/sketch_167_000106.jpg
sketch/laptop/sketch_167_000108.jpg
sketch/laptop/sketch_167_000109.jpg
sketch/laptop/sketch_167_000147.jpg
sketch/laptop/sketch_167_000209.jpg
sketch/laptop/sketch_167_000295.jpg
sketch/leaf/sketch_168_000079.jpg
sketch/leaf/sketch_168_000204.jpg
sketch/leg/sketch_169_000002.jpg
sketch/leg/sketch_169_000003.jpg
sketch/leg/sketch_169_000004.jpg
sketch/leg/sketch_169_000007.jpg
sketch/leg/sketch_169_000024.jpg
sketch/leg/sketch_169_000029.jpg
sketch/leg/sketch_169_000078.jpg
sketch/leg/sketch_169_000123.jpg
sketch/leg/sketch_169_000128.jpg
sketch/leg/sketch_169_000142.jpg
sketch/lighthouse/sketch_172_000100.jpg
sketch/lighthouse/sketch_172_000216.jpg
sketch/lightning/sketch_173_000001.jpg
sketch/lightning/sketch_173_000012.jpg
sketch/lightning/sketch_173_000013.jpg
sketch/lightning/sketch_173_000083.jpg
sketch/line/sketch_174_000020.jpg
sketch/lion/sketch_175_000021.jpg
sketch/lobster/sketch_177_000040.jpg
sketch/lobster/sketch_177_000119.jpg
sketch/lollipop/sketch_178_000101.jpg
sketch/mailbox/sketch_179_000097.jpg
sketch/matches/sketch_182_000032.jpg
sketch/matches/sketch_182_000046.jpg
sketch/microphone/sketch_185_000015.jpg
sketch/microphone/sketch_185_000077.jpg
sketch/microphone/sketch_185_000121.jpg
sketch/mosquito/sketch_189_000043.jpg
sketch/mountain/sketch_191_000054.jpg
sketch/moustache/sketch_193_000004.jpg
sketch/moustache/sketch_193_000005.jpg
sketch/moustache/sketch_193_000011.jpg
sketch/moustache/sketch_193_000013.jpg
sketch/moustache/sketch_193_000019.jpg
sketch/moustache/sketch_193_000021.jpg
sketch/moustache/sketch_193_000034.jpg
sketch/moustache/sketch_193_000047.jpg
sketch/moustache/sketch_193_000056.jpg
sketch/moustache/sketch_193_000065.jpg
sketch/moustache/sketch_193_000072.jpg
sketch/moustache/sketch_193_000079.jpg
sketch/moustache/sketch_193_000085.jpg
sketch/moustache/sketch_193_000087.jpg
sketch/mouth/sketch_194_000049.jpg
sketch/mug/sketch_195_000004.jpg
sketch/mug/sketch_195_000007.jpg
sketch/mug/sketch_195_000017.jpg
sketch/mug/sketch_195_000024.jpg
sketch/mug/sketch_195_000030.jpg
sketch/mug/sketch_195_000033.jpg
sketch/mug/sketch_195_000042.jpg
sketch/mug/sketch_195_000055.jpg
sketch/mug/sketch_195_000061.jpg
sketch/mug/sketch_195_000078.jpg
sketch/mug/sketch_195_000105.jpg
sketch/mug/sketch_195_000150.jpg
sketch/mug/sketch_195_000161.jpg
sketch/mug/sketch_195_000167.jpg
sketch/mug/sketch_195_000178.jpg
sketch/nail/sketch_197_000019.jpg
sketch/nose/sketch_199_000018.jpg
sketch/nose/sketch_199_000032.jpg
sketch/nose/sketch_199_000042.jpg
sketch/nose/sketch_199_000081.jpg
sketch/ocean/sketch_200_000027.jpg
sketch/ocean/sketch_200_000039.jpg
sketch/ocean/sketch_200_000043.jpg
sketch/ocean/sketch_200_000060.jpg
sketch/ocean/sketch_200_000071.jpg
sketch/octagon/sketch_201_000027.jpg
sketch/oven/sketch_204_000001.jpg
sketch/oven/sketch_204_000003.jpg
sketch/oven/sketch_204_000013.jpg
sketch/oven/sketch_204_000023.jpg
sketch/oven/sketch_204_000024.jpg
sketch/oven/sketch_204_000034.jpg
sketch/oven/sketch_204_000039.jpg
sketch/oven/sketch_204_000046.jpg
sketch/oven/sketch_204_000048.jpg
sketch/oven/sketch_204_000078.jpg
sketch/oven/sketch_204_000099.jpg
sketch/oven/sketch_204_000103.jpg
sketch/oven/sketch_204_000104.jpg
sketch/oven/sketch_204_000111.jpg
sketch/oven/sketch_204_000113.jpg
sketch/oven/sketch_204_000116.jpg
sketch/oven/sketch_204_000129.jpg
sketch/oven/sketch_204_000130.jpg
sketch/oven/sketch_204_000141.jpg
sketch/oven/sketch_204_000158.jpg
sketch/oven/sketch_204_000165.jpg
sketch/oven/sketch_204_000168.jpg
sketch/oven/sketch_204_000169.jpg
sketch/oven/sketch_204_000173.jpg
sketch/oven/sketch_204_000174.jpg
sketch/oven/sketch_204_000175.jpg
sketch/owl/sketch_205_000114.jpg
sketch/paint_can/sketch_207_000002.jpg
sketch/paint_can/sketch_207_000011.jpg
sketch/paint_can/sketch_207_000012.jpg
sketch/palm_tree/sketch_208_000101.jpg
sketch/palm_tree/sketch_208_000160.jpg
sketch/palm_tree/sketch_208_000163.jpg
sketch/panda/sketch_209_000035.jpg
sketch/pants/sketch_210_000089.jpg
sketch/paper_clip/sketch_211_000001.jpg
sketch/paper_clip/sketch_211_000053.jpg
sketch/parachute/sketch_212_000087.jpg
sketch/parrot/sketch_213_000083.jpg
sketch/parrot/sketch_213_000144.jpg
sketch/parrot/sketch_213_000161.jpg
sketch/parrot/sketch_213_000185.jpg
sketch/parrot/sketch_213_000209.jpg
sketch/passport/sketch_214_000043.jpg
sketch/passport/sketch_214_000073.jpg
sketch/passport/sketch_214_000086.jpg
sketch/pear/sketch_216_000157.jpg
sketch/peas/sketch_217_000035.jpg
sketch/pickup_truck/sketch_221_000098.jpg
sketch/picture_frame/sketch_222_000038.jpg
sketch/pig/sketch_223_000217.jpg
sketch/pizza/sketch_226_000009.jpg
sketch/police_car/sketch_228_000054.jpg
sketch/pond/sketch_229_000040.jpg
sketch/postcard/sketch_232_000019.jpg
sketch/potato/sketch_233_000044.jpg
sketch/potato/sketch_233_000060.jpg
sketch/power_outlet/sketch_234_000017.jpg
sketch/power_outlet/sketch_234_000090.jpg
sketch/purse/sketch_235_000058.jpg
sketch/purse/sketch_235_000068.jpg
sketch/rabbit/sketch_236_000034.jpg
sketch/rabbit/sketch_236_000045.jpg
sketch/rabbit/sketch_236_000081.jpg
sketch/raccoon/sketch_237_000106.jpg
sketch/raccoon/sketch_237_000181.jpg
sketch/rain/sketch_239_000057.jpg
sketch/rain/sketch_239_000126.jpg
sketch/rain/sketch_239_000222.jpg
sketch/rainbow/sketch_240_000006.jpg
sketch/rake/sketch_241_000007.jpg
sketch/remote_control/sketch_242_000027.jpg
sketch/river/sketch_245_000009.jpg
sketch/river/sketch_245_000011.jpg
sketch/river/sketch_245_000021.jpg
sketch/river/sketch_245_000036.jpg
sketch/river/sketch_245_000038.jpg
sketch/river/sketch_245_000045.jpg
sketch/river/sketch_245_000057.jpg
sketch/river/sketch_245_000098.jpg
sketch/river/sketch_245_000106.jpg
sketch/river/sketch_245_000108.jpg
sketch/rollerskates/sketch_247_000037.jpg
sketch/rollerskates/sketch_247_000041.jpg
sketch/rollerskates/sketch_247_000047.jpg
sketch/rollerskates/sketch_247_000050.jpg
sketch/rollerskates/sketch_247_000101.jpg
sketch/rollerskates/sketch_247_000106.jpg
sketch/rollerskates/sketch_247_000110.jpg
sketch/rollerskates/sketch_247_000132.jpg
sketch/rollerskates/sketch_247_000138.jpg
sketch/sailboat/sketch_248_000323.jpg
sketch/sandwich/sketch_249_000077.jpg
sketch/sandwich/sketch_249_000107.jpg
sketch/saw/sketch_250_000059.jpg
sketch/saw/sketch_250_000061.jpg
sketch/saw/sketch_250_000097.jpg
sketch/saw/sketch_250_000105.jpg
sketch/saxophone/sketch_251_000191.jpg
sketch/saxophone/sketch_251_000249.jpg
sketch/school_bus/sketch_252_000001.jpg
sketch/school_bus/sketch_252_000005.jpg
sketch/school_bus/sketch_252_000020.jpg
sketch/school_bus/sketch_252_000028.jpg
sketch/school_bus/sketch_252_000031.jpg
sketch/school_bus/sketch_252_000032.jpg
sketch/school_bus/sketch_252_000033.jpg
sketch/school_bus/sketch_252_000037.jpg
sketch/school_bus/sketch_252_000044.jpg
sketch/school_bus/sketch_252_000047.jpg
sketch/school_bus/sketch_252_000053.jpg
sketch/school_bus/sketch_252_000054.jpg
sketch/school_bus/sketch_252_000059.jpg
sketch/school_bus/sketch_252_000062.jpg
sketch/school_bus/sketch_252_000076.jpg
sketch/school_bus/sketch_252_000077.jpg
sketch/school_bus/sketch_252_000081.jpg
sketch/school_bus/sketch_252_000097.jpg
sketch/school_bus/sketch_252_000098.jpg
sketch/school_bus/sketch_252_000108.jpg
sketch/school_bus/sketch_252_000109.jpg
sketch/school_bus/sketch_252_000118.jpg
sketch/school_bus/sketch_252_000121.jpg
sketch/school_bus/sketch_252_000129.jpg
sketch/school_bus/sketch_252_000130.jpg
sketch/school_bus/sketch_252_000133.jpg
sketch/school_bus/sketch_252_000158.jpg
sketch/school_bus/sketch_252_000161.jpg
sketch/school_bus/sketch_252_000183.jpg
sketch/school_bus/sketch_252_000194.jpg
sketch/school_bus/sketch_252_000211.jpg
sketch/school_bus/sketch_252_000223.jpg
sketch/school_bus/sketch_252_000256.jpg
sketch/school_bus/sketch_252_000263.jpg
sketch/school_bus/sketch_252_000284.jpg
sketch/school_bus/sketch_252_000332.jpg
sketch/school_bus/sketch_252_000334.jpg
sketch/school_bus/sketch_252_000352.jpg
sketch/school_bus/sketch_252_000353.jpg
sketch/school_bus/sketch_252_000355.jpg
sketch/school_bus/sketch_252_000357.jpg
sketch/school_bus/sketch_252_000358.jpg
sketch/school_bus/sketch_252_000359.jpg
sketch/school_bus/sketch_252_000363.jpg
sketch/school_bus/sketch_252_000365.jpg
sketch/school_bus/sketch_252_000373.jpg
sketch/school_bus/sketch_252_000380.jpg
sketch/school_bus/sketch_252_000384.jpg
sketch/school_bus/sketch_252_000386.jpg
sketch/school_bus/sketch_252_000387.jpg
sketch/school_bus/sketch_252_000389.jpg
sketch/school_bus/sketch_252_000395.jpg
sketch/school_bus/sketch_252_000398.jpg
sketch/school_bus/sketch_252_000401.jpg
sketch/scissors/sketch_253_000154.jpg
sketch/scissors/sketch_253_000201.jpg
sketch/scissors/sketch_253_000224.jpg
sketch/scissors/sketch_253_000241.jpg
sketch/scissors/sketch_253_000294.jpg
sketch/scorpion/sketch_254_000141.jpg
sketch/screwdriver/sketch_255_000029.jpg
sketch/screwdriver/sketch_255_000039.jpg
sketch/screwdriver/sketch_255_000121.jpg
sketch/screwdriver/sketch_255_000149.jpg
sketch/screwdriver/sketch_255_000175.jpg
sketch/screwdriver/sketch_255_000184.jpg
sketch/screwdriver/sketch_255_000188.jpg
sketch/screwdriver/sketch_255_000193.jpg
sketch/screwdriver/sketch_255_000217.jpg
sketch/screwdriver/sketch_255_000241.jpg
sketch/screwdriver/sketch_255_000294.jpg
sketch/screwdriver/sketch_255_000302.jpg
sketch/screwdriver/sketch_255_000303.jpg
sketch/screwdriver/sketch_255_000362.jpg
sketch/screwdriver/sketch_255_000363.jpg
sketch/see_saw/sketch_257_000008.jpg
sketch/see_saw/sketch_257_000174.jpg
sketch/see_saw/sketch_257_000301.jpg
sketch/shark/sketch_258_000095.jpg
sketch/shark/sketch_258_000096.jpg
sketch/shark/sketch_258_000113.jpg
sketch/shark/sketch_258_000343.jpg
sketch/shark/sketch_258_000402.jpg
sketch/shark/sketch_258_000404.jpg
sketch/shark/sketch_258_000405.jpg
sketch/shark/sketch_258_000407.jpg
sketch/shark/sketch_258_000409.jpg
sketch/shark/sketch_258_000410.jpg
sketch/shark/sketch_258_000411.jpg
sketch/shark/sketch_258_000412.jpg
sketch/shark/sketch_258_000413.jpg
sketch/shark/sketch_258_000414.jpg
sketch/shark/sketch_258_000415.jpg
sketch/shark/sketch_258_000417.jpg
sketch/shark/sketch_258_000418.jpg
sketch/shark/sketch_258_000419.jpg
sketch/shark/sketch_258_000421.jpg
sketch/shark/sketch_258_000422.jpg
sketch/shark/sketch_258_000423.jpg
sketch/shark/sketch_258_000425.jpg
sketch/shark/sketch_258_000426.jpg
sketch/shark/sketch_258_000429.jpg
sketch/shark/sketch_258_000430.jpg
sketch/shark/sketch_258_000432.jpg
sketch/shark/sketch_258_000434.jpg
sketch/shark/sketch_258_000436.jpg
sketch/shark/sketch_258_000438.jpg
sketch/shark/sketch_258_000439.jpg
sketch/shark/sketch_258_000441.jpg
sketch/shark/sketch_258_000443.jpg
sketch/shark/sketch_258_000445.jpg
sketch/shark/sketch_258_000447.jpg
sketch/shark/sketch_258_000450.jpg
sketch/shark/sketch_258_000452.jpg
sketch/shark/sketch_258_000454.jpg
sketch/shark/sketch_258_000456.jpg
sketch/shark/sketch_258_000458.jpg
sketch/shark/sketch_258_000528.jpg
sketch/sheep/sketch_259_000110.jpg
sketch/sheep/sketch_259_000112.jpg
sketch/sheep/sketch_259_000114.jpg
sketch/sheep/sketch_259_000116.jpg
sketch/sheep/sketch_259_000125.jpg
sketch/sheep/sketch_259_000137.jpg
sketch/sheep/sketch_259_000143.jpg
sketch/sheep/sketch_259_000171.jpg
sketch/sheep/sketch_259_000182.jpg
sketch/sheep/sketch_259_000210.jpg
sketch/sheep/sketch_259_000315.jpg
sketch/sheep/sketch_259_000317.jpg
sketch/sheep/sketch_259_000326.jpg
sketch/sheep/sketch_259_000339.jpg
sketch/sheep/sketch_259_000340.jpg
sketch/sheep/sketch_259_000351.jpg
sketch/sheep/sketch_259_000358.jpg
sketch/sheep/sketch_259_000379.jpg
sketch/shoe/sketch_260_000208.jpg
sketch/shoe/sketch_260_000444.jpg
sketch/shoe/sketch_260_000568.jpg
sketch/shorts/sketch_261_000030.jpg
sketch/shorts/sketch_261_000060.jpg
sketch/shorts/sketch_261_000316.jpg
sketch/shorts/sketch_261_000321.jpg
sketch/shorts/sketch_261_000326.jpg
sketch/shorts/sketch_261_000334.jpg
sketch/shorts/sketch_261_000335.jpg
sketch/shorts/sketch_261_000341.jpg
sketch/shorts/sketch_261_000348.jpg
sketch/shorts/sketch_261_000352.jpg
sketch/shorts/sketch_261_000360.jpg
sketch/shorts/sketch_261_000401.jpg
sketch/shovel/sketch_262_000005.jpg
sketch/shovel/sketch_262_000006.jpg
sketch/shovel/sketch_262_000054.jpg
sketch/shovel/sketch_262_000056.jpg
sketch/shovel/sketch_262_000065.jpg
sketch/shovel/sketch_262_000085.jpg
sketch/shovel/sketch_262_000092.jpg
sketch/shovel/sketch_262_000146.jpg
sketch/shovel/sketch_262_000152.jpg
sketch/shovel/sketch_262_000169.jpg
sketch/shovel/sketch_262_000176.jpg
sketch/shovel/sketch_262_000205.jpg
sketch/shovel/sketch_262_000215.jpg
sketch/shovel/sketch_262_000221.jpg
sketch/shovel/sketch_262_000277.jpg
sketch/shovel/sketch_262_000304.jpg
sketch/shovel/sketch_262_000310.jpg
sketch/shovel/sketch_262_000361.jpg
sketch/shovel/sketch_262_000381.jpg
sketch/shovel/sketch_262_000413.jpg
sketch/shovel/sketch_262_000428.jpg
sketch/shovel/sketch_262_000487.jpg
sketch/shovel/sketch_262_000488.jpg
sketch/shovel/sketch_262_000523.jpg
sketch/shovel/sketch_262_000566.jpg
sketch/shovel/sketch_262_000574.jpg
sketch/shovel/sketch_262_000597.jpg
sketch/shovel/sketch_262_000604.jpg
sketch/shovel/sketch_262_000613.jpg
sketch/sink/sketch_263_000199.jpg
sketch/sink/sketch_263_000223.jpg
sketch/skateboard/sketch_264_000002.jpg
sketch/skateboard/sketch_264_000343.jpg
sketch/skateboard/sketch_264_000349.jpg
sketch/skateboard/sketch_264_000352.jpg
sketch/skull/sketch_265_000029.jpg
sketch/skull/sketch_265_000065.jpg
sketch/skull/sketch_265_000099.jpg
sketch/skull/sketch_265_000138.jpg
sketch/skull/sketch_265_000140.jpg
sketch/skull/sketch_265_000178.jpg
sketch/skull/sketch_265_000437.jpg
sketch/skull/sketch_265_000470.jpg
sketch/skull/sketch_265_000558.jpg
sketch/skull/sketch_265_000587.jpg
sketch/skull/sketch_265_000591.jpg
sketch/skull/sketch_265_000600.jpg
sketch/skyscraper/sketch_266_000384.jpg
sketch/sleeping_bag/sketch_267_000001.jpg
sketch/sleeping_bag/sketch_267_000569.jpg
sketch/smiley_face/sketch_268_000030.jpg
sketch/smiley_face/sketch_268_000263.jpg
sketch/smiley_face/sketch_268_000289.jpg
sketch/smiley_face/sketch_268_000332.jpg
sketch/snail/sketch_269_000132.jpg
sketch/snail/sketch_269_000152.jpg
sketch/snail/sketch_269_000170.jpg
sketch/snail/sketch_269_000312.jpg
sketch/snake/sketch_270_000058.jpg
sketch/snake/sketch_270_000197.jpg
sketch/snake/sketch_270_000364.jpg
sketch/snake/sketch_270_000372.jpg
sketch/snake/sketch_270_000381.jpg
sketch/snorkel/sketch_271_000070.jpg
sketch/snorkel/sketch_271_000124.jpg
sketch/snorkel/sketch_271_000131.jpg
sketch/snorkel/sketch_271_000241.jpg
sketch/snorkel/sketch_271_000337.jpg
sketch/snorkel/sketch_271_000351.jpg
sketch/snorkel/sketch_271_000363.jpg
sketch/snowflake/sketch_272_000125.jpg
sketch/snowman/sketch_273_000281.jpg
sketch/snowman/sketch_273_000456.jpg
sketch/soccer_ball/sketch_274_000042.jpg
sketch/soccer_ball/sketch_274_000175.jpg
sketch/soccer_ball/sketch_274_000334.jpg
sketch/soccer_ball/sketch_274_000343.jpg
sketch/sock/sketch_275_000047.jpg
sketch/sock/sketch_275_000086.jpg
sketch/sock/sketch_275_000121.jpg
sketch/sock/sketch_275_000175.jpg
sketch/sock/sketch_275_000221.jpg
sketch/sock/sketch_275_000305.jpg
sketch/sock/sketch_275_000321.jpg
sketch/sock/sketch_275_000364.jpg
sketch/sock/sketch_275_000397.jpg
sketch/speedboat/sketch_276_000002.jpg
sketch/speedboat/sketch_276_000058.jpg
sketch/speedboat/sketch_276_000082.jpg
sketch/speedboat/sketch_276_000088.jpg
sketch/speedboat/sketch_276_000104.jpg
sketch/speedboat/sketch_276_000125.jpg
sketch/speedboat/sketch_276_000126.jpg
sketch/speedboat/sketch_276_000204.jpg
sketch/speedboat/sketch_276_000207.jpg
sketch/speedboat/sketch_276_000218.jpg
sketch/speedboat/sketch_276_000263.jpg
sketch/speedboat/sketch_276_000334.jpg
sketch/speedboat/sketch_276_000365.jpg
sketch/speedboat/sketch_276_000409.jpg
sketch/speedboat/sketch_276_000440.jpg
sketch/spider/sketch_277_000237.jpg
sketch/spider/sketch_277_000311.jpg
sketch/spoon/sketch_278_000015.jpg
sketch/spoon/sketch_278_000076.jpg
sketch/spoon/sketch_278_000167.jpg
sketch/spoon/sketch_278_000263.jpg
sketch/spoon/sketch_278_000275.jpg
sketch/spoon/sketch_278_000299.jpg
sketch/spreadsheet/sketch_279_000012.jpg
sketch/spreadsheet/sketch_279_000174.jpg
sketch/spreadsheet/sketch_279_000384.jpg
sketch/spreadsheet/sketch_279_000450.jpg
sketch/square/sketch_280_000116.jpg
sketch/square/sketch_280_000134.jpg
sketch/square/sketch_280_000149.jpg
sketch/square/sketch_280_000262.jpg
sketch/square/sketch_280_000277.jpg
sketch/square/sketch_280_000321.jpg
sketch/square/sketch_280_000352.jpg
sketch/square/sketch_280_000469.jpg
sketch/square/sketch_280_000499.jpg
sketch/square/sketch_280_000550.jpg
sketch/square/sketch_280_000564.jpg
sketch/square/sketch_280_000598.jpg
sketch/square/sketch_280_000695.jpg
sketch/squiggle/sketch_281_000011.jpg
sketch/squiggle/sketch_281_000041.jpg
sketch/squiggle/sketch_281_000164.jpg
sketch/squiggle/sketch_281_000230.jpg
sketch/squiggle/sketch_281_000373.jpg
sketch/squirrel/sketch_282_000344.jpg
sketch/squirrel/sketch_282_000368.jpg
sketch/stairs/sketch_283_000203.jpg
sketch/stairs/sketch_283_000415.jpg
sketch/stairs/sketch_283_000446.jpg
sketch/stairs/sketch_283_000516.jpg
sketch/star/sketch_284_000037.jpg
sketch/steak/sketch_285_000067.jpg
sketch/steak/sketch_285_000099.jpg
sketch/stereo/sketch_286_000004.jpg
sketch/stereo/sketch_286_000015.jpg
sketch/stereo/sketch_286_000085.jpg
sketch/stethoscope/sketch_287_000011.jpg
sketch/stethoscope/sketch_287_000015.jpg
sketch/stethoscope/sketch_287_000052.jpg
sketch/stethoscope/sketch_287_000081.jpg
sketch/stethoscope/sketch_287_000190.jpg
sketch/stethoscope/sketch_287_000191.jpg
sketch/stethoscope/sketch_287_000206.jpg
sketch/stethoscope/sketch_287_000211.jpg
sketch/stethoscope/sketch_287_000212.jpg
sketch/stitches/sketch_288_000005.jpg
sketch/stop_sign/sketch_289_000091.jpg
sketch/stove/sketch_290_000005.jpg
sketch/stove/sketch_290_000051.jpg
sketch/stove/sketch_290_000058.jpg
sketch/stove/sketch_290_000082.jpg
sketch/stove/sketch_290_000091.jpg
sketch/stove/sketch_290_000172.jpg
sketch/stove/sketch_290_000229.jpg
sketch/strawberry/sketch_291_000047.jpg
sketch/strawberry/sketch_291_000181.jpg
sketch/streetlight/sketch_292_000061.jpg
sketch/streetlight/sketch_292_000062.jpg
sketch/streetlight/sketch_292_000063.jpg
sketch/streetlight/sketch_292_000143.jpg
sketch/streetlight/sketch_292_000195.jpg
sketch/streetlight/sketch_292_000200.jpg
sketch/streetlight/sketch_292_000230.jpg
sketch/string_bean/sketch_293_000024.jpg
sketch/string_bean/sketch_293_000049.jpg
sketch/string_bean/sketch_293_000054.jpg
sketch/string_bean/sketch_293_000056.jpg
sketch/string_bean/sketch_293_000063.jpg
sketch/submarine/sketch_294_000011.jpg
sketch/submarine/sketch_294_000049.jpg
sketch/submarine/sketch_294_000117.jpg
sketch/submarine/sketch_294_000183.jpg
sketch/sun/sketch_296_000134.jpg
sketch/sun/sketch_296_000174.jpg
sketch/swan/sketch_297_000062.jpg
sketch/swan/sketch_297_000082.jpg
sketch/swan/sketch_297_000112.jpg
sketch/swing_set/sketch_299_000015.jpg
sketch/swing_set/sketch_299_000041.jpg
sketch/swing_set/sketch_299_000042.jpg
sketch/sword/sketch_300_000062.jpg
sketch/sword/sketch_300_000084.jpg
sketch/sword/sketch_300_000089.jpg
sketch/sword/sketch_300_000139.jpg
sketch/sword/sketch_300_000146.jpg
sketch/sword/sketch_300_000152.jpg
sketch/sword/sketch_300_000156.jpg
sketch/sword/sketch_300_000188.jpg
sketch/sword/sketch_300_000243.jpg
sketch/sword/sketch_300_000292.jpg
sketch/sword/sketch_300_000378.jpg
sketch/syringe/sketch_301_000048.jpg
sketch/syringe/sketch_301_000103.jpg
sketch/syringe/sketch_301_000148.jpg
sketch/syringe/sketch_301_000149.jpg
sketch/syringe/sketch_301_000150.jpg
sketch/syringe/sketch_301_000173.jpg
sketch/syringe/sketch_301_000176.jpg
sketch/syringe/sketch_301_000195.jpg
sketch/syringe/sketch_301_000221.jpg
sketch/table/sketch_302_000004.jpg
sketch/table/sketch_302_000032.jpg
sketch/table/sketch_302_000048.jpg
sketch/table/sketch_302_000050.jpg
sketch/table/sketch_302_000065.jpg
sketch/table/sketch_302_000108.jpg
sketch/table/sketch_302_000126.jpg
sketch/table/sketch_302_000156.jpg
sketch/table/sketch_302_000172.jpg
sketch/table/sketch_302_000190.jpg
sketch/table/sketch_302_000202.jpg
sketch/table/sketch_302_000285.jpg
sketch/table/sketch_302_000299.jpg
sketch/teapot/sketch_303_000003.jpg
sketch/teapot/sketch_303_000007.jpg
sketch/teapot/sketch_303_000020.jpg
sketch/teapot/sketch_303_000043.jpg
sketch/teapot/sketch_303_000117.jpg
sketch/teapot/sketch_303_000164.jpg
sketch/teapot/sketch_303_000201.jpg
sketch/teapot/sketch_303_000215.jpg
sketch/teapot/sketch_303_000242.jpg
sketch/teapot/sketch_303_000252.jpg
sketch/teapot/sketch_303_000278.jpg
sketch/teapot/sketch_303_000320.jpg
sketch/teddy-bear/sketch_304_000001.jpg
sketch/teddy-bear/sketch_304_000002.jpg
sketch/teddy-bear/sketch_304_000004.jpg
sketch/teddy-bear/sketch_304_000005.jpg
sketch/teddy-bear/sketch_304_000006.jpg
sketch/teddy-bear/sketch_304_000007.jpg
sketch/teddy-bear/sketch_304_000009.jpg
sketch/teddy-bear/sketch_304_000010.jpg
sketch/teddy-bear/sketch_304_000012.jpg
sketch/teddy-bear/sketch_304_000017.jpg
sketch/teddy-bear/sketch_304_000018.jpg
sketch/teddy-bear/sketch_304_000019.jpg
sketch/teddy-bear/sketch_304_000020.jpg
sketch/teddy-bear/sketch_304_000022.jpg
sketch/teddy-bear/sketch_304_000023.jpg
sketch/teddy-bear/sketch_304_000029.jpg
sketch/teddy-bear/sketch_304_000031.jpg
sketch/teddy-bear/sketch_304_000032.jpg
sketch/teddy-bear/sketch_304_000033.jpg
sketch/teddy-bear/sketch_304_000035.jpg
sketch/teddy-bear/sketch_304_000036.jpg
sketch/teddy-bear/sketch_304_000044.jpg
sketch/teddy-bear/sketch_304_000047.jpg
sketch/teddy-bear/sketch_304_000049.jpg
sketch/teddy-bear/sketch_304_000053.jpg
sketch/teddy-bear/sketch_304_000059.jpg
sketch/teddy-bear/sketch_304_000062.jpg
sketch/teddy-bear/sketch_304_000072.jpg
sketch/teddy-bear/sketch_304_000080.jpg
sketch/teddy-bear/sketch_304_000081.jpg
sketch/teddy-bear/sketch_304_000084.jpg
sketch/teddy-bear/sketch_304_000144.jpg
sketch/teddy-bear/sketch_304_000156.jpg
sketch/teddy-bear/sketch_304_000157.jpg
sketch/teddy-bear/sketch_304_000160.jpg
sketch/teddy-bear/sketch_304_000172.jpg
sketch/teddy-bear/sketch_304_000216.jpg
sketch/telephone/sketch_305_000019.jpg
sketch/telephone/sketch_305_000144.jpg
sketch/telephone/sketch_305_000195.jpg
sketch/television/sketch_306_000045.jpg
sketch/television/sketch_306_000079.jpg
sketch/television/sketch_306_000114.jpg
sketch/tent/sketch_308_000015.jpg
sketch/tent/sketch_308_000219.jpg
sketch/tent/sketch_308_000229.jpg
sketch/tent/sketch_308_000243.jpg
sketch/The_Eiffel_Tower/sketch_309_000084.jpg
sketch/The_Eiffel_Tower/sketch_309_000228.jpg
sketch/The_Eiffel_Tower/sketch_309_000264.jpg
sketch/The_Great_Wall_of_China/sketch_310_000013.jpg
sketch/The_Great_Wall_of_China/sketch_310_000020.jpg
sketch/The_Great_Wall_of_China/sketch_310_000023.jpg
sketch/The_Great_Wall_of_China/sketch_310_000029.jpg
sketch/The_Great_Wall_of_China/sketch_310_000038.jpg
sketch/The_Great_Wall_of_China/sketch_310_000048.jpg
sketch/The_Great_Wall_of_China/sketch_310_000063.jpg
sketch/The_Great_Wall_of_China/sketch_310_000099.jpg
sketch/The_Great_Wall_of_China/sketch_310_000113.jpg
sketch/The_Great_Wall_of_China/sketch_310_000116.jpg
sketch/The_Great_Wall_of_China/sketch_310_000135.jpg
sketch/The_Great_Wall_of_China/sketch_310_000147.jpg
sketch/The_Mona_Lisa/sketch_311_000049.jpg
sketch/The_Mona_Lisa/sketch_311_000113.jpg
sketch/The_Mona_Lisa/sketch_311_000143.jpg
sketch/tiger/sketch_312_000005.jpg
sketch/tiger/sketch_312_000017.jpg
sketch/tiger/sketch_312_000029.jpg
sketch/tiger/sketch_312_000061.jpg
sketch/tiger/sketch_312_000213.jpg
sketch/tiger/sketch_312_000279.jpg
sketch/tiger/sketch_312_000336.jpg
sketch/toaster/sketch_313_000021.jpg
sketch/toaster/sketch_313_000025.jpg
sketch/toaster/sketch_313_000035.jpg
sketch/toaster/sketch_313_000049.jpg
sketch/toaster/sketch_313_000079.jpg
sketch/toaster/sketch_313_000181.jpg
sketch/toe/sketch_314_000005.jpg
sketch/toe/sketch_314_000006.jpg
sketch/toe/sketch_314_000008.jpg
sketch/toe/sketch_314_000010.jpg
sketch/toe/sketch_314_000012.jpg
sketch/toe/sketch_314_000013.jpg
sketch/toe/sketch_314_000015.jpg
sketch/toe/sketch_314_000020.jpg
sketch/toe/sketch_314_000024.jpg
sketch/toe/sketch_314_000027.jpg
sketch/toe/sketch_314_000036.jpg
sketch/toe/sketch_314_000043.jpg
sketch/toilet/sketch_315_000017.jpg
sketch/toilet/sketch_315_000044.jpg
sketch/toilet/sketch_315_000047.jpg
sketch/toilet/sketch_315_000055.jpg
sketch/tooth/sketch_316_000022.jpg
sketch/tooth/sketch_316_000091.jpg
sketch/tooth/sketch_316_000106.jpg
sketch/toothbrush/sketch_317_000107.jpg
sketch/toothbrush/sketch_317_000173.jpg
sketch/toothbrush/sketch_317_000174.jpg
sketch/toothpaste/sketch_318_000015.jpg
sketch/toothpaste/sketch_318_000025.jpg
sketch/toothpaste/sketch_318_000030.jpg
sketch/toothpaste/sketch_318_000036.jpg
sketch/toothpaste/sketch_318_000038.jpg
sketch/toothpaste/sketch_318_000042.jpg
sketch/toothpaste/sketch_318_000049.jpg
sketch/toothpaste/sketch_318_000054.jpg
sketch/toothpaste/sketch_318_000059.jpg
sketch/toothpaste/sketch_318_000061.jpg
sketch/toothpaste/sketch_318_000065.jpg
sketch/toothpaste/sketch_318_000067.jpg
sketch/toothpaste/sketch_318_000071.jpg
sketch/toothpaste/sketch_318_000073.jpg
sketch/toothpaste/sketch_318_000074.jpg
sketch/toothpaste/sketch_318_000080.jpg
sketch/toothpaste/sketch_318_000082.jpg
sketch/toothpaste/sketch_318_000083.jpg
sketch/toothpaste/sketch_318_000084.jpg
sketch/toothpaste/sketch_318_000087.jpg
sketch/toothpaste/sketch_318_000088.jpg
sketch/toothpaste/sketch_318_000092.jpg
sketch/toothpaste/sketch_318_000094.jpg
sketch/toothpaste/sketch_318_000096.jpg
sketch/toothpaste/sketch_318_000103.jpg
sketch/toothpaste/sketch_318_000110.jpg
sketch/toothpaste/sketch_318_000112.jpg
sketch/toothpaste/sketch_318_000114.jpg
sketch/toothpaste/sketch_318_000117.jpg
sketch/toothpaste/sketch_318_000120.jpg
sketch/toothpaste/sketch_318_000132.jpg
sketch/toothpaste/sketch_318_000139.jpg
sketch/toothpaste/sketch_318_000149.jpg
sketch/toothpaste/sketch_318_000150.jpg
sketch/toothpaste/sketch_318_000156.jpg
sketch/toothpaste/sketch_318_000161.jpg
sketch/toothpaste/sketch_318_000165.jpg
sketch/toothpaste/sketch_318_000166.jpg
sketch/toothpaste/sketch_318_000169.jpg
sketch/toothpaste/sketch_318_000176.jpg
sketch/toothpaste/sketch_318_000179.jpg
sketch/toothpaste/sketch_318_000181.jpg
sketch/toothpaste/sketch_318_000185.jpg
sketch/toothpaste/sketch_318_000186.jpg
sketch/toothpaste/sketch_318_000188.jpg
sketch/toothpaste/sketch_318_000189.jpg
sketch/toothpaste/sketch_318_000191.jpg
sketch/tornado/sketch_319_000015.jpg
sketch/tornado/sketch_319_000021.jpg
sketch/tornado/sketch_319_000037.jpg
sketch/tornado/sketch_319_000050.jpg
sketch/tornado/sketch_319_000053.jpg
sketch/tornado/sketch_319_000076.jpg
sketch/tornado/sketch_319_000099.jpg
sketch/tornado/sketch_319_000115.jpg
sketch/tornado/sketch_319_000128.jpg
sketch/tornado/sketch_319_000154.jpg
sketch/tornado/sketch_319_000156.jpg
sketch/tornado/sketch_319_000190.jpg
sketch/tornado/sketch_319_000193.jpg
sketch/tornado/sketch_319_000195.jpg
sketch/tractor/sketch_320_000023.jpg
sketch/tractor/sketch_320_000228.jpg
sketch/traffic_light/sketch_321_000010.jpg
sketch/traffic_light/sketch_321_000020.jpg
sketch/traffic_light/sketch_321_000065.jpg
sketch/traffic_light/sketch_321_000101.jpg
sketch/traffic_light/sketch_321_000120.jpg
sketch/tree/sketch_323_000015.jpg
sketch/tree/sketch_323_000033.jpg
sketch/tree/sketch_323_000111.jpg
sketch/tree/sketch_323_000122.jpg
sketch/tree/sketch_323_000144.jpg
sketch/tree/sketch_323_000201.jpg
sketch/tree/sketch_323_000261.jpg
sketch/tree/sketch_323_000357.jpg
sketch/tree/sketch_323_000391.jpg
sketch/tree/sketch_323_000396.jpg
sketch/tree/sketch_323_000410.jpg
sketch/tree/sketch_323_000417.jpg
sketch/tree/sketch_323_000422.jpg
sketch/tree/sketch_323_000469.jpg
sketch/triangle/sketch_324_000058.jpg
sketch/triangle/sketch_324_000127.jpg
sketch/triangle/sketch_324_000133.jpg
sketch/triangle/sketch_324_000149.jpg
sketch/triangle/sketch_324_000175.jpg
sketch/triangle/sketch_324_000184.jpg
sketch/triangle/sketch_324_000204.jpg
sketch/triangle/sketch_324_000265.jpg
sketch/triangle/sketch_324_000273.jpg
sketch/trombone/sketch_325_000011.jpg
sketch/trombone/sketch_325_000024.jpg
sketch/trombone/sketch_325_000027.jpg
sketch/trombone/sketch_325_000039.jpg
sketch/trombone/sketch_325_000068.jpg
sketch/truck/sketch_326_000018.jpg
sketch/truck/sketch_326_000020.jpg
sketch/truck/sketch_326_000027.jpg
sketch/truck/sketch_326_000043.jpg
sketch/truck/sketch_326_000047.jpg
sketch/truck/sketch_326_000060.jpg
sketch/truck/sketch_326_000081.jpg
sketch/truck/sketch_326_000082.jpg
sketch/truck/sketch_326_000099.jpg
sketch/truck/sketch_326_000106.jpg
sketch/truck/sketch_326_000111.jpg
sketch/truck/sketch_326_000130.jpg
sketch/truck/sketch_326_000133.jpg
sketch/truck/sketch_326_000157.jpg
sketch/truck/sketch_326_000161.jpg
sketch/truck/sketch_326_000166.jpg
sketch/truck/sketch_326_000167.jpg
sketch/truck/sketch_326_000175.jpg
sketch/truck/sketch_326_000177.jpg
sketch/truck/sketch_326_000178.jpg
sketch/truck/sketch_326_000184.jpg
sketch/truck/sketch_326_000209.jpg
sketch/truck/sketch_326_000237.jpg
sketch/truck/sketch_326_000242.jpg
sketch/truck/sketch_326_000249.jpg
sketch/truck/sketch_326_000255.jpg
sketch/trumpet/sketch_327_000001.jpg
sketch/trumpet/sketch_327_000039.jpg
sketch/trumpet/sketch_327_000040.jpg
sketch/trumpet/sketch_327_000100.jpg
sketch/trumpet/sketch_327_000129.jpg
sketch/trumpet/sketch_327_000136.jpg
sketch/trumpet/sketch_327_000154.jpg
sketch/trumpet/sketch_327_000180.jpg
sketch/t-shirt/sketch_328_000035.jpg
sketch/t-shirt/sketch_328_000060.jpg
sketch/t-shirt/sketch_328_000089.jpg
sketch/t-shirt/sketch_328_000093.jpg
sketch/t-shirt/sketch_328_000113.jpg
sketch/t-shirt/sketch_328_000130.jpg
sketch/t-shirt/sketch_328_000155.jpg
sketch/umbrella/sketch_329_000041.jpg
sketch/umbrella/sketch_329_000053.jpg
sketch/umbrella/sketch_329_000060.jpg
sketch/umbrella/sketch_329_000105.jpg
sketch/umbrella/sketch_329_000169.jpg
sketch/umbrella/sketch_329_000186.jpg
sketch/umbrella/sketch_329_000233.jpg
sketch/umbrella/sketch_329_000267.jpg
sketch/umbrella/sketch_329_000271.jpg
sketch/underwear/sketch_330_000041.jpg
sketch/underwear/sketch_330_000053.jpg
sketch/underwear/sketch_330_000090.jpg
sketch/underwear/sketch_330_000091.jpg
sketch/underwear/sketch_330_000094.jpg
sketch/underwear/sketch_330_000096.jpg
sketch/underwear/sketch_330_000106.jpg
sketch/underwear/sketch_330_000131.jpg
sketch/van/sketch_331_000001.jpg
sketch/van/sketch_331_000023.jpg
sketch/van/sketch_331_000066.jpg
sketch/van/sketch_331_000098.jpg
sketch/van/sketch_331_000137.jpg
sketch/vase/sketch_332_000016.jpg
sketch/vase/sketch_332_000097.jpg
sketch/vase/sketch_332_000112.jpg
sketch/vase/sketch_332_000166.jpg
sketch/vase/sketch_332_000183.jpg
sketch/violin/sketch_333_000103.jpg
sketch/washing_machine/sketch_334_000054.jpg
sketch/washing_machine/sketch_334_000068.jpg
sketch/washing_machine/sketch_334_000081.jpg
sketch/washing_machine/sketch_334_000084.jpg
sketch/watermelon/sketch_335_000108.jpg
sketch/waterslide/sketch_336_000032.jpg
sketch/waterslide/sketch_336_000038.jpg
sketch/waterslide/sketch_336_000046.jpg
sketch/waterslide/sketch_336_000051.jpg
sketch/waterslide/sketch_336_000056.jpg
sketch/waterslide/sketch_336_000067.jpg
sketch/waterslide/sketch_336_000088.jpg
sketch/waterslide/sketch_336_000103.jpg
sketch/whale/sketch_337_000012.jpg
sketch/whale/sketch_337_000058.jpg
sketch/whale/sketch_337_000106.jpg
sketch/wheel/sketch_338_000154.jpg
sketch/windmill/sketch_339_000021.jpg
sketch/windmill/sketch_339_000106.jpg
sketch/windmill/sketch_339_000151.jpg
sketch/wine_bottle/sketch_340_000021.jpg
sketch/wine_bottle/sketch_340_000059.jpg
sketch/wine_bottle/sketch_340_000078.jpg
sketch/wine_bottle/sketch_340_000090.jpg
sketch/wine_bottle/sketch_340_000091.jpg
sketch/wine_bottle/sketch_340_000095.jpg
sketch/wine_bottle/sketch_340_000119.jpg
sketch/wine_bottle/sketch_340_000159.jpg
sketch/wine_bottle/sketch_340_000222.jpg
sketch/wine_glass/sketch_341_000004.jpg
sketch/wine_glass/sketch_341_000006.jpg
sketch/wine_glass/sketch_341_000020.jpg
sketch/wine_glass/sketch_341_000028.jpg
sketch/wine_glass/sketch_341_000096.jpg
sketch/wine_glass/sketch_341_000105.jpg
sketch/wine_glass/sketch_341_000107.jpg
sketch/wine_glass/sketch_341_000113.jpg
sketch/wine_glass/sketch_341_000125.jpg
sketch/wine_glass/sketch_341_000127.jpg
sketch/wine_glass/sketch_341_000148.jpg
sketch/wine_glass/sketch_341_000162.jpg
sketch/wine_glass/sketch_341_000163.jpg
sketch/wine_glass/sketch_341_000167.jpg
sketch/wine_glass/sketch_341_000180.jpg
sketch/wine_glass/sketch_341_000181.jpg
sketch/wine_glass/sketch_341_000186.jpg
sketch/wine_glass/sketch_341_000209.jpg
sketch/wine_glass/sketch_341_000211.jpg
sketch/wine_glass/sketch_341_000217.jpg
sketch/wine_glass/sketch_341_000225.jpg
sketch/wine_glass/sketch_341_000239.jpg
sketch/wristwatch/sketch_342_000006.jpg
sketch/wristwatch/sketch_342_000013.jpg
sketch/wristwatch/sketch_342_000024.jpg
sketch/wristwatch/sketch_342_000077.jpg
sketch/wristwatch/sketch_342_000085.jpg
sketch/wristwatch/sketch_342_000086.jpg
sketch/wristwatch/sketch_342_000092.jpg
sketch/wristwatch/sketch_342_000094.jpg
sketch/wristwatch/sketch_342_000101.jpg
sketch/wristwatch/sketch_342_000103.jpg
sketch/wristwatch/sketch_342_000115.jpg
sketch/wristwatch/sketch_342_000143.jpg
sketch/wristwatch/sketch_342_000172.jpg
sketch/wristwatch/sketch_342_000208.jpg
sketch/wristwatch/sketch_342_000209.jpg
sketch/wristwatch/sketch_342_000217.jpg
sketch/zebra/sketch_344_000055.jpg
sketch/zebra/sketch_344_000121.jpg
sketch/zebra/sketch_344_000133.jpg
================================================
FILE: domain_generalization/swad/domainbed/models/mixstyle.py
================================================
"""
https://github.com/KaiyangZhou/mixstyle-release/blob/master/imcls/models/mixstyle.py
"""
import random
import torch
import torch.nn as nn
class MixStyle(nn.Module):
"""MixStyle.
Reference:
Zhou et al. Domain Generalization with MixStyle. ICLR 2021.
"""
def __init__(self, p=0.5, alpha=0.3, eps=1e-6):
"""
Args:
p (float): probability of using MixStyle.
alpha (float): parameter of the Beta distribution.
eps (float): scaling parameter to avoid numerical issues.
"""
super().__init__()
self.p = p
self.beta = torch.distributions.Beta(alpha, alpha)
self.eps = eps
self.alpha = alpha
print("* MixStyle params")
print(f"- p: {p}")
print(f"- alpha: {alpha}")
def __repr__(self):
return f"MixStyle(p={self.p}, alpha={self.alpha}, eps={self.eps})"
def forward(self, x):
if not self.training:
return x
if random.random() > self.p:
return x
B = x.size(0)
mu = x.mean(dim=[2, 3], keepdim=True)
var = x.var(dim=[2, 3], keepdim=True)
sig = (var + self.eps).sqrt()
mu, sig = mu.detach(), sig.detach()
x_normed = (x - mu) / sig
lmda = self.beta.sample((B, 1, 1, 1))
lmda = lmda.to(x.device)
perm = torch.randperm(B)
mu2, sig2 = mu[perm], sig[perm]
mu_mix = mu * lmda + mu2 * (1 - lmda)
sig_mix = sig * lmda + sig2 * (1 - lmda)
return x_normed * sig_mix + mu_mix
class MixStyle2(nn.Module):
"""MixStyle (w/ domain prior).
The input should contain two equal-sized mini-batches from two distinct domains.
Reference:
Zhou et al. Domain Generalization with MixStyle. ICLR 2021.
"""
def __init__(self, p=0.5, alpha=0.3, eps=1e-6):
"""
Args:
p (float): probability of using MixStyle.
alpha (float): parameter of the Beta distribution.
eps (float): scaling parameter to avoid numerical issues.
"""
super().__init__()
self.p = p
self.beta = torch.distributions.Beta(alpha, alpha)
self.eps = eps
self.alpha = alpha
print("* MixStyle params")
print(f"- p: {p}")
print(f"- alpha: {alpha}")
def __repr__(self):
return f"MixStyle(p={self.p}, alpha={self.alpha}, eps={self.eps})"
def forward(self, x):
"""
For the input x, the first half comes from one domain,
while the second half comes from the other domain.
"""
if not self.training:
return x
if random.random() > self.p:
return x
B = x.size(0)
mu = x.mean(dim=[2, 3], keepdim=True)
var = x.var(dim=[2, 3], keepdim=True)
sig = (var + self.eps).sqrt()
mu, sig = mu.detach(), sig.detach()
x_normed = (x - mu) / sig
lmda = self.beta.sample((B, 1, 1, 1))
lmda = lmda.to(x.device)
perm = torch.arange(B - 1, -1, -1) # inverse index
perm_b, perm_a = perm.chunk(2)
perm_b = perm_b[torch.randperm(B // 2)]
perm_a = perm_a[torch.randperm(B // 2)]
perm = torch.cat([perm_b, perm_a], 0)
mu2, sig2 = mu[perm], sig[perm]
mu_mix = mu * lmda + mu2 * (1 - lmda)
sig_mix = sig * lmda + sig2 * (1 - lmda)
return x_normed * sig_mix + mu_mix
================================================
FILE: domain_generalization/swad/domainbed/models/resnet_mixstyle.py
================================================
"""MixStyle w/ random shuffle
https://github.com/KaiyangZhou/mixstyle-release/blob/master/imcls/models/resnet_mixstyle.py
"""
import torch
import torch.nn as nn
import torch.utils.model_zoo as model_zoo
from .mixstyle import MixStyle
model_urls = {
"resnet18": "https://download.pytorch.org/models/resnet18-5c106cde.pth",
"resnet34": "https://download.pytorch.org/models/resnet34-333f7ec4.pth",
"resnet50": "https://download.pytorch.org/models/resnet50-19c8e357.pth",
"resnet101": "https://download.pytorch.org/models/resnet101-5d3b4d8f.pth",
"resnet152": "https://download.pytorch.org/models/resnet152-b121ed2d.pth",
}
def conv3x3(in_planes, out_planes, stride=1):
"""3x3 convolution with padding"""
return nn.Conv2d(in_planes, out_planes, kernel_size=3, stride=stride, padding=1, bias=False)
class BasicBlock(nn.Module):
expansion = 1
def __init__(self, inplanes, planes, stride=1, downsample=None):
super().__init__()
self.conv1 = conv3x3(inplanes, planes, stride)
self.bn1 = nn.BatchNorm2d(planes)
self.relu = nn.ReLU(inplace=True)
self.conv2 = conv3x3(planes, planes)
self.bn2 = nn.BatchNorm2d(planes)
self.downsample = downsample
self.stride = stride
def forward(self, x):
residual = x
out = self.conv1(x)
out = self.bn1(out)
out = self.relu(out)
out = self.conv2(out)
out = self.bn2(out)
if self.downsample is not None:
residual = self.downsample(x)
out += residual
out = self.relu(out)
return out
class Bottleneck(nn.Module):
expansion = 4
def __init__(self, inplanes, planes, stride=1, downsample=None):
super().__init__()
self.conv1 = nn.Conv2d(inplanes, planes, kernel_size=1, bias=False)
self.bn1 = nn.BatchNorm2d(planes)
self.conv2 = nn.Conv2d(planes, planes, kernel_size=3, stride=stride, padding=1, bias=False)
self.bn2 = nn.BatchNorm2d(planes)
self.conv3 = nn.Conv2d(planes, planes * self.expansion, kernel_size=1, bias=False)
self.bn3 = nn.BatchNorm2d(planes * self.expansion)
self.relu = nn.ReLU(inplace=True)
self.downsample = downsample
self.stride = stride
def forward(self, x):
residual = x
out = self.conv1(x)
out = self.bn1(out)
out = self.relu(out)
out = self.conv2(out)
out = self.bn2(out)
out = self.relu(out)
out = self.conv3(out)
out = self.bn3(out)
if self.downsample is not None:
residual = self.downsample(x)
out += residual
out = self.relu(out)
return out
class ResNet(nn.Module):
def __init__(
self, block, layers, mixstyle_layers=[], mixstyle_p=0.5, mixstyle_alpha=0.3, **kwargs
):
self.inplanes = 64
super().__init__()
# backbone network
self.conv1 = nn.Conv2d(3, 64, kernel_size=7, stride=2, padding=3, bias=False)
self.bn1 = nn.BatchNorm2d(64)
self.relu = nn.ReLU(inplace=True)
self.maxpool = nn.MaxPool2d(kernel_size=3, stride=2, padding=1)
self.layer1 = self._make_layer(block, 64, layers[0])
self.layer2 = self._make_layer(block, 128, layers[1], stride=2)
self.layer3 = self._make_layer(block, 256, layers[2], stride=2)
self.layer4 = self._make_layer(block, 512, layers[3], stride=2)
self.global_avgpool = nn.AdaptiveAvgPool2d(1)
self.mixstyle = None
if mixstyle_layers:
self.mixstyle = MixStyle(p=mixstyle_p, alpha=mixstyle_alpha)
for layer_name in mixstyle_layers:
assert layer_name in ["conv1", "conv2_x", "conv3_x", "conv4_x", "conv5_x"]
print("Insert MixStyle after the following layers: {}".format(mixstyle_layers))
self.mixstyle_layers = mixstyle_layers
self._out_features = 512 * block.expansion
self.fc = nn.Identity() # for DomainBed compatibility
self._init_params()
def _make_layer(self, block, planes, blocks, stride=1):
downsample = None
if stride != 1 or self.inplanes != planes * block.expansion:
downsample = nn.Sequential(
nn.Conv2d(
self.inplanes,
planes * block.expansion,
kernel_size=1,
stride=stride,
bias=False,
),
nn.BatchNorm2d(planes * block.expansion),
)
layers = []
layers.append(block(self.inplanes, planes, stride, downsample))
self.inplanes = planes * block.expansion
for i in range(1, blocks):
layers.append(block(self.inplanes, planes))
return nn.Sequential(*layers)
def _init_params(self):
for m in self.modules():
if isinstance(m, nn.Conv2d):
nn.init.kaiming_normal_(m.weight, mode="fan_out", nonlinearity="relu")
if m.bias is not None:
nn.init.constant_(m.bias, 0)
elif isinstance(m, nn.BatchNorm2d):
nn.init.constant_(m.weight, 1)
nn.init.constant_(m.bias, 0)
elif isinstance(m, nn.BatchNorm1d):
nn.init.constant_(m.weight, 1)
nn.init.constant_(m.bias, 0)
elif isinstance(m, nn.Linear):
nn.init.normal_(m.weight, 0, 0.01)
if m.bias is not None:
nn.init.constant_(m.bias, 0)
def compute_style(self, x):
mu = x.mean(dim=[2, 3])
sig = x.std(dim=[2, 3])
return torch.cat([mu, sig], 1)
def featuremaps(self, x):
x = self.conv1(x)
x = self.bn1(x)
x = self.relu(x)
x = self.maxpool(x)
x = self.layer1(x)
if "conv2_x" in self.mixstyle_layers:
x = self.mixstyle(x)
x = self.layer2(x)
if "conv3_x" in self.mixstyle_layers:
x = self.mixstyle(x)
x = self.layer3(x)
if "conv4_x" in self.mixstyle_layers:
x = self.mixstyle(x)
x = self.layer4(x)
if "conv5_x" in self.mixstyle_layers:
x = self.mixstyle(x)
return x
def forward(self, x):
f = self.featuremaps(x)
v = self.global_avgpool(f)
return v.view(v.size(0), -1)
def init_pretrained_weights(model, model_url):
pretrain_dict = model_zoo.load_url(model_url)
model.load_state_dict(pretrain_dict, strict=False)
def resnet18_mixstyle_L234_p0d5_a0d1(pretrained=True, **kwargs):
model = ResNet(
block=BasicBlock,
layers=[2, 2, 2, 2],
mixstyle_layers=["conv2_x", "conv3_x", "conv4_x"],
mixstyle_p=0.5,
mixstyle_alpha=0.1,
)
if pretrained:
init_pretrained_weights(model, model_urls["resnet18"])
return model
def resnet50_mixstyle_L234_p0d5_a0d1(pretrained=True, **kwargs):
model = ResNet(
block=Bottleneck,
layers=[3, 4, 6, 3],
mixstyle_layers=["conv2_x", "conv3_x", "conv4_x"],
mixstyle_p=0.5,
mixstyle_alpha=0.1,
)
if pretrained:
init_pretrained_weights(model, model_urls["resnet50"])
return model
================================================
FILE: domain_generalization/swad/domainbed/models/resnet_mixstyle2.py
================================================
"""MixStyle w/ domain label
https://github.com/KaiyangZhou/mixstyle-release/blob/master/imcls/models/resnet_mixstyle2.py
"""
import random
import torch
import torch.nn as nn
import torch.utils.model_zoo as model_zoo
from .mixstyle import MixStyle2 as MixStyle
model_urls = {
"resnet18": "https://download.pytorch.org/models/resnet18-5c106cde.pth",
"resnet34": "https://download.pytorch.org/models/resnet34-333f7ec4.pth",
"resnet50": "https://download.pytorch.org/models/resnet50-19c8e357.pth",
"resnet101": "https://download.pytorch.org/models/resnet101-5d3b4d8f.pth",
"resnet152": "https://download.pytorch.org/models/resnet152-b121ed2d.pth",
}
def conv3x3(in_planes, out_planes, stride=1):
"""3x3 convolution with padding"""
return nn.Conv2d(in_planes, out_planes, kernel_size=3, stride=stride, padding=1, bias=False)
class BasicBlock(nn.Module):
expansion = 1
def __init__(self, inplanes, planes, stride=1, downsample=None):
super().__init__()
self.conv1 = conv3x3(inplanes, planes, stride)
self.bn1 = nn.BatchNorm2d(planes)
self.relu = nn.ReLU(inplace=True)
self.conv2 = conv3x3(planes, planes)
self.bn2 = nn.BatchNorm2d(planes)
self.downsample = downsample
self.stride = stride
def forward(self, x):
residual = x
out = self.conv1(x)
out = self.bn1(out)
out = self.relu(out)
out = self.conv2(out)
out = self.bn2(out)
if self.downsample is not None:
residual = self.downsample(x)
out += residual
out = self.relu(out)
return out
class Bottleneck(nn.Module):
expansion = 4
def __init__(self, inplanes, planes, stride=1, downsample=None):
super().__init__()
self.conv1 = nn.Conv2d(inplanes, planes, kernel_size=1, bias=False)
self.bn1 = nn.BatchNorm2d(planes)
self.conv2 = nn.Conv2d(planes, planes, kernel_size=3, stride=stride, padding=1, bias=False)
self.bn2 = nn.BatchNorm2d(planes)
self.conv3 = nn.Conv2d(planes, planes * self.expansion, kernel_size=1, bias=False)
self.bn3 = nn.BatchNorm2d(planes * self.expansion)
self.relu = nn.ReLU(inplace=True)
self.downsample = downsample
self.stride = stride
def forward(self, x):
residual = x
out = self.conv1(x)
out = self.bn1(out)
out = self.relu(out)
out = self.conv2(out)
out = self.bn2(out)
out = self.relu(out)
out = self.conv3(out)
out = self.bn3(out)
if self.downsample is not None:
residual = self.downsample(x)
out += residual
out = self.relu(out)
return out
class ResNet(nn.Module):
def __init__(
self, block, layers, mixstyle_layers=[], mixstyle_p=0.5, mixstyle_alpha=0.3, **kwargs
):
self.inplanes = 64
super().__init__()
# backbone network
self.conv1 = nn.Conv2d(3, 64, kernel_size=7, stride=2, padding=3, bias=False)
self.bn1 = nn.BatchNorm2d(64)
self.relu = nn.ReLU(inplace=True)
self.maxpool = nn.MaxPool2d(kernel_size=3, stride=2, padding=1)
self.layer1 = self._make_layer(block, 64, layers[0])
self.layer2 = self._make_layer(block, 128, layers[1], stride=2)
self.layer3 = self._make_layer(block, 256, layers[2], stride=2)
self.layer4 = self._make_layer(block, 512, layers[3], stride=2)
self.global_avgpool = nn.AdaptiveAvgPool2d(1)
self.mixstyle = None
if mixstyle_layers:
self.mixstyle = MixStyle(p=mixstyle_p, alpha=mixstyle_alpha)
for layer_name in mixstyle_layers:
assert layer_name in ["conv1", "conv2_x", "conv3_x", "conv4_x", "conv5_x"]
print("Insert MixStyle after the following layers: {}".format(mixstyle_layers))
self.mixstyle_layers = mixstyle_layers
self._out_features = 512 * block.expansion
self.fc = nn.Identity() # for DomainBed compatibility
self._init_params()
def _make_layer(self, block, planes, blocks, stride=1):
downsample = None
if stride != 1 or self.inplanes != planes * block.expansion:
downsample = nn.Sequential(
nn.Conv2d(
self.inplanes,
planes * block.expansion,
kernel_size=1,
stride=stride,
bias=False,
),
nn.BatchNorm2d(planes * block.expansion),
)
layers = []
layers.append(block(self.inplanes, planes, stride, downsample))
self.inplanes = planes * block.expansion
for i in range(1, blocks):
layers.append(block(self.inplanes, planes))
return nn.Sequential(*layers)
def _init_params(self):
for m in self.modules():
if isinstance(m, nn.Conv2d):
nn.init.kaiming_normal_(m.weight, mode="fan_out", nonlinearity="relu")
if m.bias is not None:
nn.init.constant_(m.bias, 0)
elif isinstance(m, nn.BatchNorm2d):
nn.init.constant_(m.weight, 1)
nn.init.constant_(m.bias, 0)
elif isinstance(m, nn.BatchNorm1d):
nn.init.constant_(m.weight, 1)
nn.init.constant_(m.bias, 0)
elif isinstance(m, nn.Linear):
nn.init.normal_(m.weight, 0, 0.01)
if m.bias is not None:
nn.init.constant_(m.bias, 0)
def compute_style(self, x):
mu = x.mean(dim=[2, 3])
sig = x.std(dim=[2, 3])
return torch.cat([mu, sig], 1)
def featuremaps(self, x):
x = self.conv1(x)
x = self.bn1(x)
x = self.relu(x)
x = self.maxpool(x)
x = self.layer1(x)
if "conv2_x" in self.mixstyle_layers:
x = self.mixstyle(x)
x = self.layer2(x)
if "conv3_x" in self.mixstyle_layers:
x = self.mixstyle(x)
x = self.layer3(x)
if "conv4_x" in self.mixstyle_layers:
x = self.mixstyle(x)
x = self.layer4(x)
if "conv5_x" in self.mixstyle_layers:
x = self.mixstyle(x)
return x
def forward(self, x):
f = self.featuremaps(x)
v = self.global_avgpool(f)
return v.view(v.size(0), -1)
def init_pretrained_weights(model, model_url):
pretrain_dict = model_zoo.load_url(model_url)
model.load_state_dict(pretrain_dict, strict=False)
"""
Residual network configurations:
--
resnet18: block=BasicBlock, layers=[2, 2, 2, 2]
resnet34: block=BasicBlock, layers=[3, 4, 6, 3]
resnet50: block=Bottleneck, layers=[3, 4, 6, 3]
resnet101: block=Bottleneck, layers=[3, 4, 23, 3]
resnet152: block=Bottleneck, layers=[3, 8, 36, 3]
"""
def resnet18_mixstyle2_L234_p0d5_a0d1(pretrained=True, **kwargs):
model = ResNet(
block=BasicBlock,
layers=[2, 2, 2, 2],
mixstyle_layers=["conv2_x", "conv3_x", "conv4_x"],
mixstyle_p=0.5,
mixstyle_alpha=0.1,
)
if pretrained:
init_pretrained_weights(model, model_urls["resnet18"])
return model
def resnet50_mixstyle2_L234_p0d5_a0d1(pretrained=True, **kwargs):
model = ResNet(
block=Bottleneck,
layers=[3, 4, 6, 3],
mixstyle_layers=["conv2_x", "conv3_x", "conv4_x"],
mixstyle_p=0.5,
mixstyle_alpha=0.1,
)
if pretrained:
init_pretrained_weights(model, model_urls["resnet50"])
return model
================================================
FILE: domain_generalization/swad/domainbed/networks.py
================================================
# Copyright (c) Facebook, Inc. and its affiliates. All Rights Reserved
import torch
import torch.nn as nn
import torch.nn.functional as F
import torchvision.models
from domainbed.lib import wide_resnet
class Identity(nn.Module):
"""An identity layer"""
def __init__(self):
super(Identity, self).__init__()
def forward(self, x):
return x
class SqueezeLastTwo(nn.Module):
"""
A module which squeezes the last two dimensions,
ordinary squeeze can be a problem for batch size 1
"""
def __init__(self):
super(SqueezeLastTwo, self).__init__()
def forward(self, x):
return x.view(x.shape[0], x.shape[1])
class MLP(nn.Module):
"""Just an MLP"""
def __init__(self, n_inputs, n_outputs, hparams):
super(MLP, self).__init__()
self.input = nn.Linear(n_inputs, hparams["mlp_width"])
self.dropout = nn.Dropout(hparams["mlp_dropout"])
self.hiddens = nn.ModuleList(
[
nn.Linear(hparams["mlp_width"], hparams["mlp_width"])
for _ in range(hparams["mlp_depth"] - 2)
]
)
self.output = nn.Linear(hparams["mlp_width"], n_outputs)
self.n_outputs = n_outputs
def forward(self, x):
x = self.input(x)
x = self.dropout(x)
x = F.relu(x)
for hidden in self.hiddens:
x = hidden(x)
x = self.dropout(x)
x = F.relu(x)
x = self.output(x)
return x
class ResNet(torch.nn.Module):
"""ResNet with the softmax chopped off and the batchnorm frozen"""
def __init__(self, input_shape, hparams, network=None):
super(ResNet, self).__init__()
if hparams["resnet18"]:
if network is None:
network = torchvision.models.resnet18(pretrained=hparams["pretrained"])
self.network = network
self.n_outputs = 512
else:
if network is None:
network = torchvision.models.resnet50(pretrained=hparams["pretrained"])
self.network = network
self.n_outputs = 2048
# adapt number of channels
nc = input_shape[0]
if nc != 3:
tmp = self.network.conv1.weight.data.clone()
self.network.conv1 = nn.Conv2d(
nc, 64, kernel_size=(7, 7), stride=(2, 2), padding=(3, 3), bias=False
)
for i in range(nc):
self.network.conv1.weight.data[:, i, :, :] = tmp[:, i % 3, :, :]
# save memory
del self.network.fc
self.network.fc = Identity()
self.hparams = hparams
self.dropout = nn.Dropout(hparams["resnet_dropout"])
self.freeze_bn()
def forward(self, x):
"""Encode x into a feature vector of size n_outputs."""
return self.dropout(self.network(x))
def train(self, mode=True):
"""
Override the default train() to freeze the BN parameters
"""
super().train(mode)
self.freeze_bn()
def freeze_bn(self):
if self.hparams["freeze_bn"] is False:
return
for m in self.network.modules():
if isinstance(m, nn.BatchNorm2d):
m.eval()
class MNIST_CNN(nn.Module):
"""
Hand-tuned architecture for MNIST.
Weirdness I've noticed so far with this architecture:
- adding a linear layer after the mean-pool in features hurts
RotatedMNIST-100 generalization severely.
"""
n_outputs = 128
def __init__(self, input_shape):
super(MNIST_CNN, self).__init__()
self.conv1 = nn.Conv2d(input_shape[0], 64, 3, 1, padding=1)
self.conv2 = nn.Conv2d(64, 128, 3, stride=2, padding=1)
self.conv3 = nn.Conv2d(128, 128, 3, 1, padding=1)
self.conv4 = nn.Conv2d(128, 128, 3, 1, padding=1)
self.bn0 = nn.GroupNorm(8, 64)
self.bn1 = nn.GroupNorm(8, 128)
self.bn2 = nn.GroupNorm(8, 128)
self.bn3 = nn.GroupNorm(8, 128)
self.avgpool = nn.AdaptiveAvgPool2d((1, 1))
self.squeezeLastTwo = SqueezeLastTwo()
def forward(self, x):
x = self.conv1(x)
x = F.relu(x)
x = self.bn0(x)
x = self.conv2(x)
x = F.relu(x)
x = self.bn1(x)
x = self.conv3(x)
x = F.relu(x)
x = self.bn2(x)
x = self.conv4(x)
x = F.relu(x)
x = self.bn3(x)
x = self.avgpool(x)
x = self.squeezeLastTwo(x)
return x
class ContextNet(nn.Module):
def __init__(self, input_shape):
super(ContextNet, self).__init__()
# Keep same dimensions
padding = (5 - 1) // 2
self.context_net = nn.Sequential(
nn.Conv2d(input_shape[0], 64, 5, padding=padding),
nn.BatchNorm2d(64),
nn.ReLU(),
nn.Conv2d(64, 64, 5, padding=padding),
nn.BatchNorm2d(64),
nn.ReLU(),
nn.Conv2d(64, 1, 5, padding=padding),
)
def forward(self, x):
return self.context_net(x)
def Featurizer(input_shape, hparams):
"""Auto-select an appropriate featurizer for the given input shape."""
if len(input_shape) == 1:
return MLP(input_shape[0], 128, hparams)
elif input_shape[1:3] == (28, 28):
return MNIST_CNN(input_shape)
elif input_shape[1:3] == (32, 32):
return wide_resnet.Wide_ResNet(input_shape, 16, 2, 0.0)
elif input_shape[1:3] == (224, 224):
return ResNet(input_shape, hparams)
else:
raise NotImplementedError(f"Input shape {input_shape} is not supported")
================================================
FILE: domain_generalization/swad/domainbed/optimizers.py
================================================
import torch
def get_optimizer(name, params, **kwargs):
name = name.lower()
optimizers = {"adam": torch.optim.Adam, "sgd": torch.optim.SGD, "adamw": torch.optim.AdamW}
optim_cls = optimizers[name]
return optim_cls(params, **kwargs)
================================================
FILE: domain_generalization/swad/domainbed/scripts/download.py
================================================
# Copyright (c) Facebook, Inc. and its affiliates. All Rights Reserved
from torchvision.datasets import MNIST
import xml.etree.ElementTree as ET
from zipfile import ZipFile
import argparse
import tarfile
import shutil
import gdown
import uuid
import json
import os
# utils #######################################################################
def stage_path(data_dir, name):
full_path = os.path.join(data_dir, name)
if not os.path.exists(full_path):
os.makedirs(full_path)
return full_path
def download_and_extract(url, dst, remove=True):
gdown.download(url, dst, quiet=False)
if dst.endswith(".tar.gz"):
tar = tarfile.open(dst, "r:gz")
tar.extractall(os.path.dirname(dst))
tar.close()
if dst.endswith(".tar"):
tar = tarfile.open(dst, "r:")
tar.extractall(os.path.dirname(dst))
tar.close()
if dst.endswith(".zip"):
zf = ZipFile(dst, "r")
zf.extractall(os.path.dirname(dst))
zf.close()
if remove:
os.remove(dst)
# VLCS ########################################################################
# Slower, but builds dataset from the original sources
#
# def download_vlcs(data_dir):
# full_path = stage_path(data_dir, "VLCS")
#
# tmp_path = os.path.join(full_path, "tmp/")
# if not os.path.exists(tmp_path):
# os.makedirs(tmp_path)
#
# with open("domainbed/misc/vlcs_files.txt", "r") as f:
# lines = f.readlines()
# files = [line.strip().split() for line in lines]
#
# download_and_extract("http://pjreddie.com/media/files/VOCtrainval_06-Nov-2007.tar",
# os.path.join(tmp_path, "voc2007_trainval.tar"))
#
# download_and_extract("https://drive.google.com/uc?id=1I8ydxaAQunz9R_qFFdBFtw6rFTUW9goz",
# os.path.join(tmp_path, "caltech101.tar.gz"))
#
# download_and_extract("http://groups.csail.mit.edu/vision/Hcontext/data/sun09_hcontext.tar",
# os.path.join(tmp_path, "sun09_hcontext.tar"))
#
# tar = tarfile.open(os.path.join(tmp_path, "sun09.tar"), "r:")
# tar.extractall(tmp_path)
# tar.close()
#
# for src, dst in files:
# class_folder = os.path.join(data_dir, dst)
#
# if not os.path.exists(class_folder):
# os.makedirs(class_folder)
#
# dst = os.path.join(class_folder, uuid.uuid4().hex + ".jpg")
#
# if "labelme" in src:
# # download labelme from the web
# gdown.download(src, dst, quiet=False)
# else:
# src = os.path.join(tmp_path, src)
# shutil.copyfile(src, dst)
#
# shutil.rmtree(tmp_path)
def download_vlcs(data_dir):
# Original URL: http://www.eecs.qmul.ac.uk/~dl307/project_iccv2017
full_path = stage_path(data_dir, "VLCS")
download_and_extract(
"https://drive.google.com/uc?id=1skwblH1_okBwxWxmRsp9_qi15hyPpxg8",
os.path.join(data_dir, "VLCS.tar.gz"),
)
# MNIST #######################################################################
def download_mnist(data_dir):
# Original URL: http://yann.lecun.com/exdb/mnist/
full_path = stage_path(data_dir, "MNIST")
MNIST(full_path, download=True)
# PACS ########################################################################
def download_pacs(data_dir):
# Original URL: http://www.eecs.qmul.ac.uk/~dl307/project_iccv2017
full_path = stage_path(data_dir, "PACS")
download_and_extract(
"https://drive.google.com/uc?id=1JFr8f805nMUelQWWmfnJR3y4_SYoN5Pd",
os.path.join(data_dir, "PACS.zip"),
)
os.rename(os.path.join(data_dir, "kfold"), full_path)
# Office-Home #################################################################
def download_office_home(data_dir):
# Original URL: http://hemanthdv.org/OfficeHome-Dataset/
full_path = stage_path(data_dir, "office_home")
download_and_extract(
"https://drive.google.com/uc?id=0B81rNlvomiwed0V1YUxQdC1uOTg",
os.path.join(data_dir, "office_home.zip"),
)
os.rename(os.path.join(data_dir, "OfficeHomeDataset_10072016"), full_path)
# DomainNET ###################################################################
def download_domain_net(data_dir):
# Original URL: http://ai.bu.edu/M3SDA/
full_path = stage_path(data_dir, "domain_net")
urls = [
"http://csr.bu.edu/ftp/visda/2019/multi-source/groundtruth/clipart.zip",
"http://csr.bu.edu/ftp/visda/2019/multi-source/infograph.zip",
"http://csr.bu.edu/ftp/visda/2019/multi-source/groundtruth/painting.zip",
"http://csr.bu.edu/ftp/visda/2019/multi-source/quickdraw.zip",
"http://csr.bu.edu/ftp/visda/2019/multi-source/real.zip",
"http://csr.bu.edu/ftp/visda/2019/multi-source/sketch.zip",
]
for url in urls:
download_and_extract(url, os.path.join(full_path, url.split("/")[-1]))
with open("domainbed/misc/domain_net_duplicates.txt", "r") as f:
for line in f.readlines():
try:
os.remove(os.path.join(full_path, line.strip()))
except OSError:
pass
# TerraIncognita ##############################################################
def download_terra_incognita(data_dir):
# Original URL: https://beerys.github.io/CaltechCameraTraps/
# New URL: http://lila.science/datasets/caltech-camera-traps
full_path = stage_path(data_dir, "terra_incognita")
download_and_extract(
"https://lilablobssc.blob.core.windows.net/caltechcameratraps/eccv_18_all_images_sm.tar.gz",
os.path.join(full_path, "terra_incognita_images.tar.gz"),
)
download_and_extract(
"https://lilablobssc.blob.core.windows.net/caltechcameratraps/labels/caltech_camera_traps.json.zip",
os.path.join(full_path, "caltech_camera_traps.json.zip"),
)
include_locations = ["38", "46", "100", "43"]
include_categories = [
"bird",
"bobcat",
"cat",
"coyote",
"dog",
"empty",
"opossum",
"rabbit",
"raccoon",
"squirrel",
]
images_folder = os.path.join(full_path, "eccv_18_all_images_sm/")
annotations_file = os.path.join(full_path, "caltech_images_20210113.json")
destination_folder = full_path
stats = {}
if not os.path.exists(destination_folder):
os.mkdir(destination_folder)
with open(annotations_file, "r") as f:
data = json.load(f)
category_dict = {}
for item in data["categories"]:
category_dict[item["id"]] = item["name"]
for image in data["images"]:
image_location = image["location"]
if image_location not in include_locations:
continue
loc_folder = os.path.join(destination_folder, "location_" + str(image_location) + "/")
if not os.path.exists(loc_folder):
os.mkdir(loc_folder)
image_id = image["id"]
image_fname = image["file_name"]
for annotation in data["annotations"]:
if annotation["image_id"] == image_id:
if image_location not in stats:
stats[image_location] = {}
category = category_dict[annotation["category_id"]]
if category not in include_categories:
continue
if category not in stats[image_location]:
stats[image_location][category] = 0
else:
stats[image_location][category] += 1
loc_cat_folder = os.path.join(loc_folder, category + "/")
if not os.path.exists(loc_cat_folder):
os.mkdir(loc_cat_folder)
dst_path = os.path.join(loc_cat_folder, image_fname)
src_path = os.path.join(images_folder, image_fname)
shutil.copyfile(src_path, dst_path)
shutil.rmtree(images_folder)
os.remove(annotations_file)
if __name__ == "__main__":
parser = argparse.ArgumentParser(description="Download datasets")
parser.add_argument("--data_dir", type=str, required=True)
args = parser.parse_args()
download_mnist(args.data_dir)
download_pacs(args.data_dir)
download_vlcs(args.data_dir)
download_domain_net(args.data_dir)
download_office_home(args.data_dir)
download_terra_incognita(args.data_dir)
================================================
FILE: domain_generalization/swad/domainbed/swad.py
================================================
import copy
from collections import deque
import numpy as np
from domainbed.lib import swa_utils
class SWADBase:
def update_and_evaluate(self, segment_swa, val_acc, val_loss, prt_fn):
raise NotImplementedError()
def get_final_model(self):
raise NotImplementedError()
class IIDMax(SWADBase):
"""SWAD start from iid max acc and select last by iid max swa acc"""
def __init__(self, evaluator, **kwargs):
self.iid_max_acc = 0.0
self.swa_max_acc = 0.0
self.avgmodel = None
self.final_model = None
self.evaluator = evaluator
def update_and_evaluate(self, segment_swa, val_acc, val_loss, prt_fn):
if self.iid_max_acc < val_acc:
self.iid_max_acc = val_acc
self.avgmodel = swa_utils.AveragedModel(segment_swa.module, rm_optimizer=True)
self.avgmodel.start_step = segment_swa.start_step
self.avgmodel.update_parameters(segment_swa.module)
self.avgmodel.end_step = segment_swa.end_step
# evaluate
accuracies, summaries = self.evaluator.evaluate(self.avgmodel)
results = {**summaries, **accuracies}
prt_fn(results, self.avgmodel)
swa_val_acc = results["train_out"]
if swa_val_acc > self.swa_max_acc:
self.swa_max_acc = swa_val_acc
self.final_model = copy.deepcopy(self.avgmodel)
def get_final_model(self):
return self.final_model
class LossValley(SWADBase):
"""IIDMax has a potential problem that bias to validation dataset.
LossValley choose SWAD range by detecting loss valley.
"""
def __init__(self, evaluator, n_converge, n_tolerance, tolerance_ratio, **kwargs):
"""
Args:
evaluator
n_converge: converge detector window size.
n_tolerance: loss min smoothing window size
tolerance_ratio: decision ratio for dead loss valley
"""
self.evaluator = evaluator
self.n_converge = n_converge
self.n_tolerance = n_tolerance
self.tolerance_ratio = tolerance_ratio
self.converge_Q = deque(maxlen=n_converge)
self.smooth_Q = deque(maxlen=n_tolerance)
self.final_model = None
self.converge_step = None
self.dead_valley = False
self.threshold = None
def get_smooth_loss(self, idx):
smooth_loss = min([model.end_loss for model in list(self.smooth_Q)[idx:]])
return smooth_loss
@property
def is_converged(self):
return self.converge_step is not None
def update_and_evaluate(self, segment_swa, val_acc, val_loss, prt_fn):
if self.dead_valley:
return
frozen = copy.deepcopy(segment_swa)
frozen.end_loss = val_loss
self.converge_Q.append(frozen)
self.smooth_Q.append(frozen)
if not self.is_converged:
if len(self.converge_Q) < self.n_converge:
return
min_idx = np.argmin([model.end_loss for model in self.converge_Q])
untilmin_segment_swa = self.converge_Q[min_idx] # until-min segment swa.
if min_idx == 0:
self.converge_step = self.converge_Q[0].end_step
self.final_model = swa_utils.AveragedModel(untilmin_segment_swa)
th_base = np.mean([model.end_loss for model in self.converge_Q])
self.threshold = th_base * (1.0 + self.tolerance_ratio)
if self.n_tolerance < self.n_converge:
for i in range(self.n_converge - self.n_tolerance):
model = self.converge_Q[1 + i]
self.final_model.update_parameters(
model, start_step=model.start_step, end_step=model.end_step
)
elif self.n_tolerance > self.n_converge:
converge_idx = self.n_tolerance - self.n_converge
Q = list(self.smooth_Q)[: converge_idx + 1]
start_idx = 0
for i in reversed(range(len(Q))):
model = Q[i]
if model.end_loss > self.threshold:
start_idx = i + 1
break
for model in Q[start_idx + 1 :]:
self.final_model.update_parameters(
model, start_step=model.start_step, end_step=model.end_step
)
print(
f"Model converged at step {self.converge_step}, "
f"Start step = {self.final_model.start_step}; "
f"Threshold = {self.threshold:.6f}, "
)
return
if self.smooth_Q[0].end_step < self.converge_step:
return
# converged -> loss valley
min_vloss = self.get_smooth_loss(0)
if min_vloss > self.threshold:
self.dead_valley = True
print(f"Valley is dead at step {self.final_model.end_step}")
return
model = self.smooth_Q[0]
self.final_model.update_parameters(
model, start_step=model.start_step, end_step=model.end_step
)
def get_final_model(self):
if not self.is_converged:
self.evaluator.logger.error(
"Requested final model, but model is not yet converged; return last model instead"
)
return self.converge_Q[-1]
if not self.dead_valley:
self.smooth_Q.popleft()
while self.smooth_Q:
smooth_loss = self.get_smooth_loss(0)
if smooth_loss > self.threshold:
break
segment_swa = self.smooth_Q.popleft()
self.final_model.update_parameters(segment_swa, step=segment_swa.end_step)
return self.final_model
================================================
FILE: domain_generalization/swad/domainbed/trainer.py
================================================
import collections
import json
import time
import copy
from pathlib import Path
import numpy as np
import torch
import torch.utils.data
from domainbed.datasets import get_dataset, split_dataset
from domainbed import algorithms
from domainbed.evaluator import Evaluator
from domainbed.lr_scheduler import get_scheduler
from domainbed.lib import misc
from domainbed.lib import swa_utils
from domainbed.lib.query import Q
from domainbed.lib.fast_data_loader import InfiniteDataLoader, FastDataLoader
from domainbed import swad as swad_module
if torch.cuda.is_available():
device = "cuda"
else:
device = "cpu"
def json_handler(v):
if isinstance(v, (Path, range)):
return str(v)
raise TypeError(f"`{type(v)}` is not JSON Serializable")
class TransformerDecorator(torch.nn.Module):
def __init__(self, classifier, add_bt = 0, eval_batch=0):
super(TransformerDecorator, self).__init__()
self.add_bt = add_bt
self.eval_batch = eval_batch
if add_bt:
self.encoder_layers = torch.nn.TransformerEncoderLayer(classifier.in_features, 4, classifier.in_features, 0.5)
# if add_bt == 12:
# self.encoder_layers = torch.nn.TransformerEncoder(self.encoder_layers, 2)
# if add_bt == 2:
# self.encoder_layers = torch.nn.TransformerEncoderLayer(classifier.in_features, 4, 512, 0.5)
self.fc = classifier
def forward(self, x):
if self.training and self.add_bt:
old_x = x
x = x.unsqueeze(1)
x = self.encoder_layers(x)
x = x.squeeze(1)
if self.add_bt in [16, 2, 12]:
x = torch.cat([old_x, x], dim=0)
x = self.fc(x)
return x
def train(test_envs, args, hparams, n_steps, checkpoint_freq, logger, writer, target_env=None):
logger.info("")
#######################################################
# setup dataset & loader
#######################################################
args.real_test_envs = test_envs # for log
algorithm_class = algorithms.get_algorithm_class(args.algorithm)
dataset, in_splits, out_splits = get_dataset(test_envs, args, hparams, algorithm_class)
test_splits = []
if hparams.indomain_test > 0.0:
logger.info("!!! In-domain test mode On !!!")
assert hparams["val_augment"] is False, (
"indomain_test split the val set into val/test sets. "
"Therefore, the val set should be not augmented."
)
val_splits = []
for env_i, (out_split, _weights) in enumerate(out_splits):
n = len(out_split) // 2
seed = misc.seed_hash(args.trial_seed, env_i)
val_split, test_split = split_dataset(out_split, n, seed=seed)
val_splits.append((val_split, None))
test_splits.append((test_split, None))
logger.info(
"env %d: out (#%d) -> val (#%d) / test (#%d)"
% (env_i, len(out_split), len(val_split), len(test_split))
)
out_splits = val_splits
if target_env is not None:
testenv_name = f"te_{dataset.environments[target_env]}"
logger.info(f"Target env = {target_env}")
else:
testenv_properties = [str(dataset.environments[i]) for i in test_envs]
testenv_name = "te_" + "_".join(testenv_properties)
logger.info(
"Testenv name escaping {} -> {}".format(testenv_name, testenv_name.replace(".", ""))
)
testenv_name = testenv_name.replace(".", "")
logger.info(f"Test envs = {test_envs}, name = {testenv_name}")
n_envs = len(dataset)
train_envs = sorted(set(range(n_envs)) - set(test_envs))
iterator = misc.SplitIterator(test_envs)
batch_sizes = np.full([n_envs], hparams["batch_size"], dtype=np.int)
batch_sizes[test_envs] = 0
batch_sizes = batch_sizes.tolist()
logger.info(f"Batch sizes for each domain: {batch_sizes} (total={sum(batch_sizes)})")
# calculate steps per epoch
steps_per_epochs = [
len(env) / batch_size
for (env, _), batch_size in iterator.train(zip(in_splits, batch_sizes))
]
steps_per_epoch = min(steps_per_epochs)
# epoch is computed by steps_per_epoch
prt_steps = ", ".join([f"{step:.2f}" for step in steps_per_epochs])
logger.info(f"steps-per-epoch for each domain: {prt_steps} -> min = {steps_per_epoch:.2f}")
# setup loaders
train_loaders = [
InfiniteDataLoader(
dataset=env,
weights=env_weights,
batch_size=batch_size,
num_workers=dataset.N_WORKERS,
)
for (env, env_weights), batch_size in iterator.train(zip(in_splits, batch_sizes))
]
# setup eval loaders
eval_loaders_kwargs = []
for i, (env, _) in enumerate(in_splits + out_splits + test_splits):
batchsize = hparams["test_batchsize"]
loader_kwargs = {"dataset": env, "batch_size": batchsize, "num_workers": dataset.N_WORKERS}
if args.prebuild_loader:
loader_kwargs = FastDataLoader(**loader_kwargs)
eval_loaders_kwargs.append(loader_kwargs)
eval_weights = [None for _, weights in (in_splits + out_splits + test_splits)]
eval_loader_names = ["env{}_in".format(i) for i in range(len(in_splits))]
eval_loader_names += ["env{}_out".format(i) for i in range(len(out_splits))]
eval_loader_names += ["env{}_inTE".format(i) for i in range(len(test_splits))]
eval_meta = list(zip(eval_loader_names, eval_loaders_kwargs, eval_weights))
#######################################################
# setup algorithm (model)
#######################################################
algorithm = algorithm_class(
dataset.input_shape,
dataset.num_classes,
len(dataset) - len(test_envs),
hparams,
)
algorithm.classifier = TransformerDecorator(algorithm.classifier, args.add_bt)
algorithm.network = torch.nn.Sequential(algorithm.featurizer, algorithm.classifier)
from domainbed.optimizers import get_optimizer
algorithm.optimizer = get_optimizer(
hparams["optimizer"],
[{'params': [p for n, p in algorithm.network.named_parameters() if n.__contains__('encoder')],
'lr': algorithm.hparams["lr"]*args.lr_scale_bt, },
{'params': [p for n, p in algorithm.network.named_parameters() if not n.__contains__('encoder')],
'lr': algorithm.hparams["lr"]}],
lr=algorithm.hparams["lr"],
weight_decay=algorithm.hparams["weight_decay"],
)
print('begin', algorithm.hparams["lr"])
print(algorithm.optimizer)
print(algorithm.classifier)
algorithm.to(device)
n_params = sum([p.numel() for p in algorithm.parameters()])
logger.info("# of params = %d" % n_params)
# setup scheduler
scheduler = get_scheduler(
hparams["scheduler"],
algorithm.optimizer,
hparams["lr"],
n_steps,
)
train_minibatches_iterator = zip(*train_loaders)
checkpoint_vals = collections.defaultdict(lambda: [])
#######################################################
# start training loop
#######################################################
evaluator = Evaluator(
test_envs,
eval_meta,
n_envs,
logger,
evalmode=args.evalmode,
debug=args.debug,
target_env=target_env,
)
swad = None
if hparams["swad"]:
swad_algorithm = swa_utils.AveragedModel(algorithm)
swad_cls = getattr(swad_module, hparams["swad"])
swad = swad_cls(evaluator, **hparams.swad_kwargs)
last_results_keys = None
records = []
epochs_path = args.out_dir / "results.jsonl"
for step in range(n_steps):
step_start_time = time.time()
# batches_dictlist: [{env0_data_key: tensor, env0_...}, env1_..., ...]
batches_dictlist = next(train_minibatches_iterator)
# batches: {data_key: [env0_tensor, ...], ...}
batches = misc.merge_dictlist(batches_dictlist)
# to device
batches = {
key: [tensor.to(device) for tensor in tensorlist] for key, tensorlist in batches.items()
}
inputs = {**batches, "step": step}
step_vals = algorithm.update(**inputs)
for key, val in step_vals.items():
checkpoint_vals[key].append(val)
checkpoint_vals["step_time"].append(time.time() - step_start_time)
if swad:
# swad_algorithm is segment_swa for swad
swad_algorithm.update_parameters(algorithm, step=step)
scheduler.step()
if step % checkpoint_freq == 0:
results = {
"step": step,
"epoch": step / steps_per_epoch,
}
for key, val in checkpoint_vals.items():
results[key] = np.mean(val)
eval_start_time = time.time()
accuracies, summaries = evaluator.evaluate(algorithm)
results["eval_time"] = time.time() - eval_start_time
# results = (epochs, loss, step, step_time)
results_keys = list(summaries.keys()) + sorted(accuracies.keys()) + list(results.keys())
# merge results
results.update(summaries)
results.update(accuracies)
# print
if results_keys != last_results_keys:
logger.info(misc.to_row(results_keys))
last_results_keys = results_keys
logger.info(misc.to_row([results[key] for key in results_keys]))
records.append(copy.deepcopy(results))
# update results to record
results.update({"hparams": dict(hparams), "args": vars(args)})
with open(epochs_path, "a") as f:
f.write(json.dumps(results, sort_keys=True, default=json_handler) + "\n")
checkpoint_vals = collections.defaultdict(lambda: [])
writer.add_scalars_with_prefix(summaries, step, f"{testenv_name}/summary/")
writer.add_scalars_with_prefix(accuracies, step, f"{testenv_name}/all/")
if args.model_save and step >= args.model_save:
ckpt_dir = args.out_dir / "checkpoints"
ckpt_dir.mkdir(exist_ok=True)
test_env_str = ",".join(map(str, test_envs))
filename = "TE{}_{}.pth".format(test_env_str, step)
if len(test_envs) > 1 and target_env is not None:
train_env_str = ",".join(map(str, train_envs))
filename = f"TE{target_env}_TR{train_env_str}_{step}.pth"
path = ckpt_dir / filename
save_dict = {
"args": vars(args),
"model_hparams": dict(hparams),
"test_envs": test_envs,
"model_dict": algorithm.cpu().state_dict(),
}
algorithm.cuda()
if not args.debug:
torch.save(save_dict, path)
else:
logger.debug("DEBUG Mode -> no save (org path: %s)" % path)
# swad
if swad:
def prt_results_fn(results, avgmodel):
step_str = f" [{avgmodel.start_step}-{avgmodel.end_step}]"
row = misc.to_row([results[key] for key in results_keys if key in results])
logger.info(row + step_str)
swad.update_and_evaluate(
swad_algorithm, results["train_out"], results["tr_outloss"], prt_results_fn
)
if hasattr(swad, "dead_valley") and swad.dead_valley:
logger.info("SWAD valley is dead -> early stop !")
break
swad_algorithm = swa_utils.AveragedModel(algorithm) # reset
if step % args.tb_freq == 0:
# add step values only for tb log
step_vals["lr"] = scheduler.get_last_lr()[0]
writer.add_scalars_with_prefix(step_vals, step, f"{testenv_name}/summary/")
# find best
logger.info("---")
records = Q(records)
oracle_best = records.argmax("test_out")["test_in"]
iid_best = records.argmax("train_out")["test_in"]
last = records[-1]["test_in"]
if hparams.indomain_test:
# if test set exist, use test set for indomain results
in_key = "train_inTE"
else:
in_key = "train_out"
iid_best_indomain = records.argmax("train_out")[in_key]
last_indomain = records[-1][in_key]
ret = {
"oracle": oracle_best,
"iid": iid_best,
"last": last,
"last (inD)": last_indomain,
"iid (inD)": iid_best_indomain,
}
# Evaluate SWAD
if swad:
swad_algorithm = swad.get_final_model()
if hparams["freeze_bn"] is False:
n_steps = 500 if not args.debug else 10
logger.warning(f"Update SWAD BN statistics for {n_steps} steps ...")
swa_utils.update_bn(train_minibatches_iterator, swad_algorithm, n_steps)
logger.warning("Evaluate SWAD ...")
accuracies, summaries = evaluator.evaluate(swad_algorithm)
results = {**summaries, **accuracies}
start = swad_algorithm.start_step
end = swad_algorithm.end_step
step_str = f" [{start}-{end}] (N={swad_algorithm.n_averaged})"
row = misc.to_row([results[key] for key in results_keys if key in results]) + step_str
logger.info(row)
ret["SWAD"] = results["test_in"]
ret["SWAD (inD)"] = results[in_key]
for k, acc in ret.items():
logger.info(f"{k} = {acc:.3%}")
return ret, records
================================================
FILE: domain_generalization/swad/requirements.txt
================================================
prettytable==2.0.0
numpy==1.19.4
torch==1.7.0+cu92
gdown==3.12.2
imageio==2.9.0
sconf==0.2.3
torchvision==0.8.1+cu92
tqdm==4.51.0
tensorboard==2.3.0
Pillow==8.1.0
tensorboardX==2.1
================================================
FILE: domain_generalization/swad/train_all.py
================================================
import argparse
import collections
import random
import sys
from pathlib import Path
import numpy as np
import PIL
import torch
import torchvision
from sconf import Config
from prettytable import PrettyTable
from domainbed.datasets import get_dataset
from domainbed import hparams_registry
from domainbed.lib import misc
from domainbed.lib.writers import get_writer
from domainbed.lib.logger import Logger
from domainbed.trainer import train
def main():
parser = argparse.ArgumentParser(description="Domain generalization")
parser.add_argument("name", type=str)
parser.add_argument("configs", nargs="*")
parser.add_argument("--data_dir", type=str, default="datadir/")
parser.add_argument("--dataset", type=str, default="PACS")
parser.add_argument("--algorithm", type=str, default="ERM")
parser.add_argument(
"--trial_seed",
type=int,
default=0,
help="Trial number (used for seeding split_dataset and random_hparams).",
)
parser.add_argument("--seed", type=int, default=0, help="Seed for everything else")
parser.add_argument(
"--steps", type=int, default=None, help="Number of steps. Default is dataset-dependent."
)
parser.add_argument(
"--checkpoint_freq",
type=int,
default=None,
help="Checkpoint every N steps. Default is dataset-dependent.",
)
parser.add_argument("--test_envs", type=int, nargs="+", default=None) # sketch in PACS
parser.add_argument("--holdout_fraction", type=float, default=0.2)
parser.add_argument("--model_save", default=None, type=int, help="Model save start step")
parser.add_argument("--deterministic", action="store_true")
parser.add_argument("--tb_freq", default=10)
parser.add_argument("--add_bt", default=0, type=int)
parser.add_argument("--baseline", action="store_true", help="baseline")
parser.add_argument("--lr_scale_bt", default=1., type=float, help='')
parser.add_argument("--resnet18", action="store_true", help="baseline")
parser.add_argument("--debug", action="store_true", help="Run w/ debug mode")
parser.add_argument("--show", action="store_true", help="Show args and hparams w/o run")
parser.add_argument(
"--evalmode",
default="fast",
help="[fast, all]. if fast, ignore train_in datasets in evaluation time.",
)
parser.add_argument("--prebuild_loader", action="store_true", help="Pre-build eval loaders")
args, left_argv = parser.parse_known_args()
# setup hparams
hparams = hparams_registry.default_hparams(args.algorithm, args.dataset)
keys = ["config.yaml"] + args.configs
keys = [open(key, encoding="utf8") for key in keys]
hparams = Config(*keys, default=hparams)
hparams.argv_update(left_argv)
if args.baseline:
hparams['swad'] =None
if args.resnet18:
hparams['resnet18'] = (True, True)
# setup debug
if args.debug:
args.checkpoint_freq = 5
args.steps = 10
args.name += "_debug"
timestamp = misc.timestamp()
args.unique_name = f"{timestamp}_{args.name}"
# path setup
args.work_dir = Path(".")
args.data_dir = Path(args.data_dir)
args.out_root = args.work_dir / Path("train_output") / args.dataset
args.out_dir = args.out_root / args.unique_name
args.out_dir.mkdir(exist_ok=True, parents=True)
writer = get_writer(args.out_root / "runs" / args.unique_name)
logger = Logger.get(args.out_dir / "log.txt")
if args.debug:
logger.setLevel("DEBUG")
cmd = " ".join(sys.argv)
logger.info(f"Command :: {cmd}")
logger.nofmt("Environment:")
logger.nofmt("\tPython: {}".format(sys.version.split(" ")[0]))
logger.nofmt("\tPyTorch: {}".format(torch.__version__))
logger.nofmt("\tTorchvision: {}".format(torchvision.__version__))
logger.nofmt("\tCUDA: {}".format(torch.version.cuda))
logger.nofmt("\tCUDNN: {}".format(torch.backends.cudnn.version()))
logger.nofmt("\tNumPy: {}".format(np.__version__))
logger.nofmt("\tPIL: {}".format(PIL.__version__))
# Different to DomainBed, we support CUDA only.
assert torch.cuda.is_available(), "CUDA is not available"
logger.nofmt("Args:")
for k, v in sorted(vars(args).items()):
logger.nofmt("\t{}: {}".format(k, v))
logger.nofmt("HParams:")
for line in hparams.dumps().split("\n"):
logger.nofmt("\t" + line)
if args.show:
exit()
# seed
random.seed(args.seed)
np.random.seed(args.seed)
torch.manual_seed(args.seed)
torch.backends.cudnn.deterministic = args.deterministic
torch.backends.cudnn.benchmark = not args.deterministic
# Dummy datasets for logging information.
# Real dataset will be re-assigned in train function.
# test_envs only decide transforms; simply set to zero.
dataset, _in_splits, _out_splits = get_dataset([0], args, hparams)
# print dataset information
logger.nofmt("Dataset:")
logger.nofmt(f"\t[{args.dataset}] #envs={len(dataset)}, #classes={dataset.num_classes}")
for i, env_property in enumerate(dataset.environments):
logger.nofmt(f"\tenv{i}: {env_property} (#{len(dataset[i])})")
logger.nofmt("")
n_steps = args.steps or dataset.N_STEPS
checkpoint_freq = args.checkpoint_freq or dataset.CHECKPOINT_FREQ
logger.info(f"n_steps = {n_steps}")
logger.info(f"checkpoint_freq = {checkpoint_freq}")
org_n_steps = n_steps
n_steps = (n_steps // checkpoint_freq) * checkpoint_freq + 1
logger.info(f"n_steps is updated to {org_n_steps} => {n_steps} for checkpointing")
if not args.test_envs:
args.test_envs = [[te] for te in range(len(dataset))]
logger.info(f"Target test envs = {args.test_envs}")
###########################################################################
# Run
###########################################################################
all_records = []
results = collections.defaultdict(list)
for test_env in args.test_envs:
res, records = train(
test_env,
args=args,
hparams=hparams,
n_steps=n_steps,
checkpoint_freq=checkpoint_freq,
logger=logger,
writer=writer,
)
all_records.append(records)
for k, v in res.items():
results[k].append(v)
# log summary table
logger.info("=== Summary ===")
logger.info(f"Command: {' '.join(sys.argv)}")
logger.info("Unique name: %s" % args.unique_name)
logger.info("Out path: %s" % args.out_dir)
logger.info("Algorithm: %s" % args.algorithm)
logger.info("Dataset: %s" % args.dataset)
table = PrettyTable(["Selection"] + dataset.environments + ["Avg."])
for key, row in results.items():
row.append(np.mean(row))
row = [f"{acc:.3%}" for acc in row]
table.add_row([key] + row)
logger.nofmt(table)
if __name__ == "__main__":
main()
================================================
FILE: long-tailed_recognition/BalancedSoftmax/.gitignore
================================================
logs
*pkl
*pyc
.vscode
*.sh
runs
.vscode
.idea
================================================
FILE: long-tailed_recognition/BalancedSoftmax/CODE_OF_CONDUCT.md
================================================
# Code of Conduct
Facebook has adopted a Code of Conduct that we expect project participants to adhere to.
Please read the [full text](https://code.fb.com/codeofconduct/)
so that you can understand what actions will and will not be tolerated.
================================================
FILE: long-tailed_recognition/BalancedSoftmax/CONTRIBUTING.md
================================================
# Contributing to Classifier-Balancing
We want to make contributing to this project as easy and transparent as
possible.
## Our Development Process
Minor changes and improvements will be released on an ongoing basis. Larger changes (e.g., changesets implementing a new paper) will be released on a more periodic basis.
## Pull Requests
We actively welcome your pull requests.
1. Fork the repo and create your branch from `master`.
2. If you've added code that should be tested, add tests.
3. If you've changed APIs, update the documentation.
4. Ensure the test suite passes.
5. Make sure your code lints.
6. If you haven't already, complete the Contributor License Agreement ("CLA").
## Contributor License Agreement ("CLA")
In order to accept your pull request, we need you to submit a CLA. You only need
to do this once to work on any of Facebook's open source projects.
Complete your CLA here:
## Issues
We use GitHub issues to track public bugs. Please ensure your description is
clear and has sufficient instructions to be able to reproduce the issue.
Facebook has a [bounty program](https://www.facebook.com/whitehat/) for the safe
disclosure of security bugs. In those cases, please go through the process
outlined on that page and do not file a public issue.
## Coding Style
* 4 spaces for indentation rather than tabs
## License
By contributing to Classifier-Balancing, you agree that your contributions will be licensed
under the LICENSE file in the root directory of this source tree.
================================================
FILE: long-tailed_recognition/BalancedSoftmax/LICENSE
================================================
BSD License
For Classifier-Balacing software
Copyright (c) Facebook, Inc. and its affiliates. All rights reserved.
Redistribution and use in source and binary forms, with or without modification,
are permitted provided that the following conditions are met:
* Redistributions of source code must retain the above copyright notice, this
list of conditions and the following disclaimer.
* Redistributions in binary form must reproduce the above copyright notice,
this list of conditions and the following disclaimer in the documentation
and/or other materials provided with the distribution.
* Neither the name Facebook nor the names of its contributors may be used to
endorse or promote products derived from this software without specific
prior written permission.
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND
ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR
ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
(INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON
ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
Portions of the source code are from the OLTR project and are licensed under
the following terms:
BSD 3-Clause License
Copyright (c) 2019, Zhongqi Miao. All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions are met:
1. Redistributions of source code must retain the above copyright notice, this
list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright notice,
this list of conditions and the following disclaimer in the documentation
and/or other materials provided with the distribution.
3. Neither the name of the copyright holder nor the names of its
contributors may be used to endorse or promote products derived from
this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE
FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
================================================
FILE: long-tailed_recognition/BalancedSoftmax/README.md
================================================
# BatchFormer for BalancedSoftmax
Code for the paper BatchFormer on BalancedSoftmax. This repository includes the ablation study codes and thus might also include some redudanct code.
## Requirements
* Python 3
* [PyTorch](https://pytorch.org/) (version == 1.4)
* [yaml](https://pyyaml.org/wiki/PyYAMLDocumentation)
* [higher](https://github.com/facebookresearch/higher)(version == 0.2.1)
## Training
### End-to-end Training
- Balanced Softmax
```bash
python main.py --cfg ./config/ImageNet_LT/balanced_softmax_resnet10.yaml
```
- BatchFormer
```bash
python main.py --cfg ./config/ImageNet_LT/balanced_softmax_resnet10.yaml --add_bt 1
```
## Evaluation
Model evaluation can be done using the following command:
```bash
python main.py --cfg ./config/CIFAR10_LT/balanced_softmax_imba200.yaml --test
```
We use debug=4 to stat the gradients for other images in the mini-batch.
## Citation
```bibtex
@inproceedings{hou2022batch,
title={BatchFormer: Learning to Explore Sample Relationships for Robust Representation Learning},
author={Hou, Zhi and Yu, Baosheng and Tao, Dacheng},
booktitle={CVPR},
year={2022}
}
```
```bibtex
@inproceedings{
Ren2020balms,
title={Balanced Meta-Softmax for Long-Tailed Visual Recognition},
author={Jiawei Ren and Cunjun Yu and Shunan Sheng and Xiao Ma and Haiyu Zhao and Shuai Yi and Hongsheng Li},
booktitle={Proceedings of Neural Information Processing Systems(NeurIPS)},
month = {Dec},
year={2020}
}
```
Code are based on [Balanced Meta-Softmax](https://github.com/jiawei-ren/BalancedMetaSoftmax-Classification.git)
================================================
FILE: long-tailed_recognition/BalancedSoftmax/cls_freq/CIFAR-10-LT_IMBA100.json
================================================
[5000, 2997, 1796, 1077, 645, 387, 232, 139, 83, 50]
================================================
FILE: long-tailed_recognition/BalancedSoftmax/cls_freq/CIFAR-10-LT_IMBA200.json
================================================
[5000, 2775, 1540, 854, 474, 263, 146, 81, 45, 25]
================================================
FILE: long-tailed_recognition/BalancedSoftmax/cls_freq/CIFAR-100-LT_IMBA1.json
================================================
[500, 500, 500, 500, 500, 500, 500, 500, 500, 500, 500, 500, 500, 500, 500, 500, 500, 500, 500, 500, 500, 500, 500, 500, 500, 500, 500, 500, 500, 500, 500, 500, 500, 500, 500, 500, 500, 500, 500, 500, 500, 500, 500, 500, 500, 500, 500, 500, 500, 500, 500, 500, 500, 500, 500, 500, 500, 500, 500, 500, 500, 500, 500, 500, 500, 500, 500, 500, 500, 500, 500, 500, 500, 500, 500, 500, 500, 500, 500, 500, 500, 500, 500, 500, 500, 500, 500, 500, 500, 500, 500, 500, 500, 500, 500, 500, 500, 500, 500, 500]
================================================
FILE: long-tailed_recognition/BalancedSoftmax/cls_freq/CIFAR-100-LT_IMBA100.json
================================================
[500, 477, 455, 434, 415, 396, 378, 361, 344, 328, 314, 299, 286, 273, 260, 248, 237, 226, 216, 206, 197, 188, 179, 171, 163, 156, 149, 142, 135, 129, 123, 118, 112, 107, 102, 98, 93, 89, 85, 81, 77, 74, 70, 67, 64, 61, 58, 56, 53, 51, 48, 46, 44, 42, 40, 38, 36, 35, 33, 32, 30, 29, 27, 26, 25, 24, 23, 22, 21, 20, 19, 18, 17, 16, 15, 15, 14, 13, 13, 12, 12, 11, 11, 10, 10, 9, 9, 8, 8, 7, 7, 7, 6, 6, 6, 6, 5, 5, 5, 5]
================================================
FILE: long-tailed_recognition/BalancedSoftmax/cls_freq/CIFAR-100-LT_IMBA200.json
================================================
[500, 473, 449, 425, 403, 382, 362, 343, 325, 308, 292, 277, 263, 249, 236, 224, 212, 201, 190, 180, 171, 162, 154, 146, 138, 131, 124, 117, 111, 105, 100, 95, 90, 85, 81, 76, 72, 69, 65, 62, 58, 55, 52, 50, 47, 44, 42, 40, 38, 36, 34, 32, 30, 29, 27, 26, 24, 23, 22, 21, 20, 19, 18, 17, 16, 15, 14, 13, 13, 12, 11, 11, 10, 10, 9, 9, 8, 8, 7, 7, 6, 6, 6, 5, 5, 5, 5, 4, 4, 4, 4, 3, 3, 3, 3, 3, 2, 2, 2, 2]
================================================
FILE: long-tailed_recognition/BalancedSoftmax/cls_freq/ImageNet_LT.json
================================================
[50, 250, 516, 352, 143, 92, 47, 45, 150, 50, 34, 55, 25, 35, 121, 7, 232, 13, 392, 157, 87, 181, 177, 171, 114, 56, 134, 56, 279, 248, 287, 83, 65, 56, 118, 43, 68, 39, 14, 96, 80, 15, 236, 103, 64, 54, 346, 32, 201, 29, 376, 27, 189, 96, 69, 157, 16, 173, 5, 34, 34, 150, 57, 78, 31, 8, 44, 297, 349, 72, 35, 36, 428, 10, 137, 58, 80, 44, 30, 15, 126, 93, 34, 135, 226, 281, 1280, 19, 38, 502, 186, 81, 61, 222, 15, 28, 55, 819, 74, 24, 10, 93, 14, 111, 61, 32, 116, 133, 128, 66, 35, 50, 393, 47, 179, 124, 79, 604, 72, 240, 22, 148, 26, 64, 20, 23, 246, 11, 79, 161, 78, 370, 197, 155, 285, 59, 59, 178, 43, 75, 11, 23, 16, 74, 27, 87, 86, 105, 189, 37, 126, 53, 10, 30, 77, 5, 245, 30, 100, 37, 69, 119, 26, 47, 81, 100, 25, 35, 112, 35, 7, 35, 272, 469, 150, 100, 199, 137, 69, 42, 10, 20, 367, 49, 73, 38, 56, 75, 89, 24, 35, 244, 305, 23, 106, 77, 10, 59, 90, 33, 26, 214, 43, 50, 18, 125, 110, 119, 85, 43, 11, 11, 114, 171, 24, 54, 87, 39, 86, 496, 91, 52, 56, 11, 117, 48, 62, 100, 41, 138, 22, 7, 50, 51, 6, 281, 29, 13, 36, 75, 126, 127, 59, 56, 172, 119, 90, 163, 17, 53, 181, 81, 63, 167, 45, 37, 39, 282, 71, 32, 12, 137, 102, 97, 60, 14, 66, 14, 6, 132, 303, 81, 167, 28, 15, 72, 187, 20, 310, 137, 241, 123, 27, 19, 186, 12, 12, 42, 71, 96, 71, 53, 53, 19, 62, 43, 330, 41, 11, 42, 43, 22, 208, 18, 566, 46, 19, 10, 58, 306, 67, 119, 50, 51, 87, 143, 37, 16, 611, 290, 170, 114, 19, 79, 81, 128, 77, 332, 8, 446, 42, 122, 86, 1121, 48, 109, 133, 7, 174, 17, 283, 140, 27, 15, 30, 134, 167, 198, 149, 16, 122, 23, 194, 149, 23, 156, 5, 127, 11, 31, 49, 143, 145, 169, 20, 158, 87, 128, 96, 65, 49, 391, 349, 83, 123, 232, 12, 51, 137, 31, 73, 32, 41, 65, 6, 6, 85, 231, 171, 28, 108, 96, 101, 32, 122, 26, 165, 10, 372, 34, 31, 63, 679, 34, 257, 27, 83, 23, 275, 13, 22, 47, 118, 411, 159, 124, 45, 1053, 111, 184, 68, 52, 41, 150, 10, 47, 109, 32, 69, 34, 38, 10, 30, 82, 16, 157, 14, 141, 10, 58, 25, 152, 165, 112, 480, 91, 43, 113, 22, 236, 48, 137, 48, 147, 511, 17, 86, 33, 44, 36, 333, 19, 48, 15, 224, 108, 152, 16, 186, 114, 31, 82, 125, 155, 59, 186, 188, 134, 25, 121, 288, 39, 8, 32, 41, 346, 127, 232, 123, 81, 652, 6, 154, 203, 48, 66, 178, 45, 6, 22, 85, 102, 44, 105, 44, 54, 42, 20, 122, 220, 82, 44, 43, 75, 8, 142, 82, 38, 231, 619, 162, 62, 134, 8, 71, 41, 86, 55, 116, 35, 94, 100, 330, 54, 87, 143, 110, 191, 6, 48, 153, 84, 16, 55, 10, 422, 92, 25, 507, 168, 104, 83, 70, 47, 382, 148, 10, 164, 30, 310, 381, 174, 92, 87, 268, 202, 1173, 110, 8, 125, 51, 340, 37, 209, 71, 14, 14, 46, 252, 14, 239, 58, 22, 112, 22, 101, 130, 278, 18, 220, 21, 253, 39, 292, 317, 179, 36, 305, 32, 65, 169, 683, 98, 376, 125, 39, 8, 28, 298, 132, 43, 69, 80, 413, 19, 25, 354, 120, 72, 86, 50, 76, 72, 102, 310, 21, 26, 119, 140, 75, 63, 57, 10, 10, 31, 106, 54, 100, 90, 64, 88, 34, 90, 36, 85, 82, 6, 64, 86, 29, 6, 30, 204, 27, 209, 115, 24, 72, 253, 55, 10, 33, 104, 38, 46, 296, 26, 128, 170, 15, 66, 130, 138, 28, 1246, 107, 63, 75, 108, 167, 61, 26, 83, 77, 42, 113, 103, 264, 59, 16, 49, 29, 33, 33, 481, 29, 45, 24, 38, 165, 82, 36, 15, 60, 62, 72, 73, 25, 8, 182, 99, 283, 101, 297, 28, 194, 26, 141, 14, 78, 13, 67, 82, 63, 103, 454, 6, 22, 66, 93, 8, 11, 131, 137, 40, 66, 25, 472, 9, 183, 32, 14, 43, 299, 56, 40, 392, 5, 20, 31, 32, 83, 194, 81, 80, 787, 7, 137, 50, 9, 133, 34, 174, 315, 35, 290, 9, 263, 164, 86, 129, 20, 53, 175, 43, 775, 11, 54, 12, 137, 31, 87, 32, 229, 96, 161, 126, 271, 221, 27, 28, 36, 14, 85, 197, 82, 76, 15, 63, 130, 11, 85, 134, 211, 21, 328, 302, 205, 21, 53, 102, 93, 111, 19, 41, 17, 218, 128, 242, 188, 166, 105, 46, 7, 189, 20, 73, 247, 229, 35, 343, 95, 146, 57, 285, 43, 60, 84, 64, 28, 17, 44, 18, 39, 175, 25, 60, 134, 8, 54, 46, 31, 17, 10, 9, 143, 114, 101, 157, 70, 6, 75, 192, 64, 65, 5, 8, 9, 31, 15, 642, 31, 107, 11, 69, 459, 238, 19, 31, 134, 252, 25, 456, 15, 80, 102, 274, 6, 32, 50, 140, 104, 112, 133, 28, 182, 170, 76, 173, 272, 20, 102, 30, 138, 150, 116, 110, 216, 42, 39, 116, 16, 13, 215, 85, 155, 19, 7, 36, 66, 323, 22, 284, 172, 398, 110, 56, 25, 53, 534, 43, 38, 26, 418, 141, 7, 36, 55, 20, 63, 45, 18, 80, 67, 151, 94, 103, 284, 180, 23, 175, 472, 133, 99, 71, 67, 80, 32, 113, 17, 57, 172, 17, 19, 93, 124, 23, 55, 258, 108, 72, 33, 165, 95, 52, 45, 625, 73, 217, 98, 45, 34, 41, 39, 714, 143, 141, 233, 119, 63, 236, 150, 19, 15, 35, 167, 19, 25, 21, 21]
================================================
FILE: long-tailed_recognition/BalancedSoftmax/cls_freq/Places_LT.json
================================================
[219, 20, 115, 86, 229, 33, 25, 4980, 108, 2612, 33, 39, 2400, 87, 12, 225, 41, 98, 191, 1582, 37, 78, 48, 7, 1530, 56, 93, 84, 18, 183, 155, 72, 92, 78, 10, 120, 22, 6, 143, 98, 195, 36, 1487, 93, 73, 1406, 8, 40, 1380, 138, 11, 14, 1085, 43, 59, 20, 6, 129, 103, 98, 46, 41, 1036, 82, 14, 23, 1023, 841, 8, 91, 20, 30, 19, 34, 44, 11, 198, 129, 135, 132, 29, 830, 725, 11, 689, 88, 668, 48, 5, 38, 643, 605, 71, 6, 566, 72, 34, 547, 54, 72, 6, 33, 36, 97, 76, 78, 521, 97, 116, 157, 499, 8, 99, 74, 24, 62, 28, 128, 176, 72, 87, 482, 148, 86, 40, 147, 5, 108, 6, 8, 35, 24, 480, 32, 140, 7, 84, 6, 46, 22, 104, 85, 118, 59, 72, 36, 34, 80, 33, 100, 475, 470, 79, 164, 451, 7, 28, 171, 65, 15, 8, 80, 20, 208, 450, 28, 9, 60, 9, 10, 13, 431, 9, 22, 34, 428, 72, 17, 34, 42, 73, 98, 188, 422, 42, 20, 60, 21, 6, 39, 50, 23, 408, 167, 99, 5, 25, 97, 13, 105, 55, 13, 41, 392, 389, 383, 54, 12, 17, 207, 57, 18, 63, 81, 374, 371, 48, 367, 25, 16, 366, 98, 7, 104, 364, 28, 89, 68, 15, 17, 46, 113, 334, 56, 60, 12, 55, 51, 87, 58, 26, 15, 22, 331, 179, 318, 12, 18, 8, 94, 65, 71, 315, 29, 173, 9, 14, 315, 226, 124, 33, 9, 9, 51, 15, 52, 309, 34, 209, 61, 116, 298, 298, 149, 60, 70, 89, 218, 291, 290, 30, 36, 16, 186, 287, 36, 195, 80, 283, 39, 141, 14, 181, 152, 11, 6, 22, 39, 19, 280, 11, 173, 11, 43, 112, 177, 111, 279, 108, 153, 103, 19, 265, 264, 95, 72, 40, 263, 19, 229, 45, 54, 12, 260, 253, 32, 253, 27, 52, 34, 250, 10, 11, 49, 250, 17, 24, 242, 74, 14, 48, 216, 241, 16, 238, 232, 8, 33, 205, 93, 46, 7, 46, 82, 44, 230, 91, 218, 21, 69, 156, 206, 103, 14, 20]
================================================
FILE: long-tailed_recognition/BalancedSoftmax/cls_freq/iNaturalist18.json
================================================
[22, 34, 21, 27, 15, 12, 22, 23, 28, 15, 144, 28, 12, 23, 111, 554, 26, 22, 142, 23, 13, 27, 33, 10, 132, 26, 28, 19, 17, 23, 10, 29, 12, 65, 14, 17, 178, 10, 21, 17, 23, 13, 22, 19, 53, 16, 12, 24, 18, 25, 13, 28, 13, 18, 17, 31, 19, 200, 22, 34, 26, 11, 7, 11, 11, 15, 25, 9, 11, 9, 11, 15, 23, 20, 19, 22, 146, 17, 21, 11, 33, 10, 27, 13, 10, 20, 23, 126, 12, 32, 23, 16, 25, 81, 46, 18, 32, 23, 12, 16, 44, 11, 25, 13, 12, 21, 27, 19, 20, 24, 22, 19, 64, 20, 19, 15, 62, 26, 17, 28, 19, 25, 33, 21, 10, 59, 23, 15, 17, 19, 17, 17, 25, 118, 12, 11, 27, 19, 17, 22, 16, 57, 11, 13, 21, 25, 15, 25, 22, 24, 26, 17, 18, 27, 19, 26, 76, 24, 25, 16, 17, 12, 14, 16, 13, 23, 12, 38, 26, 10, 23, 272, 11, 19, 22, 17, 18, 23, 22, 25, 24, 24, 23, 20, 35, 13, 24, 18, 20, 21, 25, 23, 24, 24, 20, 20, 15, 21, 139, 12, 23, 21, 22, 26, 26, 15, 12, 41, 20, 25, 22, 25, 11, 197, 25, 14, 72, 14, 14, 20, 14, 18, 23, 16, 27, 23, 23, 22, 18, 22, 22, 16, 18, 25, 25, 17, 27, 15, 22, 696, 20, 11, 29, 26, 64, 12, 149, 24, 948, 17, 430, 25, 19, 27, 34, 13, 12, 24, 26, 25, 28, 19, 18, 18, 20, 19, 54, 25, 16, 10, 23, 34, 25, 16, 11, 13, 15, 17, 80, 27, 25, 12, 129, 31, 11, 17, 14, 29, 27, 25, 24, 34, 23, 38, 202, 11, 28, 38, 26, 27, 22, 12, 25, 8, 78, 16, 25, 28, 27, 13, 24, 14, 29, 28, 36, 18, 27, 24, 43, 29, 22, 7, 16, 11, 31, 10, 24, 17, 26, 23, 13, 16, 25, 26, 31, 8, 20, 21, 13, 22, 25, 27, 11, 14, 14, 24, 15, 10, 16, 17, 80, 16, 18, 17, 12, 13, 25, 32, 30, 19, 15, 26, 17, 25, 18, 20, 12, 19, 26, 124, 19, 26, 16, 14, 15, 14, 13, 24, 39, 21, 25, 37, 12, 33, 20, 26, 27, 25, 16, 28, 21, 25, 16, 12, 11, 12, 25, 15, 10, 59, 19, 24, 22, 21, 34, 52, 26, 27, 22, 23, 15, 16, 14, 34, 20, 11, 59, 12, 22, 119, 16, 20, 18, 384, 12, 19, 26, 14, 25, 19, 14, 25, 21, 11, 14, 20, 22, 44, 25, 12, 8, 23, 69, 19, 9, 23, 26, 19, 16, 12, 28, 7, 60, 15, 19, 12, 108, 29, 10, 23, 12, 15, 22, 15, 28, 24, 21, 23, 173, 19, 16, 23, 22, 22, 24, 16, 17, 36, 19, 36, 26, 56, 23, 24, 34, 21, 13, 21, 1000, 14, 10, 22, 25, 26, 30, 24, 358, 20, 9, 27, 14, 286, 10, 28, 15, 117, 92, 10, 81, 22, 8, 21, 18, 19, 25, 67, 8, 14, 280, 19, 28, 13, 22, 58, 16, 25, 20, 22, 25, 14, 23, 18, 9, 21, 16, 18, 24, 23, 12, 27, 13, 17, 16, 27, 17, 24, 21, 27, 14, 21, 10, 12, 10, 8, 21, 7, 15, 14, 15, 13, 25, 24, 28, 23, 25, 12, 21, 27, 18, 134, 49, 24, 15, 17, 102, 22, 26, 25, 14, 26, 9, 14, 26, 53, 20, 63, 8, 47, 21, 14, 49, 32, 18, 8, 21, 19, 26, 26, 267, 24, 17, 20, 23, 16, 12, 29, 23, 14, 9, 12, 19, 26, 19, 44, 19, 28, 16, 11, 25, 20, 10, 22, 23, 27, 11, 12, 33, 8, 21, 20, 27, 20, 19, 9, 21, 26, 18, 10, 13, 11, 20, 26, 17, 36, 18, 18, 30, 16, 21, 24, 15, 23, 16, 54, 23, 18, 15, 58, 20, 14, 23, 22, 18, 31, 16, 14, 25, 18, 25, 18, 15, 19, 8, 15, 16, 24, 25, 31, 25, 27, 23, 8, 8, 15, 13, 28, 11, 113, 12, 64, 18, 34, 38, 24, 22, 28, 9, 47, 18, 25, 15, 11, 10, 31, 24, 28, 13, 22, 14, 10, 24, 17, 23, 156, 27, 25, 26, 16, 13, 22, 14, 11, 43, 18, 23, 22, 25, 26, 16, 28, 19, 113, 28, 14, 12, 33, 27, 17, 11, 16, 10, 114, 19, 25, 22, 25, 27, 22, 26, 17, 33, 15, 22, 16, 133, 20, 14, 26, 24, 29, 23, 26, 25, 56, 49, 34, 13, 19, 31, 15, 21, 18, 24, 14, 20, 157, 92, 26, 26, 76, 15, 20, 59, 67, 28, 13, 9, 32, 25, 24, 26, 18, 11, 133, 56, 27, 73, 21, 22, 8, 24, 12, 12, 13, 27, 22, 11, 14, 27, 17, 20, 14, 18, 20, 10, 13, 33, 11, 35, 37, 8, 18, 29, 14, 12, 18, 26, 9, 14, 12, 24, 274, 21, 18, 19, 35, 26, 27, 14, 14, 16, 15, 24, 65, 23, 13, 23, 27, 28, 19, 8, 17, 29, 27, 22, 25, 31, 26, 24, 11, 27, 11, 12, 18, 23, 25, 17, 15, 18, 10, 28, 11, 19, 27, 38, 23, 128, 10, 31, 17, 25, 13, 17, 37, 13, 25, 25, 25, 20, 11, 23, 26, 46, 14, 9, 12, 26, 19, 21, 15, 26, 17, 8, 28, 29, 31, 26, 13, 20, 27, 15, 39, 16, 20, 19, 19, 12, 13, 23, 17, 14, 15, 31, 35, 39, 24, 28, 19, 21, 9, 18, 64, 27, 28, 19, 23, 25, 11, 14, 37, 13, 14, 21, 33, 21, 24, 15, 13, 37, 42, 17, 38, 9, 20, 17, 22, 13, 22, 21, 16, 16, 24, 13, 24, 25, 19, 16, 28, 35, 29, 15, 18, 9, 22, 28, 11, 19, 18, 24, 32, 12, 32, 30, 12, 27, 28, 18, 11, 14, 11, 16, 26, 16, 30, 22, 27, 23, 57, 35, 27, 16, 19, 14, 25, 80, 19, 25, 20, 25, 24, 25, 62, 28, 29, 37, 26, 18, 18, 11, 34, 15, 33, 21, 19, 13, 27, 10, 24, 22, 19, 30, 119, 13, 26, 23, 18, 33, 215, 27, 15, 14, 21, 49, 14, 11, 24, 29, 33, 20, 19, 442, 41, 18, 70, 93, 29, 29, 13, 63, 26, 96, 41, 12, 22, 20, 15, 20, 28, 783, 15, 139, 70, 27, 36, 146, 30, 33, 27, 25, 28, 20, 94, 261, 21, 86, 9, 127, 22, 29, 20, 218, 18, 26, 71, 18, 19, 24, 18, 13, 17, 14, 18, 16, 29, 73, 14, 32, 39, 76, 21, 12, 24, 23, 26, 11, 25, 18, 11, 198, 116, 23, 20, 24, 24, 15, 18, 27, 21, 14, 19, 19, 17, 8, 28, 53, 172, 36, 35, 13, 57, 23, 10, 16, 22, 81, 41, 19, 151, 75, 10, 20, 15, 472, 18, 15, 24, 148, 16, 25, 18, 18, 15, 19, 86, 17, 109, 19, 6, 8, 12, 13, 17, 30, 149, 11, 24, 25, 17, 21, 13, 26, 20, 25, 10, 29, 22, 25, 30, 131, 20, 14, 11, 14, 13, 20, 24, 20, 19, 22, 15, 25, 11, 10, 37, 19, 11, 28, 26, 45, 18, 15, 27, 28, 27, 18, 716, 26, 25, 22, 8, 22, 14, 21, 13, 26, 14, 27, 28, 17, 9, 24, 24, 9, 22, 22, 12, 21, 27, 23, 13, 10, 14, 12, 39, 23, 34, 18, 16, 16, 36, 19, 23, 21, 39, 18, 19, 15, 16, 21, 20, 9, 21, 15, 33, 17, 19, 13, 24, 19, 17, 14, 19, 19, 18, 19, 21, 20, 9, 13, 9, 31, 23, 8, 19, 67, 17, 29, 20, 28, 16, 8, 17, 31, 24, 20, 23, 40, 22, 9, 22, 29, 41, 22, 25, 28, 17, 26, 75, 18, 17, 28, 157, 30, 9, 21, 18, 24, 21, 17, 15, 26, 18, 14, 15, 11, 15, 20, 25, 9, 15, 18, 21, 34, 28, 34, 15, 16, 23, 19, 20, 17, 24, 8, 17, 54, 20, 17, 21, 9, 22, 14, 20, 20, 20, 10, 19, 19, 11, 20, 15, 17, 13, 13, 23, 22, 19, 26, 23, 14, 27, 18, 27, 18, 28, 13, 20, 14, 18, 11, 18, 12, 33, 24, 15, 19, 27, 27, 13, 22, 10, 18, 12, 22, 32, 10, 15, 234, 32, 14, 19, 113, 41, 82, 922, 52, 9, 49, 169, 75, 32, 20, 29, 26, 14, 15, 40, 22, 270, 127, 36, 34, 18, 22, 23, 26, 10, 14, 20, 26, 241, 17, 15, 19, 12, 10, 22, 150, 43, 50, 22, 54, 317, 13, 15, 12, 488, 27, 1000, 28, 42, 14, 11, 27, 28, 94, 12, 8, 14, 25, 15, 20, 14, 28, 15, 187, 46, 15, 22, 281, 42, 25, 16, 16, 9, 9, 23, 14, 25, 173, 24, 22, 28, 14, 25, 16, 18, 28, 18, 20, 12, 20, 29, 12, 1000, 8, 12, 12, 12, 15, 9, 25, 14, 26, 14, 61, 96, 30, 13, 24, 257, 327, 441, 16, 174, 25, 62, 51, 174, 30, 24, 11, 12, 100, 20, 22, 8, 15, 17, 21, 18, 31, 13, 447, 39, 9, 17, 23, 114, 18, 26, 11, 39, 189, 191, 83, 632, 49, 22, 216, 258, 45, 20, 16, 24, 28, 25, 10, 94, 10, 29, 16, 26, 256, 17, 22, 151, 1000, 615, 75, 90, 553, 26, 17, 18, 28, 23, 16, 20, 21, 14, 492, 65, 18, 14, 12, 18, 19, 25, 17, 51, 17, 6, 9, 122, 8, 846, 68, 12, 75, 36, 23, 506, 13, 6, 254, 12, 198, 25, 283, 15, 10, 22, 25, 24, 25, 27, 18, 106, 88, 27, 13, 69, 20, 26, 20, 25, 9, 19, 27, 30, 406, 110, 15, 15, 15, 23, 16, 11, 14, 13, 40, 31, 41, 17, 188, 50, 27, 220, 29, 19, 56, 45, 1000, 23, 19, 24, 18, 23, 245, 187, 16, 30, 63, 28, 42, 20, 18, 18, 29, 22, 25, 24, 20, 15, 26, 28, 19, 19, 19, 13, 28, 20, 25, 28, 19, 16, 32, 25, 13, 12, 39, 22, 21, 9, 24, 24, 12, 27, 18, 28, 30, 20, 30, 34, 19, 24, 10, 12, 24, 15, 13, 13, 24, 21, 249, 13, 189, 28, 21, 91, 16, 108, 85, 26, 11, 32, 14, 23, 14, 17, 15, 18, 31, 19, 59, 41, 17, 13, 10, 21, 12, 25, 103, 21, 80, 16, 20, 14, 295, 22, 40, 66, 21, 23, 32, 32, 21, 10, 19, 22, 25, 21, 23, 20, 21, 16, 8, 25, 23, 16, 21, 21, 25, 23, 9, 27, 21, 13, 19, 27, 19, 10, 8, 15, 18, 25, 17, 65, 16, 14, 28, 18, 16, 28, 37, 15, 22, 20, 18, 14, 28, 18, 10, 20, 18, 23, 8, 22, 24, 20, 23, 16, 21, 21, 16, 16, 11, 13, 24, 10, 13, 16, 17, 19, 19, 17, 18, 11, 22, 28, 27, 160, 33, 18, 18, 57, 8, 47, 25, 20, 20, 16, 9, 28, 28, 25, 14, 14, 19, 26, 24, 11, 23, 11, 60, 15, 27, 535, 37, 19, 27, 29, 27, 11, 150, 20, 23, 33, 27, 15, 360, 34, 30, 234, 38, 26, 25, 25, 12, 15, 23, 28, 157, 11, 11, 115, 16, 134, 41, 305, 49, 18, 28, 60, 18, 232, 38, 116, 247, 21, 29, 18, 18, 20, 30, 117, 19, 36, 237, 15, 15, 14, 43, 20, 13, 13, 92, 9, 11, 18, 18, 196, 22, 44, 25, 109, 17, 13, 34, 280, 262, 12, 256, 20, 28, 17, 124, 33, 21, 23, 12, 9, 10, 15, 15, 22, 24, 17, 20, 9, 19, 15, 37, 31, 46, 15, 47, 24, 10, 21, 9, 35, 16, 28, 25, 19, 28, 7, 20, 12, 12, 7, 18, 14, 11, 9, 19, 29, 20, 32, 10, 79, 12, 30, 75, 14, 18, 20, 17, 16, 12, 21, 22, 16, 34, 125, 11, 17, 31, 22, 20, 116, 311, 25, 30, 32, 15, 73, 44, 19, 135, 22, 61, 24, 31, 1000, 41, 15, 74, 9, 99, 40, 121, 24, 14, 107, 11, 19, 47, 155, 51, 12, 11, 81, 20, 220, 760, 90, 279, 105, 325, 40, 147, 31, 16, 13, 42, 7, 14, 22, 36, 279, 11, 17, 14, 13, 20, 12, 11, 21, 14, 24, 1000, 18, 34, 141, 58, 14, 38, 391, 983, 19, 24, 41, 19, 25, 38, 573, 139, 28, 62, 51, 18, 14, 73, 279, 17, 18, 224, 166, 11, 7, 17, 17, 20, 29, 30, 29, 10, 23, 30, 26, 19, 19, 16, 23, 27, 29, 8, 21, 25, 15, 28, 17, 14, 42, 129, 11, 9, 15, 23, 27, 19, 16, 22, 67, 22, 19, 19, 54, 21, 22, 28, 29, 40, 17, 18, 14, 12, 20, 24, 26, 14, 21, 14, 11, 24, 22, 42, 25, 26, 32, 26, 15, 27, 19, 24, 20, 16, 11, 10, 8, 12, 23, 15, 12, 9, 22, 24, 25, 15, 20, 17, 21, 11, 28, 119, 16, 30, 24, 33, 28, 19, 71, 13, 13, 210, 33, 16, 28, 28, 24, 13, 28, 20, 17, 19, 18, 25, 27, 21, 18, 26, 18, 17, 18, 40, 26, 27, 26, 28, 22, 33, 26, 31, 14, 24, 8, 243, 27, 23, 26, 20, 17, 28, 28, 10, 63, 20, 19, 28, 26, 16, 20, 17, 24, 14, 17, 16, 28, 25, 14, 21, 23, 25, 25, 20, 16, 25, 28, 111, 24, 20, 24, 12, 24, 18, 14, 56, 24, 24, 25, 23, 17, 20, 23, 14, 28, 19, 23, 21, 15, 17, 15, 15, 36, 23, 26, 21, 24, 11, 26, 16, 25, 13, 26, 16, 20, 27, 19, 28, 26, 21, 18, 17, 17, 17, 15, 11, 10, 35, 17, 74, 13, 26, 131, 27, 22, 43, 24, 18, 16, 22, 28, 18, 22, 44, 14, 18, 18, 13, 15, 27, 20, 19, 21, 15, 15, 27, 11, 19, 20, 24, 19, 25, 12, 24, 17, 22, 21, 16, 25, 17, 25, 24, 16, 10, 22, 28, 15, 26, 11, 19, 27, 19, 24, 16, 26, 28, 19, 25, 14, 18, 26, 13, 14, 10, 23, 14, 16, 24, 13, 20, 28, 17, 25, 12, 26, 27, 20, 22, 10, 20, 23, 13, 23, 18, 10, 27, 11, 15, 14, 27, 22, 18, 19, 25, 21, 23, 25, 13, 17, 18, 10, 15, 24, 18, 25, 13, 14, 21, 18, 14, 28, 25, 18, 21, 27, 20, 18, 20, 23, 12, 27, 15, 25, 24, 14, 12, 11, 21, 27, 16, 28, 28, 26, 28, 22, 20, 17, 24, 26, 28, 21, 22, 23, 16, 12, 19, 16, 12, 24, 11, 28, 24, 14, 28, 20, 19, 22, 20, 23, 22, 15, 21, 13, 16, 13, 13, 20, 14, 27, 12, 20, 23, 28, 20, 23, 28, 23, 20, 25, 26, 22, 13, 26, 18, 25, 20, 20, 14, 28, 20, 23, 17, 13, 14, 22, 25, 16, 19, 24, 13, 28, 16, 20, 21, 20, 13, 18, 21, 22, 30, 25, 18, 17, 15, 47, 13, 27, 19, 15, 62, 12, 34, 37, 25, 27, 13, 27, 26, 19, 20, 11, 23, 16, 25, 20, 18, 19, 24, 23, 12, 10, 13, 25, 13, 19, 28, 25, 23, 26, 25, 25, 27, 23, 28, 14, 15, 19, 16, 18, 15, 13, 19, 22, 15, 20, 20, 25, 19, 19, 18, 22, 13, 11, 23, 25, 27, 14, 27, 13, 16, 26, 19, 27, 17, 24, 12, 42, 11, 16, 13, 19, 24, 16, 23, 19, 25, 14, 13, 605, 238, 25, 29, 167, 173, 134, 82, 20, 145, 11, 15, 21, 22, 10, 24, 13, 622, 18, 26, 19, 6, 7, 24, 29, 28, 18, 20, 381, 20, 25, 82, 35, 285, 32, 22, 15, 21, 7, 22, 157, 28, 32, 85, 22, 178, 28, 24, 128, 87, 443, 25, 12, 27, 55, 25, 13, 14, 24, 24, 15, 24, 40, 41, 13, 25, 22, 20, 21, 27, 19, 19, 22, 15, 24, 28, 26, 184, 835, 585, 26, 28, 149, 264, 12, 185, 264, 21, 19, 14, 14, 40, 43, 11, 19, 17, 78, 12, 20, 10, 27, 25, 150, 14, 120, 43, 23, 51, 18, 28, 28, 20, 12, 82, 425, 24, 26, 14, 18, 168, 51, 21, 15, 26, 23, 26, 41, 312, 14, 28, 17, 17, 8, 460, 17, 72, 14, 162, 251, 12, 21, 19, 15, 25, 19, 14, 763, 107, 19, 19, 110, 26, 12, 23, 366, 28, 1000, 83, 776, 107, 98, 44, 16, 205, 56, 13, 12, 21, 20, 28, 31, 11, 5, 22, 9, 250, 49, 23, 23, 16, 16, 15, 23, 11, 966, 15, 15, 94, 25, 21, 14, 17, 61, 147, 11, 15, 94, 12, 16, 13, 14, 21, 49, 120, 12, 21, 15, 19, 1000, 671, 123, 20, 328, 11, 354, 28, 19, 31, 16, 1000, 27, 105, 43, 24, 10, 140, 180, 27, 6, 9, 25, 7, 26, 268, 186, 19, 485, 29, 78, 158, 22, 182, 134, 1000, 52, 659, 382, 42, 197, 35, 27, 72, 26, 90, 72, 45, 29, 28, 433, 268, 22, 71, 415, 4, 27, 6, 18, 4, 30, 266, 22, 24, 437, 280, 585, 16, 26, 69, 12, 2, 6, 5, 11, 15, 24, 110, 16, 22, 17, 25, 25, 15, 26, 13, 17, 25, 23, 33, 35, 38, 27, 21, 21, 95, 28, 19, 16, 30, 25, 16, 33, 47, 36, 17, 15, 93, 401, 876, 31, 18, 14, 27, 24, 16, 13, 20, 212, 7, 20, 11, 15, 17, 25, 21, 28, 28, 23, 34, 90, 129, 18, 27, 19, 1000, 585, 37, 16, 14, 92, 26, 11, 26, 172, 20, 24, 18, 28, 17, 13, 11, 16, 24, 62, 130, 7, 28, 253, 17, 1000, 30, 27, 21, 52, 353, 22, 34, 14, 10, 15, 46, 248, 22, 15, 129, 95, 103, 26, 14, 27, 62, 10, 15, 20, 17, 35, 179, 188, 119, 17, 17, 290, 22, 17, 350, 24, 255, 132, 15, 928, 149, 55, 124, 31, 328, 34, 157, 39, 673, 18, 21, 500, 54, 25, 23, 35, 192, 259, 116, 33, 33, 13, 78, 134, 255, 82, 43, 43, 623, 470, 13, 41, 643, 313, 162, 17, 359, 207, 131, 62, 17, 475, 26, 64, 24, 67, 130, 51, 356, 280, 26, 287, 80, 97, 17, 138, 58, 442, 679, 158, 16, 25, 15, 17, 27, 19, 27, 91, 10, 25, 13, 28, 227, 18, 11, 23, 15, 1000, 28, 130, 412, 83, 59, 22, 26, 41, 869, 46, 15, 16, 137, 34, 25, 14, 25, 12, 59, 673, 25, 9, 24, 12, 35, 26, 13, 16, 547, 32, 26, 1000, 49, 38, 76, 10, 12, 22, 15, 11, 18, 622, 13, 73, 23, 60, 24, 26, 17, 14, 12, 27, 15, 16, 23, 26, 29, 18, 22, 43, 20, 18, 14, 15, 88, 17, 209, 12, 400, 15, 62, 14, 26, 297, 27, 184, 17, 322, 23, 930, 233, 25, 10, 23, 27, 21, 19, 85, 7, 26, 376, 21, 78, 54, 72, 18, 29, 73, 18, 21, 57, 19, 15, 1000, 54, 13, 139, 23, 19, 454, 35, 88, 86, 28, 349, 40, 27, 28, 10, 15, 15, 1000, 27, 221, 28, 121, 399, 30, 29, 17, 6, 81, 88, 23, 20, 46, 22, 28, 22, 15, 26, 40, 15, 17, 24, 18, 37, 477, 17, 48, 178, 8, 25, 20, 857, 42, 21, 14, 27, 47, 21, 13, 1000, 79, 10, 76, 235, 236, 324, 15, 257, 247, 218, 127, 290, 93, 267, 18, 15, 55, 21, 12, 13, 24, 16, 582, 19, 40, 52, 910, 677, 119, 101, 11, 14, 38, 9, 66, 16, 768, 391, 15, 21, 18, 85, 30, 28, 18, 26, 22, 99, 15, 18, 75, 61, 118, 29, 165, 55, 13, 21, 20, 14, 84, 10, 23, 26, 15, 13, 20, 12, 14, 132, 23, 20, 24, 26, 13, 22, 46, 17, 14, 14, 17, 20, 62, 24, 117, 99, 24, 36, 32, 12, 24, 13, 235, 40, 1000, 164, 16, 13, 7, 67, 38, 31, 26, 19, 261, 761, 1000, 26, 8, 23, 24, 15, 16, 31, 30, 800, 11, 12, 196, 9, 138, 24, 16, 14, 31, 14, 160, 30, 19, 471, 108, 20, 1000, 34, 15, 72, 90, 46, 372, 32, 26, 131, 19, 254, 326, 78, 23, 19, 30, 18, 11, 75, 137, 157, 562, 9, 19, 37, 23, 14, 106, 989, 383, 23, 71, 232, 141, 51, 25, 7, 19, 15, 66, 14, 21, 388, 19, 19, 10, 10, 26, 25, 21, 25, 21, 9, 13, 115, 26, 18, 20, 21, 9, 204, 476, 24, 19, 30, 85, 22, 18, 1000, 11, 17, 373, 38, 136, 197, 18, 23, 24, 14, 25, 8, 20, 143, 21, 15, 26, 12, 24, 212, 24, 43, 27, 25, 22, 22, 23, 198, 28, 26, 24, 22, 21, 21, 50, 12, 23, 119, 30, 23, 16, 35, 12, 20, 18, 21, 13, 22, 16, 19, 13, 21, 13, 16, 27, 26, 63, 21, 67, 385, 152, 23, 12, 133, 14, 144, 13, 50, 733, 339, 105, 13, 18, 9, 242, 15, 27, 20, 29, 395, 17, 15, 30, 602, 26, 263, 22, 366, 15, 50, 151, 16, 25, 42, 53, 64, 128, 114, 64, 53, 1000, 43, 44, 60, 17, 26, 124, 82, 36, 184, 120, 496, 126, 10, 233, 42, 252, 149, 14, 63, 19, 12, 44, 133, 32, 17, 17, 18, 297, 1000, 7, 117, 243, 1000, 20, 506, 215, 18, 572, 242, 26, 21, 13, 12, 34, 161, 539, 86, 31, 78, 599, 122, 115, 639, 369, 46, 1000, 108, 26, 1000, 17, 23, 27, 104, 27, 12, 6, 24, 7, 13, 32, 9, 21, 27, 14, 20, 13, 26, 9, 22, 13, 19, 18, 471, 9, 26, 20, 60, 141, 38, 20, 12, 23, 10, 17, 13, 14, 638, 160, 80, 26, 10, 79, 25, 39, 228, 654, 78, 71, 23, 27, 25, 12, 118, 13, 20, 28, 22, 15, 21, 15, 22, 23, 1000, 32, 30, 14, 13, 32, 19, 28, 23, 27, 24, 48, 28, 17, 16, 19, 22, 37, 14, 35, 27, 18, 27, 19, 189, 22, 17, 27, 15, 15, 35, 67, 16, 16, 29, 35, 23, 38, 146, 23, 13, 10, 72, 109, 159, 530, 16, 13, 377, 284, 37, 20, 30, 366, 92, 50, 90, 12, 31, 141, 500, 695, 23, 14, 25, 18, 13, 124, 15, 10, 21, 28, 409, 1000, 83, 64, 12, 97, 26, 23, 47, 91, 59, 15, 13, 64, 17, 17, 12, 11, 19, 24, 9, 31, 181, 118, 28, 30, 11, 15, 169, 378, 683, 913, 690, 343, 12, 23, 24, 371, 133, 391, 332, 208, 28, 11, 16, 28, 18, 13, 140, 94, 62, 119, 25, 66, 67, 16, 1000, 306, 12, 1000, 13, 10, 15, 13, 30, 99, 21, 499, 44, 961, 441, 144, 115, 187, 1000, 344, 15, 25, 417, 21, 935, 22, 17, 10, 59, 36, 26, 754, 1000, 22, 14, 523, 19, 10, 245, 311, 101, 15, 26, 8, 7, 15, 18, 25, 20, 22, 32, 47, 24, 18, 21, 15, 27, 756, 17, 13, 13, 21, 109, 15, 21, 14, 28, 28, 52, 16, 415, 19, 327, 760, 95, 167, 495, 25, 13, 17, 10, 26, 52, 15, 13, 180, 1000, 203, 357, 25, 89, 346, 11, 19, 83, 244, 218, 122, 53, 21, 365, 793, 22, 27, 61, 15, 56, 17, 26, 34, 24, 22, 25, 27, 43, 7, 28, 17, 19, 39, 16, 23, 10, 14, 25, 23, 17, 35, 29, 13, 26, 15, 21, 12, 13, 45, 19, 18, 157, 25, 22, 22, 21, 15, 14, 26, 21, 52, 32, 195, 26, 20, 27, 18, 21, 19, 31, 18, 190, 24, 26, 14, 107, 411, 64, 28, 95, 17, 27, 28, 10, 18, 18, 234, 96, 263, 225, 9, 88, 18, 13, 13, 1000, 331, 26, 217, 115, 133, 9, 225, 34, 59, 50, 31, 24, 38, 30, 15, 23, 25, 24, 21, 9, 24, 14, 15, 19, 18, 27, 28, 24, 22, 26, 14, 16, 10, 24, 27, 16, 26, 24, 13, 25, 27, 23, 22, 167, 36, 8, 10, 93, 21, 14, 26, 37, 12, 18, 20, 11, 17, 18, 19, 17, 14, 43, 13, 10, 21, 22, 104, 19, 12, 21, 13, 37, 11, 10, 16, 19, 235, 153, 5, 9, 20, 950, 1000, 21, 36, 45, 30, 160, 86, 21, 9, 900, 26, 20, 22, 233, 8, 25, 23, 483, 25, 55, 16, 23, 18, 472, 42, 23, 39, 22, 18, 13, 20, 20, 25, 14, 31, 20, 274, 14, 14, 139, 181, 24, 20, 12, 14, 21, 11, 32, 20, 24, 24, 77, 20, 98, 20, 15, 38, 341, 11, 23, 86, 407, 31, 122, 18, 1000, 359, 39, 15, 18, 21, 20, 11, 16, 235, 14, 11, 19, 32, 20, 86, 35, 27, 18, 17, 22, 28, 80, 16, 21, 19, 27, 23, 17, 294, 20, 12, 380, 26, 24, 14, 15, 25, 12, 63, 20, 20, 125, 28, 21, 28, 16, 16, 29, 254, 64, 8, 75, 16, 223, 70, 26, 512, 16, 138, 24, 61, 8, 14, 19, 28, 28, 20, 10, 35, 21, 17, 22, 16, 19, 26, 84, 444, 11, 22, 133, 27, 15, 25, 20, 194, 78, 27, 23, 11, 82, 150, 21, 12, 149, 17, 95, 20, 11, 120, 206, 530, 206, 17, 15, 13, 82, 1000, 13, 11, 154, 1000, 81, 19, 13, 38, 23, 20, 21, 499, 11, 19, 68, 420, 15, 24, 12, 21, 35, 27, 490, 15, 61, 27, 14, 203, 10, 13, 12, 16, 19, 21, 21, 10, 39, 12, 537, 10, 14, 22, 7, 24, 25, 24, 18, 26, 17, 47, 25, 12, 17, 137, 255, 26, 53, 14, 13, 23, 20, 23, 26, 25, 231, 52, 14, 6, 19, 97, 34, 17, 60, 12, 16, 24, 28, 157, 8, 13, 51, 17, 20, 33, 6, 24, 15, 25, 13, 15, 25, 13, 51, 10, 216, 104, 162, 244, 24, 169, 113, 135, 284, 11, 13, 675, 23, 126, 47, 16, 24, 170, 43, 19, 59, 35, 131, 24, 182, 244, 698, 26, 22, 28, 23, 19, 10, 11, 73, 20, 105, 961, 8, 15, 10, 24, 379, 17, 18, 65, 49, 21, 20, 27, 24, 43, 11, 337, 9, 15, 12, 16, 17, 14, 21, 41, 142, 58, 245, 74, 13, 13, 154, 7, 25, 14, 25, 17, 33, 23, 50, 22, 101, 111, 14, 19, 15, 104, 272, 62, 12, 46, 12, 22, 25, 75, 35, 26, 106, 17, 19, 9, 26, 20, 1000, 393, 36, 64, 14, 29, 31, 63, 31, 34, 17, 16, 11, 118, 876, 24, 37, 24, 22, 22, 8, 8, 208, 14, 42, 30, 117, 192, 25, 11, 22, 15, 26, 9, 9, 20, 18, 32, 186, 47, 17, 25, 24, 57, 19, 229, 17, 25, 14, 12, 18, 15, 15, 15, 302, 368, 16, 12, 21, 649, 24, 33, 96, 43, 7, 352, 7, 28, 7, 26, 39, 66, 24, 63, 22, 12, 25, 12, 24, 117, 46, 662, 28, 84, 575, 27, 29, 10, 13, 9, 69, 540, 158, 1000, 12, 24, 26, 20, 28, 12, 24, 17, 41, 15, 29, 98, 15, 47, 9, 11, 32, 152, 28, 11, 44, 91, 150, 145, 24, 55, 14, 21, 22, 16, 22, 22, 25, 23, 20, 19, 18, 9, 20, 25, 17, 63, 20, 45, 12, 16, 24, 13, 9, 17, 33, 569, 38, 26, 12, 29, 21, 24, 173, 41, 151, 26, 23, 20, 21, 24, 12, 17, 30, 147, 15, 25, 32, 20, 26, 23, 15, 21, 24, 9, 22, 28, 24, 13, 26, 16, 141, 17, 12, 20, 14, 18, 20, 24, 15, 13, 37, 28, 27, 13, 27, 23, 28, 14, 14, 23, 20, 61, 11, 21, 18, 12, 20, 24, 20, 21, 28, 28, 15, 22, 14, 18, 15, 51, 27, 21, 20, 20, 19, 86, 28, 18, 19, 28, 27, 20, 22, 17, 19, 21, 22, 28, 27, 17, 21, 19, 19, 21, 27, 14, 22, 24, 18, 13, 13, 11, 22, 28, 27, 9, 16, 20, 18, 18, 20, 44, 45, 19, 13, 37, 27, 28, 28, 18, 10, 23, 14, 10, 24, 11, 107, 20, 14, 24, 18, 24, 28, 27, 20, 21, 22, 15, 16, 18, 23, 11, 18, 28, 21, 16, 9, 17, 19, 16, 14, 16, 15, 12, 17, 23, 18, 70, 21, 28, 11, 7, 15, 19, 25, 26, 28, 26, 27, 28, 23, 13, 11, 7, 11, 7, 9, 13, 8, 28, 7, 56, 160, 131, 7, 15, 22, 17, 9, 105, 163, 11, 26, 25, 28, 7, 28, 14, 23, 27, 307, 17, 23, 13, 26, 55, 11, 9, 21, 22, 23, 14, 14, 27, 13, 16, 21, 25, 189, 266, 9, 10, 7, 14, 15, 16, 11, 14, 15, 23, 24, 25, 36, 97, 26, 9, 27, 15, 16, 19, 18, 19, 24, 9, 13, 17, 20, 63, 706, 21, 35, 116, 18, 11, 26, 16, 10, 26, 120, 22, 13, 20, 22, 26, 21, 27, 24, 17, 10, 21, 27, 93, 28, 24, 13, 32, 22, 21, 15, 19, 19, 20, 77, 30, 23, 71, 66, 14, 28, 22, 16, 22, 25, 10, 32, 15, 12, 41, 9, 22, 28, 10, 28, 12, 47, 28, 25, 26, 19, 16, 27, 18, 57, 30, 37, 25, 9, 21, 11, 27, 16, 20, 14, 23, 14, 28, 24, 18, 17, 25, 17, 9, 53, 51, 16, 28, 12, 17, 28, 27, 11, 20, 23, 27, 23, 9, 14, 30, 9, 10, 22, 13, 23, 11, 12, 14, 10, 19, 34, 26, 9, 8, 127, 28, 27, 25, 18, 13, 25, 14, 9, 77, 16, 22, 17, 15, 23, 26, 20, 19, 21, 13, 15, 20, 17, 63, 9, 10, 28, 24, 37, 13, 9, 13, 21, 22, 10, 9, 13, 12, 20, 22, 17, 9, 20, 18, 20, 17, 28, 14, 23, 33, 25, 12, 21, 13, 13, 7, 22, 25, 9, 15, 119, 28, 20, 24, 11, 24, 15, 10, 14, 14, 12, 24, 9, 11, 12, 333, 13, 20, 13, 18, 18, 9, 14, 10, 27, 24, 16, 16, 19, 28, 27, 20, 16, 15, 9, 17, 10, 9, 18, 9, 24, 16, 25, 16, 32, 13, 10, 18, 24, 22, 14, 27, 7, 21, 19, 26, 17, 10, 18, 13, 19, 24, 10, 17, 12, 23, 18, 9, 18, 45, 16, 10, 21, 14, 14, 8, 20, 61, 10, 13, 19, 22, 10, 25, 16, 20, 8, 12, 23, 19, 14, 19, 10, 17, 7, 19, 13, 25, 26, 21, 8, 27, 13, 20, 22, 15, 23, 14, 25, 19, 22, 18, 16, 21, 10, 28, 14, 24, 26, 17, 28, 13, 17, 19, 11, 9, 28, 9, 17, 20, 16, 15, 12, 20, 22, 17, 12, 18, 15, 27, 12, 19, 24, 15, 19, 15, 19, 22, 21, 26, 14, 19, 15, 22, 14, 21, 10, 37, 25, 56, 9, 13, 16, 10, 11, 20, 20, 18, 23, 22, 24, 12, 22, 17, 10, 15, 14, 11, 79, 25, 20, 12, 18, 101, 20, 12, 23, 26, 227, 18, 24, 12, 13, 10, 22, 13, 12, 10, 26, 20, 16, 13, 30, 12, 11, 17, 17, 28, 17, 26, 15, 25, 12, 42, 22, 16, 28, 23, 21, 21, 26, 23, 12, 26, 11, 25, 11, 24, 15, 19, 22, 25, 24, 14, 70, 11, 54, 25, 36, 14, 26, 19, 30, 36, 8, 25, 56, 78, 29, 16, 11, 11, 30, 12, 14, 21, 53, 18, 38, 602, 58, 22, 20, 183, 18, 17, 44, 18, 13, 28, 15, 16, 21, 21, 18, 12, 23, 210, 49, 24, 31, 19, 16, 20, 28, 21, 19, 19, 45, 51, 14, 18, 18, 27, 16, 105, 23, 15, 9, 10, 59, 8, 23, 20, 12, 21, 11, 21, 12, 16, 14, 21, 15, 21, 14, 11, 27, 17, 20, 101, 103, 15, 11, 28, 10, 215, 15, 17, 17, 19, 8, 235, 12, 23, 16, 25, 22, 187, 29, 326, 70, 14, 25, 17, 27, 15, 16, 26, 36, 15, 22, 11, 23, 25, 29, 123, 94, 26, 16, 23, 16, 30, 24, 26, 24, 25, 27, 13, 58, 15, 16, 24, 52, 15, 28, 23, 18, 27, 59, 20, 28, 25, 123, 13, 78, 11, 21, 28, 20, 203, 15, 15, 11, 23, 16, 18, 13, 13, 15, 37, 20, 27, 14, 27, 11, 67, 23, 17, 24, 23, 23, 11, 16, 223, 20, 28, 32, 15, 18, 18, 12, 24, 17, 32, 20, 114, 19, 26, 43, 87, 18, 21, 26, 22, 14, 20, 22, 22, 24, 28, 14, 17, 13, 21, 16, 24, 24, 16, 19, 27, 16, 21, 25, 13, 17, 20, 23, 20, 27, 11, 41, 12, 23, 14, 28, 14, 10, 16, 15, 22, 17, 28, 20, 10, 9, 17, 15, 9, 27, 15, 42, 18, 65, 16, 18, 84, 45, 17, 31, 24, 17, 75, 128, 17, 30, 26, 22, 37, 40, 86, 25, 22, 11, 19, 17, 11, 30, 55, 28, 14, 40, 16, 13, 127, 45, 290, 17, 25, 73, 16, 17, 15, 24, 14, 20, 23, 29, 20, 121, 21, 17, 10, 57, 25, 15, 17, 18, 31, 270, 18, 168, 19, 96, 84, 48, 81, 39, 11, 70, 10, 24, 20, 19, 29, 14, 21, 28, 15, 27, 23, 19, 127, 18, 120, 20, 17, 23, 17, 16, 15, 25, 20, 14, 26, 18, 28, 18, 25, 24, 18, 26, 17, 31, 17, 24, 27, 21, 27, 21, 13, 27, 23, 15, 28, 13, 10, 11, 20, 17, 11, 36, 18, 18, 24, 15, 14, 15, 20, 20, 14, 22, 22, 24, 13, 23, 19, 28, 17, 23, 27, 55, 44, 18, 21, 18, 80, 22, 14, 25, 26, 47, 16, 12, 49, 11, 32, 12, 23, 20, 10, 19, 26, 71, 21, 15, 31, 71, 12, 21, 15, 16, 12, 22, 15, 24, 22, 20, 18, 19, 17, 23, 12, 22, 10, 12, 25, 20, 35, 20, 28, 17, 15, 17, 21, 17, 17, 27, 16, 17, 76, 26, 171, 16, 28, 51, 23, 17, 16, 25, 62, 54, 18, 9, 19, 21, 12, 20, 26, 9, 17, 232, 29, 25, 14, 23, 14, 18, 21, 16, 24, 45, 15, 16, 16, 8, 26, 10, 20, 19, 11, 21, 21, 138, 13, 22, 488, 19, 20, 23, 47, 13, 23, 52, 20, 270, 25, 11, 158, 12, 26, 24, 22, 18, 13, 14, 18, 161, 26, 34, 27, 43, 19, 39, 26, 23, 28, 10, 31, 15, 49, 18, 28, 48, 21, 16, 68, 32, 15, 353, 28, 20, 19, 33, 16, 35, 43, 9, 77, 19, 12, 28, 22, 18, 26, 48, 67, 14, 30, 30, 64, 22, 44, 25, 50, 25, 10, 8, 173, 47, 80, 17, 920, 17, 17, 24, 27, 65, 116, 13, 16, 89, 29, 17, 8, 45, 97, 12, 22, 137, 28, 15, 18, 17, 31, 36, 23, 21, 27, 22, 17, 12, 25, 287, 11, 134, 20, 23, 17, 19, 29, 30, 28, 21, 27, 336, 159, 15, 10, 24, 32, 9, 162, 12, 16, 19, 26, 37, 20, 23, 15, 26, 41, 19, 14, 13, 26, 85, 20, 63, 14, 28, 13, 18, 25, 13, 43, 55, 16, 21, 68, 28, 16, 18, 26, 19, 28, 11, 20, 325, 24, 20, 118, 24, 20, 19, 46, 24, 11, 21, 23, 29, 284, 84, 27, 13, 56, 11, 10, 37, 33, 36, 14, 21, 25, 23, 10, 11, 19, 22, 31, 27, 21, 18, 18, 97, 52, 69, 22, 15, 22, 11, 77, 25, 21, 68, 40, 18, 18, 62, 28, 51, 17, 99, 41, 37, 25, 51, 28, 21, 16, 250, 21, 26, 25, 22, 11, 28, 27, 19, 10, 72, 26, 17, 16, 130, 17, 13, 23, 24, 21, 20, 13, 102, 46, 17, 35, 11, 112, 18, 25, 16, 20, 57, 18, 22, 27, 38, 17, 17, 17, 19, 84, 27, 25, 17, 16, 58, 23, 21, 221, 14, 20, 16, 20, 9, 40, 20, 12, 22, 48, 22, 141, 30, 17, 12, 64, 10, 20, 32, 15, 14, 24, 20, 27, 23, 20, 18, 18, 17, 24, 36, 25, 9, 27, 25, 11, 8, 24, 8, 15, 14, 11, 21, 30, 10, 14, 17, 27, 23, 11, 29, 18, 22, 21, 28, 25, 20, 8, 16, 17, 21, 132, 255, 26, 19, 12, 15, 18, 21, 28, 22, 27, 12, 15, 27, 9, 20, 14, 16, 16, 11, 13, 26, 17, 83, 12, 10, 43, 155, 42, 49, 29, 18, 11, 28, 9, 21, 15, 21, 8, 19, 18, 23, 20, 13, 16, 11, 67, 85, 12, 13, 32, 15, 14, 10, 10, 162, 25, 23, 11, 8, 15, 19, 22, 27, 20, 88, 15, 854, 20, 47, 18, 13, 67, 31, 14, 15, 12, 211, 29, 22, 12, 35, 49, 42, 22, 17, 14, 14, 24, 22, 23, 12, 17, 112, 12, 17, 14, 14, 14, 16, 18, 19, 82, 16, 32, 21, 20, 10, 32, 192, 20, 17, 96, 14, 29, 20, 18, 15, 15, 23, 26, 20, 17, 85, 13, 15, 94, 11, 66, 20, 12, 13, 85, 23, 18, 17, 24, 59, 10, 36, 22, 24, 25, 83, 16, 18, 27, 28, 12, 20, 32, 19, 24, 20, 11, 14, 24, 21, 24, 11, 22, 26, 13, 21, 25, 22, 21, 22, 10, 20, 388, 12, 20, 18, 40, 18, 19, 9, 64, 123, 100, 141, 64, 21, 25, 15, 21, 19, 26, 21, 69, 16, 19, 27, 104, 89, 22, 13, 24, 21, 48, 45, 25, 108, 18, 11, 11, 13, 26, 55, 15, 17, 15, 21, 16, 104, 26, 25, 68, 23, 9, 16, 10, 16, 23, 25, 16, 23, 26, 24, 12, 25, 23, 34, 11, 22, 24, 15, 12, 24, 17, 16, 21, 16, 10, 20, 13, 28, 18, 27, 22, 60, 28, 25, 8, 28, 22, 19, 22, 9, 67, 84, 11, 21, 18, 31, 11, 22, 11, 21, 32, 10, 17, 13, 22, 13, 17, 13, 27, 14, 11, 23, 13, 27, 24, 10, 24, 12, 22, 14, 18, 12, 82, 10, 25, 18, 47, 19, 59, 91, 14, 13, 26, 12, 26, 22, 25, 21, 13, 25, 19, 17, 26, 13, 22, 66, 17, 17, 26, 10, 17, 26, 17, 26, 9, 9, 55, 29, 33, 60, 71, 25, 22, 23, 51, 17, 28, 10, 15, 21, 127, 15, 10, 20, 14, 8, 26, 97, 29, 20, 15, 87, 21, 28, 20, 260, 25, 30, 222, 26, 21, 27, 43, 19, 21, 34, 11, 14, 25, 443, 19, 17, 35, 23, 13, 23, 15, 24, 33, 20, 26, 10, 26, 13, 20, 18, 365, 21, 24, 56, 22, 32, 12, 11, 321, 14, 36, 29, 14, 18, 10, 8, 16, 92, 15, 35, 14, 88, 113, 13, 28, 26, 123, 20, 21, 59, 51, 35, 19, 31, 12, 26, 22, 11, 16, 22, 31, 201, 52, 129, 22, 12, 151, 26, 16, 13, 13, 28, 22, 25, 12, 10, 22, 17, 21, 23, 26, 16, 56, 15, 28, 15, 46, 53, 79, 16, 291, 118, 67, 20, 10, 131, 10, 7, 34, 14, 19, 57, 62, 16, 15, 12, 44, 19, 110, 72, 203, 34, 20, 32, 10, 14, 14, 20, 16, 83, 24, 12, 24, 28, 9, 25, 429, 85, 13, 20, 40, 20, 8, 28, 20, 253, 21, 13, 35, 122, 30, 28, 14, 20, 20, 36, 24, 19, 13, 18, 17, 26, 95, 106, 9, 138, 16, 36, 19, 13, 48, 363, 13, 70, 13, 30, 20, 27, 20, 24, 51, 28, 12, 22, 90, 10, 14, 17, 75, 30, 22, 13, 18, 23, 123, 18, 28, 28, 18, 27, 22, 64, 24, 13, 19, 26, 16, 17, 24, 19, 26, 66, 18, 16, 14, 14, 17, 34, 352, 15, 20, 6, 32, 7, 25, 20, 20, 30, 10, 37, 13, 20, 58, 13, 8, 15, 24, 34, 21, 14, 23, 25, 161, 22, 23, 14, 27, 14, 20, 17, 11, 12, 28, 24, 17, 24, 17, 121, 18, 16, 21, 45, 24, 32, 29, 12, 16, 21, 27, 17, 19, 15, 29, 24, 20, 20, 24, 20, 25, 28, 12, 20, 18, 12, 37, 18, 264, 28, 152, 24, 23, 20, 15, 67, 13, 13, 28, 27, 15, 23, 17, 51, 110, 22, 17, 15, 25, 14, 23, 9, 15, 45, 28, 28, 27, 14, 22, 21, 247, 16, 29, 104, 20, 25, 9, 33, 26, 16, 21, 18, 21, 15, 17, 140, 23, 16, 11, 27, 28, 90, 17, 45, 29, 140, 77, 16, 28, 13, 14, 16, 7, 20, 23, 9, 28, 20, 16, 16, 129, 14, 18, 10, 17, 151, 97, 17, 13, 23, 23, 30, 23, 29, 27, 15, 22, 15, 15, 38, 10, 28, 322, 492, 24, 15, 22, 78, 7, 41, 16, 23, 28, 102, 16, 80, 15, 30, 20, 22, 38, 49, 17, 8, 344, 17, 28, 302, 31, 14, 80, 58, 82, 11, 74, 23, 708, 80, 276, 312, 11, 18, 22, 17, 23, 16, 17, 24, 18, 20, 77, 21, 76, 89, 44, 54, 45, 31, 419, 15, 13, 77, 15, 12, 23, 33, 29, 40, 115, 17, 31, 67, 20, 69, 43, 8, 14, 13, 19, 24, 25, 17, 27, 40, 47, 26, 26, 26, 28, 273, 18, 82, 18, 19, 77, 23, 160, 44, 19, 23, 22, 118, 17, 61, 25, 12, 14, 15, 560, 17, 204, 56, 77, 254, 14, 27, 19, 20, 13, 8, 26, 23, 34, 29, 143, 94, 15, 22, 21, 24, 19, 28, 13, 22, 20, 23, 22, 24, 57, 21, 20, 18, 16, 11, 8, 25, 46, 207, 25, 22, 11, 12, 11, 15, 29, 25, 39, 25, 60, 39, 76, 20, 26, 10, 33, 25, 17, 233, 12, 8, 12, 27, 25, 27, 67, 16, 23, 29, 22, 233, 18, 14, 27, 21, 27, 64, 217, 15, 9, 21, 15, 151, 17, 27, 164, 15, 12, 25, 19, 19, 12, 13, 16, 20, 21, 24, 9, 130, 24, 35, 18, 21, 24, 33, 19, 24, 27, 12, 30, 13, 21, 28, 12, 32, 25, 13, 8, 20, 9, 156, 22, 22, 20, 139, 152, 35, 15, 13, 172, 25, 16, 16, 25, 31, 19, 73, 140, 46, 31, 9, 307, 12, 117, 27, 74, 16, 27, 24, 28, 35, 51, 193, 16, 13, 22, 13, 31, 14, 29, 19, 12, 28, 19, 15, 11, 17, 23, 23, 53, 24, 23, 27, 19, 31, 21, 18, 14, 13, 18, 28, 9, 18, 26, 12, 255, 29, 27, 15, 13, 24, 9, 41, 31, 16, 26, 26, 8, 20, 10, 20, 11, 8, 64, 106, 14, 21, 23, 13, 10, 18, 23, 21, 19, 28, 89, 27, 33, 26, 15, 24, 13, 28, 11, 21, 21, 24, 79, 15, 22, 26, 18, 23, 19, 24, 488, 27, 28, 36, 21, 258, 19, 10, 19, 40, 21, 27, 12, 22, 124, 63, 20, 22, 17, 20, 40, 144, 22, 22, 29, 27, 14, 182, 16, 25, 13, 16, 19, 21, 14, 24, 27, 13, 16, 23, 35, 14, 27, 23, 16, 20, 16, 13, 16, 14, 27, 251, 184, 12, 16, 23, 9, 27, 20, 17, 11, 27, 13, 107, 26, 12, 25, 11, 34, 19, 50, 23, 102, 25, 19, 51, 17, 434, 11, 167, 20, 186, 13, 19, 20, 164, 16, 27, 24, 123, 19, 16, 76, 96, 26, 21, 14, 62, 21, 21, 246, 249, 30, 28, 24, 123, 316, 16, 18, 34, 25, 18, 13, 20, 23, 16, 14, 18, 21, 15, 73, 24, 13, 8, 40, 78, 17, 22, 44, 22, 17, 36, 32, 20, 20, 19, 14, 9, 9, 28, 100, 13, 21, 23, 26, 14, 24, 7, 27, 18, 19, 17, 22, 23, 24, 13, 17, 274, 26, 19, 68, 25, 23, 24, 27, 106, 19, 25, 16, 17, 18, 94, 59, 20, 24, 12, 26, 18, 15, 31, 97, 42, 59, 15, 21, 170, 19, 26, 24, 13, 21, 11, 27, 21, 36, 19, 32, 25, 73, 16, 25, 24, 22, 16, 17, 20, 32, 35, 21, 30, 98, 21, 12, 16, 18, 27, 19, 15, 24, 11, 22, 59, 23, 32, 22, 13, 19, 32, 21, 19, 26, 27, 36, 11, 52, 12, 18, 27, 18, 11, 16, 13, 19, 19, 13, 59, 45, 10, 16, 27, 143, 18, 13, 18, 18, 24, 10, 23, 8, 20, 18, 13, 22, 27, 14, 268, 25, 24, 14, 20, 24, 11, 62, 14, 16, 34, 11, 20, 26, 26, 5, 11, 34, 7, 14, 39, 75, 111, 15, 27, 25, 15, 11, 49, 14, 19, 76, 27, 17, 9, 10, 28, 15, 198, 33, 13, 121, 34, 21, 37, 17, 11, 17, 13, 24, 12, 28, 82, 185, 21, 117, 49, 20, 25, 152, 21, 22, 22, 81, 32, 20, 21, 293, 187, 10, 14, 26, 24, 20, 26, 29, 42, 10, 26, 31, 25, 170, 74, 45, 13, 37, 391, 15, 25, 47, 20, 24, 81, 22, 27, 16, 20, 44, 161, 62, 27, 20, 758, 28, 12, 24, 21, 35, 25, 323, 19, 24, 124, 82, 85, 60, 42, 20, 12, 23, 22, 10, 54, 22, 168, 11, 9, 182, 17, 214, 16, 23, 20, 44, 18, 15, 57, 64, 22, 63, 22, 34, 29, 19, 24, 24, 20, 99, 19, 23, 34, 21, 15, 14, 37, 26, 18, 21, 26, 8, 26, 15, 13, 11, 26, 61, 13, 78, 18, 22, 21, 63, 48, 13, 17, 61, 22, 8, 16, 27, 18, 68, 15, 85, 13, 19, 19, 19, 27, 13, 17, 22, 14, 19, 51, 141, 20, 20, 14, 19, 15, 225, 11, 15, 9, 14, 16, 20, 176, 13, 14, 12, 18, 19, 16, 11, 30, 12, 15, 23, 18, 22, 21, 20, 68, 68, 12, 24, 37, 122, 59, 17, 38, 12, 33, 384, 98, 28, 17, 13, 23, 14, 41, 8, 12, 11, 154, 31, 13, 22, 9, 25, 24, 18, 15, 20, 31, 18, 223, 23, 70, 12, 23, 12, 19, 62, 14, 173, 25, 17, 96, 22, 19, 24, 10, 30, 20, 17, 66, 18, 22, 92, 172, 14, 33, 9, 14, 90, 41, 22, 135, 14, 25, 26, 27, 34, 20, 37, 30, 151, 45, 26, 12, 11, 25, 17, 42, 28, 12, 16, 196, 39, 22, 27, 18, 25, 20, 26, 26, 19, 233, 22, 17, 63, 41, 49, 66, 15, 66, 266, 56, 604, 719, 12, 13, 29, 30, 16, 14, 24, 28, 14, 18, 17, 33, 21, 13, 246, 310, 12, 216, 121, 30, 532, 112, 572, 76, 231, 18, 32, 16, 41, 18, 106, 275, 14, 10, 21, 30, 12, 34, 22, 16, 16, 25, 19, 21, 32, 21, 21, 17, 25, 33, 15, 14, 35, 23, 18, 11, 14, 18, 27, 51, 24, 28, 15, 127, 21, 25, 17, 16, 21, 28, 23, 16, 24, 13, 28, 34, 22, 21, 19, 20, 27, 21, 39, 105, 25, 11, 75, 15, 33, 26, 28, 16, 25, 118, 37, 18, 24, 36, 12, 25, 16, 16, 14, 21, 13, 18, 20, 84, 19, 14, 17, 21, 20, 25, 11, 12, 20, 23, 14, 17, 25, 20, 13, 18, 25, 29, 115, 17, 22, 21, 20, 17, 214, 27, 19, 20, 9, 13, 17, 18, 81, 33, 199, 156, 24, 62, 10, 30, 18, 9, 91, 21, 19, 29, 39, 6, 9, 17, 485, 15, 19, 15, 19, 52, 10, 10, 15, 14, 15, 15, 154, 58, 12, 18, 8, 26, 24, 52, 27, 42, 14, 29, 31, 47, 21, 14, 25, 20, 15, 210, 20, 24, 350, 109, 45, 172, 64, 9, 228, 25, 17, 15, 18, 129, 14, 160, 13, 22, 26, 14, 48, 10, 17, 16, 18, 27, 14, 25, 27, 24, 57, 19, 80, 42, 628, 13, 10, 13, 203, 422, 61, 38, 24, 28, 14, 13, 10, 23, 26, 66, 89, 9, 26, 15, 41, 12, 27, 10, 26, 12, 21, 16, 29, 19, 15, 18, 24, 23, 14, 26, 31, 98, 143, 138, 9, 34, 10, 55, 9, 11, 27, 25, 19, 63, 15, 96, 29, 34, 9, 47, 18, 96, 14, 20, 23, 22, 33, 15, 51, 32, 34, 17, 19, 21, 21, 61, 12, 44, 14, 49, 21, 37, 30, 11, 21, 29, 330, 46, 24, 26, 31, 11, 19, 23, 20, 32, 36, 14, 14, 26, 374, 21, 26, 16, 332, 10, 62, 19, 14, 19, 17, 17, 79, 401, 16, 107, 19, 12, 21, 32, 52, 54, 43, 77, 37, 9, 17, 55, 146, 20, 20, 13, 20, 16, 27, 59, 25, 22, 14, 9, 21, 76, 15, 11, 20, 26, 33, 47, 14, 14, 20, 17, 18, 23, 20, 75, 7, 17, 11, 13, 28, 26, 21, 20, 12, 14, 12, 17, 25, 14, 16, 21, 10, 15, 20, 66, 96, 22, 27]
================================================
FILE: long-tailed_recognition/BalancedSoftmax/config/CIFAR100_LT/balanced_softmax_imba100.yaml
================================================
coslr: false
coslrwarmup: true
criterions:
PerformanceLoss:
def_file: ./loss/BalancedSoftmaxLoss.py
loss_params: {freq_path: ./cls_freq/CIFAR-100-LT_IMBA100.json}
optim_params: null
weight: 1.0
endlr: 0.0
warmup_iterations: 800
base_lr: 0.05
warmup_lr: 0.1
last: false
memory: {centroids: false, init_centroids: false}
model_dir: null
networks:
classifier:
def_file: ./models/DotProductClassifier.py
optim_params: {lr: 0.05, momentum: 0.9, weight_decay: 0.0005}
params: {feat_dim: 64, num_classes: 100}
feat_model:
def_file: ./models/ResNet32Feature.py
fix: false
optim_params: {lr: 0.05, momentum: 0.9, weight_decay: 0.0005}
params: {}
shuffle: false
training_opt:
backbone: resnet32
batch_size: 512
dataset: CIFAR100_LT
display_step: 10
feature_dim: 64
log_dir: ./logs/CIFAR100_LT/models/resnet32_balanced_softmax_imba100
num_classes: 100
num_iterations: 13000
num_workers: 4
open_threshold: 0.1
sampler: null
scheduler_params: {gamma: 0.1, step_size: 3}
sub_dir: models
cifar_imb_ratio: 100
================================================
FILE: long-tailed_recognition/BalancedSoftmax/config/CIFAR100_LT/balanced_softmax_imba200.yaml
================================================
coslr: false
coslrwarmup: true
criterions:
PerformanceLoss:
def_file: ./loss/BalancedSoftmaxLoss.py
loss_params: {freq_path: ./cls_freq/CIFAR-100-LT_IMBA200.json}
optim_params: null
weight: 1.0
endlr: 0.0
warmup_iterations: 800
base_lr: 0.05
warmup_lr: 0.1
last: false
memory: {centroids: false, init_centroids: false}
model_dir: null
networks:
classifier:
def_file: ./models/DotProductClassifier.py
optim_params: {lr: 0.05, momentum: 0.9, weight_decay: 0.0005}
params: {feat_dim: 64, num_classes: 100}
feat_model:
def_file: ./models/ResNet32Feature.py
fix: false
optim_params: {lr: 0.05, momentum: 0.9, weight_decay: 0.0005}
params: {}
shuffle: false
training_opt:
backbone: resnet32
batch_size: 512
dataset: CIFAR100_LT
display_step: 10
feature_dim: 64
log_dir: ./logs/CIFAR100_LT/models/resnet32_balanced_softmax_imba200
num_classes: 100
num_iterations: 13000
num_workers: 4
open_threshold: 0.1
sampler: null
scheduler_params: {gamma: 0.1, step_size: 3}
sub_dir: models
cifar_imb_ratio: 200
================================================
FILE: long-tailed_recognition/BalancedSoftmax/config/CIFAR100_LT/balms_imba100.yaml
================================================
coslr: true
criterions:
PerformanceLoss:
def_file: ./loss/BalancedSoftmaxLoss.py
loss_params: {freq_path: ./cls_freq/CIFAR-100-LT_IMBA100.json}
optim_params: null
weight: 1.0
endlr: 0.0
last: false
memory: {centroids: false, init_centroids: false}
model_dir: ./logs/CIFAR100_LT/models/resnet32_softmax_imba100
networks:
classifier:
def_file: ./models/DotProductClassifier.py
optim_params: {lr: 0.1, momentum: 0.9, weight_decay: 0.0005}
params: {feat_dim: 64, num_classes: 100}
feat_model:
def_file: ./models/ResNet32Feature.py
fix: true
optim_params: {lr: 0.1, momentum: 0.9, weight_decay: 0.0005}
params: {}
shuffle: false
training_opt:
backbone: resnet32
batch_size: 512
dataset: CIFAR100_LT
display_step: 10
feature_dim: 64
log_dir: ./logs/CIFAR100_LT/clslearn/resnet32_balms_imba100
num_classes: 100
num_iterations: 2000
num_workers: 4
open_threshold: 0.1
sampler: {def_file: ./data/MetaSampler.py, type: MetaSampler}
scheduler_params: {gamma: 0.1, step_size: 3}
sub_dir: models
cifar_imb_ratio: 100
================================================
FILE: long-tailed_recognition/BalancedSoftmax/config/CIFAR100_LT/balms_imba200.yaml
================================================
coslr: true
criterions:
PerformanceLoss:
def_file: ./loss/BalancedSoftmaxLoss.py
loss_params: {freq_path: ./cls_freq/CIFAR-100-LT_IMBA200.json}
optim_params: null
weight: 1.0
endlr: 0.0
last: false
memory: {centroids: false, init_centroids: false}
model_dir: ./logs/CIFAR100_LT/models/resnet32_softmax_imba200
networks:
classifier:
def_file: ./models/DotProductClassifier.py
optim_params: {lr: 0.1, momentum: 0.9, weight_decay: 0.0005}
params: {feat_dim: 64, num_classes: 100}
feat_model:
def_file: ./models/ResNet32Feature.py
fix: true
optim_params: {lr: 0.1, momentum: 0.9, weight_decay: 0.0005}
params: {}
shuffle: false
training_opt:
backbone: resnet32
batch_size: 512
dataset: CIFAR100_LT
display_step: 10
feature_dim: 64
log_dir: ./logs/CIFAR100_LT/clslearn/resnet32_balms_imba200
num_classes: 100
num_iterations: 2000
num_workers: 4
open_threshold: 0.1
sampler: {def_file: ./data/MetaSampler.py, type: MetaSampler}
scheduler_params: {gamma: 0.1, step_size: 3}
sub_dir: models
cifar_imb_ratio: 200
================================================
FILE: long-tailed_recognition/BalancedSoftmax/config/CIFAR100_LT/decouple_balanced_softmax_imba100.yaml
================================================
coslr: true
criterions:
PerformanceLoss:
def_file: ./loss/BalancedSoftmaxLoss.py
loss_params: {freq_path: ./cls_freq/CIFAR-100-LT_IMBA100.json}
optim_params: null
weight: 1.0
endlr: 0.0
last: false
memory: {centroids: false, init_centroids: false}
model_dir: ./logs/CIFAR100_LT/models/resnet32_softmax_imba100
networks:
classifier:
def_file: ./models/DotProductClassifier.py
optim_params: {lr: 0.1, momentum: 0.9, weight_decay: 0.0005}
params: {feat_dim: 64, num_classes: 100}
feat_model:
def_file: ./models/ResNet32Feature.py
fix: true
optim_params: {lr: 0.1, momentum: 0.9, weight_decay: 0.0005}
params: {}
shuffle: false
training_opt:
backbone: resnet32
batch_size: 512
dataset: CIFAR100_LT
display_step: 10
feature_dim: 64
log_dir: ./logs/CIFAR100_LT/clslearn/resnet32_decouple_balanced_softmax_imba100
num_classes: 100
num_iterations: 2000
num_workers: 4
open_threshold: 0.1
sampler: null
scheduler_params: {gamma: 0.1, step_size: 3}
sub_dir: models
cifar_imb_ratio: 100
================================================
FILE: long-tailed_recognition/BalancedSoftmax/config/CIFAR100_LT/decouple_balanced_softmax_imba200.yaml
================================================
coslr: true
criterions:
PerformanceLoss:
def_file: ./loss/BalancedSoftmaxLoss.py
loss_params: {freq_path: ./cls_freq/CIFAR-100-LT_IMBA200.json}
optim_params: null
weight: 1.0
endlr: 0.0
last: false
memory: {centroids: false, init_centroids: false}
model_dir: ./logs/CIFAR100_LT/models/resnet32_softmax_imba200
networks:
classifier:
def_file: ./models/DotProductClassifier.py
optim_params: {lr: 0.1, momentum: 0.9, weight_decay: 0.0005}
params: {feat_dim: 64, num_classes: 100}
feat_model:
def_file: ./models/ResNet32Feature.py
fix: true
optim_params: {lr: 0.1, momentum: 0.9, weight_decay: 0.0005}
params: {}
shuffle: false
training_opt:
backbone: resnet32
batch_size: 512
dataset: CIFAR100_LT
display_step: 10
feature_dim: 64
log_dir: ./logs/CIFAR100_LT/clslearn/resnet32_decouple_balanced_softmax_imba200
num_classes: 100
num_iterations: 2000
num_workers: 4
open_threshold: 0.1
sampler: null
scheduler_params: {gamma: 0.1, step_size: 3}
sub_dir: models
cifar_imb_ratio: 200
================================================
FILE: long-tailed_recognition/BalancedSoftmax/config/CIFAR100_LT/softmax_imba.yaml
================================================
coslr: false
coslrwarmup: true
criterions:
PerformanceLoss:
def_file: ./loss/SoftmaxLoss.py
loss_params: {}
optim_params: null
weight: 1.0
endlr: 0.0
warmup_iterations: 800
base_lr: 0.05
warmup_lr: 0.1
last: false
memory: {centroids: false, init_centroids: false}
model_dir: null
networks:
classifier:
def_file: ./models/DotProductClassifier.py
optim_params: {lr: 0.05, momentum: 0.9, weight_decay: 0.0005}
params: {feat_dim: 64, num_classes: 100}
feat_model:
def_file: ./models/ResNet32Feature.py
fix: false
optim_params: {lr: 0.05, momentum: 0.9, weight_decay: 0.0005}
params: {}
shuffle: false
training_opt:
backbone: resnet32
batch_size: 512
dataset: CIFAR100_LT
display_step: 10
feature_dim: 64
log_dir: ./logs/CIFAR100_LT/models/resnet32_softmax_imba100
num_classes: 100
num_iterations: 13000
num_workers: 4
open_threshold: 0.1
sampler: null
scheduler_params: {gamma: 0.1, step_size: 3}
sub_dir: models
cifar_imb_ratio: 1
================================================
FILE: long-tailed_recognition/BalancedSoftmax/config/CIFAR100_LT/softmax_imba100.yaml
================================================
coslr: false
coslrwarmup: true
criterions:
PerformanceLoss:
def_file: ./loss/SoftmaxLoss.py
loss_params: {}
optim_params: null
weight: 1.0
endlr: 0.0
warmup_iterations: 800
base_lr: 0.05
warmup_lr: 0.1
last: false
memory: {centroids: false, init_centroids: false}
model_dir: null
networks:
classifier:
def_file: ./models/DotProductClassifier.py
optim_params: {lr: 0.05, momentum: 0.9, weight_decay: 0.0005}
params: {feat_dim: 64, num_classes: 100}
feat_model:
def_file: ./models/ResNet32Feature.py
fix: false
optim_params: {lr: 0.05, momentum: 0.9, weight_decay: 0.0005}
params: {}
shuffle: false
training_opt:
backbone: resnet32
batch_size: 512
dataset: CIFAR100_LT
display_step: 10
feature_dim: 64
log_dir: ./logs/CIFAR100_LT/models/resnet32_softmax_imba100
num_classes: 100
num_iterations: 13000
num_workers: 4
open_threshold: 0.1
sampler: null
scheduler_params: {gamma: 0.1, step_size: 3}
sub_dir: models
cifar_imb_ratio: 100
================================================
FILE: long-tailed_recognition/BalancedSoftmax/config/CIFAR100_LT/softmax_imba200.yaml
================================================
coslr: false
coslrwarmup: true
criterions:
PerformanceLoss:
def_file: ./loss/SoftmaxLoss.py
loss_params: {}
optim_params: null
weight: 1.0
endlr: 0.0
warmup_iterations: 800
base_lr: 0.05
warmup_lr: 0.1
last: false
memory: {centroids: false, init_centroids: false}
model_dir: null
networks:
classifier:
def_file: ./models/DotProductClassifier.py
optim_params: {lr: 0.05, momentum: 0.9, weight_decay: 0.0005}
params: {feat_dim: 64, num_classes: 100}
feat_model:
def_file: ./models/ResNet32Feature.py
fix: false
optim_params: {lr: 0.05, momentum: 0.9, weight_decay: 0.0005}
params: {}
shuffle: false
training_opt:
backbone: resnet32
batch_size: 512
dataset: CIFAR100_LT
display_step: 10
feature_dim: 64
log_dir: ./logs/CIFAR100_LT/models/resnet32_softmax_imba200
num_classes: 100
num_iterations: 13000
num_workers: 4
open_threshold: 0.1
sampler: null
scheduler_params: {gamma: 0.1, step_size: 3}
sub_dir: models
cifar_imb_ratio: 200
================================================
FILE: long-tailed_recognition/BalancedSoftmax/config/CIFAR10_LT/balanced_softmax_imba200.yaml
================================================
coslr: false
coslrwarmup: true
criterions:
PerformanceLoss:
def_file: ./loss/BalancedSoftmaxLoss.py
loss_params: {freq_path: ./cls_freq/CIFAR-10-LT_IMBA200.json}
optim_params: null
weight: 1.0
endlr: 0.0
warmup_iterations: 800
base_lr: 0.05
warmup_lr: 0.1
last: false
memory: {centroids: false, init_centroids: false}
model_dir: null
networks:
classifier:
def_file: ./models/DotProductClassifier.py
optim_params: {lr: 0.05, momentum: 0.9, weight_decay: 0.0005}
params: {feat_dim: 64, num_classes: 10}
feat_model:
def_file: ./models/ResNet32Feature.py
fix: false
optim_params: {lr: 0.05, momentum: 0.9, weight_decay: 0.0005}
params: {}
shuffle: false
training_opt:
backbone: resnet32
batch_size: 512
dataset: CIFAR10_LT
display_step: 10
feature_dim: 64
log_dir: ./logs/CIFAR10_LT/models/resnet32_balanced_softmax_imba200
num_classes: 10
num_iterations: 13000
num_workers: 4
open_threshold: 0.1
sampler: null
scheduler_params: {gamma: 0.1, step_size: 3}
sub_dir: models
cifar_imb_ratio: 200
================================================
FILE: long-tailed_recognition/BalancedSoftmax/config/CIFAR10_LT/balms_imba200.yaml
================================================
coslr: true
criterions:
PerformanceLoss:
def_file: ./loss/BalancedSoftmaxLoss.py
loss_params: {freq_path: ./cls_freq/CIFAR-10-LT_IMBA200.json}
optim_params: null
weight: 1.0
endlr: 0.0
last: false
memory: {centroids: false, init_centroids: false}
model_dir: ./logs/CIFAR10_LT/models/resnet32_softmax_imba200
networks:
classifier:
def_file: ./models/DotProductClassifier.py
optim_params: {lr: 0.1, momentum: 0.9, weight_decay: 0.0005}
params: {feat_dim: 64, num_classes: 10}
feat_model:
def_file: ./models/ResNet32Feature.py
fix: true
optim_params: {lr: 0.1, momentum: 0.9, weight_decay: 0.0005}
params: {}
shuffle: false
training_opt:
backbone: resnet32
batch_size: 512
dataset: CIFAR10_LT
display_step: 10
feature_dim: 64
log_dir: ./logs/CIFAR10_LT/clslearn/resnet32_balms_imba200
num_classes: 10
num_iterations: 2000
num_workers: 4
open_threshold: 0.1
sampler: {def_file: ./data/MetaSampler.py, type: MetaSampler}
scheduler_params: {gamma: 0.1, step_size: 3}
sub_dir: models
cifar_imb_ratio: 200
================================================
FILE: long-tailed_recognition/BalancedSoftmax/config/CIFAR10_LT/decouple_balanced_softmax_imba200.yaml
================================================
coslr: true
criterions:
PerformanceLoss:
def_file: ./loss/BalancedSoftmaxLoss.py
loss_params: {freq_path: ./cls_freq/CIFAR-10-LT_IMBA200.json}
optim_params: null
weight: 1.0
endlr: 0.0
last: false
memory: {centroids: false, init_centroids: false}
model_dir: ./logs/CIFAR10_LT/models/resnet32_softmax_imba200
networks:
classifier:
def_file: ./models/DotProductClassifier.py
optim_params: {lr: 0.1, momentum: 0.9, weight_decay: 0.0005}
params: {feat_dim: 64, num_classes: 10}
feat_model:
def_file: ./models/ResNet32Feature.py
fix: true
optim_params: {lr: 0.1, momentum: 0.9, weight_decay: 0.0005}
params: {}
shuffle: false
training_opt:
backbone: resnet32
batch_size: 512
dataset: CIFAR10_LT
display_step: 10
feature_dim: 64
log_dir: ./logs/CIFAR10_LT/clslearn/resnet32_decouple_balanced_softmax_imba200
num_classes: 10
num_iterations: 2000
num_workers: 4
open_threshold: 0.1
sampler: null
scheduler_params: {gamma: 0.1, step_size: 3}
sub_dir: models
cifar_imb_ratio: 200
================================================
FILE: long-tailed_recognition/BalancedSoftmax/config/CIFAR10_LT/softmax_imba200.yaml
================================================
coslr: false
coslrwarmup: true
criterions:
PerformanceLoss:
def_file: ./loss/SoftmaxLoss.py
loss_params: {}
optim_params: null
weight: 1.0
endlr: 0.0
warmup_iterations: 800
base_lr: 0.05
warmup_lr: 0.1
last: false
memory: {centroids: false, init_centroids: false}
model_dir: null
networks:
classifier:
def_file: ./models/DotProductClassifier.py
optim_params: {lr: 0.05, momentum: 0.9, weight_decay: 0.0005}
params: {feat_dim: 64, num_classes: 10}
feat_model:
def_file: ./models/ResNet32Feature.py
fix: false
optim_params: {lr: 0.05, momentum: 0.9, weight_decay: 0.0005}
params: {}
shuffle: false
training_opt:
backbone: resnet32
batch_size: 512
dataset: CIFAR10_LT
display_step: 10
feature_dim: 64
log_dir: ./logs/CIFAR10_LT/models/resnet32_softmax_imba200
num_classes: 10
num_iterations: 13000
num_workers: 4
open_threshold: 0.1
sampler: null
scheduler_params: {gamma: 0.1, step_size: 3}
sub_dir: models
cifar_imb_ratio: 200
================================================
FILE: long-tailed_recognition/BalancedSoftmax/config/ImageNet_LT/balanced_softmax_resnet10.yaml
================================================
coslr: true
criterions:
PerformanceLoss:
def_file: ./loss/BalancedSoftmaxLoss.py
loss_params: {freq_path: ./cls_freq/ImageNet_LT.json}
optim_params: null
weight: 1.0
endlr: 0.0
last: false
memory: {centroids: false, init_centroids: false}
model_dir: null
networks:
classifier:
def_file: ./models/DotProductClassifier.py
optim_params: {lr: 0.05, momentum: 0.9, weight_decay: 0.0005}
params: {dataset: ImageNet_LT, feat_dim: 512, log_dir: ./logs/ImageNet_LT/models/balanced_softmax_resnet10,
num_classes: 1000, stage1_weights: false}
feat_model:
def_file: ./models/ResNet10Feature.py
fix: false
optim_params: {lr: 0.05, momentum: 0.9, weight_decay: 0.0005}
params: {dataset: ImageNet_LT, dropout: null, log_dir: ./logs/ImageNet_LT/models/balanced_softmax_resnet10,
stage1_weights: false, use_fc: false, use_selfatt: false}
shuffle: false
training_opt:
backbone: resnet10
batch_size: 128
dataset: ImageNet_LT
display_step: 10
feature_dim: 512
log_dir: ./logs/ImageNet_LT/models/balanced_softmax_resnet10
log_root: /logs/ImageNet_LT
num_classes: 1000
num_epochs: 90
num_workers: 12
open_threshold: 0.1
sampler: null
scheduler_params: {gamma: 0.1, step_size: 30}
stage: balanced_softmax_resnet10
sub_dir: models
================================================
FILE: long-tailed_recognition/BalancedSoftmax/config/ImageNet_LT/balanced_softmax_resnet50.yaml
================================================
coslr: true
criterions:
PerformanceLoss:
def_file: ./loss/BalancedSoftmaxLoss.py
loss_params: {freq_path: ./cls_freq/ImageNet_LT.json}
optim_params: null
weight: 1.0
endlr: 0.0
last: false
memory: {centroids: false, init_centroids: false}
model_dir: null
networks:
classifier:
def_file: ./models/DotProductClassifier.py
optim_params: {lr: 0.05, momentum: 0.9, weight_decay: 0.0005}
params: {dataset: ImageNet_LT, feat_dim: 2048, log_dir: ./logs/ImageNet_LT/models/balanced_softmax_resnet50,
num_classes: 1000, stage1_weights: false}
feat_model:
def_file: ./models/ResNet50Feature.py
fix: false
optim_params: {lr: 0.05, momentum: 0.9, weight_decay: 0.0005}
params: {dataset: ImageNet_LT, dropout: null, log_dir: ./logs/ImageNet_LT/models/balanced_softmax_resnet50,
stage1_weights: false, use_fc: false, use_selfatt: false}
shuffle: false
training_opt:
backbone: resnet50
batch_size: 128
dataset: ImageNet_LT
display_step: 100
feature_dim: 2048
log_dir: ./logs/ImageNet_LT/models/balanced_softmax_resnet50
log_root: /logs/ImageNet_LT
num_classes: 1000
num_epochs: 90
num_workers: 12
open_threshold: 0.1
sampler: null
scheduler_params: {gamma: 0.1, step_size: 30}
stage: balanced_softmax_resnet50
sub_dir: models
================================================
FILE: long-tailed_recognition/BalancedSoftmax/config/ImageNet_LT/balms_resnet10.yaml
================================================
coslr: true
criterions:
PerformanceLoss:
def_file: ./loss/BalancedSoftmaxLoss.py
loss_params: {freq_path: ./cls_freq/ImageNet_LT.json}
optim_params: null
weight: 1.0
endlr: 0.0
last: false
memory: {centroids: false, init_centroids: false}
model_dir: ./logs/ImageNet_LT/models/resnet10_uniform_e90
networks:
classifier:
def_file: ./models/DotProductClassifier.py
optim_params: {lr: 0.125, momentum: 0.9, weight_decay: 0.0005}
params: {dataset: ImageNet_LT, feat_dim: 512, log_dir: ./logs/ImageNet_LT/clslearn/balms_resnet10,
num_classes: 1000, stage1_weights: false}
feat_model:
def_file: ./models/ResNet10Feature.py
fix: true
optim_params: {lr: 0.05, momentum: 0.9, weight_decay: 0.0005}
params: {dataset: ImageNet_LT, dropout: null, log_dir: ./logs/ImageNet_LT/clslearn/balms_resnet10,
stage1_weights: false, use_fc: false, use_selfatt: false}
shuffle: false
training_opt:
backbone: resnet10
batch_size: 128
dataset: ImageNet_LT
display_step: 10
feature_dim: 512
log_dir: ./logs/ImageNet_LT/clslearn/balms_resnet10
log_root: ./logs/ImageNet_LT
num_classes: 1000
num_epochs: 20
num_workers: 12
open_threshold: 0.1
sampler: {def_file: ./data/MetaSampler.py, type: MetaSampler, meta_batch_size: 1024, init_pow: 0.1, lr: 0.001}
scheduler_params: {gamma: 0.1, step_size: 3}
stage: balms_resnet10
sub_dir: clslearn
================================================
FILE: long-tailed_recognition/BalancedSoftmax/config/ImageNet_LT/balms_resnet50.yaml
================================================
coslr: true
criterions:
PerformanceLoss:
def_file: ./loss/BalancedSoftmaxLoss.py
loss_params: {freq_path: ./cls_freq/ImageNet_LT.json}
optim_params: null
weight: 1.0
endlr: 0.0
last: false
memory: {centroids: false, init_centroids: false}
model_dir: ./logs/ImageNet_LT/models/resnet10_uniform_e90
networks:
classifier:
def_file: ./models/DotProductClassifier.py
optim_params: {lr: 0.5, momentum: 0.9, weight_decay: 0.0005}
params: {dataset: ImageNet_LT, feat_dim: 512, log_dir: ./logs/ImageNet_LT/clslearn/balms_resnet10,
num_classes: 1000, stage1_weights: false}
feat_model:
def_file: ./models/ResNet10Feature.py
fix: true
optim_params: {lr: 0.2, momentum: 0.9, weight_decay: 0.0005}
params: {dataset: ImageNet_LT, dropout: null, log_dir: ./logs/ImageNet_LT/clslearn/balms_resnet10,
stage1_weights: false, use_fc: false, use_selfatt: false}
shuffle: false
training_opt:
backbone: resnet10
batch_size: 512
dataset: ImageNet_LT
display_step: 10
feature_dim: 512
log_dir: ./logs/ImageNet_LT/clslearn/balms_resnet10
log_root: ./logs/ImageNet_LT
num_classes: 1000
num_epochs: 20
num_workers: 12
open_threshold: 0.1
sampler: {def_file: ./data/MetaSampler.py, type: MetaSampler, meta_batch_size: 1024, init_pow: 0.1, lr: 0.001}
scheduler_params: {gamma: 0.1, step_size: 3}
stage: balms_resnet10
sub_dir: clslearn
================================================
FILE: long-tailed_recognition/BalancedSoftmax/config/ImageNet_LT/cls_crt.yaml
================================================
coslr: true
criterions:
PerformanceLoss:
def_file: ./loss/SoftmaxLoss.py
loss_params: {}
optim_params: null
weight: 1.0
endlr: 0.0
last: false
memory: {centroids: false, init_centroids: false}
model_dir: ./logs/ImageNet_LT/models/resnext50_uniform_e90
networks:
classifier:
def_file: ./models/DotProductClassifier.py
optim_params: {lr: 0.2, momentum: 0.9, weight_decay: 0.0005}
params: {dataset: ImageNet_LT, feat_dim: 2048, log_dir: ./logs/ImageNet_LT/clslearn/retrain_uni2ban,
num_classes: 1000, stage1_weights: false}
feat_model:
def_file: ./models/ResNext50Feature.py
fix: true
optim_params: {lr: 0.2, momentum: 0.9, weight_decay: 0.0005}
params: {dataset: ImageNet_LT, dropout: null, log_dir: ./logs/ImageNet_LT/clslearn/retrain_uni2ban,
stage1_weights: false, use_fc: false, use_selfatt: false}
shuffle: true
training_opt:
backbone: resnext50
batch_size: 512
dataset: ImageNet_LT
display_step: 10
feature_dim: 2048
log_dir: ./logs/ImageNet_LT/clslearn/retrain_uni2ban
log_root: ./logs/ImageNet_LT
num_classes: 1000
num_epochs: 10
num_workers: 12
open_threshold: 0.1
sampler: {def_file: ./data/ClassAwareSampler.py, num_samples_cls: 4, type: ClassAwareSampler}
scheduler_params: {gamma: 0.1, step_size: 3}
stage: retrain_uni2ban
sub_dir: clslearn
================================================
FILE: long-tailed_recognition/BalancedSoftmax/config/ImageNet_LT/cls_lws.yaml
================================================
coslr: true
criterions:
PerformanceLoss:
def_file: ./loss/SoftmaxLoss.py
loss_params: {}
optim_params: null
weight: 1.0
endlr: 0.0
last: false
memory: {centroids: false, init_centroids: false}
model_dir: ./logs/ImageNet_LT/models/resnext50_uniform_e90
networks:
classifier:
def_file: ./models/TauNormClassifier.py
optim_params: {lr: 0.2, momentum: 0.9, weight_decay: 0.0005}
params: {dataset: ImageNet_LT, feat_dim: 2048, log_dir: ./logs/ImageNet_LT/clslearn/lws_uni2ban,
num_classes: 1000, stage1_weights: false}
feat_model:
def_file: ./models/ResNext50Feature.py
fix: true
optim_params: {lr: 0.2, momentum: 0.9, weight_decay: 0.0005}
params: {dataset: ImageNet_LT, dropout: null, log_dir: ./logs/ImageNet_LT/clslearn/lws_uni2ban,
stage1_weights: false, use_fc: false, use_selfatt: false}
shuffle: true
training_opt:
backbone: resnext50
batch_size: 512
dataset: ImageNet_LT
display_step: 10
feature_dim: 2048
log_dir: ./logs/ImageNet_LT/clslearn/lws_uni2ban
log_root: ./logs/ImageNet_LT
num_classes: 1000
num_epochs: 5
num_workers: 12
open_threshold: 0.1
sampler: {def_file: ./data/ClassAwareSampler.py, num_samples_cls: 4, type: ClassAwareSampler}
scheduler_params: {gamma: 0.1, step_size: 1}
stage: lws_uni2ban
sub_dir: clslearn
================================================
FILE: long-tailed_recognition/BalancedSoftmax/config/ImageNet_LT/decouple_balanced_softmax_resnet10.yaml
================================================
coslr: true
criterions:
PerformanceLoss:
def_file: ./loss/BalancedSoftmaxLoss.py
loss_params: {freq_path: ./cls_freq/ImageNet_LT.json}
optim_params: null
weight: 1.0
endlr: 0.0
last: false
memory: {centroids: false, init_centroids: false}
model_dir: ./logs/ImageNet_LT/models/resnet10_uniform_e90
networks:
classifier:
def_file: ./models/DotProductClassifier.py
optim_params: {lr: 0.4, momentum: 0.9, weight_decay: 0.0005}
params: {dataset: ImageNet_LT, feat_dim: 512, log_dir: ./logs/ImageNet_LT/clslearn/decouple_balanced_softmax_resnet10,
num_classes: 1000, stage1_weights: false}
feat_model:
def_file: ./models/ResNet10Feature.py
fix: true
optim_params: {lr: 0.2, momentum: 0.9, weight_decay: 0.0005}
params: {dataset: ImageNet_LT, dropout: null, log_dir: ./logs/ImageNet_LT/clslearn/decouple_balanced_softmax_resnet10,
stage1_weights: false, use_fc: false, use_selfatt: false}
shuffle: true
training_opt:
backbone: resnet10
batch_size: 512
dataset: ImageNet_LT
display_step: 10
feature_dim: 512
log_dir: ./logs/ImageNet_LT/clslearn/decouple_balanced_softmax_resnet10
log_root: ./logs/ImageNet_LT
num_classes: 1000
num_epochs: 10
num_workers: 12
open_threshold: 0.1
sampler: null
scheduler_params: {gamma: 0.1, step_size: 3}
stage: decouple_balanced_softmax_resnet10
sub_dir: clslearn
================================================
FILE: long-tailed_recognition/BalancedSoftmax/config/ImageNet_LT/feat_balance.yaml
================================================
coslr: true
criterions:
PerformanceLoss:
def_file: ./loss/SoftmaxLoss.py
loss_params: {}
optim_params: null
weight: 1.0
endlr: 0.0
last: false
memory: {centroids: false, init_centroids: false}
model_dir: null
networks:
classifier:
def_file: ./models/DotProductClassifier.py
optim_params: {lr: 0.2, momentum: 0.9, weight_decay: 0.0005}
params: {dataset: ImageNet_LT, feat_dim: 2048, log_dir: ./logs/ImageNet_LT/models/resnext50_banlance_e90,
num_classes: 1000, stage1_weights: false}
feat_model:
def_file: ./models/ResNext50Feature.py
fix: false
optim_params: {lr: 0.2, momentum: 0.9, weight_decay: 0.0005}
params: {dataset: ImageNet_LT, dropout: null, log_dir: ./logs/ImageNet_LT/models/resnext50_banlance_e90,
stage1_weights: false, use_fc: false, use_selfatt: false}
shuffle: true
training_opt:
backbone: resnext50
batch_size: 128
dataset: ImageNet_LT
display_step: 10
feature_dim: 2048
log_dir: ./logs/ImageNet_LT/models/resnext50_banlance_e90
log_root: ./logs/ImageNet_LT
num_classes: 1000
num_epochs: 90
num_workers: 12
open_threshold: 0.1
sampler: {def_file: ./data/ClassAwareSampler.py, num_samples_cls: 4, type: ClassAwareSampler}
scheduler_params: {gamma: 0.1, step_size: 30}
stage: resnext50_banlance_e90
sub_dir: models
================================================
FILE: long-tailed_recognition/BalancedSoftmax/config/ImageNet_LT/feat_shift.yaml
================================================
coslr: true
criterions:
PerformanceLoss:
def_file: ./loss/SoftmaxLoss.py
loss_params: {}
optim_params: null
weight: 1.0
endlr: 0.0
last: false
memory: {centroids: false, init_centroids: false}
model_dir: null
networks:
classifier:
def_file: ./models/DotProductClassifier.py
optim_params: {lr: 0.2, momentum: 0.9, weight_decay: 0.0005}
params: {dataset: ImageNet_LT, feat_dim: 2048, log_dir: ./logs/ImageNet_LT/models/resnext50_shift_e90,
num_classes: 1000, stage1_weights: false}
feat_model:
def_file: ./models/ResNext50Feature.py
fix: false
optim_params: {lr: 0.2, momentum: 0.9, weight_decay: 0.0005}
params: {dataset: ImageNet_LT, dropout: null, log_dir: ./logs/ImageNet_LT/models/resnext50_shift_e90,
stage1_weights: false, use_fc: false, use_selfatt: false}
shuffle: false
training_opt:
backbone: resnext50
batch_size: 512
dataset: ImageNet_LT
display_step: 10
feature_dim: 2048
log_dir: ./logs/ImageNet_LT/models/resnext50_shift_e90
log_root: ./logs/ImageNet_LT
num_classes: 1000
num_epochs: 90
num_workers: 12
open_threshold: 0.1
sampler: {alpha: 1.0, cycle: 0, decay_gap: 30, def_file: ./data/ClassPrioritySampler.py,
epochs: 90, fixed_scale: 1, lam: null, manual_only: true, momentum: 0.0, nroot: null,
pri_mode: train, ptype: score, rescale: false, root_decay: null, type: ClassPrioritySampler}
scheduler_params: {gamma: 0.1, step_size: 30}
stage: resnext50_shift_e90
sub_dir: models
================================================
FILE: long-tailed_recognition/BalancedSoftmax/config/ImageNet_LT/feat_squareroot.yaml
================================================
coslr: true
criterions:
PerformanceLoss:
def_file: ./loss/SoftmaxLoss.py
loss_params: {}
optim_params: null
weight: 1.0
endlr: 0.0
last: false
memory: {centroids: false, init_centroids: false}
model_dir: null
networks:
classifier:
def_file: ./models/DotProductClassifier.py
optim_params: {lr: 0.2, momentum: 0.9, weight_decay: 0.0005}
params: {dataset: ImageNet_LT, feat_dim: 2048, log_dir: ./logs/ImageNet_LT/models/resnext50_square_e90,
num_classes: 1000, stage1_weights: false}
feat_model:
def_file: ./models/ResNext50Feature.py
fix: false
optim_params: {lr: 0.2, momentum: 0.9, weight_decay: 0.0005}
params: {dataset: ImageNet_LT, dropout: null, log_dir: ./logs/ImageNet_LT/models/resnext50_square_e90,
stage1_weights: false, use_fc: false, use_selfatt: false}
shuffle: false
training_opt:
backbone: resnext50
batch_size: 512
dataset: ImageNet_LT
display_step: 10
feature_dim: 2048
log_dir: ./logs/ImageNet_LT/models/resnext50_square_e90
log_root: ./logs/ImageNet_LT
num_classes: 1000
num_epochs: 90
num_workers: 12
open_threshold: 0.1
sampler: {alpha: 1.0, cycle: 0, decay_gap: 30, def_file: ./data/MixedPrioritizedSampler.py,
epochs: 90, fixed_scale: 1, lam: 1.0, manual_only: true, nroot: 2.0, ptype: score,
rescale: false, root_decay: null, type: MixedPrioritizedSampler}
scheduler_params: {gamma: 0.1, step_size: 30}
stage: resnext50_square_e90
sub_dir: models
================================================
FILE: long-tailed_recognition/BalancedSoftmax/config/ImageNet_LT/feat_uniform.yaml
================================================
coslr: true
criterions:
PerformanceLoss:
def_file: ./loss/SoftmaxLoss.py
loss_params: {}
optim_params: null
weight: 1.0
endlr: 0.0
last: false
memory: {centroids: false, init_centroids: false}
model_dir: null
networks:
classifier:
def_file: ./models/DotProductClassifier.py
optim_params: {lr: 0.2, momentum: 0.9, weight_decay: 0.0005}
params: {dataset: ImageNet_LT, feat_dim: 2048, log_dir: ./logs/ImageNet_LT/models/resnext50_uniform_e90,
num_classes: 1000, stage1_weights: false}
feat_model:
def_file: ./models/ResNext50Feature.py
fix: false
optim_params: {lr: 0.2, momentum: 0.9, weight_decay: 0.0005}
params: {dataset: ImageNet_LT, dropout: null, log_dir: ./logs/ImageNet_LT/models/resnext50_uniform_e90,
stage1_weights: false, use_fc: false, use_selfatt: false}
shuffle: false
training_opt:
backbone: resnext50
batch_size: 512
dataset: ImageNet_LT
display_step: 10
feature_dim: 2048
log_dir: ./logs/ImageNet_LT/models/resnext50_uniform_e90
log_root: /logs/ImageNet_LT
num_classes: 1000
num_epochs: 90
num_workers: 12
open_threshold: 0.1
sampler: null
scheduler_params: {gamma: 0.1, step_size: 30}
stage: resnext50_uniform_e90
sub_dir: models
================================================
FILE: long-tailed_recognition/BalancedSoftmax/config/ImageNet_LT/feat_uniform_resnet10.yaml
================================================
coslr: true
criterions:
PerformanceLoss:
def_file: ./loss/SoftmaxLoss.py
loss_params: {}
optim_params: null
weight: 1.0
endlr: 0.0
last: false
memory: {centroids: false, init_centroids: false}
model_dir: null
networks:
classifier:
def_file: ./models/DotProductClassifier.py
optim_params: {lr: 0.2, momentum: 0.9, weight_decay: 0.0005}
params: {dataset: ImageNet_LT, feat_dim: 512, log_dir: ./logs/ImageNet_LT/models/resnet10_uniform_e90,
num_classes: 1000, stage1_weights: false}
feat_model:
def_file: ./models/ResNet10Feature.py
fix: false
optim_params: {lr: 0.2, momentum: 0.9, weight_decay: 0.0005}
params: {dataset: ImageNet_LT, dropout: null, log_dir: ./logs/ImageNet_LT/models/resnet10_uniform_e90,
stage1_weights: false, use_fc: false, use_selfatt: false}
shuffle: false
training_opt:
backbone: resnet10
batch_size: 512
dataset: ImageNet_LT
display_step: 10
feature_dim: 512
log_dir: ./logs/ImageNet_LT/models/resnet10_uniform_e90
log_root: /logs/ImageNet_LT
num_classes: 1000
num_epochs: 90
num_workers: 12
open_threshold: 0.1
sampler: null
scheduler_params: {gamma: 0.1, step_size: 30}
stage: resnet10_uniform_e90
sub_dir: models
================================================
FILE: long-tailed_recognition/BalancedSoftmax/config/ImageNet_LT/feat_uniform_resnet50.yaml
================================================
coslr: true
criterions:
PerformanceLoss:
def_file: ./loss/SoftmaxLoss.py
loss_params: {}
optim_params: null
weight: 1.0
endlr: 0.0
last: false
memory: {centroids: false, init_centroids: false}
model_dir: null
networks:
classifier:
def_file: ./models/DotProductClassifier.py
optim_params: {lr: 0.05, momentum: 0.9, weight_decay: 0.0005}
params: {dataset: ImageNet_LT, feat_dim: 2048, log_dir: ./logs/ImageNet_LT/models/l05_resnet50_uniform_e90,
num_classes: 1000, stage1_weights: false}
feat_model:
def_file: ./models/ResNet50Feature.py
fix: false
optim_params: {lr: 0.05, momentum: 0.9, weight_decay: 0.0005}
params: {dataset: ImageNet_LT, dropout: null, log_dir: ./logs/ImageNet_LT/models/l05_resnet50_uniform_e90,
stage1_weights: false, use_fc: false, use_selfatt: false}
shuffle: false
training_opt:
backbone: resnet50
batch_size: 128
dataset: ImageNet_LT
display_step: 10
feature_dim: 2048
log_dir: ./logs/ImageNet_LT/models/l05_resnet50_uniform_e90
log_root: /logs/ImageNet_LT
num_classes: 1000
num_epochs: 90
num_workers: 12
open_threshold: 0.1
sampler: null
scheduler_params: {gamma: 0.1, step_size: 30}
stage: l05_resnet50_uniform_e90
sub_dir: models
================================================
FILE: long-tailed_recognition/BalancedSoftmax/config/ImageNet_LT/softmax_resnet10.yaml
================================================
coslr: true
criterions:
PerformanceLoss:
def_file: ./loss/SoftmaxLoss.py
loss_params: {}
optim_params: null
weight: 1.0
endlr: 0.0
last: false
memory: {centroids: false, init_centroids: false}
model_dir: ./logs/ImageNet_LT/models/resnet10_uniform_e90
networks:
classifier:
def_file: ./models/DotProductClassifier.py
optim_params: {lr: 0.2, momentum: 0.9, weight_decay: 0.0005}
params: {dataset: ImageNet_LT, feat_dim: 512, log_dir: ./logs/ImageNet_LT/clslearn/softmax_resnet10_cbs,
num_classes: 1000, stage1_weights: false}
feat_model:
def_file: ./models/ResNet10Feature.py
fix: true
optim_params: {lr: 0.2, momentum: 0.9, weight_decay: 0.0005}
params: {dataset: ImageNet_LT, dropout: null, log_dir: ./logs/ImageNet_LT/clslearn/softmax_resnet10_cbs,
stage1_weights: false, use_fc: false, use_selfatt: false}
shuffle: false
training_opt:
backbone: resnet10
batch_size: 512
dataset: ImageNet_LT
display_step: 10
feature_dim: 512
log_dir: ./logs/ImageNet_LT/clslearn/softmax_resnet10_cbs
log_root: ./logs/ImageNet_LT
num_classes: 1000
num_epochs: 10
num_workers: 12
open_threshold: 0.1
sampler: null
scheduler_params: {gamma: 0.1, step_size: 3}
stage: softmax_resnet10_cbs
sub_dir: clslearn
================================================
FILE: long-tailed_recognition/BalancedSoftmax/config/ImageNet_LT/softmax_resnet50_feat.yaml
================================================
coslr: true
criterions:
PerformanceLoss:
def_file: ./loss/SoftmaxLoss.py
loss_params: {}
optim_params: null
weight: 1.0
endlr: 0.0
last: false
memory: {centroids: false, init_centroids: false}
model_dir: null
networks:
classifier:
def_file: ./models/DotProductClassifier.py
optim_params: {lr: 0.2, momentum: 0.9, weight_decay: 0.0005}
params: {dataset: ImageNet_LT, feat_dim: 512, log_dir: ./logs/ImageNet_LT/clslearn/softmax_resnet50_cbs,
num_classes: 1000, stage1_weights: false}
feat_model:
def_file: ./models/ResNet50Feature.py
fix: false
optim_params: {lr: 0.2, momentum: 0.9, weight_decay: 0.0005}
params: {dataset: ImageNet_LT, dropout: null, log_dir: ./logs/ImageNet_LT/models/softmax_resnet50_cbs,
stage1_weights: false, use_fc: false, use_selfatt: false}
shuffle: false
training_opt:
backbone: resnet10
batch_size: 128
dataset: ImageNet_LT
display_step: 10
feature_dim: 2048
log_dir: ./logs/ImageNet_LT/models/softmax_resnet50_cbs
log_root: ./logs/ImageNet_LT
num_classes: 1000
num_epochs: 90
num_workers: 12
open_threshold: 0.1
sampler: null
scheduler_params: {gamma: 0.1, step_size: 30}
stage: softmax_resnet50_e90
sub_dir: models
================================================
FILE: long-tailed_recognition/BalancedSoftmax/config/Places_LT/balanced_softmax.yaml
================================================
coslr: true
criterions:
PerformanceLoss:
def_file: ./loss/BalancedSoftmaxLoss.py
loss_params: {freq_path: ./cls_freq/Places_LT.json}
optim_params: null
weight: 1.0
endlr: 0.0
last: false
memory: {centroids: false, init_centroids: false}
model_dir: ./logs/Places_LT/models/dir/resnet152_uniform.pth
networks:
classifier:
def_file: ./models/DotProductClassifier.py
optim_params: {lr: 0.02, momentum: 0.9, weight_decay: 0.0005}
params: {dataset: Places_LT, feat_dim: 2048, num_classes: 365, stage1_weights: false}
feat_model:
def_file: ./models/ResNet152Feature.py
fix: true
optim_params: {lr: 0.01, momentum: 0.9, weight_decay: 0.0005}
params: {dataset: Places_LT, dropout: null, stage1_weights: false,
use_fc: false, use_selfatt: false}
shuffle: true
training_opt:
batch_size: 128
dataset: Places_LT
display_step: 10
feature_dim: 2048
log_dir: ./logs/Places_LT/clslearn/balanced_softmax
log_root: ./logs/Places_LT
num_classes: 365
num_epochs: 90
num_workers: 12
open_threshold: 0.1
sampler: {def_file: ./data/ClassAwareSampler.py, num_samples_cls: 4, type: ClassAwareSampler}
scheduler_params: {gamma: 0.1, step_size: 30}
stage: balanced_softmax
sub_dir: clslearn
================================================
FILE: long-tailed_recognition/BalancedSoftmax/config/Places_LT/balms.yaml
================================================
coslr: true
criterions:
PerformanceLoss:
def_file: ./loss/BalancedSoftmaxLoss.py
loss_params: {freq_path: ./cls_freq/Places_LT.json}
optim_params: null
weight: 1.0
endlr: 0.0
last: false
memory: {centroids: false, init_centroids: false}
model_dir: ./logs/Places_LT/models/dir/resnet152_uniform.pth
networks:
classifier:
def_file: ./models/DotProductClassifier.py
optim_params: {lr: 0.02, momentum: 0.9, weight_decay: 0.0005}
params: {dataset: Places_LT, feat_dim: 2048, num_classes: 365, stage1_weights: false}
feat_model:
def_file: ./models/ResNet152Feature.py
fix: true
optim_params: {lr: 0.01, momentum: 0.9, weight_decay: 0.0005}
params: {dataset: Places_LT, dropout: null, stage1_weights: false,
use_fc: false, use_selfatt: false}
shuffle: false
training_opt:
batch_size: 256
dataset: Places_LT
display_step: 10
feature_dim: 2048
log_dir: ./logs/Places_LT/clslearn/balms
log_root: ./logs/Places_LT
num_classes: 365
num_epochs: 10
num_workers: 4
open_threshold: 0.1
sampler: {def_file: ./data/MetaSampler.py, type: MetaSampler, meta_batch_size: 512, init_pow: 0.3, lr: 0.001}
scheduler_params: {gamma: 0.1, step_size: 30}
stage: balms
sub_dir: clslearn
================================================
FILE: long-tailed_recognition/BalancedSoftmax/config/Places_LT/cls_crt.yaml
================================================
coslr: true
criterions:
PerformanceLoss:
def_file: ./loss/SoftmaxLoss.py
loss_params: {}
optim_params: null
weight: 1.0
endlr: 0.0
last: false
memory: {centroids: false, init_centroids: false}
model_dir: ./logs/Places_LT/models/dir
networks:
classifier:
def_file: ./models/DotProductClassifier.py
optim_params: {lr: 0.1, momentum: 0.9, weight_decay: 0.0005}
params: {dataset: Places_LT, feat_dim: 2048, num_classes: 365, stage1_weights: false}
feat_model:
def_file: ./models/ResNet152Feature.py
fix: true
optim_params: {lr: 0.01, momentum: 0.9, weight_decay: 0.0005}
params: {dataset: Places_LT, dropout: null, stage1_weights: false,
use_fc: false, use_selfatt: false}
shuffle: true
training_opt:
batch_size: 256
dataset: Places_LT
display_step: 10
feature_dim: 2048
log_dir: ./logs/Places_LT/clslearn/retrain_uni2ban
log_root: ./logs/Places_LT
num_classes: 365
num_epochs: 10
num_workers: 4
open_threshold: 0.1
sampler: {def_file: ./data/ClassAwareSampler.py, num_samples_cls: 4, type: ClassAwareSampler}
scheduler_params: {gamma: 0.1, step_size: 30}
stage: retrain_uni2ban
sub_dir: clslearn
================================================
FILE: long-tailed_recognition/BalancedSoftmax/config/Places_LT/cls_lws.yaml
================================================
coslr: true
criterions:
PerformanceLoss:
def_file: ./loss/SoftmaxLoss.py
loss_params: {}
optim_params: null
weight: 1.0
endlr: 0.0
last: false
memory: {centroids: false, init_centroids: false}
model_dir: ./logs/Places_LT/models/dir/resnet152_uniform.pth
networks:
classifier:
def_file: ./models/TauNormClassifier.py
optim_params: {lr: 0.1, momentum: 0.9, weight_decay: 0.0005}
params: {dataset: Places_LT, feat_dim: 2048, num_classes: 365, stage1_weights: false}
feat_model:
def_file: ./models/ResNet152Feature.py
fix: true
optim_params: {lr: 0.01, momentum: 0.9, weight_decay: 0.0005}
params: {dataset: Places_LT, dropout: null, stage1_weights: false,
use_fc: false, use_selfatt: false}
shuffle: true
training_opt:
batch_size: 256
dataset: Places_LT
display_step: 10
feature_dim: 2048
log_dir: ./logs/Places_LT/clslearn/lws_uni2ban
log_root: ./logs/Places_LT
num_classes: 365
num_epochs: 5
num_workers: 4
open_threshold: 0.1
sampler: {def_file: ./data/ClassAwareSampler.py, num_samples_cls: 4, type: ClassAwareSampler}
scheduler_params: {gamma: 0.1, step_size: 30}
stage: lws_uni2ban
sub_dir: clslearn
================================================
FILE: long-tailed_recognition/BalancedSoftmax/config/Places_LT/decouple_balanced_softmax.yaml
================================================
coslr: true
criterions:
PerformanceLoss:
def_file: ./loss/BalancedSoftmaxLoss.py
loss_params: {freq_path: ./cls_freq/Places_LT.json}
optim_params: null
weight: 1.0
endlr: 0.0
last: false
memory: {centroids: false, init_centroids: false}
model_dir: ./logs/Places_LT/models/dir/resnet152_uniform.pth
networks:
classifier:
def_file: ./models/DotProductClassifier.py
optim_params: {lr: 0.02, momentum: 0.9, weight_decay: 0.0005}
params: {dataset: Places_LT, feat_dim: 2048, num_classes: 365, stage1_weights: false}
feat_model:
def_file: ./models/ResNet152Feature.py
fix: true
optim_params: {lr: 0.01, momentum: 0.9, weight_decay: 0.0005}
params: {dataset: Places_LT, dropout: null, stage1_weights: false,
use_fc: false, use_selfatt: false}
shuffle: true
training_opt:
batch_size: 256
dataset: Places_LT
display_step: 10
feature_dim: 2048
log_dir: ./logs/Places_LT/clslearn/decouple_balanced_softmax
log_root: ./logs/Places_LT
num_classes: 365
num_epochs: 10
num_workers: 4
open_threshold: 0.1
sampler: {def_file: ./data/ClassAwareSampler.py, num_samples_cls: 4, type: ClassAwareSampler}
scheduler_params: {gamma: 0.1, step_size: 30}
stage: decouple_balanced_softmax
sub_dir: clslearn
================================================
FILE: long-tailed_recognition/BalancedSoftmax/config/Places_LT/feat_unifrom.yaml
================================================
coslr: true
criterions:
PerformanceLoss:
def_file: ./loss/BalancedSoftmaxLoss.py
loss_params: {freq_path: ./cls_freq/Places_LT.json}
optim_params: null
weight: 1.0
endlr: 0.0
last: false
memory: {centroids: false, init_centroids: false}
model_dir: null
networks:
classifier:
def_file: ./models/DotProductClassifier.py
optim_params: {lr: 0.2, momentum: 0.9, weight_decay: 0.0001}
params: {dataset: iNaturalist18, feat_dim: 2048, log_dir: ./logs/iNaturalist18/models/resnet50_uniform_e90,
num_classes: 8142, stage1_weights: false}
feat_model:
def_file: ./models/ResNet50Feature.py
fix: false
optim_params: {lr: 0.2, momentum: 0.9, weight_decay: 0.0001}
params: {dataset: iNaturalist18, dropout: null, log_dir: ./logs/iNaturalist18/models/resnet50_uniform_e90,
stage1_weights: false, use_fc: false, use_selfatt: false}
shuffle: false
training_opt:
backbone: resnet50
batch_size: 128
dataset: iNaturalist18
display_step: 10
feature_dim: 2048
log_dir: ./logs/iNaturalist18/models/resnet50_uniform_e90
log_root: ./logs/iNaturalist18
num_classes: 8142
num_epochs: 90
num_workers: 12
open_threshold: 0.1
sampler: null
scheduler_params: {gamma: 0.1, step_size: 30}
stage: resnet50_uniform_e90
sub_dir: models
================================================
FILE: long-tailed_recognition/BalancedSoftmax/config/iNaturalist18/balanced_softmax.yaml
================================================
coslr: true
criterions:
PerformanceLoss:
def_file: ./loss/BalancedSoftmaxLoss.py
loss_params: {freq_path: ./cls_freq/iNaturalist18.json}
optim_params: null
weight: 1.0
endlr: 0.0
last: false
memory: {centroids: false, init_centroids: false}
model_dir: null
networks:
classifier:
def_file: ./models/DotProductClassifier.py
optim_params: {lr: 0.05, momentum: 0.9, weight_decay: 0.0001}
params: {dataset: iNaturalist18, feat_dim: 2048, log_dir: ./logs/iNaturalist18/models/balanced_softmax_resnet50,
num_classes: 8142, stage1_weights: false}
feat_model:
def_file: ./models/ResNet50Feature.py
fix: false
optim_params: {lr: 0.05, momentum: 0.9, weight_decay: 0.0001}
params: {dataset: iNaturalist18, dropout: null, log_dir: ./logs/iNaturalist18/models/balanced_softmax_resnet50,
stage1_weights: false, use_fc: false, use_selfatt: false}
shuffle: false
training_opt:
backbone: resnet50
batch_size: 128
dataset: iNaturalist18
display_step: 10
feature_dim: 2048
log_dir: ./logs/iNaturalist18/models/balanced_softmax_resnet50
log_root: ./logs/iNaturalist18
num_classes: 8142
num_epochs: 90
num_workers: 12
open_threshold: 0.1
sampler: null
scheduler_params: {gamma: 0.1, step_size: 30}
stage: resnet50_uniform_e90
sub_dir: models
================================================
FILE: long-tailed_recognition/BalancedSoftmax/config/iNaturalist18/cls_crt.yaml
================================================
coslr: true
criterions:
PerformanceLoss:
def_file: ./loss/SoftmaxLoss.py
loss_params: {}
optim_params: null
weight: 1.0
endlr: 0.0
last: false
memory: {centroids: false, init_centroids: false}
model_dir: ./logs/iNaturalist18/models/resnet50_uniform_e90
networks:
classifier:
def_file: ./models/DotProductClassifier.py
optim_params: {lr: 0.2, momentum: 0.9, weight_decay: 0.0001}
params: {dataset: iNaturalist18, feat_dim: 2048, log_dir: ./logs/iNaturalist18/clslearn/retrain_uni2ban,
num_classes: 8142, stage1_weights: false}
feat_model:
def_file: ./models/ResNet50Feature.py
fix: true
optim_params: {lr: 0.2, momentum: 0.9, weight_decay: 0.0001}
params: {dataset: iNaturalist18, dropout: null, log_dir: ./logs/iNaturalist18/clslearn/retrain_uni2ban,
stage1_weights: false, use_fc: false, use_selfatt: false}
shuffle: true
training_opt:
backbone: resnet50
batch_size: 512
dataset: iNaturalist18
display_step: 10
feature_dim: 2048
log_dir: ./logs/iNaturalist18/clslearn/retrain_uni2ban
log_root: ./logs/iNaturalist18
num_classes: 8142
num_epochs: 30
num_workers: 12
open_threshold: 0.1
sampler: {def_file: ./data/ClassAwareSampler.py, num_samples_cls: 4, type: ClassAwareSampler}
scheduler_params: {gamma: 0.1, step_size: 10}
stage: retrain_uni2ban
sub_dir: clslearn
================================================
FILE: long-tailed_recognition/BalancedSoftmax/config/iNaturalist18/cls_lws.yaml
================================================
coslr: true
criterions:
PerformanceLoss:
def_file: ./loss/SoftmaxLoss.py
loss_params: {}
optim_params: null
weight: 1.0
endlr: 0.0
last: false
memory: {centroids: false, init_centroids: false}
model_dir: ./logs/iNaturalist18/models/resnet50_uniform_e90
networks:
classifier:
def_file: ./models/TauNormClassifier.py
optim_params: {lr: 0.2, momentum: 0.9, weight_decay: 0.0001}
params: {dataset: iNaturalist18, feat_dim: 2048, log_dir: ./logs/iNaturalist18/clslearn/lws_uni2ban,
num_classes: 8142, stage1_weights: false}
feat_model:
def_file: ./models/ResNet50Feature.py
fix: true
optim_params: {lr: 0.2, momentum: 0.9, weight_decay: 0.0001}
params: {dataset: iNaturalist18, dropout: null, log_dir: ./logs/iNaturalist18/clslearn/lws_uni2ban,
stage1_weights: false, use_fc: false, use_selfatt: false}
shuffle: true
training_opt:
backbone: resnet50
batch_size: 512
dataset: iNaturalist18
display_step: 10
feature_dim: 2048
log_dir: ./logs/iNaturalist18/clslearn/lws_uni2ban
log_root: ./logs/iNaturalist18
num_classes: 8142
num_epochs: 5
num_workers: 12
open_threshold: 0.1
sampler: {def_file: ./data/ClassAwareSampler.py, num_samples_cls: 4, type: ClassAwareSampler}
scheduler_params: {gamma: 0.1, step_size: 1}
stage: lws_uni2ban
sub_dir: clslearn
================================================
FILE: long-tailed_recognition/BalancedSoftmax/config/iNaturalist18/feat_unifrom.yaml
================================================
coslr: true
criterions:
PerformanceLoss:
def_file: ./loss/SoftmaxLoss.py
loss_params: {}
optim_params: null
weight: 1.0
endlr: 0.0
last: false
memory: {centroids: false, init_centroids: false}
model_dir: null
networks:
classifier:
def_file: ./models/DotProductClassifier.py
optim_params: {lr: 0.05, momentum: 0.9, weight_decay: 0.0001}
params: {dataset: iNaturalist18, feat_dim: 2048, log_dir: ./logs/iNaturalist18/models/resnet50_uniform_e90,
num_classes: 8142, stage1_weights: false}
feat_model:
def_file: ./models/ResNet50Feature.py
fix: false
optim_params: {lr: 0.05, momentum: 0.9, weight_decay: 0.0001}
params: {dataset: iNaturalist18, dropout: null, log_dir: ./logs/iNaturalist18/models/resnet50_uniform_e90,
stage1_weights: false, use_fc: false, use_selfatt: false}
shuffle: false
training_opt:
backbone: resnet50
batch_size: 128
dataset: iNaturalist18
display_step: 10
feature_dim: 2048
log_dir: ./logs/iNaturalist18/models/resnet50_uniform_e90
log_root: ./logs/iNaturalist18
num_classes: 8142
num_epochs: 90
num_workers: 12
open_threshold: 0.1
sampler: null
scheduler_params: {gamma: 0.1, step_size: 30}
stage: resnet50_uniform_e90
sub_dir: models
================================================
FILE: long-tailed_recognition/BalancedSoftmax/data/ClassAwareSampler.py
================================================
"""Copyright (c) Facebook, Inc. and its affiliates.
All rights reserved.
This source code is licensed under the license found in the
LICENSE file in the root directory of this source tree.
Portions of the source code are from the OLTR project which
notice below and in LICENSE in the root directory of
this source tree.
Copyright (c) 2019, Zhongqi Miao
All rights reserved.
"""
import random
import numpy as np
from torch.utils.data.sampler import Sampler
import pdb
##################################
## Class-aware sampling, partly implemented by frombeijingwithlove
##################################
class RandomCycleIter:
def __init__ (self, data, test_mode=False):
self.data_list = list(data)
self.length = len(self.data_list)
self.i = self.length - 1
self.test_mode = test_mode
def __iter__ (self):
return self
def __next__ (self):
self.i += 1
if self.i == self.length:
self.i = 0
if not self.test_mode:
random.shuffle(self.data_list)
return self.data_list[self.i]
def class_aware_sample_generator (cls_iter, data_iter_list, n, num_samples_cls=1, is_infinite=False):
i = 0
j = 0
while i < n or is_infinite:
# yield next(data_iter_list[next(cls_iter)])
if j >= num_samples_cls:
j = 0
if j == 0:
temp_tuple = next(zip(*[data_iter_list[next(cls_iter)]]*num_samples_cls))
yield temp_tuple[j]
else:
yield temp_tuple[j]
i += 1
j += 1
class ClassAwareSampler (Sampler):
def __init__(self, data_source, num_samples_cls=1, is_infinite=False):
num_classes = len(np.unique(data_source.labels))
self.class_iter = RandomCycleIter(range(num_classes))
cls_data_list = [list() for _ in range(num_classes)]
for i, label in enumerate(data_source.labels):
cls_data_list[label].append(i)
self.data_iter_list = [RandomCycleIter(x) for x in cls_data_list]
self.num_samples = max([len(x) for x in cls_data_list]) * len(cls_data_list)
self.num_samples_cls = num_samples_cls
self.is_infinite = is_infinite
def __iter__ (self):
return class_aware_sample_generator(self.class_iter, self.data_iter_list,
self.num_samples, self.num_samples_cls, self.is_infinite)
def __len__ (self):
return self.num_samples
def get_sampler():
return ClassAwareSampler
##################################
================================================
FILE: long-tailed_recognition/BalancedSoftmax/data/ClassPrioritySampler.py
================================================
"""Copyright (c) Facebook, Inc. and its affiliates.
All rights reserved.
This source code is licensed under the license found in the
LICENSE file in the root directory of this source tree.
"""
import random
import numpy as np
from torch.utils.data.sampler import Sampler
class RandomCycleIter:
def __init__ (self, data, test_mode=False):
self.data_list = list(data)
self.length = len(self.data_list)
self.i = self.length - 1
self.test_mode = test_mode
def __iter__ (self):
return self
def __next__ (self):
self.i += 1
if self.i == self.length:
self.i = 0
if not self.test_mode:
random.shuffle(self.data_list)
return self.data_list[self.i]
class PriorityTree(object):
def __init__(self, capacity, init_weights, fixed_weights=None, fixed_scale=1.0,
alpha=1.0):
"""
fixed_weights: weights that wont be updated by self.update()
"""
assert fixed_weights is None or len(fixed_weights) == capacity
assert len(init_weights) == capacity
self.alpha = alpha
self._capacity = capacity
self._tree_size = 2 * capacity - 1
self.fixed_scale = fixed_scale
self.fixed_weights = np.zeros(self._capacity) if fixed_weights is None \
else fixed_weights
self.tree = np.zeros(self._tree_size)
self._initialized = False
self.initialize(init_weights)
def initialize(self, init_weights):
"""Initialize the tree."""
# Rescale the fixed_weights if it is not zero
self.fixed_scale_init = self.fixed_scale
if self.fixed_weights.sum() > 0 and init_weights.sum() > 0:
self.fixed_scale_init *= init_weights.sum() / self.fixed_weights.sum()
self.fixed_weights *= self.fixed_scale * init_weights.sum() \
/ self.fixed_weights.sum()
print('FixedWeights: {}'.format(self.fixed_weights.sum()))
self.update_whole(init_weights + self.fixed_weights)
self._initialized = True
def reset_adaptive_weights(self, adaptive_weights):
self.update_whole(self.fixed_weights + adaptive_weights)
def reset_fixed_weights(self, fixed_weights, rescale=False):
""" Reset the manually designed weights and
update the whole tree accordingly.
@rescale: rescale the fixed_weights such that
fixed_weights.sum() = self.fixed_scale * adaptive_weights.sum()
"""
adaptive_weights = self.get_adaptive_weights()
fixed_sum = fixed_weights.sum()
if rescale and fixed_sum > 0:
# Rescale fixedweight based on adaptive weights
scale = self.fixed_scale * adaptive_weights.sum() / fixed_sum
else:
# Rescale fixedweight based on previous fixedweight
scale = self.fixed_weights.sum() / fixed_sum
self.fixed_weights = fixed_weights * scale
self.update_whole(self.fixed_weights + adaptive_weights)
def update_whole(self, total_weights):
""" Update the whole tree based on per-example sampling weights """
if self.alpha != 1:
total_weights = np.power(total_weights, self.alpha)
lefti = self.pointer_to_treeidx(0)
righti = self.pointer_to_treeidx(self.capacity-1)
self.tree[lefti:righti+1] = total_weights
# Iteratively find a parent layer
while lefti != 0 and righti != 0:
lefti = (lefti - 1) // 2 if lefti != 0 else 0
righti = (righti - 1) // 2 if righti != 0 else 0
# Assign paraent weights from right to left
for i in range(righti, lefti-1, -1):
self.tree[i] = self.tree[2*i+1] + self.tree[2*i+2]
def get_adaptive_weights(self):
""" Get the instance-aware weights, that are not mannually designed"""
if self.alpha == 1:
return self.get_total_weights() - self.fixed_weights
else:
return self.get_raw_total_weights() - self.fixed_weights
def get_total_weights(self):
""" Get the per-example sampling weights
return shape: [capacity]
"""
lefti = self.pointer_to_treeidx(0)
righti = self.pointer_to_treeidx(self.capacity-1)
return self.tree[lefti:righti+1]
def get_raw_total_weights(self):
""" Get the per-example sampling weights
return shape: [capacity]
"""
lefti = self.pointer_to_treeidx(0)
righti = self.pointer_to_treeidx(self.capacity-1)
return np.power(self.tree[lefti:righti+1], 1/self.alpha)
@property
def size(self):
return self._tree_size
@property
def capacity(self):
return self._capacity
def __len__(self):
return self.capacity
def pointer_to_treeidx(self, pointer):
assert pointer < self.capacity
return int(pointer + self.capacity - 1)
def update(self, pointer, priority):
assert pointer < self.capacity
tree_idx = self.pointer_to_treeidx(pointer)
priority += self.fixed_weights[pointer]
if self.alpha != 1:
priority = np.power(priority, self.alpha)
delta = priority - self.tree[tree_idx]
self.tree[tree_idx] = priority
while tree_idx != 0:
tree_idx = (tree_idx - 1) // 2
self.tree[tree_idx] += delta
def update_delta(self, pointer, delta):
assert pointer < self.capacity
tree_idx = self.pointer_to_treeidx(pointer)
ratio = 1- self.fixed_weights[pointer] / self.tree[tree_idx]
# delta *= ratio
if self.alpha != 1:
# Update delta
if self.tree[tree_idx] < 0 or \
np.power(self.tree[tree_idx], 1/self.alpha) + delta < 0:
import pdb; pdb.set_trace()
delta = np.power(np.power(self.tree[tree_idx], 1/self.alpha) + delta,
self.alpha) \
- self.tree[tree_idx]
self.tree[tree_idx] += delta
while tree_idx != 0:
tree_idx = (tree_idx - 1) // 2
self.tree[tree_idx] += delta
def get_leaf(self, value):
assert self._initialized, 'PriorityTree not initialized!!!!'
assert self.total > 0, 'No priority weights setted!!'
parent = 0
while True:
left_child = 2 * parent + 1
right_child = 2 * parent + 2
if left_child >= len(self.tree):
tgt_leaf = parent
break
if value < self.tree[left_child]:
parent = left_child
else:
value -= self.tree[left_child]
parent = right_child
data_idx = tgt_leaf - self.capacity + 1
return data_idx, self.tree[tgt_leaf] # data idx, priority
@property
def total(self):
assert self._initialized, 'PriorityTree not initialized!!!!'
return self.tree[0]
@property
def max(self):
return np.max(self.tree[-self.capacity:])
@property
def min(self):
assert self._initialized, 'PriorityTree not initialized!!!!'
return np.min(self.tree[-self.capacity:])
def get_weights(self):
wdict = {'fixed_weights': self.fixed_weights,
'total_weights': self.get_total_weights()}
if self.alpha != 1:
wdict.update({'raw_total_weights': self.get_raw_total_weights(),
'alpha': self.alpha})
return wdict
class ClassPrioritySampler(Sampler):
"""
A sampler combining manually designed sampling strategy and prioritized
sampling strategy.
Manually disigned strategy contains two parts:
$$ manual_weights = lam * balanced_weights + (1-lam) uniform_weights
Here we use a generalized version of balanced weights as follows,
when n limits to infinity, balanced_weights = real_balanced_weights
$$ balanced_weights = uniform_weights ^ (1/n)
Then the balanced weights are scaled such that
$$ balanced_weights.sum() = balance_scale * uniform_weights.sum()
Note: above weights are per-class weights
Overall sampling weights are given as
$$ sampling_weights = manual_weights * fixed_scale + priority_weights
Arguments:
@dataset: A dataset
@balance_scale: The scale of balanced_weights
@lam: A weight to combine balanced weights and uniform weights
- None for shifting sampling
- 0 for uniform sampling
- 1 for balanced sampling
@fixed_scale: The scale of manually designed weights
- fixed_scale < 0 means, the manually designed distribution will
be used as the backend distribution of priorities.
@cycle: shifting strategy
- 0 for linear shifting: 3 -> 2 - > 1
- 1 for periodic shifting:
3 -> 2 - > 1 -> 3 -> 2 - > 1 -> 3 -> 2 - > 1
- 2 for cosine-like periodic shifting:
3 -> 2 - > 1 -> 1 -> 2 - > 3 -> 3 -> 2 - > 1
@nroot:
- None for truly balanced weights
- >= 2 for pseudo-balanced weights
@rescale: whether to rebalance the manual weights and priority weights
every epoch
@root_decay:
- 'exp': for exponential decay
- 'linear': for linear decay
"""
def __init__(self, dataset, balance_scale=1.0, fixed_scale=1.0,
lam=None, epochs=90, cycle=0, nroot=None, manual_only=False,
rescale=False, root_decay=None, decay_gap=30, ptype='score',
pri_mode='train', momentum=0., alpha=1.0):
"""
"""
self.dataset = dataset
self.balance_scale = balance_scale
self.fixed_scale = fixed_scale
self.epochs = epochs
self.lam = lam
self.cycle = cycle
self.nroot = nroot
self.rescale = rescale
self.manual_only = manual_only
self.root_decay = root_decay
self.decay_gap = decay_gap
self.ptype = ptype
self.pri_mode = pri_mode
self.num_samples = len(dataset)
self.manual_as_backend = False
self.momentum = momentum
self.alpha = alpha
assert 0. <= self.momentum <= 1.0
assert 0. <= self.alpha
# Change the backend distribution of priority if needed
if self.fixed_scale <
## Introduction
*Transfer-Learn* is an open-source and well-documented library for Transfer Learning. It is based on pure PyTorch with high performance and friendly API. Our code is pythonic, and the design is consistent with torchvision. You can easily develop new algorithms, or readily apply existing algorithms.
The currently supported algorithms include:
##### [Domain Adaptation for Classification](/examples/domain_adaptation/image_classification)
- [Domain Adversarial Neural Network (DANN)](https://arxiv.org/abs/1505.07818)
- [Deep Adaptation Network (DAN)](https://arxiv.org/pdf/1502.02791)
- [Joint Adaptation Network (JAN)](https://arxiv.org/abs/1605.06636)
- [Adversarial Discriminative Domain Adaptation (ADDA)](https://arxiv.org/pdf/1702.05464.pdf)
- [Conditional Domain Adversarial Network (CDAN)](https://arxiv.org/abs/1705.10667)
- [Maximum Classifier Discrepancy (MCD)](https://arxiv.org/abs/1712.02560)
- [Adaptive Feature Norm (AFN)](https://arxiv.org/pdf/1811.07456v2.pdf)
- [Margin Disparity Discrepancy (MDD)](https://arxiv.org/abs/1904.05801)
- [Batch Spectral Penalization (BSP)](http://ise.thss.tsinghua.edu.cn/~mlong/doc/batch-spectral-penalization-icml19.pdf)
- [Minimum Class Confusion (MCC)](https://arxiv.org/abs/1912.03699)
##### [Partial Domain Adaptation](/examples/domain_adaptation/partial_domain_adaptation/)
- [Domain Adversarial Neural Network (DANN)](https://arxiv.org/abs/1505.07818)
- [Partial Adversarial Domain Adaptation (PADA)](https://arxiv.org/abs/1808.04205)
- [Importance Weighted Adversarial Nets (IWAN)](https://arxiv.org/abs/1803.09210)
- [Adaptive Feature Norm (AFN)](https://arxiv.org/pdf/1811.07456v2.pdf)
##### [Open-set Domain Adaptation](/examples/domain_adaptation/openset_domain_adaptation/)
- [Open Set Domain Adaptation (OSBP)](https://arxiv.org/abs/1804.10427)
##### [Domain Adaptation for Semantic Segmentation](/examples/domain_adaptation/semantic_segmentation/)
- [Cycle-Consistent Adversarial Networks (CycleGAN)](https://arxiv.org/pdf/1703.10593.pdf)
- [CyCADA: Cycle-Consistent Adversarial Domain Adaptation](https://arxiv.org/abs/1711.03213)
- [Adversarial Entropy Minimization (ADVENT)](https://arxiv.org/abs/1811.12833)
- [Fourier Domain Adaptation (FDA)](https://arxiv.org/abs/2004.05498)
##### [Domain Adaptation for Keypoint Detection](/examples/domain_adaptation/keypoint_detection)
- [Regressive Domain Adaptation for Unsupervised Keypoint Detection (RegDA, CVPR 2021)](http://ise.thss.tsinghua.edu.cn/~mlong/doc/regressive-domain-adaptation-cvpr21.pdf)
##### [Domain Adaptation for Person Re-identification](/examples/domain_adaptation/re_identification/)
- [Two at Once: Enhancing Learning and Generalization Capacities via IBN-Net (IBN-Net, ECCV 2018)](https://openaccess.thecvf.com/content_ECCV_2018/papers/Xingang_Pan_Two_at_Once_ECCV_2018_paper.pdf)
- [Mutual Mean-Teaching: Pseudo Label Refinery for Unsupervised Domain Adaptation on Person Re-identification (MMT, ICLR 2020)](https://arxiv.org/abs/2001.01526)
- [Similarity Preserving Generative Adversarial Network (SPGAN, CVPR 2018)](https://arxiv.org/pdf/1811.10551.pdf)
##### [Task Adaptation for Image Classification](/examples/task_adaptation/image_classification/)
- [Explicit inductive bias for transfer learning with convolutional networks
(L2-SP, ICML 2018)](https://arxiv.org/abs/1802.01483)
- [Catastrophic Forgetting Meets Negative Transfer: Batch Spectral Shrinkage for Safe Transfer Learning (BSS, NIPS 2019)](https://proceedings.neurips.cc/paper/2019/file/c6bff625bdb0393992c9d4db0c6bbe45-Paper.pdf)
- [DEep Learning Transfer using Fea- ture Map with Attention for convolutional networks (DELTA, ICLR 2019)](https://openreview.net/pdf?id=rkgbwsAcYm)
- [Co-Tuning for Transfer Learning (Co-Tuning, NIPS 2020)](http://ise.thss.tsinghua.edu.cn/~mlong/doc/co-tuning-for-transfer-learning-nips20.pdf)
- [Stochastic Normalization (StochNorm, NIPS 2020)](https://papers.nips.cc/paper/2020/file/bc573864331a9e42e4511de6f678aa83-Paper.pdf)
- [Learning Without Forgetting (LWF, ECCV 2016)](https://arxiv.org/abs/1606.09282)
- [Bi-tuning of Pre-trained Representations (Bi-Tuning)](https://arxiv.org/abs/2011.06182?utm_source=feedburner&utm_medium=feed&utm_campaign=Feed%3A+arxiv%2FQSXk+%28ExcitingAds%21+cs+updates+on+arXiv.org%29)
##### [Domain Generalization for Classification](/examples/domain_generalization/image_classification/)
- [Two at Once: Enhancing Learning and Generalization Capacities via IBN-Net (IBN-Net, ECCV 2018)](https://openaccess.thecvf.com/content_ECCV_2018/papers/Xingang_Pan_Two_at_Once_ECCV_2018_paper.pdf)
- [Domain Generalization with MixStyle (MixStyle, ICLR 2021)](https://arxiv.org/abs/2104.02008)
- [Learning to Generalize: Meta-Learning for Domain Generalization (MLDG, AAAI 2018)](https://arxiv.org/pdf/1710.03463.pdf)
- [Invariant Risk Minimization (IRM)](https://arxiv.org/abs/1907.02893)
- [Out-of-Distribution Generalization via Risk Extrapolation (VREx, ICML 2021)](https://arxiv.org/abs/2003.00688)
- [Distributionally Robust Neural Networks for Group Shifts: On the Importance of Regularization for Worst-Case Generalization(GroupDRO)](https://arxiv.org/abs/1911.08731)
- [Deep CORAL: Correlation Alignment for Deep Domain Adaptation (Deep Coral, ECCV 2016)](https://arxiv.org/abs/1607.01719)
##### [Domain Generalization for Person Re-identification](/examples/domain_generalization/re_identification/)
- [Two at Once: Enhancing Learning and Generalization Capacities via IBN-Net (IBN-Net, ECCV 2018)](https://openaccess.thecvf.com/content_ECCV_2018/papers/Xingang_Pan_Two_at_Once_ECCV_2018_paper.pdf)
- [Domain Generalization with MixStyle (MixStyle, ICLR 2021)](https://arxiv.org/abs/2104.02008)
We are planning to add
- DA for Object Detection
- TA for text classification
## Installation
To use ``dalib``, ``talib``, ``dglib``, and ``common`` in other places, you need to install Transfer-Learn,
```shell
python setup.py install
```
Note that we do not support *pip install* currently.
For flexible use and modification of Transfer-Learn, please git clone the library.
## Documentation
You can find the tutorial and API documentation on the website: [Documentation (please open in Firefox or Safari)](http://170.106.108.162/index.html). Note that this link is only for temporary use. You can also build the doc by yourself following the instructions in http://170.106.108.162/get_started/faq.html.
Also, we have examples in the directory `examples`. A typical usage is
```shell script
# Train a DANN on Office-31 Amazon -> Webcam task using ResNet 50.
# Assume you have put the datasets under the path `data/office-31`,
# or you are glad to download the datasets automatically from the Internet to this path
python dann.py data/office31 -d Office31 -s A -t W -a resnet50 --epochs 20
```
In the directory `examples`, you can find all the necessary running scripts to reproduce the benchmarks with specified hyper-parameters.
## Contributing
We appreciate all contributions. If you are planning to contribute back bug-fixes, please do so without any further discussion. If you plan to contribute new features, utility functions or extensions, please first open an issue and discuss the feature with us.
## Disclaimer on Datasets
This is a utility library that downloads and prepares public datasets. We do not host or distribute these datasets, vouch for their quality or fairness, or claim that you have licenses to use the dataset. It is your responsibility to determine whether you have permission to use the dataset under the dataset's license.
If you're a dataset owner and wish to update any part of it (description, citation, etc.), or do not want your dataset to be included in this library, please get in touch through a GitHub issue. Thanks for your contribution to the ML community!
## Contact
If you have any problem with our code or have some suggestions, including the future feature, feel free to contact
- Junguang Jiang (JiangJunguang1123@outlook.com)
- Baixu Chen (cbx_99_hasta@outlook.com)
- Mingsheng Long (longmingsheng@gmail.com)
or describe it in Issues.
For Q&A in Chinese, you can choose to ask questions here before sending an email. [迁移学习算法库答疑专区](https://zhuanlan.zhihu.com/p/248104070)
## Citation
If you use this toolbox or benchmark in your research, please cite this project.
```latex
@misc{dalib,
author = {Junguang Jiang, Baixu Chen, Bo Fu, Mingsheng Long},
title = {Transfer-Learning-library},
year = {2020},
publisher = {GitHub},
journal = {GitHub repository},
howpublished = {\url{https://github.com/thuml/Transfer-Learning-Library}},
}
```
## Acknowledgment
We would like to thank School of Software, Tsinghua University and The National Engineering Laboratory for Big Data Software for providing such an excellent ML research platform.
================================================
FILE: domain_generalization/Transfer-Learning-Library/common/__init__.py
================================================
__all__ = ['modules', 'utils', 'vision']
================================================
FILE: domain_generalization/Transfer-Learning-Library/common/loss/__init__.py
================================================
import torch.nn as nn
import torch.nn.functional as F
class KnowledgeDistillationLoss(nn.Module):
"""Knowledge Distillation Loss.
Args:
T (double): Temperature. Default: 1.
reduction (str, optional): Specifies the reduction to apply to the output:
``'none'`` | ``'mean'`` | ``'sum'``. ``'none'``: no reduction will be applied,
``'mean'``: the sum of the output will be divided by the number of
elements in the output, ``'sum'``: the output will be summed. Default: ``'batchmean'``
Inputs:
- y_student (tensor): logits output of the student
- y_teacher (tensor): logits output of the teacher
Shape:
- y_student: (minibatch, `num_classes`)
- y_teacher: (minibatch, `num_classes`)
"""
def __init__(self, T=1., reduction='batchmean'):
super(KnowledgeDistillationLoss, self).__init__()
self.T = T
self.kl = nn.KLDivLoss(reduction=reduction)
def forward(self, y_student, y_teacher):
""""""
return self.kl(F.log_softmax(y_student / self.T, dim=-1), F.softmax(y_teacher / self.T, dim=-1))
================================================
FILE: domain_generalization/Transfer-Learning-Library/common/modules/__init__.py
================================================
from .classifier import *
from .regressor import *
__all__ = ['classifier', 'regressor']
================================================
FILE: domain_generalization/Transfer-Learning-Library/common/modules/classifier.py
================================================
"""
@author: Junguang Jiang
@contact: JiangJunguang1123@outlook.com
"""
from typing import Tuple, Optional, List, Dict
import torch.nn as nn
import torch
__all__ = ['Classifier']
class Classifier(nn.Module):
"""A generic Classifier class for domain adaptation.
Args:
backbone (torch.nn.Module): Any backbone to extract 2-d features from data
num_classes (int): Number of classes
bottleneck (torch.nn.Module, optional): Any bottleneck layer. Use no bottleneck by default
bottleneck_dim (int, optional): Feature dimension of the bottleneck layer. Default: -1
head (torch.nn.Module, optional): Any classifier head. Use :class:`torch.nn.Linear` by default
finetune (bool): Whether finetune the classifier or train from scratch. Default: True
.. note::
Different classifiers are used in different domain adaptation algorithms to achieve better accuracy
respectively, and we provide a suggested `Classifier` for different algorithms.
Remember they are not the core of algorithms. You can implement your own `Classifier` and combine it with
the domain adaptation algorithm in this algorithm library.
.. note::
The learning rate of this classifier is set 10 times to that of the feature extractor for better accuracy
by default. If you have other optimization strategies, please over-ride :meth:`~Classifier.get_parameters`.
Inputs:
- x (tensor): input data fed to `backbone`
Outputs:
- predictions: classifier's predictions
- features: features after `bottleneck` layer and before `head` layer
Shape:
- Inputs: (minibatch, *) where * means, any number of additional dimensions
- predictions: (minibatch, `num_classes`)
- features: (minibatch, `features_dim`)
"""
def __init__(self, backbone: nn.Module, num_classes: int, bottleneck: Optional[nn.Module] = None,
bottleneck_dim: Optional[int] = -1, head: Optional[nn.Module] = None, finetune=True, pool_layer=None):
super(Classifier, self).__init__()
self.backbone = backbone
self.num_classes = num_classes
if pool_layer is None:
self.pool_layer = nn.Sequential(
nn.AdaptiveAvgPool2d(output_size=(1, 1)),
nn.Flatten()
)
else:
self.pool_layer = pool_layer
if bottleneck is None:
self.bottleneck = nn.Identity()
self._features_dim = backbone.out_features
else:
self.bottleneck = bottleneck
assert bottleneck_dim > 0
self._features_dim = bottleneck_dim
if head is None:
self.head = nn.Linear(self._features_dim, num_classes)
else:
self.head = head
self.finetune = finetune
@property
def features_dim(self) -> int:
"""The dimension of features before the final `head` layer"""
return self._features_dim
def forward(self, x: torch.Tensor) -> Tuple[torch.Tensor, torch.Tensor]:
""""""
f = self.pool_layer(self.backbone(x))
f = self.bottleneck(f)
predictions = self.head(f)
if self.training:
return predictions, f
else:
return predictions
def get_parameters(self, base_lr=1.0) -> List[Dict]:
"""A parameter list which decides optimization hyper-parameters,
such as the relative learning rate of each layer
"""
params = [
{"params": self.backbone.parameters(), "lr": 0.1 * base_lr if self.finetune else 1.0 * base_lr},
{"params": self.bottleneck.parameters(), "lr": 1.0 * base_lr},
{"params": self.head.parameters(), "lr": 1.0 * base_lr},
]
return params
class ImageClassifier(Classifier):
pass
================================================
FILE: domain_generalization/Transfer-Learning-Library/common/modules/regressor.py
================================================
"""
@author: Junguang Jiang
@contact: JiangJunguang1123@outlook.com
"""
from typing import Tuple, Optional, List, Dict
import torch.nn as nn
import torch
__all__ = ['Regressor']
class Regressor(nn.Module):
"""A generic Regressor class for domain adaptation.
Args:
backbone (torch.nn.Module): Any backbone to extract 2-d features from data
num_factors (int): Number of factors
bottleneck (torch.nn.Module, optional): Any bottleneck layer. Use no bottleneck by default
bottleneck_dim (int, optional): Feature dimension of the bottleneck layer. Default: -1
head (torch.nn.Module, optional): Any classifier head. Use `nn.Linear` by default
finetune (bool): Whether finetune the classifier or train from scratch. Default: True
.. note::
The learning rate of this regressor is set 10 times to that of the feature extractor for better accuracy
by default. If you have other optimization strategies, please over-ride :meth:`~Regressor.get_parameters`.
Inputs:
- x (tensor): input data fed to `backbone`
Outputs:
- predictions: regressor's predictions
- features: features after `bottleneck` layer and before `head` layer
Shape:
- Inputs: (minibatch, *) where * means, any number of additional dimensions
- predictions: (minibatch, `num_factors`)
- features: (minibatch, `features_dim`)
"""
def __init__(self, backbone: nn.Module, num_factors: int, bottleneck: Optional[nn.Module] = None,
bottleneck_dim=-1, head: Optional[nn.Module] = None, finetune=True):
super(Regressor, self).__init__()
self.backbone = backbone
self.num_factors = num_factors
if bottleneck is None:
feature_dim = backbone.out_features
self.bottleneck = nn.Sequential(
nn.Conv2d(feature_dim, feature_dim, kernel_size=3, stride=1, padding=1),
nn.BatchNorm2d(feature_dim, feature_dim),
nn.ReLU(),
nn.AdaptiveAvgPool2d(output_size=(1, 1)),
nn.Flatten()
)
self._features_dim = feature_dim
else:
self.bottleneck = bottleneck
assert bottleneck_dim > 0
self._features_dim = bottleneck_dim
if head is None:
self.head = nn.Sequential(
nn.Linear(self._features_dim, num_factors),
nn.Sigmoid()
)
else:
self.head = head
self.finetune = finetune
@property
def features_dim(self) -> int:
"""The dimension of features before the final `head` layer"""
return self._features_dim
def forward(self, x: torch.Tensor) -> Tuple[torch.Tensor, torch.Tensor]:
""""""
f = self.backbone(x)
f = self.bottleneck(f)
predictions = self.head(f)
if self.training:
return predictions, f
else:
return predictions
def get_parameters(self, base_lr=1.0) -> List[Dict]:
"""A parameter list which decides optimization hyper-parameters,
such as the relative learning rate of each layer
"""
params = [
{"params": self.backbone.parameters(), "lr": 0.1 * base_lr if self.finetune else 1.0 * base_lr},
{"params": self.bottleneck.parameters(), "lr": 1.0 * base_lr},
{"params": self.head.parameters(), "lr": 1.0 * base_lr},
]
return params
================================================
FILE: domain_generalization/Transfer-Learning-Library/common/utils/__init__.py
================================================
from .logger import CompleteLogger
from .meter import *
from .data import ForeverDataIterator
__all__ = ['metric', 'analysis', 'meter', 'data', 'logger']
================================================
FILE: domain_generalization/Transfer-Learning-Library/common/utils/analysis/__init__.py
================================================
import torch
from torch.utils.data import DataLoader
import torch.nn as nn
import tqdm
def collect_feature(data_loader: DataLoader, feature_extractor: nn.Module,
device: torch.device, max_num_features=None) -> torch.Tensor:
"""
Fetch data from `data_loader`, and then use `feature_extractor` to collect features
Args:
data_loader (torch.utils.data.DataLoader): Data loader.
feature_extractor (torch.nn.Module): A feature extractor.
device (torch.device)
max_num_features (int): The max number of features to return
Returns:
Features in shape (min(len(data_loader), max_num_features * mini-batch size), :math:`|\mathcal{F}|`).
"""
feature_extractor.eval()
all_features = []
with torch.no_grad():
for i, (images, target) in enumerate(tqdm.tqdm(data_loader)):
if max_num_features is not None and i >= max_num_features:
break
images = images.to(device)
feature = feature_extractor(images).cpu()
all_features.append(feature)
return torch.cat(all_features, dim=0)
================================================
FILE: domain_generalization/Transfer-Learning-Library/common/utils/analysis/a_distance.py
================================================
"""
@author: Junguang Jiang
@contact: JiangJunguang1123@outlook.com
"""
from torch.utils.data import TensorDataset
import torch
import torch.nn as nn
import torch.nn.functional as F
from torch.utils.data import DataLoader
from torch.optim import SGD
from ..meter import AverageMeter
from ..metric import binary_accuracy
class ANet(nn.Module):
def __init__(self, in_feature):
super(ANet, self).__init__()
self.layer = nn.Linear(in_feature, 1)
self.sigmoid = nn.Sigmoid()
def forward(self, x):
x = self.layer(x)
x = self.sigmoid(x)
return x
def calculate(source_feature: torch.Tensor, target_feature: torch.Tensor,
device, progress=True, training_epochs=10):
"""
Calculate the :math:`\mathcal{A}`-distance, which is a measure for distribution discrepancy.
The definition is :math:`dist_\mathcal{A} = 2 (1-2\epsilon)`, where :math:`\epsilon` is the
test error of a classifier trained to discriminate the source from the target.
Args:
source_feature (tensor): features from source domain in shape :math:`(minibatch, F)`
target_feature (tensor): features from target domain in shape :math:`(minibatch, F)`
device (torch.device)
progress (bool): if True, displays a the progress of training A-Net
training_epochs (int): the number of epochs when training the classifier
Returns:
:math:`\mathcal{A}`-distance
"""
source_label = torch.ones((source_feature.shape[0], 1))
target_label = torch.zeros((target_feature.shape[0], 1))
feature = torch.cat([source_feature, target_feature], dim=0)
label = torch.cat([source_label, target_label], dim=0)
dataset = TensorDataset(feature, label)
length = len(dataset)
train_size = int(0.8 * length)
val_size = length - train_size
train_set, val_set = torch.utils.data.random_split(dataset, [train_size, val_size])
train_loader = DataLoader(train_set, batch_size=2, shuffle=True)
val_loader = DataLoader(val_set, batch_size=8, shuffle=False)
anet = ANet(feature.shape[1]).to(device)
optimizer = SGD(anet.parameters(), lr=0.01)
a_distance = 2.0
for epoch in range(training_epochs):
anet.train()
for (x, label) in train_loader:
x = x.to(device)
label = label.to(device)
anet.zero_grad()
y = anet(x)
loss = F.binary_cross_entropy(y, label)
loss.backward()
optimizer.step()
anet.eval()
meter = AverageMeter("accuracy", ":4.2f")
with torch.no_grad():
for (x, label) in val_loader:
x = x.to(device)
label = label.to(device)
y = anet(x)
acc = binary_accuracy(y, label)
meter.update(acc, x.shape[0])
error = 1 - meter.avg / 100
a_distance = 2 * (1 - 2 * error)
if progress:
print("epoch {} accuracy: {} A-dist: {}".format(epoch, meter.avg, a_distance))
return a_distance
================================================
FILE: domain_generalization/Transfer-Learning-Library/common/utils/analysis/tsne.py
================================================
"""
@author: Junguang Jiang
@contact: JiangJunguang1123@outlook.com
"""
import torch
import matplotlib
matplotlib.use('Agg')
from sklearn.manifold import TSNE
import numpy as np
import matplotlib.pyplot as plt
import matplotlib.colors as col
def visualize(source_feature: torch.Tensor, target_feature: torch.Tensor,
filename: str, source_color='r', target_color='b'):
"""
Visualize features from different domains using t-SNE.
Args:
source_feature (tensor): features from source domain in shape :math:`(minibatch, F)`
target_feature (tensor): features from target domain in shape :math:`(minibatch, F)`
filename (str): the file name to save t-SNE
source_color (str): the color of the source features. Default: 'r'
target_color (str): the color of the target features. Default: 'b'
"""
source_feature = source_feature.numpy()
target_feature = target_feature.numpy()
features = np.concatenate([source_feature, target_feature], axis=0)
# map features to 2-d using TSNE
X_tsne = TSNE(n_components=2, random_state=33).fit_transform(features)
# domain labels, 1 represents source while 0 represents target
domains = np.concatenate((np.ones(len(source_feature)), np.zeros(len(target_feature))))
# visualize using matplotlib
fig, ax = plt.subplots(figsize=(10, 10))
ax.spines['top'].set_visible(False)
ax.spines['right'].set_visible(False)
ax.spines['bottom'].set_visible(False)
ax.spines['left'].set_visible(False)
plt.scatter(X_tsne[:, 0], X_tsne[:, 1], c=domains, cmap=col.ListedColormap([target_color, source_color]), s=20)
plt.xticks([])
plt.yticks([])
plt.savefig(filename)
================================================
FILE: domain_generalization/Transfer-Learning-Library/common/utils/data.py
================================================
"""
@author: Junguang Jiang
@contact: JiangJunguang1123@outlook.com
"""
import itertools
import random
import numpy as np
import torch
from torch.utils.data import Sampler
from torch.utils.data import DataLoader, Dataset
from typing import TypeVar, Iterable, Dict, List
T_co = TypeVar('T_co', covariant=True)
T = TypeVar('T')
def send_to_device(tensor, device):
"""
Recursively sends the elements in a nested list/tuple/dictionary of tensors to a given device.
Args:
tensor (nested list/tuple/dictionary of :obj:`torch.Tensor`):
The data to send to a given device.
device (:obj:`torch.device`):
The device to send the data to
Returns:
The same data structure as :obj:`tensor` with all tensors sent to the proper device.
"""
if isinstance(tensor, (list, tuple)):
return type(tensor)(send_to_device(t, device) for t in tensor)
elif isinstance(tensor, dict):
return type(tensor)({k: send_to_device(v, device) for k, v in tensor.items()})
elif not hasattr(tensor, "to"):
return tensor
return tensor.to(device)
class ForeverDataIterator:
r"""A data iterator that will never stop producing data"""
def __init__(self, data_loader: DataLoader, device=None):
self.data_loader = data_loader
self.iter = iter(self.data_loader)
self.device = device
def __next__(self):
try:
data = next(self.iter)
if self.device is not None:
data = send_to_device(data, self.device)
except StopIteration:
self.iter = iter(self.data_loader)
data = next(self.iter)
if self.device is not None:
data = send_to_device(data, self.device)
return data
def __len__(self):
return len(self.data_loader)
class RandomMultipleGallerySampler(Sampler):
r"""Sampler from `In defense of the Triplet Loss for Person Re-Identification
(ICCV 2017)