Showing preview only (309K chars total). Download the full file or copy to clipboard to get everything.
Repository: alibaba/cluster-contrast-reid
Branch: main
Commit: 57b62e95eb3a
Files: 44
Total size: 98.0 MB
Directory structure:
gitextract_7h4tsjq6/
├── LICENSE
├── README.md
├── clustercontrast/
│ ├── __init__.py
│ ├── datasets/
│ │ ├── __init__.py
│ │ ├── dukemtmcreid.py
│ │ ├── market1501.py
│ │ ├── msmt17.py
│ │ ├── personx.py
│ │ └── veri.py
│ ├── evaluation_metrics/
│ │ ├── __init__.py
│ │ ├── classification.py
│ │ └── ranking.py
│ ├── evaluators.py
│ ├── models/
│ │ ├── __init__.py
│ │ ├── cm.py
│ │ ├── dsbn.py
│ │ ├── kmeans.py
│ │ ├── pooling.py
│ │ ├── resnet.py
│ │ ├── resnet_ibn.py
│ │ └── resnet_ibn_a.py
│ ├── trainers.py
│ └── utils/
│ ├── __init__.py
│ ├── data/
│ │ ├── __init__.py
│ │ ├── base_dataset.py
│ │ ├── preprocessor.py
│ │ ├── sampler.py
│ │ └── transforms.py
│ ├── faiss_rerank.py
│ ├── faiss_utils.py
│ ├── infomap_cluster.py
│ ├── infomap_utils.py
│ ├── logging.py
│ ├── meters.py
│ ├── osutils.py
│ ├── rerank.py
│ └── serialization.py
├── examples/
│ ├── cluster_contrast_train_usl.py
│ ├── cluster_contrast_train_usl_infomap.py
│ ├── logs/
│ │ └── log.txt
│ ├── pretrained/
│ │ └── resnet50-19c8e357.pth
│ └── test.py
├── run_code.sh
└── setup.py
================================================
FILE CONTENTS
================================================
================================================
FILE: LICENSE
================================================
MIT License
Copyright (c) 2021 Alibaba
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
================================================
FILE: README.md
================================================
# Cluster Contrast for Unsupervised Person Re-Identification
The *official* repository for [Cluster Contrast for Unsupervised Person Re-Identification](https://arxiv.org/pdf/2103.11568v3.pdf). We achieve state-of-the-art performances on **unsupervised learning** tasks for object re-ID, including person re-ID and vehicle re-ID.
**Our unified framework**
## Updates
***11/19/2021***
1. Memory dictionary update changed from bath hard update to momentum update. Because the bath hard update is sensitive to parameters, good results need to adjust many parameters, which is not robust enough.
2. Add the results of the InforMap clustering algorithm. Compared with the DBSCAN clustering algorithm, it can achieve better results. At the same time, we found through experiments that it is more robust on each data set.
## Notes
In the process of doing experiments, we found that some settings have a greater impact on the results. Share them here to prevent everyone from stepping on the pit when applying our method.
1. The dataloader sampler uses RandomMultipleGallerySampler, see the code implementation for details. At the same time, we also provide RandomMultipleGallerySamplerNoCam sampler, which can be used in non-ReID fields.
2. Add batch normalization to the final output layer of the network, see the code for details.
3. we obtain a total number of P × Z images in the mini
batch. P represents the number of categories, Z represents the number of instances of each category. mini batch = P x Z, P is set to 16, Z changes with the mini batch.
## Requirements
### Installation
```shell
git clone https://github.com/alibaba/cluster-contrast-reid.git
cd ClusterContrast
python setup.py develop
```
### Prepare Datasets
```shell
cd examples && mkdir data
```
Download the person datasets Market-1501,MSMT17,PersonX,DukeMTMC-reID and the vehicle datasets VeRi-776 from [aliyun](https://virutalbuy-public.oss-cn-hangzhou.aliyuncs.com/share/data.zip).
Then unzip them under the directory like
```
ClusterContrast/examples/data
├── market1501
│ └── Market-1501-v15.09.15
├── msmt17
│ └── MSMT17_V1
├── personx
│ └── PersonX
├── dukemtmcreid
│ └── DukeMTMC-reID
└── veri
└── VeRi
```
### Prepare ImageNet Pre-trained Models for IBN-Net
When training with the backbone of [IBN-ResNet](https://arxiv.org/abs/1807.09441), you need to download the ImageNet-pretrained model from this [link](https://drive.google.com/drive/folders/1thS2B8UOSBi_cJX6zRy6YYRwz_nVFI_S) and save it under the path of `examples/pretrained/`.
ImageNet-pretrained models for **ResNet-50** will be automatically downloaded in the python script.
## Training
We utilize 4 GTX-2080TI GPUs for training. For more parameter configuration, please check **`run_code.sh`**.
**examples:**
Market-1501:
1. Using DBSCAN:
```shell
CUDA_VISIBLE_DEVICES=0,1,2,3 python examples/cluster_contrast_train_usl.py -b 256 -a resnet50 -d market1501 --iters 200 --momentum 0.1 --eps 0.6 --num-instances 16
```
2. Using InfoMap:
```shell
CUDA_VISIBLE_DEVICES=0,1,2,3 python examples/cluster_contrast_train_usl_infomap.py -b 256 -a resnet50 -d market1501 --iters 200 --momentum 0.1 --eps 0.5 --k1 15 --k2 4 --num-instances 16
```
MSMT17:
1. Using DBSCAN:
```shell
CUDA_VISIBLE_DEVICES=0,1,2,3 python examples/cluster_contrast_train_usl.py -b 256 -a resnet50 -d msmt17 --iters 400 --momentum 0.1 --eps 0.6 --num-instances 16
```
2. Using InfoMap:
```shell
CUDA_VISIBLE_DEVICES=0,1,2,3 python examples/cluster_contrast_train_usl_infomap.py -b 256 -a resnet50 -d msmt17 --iters 400 --momentum 0.1 --eps 0.5 --k1 15 --k2 4 --num-instances 16
```
DukeMTMC-reID:
1. Using DBSCAN:
```shell
CUDA_VISIBLE_DEVICES=0,1,2,3 python examples/cluster_contrast_train_usl.py -b 256 -a resnet50 -d dukemtmcreid --iters 200 --momentum 0.1 --eps 0.6 --num-instances 16
```
2. Using InfoMap:
```shell
CUDA_VISIBLE_DEVICES=0,1,2,3 python examples/cluster_contrast_train_usl_infomap.py -b 256 -a resnet50 -d dukemtmcreid --iters 200 --momentum 0.1 --eps 0.5 --k1 15 --k2 4 --num-instances 16
```
VeRi-776
1. Using DBSCAN:
```shell
CUDA_VISIBLE_DEVICES=0,1,2,3 python examples/cluster_contrast_train_usl.py -b 256 -a resnet50 -d veri --iters 400 --momentum 0.1 --eps 0.6 --num-instances 16 --height 224 --width 224
```
2. Using InfoMap:
```shell
CUDA_VISIBLE_DEVICES=0,1,2,3 python examples/cluster_contrast_train_usl_infomap.py -b 256 -a resnet50 -d veri --iters 400 --momentum 0.1 --eps 0.5 --k1 15 --k2 4 --num-instances 16 --height 224 --width 224
```
## Evaluation
We utilize 1 GTX-2080TI GPU for testing. **Note that**
+ use `--width 128 --height 256` (default) for person datasets, and `--height 224 --width 224` for vehicle datasets;
+ use `-a resnet50` (default) for the backbone of ResNet-50, and `-a resnet_ibn50a` for the backbone of IBN-ResNet.
To evaluate the model, run:
```shell
CUDA_VISIBLE_DEVICES=0 \
python examples/test.py \
-d $DATASET --resume $PATH
```
**Some examples:**
```shell
### Market-1501 ###
CUDA_VISIBLE_DEVICES=0 \
python examples/test.py \
-d market1501 --resume logs/spcl_usl/market_resnet50/model_best.pth.tar
```
## Results
You can download the above models in the paper from [aliyun](https://virutalbuy-public.oss-cn-hangzhou.aliyuncs.com/share/cluster-contrast.zip)
## Citation
If you find this code useful for your research, please cite our paper
```
@article{dai2021cluster,
title={Cluster Contrast for Unsupervised Person Re-Identification},
author={Dai, Zuozhuo and Wang, Guangyuan and Zhu, Siyu and Yuan, Weihao and Tan, Ping},
journal={arXiv preprint arXiv:2103.11568},
year={2021}
}
```
# Acknowledgements
Thanks to Yixiao Ge for opening source of his excellent works [SpCL](https://github.com/yxgeee/SpCL).
================================================
FILE: clustercontrast/__init__.py
================================================
from __future__ import absolute_import
from . import datasets
from . import evaluation_metrics
from . import models
from . import utils
from . import evaluators
from . import trainers
__version__ = '0.1.0'
================================================
FILE: clustercontrast/datasets/__init__.py
================================================
from __future__ import absolute_import
import warnings
from .market1501 import Market1501
from .msmt17 import MSMT17
from .personx import PersonX
from .veri import VeRi
from .dukemtmcreid import DukeMTMCreID
__factory = {
'market1501': Market1501,
'msmt17': MSMT17,
'personx': PersonX,
'veri': VeRi,
'dukemtmcreid': DukeMTMCreID
}
def names():
return sorted(__factory.keys())
def create(name, root, *args, **kwargs):
"""
Create a dataset instance.
Parameters
----------
name : str
The dataset name.
root : str
The path to the dataset directory.
split_id : int, optional
The index of data split. Default: 0
num_val : int or float, optional
When int, it means the number of validation identities. When float,
it means the proportion of validation to all the trainval. Default: 100
download : bool, optional
If True, will download the dataset. Default: False
"""
if name not in __factory:
raise KeyError("Unknown dataset:", name)
return __factory[name](root, *args, **kwargs)
def get_dataset(name, root, *args, **kwargs):
warnings.warn("get_dataset is deprecated. Use create instead.")
return create(name, root, *args, **kwargs)
================================================
FILE: clustercontrast/datasets/dukemtmcreid.py
================================================
import glob
import os.path as osp
import re
from ..utils.data import BaseImageDataset
def process_dir(dir_path, relabel=False):
img_paths = glob.glob(osp.join(dir_path, "*.jpg"))
pattern = re.compile(r"([-\d]+)_c(\d)")
# get all identities
pid_container = set()
for img_path in img_paths:
pid, _ = map(int, pattern.search(img_path).groups())
if pid == -1:
continue
pid_container.add(pid)
pid2label = {pid: label for label, pid in enumerate(pid_container)}
data = []
for img_path in img_paths:
pid, camid = map(int, pattern.search(img_path).groups())
if (pid not in pid_container) or (pid == -1):
continue
assert 1 <= camid <= 8
camid -= 1
if relabel:
pid = pid2label[pid]
data.append((img_path, pid, camid))
return data
class DukeMTMCreID(BaseImageDataset):
"""DukeMTMC-reID.
Reference:
- Ristani et al. Performance Measures and a Data Set for Multi-Target,
Multi-Camera Tracking. ECCVW 2016.
- Zheng et al. Unlabeled Samples Generated by GAN Improve the Person
Re-identification Baseline in vitro. ICCV 2017.
URL: `<https://github.com/layumi/DukeMTMC-reID_evaluation>`_
Dataset statistics:
- identities: 1404 (train + query).
- images:16522 (train) + 2228 (query) + 17661 (gallery).
- cameras: 8.
"""
dataset_dir = "DukeMTMC-reID"
def __init__(self, root, verbose=True):
super(DukeMTMCreID, self).__init__()
self.root = osp.abspath(osp.expanduser(root))
self.dataset_dir = osp.join(self.root, self.dataset_dir)
self.train_dir = osp.join(self.dataset_dir, 'bounding_box_train')
self.query_dir = osp.join(self.dataset_dir, 'query')
self.gallery_dir = osp.join(self.dataset_dir, 'bounding_box_test')
train = process_dir(dir_path=self.train_dir, relabel=True)
query = process_dir(dir_path=self.query_dir, relabel=False)
gallery = process_dir(dir_path=self.gallery_dir, relabel=False)
self.train = train
self.query = query
self.gallery = gallery
self.num_train_pids, self.num_train_imgs, self.num_train_cams = self.get_imagedata_info(self.train)
self.num_query_pids, self.num_query_imgs, self.num_query_cams = self.get_imagedata_info(self.query)
self.num_gallery_pids, self.num_gallery_imgs, self.num_gallery_cams = self.get_imagedata_info(self.gallery)
def _check_before_run(self):
"""Check if all files are available before going deeper"""
if not osp.exists(self.dataset_dir):
raise RuntimeError("'{}' is not available".format(self.dataset_dir))
if not osp.exists(self.train_dir):
raise RuntimeError("'{}' is not available".format(self.train_dir))
if not osp.exists(self.query_dir):
raise RuntimeError("'{}' is not available".format(self.query_dir))
if not osp.exists(self.gallery_dir):
raise RuntimeError("'{}' is not available".format(self.gallery_dir))
================================================
FILE: clustercontrast/datasets/market1501.py
================================================
from __future__ import print_function, absolute_import
import os.path as osp
import glob
import re
from ..utils.data import BaseImageDataset
class Market1501(BaseImageDataset):
"""
Market1501
Reference:
Zheng et al. Scalable Person Re-identification: A Benchmark. ICCV 2015.
URL: http://www.liangzheng.org/Project/project_reid.html
Dataset statistics:
# identities: 1501 (+1 for background)
# images: 12936 (train) + 3368 (query) + 15913 (gallery)
"""
dataset_dir = 'Market-1501-v15.09.15'
def __init__(self, root, verbose=True, **kwargs):
super(Market1501, self).__init__()
self.dataset_dir = osp.join(root, self.dataset_dir)
self.train_dir = osp.join(self.dataset_dir, 'bounding_box_train')
self.query_dir = osp.join(self.dataset_dir, 'query')
self.gallery_dir = osp.join(self.dataset_dir, 'bounding_box_test')
self._check_before_run()
train = self._process_dir(self.train_dir, relabel=True)
query = self._process_dir(self.query_dir, relabel=False)
gallery = self._process_dir(self.gallery_dir, relabel=False)
if verbose:
print("=> Market1501 loaded")
self.print_dataset_statistics(train, query, gallery)
self.train = train
self.query = query
self.gallery = gallery
self.num_train_pids, self.num_train_imgs, self.num_train_cams = self.get_imagedata_info(self.train)
self.num_query_pids, self.num_query_imgs, self.num_query_cams = self.get_imagedata_info(self.query)
self.num_gallery_pids, self.num_gallery_imgs, self.num_gallery_cams = self.get_imagedata_info(self.gallery)
def _check_before_run(self):
"""Check if all files are available before going deeper"""
if not osp.exists(self.dataset_dir):
raise RuntimeError("'{}' is not available".format(self.dataset_dir))
if not osp.exists(self.train_dir):
raise RuntimeError("'{}' is not available".format(self.train_dir))
if not osp.exists(self.query_dir):
raise RuntimeError("'{}' is not available".format(self.query_dir))
if not osp.exists(self.gallery_dir):
raise RuntimeError("'{}' is not available".format(self.gallery_dir))
def _process_dir(self, dir_path, relabel=False):
img_paths = glob.glob(osp.join(dir_path, '*.jpg'))
pattern = re.compile(r'([-\d]+)_c(\d)')
pid_container = set()
for img_path in img_paths:
pid, _ = map(int, pattern.search(img_path).groups())
if pid == -1:
continue # junk images are just ignored
pid_container.add(pid)
pid2label = {pid: label for label, pid in enumerate(pid_container)}
dataset = []
for img_path in img_paths:
pid, camid = map(int, pattern.search(img_path).groups())
if pid == -1:
continue # junk images are just ignored
assert 0 <= pid <= 1501 # pid == 0 means background
assert 1 <= camid <= 6
camid -= 1 # index starts from 0
if relabel:
pid = pid2label[pid]
dataset.append((img_path, pid, camid))
return dataset
================================================
FILE: clustercontrast/datasets/msmt17.py
================================================
from __future__ import print_function, absolute_import
import os.path as osp
import glob
import re
from ..utils.data import BaseImageDataset
def _process_dir(dir_path, relabel=False):
img_paths = glob.glob(osp.join(dir_path, '*.jpg'))
pattern = re.compile(r'([-\d]+)_c(\d+)')
pid_container = set()
for img_path in img_paths:
pid, _ = map(int, pattern.search(img_path).groups())
if pid == -1:
continue # junk images are just ignored
pid_container.add(pid)
pid2label = {pid: label for label, pid in enumerate(pid_container)}
dataset = []
for img_path in img_paths:
pid, camid = map(int, pattern.search(img_path).groups())
if pid == -1:
continue # junk images are just ignored
assert 1 <= camid <= 15
camid -= 1 # index starts from 0
if relabel:
pid = pid2label[pid]
dataset.append((img_path, pid, camid))
return dataset
class MSMT17(BaseImageDataset):
dataset_dir = 'MSMT17_V1'
def __init__(self, root, verbose=True, **kwargs):
super(MSMT17, self).__init__()
self.dataset_dir = osp.join(root, self.dataset_dir)
self.train_dir = osp.join(self.dataset_dir, 'bounding_box_train')
self.query_dir = osp.join(self.dataset_dir, 'query')
self.gallery_dir = osp.join(self.dataset_dir, 'bounding_box_test')
self._check_before_run()
train = _process_dir(self.train_dir, relabel=True)
query = _process_dir(self.query_dir, relabel=False)
gallery = _process_dir(self.gallery_dir, relabel=False)
if verbose:
print("=> MSMT17_V1 loaded")
self.print_dataset_statistics(train, query, gallery)
self.train = train
self.query = query
self.gallery = gallery
self.num_train_pids, self.num_train_imgs, self.num_train_cams = self.get_imagedata_info(self.train)
self.num_query_pids, self.num_query_imgs, self.num_query_cams = self.get_imagedata_info(self.query)
self.num_gallery_pids, self.num_gallery_imgs, self.num_gallery_cams = self.get_imagedata_info(self.gallery)
def _check_before_run(self):
"""Check if all files are available before going deeper"""
if not osp.exists(self.dataset_dir):
raise RuntimeError("'{}' is not available".format(self.dataset_dir))
if not osp.exists(self.train_dir):
raise RuntimeError("'{}' is not available".format(self.train_dir))
if not osp.exists(self.query_dir):
raise RuntimeError("'{}' is not available".format(self.query_dir))
if not osp.exists(self.gallery_dir):
raise RuntimeError("'{}' is not available".format(self.gallery_dir))
================================================
FILE: clustercontrast/datasets/personx.py
================================================
from __future__ import print_function, absolute_import
import os.path as osp
import glob
import re
from ..utils.data import BaseImageDataset
class PersonX(BaseImageDataset):
"""
PersonX
Reference:
Sun et al. Dissecting Person Re-identification from the Viewpoint of Viewpoint. CVPR 2019.
Dataset statistics:
# identities: 1266
# images: 9840 (train) + 5136 (query) + 30816 (gallery)
"""
dataset_dir = 'PersonX'
def __init__(self, root, verbose=True, **kwargs):
super(PersonX, self).__init__()
self.dataset_dir = osp.join(root, self.dataset_dir)
self.train_dir = osp.join(self.dataset_dir, 'bounding_box_train')
self.query_dir = osp.join(self.dataset_dir, 'query')
self.gallery_dir = osp.join(self.dataset_dir, 'bounding_box_test')
self._check_before_run()
train = self._process_dir(self.train_dir, relabel=True)
query = self._process_dir(self.query_dir, relabel=False)
gallery = self._process_dir(self.gallery_dir, relabel=False)
if verbose:
print("=> PersonX loaded")
self.print_dataset_statistics(train, query, gallery)
self.train = train
self.query = query
self.gallery = gallery
self.num_train_pids, self.num_train_imgs, self.num_train_cams = self.get_imagedata_info(self.train)
self.num_query_pids, self.num_query_imgs, self.num_query_cams = self.get_imagedata_info(self.query)
self.num_gallery_pids, self.num_gallery_imgs, self.num_gallery_cams = self.get_imagedata_info(self.gallery)
def _check_before_run(self):
"""Check if all files are available before going deeper"""
if not osp.exists(self.dataset_dir):
raise RuntimeError("'{}' is not available".format(self.dataset_dir))
if not osp.exists(self.train_dir):
raise RuntimeError("'{}' is not available".format(self.train_dir))
if not osp.exists(self.query_dir):
raise RuntimeError("'{}' is not available".format(self.query_dir))
if not osp.exists(self.gallery_dir):
raise RuntimeError("'{}' is not available".format(self.gallery_dir))
def _process_dir(self, dir_path, relabel=False):
img_paths = glob.glob(osp.join(dir_path, '*.jpg'))
pattern = re.compile(r'([-\d]+)_c([-\d]+)')
cam2label = {3: 1, 4: 2, 8: 3, 10: 4, 11: 5, 12: 6}
pid_container = set()
for img_path in img_paths:
pid, _ = map(int, pattern.search(img_path).groups())
pid_container.add(pid)
pid2label = {pid: label for label, pid in enumerate(pid_container)}
dataset = []
for img_path in img_paths:
pid, camid = map(int, pattern.search(img_path).groups())
assert (camid in cam2label.keys())
camid = cam2label[camid]
camid -= 1 # index starts from 0
if relabel: pid = pid2label[pid]
dataset.append((img_path, pid, camid))
return dataset
================================================
FILE: clustercontrast/datasets/veri.py
================================================
from __future__ import absolute_import
from __future__ import division
from __future__ import print_function
import glob
import re
import os.path as osp
from ..utils.data import BaseImageDataset
class VeRi(BaseImageDataset):
"""
VeRi
Reference:
Liu, X., Liu, W., Ma, H., Fu, H.: Large-scale vehicle re-identification in urban surveillance videos. In: IEEE %
International Conference on Multimedia and Expo. (2016) accepted.
Dataset statistics:
# identities: 776 vehicles(576 for training and 200 for testing)
# images: 37778 (train) + 11579 (query)
"""
dataset_dir = 'VeRi'
def __init__(self, root, verbose=True, **kwargs):
super(VeRi, self).__init__()
self.dataset_dir = osp.join(root, self.dataset_dir)
self.train_dir = osp.join(self.dataset_dir, 'image_train')
self.query_dir = osp.join(self.dataset_dir, 'image_query')
self.gallery_dir = osp.join(self.dataset_dir, 'image_test')
self.check_before_run()
train = self.process_dir(self.train_dir, relabel=True)
query = self.process_dir(self.query_dir, relabel=False)
gallery = self.process_dir(self.gallery_dir, relabel=False)
if verbose:
print('=> VeRi loaded')
self.print_dataset_statistics(train, query, gallery)
self.train = train
self.query = query
self.gallery = gallery
self.num_train_pids, self.num_train_imgs, self.num_train_cams = self.get_imagedata_info(self.train)
self.num_query_pids, self.num_query_imgs, self.num_query_cams = self.get_imagedata_info(self.query)
self.num_gallery_pids, self.num_gallery_imgs, self.num_gallery_cams = self.get_imagedata_info(self.gallery)
def check_before_run(self):
"""Check if all files are available before going deeper"""
if not osp.exists(self.dataset_dir):
raise RuntimeError('"{}" is not available'.format(self.dataset_dir))
if not osp.exists(self.train_dir):
raise RuntimeError('"{}" is not available'.format(self.train_dir))
if not osp.exists(self.query_dir):
raise RuntimeError('"{}" is not available'.format(self.query_dir))
if not osp.exists(self.gallery_dir):
raise RuntimeError('"{}" is not available'.format(self.gallery_dir))
def process_dir(self, dir_path, relabel=False):
img_paths = glob.glob(osp.join(dir_path, '*.jpg'))
pattern = re.compile(r'([-\d]+)_c([-\d]+)')
pid_container = set()
for img_path in img_paths:
pid, _ = map(int, pattern.search(img_path).groups())
if pid == -1:
continue # junk images are just ignored
pid_container.add(pid)
pid2label = {pid: label for label, pid in enumerate(pid_container)}
dataset = []
for img_path in img_paths:
pid, camid = map(int, pattern.search(img_path).groups())
if pid == -1:
continue # junk images are just ignored
assert 0 <= pid <= 776 # pid == 0 means background
assert 1 <= camid <= 20
camid -= 1 # index starts from 0
if relabel:
pid = pid2label[pid]
dataset.append((img_path, pid, camid))
return dataset
================================================
FILE: clustercontrast/evaluation_metrics/__init__.py
================================================
from __future__ import absolute_import
from .classification import accuracy
from .ranking import cmc, mean_ap
__all__ = [
'accuracy',
'cmc',
'mean_ap'
]
================================================
FILE: clustercontrast/evaluation_metrics/classification.py
================================================
from __future__ import absolute_import
import torch
from ..utils import to_torch
def accuracy(output, target, topk=(1,)):
with torch.no_grad():
output, target = to_torch(output), to_torch(target)
maxk = max(topk)
batch_size = target.size(0)
_, pred = output.topk(maxk, 1, True, True)
pred = pred.t()
correct = pred.eq(target.view(1, -1).expand_as(pred))
ret = []
for k in topk:
correct_k = correct[:k].view(-1).float().sum(dim=0, keepdim=True)
ret.append(correct_k.mul_(1. / batch_size))
return ret
================================================
FILE: clustercontrast/evaluation_metrics/ranking.py
================================================
from __future__ import absolute_import
from collections import defaultdict
import numpy as np
from sklearn.metrics import average_precision_score
from ..utils import to_numpy
def _unique_sample(ids_dict, num):
mask = np.zeros(num, dtype=np.bool)
for _, indices in ids_dict.items():
i = np.random.choice(indices)
mask[i] = True
return mask
def cmc(distmat, query_ids=None, gallery_ids=None,
query_cams=None, gallery_cams=None, topk=100,
separate_camera_set=False,
single_gallery_shot=False,
first_match_break=False):
distmat = to_numpy(distmat)
m, n = distmat.shape
# Fill up default values
if query_ids is None:
query_ids = np.arange(m)
if gallery_ids is None:
gallery_ids = np.arange(n)
if query_cams is None:
query_cams = np.zeros(m).astype(np.int32)
if gallery_cams is None:
gallery_cams = np.ones(n).astype(np.int32)
# Ensure numpy array
query_ids = np.asarray(query_ids)
gallery_ids = np.asarray(gallery_ids)
query_cams = np.asarray(query_cams)
gallery_cams = np.asarray(gallery_cams)
# Sort and find correct matches
indices = np.argsort(distmat, axis=1)
matches = (gallery_ids[indices] == query_ids[:, np.newaxis])
# Compute CMC for each query
ret = np.zeros(topk)
num_valid_queries = 0
for i in range(m):
# Filter out the same id and same camera
valid = ((gallery_ids[indices[i]] != query_ids[i]) |
(gallery_cams[indices[i]] != query_cams[i]))
if separate_camera_set:
# Filter out samples from same camera
valid &= (gallery_cams[indices[i]] != query_cams[i])
if not np.any(matches[i, valid]): continue
if single_gallery_shot:
repeat = 10
gids = gallery_ids[indices[i][valid]]
inds = np.where(valid)[0]
ids_dict = defaultdict(list)
for j, x in zip(inds, gids):
ids_dict[x].append(j)
else:
repeat = 1
for _ in range(repeat):
if single_gallery_shot:
# Randomly choose one instance for each id
sampled = (valid & _unique_sample(ids_dict, len(valid)))
index = np.nonzero(matches[i, sampled])[0]
else:
index = np.nonzero(matches[i, valid])[0]
delta = 1. / (len(index) * repeat)
for j, k in enumerate(index):
if k - j >= topk: break
if first_match_break:
ret[k - j] += 1
break
ret[k - j] += delta
num_valid_queries += 1
if num_valid_queries == 0:
raise RuntimeError("No valid query")
return ret.cumsum() / num_valid_queries
def mean_ap(distmat, query_ids=None, gallery_ids=None,
query_cams=None, gallery_cams=None):
distmat = to_numpy(distmat)
m, n = distmat.shape
# Fill up default values
if query_ids is None:
query_ids = np.arange(m)
if gallery_ids is None:
gallery_ids = np.arange(n)
if query_cams is None:
query_cams = np.zeros(m).astype(np.int32)
if gallery_cams is None:
gallery_cams = np.ones(n).astype(np.int32)
# Ensure numpy array
query_ids = np.asarray(query_ids)
gallery_ids = np.asarray(gallery_ids)
query_cams = np.asarray(query_cams)
gallery_cams = np.asarray(gallery_cams)
# Sort and find correct matches
indices = np.argsort(distmat, axis=1)
matches = (gallery_ids[indices] == query_ids[:, np.newaxis])
# Compute AP for each query
aps = []
for i in range(m):
# Filter out the same id and same camera
valid = ((gallery_ids[indices[i]] != query_ids[i]) |
(gallery_cams[indices[i]] != query_cams[i]))
y_true = matches[i, valid]
y_score = -distmat[i][indices[i]][valid]
if not np.any(y_true): continue
aps.append(average_precision_score(y_true, y_score))
if len(aps) == 0:
raise RuntimeError("No valid query")
return np.mean(aps)
================================================
FILE: clustercontrast/evaluators.py
================================================
from __future__ import print_function, absolute_import
import time
import collections
from collections import OrderedDict
import numpy as np
import torch
import random
import copy
from .evaluation_metrics import cmc, mean_ap
from .utils.meters import AverageMeter
from .utils.rerank import re_ranking
from .utils import to_torch
def extract_cnn_feature(model, inputs):
inputs = to_torch(inputs).cuda()
outputs = model(inputs)
outputs = outputs.data.cpu()
return outputs
def extract_features(model, data_loader, print_freq=50):
model.eval()
batch_time = AverageMeter()
data_time = AverageMeter()
features = OrderedDict()
labels = OrderedDict()
end = time.time()
with torch.no_grad():
for i, (imgs, fnames, pids, _, _) in enumerate(data_loader):
data_time.update(time.time() - end)
outputs = extract_cnn_feature(model, imgs)
for fname, output, pid in zip(fnames, outputs, pids):
features[fname] = output
labels[fname] = pid
batch_time.update(time.time() - end)
end = time.time()
if (i + 1) % print_freq == 0:
print('Extract Features: [{}/{}]\t'
'Time {:.3f} ({:.3f})\t'
'Data {:.3f} ({:.3f})\t'
.format(i + 1, len(data_loader),
batch_time.val, batch_time.avg,
data_time.val, data_time.avg))
return features, labels
def pairwise_distance(features, query=None, gallery=None):
if query is None and gallery is None:
n = len(features)
x = torch.cat(list(features.values()))
x = x.view(n, -1)
dist_m = torch.pow(x, 2).sum(dim=1, keepdim=True) * 2
dist_m = dist_m.expand(n, n) - 2 * torch.mm(x, x.t())
return dist_m
x = torch.cat([features[f].unsqueeze(0) for f, _, _ in query], 0)
y = torch.cat([features[f].unsqueeze(0) for f, _, _ in gallery], 0)
m, n = x.size(0), y.size(0)
x = x.view(m, -1)
y = y.view(n, -1)
dist_m = torch.pow(x, 2).sum(dim=1, keepdim=True).expand(m, n) + \
torch.pow(y, 2).sum(dim=1, keepdim=True).expand(n, m).t()
dist_m.addmm_(1, -2, x, y.t())
return dist_m, x.numpy(), y.numpy()
def evaluate_all(query_features, gallery_features, distmat, query=None, gallery=None,
query_ids=None, gallery_ids=None,
query_cams=None, gallery_cams=None,
cmc_topk=(1, 5, 10), cmc_flag=False):
if query is not None and gallery is not None:
query_ids = [pid for _, pid, _ in query]
gallery_ids = [pid for _, pid, _ in gallery]
query_cams = [cam for _, _, cam in query]
gallery_cams = [cam for _, _, cam in gallery]
else:
assert (query_ids is not None and gallery_ids is not None
and query_cams is not None and gallery_cams is not None)
# Compute mean AP
mAP = mean_ap(distmat, query_ids, gallery_ids, query_cams, gallery_cams)
print('Mean AP: {:4.1%}'.format(mAP))
if (not cmc_flag):
return mAP
cmc_configs = {
'market1501': dict(separate_camera_set=False,
single_gallery_shot=False,
first_match_break=True),}
cmc_scores = {name: cmc(distmat, query_ids, gallery_ids,
query_cams, gallery_cams, **params)
for name, params in cmc_configs.items()}
print('CMC Scores:')
for k in cmc_topk:
print(' top-{:<4}{:12.1%}'.format(k, cmc_scores['market1501'][k-1]))
return cmc_scores['market1501'], mAP
class Evaluator(object):
def __init__(self, model):
super(Evaluator, self).__init__()
self.model = model
def evaluate(self, data_loader, query, gallery, cmc_flag=False, rerank=False):
features, _ = extract_features(self.model, data_loader)
distmat, query_features, gallery_features = pairwise_distance(features, query, gallery)
results = evaluate_all(query_features, gallery_features, distmat, query=query, gallery=gallery, cmc_flag=cmc_flag)
if (not rerank):
return results
print('Applying person re-ranking ...')
distmat_qq, _, _ = pairwise_distance(features, query, query)
distmat_gg, _, _ = pairwise_distance(features, gallery, gallery)
distmat = re_ranking(distmat.numpy(), distmat_qq.numpy(), distmat_gg.numpy())
return evaluate_all(query_features, gallery_features, distmat, query=query, gallery=gallery, cmc_flag=cmc_flag)
================================================
FILE: clustercontrast/models/__init__.py
================================================
from __future__ import absolute_import
from .resnet import *
from .resnet_ibn import *
__factory = {
'resnet18': resnet18,
'resnet34': resnet34,
'resnet50': resnet50,
'resnet101': resnet101,
'resnet152': resnet152,
'resnet_ibn50a': resnet_ibn50a,
'resnet_ibn101a': resnet_ibn101a,
}
def names():
return sorted(__factory.keys())
def create(name, *args, **kwargs):
"""
Create a model instance.
Parameters
----------
name : str
Model name. Can be one of 'inception', 'resnet18', 'resnet34',
'resnet50', 'resnet101', and 'resnet152'.
pretrained : bool, optional
Only applied for 'resnet*' models. If True, will use ImageNet pretrained
model. Default: True
cut_at_pooling : bool, optional
If True, will cut the model before the last global pooling layer and
ignore the remaining kwargs. Default: False
num_features : int, optional
If positive, will append a Linear layer after the global pooling layer,
with this number of output units, followed by a BatchNorm layer.
Otherwise these layers will not be appended. Default: 256 for
'inception', 0 for 'resnet*'
norm : bool, optional
If True, will normalize the feature to be unit L2-norm for each sample.
Otherwise will append a ReLU layer after the above Linear layer if
num_features > 0. Default: False
dropout : float, optional
If positive, will append a Dropout layer with this dropout rate.
Default: 0
num_classes : int, optional
If positive, will append a Linear layer at the end as the classifier
with this number of output units. Default: 0
"""
if name not in __factory:
raise KeyError("Unknown model:", name)
return __factory[name](*args, **kwargs)
================================================
FILE: clustercontrast/models/cm.py
================================================
import collections
import numpy as np
from abc import ABC
import torch
import torch.nn.functional as F
from torch import nn, autograd
class CM(autograd.Function):
@staticmethod
def forward(ctx, inputs, targets, features, momentum):
ctx.features = features
ctx.momentum = momentum
ctx.save_for_backward(inputs, targets)
outputs = inputs.mm(ctx.features.t())
return outputs
@staticmethod
def backward(ctx, grad_outputs):
inputs, targets = ctx.saved_tensors
grad_inputs = None
if ctx.needs_input_grad[0]:
grad_inputs = grad_outputs.mm(ctx.features)
# momentum update
for x, y in zip(inputs, targets):
ctx.features[y] = ctx.momentum * ctx.features[y] + (1. - ctx.momentum) * x
ctx.features[y] /= ctx.features[y].norm()
return grad_inputs, None, None, None
def cm(inputs, indexes, features, momentum=0.5):
return CM.apply(inputs, indexes, features, torch.Tensor([momentum]).to(inputs.device))
class CM_Hard(autograd.Function):
@staticmethod
def forward(ctx, inputs, targets, features, momentum):
ctx.features = features
ctx.momentum = momentum
ctx.save_for_backward(inputs, targets)
outputs = inputs.mm(ctx.features.t())
return outputs
@staticmethod
def backward(ctx, grad_outputs):
inputs, targets = ctx.saved_tensors
grad_inputs = None
if ctx.needs_input_grad[0]:
grad_inputs = grad_outputs.mm(ctx.features)
batch_centers = collections.defaultdict(list)
for instance_feature, index in zip(inputs, targets.tolist()):
batch_centers[index].append(instance_feature)
for index, features in batch_centers.items():
distances = []
for feature in features:
distance = feature.unsqueeze(0).mm(ctx.features[index].unsqueeze(0).t())[0][0]
distances.append(distance.cpu().numpy())
median = np.argmin(np.array(distances))
ctx.features[index] = ctx.features[index] * ctx.momentum + (1 - ctx.momentum) * features[median]
ctx.features[index] /= ctx.features[index].norm()
return grad_inputs, None, None, None
def cm_hard(inputs, indexes, features, momentum=0.5):
return CM_Hard.apply(inputs, indexes, features, torch.Tensor([momentum]).to(inputs.device))
class ClusterMemory(nn.Module, ABC):
def __init__(self, num_features, num_samples, temp=0.05, momentum=0.2, use_hard=False):
super(ClusterMemory, self).__init__()
self.num_features = num_features
self.num_samples = num_samples
self.momentum = momentum
self.temp = temp
self.use_hard = use_hard
self.register_buffer('features', torch.zeros(num_samples, num_features))
def forward(self, inputs, targets):
inputs = F.normalize(inputs, dim=1).cuda()
if self.use_hard:
outputs = cm_hard(inputs, targets, self.features, self.momentum)
else:
outputs = cm(inputs, targets, self.features, self.momentum)
outputs /= self.temp
loss = F.cross_entropy(outputs, targets)
return loss
================================================
FILE: clustercontrast/models/dsbn.py
================================================
import torch
import torch.nn as nn
# Domain-specific BatchNorm
class DSBN2d(nn.Module):
def __init__(self, planes):
super(DSBN2d, self).__init__()
self.num_features = planes
self.BN_S = nn.BatchNorm2d(planes)
self.BN_T = nn.BatchNorm2d(planes)
def forward(self, x):
if (not self.training):
return self.BN_T(x)
bs = x.size(0)
assert (bs%2==0)
split = torch.split(x, int(bs/2), 0)
out1 = self.BN_S(split[0].contiguous())
out2 = self.BN_T(split[1].contiguous())
out = torch.cat((out1, out2), 0)
return out
class DSBN1d(nn.Module):
def __init__(self, planes):
super(DSBN1d, self).__init__()
self.num_features = planes
self.BN_S = nn.BatchNorm1d(planes)
self.BN_T = nn.BatchNorm1d(planes)
def forward(self, x):
if (not self.training):
return self.BN_T(x)
bs = x.size(0)
assert (bs%2==0)
split = torch.split(x, int(bs/2), 0)
out1 = self.BN_S(split[0].contiguous())
out2 = self.BN_T(split[1].contiguous())
out = torch.cat((out1, out2), 0)
return out
def convert_dsbn(model):
for _, (child_name, child) in enumerate(model.named_children()):
assert(not next(model.parameters()).is_cuda)
if isinstance(child, nn.BatchNorm2d):
m = DSBN2d(child.num_features)
m.BN_S.load_state_dict(child.state_dict())
m.BN_T.load_state_dict(child.state_dict())
setattr(model, child_name, m)
elif isinstance(child, nn.BatchNorm1d):
m = DSBN1d(child.num_features)
m.BN_S.load_state_dict(child.state_dict())
m.BN_T.load_state_dict(child.state_dict())
setattr(model, child_name, m)
else:
convert_dsbn(child)
def convert_bn(model, use_target=True):
for _, (child_name, child) in enumerate(model.named_children()):
assert(not next(model.parameters()).is_cuda)
if isinstance(child, DSBN2d):
m = nn.BatchNorm2d(child.num_features)
if use_target:
m.load_state_dict(child.BN_T.state_dict())
else:
m.load_state_dict(child.BN_S.state_dict())
setattr(model, child_name, m)
elif isinstance(child, DSBN1d):
m = nn.BatchNorm1d(child.num_features)
if use_target:
m.load_state_dict(child.BN_T.state_dict())
else:
m.load_state_dict(child.BN_S.state_dict())
setattr(model, child_name, m)
else:
convert_bn(child, use_target=use_target)
================================================
FILE: clustercontrast/models/kmeans.py
================================================
# Written by Yixiao Ge
import warnings
import faiss
import torch
from ..utils import to_numpy, to_torch
__all__ = ["label_generator_kmeans"]
@torch.no_grad()
def label_generator_kmeans(features, num_classes=500, cuda=True):
assert num_classes, "num_classes for kmeans is null"
# k-means cluster by faiss
cluster = faiss.Kmeans(
features.size(-1), num_classes, niter=300, verbose=True, gpu=cuda
)
cluster.train(to_numpy(features))
_, labels = cluster.index.search(to_numpy(features), 1)
labels = labels.reshape(-1)
centers = to_torch(cluster.centroids).float()
# labels = to_torch(labels).long()
# k-means does not have outlier points
assert not (-1 in labels)
return labels, centers, num_classes, None
================================================
FILE: clustercontrast/models/pooling.py
================================================
# Credit to https://github.com/JDAI-CV/fast-reid/blob/master/fastreid/layers/pooling.py
from abc import ABC
import torch
import torch.nn.functional as F
from torch import nn
__all__ = [
"GeneralizedMeanPoolingPFpn",
"GeneralizedMeanPoolingList",
"GeneralizedMeanPoolingP",
"AdaptiveAvgMaxPool2d",
"FastGlobalAvgPool2d",
"avg_pooling",
"max_pooling",
]
class GeneralizedMeanPoolingList(nn.Module, ABC):
r"""Applies a 2D power-average adaptive pooling over an input signal composed of
several input planes.
The function computed is: :math:`f(X) = pow(sum(pow(X, p)), 1/p)`
- At p = infinity, one gets Max Pooling
- At p = 1, one gets Average Pooling
The output is of size H x W, for any input size.
The number of output features is equal to the number of input planes.
Args:
output_size: the target output size of the image of the form H x W.
Can be a tuple (H, W) or a single H for a square image H x H
H and W can be either a ``int``, or ``None`` which means the size
will be the same as that of the input.
"""
def __init__(self, output_size=1, eps=1e-6):
super(GeneralizedMeanPoolingList, self).__init__()
self.output_size = output_size
self.eps = eps
def forward(self, x_list):
outs = []
for x in x_list:
x = x.clamp(min=self.eps)
out = torch.nn.functional.adaptive_avg_pool2d(x, self.output_size)
outs.append(out)
return torch.stack(outs, -1).mean(-1)
def __repr__(self):
return (
self.__class__.__name__
+ "("
+ "output_size="
+ str(self.output_size)
+ ")"
)
class GeneralizedMeanPooling(nn.Module, ABC):
r"""Applies a 2D power-average adaptive pooling over an input signal composed of
several input planes.
The function computed is: :math:`f(X) = pow(sum(pow(X, p)), 1/p)`
- At p = infinity, one gets Max Pooling
- At p = 1, one gets Average Pooling
The output is of size H x W, for any input size.
The number of output features is equal to the number of input planes.
Args:
output_size: the target output size of the image of the form H x W.
Can be a tuple (H, W) or a single H for a square image H x H
H and W can be either a ``int``, or ``None`` which means the size
will be the same as that of the input.
"""
def __init__(self, norm, output_size=1, eps=1e-6):
super(GeneralizedMeanPooling, self).__init__()
assert norm > 0
self.p = float(norm)
self.output_size = output_size
self.eps = eps
def forward(self, x):
x = x.clamp(min=self.eps).pow(self.p)
return torch.nn.functional.adaptive_avg_pool2d(x, self.output_size).pow(
1.0 / self.p
)
def __repr__(self):
return (
self.__class__.__name__
+ "("
+ str(self.p)
+ ", "
+ "output_size="
+ str(self.output_size)
+ ")"
)
class GeneralizedMeanPoolingP(GeneralizedMeanPooling, ABC):
""" Same, but norm is trainable
"""
def __init__(self, norm=3, output_size=1, eps=1e-6):
super(GeneralizedMeanPoolingP, self).__init__(norm, output_size, eps)
self.p = nn.Parameter(torch.ones(1) * norm)
class GeneralizedMeanPoolingFpn(nn.Module, ABC):
r"""Applies a 2D power-average adaptive pooling over an input signal composed of
several input planes.
The function computed is: :math:`f(X) = pow(sum(pow(X, p)), 1/p)`
- At p = infinity, one gets Max Pooling
- At p = 1, one gets Average Pooling
The output is of size H x W, for any input size.
The number of output features is equal to the number of input planes.
Args:
output_size: the target output size of the image of the form H x W.
Can be a tuple (H, W) or a single H for a square image H x H
H and W can be either a ``int``, or ``None`` which means the size
will be the same as that of the input.
"""
def __init__(self, norm, output_size=1, eps=1e-6):
super(GeneralizedMeanPoolingFpn, self).__init__()
assert norm > 0
self.p = float(norm)
self.output_size = output_size
self.eps = eps
def forward(self, x_lists):
outs = []
for x in x_lists:
x = x.clamp(min=self.eps).pow(self.p)
out = torch.nn.functional.adaptive_avg_pool2d(x, self.output_size).pow(
1.0 / self.p
)
outs.append(out)
return torch.cat(outs, 1)
def __repr__(self):
return (
self.__class__.__name__
+ "("
+ str(self.p)
+ ", "
+ "output_size="
+ str(self.output_size)
+ ")"
)
class GeneralizedMeanPoolingPFpn(GeneralizedMeanPoolingFpn, ABC):
""" Same, but norm is trainable
"""
def __init__(self, norm=3, output_size=1, eps=1e-6):
super(GeneralizedMeanPoolingPFpn, self).__init__(norm, output_size, eps)
self.p = nn.Parameter(torch.ones(1) * norm)
class AdaptiveAvgMaxPool2d(nn.Module, ABC):
def __init__(self):
super(AdaptiveAvgMaxPool2d, self).__init__()
self.avgpool = FastGlobalAvgPool2d()
def forward(self, x):
x_avg = self.avgpool(x, self.output_size)
x_max = F.adaptive_max_pool2d(x, 1)
x = x_max + x_avg
return x
class FastGlobalAvgPool2d(nn.Module, ABC):
def __init__(self, flatten=False):
super(FastGlobalAvgPool2d, self).__init__()
self.flatten = flatten
def forward(self, x):
if self.flatten:
in_size = x.size()
return x.view((in_size[0], in_size[1], -1)).mean(dim=2)
else:
return (
x.view(x.size(0), x.size(1), -1)
.mean(-1)
.view(x.size(0), x.size(1), 1, 1)
)
def avg_pooling():
return nn.AdaptiveAvgPool2d(1)
# return FastGlobalAvgPool2d()
def max_pooling():
return nn.AdaptiveMaxPool2d(1)
class Flatten(nn.Module):
def forward(self, input):
return input.view(input.size(0), -1)
__pooling_factory = {
"avg": avg_pooling,
"max": max_pooling,
"gem": GeneralizedMeanPoolingP,
"gemFpn": GeneralizedMeanPoolingPFpn,
"gemList": GeneralizedMeanPoolingList,
"avg+max": AdaptiveAvgMaxPool2d,
}
def pooling_names():
return sorted(__pooling_factory.keys())
def build_pooling_layer(name):
"""
Create a pooling layer.
Parameters
----------
name : str
The backbone name.
"""
if name not in __pooling_factory:
raise KeyError("Unknown pooling layer:", name)
return __pooling_factory[name]()
================================================
FILE: clustercontrast/models/resnet.py
================================================
from __future__ import absolute_import
from torch import nn
from torch.nn import functional as F
from torch.nn import init
import torchvision
import torch
from .pooling import build_pooling_layer
__all__ = ['ResNet', 'resnet18', 'resnet34', 'resnet50', 'resnet101',
'resnet152']
class ResNet(nn.Module):
__factory = {
18: torchvision.models.resnet18,
34: torchvision.models.resnet34,
50: torchvision.models.resnet50,
101: torchvision.models.resnet101,
152: torchvision.models.resnet152,
}
def __init__(self, depth, pretrained=True, cut_at_pooling=False,
num_features=0, norm=False, dropout=0, num_classes=0, pooling_type='avg'):
super(ResNet, self).__init__()
self.pretrained = pretrained
self.depth = depth
self.cut_at_pooling = cut_at_pooling
# Construct base (pretrained) resnet
if depth not in ResNet.__factory:
raise KeyError("Unsupported depth:", depth)
resnet = ResNet.__factory[depth](pretrained=pretrained)
resnet.layer4[0].conv2.stride = (1, 1)
resnet.layer4[0].downsample[0].stride = (1, 1)
self.base = nn.Sequential(
resnet.conv1, resnet.bn1, resnet.relu, resnet.maxpool,
resnet.layer1, resnet.layer2, resnet.layer3, resnet.layer4)
self.gap = build_pooling_layer(pooling_type)
if not self.cut_at_pooling:
self.num_features = num_features
self.norm = norm
self.dropout = dropout
self.has_embedding = num_features > 0
self.num_classes = num_classes
out_planes = resnet.fc.in_features
# Append new layers
if self.has_embedding:
self.feat = nn.Linear(out_planes, self.num_features)
self.feat_bn = nn.BatchNorm1d(self.num_features)
init.kaiming_normal_(self.feat.weight, mode='fan_out')
init.constant_(self.feat.bias, 0)
else:
# Change the num_features to CNN output channels
self.num_features = out_planes
self.feat_bn = nn.BatchNorm1d(self.num_features)
self.feat_bn.bias.requires_grad_(False)
if self.dropout > 0:
self.drop = nn.Dropout(self.dropout)
if self.num_classes > 0:
self.classifier = nn.Linear(self.num_features, self.num_classes, bias=False)
init.normal_(self.classifier.weight, std=0.001)
init.constant_(self.feat_bn.weight, 1)
init.constant_(self.feat_bn.bias, 0)
if not pretrained:
self.reset_params()
def forward(self, x):
bs = x.size(0)
x = self.base(x)
x = self.gap(x)
x = x.view(x.size(0), -1)
if self.cut_at_pooling:
return x
if self.has_embedding:
bn_x = self.feat_bn(self.feat(x))
else:
bn_x = self.feat_bn(x)
if (self.training is False):
bn_x = F.normalize(bn_x)
return bn_x
if self.norm:
bn_x = F.normalize(bn_x)
elif self.has_embedding:
bn_x = F.relu(bn_x)
if self.dropout > 0:
bn_x = self.drop(bn_x)
if self.num_classes > 0:
prob = self.classifier(bn_x)
else:
return bn_x
return prob
def reset_params(self):
for m in self.modules():
if isinstance(m, nn.Conv2d):
init.kaiming_normal_(m.weight, mode='fan_out')
if m.bias is not None:
init.constant_(m.bias, 0)
elif isinstance(m, nn.BatchNorm2d):
init.constant_(m.weight, 1)
init.constant_(m.bias, 0)
elif isinstance(m, nn.BatchNorm1d):
init.constant_(m.weight, 1)
init.constant_(m.bias, 0)
elif isinstance(m, nn.Linear):
init.normal_(m.weight, std=0.001)
if m.bias is not None:
init.constant_(m.bias, 0)
def resnet18(**kwargs):
return ResNet(18, **kwargs)
def resnet34(**kwargs):
return ResNet(34, **kwargs)
def resnet50(**kwargs):
return ResNet(50, **kwargs)
def resnet101(**kwargs):
return ResNet(101, **kwargs)
def resnet152(**kwargs):
return ResNet(152, **kwargs)
================================================
FILE: clustercontrast/models/resnet_ibn.py
================================================
from __future__ import absolute_import
from torch import nn
from torch.nn import functional as F
from torch.nn import init
import torchvision
import torch
from .pooling import build_pooling_layer
from .resnet_ibn_a import resnet50_ibn_a, resnet101_ibn_a
__all__ = ['ResNetIBN', 'resnet_ibn50a', 'resnet_ibn101a']
class ResNetIBN(nn.Module):
__factory = {
'50a': resnet50_ibn_a,
'101a': resnet101_ibn_a
}
def __init__(self, depth, pretrained=True, cut_at_pooling=False,
num_features=0, norm=False, dropout=0, num_classes=0, pooling_type='avg'):
super(ResNetIBN, self).__init__()
self.depth = depth
self.pretrained = pretrained
self.cut_at_pooling = cut_at_pooling
resnet = ResNetIBN.__factory[depth](pretrained=pretrained)
resnet.layer4[0].conv2.stride = (1, 1)
resnet.layer4[0].downsample[0].stride = (1, 1)
self.base = nn.Sequential(
resnet.conv1, resnet.bn1, resnet.relu, resnet.maxpool,
resnet.layer1, resnet.layer2, resnet.layer3, resnet.layer4)
self.gap = build_pooling_layer(pooling_type)
if not self.cut_at_pooling:
self.num_features = num_features
self.norm = norm
self.dropout = dropout
self.has_embedding = num_features > 0
self.num_classes = num_classes
out_planes = resnet.fc.in_features
# Append new layers
if self.has_embedding:
self.feat = nn.Linear(out_planes, self.num_features)
self.feat_bn = nn.BatchNorm1d(self.num_features)
init.kaiming_normal_(self.feat.weight, mode='fan_out')
init.constant_(self.feat.bias, 0)
else:
# Change the num_features to CNN output channels
self.num_features = out_planes
self.feat_bn = nn.BatchNorm1d(self.num_features)
self.feat_bn.bias.requires_grad_(False)
if self.dropout > 0:
self.drop = nn.Dropout(self.dropout)
if self.num_classes > 0:
self.classifier = nn.Linear(self.num_features, self.num_classes, bias=False)
init.normal_(self.classifier.weight, std=0.001)
init.constant_(self.feat_bn.weight, 1)
init.constant_(self.feat_bn.bias, 0)
if not pretrained:
self.reset_params()
def forward(self, x):
x = self.base(x)
x = self.gap(x)
x = x.view(x.size(0), -1)
if self.cut_at_pooling:
return x
if self.has_embedding:
bn_x = self.feat_bn(self.feat(x))
else:
bn_x = self.feat_bn(x)
if self.training is False:
bn_x = F.normalize(bn_x)
return bn_x
if self.norm:
bn_x = F.normalize(bn_x)
elif self.has_embedding:
bn_x = F.relu(bn_x)
if self.dropout > 0:
bn_x = self.drop(bn_x)
if self.num_classes > 0:
prob = self.classifier(bn_x)
else:
return bn_x
return prob
def reset_params(self):
for m in self.modules():
if isinstance(m, nn.Conv2d):
init.kaiming_normal_(m.weight, mode='fan_out')
if m.bias is not None:
init.constant_(m.bias, 0)
elif isinstance(m, nn.BatchNorm2d):
init.constant_(m.weight, 1)
init.constant_(m.bias, 0)
elif isinstance(m, nn.BatchNorm1d):
init.constant_(m.weight, 1)
init.constant_(m.bias, 0)
elif isinstance(m, nn.Linear):
init.normal_(m.weight, std=0.001)
if m.bias is not None:
init.constant_(m.bias, 0)
def resnet_ibn50a(**kwargs):
return ResNetIBN('50a', **kwargs)
def resnet_ibn101a(**kwargs):
return ResNetIBN('101a', **kwargs)
================================================
FILE: clustercontrast/models/resnet_ibn_a.py
================================================
import torch
import torch.nn as nn
import math
import torch.utils.model_zoo as model_zoo
__all__ = ['ResNet', 'resnet50_ibn_a', 'resnet101_ibn_a']
model_urls = {
'ibn_resnet50a': './examples/pretrained/resnet50_ibn_a.pth.tar',
'ibn_resnet101a': './examples/pretrained/resnet101_ibn_a.pth.tar',
}
def conv3x3(in_planes, out_planes, stride=1):
"3x3 convolution with padding"
return nn.Conv2d(in_planes, out_planes, kernel_size=3, stride=stride,
padding=1, bias=False)
class BasicBlock(nn.Module):
expansion = 1
def __init__(self, inplanes, planes, stride=1, downsample=None):
super(BasicBlock, self).__init__()
self.conv1 = conv3x3(inplanes, planes, stride)
self.bn1 = nn.BatchNorm2d(planes)
self.relu = nn.ReLU(inplace=True)
self.conv2 = conv3x3(planes, planes)
self.bn2 = nn.BatchNorm2d(planes)
self.downsample = downsample
self.stride = stride
def forward(self, x):
residual = x
out = self.conv1(x)
out = self.bn1(out)
out = self.relu(out)
out = self.conv2(out)
out = self.bn2(out)
if self.downsample is not None:
residual = self.downsample(x)
out += residual
out = self.relu(out)
return out
class IBN(nn.Module):
def __init__(self, planes):
super(IBN, self).__init__()
half1 = int(planes/2)
self.half = half1
half2 = planes - half1
self.IN = nn.InstanceNorm2d(half1, affine=True)
self.BN = nn.BatchNorm2d(half2)
def forward(self, x):
split = torch.split(x, self.half, 1)
out1 = self.IN(split[0].contiguous())
out2 = self.BN(split[1].contiguous())
out = torch.cat((out1, out2), 1)
return out
class Bottleneck(nn.Module):
expansion = 4
def __init__(self, inplanes, planes, ibn=False, stride=1, downsample=None):
super(Bottleneck, self).__init__()
self.conv1 = nn.Conv2d(inplanes, planes, kernel_size=1, bias=False)
if ibn:
self.bn1 = IBN(planes)
else:
self.bn1 = nn.BatchNorm2d(planes)
self.conv2 = nn.Conv2d(planes, planes, kernel_size=3, stride=stride,
padding=1, bias=False)
self.bn2 = nn.BatchNorm2d(planes)
self.conv3 = nn.Conv2d(planes, planes * self.expansion, kernel_size=1, bias=False)
self.bn3 = nn.BatchNorm2d(planes * self.expansion)
self.relu = nn.ReLU(inplace=True)
self.downsample = downsample
self.stride = stride
def forward(self, x):
residual = x
out = self.conv1(x)
out = self.bn1(out)
out = self.relu(out)
out = self.conv2(out)
out = self.bn2(out)
out = self.relu(out)
out = self.conv3(out)
out = self.bn3(out)
if self.downsample is not None:
residual = self.downsample(x)
out += residual
out = self.relu(out)
return out
class ResNet(nn.Module):
def __init__(self, block, layers, num_classes=1000):
scale = 64
self.inplanes = scale
super(ResNet, self).__init__()
self.conv1 = nn.Conv2d(3, scale, kernel_size=7, stride=2, padding=3,
bias=False)
self.bn1 = nn.BatchNorm2d(scale)
self.relu = nn.ReLU(inplace=True)
self.maxpool = nn.MaxPool2d(kernel_size=3, stride=2, padding=1)
self.layer1 = self._make_layer(block, scale, layers[0])
self.layer2 = self._make_layer(block, scale*2, layers[1], stride=2)
self.layer3 = self._make_layer(block, scale*4, layers[2], stride=2)
self.layer4 = self._make_layer(block, scale*8, layers[3], stride=2)
self.avgpool = nn.AvgPool2d(7)
self.fc = nn.Linear(scale * 8 * block.expansion, num_classes)
for m in self.modules():
if isinstance(m, nn.Conv2d):
n = m.kernel_size[0] * m.kernel_size[1] * m.out_channels
m.weight.data.normal_(0, math.sqrt(2. / n))
elif isinstance(m, nn.BatchNorm2d):
m.weight.data.fill_(1)
m.bias.data.zero_()
elif isinstance(m, nn.InstanceNorm2d):
m.weight.data.fill_(1)
m.bias.data.zero_()
def _make_layer(self, block, planes, blocks, stride=1):
downsample = None
if stride != 1 or self.inplanes != planes * block.expansion:
downsample = nn.Sequential(
nn.Conv2d(self.inplanes, planes * block.expansion,
kernel_size=1, stride=stride, bias=False),
nn.BatchNorm2d(planes * block.expansion),
)
layers = []
ibn = True
if planes == 512:
ibn = False
layers.append(block(self.inplanes, planes, ibn, stride, downsample))
self.inplanes = planes * block.expansion
for i in range(1, blocks):
layers.append(block(self.inplanes, planes, ibn))
return nn.Sequential(*layers)
def forward(self, x):
x = self.conv1(x)
x = self.bn1(x)
x = self.relu(x)
x = self.maxpool(x)
x = self.layer1(x)
x = self.layer2(x)
x = self.layer3(x)
x = self.layer4(x)
x = self.avgpool(x)
x = x.view(x.size(0), -1)
x = self.fc(x)
return x
def resnet50_ibn_a(pretrained=False, **kwargs):
"""Constructs a ResNet-50 model.
Args:
pretrained (bool): If True, returns a model pre-trained on ImageNet
"""
model = ResNet(Bottleneck, [3, 4, 6, 3], **kwargs)
if pretrained:
state_dict = torch.load(model_urls['ibn_resnet50a'], map_location=torch.device('cpu'))['state_dict']
state_dict = remove_module_key(state_dict)
model.load_state_dict(state_dict)
return model
def resnet101_ibn_a(pretrained=False, **kwargs):
"""Constructs a ResNet-101 model.
Args:
pretrained (bool): If True, returns a model pre-trained on ImageNet
"""
model = ResNet(Bottleneck, [3, 4, 23, 3], **kwargs)
if pretrained:
state_dict = torch.load(model_urls['ibn_resnet101a'], map_location=torch.device('cpu'))['state_dict']
state_dict = remove_module_key(state_dict)
model.load_state_dict(state_dict)
return model
def remove_module_key(state_dict):
for key in list(state_dict.keys()):
if 'module' in key:
state_dict[key.replace('module.','')] = state_dict.pop(key)
return state_dict
================================================
FILE: clustercontrast/trainers.py
================================================
from __future__ import print_function, absolute_import
import time
from .utils.meters import AverageMeter
class ClusterContrastTrainer(object):
def __init__(self, encoder, memory=None):
super(ClusterContrastTrainer, self).__init__()
self.encoder = encoder
self.memory = memory
def train(self, epoch, data_loader, optimizer, print_freq=10, train_iters=400):
self.encoder.train()
batch_time = AverageMeter()
data_time = AverageMeter()
losses = AverageMeter()
end = time.time()
for i in range(train_iters):
# load data
inputs = data_loader.next()
data_time.update(time.time() - end)
# process inputs
inputs, labels, indexes = self._parse_data(inputs)
# forward
f_out = self._forward(inputs)
# print("f_out shape: {}".format(f_out.shape))
# compute loss with the hybrid memory
# loss = self.memory(f_out, indexes)
loss = self.memory(f_out, labels)
optimizer.zero_grad()
loss.backward()
optimizer.step()
losses.update(loss.item())
# print log
batch_time.update(time.time() - end)
end = time.time()
if (i + 1) % print_freq == 0:
print('Epoch: [{}][{}/{}]\t'
'Time {:.3f} ({:.3f})\t'
'Data {:.3f} ({:.3f})\t'
'Loss {:.3f} ({:.3f})'
.format(epoch, i + 1, len(data_loader),
batch_time.val, batch_time.avg,
data_time.val, data_time.avg,
losses.val, losses.avg))
def _parse_data(self, inputs):
imgs, _, pids, _, indexes = inputs
return imgs.cuda(), pids.cuda(), indexes.cuda()
def _forward(self, inputs):
return self.encoder(inputs)
================================================
FILE: clustercontrast/utils/__init__.py
================================================
from __future__ import absolute_import
import torch
def to_numpy(tensor):
if torch.is_tensor(tensor):
return tensor.cpu().numpy()
elif type(tensor).__module__ != 'numpy':
raise ValueError("Cannot convert {} to numpy array"
.format(type(tensor)))
return tensor
def to_torch(ndarray):
if type(ndarray).__module__ == 'numpy':
return torch.from_numpy(ndarray)
elif not torch.is_tensor(ndarray):
raise ValueError("Cannot convert {} to torch tensor"
.format(type(ndarray)))
return ndarray
================================================
FILE: clustercontrast/utils/data/__init__.py
================================================
from __future__ import absolute_import
from .base_dataset import BaseDataset, BaseImageDataset
from .preprocessor import Preprocessor
class IterLoader:
def __init__(self, loader, length=None):
self.loader = loader
self.length = length
self.iter = None
def __len__(self):
if self.length is not None:
return self.length
return len(self.loader)
def new_epoch(self):
self.iter = iter(self.loader)
def next(self):
try:
return next(self.iter)
except:
self.iter = iter(self.loader)
return next(self.iter)
================================================
FILE: clustercontrast/utils/data/base_dataset.py
================================================
# encoding: utf-8
import numpy as np
class BaseDataset(object):
"""
Base class of reid dataset
"""
def get_imagedata_info(self, data):
pids, cams = [], []
for _, pid, camid in data:
pids += [pid]
cams += [camid]
pids = set(pids)
cams = set(cams)
num_pids = len(pids)
num_cams = len(cams)
num_imgs = len(data)
return num_pids, num_imgs, num_cams
def print_dataset_statistics(self):
raise NotImplementedError
@property
def images_dir(self):
return None
class BaseImageDataset(BaseDataset):
"""
Base class of image reid dataset
"""
def print_dataset_statistics(self, train, query, gallery):
num_train_pids, num_train_imgs, num_train_cams = self.get_imagedata_info(train)
num_query_pids, num_query_imgs, num_query_cams = self.get_imagedata_info(query)
num_gallery_pids, num_gallery_imgs, num_gallery_cams = self.get_imagedata_info(gallery)
print("Dataset statistics:")
print(" ----------------------------------------")
print(" subset | # ids | # images | # cameras")
print(" ----------------------------------------")
print(" train | {:5d} | {:8d} | {:9d}".format(num_train_pids, num_train_imgs, num_train_cams))
print(" query | {:5d} | {:8d} | {:9d}".format(num_query_pids, num_query_imgs, num_query_cams))
print(" gallery | {:5d} | {:8d} | {:9d}".format(num_gallery_pids, num_gallery_imgs, num_gallery_cams))
print(" ----------------------------------------")
================================================
FILE: clustercontrast/utils/data/preprocessor.py
================================================
from __future__ import absolute_import
import os
import os.path as osp
from torch.utils.data import DataLoader, Dataset
import numpy as np
import random
import math
from PIL import Image
class Preprocessor(Dataset):
def __init__(self, dataset, root=None, transform=None):
super(Preprocessor, self).__init__()
self.dataset = dataset
self.root = root
self.transform = transform
def __len__(self):
return len(self.dataset)
def __getitem__(self, indices):
return self._get_single_item(indices)
def _get_single_item(self, index):
fname, pid, camid = self.dataset[index]
fpath = fname
if self.root is not None:
fpath = osp.join(self.root, fname)
img = Image.open(fpath).convert('RGB')
if self.transform is not None:
img = self.transform(img)
return img, fname, pid, camid, index
================================================
FILE: clustercontrast/utils/data/sampler.py
================================================
from __future__ import absolute_import
from collections import defaultdict
import math
import numpy as np
import copy
import random
import torch
from torch.utils.data.sampler import (
Sampler, SequentialSampler, RandomSampler, SubsetRandomSampler,
WeightedRandomSampler)
def No_index(a, b):
assert isinstance(a, list)
return [i for i, j in enumerate(a) if j != b]
class RandomIdentitySampler(Sampler):
def __init__(self, data_source, num_instances):
self.data_source = data_source
self.num_instances = num_instances
self.index_dic = defaultdict(list)
for index, (_, pid, _) in enumerate(data_source):
self.index_dic[pid].append(index)
self.pids = list(self.index_dic.keys())
self.num_samples = len(self.pids)
def __len__(self):
return self.num_samples * self.num_instances
def __iter__(self):
indices = torch.randperm(self.num_samples).tolist()
ret = []
for i in indices:
pid = self.pids[i]
t = self.index_dic[pid]
if len(t) >= self.num_instances:
t = np.random.choice(t, size=self.num_instances, replace=False)
else:
t = np.random.choice(t, size=self.num_instances, replace=True)
ret.extend(t)
return iter(ret)
class RandomMultipleGallerySampler(Sampler):
def __init__(self, data_source, num_instances=4):
super().__init__(data_source)
self.data_source = data_source
self.index_pid = defaultdict(int)
self.pid_cam = defaultdict(list)
self.pid_index = defaultdict(list)
self.num_instances = num_instances
for index, (_, pid, cam) in enumerate(data_source):
if pid < 0:
continue
self.index_pid[index] = pid
self.pid_cam[pid].append(cam)
self.pid_index[pid].append(index)
self.pids = list(self.pid_index.keys())
self.num_samples = len(self.pids)
def __len__(self):
return self.num_samples * self.num_instances
def __iter__(self):
indices = torch.randperm(len(self.pids)).tolist()
ret = []
for kid in indices:
i = random.choice(self.pid_index[self.pids[kid]])
_, i_pid, i_cam = self.data_source[i]
ret.append(i)
pid_i = self.index_pid[i]
cams = self.pid_cam[pid_i]
index = self.pid_index[pid_i]
select_cams = No_index(cams, i_cam)
if select_cams:
if len(select_cams) >= self.num_instances:
cam_indexes = np.random.choice(select_cams, size=self.num_instances-1, replace=False)
else:
cam_indexes = np.random.choice(select_cams, size=self.num_instances-1, replace=True)
for kk in cam_indexes:
ret.append(index[kk])
else:
select_indexes = No_index(index, i)
if not select_indexes:
continue
if len(select_indexes) >= self.num_instances:
ind_indexes = np.random.choice(select_indexes, size=self.num_instances-1, replace=False)
else:
ind_indexes = np.random.choice(select_indexes, size=self.num_instances-1, replace=True)
for kk in ind_indexes:
ret.append(index[kk])
return iter(ret)
class RandomMultipleGallerySamplerNoCam(Sampler):
def __init__(self, data_source, num_instances=4):
super().__init__(data_source)
self.data_source = data_source
self.index_pid = defaultdict(int)
self.pid_index = defaultdict(list)
self.num_instances = num_instances
for index, (_, pid, cam) in enumerate(data_source):
if pid < 0:
continue
self.index_pid[index] = pid
self.pid_index[pid].append(index)
self.pids = list(self.pid_index.keys())
self.num_samples = len(self.pids)
def __len__(self):
return self.num_samples * self.num_instances
def __iter__(self):
indices = torch.randperm(len(self.pids)).tolist()
ret = []
for kid in indices:
i = random.choice(self.pid_index[self.pids[kid]])
_, i_pid, i_cam = self.data_source[i]
ret.append(i)
pid_i = self.index_pid[i]
index = self.pid_index[pid_i]
select_indexes = No_index(index, i)
if not select_indexes:
continue
if len(select_indexes) >= self.num_instances:
ind_indexes = np.random.choice(select_indexes, size=self.num_instances-1, replace=False)
else:
ind_indexes = np.random.choice(select_indexes, size=self.num_instances-1, replace=True)
for kk in ind_indexes:
ret.append(index[kk])
return iter(ret)
================================================
FILE: clustercontrast/utils/data/transforms.py
================================================
from __future__ import absolute_import
from torchvision.transforms import *
from PIL import Image
import random
import math
import numpy as np
class RectScale(object):
def __init__(self, height, width, interpolation=Image.BILINEAR):
self.height = height
self.width = width
self.interpolation = interpolation
def __call__(self, img):
w, h = img.size
if h == self.height and w == self.width:
return img
return img.resize((self.width, self.height), self.interpolation)
class RandomSizedRectCrop(object):
def __init__(self, height, width, interpolation=Image.BILINEAR):
self.height = height
self.width = width
self.interpolation = interpolation
def __call__(self, img):
for attempt in range(10):
area = img.size[0] * img.size[1]
target_area = random.uniform(0.64, 1.0) * area
aspect_ratio = random.uniform(2, 3)
h = int(round(math.sqrt(target_area * aspect_ratio)))
w = int(round(math.sqrt(target_area / aspect_ratio)))
if w <= img.size[0] and h <= img.size[1]:
x1 = random.randint(0, img.size[0] - w)
y1 = random.randint(0, img.size[1] - h)
img = img.crop((x1, y1, x1 + w, y1 + h))
assert(img.size == (w, h))
return img.resize((self.width, self.height), self.interpolation)
# Fallback
scale = RectScale(self.height, self.width,
interpolation=self.interpolation)
return scale(img)
class RandomErasing(object):
""" Randomly selects a rectangle region in an image and erases its pixels.
'Random Erasing Data Augmentation' by Zhong et al.
See https://arxiv.org/pdf/1708.04896.pdf
Args:
probability: The probability that the Random Erasing operation will be performed.
sl: Minimum proportion of erased area against input image.
sh: Maximum proportion of erased area against input image.
r1: Minimum aspect ratio of erased area.
mean: Erasing value.
"""
def __init__(self, probability=0.5, sl=0.02, sh=0.4, r1=0.3, mean=(0.4914, 0.4822, 0.4465)):
self.probability = probability
self.mean = mean
self.sl = sl
self.sh = sh
self.r1 = r1
def __call__(self, img):
if random.uniform(0, 1) >= self.probability:
return img
for attempt in range(100):
area = img.size()[1] * img.size()[2]
target_area = random.uniform(self.sl, self.sh) * area
aspect_ratio = random.uniform(self.r1, 1 / self.r1)
h = int(round(math.sqrt(target_area * aspect_ratio)))
w = int(round(math.sqrt(target_area / aspect_ratio)))
if w < img.size()[2] and h < img.size()[1]:
x1 = random.randint(0, img.size()[1] - h)
y1 = random.randint(0, img.size()[2] - w)
if img.size()[0] == 3:
img[0, x1:x1 + h, y1:y1 + w] = self.mean[0]
img[1, x1:x1 + h, y1:y1 + w] = self.mean[1]
img[2, x1:x1 + h, y1:y1 + w] = self.mean[2]
else:
img[0, x1:x1 + h, y1:y1 + w] = self.mean[0]
return img
return img
================================================
FILE: clustercontrast/utils/faiss_rerank.py
================================================
#!/usr/bin/env python3
# -*- coding: utf-8 -*-
"""
CVPR2017 paper:Zhong Z, Zheng L, Cao D, et al. Re-ranking Person Re-identification with k-reciprocal Encoding[J]. 2017.
url:http://openaccess.thecvf.com/content_cvpr_2017/papers/Zhong_Re-Ranking_Person_Re-Identification_CVPR_2017_paper.pdf
Matlab version: https://github.com/zhunzhong07/person-re-ranking
"""
import os, sys
import time
import numpy as np
from scipy.spatial.distance import cdist
import gc
import faiss
import torch
import torch.nn.functional as F
from .faiss_utils import search_index_pytorch, search_raw_array_pytorch, \
index_init_gpu, index_init_cpu
def k_reciprocal_neigh(initial_rank, i, k1):
forward_k_neigh_index = initial_rank[i,:k1+1]
backward_k_neigh_index = initial_rank[forward_k_neigh_index,:k1+1]
fi = np.where(backward_k_neigh_index==i)[0]
return forward_k_neigh_index[fi]
def compute_jaccard_distance(target_features, k1=20, k2=6, print_flag=True, search_option=0, use_float16=False):
end = time.time()
if print_flag:
print('Computing jaccard distance...')
ngpus = faiss.get_num_gpus()
N = target_features.size(0)
mat_type = np.float16 if use_float16 else np.float32
if (search_option==0):
# GPU + PyTorch CUDA Tensors (1)
res = faiss.StandardGpuResources()
res.setDefaultNullStreamAllDevices()
_, initial_rank = search_raw_array_pytorch(res, target_features, target_features, k1)
initial_rank = initial_rank.cpu().numpy()
elif (search_option==1):
# GPU + PyTorch CUDA Tensors (2)
res = faiss.StandardGpuResources()
index = faiss.GpuIndexFlatL2(res, target_features.size(-1))
index.add(target_features.cpu().numpy())
_, initial_rank = search_index_pytorch(index, target_features, k1)
res.syncDefaultStreamCurrentDevice()
initial_rank = initial_rank.cpu().numpy()
elif (search_option==2):
# GPU
index = index_init_gpu(ngpus, target_features.size(-1))
index.add(target_features.cpu().numpy())
_, initial_rank = index.search(target_features.cpu().numpy(), k1)
else:
# CPU
index = index_init_cpu(target_features.size(-1))
index.add(target_features.cpu().numpy())
_, initial_rank = index.search(target_features.cpu().numpy(), k1)
nn_k1 = []
nn_k1_half = []
for i in range(N):
nn_k1.append(k_reciprocal_neigh(initial_rank, i, k1))
nn_k1_half.append(k_reciprocal_neigh(initial_rank, i, int(np.around(k1/2))))
V = np.zeros((N, N), dtype=mat_type)
for i in range(N):
k_reciprocal_index = nn_k1[i]
k_reciprocal_expansion_index = k_reciprocal_index
for candidate in k_reciprocal_index:
candidate_k_reciprocal_index = nn_k1_half[candidate]
if (len(np.intersect1d(candidate_k_reciprocal_index,k_reciprocal_index)) > 2/3*len(candidate_k_reciprocal_index)):
k_reciprocal_expansion_index = np.append(k_reciprocal_expansion_index,candidate_k_reciprocal_index)
k_reciprocal_expansion_index = np.unique(k_reciprocal_expansion_index) ## element-wise unique
dist = 2-2*torch.mm(target_features[i].unsqueeze(0).contiguous(), target_features[k_reciprocal_expansion_index].t())
if use_float16:
V[i,k_reciprocal_expansion_index] = F.softmax(-dist, dim=1).view(-1).cpu().numpy().astype(mat_type)
else:
V[i,k_reciprocal_expansion_index] = F.softmax(-dist, dim=1).view(-1).cpu().numpy()
del nn_k1, nn_k1_half
if k2 != 1:
V_qe = np.zeros_like(V, dtype=mat_type)
for i in range(N):
V_qe[i,:] = np.mean(V[initial_rank[i,:k2],:], axis=0)
V = V_qe
del V_qe
del initial_rank
invIndex = []
for i in range(N):
invIndex.append(np.where(V[:,i] != 0)[0]) #len(invIndex)=all_num
jaccard_dist = np.zeros((N, N), dtype=mat_type)
for i in range(N):
temp_min = np.zeros((1, N), dtype=mat_type)
# temp_max = np.zeros((1,N), dtype=mat_type)
indNonZero = np.where(V[i, :] != 0)[0]
indImages = []
indImages = [invIndex[ind] for ind in indNonZero]
for j in range(len(indNonZero)):
temp_min[0, indImages[j]] = temp_min[0, indImages[j]]+np.minimum(V[i, indNonZero[j]], V[indImages[j], indNonZero[j]])
# temp_max[0,indImages[j]] = temp_max[0,indImages[j]]+np.maximum(V[i,indNonZero[j]],V[indImages[j],indNonZero[j]])
jaccard_dist[i] = 1-temp_min/(2-temp_min)
# jaccard_dist[i] = 1-temp_min/(temp_max+1e-6)
del invIndex, V
pos_bool = (jaccard_dist < 0)
jaccard_dist[pos_bool] = 0.0
if print_flag:
print("Jaccard distance computing time cost: {}".format(time.time()-end))
return jaccard_dist
================================================
FILE: clustercontrast/utils/faiss_utils.py
================================================
import os
import numpy as np
import faiss
import torch
def swig_ptr_from_FloatTensor(x):
assert x.is_contiguous()
assert x.dtype == torch.float32
return faiss.cast_integer_to_float_ptr(
x.storage().data_ptr() + x.storage_offset() * 4)
def swig_ptr_from_LongTensor(x):
assert x.is_contiguous()
assert x.dtype == torch.int64, 'dtype=%s' % x.dtype
return faiss.cast_integer_to_idx_t_ptr(
x.storage().data_ptr() + x.storage_offset() * 8)
def search_index_pytorch(index, x, k, D=None, I=None):
"""call the search function of an index with pytorch tensor I/O (CPU
and GPU supported)"""
assert x.is_contiguous()
n, d = x.size()
assert d == index.d
if D is None:
D = torch.empty((n, k), dtype=torch.float32, device=x.device)
else:
assert D.size() == (n, k)
if I is None:
I = torch.empty((n, k), dtype=torch.int64, device=x.device)
else:
assert I.size() == (n, k)
torch.cuda.synchronize()
xptr = swig_ptr_from_FloatTensor(x)
Iptr = swig_ptr_from_LongTensor(I)
Dptr = swig_ptr_from_FloatTensor(D)
index.search_c(n, xptr,
k, Dptr, Iptr)
torch.cuda.synchronize()
return D, I
def search_raw_array_pytorch(res, xb, xq, k, D=None, I=None,
metric=faiss.METRIC_L2):
assert xb.device == xq.device
nq, d = xq.size()
if xq.is_contiguous():
xq_row_major = True
elif xq.t().is_contiguous():
xq = xq.t() # I initially wrote xq:t(), Lua is still haunting me :-)
xq_row_major = False
else:
raise TypeError('matrix should be row or column-major')
xq_ptr = swig_ptr_from_FloatTensor(xq)
nb, d2 = xb.size()
assert d2 == d
if xb.is_contiguous():
xb_row_major = True
elif xb.t().is_contiguous():
xb = xb.t()
xb_row_major = False
else:
raise TypeError('matrix should be row or column-major')
xb_ptr = swig_ptr_from_FloatTensor(xb)
if D is None:
D = torch.empty(nq, k, device=xb.device, dtype=torch.float32)
else:
assert D.shape == (nq, k)
assert D.device == xb.device
if I is None:
I = torch.empty(nq, k, device=xb.device, dtype=torch.int64)
else:
assert I.shape == (nq, k)
assert I.device == xb.device
D_ptr = swig_ptr_from_FloatTensor(D)
I_ptr = swig_ptr_from_LongTensor(I)
faiss.bruteForceKnn(res, metric,
xb_ptr, xb_row_major, nb,
xq_ptr, xq_row_major, nq,
d, k, D_ptr, I_ptr)
return D, I
def index_init_gpu(ngpus, feat_dim):
flat_config = []
for i in range(ngpus):
cfg = faiss.GpuIndexFlatConfig()
cfg.useFloat16 = False
cfg.device = i
flat_config.append(cfg)
res = [faiss.StandardGpuResources() for i in range(ngpus)]
indexes = [faiss.GpuIndexFlatL2(res[i], feat_dim, flat_config[i]) for i in range(ngpus)]
index = faiss.IndexShards(feat_dim)
for sub_index in indexes:
index.add_shard(sub_index)
index.reset()
return index
def index_init_cpu(feat_dim):
return faiss.IndexFlatL2(feat_dim)
================================================
FILE: clustercontrast/utils/infomap_cluster.py
================================================
import numpy as np
from tqdm import tqdm
import infomap
import faiss
import math
import multiprocessing as mp
from clustercontrast.utils.infomap_utils import Timer
def l2norm(vec):
"""
归一化
:param vec:
:return:
"""
vec /= np.linalg.norm(vec, axis=1).reshape(-1, 1)
return vec
def intdict2ndarray(d, default_val=-1):
arr = np.zeros(len(d)) + default_val
for k, v in d.items():
arr[k] = v
return arr
def read_meta(fn_meta, start_pos=0, verbose=True):
"""
idx2lb：每一个顶点对应一个类
lb2idxs：每个类对应一个id
"""
lb2idxs = {}
idx2lb = {}
with open(fn_meta) as f:
for idx, x in enumerate(f.readlines()[start_pos:]):
lb = int(x.strip())
if lb not in lb2idxs:
lb2idxs[lb] = []
lb2idxs[lb] += [idx]
idx2lb[idx] = lb
inst_num = len(idx2lb)
cls_num = len(lb2idxs)
if verbose:
print('[{}] #cls: {}, #inst: {}'.format(fn_meta, cls_num, inst_num))
return lb2idxs, idx2lb
class knn_faiss():
"""
内积暴力循环
归一化特征的内积等价于余弦相似度
"""
def __init__(self, feats, k, knn_method='faiss-cpu', verbose=True):
self.verbose = verbose
with Timer('[{}] build index {}'.format(knn_method, k), verbose):
feats = feats.astype('float32')
size, dim = feats.shape
if knn_method == 'faiss-gpu':
i = math.ceil(size / 1000000)
if i > 1:
i = (i - 1) * 4
res = faiss.StandardGpuResources()
res.setTempMemory(i * 1024 * 1024 * 1024)
index = faiss.GpuIndexFlatIP(res, dim)
else:
index = faiss.IndexFlatIP(dim)
index.add(feats)
with Timer('[{}] query topk {}'.format(knn_method, k), verbose):
sims, nbrs = index.search(feats, k=k)
self.knns = [(np.array(nbr, dtype=np.int32),
1 - np.array(sim, dtype=np.float32))
for nbr, sim in zip(nbrs, sims)]
def filter_by_th(self, i):
th_nbrs = []
th_dists = []
nbrs, dists = self.knns[i]
for n, dist in zip(nbrs, dists):
if 1 - dist < self.th:
continue
th_nbrs.append(n)
th_dists.append(dist)
th_nbrs = np.array(th_nbrs)
th_dists = np.array(th_dists)
return th_nbrs, th_dists
def get_knns(self, th=None):
if th is None or th <= 0.:
return self.knns
# TODO: optimize the filtering process by numpy
# nproc = mp.cpu_count()
nproc = 1
with Timer('filter edges by th {} (CPU={})'.format(th, nproc),
self.verbose):
self.th = th
self.th_knns = []
tot = len(self.knns)
if nproc > 1:
pool = mp.Pool(nproc)
th_knns = list(
tqdm(pool.imap(self.filter_by_th, range(tot)), total=tot))
pool.close()
else:
th_knns = [self.filter_by_th(i) for i in range(tot)]
return th_knns
def knns2ordered_nbrs(knns, sort=True):
if isinstance(knns, list):
knns = np.array(knns)
nbrs = knns[:, 0, :].astype(np.int32)
dists = knns[:, 1, :]
if sort:
# sort dists from low to high
nb_idx = np.argsort(dists, axis=1)
idxs = np.arange(nb_idx.shape[0]).reshape(-1, 1)
dists = dists[idxs, nb_idx]
nbrs = nbrs[idxs, nb_idx]
return dists, nbrs
# 构造边
def get_links(single, links, nbrs, dists, min_sim):
for i in tqdm(range(nbrs.shape[0])):
count = 0
for j in range(0, len(nbrs[i])):
# 排除本身节点
if i == nbrs[i][j]:
pass
elif dists[i][j] <= 1 - min_sim:
count += 1
links[(i, nbrs[i][j])] = float(1 - dists[i][j])
else:
break
# 统计孤立点
if count == 0:
single.append(i)
return single, links
def cluster_by_infomap(nbrs, dists, min_sim, cluster_num=2):
"""
基于infomap的聚类
:param nbrs:
:param dists:
:param pred_label_path:
:return:
"""
single = []
links = {}
with Timer('get links', verbose=True):
single, links = get_links(single=single, links=links, nbrs=nbrs, dists=dists, min_sim=min_sim)
infomapWrapper = infomap.Infomap("--two-level --directed")
for (i, j), sim in tqdm(links.items()):
_ = infomapWrapper.addLink(int(i), int(j), sim)
# 聚类运算
infomapWrapper.run()
label2idx = {}
idx2label = {}
# 聚类结果统计
for node in infomapWrapper.iterTree():
# node.physicalId 特征向量的编号
# node.moduleIndex() 聚类的编号
if node.moduleIndex() not in label2idx:
label2idx[node.moduleIndex()] = []
label2idx[node.moduleIndex()].append(node.physicalId)
node_count = 0
for k, v in label2idx.items():
if k == 0:
each_index_list = v[2:]
node_count += len(each_index_list)
label2idx[k] = each_index_list
else:
each_index_list = v[1:]
node_count += len(each_index_list)
label2idx[k] = each_index_list
for each_index in each_index_list:
idx2label[each_index] = k
keys_len = len(list(label2idx.keys()))
# 孤立点放入到结果中
for single_node in single:
idx2label[single_node] = keys_len
label2idx[keys_len] = [single_node]
keys_len += 1
node_count += 1
# 孤立点个数
print("孤立点数：{}".format(len(single)))
idx_len = len(list(idx2label.keys()))
assert idx_len == node_count, 'idx_len not equal node_count!'
print("总节点数：{}".format(idx_len))
old_label_container = set()
for each_label, each_index_list in label2idx.items():
if len(each_index_list) <= cluster_num:
for each_index in each_index_list:
idx2label[each_index] = -1
else:
old_label_container.add(each_label)
old2new = {old_label: new_label for new_label, old_label in enumerate(old_label_container)}
for each_index, each_label in idx2label.items():
if each_label == -1:
continue
idx2label[each_index] = old2new[each_label]
pre_labels = intdict2ndarray(idx2label)
print("总类别数：{}/{}".format(keys_len, len(set(pre_labels)) - (1 if -1 in pre_labels else 0)))
return pre_labels
def get_dist_nbr(features, k=80, knn_method='faiss-cpu'):
index = knn_faiss(feats=features, k=k, knn_method=knn_method)
knns = index.get_knns()
dists, nbrs = knns2ordered_nbrs(knns)
return dists, nbrs
================================================
FILE: clustercontrast/utils/infomap_utils.py
================================================
import time
class TextColors:
HEADER = '\033[35m'
OKBLUE = '\033[34m'
OKGREEN = '\033[32m'
WARNING = '\033[33m'
FATAL = '\033[31m'
ENDC = '\033[0m'
BOLD = '\033[1m'
UNDERLINE = '\033[4m'
class Timer():
def __init__(self, name='task', verbose=True):
self.name = name
self.verbose = verbose
def __enter__(self):
self.start = time.time()
return self
def __exit__(self, exc_type, exc_val, exc_tb):
if self.verbose:
print('[Time] {} consumes {:.4f} s'.format(
self.name,
time.time() - self.start))
return exc_type is None
================================================
FILE: clustercontrast/utils/logging.py
================================================
from __future__ import absolute_import
import os
import sys
from .osutils import mkdir_if_missing
class Logger(object):
def __init__(self, fpath=None):
self.console = sys.stdout
self.file = None
if fpath is not None:
mkdir_if_missing(os.path.dirname(fpath))
self.file = open(fpath, 'w')
def __del__(self):
self.close()
def __enter__(self):
pass
def __exit__(self, *args):
self.close()
def write(self, msg):
self.console.write(msg)
if self.file is not None:
self.file.write(msg)
def flush(self):
self.console.flush()
if self.file is not None:
self.file.flush()
os.fsync(self.file.fileno())
def close(self):
self.console.close()
if self.file is not None:
self.file.close()
================================================
FILE: clustercontrast/utils/meters.py
================================================
from __future__ import absolute_import
class AverageMeter(object):
"""Computes and stores the average and current value"""
def __init__(self):
self.val = 0
self.avg = 0
self.sum = 0
self.count = 0
def reset(self):
self.val = 0
self.avg = 0
self.sum = 0
self.count = 0
def update(self, val, n=1):
self.val = val
self.sum += val * n
self.count += n
self.avg = self.sum / self.count
================================================
FILE: clustercontrast/utils/osutils.py
================================================
from __future__ import absolute_import
import os
import errno
def mkdir_if_missing(dir_path):
try:
os.makedirs(dir_path)
except OSError as e:
if e.errno != errno.EEXIST:
raise
================================================
FILE: clustercontrast/utils/rerank.py
================================================
#!/usr/bin/env python2/python3
# -*- coding: utf-8 -*-
"""
Source: https://github.com/zhunzhong07/person-re-ranking
Created on Mon Jun 26 14:46:56 2017
@author: luohao
Modified by Houjing Huang, 2017-12-22.
- This version accepts distance matrix instead of raw features.
- The difference of `/` division between python 2 and 3 is handled.
- numpy.float16 is replaced by numpy.float32 for numerical precision.
CVPR2017 paper:Zhong Z, Zheng L, Cao D, et al. Re-ranking Person Re-identification with k-reciprocal Encoding[J]. 2017.
url:http://openaccess.thecvf.com/content_cvpr_2017/papers/Zhong_Re-Ranking_Person_Re-Identification_CVPR_2017_paper.pdf
Matlab version: https://github.com/zhunzhong07/person-re-ranking
API
q_g_dist: query-gallery distance matrix, numpy array, shape [num_query, num_gallery]
q_q_dist: query-query distance matrix, numpy array, shape [num_query, num_query]
g_g_dist: gallery-gallery distance matrix, numpy array, shape [num_gallery, num_gallery]
k1, k2, lambda_value: parameters, the original paper is (k1=20, k2=6, lambda_value=0.3)
Returns:
final_dist: re-ranked distance, numpy array, shape [num_query, num_gallery]
"""
from __future__ import absolute_import
from __future__ import print_function
from __future__ import division
__all__ = ['re_ranking']
import numpy as np
def re_ranking(q_g_dist, q_q_dist, g_g_dist, k1=20, k2=6, lambda_value=0.3):
# The following naming, e.g. gallery_num, is different from outer scope.
# Don't care about it.
original_dist = np.concatenate(
[np.concatenate([q_q_dist, q_g_dist], axis=1),
np.concatenate([q_g_dist.T, g_g_dist], axis=1)],
axis=0)
original_dist = np.power(original_dist, 2).astype(np.float32)
original_dist = np.transpose(1. * original_dist/np.max(original_dist,axis = 0))
V = np.zeros_like(original_dist).astype(np.float32)
initial_rank = np.argsort(original_dist).astype(np.int32)
query_num = q_g_dist.shape[0]
gallery_num = q_g_dist.shape[0] + q_g_dist.shape[1]
all_num = gallery_num
for i in range(all_num):
# k-reciprocal neighbors
forward_k_neigh_index = initial_rank[i,:k1+1]
backward_k_neigh_index = initial_rank[forward_k_neigh_index,:k1+1]
fi = np.where(backward_k_neigh_index==i)[0]
k_reciprocal_index = forward_k_neigh_index[fi]
k_reciprocal_expansion_index = k_reciprocal_index
for j in range(len(k_reciprocal_index)):
candidate = k_reciprocal_index[j]
candidate_forward_k_neigh_index = initial_rank[candidate,:int(np.around(k1/2.))+1]
candidate_backward_k_neigh_index = initial_rank[candidate_forward_k_neigh_index,:int(np.around(k1/2.))+1]
fi_candidate = np.where(candidate_backward_k_neigh_index == candidate)[0]
candidate_k_reciprocal_index = candidate_forward_k_neigh_index[fi_candidate]
if len(np.intersect1d(candidate_k_reciprocal_index,k_reciprocal_index))> 2./3*len(candidate_k_reciprocal_index):
k_reciprocal_expansion_index = np.append(k_reciprocal_expansion_index,candidate_k_reciprocal_index)
k_reciprocal_expansion_index = np.unique(k_reciprocal_expansion_index)
weight = np.exp(-original_dist[i,k_reciprocal_expansion_index])
V[i,k_reciprocal_expansion_index] = 1.*weight/np.sum(weight)
original_dist = original_dist[:query_num,]
if k2 != 1:
V_qe = np.zeros_like(V,dtype=np.float32)
for i in range(all_num):
V_qe[i,:] = np.mean(V[initial_rank[i,:k2],:],axis=0)
V = V_qe
del V_qe
del initial_rank
invIndex = []
for i in range(gallery_num):
invIndex.append(np.where(V[:,i] != 0)[0])
jaccard_dist = np.zeros_like(original_dist,dtype = np.float32)
for i in range(query_num):
temp_min = np.zeros(shape=[1,gallery_num],dtype=np.float32)
indNonZero = np.where(V[i,:] != 0)[0]
indImages = []
indImages = [invIndex[ind] for ind in indNonZero]
for j in range(len(indNonZero)):
temp_min[0,indImages[j]] = temp_min[0,indImages[j]]+ np.minimum(V[i,indNonZero[j]],V[indImages[j],indNonZero[j]])
jaccard_dist[i] = 1-temp_min/(2.-temp_min)
final_dist = jaccard_dist*(1-lambda_value) + original_dist*lambda_value
del original_dist
del V
del jaccard_dist
final_dist = final_dist[:query_num,query_num:]
return final_dist
================================================
FILE: clustercontrast/utils/serialization.py
================================================
from __future__ import print_function, absolute_import
import json
import os.path as osp
import shutil
import torch
from torch.nn import Parameter
from .osutils import mkdir_if_missing
def read_json(fpath):
with open(fpath, 'r') as f:
obj = json.load(f)
return obj
def write_json(obj, fpath):
mkdir_if_missing(osp.dirname(fpath))
with open(fpath, 'w') as f:
json.dump(obj, f, indent=4, separators=(',', ': '))
def save_checkpoint(state, is_best, fpath='checkpoint.pth.tar'):
mkdir_if_missing(osp.dirname(fpath))
torch.save(state, fpath)
if is_best:
shutil.copy(fpath, osp.join(osp.dirname(fpath), 'model_best.pth.tar'))
def load_checkpoint(fpath):
if osp.isfile(fpath):
# checkpoint = torch.load(fpath)
checkpoint = torch.load(fpath, map_location=torch.device('cpu'))
print("=> Loaded checkpoint '{}'".format(fpath))
return checkpoint
else:
raise ValueError("=> No checkpoint found at '{}'".format(fpath))
def copy_state_dict(state_dict, model, strip=None):
tgt_state = model.state_dict()
copied_names = set()
for name, param in state_dict.items():
if strip is not None and name.startswith(strip):
name = name[len(strip):]
if name not in tgt_state:
continue
if isinstance(param, Parameter):
param = param.data
if param.size() != tgt_state[name].size():
print('mismatch:', name, param.size(), tgt_state[name].size())
continue
tgt_state[name].copy_(param)
copied_names.add(name)
missing = set(tgt_state.keys()) - copied_names
if len(missing) > 0:
print("missing keys in state_dict:", missing)
return model
================================================
FILE: examples/cluster_contrast_train_usl.py
================================================
# -*- coding: utf-8 -*-
from __future__ import print_function, absolute_import
import argparse
import os.path as osp
import random
import numpy as np
import sys
import collections
import time
from datetime import timedelta
from sklearn.cluster import DBSCAN
import torch
from torch import nn
from torch.backends import cudnn
from torch.utils.data import DataLoader
import torch.nn.functional as F
from clustercontrast import datasets
from clustercontrast import models
from clustercontrast.models.cm import ClusterMemory
from clustercontrast.trainers import ClusterContrastTrainer
from clustercontrast.evaluators import Evaluator, extract_features
from clustercontrast.utils.data import IterLoader
from clustercontrast.utils.data import transforms as T
from clustercontrast.utils.data.preprocessor import Preprocessor
from clustercontrast.utils.logging import Logger
from clustercontrast.utils.serialization import load_checkpoint, save_checkpoint
from clustercontrast.utils.faiss_rerank import compute_jaccard_distance
from clustercontrast.utils.data.sampler import RandomMultipleGallerySampler, RandomMultipleGallerySamplerNoCam
start_epoch = best_mAP = 0
def get_data(name, data_dir):
root = osp.join(data_dir, name)
dataset = datasets.create(name, root)
return dataset
def get_train_loader(args, dataset, height, width, batch_size, workers,
num_instances, iters, trainset=None, no_cam=False):
normalizer = T.Normalize(mean=[0.485, 0.456, 0.406],
std=[0.229, 0.224, 0.225])
train_transformer = T.Compose([
T.Resize((height, width), interpolation=3),
T.RandomHorizontalFlip(p=0.5),
T.Pad(10),
T.RandomCrop((height, width)),
T.ToTensor(),
normalizer,
T.RandomErasing(probability=0.5, mean=[0.485, 0.456, 0.406])
])
train_set = sorted(dataset.train) if trainset is None else sorted(trainset)
rmgs_flag = num_instances > 0
if rmgs_flag:
if no_cam:
sampler = RandomMultipleGallerySamplerNoCam(train_set, num_instances)
else:
sampler = RandomMultipleGallerySampler(train_set, num_instances)
else:
sampler = None
train_loader = IterLoader(
DataLoader(Preprocessor(train_set, root=dataset.images_dir, transform=train_transformer),
batch_size=batch_size, num_workers=workers, sampler=sampler,
shuffle=not rmgs_flag, pin_memory=True, drop_last=True), length=iters)
return train_loader
def get_test_loader(dataset, height, width, batch_size, workers, testset=None):
normalizer = T.Normalize(mean=[0.485, 0.456, 0.406],
std=[0.229, 0.224, 0.225])
test_transformer = T.Compose([
T.Resize((height, width), interpolation=3),
T.ToTensor(),
normalizer
])
if testset is None:
testset = list(set(dataset.query) | set(dataset.gallery))
test_loader = DataLoader(
Preprocessor(testset, root=dataset.images_dir, transform=test_transformer),
batch_size=batch_size, num_workers=workers,
shuffle=False, pin_memory=True)
return test_loader
def create_model(args):
model = models.create(args.arch, num_features=args.features, norm=True, dropout=args.dropout,
num_classes=0, pooling_type=args.pooling_type)
# use CUDA
model.cuda()
model = nn.DataParallel(model)
return model
def main():
args = parser.parse_args()
if args.seed is not None:
random.seed(args.seed)
np.random.seed(args.seed)
torch.manual_seed(args.seed)
cudnn.deterministic = True
main_worker(args)
def main_worker(args):
global start_epoch, best_mAP
start_time = time.monotonic()
cudnn.benchmark = True
sys.stdout = Logger(osp.join(args.logs_dir, 'log.txt'))
print("==========\nArgs:{}\n==========".format(args))
# Create datasets
iters = args.iters if (args.iters > 0) else None
print("==> Load unlabeled dataset")
dataset = get_data(args.dataset, args.data_dir)
test_loader = get_test_loader(dataset, args.height, args.width, args.batch_size, args.workers)
# Create model
model = create_model(args)
# Evaluator
evaluator = Evaluator(model)
# Optimizer
params = [{"params": [value]} for _, value in model.named_parameters() if value.requires_grad]
optimizer = torch.optim.Adam(params, lr=args.lr, weight_decay=args.weight_decay)
lr_scheduler = torch.optim.lr_scheduler.StepLR(optimizer, step_size=args.step_size, gamma=0.1)
# Trainer
trainer = ClusterContrastTrainer(model)
for epoch in range(args.epochs):
with torch.no_grad():
print('==> Create pseudo labels for unlabeled data')
cluster_loader = get_test_loader(dataset, args.height, args.width,
args.batch_size, args.workers, testset=sorted(dataset.train))
features, _ = extract_features(model, cluster_loader, print_freq=50)
features = torch.cat([features[f].unsqueeze(0) for f, _, _ in sorted(dataset.train)], 0)
rerank_dist = compute_jaccard_distance(features, k1=args.k1, k2=args.k2)
if epoch == 0:
# DBSCAN cluster
eps = args.eps
print('Clustering criterion: eps: {:.3f}'.format(eps))
cluster = DBSCAN(eps=eps, min_samples=4, metric='precomputed', n_jobs=-1)
# select & cluster images as training set of this epochs
pseudo_labels = cluster.fit_predict(rerank_dist)
num_cluster = len(set(pseudo_labels)) - (1 if -1 in pseudo_labels else 0)
# print("epoch: {} \n pseudo_labels: {}".format(epoch, pseudo_labels.tolist()[:100]))
# generate new dataset and calculate cluster centers
@torch.no_grad()
def generate_cluster_features(labels, features):
centers = collections.defaultdict(list)
for i, label in enumerate(labels):
if label == -1:
continue
centers[labels[i]].append(features[i])
centers = [
torch.stack(centers[idx], dim=0).mean(0) for idx in sorted(centers.keys())
]
centers = torch.stack(centers, dim=0)
return centers
cluster_features = generate_cluster_features(pseudo_labels, features)
del cluster_loader, features
# Create hybrid memory
memory = ClusterMemory(model.module.num_features, num_cluster, temp=args.temp,
momentum=args.momentum, use_hard=args.use_hard).cuda()
memory.features = F.normalize(cluster_features, dim=1).cuda()
trainer.memory = memory
pseudo_labeled_dataset = []
for i, ((fname, _, cid), label) in enumerate(zip(sorted(dataset.train), pseudo_labels)):
if label != -1:
pseudo_labeled_dataset.append((fname, label.item(), cid))
print('==> Statistics for epoch {}: {} clusters'.format(epoch, num_cluster))
train_loader = get_train_loader(args, dataset, args.height, args.width,
args.batch_size, args.workers, args.num_instances, iters,
trainset=pseudo_labeled_dataset, no_cam=args.no_cam)
train_loader.new_epoch()
trainer.train(epoch, train_loader, optimizer,
print_freq=args.print_freq, train_iters=len(train_loader))
if (epoch + 1) % args.eval_step == 0 or (epoch == args.epochs - 1):
mAP = evaluator.evaluate(test_loader, dataset.query, dataset.gallery, cmc_flag=False)
is_best = (mAP > best_mAP)
best_mAP = max(mAP, best_mAP)
save_checkpoint({
'state_dict': model.state_dict(),
'epoch': epoch + 1,
'best_mAP': best_mAP,
}, is_best, fpath=osp.join(args.logs_dir, 'checkpoint.pth.tar'))
print('\n * Finished epoch {:3d} model mAP: {:5.1%} best: {:5.1%}{}\n'.
format(epoch, mAP, best_mAP, ' *' if is_best else ''))
lr_scheduler.step()
print('==> Test with the best model:')
checkpoint = load_checkpoint(osp.join(args.logs_dir, 'model_best.pth.tar'))
model.load_state_dict(checkpoint['state_dict'])
evaluator.evaluate(test_loader, dataset.query, dataset.gallery, cmc_flag=True)
end_time = time.monotonic()
print('Total running time: ', timedelta(seconds=end_time - start_time))
if __name__ == '__main__':
parser = argparse.ArgumentParser(description="Self-paced contrastive learning on unsupervised re-ID")
# data
parser.add_argument('-d', '--dataset', type=str, default='dukemtmcreid',
choices=datasets.names())
parser.add_argument('-b', '--batch-size', type=int, default=2)
parser.add_argument('-j', '--workers', type=int, default=4)
parser.add_argument('--height', type=int, default=256, help="input height")
parser.add_argument('--width', type=int, default=128, help="input width")
parser.add_argument('--num-instances', type=int, default=4,
help="each minibatch consist of "
"(batch_size // num_instances) identities, and "
"each identity has num_instances instances, "
"default: 0 (NOT USE)")
# cluster
parser.add_argument('--eps', type=float, default=0.6,
help="max neighbor distance for DBSCAN")
parser.add_argument('--eps-gap', type=float, default=0.02,
help="multi-scale criterion for measuring cluster reliability")
parser.add_argument('--k1', type=int, default=30,
help="hyperparameter for jaccard distance")
parser.add_argument('--k2', type=int, default=6,
help="hyperparameter for jaccard distance")
# model
parser.add_argument('-a', '--arch', type=str, default='resnet50',
choices=models.names())
parser.add_argument('--features', type=int, default=0)
parser.add_argument('--dropout', type=float, default=0)
parser.add_argument('--momentum', type=float, default=0.2,
help="update momentum for the hybrid memory")
# optimizer
parser.add_argument('--lr', type=float, default=0.00035,
help="learning rate")
parser.add_argument('--weight-decay', type=float, default=5e-4)
parser.add_argument('--epochs', type=int, default=50)
parser.add_argument('--iters', type=int, default=400)
parser.add_argument('--step-size', type=int, default=20)
# training configs
parser.add_argument('--seed', type=int, default=1)
parser.add_argument('--print-freq', type=int, default=10)
parser.add_argument('--eval-step', type=int, default=10)
parser.add_argument('--temp', type=float, default=0.05,
help="temperature for scaling contrastive loss")
# path
working_dir = osp.dirname(osp.abspath(__file__))
parser.add_argument('--data-dir', type=str, metavar='PATH',
default=osp.join(working_dir, 'data'))
parser.add_argument('--logs-dir', type=str, metavar='PATH',
default=osp.join(working_dir, 'logs'))
parser.add_argument('--pooling-type', type=str, default='gem')
parser.add_argument('--use-hard', action="store_true")
parser.add_argument('--no-cam', action="store_true")
main()
================================================
FILE: examples/cluster_contrast_train_usl_infomap.py
================================================
# -*- coding: utf-8 -*-
from __future__ import print_function, absolute_import
import argparse
import os.path as osp
import random
import numpy as np
import sys
import collections
import time
from datetime import timedelta
import torch
from torch import nn
from torch.backends import cudnn
from torch.utils.data import DataLoader
import torch.nn.functional as F
from clustercontrast import datasets
from clustercontrast import models
from clustercontrast.models.cm import ClusterMemory
from clustercontrast.trainers import ClusterContrastTrainer
from clustercontrast.evaluators import Evaluator, extract_features
from clustercontrast.utils.data import IterLoader
from clustercontrast.utils.data import transforms as T
from clustercontrast.utils.data.sampler import RandomMultipleGallerySampler, RandomMultipleGallerySamplerNoCam
from clustercontrast.utils.data.preprocessor import Preprocessor
from clustercontrast.utils.logging import Logger
from clustercontrast.utils.serialization import load_checkpoint, save_checkpoint
from clustercontrast.utils.infomap_cluster import get_dist_nbr, cluster_by_infomap
start_epoch = best_mAP = 0
def str2bool(v):
if isinstance(v, bool):
return v
if v.lower() in ('yes', 'true', 't', 'y', '1'):
return True
elif v.lower() in ('no', 'false', 'f', 'n', '0'):
return False
else:
raise argparse.ArgumentTypeError('Boolean value expected.')
def get_data(name, data_dir):
root = osp.join(data_dir, name)
dataset = datasets.create(name, root)
return dataset
def get_train_loader(args, dataset, height, width, batch_size, workers,
num_instances, iters, trainset=None, no_cam=False):
normalizer = T.Normalize(mean=[0.485, 0.456, 0.406],
std=[0.229, 0.224, 0.225])
train_transformer = T.Compose([
T.Resize((height, width), interpolation=3),
T.RandomHorizontalFlip(p=0.5),
T.Pad(10),
T.RandomCrop((height, width)),
T.ToTensor(),
normalizer,
T.RandomErasing(probability=0.5, mean=[0.485, 0.456, 0.406])
])
train_set = sorted(dataset.train) if trainset is None else sorted(trainset)
rmgs_flag = num_instances > 0
if rmgs_flag:
if no_cam:
sampler = RandomMultipleGallerySamplerNoCam(train_set, num_instances)
else:
sampler = RandomMultipleGallerySampler(train_set, num_instances)
else:
sampler = None
train_loader = IterLoader(
DataLoader(Preprocessor(train_set, root=dataset.images_dir, transform=train_transformer),
batch_size=batch_size, num_workers=workers, sampler=sampler,
shuffle=not rmgs_flag, pin_memory=True, drop_last=True), length=iters)
return train_loader
def get_test_loader(dataset, height, width, batch_size, workers, testset=None):
normalizer = T.Normalize(mean=[0.485, 0.456, 0.406],
std=[0.229, 0.224, 0.225])
test_transformer = T.Compose([
T.Resize((height, width), interpolation=3),
T.ToTensor(),
normalizer
])
if testset is None:
testset = list(set(dataset.query) | set(dataset.gallery))
test_loader = DataLoader(
Preprocessor(testset, root=dataset.images_dir, transform=test_transformer),
batch_size=batch_size, num_workers=workers,
shuffle=False, pin_memory=True)
return test_loader
def create_model(args):
model = models.create(args.arch, num_features=args.features, norm=True, dropout=args.dropout,
num_classes=0, pooling_type=args.pooling_type)
# use CUDA
model.cuda()
model = nn.DataParallel(model)
return model
def main():
args = parser.parse_args()
if args.seed is not None:
random.seed(args.seed)
np.random.seed(args.seed)
torch.manual_seed(args.seed)
cudnn.deterministic = True
main_worker(args)
def main_worker(args):
global start_epoch, best_mAP
start_time = time.monotonic()
cudnn.benchmark = True
sys.stdout = Logger(osp.join(args.logs_dir, 'log.txt'))
print("==========\nArgs:{}\n==========".format(args))
# Create datasets
iters = args.iters if (args.iters > 0) else None
print("==> Load unlabeled dataset")
dataset = get_data(args.dataset, args.data_dir)
test_loader = get_test_loader(dataset, args.height, args.width, args.batch_size, args.workers)
# Create model
model = create_model(args)
# Evaluator
evaluator = Evaluator(model)
# Optimizer
params = [{"params": [value]} for _, value in model.named_parameters() if value.requires_grad]
optimizer = torch.optim.Adam(params, lr=args.lr, weight_decay=args.weight_decay)
lr_scheduler = torch.optim.lr_scheduler.StepLR(optimizer, step_size=args.step_size, gamma=0.1)
# Trainer
trainer = ClusterContrastTrainer(model)
for epoch in range(args.epochs):
with torch.no_grad():
print('==> Create pseudo labels for unlabeled data')
cluster_loader = get_test_loader(dataset, args.height, args.width,
args.batch_size, args.workers, testset=sorted(dataset.train))
features, _ = extract_features(model, cluster_loader, print_freq=50)
features = torch.cat([features[f].unsqueeze(0) for f, _, _ in sorted(dataset.train)], 0)
features_array = F.normalize(features, dim=1).cpu().numpy()
feat_dists, feat_nbrs = get_dist_nbr(features=features_array, k=args.k1, knn_method='faiss-gpu')
del features_array
s = time.time()
pseudo_labels = cluster_by_infomap(feat_nbrs, feat_dists, min_sim=args.eps, cluster_num=args.k2)
pseudo_labels = pseudo_labels.astype(np.intp)
print('cluster cost time: {}'.format(time.time() - s))
num_cluster = len(set(pseudo_labels)) - (1 if -1 in pseudo_labels else 0)
# generate new dataset and calculate cluster centers
@torch.no_grad()
def generate_cluster_features(labels, features):
centers = collections.defaultdict(list)
for i, label in enumerate(labels):
if label == -1:
continue
centers[labels[i]].append(features[i])
centers = [
torch.stack(centers[idx], dim=0).mean(0) for idx in sorted(centers.keys())
]
centers = torch.stack(centers, dim=0)
return centers
cluster_features = generate_cluster_features(pseudo_labels, features)
del cluster_loader, features
# Create hybrid memory
memory = ClusterMemory(model.module.num_features, num_cluster, temp=args.temp,
momentum=args.momentum, use_hard=args.use_hard).cuda()
memory.features = F.normalize(cluster_features, dim=1).cuda()
trainer.memory = memory
pseudo_labeled_dataset = []
for i, ((fname, _, cid), label) in enumerate(zip(sorted(dataset.train), pseudo_labels)):
if label != -1:
pseudo_labeled_dataset.append((fname, label.item(), cid))
print('==> Statistics for epoch {}: {} clusters'.format(epoch, num_cluster))
train_loader = get_train_loader(args, dataset, args.height, args.width,
args.batch_size, args.workers, args.num_instances, iters,
trainset=pseudo_labeled_dataset, no_cam=args.no_cam)
train_loader.new_epoch()
trainer.train(epoch, train_loader, optimizer,
print_freq=args.print_freq, train_iters=len(train_loader))
if (epoch + 1) % args.eval_step == 0 or (epoch == args.epochs - 1):
mAP = evaluator.evaluate(test_loader, dataset.query, dataset.gallery, cmc_flag=False)
is_best = (mAP > best_mAP)
best_mAP = max(mAP, best_mAP)
save_checkpoint({
'state_dict': model.state_dict(),
'epoch': epoch + 1,
'best_mAP': best_mAP,
}, is_best, fpath=osp.join(args.logs_dir, 'checkpoint.pth.tar'))
print('\n * Finished epoch {:3d} model mAP: {:5.1%} best: {:5.1%}{}\n'.
format(epoch, mAP, best_mAP, ' *' if is_best else ''))
lr_scheduler.step()
print('==> Test with the best model:')
checkpoint = load_checkpoint(osp.join(args.logs_dir, 'model_best.pth.tar'))
model.load_state_dict(checkpoint['state_dict'])
evaluator.evaluate(test_loader, dataset.query, dataset.gallery, cmc_flag=True)
end_time = time.monotonic()
print('Total running time: ', timedelta(seconds=end_time - start_time))
if __name__ == '__main__':
parser = argparse.ArgumentParser(description="Self-paced contrastive learning on unsupervised re-ID")
# data
parser.add_argument('-d', '--dataset', type=str, default='dukemtmcreid',
choices=datasets.names())
parser.add_argument('-b', '--batch-size', type=int, default=2)
parser.add_argument('-j', '--workers', type=int, default=4)
parser.add_argument('--height', type=int, default=256, help="input height")
parser.add_argument('--width', type=int, default=128, help="input width")
parser.add_argument('--num-instances', type=int, default=4,
help="each minibatch consist of "
"(batch_size // num_instances) identities, and "
"each identity has num_instances instances, "
"default: 0 (NOT USE)")
# cluster
parser.add_argument('--eps', type=float, default=0.5,
help="max neighbor distance for DBSCAN")
parser.add_argument('--eps-gap', type=float, default=0.02,
help="multi-scale criterion for measuring cluster reliability")
parser.add_argument('--k1', type=int, default=15,
help="hyperparameter for KNN")
parser.add_argument('--k2', type=int, default=4,
help="hyperparameter for outline")
# model
parser.add_argument('-a', '--arch', type=str, default='resnet50',
choices=models.names())
parser.add_argument('--features', type=int, default=0)
parser.add_argument('--dropout', type=float, default=0)
parser.add_argument('--momentum', type=float, default=0.2,
help="update momentum for the hybrid memory")
# optimizer
parser.add_argument('--lr', type=float, default=0.00035,
help="learning rate")
parser.add_argument('--weight-decay', type=float, default=5e-4)
parser.add_argument('--epochs', type=int, default=50)
parser.add_argument('--iters', type=int, default=400)
parser.add_argument('--step-size', type=int, default=20)
# training configs
parser.add_argument('--seed', type=int, default=1)
parser.add_argument('--print-freq', type=int, default=10)
parser.add_argument('--eval-step', type=int, default=10)
parser.add_argument('--temp', type=float, default=0.05,
help="temperature for scaling contrastive loss")
# path
working_dir = osp.dirname(osp.abspath(__file__))
parser.add_argument('--data-dir', type=str, metavar='PATH',
default=osp.join(working_dir, 'data'))
parser.add_argument('--logs-dir', type=str, metavar='PATH',
default=osp.join(working_dir, 'logs'))
parser.add_argument('--pooling-type', type=str, default='gem')
parser.add_argument('--use-hard', action="store_true")
parser.add_argument('--no-cam', action="store_true")
main()
================================================
FILE: examples/logs/log.txt
================================================
==========
Args:Namespace(arch='resnet_ibn50a', batch_size=256, data_dir='/data0/developer/cluster-contrast/examples/data', dataset='market1501', dropout=0, epochs=50, eps=0.4, eps_gap=0.02, eval_step=10, features=0, height=256, iters=400, k1=30, k2=6, logs_dir='/data0/developer/cluster-contrast/examples/logs/gem-hard', lr=0.00035, momentum=0.1, num_instances=16, pooling_type='gem', print_freq=10, seed=1, step_size=20, temp=0.05, use_hard=True, weight_decay=0.0005, width=128, workers=4)
==========
==> Load unlabeled dataset
=> Market1501 loaded
Dataset statistics:
----------------------------------------
subset | # ids | # images | # cameras
----------------------------------------
train | 751 | 12936 | 6
query | 750 | 3368 | 6
gallery | 751 | 15913 | 6
----------------------------------------
pooling_type: gem
==> Create pseudo labels for unlabeled data
Extract Features: [50/51] Time 0.128 (0.379) Data 0.000 (0.019)
Computing jaccard distance...
Jaccard distance computing time cost: 23.047871351242065
Clustering criterion: eps: 0.400
==> Statistics for epoch 0: 80 clusters
Epoch: [0][10/400] Time 0.370 (0.775) Data 0.000 (0.211) Loss 4.447 (3.906)
Epoch: [0][20/400] Time 0.362 (0.733) Data 0.000 (0.271) Loss 3.082 (3.512)
Epoch: [0][30/400] Time 0.361 (0.722) Data 0.000 (0.293) Loss 3.117 (3.283)
Epoch: [0][40/400] Time 0.361 (0.714) Data 0.000 (0.301) Loss 3.093 (3.113)
Epoch: [0][50/400] Time 0.362 (0.712) Data 0.000 (0.309) Loss 2.184 (2.973)
Epoch: [0][60/400] Time 0.361 (0.711) Data 0.000 (0.315) Loss 2.049 (2.829)
Epoch: [0][70/400] Time 0.354 (0.711) Data 0.000 (0.319) Loss 2.118 (2.714)
Epoch: [0][80/400] Time 0.362 (0.709) Data 0.000 (0.321) Loss 2.413 (2.657)
Epoch: [0][90/400] Time 0.360 (0.707) Data 0.000 (0.321) Loss 2.092 (2.576)
Epoch: [0][100/400] Time 0.362 (0.706) Data 0.000 (0.322) Loss 1.619 (2.488)
Epoch: [0][110/400] Time 0.374 (0.704) Data 0.000 (0.322) Loss 2.173 (2.423)
Epoch: [0][120/400] Time 0.351 (0.703) Data 0.000 (0.323) Loss 1.402 (2.359)
Epoch: [0][130/400] Time 0.369 (0.703) Data 0.000 (0.324) Loss 2.615 (2.307)
Epoch: [0][140/400] Time 0.368 (0.703) Data 0.000 (0.325) Loss 1.323 (2.262)
Epoch: [0][150/400] Time 0.369 (0.703) Data 0.000 (0.325) Loss 1.811 (2.209)
Epoch: [0][160/400] Time 0.369 (0.704) Data 0.000 (0.326) Loss 1.369 (2.153)
Epoch: [0][170/400] Time 0.368 (0.704) Data 0.000 (0.327) Loss 1.734 (2.109)
Epoch: [0][180/400] Time 0.369 (0.704) Data 0.000 (0.327) Loss 0.889 (2.057)
Epoch: [0][190/400] Time 0.366 (0.704) Data 0.000 (0.328) Loss 1.458 (2.011)
Epoch: [0][200/400] Time 0.361 (0.705) Data 0.000 (0.329) Loss 1.066 (1.967)
Epoch: [0][210/400] Time 0.363 (0.705) Data 0.000 (0.329) Loss 1.535 (1.928)
Epoch: [0][220/400] Time 0.362 (0.705) Data 0.000 (0.330) Loss 1.193 (1.898)
Epoch: [0][230/400] Time 0.362 (0.705) Data 0.000 (0.330) Loss 1.210 (1.862)
Epoch: [0][240/400] Time 0.360 (0.704) Data 0.000 (0.330) Loss 1.218 (1.829)
Epoch: [0][250/400] Time 0.363 (0.704) Data 0.000 (0.330) Loss 0.901 (1.800)
Epoch: [0][260/400] Time 0.363 (0.704) Data 0.000 (0.330) Loss 1.281 (1.770)
Epoch: [0][270/400] Time 0.360 (0.704) Data 0.000 (0.331) Loss 1.332 (1.740)
Epoch: [0][280/400] Time 0.362 (0.704) Data 0.000 (0.331) Loss 1.559 (1.710)
Epoch: [0][290/400] Time 0.362 (0.704) Data 0.000 (0.331) Loss 0.889 (1.682)
Epoch: [0][300/400] Time 0.362 (0.704) Data 0.000 (0.331) Loss 1.134 (1.657)
Epoch: [0][310/400] Time 0.363 (0.703) Data 0.000 (0.331) Loss 1.001 (1.630)
Epoch: [0][320/400] Time 0.372 (0.703) Data 0.000 (0.331) Loss 0.619 (1.602)
Epoch: [0][330/400] Time 0.361 (0.703) Data 0.000 (0.332) Loss 0.942 (1.576)
Epoch: [0][340/400] Time 0.363 (0.703) Data 0.000 (0.332) Loss 0.700 (1.552)
Epoch: [0][350/400] Time 0.363 (0.703) Data 0.000 (0.332) Loss 0.434 (1.530)
Epoch: [0][360/400] Time 0.362 (0.703) Data 0.000 (0.332) Loss 0.739 (1.507)
Epoch: [0][370/400] Time 0.364 (0.703) Data 0.000 (0.332) Loss 0.631 (1.485)
Epoch: [0][380/400] Time 0.365 (0.702) Data 0.000 (0.332) Loss 0.529 (1.465)
Epoch: [0][390/400] Time 0.364 (0.702) Data 0.000 (0.332) Loss 0.699 (1.445)
Epoch: [0][400/400] Time 0.363 (0.702) Data 0.000 (0.332) Loss 0.945 (1.426)
==> Create pseudo labels for unlabeled data
Extract Features: [50/51] Time 0.130 (0.158) Data 0.000 (0.027)
Computing jaccard distance...
Jaccard distance computing time cost: 20.340964555740356
==> Statistics for epoch 1: 286 clusters
Epoch: [1][10/400] Time 0.352 (0.418) Data 0.001 (0.053) Loss 0.530 (0.732)
Epoch: [1][20/400] Time 0.360 (0.480) Data 0.000 (0.117) Loss 3.473 (1.095)
Epoch: [1][30/400] Time 0.361 (0.441) Data 0.000 (0.078) Loss 3.468 (1.908)
Epoch: [1][40/400] Time 0.363 (0.461) Data 0.000 (0.099) Loss 3.717 (2.348)
Epoch: [1][50/400] Time 0.366 (0.442) Data 0.000 (0.079) Loss 3.763 (2.694)
Epoch: [1][60/400] Time 0.363 (0.456) Data 0.000 (0.094) Loss 3.590 (2.891)
Epoch: [1][70/400] Time 0.345 (0.468) Data 0.000 (0.106) Loss 3.950 (3.029)
Epoch: [1][80/400] Time 0.362 (0.455) Data 0.000 (0.093) Loss 3.343 (3.112)
Epoch: [1][90/400] Time 0.362 (0.465) Data 0.000 (0.102) Loss 3.418 (3.184)
Epoch: [1][100/400] Time 0.363 (0.454) Data 0.000 (0.092) Loss 3.445 (3.229)
Epoch: [1][110/400] Time 0.362 (0.462) Data 0.001 (0.099) Loss 3.722 (3.268)
Epoch: [1][120/400] Time 2.066 (0.468) Data 1.640 (0.105) Loss 2.855 (3.295)
Epoch: [1][130/400] Time 0.362 (0.460) Data 0.000 (0.097) Loss 3.671 (3.306)
Epoch: [1][140/400] Time 0.363 (0.465) Data 0.000 (0.102) Loss 2.616 (3.296)
Epoch: [1][150/400] Time 0.363 (0.458) Data 0.000 (0.095) Loss 3.351 (3.290)
Epoch: [1][160/400] Time 0.363 (0.463) Data 0.000 (0.100) Loss 3.071 (3.285)
Epoch: [1][170/400] Time 0.363 (0.458) Data 0.000 (0.094) Loss 3.275 (3.282)
Epoch: [1][180/400] Time 0.364 (0.461) Data 0.000 (0.098) Loss 3.301 (3.270)
Epoch: [1][190/400] Time 0.381 (0.466) Data 0.001 (0.102) Loss 2.709 (3.262)
Epoch: [1][200/400] Time 0.365 (0.460) Data 0.000 (0.097) Loss 3.041 (3.249)
Epoch: [1][210/400] Time 0.370 (0.464) Data 0.001 (0.100) Loss 3.036 (3.237)
Epoch: [1][220/400] Time 0.369 (0.460) Data 0.000 (0.096) Loss 3.528 (3.245)
Epoch: [1][230/400] Time 0.362 (0.463) Data 0.001 (0.099) Loss 2.933 (3.224)
Epoch: [1][240/400] Time 0.356 (0.466) Data 0.001 (0.101) Loss 2.484 (3.218)
Epoch: [1][250/400] Time 0.362 (0.462) Data 0.000 (0.097) Loss 2.455 (3.207)
Epoch: [1][260/400] Time 0.414 (0.464) Data 0.001 (0.100) Loss 2.943 (3.191)
Epoch: [1][270/400] Time 0.369 (0.461) Data 0.000 (0.096) Loss 2.580 (3.175)
Epoch: [1][280/400] Time 0.352 (0.464) Data 0.000 (0.099) Loss 2.666 (3.156)
Epoch: [1][290/400] Time 1.998 (0.466) Data 1.634 (0.101) Loss 2.325 (3.150)
Epoch: [1][300/400] Time 0.363 (0.463) Data 0.000 (0.098) Loss 3.091 (3.134)
Epoch: [1][310/400] Time 0.361 (0.465) Data 0.000 (0.100) Loss 1.971 (3.123)
Epoch: [1][320/400] Time 0.364 (0.462) Data 0.000 (0.097) Loss 3.067 (3.111)
Epoch: [1][330/400] Time 0.362 (0.464) Data 0.001 (0.099) Loss 2.665 (3.097)
Epoch: [1][340/400] Time 0.363 (0.461) Data 0.000 (0.096) Loss 2.564 (3.083)
Epoch: [1][350/400] Time 0.363 (0.463) Data 0.000 (0.098) Loss 2.617 (3.072)
Epoch: [1][360/400] Time 0.361 (0.465) Data 0.001 (0.100) Loss 2.341 (3.057)
Epoch: [1][370/400] Time 0.365 (0.463) Data 0.000 (0.098) Loss 2.255 (3.041)
Epoch: [1][380/400] Time 0.364 (0.464) Data 0.001 (0.099) Loss 2.267 (3.029)
Epoch: [1][390/400] Time 0.366 (0.462) Data 0.000 (0.097) Loss 2.701 (3.015)
Epoch: [1][400/400] Time 0.376 (0.464) Data 0.001 (0.099) Loss 1.968 (3.001)
==> Create pseudo labels for unlabeled data
Extract Features: [50/51] Time 0.130 (0.160) Data 0.000 (0.031)
Computing jaccard distance...
Jaccard distance computing time cost: 21.161827564239502
==> Statistics for epoch 2: 561 clusters
Epoch: [2][10/400] Time 0.362 (0.422) Data 0.001 (0.054) Loss 0.609 (0.845)
Epoch: [2][20/400] Time 0.361 (0.392) Data 0.001 (0.027) Loss 0.627 (0.767)
Epoch: [2][30/400] Time 0.361 (0.382) Data 0.000 (0.018) Loss 0.408 (0.696)
Epoch: [2][40/400] Time 0.363 (0.420) Data 0.001 (0.056) Loss 4.156 (1.033)
Epoch: [2][50/400] Time 0.362 (0.408) Data 0.001 (0.045) Loss 3.165 (1.537)
Epoch: [2][60/400] Time 0.386 (0.401) Data 0.000 (0.038) Loss 4.043 (1.907)
Epoch: [2][70/400] Time 0.363 (0.396) Data 0.000 (0.032) Loss 4.042 (2.221)
Epoch: [2][80/400] Time 0.364 (0.415) Data 0.001 (0.051) Loss 3.583 (2.379)
Epoch: [2][90/400] Time 0.364 (0.409) Data 0.000 (0.046) Loss 3.396 (2.509)
Epoch: [2][100/400] Time 0.365 (0.405) Data 0.000 (0.041) Loss 3.422 (2.619)
Epoch: [2][110/400] Time 0.363 (0.418) Data 0.001 (0.054) Loss 4.696 (2.711)
Epoch: [2][120/400] Time 0.363 (0.414) Data 0.001 (0.050) Loss 3.733 (2.794)
Epoch: [2][130/400] Time 0.391 (0.410) Data 0.001 (0.046) Loss 3.354 (2.860)
Epoch: [2][140/400] Time 0.365 (0.407) Data 0.000 (0.043) Loss 3.528 (2.919)
Epoch: [2][150/400] Time 0.362 (0.416) Data 0.001 (0.051) Loss 4.046 (2.952)
Epoch: [2][160/400] Time 0.364 (0.413) Data 0.000 (0.048) Loss 3.927 (2.989)
Epoch: [2][170/400] Time 0.364 (0.410) Data 0.000 (0.045) Loss 3.270 (3.019)
Epoch: [2][180/400] Time 0.362 (0.416) Data 0.001 (0.052) Loss 3.555 (3.050)
Epoch: [2][190/400] Time 0.364 (0.414) Data 0.001 (0.049) Loss 3.980 (3.071)
Epoch: [2][200/400] Time 0.363 (0.411) Data 0.001 (0.047) Loss 3.768 (3.093)
Epoch: [2][210/400] Time 0.363 (0.409) Data 0.000 (0.045) Loss 3.659 (3.104)
Epoch: [2][220/400] Time 0.364 (0.415) Data 0.001 (0.050) Loss 2.920 (3.109)
Epoch: [2][230/400] Time 0.365 (0.413) Data 0.001 (0.048) Loss 3.674 (3.115)
Epoch: [2][240/400] Time 0.365 (0.411) Data 0.000 (0.046) Loss 2.956 (3.118)
Epoch: [2][250/400] Time 0.363 (0.416) Data 0.001 (0.051) Loss 2.949 (3.129)
Epoch: [2][260/400] Time 0.363 (0.414) Data 0.000 (0.049) Loss 3.126 (3.132)
Epoch: [2][270/400] Time 0.365 (0.412) Data 0.001 (0.047) Loss 2.763 (3.149)
Epoch: [2][280/400] Time 0.363 (0.410) Data 0.000 (0.046) Loss 2.874 (3.153)
Epoch: [2][290/400] Time 0.350 (0.414) Data 0.001 (0.050) Loss 3.045 (3.151)
Epoch: [2][300/400] Time 0.366 (0.412) Data 0.001 (0.048) Loss 2.985 (3.155)
Epoch: [2][310/400] Time 0.365 (0.411) Data 0.000 (0.046) Loss 3.784 (3.154)
Epoch: [2][320/400] Time 0.364 (0.415) Data 0.000 (0.050) Loss 2.958 (3.151)
Epoch: [2][330/400] Time 0.366 (0.413) Data 0.001 (0.049) Loss 2.676 (3.160)
Epoch: [2][340/400] Time 0.366 (0.412) Data 0.001 (0.047) Loss 3.050 (3.167)
Epoch: [2][350/400] Time 0.363 (0.410) Data 0.000 (0.046) Loss 3.733 (3.167)
Epoch: [2][360/400] Time 0.365 (0.414) Data 0.001 (0.049) Loss 3.433 (3.169)
Epoch: [2][370/400] Time 0.366 (0.413) Data 0.000 (0.048) Loss 3.044 (3.165)
Epoch: [2][380/400] Time 0.364 (0.412) Data 0.000 (0.047) Loss 3.781 (3.168)
Epoch: [2][390/400] Time 0.364 (0.415) Data 0.000 (0.050) Loss 3.573 (3.170)
Epoch: [2][400/400] Time 0.365 (0.414) Data 0.001 (0.049) Loss 3.600 (3.168)
==> Create pseudo labels for unlabeled data
Extract Features: [50/51] Time 0.129 (0.158) Data 0.000 (0.030)
Computing jaccard distance...
Jaccard distance computing time cost: 22.187981843948364
==> Statistics for epoch 3: 669 clusters
Epoch: [3][10/400] Time 0.363 (0.432) Data 0.001 (0.064) Loss 0.745 (0.655)
Epoch: [3][20/400] Time 0.362 (0.396) Data 0.000 (0.032) Loss 0.646 (0.626)
Epoch: [3][30/400] Time 0.363 (0.384) Data 0.001 (0.022) Loss 0.527 (0.619)
Epoch: [3][40/400] Time 0.363 (0.379) Data 0.000 (0.016) Loss 0.470 (0.578)
Epoch: [3][50/400] Time 0.361 (0.413) Data 0.000 (0.050) Loss 3.554 (1.092)
Epoch: [3][60/400] Time 0.364 (0.405) Data 0.000 (0.042) Loss 3.596 (1.479)
Epoch: [3][70/400] Time 0.360 (0.399) Data 0.000 (0.036) Loss 3.852 (1.760)
Epoch: [3][80/400] Time 0.364 (0.394) Data 0.000 (0.031) Loss 3.392 (1.988)
Epoch: [3][90/400] Time 0.359 (0.410) Data 0.000 (0.047) Loss 3.664 (2.139)
Epoch: [3][100/400] Time 0.364 (0.406) Data 0.001 (0.043) Loss 3.258 (2.264)
Epoch: [3][110/400] Time 0.351 (0.402) Data 0.000 (0.039) Loss 4.049 (2.381)
Epoch: [3][120/400] Time 0.362 (0.398) Data 0.000 (0.036) Loss 3.549 (2.481)
Epoch: [3][130/400] Time 0.356 (0.410) Data 0.001 (0.047) Loss 3.922 (2.562)
Epoch: [3][140/400] Time 0.356 (0.406) Data 0.001 (0.044) Loss 3.350 (2.643)
Epoch: [3][150/400] Time 0.358 (0.403) Data 0.001 (0.041) Loss 3.777 (2.697)
Epoch: [3][160/400] Time 0.363 (0.400) Data 0.000 (0.038) Loss 3.202 (2.751)
Epoch: [3][170/400] Time 0.363 (0.409) Data 0.000 (0.046) Loss 3.472 (2.796)
Epoch: [3][180/400] Time 0.353 (0.406) Data 0.001 (0.044) Loss 4.451 (2.837)
Epoch: [3][190/400] Time 0.355 (0.404) Data 0.001 (0.042) Loss 3.997 (2.881)
Epoch: [3][200/400] Time 0.364 (0.402) Data 0.000 (0.040) Loss 3.018 (2.906)
Epoch: [3][210/400] Time 0.358 (0.408) Data 0.001 (0.046) Loss 4.063 (2.924)
Epoch: [3][220/400] Time 0.394 (0.406) Data 0.001 (0.044) Loss 3.438 (2.944)
Epoch: [3][230/400] Time 0.365 (0.404) Data 0.000 (0.042) Loss 4.328 (2.966)
Epoch: [3][240/400] Time 0.363 (0.403) Data 0.000 (0.040) Loss 3.005 (2.975)
Epoch: [3][250/400] Time 0.357 (0.408) Data 0.001 (0.046) Loss 3.439 (2.991)
Epoch: [3][260/400] Time 0.366 (0.406) Data 0.001 (0.044) Loss 3.608 (3.002)
Epoch: [3][270/400] Time 0.366 (0.405) Data 0.000 (0.042) Loss 3.404 (3.014)
Epoch: [3][280/400] Time 0.363 (0.403) Data 0.001 (0.041) Loss 3.531 (3.033)
Epoch: [3][290/400] Time 0.355 (0.408) Data 0.000 (0.046) Loss 3.654 (3.046)
Epoch: [3][300/400] Time 0.354 (0.407) Data 0.000 (0.044) Loss 3.869 (3.054)
Epoch: [3][310/400] Time 0.353 (0.405) Data 0.001 (0.043) Loss 3.166 (3.064)
Epoch: [3][320/400] Time 0.365 (0.404) Data 0.000 (0.042) Loss 3.749 (3.077)
Epoch: [3][330/400] Time 0.349 (0.408) Data 0.001 (0.046) Loss 3.487 (3.084)
Epoch: [3][340/400] Time 0.365 (0.407) Data 0.001 (0.045) Loss 3.068 (3.092)
Epoch: [3][350/400] Time 0.366 (0.406) Data 0.001 (0.043) Loss 2.509 (3.098)
Epoch: [3][360/400] Time 0.366 (0.404) Data 0.000 (0.042) Loss 3.394 (3.107)
Epoch: [3][370/400] Time 2.231 (0.408) Data 1.838 (0.046) Loss 2.484 (3.108)
Epoch: [3][380/400] Time 0.404 (0.407) Data 0.001 (0.045) Loss 3.090 (3.111)
Epoch: [3][390/400] Time 0.363 (0.406) Data 0.000 (0.044) Loss 2.966 (3.116)
Epoch: [3][400/400] Time 0.355 (0.405) Data 0.001 (0.043) Loss 2.738 (3.121)
==> Create pseudo labels for unlabeled data
Extract Features: [50/51] Time 0.129 (0.159) Data 0.000 (0.029)
Computing jaccard distance...
Jaccard distance computing time cost: 20.534826517105103
==> Statistics for epoch 4: 718 clusters
Epoch: [4][10/400] Time 0.363 (0.422) Data 0.001 (0.059) Loss 0.710 (0.738)
Epoch: [4][20/400] Time 0.363 (0.390) Data 0.001 (0.030) Loss 0.527 (0.702)
Epoch: [4][30/400] Time 0.364 (0.380) Data 0.000 (0.020) Loss 0.632 (0.662)
Epoch: [4][40/400] Time 0.354 (0.375) Data 0.000 (0.015) Loss 0.507 (0.615)
Epoch: [4][50/400] Time 0.361 (0.407) Data 0.001 (0.046) Loss 3.590 (0.947)
Epoch: [4][60/400] Time 0.365 (0.400) Data 0.000 (0.038) Loss 3.781 (1.381)
Epoch: [4][70/400] Time 0.380 (0.395) Data 0.001 (0.033) Loss 3.560 (1.677)
Epoch: [4][80/400] Time 0.353 (0.391) Data 0.000 (0.029) Loss 3.385 (1.940)
Epoch: [4][90/400] Time 0.336 (0.408) Data 0.001 (0.046) Loss 2.380 (2.110)
Epoch: [4][100/400] Time 0.365 (0.403) Data 0.000 (0.041) Loss 3.410 (2.233)
Epoch: [4][110/400] Time 0.356 (0.399) Data 0.001 (0.038) Loss 3.001 (2.355)
Epoch: [4][120/400] Time 0.365 (0.396) Data 0.000 (0.034) Loss 3.813 (2.443)
Epoch: [4][130/400] Time 0.364 (0.393) Data 0.000 (0.032) Loss 3.336 (2.522)
Epoch: [4][140/400] Time 0.353 (0.404) Data 0.001 (0.043) Loss 3.169 (2.593)
Epoch: [4][150/400] Time 0.364 (0.402) Data 0.001 (0.040) Loss 3.389 (2.639)
Epoch: [4][160/400] Time 0.365 (0.399) Data 0.001 (0.037) Loss 3.830 (2.694)
Epoch: [4][170/400] Time 0.364 (0.397) Data 0.000 (0.035) Loss 3.357 (2.738)
Epoch: [4][180/400] Time 0.365 (0.405) Data 0.001 (0.043) Loss 2.959 (2.768)
Epoch: [4][190/400] Time 0.367 (0.403) Data 0.001 (0.041) Loss 3.563 (2.808)
Epoch: [4][200/400] Time 0.365 (0.401) Data 0.000 (0.039) Loss 3.389 (2.831)
Epoch: [4][210/400] Time 0.365 (0.399) Data 0.001 (0.037) Loss 3.796 (2.872)
Epoch: [4][220/400] Time 0.364 (0.397) Data 0.000 (0.036) Loss 3.907 (2.902)
Epoch: [4][230/400] Time 0.360 (0.404) Data 0.001 (0.042) Loss 3.402 (2.915)
Epoch: [4][240/400] Time 0.356 (0.402) Data 0.001 (0.040) Loss 3.342 (2.935)
Epoch: [4][250/400] Time 0.366 (0.400) Data 0.001 (0.038) Loss 2.512 (2.949)
Epoch: [4][260/400] Time 0.366 (0.399) Data 0.000 (0.037) Loss 3.143 (2.970)
Epoch: [4][270/400] Time 0.364 (0.404) Data 0.000 (0.042) Loss 2.611 (2.978)
Epoch: [4][280/400] Time 0.351 (0.402) Data 0.001 (0.040) Loss 3.454 (2.994)
Epoch: [4][290/400] Time 0.364 (0.401) Data 0.001 (0.039) Loss 3.459 (3.013)
Epoch: [4][300/400] Time 0.365 (0.400) Data 0.001 (0.038) Loss 4.128 (3.031)
Epoch: [4][310/400] Time 0.348 (0.404) Data 0.001 (0.042) Loss 3.815 (3.045)
Epoch: [4][320/400] Time 0.353 (0.403) Data 0.001 (0.041) Loss 2.844 (3.051)
Epoch: [4][330/400] Time 0.364 (0.402) Data 0.000 (0.040) Loss 2.982 (3.059)
Epoch: [4][340/400] Time 0.363 (0.400) Data 0.001 (0.039) Loss 2.965 (3.071)
Epoch: [4][350/400] Time 0.364 (0.399) Data 0.000 (0.037) Loss 3.538 (3.081)
Epoch: [4][360/400] Time 0.353 (0.403) Data 0.000 (0.041) Loss 2.585 (3.086)
Epoch: [4][370/400] Time 0.352 (0.402) Data 0.000 (0.040) Loss 3.671 (3.091)
Epoch: [4][380/400] Time 0.353 (0.401) Data 0.000 (0.039) Loss 3.471 (3.093)
Epoch: [4][390/400] Time 0.354 (0.400) Data 0.000 (0.038) Loss 3.564 (3.103)
Epoch: [4][400/400] Time 0.363 (0.403) Data 0.000 (0.042) Loss 3.386 (3.105)
==> Create pseudo labels for unlabeled data
Extract Features: [50/51] Time 0.129 (0.159) Data 0.000 (0.029)
Computing jaccard distance...
Jaccard distance computing time cost: 19.915706157684326
==> Statistics for epoch 5: 738 clusters
Epoch: [5][10/400] Time 0.363 (0.427) Data 0.001 (0.058) Loss 0.871 (0.671)
Epoch: [5][20/400] Time 0.363 (0.395) Data 0.000 (0.029) Loss 0.594 (0.658)
Epoch: [5][30/400] Time 0.361 (0.384) Data 0.001 (0.020) Loss 0.571 (0.646)
Epoch: [5][40/400] Time 0.362 (0.379) Data 0.000 (0.015) Loss 0.434 (0.608)
Epoch: [5][50/400] Time 0.363 (0.411) Data 0.001 (0.047) Loss 3.662 (0.826)
Epoch: [5][60/400] Time 0.363 (0.404) Data 0.001 (0.039) Loss 3.928 (1.278)
Epoch: [5][70/400] Time 0.363 (0.398) Data 0.001 (0.034) Loss 3.518 (1.610)
Epoch: [5][80/400] Time 0.363 (0.394) Data 0.001 (0.030) Loss 4.034 (1.891)
Epoch: [5][90/400] Time 0.360 (0.391) Data 0.000 (0.026) Loss 3.181 (2.090)
Epoch: [5][100/400] Time 0.363 (0.406) Data 0.001 (0.042) Loss 3.906 (2.223)
Epoch: [5][110/400] Time 0.363 (0.402) Data 0.001 (0.038) Loss 2.889 (2.327)
Epoch: [5][120/400] Time 0.364 (0.399) Data 0.001 (0.035) Loss 3.901 (2.436)
Epoch: [5][130/400] Time 0.372 (0.396) Data 0.001 (0.032) Loss 3.618 (2.512)
Epoch: [5][140/400] Time 0.364 (0.406) Data 0.001 (0.042) Loss 3.443 (2.572)
Epoch: [5][150/400] Time 0.362 (0.403) Data 0.001 (0.039) Loss 3.673 (2.634)
Epoch: [5][160/400] Time 0.362 (0.401) Data 0.001 (0.037) Loss 3.939 (2.700)
Epoch: [5][170/400] Time 0.363 (0.398) Data 0.001 (0.035) Loss 3.513 (2.750)
Epoch: [5][180/400] Time 0.362 (0.396) Data 0.000 (0.033) Loss 3.440 (2.786)
Epoch: [5][190/400] Time 0.364 (0.404) Data 0.001 (0.040) Loss 3.323 (2.821)
Epoch: [5][200/400] Time 0.379 (0.402) Data 0.001 (0.038) Loss 3.773 (2.852)
Epoch: [5][210/400] Time 0.364 (0.400) Data 0.001 (0.037) Loss 3.739 (2.879)
Epoch: [5][220/400] Time 0.365 (0.399) Data 0.001 (0.035) Loss 2.810 (2.905)
Epoch: [5][230/400] Time 0.363 (0.397) Data 0.000 (0.033) Loss 2.995 (2.926)
Epoch: [5][240/400] Time 0.363 (0.403) Data 0.001 (0.039) Loss 3.483 (2.945)
Epoch: [5][250/400] Time 0.364 (0.402) Data 0.001 (0.038) Loss 2.913 (2.960)
Epoch: [5][260/400] Time 0.364 (0.400) Data 0.001 (0.036) Loss 3.559 (2.984)
Epoch: [5][270/400] Time 0.363 (0.399) Data 0.000 (0.035) Loss 2.889 (2.996)
Epoch: [5][280/400] Time 0.363 (0.404) Data 0.000 (0.040) Loss 3.172 (3.010)
Epoch: [5][290/400] Time 0.397 (0.402) Data 0.001 (0.038) Loss 2.851 (3.031)
Epoch: [5][300/400] Time 0.363 (0.401) Data 0.001 (0.037) Loss 3.221 (3.040)
Epoch: [5][310/400] Time 0.404 (0.400) Data 0.001 (0.036) Loss 4.441 (3.056)
Epoch: [5][320/400] Time 0.362 (0.399) Data 0.000 (0.035) Loss 3.435 (3.065)
Epoch: [5][330/400] Time 0.363 (0.403) Data 0.000 (0.039) Loss 2.926 (3.070)
Epoch: [5][340/400] Time 0.363 (0.402) Data 0.001 (0.038) Loss 3.199 (3.078)
Epoch: [5][350/400] Time 0.363 (0.401) Data 0.000 (0.037) Loss 3.741 (3.090)
Epoch: [5][360/400] Time 0.364 (0.400) Data 0.000 (0.036) Loss 3.576 (3.102)
Epoch: [5][370/400] Time 0.354 (0.404) Data 0.000 (0.040) Loss 3.012 (3.114)
Epoch: [5][380/400] Time 0.362 (0.403) Data 0.001 (0.039) Loss 3.448 (3.122)
Epoch: [5][390/400] Time 0.362 (0.402) Data 0.001 (0.038) Loss 3.214 (3.128)
Epoch: [5][400/400] Time 0.363 (0.401) Data 0.000 (0.037) Loss 2.759 (3.135)
==> Create pseudo labels for unlabeled data
Extract Features: [50/51] Time 0.129 (0.156) Data 0.000 (0.028)
Computing jaccard distance...
Jaccard distance computing time cost: 21.006665468215942
==> Statistics for epoch 6: 774 clusters
Epoch: [6][10/400] Time 0.361 (0.421) Data 0.000 (0.055) Loss 0.654 (0.630)
Epoch: [6][20/400] Time 0.361 (0.391) Data 0.000 (0.028) Loss 0.532 (0.637)
Epoch: [6][30/400] Time 0.349 (0.381) Data 0.001 (0.019) Loss 0.706 (0.609)
Epoch: [6][40/400] Time 0.351 (0.376) Data 0.001 (0.014) Loss 0.388 (0.561)
Epoch: [6][50/400] Time 0.351 (0.409) Data 0.001 (0.046) Loss 3.556 (0.656)
Epoch: [6][60/400] Time 0.363 (0.401) Data 0.001 (0.039) Loss 3.634 (1.125)
Epoch: [6][70/400] Time 0.363 (0.394) Data 0.001 (0.033) Loss 3.379 (1.449)
Epoch: [6][80/400] Time 0.363 (0.390) Data 0.001 (0.029) Loss 4.524 (1.719)
Epoch: [6][90/400] Time 0.361 (0.386) Data 0.000 (0.026) Loss 4.387 (1.916)
Epoch: [6][100/400] Time 0.363 (0.401) Data 0.001 (0.040) Loss 3.475 (2.080)
Epoch: [6][110/400] Time 0.362 (0.398) Data 0.001 (0.037) Loss 3.509 (2.204)
Epoch: [6][120/400] Time 0.363 (0.395) Data 0.000 (0.034) Loss 4.077 (2.310)
Epoch: [6][130/400] Time 0.363 (0.392) Data 0.001 (0.031) Loss 3.712 (2.401)
Epoch: [6][140/400] Time 0.363 (0.390) Data 0.000 (0.029) Loss 3.296 (2.459)
Epoch: [6][150/400] Time 0.363 (0.400) Data 0.001 (0.038) Loss 2.994 (2.518)
Epoch: [6][160/400] Time 0.367 (0.397) Data 0.000 (0.036) Loss 3.683 (2.580)
Epoch: [6][170/400] Time 0.363 (0.395) Data 0.001 (0.034) Loss 3.782 (2.641)
Epoch: [6][180/400] Time 0.356 (0.393) Data 0.000 (0.032) Loss 3.207 (2.689)
Epoch: [6][190/400] Time 0.362 (0.392) Data 0.000 (0.030) Loss 3.582 (2.743)
Epoch: [6][200/400] Time 0.354 (0.399) Data 0.001 (0.038) Loss 3.799 (2.784)
Epoch: [6][210/400] Time 0.356 (0.397) Data 0.001 (0.036) Loss 4.227 (2.824)
Epoch: [6][220/400] Time 0.365 (0.396) Data 0.000 (0.034) Loss 3.427 (2.857)
Epoch: [6][230/400] Time 0.365 (0.394) Data 0.001 (0.033) Loss 3.153 (2.878)
Epoch: [6][240/400] Time 0.361 (0.393) Data 0.000 (0.032) Loss 2.961 (2.900)
Epoch: [6][250/400] Time 0.365 (0.399) Data 0.001 (0.037) Loss 3.756 (2.919)
Epoch: [6][260/400] Time 0.365 (0.398) Data 0.000 (0.036) Loss 3.699 (2.935)
Epoch: [6][270/400] Time 0.363 (0.397) Data 0.001 (0.035) Loss 3.538 (2.951)
Epoch: [6][280/400] Time 0.356 (0.395) Data 0.000 (0.033) Loss 3.931 (2.973)
Epoch: [6][290/400] Time 0.397 (0.401) Data 0.000 (0.039) Loss 3.717 (2.991)
Epoch: [6][300/400] Time 0.364 (0.399) Data 0.000 (0.037) Loss 3.070 (3.006)
Epoch: [6][310/400] Time 0.364 (0.398) Data 0.001 (0.036) Loss 3.666 (3.022)
Epoch: [6][320/400] Time 0.364 (0.397) Data 0.000 (0.035) Loss 3.174 (3.034)
Epoch: [6][330/400] Time 0.365 (0.396) Data 0.000 (0.034) Loss 3.553 (3.049)
Epoch: [6][340/400] Time 0.364 (0.401) Data 0.001 (0.038) Loss 3.590 (3.060)
Epoch: [6][350/400] Time 0.360 (0.400) Data 0.001 (0.037) Loss 3.599 (3.066)
Epoch: [6][360/400] Time 0.353 (0.399) Data 0.001 (0.036) Loss 4.292 (3.077)
Epoch: [6][370/400] Time 0.364 (0.398) Data 0.001 (0.035) Loss 3.490 (3.088)
Epoch: [6][380/400] Time 0.365 (0.397) Data 0.000 (0.034) Loss 2.599 (3.097)
Epoch: [6][390/400] Time 0.367 (0.400) Data 0.001 (0.038) Loss 2.956 (3.100)
Epoch: [6][400/400] Time 0.358 (0.399) Data 0.000 (0.037) Loss 3.397 (3.106)
==> Create pseudo labels for unlabeled data
Extract Features: [50/51] Time 0.128 (0.158) Data 0.000 (0.028)
Computing jaccard distance...
Jaccard distance computing time cost: 20.961553812026978
==> Statistics for epoch 7: 803 clusters
Epoch: [7][10/400] Time 0.362 (0.417) Data 0.001 (0.049) Loss 0.741 (0.740)
Epoch: [7][20/400] Time 0.364 (0.392) Data 0.000 (0.025) Loss 0.576 (0.666)
Epoch: [7][30/400] Time 0.363 (0.381) Data 0.001 (0.017) Loss 0.634 (0.641)
Epoch: [7][40/400] Time 0.363 (0.376) Data 0.000 (0.013) Loss 0.411 (0.603)
Epoch: [7][50/400] Time 0.363 (0.374) Data 0.000 (0.010) Loss 0.468 (0.554)
Epoch: [7][60/400] Time 0.364 (0.401) Data 0.001 (0.038) Loss 3.117 (1.051)
Epoch: [7][70/400] Time 0.364 (0.396) Data 0.000 (0.033) Loss 3.634 (1.421)
Epoch: [7][80/400] Time 0.365 (0.392) Data 0.000 (0.029) Loss 3.693 (1.686)
Epoch: [7][90/400] Time 0.365 (0.388) Data 0.000 (0.025) Loss 3.826 (1.910)
Epoch: [7][100/400] Time 0.363 (0.386) Data 0.000 (0.023) Loss 3.239 (2.086)
Epoch: [7][110/400] Time 0.363 (0.399) Data 0.001 (0.037) Loss 3.253 (2.211)
Epoch: [7][120/400] Time 0.359 (0.397) Data 0.001 (0.034) Loss 2.935 (2.302)
Epoch: [7][130/400] Time 0.363 (0.394) Data 0.001 (0.031) Loss 3.105 (2.382)
Epoch: [7][140/400] Time 0.365 (0.392) Data 0.001 (0.029) Loss 2.918 (2.452)
Epoch: [7][150/400] Time 0.364 (0.390) Data 0.000 (0.027) Loss 3.601 (2.498)
Epoch: [7][160/400] Time 0.356 (0.399) Data 0.001 (0.036) Loss 3.396 (2.568)
Epoch: [7][170/400] Time 0.355 (0.396) Data 0.001 (0.033) Loss 3.363 (2.623)
Epoch: [7][180/400] Time 0.356 (0.394) Data 0.000 (0.032) Loss 3.293 (2.673)
Epoch: [7][190/400] Time 0.353 (0.393) Data 0.001 (0.030) Loss 3.481 (2.716)
Epoch: [7][200/400] Time 0.363 (0.391) Data 0.000 (0.029) Loss 3.512 (2.769)
Epoch: [7][210/400] Time 0.407 (0.398) Data 0.001 (0.035) Loss 3.056 (2.812)
Epoch: [7][220/400] Time 0.364 (0.397) Data 0.001 (0.034) Loss 3.987 (2.845)
Epoch: [7][230/400] Time 0.358 (0.395) Data 0.000 (0.032) Loss 4.364 (2.876)
Epoch: [7][240/400] Time 0.364 (0.394) Data 0.000 (0.031) Loss 4.100 (2.909)
Epoch: [7][250/400] Time 0.363 (0.393) Data 0.000 (0.030) Loss 3.368 (2.932)
Epoch: [7][260/400] Time 0.366 (0.398) Data 0.000 (0.035) Loss 3.815 (2.958)
Epoch: [7][270/400] Time 0.364 (0.397) Data 0.001 (0.034) Loss 4.178 (2.977)
Epoch: [7][280/400] Time 0.366 (0.395) Data 0.001 (0.033) Loss 3.080 (2.994)
Epoch: [7][290/400] Time 0.366 (0.394) Data 0.000 (0.031) Loss 3.532 (3.008)
Epoch: [7][300/400] Time 0.364 (0.393) Data 0.000 (0.030) Loss 3.602 (3.028)
Epoch: [7][310/400] Time 0.363 (0.398) Data 0.001 (0.035) Loss 2.980 (3.041)
Epoch: [7][320/400] Time 0.354 (0.397) Data 0.001 (0.034) Loss 3.400 (3.052)
Epoch: [7][330/400] Time 0.362 (0.396) Data 0.001 (0.033) Loss 3.165 (3.062)
Epoch: [7][340/400] Time 0.353 (0.395) Data 0.001 (0.032) Loss 3.562 (3.073)
Epoch: [7][350/400] Time 0.363 (0.394) Data 0.000 (0.031) Loss 3.479 (3.082)
Epoch: [7][360/400] Time 0.363 (0.398) Data 0.001 (0.035) Loss 2.918 (3.089)
Epoch: [7][370/400] Time 0.364 (0.397) Data 0.001 (0.034) Loss 3.367 (3.101)
Epoch: [7][380/400] Time 0.365 (0.396) Data 0.001 (0.033) Loss 3.145 (3.108)
Epoch: [7][390/400] Time 0.393 (0.395) Data 0.001 (0.033) Loss 2.261 (3.115)
Epoch: [7][400/400] Time 0.364 (0.394) Data 0.000 (0.032) Loss 3.137 (3.124)
==> Create pseudo labels for unlabeled data
Extract Features: [50/51] Time 0.127 (0.161) Data 0.000 (0.030)
Computing jaccard distance...
Jaccard distance computing time cost: 20.03676414489746
==> Statistics for epoch 8: 799 clusters
Epoch: [8][10/400] Time 0.365 (0.420) Data 0.001 (0.049) Loss 0.598 (0.707)
Epoch: [8][20/400] Time 0.359 (0.392) Data 0.001 (0.025) Loss 0.571 (0.668)
Epoch: [8][30/400] Time 0.366 (0.382) Data 0.001 (0.017) Loss 0.589 (0.644)
Epoch: [8][40/400] Time 0.364 (0.377) Data 0.001 (0.013) Loss 0.573 (0.621)
Epoch: [8][50/400] Time 2.100 (0.409) Data 1.716 (0.044) Loss 3.106 (0.639)
Epoch: [8][60/400] Time 0.356 (0.401) Data 0.000 (0.037) Loss 3.788 (1.123)
Epoch: [8][70/400] Time 0.352 (0.395) Data 0.001 (0.032) Loss 3.493 (1.449)
Epoch: [8][80/400] Time 0.353 (0.390) Data 0.001 (0.028) Loss 2.146 (1.712)
Epoch: [8][90/400] Time 0.353 (0.387) Data 0.001 (0.025) Loss 2.986 (1.912)
Epoch: [8][100/400] Time 0.352 (0.402) Data 0.001 (0.040) Loss 3.582 (2.088)
Epoch: [8][110/400] Time 0.365 (0.399) Data 0.001 (0.036) Loss 3.188 (2.201)
Epoch: [8][120/400] Time 0.364 (0.396) Data 0.001 (0.033) Loss 3.841 (2.302)
Epoch: [8][130/400] Time 0.366 (0.394) Data 0.001 (0.031) Loss 2.937 (2.373)
Epoch: [8][140/400] Time 0.360 (0.392) Data 0.001 (0.029) Loss 3.857 (2.442)
Epoch: [8][150/400] Time 0.418 (0.402) Data 0.001 (0.039) Loss 3.660 (2.511)
Epoch: [8][160/400] Time 0.364 (0.399) Data 0.000 (0.036) Loss 3.513 (2.573)
Epoch: [8][170/400] Time 0.362 (0.397) Data 0.001 (0.034) Loss 3.805 (2.619)
Epoch: [8][180/400] Time 0.361 (0.395) Data 0.001 (0.032) Loss 3.434 (2.668)
Epoch: [8][190/400] Time 0.363 (0.393) Data 0.000 (0.031) Loss 3.731 (2.715)
Epoch: [8][200/400] Time 0.358 (0.401) Data 0.000 (0.038) Loss 2.796 (2.752)
Epoch: [8][210/400] Time 0.364 (0.399) Data 0.001 (0.036) Loss 3.727 (2.783)
Epoch: [8][220/400] Time 0.363 (0.397) Data 0.000 (0.034) Loss 3.256 (2.807)
Epoch: [8][230/400] Time 0.363 (0.395) Data 0.001 (0.033) Loss 3.585 (2.840)
Epoch: [8][240/400] Time 0.363 (0.394) Data 0.000 (0.032) Loss 3.574 (2.871)
Epoch: [8][250/400] Time 0.363 (0.400) Data 0.000 (0.037) Loss 3.430 (2.897)
Epoch: [8][260/400] Time 0.355 (0.398) Data 0.000 (0.036) Loss 3.468 (2.922)
Epoch: [8][270/400] Time 0.363 (0.397) Data 0.000 (0.034) Loss 3.489 (2.942)
Epoch: [8][280/400] Time 0.353 (0.396) Data 0.001 (0.033) Loss 3.625 (2.956)
Epoch: [8][290/400] Time 0.363 (0.394) Data 0.000 (0.032) Loss 3.313 (2.974)
Epoch: [8][300/400] Time 0.365 (0.399) Data 0.001 (0.037) Loss 3.589 (2.987)
Epoch: [8][310/400] Time 0.365 (0.398) Data 0.001 (0.036) Loss 3.810 (3.009)
Epoch: [8][320/400] Time 0.355 (0.397) Data 0.001 (0.035) Loss 3.761 (3.022)
Epoch: [8][330/400] Time 0.364 (0.396) Data 0.000 (0.034) Loss 4.115 (3.036)
Epoch: [8][340/400] Time 0.361 (0.395) Data 0.000 (0.033) Loss 4.035 (3.048)
Epoch: [8][350/400] Time 0.364 (0.399) Data 0.001 (0.037) Loss 3.287 (3.061)
Epoch: [8][360/400] Time 0.363 (0.398) Data 0.000 (0.036) Loss 3.489 (3.067)
Epoch: [8][370/400] Time 0.364 (0.397) Data 0.001 (0.035) Loss 3.295 (3.074)
Epoch: [8][380/400] Time 0.364 (0.396) Data 0.000 (0.034) Loss 4.110 (3.090)
Epoch: [8][390/400] Time 0.364 (0.395) Data 0.000 (0.033) Loss 2.863 (3.096)
Epoch: [8][400/400] Time 0.354 (0.399) Data 0.001 (0.037) Loss 3.270 (3.097)
==> Create pseudo labels for unlabeled data
Extract Features: [50/51] Time 0.129 (0.160) Data 0.000 (0.030)
Computing jaccard distance...
Jaccard distance computing time cost: 19.664186716079712
==> Statistics for epoch 9: 824 clusters
Epoch: [9][10/400] Time 0.363 (0.416) Data 0.001 (0.058) Loss 0.592 (0.635)
Epoch: [9][20/400] Time 0.364 (0.389) Data 0.001 (0.029) Loss 0.543 (0.622)
Epoch: [9][30/400] Time 0.373 (0.379) Data 0.001 (0.020) Loss 0.464 (0.579)
Epoch: [9][40/400] Time 0.355 (0.373) Data 0.001 (0.015) Loss 0.486 (0.557)
Epoch: [9][50/400] Time 0.363 (0.370) Data 0.000 (0.012) Loss 0.417 (0.524)
Epoch: [9][60/400] Time 0.352 (0.397) Data 0.001 (0.039) Loss 3.998 (0.975)
Epoch: [9][70/400] Time 0.363 (0.394) Data 0.001 (0.034) Loss 3.432 (1.335)
Epoch: [9][80/400] Time 0.352 (0.389) Data 0.000 (0.030) Loss 2.783 (1.618)
Epoch: [9][90/400] Time 0.353 (0.386) Data 0.000 (0.026) Loss 3.003 (1.821)
Epoch: [9][100/400] Time 0.354 (0.383) Data 0.000 (0.024) Loss 3.782 (2.005)
Epoch: [9][110/400] Time 0.354 (0.398) Data 0.001 (0.038) Loss 3.219 (2.122)
Epoch: [9][120/400] Time 0.369 (0.395) Data 0.001 (0.035) Loss 3.513 (2.217)
Epoch: [9][130/400] Time 0.356 (0.392) Data 0.001 (0.032) Loss 3.717 (2.314)
Epoch: [9][140/400] Time 0.354 (0.389) Data 0.001 (0.030) Loss 3.731 (2.382)
Epoch: [9][150/400] Time 0.364 (0.387) Data 0.000 (0.028) Loss 3.522 (2.460)
Epoch: [9][160/400] Time 0.364 (0.398) Data 0.000 (0.038) Loss 3.247 (2.517)
Epoch: [9][170/400] Time 0.358 (0.395) Data 0.001 (0.036) Loss 3.656 (2.571)
Epoch: [9][180/400] Time 0.370 (0.393) Data 0.000 (0.034) Loss 3.259 (2.614)
Epoch: [9][190/400] Time 0.360 (0.391) Data 0.000 (0.032) Loss 3.847 (2.663)
Epoch: [9][200/400] Time 0.365 (0.390) Data 0.000 (0.030) Loss 2.787 (2.698)
Epoch: [9][210/400] Time 0.353 (0.397) Data 0.001 (0.037) Loss 3.997 (2.739)
Epoch: [9][220/400] Time 0.353 (0.395) Data 0.001 (0.036) Loss 3.220 (2.773)
Epoch: [9][230/400] Time 0.363 (0.393) Data 0.000 (0.034) Loss 2.794 (2.797)
Epoch: [9][240/400] Time 0.353 (0.391) Data 0.000 (0.033) Loss 3.842 (2.828)
Epoch: [9][250/400] Time 0.364 (0.390) Data 0.000 (0.031) Loss 3.323 (2.856)
Epoch: [9][260/400] Time 0.355 (0.396) Data 0.000 (0.037) Loss 2.733 (2.875)
Epoch: [9][270/400] Time 0.353 (0.395) Data 0.001 (0.036) Loss 2.985 (2.894)
Epoch: [9][280/400] Time 0.352 (0.393) Data 0.001 (0.035) Loss 3.120 (2.913)
Epoch: [9][290/400] Time 0.352 (0.392) Data 0.001 (0.034) Loss 2.927 (2.932)
Epoch: [9][300/400] Time 0.364 (0.391) Data 0.000 (0.032) Loss 3.177 (2.946)
Epoch: [9][310/400] Time 0.353 (0.396) Data 0.001 (0.037) Loss 3.438 (2.960)
Epoch: [9][320/400] Time 0.353 (0.395) Data 0.001 (0.036) Loss 3.001 (2.971)
Epoch: [9][330/400] Time 0.374 (0.394) Data 0.001 (0.035) Loss 2.812 (2.981)
Epoch: [9][340/400] Time 0.355 (0.393) Data 0.001 (0.034) Loss 3.710 (3.000)
Epoch: [9][350/400] Time 0.356 (0.392) Data 0.001 (0.033) Loss 3.407 (3.016)
Epoch: [9][360/400] Time 0.354 (0.395) Data 0.001 (0.037) Loss 2.849 (3.028)
Epoch: [9][370/400] Time 0.352 (0.394) Data 0.001 (0.036) Loss 3.054 (3.037)
Epoch: [9][380/400] Time 0.354 (0.394) Data 0.001 (0.035) Loss 3.148 (3.037)
Epoch: [9][390/400] Time 0.351 (0.393) Data 0.001 (0.034) Loss 3.203 (3.042)
Epoch: [9][400/400] Time 0.354 (0.392) Data 0.001 (0.033) Loss 3.732 (3.048)
Extract Features: [50/76] Time 0.133 (0.159) Data 0.000 (0.029)
Mean AP: 53.8%
* Finished epoch 9 model mAP: 53.8% best: 53.8% *
==> Create pseudo labels for unlabeled data
Extract Features: [50/51] Time 0.128 (0.161) Data 0.000 (0.029)
Computing jaccard distance...
Jaccard distance computing time cost: 19.404287099838257
==> Statistics for epoch 10: 832 clusters
Epoch: [10][10/400] Time 0.352 (0.426) Data 0.001 (0.059) Loss 0.724 (0.652)
Epoch: [10][20/400] Time 0.352 (0.389) Data 0.001 (0.030) Loss 0.703 (0.629)
Epoch: [10][30/400] Time 0.352 (0.377) Data 0.001 (0.020) Loss 0.772 (0.608)
Epoch: [10][40/400] Time 0.353 (0.371) Data 0.001 (0.015) Loss 0.289 (0.561)
Epoch: [10][50/400] Time 0.363 (0.370) Data 0.000 (0.012) Loss 0.215 (0.538)
Epoch: [10][60/400] Time 0.354 (0.396) Data 0.000 (0.039) Loss 3.770 (0.915)
Epoch: [10][70/400] Time 0.354 (0.391) Data 0.001 (0.033) Loss 3.789 (1.271)
Epoch: [10][80/400] Time 0.354 (0.387) Data 0.000 (0.029) Loss 3.113 (1.542)
Epoch: [10][90/400] Time 0.352 (0.383) Data 0.001 (0.026) Loss 3.385 (1.774)
Epoch: [10][100/400] Time 0.363 (0.380) Data 0.000 (0.024) Loss 3.141 (1.958)
Epoch: [10][110/400] Time 0.353 (0.395) Data 0.001 (0.038) Loss 3.547 (2.085)
Epoch: [10][120/400] Time 0.352 (0.392) Data 0.001 (0.035) Loss 3.849 (2.198)
Epoch: [10][130/400] Time 0.355 (0.389) Data 0.001 (0.032) Loss 3.358 (2.279)
Epoch: [10][140/400] Time 0.357 (0.388) Data 0.000 (0.030) Loss 2.969 (2.351)
Epoch: [10][150/400] Time 0.369 (0.386) Data 0.000 (0.028) Loss 3.277 (2.412)
Epoch: [10][160/400] Time 0.350 (0.395) Data 0.001 (0.037) Loss 2.731 (2.463)
Epoch: [10][170/400] Time 0.369 (0.393) Data 0.001 (0.035) Loss 3.326 (2.526)
Epoch: [10][180/400] Time 0.353 (0.391) Data 0.000 (0.033) Loss 3.250 (2.590)
Epoch: [10][190/400] Time 0.359 (0.389) Data 0.000 (0.031) Loss 4.185 (2.639)
Epoch: [10][200/400] Time 0.354 (0.387) Data 0.000 (0.030) Loss 3.112 (2.672)
Epoch: [10][210/400] Time 0.364 (0.395) Data 0.001 (0.037) Loss 3.719 (2.719)
Epoch: [10][220/400] Time 0.354 (0.393) Data 0.001 (0.035) Loss 3.780 (2.747)
Epoch: [10][230/400] Time 0.353 (0.391) Data 0.001 (0.033) Loss 3.101 (2.770)
Epoch: [10][240/400] Time 0.342 (0.390) Data 0.001 (0.032) Loss 3.630 (2.795)
Epoch: [10][250/400] Time 0.353 (0.388) Data 0.001 (0.031) Loss 3.657 (2.832)
Epoch: [10][260/400] Time 0.365 (0.387) Data 0.000 (0.030) Loss 3.088 (2.845)
Epoch: [10][270/400] Time 0.370 (0.393) Data 0.001 (0.035) Loss 3.160 (2.863)
Epoch: [10][280/400] Time 0.377 (0.392) Data 0.000 (0.034) Loss 3.240 (2.878)
Epoch: [10][290/400] Time 0.362 (0.391) Data 0.001 (0.033) Loss 2.841 (2.893)
Epoch: [10][300/400] Time 0.350 (0.390) Data 0.001 (0.032) Loss 3.499 (2.909)
Epoch: [10][310/400] Time 0.363 (0.389) Data 0.000 (0.031) Loss 3.529 (2.928)
Epoch: [10][320/400] Time 0.352 (0.394) Data 0.001 (0.035) Loss 3.590 (2.945)
Epoch: [10][330/400] Time 0.352 (0.392) Data 0.001 (0.034) Loss 3.562 (2.963)
Epoch: [10][340/400] Time 0.353 (0.391) Data 0.000 (0.033) Loss 3.800 (2.979)
Epoch: [10][350/400] Time 0.352 (0.390) Data 0.001 (0.032) Loss 3.227 (2.986)
Epoch: [10][360/400] Time 0.364 (0.389) Data 0.000 (0.032) Loss 4.077 (3.000)
Epoch: [10][370/400] Time 0.355 (0.394) Data 0.000 (0.036) Loss 3.087 (3.014)
Epoch: [10][380/400] Time 0.361 (0.393) Data 0.000 (0.035) Loss 3.346 (3.021)
Epoch: [10][390/400] Time 0.371 (0.392) Data 0.000 (0.034) Loss 3.906 (3.031)
Epoch: [10][400/400] Time 0.352 (0.391) Data 0.001 (0.033) Loss 3.951 (3.036)
==> Create pseudo labels for unlabeled data
Extract Features: [50/51] Time 0.129 (0.161) Data 0.000 (0.028)
Computing jaccard distance...
Jaccard distance computing time cost: 19.264953136444092
==> Statistics for epoch 11: 820 clusters
Epoch: [11][10/400] Time 0.361 (0.425) Data 0.001 (0.066) Loss 0.612 (0.591)
Epoch: [11][20/400] Time 0.364 (0.391) Data 0.000 (0.033) Loss 0.553 (0.592)
Epoch: [11][30/400] Time 0.361 (0.379) Data 0.001 (0.022) Loss 0.460 (0.592)
Epoch: [11][40/400] Time 0.351 (0.374) Data 0.000 (0.017) Loss 0.447 (0.555)
Epoch: [11][50/400] Time 0.362 (0.371) Data 0.000 (0.013) Loss 0.305 (0.512)
Epoch: [11][60/400] Time 0.355 (0.399) Data 0.000 (0.041) Loss 3.779 (0.909)
Epoch: [11][70/400] Time 0.351 (0.393) Data 0.000 (0.035) Loss 3.464 (1.264)
Epoch: [11][80/400] Time 0.354 (0.389) Data 0.000 (0.031) Loss 3.233 (1.546)
Epoch: [11][90/400] Time 0.354 (0.385) Data 0.000 (0.027) Loss 3.937 (1.768)
Epoch: [11][100/400] Time 0.362 (0.382) Data 0.000 (0.025) Loss 3.272 (1.956)
Epoch: [11][110/400] Time 0.351 (0.396) Data 0.000 (0.038) Loss 3.169 (2.079)
Epoch: [11][120/400] Time 0.357 (0.393) Data 0.001 (0.035) Loss 3.332 (2.180)
Epoch: [11][130/400] Time 0.352 (0.389) Data 0.000 (0.032) Loss 3.293 (2.266)
Epoch: [11][140/400] Time 0.351 (0.387) Data 0.001 (0.030) Loss 3.317 (2.349)
Epoch: [11][150/400] Time 0.362 (0.385) Data 0.000 (0.028) Loss 3.531 (2.424)
Epoch: [11][160/400] Time 0.351 (0.394) Data 0.001 (0.037) Loss 3.524 (2.475)
Epoch: [11][170/400] Time 0.351 (0.392) Data 0.001 (0.035) Loss 2.960 (2.529)
Epoch: [11][180/400] Time 0.351 (0.390) Data 0.001 (0.033) Loss 3.660 (2.580)
Epoch: [11][190/400] Time 0.365 (0.388) Data 0.001 (0.032) Loss 2.995 (2.623)
Epoch: [11][200/400] Time 0.363 (0.387) Data 0.000 (0.030) Loss 3.274 (2.664)
Epoch: [11][210/400] Time 0.362 (0.395) Data 0.001 (0.038) Loss 3.163 (2.692)
Epoch: [11][220/400] Time 0.352 (0.393) Data 0.001 (0.036) Loss 3.408 (2.712)
Epoch: [11][230/400] Time 0.363 (0.392) Data 0.001 (0.035) Loss 3.713 (2.756)
Epoch: [11][240/400] Time 0.354 (0.390) Data 0.001 (0.033) Loss 2.348 (2.772)
Epoch: [11][250/400] Time 0.364 (0.389) Data 0.000 (0.032) Loss 3.499 (2.798)
Epoch: [11][260/400] Time 0.352 (0.395) Data 0.000 (0.038) Loss 4.510 (2.814)
Epoch: [11][270/400] Time 0.372 (0.394) Data 0.001 (0.036) Loss 3.749 (2.832)
Epoch: [11][280/400] Time 0.352 (0.393) Data 0.001 (0.035) Loss 2.962 (2.854)
Epoch: [11][290/400] Time 0.362 (0.392) Data 0.001 (0.034) Loss 3.504 (2.875)
Epoch: [11][300/400] Time 0.363 (0.390) Data 0.000 (0.033) Loss 3.720 (2.880)
Epoch: [11][310/400] Time 0.351 (0.395) Data 0.000 (0.037) Loss 2.789 (2.896)
Epoch: [11][320/400] Time 0.363 (0.394) Data 0.001 (0.036) Loss 3.354 (2.910)
Epoch: [11][330/400] Time 0.355 (0.393) Data 0.001 (0.035) Loss 2.951 (2.927)
Epoch: [11][340/400] Time 0.364 (0.392) Data 0.001 (0.034) Loss 3.100 (2.945)
Epoch: [11][350/400] Time 0.354 (0.391) Data 0.001 (0.033) Loss 3.269 (2.954)
Epoch: [11][360/400] Time 0.353 (0.395) Data 0.001 (0.037) Loss 3.439 (2.968)
Epoch: [11][370/400] Time 0.353 (0.394) Data 0.001 (0.036) Loss 2.336 (2.975)
Epoch: [11][380/400] Time 0.353 (0.393) Data 0.001 (0.035) Loss 3.314 (2.985)
Epoch: [11][390/400] Time 0.352 (0.392) Data 0.001 (0.034) Loss 3.860 (2.992)
Epoch: [11][400/400] Time 0.354 (0.391) Data 0.001 (0.033) Loss 2.886 (3.006)
==> Create pseudo labels for unlabeled data
Extract Features: [50/51] Time 0.129 (0.161) Data 0.000 (0.028)
Computing jaccard distance...
Jaccard distance computing time cost: 19.479636669158936
==> Statistics for epoch 12: 823 clusters
Epoch: [12][10/400] Time 0.350 (0.406) Data 0.000 (0.049) Loss 0.580 (0.660)
Epoch: [12][20/400] Time 0.352 (0.381) Data 0.000 (0.025) Loss 0.551 (0.625)
Epoch: [12][30/400] Time 0.368 (0.373) Data 0.001 (0.017) Loss 0.421 (0.581)
Epoch: [12][40/400] Time 0.355 (0.369) Data 0.001 (0.013) Loss 0.313 (0.540)
Epoch: [12][50/400] Time 0.364 (0.367) Data 0.000 (0.010) Loss 0.316 (0.492)
Epoch: [12][60/400] Time 0.352 (0.394) Data 0.000 (0.037) Loss 3.107 (0.885)
Epoch: [12][70/400] Time 0.377 (0.389) Data 0.000 (0.031) Loss 3.758 (1.241)
Epoch: [12][80/400] Time 0.352 (0.385) Data 0.000 (0.028) Loss 3.670 (1.538)
Epoch: [12][90/400] Time 0.351 (0.381) Data 0.001 (0.025) Loss 2.967 (1.747)
Epoch: [12][100/400] Time 0.363 (0.379) Data 0.000 (0.022) Loss 3.561 (1.928)
Epoch: [12][110/400] Time 0.353 (0.394) Data 0.001 (0.036) Loss 3.007 (2.034)
Epoch: [12][120/400] Time 0.352 (0.390) Data 0.000 (0.033) Loss 3.539 (2.135)
Epoch: [12][130/400] Time 0.352 (0.388) Data 0.001 (0.031) Loss 3.415 (2.224)
Epoch: [12][140/400] Time 0.375 (0.385) Data 0.001 (0.029) Loss 3.808 (2.296)
Epoch: [12][150/400] Time 0.364 (0.384) Data 0.000 (0.027) Loss 3.084 (2.358)
Epoch: [12][160/400] Time 0.352 (0.393) Data 0.000 (0.036) Loss 2.886 (2.419)
Epoch: [12][170/400] Time 0.354 (0.391) Data 0.000 (0.034) Loss 3.283 (2.467)
Epoch: [12][180/400] Time 0.354 (0.389) Data 0.000 (0.032) Loss 3.750 (2.515)
Epoch: [12][190/400] Time 0.353 (0.387) Data 0.000 (0.030) Loss 3.949 (2.560)
Epoch: [12][200/400] Time 0.363 (0.386) Data 0.000 (0.029) Loss 3.164 (2.605)
Epoch: [12][210/400] Time 0.352 (0.394) Data 0.001 (0.037) Loss 3.314 (2.634)
Epoch: [12][220/400] Time 0.351 (0.392) Data 0.000 (0.035) Loss 3.724 (2.659)
Epoch: [12][230/400] Time 0.353 (0.391) Data 0.001 (0.034) Loss 3.433 (2.687)
Epoch: [12][240/400] Time 0.351 (0.389) Data 0.000 (0.032) Loss 3.072 (2.719)
Epoch: [12][250/400] Time 0.361 (0.388) Data 0.000 (0.031) Loss 2.990 (2.742)
Epoch: [12][260/400] Time 0.354 (0.393) Data 0.000 (0.037) Loss 2.773 (2.763)
Epoch: [12][270/400] Time 0.362 (0.392) Data 0.001 (0.035) Loss 3.360 (2.783)
Epoch: [12][280/400] Time 0.353 (0.391) Data 0.000 (0.034) Loss 3.166 (2.801)
Epoch: [12][290/400] Time 0.362 (0.390) Data 0.000 (0.033) Loss 2.688 (2.824)
Epoch: [12][300/400] Time 0.362 (0.389) Data 0.000 (0.032) Loss 2.819 (2.839)
Epoch: [12][310/400] Time 0.352 (0.394) Data 0.000 (0.037) Loss 3.205 (2.846)
Epoch: [12][320/400] Time 0.394 (0.393) Data 0.001 (0.036) Loss 3.492 (2.863)
Epoch: [12][330/400] Time 0.353 (0.392) Data 0.000 (0.035) Loss 3.305 (2.868)
Epoch: [12][340/400] Time 0.362 (0.391) Data 0.000 (0.034) Loss 3.045 (2.877)
Epoch: [12][350/400] Time 0.354 (0.390) Data 0.000 (0.033) Loss 3.212 (2.886)
Epoch: [12][360/400] Time 0.351 (0.394) Data 0.000 (0.037) Loss 2.953 (2.895)
Epoch: [12][370/400] Time 0.355 (0.393) Data 0.000 (0.036) Loss 3.228 (2.902)
Epoch: [12][380/400] Time 0.353 (0.392) Data 0.001 (0.035) Loss 2.765 (2.906)
Epoch: [12][390/400] Time 0.376 (0.392) Data 0.000 (0.034) Loss 3.026 (2.912)
Epoch: [12][400/400] Time 0.363 (0.391) Data 0.001 (0.033) Loss 2.972 (2.922)
==> Create pseudo labels for unlabeled data
Extract Features: [50/51] Time 0.129 (0.160) Data 0.000 (0.029)
Computing jaccard distance...
Jaccard distance computing time cost: 19.265992641448975
==> Statistics for epoch 13: 795 clusters
Epoch: [13][10/400] Time 0.350 (0.426) Data 0.000 (0.053) Loss 0.824 (0.678)
Epoch: [13][20/400] Time 0.352 (0.391) Data 0.001 (0.027) Loss 0.324 (0.580)
Epoch: [13][30/400] Time 0.352 (0.378) Data 0.000 (0.018) Loss 0.401 (0.550)
Epoch: [13][40/400] Time 0.351 (0.372) Data 0.000 (0.014) Loss 0.377 (0.512)
Epoch: [13][50/400] Time 2.274 (0.408) Data 1.869 (0.048) Loss 3.357 (0.535)
Epoch: [13][60/400] Time 0.352 (0.399) Data 0.000 (0.040) Loss 3.581 (0.979)
Epoch: [13][70/400] Time 0.355 (0.392) Data 0.000 (0.035) Loss 4.060 (1.349)
Epoch: [13][80/400] Time 0.352 (0.387) Data 0.000 (0.030) Loss 3.170 (1.605)
Epoch: [13][90/400] Time 0.353 (0.384) Data 0.000 (0.027) Loss 3.976 (1.797)
Epoch: [13][100/400] Time 0.378 (0.401) Data 0.000 (0.043) Loss 2.443 (1.947)
Epoch: [13][110/400] Time 0.362 (0.397) Data 0.000 (0.039) Loss 3.263 (2.076)
Epoch: [13][120/400] Time 0.362 (0.394) Data 0.000 (0.036) Loss 3.032 (2.173)
Epoch: [13][130/400] Time 0.364 (0.391) Data 0.000 (0.033) Loss 3.170 (2.243)
Epoch: [13][140/400] Time 0.362 (0.389) Data 0.001 (0.031) Loss 3.357 (2.306)
Epoch: [13][150/400] Time 0.356 (0.399) Data 0.000 (0.040) Loss 3.208 (2.359)
Epoch: [13][160/400] Time 0.359 (0.396) Data 0.001 (0.038) Loss 3.320 (2.415)
Epoch: [13][170/400] Time 0.356 (0.394) Data 0.000 (0.036) Loss 3.595 (2.476)
Epoch: [13][180/400] Time 0.353 (0.392) Data 0.001 (0.034) Loss 3.498 (2.532)
Epoch: [13][190/400] Time 0.363 (0.390) Data 0.000 (0.032) Loss 3.206 (2.583)
Epoch: [13][200/400] Time 0.354 (0.399) Data 0.000 (0.040) Loss 3.678 (2.616)
Epoch: [13][210/400] Time 0.353 (0.397) Data 0.001 (0.038) Loss 3.085 (2.651)
Epoch: [13][220/400] Time 0.354 (0.395) Data 0.000 (0.036) Loss 2.938 (2.679)
Epoch: [13][230/400] Time 0.352 (0.393) Data 0.001 (0.035) Loss 3.427 (2.707)
Epoch: [13][240/400] Time 0.363 (0.392) Data 0.000 (0.033) Loss 3.214 (2.734)
Epoch: [13][250/400] Time 0.354 (0.397) Data 0.001 (0.039) Loss 3.185 (2.753)
Epoch: [13][260/400] Time 0.351 (0.396) Data 0.001 (0.037) Loss 3.228 (2.780)
Epoch: [13][270/400] Time 0.353 (0.394) Data 0.001 (0.036) Loss 3.484 (2.799)
Epoch: [13][280/400] Time 0.351 (0.393) Data 0.001 (0.035) Loss 3.392 (2.818)
Epoch: [13][290/400] Time 0.364 (0.392) Data 0.000 (0.034) Loss 3.271 (2.830)
Epoch: [13][300/400] Time 0.358 (0.397) Data 0.001 (0.039) Loss 3.744 (2.848)
Epoch: [13][310/400] Time 0.354 (0.396) Data 0.001 (0.037) Loss 3.242 (2.856)
Epoch: [13][320/400] Time 0.351 (0.394) Data 0.001 (0.036) Loss 4.199 (2.868)
Epoch: [13][330/400] Time 0.353 (0.393) Data 0.000 (0.035) Loss 3.117 (2.881)
Epoch: [13][340/400] Time 0.364 (0.393) Data 0.000 (0.034) Loss 3.464 (2.896)
Epoch: [13][350/400] Time 0.376 (0.397) Data 0.000 (0.038) Loss 2.727 (2.909)
Epoch: [13][360/400] Time 0.354 (0.395) Data 0.000 (0.037) Loss 3.185 (2.920)
Epoch: [13][370/400] Time 0.354 (0.394) Data 0.000 (0.036) Loss 3.677 (2.932)
Epoch: [13][380/400] Time 0.354 (0.393) Data 0.001 (0.035) Loss 3.252 (2.945)
Epoch: [13][390/400] Time 0.375 (0.393) Data 0.000 (0.034) Loss 2.978 (2.949)
Epoch: [13][400/400] Time 0.354 (0.396) Data 0.001 (0.038) Loss 3.420 (2.955)
==> Create pseudo labels for unlabeled data
Extract Features: [50/51] Time 0.129 (0.159) Data 0.000 (0.027)
Computing jaccard distance...
Jaccard distance computing time cost: 19.923077821731567
==> Statistics for epoch 14: 790 clusters
Epoch: [14][10/400] Time 0.362 (0.416) Data 0.000 (0.048) Loss 0.610 (0.566)
Epoch: [14][20/400] Time 0.351 (0.386) Data 0.001 (0.024) Loss 0.554 (0.538)
Epoch: [14][30/400] Time 0.352 (0.376) Data 0.001 (0.016) Loss 0.321 (0.498)
Epoch: [14][40/400] Time 0.361 (0.372) Data 0.001 (0.012) Loss 0.381 (0.475)
Epoch: [14][50/400] Time 2.216 (0.406) Data 1.846 (0.047) Loss 2.810 (0.496)
Epoch: [14][60/400] Time 0.351 (0.399) Data 0.001 (0.039) Loss 3.211 (0.944)
Epoch: [14][70/400] Time 0.353 (0.393) Data 0.001 (0.034) Loss 3.647 (1.274)
Epoch: [14][80/400] Time 0.351 (0.389) Data 0.001 (0.030) Loss 4.010 (1.545)
Epoch: [14][90/400] Time 0.353 (0.385) Data 0.001 (0.026) Loss 3.302 (1.752)
Epoch: [14][100/400] Time 0.348 (0.401) Data 0.000 (0.041) Loss 3.138 (1.925)
Epoch: [14][110/400] Time 0.352 (0.397) Data 0.000 (0.038) Loss 3.270 (2.021)
Epoch: [14][120/400] Time 0.362 (0.394) Data 0.001 (0.035) Loss 3.810 (2.130)
Epoch: [14][130/400] Time 0.355 (0.391) Data 0.001 (0.032) Loss 3.018 (2.196)
Epoch: [14][140/400] Time 0.352 (0.389) Data 0.001 (0.030) Loss 3.428 (2.267)
Epoch: [14][150/400] Time 0.351 (0.399) Data 0.000 (0.040) Loss 2.540 (2.318)
Epoch: [14][160/400] Time 0.362 (0.396) Data 0.001 (0.037) Loss 3.774 (2.392)
Epoch: [14][170/400] Time 0.352 (0.394) Data 0.000 (0.035) Loss 3.386 (2.439)
Epoch: [14][180/400] Time 0.352 (0.392) Data 0.000 (0.033) Loss 3.828 (2.493)
Epoch: [14][190/400] Time 0.363 (0.390) Data 0.000 (0.031) Loss 3.470 (2.540)
Epoch: [14][200/400] Time 0.353 (0.398) Data 0.000 (0.039) Loss 3.056 (2.566)
Epoch: [14][210/400] Time 0.361 (0.396) Data 0.001 (0.038) Loss 3.308 (2.603)
Epoch: [14][220/400] Time 0.352 (0.395) Data 0.001 (0.036) Loss 3.197 (2.634)
Epoch: [14][230/400] Time 0.406 (0.393) Data 0.001 (0.034) Loss 3.360 (2.657)
Epoch: [14][240/400] Time 0.362 (0.392) Data 0.000 (0.033) Loss 3.102 (2.685)
Epoch: [14][250/400] Time 0.351 (0.398) Data 0.001 (0.039) Loss 3.292 (2.709)
Epoch: [14][260/400] Time 0.355 (0.396) Data 0.001 (0.037) Loss 3.541 (2.733)
Epoch: [14][270/400] Time 0.352 (0.395) Data 0.001 (0.036) Loss 3.391 (2.754)
Epoch: [14][280/400] Time 0.357 (0.393) Data 0.001 (0.035) Loss 3.009 (2.780)
Epoch: [14][290/400] Time 0.364 (0.392) Data 0.000 (0.033) Loss 3.992 (2.795)
Epoch: [14][300/400] Time 0.353 (0.398) Data 0.000 (0.039) Loss 3.267 (2.809)
Epoch: [14][310/400] Time 0.368 (0.396) Data 0.001 (0.037) Loss 3.523 (2.826)
Epoch: [14][320/400] Time 0.363 (0.395) Data 0.001 (0.036) Loss 2.903 (2.833)
Epoch: [14][330/400] Time 0.351 (0.394) Data 0.000 (0.035) Loss 2.882 (2.840)
Epoch: [14][340/400] Time 0.364 (0.393) Data 0.000 (0.034) Loss 3.980 (2.858)
Epoch: [14][350/400] Time 0.351 (0.397) Data 0.001 (0.038) Loss 4.101 (2.872)
Epoch: [14][360/400] Time 0.361 (0.396) Data 0.001 (0.037) Loss 3.365 (2.882)
Epoch: [14][370/400] Time 0.352 (0.395) Data 0.001 (0.036) Loss 3.678 (2.894)
Epoch: [14][380/400] Time 0.351 (0.394) Data 0.001 (0.035) Loss 3.371 (2.905)
Epoch: [14][390/400] Time 0.363 (0.393) Data 0.000 (0.034) Loss 3.487 (2.907)
Epoch: [14][400/400] Time 0.352 (0.397) Data 0.001 (0.038) Loss 2.707 (2.913)
==> Create pseudo labels for unlabeled data
Extract Features: [50/51] Time 0.129 (0.159) Data 0.000 (0.029)
Computing jaccard distance...
Jaccard distance computing time cost: 18.7038893699646
==> Statistics for epoch 15: 762 clusters
Epoch: [15][10/400] Time 0.361 (0.408) Data 0.001 (0.052) Loss 0.427 (0.511)
Epoch: [15][20/400] Time 0.344 (0.379) Data 0.001 (0.027) Loss 0.680 (0.525)
Epoch: [15][30/400] Time 0.350 (0.369) Data 0.001 (0.018) Loss 0.465 (0.490)
Epoch: [15][40/400] Time 0.379 (0.365) Data 0.001 (0.014) Loss 0.393 (0.465)
Epoch: [15][50/400] Time 0.353 (0.403) Data 0.001 (0.048) Loss 3.886 (0.625)
Epoch: [15][60/400] Time 0.363 (0.395) Data 0.001 (0.041) Loss 4.075 (1.036)
Epoch: [15][70/400] Time 0.354 (0.389) Data 0.001 (0.035) Loss 3.462 (1.347)
Epoch: [15][80/400] Time 0.351 (0.385) Data 0.001 (0.031) Loss 3.275 (1.604)
Epoch: [15][90/400] Time 0.361 (0.382) Data 0.000 (0.027) Loss 3.815 (1.821)
Epoch: [15][100/400] Time 0.352 (0.397) Data 0.001 (0.042) Loss 3.366 (1.957)
Epoch: [15][110/400] Time 0.352 (0.393) Data 0.001 (0.038) Loss 3.331 (2.063)
Epoch: [15][120/400] Time 0.358 (0.391) Data 0.001 (0.035) Loss 2.829 (2.147)
Epoch: [15][130/400] Time 0.351 (0.388) Data 0.001 (0.032) Loss 3.148 (2.216)
Epoch: [15][140/400] Time 0.362 (0.386) Data 0.000 (0.030) Loss 3.133 (2.283)
Epoch: [15][150/400] Time 0.353 (0.395) Data 0.000 (0.039) Loss 3.000 (2.326)
Epoch: [15][160/400] Time 0.351 (0.392) Data 0.000 (0.037) Loss 2.966 (2.371)
Epoch: [15][170/400] Time 0.352 (0.390) Data 0.001 (0.035) Loss 3.734 (2.429)
Epoch: [15][180/400] Time 0.352 (0.388) Data 0.000 (0.033) Loss 3.507 (2.491)
Epoch: [15][190/400] Time 0.351 (0.396) Data 0.000 (0.040) Loss 2.590 (2.533)
Epoch: [15][200/400] Time 0.352 (0.394) Data 0.001 (0.038) Loss 3.072 (2.566)
Epoch: [15][210/400] Time 0.351 (0.392) Data 0.000 (0.037) Loss 3.743 (2.606)
Epoch: [15][220/400] Time 0.351 (0.390) Data 0.001 (0.035) Loss 4.051 (2.639)
Epoch: [15][230/400] Time 0.363 (0.389) Data 0.000 (0.033) Loss 2.909 (2.665)
Epoch: [15][240/400] Time 0.353 (0.395) Data 0.001 (0.039) Loss 3.017 (2.698)
Epoch: [15][250/400] Time 0.353 (0.394) Data 0.001 (0.038) Loss 3.426 (2.713)
Epoch: [15][260/400] Time 0.374 (0.392) Data 0.001 (0.036) Loss 2.636 (2.730)
Epoch: [15][270/400] Time 0.355 (0.391) Data 0.001 (0.035) Loss 2.961 (2.754)
Epoch: [15][280/400] Time 0.364 (0.390) Data 0.000 (0.034) Loss 3.096 (2.773)
Epoch: [15][290/400] Time 0.355 (0.395) Data 0.001 (0.039) Loss 2.648 (2.790)
Epoch: [15][300/400] Time 0.354 (0.394) Data 0.001 (0.038) Loss 3.332 (2.796)
Epoch: [15][310/400] Time 0.363 (0.393) Data 0.001 (0.036) Loss 2.959 (2.814)
Epoch: [15][320/400] Time 0.352 (0.392) Data 0.001 (0.035) Loss 3.060 (2.828)
Epoch: [15][330/400] Time 2.109 (0.396) Data 1.729 (0.039) Loss 3.621 (2.842)
Epoch: [15][340/400] Time 0.353 (0.395) Data 0.001 (0.038) Loss 3.697 (2.851)
Epoch: [15][350/400] Time 0.354 (0.394) Data 0.001 (0.037) Loss 3.061 (2.856)
Epoch: [15][360/400] Time 0.353 (0.393) Data 0.000 (0.036) Loss 3.597 (2.868)
Epoch: [15][370/400] Time 0.365 (0.392) Data 0.000 (0.035) Loss 2.889 (2.879)
Epoch: [15][380/400] Time 0.353 (0.396) Data 0.000 (0.039) Loss 3.172 (2.887)
Epoch: [15][390/400] Time 0.405 (0.395) Data 0.001 (0.038) Loss 3.439 (2.896)
Epoch: [15][400/400] Time 0.353 (0.394) Data 0.000 (0.037) Loss 3.548 (2.901)
==> Create pseudo labels for unlabeled data
Extract Features: [50/51] Time 0.129 (0.162) Data 0.000 (0.028)
Computing jaccard distance...
Jaccard distance computing time cost: 19.068045377731323
==> Statistics for epoch 16: 752 clusters
Epoch: [16][10/400] Time 0.349 (0.414) Data 0.000 (0.053) Loss 0.500 (0.518)
Epoch: [16][20/400] Time 0.351 (0.382) Data 0.000 (0.027) Loss 0.438 (0.509)
Epoch: [16][30/400] Time 0.351 (0.374) Data 0.000 (0.018) Loss 0.239 (0.498)
Epoch: [16][40/400] Time 0.351 (0.370) Data 0.001 (0.014) Loss 0.273 (0.460)
Epoch: [16][50/400] Time 0.351 (0.406) Data 0.000 (0.049) Loss 3.233 (0.605)
Epoch: [16][60/400] Time 0.371 (0.399) Data 0.000 (0.041) Loss 2.734 (1.023)
Epoch: [16][70/400] Time 0.350 (0.392) Data 0.000 (0.035) Loss 3.455 (1.349)
Epoch: [16][80/400] Time 0.354 (0.388) Data 0.001 (0.031) Loss 3.746 (1.599)
Epoch: [16][90/400] Time 0.362 (0.384) Data 0.000 (0.027) Loss 3.622 (1.805)
Epoch: [16][100/400] Time 0.357 (0.400) Data 0.001 (0.042) Loss 2.684 (1.938)
Epoch: [16][110/400] Time 0.351 (0.396) Data 0.001 (0.039) Loss 3.758 (2.046)
Epoch: [16][120/400] Time 0.361 (0.393) Data 0.000 (0.035) Loss 2.084 (2.116)
Epoch: [16][130/400] Time 0.371 (0.390) Data 0.001 (0.033) Loss 3.511 (2.181)
Epoch: [16][140/400] Time 0.363 (0.388) Data 0.000 (0.030) Loss 2.800 (2.250)
Epoch: [16][150/400] Time 0.375 (0.399) Data 0.000 (0.041) Loss 2.657 (2.301)
Epoch: [16][160/400] Time 0.352 (0.396) Data 0.001 (0.038) Loss 3.241 (2.360)
Epoch: [16][170/400] Time 0.350 (0.394) Data 0.001 (0.036) Loss 3.451 (2.422)
Epoch: [16][180/400] Time 0.352 (0.391) Data 0.001 (0.034) Loss 3.335 (2.469)
Epoch: [16][190/400] Time 0.375 (0.399) Data 0.001 (0.042) Loss 3.002 (2.510)
Epoch: [16][200/400] Time 0.355 (0.397) Data 0.000 (0.039) Loss 2.965 (2.534)
Epoch: [16][210/400] Time 0.365 (0.395) Data 0.001 (0.038) Loss 3.615 (2.571)
Epoch: [16][220/400] Time 0.361 (0.394) Data 0.001 (0.036) Loss 3.552 (2.598)
Epoch: [16][230/400] Time 0.362 (0.392) Data 0.000 (0.034) Loss 3.023 (2.618)
Epoch: [16][240/400] Time 0.364 (0.398) Data 0.000 (0.040) Loss 3.833 (2.643)
Epoch: [16][250/400] Time 0.350 (0.396) Data 0.001 (0.038) Loss 2.741 (2.667)
Epoch: [16][260/400] Time 0.352 (0.395) Data 0.000 (0.037) Loss 2.653 (2.688)
Epoch: [16][270/400] Time 0.354 (0.393) Data 0.000 (0.036) Loss 3.082 (2.706)
Epoch: [16][280/400] Time 0.363 (0.392) Data 0.000 (0.034) Loss 3.040 (2.733)
Epoch: [16][290/400] Time 0.364 (0.397) Data 0.000 (0.039) Loss 3.054 (2.752)
Epoch: [16][300/400] Time 0.356 (0.396) Data 0.001 (0.038) Loss 3.046 (2.761)
Epoch: [16][310/400] Time 0.366 (0.395) Data 0.000 (0.037) Loss 3.188 (2.774)
Epoch: [16][320/400] Time 0.364 (0.394) Data 0.001 (0.036) Loss 3.074 (2.783)
Epoch: [16][330/400] Time 2.142 (0.398) Data 1.691 (0.040) Loss 3.194 (2.789)
Epoch: [16][340/400] Time 0.353 (0.397) Data 0.001 (0.038) Loss 1.998 (2.798)
Epoch: [16][350/400] Time 0.354 (0.396) Data 0.000 (0.037) Loss 2.835 (2.806)
Epoch: [16][360/400] Time 0.354 (0.394) Data 0.000 (0.036) Loss 3.600 (2.824)
Epoch: [16][370/400] Time 0.363 (0.393) Data 0.000 (0.035) Loss 3.361 (2.837)
Epoch: [16][380/400] Time 0.350 (0.397) Data 0.000 (0.039) Loss 3.627 (2.846)
Epoch: [16][390/400] Time 0.351 (0.396) Data 0.001 (0.038) Loss 3.628 (2.857)
Epoch: [16][400/400] Time 0.354 (0.395) Data 0.001 (0.037) Loss 2.607 (2.863)
==> Create pseudo labels for unlabeled data
Extract Features: [50/51] Time 0.129 (0.158) Data 0.000 (0.027)
Computing jaccard distance...
Jaccard distance computing time cost: 18.778992891311646
==> Statistics for epoch 17: 744 clusters
Epoch: [17][10/400] Time 0.350 (0.404) Data 0.001 (0.046) Loss 0.413 (0.525)
Epoch: [17][20/400] Time 0.351 (0.382) Data 0.001 (0.023) Loss 0.344 (0.506)
Epoch: [17][30/400] Time 0.362 (0.374) Data 0.001 (0.016) Loss 0.320 (0.475)
Epoch: [17][40/400] Time 0.363 (0.369) Data 0.000 (0.012) Loss 0.243 (0.429)
Epoch: [17][50/400] Time 0.352 (0.405) Data 0.000 (0.046) Loss 2.625 (0.619)
Epoch: [17][60/400] Time 0.363 (0.397) Data 0.001 (0.039) Loss 3.538 (1.028)
Epoch: [17][70/400] Time 0.356 (0.391) Data 0.000 (0.033) Loss 3.579 (1.337)
Epoch: [17][80/400] Time 0.364 (0.387) Data 0.001 (0.029) Loss 2.884 (1.575)
Epoch: [17][90/400] Time 0.363 (0.384) Data 0.000 (0.026) Loss 3.305 (1.786)
Epoch: [17][100/400] Time 0.352 (0.399) Data 0.000 (0.041) Loss 2.545 (1.909)
Epoch: [17][110/400] Time 0.350 (0.395) Data 0.001 (0.038) Loss 3.551 (2.010)
Epoch: [17][120/400] Time 0.379 (0.392) Data 0.001 (0.035) Loss 2.456 (2.070)
Epoch: [17][130/400] Time 0.358 (0.389) Data 0.001 (0.032) Loss 2.775 (2.147)
Epoch: [17][140/400] Time 0.351 (0.400) Data 0.000 (0.042) Loss 2.695 (2.210)
Epoch: [17][150/400] Time 0.351 (0.397) Data 0.000 (0.039) Loss 2.791 (2.272)
Epoch: [17][160/400] Time 0.354 (0.395) Data 0.001 (0.037) Loss 3.777 (2.328)
Epoch: [17][170/400] Time 0.353 (0.392) Data 0.001 (0.035) Loss 3.818 (2.384)
Epoch: [17][180/400] Time 0.363 (0.390) Data 0.000 (0.033) Loss 3.937 (2.442)
Epoch: [17][190/400] Time 0.354 (0.398) Data 0.001 (0.040) Loss 3.735 (2.481)
Epoch: [17][200/400] Time 0.350 (0.396) Data 0.001 (0.038) Loss 3.109 (2.507)
Epoch: [17][210/400] Time 0.362 (0.394) Data 0.001 (0.036) Loss 2.810 (2.538)
Epoch: [17][220/400] Time 0.352 (0.393) Data 0.001 (0.035) Loss 2.615 (2.563)
Epoch: [17][230/400] Time 0.362 (0.391) Data 0.000 (0.033) Loss 3.619 (2.587)
Epoch: [17][240/400] Time 0.362 (0.397) Data 0.001 (0.039) Loss 2.995 (2.613)
Epoch: [17][250/400] Time 0.352 (0.395) Data 0.001 (0.038) Loss 3.345 (2.634)
Epoch: [17][260/400] Time 0.354 (0.394) Data 0.001 (0.036) Loss 2.918 (2.650)
Epoch: [17][270/400] Time 0.364 (0.392) Data 0.000 (0.035) Loss 2.958 (2.669)
Epoch: [17][280/400] Time 0.405 (0.398) Data 0.000 (0.040) Loss 2.801 (2.687)
Epoch: [17][290/400] Time 0.349 (0.396) Data 0.001 (0.038) Loss 3.234 (2.700)
Epoch: [17][300/400] Time 0.362 (0.395) Data 0.001 (0.037) Loss 2.987 (2.710)
Epoch: [17][310/400] Time 0.353 (0.394) Data 0.001 (0.036) Loss 3.327 (2.726)
Epoch: [17][320/400] Time 0.363 (0.393) Data 0.000 (0.035) Loss 3.132 (2.733)
Epoch: [17][330/400] Time 0.352 (0.397) Data 0.001 (0.039) Loss 2.924 (2.742)
Epoch: [17][340/400] Time 0.364 (0.396) Data 0.001 (0.038) Loss 3.477 (2.755)
Epoch: [17][350/400] Time 0.353 (0.395) Data 0.000 (0.037) Loss 4.079 (2.770)
Epoch: [17][360/400] Time 0.364 (0.394) Data 0.000 (0.036) Loss 3.004 (2.781)
Epoch: [17][370/400] Time 0.351 (0.397) Data 0.001 (0.039) Loss 2.665 (2.791)
Epoch: [17][380/400] Time 0.353 (0.396) Data 0.001 (0.038) Loss 2.900 (2.793)
Epoch: [17][390/400] Time 0.351 (0.395) Data 0.001 (0.037) Loss 2.701 (2.798)
Epoch: [17][400/400] Time 0.352 (0.394) Data 0.001 (0.037) Loss 2.122 (2.806)
==> Create pseudo labels for unlabeled data
Extract Features: [50/51] Time 0.124 (0.159) Data 0.000 (0.029)
Computing jaccard distance...
Jaccard distance computing time cost: 18.61119508743286
==> Statistics for epoch 18: 725 clusters
Epoch: [18][10/400] Time 0.360 (0.419) Data 0.001 (0.058) Loss 0.515 (0.526)
Epoch: [18][20/400] Time 0.366 (0.389) Data 0.001 (0.029) Loss 0.368 (0.455)
Epoch: [18][30/400] Time 0.369 (0.379) Data 0.000 (0.020) Loss 0.421 (0.432)
Epoch: [18][40/400] Time 0.362 (0.374) Data 0.000 (0.015) Loss 0.357 (0.407)
Epoch: [18][50/400] Time 0.353 (0.405) Data 0.000 (0.046) Loss 2.700 (0.637)
Epoch: [18][60/400] Time 0.371 (0.398) Data 0.001 (0.038) Loss 2.934 (1.057)
Epoch: [18][70/400] Time 0.350 (0.392) Data 0.001 (0.033) Loss 3.014 (1.374)
Epoch: [18][80/400] Time 0.364 (0.388) Data 0.001 (0.029) Loss 2.810 (1.612)
Epoch: [18][90/400] Time 0.362 (0.385) Data 0.000 (0.026) Loss 3.201 (1.804)
Epoch: [18][100/400] Time 0.354 (0.401) Data 0.000 (0.041) Loss 2.767 (1.907)
Epoch: [18][110/400] Time 0.351 (0.397) Data 0.001 (0.037) Loss 3.115 (1.999)
Epoch: [18][120/400] Time 0.353 (0.393) Data 0.000 (0.034) Loss 2.943 (2.105)
Epoch: [18][130/400] Time 0.364 (0.390) Data 0.000 (0.032) Loss 2.687 (2.179)
Epoch: [18][140/400] Time 0.353 (0.401) Data 0.000 (0.042) Loss 2.881 (2.230)
Epoch: [18][150/400] Time 0.362 (0.398) Data 0.001 (0.039) Loss 3.405 (2.303)
Epoch: [18][160/400] Time 0.352 (0.395) Data 0.001 (0.037) Loss 3.115 (2.353)
Epoch: [18][170/400] Time 0.369 (0.393) Data 0.001 (0.035) Loss 3.282 (2.411)
Epoch: [18][180/400] Time 0.362 (0.391) Data 0.000 (0.033) Loss 3.578 (2.455)
Epoch: [18][190/400] Time 0.352 (0.400) Data 0.001 (0.041) Loss 2.819 (2.486)
Epoch: [18][200/400] Time 0.351 (0.397) Data 0.000 (0.039) Loss 3.273 (2.520)
Epoch: [18][210/400] Time 0.352 (0.395) Data 0.001 (0.037) Loss 3.079 (2.542)
Epoch: [18][220/400] Time 0.362 (0.394) Data 0.000 (0.036) Loss 3.234 (2.572)
Epoch: [18][230/400] Time 0.354 (0.400) Data 0.000 (0.042) Loss 2.924 (2.595)
Epoch: [18][240/400] Time 0.349 (0.398) Data 0.000 (0.040) Loss 3.282 (2.626)
Epoch: [18][250/400] Time 0.353 (0.397) Data 0.001 (0.038) Loss 3.278 (2.646)
Epoch: [18][260/400] Time 0.352 (0.395) Data 0.000 (0.037) Loss 3.142 (2.666)
Epoch: [18][270/400] Time 0.362 (0.394) Data 0.000 (0.035) Loss 3.349 (2.683)
Epoch: [18][280/400] Time 0.363 (0.399) Data 0.001 (0.041) Loss 2.549 (2.701)
Epoch: [18][290/400] Time 0.353 (0.398) Data 0.000 (0.039) Loss 2.947 (2.714)
Epoch: [18][300/400] Time 0.356 (0.397) Data 0.001 (0.038) Loss 3.413 (2.728)
Epoch: [18][310/400] Time 0.361 (0.396) Data 0.000 (0.037) Loss 3.380 (2.737)
Epoch: [18][320/400] Time 0.352 (0.400) Data 0.001 (0.041) Loss 3.836 (2.751)
Epoch: [18][330/400] Time 0.353 (0.399) Data 0.001 (0.040) Loss 2.782 (2.762)
Epoch: [18][340/400] Time 0.352 (0.397) Data 0.001 (0.039) Loss 3.140 (2.770)
Epoch: [18][350/400] Time 0.353 (0.396) Data 0.001 (0.038) Loss 3.102 (2.777)
Epoch: [18][360/400] Time 0.364 (0.395) Data 0.000 (0.037) Loss 3.474 (2.785)
Epoch: [18][370/400] Time 0.364 (0.400) Data 0.001 (0.041) Loss 3.068 (2.792)
Epoch: [18][380/400] Time 0.398 (0.399) Data 0.000 (0.040) Loss 2.818 (2.802)
Epoch: [18][390/400] Time 0.364 (0.397) Data 0.001 (0.039) Loss 2.980 (2.805)
Epoch: [18][400/400] Time 0.363 (0.396) Data 0.000 (0.038) Loss 2.303 (2.812)
==> Create pseudo labels for unlabeled data
Extract Features: [50/51] Time 0.129 (0.164) Data 0.000 (0.030)
Computing jaccard distance...
Jaccard distance computing time cost: 18.65268301963806
==> Statistics for epoch 19: 718 clusters
Epoch: [19][10/400] Time 0.354 (0.409) Data 0.001 (0.048) Loss 0.564 (0.489)
Epoch: [19][20/400] Time 0.351 (0.383) Data 0.001 (0.024) Loss 0.433 (0.468)
Epoch: [19][30/400] Time 0.350 (0.372) Data 0.001 (0.016) Loss 0.333 (0.432)
Epoch: [19][40/400] Time 0.362 (0.368) Data 0.000 (0.012) Loss 0.277 (0.401)
Epoch: [19][50/400] Time 0.362 (0.401) Data 0.001 (0.044) Loss 3.021 (0.724)
Epoch: [19][60/400] Time 0.353 (0.393) Data 0.000 (0.036) Loss 3.214 (1.109)
Epoch: [19][70/400] Time 0.352 (0.388) Data 0.001 (0.031) Loss 3.490 (1.362)
Epoch: [19][80/400] Time 0.352 (0.383) Data 0.000 (0.027) Loss 2.828 (1.590)
Epoch: [19][90/400] Time 0.349 (0.400) Data 0.001 (0.044) Loss 3.026 (1.755)
Epoch: [19][100/400] Time 0.363 (0.396) Data 0.001 (0.040) Loss 2.432 (1.860)
Epoch: [19][110/400] Time 0.353 (0.393) Data 0.000 (0.036) Loss 2.974 (1.940)
Epoch: [19][120/400] Time 0.361 (0.390) Data 0.001 (0.033) Loss 2.866 (2.025)
Epoch: [19][130/400] Time 0.363 (0.387) Data 0.000 (0.031) Loss 2.962 (2.082)
Epoch: [19][140/400] Time 0.351 (0.399) Data 0.001 (0.041) Loss 2.721 (2.155)
Epoch: [19][150/400] Time 0.350 (0.396) Data 0.001 (0.039) Loss 3.033 (2.206)
Epoch: [19][160/400] Time 0.351 (0.393) Data 0.001 (0.036) Loss 3.136 (2.266)
Epoch: [19][170/400] Time 0.364 (0.391) Data 0.000 (0.034) Loss 2.575 (2.304)
Epoch: [19][180/400] Time 0.351 (0.400) Data 0.001 (0.042) Loss 3.043 (2.341)
Epoch: [19][190/400] Time 0.353 (0.398) Data 0.001 (0.040) Loss 2.589 (2.358)
Epoch: [19][200/400] Time 0.354 (0.395) Data 0.001 (0.038) Loss 2.823 (2.389)
Epoch: [19][210/400] Time 0.354 (0.393) Data 0.001 (0.036) Loss 2.407 (2.422)
Epoch: [19][220/400] Time 0.355 (0.392) Data 0.000 (0.035) Loss 3.007 (2.451)
Epoch: [19][230/400] Time 0.355 (0.398) Data 0.001 (0.041) Loss 3.365 (2.478)
Epoch: [19][240/400] Time 0.354 (0.397) Data 0.000 (0.039) Loss 2.966 (2.501)
Epoch: [19][250/400] Time 0.354 (0.395) Data 0.001 (0.038) Loss 3.292 (2.523)
Epoch: [19][260/400] Time 0.374 (0.394) Data 0.000 (0.036) Loss 2.445 (2.539)
Epoch: [19][270/400] Time 0.363 (0.399) Data 0.001 (0.042) Loss 3.306 (2.552)
Epoch: [19][280/400] Time 0.354 (0.397) Data 0.001 (0.040) Loss 3.214 (2.571)
Epoch: [19][290/400] Time 0.375 (0.396) Data 0.001 (0.039) Loss 3.514 (2.592)
Epoch: [19][300/400] Time 0.351 (0.395) Data 0.001 (0.037) Loss 2.527 (2.602)
Epoch: [19][310/400] Time 0.351 (0.400) Data 0.001 (0.042) Loss 3.175 (2.617)
Epoch: [19][320/400] Time 0.353 (0.398) Data 0.001 (0.041) Loss 3.120 (2.630)
Epoch: [19][330/400] Time 0.362 (0.397) Data 0.001 (0.039) Loss 2.933 (2.646)
Epoch: [19][340/400] Time 0.354 (0.396) Data 0.000 (0.038) Loss 2.819 (2.662)
Epoch: [19][350/400] Time 0.365 (0.395) Data 0.000 (0.037) Loss 3.588 (2.676)
Epoch: [19][360/400] Time 0.353 (0.399) Data 0.000 (0.041) Loss 2.967 (2.684)
Epoch: [19][370/400] Time 0.352 (0.398) Data 0.001 (0.040) Loss 2.823 (2.688)
Epoch: [19][380/400] Time 0.375 (0.397) Data 0.000 (0.039) Loss 3.236 (2.693)
Epoch: [19][390/400] Time 0.364 (0.396) Data 0.000 (0.038) Loss 2.652 (2.697)
Epoch: [19][400/400] Time 0.354 (0.400) Data 0.001 (0.042) Loss 2.646 (2.707)
Extract Features: [50/76] Time 0.133 (0.163) Data 0.000 (0.029)
Mean AP: 76.9%
* Finished epoch 19 model mAP: 76.9% best: 76.9% *
==> Create pseudo labels for unlabeled data
Extract Features: [50/51] Time 0.136 (0.161) Data 0.000 (0.029)
Computing jaccard distance...
Jaccard distance computing time cost: 19.028404235839844
==> Statistics for epoch 20: 710 clusters
Epoch: [20][10/400] Time 0.361 (0.421) Data 0.001 (0.052) Loss 0.460 (0.458)
Epoch: [20][20/400] Time 0.349 (0.387) Data 0.001 (0.026) Loss 0.355 (0.444)
Epoch: [20][30/400] Time 0.362 (0.378) Data 0.001 (0.018) Loss 0.334 (0.431)
Epoch: [20][40/400] Time 0.363 (0.373) Data 0.000 (0.014) Loss 0.351 (0.403)
Epoch: [20][50/400] Time 0.349 (0.407) Data 0.001 (0.047) Loss 2.913 (0.716)
Epoch: [20][60/400] Time 0.351 (0.399) Data 0.001 (0.040) Loss 3.591 (1.113)
Epoch: [20][70/400] Time 0.351 (0.392) Data 0.001 (0.034) Loss 2.805 (1.384)
Epoch: [20][80/400] Time 0.350 (0.387) Data 0.001 (0.030) Loss 2.635 (1.594)
Epoch: [20][90/400] Time 0.351 (0.405) Data 0.001 (0.047) Loss 1.877 (1.735)
Epoch: [20][100/400] Time 0.354 (0.401) Data 0.001 (0.042) Loss 2.308 (1.833)
Epoch: [20][110/400] Time 0.381 (0.397) Data 0.001 (0.038) Loss 2.102 (1.879)
Epoch: [20][120/400] Time 0.352 (0.393) Data 0.001 (0.035) Loss 2.467 (1.931)
Epoch: [20][130/400] Time 0.363 (0.390) Data 0.000 (0.033) Loss 2.248 (1.984)
Epoch: [20][140/400] Time 0.353 (0.400) Data 0.000 (0.043) Loss 2.529 (2.044)
Epoch: [20][150/400] Time 0.364 (0.398) Data 0.001 (0.040) Loss 2.474 (2.089)
Epoch: [20][160/400] Time 0.351 (0.395) Data 0.001 (0.037) Loss 2.442 (2.117)
Epoch: [20][170/400] Time 0.363 (0.393) Data 0.000 (0.035) Loss 2.734 (2.156)
Epoch: [20][180/400] Time 0.408 (0.402) Data 0.000 (0.044) Loss 2.851 (2.183)
Epoch: [20][190/400] Time 0.351 (0.400) Data 0.001 (0.041) Loss 2.188 (2.217)
Epoch: [20][200/400] Time 0.353 (0.397) Data 0.001 (0.039) Loss 1.985 (2.231)
Epoch:
gitextract_7h4tsjq6/ ├── LICENSE ├── README.md ├── clustercontrast/ │ ├── __init__.py │ ├── datasets/ │ │ ├── __init__.py │ │ ├── dukemtmcreid.py │ │ ├── market1501.py │ │ ├── msmt17.py │ │ ├── personx.py │ │ └── veri.py │ ├── evaluation_metrics/ │ │ ├── __init__.py │ │ ├── classification.py │ │ └── ranking.py │ ├── evaluators.py │ ├── models/ │ │ ├── __init__.py │ │ ├── cm.py │ │ ├── dsbn.py │ │ ├── kmeans.py │ │ ├── pooling.py │ │ ├── resnet.py │ │ ├── resnet_ibn.py │ │ └── resnet_ibn_a.py │ ├── trainers.py │ └── utils/ │ ├── __init__.py │ ├── data/ │ │ ├── __init__.py │ │ ├── base_dataset.py │ │ ├── preprocessor.py │ │ ├── sampler.py │ │ └── transforms.py │ ├── faiss_rerank.py │ ├── faiss_utils.py │ ├── infomap_cluster.py │ ├── infomap_utils.py │ ├── logging.py │ ├── meters.py │ ├── osutils.py │ ├── rerank.py │ └── serialization.py ├── examples/ │ ├── cluster_contrast_train_usl.py │ ├── cluster_contrast_train_usl_infomap.py │ ├── logs/ │ │ └── log.txt │ ├── pretrained/ │ │ └── resnet50-19c8e357.pth │ └── test.py ├── run_code.sh └── setup.py
SYMBOL INDEX (220 symbols across 36 files)
FILE: clustercontrast/datasets/__init__.py
function names (line 20) | def names():
function create (line 24) | def create(name, root, *args, **kwargs):
function get_dataset (line 47) | def get_dataset(name, root, *args, **kwargs):
FILE: clustercontrast/datasets/dukemtmcreid.py
function process_dir (line 7) | def process_dir(dir_path, relabel=False):
class DukeMTMCreID (line 37) | class DukeMTMCreID(BaseImageDataset):
method __init__ (line 55) | def __init__(self, root, verbose=True):
method _check_before_run (line 76) | def _check_before_run(self):
FILE: clustercontrast/datasets/market1501.py
class Market1501 (line 8) | class Market1501(BaseImageDataset):
method __init__ (line 21) | def __init__(self, root, verbose=True, **kwargs):
method _check_before_run (line 46) | def _check_before_run(self):
method _process_dir (line 57) | def _process_dir(self, dir_path, relabel=False):
FILE: clustercontrast/datasets/msmt17.py
function _process_dir (line 9) | def _process_dir(dir_path, relabel=False):
class MSMT17 (line 34) | class MSMT17(BaseImageDataset):
method __init__ (line 37) | def __init__(self, root, verbose=True, **kwargs):
method _check_before_run (line 62) | def _check_before_run(self):
FILE: clustercontrast/datasets/personx.py
class PersonX (line 9) | class PersonX(BaseImageDataset):
method __init__ (line 21) | def __init__(self, root, verbose=True, **kwargs):
method _check_before_run (line 46) | def _check_before_run(self):
method _process_dir (line 57) | def _process_dir(self, dir_path, relabel=False):
FILE: clustercontrast/datasets/veri.py
class VeRi (line 12) | class VeRi(BaseImageDataset):
method __init__ (line 24) | def __init__(self, root, verbose=True, **kwargs):
method check_before_run (line 49) | def check_before_run(self):
method process_dir (line 60) | def process_dir(self, dir_path, relabel=False):
FILE: clustercontrast/evaluation_metrics/classification.py
function accuracy (line 7) | def accuracy(output, target, topk=(1,)):
FILE: clustercontrast/evaluation_metrics/ranking.py
function _unique_sample (line 10) | def _unique_sample(ids_dict, num):
function cmc (line 18) | def cmc(distmat, query_ids=None, gallery_ids=None,
function mean_ap (line 82) | def mean_ap(distmat, query_ids=None, gallery_ids=None,
FILE: clustercontrast/evaluators.py
function extract_cnn_feature (line 16) | def extract_cnn_feature(model, inputs):
function extract_features (line 23) | def extract_features(model, data_loader, print_freq=50):
function pairwise_distance (line 55) | def pairwise_distance(features, query=None, gallery=None):
function evaluate_all (line 75) | def evaluate_all(query_features, gallery_features, distmat, query=None, ...
class Evaluator (line 109) | class Evaluator(object):
method __init__ (line 110) | def __init__(self, model):
method evaluate (line 114) | def evaluate(self, data_loader, query, gallery, cmc_flag=False, rerank...
FILE: clustercontrast/models/__init__.py
function names (line 16) | def names():
function create (line 20) | def create(name, *args, **kwargs):
FILE: clustercontrast/models/cm.py
class CM (line 9) | class CM(autograd.Function):
method forward (line 12) | def forward(ctx, inputs, targets, features, momentum):
method backward (line 21) | def backward(ctx, grad_outputs):
function cm (line 35) | def cm(inputs, indexes, features, momentum=0.5):
class CM_Hard (line 39) | class CM_Hard(autograd.Function):
method forward (line 42) | def forward(ctx, inputs, targets, features, momentum):
method backward (line 51) | def backward(ctx, grad_outputs):
function cm_hard (line 74) | def cm_hard(inputs, indexes, features, momentum=0.5):
class ClusterMemory (line 78) | class ClusterMemory(nn.Module, ABC):
method __init__ (line 79) | def __init__(self, num_features, num_samples, temp=0.05, momentum=0.2,...
method forward (line 90) | def forward(self, inputs, targets):
FILE: clustercontrast/models/dsbn.py
class DSBN2d (line 6) | class DSBN2d(nn.Module):
method __init__ (line 7) | def __init__(self, planes):
method forward (line 13) | def forward(self, x):
class DSBN1d (line 25) | class DSBN1d(nn.Module):
method __init__ (line 26) | def __init__(self, planes):
method forward (line 32) | def forward(self, x):
function convert_dsbn (line 44) | def convert_dsbn(model):
function convert_bn (line 60) | def convert_bn(model, use_target=True):
FILE: clustercontrast/models/kmeans.py
function label_generator_kmeans (line 14) | def label_generator_kmeans(features, num_classes=500, cuda=True):
FILE: clustercontrast/models/pooling.py
class GeneralizedMeanPoolingList (line 19) | class GeneralizedMeanPoolingList(nn.Module, ABC):
method __init__ (line 34) | def __init__(self, output_size=1, eps=1e-6):
method forward (line 39) | def forward(self, x_list):
method __repr__ (line 47) | def __repr__(self):
class GeneralizedMeanPooling (line 57) | class GeneralizedMeanPooling(nn.Module, ABC):
method __init__ (line 72) | def __init__(self, norm, output_size=1, eps=1e-6):
method forward (line 79) | def forward(self, x):
method __repr__ (line 85) | def __repr__(self):
class GeneralizedMeanPoolingP (line 97) | class GeneralizedMeanPoolingP(GeneralizedMeanPooling, ABC):
method __init__ (line 101) | def __init__(self, norm=3, output_size=1, eps=1e-6):
class GeneralizedMeanPoolingFpn (line 106) | class GeneralizedMeanPoolingFpn(nn.Module, ABC):
method __init__ (line 121) | def __init__(self, norm, output_size=1, eps=1e-6):
method forward (line 128) | def forward(self, x_lists):
method __repr__ (line 138) | def __repr__(self):
class GeneralizedMeanPoolingPFpn (line 150) | class GeneralizedMeanPoolingPFpn(GeneralizedMeanPoolingFpn, ABC):
method __init__ (line 154) | def __init__(self, norm=3, output_size=1, eps=1e-6):
class AdaptiveAvgMaxPool2d (line 159) | class AdaptiveAvgMaxPool2d(nn.Module, ABC):
method __init__ (line 160) | def __init__(self):
method forward (line 164) | def forward(self, x):
class FastGlobalAvgPool2d (line 171) | class FastGlobalAvgPool2d(nn.Module, ABC):
method __init__ (line 172) | def __init__(self, flatten=False):
method forward (line 176) | def forward(self, x):
function avg_pooling (line 188) | def avg_pooling():
function max_pooling (line 193) | def max_pooling():
class Flatten (line 197) | class Flatten(nn.Module):
method forward (line 198) | def forward(self, input):
function pooling_names (line 212) | def pooling_names():
function build_pooling_layer (line 216) | def build_pooling_layer(name):
FILE: clustercontrast/models/resnet.py
class ResNet (line 14) | class ResNet(nn.Module):
method __init__ (line 23) | def __init__(self, depth, pretrained=True, cut_at_pooling=False,
method forward (line 72) | def forward(self, x):
method reset_params (line 106) | def reset_params(self):
function resnet18 (line 124) | def resnet18(**kwargs):
function resnet34 (line 128) | def resnet34(**kwargs):
function resnet50 (line 132) | def resnet50(**kwargs):
function resnet101 (line 136) | def resnet101(**kwargs):
function resnet152 (line 140) | def resnet152(**kwargs):
FILE: clustercontrast/models/resnet_ibn.py
class ResNetIBN (line 16) | class ResNetIBN(nn.Module):
method __init__ (line 22) | def __init__(self, depth, pretrained=True, cut_at_pooling=False,
method forward (line 73) | def forward(self, x):
method reset_params (line 106) | def reset_params(self):
function resnet_ibn50a (line 124) | def resnet_ibn50a(**kwargs):
function resnet_ibn101a (line 128) | def resnet_ibn101a(**kwargs):
FILE: clustercontrast/models/resnet_ibn_a.py
function conv3x3 (line 16) | def conv3x3(in_planes, out_planes, stride=1):
class BasicBlock (line 22) | class BasicBlock(nn.Module):
method __init__ (line 25) | def __init__(self, inplanes, planes, stride=1, downsample=None):
method forward (line 35) | def forward(self, x):
class IBN (line 54) | class IBN(nn.Module):
method __init__ (line 55) | def __init__(self, planes):
method forward (line 63) | def forward(self, x):
class Bottleneck (line 70) | class Bottleneck(nn.Module):
method __init__ (line 73) | def __init__(self, inplanes, planes, ibn=False, stride=1, downsample=N...
method forward (line 89) | def forward(self, x):
class ResNet (line 112) | class ResNet(nn.Module):
method __init__ (line 114) | def __init__(self, block, layers, num_classes=1000):
method _make_layer (line 141) | def _make_layer(self, block, planes, blocks, stride=1):
method forward (line 161) | def forward(self, x):
function resnet50_ibn_a (line 179) | def resnet50_ibn_a(pretrained=False, **kwargs):
function resnet101_ibn_a (line 192) | def resnet101_ibn_a(pretrained=False, **kwargs):
function remove_module_key (line 205) | def remove_module_key(state_dict):
FILE: clustercontrast/trainers.py
class ClusterContrastTrainer (line 6) | class ClusterContrastTrainer(object):
method __init__ (line 7) | def __init__(self, encoder, memory=None):
method train (line 12) | def train(self, epoch, data_loader, optimizer, print_freq=10, train_it...
method _parse_data (line 56) | def _parse_data(self, inputs):
method _forward (line 60) | def _forward(self, inputs):
FILE: clustercontrast/utils/__init__.py
function to_numpy (line 6) | def to_numpy(tensor):
function to_torch (line 15) | def to_torch(ndarray):
FILE: clustercontrast/utils/data/__init__.py
class IterLoader (line 7) | class IterLoader:
method __init__ (line 8) | def __init__(self, loader, length=None):
method __len__ (line 13) | def __len__(self):
method new_epoch (line 19) | def new_epoch(self):
method next (line 22) | def next(self):
FILE: clustercontrast/utils/data/base_dataset.py
class BaseDataset (line 5) | class BaseDataset(object):
method get_imagedata_info (line 10) | def get_imagedata_info(self, data):
method print_dataset_statistics (line 22) | def print_dataset_statistics(self):
method images_dir (line 26) | def images_dir(self):
class BaseImageDataset (line 30) | class BaseImageDataset(BaseDataset):
method print_dataset_statistics (line 35) | def print_dataset_statistics(self, train, query, gallery):
FILE: clustercontrast/utils/data/preprocessor.py
class Preprocessor (line 11) | class Preprocessor(Dataset):
method __init__ (line 12) | def __init__(self, dataset, root=None, transform=None):
method __len__ (line 18) | def __len__(self):
method __getitem__ (line 21) | def __getitem__(self, indices):
method _get_single_item (line 24) | def _get_single_item(self, index):
FILE: clustercontrast/utils/data/sampler.py
function No_index (line 14) | def No_index(a, b):
class RandomIdentitySampler (line 19) | class RandomIdentitySampler(Sampler):
method __init__ (line 20) | def __init__(self, data_source, num_instances):
method __len__ (line 29) | def __len__(self):
method __iter__ (line 32) | def __iter__(self):
class RandomMultipleGallerySampler (line 46) | class RandomMultipleGallerySampler(Sampler):
method __init__ (line 47) | def __init__(self, data_source, num_instances=4):
method __len__ (line 65) | def __len__(self):
method __iter__ (line 68) | def __iter__(self):
class RandomMultipleGallerySamplerNoCam (line 109) | class RandomMultipleGallerySamplerNoCam(Sampler):
method __init__ (line 110) | def __init__(self, data_source, num_instances=4):
method __len__ (line 127) | def __len__(self):
method __iter__ (line 130) | def __iter__(self):
FILE: clustercontrast/utils/data/transforms.py
class RectScale (line 9) | class RectScale(object):
method __init__ (line 10) | def __init__(self, height, width, interpolation=Image.BILINEAR):
method __call__ (line 15) | def __call__(self, img):
class RandomSizedRectCrop (line 22) | class RandomSizedRectCrop(object):
method __init__ (line 23) | def __init__(self, height, width, interpolation=Image.BILINEAR):
method __call__ (line 28) | def __call__(self, img):
class RandomErasing (line 52) | class RandomErasing(object):
method __init__ (line 64) | def __init__(self, probability=0.5, sl=0.02, sh=0.4, r1=0.3, mean=(0.4...
method __call__ (line 71) | def __call__(self, img):
FILE: clustercontrast/utils/faiss_rerank.py
function k_reciprocal_neigh (line 23) | def k_reciprocal_neigh(initial_rank, i, k1):
function compute_jaccard_distance (line 30) | def compute_jaccard_distance(target_features, k1=20, k2=6, print_flag=Tr...
FILE: clustercontrast/utils/faiss_utils.py
function swig_ptr_from_FloatTensor (line 6) | def swig_ptr_from_FloatTensor(x):
function swig_ptr_from_LongTensor (line 12) | def swig_ptr_from_LongTensor(x):
function search_index_pytorch (line 19) | def search_index_pytorch(index, x, k, D=None, I=None):
function search_raw_array_pytorch (line 44) | def search_raw_array_pytorch(res, xb, xq, k, D=None, I=None,
function index_init_gpu (line 92) | def index_init_gpu(ngpus, feat_dim):
function index_init_cpu (line 108) | def index_init_cpu(feat_dim):
FILE: clustercontrast/utils/infomap_cluster.py
function l2norm (line 12) | def l2norm(vec):
function intdict2ndarray (line 22) | def intdict2ndarray(d, default_val=-1):
function read_meta (line 29) | def read_meta(fn_meta, start_pos=0, verbose=True):
class knn_faiss (line 51) | class knn_faiss():
method __init__ (line 57) | def __init__(self, feats, k, knn_method='faiss-cpu', verbose=True):
method filter_by_th (line 80) | def filter_by_th(self, i):
method get_knns (line 93) | def get_knns(self, th=None):
function knns2ordered_nbrs (line 114) | def knns2ordered_nbrs(knns, sort=True):
function get_links (line 129) | def get_links(single, links, nbrs, dists, min_sim):
function cluster_by_infomap (line 147) | def cluster_by_infomap(nbrs, dists, min_sim, cluster_num=2):
function get_dist_nbr (line 230) | def get_dist_nbr(features, k=80, knn_method='faiss-cpu'):
FILE: clustercontrast/utils/infomap_utils.py
class TextColors (line 4) | class TextColors:
class Timer (line 15) | class Timer():
method __init__ (line 16) | def __init__(self, name='task', verbose=True):
method __enter__ (line 20) | def __enter__(self):
method __exit__ (line 24) | def __exit__(self, exc_type, exc_val, exc_tb):
FILE: clustercontrast/utils/logging.py
class Logger (line 8) | class Logger(object):
method __init__ (line 9) | def __init__(self, fpath=None):
method __del__ (line 16) | def __del__(self):
method __enter__ (line 19) | def __enter__(self):
method __exit__ (line 22) | def __exit__(self, *args):
method write (line 25) | def write(self, msg):
method flush (line 30) | def flush(self):
method close (line 36) | def close(self):
FILE: clustercontrast/utils/meters.py
class AverageMeter (line 4) | class AverageMeter(object):
method __init__ (line 7) | def __init__(self):
method reset (line 13) | def reset(self):
method update (line 19) | def update(self, val, n=1):
FILE: clustercontrast/utils/osutils.py
function mkdir_if_missing (line 6) | def mkdir_if_missing(dir_path):
FILE: clustercontrast/utils/rerank.py
function re_ranking (line 31) | def re_ranking(q_g_dist, q_q_dist, g_g_dist, k1=20, k2=6, lambda_value=0...
FILE: clustercontrast/utils/serialization.py
function read_json (line 12) | def read_json(fpath):
function write_json (line 18) | def write_json(obj, fpath):
function save_checkpoint (line 24) | def save_checkpoint(state, is_best, fpath='checkpoint.pth.tar'):
function load_checkpoint (line 31) | def load_checkpoint(fpath):
function copy_state_dict (line 41) | def copy_state_dict(state_dict, model, strip=None):
FILE: examples/cluster_contrast_train_usl.py
function get_data (line 36) | def get_data(name, data_dir):
function get_train_loader (line 42) | def get_train_loader(args, dataset, height, width, batch_size, workers,
function get_test_loader (line 75) | def get_test_loader(dataset, height, width, batch_size, workers, testset...
function create_model (line 96) | def create_model(args):
function main (line 105) | def main():
function main_worker (line 117) | def main_worker(args):
FILE: examples/cluster_contrast_train_usl_infomap.py
function str2bool (line 34) | def str2bool(v):
function get_data (line 44) | def get_data(name, data_dir):
function get_train_loader (line 50) | def get_train_loader(args, dataset, height, width, batch_size, workers,
function get_test_loader (line 82) | def get_test_loader(dataset, height, width, batch_size, workers, testset...
function create_model (line 104) | def create_model(args):
function main (line 113) | def main():
function main_worker (line 125) | def main_worker(args):
FILE: examples/test.py
function get_data (line 23) | def get_data(name, data_dir, height, width, batch_size, workers):
function main (line 45) | def main():
function main_worker (line 57) | def main_worker(args):
Condensed preview — 44 files, each showing path, character count, and a content snippet. Download the .json file or copy for the full structured content (316K chars).
[
{
"path": "LICENSE",
"chars": 1064,
"preview": "MIT License\n\nCopyright (c) 2021 Alibaba\n\nPermission is hereby granted, free of charge, to any person obtaining a copy\nof"
},
{
"path": "README.md",
"chars": 5962,
"preview": "\n![PyTorch >=1.6](https://img.shields.io/badge/PyTorc"
},
{
"path": "clustercontrast/__init__.py",
"chars": 208,
"preview": "from __future__ import absolute_import\n\nfrom . import datasets\nfrom . import evaluation_metrics\nfrom . import models\nfro"
},
{
"path": "clustercontrast/datasets/__init__.py",
"chars": 1273,
"preview": "from __future__ import absolute_import\nimport warnings\n\nfrom .market1501 import Market1501\nfrom .msmt17 import MSMT17\nfr"
},
{
"path": "clustercontrast/datasets/dukemtmcreid.py",
"chars": 3121,
"preview": "import glob\nimport os.path as osp\nimport re\nfrom ..utils.data import BaseImageDataset\n\n\ndef process_dir(dir_path, relabe"
},
{
"path": "clustercontrast/datasets/market1501.py",
"chars": 3255,
"preview": "from __future__ import print_function, absolute_import\nimport os.path as osp\nimport glob\nimport re\nfrom ..utils.data imp"
},
{
"path": "clustercontrast/datasets/msmt17.py",
"chars": 2779,
"preview": "from __future__ import print_function, absolute_import\nimport os.path as osp\n\nimport glob\nimport re\nfrom ..utils.data im"
},
{
"path": "clustercontrast/datasets/personx.py",
"chars": 3033,
"preview": "from __future__ import print_function, absolute_import\nimport os.path as osp\nimport glob\nimport re\n\nfrom ..utils.data im"
},
{
"path": "clustercontrast/datasets/veri.py",
"chars": 3320,
"preview": "from __future__ import absolute_import\nfrom __future__ import division\nfrom __future__ import print_function\n\nimport glo"
},
{
"path": "clustercontrast/evaluation_metrics/__init__.py",
"chars": 167,
"preview": "from __future__ import absolute_import\n\nfrom .classification import accuracy\nfrom .ranking import cmc, mean_ap\n\n__all__ "
},
{
"path": "clustercontrast/evaluation_metrics/classification.py",
"chars": 604,
"preview": "from __future__ import absolute_import\n\nimport torch\nfrom ..utils import to_torch\n\n\ndef accuracy(output, target, topk=(1"
},
{
"path": "clustercontrast/evaluation_metrics/ranking.py",
"chars": 4126,
"preview": "from __future__ import absolute_import\nfrom collections import defaultdict\n\nimport numpy as np\nfrom sklearn.metrics impo"
},
{
"path": "clustercontrast/evaluators.py",
"chars": 4623,
"preview": "from __future__ import print_function, absolute_import\nimport time\nimport collections\nfrom collections import OrderedDic"
},
{
"path": "clustercontrast/models/__init__.py",
"chars": 1839,
"preview": "from __future__ import absolute_import\n\nfrom .resnet import *\nfrom .resnet_ibn import *\n__factory = {\n 'resnet18': re"
},
{
"path": "clustercontrast/models/cm.py",
"chars": 3243,
"preview": "import collections\nimport numpy as np\nfrom abc import ABC\nimport torch\nimport torch.nn.functional as F\nfrom torch import"
},
{
"path": "clustercontrast/models/dsbn.py",
"chars": 2680,
"preview": "import torch\nimport torch.nn as nn\n\n# Domain-specific BatchNorm\n\nclass DSBN2d(nn.Module):\n def __init__(self, planes)"
},
{
"path": "clustercontrast/models/kmeans.py",
"chars": 771,
"preview": "# Written by Yixiao Ge\n\nimport warnings\n\nimport faiss\nimport torch\n\nfrom ..utils import to_numpy, to_torch\n\n__all__ = [\""
},
{
"path": "clustercontrast/models/pooling.py",
"chars": 7009,
"preview": "# Credit to https://github.com/JDAI-CV/fast-reid/blob/master/fastreid/layers/pooling.py\nfrom abc import ABC\n\nimport torc"
},
{
"path": "clustercontrast/models/resnet.py",
"chars": 4417,
"preview": "from __future__ import absolute_import\nfrom torch import nn\nfrom torch.nn import functional as F\nfrom torch.nn import in"
},
{
"path": "clustercontrast/models/resnet_ibn.py",
"chars": 3988,
"preview": "from __future__ import absolute_import\n\nfrom torch import nn\nfrom torch.nn import functional as F\nfrom torch.nn import i"
},
{
"path": "clustercontrast/models/resnet_ibn_a.py",
"chars": 6595,
"preview": "import torch\nimport torch.nn as nn\nimport math\nimport torch.utils.model_zoo as model_zoo\n\n\n__all__ = ['ResNet', 'resnet5"
},
{
"path": "clustercontrast/trainers.py",
"chars": 1971,
"preview": "from __future__ import print_function, absolute_import\nimport time\nfrom .utils.meters import AverageMeter\n\n\nclass Cluste"
},
{
"path": "clustercontrast/utils/__init__.py",
"chars": 594,
"preview": "from __future__ import absolute_import\n\nimport torch\n\n\ndef to_numpy(tensor):\n if torch.is_tensor(tensor):\n ret"
},
{
"path": "clustercontrast/utils/data/__init__.py",
"chars": 633,
"preview": "from __future__ import absolute_import\n\nfrom .base_dataset import BaseDataset, BaseImageDataset\nfrom .preprocessor impor"
},
{
"path": "clustercontrast/utils/data/base_dataset.py",
"chars": 1620,
"preview": "# encoding: utf-8\nimport numpy as np\n\n\nclass BaseDataset(object):\n \"\"\"\n Base class of reid dataset\n \"\"\"\n\n de"
},
{
"path": "clustercontrast/utils/data/preprocessor.py",
"chars": 917,
"preview": "from __future__ import absolute_import\nimport os\nimport os.path as osp\nfrom torch.utils.data import DataLoader, Dataset\n"
},
{
"path": "clustercontrast/utils/data/sampler.py",
"chars": 5000,
"preview": "from __future__ import absolute_import\nfrom collections import defaultdict\nimport math\n\nimport numpy as np\nimport copy\ni"
},
{
"path": "clustercontrast/utils/data/transforms.py",
"chars": 3359,
"preview": "from __future__ import absolute_import\n\nfrom torchvision.transforms import *\nfrom PIL import Image\nimport random\nimport "
},
{
"path": "clustercontrast/utils/faiss_rerank.py",
"chars": 4846,
"preview": "#!/usr/bin/env python3\n# -*- coding: utf-8 -*-\n\"\"\"\nCVPR2017 paper:Zhong Z, Zheng L, Cao D, et al. Re-ranking Person Re-i"
},
{
"path": "clustercontrast/utils/faiss_utils.py",
"chars": 3184,
"preview": "import os\nimport numpy as np\nimport faiss\nimport torch\n\ndef swig_ptr_from_FloatTensor(x):\n assert x.is_contiguous()\n "
},
{
"path": "clustercontrast/utils/infomap_cluster.py",
"chars": 6725,
"preview": "import numpy as np\nfrom tqdm import tqdm\nimport infomap\nimport faiss\nimport math\nimport multiprocessing as mp\nfrom clust"
},
{
"path": "clustercontrast/utils/infomap_utils.py",
"chars": 658,
"preview": "import time\n\n\nclass TextColors:\n HEADER = '\\033[35m'\n OKBLUE = '\\033[34m'\n OKGREEN = '\\033[32m'\n WARNING = '"
},
{
"path": "clustercontrast/utils/logging.py",
"chars": 876,
"preview": "from __future__ import absolute_import\nimport os\nimport sys\n\nfrom .osutils import mkdir_if_missing\n\n\nclass Logger(object"
},
{
"path": "clustercontrast/utils/meters.py",
"chars": 497,
"preview": "from __future__ import absolute_import\n\n\nclass AverageMeter(object):\n \"\"\"Computes and stores the average and current "
},
{
"path": "clustercontrast/utils/osutils.py",
"chars": 214,
"preview": "from __future__ import absolute_import\nimport os\nimport errno\n\n\ndef mkdir_if_missing(dir_path):\n try:\n os.make"
},
{
"path": "clustercontrast/utils/rerank.py",
"chars": 4421,
"preview": "#!/usr/bin/env python2/python3\n# -*- coding: utf-8 -*-\n\"\"\"\nSource: https://github.com/zhunzhong07/person-re-ranking\nCrea"
},
{
"path": "clustercontrast/utils/serialization.py",
"chars": 1758,
"preview": "from __future__ import print_function, absolute_import\nimport json\nimport os.path as osp\nimport shutil\n\nimport torch\nfro"
},
{
"path": "examples/cluster_contrast_train_usl.py",
"chars": 11650,
"preview": "# -*- coding: utf-8 -*-\nfrom __future__ import print_function, absolute_import\nimport argparse\nimport os.path as osp\nimp"
},
{
"path": "examples/cluster_contrast_train_usl_infomap.py",
"chars": 11798,
"preview": "# -*- coding: utf-8 -*-\nfrom __future__ import print_function, absolute_import\nimport argparse\nimport os.path as osp\nimp"
},
{
"path": "examples/logs/log.txt",
"chars": 168954,
"preview": "==========\nArgs:Namespace(arch='resnet_ibn50a', batch_size=256, data_dir='/data0/developer/cluster-contrast/examples/dat"
},
{
"path": "examples/test.py",
"chars": 4743,
"preview": "from __future__ import print_function, absolute_import\nimport argparse\nimport os.path as osp\nimport random\nimport numpy "
},
{
"path": "run_code.sh",
"chars": 1457,
"preview": "CUDA_VISIBLE_DEVICES=0,1,2,3 python examples/cluster_contrast_train_usl.py -b 256 -a resnet50 -d market1501 --iters 200 "
},
{
"path": "setup.py",
"chars": 607,
"preview": "from setuptools import setup, find_packages\n\n\nsetup(name='ClusterContrast',\n version='1.0.0',\n description='Cl"
}
]
// ... and 1 more files (download for full content)
About this extraction
This page contains the full source code of the alibaba/cluster-contrast-reid GitHub repository, extracted and formatted as plain text for AI agents and large language models (LLMs). The extraction includes 44 files (98.0 MB), approximately 118.3k tokens, and a symbol index with 220 extracted functions, classes, methods, constants, and types. Use this with OpenClaw, Claude, ChatGPT, Cursor, Windsurf, or any other AI tool that accepts text input. You can copy the full output to your clipboard or download it as a .txt file.
Extracted by GitExtract — free GitHub repo to text converter for AI. Built by Nikandr Surkov.